2024-09-12T02:38:16.730 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-12T02:38:16.753 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-12T02:38:16.845 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901638 branch: squid description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-quiesce/with-quiesce 6-workunit/suites/fsx}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7901638' kernel: client: sha1: distro kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: cephfs: max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 mon warn on pool no app: false osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - slow metadata IO - SLOW_OPS - slow request sha1: b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc subvols: create: 2 subvol_options: --namespace-isolated ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - mds.dir_split install: ceph: flavor: default sha1: b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=secure - wsync syntax: v2 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: 086c737fa75c22d521a3932c59b31c5e1d1c9ef4 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 481 sha1: b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc sleep_before_teardown: 0 subset: 57/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 086c737fa75c22d521a3932c59b31c5e1d1c9ef4 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi005.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCKq6h/BnD3mJvrm3g/VBycB1LVIUoqjnkARr2Of5U4Tm0P1L3WydtXc/MTEDsjMueOS/Uh5nlIVs0RCMJP2gIc= smithi146.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJOuolBQS9Gwm0ugvjNDwekL45E58ulbk68e06x9MMtRjq8pBPnkxJqVse151JY0cZOd4UU5UPTYjJeHt7cTgd4= smithi148.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBC1K99Yxsg6VwKFdkBGRpRJhhCzGLHvRcsdhLmwHXuzcyn95JsIKSpSOESq0srxhK6TWo4g0jbdqdI4PsUUwKBs= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - quiescer: null - workunit: clients: all: - suites/fsx.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-11_21:24:17 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-12T02:38:16.845 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa; will attempt to use it 2024-09-12T02:38:16.846 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa/tasks 2024-09-12T02:38:16.846 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-12T02:38:16.848 INFO:teuthology.task.internal:Checking packages... 2024-09-12T02:38:16.867 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc' 2024-09-12T02:38:16.867 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-12T02:38:16.867 INFO:teuthology.packaging:ref: None 2024-09-12T02:38:16.867 INFO:teuthology.packaging:tag: None 2024-09-12T02:38:16.867 INFO:teuthology.packaging:branch: squid 2024-09-12T02:38:16.867 INFO:teuthology.packaging:sha1: b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T02:38:16.867 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-09-12T02:38:17.079 INFO:teuthology.task.internal:Found packages for ceph version 19.1.1-314.g086c737f 2024-09-12T02:38:17.080 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-12T02:38:17.087 INFO:teuthology.task.internal:no buildpackages task found 2024-09-12T02:38:17.087 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-12T02:38:17.123 INFO:teuthology.task.internal:Saving configuration 2024-09-12T02:38:17.142 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-12T02:38:17.158 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-12T02:38:17.184 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi005.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901638', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-12 02:33:07.724604', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCKq6h/BnD3mJvrm3g/VBycB1LVIUoqjnkARr2Of5U4Tm0P1L3WydtXc/MTEDsjMueOS/Uh5nlIVs0RCMJP2gIc='} 2024-09-12T02:38:17.207 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi146.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901638', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-12 02:33:07.721759', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJOuolBQS9Gwm0ugvjNDwekL45E58ulbk68e06x9MMtRjq8pBPnkxJqVse151JY0cZOd4UU5UPTYjJeHt7cTgd4='} 2024-09-12T02:38:17.231 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi148.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901638', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-12 02:33:07.723316', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBC1K99Yxsg6VwKFdkBGRpRJhhCzGLHvRcsdhLmwHXuzcyn95JsIKSpSOESq0srxhK6TWo4g0jbdqdI4PsUUwKBs='} 2024-09-12T02:38:17.231 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-12T02:38:17.236 INFO:teuthology.task.internal:roles: ubuntu@smithi005.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-09-12T02:38:17.237 INFO:teuthology.task.internal:roles: ubuntu@smithi146.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-09-12T02:38:17.237 INFO:teuthology.task.internal:roles: ubuntu@smithi148.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-09-12T02:38:17.237 INFO:teuthology.run_tasks:Running task console_log... 2024-09-12T02:38:17.339 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f17649f3490>, signals=[15]) 2024-09-12T02:38:17.339 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-12T02:38:17.345 INFO:teuthology.task.internal:Opening connections... 2024-09-12T02:38:17.345 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi005.front.sepia.ceph.com 2024-09-12T02:38:17.346 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi005.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T02:38:17.421 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi146.front.sepia.ceph.com 2024-09-12T02:38:17.421 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T02:38:17.499 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi148.front.sepia.ceph.com 2024-09-12T02:38:17.500 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T02:38:17.572 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-12T02:38:17.580 DEBUG:teuthology.orchestra.run.smithi005:> uname -m 2024-09-12T02:38:17.598 INFO:teuthology.orchestra.run.smithi005.stdout:x86_64 2024-09-12T02:38:17.598 DEBUG:teuthology.orchestra.run.smithi005:> cat /etc/os-release 2024-09-12T02:38:17.656 INFO:teuthology.orchestra.run.smithi005.stdout:NAME="CentOS Stream" 2024-09-12T02:38:17.656 INFO:teuthology.orchestra.run.smithi005.stdout:VERSION="9" 2024-09-12T02:38:17.656 INFO:teuthology.orchestra.run.smithi005.stdout:ID="centos" 2024-09-12T02:38:17.656 INFO:teuthology.orchestra.run.smithi005.stdout:ID_LIKE="rhel fedora" 2024-09-12T02:38:17.656 INFO:teuthology.orchestra.run.smithi005.stdout:VERSION_ID="9" 2024-09-12T02:38:17.656 INFO:teuthology.orchestra.run.smithi005.stdout:PLATFORM_ID="platform:el9" 2024-09-12T02:38:17.656 INFO:teuthology.orchestra.run.smithi005.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-12T02:38:17.656 INFO:teuthology.orchestra.run.smithi005.stdout:ANSI_COLOR="0;31" 2024-09-12T02:38:17.656 INFO:teuthology.orchestra.run.smithi005.stdout:LOGO="fedora-logo-icon" 2024-09-12T02:38:17.657 INFO:teuthology.orchestra.run.smithi005.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-12T02:38:17.657 INFO:teuthology.orchestra.run.smithi005.stdout:HOME_URL="https://centos.org/" 2024-09-12T02:38:17.657 INFO:teuthology.orchestra.run.smithi005.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-12T02:38:17.657 INFO:teuthology.orchestra.run.smithi005.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-12T02:38:17.657 INFO:teuthology.orchestra.run.smithi005.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-12T02:38:17.657 INFO:teuthology.lock.ops:Updating smithi005.front.sepia.ceph.com on lock server 2024-09-12T02:38:17.681 DEBUG:teuthology.orchestra.run.smithi146:> uname -m 2024-09-12T02:38:17.699 INFO:teuthology.orchestra.run.smithi146.stdout:x86_64 2024-09-12T02:38:17.699 DEBUG:teuthology.orchestra.run.smithi146:> cat /etc/os-release 2024-09-12T02:38:17.757 INFO:teuthology.orchestra.run.smithi146.stdout:NAME="CentOS Stream" 2024-09-12T02:38:17.757 INFO:teuthology.orchestra.run.smithi146.stdout:VERSION="9" 2024-09-12T02:38:17.757 INFO:teuthology.orchestra.run.smithi146.stdout:ID="centos" 2024-09-12T02:38:17.757 INFO:teuthology.orchestra.run.smithi146.stdout:ID_LIKE="rhel fedora" 2024-09-12T02:38:17.757 INFO:teuthology.orchestra.run.smithi146.stdout:VERSION_ID="9" 2024-09-12T02:38:17.757 INFO:teuthology.orchestra.run.smithi146.stdout:PLATFORM_ID="platform:el9" 2024-09-12T02:38:17.757 INFO:teuthology.orchestra.run.smithi146.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-12T02:38:17.758 INFO:teuthology.orchestra.run.smithi146.stdout:ANSI_COLOR="0;31" 2024-09-12T02:38:17.758 INFO:teuthology.orchestra.run.smithi146.stdout:LOGO="fedora-logo-icon" 2024-09-12T02:38:17.758 INFO:teuthology.orchestra.run.smithi146.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-12T02:38:17.758 INFO:teuthology.orchestra.run.smithi146.stdout:HOME_URL="https://centos.org/" 2024-09-12T02:38:17.758 INFO:teuthology.orchestra.run.smithi146.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-12T02:38:17.758 INFO:teuthology.orchestra.run.smithi146.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-12T02:38:17.758 INFO:teuthology.orchestra.run.smithi146.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-12T02:38:17.758 INFO:teuthology.lock.ops:Updating smithi146.front.sepia.ceph.com on lock server 2024-09-12T02:38:17.780 DEBUG:teuthology.orchestra.run.smithi148:> uname -m 2024-09-12T02:38:17.799 INFO:teuthology.orchestra.run.smithi148.stdout:x86_64 2024-09-12T02:38:17.799 DEBUG:teuthology.orchestra.run.smithi148:> cat /etc/os-release 2024-09-12T02:38:17.856 INFO:teuthology.orchestra.run.smithi148.stdout:NAME="CentOS Stream" 2024-09-12T02:38:17.856 INFO:teuthology.orchestra.run.smithi148.stdout:VERSION="9" 2024-09-12T02:38:17.856 INFO:teuthology.orchestra.run.smithi148.stdout:ID="centos" 2024-09-12T02:38:17.856 INFO:teuthology.orchestra.run.smithi148.stdout:ID_LIKE="rhel fedora" 2024-09-12T02:38:17.856 INFO:teuthology.orchestra.run.smithi148.stdout:VERSION_ID="9" 2024-09-12T02:38:17.857 INFO:teuthology.orchestra.run.smithi148.stdout:PLATFORM_ID="platform:el9" 2024-09-12T02:38:17.857 INFO:teuthology.orchestra.run.smithi148.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-12T02:38:17.857 INFO:teuthology.orchestra.run.smithi148.stdout:ANSI_COLOR="0;31" 2024-09-12T02:38:17.857 INFO:teuthology.orchestra.run.smithi148.stdout:LOGO="fedora-logo-icon" 2024-09-12T02:38:17.857 INFO:teuthology.orchestra.run.smithi148.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-12T02:38:17.857 INFO:teuthology.orchestra.run.smithi148.stdout:HOME_URL="https://centos.org/" 2024-09-12T02:38:17.857 INFO:teuthology.orchestra.run.smithi148.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-12T02:38:17.857 INFO:teuthology.orchestra.run.smithi148.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-12T02:38:17.857 INFO:teuthology.orchestra.run.smithi148.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-12T02:38:17.857 INFO:teuthology.lock.ops:Updating smithi148.front.sepia.ceph.com on lock server 2024-09-12T02:38:17.880 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-12T02:38:17.891 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-12T02:38:17.927 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-12T02:38:17.927 DEBUG:teuthology.orchestra.run.smithi005:> test '!' -e /home/ubuntu/cephtest 2024-09-12T02:38:17.929 DEBUG:teuthology.orchestra.run.smithi146:> test '!' -e /home/ubuntu/cephtest 2024-09-12T02:38:17.932 DEBUG:teuthology.orchestra.run.smithi148:> test '!' -e /home/ubuntu/cephtest 2024-09-12T02:38:17.949 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-12T02:38:17.955 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-12T02:38:17.955 DEBUG:teuthology.orchestra.run.smithi005:> test -z $(ls -A /var/lib/ceph) 2024-09-12T02:38:17.986 DEBUG:teuthology.orchestra.run.smithi146:> test -z $(ls -A /var/lib/ceph) 2024-09-12T02:38:17.990 DEBUG:teuthology.orchestra.run.smithi148:> test -z $(ls -A /var/lib/ceph) 2024-09-12T02:38:18.113 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-12T02:38:18.185 INFO:teuthology.run_tasks:Running task kernel... 2024-09-12T02:38:18.200 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'} 2024-09-12T02:38:18.200 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-09-12T02:38:18.200 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-09-12T02:38:18.200 DEBUG:teuthology.orchestra.run.smithi005:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-12T02:38:18.217 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T02:38:18.218 DEBUG:teuthology.orchestra.run.smithi005:> uname -r 2024-09-12T02:38:18.276 INFO:teuthology.orchestra.run.smithi005.stdout:5.14.0-503.el9.x86_64 2024-09-12T02:38:18.276 INFO:teuthology.task.kernel:Running kernel on smithi005: 5.14.0-503.el9.x86_64 2024-09-12T02:38:18.276 DEBUG:teuthology.orchestra.run.smithi005:> sudo yum install -y kernel 2024-09-12T02:38:21.281 INFO:teuthology.orchestra.run.smithi005.stdout:CentOS Stream 9 - BaseOS 6.5 MB/s | 8.2 MB 00:01 2024-09-12T02:38:23.349 INFO:teuthology.orchestra.run.smithi005.stdout:CentOS Stream 9 - AppStream 23 MB/s | 20 MB 00:00 2024-09-12T02:38:28.149 INFO:teuthology.orchestra.run.smithi005.stdout:CentOS Stream 9 - CRB 7.3 MB/s | 6.5 MB 00:00 2024-09-12T02:38:29.792 INFO:teuthology.orchestra.run.smithi005.stdout:CentOS Stream 9 - Extras packages 52 kB/s | 19 kB 00:00 2024-09-12T02:38:30.662 INFO:teuthology.orchestra.run.smithi005.stdout:Extra Packages for Enterprise Linux 31 MB/s | 23 MB 00:00 2024-09-12T02:38:36.335 INFO:teuthology.orchestra.run.smithi005.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-09-12T02:38:38.425 INFO:teuthology.orchestra.run.smithi005.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-12T02:38:38.425 INFO:teuthology.orchestra.run.smithi005.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-12T02:38:38.524 INFO:teuthology.orchestra.run.smithi005.stdout:Dependencies resolved. 2024-09-12T02:38:38.536 INFO:teuthology.orchestra.run.smithi005.stdout:================================================================================ 2024-09-12T02:38:38.536 INFO:teuthology.orchestra.run.smithi005.stdout: Package Architecture Version Repository Size 2024-09-12T02:38:38.536 INFO:teuthology.orchestra.run.smithi005.stdout:================================================================================ 2024-09-12T02:38:38.536 INFO:teuthology.orchestra.run.smithi005.stdout:Installing: 2024-09-12T02:38:38.536 INFO:teuthology.orchestra.run.smithi005.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-12T02:38:38.536 INFO:teuthology.orchestra.run.smithi005.stdout:Installing dependencies: 2024-09-12T02:38:38.536 INFO:teuthology.orchestra.run.smithi005.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-12T02:38:38.536 INFO:teuthology.orchestra.run.smithi005.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-12T02:38:38.536 INFO:teuthology.orchestra.run.smithi005.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-12T02:38:38.537 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-12T02:38:38.537 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction Summary 2024-09-12T02:38:38.537 INFO:teuthology.orchestra.run.smithi005.stdout:================================================================================ 2024-09-12T02:38:38.537 INFO:teuthology.orchestra.run.smithi005.stdout:Install 4 Packages 2024-09-12T02:38:38.537 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-12T02:38:38.537 INFO:teuthology.orchestra.run.smithi005.stdout:Total download size: 79 M 2024-09-12T02:38:38.537 INFO:teuthology.orchestra.run.smithi005.stdout:Installed size: 126 M 2024-09-12T02:38:38.537 INFO:teuthology.orchestra.run.smithi005.stdout:Downloading Packages: 2024-09-12T02:38:38.786 INFO:teuthology.orchestra.run.smithi005.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 185 kB/s | 23 kB 00:00 2024-09-12T02:38:39.886 INFO:teuthology.orchestra.run.smithi005.stdout:(2/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 28 MB/s | 35 MB 00:01 2024-09-12T02:38:40.228 INFO:teuthology.orchestra.run.smithi005.stdout:(3/4): kernel-core-5.14.0-505.el9.x86_64.rpm 10 MB/s | 16 MB 00:01 2024-09-12T02:38:40.637 INFO:teuthology.orchestra.run.smithi005.stdout:(4/4): kernel-modules-core-5.14.0-505.el9.x86_6 15 MB/s | 29 MB 00:01 2024-09-12T02:38:40.637 INFO:teuthology.orchestra.run.smithi005.stdout:-------------------------------------------------------------------------------- 2024-09-12T02:38:40.637 INFO:teuthology.orchestra.run.smithi005.stdout:Total 38 MB/s | 79 MB 00:02 2024-09-12T02:38:41.343 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction check 2024-09-12T02:38:41.714 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction check succeeded. 2024-09-12T02:38:41.714 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction test 2024-09-12T02:38:42.691 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction test succeeded. 2024-09-12T02:38:42.691 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction 2024-09-12T02:38:45.277 INFO:teuthology.orchestra.run.smithi005.stdout: Preparing : 1/1 2024-09-12T02:38:45.947 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-12T02:38:46.045 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-12T02:38:48.821 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-12T02:38:48.957 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-12T02:38:54.081 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-12T02:38:54.199 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-12T02:38:59.347 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-12T02:39:38.210 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-12T02:39:38.251 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-12T02:39:38.594 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-12T02:39:38.594 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-12T02:39:38.594 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-12T02:39:38.595 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-12T02:39:39.399 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-12T02:39:39.399 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-12T02:39:39.399 INFO:teuthology.orchestra.run.smithi005.stdout:Installed: 2024-09-12T02:39:39.399 INFO:teuthology.orchestra.run.smithi005.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-12T02:39:39.399 INFO:teuthology.orchestra.run.smithi005.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-12T02:39:39.400 INFO:teuthology.orchestra.run.smithi005.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-12T02:39:39.400 INFO:teuthology.orchestra.run.smithi005.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-12T02:39:39.400 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-12T02:39:39.400 INFO:teuthology.orchestra.run.smithi005.stdout:Complete! 2024-09-12T02:39:39.752 DEBUG:teuthology.orchestra.run.smithi005:> rpm -q kernel | sort -rV | head -n 1 2024-09-12T02:39:39.849 INFO:teuthology.orchestra.run.smithi005.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-12T02:39:39.850 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-12T02:39:39.850 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-12T02:39:39.850 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-12T02:39:39.850 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-12T02:39:39.850 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-09-12T02:39:39.850 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi005.front.sepia.ceph.com, path=None, version=distro) 2024-09-12T02:39:39.850 DEBUG:teuthology.orchestra.run.smithi005:> rpm -q kernel | sort -rV | head -n 1 2024-09-12T02:39:39.909 INFO:teuthology.orchestra.run.smithi005.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-12T02:39:39.910 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-12T02:39:39.910 DEBUG:teuthology.orchestra.run.smithi005:> sudo rpm -qi grub2-tools 2024-09-12T02:39:39.941 INFO:teuthology.orchestra.run.smithi005.stdout:Name : grub2-tools 2024-09-12T02:39:39.941 INFO:teuthology.orchestra.run.smithi005.stdout:Epoch : 1 2024-09-12T02:39:39.941 INFO:teuthology.orchestra.run.smithi005.stdout:Version : 2.06 2024-09-12T02:39:39.941 INFO:teuthology.orchestra.run.smithi005.stdout:Release : 82.el9 2024-09-12T02:39:39.941 INFO:teuthology.orchestra.run.smithi005.stdout:Architecture: x86_64 2024-09-12T02:39:39.941 INFO:teuthology.orchestra.run.smithi005.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-12T02:39:39.941 INFO:teuthology.orchestra.run.smithi005.stdout:Group : Unspecified 2024-09-12T02:39:39.941 INFO:teuthology.orchestra.run.smithi005.stdout:Size : 8274670 2024-09-12T02:39:39.941 INFO:teuthology.orchestra.run.smithi005.stdout:License : GPLv3+ 2024-09-12T02:39:39.941 INFO:teuthology.orchestra.run.smithi005.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-12T02:39:39.942 INFO:teuthology.orchestra.run.smithi005.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-12T02:39:39.942 INFO:teuthology.orchestra.run.smithi005.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-12T02:39:39.942 INFO:teuthology.orchestra.run.smithi005.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-12T02:39:39.942 INFO:teuthology.orchestra.run.smithi005.stdout:Packager : builder@centos.org 2024-09-12T02:39:39.942 INFO:teuthology.orchestra.run.smithi005.stdout:Vendor : CentOS 2024-09-12T02:39:39.942 INFO:teuthology.orchestra.run.smithi005.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-12T02:39:39.942 INFO:teuthology.orchestra.run.smithi005.stdout:Summary : Support tools for GRUB. 2024-09-12T02:39:39.942 INFO:teuthology.orchestra.run.smithi005.stdout:Description : 2024-09-12T02:39:39.942 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-12T02:39:39.942 INFO:teuthology.orchestra.run.smithi005.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-12T02:39:39.942 INFO:teuthology.orchestra.run.smithi005.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-12T02:39:39.942 INFO:teuthology.orchestra.run.smithi005.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-12T02:39:39.942 INFO:teuthology.orchestra.run.smithi005.stdout:hardware devices. 2024-09-12T02:39:39.942 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-12T02:39:39.943 INFO:teuthology.orchestra.run.smithi005.stdout:This subpackage provides tools for support of all platforms. 2024-09-12T02:39:39.943 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-12T02:39:39.943 INFO:teuthology.task.kernel:Updating grub on smithi005 to boot 5.14.0-505.el9.x86_64 2024-09-12T02:39:39.944 DEBUG:teuthology.orchestra.run.smithi005:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-12T02:39:40.592 INFO:teuthology.orchestra.run.smithi005.stderr:Generating grub configuration file ... 2024-09-12T02:39:41.920 INFO:teuthology.orchestra.run.smithi005.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-12T02:39:41.950 INFO:teuthology.orchestra.run.smithi005.stderr:done 2024-09-12T02:39:41.952 DEBUG:teuthology.orchestra.run.smithi005:> mktemp 2024-09-12T02:39:41.967 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/tmp.HNVBRrGRpt 2024-09-12T02:39:41.967 DEBUG:teuthology.orchestra.run.smithi005:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.HNVBRrGRpt 2024-09-12T02:39:42.032 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 0666 /tmp/tmp.HNVBRrGRpt 2024-09-12T02:39:42.173 DEBUG:teuthology.orchestra.remote:smithi005:/tmp/tmp.HNVBRrGRpt is 6KB 2024-09-12T02:39:42.222 DEBUG:teuthology.orchestra.run.smithi005:> rm -fr /tmp/tmp.HNVBRrGRpt 2024-09-12T02:39:42.237 DEBUG:teuthology.orchestra.run.smithi005:> sudo /bin/ls /boot/loader/entries || true 2024-09-12T02:39:42.307 INFO:teuthology.orchestra.run.smithi005.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-12T02:39:42.307 INFO:teuthology.orchestra.run.smithi005.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-12T02:39:42.308 INFO:teuthology.orchestra.run.smithi005.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-12T02:39:42.308 INFO:teuthology.orchestra.run.smithi005.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-12T02:39:42.309 DEBUG:teuthology.orchestra.run.smithi005:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-12T02:39:42.466 DEBUG:teuthology.orchestra.run.smithi005:> sudo shutdown -r now 2024-09-12T02:40:12.469 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-12T02:40:12.470 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi005.front.sepia.ceph.com' 2024-09-12T02:40:12.471 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi005.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T02:40:31.037 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.5 2024-09-12T02:40:40.038 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi005.front.sepia.ceph.com' 2024-09-12T02:40:40.038 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi005.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T02:40:43.101 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.5 2024-09-12T02:40:55.102 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi005.front.sepia.ceph.com' 2024-09-12T02:40:55.102 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi005.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T02:40:58.173 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.5 2024-09-12T02:41:13.173 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi005.front.sepia.ceph.com' 2024-09-12T02:41:13.174 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi005.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T02:41:45.082 DEBUG:teuthology.orchestra.run.smithi005:> true 2024-09-12T02:41:45.479 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi005.front.sepia.ceph.com' 2024-09-12T02:41:45.480 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-505.el9.x86_64"... 2024-09-12T02:41:45.480 DEBUG:teuthology.orchestra.run.smithi005:> uname -r 2024-09-12T02:41:45.496 INFO:teuthology.orchestra.run.smithi005.stdout:5.14.0-505.el9.x86_64 2024-09-12T02:41:45.496 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-12T02:41:45.496 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-12T02:41:45.496 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-12T02:41:46.497 DEBUG:teuthology.parallel:result is None 2024-09-12T02:41:46.497 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-12T02:41:46.504 INFO:teuthology.task.internal:Creating test directory... 2024-09-12T02:41:46.505 DEBUG:teuthology.orchestra.run.smithi005:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-12T02:41:46.507 DEBUG:teuthology.orchestra.run.smithi146:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-12T02:41:46.510 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-12T02:41:46.528 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-12T02:41:46.554 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-12T02:41:46.596 INFO:teuthology.task.internal:Creating archive directory... 2024-09-12T02:41:46.596 DEBUG:teuthology.orchestra.run.smithi005:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-12T02:41:46.598 DEBUG:teuthology.orchestra.run.smithi146:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-12T02:41:46.601 DEBUG:teuthology.orchestra.run.smithi148:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-12T02:41:46.736 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-12T02:41:46.745 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-12T02:41:46.745 DEBUG:teuthology.orchestra.run.smithi146:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-12T02:41:46.762 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T02:41:46.762 DEBUG:teuthology.orchestra.run.smithi148:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-12T02:41:46.779 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T02:41:46.780 DEBUG:teuthology.orchestra.run.smithi005:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-12T02:41:46.782 DEBUG:teuthology.orchestra.run.smithi146:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-12T02:41:46.805 DEBUG:teuthology.orchestra.run.smithi148:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-12T02:41:46.842 INFO:teuthology.orchestra.run.smithi146.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-12T02:41:46.863 INFO:teuthology.orchestra.run.smithi146.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-12T02:41:46.888 INFO:teuthology.orchestra.run.smithi148.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-12T02:41:46.890 INFO:teuthology.orchestra.run.smithi005.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-12T02:41:46.906 INFO:teuthology.orchestra.run.smithi148.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-12T02:41:46.922 INFO:teuthology.orchestra.run.smithi005.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-12T02:41:46.924 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-12T02:41:46.932 INFO:teuthology.task.internal:Configuring sudo... 2024-09-12T02:41:46.932 DEBUG:teuthology.orchestra.run.smithi005:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-12T02:41:46.934 DEBUG:teuthology.orchestra.run.smithi146:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-12T02:41:46.936 DEBUG:teuthology.orchestra.run.smithi148:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-12T02:41:46.965 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-12T02:41:46.973 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-12T02:41:46.973 DEBUG:teuthology.orchestra.run.smithi005:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-12T02:41:47.000 DEBUG:teuthology.orchestra.run.smithi146:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-12T02:41:47.006 DEBUG:teuthology.orchestra.run.smithi148:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-12T02:41:47.023 DEBUG:teuthology.orchestra.run.smithi005:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-12T02:41:47.097 DEBUG:teuthology.orchestra.run.smithi005:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-12T02:41:47.150 DEBUG:teuthology.orchestra.run.smithi005:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-12T02:41:47.232 DEBUG:teuthology.orchestra.run.smithi005:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-12T02:41:47.269 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-09-12T02:41:47.269 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-12T02:41:47.351 DEBUG:teuthology.orchestra.run.smithi146:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-12T02:41:47.399 DEBUG:teuthology.orchestra.run.smithi146:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-12T02:41:47.458 DEBUG:teuthology.orchestra.run.smithi146:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-12T02:41:47.503 DEBUG:teuthology.orchestra.run.smithi146:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-12T02:41:47.546 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-12T02:41:47.546 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-12T02:41:47.614 DEBUG:teuthology.orchestra.run.smithi148:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-12T02:41:47.657 DEBUG:teuthology.orchestra.run.smithi148:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-12T02:41:47.716 DEBUG:teuthology.orchestra.run.smithi148:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-12T02:41:47.801 DEBUG:teuthology.orchestra.run.smithi148:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-12T02:41:47.844 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-12T02:41:47.844 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-12T02:41:47.910 DEBUG:teuthology.orchestra.run.smithi005:> sudo service rsyslog restart 2024-09-12T02:41:47.912 DEBUG:teuthology.orchestra.run.smithi146:> sudo service rsyslog restart 2024-09-12T02:41:47.915 DEBUG:teuthology.orchestra.run.smithi148:> sudo service rsyslog restart 2024-09-12T02:41:47.953 INFO:teuthology.orchestra.run.smithi146.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-12T02:41:47.958 INFO:teuthology.orchestra.run.smithi005.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-12T02:41:47.990 INFO:teuthology.orchestra.run.smithi148.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-12T02:41:48.455 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-12T02:41:48.462 INFO:teuthology.task.internal:Starting timer... 2024-09-12T02:41:48.463 INFO:teuthology.run_tasks:Running task pcp... 2024-09-12T02:41:48.491 INFO:teuthology.run_tasks:Running task selinux... 2024-09-12T02:41:48.531 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-09-12T02:41:48.532 DEBUG:teuthology.orchestra.run.smithi005:> sudo service auditd rotate 2024-09-12T02:41:48.609 INFO:teuthology.orchestra.run.smithi005.stdout:Rotating logs: 2024-09-12T02:41:48.610 DEBUG:teuthology.orchestra.run.smithi146:> sudo service auditd rotate 2024-09-12T02:41:48.665 INFO:teuthology.orchestra.run.smithi146.stdout:Rotating logs: 2024-09-12T02:41:48.667 DEBUG:teuthology.orchestra.run.smithi148:> sudo service auditd rotate 2024-09-12T02:41:48.716 INFO:teuthology.orchestra.run.smithi148.stdout:Rotating logs: 2024-09-12T02:41:48.718 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-12T02:41:48.718 DEBUG:teuthology.orchestra.run.smithi005:> /usr/sbin/getenforce 2024-09-12T02:41:48.744 INFO:teuthology.orchestra.run.smithi005.stdout:Permissive 2024-09-12T02:41:48.745 DEBUG:teuthology.orchestra.run.smithi146:> /usr/sbin/getenforce 2024-09-12T02:41:48.769 INFO:teuthology.orchestra.run.smithi146.stdout:Permissive 2024-09-12T02:41:48.770 DEBUG:teuthology.orchestra.run.smithi148:> /usr/sbin/getenforce 2024-09-12T02:41:48.789 INFO:teuthology.orchestra.run.smithi148.stdout:Permissive 2024-09-12T02:41:48.790 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi005.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi146.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi148.front.sepia.ceph.com': 'permissive'} 2024-09-12T02:41:48.790 DEBUG:teuthology.orchestra.run.smithi005:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-12T02:41:48.815 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T02:41:48.816 DEBUG:teuthology.orchestra.run.smithi146:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-12T02:41:48.846 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T02:41:48.847 DEBUG:teuthology.orchestra.run.smithi148:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-12T02:41:48.873 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T02:41:48.876 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-12T02:41:48.876 DEBUG:teuthology.orchestra.run.smithi005:> sudo /usr/sbin/setenforce permissive 2024-09-12T02:41:48.957 DEBUG:teuthology.orchestra.run.smithi146:> sudo /usr/sbin/setenforce permissive 2024-09-12T02:41:48.999 DEBUG:teuthology.orchestra.run.smithi148:> sudo /usr/sbin/setenforce permissive 2024-09-12T02:41:49.045 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-12T02:41:49.234 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-12T02:41:49.246 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-12T02:41:49.247 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi005.front.sepia.ceph.com,smithi146.front.sepia.ceph.com,smithi148.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-12T02:45:57.232 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi005.front.sepia.ceph.com'), Remote(name='ubuntu@smithi146.front.sepia.ceph.com'), Remote(name='ubuntu@smithi148.front.sepia.ceph.com')] 2024-09-12T02:45:57.234 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi005.front.sepia.ceph.com' 2024-09-12T02:45:57.235 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi005.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T02:45:57.317 DEBUG:teuthology.orchestra.run.smithi005:> true 2024-09-12T02:45:57.399 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi005.front.sepia.ceph.com' 2024-09-12T02:45:57.399 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi146.front.sepia.ceph.com' 2024-09-12T02:45:57.400 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi146.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T02:45:57.475 DEBUG:teuthology.orchestra.run.smithi146:> true 2024-09-12T02:45:57.561 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi146.front.sepia.ceph.com' 2024-09-12T02:45:57.562 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi148.front.sepia.ceph.com' 2024-09-12T02:45:57.562 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi148.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-12T02:45:57.639 DEBUG:teuthology.orchestra.run.smithi148:> true 2024-09-12T02:45:57.720 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi148.front.sepia.ceph.com' 2024-09-12T02:45:57.721 INFO:teuthology.run_tasks:Running task clock... 2024-09-12T02:45:57.735 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-12T02:45:57.736 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-12T02:45:57.736 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-12T02:45:57.740 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-12T02:45:57.740 DEBUG:teuthology.orchestra.run.smithi146:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-12T02:45:57.743 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-12T02:45:57.744 DEBUG:teuthology.orchestra.run.smithi148:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-12T02:45:57.772 INFO:teuthology.orchestra.run.smithi005.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-12T02:45:57.772 INFO:teuthology.orchestra.run.smithi146.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-12T02:45:57.788 INFO:teuthology.orchestra.run.smithi146.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-12T02:45:57.788 INFO:teuthology.orchestra.run.smithi005.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-12T02:45:57.792 INFO:teuthology.orchestra.run.smithi148.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-12T02:45:57.807 INFO:teuthology.orchestra.run.smithi148.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-12T02:45:57.813 INFO:teuthology.orchestra.run.smithi146.stderr:sudo: ntpd: command not found 2024-09-12T02:45:57.816 INFO:teuthology.orchestra.run.smithi005.stderr:sudo: ntpd: command not found 2024-09-12T02:45:57.824 INFO:teuthology.orchestra.run.smithi146.stdout:506 Cannot talk to daemon 2024-09-12T02:45:57.829 INFO:teuthology.orchestra.run.smithi005.stdout:506 Cannot talk to daemon 2024-09-12T02:45:57.831 INFO:teuthology.orchestra.run.smithi148.stderr:sudo: ntpd: command not found 2024-09-12T02:45:57.839 INFO:teuthology.orchestra.run.smithi146.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-12T02:45:57.843 INFO:teuthology.orchestra.run.smithi148.stdout:506 Cannot talk to daemon 2024-09-12T02:45:57.844 INFO:teuthology.orchestra.run.smithi005.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-12T02:45:57.853 INFO:teuthology.orchestra.run.smithi146.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-12T02:45:57.857 INFO:teuthology.orchestra.run.smithi148.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-12T02:45:57.859 INFO:teuthology.orchestra.run.smithi005.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-12T02:45:57.871 INFO:teuthology.orchestra.run.smithi148.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-12T02:45:57.910 INFO:teuthology.orchestra.run.smithi146.stderr:bash: line 1: ntpq: command not found 2024-09-12T02:45:57.914 INFO:teuthology.orchestra.run.smithi146.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-12T02:45:57.914 INFO:teuthology.orchestra.run.smithi146.stdout:=============================================================================== 2024-09-12T02:45:57.914 INFO:teuthology.orchestra.run.smithi146.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T02:45:57.914 INFO:teuthology.orchestra.run.smithi146.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T02:45:57.914 INFO:teuthology.orchestra.run.smithi146.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T02:45:57.914 INFO:teuthology.orchestra.run.smithi146.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T02:45:57.917 INFO:teuthology.orchestra.run.smithi005.stderr:bash: line 1: ntpq: command not found 2024-09-12T02:45:57.921 INFO:teuthology.orchestra.run.smithi005.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-12T02:45:57.921 INFO:teuthology.orchestra.run.smithi005.stdout:=============================================================================== 2024-09-12T02:45:57.921 INFO:teuthology.orchestra.run.smithi005.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T02:45:57.922 INFO:teuthology.orchestra.run.smithi005.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T02:45:57.922 INFO:teuthology.orchestra.run.smithi005.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T02:45:57.922 INFO:teuthology.orchestra.run.smithi005.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T02:45:57.928 INFO:teuthology.orchestra.run.smithi148.stderr:bash: line 1: ntpq: command not found 2024-09-12T02:45:57.931 INFO:teuthology.orchestra.run.smithi148.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-12T02:45:57.931 INFO:teuthology.orchestra.run.smithi148.stdout:=============================================================================== 2024-09-12T02:45:57.931 INFO:teuthology.orchestra.run.smithi148.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T02:45:57.932 INFO:teuthology.orchestra.run.smithi148.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T02:45:57.932 INFO:teuthology.orchestra.run.smithi148.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T02:45:57.932 INFO:teuthology.orchestra.run.smithi148.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T02:45:57.933 INFO:teuthology.run_tasks:Running task install... 2024-09-12T02:45:57.941 DEBUG:teuthology.task.install:project ceph 2024-09-12T02:45:57.941 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc'}} 2024-09-12T02:45:57.941 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc'} 2024-09-12T02:45:57.941 INFO:teuthology.task.install:Using flavor: default 2024-09-12T02:45:57.948 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-12T02:45:57.948 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-12T02:45:57.948 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'tag': None, 'wait_for_package': False} 2024-09-12T02:45:57.949 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T02:45:57.951 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'tag': None, 'wait_for_package': False} 2024-09-12T02:45:57.951 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T02:45:57.952 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'tag': None, 'wait_for_package': False} 2024-09-12T02:45:57.953 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T02:45:58.106 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/ 2024-09-12T02:45:58.106 INFO:teuthology.task.install.rpm:Package version is 19.1.1-298.gb53b6d3e 2024-09-12T02:45:58.109 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/ 2024-09-12T02:45:58.109 INFO:teuthology.task.install.rpm:Package version is 19.1.1-298.gb53b6d3e 2024-09-12T02:45:58.117 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/ 2024-09-12T02:45:58.117 INFO:teuthology.task.install.rpm:Package version is 19.1.1-298.gb53b6d3e 2024-09-12T02:45:58.220 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-12T02:45:58.220 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-09-12T02:45:58.220 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-12T02:45:58.228 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-12T02:45:58.228 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-12T02:45:58.228 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-12T02:45:58.232 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-12T02:45:58.232 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-12T02:45:58.233 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-12T02:45:58.248 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-12T02:45:58.249 DEBUG:teuthology.orchestra.run.smithi005:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-12T02:45:58.253 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-12T02:45:58.253 DEBUG:teuthology.orchestra.run.smithi148:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-12T02:45:58.258 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-12T02:45:58.258 DEBUG:teuthology.orchestra.run.smithi146:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-12T02:45:58.316 DEBUG:teuthology.orchestra.run.smithi005:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-12T02:45:58.321 DEBUG:teuthology.orchestra.run.smithi148:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-12T02:45:58.323 DEBUG:teuthology.orchestra.run.smithi146:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-12T02:45:58.394 DEBUG:teuthology.orchestra.run.smithi005:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-12T02:45:58.395 DEBUG:teuthology.orchestra.run.smithi148:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-12T02:45:58.398 DEBUG:teuthology.orchestra.run.smithi146:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-12T02:45:58.460 INFO:teuthology.orchestra.run.smithi148.stdout:check_obsoletes = 1 2024-09-12T02:45:58.460 INFO:teuthology.orchestra.run.smithi005.stdout:check_obsoletes = 1 2024-09-12T02:45:58.462 DEBUG:teuthology.orchestra.run.smithi005:> sudo yum clean all 2024-09-12T02:45:58.462 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum clean all 2024-09-12T02:45:58.463 INFO:teuthology.orchestra.run.smithi146.stdout:check_obsoletes = 1 2024-09-12T02:45:58.464 DEBUG:teuthology.orchestra.run.smithi146:> sudo yum clean all 2024-09-12T02:45:58.688 INFO:teuthology.orchestra.run.smithi148.stdout:45 files removed 2024-09-12T02:45:58.692 INFO:teuthology.orchestra.run.smithi146.stdout:45 files removed 2024-09-12T02:45:58.693 INFO:teuthology.orchestra.run.smithi005.stdout:45 files removed 2024-09-12T02:45:58.715 DEBUG:teuthology.orchestra.run.smithi148:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-12T02:45:58.719 DEBUG:teuthology.orchestra.run.smithi146:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-12T02:45:58.720 DEBUG:teuthology.orchestra.run.smithi005:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-12T02:45:59.270 INFO:teuthology.orchestra.run.smithi148.stdout:ceph packages for x86_64 303 kB/s | 84 kB 00:00 2024-09-12T02:45:59.276 INFO:teuthology.orchestra.run.smithi146.stdout:ceph packages for x86_64 302 kB/s | 84 kB 00:00 2024-09-12T02:45:59.296 INFO:teuthology.orchestra.run.smithi005.stdout:ceph packages for x86_64 303 kB/s | 84 kB 00:00 2024-09-12T02:45:59.546 INFO:teuthology.orchestra.run.smithi148.stdout:ceph noarch packages 50 kB/s | 12 kB 00:00 2024-09-12T02:45:59.569 INFO:teuthology.orchestra.run.smithi146.stdout:ceph noarch packages 47 kB/s | 12 kB 00:00 2024-09-12T02:45:59.588 INFO:teuthology.orchestra.run.smithi005.stdout:ceph noarch packages 48 kB/s | 12 kB 00:00 2024-09-12T02:45:59.822 INFO:teuthology.orchestra.run.smithi148.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-12T02:45:59.844 INFO:teuthology.orchestra.run.smithi146.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-12T02:45:59.863 INFO:teuthology.orchestra.run.smithi005.stdout:ceph source packages 8.0 kB/s | 1.9 kB 00:00 2024-09-12T02:46:00.313 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 9 - BaseOS 18 MB/s | 8.2 MB 00:00 2024-09-12T02:46:01.644 INFO:teuthology.orchestra.run.smithi146.stdout:CentOS Stream 9 - BaseOS 4.7 MB/s | 8.2 MB 00:01 2024-09-12T02:46:01.797 INFO:teuthology.orchestra.run.smithi005.stdout:CentOS Stream 9 - BaseOS 4.3 MB/s | 8.2 MB 00:01 2024-09-12T02:46:04.470 INFO:teuthology.orchestra.run.smithi146.stdout:CentOS Stream 9 - AppStream 12 MB/s | 20 MB 00:01 2024-09-12T02:46:06.014 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 9 - AppStream 4.4 MB/s | 20 MB 00:04 2024-09-12T02:46:06.473 INFO:teuthology.orchestra.run.smithi005.stdout:CentOS Stream 9 - AppStream 5.7 MB/s | 20 MB 00:03 2024-09-12T02:46:08.721 INFO:teuthology.orchestra.run.smithi146.stdout:CentOS Stream 9 - CRB 15 MB/s | 6.5 MB 00:00 2024-09-12T02:46:10.340 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.5 MB 00:00 2024-09-12T02:46:10.613 INFO:teuthology.orchestra.run.smithi146.stdout:CentOS Stream 9 - Extras packages 27 kB/s | 19 kB 00:00 2024-09-12T02:46:10.840 INFO:teuthology.orchestra.run.smithi005.stdout:CentOS Stream 9 - CRB 16 MB/s | 6.5 MB 00:00 2024-09-12T02:46:11.493 INFO:teuthology.orchestra.run.smithi146.stdout:Extra Packages for Enterprise Linux 30 MB/s | 23 MB 00:00 2024-09-12T02:46:11.865 INFO:teuthology.orchestra.run.smithi148.stdout:CentOS Stream 9 - Extras packages 61 kB/s | 19 kB 00:00 2024-09-12T02:46:12.524 INFO:teuthology.orchestra.run.smithi005.stdout:CentOS Stream 9 - Extras packages 45 kB/s | 19 kB 00:00 2024-09-12T02:46:12.645 INFO:teuthology.orchestra.run.smithi148.stdout:Extra Packages for Enterprise Linux 34 MB/s | 23 MB 00:00 2024-09-12T02:46:13.329 INFO:teuthology.orchestra.run.smithi005.stdout:Extra Packages for Enterprise Linux 34 MB/s | 23 MB 00:00 2024-09-12T02:46:16.998 INFO:teuthology.orchestra.run.smithi146.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-09-12T02:46:18.093 INFO:teuthology.orchestra.run.smithi148.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-09-12T02:46:18.951 INFO:teuthology.orchestra.run.smithi005.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-09-12T02:46:19.325 INFO:teuthology.orchestra.run.smithi146.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-12T02:46:19.326 INFO:teuthology.orchestra.run.smithi146.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-12T02:46:19.337 INFO:teuthology.orchestra.run.smithi146.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-12T02:46:19.338 INFO:teuthology.orchestra.run.smithi146.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-12T02:46:19.339 INFO:teuthology.orchestra.run.smithi146.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-12T02:46:19.340 INFO:teuthology.orchestra.run.smithi146.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-12T02:46:19.341 INFO:teuthology.orchestra.run.smithi146.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-12T02:46:19.344 INFO:teuthology.orchestra.run.smithi146.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-12T02:46:19.345 INFO:teuthology.orchestra.run.smithi146.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-12T02:46:19.407 INFO:teuthology.orchestra.run.smithi146.stdout:Dependencies resolved. 2024-09-12T02:46:19.423 INFO:teuthology.orchestra.run.smithi146.stdout:====================================================================================== 2024-09-12T02:46:19.423 INFO:teuthology.orchestra.run.smithi146.stdout: Package Arch Version Repository Size 2024-09-12T02:46:19.423 INFO:teuthology.orchestra.run.smithi146.stdout:====================================================================================== 2024-09-12T02:46:19.423 INFO:teuthology.orchestra.run.smithi146.stdout:Installing: 2024-09-12T02:46:19.423 INFO:teuthology.orchestra.run.smithi146.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-12T02:46:19.423 INFO:teuthology.orchestra.run.smithi146.stdout: ceph x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 6.5 k 2024-09-12T02:46:19.423 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-base x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.4 M 2024-09-12T02:46:19.423 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 897 k 2024-09-12T02:46:19.424 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 145 k 2024-09-12T02:46:19.424 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 1.5 M 2024-09-12T02:46:19.424 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 146 k 2024-09-12T02:46:19.424 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-dashboard noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 3.6 M 2024-09-12T02:46:19.424 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 7.4 M 2024-09-12T02:46:19.424 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-rook noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 49 k 2024-09-12T02:46:19.424 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-radosgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 11 M 2024-09-12T02:46:19.424 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-test x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 49 M 2024-09-12T02:46:19.424 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-volume noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 285 k 2024-09-12T02:46:19.424 INFO:teuthology.orchestra.run.smithi146.stdout: cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 765 k 2024-09-12T02:46:19.425 INFO:teuthology.orchestra.run.smithi146.stdout: cephfs-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 228 k 2024-09-12T02:46:19.425 INFO:teuthology.orchestra.run.smithi146.stdout: cephfs-shell x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 27 k 2024-09-12T02:46:19.425 INFO:teuthology.orchestra.run.smithi146.stdout: cephfs-top noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 18 k 2024-09-12T02:46:19.425 INFO:teuthology.orchestra.run.smithi146.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-12T02:46:19.425 INFO:teuthology.orchestra.run.smithi146.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-12T02:46:19.425 INFO:teuthology.orchestra.run.smithi146.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-12T02:46:19.426 INFO:teuthology.orchestra.run.smithi146.stdout: libcephfs-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 32 k 2024-09-12T02:46:19.426 INFO:teuthology.orchestra.run.smithi146.stdout: libcephfs2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 752 k 2024-09-12T02:46:19.426 INFO:teuthology.orchestra.run.smithi146.stdout: librados-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 127 k 2024-09-12T02:46:19.426 INFO:teuthology.orchestra.run.smithi146.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-12T02:46:19.426 INFO:teuthology.orchestra.run.smithi146.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-12T02:46:19.426 INFO:teuthology.orchestra.run.smithi146.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-12T02:46:19.426 INFO:teuthology.orchestra.run.smithi146.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-12T02:46:19.426 INFO:teuthology.orchestra.run.smithi146.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-12T02:46:19.427 INFO:teuthology.orchestra.run.smithi146.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-12T02:46:19.427 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cephfs x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 162 k 2024-09-12T02:46:19.427 INFO:teuthology.orchestra.run.smithi146.stdout: python3-rados x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 321 k 2024-09-12T02:46:19.427 INFO:teuthology.orchestra.run.smithi146.stdout: python3-rbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 301 k 2024-09-12T02:46:19.427 INFO:teuthology.orchestra.run.smithi146.stdout: python3-rgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 100 k 2024-09-12T02:46:19.427 INFO:teuthology.orchestra.run.smithi146.stdout: rbd-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 85 k 2024-09-12T02:46:19.427 INFO:teuthology.orchestra.run.smithi146.stdout: rbd-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.1 M 2024-09-12T02:46:19.427 INFO:teuthology.orchestra.run.smithi146.stdout: rbd-nbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 171 k 2024-09-12T02:46:19.428 INFO:teuthology.orchestra.run.smithi146.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-12T02:46:19.428 INFO:teuthology.orchestra.run.smithi146.stdout:Upgrading: 2024-09-12T02:46:19.428 INFO:teuthology.orchestra.run.smithi146.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-12T02:46:19.428 INFO:teuthology.orchestra.run.smithi146.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-12T02:46:19.428 INFO:teuthology.orchestra.run.smithi146.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-12T02:46:19.428 INFO:teuthology.orchestra.run.smithi146.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-12T02:46:19.428 INFO:teuthology.orchestra.run.smithi146.stdout: librados2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.4 M 2024-09-12T02:46:19.428 INFO:teuthology.orchestra.run.smithi146.stdout: librbd1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.2 M 2024-09-12T02:46:19.428 INFO:teuthology.orchestra.run.smithi146.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-12T02:46:19.428 INFO:teuthology.orchestra.run.smithi146.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-12T02:46:19.428 INFO:teuthology.orchestra.run.smithi146.stdout:Installing dependencies: 2024-09-12T02:46:19.429 INFO:teuthology.orchestra.run.smithi146.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-12T02:46:19.429 INFO:teuthology.orchestra.run.smithi146.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-12T02:46:19.429 INFO:teuthology.orchestra.run.smithi146.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-12T02:46:19.429 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 21 M 2024-09-12T02:46:19.429 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-grafana-dashboards noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 32 k 2024-09-12T02:46:19.429 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mds x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 2.4 M 2024-09-12T02:46:19.429 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-modules-core noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 249 k 2024-09-12T02:46:19.429 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mon x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 4.7 M 2024-09-12T02:46:19.429 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-osd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 17 M 2024-09-12T02:46:19.429 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-prometheus-alerts noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 16 k 2024-09-12T02:46:19.429 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-selinux x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 25 k 2024-09-12T02:46:19.429 INFO:teuthology.orchestra.run.smithi146.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-12T02:46:19.430 INFO:teuthology.orchestra.run.smithi146.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-12T02:46:19.430 INFO:teuthology.orchestra.run.smithi146.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-12T02:46:19.430 INFO:teuthology.orchestra.run.smithi146.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-12T02:46:19.430 INFO:teuthology.orchestra.run.smithi146.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-12T02:46:19.430 INFO:teuthology.orchestra.run.smithi146.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-12T02:46:19.430 INFO:teuthology.orchestra.run.smithi146.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-12T02:46:19.430 INFO:teuthology.orchestra.run.smithi146.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-12T02:46:19.430 INFO:teuthology.orchestra.run.smithi146.stdout: libcephsqlite x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 163 k 2024-09-12T02:46:19.430 INFO:teuthology.orchestra.run.smithi146.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-12T02:46:19.430 INFO:teuthology.orchestra.run.smithi146.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-12T02:46:19.430 INFO:teuthology.orchestra.run.smithi146.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-12T02:46:19.430 INFO:teuthology.orchestra.run.smithi146.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-12T02:46:19.430 INFO:teuthology.orchestra.run.smithi146.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-12T02:46:19.431 INFO:teuthology.orchestra.run.smithi146.stdout: libradosstriper1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 506 k 2024-09-12T02:46:19.431 INFO:teuthology.orchestra.run.smithi146.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-12T02:46:19.431 INFO:teuthology.orchestra.run.smithi146.stdout: librgw2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.3 M 2024-09-12T02:46:19.431 INFO:teuthology.orchestra.run.smithi146.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-12T02:46:19.431 INFO:teuthology.orchestra.run.smithi146.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-12T02:46:19.431 INFO:teuthology.orchestra.run.smithi146.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-12T02:46:19.431 INFO:teuthology.orchestra.run.smithi146.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-12T02:46:19.431 INFO:teuthology.orchestra.run.smithi146.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-12T02:46:19.431 INFO:teuthology.orchestra.run.smithi146.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-12T02:46:19.431 INFO:teuthology.orchestra.run.smithi146.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-12T02:46:19.431 INFO:teuthology.orchestra.run.smithi146.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-12T02:46:19.431 INFO:teuthology.orchestra.run.smithi146.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-12T02:46:19.431 INFO:teuthology.orchestra.run.smithi146.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-12T02:46:19.431 INFO:teuthology.orchestra.run.smithi146.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-12T02:46:19.432 INFO:teuthology.orchestra.run.smithi146.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-12T02:46:19.432 INFO:teuthology.orchestra.run.smithi146.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-12T02:46:19.432 INFO:teuthology.orchestra.run.smithi146.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-12T02:46:19.432 INFO:teuthology.orchestra.run.smithi146.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-12T02:46:19.432 INFO:teuthology.orchestra.run.smithi146.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-12T02:46:19.432 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-12T02:46:19.432 INFO:teuthology.orchestra.run.smithi146.stdout: python3-ceph-argparse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 45 k 2024-09-12T02:46:19.432 INFO:teuthology.orchestra.run.smithi146.stdout: python3-ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 136 k 2024-09-12T02:46:19.432 INFO:teuthology.orchestra.run.smithi146.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-12T02:46:19.432 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-12T02:46:19.432 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-12T02:46:19.432 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-12T02:46:19.433 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-12T02:46:19.433 INFO:teuthology.orchestra.run.smithi146.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-12T02:46:19.433 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-12T02:46:19.433 INFO:teuthology.orchestra.run.smithi146.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-12T02:46:19.433 INFO:teuthology.orchestra.run.smithi146.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-12T02:46:19.433 INFO:teuthology.orchestra.run.smithi146.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-12T02:46:19.433 INFO:teuthology.orchestra.run.smithi146.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-12T02:46:19.433 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-12T02:46:19.433 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-12T02:46:19.433 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-12T02:46:19.433 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-12T02:46:19.434 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-12T02:46:19.434 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-12T02:46:19.434 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-12T02:46:19.434 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-12T02:46:19.434 INFO:teuthology.orchestra.run.smithi146.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-12T02:46:19.434 INFO:teuthology.orchestra.run.smithi146.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-12T02:46:19.434 INFO:teuthology.orchestra.run.smithi146.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-12T02:46:19.434 INFO:teuthology.orchestra.run.smithi146.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-12T02:46:19.434 INFO:teuthology.orchestra.run.smithi146.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-12T02:46:19.434 INFO:teuthology.orchestra.run.smithi146.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-12T02:46:19.434 INFO:teuthology.orchestra.run.smithi146.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-12T02:46:19.434 INFO:teuthology.orchestra.run.smithi146.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-12T02:46:19.435 INFO:teuthology.orchestra.run.smithi146.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-12T02:46:19.435 INFO:teuthology.orchestra.run.smithi146.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-12T02:46:19.435 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-12T02:46:19.435 INFO:teuthology.orchestra.run.smithi146.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-12T02:46:19.435 INFO:teuthology.orchestra.run.smithi146.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-12T02:46:19.435 INFO:teuthology.orchestra.run.smithi146.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-12T02:46:19.436 INFO:teuthology.orchestra.run.smithi146.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-12T02:46:19.436 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-12T02:46:19.436 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-12T02:46:19.436 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-12T02:46:19.436 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-12T02:46:19.437 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-12T02:46:19.437 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-12T02:46:19.437 INFO:teuthology.orchestra.run.smithi146.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-12T02:46:19.437 INFO:teuthology.orchestra.run.smithi146.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-12T02:46:19.437 INFO:teuthology.orchestra.run.smithi146.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-12T02:46:19.438 INFO:teuthology.orchestra.run.smithi146.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-12T02:46:19.438 INFO:teuthology.orchestra.run.smithi146.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-12T02:46:19.438 INFO:teuthology.orchestra.run.smithi146.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-12T02:46:19.438 INFO:teuthology.orchestra.run.smithi146.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-12T02:46:19.438 INFO:teuthology.orchestra.run.smithi146.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-12T02:46:19.438 INFO:teuthology.orchestra.run.smithi146.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-12T02:46:19.438 INFO:teuthology.orchestra.run.smithi146.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-12T02:46:19.439 INFO:teuthology.orchestra.run.smithi146.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-12T02:46:19.439 INFO:teuthology.orchestra.run.smithi146.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-12T02:46:19.439 INFO:teuthology.orchestra.run.smithi146.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-12T02:46:19.439 INFO:teuthology.orchestra.run.smithi146.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-12T02:46:19.439 INFO:teuthology.orchestra.run.smithi146.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-12T02:46:19.439 INFO:teuthology.orchestra.run.smithi146.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-12T02:46:19.439 INFO:teuthology.orchestra.run.smithi146.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-12T02:46:19.439 INFO:teuthology.orchestra.run.smithi146.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-12T02:46:19.439 INFO:teuthology.orchestra.run.smithi146.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-12T02:46:19.439 INFO:teuthology.orchestra.run.smithi146.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-12T02:46:19.439 INFO:teuthology.orchestra.run.smithi146.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-12T02:46:19.439 INFO:teuthology.orchestra.run.smithi146.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-12T02:46:19.439 INFO:teuthology.orchestra.run.smithi146.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-12T02:46:19.440 INFO:teuthology.orchestra.run.smithi146.stdout:Installing weak dependencies: 2024-09-12T02:46:19.440 INFO:teuthology.orchestra.run.smithi146.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-12T02:46:19.440 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-12T02:46:19.440 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction Summary 2024-09-12T02:46:19.440 INFO:teuthology.orchestra.run.smithi146.stdout:====================================================================================== 2024-09-12T02:46:19.440 INFO:teuthology.orchestra.run.smithi146.stdout:Install 148 Packages 2024-09-12T02:46:19.440 INFO:teuthology.orchestra.run.smithi146.stdout:Upgrade 8 Packages 2024-09-12T02:46:19.440 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-12T02:46:19.441 INFO:teuthology.orchestra.run.smithi146.stdout:Total download size: 226 M 2024-09-12T02:46:19.441 INFO:teuthology.orchestra.run.smithi146.stdout:Downloading Packages: 2024-09-12T02:46:20.091 INFO:teuthology.orchestra.run.smithi146.stdout:(1/156): ceph-19.1.1-298.gb53b6d3e.el9.x86_64.r 50 kB/s | 6.5 kB 00:00 2024-09-12T02:46:20.325 INFO:teuthology.orchestra.run.smithi146.stdout:(2/156): ceph-fuse-19.1.1-298.gb53b6d3e.el9.x86 3.8 MB/s | 897 kB 00:00 2024-09-12T02:46:20.370 INFO:teuthology.orchestra.run.smithi148.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-12T02:46:20.370 INFO:teuthology.orchestra.run.smithi148.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-12T02:46:20.382 INFO:teuthology.orchestra.run.smithi148.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-12T02:46:20.383 INFO:teuthology.orchestra.run.smithi148.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-12T02:46:20.383 INFO:teuthology.orchestra.run.smithi148.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-12T02:46:20.384 INFO:teuthology.orchestra.run.smithi148.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-12T02:46:20.386 INFO:teuthology.orchestra.run.smithi148.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-12T02:46:20.389 INFO:teuthology.orchestra.run.smithi148.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-12T02:46:20.389 INFO:teuthology.orchestra.run.smithi148.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-12T02:46:20.442 INFO:teuthology.orchestra.run.smithi146.stdout:(3/156): ceph-base-19.1.1-298.gb53b6d3e.el9.x86 11 MB/s | 5.4 MB 00:00 2024-09-12T02:46:20.452 INFO:teuthology.orchestra.run.smithi148.stdout:Dependencies resolved. 2024-09-12T02:46:20.467 INFO:teuthology.orchestra.run.smithi146.stdout:(4/156): ceph-immutable-object-cache-19.1.1-298 1.0 MB/s | 145 kB 00:00 2024-09-12T02:46:20.476 INFO:teuthology.orchestra.run.smithi148.stdout:====================================================================================== 2024-09-12T02:46:20.476 INFO:teuthology.orchestra.run.smithi148.stdout: Package Arch Version Repository Size 2024-09-12T02:46:20.477 INFO:teuthology.orchestra.run.smithi148.stdout:====================================================================================== 2024-09-12T02:46:20.477 INFO:teuthology.orchestra.run.smithi148.stdout:Installing: 2024-09-12T02:46:20.477 INFO:teuthology.orchestra.run.smithi148.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-12T02:46:20.477 INFO:teuthology.orchestra.run.smithi148.stdout: ceph x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 6.5 k 2024-09-12T02:46:20.477 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-base x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.4 M 2024-09-12T02:46:20.477 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 897 k 2024-09-12T02:46:20.477 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 145 k 2024-09-12T02:46:20.477 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 1.5 M 2024-09-12T02:46:20.477 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 146 k 2024-09-12T02:46:20.477 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-dashboard noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 3.6 M 2024-09-12T02:46:20.477 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 7.4 M 2024-09-12T02:46:20.477 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-rook noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 49 k 2024-09-12T02:46:20.478 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-radosgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 11 M 2024-09-12T02:46:20.478 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-test x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 49 M 2024-09-12T02:46:20.478 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-volume noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 285 k 2024-09-12T02:46:20.478 INFO:teuthology.orchestra.run.smithi148.stdout: cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 765 k 2024-09-12T02:46:20.478 INFO:teuthology.orchestra.run.smithi148.stdout: cephfs-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 228 k 2024-09-12T02:46:20.478 INFO:teuthology.orchestra.run.smithi148.stdout: cephfs-shell x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 27 k 2024-09-12T02:46:20.478 INFO:teuthology.orchestra.run.smithi148.stdout: cephfs-top noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 18 k 2024-09-12T02:46:20.478 INFO:teuthology.orchestra.run.smithi148.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-12T02:46:20.478 INFO:teuthology.orchestra.run.smithi148.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-12T02:46:20.478 INFO:teuthology.orchestra.run.smithi148.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-12T02:46:20.478 INFO:teuthology.orchestra.run.smithi148.stdout: libcephfs-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 32 k 2024-09-12T02:46:20.478 INFO:teuthology.orchestra.run.smithi148.stdout: libcephfs2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 752 k 2024-09-12T02:46:20.479 INFO:teuthology.orchestra.run.smithi148.stdout: librados-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 127 k 2024-09-12T02:46:20.479 INFO:teuthology.orchestra.run.smithi148.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-12T02:46:20.479 INFO:teuthology.orchestra.run.smithi148.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-12T02:46:20.479 INFO:teuthology.orchestra.run.smithi148.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-12T02:46:20.479 INFO:teuthology.orchestra.run.smithi148.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-12T02:46:20.479 INFO:teuthology.orchestra.run.smithi148.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-12T02:46:20.479 INFO:teuthology.orchestra.run.smithi148.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-12T02:46:20.479 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cephfs x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 162 k 2024-09-12T02:46:20.479 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rados x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 321 k 2024-09-12T02:46:20.479 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 301 k 2024-09-12T02:46:20.479 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 100 k 2024-09-12T02:46:20.479 INFO:teuthology.orchestra.run.smithi148.stdout: rbd-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 85 k 2024-09-12T02:46:20.479 INFO:teuthology.orchestra.run.smithi148.stdout: rbd-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.1 M 2024-09-12T02:46:20.479 INFO:teuthology.orchestra.run.smithi148.stdout: rbd-nbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 171 k 2024-09-12T02:46:20.480 INFO:teuthology.orchestra.run.smithi148.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-12T02:46:20.480 INFO:teuthology.orchestra.run.smithi148.stdout:Upgrading: 2024-09-12T02:46:20.480 INFO:teuthology.orchestra.run.smithi148.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-12T02:46:20.480 INFO:teuthology.orchestra.run.smithi148.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-12T02:46:20.480 INFO:teuthology.orchestra.run.smithi148.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-12T02:46:20.480 INFO:teuthology.orchestra.run.smithi148.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-12T02:46:20.480 INFO:teuthology.orchestra.run.smithi148.stdout: librados2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.4 M 2024-09-12T02:46:20.480 INFO:teuthology.orchestra.run.smithi148.stdout: librbd1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.2 M 2024-09-12T02:46:20.480 INFO:teuthology.orchestra.run.smithi148.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-12T02:46:20.480 INFO:teuthology.orchestra.run.smithi148.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-12T02:46:20.480 INFO:teuthology.orchestra.run.smithi148.stdout:Installing dependencies: 2024-09-12T02:46:20.481 INFO:teuthology.orchestra.run.smithi148.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-12T02:46:20.481 INFO:teuthology.orchestra.run.smithi148.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-12T02:46:20.481 INFO:teuthology.orchestra.run.smithi148.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-12T02:46:20.481 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 21 M 2024-09-12T02:46:20.481 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-grafana-dashboards noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 32 k 2024-09-12T02:46:20.481 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mds x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 2.4 M 2024-09-12T02:46:20.481 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-modules-core noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 249 k 2024-09-12T02:46:20.481 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mon x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 4.7 M 2024-09-12T02:46:20.481 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-osd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 17 M 2024-09-12T02:46:20.481 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-prometheus-alerts noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 16 k 2024-09-12T02:46:20.481 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-selinux x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 25 k 2024-09-12T02:46:20.481 INFO:teuthology.orchestra.run.smithi148.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-12T02:46:20.481 INFO:teuthology.orchestra.run.smithi148.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-12T02:46:20.482 INFO:teuthology.orchestra.run.smithi148.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-12T02:46:20.482 INFO:teuthology.orchestra.run.smithi148.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-12T02:46:20.482 INFO:teuthology.orchestra.run.smithi148.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-12T02:46:20.482 INFO:teuthology.orchestra.run.smithi148.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-12T02:46:20.482 INFO:teuthology.orchestra.run.smithi148.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-12T02:46:20.482 INFO:teuthology.orchestra.run.smithi148.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-12T02:46:20.482 INFO:teuthology.orchestra.run.smithi148.stdout: libcephsqlite x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 163 k 2024-09-12T02:46:20.482 INFO:teuthology.orchestra.run.smithi148.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-12T02:46:20.482 INFO:teuthology.orchestra.run.smithi148.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-12T02:46:20.482 INFO:teuthology.orchestra.run.smithi148.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-12T02:46:20.482 INFO:teuthology.orchestra.run.smithi148.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-12T02:46:20.483 INFO:teuthology.orchestra.run.smithi148.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-12T02:46:20.483 INFO:teuthology.orchestra.run.smithi148.stdout: libradosstriper1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 506 k 2024-09-12T02:46:20.483 INFO:teuthology.orchestra.run.smithi148.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-12T02:46:20.483 INFO:teuthology.orchestra.run.smithi148.stdout: librgw2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.3 M 2024-09-12T02:46:20.483 INFO:teuthology.orchestra.run.smithi148.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-12T02:46:20.483 INFO:teuthology.orchestra.run.smithi148.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-12T02:46:20.483 INFO:teuthology.orchestra.run.smithi148.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-12T02:46:20.483 INFO:teuthology.orchestra.run.smithi148.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-12T02:46:20.483 INFO:teuthology.orchestra.run.smithi148.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-12T02:46:20.483 INFO:teuthology.orchestra.run.smithi148.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-12T02:46:20.483 INFO:teuthology.orchestra.run.smithi148.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-12T02:46:20.483 INFO:teuthology.orchestra.run.smithi148.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-12T02:46:20.483 INFO:teuthology.orchestra.run.smithi148.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-12T02:46:20.484 INFO:teuthology.orchestra.run.smithi148.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-12T02:46:20.484 INFO:teuthology.orchestra.run.smithi148.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-12T02:46:20.484 INFO:teuthology.orchestra.run.smithi148.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-12T02:46:20.484 INFO:teuthology.orchestra.run.smithi148.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-12T02:46:20.484 INFO:teuthology.orchestra.run.smithi148.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-12T02:46:20.484 INFO:teuthology.orchestra.run.smithi148.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-12T02:46:20.484 INFO:teuthology.orchestra.run.smithi148.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-12T02:46:20.484 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-12T02:46:20.484 INFO:teuthology.orchestra.run.smithi148.stdout: python3-ceph-argparse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 45 k 2024-09-12T02:46:20.484 INFO:teuthology.orchestra.run.smithi148.stdout: python3-ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 136 k 2024-09-12T02:46:20.484 INFO:teuthology.orchestra.run.smithi148.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-12T02:46:20.484 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-12T02:46:20.484 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-12T02:46:20.485 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-12T02:46:20.485 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-12T02:46:20.485 INFO:teuthology.orchestra.run.smithi148.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-12T02:46:20.485 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-12T02:46:20.485 INFO:teuthology.orchestra.run.smithi148.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-12T02:46:20.485 INFO:teuthology.orchestra.run.smithi148.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-12T02:46:20.485 INFO:teuthology.orchestra.run.smithi148.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-12T02:46:20.485 INFO:teuthology.orchestra.run.smithi148.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-12T02:46:20.485 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-12T02:46:20.485 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-12T02:46:20.485 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-12T02:46:20.485 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-12T02:46:20.485 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-12T02:46:20.486 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-12T02:46:20.486 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-12T02:46:20.486 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-12T02:46:20.486 INFO:teuthology.orchestra.run.smithi148.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-12T02:46:20.486 INFO:teuthology.orchestra.run.smithi148.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-12T02:46:20.486 INFO:teuthology.orchestra.run.smithi148.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-12T02:46:20.486 INFO:teuthology.orchestra.run.smithi148.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-12T02:46:20.486 INFO:teuthology.orchestra.run.smithi148.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-12T02:46:20.486 INFO:teuthology.orchestra.run.smithi148.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-12T02:46:20.486 INFO:teuthology.orchestra.run.smithi148.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-12T02:46:20.486 INFO:teuthology.orchestra.run.smithi148.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-12T02:46:20.486 INFO:teuthology.orchestra.run.smithi148.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-12T02:46:20.486 INFO:teuthology.orchestra.run.smithi148.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-12T02:46:20.487 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-12T02:46:20.487 INFO:teuthology.orchestra.run.smithi148.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-12T02:46:20.487 INFO:teuthology.orchestra.run.smithi148.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-12T02:46:20.487 INFO:teuthology.orchestra.run.smithi148.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-12T02:46:20.487 INFO:teuthology.orchestra.run.smithi148.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-12T02:46:20.487 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-12T02:46:20.487 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-12T02:46:20.487 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-12T02:46:20.487 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-12T02:46:20.487 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-12T02:46:20.487 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-12T02:46:20.487 INFO:teuthology.orchestra.run.smithi148.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-12T02:46:20.487 INFO:teuthology.orchestra.run.smithi148.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-12T02:46:20.487 INFO:teuthology.orchestra.run.smithi148.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-12T02:46:20.488 INFO:teuthology.orchestra.run.smithi148.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-12T02:46:20.488 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-12T02:46:20.488 INFO:teuthology.orchestra.run.smithi148.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-12T02:46:20.488 INFO:teuthology.orchestra.run.smithi148.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-12T02:46:20.488 INFO:teuthology.orchestra.run.smithi148.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-12T02:46:20.488 INFO:teuthology.orchestra.run.smithi148.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-12T02:46:20.488 INFO:teuthology.orchestra.run.smithi148.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-12T02:46:20.488 INFO:teuthology.orchestra.run.smithi148.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-12T02:46:20.488 INFO:teuthology.orchestra.run.smithi148.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-12T02:46:20.488 INFO:teuthology.orchestra.run.smithi148.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-12T02:46:20.488 INFO:teuthology.orchestra.run.smithi148.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-12T02:46:20.488 INFO:teuthology.orchestra.run.smithi148.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-12T02:46:20.488 INFO:teuthology.orchestra.run.smithi148.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-12T02:46:20.489 INFO:teuthology.orchestra.run.smithi148.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-12T02:46:20.489 INFO:teuthology.orchestra.run.smithi148.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-12T02:46:20.489 INFO:teuthology.orchestra.run.smithi148.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-12T02:46:20.489 INFO:teuthology.orchestra.run.smithi148.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-12T02:46:20.489 INFO:teuthology.orchestra.run.smithi148.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-12T02:46:20.489 INFO:teuthology.orchestra.run.smithi148.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-12T02:46:20.489 INFO:teuthology.orchestra.run.smithi148.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-12T02:46:20.489 INFO:teuthology.orchestra.run.smithi148.stdout:Installing weak dependencies: 2024-09-12T02:46:20.489 INFO:teuthology.orchestra.run.smithi148.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-12T02:46:20.489 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-12T02:46:20.489 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction Summary 2024-09-12T02:46:20.489 INFO:teuthology.orchestra.run.smithi148.stdout:====================================================================================== 2024-09-12T02:46:20.489 INFO:teuthology.orchestra.run.smithi148.stdout:Install 148 Packages 2024-09-12T02:46:20.489 INFO:teuthology.orchestra.run.smithi148.stdout:Upgrade 8 Packages 2024-09-12T02:46:20.490 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-12T02:46:20.490 INFO:teuthology.orchestra.run.smithi148.stdout:Total download size: 226 M 2024-09-12T02:46:20.490 INFO:teuthology.orchestra.run.smithi148.stdout:Downloading Packages: 2024-09-12T02:46:20.551 INFO:teuthology.orchestra.run.smithi146.stdout:(5/156): ceph-mds-19.1.1-298.gb53b6d3e.el9.x86_ 22 MB/s | 2.4 MB 00:00 2024-09-12T02:46:20.684 INFO:teuthology.orchestra.run.smithi146.stdout:(6/156): ceph-mgr-19.1.1-298.gb53b6d3e.el9.x86_ 7.0 MB/s | 1.5 MB 00:00 2024-09-12T02:46:20.818 INFO:teuthology.orchestra.run.smithi146.stdout:(7/156): ceph-mon-19.1.1-298.gb53b6d3e.el9.x86_ 18 MB/s | 4.7 MB 00:00 2024-09-12T02:46:21.110 INFO:teuthology.orchestra.run.smithi146.stdout:(8/156): ceph-common-19.1.1-298.gb53b6d3e.el9.x 18 MB/s | 21 MB 00:01 2024-09-12T02:46:21.152 INFO:teuthology.orchestra.run.smithi146.stdout:(9/156): ceph-selinux-19.1.1-298.gb53b6d3e.el9. 608 kB/s | 25 kB 00:00 2024-09-12T02:46:21.363 INFO:teuthology.orchestra.run.smithi005.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-12T02:46:21.364 INFO:teuthology.orchestra.run.smithi005.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-12T02:46:21.376 INFO:teuthology.orchestra.run.smithi005.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-12T02:46:21.377 INFO:teuthology.orchestra.run.smithi005.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-12T02:46:21.378 INFO:teuthology.orchestra.run.smithi005.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-12T02:46:21.378 INFO:teuthology.orchestra.run.smithi005.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-12T02:46:21.380 INFO:teuthology.orchestra.run.smithi005.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-12T02:46:21.383 INFO:teuthology.orchestra.run.smithi005.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-12T02:46:21.384 INFO:teuthology.orchestra.run.smithi005.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-12T02:46:21.444 INFO:teuthology.orchestra.run.smithi146.stdout:(10/156): ceph-radosgw-19.1.1-298.gb53b6d3e.el9 17 MB/s | 11 MB 00:00 2024-09-12T02:46:21.450 INFO:teuthology.orchestra.run.smithi005.stdout:Dependencies resolved. 2024-09-12T02:46:21.460 INFO:teuthology.orchestra.run.smithi148.stdout:(1/156): ceph-19.1.1-298.gb53b6d3e.el9.x86_64.r 28 kB/s | 6.5 kB 00:00 2024-09-12T02:46:21.471 INFO:teuthology.orchestra.run.smithi005.stdout:====================================================================================== 2024-09-12T02:46:21.471 INFO:teuthology.orchestra.run.smithi005.stdout: Package Arch Version Repository Size 2024-09-12T02:46:21.471 INFO:teuthology.orchestra.run.smithi005.stdout:====================================================================================== 2024-09-12T02:46:21.472 INFO:teuthology.orchestra.run.smithi005.stdout:Installing: 2024-09-12T02:46:21.472 INFO:teuthology.orchestra.run.smithi005.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-12T02:46:21.472 INFO:teuthology.orchestra.run.smithi005.stdout: ceph x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 6.5 k 2024-09-12T02:46:21.472 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-base x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.4 M 2024-09-12T02:46:21.472 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 897 k 2024-09-12T02:46:21.472 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 145 k 2024-09-12T02:46:21.472 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 1.5 M 2024-09-12T02:46:21.472 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 146 k 2024-09-12T02:46:21.472 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-dashboard noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 3.6 M 2024-09-12T02:46:21.472 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 7.4 M 2024-09-12T02:46:21.472 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-rook noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 49 k 2024-09-12T02:46:21.472 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-radosgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 11 M 2024-09-12T02:46:21.473 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-test x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 49 M 2024-09-12T02:46:21.473 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-volume noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 285 k 2024-09-12T02:46:21.473 INFO:teuthology.orchestra.run.smithi005.stdout: cephadm noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 765 k 2024-09-12T02:46:21.473 INFO:teuthology.orchestra.run.smithi005.stdout: cephfs-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 228 k 2024-09-12T02:46:21.473 INFO:teuthology.orchestra.run.smithi005.stdout: cephfs-shell x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 27 k 2024-09-12T02:46:21.473 INFO:teuthology.orchestra.run.smithi005.stdout: cephfs-top noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 18 k 2024-09-12T02:46:21.473 INFO:teuthology.orchestra.run.smithi005.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-12T02:46:21.473 INFO:teuthology.orchestra.run.smithi005.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-12T02:46:21.473 INFO:teuthology.orchestra.run.smithi005.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-12T02:46:21.473 INFO:teuthology.orchestra.run.smithi005.stdout: libcephfs-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 32 k 2024-09-12T02:46:21.473 INFO:teuthology.orchestra.run.smithi005.stdout: libcephfs2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 752 k 2024-09-12T02:46:21.473 INFO:teuthology.orchestra.run.smithi005.stdout: librados-devel x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 127 k 2024-09-12T02:46:21.474 INFO:teuthology.orchestra.run.smithi005.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-12T02:46:21.474 INFO:teuthology.orchestra.run.smithi005.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-12T02:46:21.474 INFO:teuthology.orchestra.run.smithi005.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-12T02:46:21.474 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-12T02:46:21.474 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-12T02:46:21.474 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-12T02:46:21.474 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cephfs x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 162 k 2024-09-12T02:46:21.474 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rados x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 321 k 2024-09-12T02:46:21.474 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 301 k 2024-09-12T02:46:21.474 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rgw x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 100 k 2024-09-12T02:46:21.474 INFO:teuthology.orchestra.run.smithi005.stdout: rbd-fuse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 85 k 2024-09-12T02:46:21.474 INFO:teuthology.orchestra.run.smithi005.stdout: rbd-mirror x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.1 M 2024-09-12T02:46:21.474 INFO:teuthology.orchestra.run.smithi005.stdout: rbd-nbd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 171 k 2024-09-12T02:46:21.475 INFO:teuthology.orchestra.run.smithi005.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-12T02:46:21.475 INFO:teuthology.orchestra.run.smithi005.stdout:Upgrading: 2024-09-12T02:46:21.475 INFO:teuthology.orchestra.run.smithi005.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-12T02:46:21.475 INFO:teuthology.orchestra.run.smithi005.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-12T02:46:21.475 INFO:teuthology.orchestra.run.smithi005.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-12T02:46:21.475 INFO:teuthology.orchestra.run.smithi005.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-12T02:46:21.475 INFO:teuthology.orchestra.run.smithi005.stdout: librados2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.4 M 2024-09-12T02:46:21.475 INFO:teuthology.orchestra.run.smithi005.stdout: librbd1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 3.2 M 2024-09-12T02:46:21.475 INFO:teuthology.orchestra.run.smithi005.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-12T02:46:21.475 INFO:teuthology.orchestra.run.smithi005.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-12T02:46:21.475 INFO:teuthology.orchestra.run.smithi005.stdout:Installing dependencies: 2024-09-12T02:46:21.475 INFO:teuthology.orchestra.run.smithi005.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-12T02:46:21.475 INFO:teuthology.orchestra.run.smithi005.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-12T02:46:21.475 INFO:teuthology.orchestra.run.smithi005.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-12T02:46:21.476 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 21 M 2024-09-12T02:46:21.476 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-grafana-dashboards noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 32 k 2024-09-12T02:46:21.476 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mds x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 2.4 M 2024-09-12T02:46:21.476 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-modules-core noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 249 k 2024-09-12T02:46:21.476 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mon x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 4.7 M 2024-09-12T02:46:21.476 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-osd x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 17 M 2024-09-12T02:46:21.476 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-prometheus-alerts noarch 2:19.1.1-298.gb53b6d3e.el9 ceph-noarch 16 k 2024-09-12T02:46:21.476 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-selinux x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 25 k 2024-09-12T02:46:21.476 INFO:teuthology.orchestra.run.smithi005.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-12T02:46:21.476 INFO:teuthology.orchestra.run.smithi005.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-12T02:46:21.476 INFO:teuthology.orchestra.run.smithi005.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-12T02:46:21.476 INFO:teuthology.orchestra.run.smithi005.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-12T02:46:21.477 INFO:teuthology.orchestra.run.smithi005.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-12T02:46:21.477 INFO:teuthology.orchestra.run.smithi005.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-12T02:46:21.477 INFO:teuthology.orchestra.run.smithi005.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-12T02:46:21.477 INFO:teuthology.orchestra.run.smithi005.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-12T02:46:21.477 INFO:teuthology.orchestra.run.smithi005.stdout: libcephsqlite x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 163 k 2024-09-12T02:46:21.477 INFO:teuthology.orchestra.run.smithi005.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-12T02:46:21.477 INFO:teuthology.orchestra.run.smithi005.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-12T02:46:21.477 INFO:teuthology.orchestra.run.smithi005.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-12T02:46:21.477 INFO:teuthology.orchestra.run.smithi005.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-12T02:46:21.477 INFO:teuthology.orchestra.run.smithi005.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-12T02:46:21.477 INFO:teuthology.orchestra.run.smithi005.stdout: libradosstriper1 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 506 k 2024-09-12T02:46:21.477 INFO:teuthology.orchestra.run.smithi005.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-12T02:46:21.477 INFO:teuthology.orchestra.run.smithi005.stdout: librgw2 x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 5.3 M 2024-09-12T02:46:21.478 INFO:teuthology.orchestra.run.smithi005.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-12T02:46:21.478 INFO:teuthology.orchestra.run.smithi005.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-12T02:46:21.478 INFO:teuthology.orchestra.run.smithi005.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-12T02:46:21.478 INFO:teuthology.orchestra.run.smithi005.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-12T02:46:21.478 INFO:teuthology.orchestra.run.smithi005.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-12T02:46:21.478 INFO:teuthology.orchestra.run.smithi005.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-12T02:46:21.478 INFO:teuthology.orchestra.run.smithi005.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-12T02:46:21.478 INFO:teuthology.orchestra.run.smithi005.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-12T02:46:21.478 INFO:teuthology.orchestra.run.smithi005.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-12T02:46:21.478 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-12T02:46:21.478 INFO:teuthology.orchestra.run.smithi005.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-12T02:46:21.478 INFO:teuthology.orchestra.run.smithi005.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-12T02:46:21.478 INFO:teuthology.orchestra.run.smithi005.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-12T02:46:21.478 INFO:teuthology.orchestra.run.smithi005.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-12T02:46:21.479 INFO:teuthology.orchestra.run.smithi005.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-12T02:46:21.479 INFO:teuthology.orchestra.run.smithi005.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-12T02:46:21.479 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-12T02:46:21.479 INFO:teuthology.orchestra.run.smithi005.stdout: python3-ceph-argparse x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 45 k 2024-09-12T02:46:21.479 INFO:teuthology.orchestra.run.smithi005.stdout: python3-ceph-common x86_64 2:19.1.1-298.gb53b6d3e.el9 ceph 136 k 2024-09-12T02:46:21.479 INFO:teuthology.orchestra.run.smithi005.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-12T02:46:21.479 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-12T02:46:21.479 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-12T02:46:21.479 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-12T02:46:21.479 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-12T02:46:21.479 INFO:teuthology.orchestra.run.smithi005.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-12T02:46:21.479 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-12T02:46:21.479 INFO:teuthology.orchestra.run.smithi005.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-12T02:46:21.480 INFO:teuthology.orchestra.run.smithi005.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-12T02:46:21.480 INFO:teuthology.orchestra.run.smithi005.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-12T02:46:21.480 INFO:teuthology.orchestra.run.smithi005.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-12T02:46:21.480 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-12T02:46:21.480 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-12T02:46:21.480 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-12T02:46:21.480 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-12T02:46:21.480 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-12T02:46:21.480 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-12T02:46:21.480 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-12T02:46:21.480 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-12T02:46:21.481 INFO:teuthology.orchestra.run.smithi005.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-12T02:46:21.481 INFO:teuthology.orchestra.run.smithi005.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-12T02:46:21.481 INFO:teuthology.orchestra.run.smithi005.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-12T02:46:21.481 INFO:teuthology.orchestra.run.smithi005.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-12T02:46:21.481 INFO:teuthology.orchestra.run.smithi005.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-12T02:46:21.481 INFO:teuthology.orchestra.run.smithi005.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-12T02:46:21.481 INFO:teuthology.orchestra.run.smithi005.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-12T02:46:21.482 INFO:teuthology.orchestra.run.smithi005.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-12T02:46:21.482 INFO:teuthology.orchestra.run.smithi005.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-12T02:46:21.482 INFO:teuthology.orchestra.run.smithi005.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-12T02:46:21.482 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-12T02:46:21.482 INFO:teuthology.orchestra.run.smithi005.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-12T02:46:21.482 INFO:teuthology.orchestra.run.smithi005.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-12T02:46:21.482 INFO:teuthology.orchestra.run.smithi005.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-12T02:46:21.482 INFO:teuthology.orchestra.run.smithi005.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-12T02:46:21.482 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-12T02:46:21.482 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-12T02:46:21.482 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-12T02:46:21.482 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-12T02:46:21.482 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-12T02:46:21.483 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-12T02:46:21.483 INFO:teuthology.orchestra.run.smithi005.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-12T02:46:21.483 INFO:teuthology.orchestra.run.smithi005.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-12T02:46:21.483 INFO:teuthology.orchestra.run.smithi005.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-12T02:46:21.483 INFO:teuthology.orchestra.run.smithi005.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-12T02:46:21.483 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-12T02:46:21.483 INFO:teuthology.orchestra.run.smithi005.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-12T02:46:21.483 INFO:teuthology.orchestra.run.smithi005.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-12T02:46:21.483 INFO:teuthology.orchestra.run.smithi005.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-12T02:46:21.483 INFO:teuthology.orchestra.run.smithi005.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-12T02:46:21.483 INFO:teuthology.orchestra.run.smithi005.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-12T02:46:21.483 INFO:teuthology.orchestra.run.smithi005.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-12T02:46:21.483 INFO:teuthology.orchestra.run.smithi005.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-12T02:46:21.484 INFO:teuthology.orchestra.run.smithi005.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-12T02:46:21.484 INFO:teuthology.orchestra.run.smithi005.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-12T02:46:21.484 INFO:teuthology.orchestra.run.smithi005.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-12T02:46:21.484 INFO:teuthology.orchestra.run.smithi005.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-12T02:46:21.484 INFO:teuthology.orchestra.run.smithi005.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-12T02:46:21.484 INFO:teuthology.orchestra.run.smithi005.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-12T02:46:21.484 INFO:teuthology.orchestra.run.smithi005.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-12T02:46:21.484 INFO:teuthology.orchestra.run.smithi005.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-12T02:46:21.484 INFO:teuthology.orchestra.run.smithi005.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-12T02:46:21.484 INFO:teuthology.orchestra.run.smithi005.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-12T02:46:21.484 INFO:teuthology.orchestra.run.smithi005.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-12T02:46:21.484 INFO:teuthology.orchestra.run.smithi005.stdout:Installing weak dependencies: 2024-09-12T02:46:21.484 INFO:teuthology.orchestra.run.smithi005.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-12T02:46:21.485 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-12T02:46:21.485 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction Summary 2024-09-12T02:46:21.485 INFO:teuthology.orchestra.run.smithi005.stdout:====================================================================================== 2024-09-12T02:46:21.485 INFO:teuthology.orchestra.run.smithi005.stdout:Install 148 Packages 2024-09-12T02:46:21.485 INFO:teuthology.orchestra.run.smithi005.stdout:Upgrade 8 Packages 2024-09-12T02:46:21.485 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-12T02:46:21.485 INFO:teuthology.orchestra.run.smithi005.stdout:Total download size: 226 M 2024-09-12T02:46:21.485 INFO:teuthology.orchestra.run.smithi005.stdout:Downloading Packages: 2024-09-12T02:46:21.511 INFO:teuthology.orchestra.run.smithi146.stdout:(11/156): cephfs-mirror-19.1.1-298.gb53b6d3e.el 3.4 MB/s | 228 kB 00:00 2024-09-12T02:46:21.719 INFO:teuthology.orchestra.run.smithi146.stdout:(12/156): ceph-osd-19.1.1-298.gb53b6d3e.el9.x86 16 MB/s | 17 MB 00:01 2024-09-12T02:46:21.736 INFO:teuthology.orchestra.run.smithi146.stdout:(13/156): cephfs-shell-19.1.1-298.gb53b6d3e.el9 120 kB/s | 27 kB 00:00 2024-09-12T02:46:21.752 INFO:teuthology.orchestra.run.smithi146.stdout:(14/156): libcephfs-devel-19.1.1-298.gb53b6d3e. 954 kB/s | 32 kB 00:00 2024-09-12T02:46:21.803 INFO:teuthology.orchestra.run.smithi146.stdout:(15/156): libcephfs2-19.1.1-298.gb53b6d3e.el9.x 11 MB/s | 752 kB 00:00 2024-09-12T02:46:21.828 INFO:teuthology.orchestra.run.smithi146.stdout:(16/156): libcephsqlite-19.1.1-298.gb53b6d3e.el 2.1 MB/s | 163 kB 00:00 2024-09-12T02:46:21.869 INFO:teuthology.orchestra.run.smithi148.stdout:(2/156): ceph-fuse-19.1.1-298.gb53b6d3e.el9.x86 2.1 MB/s | 897 kB 00:00 2024-09-12T02:46:21.869 INFO:teuthology.orchestra.run.smithi146.stdout:(17/156): librados-devel-19.1.1-298.gb53b6d3e.e 1.9 MB/s | 127 kB 00:00 2024-09-12T02:46:21.903 INFO:teuthology.orchestra.run.smithi146.stdout:(18/156): libradosstriper1-19.1.1-298.gb53b6d3e 6.7 MB/s | 506 kB 00:00 2024-09-12T02:46:21.969 INFO:teuthology.orchestra.run.smithi148.stdout:(3/156): ceph-immutable-object-cache-19.1.1-298 1.4 MB/s | 145 kB 00:00 2024-09-12T02:46:21.978 INFO:teuthology.orchestra.run.smithi146.stdout:(19/156): python3-ceph-argparse-19.1.1-298.gb53 602 kB/s | 45 kB 00:00 2024-09-12T02:46:22.070 INFO:teuthology.orchestra.run.smithi146.stdout:(20/156): python3-ceph-common-19.1.1-298.gb53b6 1.4 MB/s | 136 kB 00:00 2024-09-12T02:46:22.187 INFO:teuthology.orchestra.run.smithi146.stdout:(21/156): librgw2-19.1.1-298.gb53b6d3e.el9.x86_ 17 MB/s | 5.3 MB 00:00 2024-09-12T02:46:22.213 INFO:teuthology.orchestra.run.smithi146.stdout:(22/156): python3-cephfs-19.1.1-298.gb53b6d3e.e 1.1 MB/s | 162 kB 00:00 2024-09-12T02:46:22.219 INFO:teuthology.orchestra.run.smithi005.stdout:(1/156): ceph-19.1.1-298.gb53b6d3e.el9.x86_64.r 27 kB/s | 6.5 kB 00:00 2024-09-12T02:46:22.277 INFO:teuthology.orchestra.run.smithi146.stdout:(23/156): python3-rados-19.1.1-298.gb53b6d3e.el 3.5 MB/s | 321 kB 00:00 2024-09-12T02:46:22.405 INFO:teuthology.orchestra.run.smithi146.stdout:(24/156): python3-rbd-19.1.1-298.gb53b6d3e.el9. 1.5 MB/s | 301 kB 00:00 2024-09-12T02:46:22.430 INFO:teuthology.orchestra.run.smithi005.stdout:(2/156): ceph-fuse-19.1.1-298.gb53b6d3e.el9.x86 4.2 MB/s | 897 kB 00:00 2024-09-12T02:46:22.430 INFO:teuthology.orchestra.run.smithi146.stdout:(25/156): python3-rgw-19.1.1-298.gb53b6d3e.el9. 650 kB/s | 100 kB 00:00 2024-09-12T02:46:22.456 INFO:teuthology.orchestra.run.smithi146.stdout:(26/156): rbd-fuse-19.1.1-298.gb53b6d3e.el9.x86 1.7 MB/s | 85 kB 00:00 2024-09-12T02:46:22.478 INFO:teuthology.orchestra.run.smithi148.stdout:(4/156): ceph-base-19.1.1-298.gb53b6d3e.el9.x86 4.3 MB/s | 5.4 MB 00:01 2024-09-12T02:46:22.480 INFO:teuthology.orchestra.run.smithi005.stdout:(3/156): ceph-immutable-object-cache-19.1.1-298 2.8 MB/s | 145 kB 00:00 2024-09-12T02:46:22.522 INFO:teuthology.orchestra.run.smithi146.stdout:(27/156): rbd-nbd-19.1.1-298.gb53b6d3e.el9.x86_ 2.5 MB/s | 171 kB 00:00 2024-09-12T02:46:22.586 INFO:teuthology.orchestra.run.smithi148.stdout:(5/156): ceph-mds-19.1.1-298.gb53b6d3e.el9.x86_ 3.9 MB/s | 2.4 MB 00:00 2024-09-12T02:46:22.589 INFO:teuthology.orchestra.run.smithi146.stdout:(28/156): rbd-mirror-19.1.1-298.gb53b6d3e.el9.x 20 MB/s | 3.1 MB 00:00 2024-09-12T02:46:22.614 INFO:teuthology.orchestra.run.smithi146.stdout:(29/156): ceph-grafana-dashboards-19.1.1-298.gb 347 kB/s | 32 kB 00:00 2024-09-12T02:46:22.673 INFO:teuthology.orchestra.run.smithi146.stdout:(30/156): ceph-mgr-cephadm-19.1.1-298.gb53b6d3e 1.7 MB/s | 146 kB 00:00 2024-09-12T02:46:22.872 INFO:teuthology.orchestra.run.smithi005.stdout:(4/156): ceph-mds-19.1.1-298.gb53b6d3e.el9.x86_ 6.1 MB/s | 2.4 MB 00:00 2024-09-12T02:46:22.887 INFO:teuthology.orchestra.run.smithi148.stdout:(6/156): ceph-mgr-19.1.1-298.gb53b6d3e.el9.x86_ 3.7 MB/s | 1.5 MB 00:00 2024-09-12T02:46:22.957 INFO:teuthology.orchestra.run.smithi146.stdout:(31/156): ceph-mgr-dashboard-19.1.1-298.gb53b6d 10 MB/s | 3.6 MB 00:00 2024-09-12T02:46:23.072 INFO:teuthology.orchestra.run.smithi005.stdout:(5/156): ceph-mgr-19.1.1-298.gb53b6d3e.el9.x86_ 7.6 MB/s | 1.5 MB 00:00 2024-09-12T02:46:23.082 INFO:teuthology.orchestra.run.smithi146.stdout:(32/156): ceph-mgr-modules-core-19.1.1-298.gb53 2.0 MB/s | 249 kB 00:00 2024-09-12T02:46:23.191 INFO:teuthology.orchestra.run.smithi146.stdout:(33/156): ceph-mgr-rook-19.1.1-298.gb53b6d3e.el 454 kB/s | 49 kB 00:00 2024-09-12T02:46:23.308 INFO:teuthology.orchestra.run.smithi146.stdout:(34/156): ceph-prometheus-alerts-19.1.1-298.gb5 141 kB/s | 16 kB 00:00 2024-09-12T02:46:23.408 INFO:teuthology.orchestra.run.smithi146.stdout:(35/156): ceph-volume-19.1.1-298.gb53b6d3e.el9. 2.8 MB/s | 285 kB 00:00 2024-09-12T02:46:23.673 INFO:teuthology.orchestra.run.smithi005.stdout:(6/156): ceph-mon-19.1.1-298.gb53b6d3e.el9.x86_ 7.8 MB/s | 4.7 MB 00:00 2024-09-12T02:46:23.756 INFO:teuthology.orchestra.run.smithi005.stdout:(7/156): ceph-base-19.1.1-298.gb53b6d3e.el9.x86 3.0 MB/s | 5.4 MB 00:01 2024-09-12T02:46:23.875 INFO:teuthology.orchestra.run.smithi146.stdout:(36/156): ceph-test-19.1.1-298.gb53b6d3e.el9.x8 18 MB/s | 49 MB 00:02 2024-09-12T02:46:23.917 INFO:teuthology.orchestra.run.smithi146.stdout:(37/156): cephadm-19.1.1-298.gb53b6d3e.el9.noar 1.5 MB/s | 765 kB 00:00 2024-09-12T02:46:23.937 INFO:teuthology.orchestra.run.smithi148.stdout:(7/156): ceph-mon-19.1.1-298.gb53b6d3e.el9.x86_ 3.5 MB/s | 4.7 MB 00:01 2024-09-12T02:46:23.942 INFO:teuthology.orchestra.run.smithi146.stdout:(38/156): cephfs-top-19.1.1-298.gb53b6d3e.el9.n 274 kB/s | 18 kB 00:00 2024-09-12T02:46:24.035 INFO:teuthology.orchestra.run.smithi146.stdout:(39/156): ceph-mgr-diskprediction-local-19.1.1- 5.4 MB/s | 7.4 MB 00:01 2024-09-12T02:46:24.134 INFO:teuthology.orchestra.run.smithi146.stdout:(40/156): libquadmath-11.5.0-2.el9.x86_64.rpm 1.9 MB/s | 191 kB 00:00 2024-09-12T02:46:24.151 INFO:teuthology.orchestra.run.smithi146.stdout:(41/156): fuse-2.9.9-16.el9.x86_64.rpm 341 kB/s | 80 kB 00:00 2024-09-12T02:46:24.193 INFO:teuthology.orchestra.run.smithi146.stdout:(42/156): libgfortran-11.5.0-2.el9.x86_64.rpm 3.1 MB/s | 801 kB 00:00 2024-09-12T02:46:24.218 INFO:teuthology.orchestra.run.smithi146.stdout:(43/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.0 MB/s | 253 kB 00:00 2024-09-12T02:46:24.220 INFO:teuthology.orchestra.run.smithi148.stdout:(8/156): ceph-common-19.1.1-298.gb53b6d3e.el9.x 7.1 MB/s | 21 MB 00:02 2024-09-12T02:46:24.270 INFO:teuthology.orchestra.run.smithi148.stdout:(9/156): ceph-selinux-19.1.1-298.gb53b6d3e.el9. 506 kB/s | 25 kB 00:00 2024-09-12T02:46:24.286 INFO:teuthology.orchestra.run.smithi146.stdout:(44/156): python3-cryptography-36.0.1-4.el9.x86 9.2 MB/s | 1.2 MB 00:00 2024-09-12T02:46:24.311 INFO:teuthology.orchestra.run.smithi146.stdout:(45/156): python3-ply-3.11-14.el9.noarch.rpm 909 kB/s | 106 kB 00:00 2024-09-12T02:46:24.336 INFO:teuthology.orchestra.run.smithi146.stdout:(46/156): python3-pycparser-2.20-6.el9.noarch.r 1.1 MB/s | 135 kB 00:00 2024-09-12T02:46:24.361 INFO:teuthology.orchestra.run.smithi146.stdout:(47/156): python3-requests-2.25.1-8.el9.noarch. 1.6 MB/s | 125 kB 00:00 2024-09-12T02:46:24.395 INFO:teuthology.orchestra.run.smithi146.stdout:(48/156): python3-urllib3-1.26.5-6.el9.noarch.r 2.5 MB/s | 215 kB 00:00 2024-09-12T02:46:25.006 INFO:teuthology.orchestra.run.smithi005.stdout:(8/156): ceph-osd-19.1.1-298.gb53b6d3e.el9.x86_ 13 MB/s | 17 MB 00:01 2024-09-12T02:46:25.081 INFO:teuthology.orchestra.run.smithi005.stdout:(9/156): ceph-selinux-19.1.1-298.gb53b6d3e.el9. 337 kB/s | 25 kB 00:00 2024-09-12T02:46:25.111 INFO:teuthology.orchestra.run.smithi146.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 149 kB/s | 107 kB 00:00 2024-09-12T02:46:25.220 INFO:teuthology.orchestra.run.smithi146.stdout:(50/156): automake-1.16.2-8.el9.noarch.rpm 779 kB/s | 689 kB 00:00 2024-09-12T02:46:25.262 INFO:teuthology.orchestra.run.smithi146.stdout:(51/156): bison-3.7.4-5.el9.x86_64.rpm 1.0 MB/s | 943 kB 00:00 2024-09-12T02:46:25.657 INFO:teuthology.orchestra.run.smithi005.stdout:(10/156): ceph-common-19.1.1-298.gb53b6d3e.el9. 5.8 MB/s | 21 MB 00:03 2024-09-12T02:46:25.729 INFO:teuthology.orchestra.run.smithi146.stdout:(52/156): elfutils-libelf-devel-0.191-4.el9.x86 35 kB/s | 22 kB 00:00 2024-09-12T02:46:25.804 INFO:teuthology.orchestra.run.smithi146.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 60 kB/s | 33 kB 00:00 2024-09-12T02:46:25.832 INFO:teuthology.orchestra.run.smithi005.stdout:(11/156): ceph-radosgw-19.1.1-298.gb53b6d3e.el9 5.1 MB/s | 11 MB 00:02 2024-09-12T02:46:25.857 INFO:teuthology.orchestra.run.smithi005.stdout:(12/156): cephfs-mirror-19.1.1-298.gb53b6d3e.el 1.1 MB/s | 228 kB 00:00 2024-09-12T02:46:25.882 INFO:teuthology.orchestra.run.smithi005.stdout:(13/156): cephfs-shell-19.1.1-298.gb53b6d3e.el9 540 kB/s | 27 kB 00:00 2024-09-12T02:46:25.908 INFO:teuthology.orchestra.run.smithi005.stdout:(14/156): libcephfs-devel-19.1.1-298.gb53b6d3e. 632 kB/s | 32 kB 00:00 2024-09-12T02:46:25.958 INFO:teuthology.orchestra.run.smithi005.stdout:(15/156): libcephsqlite-19.1.1-298.gb53b6d3e.el 3.2 MB/s | 163 kB 00:00 2024-09-12T02:46:25.983 INFO:teuthology.orchestra.run.smithi005.stdout:(16/156): libcephfs2-19.1.1-298.gb53b6d3e.el9.x 7.3 MB/s | 752 kB 00:00 2024-09-12T02:46:25.996 INFO:teuthology.orchestra.run.smithi146.stdout:(54/156): flex-2.6.4-9.el9.x86_64.rpm 404 kB/s | 313 kB 00:00 2024-09-12T02:46:26.000 INFO:teuthology.orchestra.run.smithi005.stdout:(17/156): librados-devel-19.1.1-298.gb53b6d3e.e 3.0 MB/s | 127 kB 00:00 2024-09-12T02:46:26.059 INFO:teuthology.orchestra.run.smithi005.stdout:(18/156): libradosstriper1-19.1.1-298.gb53b6d3e 6.6 MB/s | 506 kB 00:00 2024-09-12T02:46:26.109 INFO:teuthology.orchestra.run.smithi005.stdout:(19/156): python3-ceph-argparse-19.1.1-298.gb53 905 kB/s | 45 kB 00:00 2024-09-12T02:46:26.159 INFO:teuthology.orchestra.run.smithi005.stdout:(20/156): python3-ceph-common-19.1.1-298.gb53b6 2.7 MB/s | 136 kB 00:00 2024-09-12T02:46:26.210 INFO:teuthology.orchestra.run.smithi005.stdout:(21/156): python3-cephfs-19.1.1-298.gb53b6d3e.e 3.2 MB/s | 162 kB 00:00 2024-09-12T02:46:26.212 INFO:teuthology.orchestra.run.smithi148.stdout:(10/156): ceph-test-19.1.1-298.gb53b6d3e.el9.x8 25 MB/s | 49 MB 00:01 2024-09-12T02:46:26.271 INFO:teuthology.orchestra.run.smithi148.stdout:(11/156): cephfs-mirror-19.1.1-298.gb53b6d3e.el 3.8 MB/s | 228 kB 00:00 2024-09-12T02:46:26.271 INFO:teuthology.orchestra.run.smithi146.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 39 kB/s | 18 kB 00:00 2024-09-12T02:46:26.293 INFO:teuthology.orchestra.run.smithi005.stdout:(22/156): python3-rados-19.1.1-298.gb53b6d3e.el 3.8 MB/s | 321 kB 00:00 2024-09-12T02:46:26.321 INFO:teuthology.orchestra.run.smithi148.stdout:(12/156): cephfs-shell-19.1.1-298.gb53b6d3e.el9 545 kB/s | 27 kB 00:00 2024-09-12T02:46:26.369 INFO:teuthology.orchestra.run.smithi005.stdout:(23/156): python3-rbd-19.1.1-298.gb53b6d3e.el9. 3.9 MB/s | 301 kB 00:00 2024-09-12T02:46:26.419 INFO:teuthology.orchestra.run.smithi005.stdout:(24/156): python3-rgw-19.1.1-298.gb53b6d3e.el9. 1.9 MB/s | 100 kB 00:00 2024-09-12T02:46:26.455 INFO:teuthology.orchestra.run.smithi146.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 24 kB/s | 11 kB 00:00 2024-09-12T02:46:26.494 INFO:teuthology.orchestra.run.smithi005.stdout:(25/156): librgw2-19.1.1-298.gb53b6d3e.el9.x86_ 11 MB/s | 5.3 MB 00:00 2024-09-12T02:46:26.496 INFO:teuthology.orchestra.run.smithi148.stdout:(13/156): ceph-osd-19.1.1-298.gb53b6d3e.el9.x86 4.7 MB/s | 17 MB 00:03 2024-09-12T02:46:26.519 INFO:teuthology.orchestra.run.smithi005.stdout:(26/156): rbd-fuse-19.1.1-298.gb53b6d3e.el9.x86 852 kB/s | 85 kB 00:00 2024-09-12T02:46:26.521 INFO:teuthology.orchestra.run.smithi148.stdout:(14/156): libcephfs-devel-19.1.1-298.gb53b6d3e. 158 kB/s | 32 kB 00:00 2024-09-12T02:46:26.570 INFO:teuthology.orchestra.run.smithi005.stdout:(27/156): rbd-nbd-19.1.1-298.gb53b6d3e.el9.x86_ 3.3 MB/s | 171 kB 00:00 2024-09-12T02:46:26.620 INFO:teuthology.orchestra.run.smithi005.stdout:(28/156): ceph-grafana-dashboards-19.1.1-298.gb 636 kB/s | 32 kB 00:00 2024-09-12T02:46:26.639 INFO:teuthology.orchestra.run.smithi148.stdout:(15/156): ceph-radosgw-19.1.1-298.gb53b6d3e.el9 3.9 MB/s | 11 MB 00:02 2024-09-12T02:46:26.670 INFO:teuthology.orchestra.run.smithi005.stdout:(29/156): ceph-mgr-cephadm-19.1.1-298.gb53b6d3e 2.9 MB/s | 146 kB 00:00 2024-09-12T02:46:26.672 INFO:teuthology.orchestra.run.smithi148.stdout:(16/156): libcephsqlite-19.1.1-298.gb53b6d3e.el 1.1 MB/s | 163 kB 00:00 2024-09-12T02:46:26.706 INFO:teuthology.orchestra.run.smithi148.stdout:(17/156): libcephfs2-19.1.1-298.gb53b6d3e.el9.x 3.5 MB/s | 752 kB 00:00 2024-09-12T02:46:26.730 INFO:teuthology.orchestra.run.smithi148.stdout:(18/156): librados-devel-19.1.1-298.gb53b6d3e.e 1.3 MB/s | 127 kB 00:00 2024-09-12T02:46:26.764 INFO:teuthology.orchestra.run.smithi148.stdout:(19/156): libradosstriper1-19.1.1-298.gb53b6d3e 5.4 MB/s | 506 kB 00:00 2024-09-12T02:46:26.772 INFO:teuthology.orchestra.run.smithi146.stdout:(57/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 2.9 MB/s | 3.0 MB 00:01 2024-09-12T02:46:26.779 INFO:teuthology.orchestra.run.smithi005.stdout:(30/156): rbd-mirror-19.1.1-298.gb53b6d3e.el9.x 11 MB/s | 3.1 MB 00:00 2024-09-12T02:46:26.790 INFO:teuthology.orchestra.run.smithi148.stdout:(20/156): python3-ceph-argparse-19.1.1-298.gb53 766 kB/s | 45 kB 00:00 2024-09-12T02:46:26.832 INFO:teuthology.orchestra.run.smithi148.stdout:(21/156): python3-ceph-common-19.1.1-298.gb53b6 2.0 MB/s | 136 kB 00:00 2024-09-12T02:46:26.849 INFO:teuthology.orchestra.run.smithi148.stdout:(22/156): python3-cephfs-19.1.1-298.gb53b6d3e.e 2.7 MB/s | 162 kB 00:00 2024-09-12T02:46:26.891 INFO:teuthology.orchestra.run.smithi148.stdout:(23/156): python3-rados-19.1.1-298.gb53b6d3e.el 5.4 MB/s | 321 kB 00:00 2024-09-12T02:46:26.941 INFO:teuthology.orchestra.run.smithi148.stdout:(24/156): python3-rbd-19.1.1-298.gb53b6d3e.el9. 3.2 MB/s | 301 kB 00:00 2024-09-12T02:46:26.966 INFO:teuthology.orchestra.run.smithi148.stdout:(25/156): python3-rgw-19.1.1-298.gb53b6d3e.el9. 1.3 MB/s | 100 kB 00:00 2024-09-12T02:46:26.991 INFO:teuthology.orchestra.run.smithi148.stdout:(26/156): rbd-fuse-19.1.1-298.gb53b6d3e.el9.x86 1.7 MB/s | 85 kB 00:00 2024-09-12T02:46:26.997 INFO:teuthology.orchestra.run.smithi146.stdout:(58/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 221 kB/s | 160 kB 00:00 2024-09-12T02:46:27.056 INFO:teuthology.orchestra.run.smithi146.stdout:(59/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 75 kB/s | 45 kB 00:00 2024-09-12T02:46:27.083 INFO:teuthology.orchestra.run.smithi148.stdout:(27/156): rbd-nbd-19.1.1-298.gb53b6d3e.el9.x86_ 1.8 MB/s | 171 kB 00:00 2024-09-12T02:46:27.150 INFO:teuthology.orchestra.run.smithi148.stdout:(28/156): rbd-mirror-19.1.1-298.gb53b6d3e.el9.x 17 MB/s | 3.1 MB 00:00 2024-09-12T02:46:27.171 INFO:teuthology.orchestra.run.smithi005.stdout:(31/156): ceph-mgr-dashboard-19.1.1-298.gb53b6d 7.1 MB/s | 3.6 MB 00:00 2024-09-12T02:46:27.175 INFO:teuthology.orchestra.run.smithi148.stdout:(29/156): ceph-grafana-dashboards-19.1.1-298.gb 347 kB/s | 32 kB 00:00 2024-09-12T02:46:27.201 INFO:teuthology.orchestra.run.smithi148.stdout:(30/156): ceph-mgr-cephadm-19.1.1-298.gb53b6d3e 2.9 MB/s | 146 kB 00:00 2024-09-12T02:46:27.221 INFO:teuthology.orchestra.run.smithi005.stdout:(32/156): ceph-mgr-modules-core-19.1.1-298.gb53 4.9 MB/s | 249 kB 00:00 2024-09-12T02:46:27.313 INFO:teuthology.orchestra.run.smithi005.stdout:(33/156): ceph-mgr-rook-19.1.1-298.gb53b6d3e.el 537 kB/s | 49 kB 00:00 2024-09-12T02:46:27.405 INFO:teuthology.orchestra.run.smithi005.stdout:(34/156): ceph-prometheus-alerts-19.1.1-298.gb5 180 kB/s | 16 kB 00:00 2024-09-12T02:46:27.489 INFO:teuthology.orchestra.run.smithi005.stdout:(35/156): ceph-volume-19.1.1-298.gb53b6d3e.el9. 3.3 MB/s | 285 kB 00:00 2024-09-12T02:46:27.526 INFO:teuthology.orchestra.run.smithi148.stdout:(31/156): ceph-mgr-diskprediction-local-19.1.1- 23 MB/s | 7.4 MB 00:00 2024-09-12T02:46:27.576 INFO:teuthology.orchestra.run.smithi148.stdout:(32/156): ceph-mgr-modules-core-19.1.1-298.gb53 4.9 MB/s | 249 kB 00:00 2024-09-12T02:46:27.598 INFO:teuthology.orchestra.run.smithi146.stdout:(60/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 45 kB/s | 24 kB 00:00 2024-09-12T02:46:27.623 INFO:teuthology.orchestra.run.smithi005.stdout:(36/156): ceph-mgr-diskprediction-local-19.1.1- 8.8 MB/s | 7.4 MB 00:00 2024-09-12T02:46:27.656 INFO:teuthology.orchestra.run.smithi005.stdout:(37/156): cephadm-19.1.1-298.gb53b6d3e.el9.noar 4.5 MB/s | 765 kB 00:00 2024-09-12T02:46:27.657 INFO:teuthology.orchestra.run.smithi146.stdout:(61/156): librdkafka-1.6.1-102.el9.x86_64.rpm 750 kB/s | 662 kB 00:00 2024-09-12T02:46:27.668 INFO:teuthology.orchestra.run.smithi148.stdout:(33/156): librgw2-19.1.1-298.gb53b6d3e.el9.x86_ 5.6 MB/s | 5.3 MB 00:00 2024-09-12T02:46:27.682 INFO:teuthology.orchestra.run.smithi005.stdout:(38/156): cephfs-top-19.1.1-298.gb53b6d3e.el9.n 311 kB/s | 18 kB 00:00 2024-09-12T02:46:27.685 INFO:teuthology.orchestra.run.smithi148.stdout:(34/156): ceph-mgr-rook-19.1.1-298.gb53b6d3e.el 454 kB/s | 49 kB 00:00 2024-09-12T02:46:27.702 INFO:teuthology.orchestra.run.smithi148.stdout:(35/156): ceph-prometheus-alerts-19.1.1-298.gb5 493 kB/s | 16 kB 00:00 2024-09-12T02:46:27.736 INFO:teuthology.orchestra.run.smithi148.stdout:(36/156): ceph-volume-19.1.1-298.gb53b6d3e.el9. 5.5 MB/s | 285 kB 00:00 2024-09-12T02:46:27.798 INFO:teuthology.orchestra.run.smithi148.stdout:(37/156): ceph-mgr-dashboard-19.1.1-298.gb53b6d 5.8 MB/s | 3.6 MB 00:00 2024-09-12T02:46:27.811 INFO:teuthology.orchestra.run.smithi148.stdout:(38/156): cephfs-top-19.1.1-298.gb53b6d3e.el9.n 243 kB/s | 18 kB 00:00 2024-09-12T02:46:27.815 INFO:teuthology.orchestra.run.smithi005.stdout:(39/156): fuse-2.9.9-16.el9.x86_64.rpm 503 kB/s | 80 kB 00:00 2024-09-12T02:46:27.840 INFO:teuthology.orchestra.run.smithi146.stdout:(62/156): libtool-2.4.6-46.el9.x86_64.rpm 696 kB/s | 581 kB 00:00 2024-09-12T02:46:27.845 INFO:teuthology.orchestra.run.smithi148.stdout:(39/156): cephadm-19.1.1-298.gb53b6d3e.el9.noar 5.2 MB/s | 765 kB 00:00 2024-09-12T02:46:27.907 INFO:teuthology.orchestra.run.smithi005.stdout:(40/156): libgfortran-11.5.0-2.el9.x86_64.rpm 3.5 MB/s | 801 kB 00:00 2024-09-12T02:46:27.932 INFO:teuthology.orchestra.run.smithi005.stdout:(41/156): libquadmath-11.5.0-2.el9.x86_64.rpm 1.6 MB/s | 191 kB 00:00 2024-09-12T02:46:27.953 INFO:teuthology.orchestra.run.smithi148.stdout:(40/156): fuse-2.9.9-16.el9.x86_64.rpm 503 kB/s | 80 kB 00:00 2024-09-12T02:46:27.974 INFO:teuthology.orchestra.run.smithi005.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.7 MB/s | 253 kB 00:00 2024-09-12T02:46:28.003 INFO:teuthology.orchestra.run.smithi148.stdout:(41/156): libquadmath-11.5.0-2.el9.x86_64.rpm 1.2 MB/s | 191 kB 00:00 2024-09-12T02:46:28.016 INFO:teuthology.orchestra.run.smithi005.stdout:(43/156): python3-cryptography-36.0.1-4.el9.x86 15 MB/s | 1.2 MB 00:00 2024-09-12T02:46:28.037 INFO:teuthology.orchestra.run.smithi148.stdout:(42/156): libgfortran-11.5.0-2.el9.x86_64.rpm 3.5 MB/s | 801 kB 00:00 2024-09-12T02:46:28.041 INFO:teuthology.orchestra.run.smithi005.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 1.6 MB/s | 106 kB 00:00 2024-09-12T02:46:28.067 INFO:teuthology.orchestra.run.smithi005.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-09-12T02:46:28.070 INFO:teuthology.orchestra.run.smithi148.stdout:(43/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.1 MB/s | 253 kB 00:00 2024-09-12T02:46:28.092 INFO:teuthology.orchestra.run.smithi005.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-09-12T02:46:28.096 INFO:teuthology.orchestra.run.smithi148.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-09-12T02:46:28.117 INFO:teuthology.orchestra.run.smithi005.stdout:(47/156): python3-urllib3-1.26.5-6.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-09-12T02:46:28.130 INFO:teuthology.orchestra.run.smithi148.stdout:(45/156): python3-cryptography-36.0.1-4.el9.x86 9.8 MB/s | 1.2 MB 00:00 2024-09-12T02:46:28.155 INFO:teuthology.orchestra.run.smithi148.stdout:(46/156): python3-pycparser-2.20-6.el9.noarch.r 1.6 MB/s | 135 kB 00:00 2024-09-12T02:46:28.173 INFO:teuthology.orchestra.run.smithi146.stdout:(63/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 82 kB/s | 47 kB 00:00 2024-09-12T02:46:28.180 INFO:teuthology.orchestra.run.smithi148.stdout:(47/156): python3-requests-2.25.1-8.el9.noarch. 1.5 MB/s | 125 kB 00:00 2024-09-12T02:46:28.205 INFO:teuthology.orchestra.run.smithi148.stdout:(48/156): python3-urllib3-1.26.5-6.el9.noarch.r 2.8 MB/s | 215 kB 00:00 2024-09-12T02:46:28.323 INFO:teuthology.orchestra.run.smithi148.stdout:(49/156): automake-1.16.2-8.el9.noarch.rpm 4.0 MB/s | 689 kB 00:00 2024-09-12T02:46:28.356 INFO:teuthology.orchestra.run.smithi148.stdout:(50/156): bison-3.7.4-5.el9.x86_64.rpm 5.2 MB/s | 943 kB 00:00 2024-09-12T02:46:28.382 INFO:teuthology.orchestra.run.smithi148.stdout:(51/156): elfutils-libelf-devel-0.191-4.el9.x86 371 kB/s | 22 kB 00:00 2024-09-12T02:46:28.398 INFO:teuthology.orchestra.run.smithi148.stdout:(52/156): boost-program-options-1.75.0-8.el9.x8 554 kB/s | 107 kB 00:00 2024-09-12T02:46:28.424 INFO:teuthology.orchestra.run.smithi148.stdout:(53/156): flex-2.6.4-9.el9.x86_64.rpm 4.6 MB/s | 313 kB 00:00 2024-09-12T02:46:28.440 INFO:teuthology.orchestra.run.smithi146.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 373 kB/s | 292 kB 00:00 2024-09-12T02:46:28.457 INFO:teuthology.orchestra.run.smithi148.stdout:(54/156): flexiblas-3.0.4-8.el9.x86_64.rpm 434 kB/s | 33 kB 00:00 2024-09-12T02:46:28.483 INFO:teuthology.orchestra.run.smithi148.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 309 kB/s | 18 kB 00:00 2024-09-12T02:46:28.525 INFO:teuthology.orchestra.run.smithi148.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 166 kB/s | 11 kB 00:00 2024-09-12T02:46:28.550 INFO:teuthology.orchestra.run.smithi148.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.3 MB/s | 160 kB 00:00 2024-09-12T02:46:28.593 INFO:teuthology.orchestra.run.smithi005.stdout:(48/156): ceph-test-19.1.1-298.gb53b6d3e.el9.x8 14 MB/s | 49 MB 00:03 2024-09-12T02:46:28.600 INFO:teuthology.orchestra.run.smithi148.stdout:(58/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 15 MB/s | 3.0 MB 00:00 2024-09-12T02:46:28.624 INFO:teuthology.orchestra.run.smithi146.stdout:(65/156): lua-5.4.4-4.el9.x86_64.rpm 241 kB/s | 188 kB 00:00 2024-09-12T02:46:28.625 INFO:teuthology.orchestra.run.smithi148.stdout:(59/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 450 kB/s | 45 kB 00:00 2024-09-12T02:46:28.651 INFO:teuthology.orchestra.run.smithi148.stdout:(60/156): librdkafka-1.6.1-102.el9.x86_64.rpm 6.4 MB/s | 662 kB 00:00 2024-09-12T02:46:28.684 INFO:teuthology.orchestra.run.smithi148.stdout:(61/156): libtool-2.4.6-46.el9.x86_64.rpm 6.8 MB/s | 581 kB 00:00 2024-09-12T02:46:28.710 INFO:teuthology.orchestra.run.smithi148.stdout:(62/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 292 kB/s | 24 kB 00:00 2024-09-12T02:46:28.735 INFO:teuthology.orchestra.run.smithi148.stdout:(63/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 566 kB/s | 47 kB 00:00 2024-09-12T02:46:28.760 INFO:teuthology.orchestra.run.smithi148.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.8 MB/s | 292 kB 00:00 2024-09-12T02:46:28.768 INFO:teuthology.orchestra.run.smithi005.stdout:(49/156): automake-1.16.2-8.el9.noarch.rpm 1.0 MB/s | 689 kB 00:00 2024-09-12T02:46:28.774 INFO:teuthology.orchestra.run.smithi146.stdout:(66/156): openblas-0.3.26-2.el9.x86_64.rpm 64 kB/s | 38 kB 00:00 2024-09-12T02:46:28.785 INFO:teuthology.orchestra.run.smithi148.stdout:(65/156): lua-5.4.4-4.el9.x86_64.rpm 2.4 MB/s | 188 kB 00:00 2024-09-12T02:46:28.793 INFO:teuthology.orchestra.run.smithi005.stdout:(50/156): boost-program-options-1.75.0-8.el9.x8 534 kB/s | 107 kB 00:00 2024-09-12T02:46:28.811 INFO:teuthology.orchestra.run.smithi148.stdout:(66/156): openblas-0.3.26-2.el9.x86_64.rpm 510 kB/s | 38 kB 00:00 2024-09-12T02:46:28.835 INFO:teuthology.orchestra.run.smithi005.stdout:(51/156): bison-3.7.4-5.el9.x86_64.rpm 1.3 MB/s | 943 kB 00:00 2024-09-12T02:46:28.852 INFO:teuthology.orchestra.run.smithi005.stdout:(52/156): elfutils-libelf-devel-0.191-4.el9.x86 259 kB/s | 22 kB 00:00 2024-09-12T02:46:28.894 INFO:teuthology.orchestra.run.smithi148.stdout:(67/156): postgresql-13.16-1.el9.x86_64.rpm 19 MB/s | 1.6 MB 00:00 2024-09-12T02:46:28.902 INFO:teuthology.orchestra.run.smithi005.stdout:(53/156): flex-2.6.4-9.el9.x86_64.rpm 2.8 MB/s | 313 kB 00:00 2024-09-12T02:46:28.919 INFO:teuthology.orchestra.run.smithi005.stdout:(54/156): flexiblas-3.0.4-8.el9.x86_64.rpm 390 kB/s | 33 kB 00:00 2024-09-12T02:46:28.953 INFO:teuthology.orchestra.run.smithi005.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 362 kB/s | 18 kB 00:00 2024-09-12T02:46:28.961 INFO:teuthology.orchestra.run.smithi148.stdout:(68/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 25 MB/s | 4.4 MB 00:00 2024-09-12T02:46:28.978 INFO:teuthology.orchestra.run.smithi005.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 191 kB/s | 11 kB 00:00 2024-09-12T02:46:29.012 INFO:teuthology.orchestra.run.smithi005.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-09-12T02:46:29.028 INFO:teuthology.orchestra.run.smithi148.stdout:(69/156): openblas-openmp-0.3.26-2.el9.x86_64.r 18 MB/s | 4.9 MB 00:00 2024-09-12T02:46:29.037 INFO:teuthology.orchestra.run.smithi005.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 772 kB/s | 45 kB 00:00 2024-09-12T02:46:29.062 INFO:teuthology.orchestra.run.smithi148.stdout:(70/156): postgresql-contrib-13.16-1.el9.x86_64 5.3 MB/s | 910 kB 00:00 2024-09-12T02:46:29.087 INFO:teuthology.orchestra.run.smithi148.stdout:(71/156): postgresql-private-libs-13.16-1.el9.x 1.1 MB/s | 137 kB 00:00 2024-09-12T02:46:29.087 INFO:teuthology.orchestra.run.smithi005.stdout:(59/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 13 MB/s | 3.0 MB 00:00 2024-09-12T02:46:29.121 INFO:teuthology.orchestra.run.smithi005.stdout:(60/156): librdkafka-1.6.1-102.el9.x86_64.rpm 5.9 MB/s | 662 kB 00:00 2024-09-12T02:46:29.137 INFO:teuthology.orchestra.run.smithi148.stdout:(72/156): protobuf-3.14.0-13.el9.x86_64.rpm 13 MB/s | 1.0 MB 00:00 2024-09-12T02:46:29.146 INFO:teuthology.orchestra.run.smithi005.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 420 kB/s | 24 kB 00:00 2024-09-12T02:46:29.162 INFO:teuthology.orchestra.run.smithi148.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.1 MB/s | 87 kB 00:00 2024-09-12T02:46:29.172 INFO:teuthology.orchestra.run.smithi005.stdout:(62/156): libtool-2.4.6-46.el9.x86_64.rpm 4.2 MB/s | 581 kB 00:00 2024-09-12T02:46:29.197 INFO:teuthology.orchestra.run.smithi005.stdout:(63/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 627 kB/s | 47 kB 00:00 2024-09-12T02:46:29.222 INFO:teuthology.orchestra.run.smithi005.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.8 MB/s | 292 kB 00:00 2024-09-12T02:46:29.248 INFO:teuthology.orchestra.run.smithi005.stdout:(65/156): lua-5.4.4-4.el9.x86_64.rpm 2.4 MB/s | 188 kB 00:00 2024-09-12T02:46:29.271 INFO:teuthology.orchestra.run.smithi148.stdout:(74/156): postgresql-server-13.16-1.el9.x86_64. 24 MB/s | 5.8 MB 00:00 2024-09-12T02:46:29.273 INFO:teuthology.orchestra.run.smithi005.stdout:(66/156): openblas-0.3.26-2.el9.x86_64.rpm 511 kB/s | 38 kB 00:00 2024-09-12T02:46:29.296 INFO:teuthology.orchestra.run.smithi148.stdout:(75/156): python3-colorama-0.4.6-3.el9.noarch.r 459 kB/s | 61 kB 00:00 2024-09-12T02:46:29.380 INFO:teuthology.orchestra.run.smithi148.stdout:(76/156): python3-babel-2.9.1-2.el9.noarch.rpm 25 MB/s | 6.0 MB 00:00 2024-09-12T02:46:29.405 INFO:teuthology.orchestra.run.smithi148.stdout:(77/156): python3-devel-3.9.19-8.el9.x86_64.rpm 1.8 MB/s | 245 kB 00:00 2024-09-12T02:46:29.430 INFO:teuthology.orchestra.run.smithi148.stdout:(78/156): python3-jinja2-2.11.3-6.el9.noarch.rp 1.8 MB/s | 248 kB 00:00 2024-09-12T02:46:29.431 INFO:teuthology.orchestra.run.smithi005.stdout:(67/156): openblas-openmp-0.3.26-2.el9.x86_64.r 24 MB/s | 4.9 MB 00:00 2024-09-12T02:46:29.455 INFO:teuthology.orchestra.run.smithi148.stdout:(79/156): python3-jsonpatch-1.21-16.el9.noarch. 342 kB/s | 26 kB 00:00 2024-09-12T02:46:29.465 INFO:teuthology.orchestra.run.smithi005.stdout:(68/156): postgresql-13.16-1.el9.x86_64.rpm 8.3 MB/s | 1.6 MB 00:00 2024-09-12T02:46:29.497 INFO:teuthology.orchestra.run.smithi148.stdout:(80/156): python3-jsonpointer-2.0-4.el9.noarch. 288 kB/s | 19 kB 00:00 2024-09-12T02:46:29.498 INFO:teuthology.orchestra.run.smithi005.stdout:(69/156): postgresql-contrib-13.16-1.el9.x86_64 13 MB/s | 910 kB 00:00 2024-09-12T02:46:29.523 INFO:teuthology.orchestra.run.smithi148.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 518 kB/s | 35 kB 00:00 2024-09-12T02:46:29.524 INFO:teuthology.orchestra.run.smithi005.stdout:(70/156): postgresql-private-libs-13.16-1.el9.x 2.3 MB/s | 137 kB 00:00 2024-09-12T02:46:29.548 INFO:teuthology.orchestra.run.smithi148.stdout:(82/156): python3-mako-1.1.4-6.el9.noarch.rpm 1.4 MB/s | 172 kB 00:00 2024-09-12T02:46:29.591 INFO:teuthology.orchestra.run.smithi005.stdout:(71/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 13 MB/s | 4.4 MB 00:00 2024-09-12T02:46:29.598 INFO:teuthology.orchestra.run.smithi148.stdout:(83/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 3.3 MB/s | 250 kB 00:00 2024-09-12T02:46:29.625 INFO:teuthology.orchestra.run.smithi005.stdout:(72/156): protobuf-3.14.0-13.el9.x86_64.rpm 10 MB/s | 1.0 MB 00:00 2024-09-12T02:46:29.632 INFO:teuthology.orchestra.run.smithi148.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 2.6 MB/s | 222 kB 00:00 2024-09-12T02:46:29.649 INFO:teuthology.orchestra.run.smithi146.stdout:(67/156): openblas-openmp-0.3.26-2.el9.x86_64.r 4.1 MB/s | 4.9 MB 00:01 2024-09-12T02:46:29.650 INFO:teuthology.orchestra.run.smithi005.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-09-12T02:46:29.657 INFO:teuthology.orchestra.run.smithi148.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-09-12T02:46:29.708 INFO:teuthology.orchestra.run.smithi005.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 1.0 MB/s | 61 kB 00:00 2024-09-12T02:46:29.733 INFO:teuthology.orchestra.run.smithi148.stdout:(86/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 22 MB/s | 5.1 MB 00:00 2024-09-12T02:46:29.741 INFO:teuthology.orchestra.run.smithi146.stdout:(68/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 4.0 MB/s | 4.4 MB 00:01 2024-09-12T02:46:29.766 INFO:teuthology.orchestra.run.smithi148.stdout:(87/156): python3-prettytable-0.7.2-27.el9.noar 315 kB/s | 42 kB 00:00 2024-09-12T02:46:29.767 INFO:teuthology.orchestra.run.smithi005.stdout:(75/156): python3-devel-3.9.19-8.el9.x86_64.rpm 4.1 MB/s | 245 kB 00:00 2024-09-12T02:46:29.792 INFO:teuthology.orchestra.run.smithi148.stdout:(88/156): python3-protobuf-3.14.0-13.el9.noarch 2.0 MB/s | 269 kB 00:00 2024-09-12T02:46:29.817 INFO:teuthology.orchestra.run.smithi148.stdout:(89/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.9 MB/s | 159 kB 00:00 2024-09-12T02:46:29.826 INFO:teuthology.orchestra.run.smithi005.stdout:(76/156): python3-jinja2-2.11.3-6.el9.noarch.rp 4.2 MB/s | 248 kB 00:00 2024-09-12T02:46:29.842 INFO:teuthology.orchestra.run.smithi148.stdout:(90/156): python3-pyasn1-modules-0.4.8-6.el9.no 3.6 MB/s | 279 kB 00:00 2024-09-12T02:46:29.866 INFO:teuthology.orchestra.run.smithi146.stdout:(69/156): postgresql-13.16-1.el9.x86_64.rpm 1.5 MB/s | 1.6 MB 00:01 2024-09-12T02:46:29.867 INFO:teuthology.orchestra.run.smithi148.stdout:(91/156): python3-pytz-2021.1-5.el9.noarch.rpm 677 kB/s | 51 kB 00:00 2024-09-12T02:46:29.893 INFO:teuthology.orchestra.run.smithi005.stdout:(77/156): postgresql-server-13.16-1.el9.x86_64. 15 MB/s | 5.8 MB 00:00 2024-09-12T02:46:29.893 INFO:teuthology.orchestra.run.smithi148.stdout:(92/156): python3-requests-oauthlib-1.3.0-12.el 711 kB/s | 54 kB 00:00 2024-09-12T02:46:29.918 INFO:teuthology.orchestra.run.smithi005.stdout:(78/156): python3-jsonpatch-1.21-16.el9.noarch. 279 kB/s | 26 kB 00:00 2024-09-12T02:46:29.934 INFO:teuthology.orchestra.run.smithi148.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 623 kB/s | 42 kB 00:00 2024-09-12T02:46:29.960 INFO:teuthology.orchestra.run.smithi148.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 661 kB/s | 44 kB 00:00 2024-09-12T02:46:29.993 INFO:teuthology.orchestra.run.smithi005.stdout:(79/156): python3-babel-2.9.1-2.el9.noarch.rpm 16 MB/s | 6.0 MB 00:00 2024-09-12T02:46:29.994 INFO:teuthology.orchestra.run.smithi148.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-09-12T02:46:30.018 INFO:teuthology.orchestra.run.smithi005.stdout:(80/156): python3-jsonpointer-2.0-4.el9.noarch. 153 kB/s | 19 kB 00:00 2024-09-12T02:46:30.019 INFO:teuthology.orchestra.run.smithi148.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 613 kB/s | 36 kB 00:00 2024-09-12T02:46:30.044 INFO:teuthology.orchestra.run.smithi005.stdout:(81/156): python3-mako-1.1.4-6.el9.noarch.rpm 1.3 MB/s | 172 kB 00:00 2024-09-12T02:46:30.052 INFO:teuthology.orchestra.run.smithi148.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 810 kB/s | 47 kB 00:00 2024-09-12T02:46:30.069 INFO:teuthology.orchestra.run.smithi005.stdout:(82/156): python3-markupsafe-1.1.1-12.el9.x86_6 461 kB/s | 35 kB 00:00 2024-09-12T02:46:30.086 INFO:teuthology.orchestra.run.smithi148.stdout:(98/156): socat-1.7.4.1-6.el9.x86_64.rpm 4.4 MB/s | 304 kB 00:00 2024-09-12T02:46:30.103 INFO:teuthology.orchestra.run.smithi005.stdout:(83/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.2 MB/s | 250 kB 00:00 2024-09-12T02:46:30.111 INFO:teuthology.orchestra.run.smithi148.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 989 kB/s | 58 kB 00:00 2024-09-12T02:46:30.128 INFO:teuthology.orchestra.run.smithi005.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 3.7 MB/s | 222 kB 00:00 2024-09-12T02:46:30.161 INFO:teuthology.orchestra.run.smithi005.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-09-12T02:46:30.162 INFO:teuthology.orchestra.run.smithi148.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.2 MB/s | 96 kB 00:00 2024-09-12T02:46:30.186 INFO:teuthology.orchestra.run.smithi148.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 846 kB/s | 64 kB 00:00 2024-09-12T02:46:30.187 INFO:teuthology.orchestra.run.smithi005.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 723 kB/s | 42 kB 00:00 2024-09-12T02:46:30.220 INFO:teuthology.orchestra.run.smithi005.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 4.5 MB/s | 269 kB 00:00 2024-09-12T02:46:30.246 INFO:teuthology.orchestra.run.smithi005.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 159 kB 00:00 2024-09-12T02:46:30.288 INFO:teuthology.orchestra.run.smithi005.stdout:(89/156): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 279 kB 00:00 2024-09-12T02:46:30.305 INFO:teuthology.orchestra.run.smithi005.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 870 kB/s | 51 kB 00:00 2024-09-12T02:46:30.337 INFO:teuthology.orchestra.run.smithi148.stdout:(102/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 33 MB/s | 16 MB 00:00 2024-09-12T02:46:30.338 INFO:teuthology.orchestra.run.smithi005.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-09-12T02:46:30.362 INFO:teuthology.orchestra.run.smithi148.stdout:(103/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 224 kB/s | 45 kB 00:00 2024-09-12T02:46:30.389 INFO:teuthology.orchestra.run.smithi005.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 837 kB/s | 42 kB 00:00 2024-09-12T02:46:30.413 INFO:teuthology.orchestra.run.smithi148.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 11 MB/s | 548 kB 00:00 2024-09-12T02:46:30.446 INFO:teuthology.orchestra.run.smithi148.stdout:(105/156): gperftools-libs-2.9.1-2.el9.x86_64.r 9.1 MB/s | 309 kB 00:00 2024-09-12T02:46:30.447 INFO:teuthology.orchestra.run.smithi005.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 758 kB/s | 44 kB 00:00 2024-09-12T02:46:30.449 INFO:teuthology.orchestra.run.smithi146.stdout:(70/156): postgresql-private-libs-13.16-1.el9.x 194 kB/s | 137 kB 00:00 2024-09-12T02:46:30.472 INFO:teuthology.orchestra.run.smithi148.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 784 kB/s | 19 kB 00:00 2024-09-12T02:46:30.558 INFO:teuthology.orchestra.run.smithi146.stdout:(71/156): postgresql-contrib-13.16-1.el9.x86_64 1.0 MB/s | 910 kB 00:00 2024-09-12T02:46:30.573 INFO:teuthology.orchestra.run.smithi005.stdout:(94/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 9.2 MB/s | 5.1 MB 00:00 2024-09-12T02:46:30.588 INFO:teuthology.orchestra.run.smithi148.stdout:(107/156): libarrow-9.0.0-11.el9.x86_64.rpm 38 MB/s | 4.4 MB 00:00 2024-09-12T02:46:30.598 INFO:teuthology.orchestra.run.smithi005.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 1.4 MB/s | 221 kB 00:00 2024-09-12T02:46:30.613 INFO:teuthology.orchestra.run.smithi148.stdout:(108/156): lua-devel-5.4.4-4.el9.x86_64.rpm 52 kB/s | 22 kB 00:00 2024-09-12T02:46:30.623 INFO:teuthology.orchestra.run.smithi005.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 717 kB/s | 36 kB 00:00 2024-09-12T02:46:30.647 INFO:teuthology.orchestra.run.smithi148.stdout:(109/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 444 kB/s | 26 kB 00:00 2024-09-12T02:46:30.657 INFO:teuthology.orchestra.run.smithi005.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 812 kB/s | 47 kB 00:00 2024-09-12T02:46:30.672 INFO:teuthology.orchestra.run.smithi148.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 2.8 MB/s | 67 kB 00:00 2024-09-12T02:46:30.682 INFO:teuthology.orchestra.run.smithi005.stdout:(98/156): socat-1.7.4.1-6.el9.x86_64.rpm 5.1 MB/s | 304 kB 00:00 2024-09-12T02:46:30.698 INFO:teuthology.orchestra.run.smithi148.stdout:(111/156): liboath-2.6.7-2.el9.x86_64.rpm 579 kB/s | 49 kB 00:00 2024-09-12T02:46:30.716 INFO:teuthology.orchestra.run.smithi005.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 988 kB/s | 58 kB 00:00 2024-09-12T02:46:30.723 INFO:teuthology.orchestra.run.smithi148.stdout:(112/156): luarocks-3.9.2-5.el9.noarch.rpm 2.9 MB/s | 151 kB 00:00 2024-09-12T02:46:30.741 INFO:teuthology.orchestra.run.smithi005.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.6 MB/s | 96 kB 00:00 2024-09-12T02:46:30.765 INFO:teuthology.orchestra.run.smithi148.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 13 MB/s | 548 kB 00:00 2024-09-12T02:46:30.774 INFO:teuthology.orchestra.run.smithi005.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-09-12T02:46:30.790 INFO:teuthology.orchestra.run.smithi148.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-12T02:46:30.800 INFO:teuthology.orchestra.run.smithi005.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 763 kB/s | 45 kB 00:00 2024-09-12T02:46:30.815 INFO:teuthology.orchestra.run.smithi148.stdout:(115/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 7.0 MB/s | 839 kB 00:00 2024-09-12T02:46:30.841 INFO:teuthology.orchestra.run.smithi148.stdout:(116/156): python3-cachetools-4.2.4-1.el9.noarc 639 kB/s | 32 kB 00:00 2024-09-12T02:46:30.866 INFO:teuthology.orchestra.run.smithi148.stdout:(117/156): python3-certifi-2023.05.07-4.el9.noa 281 kB/s | 14 kB 00:00 2024-09-12T02:46:30.900 INFO:teuthology.orchestra.run.smithi148.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-09-12T02:46:30.925 INFO:teuthology.orchestra.run.smithi148.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 5.9 MB/s | 358 kB 00:00 2024-09-12T02:46:30.958 INFO:teuthology.orchestra.run.smithi148.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.1 MB/s | 303 kB 00:00 2024-09-12T02:46:30.984 INFO:teuthology.orchestra.run.smithi148.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 3.7 MB/s | 220 kB 00:00 2024-09-12T02:46:30.992 INFO:teuthology.orchestra.run.smithi005.stdout:(103/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 24 MB/s | 16 MB 00:00 2024-09-12T02:46:31.000 INFO:teuthology.orchestra.run.smithi146.stdout:(72/156): postgresql-server-13.16-1.el9.x86_64. 5.1 MB/s | 5.8 MB 00:01 2024-09-12T02:46:31.017 INFO:teuthology.orchestra.run.smithi005.stdout:(104/156): lua-devel-5.4.4-4.el9.x86_64.rpm 92 kB/s | 22 kB 00:00 2024-09-12T02:46:31.018 INFO:teuthology.orchestra.run.smithi148.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 4.3 MB/s | 144 kB 00:00 2024-09-12T02:46:31.043 INFO:teuthology.orchestra.run.smithi148.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 428 kB/s | 11 kB 00:00 2024-09-12T02:46:31.068 INFO:teuthology.orchestra.run.smithi148.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 704 kB/s | 18 kB 00:00 2024-09-12T02:46:31.076 INFO:teuthology.orchestra.run.smithi005.stdout:(105/156): abseil-cpp-20211102.0-3.el9.x86_64.r 6.4 MB/s | 548 kB 00:00 2024-09-12T02:46:31.093 INFO:teuthology.orchestra.run.smithi148.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 933 kB/s | 23 kB 00:00 2024-09-12T02:46:31.101 INFO:teuthology.orchestra.run.smithi005.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 783 kB/s | 19 kB 00:00 2024-09-12T02:46:31.119 INFO:teuthology.orchestra.run.smithi148.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 776 kB/s | 19 kB 00:00 2024-09-12T02:46:31.126 INFO:teuthology.orchestra.run.smithi005.stdout:(107/156): gperftools-libs-2.9.1-2.el9.x86_64.r 2.8 MB/s | 309 kB 00:00 2024-09-12T02:46:31.144 INFO:teuthology.orchestra.run.smithi148.stdout:(127/156): python3-jaraco-text-3.2.0-6.el9.noar 793 kB/s | 20 kB 00:00 2024-09-12T02:46:31.151 INFO:teuthology.orchestra.run.smithi005.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-12T02:46:31.177 INFO:teuthology.orchestra.run.smithi005.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-12T02:46:31.202 INFO:teuthology.orchestra.run.smithi148.stdout:(128/156): python3-grpcio-1.46.7-10.el9.x86_64. 8.4 MB/s | 2.0 MB 00:00 2024-09-12T02:46:31.209 INFO:teuthology.orchestra.run.smithi146.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 135 kB/s | 87 kB 00:00 2024-09-12T02:46:31.210 INFO:teuthology.orchestra.run.smithi005.stdout:(110/156): protobuf-compiler-3.14.0-13.el9.x86_ 2.1 MB/s | 863 kB 00:00 2024-09-12T02:46:31.227 INFO:teuthology.orchestra.run.smithi148.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-12T02:46:31.236 INFO:teuthology.orchestra.run.smithi005.stdout:(111/156): libunwind-1.6.2-1.el9.x86_64.rpm 1.1 MB/s | 67 kB 00:00 2024-09-12T02:46:31.253 INFO:teuthology.orchestra.run.smithi148.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-12T02:46:31.287 INFO:teuthology.orchestra.run.smithi148.stdout:(131/156): python3-kubernetes-26.1.0-2.el9.noar 7.2 MB/s | 1.0 MB 00:00 2024-09-12T02:46:31.294 INFO:teuthology.orchestra.run.smithi005.stdout:(112/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 17 MB/s | 839 kB 00:00 2024-09-12T02:46:31.313 INFO:teuthology.orchestra.run.smithi148.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 986 kB/s | 58 kB 00:00 2024-09-12T02:46:31.319 INFO:teuthology.orchestra.run.smithi005.stdout:(113/156): luarocks-3.9.2-5.el9.noarch.rpm 1.4 MB/s | 151 kB 00:00 2024-09-12T02:46:31.337 INFO:teuthology.orchestra.run.smithi148.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 666 kB/s | 16 kB 00:00 2024-09-12T02:46:31.345 INFO:teuthology.orchestra.run.smithi005.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-12T02:46:31.362 INFO:teuthology.orchestra.run.smithi148.stdout:(134/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 3.5 MB/s | 90 kB 00:00 2024-09-12T02:46:31.370 INFO:teuthology.orchestra.run.smithi005.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-12T02:46:31.387 INFO:teuthology.orchestra.run.smithi148.stdout:(135/156): python3-pecan-1.4.2-3.el9.noarch.rpm 2.6 MB/s | 272 kB 00:00 2024-09-12T02:46:31.396 INFO:teuthology.orchestra.run.smithi005.stdout:(116/156): python3-certifi-2023.05.07-4.el9.noa 567 kB/s | 14 kB 00:00 2024-09-12T02:46:31.421 INFO:teuthology.orchestra.run.smithi148.stdout:(136/156): protobuf-compiler-3.14.0-13.el9.x86_ 797 kB/s | 863 kB 00:01 2024-09-12T02:46:31.429 INFO:teuthology.orchestra.run.smithi005.stdout:(117/156): python3-asyncssh-2.13.2-5.el9.noarch 4.0 MB/s | 548 kB 00:00 2024-09-12T02:46:31.438 INFO:teuthology.orchestra.run.smithi148.stdout:(137/156): python3-pyperclip-1.8.2-2.el9.noarch 375 kB/s | 28 kB 00:00 2024-09-12T02:46:31.455 INFO:teuthology.orchestra.run.smithi148.stdout:(138/156): python3-repoze-lru-0.7-16.el9.noarch 459 kB/s | 31 kB 00:00 2024-09-12T02:46:31.463 INFO:teuthology.orchestra.run.smithi005.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 2.5 MB/s | 172 kB 00:00 2024-09-12T02:46:31.480 INFO:teuthology.orchestra.run.smithi148.stdout:(139/156): python3-tempora-5.0.0-2.el9.noarch.r 2.0 MB/s | 36 kB 00:00 2024-09-12T02:46:31.484 INFO:teuthology.orchestra.run.smithi146.stdout:(74/156): protobuf-3.14.0-13.el9.x86_64.rpm 1.0 MB/s | 1.0 MB 00:01 2024-09-12T02:46:31.488 INFO:teuthology.orchestra.run.smithi005.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-09-12T02:46:31.517 INFO:teuthology.orchestra.run.smithi148.stdout:(140/156): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-09-12T02:46:31.522 INFO:teuthology.orchestra.run.smithi005.stdout:(120/156): python3-google-auth-2.34.0-1.el9.noa 6.5 MB/s | 220 kB 00:00 2024-09-12T02:46:31.530 INFO:teuthology.orchestra.run.smithi148.stdout:(141/156): python3-rsa-4.9-2.el9.noarch.rpm 642 kB/s | 59 kB 00:00 2024-09-12T02:46:31.547 INFO:teuthology.orchestra.run.smithi005.stdout:(121/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.5 MB/s | 303 kB 00:00 2024-09-12T02:46:31.548 INFO:teuthology.orchestra.run.smithi148.stdout:(142/156): python3-typing-extensions-4.12.2-2.e 1.1 MB/s | 74 kB 00:00 2024-09-12T02:46:31.573 INFO:teuthology.orchestra.run.smithi148.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 2.1 MB/s | 90 kB 00:00 2024-09-12T02:46:31.581 INFO:teuthology.orchestra.run.smithi005.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 4.2 MB/s | 144 kB 00:00 2024-09-12T02:46:31.590 INFO:teuthology.orchestra.run.smithi148.stdout:(144/156): python3-zc-lockfile-2.0-10.el9.noarc 1.2 MB/s | 20 kB 00:00 2024-09-12T02:46:31.606 INFO:teuthology.orchestra.run.smithi005.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 427 kB/s | 11 kB 00:00 2024-09-12T02:46:31.615 INFO:teuthology.orchestra.run.smithi148.stdout:(145/156): python3-webob-1.8.8-2.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-09-12T02:46:31.631 INFO:teuthology.orchestra.run.smithi005.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 713 kB/s | 18 kB 00:00 2024-09-12T02:46:31.640 INFO:teuthology.orchestra.run.smithi148.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-12T02:46:31.657 INFO:teuthology.orchestra.run.smithi005.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 929 kB/s | 23 kB 00:00 2024-09-12T02:46:31.665 INFO:teuthology.orchestra.run.smithi148.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.6 MB/s | 427 kB 00:00 2024-09-12T02:46:31.682 INFO:teuthology.orchestra.run.smithi005.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 780 kB/s | 19 kB 00:00 2024-09-12T02:46:31.707 INFO:teuthology.orchestra.run.smithi005.stdout:(127/156): python3-jaraco-text-3.2.0-6.el9.noar 790 kB/s | 20 kB 00:00 2024-09-12T02:46:31.717 INFO:teuthology.orchestra.run.smithi148.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-09-12T02:46:31.791 INFO:teuthology.orchestra.run.smithi005.stdout:(128/156): libarrow-9.0.0-11.el9.x86_64.rpm 6.4 MB/s | 4.4 MB 00:00 2024-09-12T02:46:31.816 INFO:teuthology.orchestra.run.smithi005.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-12T02:46:31.842 INFO:teuthology.orchestra.run.smithi005.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-12T02:46:31.858 INFO:teuthology.orchestra.run.smithi148.stdout:(149/156): NetworkManager-1.49.90-1.el9.x86_64. 17 MB/s | 2.3 MB 00:00 2024-09-12T02:46:31.875 INFO:teuthology.orchestra.run.smithi005.stdout:(131/156): python3-kubernetes-26.1.0-2.el9.noar 6.1 MB/s | 1.0 MB 00:00 2024-09-12T02:46:31.875 INFO:teuthology.orchestra.run.smithi146.stdout:(75/156): python3-colorama-0.4.6-3.el9.noarch.r 92 kB/s | 61 kB 00:00 2024-09-12T02:46:31.900 INFO:teuthology.orchestra.run.smithi005.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 988 kB/s | 58 kB 00:00 2024-09-12T02:46:31.926 INFO:teuthology.orchestra.run.smithi005.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 659 kB/s | 16 kB 00:00 2024-09-12T02:46:31.933 INFO:teuthology.orchestra.run.smithi148.stdout:(150/156): NetworkManager-libnm-1.49.90-1.el9.x 24 MB/s | 1.8 MB 00:00 2024-09-12T02:46:31.959 INFO:teuthology.orchestra.run.smithi005.stdout:(134/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 2.6 MB/s | 90 kB 00:00 2024-09-12T02:46:31.975 INFO:teuthology.orchestra.run.smithi148.stdout:(151/156): NetworkManager-team-1.49.90-1.el9.x8 986 kB/s | 41 kB 00:00 2024-09-12T02:46:32.001 INFO:teuthology.orchestra.run.smithi005.stdout:(135/156): python3-grpcio-1.46.7-10.el9.x86_64. 4.3 MB/s | 2.0 MB 00:00 2024-09-12T02:46:32.026 INFO:teuthology.orchestra.run.smithi148.stdout:(152/156): NetworkManager-tui-1.49.90-1.el9.x86 4.9 MB/s | 249 kB 00:00 2024-09-12T02:46:32.026 INFO:teuthology.orchestra.run.smithi005.stdout:(136/156): python3-pecan-1.4.2-3.el9.noarch.rpm 1.8 MB/s | 272 kB 00:00 2024-09-12T02:46:32.052 INFO:teuthology.orchestra.run.smithi005.stdout:(137/156): python3-pyperclip-1.8.2-2.el9.noarch 306 kB/s | 28 kB 00:00 2024-09-12T02:46:32.077 INFO:teuthology.orchestra.run.smithi005.stdout:(138/156): python3-repoze-lru-0.7-16.el9.noarch 407 kB/s | 31 kB 00:00 2024-09-12T02:46:32.084 INFO:teuthology.orchestra.run.smithi148.stdout:(153/156): librbd1-19.1.1-298.gb53b6d3e.el9.x86 7.6 MB/s | 3.2 MB 00:00 2024-09-12T02:46:32.102 INFO:teuthology.orchestra.run.smithi005.stdout:(139/156): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-09-12T02:46:32.126 INFO:teuthology.orchestra.run.smithi148.stdout:(154/156): openssl-3.2.2-6.el9.x86_64.rpm 14 MB/s | 1.4 MB 00:00 2024-09-12T02:46:32.127 INFO:teuthology.orchestra.run.smithi005.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 712 kB/s | 36 kB 00:00 2024-09-12T02:46:32.145 INFO:teuthology.orchestra.run.smithi005.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 1.7 MB/s | 74 kB 00:00 2024-09-12T02:46:32.151 INFO:teuthology.orchestra.run.smithi146.stdout:(76/156): python3-babel-2.9.1-2.el9.noarch.rpm 5.2 MB/s | 6.0 MB 00:01 2024-09-12T02:46:32.170 INFO:teuthology.orchestra.run.smithi005.stdout:(142/156): python3-routes-2.5.1-5.el9.noarch.rp 1.3 MB/s | 188 kB 00:00 2024-09-12T02:46:32.193 INFO:teuthology.orchestra.run.smithi148.stdout:(155/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 22 MB/s | 2.4 MB 00:00 2024-09-12T02:46:32.196 INFO:teuthology.orchestra.run.smithi005.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.8 MB/s | 90 kB 00:00 2024-09-12T02:46:32.220 INFO:teuthology.orchestra.run.smithi005.stdout:(144/156): python3-webob-1.8.8-2.el9.noarch.rpm 2.4 MB/s | 230 kB 00:00 2024-09-12T02:46:32.238 INFO:teuthology.orchestra.run.smithi005.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 477 kB/s | 20 kB 00:00 2024-09-12T02:46:32.251 INFO:teuthology.orchestra.run.smithi148.stdout:(156/156): librados2-19.1.1-298.gb53b6d3e.el9.x 5.6 MB/s | 3.4 MB 00:00 2024-09-12T02:46:32.258 INFO:teuthology.orchestra.run.smithi148.stdout:-------------------------------------------------------------------------------- 2024-09-12T02:46:32.258 INFO:teuthology.orchestra.run.smithi148.stdout:Total 19 MB/s | 226 MB 00:11 2024-09-12T02:46:32.263 INFO:teuthology.orchestra.run.smithi005.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 4.4 MB/s | 191 kB 00:00 2024-09-12T02:46:32.285 INFO:teuthology.orchestra.run.smithi146.stdout:(77/156): python3-devel-3.9.19-8.el9.x86_64.rpm 306 kB/s | 245 kB 00:00 2024-09-12T02:46:32.288 INFO:teuthology.orchestra.run.smithi005.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.5 MB/s | 427 kB 00:00 2024-09-12T02:46:32.339 INFO:teuthology.orchestra.run.smithi005.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-09-12T02:46:32.455 INFO:teuthology.orchestra.run.smithi005.stdout:(149/156): NetworkManager-1.49.90-1.el9.x86_64. 20 MB/s | 2.3 MB 00:00 2024-09-12T02:46:32.523 INFO:teuthology.orchestra.run.smithi005.stdout:(150/156): NetworkManager-libnm-1.49.90-1.el9.x 27 MB/s | 1.8 MB 00:00 2024-09-12T02:46:32.564 INFO:teuthology.orchestra.run.smithi005.stdout:(151/156): NetworkManager-team-1.49.90-1.el9.x8 989 kB/s | 41 kB 00:00 2024-09-12T02:46:32.623 INFO:teuthology.orchestra.run.smithi005.stdout:(152/156): librados2-19.1.1-298.gb53b6d3e.el9.x 9.5 MB/s | 3.4 MB 00:00 2024-09-12T02:46:32.656 INFO:teuthology.orchestra.run.smithi005.stdout:(153/156): NetworkManager-tui-1.49.90-1.el9.x86 2.6 MB/s | 249 kB 00:00 2024-09-12T02:46:32.698 INFO:teuthology.orchestra.run.smithi005.stdout:(154/156): openssl-3.2.2-6.el9.x86_64.rpm 18 MB/s | 1.4 MB 00:00 2024-09-12T02:46:32.711 INFO:teuthology.orchestra.run.smithi146.stdout:(78/156): python3-jinja2-2.11.3-6.el9.noarch.rp 298 kB/s | 248 kB 00:00 2024-09-12T02:46:32.752 INFO:teuthology.orchestra.run.smithi146.stdout:(79/156): python3-jsonpatch-1.21-16.el9.noarch. 43 kB/s | 26 kB 00:00 2024-09-12T02:46:32.757 INFO:teuthology.orchestra.run.smithi005.stdout:(155/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 24 MB/s | 2.4 MB 00:00 2024-09-12T02:46:32.777 INFO:teuthology.orchestra.run.smithi146.stdout:(80/156): python3-jsonpointer-2.0-4.el9.noarch. 39 kB/s | 19 kB 00:00 2024-09-12T02:46:32.815 INFO:teuthology.orchestra.run.smithi005.stdout:(156/156): librbd1-19.1.1-298.gb53b6d3e.el9.x86 6.0 MB/s | 3.2 MB 00:00 2024-09-12T02:46:32.821 INFO:teuthology.orchestra.run.smithi005.stdout:-------------------------------------------------------------------------------- 2024-09-12T02:46:32.822 INFO:teuthology.orchestra.run.smithi005.stdout:Total 20 MB/s | 226 MB 00:11 2024-09-12T02:46:33.310 INFO:teuthology.orchestra.run.smithi146.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 62 kB/s | 35 kB 00:00 2024-09-12T02:46:33.444 INFO:teuthology.orchestra.run.smithi146.stdout:(82/156): python3-mako-1.1.4-6.el9.noarch.rpm 235 kB/s | 172 kB 00:00 2024-09-12T02:46:33.477 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction check 2024-09-12T02:46:33.628 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction check succeeded. 2024-09-12T02:46:33.628 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction test 2024-09-12T02:46:33.908 INFO:teuthology.orchestra.run.smithi146.stdout:(83/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 4.5 MB/s | 5.1 MB 00:01 2024-09-12T02:46:34.054 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction check 2024-09-12T02:46:34.103 INFO:teuthology.orchestra.run.smithi146.stdout:(84/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 316 kB/s | 250 kB 00:00 2024-09-12T02:46:34.221 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction check succeeded. 2024-09-12T02:46:34.222 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction test 2024-09-12T02:46:34.320 INFO:teuthology.orchestra.run.smithi146.stdout:(85/156): python3-oauthlib-3.1.1-5.el9.noarch.r 254 kB/s | 222 kB 00:00 2024-09-12T02:46:34.645 INFO:teuthology.orchestra.run.smithi146.stdout:(86/156): python3-packaging-20.9-5.el9.noarch.r 105 kB/s | 77 kB 00:00 2024-09-12T02:46:34.679 INFO:teuthology.orchestra.run.smithi146.stdout:(87/156): python3-prettytable-0.7.2-27.el9.noar 73 kB/s | 42 kB 00:00 2024-09-12T02:46:35.087 INFO:teuthology.orchestra.run.smithi146.stdout:(88/156): python3-protobuf-3.14.0-13.el9.noarch 355 kB/s | 269 kB 00:00 2024-09-12T02:46:35.371 INFO:teuthology.orchestra.run.smithi146.stdout:(89/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 219 kB/s | 159 kB 00:00 2024-09-12T02:46:35.424 INFO:teuthology.orchestra.run.smithi148.stdout:Transaction test succeeded. 2024-09-12T02:46:35.424 INFO:teuthology.orchestra.run.smithi148.stdout:Running transaction 2024-09-12T02:46:35.446 INFO:teuthology.orchestra.run.smithi146.stdout:(90/156): python3-pyasn1-modules-0.4.8-6.el9.no 363 kB/s | 279 kB 00:00 2024-09-12T02:46:35.696 INFO:teuthology.orchestra.run.smithi146.stdout:(91/156): python3-pytz-2021.1-5.el9.noarch.rpm 84 kB/s | 51 kB 00:00 2024-09-12T02:46:35.988 INFO:teuthology.orchestra.run.smithi146.stdout:(92/156): python3-requests-oauthlib-1.3.0-12.el 87 kB/s | 54 kB 00:00 2024-09-12T02:46:36.305 INFO:teuthology.orchestra.run.smithi146.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 68 kB/s | 42 kB 00:00 2024-09-12T02:46:36.506 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction test succeeded. 2024-09-12T02:46:36.507 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction 2024-09-12T02:46:36.580 INFO:teuthology.orchestra.run.smithi146.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 75 kB/s | 44 kB 00:00 2024-09-12T02:46:36.906 INFO:teuthology.orchestra.run.smithi146.stdout:(95/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 11 MB/s | 16 MB 00:01 2024-09-12T02:46:37.172 INFO:teuthology.orchestra.run.smithi146.stdout:(96/156): qatlib-24.02.0-1.el9.x86_64.rpm 255 kB/s | 221 kB 00:00 2024-09-12T02:46:37.281 INFO:teuthology.orchestra.run.smithi146.stdout:(97/156): qatlib-service-24.02.0-1.el9.x86_64.r 51 kB/s | 36 kB 00:00 2024-09-12T02:46:37.498 INFO:teuthology.orchestra.run.smithi146.stdout:(98/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 81 kB/s | 47 kB 00:00 2024-09-12T02:46:37.898 INFO:teuthology.orchestra.run.smithi146.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 94 kB/s | 58 kB 00:00 2024-09-12T02:46:38.040 INFO:teuthology.orchestra.run.smithi146.stdout:(100/156): socat-1.7.4.1-6.el9.x86_64.rpm 351 kB/s | 304 kB 00:00 2024-09-12T02:46:38.223 INFO:teuthology.orchestra.run.smithi146.stdout:(101/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 134 kB/s | 96 kB 00:00 2024-09-12T02:46:38.291 INFO:teuthology.orchestra.run.smithi146.stdout:(102/156): lua-devel-5.4.4-4.el9.x86_64.rpm 336 kB/s | 22 kB 00:00 2024-09-12T02:46:38.441 INFO:teuthology.orchestra.run.smithi146.stdout:(103/156): protobuf-compiler-3.14.0-13.el9.x86_ 5.6 MB/s | 863 kB 00:00 2024-09-12T02:46:38.499 INFO:teuthology.orchestra.run.smithi146.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 9.2 MB/s | 548 kB 00:00 2024-09-12T02:46:38.533 INFO:teuthology.orchestra.run.smithi146.stdout:(105/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 100 kB/s | 64 kB 00:00 2024-09-12T02:46:38.566 INFO:teuthology.orchestra.run.smithi146.stdout:(106/156): gperftools-libs-2.9.1-2.el9.x86_64.r 4.6 MB/s | 309 kB 00:00 2024-09-12T02:46:38.592 INFO:teuthology.orchestra.run.smithi146.stdout:(107/156): grpc-data-1.46.7-10.el9.noarch.rpm 333 kB/s | 19 kB 00:00 2024-09-12T02:46:38.617 INFO:teuthology.orchestra.run.smithi146.stdout:(108/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 78 kB/s | 45 kB 00:00 2024-09-12T02:46:38.642 INFO:teuthology.orchestra.run.smithi146.stdout:(109/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 531 kB/s | 26 kB 00:00 2024-09-12T02:46:38.676 INFO:teuthology.orchestra.run.smithi146.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-12T02:46:38.709 INFO:teuthology.orchestra.run.smithi146.stdout:(111/156): liboath-2.6.7-2.el9.x86_64.rpm 527 kB/s | 49 kB 00:00 2024-09-12T02:46:38.735 INFO:teuthology.orchestra.run.smithi146.stdout:(112/156): luarocks-3.9.2-5.el9.noarch.rpm 2.6 MB/s | 151 kB 00:00 2024-09-12T02:46:38.777 INFO:teuthology.orchestra.run.smithi146.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 13 MB/s | 548 kB 00:00 2024-09-12T02:46:38.802 INFO:teuthology.orchestra.run.smithi146.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-12T02:46:38.827 INFO:teuthology.orchestra.run.smithi146.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-12T02:46:38.853 INFO:teuthology.orchestra.run.smithi146.stdout:(116/156): python3-certifi-2023.05.07-4.el9.noa 559 kB/s | 14 kB 00:00 2024-09-12T02:46:38.895 INFO:teuthology.orchestra.run.smithi146.stdout:(117/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 4.4 MB/s | 839 kB 00:00 2024-09-12T02:46:38.920 INFO:teuthology.orchestra.run.smithi146.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 2.5 MB/s | 172 kB 00:00 2024-09-12T02:46:38.953 INFO:teuthology.orchestra.run.smithi146.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-09-12T02:46:38.978 INFO:teuthology.orchestra.run.smithi146.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.1 MB/s | 303 kB 00:00 2024-09-12T02:46:39.012 INFO:teuthology.orchestra.run.smithi146.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 3.7 MB/s | 220 kB 00:00 2024-09-12T02:46:39.038 INFO:teuthology.orchestra.run.smithi146.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 5.6 MB/s | 144 kB 00:00 2024-09-12T02:46:39.054 INFO:teuthology.orchestra.run.smithi146.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 649 kB/s | 11 kB 00:00 2024-09-12T02:46:39.071 INFO:teuthology.orchestra.run.smithi146.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 1.0 MB/s | 18 kB 00:00 2024-09-12T02:46:39.088 INFO:teuthology.orchestra.run.smithi146.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 1.4 MB/s | 23 kB 00:00 2024-09-12T02:46:39.105 INFO:teuthology.orchestra.run.smithi146.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 1.1 MB/s | 19 kB 00:00 2024-09-12T02:46:39.122 INFO:teuthology.orchestra.run.smithi146.stdout:(127/156): python3-jaraco-text-3.2.0-6.el9.noar 1.2 MB/s | 20 kB 00:00 2024-09-12T02:46:39.300 INFO:teuthology.orchestra.run.smithi146.stdout:(128/156): python3-kubernetes-26.1.0-2.el9.noar 5.8 MB/s | 1.0 MB 00:00 2024-09-12T02:46:39.323 INFO:teuthology.orchestra.run.smithi146.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-12T02:46:39.365 INFO:teuthology.orchestra.run.smithi146.stdout:(130/156): python3-grpcio-1.46.7-10.el9.x86_64. 5.3 MB/s | 2.0 MB 00:00 2024-09-12T02:46:39.390 INFO:teuthology.orchestra.run.smithi146.stdout:(131/156): python3-more-itertools-8.12.0-2.el9. 1.1 MB/s | 79 kB 00:00 2024-09-12T02:46:39.415 INFO:teuthology.orchestra.run.smithi146.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-09-12T02:46:39.441 INFO:teuthology.orchestra.run.smithi146.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 659 kB/s | 16 kB 00:00 2024-09-12T02:46:39.499 INFO:teuthology.orchestra.run.smithi146.stdout:(134/156): libarrow-9.0.0-11.el9.x86_64.rpm 4.7 MB/s | 4.4 MB 00:00 2024-09-12T02:46:39.524 INFO:teuthology.orchestra.run.smithi146.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.1 MB/s | 90 kB 00:00 2024-09-12T02:46:39.550 INFO:teuthology.orchestra.run.smithi146.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 563 kB/s | 28 kB 00:00 2024-09-12T02:46:39.575 INFO:teuthology.orchestra.run.smithi146.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 614 kB/s | 31 kB 00:00 2024-09-12T02:46:39.600 INFO:teuthology.orchestra.run.smithi146.stdout:(138/156): python3-pecan-1.4.2-3.el9.noarch.rpm 1.3 MB/s | 272 kB 00:00 2024-09-12T02:46:39.625 INFO:teuthology.orchestra.run.smithi146.stdout:(139/156): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-09-12T02:46:39.651 INFO:teuthology.orchestra.run.smithi146.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 713 kB/s | 36 kB 00:00 2024-09-12T02:46:39.668 INFO:teuthology.orchestra.run.smithi146.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 1.7 MB/s | 74 kB 00:00 2024-09-12T02:46:39.693 INFO:teuthology.orchestra.run.smithi146.stdout:(142/156): python3-routes-2.5.1-5.el9.noarch.rp 1.3 MB/s | 188 kB 00:00 2024-09-12T02:46:39.718 INFO:teuthology.orchestra.run.smithi146.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-09-12T02:46:39.743 INFO:teuthology.orchestra.run.smithi146.stdout:(144/156): python3-webob-1.8.8-2.el9.noarch.rpm 2.4 MB/s | 230 kB 00:00 2024-09-12T02:46:39.769 INFO:teuthology.orchestra.run.smithi146.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 399 kB/s | 20 kB 00:00 2024-09-12T02:46:39.794 INFO:teuthology.orchestra.run.smithi146.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-12T02:46:39.819 INFO:teuthology.orchestra.run.smithi146.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-09-12T02:46:39.870 INFO:teuthology.orchestra.run.smithi146.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-09-12T02:46:39.978 INFO:teuthology.orchestra.run.smithi146.stdout:(149/156): NetworkManager-1.49.90-1.el9.x86_64. 21 MB/s | 2.3 MB 00:00 2024-09-12T02:46:40.087 INFO:teuthology.orchestra.run.smithi146.stdout:(150/156): librados2-19.1.1-298.gb53b6d3e.el9.x 12 MB/s | 3.4 MB 00:00 2024-09-12T02:46:40.129 INFO:teuthology.orchestra.run.smithi146.stdout:(151/156): NetworkManager-libnm-1.49.90-1.el9.x 12 MB/s | 1.8 MB 00:00 2024-09-12T02:46:40.196 INFO:teuthology.orchestra.run.smithi146.stdout:(152/156): librbd1-19.1.1-298.gb53b6d3e.el9.x86 8.4 MB/s | 3.2 MB 00:00 2024-09-12T02:46:40.221 INFO:teuthology.orchestra.run.smithi146.stdout:(153/156): NetworkManager-team-1.49.90-1.el9.x8 305 kB/s | 41 kB 00:00 2024-09-12T02:46:40.246 INFO:teuthology.orchestra.run.smithi146.stdout:(154/156): NetworkManager-tui-1.49.90-1.el9.x86 2.1 MB/s | 249 kB 00:00 2024-09-12T02:46:40.296 INFO:teuthology.orchestra.run.smithi146.stdout:(155/156): openssl-3.2.2-6.el9.x86_64.rpm 14 MB/s | 1.4 MB 00:00 2024-09-12T02:46:40.355 INFO:teuthology.orchestra.run.smithi146.stdout:(156/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 18 MB/s | 2.4 MB 00:00 2024-09-12T02:46:40.361 INFO:teuthology.orchestra.run.smithi146.stdout:-------------------------------------------------------------------------------- 2024-09-12T02:46:40.361 INFO:teuthology.orchestra.run.smithi146.stdout:Total 11 MB/s | 226 MB 00:20 2024-09-12T02:46:41.537 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction check 2024-09-12T02:46:41.689 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction check succeeded. 2024-09-12T02:46:41.690 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction test 2024-09-12T02:46:41.852 INFO:teuthology.orchestra.run.smithi148.stdout: Preparing : 1/1 2024-09-12T02:46:41.955 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-12T02:46:41.993 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-12T02:46:42.055 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-12T02:46:42.088 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-12T02:46:42.114 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-12T02:46:42.156 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-12T02:46:42.481 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-12T02:46:42.591 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T02:46:42.713 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T02:46:42.750 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T02:46:42.858 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T02:46:42.887 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-12T02:46:43.033 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-12T02:46:43.053 INFO:teuthology.orchestra.run.smithi005.stdout: Preparing : 1/1 2024-09-12T02:46:43.084 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-12T02:46:43.112 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T02:46:43.186 INFO:teuthology.orchestra.run.smithi005.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-12T02:46:43.217 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T02:46:43.225 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-12T02:46:43.246 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-12T02:46:43.279 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-12T02:46:43.309 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-12T02:46:43.323 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-12T02:46:43.331 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-12T02:46:43.349 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-12T02:46:43.353 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-12T02:46:43.402 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-12T02:46:43.492 INFO:teuthology.orchestra.run.smithi146.stdout:Transaction test succeeded. 2024-09-12T02:46:43.493 INFO:teuthology.orchestra.run.smithi146.stdout:Running transaction 2024-09-12T02:46:43.532 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-12T02:46:43.675 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-12T02:46:43.749 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-12T02:46:43.779 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-12T02:46:43.880 INFO:teuthology.orchestra.run.smithi005.stdout: Upgrading : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T02:46:43.889 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T02:46:43.964 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T02:46:44.008 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T02:46:44.042 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T02:46:44.135 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T02:46:44.163 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-12T02:46:44.188 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T02:46:44.300 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-12T02:46:44.345 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-12T02:46:44.403 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-12T02:46:44.415 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-12T02:46:44.430 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T02:46:44.445 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-12T02:46:44.488 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-12T02:46:44.536 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T02:46:44.561 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-12T02:46:44.564 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-12T02:46:44.605 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-12T02:46:44.628 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-12T02:46:44.645 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-12T02:46:44.657 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-12T02:46:44.681 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-12T02:46:44.689 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-12T02:46:44.721 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-12T02:46:44.749 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-12T02:46:44.818 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-12T02:46:44.871 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-12T02:46:44.876 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-12T02:46:44.901 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-12T02:46:44.945 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-12T02:46:45.018 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-12T02:46:45.041 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-12T02:46:45.051 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 34/164 2024-09-12T02:46:45.094 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 35/164 2024-09-12T02:46:45.123 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 36/164 2024-09-12T02:46:45.155 INFO:teuthology.orchestra.run.smithi005.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-12T02:46:45.194 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-12T02:46:45.238 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-12T02:46:45.274 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T02:46:45.312 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-12T02:46:45.349 INFO:teuthology.orchestra.run.smithi005.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T02:46:45.444 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-12T02:46:45.498 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-12T02:46:45.577 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T02:46:45.692 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-12T02:46:45.788 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-12T02:46:45.821 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-12T02:46:45.856 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-12T02:46:45.932 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-12T02:46:45.983 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-12T02:46:46.022 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-12T02:46:46.038 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-12T02:46:46.066 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-12T02:46:46.099 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-12T02:46:46.119 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-12T02:46:46.126 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-12T02:46:46.148 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-12T02:46:46.182 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-12T02:46:46.189 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-12T02:46:46.229 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 46/164 2024-09-12T02:46:46.250 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-12T02:46:46.270 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-12T02:46:46.280 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-12T02:46:46.323 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-12T02:46:46.403 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-12T02:46:46.425 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-12T02:46:46.433 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 34/164 2024-09-12T02:46:46.473 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-12T02:46:46.477 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 35/164 2024-09-12T02:46:46.505 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 36/164 2024-09-12T02:46:46.536 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-12T02:46:46.579 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-12T02:46:46.609 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-12T02:46:46.629 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-12T02:46:46.641 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-12T02:46:46.677 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-12T02:46:46.706 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-12T02:46:46.723 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-12T02:46:46.760 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-12T02:46:46.802 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-12T02:46:46.845 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-12T02:46:46.852 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-12T02:46:46.908 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-12T02:46:46.990 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-12T02:46:47.111 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-12T02:46:47.373 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-12T02:46:47.463 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-12T02:46:47.552 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-12T02:46:47.555 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-12T02:46:47.581 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-12T02:46:47.589 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-12T02:46:47.620 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-12T02:46:47.632 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-12T02:46:47.646 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-09-12T02:46:47.671 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 46/164 2024-09-12T02:46:47.671 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-12T02:46:47.713 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-12T02:46:47.745 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-12T02:46:47.872 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-12T02:46:47.898 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-12T02:46:47.911 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-12T02:46:47.939 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-12T02:46:47.957 INFO:teuthology.orchestra.run.smithi005.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-12T02:46:47.973 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-12T02:46:48.013 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-12T02:46:48.013 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-12T02:46:48.062 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-12T02:46:48.088 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-12T02:46:48.090 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-12T02:46:48.125 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-12T02:46:48.125 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-12T02:46:48.158 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-12T02:46:48.161 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-12T02:46:48.191 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-12T02:46:48.209 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-12T02:46:48.226 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-12T02:46:48.243 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-12T02:46:48.287 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-12T02:46:48.312 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-12T02:46:48.329 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-12T02:46:48.356 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-12T02:46:48.387 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-12T02:46:48.481 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-12T02:46:48.490 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-12T02:46:48.540 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-12T02:46:48.739 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-12T02:46:48.899 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-12T02:46:49.009 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-12T02:46:49.056 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-12T02:46:49.097 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-12T02:46:49.129 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-09-12T02:46:49.154 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-12T02:46:49.251 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-12T02:46:49.251 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-12T02:46:49.294 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-12T02:46:49.327 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-12T02:46:49.386 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-12T02:46:49.425 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-12T02:46:49.428 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-12T02:46:49.465 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-12T02:46:49.474 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-12T02:46:49.507 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-12T02:46:49.518 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-12T02:46:49.546 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-12T02:46:49.579 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-12T02:46:49.606 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-12T02:46:49.611 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-12T02:46:49.638 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-12T02:46:49.654 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-12T02:46:49.687 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-12T02:46:49.729 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-12T02:46:49.763 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-12T02:46:49.822 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-12T02:46:49.852 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-12T02:46:49.854 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-12T02:46:49.911 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-12T02:46:49.923 INFO:teuthology.orchestra.run.smithi146.stdout: Preparing : 1/1 2024-09-12T02:46:49.939 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-12T02:46:49.942 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-12T02:46:50.041 INFO:teuthology.orchestra.run.smithi146.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-12T02:46:50.049 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-12T02:46:50.080 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-12T02:46:50.102 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-12T02:46:50.133 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-12T02:46:50.167 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-12T02:46:50.193 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-12T02:46:50.236 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-12T02:46:50.567 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-12T02:46:50.597 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-12T02:46:50.704 INFO:teuthology.orchestra.run.smithi146.stdout: Upgrading : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T02:46:50.709 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-12T02:46:50.815 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T02:46:50.845 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T02:46:50.861 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-12T02:46:50.907 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-12T02:46:50.929 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T02:46:50.941 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-12T02:46:50.957 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-12T02:46:51.052 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-12T02:46:51.105 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-12T02:46:51.108 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-12T02:46:51.150 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-12T02:46:51.171 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-12T02:46:51.207 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T02:46:51.240 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-12T02:46:51.276 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-12T02:46:51.311 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T02:46:51.333 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-12T02:46:51.396 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 14/164 2024-09-12T02:46:51.417 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-12T02:46:51.437 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-12T02:46:51.460 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-12T02:46:51.482 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-12T02:46:51.563 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-12T02:46:51.635 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-12T02:46:51.771 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-12T02:46:51.899 INFO:teuthology.orchestra.run.smithi146.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-12T02:46:51.933 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-12T02:46:51.998 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T02:46:52.025 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-12T02:46:52.052 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-12T02:46:52.086 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-12T02:46:52.087 INFO:teuthology.orchestra.run.smithi146.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T02:46:52.251 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-12T02:46:52.310 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-12T02:46:52.371 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-12T02:46:52.395 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-12T02:46:52.427 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-12T02:46:52.440 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-12T02:46:52.524 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-12T02:46:52.531 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-12T02:46:52.562 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 102/164 2024-09-12T02:46:52.567 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-12T02:46:52.597 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-12T02:46:52.603 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-12T02:46:52.716 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-12T02:46:52.784 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-12T02:46:52.824 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-12T02:46:52.884 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-12T02:46:52.925 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-12T02:46:52.957 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-12T02:46:53.066 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-12T02:46:53.068 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-12T02:46:53.110 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-12T02:46:53.138 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-12T02:46:53.177 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-12T02:46:53.183 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-12T02:46:53.203 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 106/164 2024-09-12T02:46:53.237 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-12T02:46:53.312 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-12T02:46:53.346 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 34/164 2024-09-12T02:46:53.389 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 35/164 2024-09-12T02:46:53.418 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 36/164 2024-09-12T02:46:53.488 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-12T02:46:53.533 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-12T02:46:53.607 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-12T02:46:53.691 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-12T02:46:53.749 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-12T02:46:53.773 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-12T02:46:53.801 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-12T02:46:53.810 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-12T02:46:53.834 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-12T02:46:54.162 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-12T02:46:54.213 INFO:teuthology.orchestra.run.smithi005.stdout: Upgrading : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-12T02:46:54.314 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-12T02:46:54.337 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-12T02:46:54.343 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 102/164 2024-09-12T02:46:54.380 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-12T02:46:54.422 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-12T02:46:54.467 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-12T02:46:54.501 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-12T02:46:54.540 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 46/164 2024-09-12T02:46:54.581 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-12T02:46:54.733 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-12T02:46:54.784 INFO:teuthology.orchestra.run.smithi146.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-12T02:46:54.839 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-12T02:46:54.895 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-12T02:46:54.912 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-12T02:46:54.925 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-12T02:46:54.944 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-12T02:46:54.988 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-12T02:46:54.997 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-12T02:46:55.027 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 106/164 2024-09-12T02:46:55.034 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-12T02:46:55.070 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-12T02:46:55.114 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-12T02:46:55.156 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-12T02:46:55.294 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-12T02:46:55.341 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-12T02:46:55.382 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-12T02:46:55.465 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T02:46:55.485 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-12T02:46:55.499 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T02:46:55.613 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-12T02:46:55.664 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-12T02:46:55.690 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-09-12T02:46:55.715 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-12T02:46:55.790 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-12T02:46:55.916 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-12T02:46:55.963 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-12T02:46:55.991 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-12T02:46:56.025 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-12T02:46:56.043 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T02:46:56.065 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-12T02:46:56.078 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-12T02:46:56.096 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-12T02:46:56.126 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-12T02:46:56.169 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-12T02:46:56.174 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-12T02:46:56.174 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-12T02:46:56.175 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-12T02:46:56.175 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-12T02:46:56.202 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-12T02:46:56.204 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-12T02:46:56.244 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-12T02:46:56.278 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-12T02:46:56.365 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-12T02:46:56.408 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-12T02:46:56.448 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-12T02:46:56.559 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-12T02:46:56.615 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-12T02:46:57.345 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-12T02:46:57.404 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-12T02:46:57.438 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-12T02:46:57.448 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T02:46:57.481 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T02:46:57.537 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-12T02:46:57.593 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-12T02:46:57.638 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-12T02:46:57.725 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-12T02:46:57.757 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-12T02:46:57.941 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-12T02:46:57.971 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-12T02:46:58.049 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-12T02:46:58.061 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T02:46:58.110 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-12T02:46:58.202 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-12T02:46:58.202 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-12T02:46:58.202 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-12T02:46:58.203 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-12T02:46:58.245 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-12T02:46:58.702 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-12T02:46:58.811 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-12T02:47:00.045 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-12T02:47:00.124 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-12T02:47:00.152 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-12T02:47:00.185 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-12T02:47:00.496 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-12T02:47:00.540 INFO:teuthology.orchestra.run.smithi146.stdout: Upgrading : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-12T02:47:00.631 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 101/164 2024-09-12T02:47:00.661 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 102/164 2024-09-12T02:47:00.697 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-12T02:47:01.182 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-12T02:47:01.368 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-12T02:47:01.589 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 105/164 2024-09-12T02:47:01.619 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 106/164 2024-09-12T02:47:03.918 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T02:47:03.951 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T02:47:04.496 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 107/164 2024-09-12T02:47:04.538 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-12T02:47:04.618 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 108/164 2024-09-12T02:47:04.618 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-12T02:47:04.619 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-12T02:47:04.619 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-12T02:47:04.655 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-12T02:47:08.006 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-12T02:47:08.006 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /sys 2024-09-12T02:47:08.006 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /proc 2024-09-12T02:47:08.006 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /mnt 2024-09-12T02:47:08.006 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /var/tmp 2024-09-12T02:47:08.006 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /home 2024-09-12T02:47:08.006 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /root 2024-09-12T02:47:08.006 INFO:teuthology.orchestra.run.smithi148.stdout:skipping the directory /tmp 2024-09-12T02:47:08.007 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-12T02:47:08.174 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-12T02:47:08.190 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-12T02:47:08.190 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:47:08.190 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-12T02:47:08.190 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-12T02:47:09.677 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-12T02:47:09.704 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-12T02:47:09.704 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:47:09.704 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-12T02:47:09.704 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-12T02:47:09.704 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-12T02:47:09.704 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-12T02:47:09.928 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-12T02:47:09.951 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-12T02:47:09.951 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:47:09.951 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-12T02:47:09.951 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-12T02:47:09.951 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-12T02:47:09.951 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-12T02:47:10.378 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-12T02:47:10.390 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-12T02:47:10.390 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /sys 2024-09-12T02:47:10.391 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /proc 2024-09-12T02:47:10.391 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /mnt 2024-09-12T02:47:10.391 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /var/tmp 2024-09-12T02:47:10.391 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /home 2024-09-12T02:47:10.391 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /root 2024-09-12T02:47:10.391 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /tmp 2024-09-12T02:47:10.391 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-12T02:47:10.402 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-12T02:47:10.403 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:47:10.403 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-12T02:47:10.403 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-12T02:47:10.403 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-12T02:47:10.403 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-12T02:47:10.508 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-12T02:47:10.557 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-12T02:47:10.575 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-12T02:47:10.592 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-12T02:47:10.592 INFO:teuthology.orchestra.run.smithi005.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:47:10.592 INFO:teuthology.orchestra.run.smithi005.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-12T02:47:10.592 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-12T02:47:10.611 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-12T02:47:10.796 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-12T02:47:10.845 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-12T02:47:10.914 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-12T02:47:11.003 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-12T02:47:11.141 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-12T02:47:11.187 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-12T02:47:11.219 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-12T02:47:11.247 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-12T02:47:11.697 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-12T02:47:11.728 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-12T02:47:11.760 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 126/164 2024-09-12T02:47:11.821 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 127/164 2024-09-12T02:47:11.851 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-12T02:47:12.130 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-12T02:47:12.158 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-12T02:47:12.158 INFO:teuthology.orchestra.run.smithi005.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:47:12.158 INFO:teuthology.orchestra.run.smithi005.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-12T02:47:12.158 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-12T02:47:12.158 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-12T02:47:12.158 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-12T02:47:12.397 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-12T02:47:12.422 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-12T02:47:12.422 INFO:teuthology.orchestra.run.smithi005.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:47:12.422 INFO:teuthology.orchestra.run.smithi005.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-12T02:47:12.422 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-12T02:47:12.422 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-12T02:47:12.422 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-12T02:47:12.888 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-12T02:47:12.889 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-12T02:47:12.914 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-12T02:47:12.914 INFO:teuthology.orchestra.run.smithi005.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:47:12.914 INFO:teuthology.orchestra.run.smithi005.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-12T02:47:12.914 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-12T02:47:12.914 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-12T02:47:12.915 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-12T02:47:12.928 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-12T02:47:13.020 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-12T02:47:13.078 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-12T02:47:13.142 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-12T02:47:13.345 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-12T02:47:13.391 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-12T02:47:13.471 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-12T02:47:13.561 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-12T02:47:13.712 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-12T02:47:13.767 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-12T02:47:13.797 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-12T02:47:13.825 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-12T02:47:13.889 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-12T02:47:13.929 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-12T02:47:14.030 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-12T02:47:14.190 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 131/164 2024-09-12T02:47:14.222 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-12T02:47:14.246 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-12T02:47:14.246 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:47:14.246 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-12T02:47:14.246 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-12T02:47:14.246 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-12T02:47:14.247 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-12T02:47:14.290 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-12T02:47:14.298 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-12T02:47:14.310 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-12T02:47:14.316 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-12T02:47:14.348 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 126/164 2024-09-12T02:47:14.414 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 134/164 2024-09-12T02:47:14.415 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 127/164 2024-09-12T02:47:14.438 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-12T02:47:14.462 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 135/164 2024-09-12T02:47:15.384 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-12T02:47:15.409 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-12T02:47:15.432 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-12T02:47:15.432 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:47:15.432 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-12T02:47:15.432 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-12T02:47:15.432 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-12T02:47:15.433 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-12T02:47:15.484 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-12T02:47:15.507 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-12T02:47:15.507 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:47:15.507 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-12T02:47:15.507 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-12T02:47:15.545 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-12T02:47:15.560 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-12T02:47:15.583 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-12T02:47:15.583 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:47:15.583 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-12T02:47:15.584 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-12T02:47:15.584 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-12T02:47:15.584 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-12T02:47:15.593 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-12T02:47:15.886 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-12T02:47:15.909 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-12T02:47:15.909 INFO:teuthology.orchestra.run.smithi148.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:47:15.909 INFO:teuthology.orchestra.run.smithi148.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-12T02:47:15.909 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-12T02:47:15.909 INFO:teuthology.orchestra.run.smithi148.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-12T02:47:15.909 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-12T02:47:16.268 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 109/164 2024-09-12T02:47:16.268 INFO:teuthology.orchestra.run.smithi146.stdout:skipping the directory /sys 2024-09-12T02:47:16.268 INFO:teuthology.orchestra.run.smithi146.stdout:skipping the directory /proc 2024-09-12T02:47:16.268 INFO:teuthology.orchestra.run.smithi146.stdout:skipping the directory /mnt 2024-09-12T02:47:16.269 INFO:teuthology.orchestra.run.smithi146.stdout:skipping the directory /var/tmp 2024-09-12T02:47:16.269 INFO:teuthology.orchestra.run.smithi146.stdout:skipping the directory /home 2024-09-12T02:47:16.269 INFO:teuthology.orchestra.run.smithi146.stdout:skipping the directory /root 2024-09-12T02:47:16.269 INFO:teuthology.orchestra.run.smithi146.stdout:skipping the directory /tmp 2024-09-12T02:47:16.269 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-12T02:47:16.449 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-12T02:47:16.465 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 110/164 2024-09-12T02:47:16.465 INFO:teuthology.orchestra.run.smithi146.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:47:16.465 INFO:teuthology.orchestra.run.smithi146.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-12T02:47:16.465 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-12T02:47:16.611 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-12T02:47:16.641 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-12T02:47:16.751 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-12T02:47:16.918 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 131/164 2024-09-12T02:47:16.943 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-12T02:47:16.968 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-12T02:47:16.969 INFO:teuthology.orchestra.run.smithi005.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:47:16.969 INFO:teuthology.orchestra.run.smithi005.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-12T02:47:16.969 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-12T02:47:16.969 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-12T02:47:16.969 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-12T02:47:17.019 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-12T02:47:17.031 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-12T02:47:17.133 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 134/164 2024-09-12T02:47:17.199 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 135/164 2024-09-12T02:47:17.960 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-12T02:47:17.986 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 111/164 2024-09-12T02:47:17.986 INFO:teuthology.orchestra.run.smithi146.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:47:17.987 INFO:teuthology.orchestra.run.smithi146.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-12T02:47:17.987 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-12T02:47:17.987 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-12T02:47:17.987 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-12T02:47:18.181 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-12T02:47:18.211 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-12T02:47:18.213 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-12T02:47:18.235 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 112/164 2024-09-12T02:47:18.235 INFO:teuthology.orchestra.run.smithi146.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:47:18.235 INFO:teuthology.orchestra.run.smithi146.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-12T02:47:18.236 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-12T02:47:18.236 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-12T02:47:18.236 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-12T02:47:18.238 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-12T02:47:18.238 INFO:teuthology.orchestra.run.smithi005.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:47:18.238 INFO:teuthology.orchestra.run.smithi005.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-12T02:47:18.238 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-12T02:47:18.238 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-12T02:47:18.238 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-12T02:47:18.280 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-12T02:47:18.304 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-12T02:47:18.304 INFO:teuthology.orchestra.run.smithi005.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:47:18.304 INFO:teuthology.orchestra.run.smithi005.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-12T02:47:18.304 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-12T02:47:18.356 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-12T02:47:18.380 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-12T02:47:18.380 INFO:teuthology.orchestra.run.smithi005.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:47:18.380 INFO:teuthology.orchestra.run.smithi005.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-12T02:47:18.380 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-12T02:47:18.380 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-12T02:47:18.381 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-12T02:47:18.669 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-12T02:47:18.693 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 113/164 2024-09-12T02:47:18.693 INFO:teuthology.orchestra.run.smithi146.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:47:18.693 INFO:teuthology.orchestra.run.smithi146.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-12T02:47:18.693 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-12T02:47:18.693 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-12T02:47:18.693 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-12T02:47:18.698 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-12T02:47:18.723 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-12T02:47:18.723 INFO:teuthology.orchestra.run.smithi005.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:47:18.723 INFO:teuthology.orchestra.run.smithi005.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-12T02:47:18.723 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-12T02:47:18.723 INFO:teuthology.orchestra.run.smithi005.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-12T02:47:18.723 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-12T02:47:18.800 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-12T02:47:18.857 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-12T02:47:18.912 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-12T02:47:19.098 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-12T02:47:19.178 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-12T02:47:19.265 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-12T02:47:19.354 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-12T02:47:19.494 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-12T02:47:19.554 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-12T02:47:19.585 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-12T02:47:19.613 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-12T02:47:20.013 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-12T02:47:20.045 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 125/164 2024-09-12T02:47:20.076 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 126/164 2024-09-12T02:47:20.138 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 127/164 2024-09-12T02:47:20.169 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-12T02:47:20.439 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 141/164 2024-09-12T02:47:20.481 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 142/164 2024-09-12T02:47:20.521 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 143/164 2024-09-12T02:47:20.553 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-12T02:47:20.581 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-12T02:47:20.761 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 146/164 2024-09-12T02:47:20.811 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-12T02:47:20.841 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 148/164 2024-09-12T02:47:20.879 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-12T02:47:20.925 INFO:teuthology.orchestra.run.smithi148.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-12T02:47:21.207 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 128/164 2024-09-12T02:47:21.261 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-12T02:47:21.515 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T02:47:21.565 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T02:47:21.585 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T02:47:22.223 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 129/164 2024-09-12T02:47:22.253 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-12T02:47:22.355 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 130/164 2024-09-12T02:47:22.430 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 152/164 2024-09-12T02:47:22.517 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 131/164 2024-09-12T02:47:22.527 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-12T02:47:22.547 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-12T02:47:22.569 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-12T02:47:22.571 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 132/164 2024-09-12T02:47:22.571 INFO:teuthology.orchestra.run.smithi146.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:47:22.571 INFO:teuthology.orchestra.run.smithi146.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-12T02:47:22.571 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-12T02:47:22.571 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-12T02:47:22.571 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-12T02:47:22.631 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-12T02:47:22.642 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 133/164 2024-09-12T02:47:22.658 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-12T02:47:22.698 INFO:teuthology.orchestra.run.smithi148.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-12T02:47:22.723 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-12T02:47:22.741 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 134/164 2024-09-12T02:47:22.749 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-12T02:47:22.749 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-12T02:47:22.804 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 135/164 2024-09-12T02:47:22.824 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-12T02:47:22.824 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-12T02:47:22.892 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-12T02:47:22.917 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-12T02:47:22.922 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T02:47:22.922 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T02:47:23.110 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T02:47:23.110 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-12T02:47:23.194 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-12T02:47:23.257 INFO:teuthology.orchestra.run.smithi148.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T02:47:23.578 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 141/164 2024-09-12T02:47:23.619 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 142/164 2024-09-12T02:47:23.661 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 143/164 2024-09-12T02:47:23.692 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-12T02:47:23.719 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-12T02:47:23.734 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-12T02:47:23.767 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-12T02:47:23.790 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 137/164 2024-09-12T02:47:23.790 INFO:teuthology.orchestra.run.smithi146.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:47:23.791 INFO:teuthology.orchestra.run.smithi146.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-12T02:47:23.791 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-12T02:47:23.791 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-12T02:47:23.791 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-12T02:47:23.834 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-12T02:47:23.857 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 138/164 2024-09-12T02:47:23.857 INFO:teuthology.orchestra.run.smithi146.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:47:23.857 INFO:teuthology.orchestra.run.smithi146.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-12T02:47:23.857 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-12T02:47:23.901 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-12T02:47:23.928 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 139/164 2024-09-12T02:47:23.929 INFO:teuthology.orchestra.run.smithi146.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:47:23.929 INFO:teuthology.orchestra.run.smithi146.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-12T02:47:23.929 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-12T02:47:23.929 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-12T02:47:23.929 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-12T02:47:23.933 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 146/164 2024-09-12T02:47:23.982 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-12T02:47:24.013 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 148/164 2024-09-12T02:47:24.042 INFO:teuthology.orchestra.run.smithi005.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-12T02:47:24.088 INFO:teuthology.orchestra.run.smithi005.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-12T02:47:24.244 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-12T02:47:24.267 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 140/164 2024-09-12T02:47:24.268 INFO:teuthology.orchestra.run.smithi146.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-12T02:47:24.268 INFO:teuthology.orchestra.run.smithi146.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-12T02:47:24.268 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-12T02:47:24.268 INFO:teuthology.orchestra.run.smithi146.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-12T02:47:24.268 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-12T02:47:24.725 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T02:47:24.761 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T02:47:24.782 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T02:47:25.705 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 152/164 2024-09-12T02:47:25.800 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-12T02:47:25.882 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-12T02:47:25.978 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-12T02:47:26.011 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-12T02:47:26.037 INFO:teuthology.orchestra.run.smithi005.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-12T02:47:26.063 INFO:teuthology.orchestra.run.smithi005.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-12T02:47:26.064 INFO:teuthology.orchestra.run.smithi005.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-12T02:47:26.147 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-12T02:47:26.147 INFO:teuthology.orchestra.run.smithi005.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-12T02:47:26.231 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-12T02:47:26.256 INFO:teuthology.orchestra.run.smithi005.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-12T02:47:26.262 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T02:47:26.262 INFO:teuthology.orchestra.run.smithi005.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T02:47:26.441 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T02:47:26.441 INFO:teuthology.orchestra.run.smithi005.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-12T02:47:26.534 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-12T02:47:26.601 INFO:teuthology.orchestra.run.smithi005.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T02:47:28.814 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 141/164 2024-09-12T02:47:28.826 INFO:teuthology.orchestra.run.smithi148.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T02:47:28.826 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/164 2024-09-12T02:47:28.826 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2/164 2024-09-12T02:47:28.826 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 3/164 2024-09-12T02:47:28.826 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 4/164 2024-09-12T02:47:28.826 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 5/164 2024-09-12T02:47:28.826 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 6/164 2024-09-12T02:47:28.826 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 7/164 2024-09-12T02:47:28.826 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T02:47:28.826 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T02:47:28.827 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-12T02:47:28.827 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 11/164 2024-09-12T02:47:28.827 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 12/164 2024-09-12T02:47:28.827 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T02:47:28.827 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 14/164 2024-09-12T02:47:28.828 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 15/164 2024-09-12T02:47:28.828 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 16/164 2024-09-12T02:47:28.828 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 17/164 2024-09-12T02:47:28.828 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 18/164 2024-09-12T02:47:28.828 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 19/164 2024-09-12T02:47:28.828 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 20/164 2024-09-12T02:47:28.828 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 21/164 2024-09-12T02:47:28.828 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 22/164 2024-09-12T02:47:28.828 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 23/164 2024-09-12T02:47:28.828 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 24/164 2024-09-12T02:47:28.829 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 25/164 2024-09-12T02:47:28.829 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 26/164 2024-09-12T02:47:28.829 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 27/164 2024-09-12T02:47:28.829 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 28/164 2024-09-12T02:47:28.829 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 29/164 2024-09-12T02:47:28.829 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 30/164 2024-09-12T02:47:28.841 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 31/164 2024-09-12T02:47:28.841 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 32/164 2024-09-12T02:47:28.841 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 33/164 2024-09-12T02:47:28.841 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 34/164 2024-09-12T02:47:28.841 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 35/164 2024-09-12T02:47:28.841 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 36/164 2024-09-12T02:47:28.841 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 37/164 2024-09-12T02:47:28.841 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 38/164 2024-09-12T02:47:28.841 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 39/164 2024-09-12T02:47:28.841 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-12T02:47:28.842 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-12T02:47:28.842 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-12T02:47:28.842 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-12T02:47:28.842 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-12T02:47:28.842 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-12T02:47:28.842 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-12T02:47:28.842 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-12T02:47:28.842 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-12T02:47:28.842 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-12T02:47:28.842 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-12T02:47:28.842 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-12T02:47:28.842 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-12T02:47:28.843 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-12T02:47:28.843 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-12T02:47:28.843 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-12T02:47:28.843 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-12T02:47:28.843 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-12T02:47:28.843 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-12T02:47:28.843 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-12T02:47:28.843 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-12T02:47:28.843 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-12T02:47:28.843 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-12T02:47:28.843 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-12T02:47:28.843 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-12T02:47:28.844 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-12T02:47:28.844 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-12T02:47:28.844 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-12T02:47:28.844 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-12T02:47:28.844 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-12T02:47:28.844 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-12T02:47:28.844 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-12T02:47:28.844 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-12T02:47:28.844 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-12T02:47:28.844 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-12T02:47:28.844 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-12T02:47:28.844 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-12T02:47:28.845 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-12T02:47:28.845 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-12T02:47:28.845 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-12T02:47:28.845 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-12T02:47:28.845 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-12T02:47:28.845 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-12T02:47:28.845 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-12T02:47:28.845 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-12T02:47:28.845 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-12T02:47:28.845 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-12T02:47:28.845 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-12T02:47:28.845 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-12T02:47:28.845 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-12T02:47:28.846 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-12T02:47:28.846 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-12T02:47:28.846 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-12T02:47:28.846 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-12T02:47:28.846 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-12T02:47:28.846 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-12T02:47:28.846 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-12T02:47:28.846 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-12T02:47:28.846 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-12T02:47:28.846 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-12T02:47:28.846 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-12T02:47:28.846 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-12T02:47:28.846 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-12T02:47:28.847 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-12T02:47:28.847 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-12T02:47:28.847 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-12T02:47:28.847 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-12T02:47:28.847 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/164 2024-09-12T02:47:28.847 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-12T02:47:28.847 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-12T02:47:28.847 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-12T02:47:28.847 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-12T02:47:28.847 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-12T02:47:28.847 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-12T02:47:28.847 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-12T02:47:28.847 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-12T02:47:28.848 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-12T02:47:28.848 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-12T02:47:28.848 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-12T02:47:28.848 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-12T02:47:28.848 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-12T02:47:28.848 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-12T02:47:28.848 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-12T02:47:28.848 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-12T02:47:28.848 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-12T02:47:28.848 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-12T02:47:28.848 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-12T02:47:28.848 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-12T02:47:28.848 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-12T02:47:28.849 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-12T02:47:28.849 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-12T02:47:28.849 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-12T02:47:28.849 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-12T02:47:28.849 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-12T02:47:28.849 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-12T02:47:28.849 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-12T02:47:28.849 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-12T02:47:28.849 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-12T02:47:28.849 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-12T02:47:28.849 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-12T02:47:28.849 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-12T02:47:28.849 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-12T02:47:28.850 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-12T02:47:28.850 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-12T02:47:28.850 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-12T02:47:28.850 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-12T02:47:28.850 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-12T02:47:28.850 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-12T02:47:28.850 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-12T02:47:28.850 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 149/164 2024-09-12T02:47:28.850 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-12T02:47:28.850 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 151/164 2024-09-12T02:47:28.850 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-12T02:47:28.850 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-12T02:47:28.851 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-12T02:47:28.851 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-12T02:47:28.851 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-12T02:47:28.851 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-12T02:47:28.851 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-12T02:47:28.851 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-12T02:47:28.851 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-12T02:47:28.851 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-12T02:47:28.851 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-12T02:47:28.851 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-12T02:47:28.855 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 142/164 2024-09-12T02:47:28.904 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 143/164 2024-09-12T02:47:28.937 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-12T02:47:28.972 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-12T02:47:29.155 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 146/164 2024-09-12T02:47:29.202 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-12T02:47:29.232 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 148/164 2024-09-12T02:47:29.262 INFO:teuthology.orchestra.run.smithi146.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-12T02:47:29.308 INFO:teuthology.orchestra.run.smithi146.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-12T02:47:29.882 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T02:47:29.922 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T02:47:29.942 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-12T02:47:30.796 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 152/164 2024-09-12T02:47:30.868 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-12T02:47:30.909 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-12T02:47:30.991 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-12T02:47:31.021 INFO:teuthology.orchestra.run.smithi146.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-12T02:47:31.054 INFO:teuthology.orchestra.run.smithi146.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-12T02:47:31.081 INFO:teuthology.orchestra.run.smithi146.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-12T02:47:31.081 INFO:teuthology.orchestra.run.smithi146.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-12T02:47:31.173 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-12T02:47:31.173 INFO:teuthology.orchestra.run.smithi146.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-12T02:47:31.265 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-12T02:47:31.290 INFO:teuthology.orchestra.run.smithi146.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-12T02:47:31.295 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T02:47:31.295 INFO:teuthology.orchestra.run.smithi146.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T02:47:31.475 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-12T02:47:31.475 INFO:teuthology.orchestra.run.smithi146.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-12T02:47:31.552 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-12T02:47:31.565 INFO:teuthology.orchestra.run.smithi148.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T02:47:31.565 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-12T02:47:31.565 INFO:teuthology.orchestra.run.smithi148.stdout:Upgraded: 2024-09-12T02:47:31.566 INFO:teuthology.orchestra.run.smithi148.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-12T02:47:31.566 INFO:teuthology.orchestra.run.smithi148.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-12T02:47:31.566 INFO:teuthology.orchestra.run.smithi148.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-12T02:47:31.566 INFO:teuthology.orchestra.run.smithi148.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-12T02:47:31.566 INFO:teuthology.orchestra.run.smithi148.stdout: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:47:31.566 INFO:teuthology.orchestra.run.smithi148.stdout: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:47:31.566 INFO:teuthology.orchestra.run.smithi148.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-12T02:47:31.566 INFO:teuthology.orchestra.run.smithi148.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-12T02:47:31.566 INFO:teuthology.orchestra.run.smithi148.stdout:Installed: 2024-09-12T02:47:31.566 INFO:teuthology.orchestra.run.smithi148.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-12T02:47:31.566 INFO:teuthology.orchestra.run.smithi148.stdout: automake-1.16.2-8.el9.noarch 2024-09-12T02:47:31.566 INFO:teuthology.orchestra.run.smithi148.stdout: bison-3.7.4-5.el9.x86_64 2024-09-12T02:47:31.567 INFO:teuthology.orchestra.run.smithi148.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-12T02:47:31.567 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:47:31.567 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:47:31.567 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:47:31.567 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:47:31.567 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:47:31.567 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:47:31.567 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:47:31.567 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:47:31.567 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:47:31.567 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:47:31.567 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:47:31.568 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:47:31.568 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:47:31.568 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:47:31.568 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:47:31.568 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:47:31.568 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:47:31.568 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:47:31.568 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:47:31.568 INFO:teuthology.orchestra.run.smithi148.stdout: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:47:31.568 INFO:teuthology.orchestra.run.smithi148.stdout: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:47:31.568 INFO:teuthology.orchestra.run.smithi148.stdout: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:47:31.568 INFO:teuthology.orchestra.run.smithi148.stdout: cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:47:31.569 INFO:teuthology.orchestra.run.smithi148.stdout: cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:47:31.569 INFO:teuthology.orchestra.run.smithi148.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-12T02:47:31.569 INFO:teuthology.orchestra.run.smithi148.stdout: flex-2.6.4-9.el9.x86_64 2024-09-12T02:47:31.569 INFO:teuthology.orchestra.run.smithi148.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-12T02:47:31.569 INFO:teuthology.orchestra.run.smithi148.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-12T02:47:31.569 INFO:teuthology.orchestra.run.smithi148.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-12T02:47:31.569 INFO:teuthology.orchestra.run.smithi148.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-12T02:47:31.569 INFO:teuthology.orchestra.run.smithi148.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-12T02:47:31.569 INFO:teuthology.orchestra.run.smithi148.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-12T02:47:31.569 INFO:teuthology.orchestra.run.smithi148.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-12T02:47:31.569 INFO:teuthology.orchestra.run.smithi148.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-12T02:47:31.569 INFO:teuthology.orchestra.run.smithi148.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-12T02:47:31.569 INFO:teuthology.orchestra.run.smithi148.stdout: libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:47:31.570 INFO:teuthology.orchestra.run.smithi148.stdout: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:47:31.570 INFO:teuthology.orchestra.run.smithi148.stdout: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:47:31.570 INFO:teuthology.orchestra.run.smithi148.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-12T02:47:31.570 INFO:teuthology.orchestra.run.smithi148.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-12T02:47:31.570 INFO:teuthology.orchestra.run.smithi148.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-12T02:47:31.570 INFO:teuthology.orchestra.run.smithi148.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-12T02:47:31.570 INFO:teuthology.orchestra.run.smithi148.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-12T02:47:31.570 INFO:teuthology.orchestra.run.smithi148.stdout: librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:47:31.570 INFO:teuthology.orchestra.run.smithi148.stdout: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:47:31.570 INFO:teuthology.orchestra.run.smithi148.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-12T02:47:31.570 INFO:teuthology.orchestra.run.smithi148.stdout: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:47:31.570 INFO:teuthology.orchestra.run.smithi148.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-12T02:47:31.571 INFO:teuthology.orchestra.run.smithi148.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-12T02:47:31.571 INFO:teuthology.orchestra.run.smithi148.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-12T02:47:31.571 INFO:teuthology.orchestra.run.smithi148.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-12T02:47:31.571 INFO:teuthology.orchestra.run.smithi148.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-12T02:47:31.571 INFO:teuthology.orchestra.run.smithi148.stdout: lua-5.4.4-4.el9.x86_64 2024-09-12T02:47:31.571 INFO:teuthology.orchestra.run.smithi148.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-12T02:47:31.571 INFO:teuthology.orchestra.run.smithi148.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-12T02:47:31.571 INFO:teuthology.orchestra.run.smithi148.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-12T02:47:31.571 INFO:teuthology.orchestra.run.smithi148.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-12T02:47:31.571 INFO:teuthology.orchestra.run.smithi148.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-12T02:47:31.571 INFO:teuthology.orchestra.run.smithi148.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-12T02:47:31.571 INFO:teuthology.orchestra.run.smithi148.stdout: postgresql-13.16-1.el9.x86_64 2024-09-12T02:47:31.572 INFO:teuthology.orchestra.run.smithi148.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-12T02:47:31.572 INFO:teuthology.orchestra.run.smithi148.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-12T02:47:31.572 INFO:teuthology.orchestra.run.smithi148.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-12T02:47:31.572 INFO:teuthology.orchestra.run.smithi148.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-12T02:47:31.572 INFO:teuthology.orchestra.run.smithi148.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-12T02:47:31.572 INFO:teuthology.orchestra.run.smithi148.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-12T02:47:31.572 INFO:teuthology.orchestra.run.smithi148.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-12T02:47:31.572 INFO:teuthology.orchestra.run.smithi148.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-12T02:47:31.572 INFO:teuthology.orchestra.run.smithi148.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-12T02:47:31.572 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-12T02:47:31.572 INFO:teuthology.orchestra.run.smithi148.stdout: python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:47:31.572 INFO:teuthology.orchestra.run.smithi148.stdout: python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:47:31.572 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:47:31.572 INFO:teuthology.orchestra.run.smithi148.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-12T02:47:31.573 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-12T02:47:31.573 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-12T02:47:31.573 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-12T02:47:31.573 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-12T02:47:31.573 INFO:teuthology.orchestra.run.smithi148.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-12T02:47:31.573 INFO:teuthology.orchestra.run.smithi148.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-12T02:47:31.573 INFO:teuthology.orchestra.run.smithi148.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-12T02:47:31.573 INFO:teuthology.orchestra.run.smithi148.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-12T02:47:31.573 INFO:teuthology.orchestra.run.smithi148.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-12T02:47:31.573 INFO:teuthology.orchestra.run.smithi148.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-12T02:47:31.573 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-12T02:47:31.573 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-12T02:47:31.573 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-12T02:47:31.574 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-12T02:47:31.574 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-12T02:47:31.574 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-12T02:47:31.574 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-12T02:47:31.574 INFO:teuthology.orchestra.run.smithi148.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-12T02:47:31.574 INFO:teuthology.orchestra.run.smithi148.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-12T02:47:31.574 INFO:teuthology.orchestra.run.smithi148.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-12T02:47:31.574 INFO:teuthology.orchestra.run.smithi148.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-12T02:47:31.574 INFO:teuthology.orchestra.run.smithi148.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-12T02:47:31.574 INFO:teuthology.orchestra.run.smithi148.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-12T02:47:31.574 INFO:teuthology.orchestra.run.smithi148.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-12T02:47:31.574 INFO:teuthology.orchestra.run.smithi148.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-12T02:47:31.574 INFO:teuthology.orchestra.run.smithi148.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-12T02:47:31.575 INFO:teuthology.orchestra.run.smithi148.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-12T02:47:31.575 INFO:teuthology.orchestra.run.smithi148.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-12T02:47:31.575 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-12T02:47:31.575 INFO:teuthology.orchestra.run.smithi148.stdout: python3-ply-3.11-14.el9.noarch 2024-09-12T02:47:31.575 INFO:teuthology.orchestra.run.smithi148.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-12T02:47:31.575 INFO:teuthology.orchestra.run.smithi148.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-12T02:47:31.575 INFO:teuthology.orchestra.run.smithi148.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-12T02:47:31.575 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-12T02:47:31.575 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-12T02:47:31.575 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-12T02:47:31.575 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-12T02:47:31.575 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-12T02:47:31.575 INFO:teuthology.orchestra.run.smithi148.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-12T02:47:31.576 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:47:31.576 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:47:31.576 INFO:teuthology.orchestra.run.smithi148.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-12T02:47:31.576 INFO:teuthology.orchestra.run.smithi148.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-12T02:47:31.576 INFO:teuthology.orchestra.run.smithi148.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-12T02:47:31.576 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:47:31.576 INFO:teuthology.orchestra.run.smithi148.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-12T02:47:31.576 INFO:teuthology.orchestra.run.smithi148.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-12T02:47:31.576 INFO:teuthology.orchestra.run.smithi148.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-12T02:47:31.576 INFO:teuthology.orchestra.run.smithi148.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-12T02:47:31.576 INFO:teuthology.orchestra.run.smithi148.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-12T02:47:31.576 INFO:teuthology.orchestra.run.smithi148.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-12T02:47:31.576 INFO:teuthology.orchestra.run.smithi148.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-12T02:47:31.577 INFO:teuthology.orchestra.run.smithi148.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-12T02:47:31.577 INFO:teuthology.orchestra.run.smithi148.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-12T02:47:31.577 INFO:teuthology.orchestra.run.smithi148.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-12T02:47:31.577 INFO:teuthology.orchestra.run.smithi148.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-12T02:47:31.577 INFO:teuthology.orchestra.run.smithi148.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-12T02:47:31.577 INFO:teuthology.orchestra.run.smithi148.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-12T02:47:31.577 INFO:teuthology.orchestra.run.smithi148.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-12T02:47:31.577 INFO:teuthology.orchestra.run.smithi148.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-12T02:47:31.577 INFO:teuthology.orchestra.run.smithi148.stdout: rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:47:31.577 INFO:teuthology.orchestra.run.smithi148.stdout: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:47:31.577 INFO:teuthology.orchestra.run.smithi148.stdout: rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:47:31.577 INFO:teuthology.orchestra.run.smithi148.stdout: re2-1:20211101-20.el9.x86_64 2024-09-12T02:47:31.577 INFO:teuthology.orchestra.run.smithi148.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-12T02:47:31.577 INFO:teuthology.orchestra.run.smithi148.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-12T02:47:31.578 INFO:teuthology.orchestra.run.smithi148.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-12T02:47:31.578 INFO:teuthology.orchestra.run.smithi148.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-12T02:47:31.578 INFO:teuthology.orchestra.run.smithi148.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-12T02:47:31.578 INFO:teuthology.orchestra.run.smithi148.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-12T02:47:31.578 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-12T02:47:31.578 INFO:teuthology.orchestra.run.smithi148.stdout:Complete! 2024-09-12T02:47:31.614 INFO:teuthology.orchestra.run.smithi146.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T02:47:31.955 DEBUG:teuthology.parallel:result is None 2024-09-12T02:47:32.101 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T02:47:32.101 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/164 2024-09-12T02:47:32.101 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2/164 2024-09-12T02:47:32.101 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 3/164 2024-09-12T02:47:32.101 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 4/164 2024-09-12T02:47:32.101 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 5/164 2024-09-12T02:47:32.101 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 6/164 2024-09-12T02:47:32.101 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 7/164 2024-09-12T02:47:32.101 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T02:47:32.101 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T02:47:32.101 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-12T02:47:32.102 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 11/164 2024-09-12T02:47:32.102 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 12/164 2024-09-12T02:47:32.102 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T02:47:32.102 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 14/164 2024-09-12T02:47:32.102 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 15/164 2024-09-12T02:47:32.102 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 16/164 2024-09-12T02:47:32.102 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 17/164 2024-09-12T02:47:32.102 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 18/164 2024-09-12T02:47:32.102 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 19/164 2024-09-12T02:47:32.102 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 20/164 2024-09-12T02:47:32.102 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 21/164 2024-09-12T02:47:32.102 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 22/164 2024-09-12T02:47:32.102 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 23/164 2024-09-12T02:47:32.103 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 24/164 2024-09-12T02:47:32.103 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 25/164 2024-09-12T02:47:32.116 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 26/164 2024-09-12T02:47:32.117 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 27/164 2024-09-12T02:47:32.117 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 28/164 2024-09-12T02:47:32.117 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 29/164 2024-09-12T02:47:32.117 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 30/164 2024-09-12T02:47:32.117 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 31/164 2024-09-12T02:47:32.117 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 32/164 2024-09-12T02:47:32.117 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 33/164 2024-09-12T02:47:32.117 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 34/164 2024-09-12T02:47:32.117 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 35/164 2024-09-12T02:47:32.117 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 36/164 2024-09-12T02:47:32.117 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 37/164 2024-09-12T02:47:32.117 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 38/164 2024-09-12T02:47:32.118 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 39/164 2024-09-12T02:47:32.118 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-12T02:47:32.118 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-12T02:47:32.118 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-12T02:47:32.118 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-12T02:47:32.118 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-12T02:47:32.118 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-12T02:47:32.118 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-12T02:47:32.118 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-12T02:47:32.118 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-12T02:47:32.118 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-12T02:47:32.118 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-12T02:47:32.118 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-12T02:47:32.119 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-12T02:47:32.119 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-12T02:47:32.119 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-12T02:47:32.119 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-12T02:47:32.119 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-12T02:47:32.119 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-12T02:47:32.119 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-12T02:47:32.119 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-12T02:47:32.119 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-12T02:47:32.119 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-12T02:47:32.119 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-12T02:47:32.119 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-12T02:47:32.119 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-12T02:47:32.120 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-12T02:47:32.120 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-12T02:47:32.120 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-12T02:47:32.120 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-12T02:47:32.120 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-12T02:47:32.120 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-12T02:47:32.120 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-12T02:47:32.120 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-12T02:47:32.121 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-12T02:47:32.122 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-12T02:47:32.122 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-12T02:47:32.122 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-12T02:47:32.122 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-12T02:47:32.122 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-12T02:47:32.122 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-12T02:47:32.122 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-12T02:47:32.122 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-12T02:47:32.122 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-12T02:47:32.122 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-12T02:47:32.122 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-12T02:47:32.122 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-12T02:47:32.123 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-12T02:47:32.124 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-12T02:47:32.124 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-12T02:47:32.124 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-12T02:47:32.124 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-12T02:47:32.124 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-12T02:47:32.124 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-12T02:47:32.124 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-12T02:47:32.124 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-12T02:47:32.125 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-12T02:47:32.125 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-12T02:47:32.125 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-12T02:47:32.125 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-12T02:47:32.125 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-12T02:47:32.125 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-12T02:47:32.125 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-12T02:47:32.125 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-12T02:47:32.125 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-12T02:47:32.125 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-12T02:47:32.125 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-12T02:47:32.125 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-12T02:47:32.125 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/164 2024-09-12T02:47:32.126 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-12T02:47:32.126 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-12T02:47:32.126 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-12T02:47:32.126 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-12T02:47:32.126 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-12T02:47:32.126 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-12T02:47:32.126 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-12T02:47:32.126 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-12T02:47:32.126 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-12T02:47:32.126 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-12T02:47:32.127 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-12T02:47:32.127 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-12T02:47:32.127 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-12T02:47:32.127 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-12T02:47:32.127 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-12T02:47:32.127 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-12T02:47:32.127 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-12T02:47:32.127 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-12T02:47:32.127 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-12T02:47:32.127 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-12T02:47:32.127 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-12T02:47:32.127 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-12T02:47:32.127 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-12T02:47:32.128 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-12T02:47:32.128 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-12T02:47:32.128 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-12T02:47:32.128 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-12T02:47:32.128 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-12T02:47:32.128 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-12T02:47:32.128 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-12T02:47:32.128 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-12T02:47:32.128 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-12T02:47:32.128 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-12T02:47:32.128 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-12T02:47:32.128 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-12T02:47:32.128 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-12T02:47:32.129 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-12T02:47:32.129 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-12T02:47:32.129 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-12T02:47:32.129 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-12T02:47:32.129 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-12T02:47:32.129 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 149/164 2024-09-12T02:47:32.129 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-12T02:47:32.129 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 151/164 2024-09-12T02:47:32.129 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-12T02:47:32.129 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-12T02:47:32.129 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-12T02:47:32.129 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-12T02:47:32.129 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-12T02:47:32.130 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-12T02:47:32.130 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-12T02:47:32.130 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-12T02:47:32.130 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-12T02:47:32.130 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-12T02:47:32.130 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-12T02:47:32.130 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-12T02:47:34.872 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T02:47:34.872 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-12T02:47:34.872 INFO:teuthology.orchestra.run.smithi005.stdout:Upgraded: 2024-09-12T02:47:34.873 INFO:teuthology.orchestra.run.smithi005.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-12T02:47:34.873 INFO:teuthology.orchestra.run.smithi005.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-12T02:47:34.873 INFO:teuthology.orchestra.run.smithi005.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-12T02:47:34.873 INFO:teuthology.orchestra.run.smithi005.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-12T02:47:34.873 INFO:teuthology.orchestra.run.smithi005.stdout: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:47:34.873 INFO:teuthology.orchestra.run.smithi005.stdout: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:47:34.873 INFO:teuthology.orchestra.run.smithi005.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-12T02:47:34.873 INFO:teuthology.orchestra.run.smithi005.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-12T02:47:34.873 INFO:teuthology.orchestra.run.smithi005.stdout:Installed: 2024-09-12T02:47:34.873 INFO:teuthology.orchestra.run.smithi005.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-12T02:47:34.873 INFO:teuthology.orchestra.run.smithi005.stdout: automake-1.16.2-8.el9.noarch 2024-09-12T02:47:34.874 INFO:teuthology.orchestra.run.smithi005.stdout: bison-3.7.4-5.el9.x86_64 2024-09-12T02:47:34.874 INFO:teuthology.orchestra.run.smithi005.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-12T02:47:34.874 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:47:34.874 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:47:34.874 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:47:34.874 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:47:34.874 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:47:34.874 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:47:34.874 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:47:34.874 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:47:34.874 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:47:34.874 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:47:34.874 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:47:34.874 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:47:34.875 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:47:34.875 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:47:34.875 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:47:34.875 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:47:34.875 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:47:34.875 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:47:34.875 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:47:34.875 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:47:34.875 INFO:teuthology.orchestra.run.smithi005.stdout: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:47:34.875 INFO:teuthology.orchestra.run.smithi005.stdout: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:47:34.875 INFO:teuthology.orchestra.run.smithi005.stdout: cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:47:34.875 INFO:teuthology.orchestra.run.smithi005.stdout: cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:47:34.876 INFO:teuthology.orchestra.run.smithi005.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-12T02:47:34.876 INFO:teuthology.orchestra.run.smithi005.stdout: flex-2.6.4-9.el9.x86_64 2024-09-12T02:47:34.876 INFO:teuthology.orchestra.run.smithi005.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-12T02:47:34.876 INFO:teuthology.orchestra.run.smithi005.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-12T02:47:34.876 INFO:teuthology.orchestra.run.smithi005.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-12T02:47:34.876 INFO:teuthology.orchestra.run.smithi005.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-12T02:47:34.876 INFO:teuthology.orchestra.run.smithi005.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-12T02:47:34.876 INFO:teuthology.orchestra.run.smithi005.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-12T02:47:34.876 INFO:teuthology.orchestra.run.smithi005.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-12T02:47:34.876 INFO:teuthology.orchestra.run.smithi005.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-12T02:47:34.876 INFO:teuthology.orchestra.run.smithi005.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-12T02:47:34.876 INFO:teuthology.orchestra.run.smithi005.stdout: libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:47:34.877 INFO:teuthology.orchestra.run.smithi005.stdout: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:47:34.877 INFO:teuthology.orchestra.run.smithi005.stdout: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:47:34.877 INFO:teuthology.orchestra.run.smithi005.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-12T02:47:34.877 INFO:teuthology.orchestra.run.smithi005.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-12T02:47:34.877 INFO:teuthology.orchestra.run.smithi005.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-12T02:47:34.877 INFO:teuthology.orchestra.run.smithi005.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-12T02:47:34.877 INFO:teuthology.orchestra.run.smithi005.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-12T02:47:34.877 INFO:teuthology.orchestra.run.smithi005.stdout: librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:47:34.877 INFO:teuthology.orchestra.run.smithi005.stdout: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:47:34.877 INFO:teuthology.orchestra.run.smithi005.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-12T02:47:34.877 INFO:teuthology.orchestra.run.smithi005.stdout: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:47:34.877 INFO:teuthology.orchestra.run.smithi005.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-12T02:47:34.878 INFO:teuthology.orchestra.run.smithi005.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-12T02:47:34.878 INFO:teuthology.orchestra.run.smithi005.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-12T02:47:34.878 INFO:teuthology.orchestra.run.smithi005.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-12T02:47:34.878 INFO:teuthology.orchestra.run.smithi005.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-12T02:47:34.878 INFO:teuthology.orchestra.run.smithi005.stdout: lua-5.4.4-4.el9.x86_64 2024-09-12T02:47:34.878 INFO:teuthology.orchestra.run.smithi005.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-12T02:47:34.878 INFO:teuthology.orchestra.run.smithi005.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-12T02:47:34.878 INFO:teuthology.orchestra.run.smithi005.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-12T02:47:34.878 INFO:teuthology.orchestra.run.smithi005.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-12T02:47:34.878 INFO:teuthology.orchestra.run.smithi005.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-12T02:47:34.878 INFO:teuthology.orchestra.run.smithi005.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-12T02:47:34.878 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql-13.16-1.el9.x86_64 2024-09-12T02:47:34.878 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-12T02:47:34.878 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-12T02:47:34.879 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-12T02:47:34.879 INFO:teuthology.orchestra.run.smithi005.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-12T02:47:34.879 INFO:teuthology.orchestra.run.smithi005.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-12T02:47:34.879 INFO:teuthology.orchestra.run.smithi005.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-12T02:47:34.879 INFO:teuthology.orchestra.run.smithi005.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-12T02:47:34.879 INFO:teuthology.orchestra.run.smithi005.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-12T02:47:34.879 INFO:teuthology.orchestra.run.smithi005.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-12T02:47:34.879 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-12T02:47:34.879 INFO:teuthology.orchestra.run.smithi005.stdout: python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:47:34.879 INFO:teuthology.orchestra.run.smithi005.stdout: python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:47:34.879 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:47:34.879 INFO:teuthology.orchestra.run.smithi005.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-12T02:47:34.879 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-12T02:47:34.880 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-12T02:47:34.880 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-12T02:47:34.880 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-12T02:47:34.880 INFO:teuthology.orchestra.run.smithi005.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-12T02:47:34.880 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-12T02:47:34.880 INFO:teuthology.orchestra.run.smithi005.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-12T02:47:34.880 INFO:teuthology.orchestra.run.smithi005.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-12T02:47:34.880 INFO:teuthology.orchestra.run.smithi005.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-12T02:47:34.880 INFO:teuthology.orchestra.run.smithi005.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-12T02:47:34.880 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-12T02:47:34.880 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-12T02:47:34.880 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-12T02:47:34.880 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-12T02:47:34.881 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-12T02:47:34.881 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-12T02:47:34.881 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-12T02:47:34.881 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-12T02:47:34.881 INFO:teuthology.orchestra.run.smithi005.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-12T02:47:34.881 INFO:teuthology.orchestra.run.smithi005.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-12T02:47:34.881 INFO:teuthology.orchestra.run.smithi005.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-12T02:47:34.881 INFO:teuthology.orchestra.run.smithi005.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-12T02:47:34.881 INFO:teuthology.orchestra.run.smithi005.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-12T02:47:34.881 INFO:teuthology.orchestra.run.smithi005.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-12T02:47:34.881 INFO:teuthology.orchestra.run.smithi005.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-12T02:47:34.881 INFO:teuthology.orchestra.run.smithi005.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-12T02:47:34.881 INFO:teuthology.orchestra.run.smithi005.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-12T02:47:34.882 INFO:teuthology.orchestra.run.smithi005.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-12T02:47:34.882 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-12T02:47:34.882 INFO:teuthology.orchestra.run.smithi005.stdout: python3-ply-3.11-14.el9.noarch 2024-09-12T02:47:34.882 INFO:teuthology.orchestra.run.smithi005.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-12T02:47:34.882 INFO:teuthology.orchestra.run.smithi005.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-12T02:47:34.882 INFO:teuthology.orchestra.run.smithi005.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-12T02:47:34.882 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-12T02:47:34.882 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-12T02:47:34.882 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-12T02:47:34.882 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-12T02:47:34.882 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-12T02:47:34.882 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-12T02:47:34.882 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:47:34.883 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:47:34.883 INFO:teuthology.orchestra.run.smithi005.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-12T02:47:34.883 INFO:teuthology.orchestra.run.smithi005.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-12T02:47:34.883 INFO:teuthology.orchestra.run.smithi005.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-12T02:47:34.883 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:47:34.883 INFO:teuthology.orchestra.run.smithi005.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-12T02:47:34.883 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-12T02:47:34.883 INFO:teuthology.orchestra.run.smithi005.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-12T02:47:34.883 INFO:teuthology.orchestra.run.smithi005.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-12T02:47:34.883 INFO:teuthology.orchestra.run.smithi005.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-12T02:47:34.883 INFO:teuthology.orchestra.run.smithi005.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-12T02:47:34.883 INFO:teuthology.orchestra.run.smithi005.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-12T02:47:34.884 INFO:teuthology.orchestra.run.smithi005.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-12T02:47:34.884 INFO:teuthology.orchestra.run.smithi005.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-12T02:47:34.884 INFO:teuthology.orchestra.run.smithi005.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-12T02:47:34.884 INFO:teuthology.orchestra.run.smithi005.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-12T02:47:34.884 INFO:teuthology.orchestra.run.smithi005.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-12T02:47:34.884 INFO:teuthology.orchestra.run.smithi005.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-12T02:47:34.884 INFO:teuthology.orchestra.run.smithi005.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-12T02:47:34.884 INFO:teuthology.orchestra.run.smithi005.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-12T02:47:34.884 INFO:teuthology.orchestra.run.smithi005.stdout: rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:47:34.884 INFO:teuthology.orchestra.run.smithi005.stdout: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:47:34.884 INFO:teuthology.orchestra.run.smithi005.stdout: rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:47:34.884 INFO:teuthology.orchestra.run.smithi005.stdout: re2-1:20211101-20.el9.x86_64 2024-09-12T02:47:34.885 INFO:teuthology.orchestra.run.smithi005.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-12T02:47:34.885 INFO:teuthology.orchestra.run.smithi005.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-12T02:47:34.885 INFO:teuthology.orchestra.run.smithi005.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-12T02:47:34.885 INFO:teuthology.orchestra.run.smithi005.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-12T02:47:34.885 INFO:teuthology.orchestra.run.smithi005.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-12T02:47:34.885 INFO:teuthology.orchestra.run.smithi005.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-12T02:47:34.886 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-12T02:47:34.886 INFO:teuthology.orchestra.run.smithi005.stdout:Complete! 2024-09-12T02:47:35.468 DEBUG:teuthology.parallel:result is None 2024-09-12T02:47:37.320 INFO:teuthology.orchestra.run.smithi146.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T02:47:37.321 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 1/164 2024-09-12T02:47:37.321 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2/164 2024-09-12T02:47:37.321 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 3/164 2024-09-12T02:47:37.321 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 4/164 2024-09-12T02:47:37.321 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-298.gb53b6d 5/164 2024-09-12T02:47:37.321 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 6/164 2024-09-12T02:47:37.321 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 7/164 2024-09-12T02:47:37.321 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 8/164 2024-09-12T02:47:37.321 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 9/164 2024-09-12T02:47:37.321 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 10/164 2024-09-12T02:47:37.322 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 11/164 2024-09-12T02:47:37.322 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 12/164 2024-09-12T02:47:37.322 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 13/164 2024-09-12T02:47:37.322 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 14/164 2024-09-12T02:47:37.322 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_6 15/164 2024-09-12T02:47:37.338 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 16/164 2024-09-12T02:47:37.338 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 17/164 2024-09-12T02:47:37.338 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 18/164 2024-09-12T02:47:37.338 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_ 19/164 2024-09-12T02:47:37.338 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 20/164 2024-09-12T02:47:37.338 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9 21/164 2024-09-12T02:47:37.339 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x 22/164 2024-09-12T02:47:37.339 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 23/164 2024-09-12T02:47:37.339 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 24/164 2024-09-12T02:47:37.339 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 25/164 2024-09-12T02:47:37.339 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 26/164 2024-09-12T02:47:37.339 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 27/164 2024-09-12T02:47:37.339 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 28/164 2024-09-12T02:47:37.339 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 29/164 2024-09-12T02:47:37.339 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.e 30/164 2024-09-12T02:47:37.339 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noar 31/164 2024-09-12T02:47:37.339 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.no 32/164 2024-09-12T02:47:37.339 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b 33/164 2024-09-12T02:47:37.340 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9 34/164 2024-09-12T02:47:37.340 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 35/164 2024-09-12T02:47:37.340 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el 36/164 2024-09-12T02:47:37.340 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 37/164 2024-09-12T02:47:37.340 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 38/164 2024-09-12T02:47:37.341 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 39/164 2024-09-12T02:47:37.341 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-12T02:47:37.341 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-12T02:47:37.341 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-12T02:47:37.341 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-12T02:47:37.341 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-12T02:47:37.342 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-12T02:47:37.342 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-12T02:47:37.342 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-12T02:47:37.342 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-12T02:47:37.342 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-12T02:47:37.342 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-12T02:47:37.342 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-12T02:47:37.342 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-12T02:47:37.342 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-12T02:47:37.342 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-12T02:47:37.342 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-12T02:47:37.342 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-12T02:47:37.342 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-12T02:47:37.343 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-12T02:47:37.343 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-12T02:47:37.343 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-12T02:47:37.343 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-12T02:47:37.343 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-12T02:47:37.343 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-12T02:47:37.343 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-12T02:47:37.344 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-12T02:47:37.344 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-12T02:47:37.344 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-12T02:47:37.344 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-12T02:47:37.344 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-12T02:47:37.344 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-12T02:47:37.344 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-12T02:47:37.344 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-12T02:47:37.344 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-12T02:47:37.344 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-12T02:47:37.344 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-12T02:47:37.344 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-12T02:47:37.345 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-12T02:47:37.345 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-12T02:47:37.345 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-12T02:47:37.345 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-12T02:47:37.345 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-12T02:47:37.345 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-12T02:47:37.345 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-12T02:47:37.345 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-12T02:47:37.345 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-12T02:47:37.345 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-12T02:47:37.345 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-12T02:47:37.345 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-12T02:47:37.345 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-12T02:47:37.346 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-12T02:47:37.346 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-12T02:47:37.346 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-12T02:47:37.346 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-12T02:47:37.346 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-12T02:47:37.346 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-12T02:47:37.346 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-12T02:47:37.346 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-12T02:47:37.346 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-12T02:47:37.346 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-12T02:47:37.346 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-12T02:47:37.346 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-12T02:47:37.346 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-12T02:47:37.347 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-12T02:47:37.347 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-12T02:47:37.347 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-12T02:47:37.347 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-12T02:47:37.347 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/164 2024-09-12T02:47:37.347 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-12T02:47:37.347 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-12T02:47:37.347 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-12T02:47:37.347 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-12T02:47:37.347 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-12T02:47:37.347 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-12T02:47:37.347 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-12T02:47:37.348 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-12T02:47:37.348 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-12T02:47:37.348 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-12T02:47:37.348 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-12T02:47:37.348 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-12T02:47:37.348 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-12T02:47:37.348 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-12T02:47:37.348 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-12T02:47:37.349 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-12T02:47:37.349 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-12T02:47:37.349 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-12T02:47:37.349 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-12T02:47:37.349 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-12T02:47:37.349 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-12T02:47:37.349 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-12T02:47:37.349 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-12T02:47:37.349 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-12T02:47:37.349 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-12T02:47:37.349 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-12T02:47:37.349 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-12T02:47:37.350 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-12T02:47:37.350 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-12T02:47:37.350 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-12T02:47:37.350 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-12T02:47:37.350 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-12T02:47:37.350 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-12T02:47:37.350 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-12T02:47:37.350 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-12T02:47:37.350 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-12T02:47:37.350 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-12T02:47:37.350 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-12T02:47:37.350 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-12T02:47:37.350 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-12T02:47:37.351 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-12T02:47:37.351 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 149/164 2024-09-12T02:47:37.351 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-12T02:47:37.351 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 151/164 2024-09-12T02:47:37.351 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-12T02:47:37.351 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-12T02:47:37.351 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-12T02:47:37.351 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-12T02:47:37.351 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-12T02:47:37.351 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-12T02:47:37.351 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-12T02:47:37.351 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-12T02:47:37.351 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-12T02:47:37.352 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-12T02:47:37.352 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-12T02:47:37.352 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-12T02:47:39.935 INFO:teuthology.orchestra.run.smithi146.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-12T02:47:39.935 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-12T02:47:39.935 INFO:teuthology.orchestra.run.smithi146.stdout:Upgraded: 2024-09-12T02:47:39.935 INFO:teuthology.orchestra.run.smithi146.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-12T02:47:39.935 INFO:teuthology.orchestra.run.smithi146.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-12T02:47:39.935 INFO:teuthology.orchestra.run.smithi146.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-12T02:47:39.935 INFO:teuthology.orchestra.run.smithi146.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-12T02:47:39.935 INFO:teuthology.orchestra.run.smithi146.stdout: librados2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:47:39.936 INFO:teuthology.orchestra.run.smithi146.stdout: librbd1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:47:39.936 INFO:teuthology.orchestra.run.smithi146.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-12T02:47:39.936 INFO:teuthology.orchestra.run.smithi146.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-12T02:47:39.936 INFO:teuthology.orchestra.run.smithi146.stdout:Installed: 2024-09-12T02:47:39.936 INFO:teuthology.orchestra.run.smithi146.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-12T02:47:39.936 INFO:teuthology.orchestra.run.smithi146.stdout: automake-1.16.2-8.el9.noarch 2024-09-12T02:47:39.936 INFO:teuthology.orchestra.run.smithi146.stdout: bison-3.7.4-5.el9.x86_64 2024-09-12T02:47:39.936 INFO:teuthology.orchestra.run.smithi146.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-12T02:47:39.936 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:47:39.936 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-base-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:47:39.936 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:47:39.937 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:47:39.937 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-grafana-dashboards-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:47:39.937 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-immutable-object-cache-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:47:39.937 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mds-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:47:39.937 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:47:39.937 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:47:39.937 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-dashboard-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:47:39.937 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-diskprediction-local-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:47:39.937 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-modules-core-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:47:39.937 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mgr-rook-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:47:39.937 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-mon-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:47:39.937 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-osd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:47:39.938 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-prometheus-alerts-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:47:39.938 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-radosgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:47:39.938 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-selinux-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:47:39.938 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-test-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:47:39.938 INFO:teuthology.orchestra.run.smithi146.stdout: ceph-volume-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:47:39.938 INFO:teuthology.orchestra.run.smithi146.stdout: cephadm-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:47:39.938 INFO:teuthology.orchestra.run.smithi146.stdout: cephfs-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:47:39.938 INFO:teuthology.orchestra.run.smithi146.stdout: cephfs-shell-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:47:39.938 INFO:teuthology.orchestra.run.smithi146.stdout: cephfs-top-2:19.1.1-298.gb53b6d3e.el9.noarch 2024-09-12T02:47:39.938 INFO:teuthology.orchestra.run.smithi146.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-12T02:47:39.938 INFO:teuthology.orchestra.run.smithi146.stdout: flex-2.6.4-9.el9.x86_64 2024-09-12T02:47:39.938 INFO:teuthology.orchestra.run.smithi146.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-12T02:47:39.939 INFO:teuthology.orchestra.run.smithi146.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-12T02:47:39.939 INFO:teuthology.orchestra.run.smithi146.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-12T02:47:39.939 INFO:teuthology.orchestra.run.smithi146.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-12T02:47:39.939 INFO:teuthology.orchestra.run.smithi146.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-12T02:47:39.939 INFO:teuthology.orchestra.run.smithi146.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-12T02:47:39.939 INFO:teuthology.orchestra.run.smithi146.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-12T02:47:39.939 INFO:teuthology.orchestra.run.smithi146.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-12T02:47:39.939 INFO:teuthology.orchestra.run.smithi146.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-12T02:47:39.939 INFO:teuthology.orchestra.run.smithi146.stdout: libcephfs-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:47:39.939 INFO:teuthology.orchestra.run.smithi146.stdout: libcephfs2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:47:39.939 INFO:teuthology.orchestra.run.smithi146.stdout: libcephsqlite-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:47:39.940 INFO:teuthology.orchestra.run.smithi146.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-12T02:47:39.940 INFO:teuthology.orchestra.run.smithi146.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-12T02:47:39.940 INFO:teuthology.orchestra.run.smithi146.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-12T02:47:39.940 INFO:teuthology.orchestra.run.smithi146.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-12T02:47:39.940 INFO:teuthology.orchestra.run.smithi146.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-12T02:47:39.940 INFO:teuthology.orchestra.run.smithi146.stdout: librados-devel-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:47:39.940 INFO:teuthology.orchestra.run.smithi146.stdout: libradosstriper1-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:47:39.940 INFO:teuthology.orchestra.run.smithi146.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-12T02:47:39.940 INFO:teuthology.orchestra.run.smithi146.stdout: librgw2-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:47:39.940 INFO:teuthology.orchestra.run.smithi146.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-12T02:47:39.940 INFO:teuthology.orchestra.run.smithi146.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-12T02:47:39.941 INFO:teuthology.orchestra.run.smithi146.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-12T02:47:39.941 INFO:teuthology.orchestra.run.smithi146.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-12T02:47:39.941 INFO:teuthology.orchestra.run.smithi146.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-12T02:47:39.941 INFO:teuthology.orchestra.run.smithi146.stdout: lua-5.4.4-4.el9.x86_64 2024-09-12T02:47:39.941 INFO:teuthology.orchestra.run.smithi146.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-12T02:47:39.941 INFO:teuthology.orchestra.run.smithi146.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-12T02:47:39.941 INFO:teuthology.orchestra.run.smithi146.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-12T02:47:39.941 INFO:teuthology.orchestra.run.smithi146.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-12T02:47:39.941 INFO:teuthology.orchestra.run.smithi146.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-12T02:47:39.941 INFO:teuthology.orchestra.run.smithi146.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-12T02:47:39.941 INFO:teuthology.orchestra.run.smithi146.stdout: postgresql-13.16-1.el9.x86_64 2024-09-12T02:47:39.941 INFO:teuthology.orchestra.run.smithi146.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-12T02:47:39.942 INFO:teuthology.orchestra.run.smithi146.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-12T02:47:39.942 INFO:teuthology.orchestra.run.smithi146.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-12T02:47:39.942 INFO:teuthology.orchestra.run.smithi146.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-12T02:47:39.942 INFO:teuthology.orchestra.run.smithi146.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-12T02:47:39.942 INFO:teuthology.orchestra.run.smithi146.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-12T02:47:39.942 INFO:teuthology.orchestra.run.smithi146.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-12T02:47:39.942 INFO:teuthology.orchestra.run.smithi146.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-12T02:47:39.942 INFO:teuthology.orchestra.run.smithi146.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-12T02:47:39.942 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-12T02:47:39.942 INFO:teuthology.orchestra.run.smithi146.stdout: python3-ceph-argparse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:47:39.942 INFO:teuthology.orchestra.run.smithi146.stdout: python3-ceph-common-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:47:39.942 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cephfs-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:47:39.943 INFO:teuthology.orchestra.run.smithi146.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-12T02:47:39.943 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-12T02:47:39.943 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-12T02:47:39.943 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-12T02:47:39.943 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-12T02:47:39.943 INFO:teuthology.orchestra.run.smithi146.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-12T02:47:39.943 INFO:teuthology.orchestra.run.smithi146.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-12T02:47:39.943 INFO:teuthology.orchestra.run.smithi146.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-12T02:47:39.943 INFO:teuthology.orchestra.run.smithi146.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-12T02:47:39.943 INFO:teuthology.orchestra.run.smithi146.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-12T02:47:39.943 INFO:teuthology.orchestra.run.smithi146.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-12T02:47:39.943 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-12T02:47:39.943 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-12T02:47:39.944 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-12T02:47:39.944 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-12T02:47:39.944 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-12T02:47:39.944 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-12T02:47:39.944 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-12T02:47:39.944 INFO:teuthology.orchestra.run.smithi146.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-12T02:47:39.944 INFO:teuthology.orchestra.run.smithi146.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-12T02:47:39.944 INFO:teuthology.orchestra.run.smithi146.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-12T02:47:39.945 INFO:teuthology.orchestra.run.smithi146.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-12T02:47:39.945 INFO:teuthology.orchestra.run.smithi146.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-12T02:47:39.945 INFO:teuthology.orchestra.run.smithi146.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-12T02:47:39.945 INFO:teuthology.orchestra.run.smithi146.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-12T02:47:39.945 INFO:teuthology.orchestra.run.smithi146.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-12T02:47:39.946 INFO:teuthology.orchestra.run.smithi146.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-12T02:47:39.946 INFO:teuthology.orchestra.run.smithi146.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-12T02:47:39.946 INFO:teuthology.orchestra.run.smithi146.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-12T02:47:39.946 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-12T02:47:39.946 INFO:teuthology.orchestra.run.smithi146.stdout: python3-ply-3.11-14.el9.noarch 2024-09-12T02:47:39.947 INFO:teuthology.orchestra.run.smithi146.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-12T02:47:39.947 INFO:teuthology.orchestra.run.smithi146.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-12T02:47:39.947 INFO:teuthology.orchestra.run.smithi146.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-12T02:47:39.947 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-12T02:47:39.947 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-12T02:47:39.947 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-12T02:47:39.948 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-12T02:47:39.948 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-12T02:47:39.948 INFO:teuthology.orchestra.run.smithi146.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-12T02:47:39.948 INFO:teuthology.orchestra.run.smithi146.stdout: python3-rados-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:47:39.948 INFO:teuthology.orchestra.run.smithi146.stdout: python3-rbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:47:39.948 INFO:teuthology.orchestra.run.smithi146.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-12T02:47:39.949 INFO:teuthology.orchestra.run.smithi146.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-12T02:47:39.949 INFO:teuthology.orchestra.run.smithi146.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-12T02:47:39.949 INFO:teuthology.orchestra.run.smithi146.stdout: python3-rgw-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:47:39.949 INFO:teuthology.orchestra.run.smithi146.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-12T02:47:39.949 INFO:teuthology.orchestra.run.smithi146.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-12T02:47:39.950 INFO:teuthology.orchestra.run.smithi146.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-12T02:47:39.950 INFO:teuthology.orchestra.run.smithi146.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-12T02:47:39.950 INFO:teuthology.orchestra.run.smithi146.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-12T02:47:39.950 INFO:teuthology.orchestra.run.smithi146.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-12T02:47:39.950 INFO:teuthology.orchestra.run.smithi146.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-12T02:47:39.950 INFO:teuthology.orchestra.run.smithi146.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-12T02:47:39.951 INFO:teuthology.orchestra.run.smithi146.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-12T02:47:39.951 INFO:teuthology.orchestra.run.smithi146.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-12T02:47:39.951 INFO:teuthology.orchestra.run.smithi146.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-12T02:47:39.951 INFO:teuthology.orchestra.run.smithi146.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-12T02:47:39.951 INFO:teuthology.orchestra.run.smithi146.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-12T02:47:39.952 INFO:teuthology.orchestra.run.smithi146.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-12T02:47:39.952 INFO:teuthology.orchestra.run.smithi146.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-12T02:47:39.952 INFO:teuthology.orchestra.run.smithi146.stdout: rbd-fuse-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:47:39.952 INFO:teuthology.orchestra.run.smithi146.stdout: rbd-mirror-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:47:39.952 INFO:teuthology.orchestra.run.smithi146.stdout: rbd-nbd-2:19.1.1-298.gb53b6d3e.el9.x86_64 2024-09-12T02:47:39.953 INFO:teuthology.orchestra.run.smithi146.stdout: re2-1:20211101-20.el9.x86_64 2024-09-12T02:47:39.953 INFO:teuthology.orchestra.run.smithi146.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-12T02:47:39.953 INFO:teuthology.orchestra.run.smithi146.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-12T02:47:39.953 INFO:teuthology.orchestra.run.smithi146.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-12T02:47:39.953 INFO:teuthology.orchestra.run.smithi146.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-12T02:47:39.954 INFO:teuthology.orchestra.run.smithi146.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-12T02:47:39.954 INFO:teuthology.orchestra.run.smithi146.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-12T02:47:39.954 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-12T02:47:39.955 INFO:teuthology.orchestra.run.smithi146.stdout:Complete! 2024-09-12T02:47:40.352 DEBUG:teuthology.parallel:result is None 2024-09-12T02:47:40.353 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T02:47:40.503 DEBUG:teuthology.orchestra.run.smithi005:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-12T02:47:40.531 INFO:teuthology.orchestra.run.smithi005.stdout:19.1.1-298.gb53b6d3e.el9 2024-09-12T02:47:40.531 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-298.gb53b6d3e.el9 2024-09-12T02:47:40.532 INFO:teuthology.task.install:The correct ceph version 19.1.1-298.gb53b6d3e is installed. 2024-09-12T02:47:40.533 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T02:47:40.673 DEBUG:teuthology.orchestra.run.smithi146:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-12T02:47:40.695 INFO:teuthology.orchestra.run.smithi146.stdout:19.1.1-298.gb53b6d3e.el9 2024-09-12T02:47:40.696 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-298.gb53b6d3e.el9 2024-09-12T02:47:40.696 INFO:teuthology.task.install:The correct ceph version 19.1.1-298.gb53b6d3e is installed. 2024-09-12T02:47:40.698 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T02:47:40.839 DEBUG:teuthology.orchestra.run.smithi148:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-12T02:47:40.867 INFO:teuthology.orchestra.run.smithi148.stdout:19.1.1-298.gb53b6d3e.el9 2024-09-12T02:47:40.867 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-298.gb53b6d3e.el9 2024-09-12T02:47:40.867 INFO:teuthology.task.install:The correct ceph version 19.1.1-298.gb53b6d3e is installed. 2024-09-12T02:47:40.869 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-12T02:47:40.870 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-09-12T02:47:40.870 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-12T02:47:40.904 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-12T02:47:40.904 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-12T02:47:40.932 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-12T02:47:40.933 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-12T02:47:40.965 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-12T02:47:40.966 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-09-12T02:47:40.966 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/usr/bin/daemon-helper 2024-09-12T02:47:40.995 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-12T02:47:41.063 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-12T02:47:41.063 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd of=/usr/bin/daemon-helper 2024-09-12T02:47:41.090 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-12T02:47:41.155 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-12T02:47:41.156 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/usr/bin/daemon-helper 2024-09-12T02:47:41.186 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-12T02:47:41.257 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-12T02:47:41.257 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-09-12T02:47:41.258 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-12T02:47:41.286 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-12T02:47:41.353 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-12T02:47:41.353 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-12T02:47:41.379 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-12T02:47:41.444 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-12T02:47:41.444 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-12T02:47:41.474 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-12T02:47:41.542 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-12T02:47:41.542 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-09-12T02:47:41.542 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/usr/bin/stdin-killer 2024-09-12T02:47:41.570 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-12T02:47:41.637 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-12T02:47:41.637 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd of=/usr/bin/stdin-killer 2024-09-12T02:47:41.662 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-12T02:47:41.727 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-12T02:47:41.727 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/usr/bin/stdin-killer 2024-09-12T02:47:41.755 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-12T02:47:41.822 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-12T02:47:41.926 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-09-12T02:47:41.926 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T02:47:41.927 INFO:tasks.cephadm:Cluster fsid is 618305f8-70b1-11ef-bcea-c7b262605968 2024-09-12T02:47:41.927 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-12T02:47:41.927 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.5', 'mon.b': '172.21.15.146', 'mon.c': '172.21.15.148'} 2024-09-12T02:47:41.927 INFO:tasks.cephadm:First mon is mon.a on smithi005 2024-09-12T02:47:41.928 INFO:tasks.cephadm:First mgr is x 2024-09-12T02:47:41.928 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-12T02:47:41.928 DEBUG:teuthology.orchestra.run.smithi005:> sudo hostname $(hostname -s) 2024-09-12T02:47:41.957 DEBUG:teuthology.orchestra.run.smithi146:> sudo hostname $(hostname -s) 2024-09-12T02:47:41.983 DEBUG:teuthology.orchestra.run.smithi148:> sudo hostname $(hostname -s) 2024-09-12T02:47:42.011 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-09-12T02:47:42.011 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T02:47:42.155 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/56038/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.1.1-298-gb53b6d3e', 'node_name': '172.21.5.36+adami06', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.1.1-298.gb53b6d3e'}, 'url': 'https://2.chacra.ceph.com/r/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-09 17:24:58.599585', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://2.chacra.ceph.com/repos/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-09-12T02:47:42.307 INFO:tasks.util.chacra:got chacra host 2.chacra.ceph.com, ref squid, sha1 b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T02:47:42.309 INFO:tasks.cephadm:Discovered cachra url: https://2.chacra.ceph.com/binaries/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/x86_64/flavors/default/cephadm 2024-09-12T02:47:42.309 INFO:tasks.cephadm:Downloading cephadm from url: https://2.chacra.ceph.com/binaries/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/x86_64/flavors/default/cephadm 2024-09-12T02:47:42.309 DEBUG:teuthology.orchestra.run.smithi005:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-12T02:47:42.692 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-r--r--. 1 ubuntu ubuntu 784498 Sep 12 02:47 /home/ubuntu/cephtest/cephadm 2024-09-12T02:47:42.693 DEBUG:teuthology.orchestra.run.smithi146:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-12T02:47:43.135 INFO:teuthology.orchestra.run.smithi146.stdout:-rw-r--r--. 1 ubuntu ubuntu 784498 Sep 12 02:47 /home/ubuntu/cephtest/cephadm 2024-09-12T02:47:43.135 DEBUG:teuthology.orchestra.run.smithi148:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/squid/b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-12T02:47:43.628 INFO:teuthology.orchestra.run.smithi148.stdout:-rw-r--r--. 1 ubuntu ubuntu 784498 Sep 12 02:47 /home/ubuntu/cephtest/cephadm 2024-09-12T02:47:43.628 DEBUG:teuthology.orchestra.run.smithi005:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-12T02:47:43.650 DEBUG:teuthology.orchestra.run.smithi146:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-12T02:47:43.671 DEBUG:teuthology.orchestra.run.smithi148:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-12T02:47:43.706 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc on all hosts... 2024-09-12T02:47:43.706 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc pull 2024-09-12T02:47:43.708 DEBUG:teuthology.orchestra.run.smithi146:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc pull 2024-09-12T02:47:43.713 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc pull 2024-09-12T02:47:45.240 INFO:teuthology.orchestra.run.smithi005.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc... 2024-09-12T02:47:45.247 INFO:teuthology.orchestra.run.smithi146.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc... 2024-09-12T02:47:45.307 INFO:teuthology.orchestra.run.smithi148.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc... 2024-09-12T02:48:18.082 INFO:teuthology.orchestra.run.smithi146.stdout:{ 2024-09-12T02:48:18.082 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", 2024-09-12T02:48:18.082 INFO:teuthology.orchestra.run.smithi146.stdout: "image_id": "54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594", 2024-09-12T02:48:18.082 INFO:teuthology.orchestra.run.smithi146.stdout: "repo_digests": [ 2024-09-12T02:48:18.082 INFO:teuthology.orchestra.run.smithi146.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66" 2024-09-12T02:48:18.082 INFO:teuthology.orchestra.run.smithi146.stdout: ] 2024-09-12T02:48:18.082 INFO:teuthology.orchestra.run.smithi146.stdout:} 2024-09-12T02:48:27.499 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-12T02:48:27.499 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", 2024-09-12T02:48:27.499 INFO:teuthology.orchestra.run.smithi148.stdout: "image_id": "54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594", 2024-09-12T02:48:27.499 INFO:teuthology.orchestra.run.smithi148.stdout: "repo_digests": [ 2024-09-12T02:48:27.499 INFO:teuthology.orchestra.run.smithi148.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66" 2024-09-12T02:48:27.500 INFO:teuthology.orchestra.run.smithi148.stdout: ] 2024-09-12T02:48:27.500 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-12T02:48:33.200 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-12T02:48:33.200 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_version": "ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable)", 2024-09-12T02:48:33.200 INFO:teuthology.orchestra.run.smithi005.stdout: "image_id": "54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594", 2024-09-12T02:48:33.200 INFO:teuthology.orchestra.run.smithi005.stdout: "repo_digests": [ 2024-09-12T02:48:33.200 INFO:teuthology.orchestra.run.smithi005.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66" 2024-09-12T02:48:33.200 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-12T02:48:33.201 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-12T02:48:33.226 DEBUG:teuthology.orchestra.run.smithi005:> sudo mkdir -p /etc/ceph 2024-09-12T02:48:33.267 DEBUG:teuthology.orchestra.run.smithi146:> sudo mkdir -p /etc/ceph 2024-09-12T02:48:33.302 DEBUG:teuthology.orchestra.run.smithi148:> sudo mkdir -p /etc/ceph 2024-09-12T02:48:33.336 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 777 /etc/ceph 2024-09-12T02:48:33.361 DEBUG:teuthology.orchestra.run.smithi146:> sudo chmod 777 /etc/ceph 2024-09-12T02:48:33.389 DEBUG:teuthology.orchestra.run.smithi148:> sudo chmod 777 /etc/ceph 2024-09-12T02:48:33.417 INFO:tasks.cephadm:Writing seed config... 2024-09-12T02:48:33.419 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-09-12T02:48:33.419 INFO:tasks.cephadm: override: [client] debug client = 20 2024-09-12T02:48:33.419 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-09-12T02:48:33.419 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-09-12T02:48:33.419 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-09-12T02:48:33.419 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-09-12T02:48:33.419 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-09-12T02:48:33.419 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-09-12T02:48:33.420 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-09-12T02:48:33.420 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-09-12T02:48:33.420 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-09-12T02:48:33.420 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-09-12T02:48:33.420 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-09-12T02:48:33.420 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-09-12T02:48:33.420 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-09-12T02:48:33.420 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-09-12T02:48:33.420 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-09-12T02:48:33.420 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-09-12T02:48:33.420 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-09-12T02:48:33.420 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-09-12T02:48:33.420 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-09-12T02:48:33.421 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-09-12T02:48:33.421 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-09-12T02:48:33.421 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-12T02:48:33.421 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-12T02:48:33.421 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-12T02:48:33.421 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-12T02:48:33.421 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-12T02:48:33.421 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-09-12T02:48:33.421 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-09-12T02:48:33.421 INFO:tasks.cephadm: override: [mon] mon warn on pool no app = False 2024-09-12T02:48:33.421 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-09-12T02:48:33.421 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-09-12T02:48:33.421 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-09-12T02:48:33.422 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-09-12T02:48:33.422 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-09-12T02:48:33.422 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-09-12T02:48:33.422 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-09-12T02:48:33.422 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-12T02:48:33.422 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-12T02:48:33.422 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-09-12T02:48:33.422 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-09-12T02:48:33.422 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-09-12T02:48:33.422 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-09-12T02:48:33.422 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-09-12T02:48:33.422 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-09-12T02:48:33.422 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-09-12T02:48:33.423 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-12T02:48:33.423 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-09-12T02:48:33.423 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-09-12T02:48:33.423 DEBUG:teuthology.orchestra.run.smithi005:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-12T02:48:33.439 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 618305f8-70b1-11ef-bcea-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 mon warn on pool no app = False [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-09-12T02:48:33.439 DEBUG:teuthology.orchestra.run.smithi005:mon.a> sudo journalctl -f -n 0 -u ceph-618305f8-70b1-11ef-bcea-c7b262605968@mon.a.service 2024-09-12T02:48:33.482 DEBUG:teuthology.orchestra.run.smithi005:mgr.x> sudo journalctl -f -n 0 -u ceph-618305f8-70b1-11ef-bcea-c7b262605968@mgr.x.service 2024-09-12T02:48:33.524 INFO:tasks.cephadm:Bootstrapping... 2024-09-12T02:48:33.524 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc -v bootstrap --fsid 618305f8-70b1-11ef-bcea-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.5 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-12T02:48:33.733 INFO:teuthology.orchestra.run.smithi005.stdout:-------------------------------------------------------------------------------- 2024-09-12T02:48:33.733 INFO:teuthology.orchestra.run.smithi005.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc', '-v', 'bootstrap', '--fsid', '618305f8-70b1-11ef-bcea-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.5', '--skip-admin-label'] 2024-09-12T02:48:33.733 INFO:teuthology.orchestra.run.smithi005.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-12T02:48:33.734 INFO:teuthology.orchestra.run.smithi005.stdout:Verifying podman|docker is present... 2024-09-12T02:48:33.769 INFO:teuthology.orchestra.run.smithi005.stdout:/bin/podman: stdout 5.2.2 2024-09-12T02:48:33.769 INFO:teuthology.orchestra.run.smithi005.stdout:Verifying lvm2 is present... 2024-09-12T02:48:33.769 INFO:teuthology.orchestra.run.smithi005.stdout:Verifying time synchronization is in place... 2024-09-12T02:48:33.777 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-12T02:48:33.777 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-12T02:48:33.783 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-12T02:48:33.783 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout inactive 2024-09-12T02:48:33.790 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout enabled 2024-09-12T02:48:33.797 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout active 2024-09-12T02:48:33.797 INFO:teuthology.orchestra.run.smithi005.stdout:Unit chronyd.service is enabled and running 2024-09-12T02:48:33.797 INFO:teuthology.orchestra.run.smithi005.stdout:Repeating the final host check... 2024-09-12T02:48:33.820 INFO:teuthology.orchestra.run.smithi005.stdout:/bin/podman: stdout 5.2.2 2024-09-12T02:48:33.820 INFO:teuthology.orchestra.run.smithi005.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-12T02:48:33.820 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl is present 2024-09-12T02:48:33.820 INFO:teuthology.orchestra.run.smithi005.stdout:lvcreate is present 2024-09-12T02:48:33.827 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-12T02:48:33.827 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-12T02:48:33.833 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-12T02:48:33.834 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout inactive 2024-09-12T02:48:33.840 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout enabled 2024-09-12T02:48:33.847 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout active 2024-09-12T02:48:33.847 INFO:teuthology.orchestra.run.smithi005.stdout:Unit chronyd.service is enabled and running 2024-09-12T02:48:33.847 INFO:teuthology.orchestra.run.smithi005.stdout:Host looks OK 2024-09-12T02:48:33.847 INFO:teuthology.orchestra.run.smithi005.stdout:Cluster fsid: 618305f8-70b1-11ef-bcea-c7b262605968 2024-09-12T02:48:33.847 INFO:teuthology.orchestra.run.smithi005.stdout:Acquiring lock 139922392249344 on /run/cephadm/618305f8-70b1-11ef-bcea-c7b262605968.lock 2024-09-12T02:48:33.847 INFO:teuthology.orchestra.run.smithi005.stdout:Lock 139922392249344 acquired on /run/cephadm/618305f8-70b1-11ef-bcea-c7b262605968.lock 2024-09-12T02:48:33.848 INFO:teuthology.orchestra.run.smithi005.stdout:Verifying IP 172.21.15.5 port 3300 ... 2024-09-12T02:48:33.848 INFO:teuthology.orchestra.run.smithi005.stdout:Verifying IP 172.21.15.5 port 6789 ... 2024-09-12T02:48:33.848 INFO:teuthology.orchestra.run.smithi005.stdout:Base mon IP(s) is [172.21.15.5:3300, 172.21.15.5:6789], mon addrv is [v2:172.21.15.5:3300,v1:172.21.15.5:6789] 2024-09-12T02:48:33.852 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.5 metric 100 2024-09-12T02:48:33.852 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.5 metric 100 2024-09-12T02:48:33.854 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-12T02:48:33.854 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 256 pref medium 2024-09-12T02:48:33.854 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 1024 expires 1789sec hoplimit 64 pref medium 2024-09-12T02:48:33.857 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-12T02:48:33.857 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-12T02:48:33.857 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-12T02:48:33.857 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-09-12T02:48:33.857 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:1442/64 scope link 2024-09-12T02:48:33.857 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-12T02:48:33.858 INFO:teuthology.orchestra.run.smithi005.stdout:Mon IP `172.21.15.5` is in CIDR network `172.21.0.0/20` 2024-09-12T02:48:33.858 INFO:teuthology.orchestra.run.smithi005.stdout:Mon IP `172.21.15.5` is in CIDR network `172.21.0.0/20` 2024-09-12T02:48:33.858 INFO:teuthology.orchestra.run.smithi005.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-12T02:48:33.858 INFO:teuthology.orchestra.run.smithi005.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-12T02:48:33.859 INFO:teuthology.orchestra.run.smithi005.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc... 2024-09-12T02:48:34.420 INFO:teuthology.orchestra.run.smithi005.stdout:/bin/podman: stdout 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 2024-09-12T02:48:34.420 INFO:teuthology.orchestra.run.smithi005.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc... 2024-09-12T02:48:34.420 INFO:teuthology.orchestra.run.smithi005.stdout:/bin/podman: stderr Getting image source signatures 2024-09-12T02:48:34.420 INFO:teuthology.orchestra.run.smithi005.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-12T02:48:34.420 INFO:teuthology.orchestra.run.smithi005.stdout:/bin/podman: stderr Copying blob sha256:60180723200353b4e13b39fd162aecaf1bc097c859c7c6537324229e97c78567 2024-09-12T02:48:34.420 INFO:teuthology.orchestra.run.smithi005.stdout:/bin/podman: stderr Copying config sha256:54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 2024-09-12T02:48:34.420 INFO:teuthology.orchestra.run.smithi005.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-12T02:48:35.307 INFO:teuthology.orchestra.run.smithi005.stdout:ceph: stdout ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable) 2024-09-12T02:48:35.307 INFO:teuthology.orchestra.run.smithi005.stdout:Ceph version: ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable) 2024-09-12T02:48:35.308 INFO:teuthology.orchestra.run.smithi005.stdout:Extracting ceph user uid/gid from container image... 2024-09-12T02:48:36.139 INFO:teuthology.orchestra.run.smithi005.stdout:stat: stdout 167 167 2024-09-12T02:48:36.139 INFO:teuthology.orchestra.run.smithi005.stdout:Creating initial keys... 2024-09-12T02:48:37.025 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph-authtool: stdout AQCEVuJmTmg4HRAA8E7aE596g8Iv0IW6Ooaeaw== 2024-09-12T02:48:38.075 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph-authtool: stdout AQCFVuJmaEacFRAAMWzvUTayM1gURcO/xl9szQ== 2024-09-12T02:48:38.989 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph-authtool: stdout AQCGVuJmijDmFRAAH2vN10auPmiU0Lip3MsnTw== 2024-09-12T02:48:38.990 INFO:teuthology.orchestra.run.smithi005.stdout:Creating initial monmap... 2024-09-12T02:48:39.937 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-12T02:48:39.938 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-09-12T02:48:39.938 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 618305f8-70b1-11ef-bcea-c7b262605968 2024-09-12T02:48:39.938 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-12T02:48:39.938 INFO:teuthology.orchestra.run.smithi005.stdout:monmaptool for a [v2:172.21.15.5:3300,v1:172.21.15.5:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-12T02:48:39.938 INFO:teuthology.orchestra.run.smithi005.stdout:setting min_mon_release = quincy 2024-09-12T02:48:39.938 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/monmaptool: set fsid to 618305f8-70b1-11ef-bcea-c7b262605968 2024-09-12T02:48:39.938 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-12T02:48:39.938 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-12T02:48:39.938 INFO:teuthology.orchestra.run.smithi005.stdout:Creating mon... 2024-09-12T02:48:40.851 INFO:teuthology.orchestra.run.smithi005.stdout:create mon.a on 2024-09-12T02:48:41.067 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-12T02:48:41.212 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-12T02:48:41.401 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-618305f8-70b1-11ef-bcea-c7b262605968.target → /etc/systemd/system/ceph-618305f8-70b1-11ef-bcea-c7b262605968.target. 2024-09-12T02:48:41.401 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-618305f8-70b1-11ef-bcea-c7b262605968.target → /etc/systemd/system/ceph-618305f8-70b1-11ef-bcea-c7b262605968.target. 2024-09-12T02:48:41.637 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-618305f8-70b1-11ef-bcea-c7b262605968@mon.a 2024-09-12T02:48:41.637 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stderr Failed to reset failed state of unit ceph-618305f8-70b1-11ef-bcea-c7b262605968@mon.a.service: Unit ceph-618305f8-70b1-11ef-bcea-c7b262605968@mon.a.service not loaded. 2024-09-12T02:48:41.877 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-618305f8-70b1-11ef-bcea-c7b262605968.target.wants/ceph-618305f8-70b1-11ef-bcea-c7b262605968@mon.a.service → /etc/systemd/system/ceph-618305f8-70b1-11ef-bcea-c7b262605968@.service. 2024-09-12T02:48:42.401 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-12T02:48:42.401 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout disabled 2024-09-12T02:48:42.408 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-12T02:48:42.408 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout inactive 2024-09-12T02:48:42.408 INFO:teuthology.orchestra.run.smithi005.stdout:firewalld.service is not enabled 2024-09-12T02:48:42.408 INFO:teuthology.orchestra.run.smithi005.stdout:Not possible to enable service . firewalld.service is not available 2024-09-12T02:48:42.409 INFO:teuthology.orchestra.run.smithi005.stdout:Waiting for mon to start... 2024-09-12T02:48:42.409 INFO:teuthology.orchestra.run.smithi005.stdout:Waiting for mon... 2024-09-12T02:48:43.385 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout cluster: 2024-09-12T02:48:43.385 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout id: 618305f8-70b1-11ef-bcea-c7b262605968 2024-09-12T02:48:43.385 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-12T02:48:43.385 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2024-09-12T02:48:43.385 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout services: 2024-09-12T02:48:43.386 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.543341s) 2024-09-12T02:48:43.386 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-12T02:48:43.386 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-12T02:48:43.386 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2024-09-12T02:48:43.386 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout data: 2024-09-12T02:48:43.386 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-12T02:48:43.386 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-12T02:48:43.386 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-12T02:48:43.387 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout pgs: 2024-09-12T02:48:43.387 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2024-09-12T02:48:43.387 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:42.822+0000 7f4d14f72640 1 Processor -- start 2024-09-12T02:48:43.387 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:42.823+0000 7f4d14f72640 1 -- start start 2024-09-12T02:48:43.387 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:42.823+0000 7f4d14f72640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d101081c0 0x7f4d1010a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:43.387 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:42.823+0000 7f4d14f72640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4d1007bdf0 con 0x7f4d101081c0 2024-09-12T02:48:43.387 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:42.823+0000 7f4d0e59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d101081c0 0x7f4d1010a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:43.387 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:42.823+0000 7f4d0e59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d101081c0 0x7f4d1010a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43636/0 (socket says 172.21.15.5:43636) 2024-09-12T02:48:43.387 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:42.823+0000 7f4d0e59c640 1 -- 172.21.15.5:0/1153525230 learned_addr learned my addr 172.21.15.5:0/1153525230 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:48:43.387 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:42.824+0000 7f4d0e59c640 1 -- 172.21.15.5:0/1153525230 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d1010aaf0 con 0x7f4d101081c0 2024-09-12T02:48:43.387 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:42.825+0000 7f4d0e59c640 1 --2- 172.21.15.5:0/1153525230 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d101081c0 0x7f4d1010a5b0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f4d00009820 tx=0x7f4d0002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a6f5aadbf97ad7a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:48:43.387 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:42.825+0000 7f4d0d59a640 1 -- 172.21.15.5:0/1153525230 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4d00037070 con 0x7f4d101081c0 2024-09-12T02:48:43.387 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:42.825+0000 7f4d0d59a640 1 -- 172.21.15.5:0/1153525230 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d00032440 con 0x7f4d101081c0 2024-09-12T02:48:43.388 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:42.825+0000 7f4d0d59a640 1 -- 172.21.15.5:0/1153525230 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4d00032740 con 0x7f4d101081c0 2024-09-12T02:48:43.388 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:42.826+0000 7f4d14f72640 1 -- 172.21.15.5:0/1153525230 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d101081c0 msgr2=0x7f4d1010a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:43.388 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:42.826+0000 7f4d14f72640 1 --2- 172.21.15.5:0/1153525230 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d101081c0 0x7f4d1010a5b0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f4d00009820 tx=0x7f4d0002a270 comp rx=0 tx=0).stop 2024-09-12T02:48:43.388 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:42.826+0000 7f4d14f72640 1 -- 172.21.15.5:0/1153525230 shutdown_connections 2024-09-12T02:48:43.388 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:42.826+0000 7f4d14f72640 1 --2- 172.21.15.5:0/1153525230 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d101081c0 0x7f4d1010a5b0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:43.388 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:42.826+0000 7f4d14f72640 1 -- 172.21.15.5:0/1153525230 >> 172.21.15.5:0/1153525230 conn(0x7f4d10101e20 msgr2=0x7f4d10104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:48:43.388 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:42.826+0000 7f4d14f72640 1 -- 172.21.15.5:0/1153525230 shutdown_connections 2024-09-12T02:48:43.388 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:42.826+0000 7f4d14f72640 1 -- 172.21.15.5:0/1153525230 wait complete. 2024-09-12T02:48:43.388 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:42.827+0000 7f4d14f72640 1 Processor -- start 2024-09-12T02:48:43.388 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:42.827+0000 7f4d14f72640 1 -- start start 2024-09-12T02:48:43.388 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:42.827+0000 7f4d14f72640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d101081c0 0x7f4d1019aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:43.388 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:42.827+0000 7f4d14f72640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4d1010afd0 con 0x7f4d101081c0 2024-09-12T02:48:43.389 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:42.827+0000 7f4d0e59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d101081c0 0x7f4d1019aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:43.389 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:42.827+0000 7f4d0e59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d101081c0 0x7f4d1019aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43650/0 (socket says 172.21.15.5:43650) 2024-09-12T02:48:43.389 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:42.827+0000 7f4d0e59c640 1 -- 172.21.15.5:0/2831767485 learned_addr learned my addr 172.21.15.5:0/2831767485 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:48:43.389 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:42.828+0000 7f4d0e59c640 1 -- 172.21.15.5:0/2831767485 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d1019b420 con 0x7f4d101081c0 2024-09-12T02:48:43.389 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:42.828+0000 7f4d0e59c640 1 --2- 172.21.15.5:0/2831767485 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d101081c0 0x7f4d1019aee0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f4d000090b0 tx=0x7f4d000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:48:43.389 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:42.828+0000 7f4cff7fe640 1 -- 172.21.15.5:0/2831767485 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4d00041070 con 0x7f4d101081c0 2024-09-12T02:48:43.389 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:42.828+0000 7f4cff7fe640 1 -- 172.21.15.5:0/2831767485 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d0003c7a0 con 0x7f4d101081c0 2024-09-12T02:48:43.389 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:42.828+0000 7f4d14f72640 1 -- 172.21.15.5:0/2831767485 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d1019b6b0 con 0x7f4d101081c0 2024-09-12T02:48:43.389 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:42.828+0000 7f4cff7fe640 1 -- 172.21.15.5:0/2831767485 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4d00037040 con 0x7f4d101081c0 2024-09-12T02:48:43.389 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:42.828+0000 7f4d14f72640 1 -- 172.21.15.5:0/2831767485 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d1019e3a0 con 0x7f4d101081c0 2024-09-12T02:48:43.389 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:42.828+0000 7f4cff7fe640 1 -- 172.21.15.5:0/2831767485 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f4d00044070 con 0x7f4d101081c0 2024-09-12T02:48:43.389 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:42.828+0000 7f4cff7fe640 1 -- 172.21.15.5:0/2831767485 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f4d0003d3f0 con 0x7f4d101081c0 2024-09-12T02:48:43.390 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:42.829+0000 7f4d14f72640 1 -- 172.21.15.5:0/2831767485 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d10108620 con 0x7f4d101081c0 2024-09-12T02:48:43.390 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:42.831+0000 7f4cff7fe640 1 -- 172.21.15.5:0/2831767485 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f4d00046450 con 0x7f4d101081c0 2024-09-12T02:48:43.390 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:42.893+0000 7f4d14f72640 1 -- 172.21.15.5:0/2831767485 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f4d1002d940 con 0x7f4d101081c0 2024-09-12T02:48:43.390 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:42.894+0000 7f4cff7fe640 1 -- 172.21.15.5:0/2831767485 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f4d00046630 con 0x7f4d101081c0 2024-09-12T02:48:43.390 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:42.894+0000 7f4d14f72640 1 -- 172.21.15.5:0/2831767485 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d101081c0 msgr2=0x7f4d1019aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:43.390 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:42.894+0000 7f4d14f72640 1 --2- 172.21.15.5:0/2831767485 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d101081c0 0x7f4d1019aee0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f4d000090b0 tx=0x7f4d000046c0 comp rx=0 tx=0).stop 2024-09-12T02:48:43.390 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:42.894+0000 7f4d14f72640 1 -- 172.21.15.5:0/2831767485 shutdown_connections 2024-09-12T02:48:43.390 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:42.894+0000 7f4d14f72640 1 --2- 172.21.15.5:0/2831767485 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d101081c0 0x7f4d1019aee0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:43.390 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:42.894+0000 7f4d14f72640 1 -- 172.21.15.5:0/2831767485 >> 172.21.15.5:0/2831767485 conn(0x7f4d10101e20 msgr2=0x7f4d10102ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:48:43.390 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:42.895+0000 7f4d14f72640 1 -- 172.21.15.5:0/2831767485 shutdown_connections 2024-09-12T02:48:43.390 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:42.895+0000 7f4d14f72640 1 -- 172.21.15.5:0/2831767485 wait complete. 2024-09-12T02:48:43.390 INFO:teuthology.orchestra.run.smithi005.stdout:mon is available 2024-09-12T02:48:43.390 INFO:teuthology.orchestra.run.smithi005.stdout:Assimilating anything we can from ceph.conf... 2024-09-12T02:48:44.400 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2024-09-12T02:48:44.400 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout [global] 2024-09-12T02:48:44.400 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout fsid = 618305f8-70b1-11ef-bcea-c7b262605968 2024-09-12T02:48:44.400 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.5:3300,v1:172.21.15.5:6789] 2024-09-12T02:48:44.400 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-12T02:48:44.400 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-12T02:48:44.400 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-12T02:48:44.400 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-12T02:48:44.400 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2024-09-12T02:48:44.400 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-12T02:48:44.401 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-12T02:48:44.401 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2024-09-12T02:48:44.401 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout [osd] 2024-09-12T02:48:44.401 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-12T02:48:44.401 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-12T02:48:44.401 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-12T02:48:44.401 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-12T02:48:44.401 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-12T02:48:44.401 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:43.810+0000 7ffb23c4c640 1 Processor -- start 2024-09-12T02:48:44.403 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:43.810+0000 7ffb23c4c640 1 -- start start 2024-09-12T02:48:44.404 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:43.810+0000 7ffb23c4c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb1c1081d0 0x7ffb1c10a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:44.404 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:43.810+0000 7ffb23c4c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ffb1c07bdf0 con 0x7ffb1c1081d0 2024-09-12T02:48:44.404 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:43.811+0000 7ffb219e8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb1c1081d0 0x7ffb1c10a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:44.404 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:43.811+0000 7ffb219e8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb1c1081d0 0x7ffb1c10a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52768/0 (socket says 172.21.15.5:52768) 2024-09-12T02:48:44.404 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:43.811+0000 7ffb219e8640 1 -- 172.21.15.5:0/2812574046 learned_addr learned my addr 172.21.15.5:0/2812574046 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:48:44.404 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:43.811+0000 7ffb219e8640 1 -- 172.21.15.5:0/2812574046 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb1c10ab00 con 0x7ffb1c1081d0 2024-09-12T02:48:44.404 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:43.812+0000 7ffb219e8640 1 --2- 172.21.15.5:0/2812574046 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb1c1081d0 0x7ffb1c10a5c0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7ffb18009820 tx=0x7ffb1802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0fac38f3a921b2f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:48:44.404 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:43.812+0000 7ffb209e6640 1 -- 172.21.15.5:0/2812574046 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffb18037070 con 0x7ffb1c1081d0 2024-09-12T02:48:44.404 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:43.812+0000 7ffb209e6640 1 -- 172.21.15.5:0/2812574046 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb18032440 con 0x7ffb1c1081d0 2024-09-12T02:48:44.404 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:43.812+0000 7ffb209e6640 1 -- 172.21.15.5:0/2812574046 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffb18032740 con 0x7ffb1c1081d0 2024-09-12T02:48:44.404 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:43.813+0000 7ffb23c4c640 1 -- 172.21.15.5:0/2812574046 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb1c1081d0 msgr2=0x7ffb1c10a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:44.405 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:43.813+0000 7ffb23c4c640 1 --2- 172.21.15.5:0/2812574046 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb1c1081d0 0x7ffb1c10a5c0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7ffb18009820 tx=0x7ffb1802a270 comp rx=0 tx=0).stop 2024-09-12T02:48:44.405 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:43.813+0000 7ffb23c4c640 1 -- 172.21.15.5:0/2812574046 shutdown_connections 2024-09-12T02:48:44.405 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:43.813+0000 7ffb23c4c640 1 --2- 172.21.15.5:0/2812574046 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb1c1081d0 0x7ffb1c10a5c0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:44.405 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:43.813+0000 7ffb23c4c640 1 -- 172.21.15.5:0/2812574046 >> 172.21.15.5:0/2812574046 conn(0x7ffb1c101e70 msgr2=0x7ffb1c104290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:48:44.405 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:43.813+0000 7ffb23c4c640 1 -- 172.21.15.5:0/2812574046 shutdown_connections 2024-09-12T02:48:44.405 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:43.813+0000 7ffb23c4c640 1 -- 172.21.15.5:0/2812574046 wait complete. 2024-09-12T02:48:44.405 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:43.813+0000 7ffb23c4c640 1 Processor -- start 2024-09-12T02:48:44.405 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:43.813+0000 7ffb23c4c640 1 -- start start 2024-09-12T02:48:44.405 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:43.814+0000 7ffb23c4c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb1c1081d0 0x7ffb1c107a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:44.405 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:43.814+0000 7ffb23c4c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ffb1c10afe0 con 0x7ffb1c1081d0 2024-09-12T02:48:44.405 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:43.814+0000 7ffb219e8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb1c1081d0 0x7ffb1c107a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:44.405 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:43.814+0000 7ffb219e8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb1c1081d0 0x7ffb1c107a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52780/0 (socket says 172.21.15.5:52780) 2024-09-12T02:48:44.405 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:43.814+0000 7ffb219e8640 1 -- 172.21.15.5:0/4061188736 learned_addr learned my addr 172.21.15.5:0/4061188736 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:48:44.406 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:43.814+0000 7ffb219e8640 1 -- 172.21.15.5:0/4061188736 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb1c107f90 con 0x7ffb1c1081d0 2024-09-12T02:48:44.406 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:43.814+0000 7ffb219e8640 1 --2- 172.21.15.5:0/4061188736 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb1c1081d0 0x7ffb1c107a50 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7ffb180090b0 tx=0x7ffb180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:48:44.406 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:43.814+0000 7ffb12ffd640 1 -- 172.21.15.5:0/4061188736 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffb18041070 con 0x7ffb1c1081d0 2024-09-12T02:48:44.406 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:43.814+0000 7ffb12ffd640 1 -- 172.21.15.5:0/4061188736 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb1803c7a0 con 0x7ffb1c1081d0 2024-09-12T02:48:44.406 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:43.814+0000 7ffb23c4c640 1 -- 172.21.15.5:0/4061188736 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb1c106180 con 0x7ffb1c1081d0 2024-09-12T02:48:44.406 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:43.814+0000 7ffb12ffd640 1 -- 172.21.15.5:0/4061188736 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffb18037040 con 0x7ffb1c1081d0 2024-09-12T02:48:44.406 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:43.814+0000 7ffb23c4c640 1 -- 172.21.15.5:0/4061188736 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb1c1066c0 con 0x7ffb1c1081d0 2024-09-12T02:48:44.406 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:43.815+0000 7ffb12ffd640 1 -- 172.21.15.5:0/4061188736 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ffb18044070 con 0x7ffb1c1081d0 2024-09-12T02:48:44.406 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:43.815+0000 7ffb12ffd640 1 -- 172.21.15.5:0/4061188736 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ffb1803d3f0 con 0x7ffb1c1081d0 2024-09-12T02:48:44.406 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:43.816+0000 7ffb23c4c640 1 -- 172.21.15.5:0/4061188736 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb1c108630 con 0x7ffb1c1081d0 2024-09-12T02:48:44.406 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:43.818+0000 7ffb12ffd640 1 -- 172.21.15.5:0/4061188736 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7ffb18046450 con 0x7ffb1c1081d0 2024-09-12T02:48:44.406 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:43.878+0000 7ffb23c4c640 1 -- 172.21.15.5:0/4061188736 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7ffb1c194890 con 0x7ffb1c1081d0 2024-09-12T02:48:44.406 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:43.881+0000 7ffb12ffd640 1 -- 172.21.15.5:0/4061188736 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+427 (secure 0 0 0) 0x7ffb18046630 con 0x7ffb1c1081d0 2024-09-12T02:48:44.407 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:43.881+0000 7ffb12ffd640 1 -- 172.21.15.5:0/4061188736 <== mon.0 v2:172.21.15.5:3300/0 8 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ffb18056070 con 0x7ffb1c1081d0 2024-09-12T02:48:44.407 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:43.882+0000 7ffb23c4c640 1 -- 172.21.15.5:0/4061188736 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb1c1081d0 msgr2=0x7ffb1c107a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:44.407 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:43.882+0000 7ffb23c4c640 1 --2- 172.21.15.5:0/4061188736 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb1c1081d0 0x7ffb1c107a50 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7ffb180090b0 tx=0x7ffb180046c0 comp rx=0 tx=0).stop 2024-09-12T02:48:44.407 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:43.882+0000 7ffb23c4c640 1 -- 172.21.15.5:0/4061188736 shutdown_connections 2024-09-12T02:48:44.407 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:43.882+0000 7ffb23c4c640 1 --2- 172.21.15.5:0/4061188736 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb1c1081d0 0x7ffb1c107a50 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:44.407 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:43.882+0000 7ffb23c4c640 1 -- 172.21.15.5:0/4061188736 >> 172.21.15.5:0/4061188736 conn(0x7ffb1c101e70 msgr2=0x7ffb1c102f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:48:44.407 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:43.882+0000 7ffb23c4c640 1 -- 172.21.15.5:0/4061188736 shutdown_connections 2024-09-12T02:48:44.407 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:43.882+0000 7ffb23c4c640 1 -- 172.21.15.5:0/4061188736 wait complete. 2024-09-12T02:48:44.407 INFO:teuthology.orchestra.run.smithi005.stdout:Generating new minimal ceph.conf... 2024-09-12T02:48:45.434 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:44.816+0000 7fb22dd40640 1 Processor -- start 2024-09-12T02:48:45.434 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:44.816+0000 7fb22dd40640 1 -- start start 2024-09-12T02:48:45.434 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:44.816+0000 7fb22dd40640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb228105a80 0x7fb228105e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:45.438 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:44.816+0000 7fb22dd40640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb228106450 con 0x7fb228105a80 2024-09-12T02:48:45.439 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:44.816+0000 7fb2277fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb228105a80 0x7fb228105e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:45.439 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:44.816+0000 7fb2277fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb228105a80 0x7fb228105e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52796/0 (socket says 172.21.15.5:52796) 2024-09-12T02:48:45.439 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:44.817+0000 7fb2277fe640 1 -- 172.21.15.5:0/2166998418 learned_addr learned my addr 172.21.15.5:0/2166998418 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:48:45.439 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:44.817+0000 7fb2277fe640 1 -- 172.21.15.5:0/2166998418 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2281065d0 con 0x7fb228105a80 2024-09-12T02:48:45.439 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:44.817+0000 7fb2277fe640 1 --2- 172.21.15.5:0/2166998418 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb228105a80 0x7fb228105e80 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fb218009820 tx=0x7fb21802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd7a7d9913b0fc7c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:48:45.439 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:44.818+0000 7fb2267fc640 1 -- 172.21.15.5:0/2166998418 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb218037070 con 0x7fb228105a80 2024-09-12T02:48:45.439 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:44.818+0000 7fb2267fc640 1 -- 172.21.15.5:0/2166998418 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb218032900 con 0x7fb228105a80 2024-09-12T02:48:45.440 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:44.818+0000 7fb2267fc640 1 -- 172.21.15.5:0/2166998418 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb218030850 con 0x7fb228105a80 2024-09-12T02:48:45.440 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:44.818+0000 7fb22dd40640 1 -- 172.21.15.5:0/2166998418 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb228105a80 msgr2=0x7fb228105e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:45.440 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:44.818+0000 7fb22dd40640 1 --2- 172.21.15.5:0/2166998418 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb228105a80 0x7fb228105e80 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fb218009820 tx=0x7fb21802a270 comp rx=0 tx=0).stop 2024-09-12T02:48:45.440 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:44.818+0000 7fb22dd40640 1 -- 172.21.15.5:0/2166998418 shutdown_connections 2024-09-12T02:48:45.440 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:44.818+0000 7fb22dd40640 1 --2- 172.21.15.5:0/2166998418 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb228105a80 0x7fb228105e80 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:45.441 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:44.818+0000 7fb22dd40640 1 -- 172.21.15.5:0/2166998418 >> 172.21.15.5:0/2166998418 conn(0x7fb2281015f0 msgr2=0x7fb228103a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:48:45.441 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:44.818+0000 7fb22dd40640 1 -- 172.21.15.5:0/2166998418 shutdown_connections 2024-09-12T02:48:45.441 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:44.818+0000 7fb22dd40640 1 -- 172.21.15.5:0/2166998418 wait complete. 2024-09-12T02:48:45.441 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:44.819+0000 7fb22dd40640 1 Processor -- start 2024-09-12T02:48:45.441 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:44.819+0000 7fb22dd40640 1 -- start start 2024-09-12T02:48:45.441 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:44.819+0000 7fb22dd40640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb228105a80 0x7fb22819b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:45.442 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:44.819+0000 7fb22dd40640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb2281098f0 con 0x7fb228105a80 2024-09-12T02:48:45.442 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:44.819+0000 7fb2277fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb228105a80 0x7fb22819b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:45.442 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:44.819+0000 7fb2277fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb228105a80 0x7fb22819b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52806/0 (socket says 172.21.15.5:52806) 2024-09-12T02:48:45.442 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:44.819+0000 7fb2277fe640 1 -- 172.21.15.5:0/2685520177 learned_addr learned my addr 172.21.15.5:0/2685520177 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:48:45.442 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:44.820+0000 7fb2277fe640 1 -- 172.21.15.5:0/2685520177 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb22819bc20 con 0x7fb228105a80 2024-09-12T02:48:45.442 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:44.820+0000 7fb2277fe640 1 --2- 172.21.15.5:0/2685520177 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb228105a80 0x7fb22819b6e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb218009940 tx=0x7fb218032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:48:45.442 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:44.820+0000 7fb224ff9640 1 -- 172.21.15.5:0/2685520177 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb218042070 con 0x7fb228105a80 2024-09-12T02:48:45.442 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:44.820+0000 7fb224ff9640 1 -- 172.21.15.5:0/2685520177 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb2180308c0 con 0x7fb228105a80 2024-09-12T02:48:45.442 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:44.820+0000 7fb22dd40640 1 -- 172.21.15.5:0/2685520177 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb22819bf10 con 0x7fb228105a80 2024-09-12T02:48:45.442 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:44.820+0000 7fb224ff9640 1 -- 172.21.15.5:0/2685520177 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb218037040 con 0x7fb228105a80 2024-09-12T02:48:45.442 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:44.820+0000 7fb22dd40640 1 -- 172.21.15.5:0/2685520177 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb22819eba0 con 0x7fb228105a80 2024-09-12T02:48:45.443 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:44.820+0000 7fb224ff9640 1 -- 172.21.15.5:0/2685520177 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb218045070 con 0x7fb228105a80 2024-09-12T02:48:45.443 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:44.820+0000 7fb224ff9640 1 -- 172.21.15.5:0/2685520177 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb218047c10 con 0x7fb228105a80 2024-09-12T02:48:45.443 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:44.821+0000 7fb22dd40640 1 -- 172.21.15.5:0/2685520177 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb228105f00 con 0x7fb228105a80 2024-09-12T02:48:45.443 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:44.823+0000 7fb224ff9640 1 -- 172.21.15.5:0/2685520177 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fb21804c030 con 0x7fb228105a80 2024-09-12T02:48:45.443 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:44.884+0000 7fb22dd40640 1 -- 172.21.15.5:0/2685520177 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fb22819f1f0 con 0x7fb228105a80 2024-09-12T02:48:45.443 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:44.884+0000 7fb224ff9640 1 -- 172.21.15.5:0/2685520177 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+173 (secure 0 0 0) 0x7fb2180327b0 con 0x7fb228105a80 2024-09-12T02:48:45.443 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:44.885+0000 7fb22dd40640 1 -- 172.21.15.5:0/2685520177 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb228105a80 msgr2=0x7fb22819b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:45.443 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:44.885+0000 7fb22dd40640 1 --2- 172.21.15.5:0/2685520177 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb228105a80 0x7fb22819b6e0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb218009940 tx=0x7fb218032ce0 comp rx=0 tx=0).stop 2024-09-12T02:48:45.443 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:44.885+0000 7fb22dd40640 1 -- 172.21.15.5:0/2685520177 shutdown_connections 2024-09-12T02:48:45.443 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:44.885+0000 7fb22dd40640 1 --2- 172.21.15.5:0/2685520177 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb228105a80 0x7fb22819b6e0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:45.443 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:44.885+0000 7fb22dd40640 1 -- 172.21.15.5:0/2685520177 >> 172.21.15.5:0/2685520177 conn(0x7fb2281015f0 msgr2=0x7fb2281021a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:48:45.443 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:44.885+0000 7fb22dd40640 1 -- 172.21.15.5:0/2685520177 shutdown_connections 2024-09-12T02:48:45.444 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:44.885+0000 7fb22dd40640 1 -- 172.21.15.5:0/2685520177 wait complete. 2024-09-12T02:48:45.444 INFO:teuthology.orchestra.run.smithi005.stdout:Restarting the monitor... 2024-09-12T02:48:45.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:45 smithi005 systemd[1]: Stopping Ceph mon.a for 618305f8-70b1-11ef-bcea-c7b262605968... 2024-09-12T02:48:46.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:45 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mon-a[30825]: 2024-09-12T02:48:45.731+0000 7fd268aae640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T02:48:46.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:45 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mon-a[30825]: 2024-09-12T02:48:45.731+0000 7fd268aae640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-12T02:48:46.061 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:45 smithi005 podman[31071]: 2024-09-12 02:48:45.874978961 +0000 UTC m=+0.267163602 container died fb1a104701f07aecb14fb2867344e52575fde226a4784ef02a314161ff26fb83 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-mon-a, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-12T02:48:46.410 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:46 smithi005 podman[31071]: 2024-09-12 02:48:46.192882568 +0000 UTC m=+0.585067207 container cleanup fb1a104701f07aecb14fb2867344e52575fde226a4784ef02a314161ff26fb83 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-mon-a, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-12T02:48:46.411 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:46 smithi005 bash[31071]: ceph-618305f8-70b1-11ef-bcea-c7b262605968-mon-a 2024-09-12T02:48:46.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:46 smithi005 podman[31084]: 2024-09-12 02:48:46.410522565 +0000 UTC m=+0.529126559 container remove fb1a104701f07aecb14fb2867344e52575fde226a4784ef02a314161ff26fb83 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-mon-a, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-12T02:48:46.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:46 smithi005 systemd[1]: ceph-618305f8-70b1-11ef-bcea-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-12T02:48:46.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:46 smithi005 systemd[1]: Stopped Ceph mon.a for 618305f8-70b1-11ef-bcea-c7b262605968. 2024-09-12T02:48:46.811 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:46 smithi005 systemd[1]: Starting Ceph mon.a for 618305f8-70b1-11ef-bcea-c7b262605968... 2024-09-12T02:48:47.097 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:46 smithi005 podman[31178]: 2024-09-12 02:48:46.744472545 +0000 UTC m=+0.016774633 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc 2024-09-12T02:48:47.097 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:46 smithi005 podman[31178]: 2024-09-12 02:48:46.861312654 +0000 UTC m=+0.133614726 container create 62861099a2acfdd3da3b829c4cbab84623c36b01d03f445532a93ebf5c6735dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-mon-a, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T02:48:47.097 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 podman[31178]: 2024-09-12 02:48:47.072079079 +0000 UTC m=+0.344381160 container init 62861099a2acfdd3da3b829c4cbab84623c36b01d03f445532a93ebf5c6735dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-mon-a, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T02:48:47.098 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 podman[31178]: 2024-09-12 02:48:47.075181948 +0000 UTC m=+0.347484019 container start 62861099a2acfdd3da3b829c4cbab84623c36b01d03f445532a93ebf5c6735dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-mon-a, GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True) 2024-09-12T02:48:47.159 INFO:teuthology.orchestra.run.smithi005.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-12T02:48:47.359 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: set uid:gid to 167:167 (ceph:ceph) 2024-09-12T02:48:47.359 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable), process ceph-mon, pid 2 2024-09-12T02:48:47.359 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: pidfile_write: ignore empty --pid-file 2024-09-12T02:48:47.359 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: load: jerasure load: lrc 2024-09-12T02:48:47.360 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: RocksDB version: 7.9.2 2024-09-12T02:48:47.360 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Git sha 0 2024-09-12T02:48:47.360 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Compile date 2024-09-09 16:23:28 2024-09-12T02:48:47.360 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: DB SUMMARY 2024-09-12T02:48:47.360 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: DB Session ID: NXHYAEROEXNYP8FV1NYZ 2024-09-12T02:48:47.360 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: CURRENT file: CURRENT 2024-09-12T02:48:47.360 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: IDENTITY file: IDENTITY 2024-09-12T02:48:47.360 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-12T02:48:47.360 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-12T02:48:47.360 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 106190 ; 2024-09-12T02:48:47.360 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.error_if_exists: 0 2024-09-12T02:48:47.360 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.create_if_missing: 0 2024-09-12T02:48:47.360 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.paranoid_checks: 1 2024-09-12T02:48:47.361 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-12T02:48:47.361 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-12T02:48:47.361 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-12T02:48:47.361 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.env: 0x55a0b0159940 2024-09-12T02:48:47.361 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.fs: PosixFileSystem 2024-09-12T02:48:47.361 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.info_log: 0x55a0b172d6e0 2024-09-12T02:48:47.361 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.max_file_opening_threads: 16 2024-09-12T02:48:47.361 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.statistics: (nil) 2024-09-12T02:48:47.361 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.use_fsync: 0 2024-09-12T02:48:47.361 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.max_log_file_size: 0 2024-09-12T02:48:47.361 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-12T02:48:47.361 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-12T02:48:47.361 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.keep_log_file_num: 1000 2024-09-12T02:48:47.362 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.recycle_log_file_num: 0 2024-09-12T02:48:47.362 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.allow_fallocate: 1 2024-09-12T02:48:47.362 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.allow_mmap_reads: 0 2024-09-12T02:48:47.362 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.allow_mmap_writes: 0 2024-09-12T02:48:47.362 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.use_direct_reads: 0 2024-09-12T02:48:47.362 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-12T02:48:47.362 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.create_missing_column_families: 0 2024-09-12T02:48:47.362 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.db_log_dir: 2024-09-12T02:48:47.362 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.wal_dir: 2024-09-12T02:48:47.362 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-12T02:48:47.364 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-12T02:48:47.364 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-12T02:48:47.364 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-12T02:48:47.364 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-12T02:48:47.364 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-12T02:48:47.365 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.advise_random_on_open: 1 2024-09-12T02:48:47.365 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.db_write_buffer_size: 0 2024-09-12T02:48:47.365 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.write_buffer_manager: 0x55a0b173cdc0 2024-09-12T02:48:47.365 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-12T02:48:47.365 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-12T02:48:47.365 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-12T02:48:47.365 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.rate_limiter: (nil) 2024-09-12T02:48:47.365 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-12T02:48:47.365 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.wal_recovery_mode: 2 2024-09-12T02:48:47.365 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.enable_thread_tracking: 0 2024-09-12T02:48:47.366 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.enable_pipelined_write: 0 2024-09-12T02:48:47.366 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.unordered_write: 0 2024-09-12T02:48:47.366 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-12T02:48:47.366 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-12T02:48:47.366 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-12T02:48:47.366 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-12T02:48:47.366 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.row_cache: None 2024-09-12T02:48:47.366 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.wal_filter: None 2024-09-12T02:48:47.366 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-12T02:48:47.366 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.allow_ingest_behind: 0 2024-09-12T02:48:47.366 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.two_write_queues: 0 2024-09-12T02:48:47.366 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.manual_wal_flush: 0 2024-09-12T02:48:47.366 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.wal_compression: 0 2024-09-12T02:48:47.367 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.atomic_flush: 0 2024-09-12T02:48:47.367 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-12T02:48:47.367 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-12T02:48:47.367 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-12T02:48:47.368 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.log_readahead_size: 0 2024-09-12T02:48:47.368 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-12T02:48:47.368 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.best_efforts_recovery: 0 2024-09-12T02:48:47.368 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-12T02:48:47.368 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-12T02:48:47.369 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.allow_data_in_errors: 0 2024-09-12T02:48:47.369 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.db_host_id: __hostname__ 2024-09-12T02:48:47.369 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-12T02:48:47.369 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.max_background_jobs: 2 2024-09-12T02:48:47.369 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.max_background_compactions: -1 2024-09-12T02:48:47.369 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.max_subcompactions: 1 2024-09-12T02:48:47.369 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-12T02:48:47.369 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-12T02:48:47.369 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-12T02:48:47.369 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.max_total_wal_size: 0 2024-09-12T02:48:47.369 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-12T02:48:47.369 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-12T02:48:47.369 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-12T02:48:47.369 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-12T02:48:47.370 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.max_open_files: -1 2024-09-12T02:48:47.370 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.bytes_per_sync: 0 2024-09-12T02:48:47.370 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-12T02:48:47.370 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-12T02:48:47.370 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.compaction_readahead_size: 0 2024-09-12T02:48:47.370 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.max_background_flushes: -1 2024-09-12T02:48:47.370 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Compression algorithms supported: 2024-09-12T02:48:47.370 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: kZSTD supported: 0 2024-09-12T02:48:47.370 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: kXpressCompression supported: 0 2024-09-12T02:48:47.370 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: kBZip2Compression supported: 0 2024-09-12T02:48:47.370 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-12T02:48:47.370 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: kLZ4Compression supported: 1 2024-09-12T02:48:47.370 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: kZlibCompression supported: 1 2024-09-12T02:48:47.370 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: kLZ4HCCompression supported: 1 2024-09-12T02:48:47.371 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: kSnappyCompression supported: 1 2024-09-12T02:48:47.371 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-12T02:48:47.371 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-12T02:48:47.371 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-09-12T02:48:47.371 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-12T02:48:47.371 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-12T02:48:47.371 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.merge_operator: 2024-09-12T02:48:47.371 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.compaction_filter: None 2024-09-12T02:48:47.371 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.compaction_filter_factory: None 2024-09-12T02:48:47.372 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.sst_partitioner_factory: None 2024-09-12T02:48:47.372 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-12T02:48:47.372 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-12T02:48:47.372 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55a0b172d320) 2024-09-12T02:48:47.372 INFO:journalctl@ceph.mon.a.smithi005.stdout: cache_index_and_filter_blocks: 1 2024-09-12T02:48:47.372 INFO:journalctl@ceph.mon.a.smithi005.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-12T02:48:47.373 INFO:journalctl@ceph.mon.a.smithi005.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-12T02:48:47.373 INFO:journalctl@ceph.mon.a.smithi005.stdout: pin_top_level_index_and_filter: 1 2024-09-12T02:48:47.373 INFO:journalctl@ceph.mon.a.smithi005.stdout: index_type: 0 2024-09-12T02:48:47.373 INFO:journalctl@ceph.mon.a.smithi005.stdout: data_block_index_type: 0 2024-09-12T02:48:47.373 INFO:journalctl@ceph.mon.a.smithi005.stdout: index_shortening: 1 2024-09-12T02:48:47.373 INFO:journalctl@ceph.mon.a.smithi005.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-12T02:48:47.373 INFO:journalctl@ceph.mon.a.smithi005.stdout: checksum: 4 2024-09-12T02:48:47.373 INFO:journalctl@ceph.mon.a.smithi005.stdout: no_block_cache: 0 2024-09-12T02:48:47.373 INFO:journalctl@ceph.mon.a.smithi005.stdout: block_cache: 0x55a0b17211f0 2024-09-12T02:48:47.373 INFO:journalctl@ceph.mon.a.smithi005.stdout: block_cache_name: BinnedLRUCache 2024-09-12T02:48:47.374 INFO:journalctl@ceph.mon.a.smithi005.stdout: block_cache_options: 2024-09-12T02:48:47.374 INFO:journalctl@ceph.mon.a.smithi005.stdout: capacity : 536870912 2024-09-12T02:48:47.374 INFO:journalctl@ceph.mon.a.smithi005.stdout: num_shard_bits : 4 2024-09-12T02:48:47.374 INFO:journalctl@ceph.mon.a.smithi005.stdout: strict_capacity_limit : 0 2024-09-12T02:48:47.374 INFO:journalctl@ceph.mon.a.smithi005.stdout: high_pri_pool_ratio: 0.000 2024-09-12T02:48:47.374 INFO:journalctl@ceph.mon.a.smithi005.stdout: block_cache_compressed: (nil) 2024-09-12T02:48:47.374 INFO:journalctl@ceph.mon.a.smithi005.stdout: persistent_cache: (nil) 2024-09-12T02:48:47.374 INFO:journalctl@ceph.mon.a.smithi005.stdout: block_size: 4096 2024-09-12T02:48:47.374 INFO:journalctl@ceph.mon.a.smithi005.stdout: block_size_deviation: 10 2024-09-12T02:48:47.374 INFO:journalctl@ceph.mon.a.smithi005.stdout: block_restart_interval: 16 2024-09-12T02:48:47.374 INFO:journalctl@ceph.mon.a.smithi005.stdout: index_block_restart_interval: 1 2024-09-12T02:48:47.374 INFO:journalctl@ceph.mon.a.smithi005.stdout: metadata_block_size: 4096 2024-09-12T02:48:47.375 INFO:journalctl@ceph.mon.a.smithi005.stdout: partition_filters: 0 2024-09-12T02:48:47.375 INFO:journalctl@ceph.mon.a.smithi005.stdout: use_delta_encoding: 1 2024-09-12T02:48:47.375 INFO:journalctl@ceph.mon.a.smithi005.stdout: filter_policy: bloomfilter 2024-09-12T02:48:47.375 INFO:journalctl@ceph.mon.a.smithi005.stdout: whole_key_filtering: 1 2024-09-12T02:48:47.375 INFO:journalctl@ceph.mon.a.smithi005.stdout: verify_compression: 0 2024-09-12T02:48:47.375 INFO:journalctl@ceph.mon.a.smithi005.stdout: read_amp_bytes_per_bit: 0 2024-09-12T02:48:47.375 INFO:journalctl@ceph.mon.a.smithi005.stdout: format_version: 5 2024-09-12T02:48:47.375 INFO:journalctl@ceph.mon.a.smithi005.stdout: enable_index_compression: 1 2024-09-12T02:48:47.375 INFO:journalctl@ceph.mon.a.smithi005.stdout: block_align: 0 2024-09-12T02:48:47.375 INFO:journalctl@ceph.mon.a.smithi005.stdout: max_auto_readahead_size: 262144 2024-09-12T02:48:47.375 INFO:journalctl@ceph.mon.a.smithi005.stdout: prepopulate_block_cache: 0 2024-09-12T02:48:47.376 INFO:journalctl@ceph.mon.a.smithi005.stdout: initial_auto_readahead_size: 8192 2024-09-12T02:48:47.376 INFO:journalctl@ceph.mon.a.smithi005.stdout: num_file_reads_for_auto_readahead: 2 2024-09-12T02:48:47.376 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.write_buffer_size: 33554432 2024-09-12T02:48:47.376 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.max_write_buffer_number: 2 2024-09-12T02:48:47.376 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.compression: NoCompression 2024-09-12T02:48:47.376 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.bottommost_compression: Disabled 2024-09-12T02:48:47.376 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.prefix_extractor: nullptr 2024-09-12T02:48:47.376 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-12T02:48:47.376 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.num_levels: 7 2024-09-12T02:48:47.376 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-12T02:48:47.376 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-12T02:48:47.377 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-12T02:48:47.377 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-12T02:48:47.377 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-12T02:48:47.377 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-12T02:48:47.377 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-12T02:48:47.377 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-12T02:48:47.377 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-12T02:48:47.378 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-12T02:48:47.378 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-12T02:48:47.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-12T02:48:47.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-12T02:48:47.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.compression_opts.level: 32767 2024-09-12T02:48:47.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.compression_opts.strategy: 0 2024-09-12T02:48:47.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-12T02:48:47.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-12T02:48:47.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-12T02:48:47.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-12T02:48:47.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.compression_opts.enabled: false 2024-09-12T02:48:47.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-12T02:48:47.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-12T02:48:47.379 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-12T02:48:47.380 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-12T02:48:47.380 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.target_file_size_base: 67108864 2024-09-12T02:48:47.380 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-12T02:48:47.380 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-12T02:48:47.380 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-12T02:48:47.380 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-12T02:48:47.380 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-12T02:48:47.380 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-12T02:48:47.380 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-12T02:48:47.380 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-12T02:48:47.380 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-12T02:48:47.380 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-12T02:48:47.380 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-12T02:48:47.381 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-12T02:48:47.381 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-12T02:48:47.381 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-12T02:48:47.381 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.arena_block_size: 1048576 2024-09-12T02:48:47.381 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-12T02:48:47.381 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-12T02:48:47.381 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.disable_auto_compactions: 0 2024-09-12T02:48:47.381 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-12T02:48:47.381 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-12T02:48:47.381 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-12T02:48:47.381 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-12T02:48:47.382 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-12T02:48:47.382 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-12T02:48:47.382 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-12T02:48:47.382 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-12T02:48:47.382 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-12T02:48:47.382 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-12T02:48:47.382 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-12T02:48:47.382 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.inplace_update_support: 0 2024-09-12T02:48:47.382 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-12T02:48:47.382 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-12T02:48:47.382 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-12T02:48:47.382 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-12T02:48:47.382 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.bloom_locality: 0 2024-09-12T02:48:47.383 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.max_successive_merges: 0 2024-09-12T02:48:47.383 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-12T02:48:47.383 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.paranoid_file_checks: 0 2024-09-12T02:48:47.383 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.force_consistency_checks: 1 2024-09-12T02:48:47.383 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.report_bg_io_stats: 0 2024-09-12T02:48:47.383 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.ttl: 2592000 2024-09-12T02:48:47.383 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-12T02:48:47.383 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-12T02:48:47.383 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-12T02:48:47.383 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.enable_blob_files: false 2024-09-12T02:48:47.383 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.min_blob_size: 0 2024-09-12T02:48:47.383 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.blob_file_size: 268435456 2024-09-12T02:48:47.383 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-12T02:48:47.383 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-12T02:48:47.384 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-12T02:48:47.384 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-12T02:48:47.384 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-12T02:48:47.384 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.blob_file_starting_level: 0 2024-09-12T02:48:47.384 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-12T02:48:47.384 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-12T02:48:47.384 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-12T02:48:47.384 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 97d41d94-16e0-4ac7-acc7-b5d851a82d3a 2024-09-12T02:48:47.384 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726109327103329, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-12T02:48:47.384 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-12T02:48:47.384 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726109327104526, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 101146, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 332, "table_properties": {"data_size": 99031, "index_size": 307, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14538, "raw_average_key_size": 49, "raw_value_size": 91187, "raw_average_value_size": 313, "num_data_blocks": 12, "num_entries": 291, "num_filter_entries": 291, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726109327, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "97d41d94-16e0-4ac7-acc7-b5d851a82d3a", "db_session_id": "NXHYAEROEXNYP8FV1NYZ", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-12T02:48:47.384 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726109327104624, "job": 1, "event": "recovery_finished"} 2024-09-12T02:48:47.384 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-12T02:48:47.385 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-12T02:48:47.385 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55a0b174ee00 2024-09-12T02:48:47.385 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: DB pointer 0x55a0b184a000 2024-09-12T02:48:47.385 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-12T02:48:47.386 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-12T02:48:47.386 INFO:journalctl@ceph.mon.a.smithi005.stdout: ** DB Stats ** 2024-09-12T02:48:47.386 INFO:journalctl@ceph.mon.a.smithi005.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-12T02:48:47.386 INFO:journalctl@ceph.mon.a.smithi005.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-12T02:48:47.386 INFO:journalctl@ceph.mon.a.smithi005.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-12T02:48:47.386 INFO:journalctl@ceph.mon.a.smithi005.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-12T02:48:47.386 INFO:journalctl@ceph.mon.a.smithi005.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-12T02:48:47.386 INFO:journalctl@ceph.mon.a.smithi005.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-12T02:48:47.386 INFO:journalctl@ceph.mon.a.smithi005.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-12T02:48:47.386 INFO:journalctl@ceph.mon.a.smithi005.stdout: 2024-09-12T02:48:47.386 INFO:journalctl@ceph.mon.a.smithi005.stdout: ** Compaction Stats [default] ** 2024-09-12T02:48:47.386 INFO:journalctl@ceph.mon.a.smithi005.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-12T02:48:47.387 INFO:journalctl@ceph.mon.a.smithi005.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-12T02:48:47.387 INFO:journalctl@ceph.mon.a.smithi005.stdout: L0 2/0 100.64 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 122.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-12T02:48:47.387 INFO:journalctl@ceph.mon.a.smithi005.stdout: Sum 2/0 100.64 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 122.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-12T02:48:47.387 INFO:journalctl@ceph.mon.a.smithi005.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 122.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-12T02:48:47.387 INFO:journalctl@ceph.mon.a.smithi005.stdout: 2024-09-12T02:48:47.387 INFO:journalctl@ceph.mon.a.smithi005.stdout: ** Compaction Stats [default] ** 2024-09-12T02:48:47.387 INFO:journalctl@ceph.mon.a.smithi005.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-12T02:48:47.387 INFO:journalctl@ceph.mon.a.smithi005.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-12T02:48:47.387 INFO:journalctl@ceph.mon.a.smithi005.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 122.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-12T02:48:47.387 INFO:journalctl@ceph.mon.a.smithi005.stdout: 2024-09-12T02:48:47.387 INFO:journalctl@ceph.mon.a.smithi005.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-12T02:48:47.387 INFO:journalctl@ceph.mon.a.smithi005.stdout: 2024-09-12T02:48:47.388 INFO:journalctl@ceph.mon.a.smithi005.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-12T02:48:47.388 INFO:journalctl@ceph.mon.a.smithi005.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-12T02:48:47.388 INFO:journalctl@ceph.mon.a.smithi005.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-12T02:48:47.388 INFO:journalctl@ceph.mon.a.smithi005.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-12T02:48:47.388 INFO:journalctl@ceph.mon.a.smithi005.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-12T02:48:47.388 INFO:journalctl@ceph.mon.a.smithi005.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-12T02:48:47.388 INFO:journalctl@ceph.mon.a.smithi005.stdout: Cumulative compaction: 0.00 GB write, 28.21 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-12T02:48:47.388 INFO:journalctl@ceph.mon.a.smithi005.stdout: Interval compaction: 0.00 GB write, 28.21 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-12T02:48:47.388 INFO:journalctl@ceph.mon.a.smithi005.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-12T02:48:47.388 INFO:journalctl@ceph.mon.a.smithi005.stdout: Block cache BinnedLRUCache@0x55a0b17211f0#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-09-12T02:48:47.388 INFO:journalctl@ceph.mon.a.smithi005.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-09-12T02:48:47.388 INFO:journalctl@ceph.mon.a.smithi005.stdout: 2024-09-12T02:48:47.389 INFO:journalctl@ceph.mon.a.smithi005.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-12T02:48:47.389 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: starting mon.a rank 0 at public addrs [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] at bind addrs [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 618305f8-70b1-11ef-bcea-c7b262605968 2024-09-12T02:48:47.389 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: mon.a@-1(???) e1 preinit fsid 618305f8-70b1-11ef-bcea-c7b262605968 2024-09-12T02:48:47.389 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: mon.a@-1(???).mds e1 new map 2024-09-12T02:48:47.389 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: mon.a@-1(???).mds e1 print_map 2024-09-12T02:48:47.389 INFO:journalctl@ceph.mon.a.smithi005.stdout: e1 2024-09-12T02:48:47.389 INFO:journalctl@ceph.mon.a.smithi005.stdout: btime 2024-09-12T02:48:42:351994+0000 2024-09-12T02:48:47.389 INFO:journalctl@ceph.mon.a.smithi005.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-12T02:48:47.389 INFO:journalctl@ceph.mon.a.smithi005.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-12T02:48:47.389 INFO:journalctl@ceph.mon.a.smithi005.stdout: legacy client fscid: -1 2024-09-12T02:48:47.389 INFO:journalctl@ceph.mon.a.smithi005.stdout: 2024-09-12T02:48:47.389 INFO:journalctl@ceph.mon.a.smithi005.stdout: No filesystems configured 2024-09-12T02:48:47.389 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-12T02:48:47.390 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-12T02:48:47.390 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-12T02:48:47.390 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-12T02:48:47.390 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-12T02:48:47.390 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 bash[31178]: 62861099a2acfdd3da3b829c4cbab84623c36b01d03f445532a93ebf5c6735dd 2024-09-12T02:48:47.390 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 systemd[1]: Started Ceph mon.a for 618305f8-70b1-11ef-bcea-c7b262605968. 2024-09-12T02:48:47.390 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-12T02:48:47.390 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: monmap epoch 1 2024-09-12T02:48:47.390 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: fsid 618305f8-70b1-11ef-bcea-c7b262605968 2024-09-12T02:48:47.390 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: last_changed 2024-09-12T02:48:39.314065+0000 2024-09-12T02:48:47.390 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: created 2024-09-12T02:48:39.314065+0000 2024-09-12T02:48:47.390 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: min_mon_release 19 (squid) 2024-09-12T02:48:47.390 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: election_strategy: 1 2024-09-12T02:48:47.391 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: 0: [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] mon.a 2024-09-12T02:48:47.391 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: fsmap 2024-09-12T02:48:47.391 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: osdmap e1: 0 total, 0 up, 0 in 2024-09-12T02:48:47.391 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:47 smithi005 ceph-mon[31197]: mgrmap e1: no daemons active 2024-09-12T02:48:48.132 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:47.534+0000 7fb4cc13c640 1 Processor -- start 2024-09-12T02:48:48.137 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:47.535+0000 7fb4cc13c640 1 -- start start 2024-09-12T02:48:48.137 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:47.535+0000 7fb4cc13c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb4c4107780 0x7fb4c4107b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:48.137 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:47.535+0000 7fb4cc13c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb4c4108150 con 0x7fb4c4107780 2024-09-12T02:48:48.137 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:47.535+0000 7fb4c9ed8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb4c4107780 0x7fb4c4107b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:48.137 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:47.535+0000 7fb4c9ed8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb4c4107780 0x7fb4c4107b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52812/0 (socket says 172.21.15.5:52812) 2024-09-12T02:48:48.137 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:47.535+0000 7fb4c9ed8640 1 -- 172.21.15.5:0/3663106830 learned_addr learned my addr 172.21.15.5:0/3663106830 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:48:48.137 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:47.536+0000 7fb4c9ed8640 1 -- 172.21.15.5:0/3663106830 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4c4108980 con 0x7fb4c4107780 2024-09-12T02:48:48.137 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:47.537+0000 7fb4c9ed8640 1 --2- 172.21.15.5:0/3663106830 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb4c4107780 0x7fb4c4107b80 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fb4c0009820 tx=0x7fb4c002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d08cc902539d52d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:48:48.137 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:47.538+0000 7fb4c8ed6640 1 -- 172.21.15.5:0/3663106830 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb4c0037070 con 0x7fb4c4107780 2024-09-12T02:48:48.138 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:47.538+0000 7fb4c8ed6640 1 -- 172.21.15.5:0/3663106830 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb4c0032900 con 0x7fb4c4107780 2024-09-12T02:48:48.138 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:47.538+0000 7fb4c8ed6640 1 -- 172.21.15.5:0/3663106830 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb4c0030850 con 0x7fb4c4107780 2024-09-12T02:48:48.138 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:47.538+0000 7fb4cc13c640 1 -- 172.21.15.5:0/3663106830 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb4c4107780 msgr2=0x7fb4c4107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:48.138 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:47.538+0000 7fb4cc13c640 1 --2- 172.21.15.5:0/3663106830 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb4c4107780 0x7fb4c4107b80 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fb4c0009820 tx=0x7fb4c002a270 comp rx=0 tx=0).stop 2024-09-12T02:48:48.138 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:47.538+0000 7fb4cc13c640 1 -- 172.21.15.5:0/3663106830 shutdown_connections 2024-09-12T02:48:48.138 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:47.538+0000 7fb4cc13c640 1 --2- 172.21.15.5:0/3663106830 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb4c4107780 0x7fb4c4107b80 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:48.138 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:47.538+0000 7fb4cc13c640 1 -- 172.21.15.5:0/3663106830 >> 172.21.15.5:0/3663106830 conn(0x7fb4c4102f30 msgr2=0x7fb4c4105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:48:48.138 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:47.538+0000 7fb4cc13c640 1 -- 172.21.15.5:0/3663106830 shutdown_connections 2024-09-12T02:48:48.138 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:47.538+0000 7fb4cc13c640 1 -- 172.21.15.5:0/3663106830 wait complete. 2024-09-12T02:48:48.138 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:47.539+0000 7fb4cc13c640 1 Processor -- start 2024-09-12T02:48:48.138 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:47.539+0000 7fb4cc13c640 1 -- start start 2024-09-12T02:48:48.138 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:47.539+0000 7fb4cc13c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb4c4107780 0x7fb4c407daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:48.138 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:47.539+0000 7fb4cc13c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb4c410bd30 con 0x7fb4c4107780 2024-09-12T02:48:48.139 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:47.539+0000 7fb4c9ed8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb4c4107780 0x7fb4c407daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:48.139 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:47.539+0000 7fb4c9ed8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb4c4107780 0x7fb4c407daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52818/0 (socket says 172.21.15.5:52818) 2024-09-12T02:48:48.139 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:47.539+0000 7fb4c9ed8640 1 -- 172.21.15.5:0/489066968 learned_addr learned my addr 172.21.15.5:0/489066968 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:48:48.139 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:47.540+0000 7fb4c9ed8640 1 -- 172.21.15.5:0/489066968 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4c407e030 con 0x7fb4c4107780 2024-09-12T02:48:48.139 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:47.540+0000 7fb4c9ed8640 1 --2- 172.21.15.5:0/489066968 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb4c4107780 0x7fb4c407daf0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fb4c0009940 tx=0x7fb4c0032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:48:48.139 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:47.540+0000 7fb4baffd640 1 -- 172.21.15.5:0/489066968 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb4c0042070 con 0x7fb4c4107780 2024-09-12T02:48:48.139 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:47.540+0000 7fb4baffd640 1 -- 172.21.15.5:0/489066968 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fb4c00308c0 con 0x7fb4c4107780 2024-09-12T02:48:48.139 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:47.540+0000 7fb4baffd640 1 -- 172.21.15.5:0/489066968 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb4c0037040 con 0x7fb4c4107780 2024-09-12T02:48:48.139 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:47.540+0000 7fb4cc13c640 1 -- 172.21.15.5:0/489066968 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4c407c220 con 0x7fb4c4107780 2024-09-12T02:48:48.139 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:47.540+0000 7fb4cc13c640 1 -- 172.21.15.5:0/489066968 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4c407c760 con 0x7fb4c4107780 2024-09-12T02:48:48.139 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:47.540+0000 7fb4baffd640 1 -- 172.21.15.5:0/489066968 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb4c0045070 con 0x7fb4c4107780 2024-09-12T02:48:48.139 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:47.540+0000 7fb4baffd640 1 -- 172.21.15.5:0/489066968 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb4c0047bc0 con 0x7fb4c4107780 2024-09-12T02:48:48.139 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:47.541+0000 7fb4cc13c640 1 -- 172.21.15.5:0/489066968 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4c4107c00 con 0x7fb4c4107780 2024-09-12T02:48:48.140 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:47.543+0000 7fb4baffd640 1 -- 172.21.15.5:0/489066968 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fb4c00327b0 con 0x7fb4c4107780 2024-09-12T02:48:48.140 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:47.603+0000 7fb4cc13c640 1 -- 172.21.15.5:0/489066968 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7fb4c407ca60 con 0x7fb4c4107780 2024-09-12T02:48:48.140 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:47.604+0000 7fb4baffd640 1 -- 172.21.15.5:0/489066968 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7fb4c003c870 con 0x7fb4c4107780 2024-09-12T02:48:48.140 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:47.605+0000 7fb4cc13c640 1 -- 172.21.15.5:0/489066968 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb4c4107780 msgr2=0x7fb4c407daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:48.140 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:47.605+0000 7fb4cc13c640 1 --2- 172.21.15.5:0/489066968 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb4c4107780 0x7fb4c407daf0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fb4c0009940 tx=0x7fb4c0032ce0 comp rx=0 tx=0).stop 2024-09-12T02:48:48.140 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:47.605+0000 7fb4cc13c640 1 -- 172.21.15.5:0/489066968 shutdown_connections 2024-09-12T02:48:48.140 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:47.605+0000 7fb4cc13c640 1 --2- 172.21.15.5:0/489066968 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb4c4107780 0x7fb4c407daf0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:48.140 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:47.605+0000 7fb4cc13c640 1 -- 172.21.15.5:0/489066968 >> 172.21.15.5:0/489066968 conn(0x7fb4c4102f30 msgr2=0x7fb4c4103aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:48:48.140 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:47.605+0000 7fb4cc13c640 1 -- 172.21.15.5:0/489066968 shutdown_connections 2024-09-12T02:48:48.140 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:47.605+0000 7fb4cc13c640 1 -- 172.21.15.5:0/489066968 wait complete. 2024-09-12T02:48:48.140 INFO:teuthology.orchestra.run.smithi005.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-12T02:48:48.154 INFO:teuthology.orchestra.run.smithi005.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-12T02:48:48.154 INFO:teuthology.orchestra.run.smithi005.stdout:Creating mgr... 2024-09-12T02:48:48.154 INFO:teuthology.orchestra.run.smithi005.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-12T02:48:48.155 INFO:teuthology.orchestra.run.smithi005.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-12T02:48:48.394 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-618305f8-70b1-11ef-bcea-c7b262605968@mgr.x 2024-09-12T02:48:48.395 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stderr Failed to reset failed state of unit ceph-618305f8-70b1-11ef-bcea-c7b262605968@mgr.x.service: Unit ceph-618305f8-70b1-11ef-bcea-c7b262605968@mgr.x.service not loaded. 2024-09-12T02:48:48.539 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-618305f8-70b1-11ef-bcea-c7b262605968.target.wants/ceph-618305f8-70b1-11ef-bcea-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-618305f8-70b1-11ef-bcea-c7b262605968@.service. 2024-09-12T02:48:48.765 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:48 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/489066968' entity='client.admin' 2024-09-12T02:48:49.034 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:48:48 smithi005 podman[31408]: 2024-09-12 02:48:48.962332091 +0000 UTC m=+0.298566593 container start 929a8c33032e68c39cb5f45436bea78f8b9569c82e7e400dbd5f966b6c9228ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T02:48:49.034 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:48:49 smithi005 bash[31408]: 929a8c33032e68c39cb5f45436bea78f8b9569c82e7e400dbd5f966b6c9228ad 2024-09-12T02:48:49.049 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-12T02:48:49.049 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout disabled 2024-09-12T02:48:49.057 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-12T02:48:49.057 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout inactive 2024-09-12T02:48:49.057 INFO:teuthology.orchestra.run.smithi005.stdout:firewalld.service is not enabled 2024-09-12T02:48:49.057 INFO:teuthology.orchestra.run.smithi005.stdout:Not possible to enable service . firewalld.service is not available 2024-09-12T02:48:49.066 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-12T02:48:49.067 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout disabled 2024-09-12T02:48:49.074 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-12T02:48:49.074 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout inactive 2024-09-12T02:48:49.074 INFO:teuthology.orchestra.run.smithi005.stdout:firewalld.service is not enabled 2024-09-12T02:48:49.074 INFO:teuthology.orchestra.run.smithi005.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-12T02:48:49.075 INFO:teuthology.orchestra.run.smithi005.stdout:Waiting for mgr to start... 2024-09-12T02:48:49.075 INFO:teuthology.orchestra.run.smithi005.stdout:Waiting for mgr... 2024-09-12T02:48:49.285 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:48:49 smithi005 systemd[1]: Started Ceph mgr.x for 618305f8-70b1-11ef-bcea-c7b262605968. 2024-09-12T02:48:49.286 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:48:49 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: 2024-09-12T02:48:49.088+0000 7f8b9aacc040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-12T02:48:49.286 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:48:49 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: 2024-09-12T02:48:49.154+0000 7f8b9aacc040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-12T02:48:49.543 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:48:49 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: 2024-09-12T02:48:49.314+0000 7f8b9aacc040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-12T02:48:49.543 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:48:49 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: 2024-09-12T02:48:49.383+0000 7f8b9aacc040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-12T02:48:49.543 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:48:49 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: 2024-09-12T02:48:49.542+0000 7f8b9aacc040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-12T02:48:49.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:49 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/3434680673' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-12T02:48:49.810 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:48:49 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: 2024-09-12T02:48:49.693+0000 7f8b9aacc040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-12T02:48:50.479 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:48:50 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: 2024-09-12T02:48:50.221+0000 7f8b9aacc040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-12T02:48:50.479 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:48:50 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: 2024-09-12T02:48:50.284+0000 7f8b9aacc040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-12T02:48:50.479 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:48:50 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: 2024-09-12T02:48:50.478+0000 7f8b9aacc040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-12T02:48:51.810 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:48:51 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: 2024-09-12T02:48:51.408+0000 7f8b9aacc040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-12T02:48:51.810 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:48:51 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: 2024-09-12T02:48:51.479+0000 7f8b9aacc040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-12T02:48:52.506 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:48:52 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: 2024-09-12T02:48:52.236+0000 7f8b9aacc040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-12T02:48:52.507 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:48:52 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: 2024-09-12T02:48:52.356+0000 7f8b9aacc040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-12T02:48:52.798 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:48:52 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: 2024-09-12T02:48:52.506+0000 7f8b9aacc040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-12T02:48:52.798 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:48:52 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: 2024-09-12T02:48:52.570+0000 7f8b9aacc040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-12T02:48:52.799 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:48:52 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: 2024-09-12T02:48:52.726+0000 7f8b9aacc040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-12T02:48:52.799 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:48:52 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: 2024-09-12T02:48:52.798+0000 7f8b9aacc040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-12T02:48:52.873 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2024-09-12T02:48:52.874 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout { 2024-09-12T02:48:52.874 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "fsid": "618305f8-70b1-11ef-bcea-c7b262605968", 2024-09-12T02:48:52.874 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "health": { 2024-09-12T02:48:52.874 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-12T02:48:52.874 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-12T02:48:52.877 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-12T02:48:52.877 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-09-12T02:48:52.877 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-12T02:48:52.877 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-12T02:48:52.877 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 0 2024-09-12T02:48:52.877 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout ], 2024-09-12T02:48:52.877 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-12T02:48:52.877 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "a" 2024-09-12T02:48:52.877 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout ], 2024-09-12T02:48:52.877 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-09-12T02:48:52.877 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-12T02:48:52.878 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T02:48:52.878 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-12T02:48:52.878 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-12T02:48:52.878 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-09-12T02:48:52.878 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-12T02:48:52.878 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T02:48:52.878 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-12T02:48:52.878 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-12T02:48:52.878 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-12T02:48:52.878 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-12T02:48:52.878 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-12T02:48:52.878 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-12T02:48:52.878 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-09-12T02:48:52.878 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-12T02:48:52.879 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-12T02:48:52.879 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-12T02:48:52.879 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-12T02:48:52.879 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-12T02:48:52.879 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-12T02:48:52.879 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-12T02:48:52.879 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-12T02:48:52.879 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-12T02:48:52.879 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-09-12T02:48:52.879 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-12T02:48:52.879 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T02:48:52.879 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "btime": "2024-09-12T02:48:42:351994+0000", 2024-09-12T02:48:52.879 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-12T02:48:52.879 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-12T02:48:52.880 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-09-12T02:48:52.880 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-12T02:48:52.880 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-12T02:48:52.880 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-12T02:48:52.880 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-12T02:48:52.880 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-12T02:48:52.880 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-12T02:48:52.880 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "restful" 2024-09-12T02:48:52.880 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout ], 2024-09-12T02:48:52.880 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-12T02:48:52.880 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-09-12T02:48:52.880 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-12T02:48:52.880 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T02:48:52.881 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "modified": "2024-09-12T02:48:42.352613+0000", 2024-09-12T02:48:52.881 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-12T02:48:52.881 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-09-12T02:48:52.881 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-12T02:48:52.881 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout } 2024-09-12T02:48:52.881 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:49.477+0000 7fd00a2ff640 1 Processor -- start 2024-09-12T02:48:52.881 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:49.477+0000 7fd00a2ff640 1 -- start start 2024-09-12T02:48:52.881 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:49.477+0000 7fd00a2ff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0041065d0 0x7fd0041069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:52.881 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:49.477+0000 7fd00a2ff640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd004106fa0 con 0x7fd0041065d0 2024-09-12T02:48:52.881 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:49.477+0000 7fd003fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0041065d0 0x7fd0041069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:52.881 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:49.477+0000 7fd003fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0041065d0 0x7fd0041069d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52848/0 (socket says 172.21.15.5:52848) 2024-09-12T02:48:52.881 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:49.477+0000 7fd003fff640 1 -- 172.21.15.5:0/2492058515 learned_addr learned my addr 172.21.15.5:0/2492058515 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:48:52.881 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:49.478+0000 7fd003fff640 1 -- 172.21.15.5:0/2492058515 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0041077d0 con 0x7fd0041065d0 2024-09-12T02:48:52.882 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:49.479+0000 7fd003fff640 1 --2- 172.21.15.5:0/2492058515 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0041065d0 0x7fd0041069d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fcff4009820 tx=0x7fcff402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be7e337524d2d10e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:48:52.882 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:49.479+0000 7fd002ffd640 1 -- 172.21.15.5:0/2492058515 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcff4037070 con 0x7fd0041065d0 2024-09-12T02:48:52.882 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:49.479+0000 7fd002ffd640 1 -- 172.21.15.5:0/2492058515 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fcff4032900 con 0x7fd0041065d0 2024-09-12T02:48:52.882 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:49.479+0000 7fd002ffd640 1 -- 172.21.15.5:0/2492058515 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcff4030850 con 0x7fd0041065d0 2024-09-12T02:48:52.882 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:49.480+0000 7fd00a2ff640 1 -- 172.21.15.5:0/2492058515 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0041065d0 msgr2=0x7fd0041069d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:52.882 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:49.480+0000 7fd00a2ff640 1 --2- 172.21.15.5:0/2492058515 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0041065d0 0x7fd0041069d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fcff4009820 tx=0x7fcff402a270 comp rx=0 tx=0).stop 2024-09-12T02:48:52.882 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:49.480+0000 7fd00a2ff640 1 -- 172.21.15.5:0/2492058515 shutdown_connections 2024-09-12T02:48:52.882 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:49.480+0000 7fd00a2ff640 1 --2- 172.21.15.5:0/2492058515 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0041065d0 0x7fd0041069d0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:52.882 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:49.480+0000 7fd00a2ff640 1 -- 172.21.15.5:0/2492058515 >> 172.21.15.5:0/2492058515 conn(0x7fd004101e20 msgr2=0x7fd004104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:48:52.882 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:49.480+0000 7fd00a2ff640 1 -- 172.21.15.5:0/2492058515 shutdown_connections 2024-09-12T02:48:52.882 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:49.480+0000 7fd00a2ff640 1 -- 172.21.15.5:0/2492058515 wait complete. 2024-09-12T02:48:52.882 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:49.481+0000 7fd00a2ff640 1 Processor -- start 2024-09-12T02:48:52.882 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:49.481+0000 7fd00a2ff640 1 -- start start 2024-09-12T02:48:52.882 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:49.481+0000 7fd00a2ff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0041065d0 0x7fd004191800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:52.883 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:49.481+0000 7fd00a2ff640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd00410ab80 con 0x7fd0041065d0 2024-09-12T02:48:52.883 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:49.481+0000 7fd003fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0041065d0 0x7fd004191800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:52.883 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:49.481+0000 7fd003fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0041065d0 0x7fd004191800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52864/0 (socket says 172.21.15.5:52864) 2024-09-12T02:48:52.883 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:49.481+0000 7fd003fff640 1 -- 172.21.15.5:0/3434680673 learned_addr learned my addr 172.21.15.5:0/3434680673 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:48:52.883 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:49.481+0000 7fd003fff640 1 -- 172.21.15.5:0/3434680673 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd004191d40 con 0x7fd0041065d0 2024-09-12T02:48:52.884 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:49.482+0000 7fd003fff640 1 --2- 172.21.15.5:0/3434680673 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0041065d0 0x7fd004191800 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fcff4009940 tx=0x7fcff4032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:48:52.884 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:49.482+0000 7fd0017fa640 1 -- 172.21.15.5:0/3434680673 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcff4042070 con 0x7fd0041065d0 2024-09-12T02:48:52.884 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:49.482+0000 7fd0017fa640 1 -- 172.21.15.5:0/3434680673 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fcff40308c0 con 0x7fd0041065d0 2024-09-12T02:48:52.884 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:49.482+0000 7fd00a2ff640 1 -- 172.21.15.5:0/3434680673 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd00418ff30 con 0x7fd0041065d0 2024-09-12T02:48:52.884 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:49.482+0000 7fd0017fa640 1 -- 172.21.15.5:0/3434680673 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcff4037040 con 0x7fd0041065d0 2024-09-12T02:48:52.884 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:49.482+0000 7fd00a2ff640 1 -- 172.21.15.5:0/3434680673 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd004190470 con 0x7fd0041065d0 2024-09-12T02:48:52.884 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:49.482+0000 7fd0017fa640 1 -- 172.21.15.5:0/3434680673 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fcff4045070 con 0x7fd0041065d0 2024-09-12T02:48:52.885 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:49.482+0000 7fd0017fa640 1 -- 172.21.15.5:0/3434680673 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fcff4047bc0 con 0x7fd0041065d0 2024-09-12T02:48:52.885 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:49.483+0000 7fd00a2ff640 1 -- 172.21.15.5:0/3434680673 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd00410ad00 con 0x7fd0041065d0 2024-09-12T02:48:52.885 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:49.485+0000 7fd0017fa640 1 -- 172.21.15.5:0/3434680673 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fcff40327b0 con 0x7fd0041065d0 2024-09-12T02:48:52.885 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:49.548+0000 7fd00a2ff640 1 -- 172.21.15.5:0/3434680673 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fd004195490 con 0x7fd0041065d0 2024-09-12T02:48:52.885 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:49.549+0000 7fd0017fa640 1 -- 172.21.15.5:0/3434680673 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7fcff403c870 con 0x7fd0041065d0 2024-09-12T02:48:52.886 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:49.549+0000 7fd00a2ff640 1 -- 172.21.15.5:0/3434680673 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0041065d0 msgr2=0x7fd004191800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:52.886 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:49.549+0000 7fd00a2ff640 1 --2- 172.21.15.5:0/3434680673 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0041065d0 0x7fd004191800 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fcff4009940 tx=0x7fcff4032ce0 comp rx=0 tx=0).stop 2024-09-12T02:48:52.886 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:49.550+0000 7fd00a2ff640 1 -- 172.21.15.5:0/3434680673 shutdown_connections 2024-09-12T02:48:52.886 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:49.550+0000 7fd00a2ff640 1 --2- 172.21.15.5:0/3434680673 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd0041065d0 0x7fd004191800 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:52.887 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:49.550+0000 7fd00a2ff640 1 -- 172.21.15.5:0/3434680673 >> 172.21.15.5:0/3434680673 conn(0x7fd004101e20 msgr2=0x7fd004102d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:48:52.887 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:49.550+0000 7fd00a2ff640 1 -- 172.21.15.5:0/3434680673 shutdown_connections 2024-09-12T02:48:52.887 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:49.550+0000 7fd00a2ff640 1 -- 172.21.15.5:0/3434680673 wait complete. 2024-09-12T02:48:52.887 INFO:teuthology.orchestra.run.smithi005.stdout:mgr not available, waiting (1/15)... 2024-09-12T02:48:53.060 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:48:52 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: 2024-09-12T02:48:52.861+0000 7f8b9aacc040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-12T02:48:53.060 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:48:53 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: 2024-09-12T02:48:53.028+0000 7f8b9aacc040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-12T02:48:53.399 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:48:53 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: 2024-09-12T02:48:53.100+0000 7f8b9aacc040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-12T02:48:53.399 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:48:53 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: 2024-09-12T02:48:53.174+0000 7f8b9aacc040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-12T02:48:53.399 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:48:53 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: 2024-09-12T02:48:53.268+0000 7f8b9aacc040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-12T02:48:53.399 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:48:53 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: 2024-09-12T02:48:53.335+0000 7f8b9aacc040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-12T02:48:53.400 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:48:53 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: 2024-09-12T02:48:53.398+0000 7f8b9aacc040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-12T02:48:54.560 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:48:54 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: 2024-09-12T02:48:54.232+0000 7f8b9aacc040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-12T02:48:54.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:54 smithi005 ceph-mon[31197]: Activating manager daemon x 2024-09-12T02:48:54.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:54 smithi005 ceph-mon[31197]: mgrmap e2: x(active, starting, since 0.0030373s) 2024-09-12T02:48:54.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:54 smithi005 ceph-mon[31197]: from='mgr.14100 172.21.15.5:0/2143088274' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-12T02:48:54.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:54 smithi005 ceph-mon[31197]: from='mgr.14100 172.21.15.5:0/2143088274' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-12T02:48:54.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:54 smithi005 ceph-mon[31197]: from='mgr.14100 172.21.15.5:0/2143088274' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-12T02:48:54.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:54 smithi005 ceph-mon[31197]: from='mgr.14100 172.21.15.5:0/2143088274' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T02:48:54.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:54 smithi005 ceph-mon[31197]: from='mgr.14100 172.21.15.5:0/2143088274' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-12T02:48:54.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:54 smithi005 ceph-mon[31197]: Manager daemon x is now available 2024-09-12T02:48:54.562 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:54 smithi005 ceph-mon[31197]: from='mgr.14100 172.21.15.5:0/2143088274' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-12T02:48:54.562 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:54 smithi005 ceph-mon[31197]: from='mgr.14100 172.21.15.5:0/2143088274' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-12T02:48:54.562 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:54 smithi005 ceph-mon[31197]: from='mgr.14100 172.21.15.5:0/2143088274' entity='mgr.x' 2024-09-12T02:48:54.562 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:54 smithi005 ceph-mon[31197]: from='mgr.14100 172.21.15.5:0/2143088274' entity='mgr.x' 2024-09-12T02:48:54.562 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:54 smithi005 ceph-mon[31197]: from='mgr.14100 172.21.15.5:0/2143088274' entity='mgr.x' 2024-09-12T02:48:56.070 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2024-09-12T02:48:56.070 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout { 2024-09-12T02:48:56.070 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "fsid": "618305f8-70b1-11ef-bcea-c7b262605968", 2024-09-12T02:48:56.070 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "health": { 2024-09-12T02:48:56.071 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-12T02:48:56.071 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-12T02:48:56.071 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-12T02:48:56.071 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-09-12T02:48:56.071 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-12T02:48:56.071 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-12T02:48:56.071 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 0 2024-09-12T02:48:56.071 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout ], 2024-09-12T02:48:56.071 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-12T02:48:56.071 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "a" 2024-09-12T02:48:56.071 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout ], 2024-09-12T02:48:56.071 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-09-12T02:48:56.071 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-12T02:48:56.072 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T02:48:56.072 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-12T02:48:56.072 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-12T02:48:56.072 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-09-12T02:48:56.072 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-12T02:48:56.072 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T02:48:56.072 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-12T02:48:56.072 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-12T02:48:56.072 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-12T02:48:56.072 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-12T02:48:56.072 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-12T02:48:56.072 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-12T02:48:56.073 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-09-12T02:48:56.073 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-12T02:48:56.073 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-12T02:48:56.073 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-12T02:48:56.073 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-12T02:48:56.073 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-12T02:48:56.073 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-12T02:48:56.073 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-12T02:48:56.073 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-12T02:48:56.073 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-12T02:48:56.073 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-09-12T02:48:56.073 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-12T02:48:56.073 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T02:48:56.074 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "btime": "2024-09-12T02:48:42:351994+0000", 2024-09-12T02:48:56.074 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-12T02:48:56.074 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-12T02:48:56.074 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-09-12T02:48:56.074 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-12T02:48:56.074 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-12T02:48:56.074 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-12T02:48:56.074 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-12T02:48:56.074 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-12T02:48:56.075 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-12T02:48:56.075 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "restful" 2024-09-12T02:48:56.075 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout ], 2024-09-12T02:48:56.075 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-12T02:48:56.075 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-09-12T02:48:56.075 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-12T02:48:56.075 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-12T02:48:56.075 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "modified": "2024-09-12T02:48:42.352613+0000", 2024-09-12T02:48:56.075 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-12T02:48:56.075 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2024-09-12T02:48:56.075 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-12T02:48:56.075 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout } 2024-09-12T02:48:56.075 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:55.281+0000 7f753f7bc640 1 Processor -- start 2024-09-12T02:48:56.075 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:55.282+0000 7f753f7bc640 1 -- start start 2024-09-12T02:48:56.076 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:55.282+0000 7f753f7bc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7538107700 0x7f7538107b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:56.076 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:55.282+0000 7f753f7bc640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f75381080d0 con 0x7f7538107700 2024-09-12T02:48:56.076 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:55.282+0000 7f753d558640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7538107700 0x7f7538107b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:56.076 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:55.282+0000 7f753d558640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7538107700 0x7f7538107b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59104/0 (socket says 172.21.15.5:59104) 2024-09-12T02:48:56.076 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:55.282+0000 7f753d558640 1 -- 172.21.15.5:0/2066002896 learned_addr learned my addr 172.21.15.5:0/2066002896 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:48:56.076 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:55.282+0000 7f753d558640 1 -- 172.21.15.5:0/2066002896 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7538108900 con 0x7f7538107700 2024-09-12T02:48:56.076 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:55.283+0000 7f753d558640 1 --2- 172.21.15.5:0/2066002896 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7538107700 0x7f7538107b00 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f7534009e80 tx=0x7f753402a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d58c8a54a56be09 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:48:56.076 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:55.283+0000 7f752ffff640 1 -- 172.21.15.5:0/2066002896 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7534036070 con 0x7f7538107700 2024-09-12T02:48:56.077 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:55.283+0000 7f752ffff640 1 -- 172.21.15.5:0/2066002896 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f75340318c0 con 0x7f7538107700 2024-09-12T02:48:56.077 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:55.283+0000 7f752ffff640 1 -- 172.21.15.5:0/2066002896 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7534007ab0 con 0x7f7538107700 2024-09-12T02:48:56.077 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:55.284+0000 7f753f7bc640 1 -- 172.21.15.5:0/2066002896 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7538107700 msgr2=0x7f7538107b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:56.077 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:55.284+0000 7f753f7bc640 1 --2- 172.21.15.5:0/2066002896 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7538107700 0x7f7538107b00 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f7534009e80 tx=0x7f753402a7a0 comp rx=0 tx=0).stop 2024-09-12T02:48:56.077 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:55.284+0000 7f753f7bc640 1 -- 172.21.15.5:0/2066002896 shutdown_connections 2024-09-12T02:48:56.077 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:55.284+0000 7f753f7bc640 1 --2- 172.21.15.5:0/2066002896 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7538107700 0x7f7538107b00 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:56.077 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:55.284+0000 7f753f7bc640 1 -- 172.21.15.5:0/2066002896 >> 172.21.15.5:0/2066002896 conn(0x7f7538102f30 msgr2=0x7f7538105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:48:56.077 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:55.284+0000 7f753f7bc640 1 -- 172.21.15.5:0/2066002896 shutdown_connections 2024-09-12T02:48:56.077 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:55.284+0000 7f753f7bc640 1 -- 172.21.15.5:0/2066002896 wait complete. 2024-09-12T02:48:56.077 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:55.285+0000 7f753f7bc640 1 Processor -- start 2024-09-12T02:48:56.077 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:55.285+0000 7f753f7bc640 1 -- start start 2024-09-12T02:48:56.077 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:55.285+0000 7f753f7bc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7538107700 0x7f753807daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:56.077 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:55.285+0000 7f753f7bc640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f753810bcb0 con 0x7f7538107700 2024-09-12T02:48:56.078 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:55.285+0000 7f753d558640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7538107700 0x7f753807daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:56.078 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:55.285+0000 7f753d558640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7538107700 0x7f753807daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59118/0 (socket says 172.21.15.5:59118) 2024-09-12T02:48:56.078 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:55.285+0000 7f753d558640 1 -- 172.21.15.5:0/2812080905 learned_addr learned my addr 172.21.15.5:0/2812080905 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:48:56.078 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:55.285+0000 7f753d558640 1 -- 172.21.15.5:0/2812080905 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f753807e030 con 0x7f7538107700 2024-09-12T02:48:56.078 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:55.286+0000 7f753d558640 1 --2- 172.21.15.5:0/2812080905 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7538107700 0x7f753807daf0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f753402acc0 tx=0x7f7534031ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:48:56.078 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:55.286+0000 7f752e7fc640 1 -- 172.21.15.5:0/2812080905 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7534036050 con 0x7f7538107700 2024-09-12T02:48:56.078 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:55.286+0000 7f752e7fc640 1 -- 172.21.15.5:0/2812080905 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7534037040 con 0x7f7538107700 2024-09-12T02:48:56.078 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:55.286+0000 7f753f7bc640 1 -- 172.21.15.5:0/2812080905 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f753807c220 con 0x7f7538107700 2024-09-12T02:48:56.078 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:55.286+0000 7f752e7fc640 1 -- 172.21.15.5:0/2812080905 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f753403d8c0 con 0x7f7538107700 2024-09-12T02:48:56.078 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:55.286+0000 7f753f7bc640 1 -- 172.21.15.5:0/2812080905 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f753807c760 con 0x7f7538107700 2024-09-12T02:48:56.078 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:55.287+0000 7f752e7fc640 1 -- 172.21.15.5:0/2812080905 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 3) ==== 46170+0+0 (secure 0 0 0) 0x7f7534044020 con 0x7f7538107700 2024-09-12T02:48:56.078 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:55.287+0000 7f753f7bc640 1 -- 172.21.15.5:0/2812080905 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7538107b80 con 0x7f7538107700 2024-09-12T02:48:56.078 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:55.287+0000 7f752e7fc640 1 --2- 172.21.15.5:0/2812080905 >> [v2:172.21.15.5:6800/2595781680,v1:172.21.15.5:6801/2595781680] conn(0x7f7524038e60 0x7f752403b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:56.079 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:55.287+0000 7f752e7fc640 1 -- 172.21.15.5:0/2812080905 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f7534042020 con 0x7f7538107700 2024-09-12T02:48:56.079 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:55.287+0000 7f753cd57640 1 --2- 172.21.15.5:0/2812080905 >> [v2:172.21.15.5:6800/2595781680,v1:172.21.15.5:6801/2595781680] conn(0x7f7524038e60 0x7f752403b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:56.079 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:55.288+0000 7f753cd57640 1 --2- 172.21.15.5:0/2812080905 >> [v2:172.21.15.5:6800/2595781680,v1:172.21.15.5:6801/2595781680] conn(0x7f7524038e60 0x7f752403b320 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f75280098c0 tx=0x7f7528006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:48:56.079 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:55.292+0000 7f752e7fc640 1 -- 172.21.15.5:0/2812080905 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f753404b020 con 0x7f7538107700 2024-09-12T02:48:56.079 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:55.535+0000 7f753f7bc640 1 -- 172.21.15.5:0/2812080905 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f753807c9e0 con 0x7f7538107700 2024-09-12T02:48:56.079 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:55.536+0000 7f752e7fc640 1 -- 172.21.15.5:0/2812080905 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1290 (secure 0 0 0) 0x7f7534039300 con 0x7f7538107700 2024-09-12T02:48:56.079 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:55.538+0000 7f753f7bc640 1 -- 172.21.15.5:0/2812080905 >> [v2:172.21.15.5:6800/2595781680,v1:172.21.15.5:6801/2595781680] conn(0x7f7524038e60 msgr2=0x7f752403b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:56.079 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:55.538+0000 7f753f7bc640 1 --2- 172.21.15.5:0/2812080905 >> [v2:172.21.15.5:6800/2595781680,v1:172.21.15.5:6801/2595781680] conn(0x7f7524038e60 0x7f752403b320 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f75280098c0 tx=0x7f7528006db0 comp rx=0 tx=0).stop 2024-09-12T02:48:56.079 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:55.538+0000 7f753f7bc640 1 -- 172.21.15.5:0/2812080905 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7538107700 msgr2=0x7f753807daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:56.079 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:55.538+0000 7f753f7bc640 1 --2- 172.21.15.5:0/2812080905 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7538107700 0x7f753807daf0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f753402acc0 tx=0x7f7534031ca0 comp rx=0 tx=0).stop 2024-09-12T02:48:56.079 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:55.538+0000 7f753f7bc640 1 -- 172.21.15.5:0/2812080905 shutdown_connections 2024-09-12T02:48:56.079 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:55.538+0000 7f753f7bc640 1 --2- 172.21.15.5:0/2812080905 >> [v2:172.21.15.5:6800/2595781680,v1:172.21.15.5:6801/2595781680] conn(0x7f7524038e60 0x7f752403b320 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:56.079 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:55.538+0000 7f753f7bc640 1 --2- 172.21.15.5:0/2812080905 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7538107700 0x7f753807daf0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:56.079 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:55.538+0000 7f753f7bc640 1 -- 172.21.15.5:0/2812080905 >> 172.21.15.5:0/2812080905 conn(0x7f7538102f30 msgr2=0x7f7538103a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:48:56.080 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:55.538+0000 7f753f7bc640 1 -- 172.21.15.5:0/2812080905 shutdown_connections 2024-09-12T02:48:56.080 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:55.538+0000 7f753f7bc640 1 -- 172.21.15.5:0/2812080905 wait complete. 2024-09-12T02:48:56.080 INFO:teuthology.orchestra.run.smithi005.stdout:mgr is available 2024-09-12T02:48:56.476 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:56 smithi005 ceph-mon[31197]: mgrmap e3: x(active, since 1.00722s) 2024-09-12T02:48:56.476 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:56 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/2812080905' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-12T02:48:57.289 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2024-09-12T02:48:57.289 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout [global] 2024-09-12T02:48:57.289 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout fsid = 618305f8-70b1-11ef-bcea-c7b262605968 2024-09-12T02:48:57.289 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.5:3300,v1:172.21.15.5:6789] 2024-09-12T02:48:57.289 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-12T02:48:57.289 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-12T02:48:57.289 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-12T02:48:57.289 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-12T02:48:57.290 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2024-09-12T02:48:57.290 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-12T02:48:57.290 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-12T02:48:57.290 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2024-09-12T02:48:57.290 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout [osd] 2024-09-12T02:48:57.290 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-12T02:48:57.290 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-12T02:48:57.290 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-12T02:48:57.290 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-12T02:48:57.291 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-12T02:48:57.291 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:56.483+0000 7fbc6075e640 1 Processor -- start 2024-09-12T02:48:57.291 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:56.483+0000 7fbc6075e640 1 -- start start 2024-09-12T02:48:57.291 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:56.484+0000 7fbc6075e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc581081f0 0x7fbc5810a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:57.291 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:56.484+0000 7fbc6075e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbc5807bdd0 con 0x7fbc581081f0 2024-09-12T02:48:57.291 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:56.484+0000 7fbc5e4fa640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc581081f0 0x7fbc5810a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:57.291 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:56.484+0000 7fbc5e4fa640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc581081f0 0x7fbc5810a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59134/0 (socket says 172.21.15.5:59134) 2024-09-12T02:48:57.291 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:56.484+0000 7fbc5e4fa640 1 -- 172.21.15.5:0/2843796380 learned_addr learned my addr 172.21.15.5:0/2843796380 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:48:57.291 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:56.484+0000 7fbc5e4fa640 1 -- 172.21.15.5:0/2843796380 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc5810ab20 con 0x7fbc581081f0 2024-09-12T02:48:57.291 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:56.485+0000 7fbc5e4fa640 1 --2- 172.21.15.5:0/2843796380 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc581081f0 0x7fbc5810a5e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fbc54009820 tx=0x7fbc5402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ae47366ad197b33 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:48:57.292 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:56.485+0000 7fbc5d4f8640 1 -- 172.21.15.5:0/2843796380 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc54037070 con 0x7fbc581081f0 2024-09-12T02:48:57.292 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:56.485+0000 7fbc5d4f8640 1 -- 172.21.15.5:0/2843796380 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fbc54032900 con 0x7fbc581081f0 2024-09-12T02:48:57.292 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:56.485+0000 7fbc5d4f8640 1 -- 172.21.15.5:0/2843796380 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc54030850 con 0x7fbc581081f0 2024-09-12T02:48:57.292 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:56.486+0000 7fbc6075e640 1 -- 172.21.15.5:0/2843796380 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc581081f0 msgr2=0x7fbc5810a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:57.292 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:56.486+0000 7fbc6075e640 1 --2- 172.21.15.5:0/2843796380 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc581081f0 0x7fbc5810a5e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fbc54009820 tx=0x7fbc5402a270 comp rx=0 tx=0).stop 2024-09-12T02:48:57.292 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:56.486+0000 7fbc6075e640 1 -- 172.21.15.5:0/2843796380 shutdown_connections 2024-09-12T02:48:57.292 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:56.486+0000 7fbc6075e640 1 --2- 172.21.15.5:0/2843796380 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc581081f0 0x7fbc5810a5e0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:57.292 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:56.486+0000 7fbc6075e640 1 -- 172.21.15.5:0/2843796380 >> 172.21.15.5:0/2843796380 conn(0x7fbc58101e70 msgr2=0x7fbc58104290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:48:57.293 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:56.486+0000 7fbc6075e640 1 -- 172.21.15.5:0/2843796380 shutdown_connections 2024-09-12T02:48:57.293 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:56.486+0000 7fbc6075e640 1 -- 172.21.15.5:0/2843796380 wait complete. 2024-09-12T02:48:57.293 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:56.487+0000 7fbc6075e640 1 Processor -- start 2024-09-12T02:48:57.293 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:56.487+0000 7fbc6075e640 1 -- start start 2024-09-12T02:48:57.293 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:56.487+0000 7fbc6075e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc581081f0 0x7fbc58107a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:57.293 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:56.487+0000 7fbc6075e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbc5810bb60 con 0x7fbc581081f0 2024-09-12T02:48:57.293 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:56.487+0000 7fbc5e4fa640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc581081f0 0x7fbc58107a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:57.293 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:56.487+0000 7fbc5e4fa640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc581081f0 0x7fbc58107a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59140/0 (socket says 172.21.15.5:59140) 2024-09-12T02:48:57.293 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:56.487+0000 7fbc5e4fa640 1 -- 172.21.15.5:0/3126217581 learned_addr learned my addr 172.21.15.5:0/3126217581 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:48:57.293 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:56.487+0000 7fbc5e4fa640 1 -- 172.21.15.5:0/3126217581 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc58107fb0 con 0x7fbc581081f0 2024-09-12T02:48:57.293 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:56.487+0000 7fbc5e4fa640 1 --2- 172.21.15.5:0/3126217581 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc581081f0 0x7fbc58107a70 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fbc54032ce0 tx=0x7fbc54032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:48:57.293 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:56.487+0000 7fbc4f7fe640 1 -- 172.21.15.5:0/3126217581 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc54042070 con 0x7fbc581081f0 2024-09-12T02:48:57.293 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:56.488+0000 7fbc6075e640 1 -- 172.21.15.5:0/3126217581 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc581061a0 con 0x7fbc581081f0 2024-09-12T02:48:57.294 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:56.488+0000 7fbc4f7fe640 1 -- 172.21.15.5:0/3126217581 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fbc540308c0 con 0x7fbc581081f0 2024-09-12T02:48:57.294 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:56.488+0000 7fbc6075e640 1 -- 172.21.15.5:0/3126217581 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc581066e0 con 0x7fbc581081f0 2024-09-12T02:48:57.294 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:56.488+0000 7fbc4f7fe640 1 -- 172.21.15.5:0/3126217581 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc54037040 con 0x7fbc581081f0 2024-09-12T02:48:57.294 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:56.489+0000 7fbc4f7fe640 1 -- 172.21.15.5:0/3126217581 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7fbc54045070 con 0x7fbc581081f0 2024-09-12T02:48:57.294 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:56.489+0000 7fbc4f7fe640 1 --2- 172.21.15.5:0/3126217581 >> [v2:172.21.15.5:6800/2595781680,v1:172.21.15.5:6801/2595781680] conn(0x7fbc44038f20 0x7fbc4403b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:57.294 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:56.489+0000 7fbc6075e640 1 -- 172.21.15.5:0/3126217581 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc58108650 con 0x7fbc581081f0 2024-09-12T02:48:57.294 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:56.489+0000 7fbc4f7fe640 1 -- 172.21.15.5:0/3126217581 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fbc5406c090 con 0x7fbc581081f0 2024-09-12T02:48:57.294 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:56.489+0000 7fbc5dcf9640 1 --2- 172.21.15.5:0/3126217581 >> [v2:172.21.15.5:6800/2595781680,v1:172.21.15.5:6801/2595781680] conn(0x7fbc44038f20 0x7fbc4403b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:57.294 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:56.489+0000 7fbc5dcf9640 1 --2- 172.21.15.5:0/3126217581 >> [v2:172.21.15.5:6800/2595781680,v1:172.21.15.5:6801/2595781680] conn(0x7fbc44038f20 0x7fbc4403b3e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbc480098c0 tx=0x7fbc48006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:48:57.294 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:56.494+0000 7fbc4f7fe640 1 -- 172.21.15.5:0/3126217581 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbc5403cd70 con 0x7fbc581081f0 2024-09-12T02:48:57.294 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:56.672+0000 7fbc6075e640 1 -- 172.21.15.5:0/3126217581 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fbc58106ef0 con 0x7fbc581081f0 2024-09-12T02:48:57.294 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:56.673+0000 7fbc4f7fe640 1 -- 172.21.15.5:0/3126217581 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+427 (secure 0 0 0) 0x7fbc5403c850 con 0x7fbc581081f0 2024-09-12T02:48:57.294 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:56.676+0000 7fbc6075e640 1 -- 172.21.15.5:0/3126217581 >> [v2:172.21.15.5:6800/2595781680,v1:172.21.15.5:6801/2595781680] conn(0x7fbc44038f20 msgr2=0x7fbc4403b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:57.295 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:56.676+0000 7fbc6075e640 1 --2- 172.21.15.5:0/3126217581 >> [v2:172.21.15.5:6800/2595781680,v1:172.21.15.5:6801/2595781680] conn(0x7fbc44038f20 0x7fbc4403b3e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbc480098c0 tx=0x7fbc48006db0 comp rx=0 tx=0).stop 2024-09-12T02:48:57.295 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:56.676+0000 7fbc6075e640 1 -- 172.21.15.5:0/3126217581 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc581081f0 msgr2=0x7fbc58107a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:57.295 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:56.676+0000 7fbc6075e640 1 --2- 172.21.15.5:0/3126217581 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc581081f0 0x7fbc58107a70 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fbc54032ce0 tx=0x7fbc54032d10 comp rx=0 tx=0).stop 2024-09-12T02:48:57.295 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:56.676+0000 7fbc6075e640 1 -- 172.21.15.5:0/3126217581 shutdown_connections 2024-09-12T02:48:57.295 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:56.676+0000 7fbc6075e640 1 --2- 172.21.15.5:0/3126217581 >> [v2:172.21.15.5:6800/2595781680,v1:172.21.15.5:6801/2595781680] conn(0x7fbc44038f20 0x7fbc4403b3e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:57.295 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:56.676+0000 7fbc6075e640 1 --2- 172.21.15.5:0/3126217581 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbc581081f0 0x7fbc58107a70 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:57.295 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:56.676+0000 7fbc6075e640 1 -- 172.21.15.5:0/3126217581 >> 172.21.15.5:0/3126217581 conn(0x7fbc58101e70 msgr2=0x7fbc58102a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:48:57.295 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:56.676+0000 7fbc6075e640 1 -- 172.21.15.5:0/3126217581 shutdown_connections 2024-09-12T02:48:57.295 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:56.676+0000 7fbc6075e640 1 -- 172.21.15.5:0/3126217581 wait complete. 2024-09-12T02:48:57.295 INFO:teuthology.orchestra.run.smithi005.stdout:Enabling cephadm module... 2024-09-12T02:48:57.515 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:57 smithi005 ceph-mon[31197]: mgrmap e4: x(active, since 2s) 2024-09-12T02:48:57.515 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:57 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/3126217581' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-12T02:48:58.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:58 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/1805376622' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-09-12T02:48:58.561 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:48:58 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: ignoring --setuser ceph since I am not root 2024-09-12T02:48:58.561 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:48:58 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: ignoring --setgroup ceph since I am not root 2024-09-12T02:48:58.561 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:48:58 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: 2024-09-12T02:48:58.387+0000 7f854a834040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-12T02:48:58.561 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:48:58 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: 2024-09-12T02:48:58.447+0000 7f854a834040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-12T02:48:58.827 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:48:58 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: 2024-09-12T02:48:58.604+0000 7f854a834040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-12T02:48:58.828 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:48:58 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: 2024-09-12T02:48:58.673+0000 7f854a834040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-12T02:48:58.828 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:48:58 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: 2024-09-12T02:48:58.827+0000 7f854a834040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-12T02:48:58.844 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:57.677+0000 7f4e58a8f640 1 Processor -- start 2024-09-12T02:48:58.845 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:57.677+0000 7f4e58a8f640 1 -- start start 2024-09-12T02:48:58.845 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:57.678+0000 7f4e58a8f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e541065d0 0x7f4e541069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:58.845 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:57.678+0000 7f4e58a8f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4e54106fa0 con 0x7f4e541065d0 2024-09-12T02:48:58.845 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:57.678+0000 7f4e5259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e541065d0 0x7f4e541069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:58.845 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:57.678+0000 7f4e5259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e541065d0 0x7f4e541069d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59150/0 (socket says 172.21.15.5:59150) 2024-09-12T02:48:58.845 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:57.678+0000 7f4e5259c640 1 -- 172.21.15.5:0/1270941468 learned_addr learned my addr 172.21.15.5:0/1270941468 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:48:58.845 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:57.678+0000 7f4e5259c640 1 -- 172.21.15.5:0/1270941468 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e541077d0 con 0x7f4e541065d0 2024-09-12T02:48:58.845 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:57.679+0000 7f4e5259c640 1 --2- 172.21.15.5:0/1270941468 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e541065d0 0x7f4e541069d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c009820 tx=0x7f4e4c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0db645683c69ad1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:48:58.845 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:57.679+0000 7f4e5159a640 1 -- 172.21.15.5:0/1270941468 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4e4c037070 con 0x7f4e541065d0 2024-09-12T02:48:58.846 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:57.679+0000 7f4e5159a640 1 -- 172.21.15.5:0/1270941468 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4e4c032900 con 0x7f4e541065d0 2024-09-12T02:48:58.846 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:57.679+0000 7f4e5159a640 1 -- 172.21.15.5:0/1270941468 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4e4c030850 con 0x7f4e541065d0 2024-09-12T02:48:58.846 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:57.680+0000 7f4e58a8f640 1 -- 172.21.15.5:0/1270941468 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e541065d0 msgr2=0x7f4e541069d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:58.846 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:57.680+0000 7f4e58a8f640 1 --2- 172.21.15.5:0/1270941468 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e541065d0 0x7f4e541069d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c009820 tx=0x7f4e4c02a270 comp rx=0 tx=0).stop 2024-09-12T02:48:58.846 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:57.680+0000 7f4e58a8f640 1 -- 172.21.15.5:0/1270941468 shutdown_connections 2024-09-12T02:48:58.846 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:57.680+0000 7f4e58a8f640 1 --2- 172.21.15.5:0/1270941468 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e541065d0 0x7f4e541069d0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:58.846 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:57.680+0000 7f4e58a8f640 1 -- 172.21.15.5:0/1270941468 >> 172.21.15.5:0/1270941468 conn(0x7f4e54101e20 msgr2=0x7f4e54104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:48:58.846 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:57.680+0000 7f4e58a8f640 1 -- 172.21.15.5:0/1270941468 shutdown_connections 2024-09-12T02:48:58.846 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:57.680+0000 7f4e58a8f640 1 -- 172.21.15.5:0/1270941468 wait complete. 2024-09-12T02:48:58.846 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:57.681+0000 7f4e58a8f640 1 Processor -- start 2024-09-12T02:48:58.846 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:57.681+0000 7f4e58a8f640 1 -- start start 2024-09-12T02:48:58.846 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:57.681+0000 7f4e58a8f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e541065d0 0x7f4e5419b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:58.846 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:57.681+0000 7f4e58a8f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4e5410ab80 con 0x7f4e541065d0 2024-09-12T02:48:58.847 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:57.681+0000 7f4e5259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e541065d0 0x7f4e5419b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:58.847 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:57.681+0000 7f4e5259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e541065d0 0x7f4e5419b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59162/0 (socket says 172.21.15.5:59162) 2024-09-12T02:48:58.847 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:57.681+0000 7f4e5259c640 1 -- 172.21.15.5:0/1805376622 learned_addr learned my addr 172.21.15.5:0/1805376622 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:48:58.847 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:57.681+0000 7f4e5259c640 1 -- 172.21.15.5:0/1805376622 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e5419bcc0 con 0x7f4e541065d0 2024-09-12T02:48:58.847 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:57.682+0000 7f4e5259c640 1 --2- 172.21.15.5:0/1805376622 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e541065d0 0x7f4e5419b780 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c009940 tx=0x7f4e4c032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:48:58.847 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:57.682+0000 7f4e4b7fe640 1 -- 172.21.15.5:0/1805376622 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4e4c042070 con 0x7f4e541065d0 2024-09-12T02:48:58.847 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:57.682+0000 7f4e4b7fe640 1 -- 172.21.15.5:0/1805376622 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f4e4c0308c0 con 0x7f4e541065d0 2024-09-12T02:48:58.847 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:57.682+0000 7f4e58a8f640 1 -- 172.21.15.5:0/1805376622 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e5419bf50 con 0x7f4e541065d0 2024-09-12T02:48:58.847 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:57.682+0000 7f4e4b7fe640 1 -- 172.21.15.5:0/1805376622 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4e4c037040 con 0x7f4e541065d0 2024-09-12T02:48:58.847 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:57.682+0000 7f4e58a8f640 1 -- 172.21.15.5:0/1805376622 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e5419ec40 con 0x7f4e541065d0 2024-09-12T02:48:58.847 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:57.683+0000 7f4e4b7fe640 1 -- 172.21.15.5:0/1805376622 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f4e4c045070 con 0x7f4e541065d0 2024-09-12T02:48:58.847 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:57.683+0000 7f4e58a8f640 1 -- 172.21.15.5:0/1805376622 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e5410ad00 con 0x7f4e541065d0 2024-09-12T02:48:58.848 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:57.683+0000 7f4e4b7fe640 1 --2- 172.21.15.5:0/1805376622 >> [v2:172.21.15.5:6800/2595781680,v1:172.21.15.5:6801/2595781680] conn(0x7f4e40038f20 0x7f4e4003b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:48:58.848 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:57.683+0000 7f4e4b7fe640 1 -- 172.21.15.5:0/1805376622 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f4e4c06bfb0 con 0x7f4e541065d0 2024-09-12T02:48:58.848 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:57.683+0000 7f4e51d9b640 1 --2- 172.21.15.5:0/1805376622 >> [v2:172.21.15.5:6800/2595781680,v1:172.21.15.5:6801/2595781680] conn(0x7f4e40038f20 0x7f4e4003b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:48:58.848 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:57.684+0000 7f4e51d9b640 1 --2- 172.21.15.5:0/1805376622 >> [v2:172.21.15.5:6800/2595781680,v1:172.21.15.5:6801/2595781680] conn(0x7f4e40038f20 0x7f4e4003b3e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c0098c0 tx=0x7f4e3c006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:48:58.848 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:57.688+0000 7f4e4b7fe640 1 -- 172.21.15.5:0/1805376622 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4e4c04b9d0 con 0x7f4e541065d0 2024-09-12T02:48:58.848 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:57.911+0000 7f4e58a8f640 1 -- 172.21.15.5:0/1805376622 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f4e5402da30 con 0x7f4e541065d0 2024-09-12T02:48:58.848 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:58.243+0000 7f4e4b7fe640 1 -- 172.21.15.5:0/1805376622 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f4e4c045e30 con 0x7f4e541065d0 2024-09-12T02:48:58.848 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:58.245+0000 7f4e4b7fe640 1 -- 172.21.15.5:0/1805376622 <== mon.0 v2:172.21.15.5:3300/0 8 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f4e4c0327b0 con 0x7f4e541065d0 2024-09-12T02:48:58.848 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:58.246+0000 7f4e58a8f640 1 -- 172.21.15.5:0/1805376622 >> [v2:172.21.15.5:6800/2595781680,v1:172.21.15.5:6801/2595781680] conn(0x7f4e40038f20 msgr2=0x7f4e4003b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:58.848 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:58.246+0000 7f4e58a8f640 1 --2- 172.21.15.5:0/1805376622 >> [v2:172.21.15.5:6800/2595781680,v1:172.21.15.5:6801/2595781680] conn(0x7f4e40038f20 0x7f4e4003b3e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c0098c0 tx=0x7f4e3c006db0 comp rx=0 tx=0).stop 2024-09-12T02:48:58.848 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:58.246+0000 7f4e58a8f640 1 -- 172.21.15.5:0/1805376622 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e541065d0 msgr2=0x7f4e5419b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:48:58.848 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:58.246+0000 7f4e58a8f640 1 --2- 172.21.15.5:0/1805376622 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e541065d0 0x7f4e5419b780 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c009940 tx=0x7f4e4c032ce0 comp rx=0 tx=0).stop 2024-09-12T02:48:58.849 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:58.246+0000 7f4e58a8f640 1 -- 172.21.15.5:0/1805376622 shutdown_connections 2024-09-12T02:48:58.849 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:58.246+0000 7f4e58a8f640 1 --2- 172.21.15.5:0/1805376622 >> [v2:172.21.15.5:6800/2595781680,v1:172.21.15.5:6801/2595781680] conn(0x7f4e40038f20 0x7f4e4003b3e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:58.849 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:58.246+0000 7f4e58a8f640 1 --2- 172.21.15.5:0/1805376622 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4e541065d0 0x7f4e5419b780 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:48:58.849 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:58.246+0000 7f4e58a8f640 1 -- 172.21.15.5:0/1805376622 >> 172.21.15.5:0/1805376622 conn(0x7f4e54101e20 msgr2=0x7f4e54102d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:48:58.849 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:58.246+0000 7f4e58a8f640 1 -- 172.21.15.5:0/1805376622 shutdown_connections 2024-09-12T02:48:58.849 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:48:58.246+0000 7f4e58a8f640 1 -- 172.21.15.5:0/1805376622 wait complete. 2024-09-12T02:48:59.244 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:48:58 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: 2024-09-12T02:48:58.985+0000 7f854a834040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-12T02:48:59.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:59 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/1805376622' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-12T02:48:59.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:48:59 smithi005 ceph-mon[31197]: mgrmap e5: x(active, since 4s) 2024-09-12T02:48:59.560 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:48:59 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: 2024-09-12T02:48:59.312+0000 7f854a834040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-12T02:48:59.560 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:48:59 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: 2024-09-12T02:48:59.376+0000 7f854a834040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-12T02:49:00.060 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:48:59 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: 2024-09-12T02:48:59.571+0000 7f854a834040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-12T02:49:00.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:00 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/595160129' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-12T02:49:00.811 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:49:00 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: 2024-09-12T02:49:00.499+0000 7f854a834040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-12T02:49:00.811 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:49:00 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: 2024-09-12T02:49:00.573+0000 7f854a834040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-12T02:49:01.443 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:49:01 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: 2024-09-12T02:49:01.322+0000 7f854a834040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-12T02:49:01.444 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:49:01 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: 2024-09-12T02:49:01.443+0000 7f854a834040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-12T02:49:01.810 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:49:01 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: 2024-09-12T02:49:01.593+0000 7f854a834040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-12T02:49:01.810 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:49:01 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: 2024-09-12T02:49:01.657+0000 7f854a834040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-12T02:49:02.114 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:49:01 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: 2024-09-12T02:49:01.814+0000 7f854a834040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-12T02:49:02.115 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:49:01 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: 2024-09-12T02:49:01.885+0000 7f854a834040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-12T02:49:02.115 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:49:01 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: 2024-09-12T02:49:01.948+0000 7f854a834040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-12T02:49:02.115 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:49:02 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: 2024-09-12T02:49:02.114+0000 7f854a834040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-12T02:49:02.402 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:49:02 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: 2024-09-12T02:49:02.183+0000 7f854a834040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-12T02:49:02.402 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:49:02 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: 2024-09-12T02:49:02.252+0000 7f854a834040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-12T02:49:02.402 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:49:02 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: 2024-09-12T02:49:02.339+0000 7f854a834040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-12T02:49:02.403 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:49:02 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: 2024-09-12T02:49:02.401+0000 7f854a834040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-12T02:49:02.810 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:49:02 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: 2024-09-12T02:49:02.463+0000 7f854a834040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-12T02:49:03.400 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:49:03 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: 2024-09-12T02:49:03.298+0000 7f854a834040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-12T02:49:03.400 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:03 smithi005 ceph-mon[31197]: Active manager daemon x restarted 2024-09-12T02:49:03.400 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:03 smithi005 ceph-mon[31197]: Activating manager daemon x 2024-09-12T02:49:03.400 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:03 smithi005 ceph-mon[31197]: osdmap e2: 0 total, 0 up, 0 in 2024-09-12T02:49:03.400 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:03 smithi005 ceph-mon[31197]: mgrmap e6: x(active, starting, since 0.0043611s) 2024-09-12T02:49:03.400 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:03 smithi005 ceph-mon[31197]: from='mgr.14116 172.21.15.5:0/613993494' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T02:49:03.401 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:03 smithi005 ceph-mon[31197]: from='mgr.14116 172.21.15.5:0/613993494' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-12T02:49:03.401 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:03 smithi005 ceph-mon[31197]: from='mgr.14116 172.21.15.5:0/613993494' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-12T02:49:03.401 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:03 smithi005 ceph-mon[31197]: from='mgr.14116 172.21.15.5:0/613993494' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-12T02:49:03.401 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:03 smithi005 ceph-mon[31197]: from='mgr.14116 172.21.15.5:0/613993494' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-12T02:49:03.401 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:03 smithi005 ceph-mon[31197]: Manager daemon x is now available 2024-09-12T02:49:03.401 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:03 smithi005 ceph-mon[31197]: from='mgr.14116 172.21.15.5:0/613993494' entity='mgr.x' 2024-09-12T02:49:03.401 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:03 smithi005 ceph-mon[31197]: from='mgr.14116 172.21.15.5:0/613993494' entity='mgr.x' 2024-09-12T02:49:03.401 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:03 smithi005 ceph-mon[31197]: from='mgr.14116 172.21.15.5:0/613993494' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:49:03.401 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:03 smithi005 ceph-mon[31197]: from='mgr.14116 172.21.15.5:0/613993494' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:49:03.401 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:03 smithi005 ceph-mon[31197]: from='mgr.14116 172.21.15.5:0/613993494' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-12T02:49:03.401 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:03 smithi005 ceph-mon[31197]: from='mgr.14116 172.21.15.5:0/613993494' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-12T02:49:05.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:05 smithi005 ceph-mon[31197]: Found migration_current of "None". Setting to last migration. 2024-09-12T02:49:05.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:05 smithi005 ceph-mon[31197]: from='mgr.14116 172.21.15.5:0/613993494' entity='mgr.x' 2024-09-12T02:49:05.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:05 smithi005 ceph-mon[31197]: from='mgr.14116 172.21.15.5:0/613993494' entity='mgr.x' 2024-09-12T02:49:05.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:05 smithi005 ceph-mon[31197]: mgrmap e7: x(active, since 1.00529s) 2024-09-12T02:49:06.152 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout { 2024-09-12T02:49:06.152 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-12T02:49:06.152 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-12T02:49:06.152 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-12T02:49:06.152 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-12T02:49:06.152 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout } 2024-09-12T02:49:06.152 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:00.262+0000 7fa8ec234640 1 Processor -- start 2024-09-12T02:49:06.152 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:00.262+0000 7fa8ec234640 1 -- start start 2024-09-12T02:49:06.153 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:00.262+0000 7fa8ec234640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8e4105f70 0x7fa8e4106370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:06.153 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:00.262+0000 7fa8ec234640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa8e407bdf0 con 0x7fa8e4105f70 2024-09-12T02:49:06.160 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:00.262+0000 7fa8e9fd0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8e4105f70 0x7fa8e4106370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:06.160 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:00.262+0000 7fa8e9fd0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8e4105f70 0x7fa8e4106370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59194/0 (socket says 172.21.15.5:59194) 2024-09-12T02:49:06.160 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:00.262+0000 7fa8e9fd0640 1 -- 172.21.15.5:0/65677866 learned_addr learned my addr 172.21.15.5:0/65677866 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:49:06.160 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:00.263+0000 7fa8e9fd0640 1 -- 172.21.15.5:0/65677866 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8e41068b0 con 0x7fa8e4105f70 2024-09-12T02:49:06.160 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:00.263+0000 7fa8e9fd0640 1 --2- 172.21.15.5:0/65677866 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8e4105f70 0x7fa8e4106370 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fa8e0009820 tx=0x7fa8e002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b88107a5dc38f26e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:06.160 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:00.264+0000 7fa8e8fce640 1 -- 172.21.15.5:0/65677866 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa8e0037070 con 0x7fa8e4105f70 2024-09-12T02:49:06.160 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:00.264+0000 7fa8e8fce640 1 -- 172.21.15.5:0/65677866 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa8e0032900 con 0x7fa8e4105f70 2024-09-12T02:49:06.160 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:00.264+0000 7fa8e8fce640 1 -- 172.21.15.5:0/65677866 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa8e0030850 con 0x7fa8e4105f70 2024-09-12T02:49:06.160 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:00.264+0000 7fa8ec234640 1 -- 172.21.15.5:0/65677866 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8e4105f70 msgr2=0x7fa8e4106370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:06.160 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:00.264+0000 7fa8ec234640 1 --2- 172.21.15.5:0/65677866 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8e4105f70 0x7fa8e4106370 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fa8e0009820 tx=0x7fa8e002a270 comp rx=0 tx=0).stop 2024-09-12T02:49:06.160 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:00.264+0000 7fa8ec234640 1 -- 172.21.15.5:0/65677866 shutdown_connections 2024-09-12T02:49:06.161 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:00.264+0000 7fa8ec234640 1 --2- 172.21.15.5:0/65677866 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8e4105f70 0x7fa8e4106370 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:06.161 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:00.264+0000 7fa8ec234640 1 -- 172.21.15.5:0/65677866 >> 172.21.15.5:0/65677866 conn(0x7fa8e4101e20 msgr2=0x7fa8e4104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:06.161 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:00.264+0000 7fa8ec234640 1 -- 172.21.15.5:0/65677866 shutdown_connections 2024-09-12T02:49:06.161 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:00.264+0000 7fa8ec234640 1 -- 172.21.15.5:0/65677866 wait complete. 2024-09-12T02:49:06.161 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:00.265+0000 7fa8ec234640 1 Processor -- start 2024-09-12T02:49:06.161 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:00.265+0000 7fa8ec234640 1 -- start start 2024-09-12T02:49:06.161 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:00.265+0000 7fa8ec234640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8e4105f70 0x7fa8e419d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:06.161 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:00.265+0000 7fa8ec234640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa8e41079b0 con 0x7fa8e4105f70 2024-09-12T02:49:06.161 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:00.265+0000 7fa8e9fd0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8e4105f70 0x7fa8e419d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:06.161 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:00.265+0000 7fa8e9fd0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8e4105f70 0x7fa8e419d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59196/0 (socket says 172.21.15.5:59196) 2024-09-12T02:49:06.161 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:00.265+0000 7fa8e9fd0640 1 -- 172.21.15.5:0/595160129 learned_addr learned my addr 172.21.15.5:0/595160129 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:49:06.161 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:00.266+0000 7fa8e9fd0640 1 -- 172.21.15.5:0/595160129 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8e419de10 con 0x7fa8e4105f70 2024-09-12T02:49:06.161 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:00.266+0000 7fa8e9fd0640 1 --2- 172.21.15.5:0/595160129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8e4105f70 0x7fa8e419d8d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa8e0032ce0 tx=0x7fa8e0032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:06.162 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:00.266+0000 7fa8d6ffd640 1 -- 172.21.15.5:0/595160129 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa8e0042070 con 0x7fa8e4105f70 2024-09-12T02:49:06.162 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:00.266+0000 7fa8d6ffd640 1 -- 172.21.15.5:0/595160129 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa8e00308c0 con 0x7fa8e4105f70 2024-09-12T02:49:06.162 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:00.266+0000 7fa8ec234640 1 -- 172.21.15.5:0/595160129 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8e419e0a0 con 0x7fa8e4105f70 2024-09-12T02:49:06.162 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:00.266+0000 7fa8d6ffd640 1 -- 172.21.15.5:0/595160129 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa8e0037040 con 0x7fa8e4105f70 2024-09-12T02:49:06.162 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:00.266+0000 7fa8ec234640 1 -- 172.21.15.5:0/595160129 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8e41a0d90 con 0x7fa8e4105f70 2024-09-12T02:49:06.162 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:00.267+0000 7fa8d6ffd640 1 -- 172.21.15.5:0/595160129 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7fa8e0045070 con 0x7fa8e4105f70 2024-09-12T02:49:06.162 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:00.267+0000 7fa8ec234640 1 -- 172.21.15.5:0/595160129 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8e4106370 con 0x7fa8e4105f70 2024-09-12T02:49:06.162 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:00.267+0000 7fa8d6ffd640 1 --2- 172.21.15.5:0/595160129 >> [v2:172.21.15.5:6800/2595781680,v1:172.21.15.5:6801/2595781680] conn(0x7fa8d0038fc0 0x7fa8d003b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:06.162 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:00.267+0000 7fa8d6ffd640 1 -- 172.21.15.5:0/595160129 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fa8e006bfb0 con 0x7fa8e4105f70 2024-09-12T02:49:06.162 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:00.267+0000 7fa8e97cf640 1 -- 172.21.15.5:0/595160129 >> [v2:172.21.15.5:6800/2595781680,v1:172.21.15.5:6801/2595781680] conn(0x7fa8d0038fc0 msgr2=0x7fa8d003b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/2595781680 2024-09-12T02:49:06.162 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:00.267+0000 7fa8e97cf640 1 --2- 172.21.15.5:0/595160129 >> [v2:172.21.15.5:6800/2595781680,v1:172.21.15.5:6801/2595781680] conn(0x7fa8d0038fc0 0x7fa8d003b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T02:49:06.162 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:00.272+0000 7fa8d6ffd640 1 -- 172.21.15.5:0/595160129 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa8e004b170 con 0x7fa8e4105f70 2024-09-12T02:49:06.162 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:00.468+0000 7fa8e97cf640 1 -- 172.21.15.5:0/595160129 >> [v2:172.21.15.5:6800/2595781680,v1:172.21.15.5:6801/2595781680] conn(0x7fa8d0038fc0 msgr2=0x7fa8d003b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/2595781680 2024-09-12T02:49:06.163 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:00.468+0000 7fa8e97cf640 1 --2- 172.21.15.5:0/595160129 >> [v2:172.21.15.5:6800/2595781680,v1:172.21.15.5:6801/2595781680] conn(0x7fa8d0038fc0 0x7fa8d003b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-12T02:49:06.163 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:00.497+0000 7fa8ec234640 1 -- 172.21.15.5:0/595160129 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fa8e4196d40 con 0x7fa8e4105f70 2024-09-12T02:49:06.163 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:00.497+0000 7fa8d6ffd640 1 -- 172.21.15.5:0/595160129 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7fa8e006b9d0 con 0x7fa8e4105f70 2024-09-12T02:49:06.163 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:00.500+0000 7fa8ec234640 1 -- 172.21.15.5:0/595160129 >> [v2:172.21.15.5:6800/2595781680,v1:172.21.15.5:6801/2595781680] conn(0x7fa8d0038fc0 msgr2=0x7fa8d003b480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T02:49:06.163 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:00.500+0000 7fa8ec234640 1 --2- 172.21.15.5:0/595160129 >> [v2:172.21.15.5:6800/2595781680,v1:172.21.15.5:6801/2595781680] conn(0x7fa8d0038fc0 0x7fa8d003b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:06.163 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:00.500+0000 7fa8ec234640 1 -- 172.21.15.5:0/595160129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8e4105f70 msgr2=0x7fa8e419d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:06.163 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:00.500+0000 7fa8ec234640 1 --2- 172.21.15.5:0/595160129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8e4105f70 0x7fa8e419d8d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa8e0032ce0 tx=0x7fa8e0032d10 comp rx=0 tx=0).stop 2024-09-12T02:49:06.163 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:00.500+0000 7fa8ec234640 1 -- 172.21.15.5:0/595160129 shutdown_connections 2024-09-12T02:49:06.163 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:00.500+0000 7fa8ec234640 1 --2- 172.21.15.5:0/595160129 >> [v2:172.21.15.5:6800/2595781680,v1:172.21.15.5:6801/2595781680] conn(0x7fa8d0038fc0 0x7fa8d003b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:06.163 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:00.500+0000 7fa8ec234640 1 --2- 172.21.15.5:0/595160129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8e4105f70 0x7fa8e419d8d0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:06.163 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:00.500+0000 7fa8ec234640 1 -- 172.21.15.5:0/595160129 >> 172.21.15.5:0/595160129 conn(0x7fa8e4101e20 msgr2=0x7fa8e4191bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:06.163 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:00.500+0000 7fa8ec234640 1 -- 172.21.15.5:0/595160129 shutdown_connections 2024-09-12T02:49:06.164 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:00.500+0000 7fa8ec234640 1 -- 172.21.15.5:0/595160129 wait complete. 2024-09-12T02:49:06.164 INFO:teuthology.orchestra.run.smithi005.stdout:Waiting for the mgr to restart... 2024-09-12T02:49:06.164 INFO:teuthology.orchestra.run.smithi005.stdout:Waiting for mgr epoch 5... 2024-09-12T02:49:07.136 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout { 2024-09-12T02:49:07.136 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 8, 2024-09-12T02:49:07.136 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-12T02:49:07.136 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout } 2024-09-12T02:49:07.137 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:06.580+0000 7ffbdb34a640 1 Processor -- start 2024-09-12T02:49:07.137 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:06.580+0000 7ffbdb34a640 1 -- start start 2024-09-12T02:49:07.137 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:06.581+0000 7ffbdb34a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbd407dd60 0x7ffbd407c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:07.137 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:06.581+0000 7ffbdb34a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ffbd407c7a0 con 0x7ffbd407dd60 2024-09-12T02:49:07.137 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:06.581+0000 7ffbd90e6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbd407dd60 0x7ffbd407c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:07.137 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:06.581+0000 7ffbd90e6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbd407dd60 0x7ffbd407c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36600/0 (socket says 172.21.15.5:36600) 2024-09-12T02:49:07.137 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:06.581+0000 7ffbd90e6640 1 -- 172.21.15.5:0/3246334508 learned_addr learned my addr 172.21.15.5:0/3246334508 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:49:07.137 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:06.581+0000 7ffbd90e6640 1 -- 172.21.15.5:0/3246334508 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbd407c920 con 0x7ffbd407dd60 2024-09-12T02:49:07.137 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:06.582+0000 7ffbd90e6640 1 --2- 172.21.15.5:0/3246334508 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbd407dd60 0x7ffbd407c260 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ffbd0009820 tx=0x7ffbd002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bde6eb000f0bf2c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:07.137 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:06.582+0000 7ffbc7fff640 1 -- 172.21.15.5:0/3246334508 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffbd0037070 con 0x7ffbd407dd60 2024-09-12T02:49:07.137 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:06.582+0000 7ffbc7fff640 1 -- 172.21.15.5:0/3246334508 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ffbd0032900 con 0x7ffbd407dd60 2024-09-12T02:49:07.138 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:06.582+0000 7ffbc7fff640 1 -- 172.21.15.5:0/3246334508 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffbd0030850 con 0x7ffbd407dd60 2024-09-12T02:49:07.138 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:06.583+0000 7ffbdb34a640 1 -- 172.21.15.5:0/3246334508 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbd407dd60 msgr2=0x7ffbd407c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:07.138 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:06.583+0000 7ffbdb34a640 1 --2- 172.21.15.5:0/3246334508 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbd407dd60 0x7ffbd407c260 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ffbd0009820 tx=0x7ffbd002a270 comp rx=0 tx=0).stop 2024-09-12T02:49:07.138 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:06.583+0000 7ffbdb34a640 1 -- 172.21.15.5:0/3246334508 shutdown_connections 2024-09-12T02:49:07.138 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:06.583+0000 7ffbdb34a640 1 --2- 172.21.15.5:0/3246334508 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbd407dd60 0x7ffbd407c260 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:07.138 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:06.583+0000 7ffbdb34a640 1 -- 172.21.15.5:0/3246334508 >> 172.21.15.5:0/3246334508 conn(0x7ffbd4102f30 msgr2=0x7ffbd41053b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:07.138 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:06.583+0000 7ffbdb34a640 1 -- 172.21.15.5:0/3246334508 shutdown_connections 2024-09-12T02:49:07.138 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:06.583+0000 7ffbdb34a640 1 -- 172.21.15.5:0/3246334508 wait complete. 2024-09-12T02:49:07.138 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:06.584+0000 7ffbdb34a640 1 Processor -- start 2024-09-12T02:49:07.138 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:06.584+0000 7ffbdb34a640 1 -- start start 2024-09-12T02:49:07.138 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:06.584+0000 7ffbdb34a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbd407dd60 0x7ffbd419d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:07.138 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:06.584+0000 7ffbdb34a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ffbd410dd50 con 0x7ffbd407dd60 2024-09-12T02:49:07.138 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:06.584+0000 7ffbd90e6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbd407dd60 0x7ffbd419d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:07.139 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:06.584+0000 7ffbd90e6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbd407dd60 0x7ffbd419d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36602/0 (socket says 172.21.15.5:36602) 2024-09-12T02:49:07.139 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:06.584+0000 7ffbd90e6640 1 -- 172.21.15.5:0/1295745916 learned_addr learned my addr 172.21.15.5:0/1295745916 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:49:07.139 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:06.584+0000 7ffbd90e6640 1 -- 172.21.15.5:0/1295745916 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbd419de70 con 0x7ffbd407dd60 2024-09-12T02:49:07.139 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:06.584+0000 7ffbd90e6640 1 --2- 172.21.15.5:0/1295745916 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbd407dd60 0x7ffbd419d930 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ffbd00097f0 tx=0x7ffbd0030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:07.139 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:06.585+0000 7ffbc67fc640 1 -- 172.21.15.5:0/1295745916 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffbd0042070 con 0x7ffbd407dd60 2024-09-12T02:49:07.139 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:06.585+0000 7ffbc67fc640 1 -- 172.21.15.5:0/1295745916 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ffbd0032ca0 con 0x7ffbd407dd60 2024-09-12T02:49:07.139 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:06.585+0000 7ffbdb34a640 1 -- 172.21.15.5:0/1295745916 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbd419e100 con 0x7ffbd407dd60 2024-09-12T02:49:07.139 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:06.585+0000 7ffbdb34a640 1 -- 172.21.15.5:0/1295745916 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbd41a0df0 con 0x7ffbd407dd60 2024-09-12T02:49:07.139 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:06.585+0000 7ffbc67fc640 1 -- 172.21.15.5:0/1295745916 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffbd0037040 con 0x7ffbd407dd60 2024-09-12T02:49:07.140 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:06.586+0000 7ffbc67fc640 1 -- 172.21.15.5:0/1295745916 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7ffbd0045070 con 0x7ffbd407dd60 2024-09-12T02:49:07.140 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:06.586+0000 7ffbc67fc640 1 --2- 172.21.15.5:0/1295745916 >> [v2:172.21.15.5:6800/3125821798,v1:172.21.15.5:6801/3125821798] conn(0x7ffbc0038fc0 0x7ffbc003b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:07.140 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:06.586+0000 7ffbc67fc640 1 -- 172.21.15.5:0/1295745916 --> [v2:172.21.15.5:6800/3125821798,v1:172.21.15.5:6801/3125821798] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ffbc003bb90 con 0x7ffbc0038fc0 2024-09-12T02:49:07.140 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:06.586+0000 7ffbc67fc640 1 -- 172.21.15.5:0/1295745916 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ffbd006bf40 con 0x7ffbd407dd60 2024-09-12T02:49:07.140 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:06.586+0000 7ffbd88e5640 1 --2- 172.21.15.5:0/1295745916 >> [v2:172.21.15.5:6800/3125821798,v1:172.21.15.5:6801/3125821798] conn(0x7ffbc0038fc0 0x7ffbc003b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:07.140 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:06.586+0000 7ffbd88e5640 1 --2- 172.21.15.5:0/1295745916 >> [v2:172.21.15.5:6800/3125821798,v1:172.21.15.5:6801/3125821798] conn(0x7ffbc0038fc0 0x7ffbc003b480 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ffbc80098c0 tx=0x7ffbc8006db0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:07.140 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:06.587+0000 7ffbc67fc640 1 -- 172.21.15.5:0/1295745916 <== mgr.14116 v2:172.21.15.5:6800/3125821798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7ffbc003bb90 con 0x7ffbc0038fc0 2024-09-12T02:49:07.140 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:06.593+0000 7ffbdb34a640 1 -- 172.21.15.5:0/1295745916 --> [v2:172.21.15.5:6800/3125821798,v1:172.21.15.5:6801/3125821798] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7ffbd407c2e0 con 0x7ffbc0038fc0 2024-09-12T02:49:07.140 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:06.594+0000 7ffbc67fc640 1 -- 172.21.15.5:0/1295745916 <== mgr.14116 v2:172.21.15.5:6800/3125821798 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7ffbd407c2e0 con 0x7ffbc0038fc0 2024-09-12T02:49:07.140 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:06.594+0000 7ffbdb34a640 1 -- 172.21.15.5:0/1295745916 >> [v2:172.21.15.5:6800/3125821798,v1:172.21.15.5:6801/3125821798] conn(0x7ffbc0038fc0 msgr2=0x7ffbc003b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:07.140 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:06.594+0000 7ffbdb34a640 1 --2- 172.21.15.5:0/1295745916 >> [v2:172.21.15.5:6800/3125821798,v1:172.21.15.5:6801/3125821798] conn(0x7ffbc0038fc0 0x7ffbc003b480 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ffbc80098c0 tx=0x7ffbc8006db0 comp rx=0 tx=0).stop 2024-09-12T02:49:07.140 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:06.594+0000 7ffbdb34a640 1 -- 172.21.15.5:0/1295745916 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbd407dd60 msgr2=0x7ffbd419d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:07.140 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:06.594+0000 7ffbdb34a640 1 --2- 172.21.15.5:0/1295745916 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbd407dd60 0x7ffbd419d930 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ffbd00097f0 tx=0x7ffbd0030ec0 comp rx=0 tx=0).stop 2024-09-12T02:49:07.141 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:06.594+0000 7ffbdb34a640 1 -- 172.21.15.5:0/1295745916 shutdown_connections 2024-09-12T02:49:07.141 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:06.594+0000 7ffbdb34a640 1 --2- 172.21.15.5:0/1295745916 >> [v2:172.21.15.5:6800/3125821798,v1:172.21.15.5:6801/3125821798] conn(0x7ffbc0038fc0 0x7ffbc003b480 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:07.141 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:06.594+0000 7ffbdb34a640 1 --2- 172.21.15.5:0/1295745916 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbd407dd60 0x7ffbd419d930 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:07.141 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:06.594+0000 7ffbdb34a640 1 -- 172.21.15.5:0/1295745916 >> 172.21.15.5:0/1295745916 conn(0x7ffbd4102f30 msgr2=0x7ffbd4103b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:07.141 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:06.594+0000 7ffbdb34a640 1 -- 172.21.15.5:0/1295745916 shutdown_connections 2024-09-12T02:49:07.141 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:06.594+0000 7ffbdb34a640 1 -- 172.21.15.5:0/1295745916 wait complete. 2024-09-12T02:49:07.141 INFO:teuthology.orchestra.run.smithi005.stdout:mgr epoch 5 is available 2024-09-12T02:49:07.141 INFO:teuthology.orchestra.run.smithi005.stdout:Setting orchestrator backend to cephadm... 2024-09-12T02:49:07.270 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:07 smithi005 ceph-mon[31197]: mgrmap e8: x(active, since 2s) 2024-09-12T02:49:08.301 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:08 smithi005 ceph-mon[31197]: from='client.14126 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-12T02:49:08.301 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:08 smithi005 ceph-mon[31197]: from='client.14126 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-12T02:49:08.301 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:08 smithi005 ceph-mon[31197]: from='mgr.14116 172.21.15.5:0/613993494' entity='mgr.x' 2024-09-12T02:49:08.301 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:08 smithi005 ceph-mon[31197]: from='mgr.14116 172.21.15.5:0/613993494' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:49:08.301 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:08 smithi005 ceph-mon[31197]: from='mgr.14116 172.21.15.5:0/613993494' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:49:08.301 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:49:07 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: 2024-09-12T02:49:07.886+0000 7f84b7319640 -1 log_channel(cephadm) log [ERR] : [12/Sep/2024:02:49:07] ENGINE Error in HTTPServer.serve 2024-09-12T02:49:08.302 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:49:07 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: Traceback (most recent call last): 2024-09-12T02:49:08.302 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:49:07 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-12T02:49:08.302 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:49:07 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: self._connections.run(self.expiration_interval) 2024-09-12T02:49:08.302 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:49:07 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-12T02:49:08.302 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:49:07 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: self._run(expiration_interval) 2024-09-12T02:49:08.302 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:49:07 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-12T02:49:08.302 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:49:07 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: new_conn = self._from_server_socket(self.server.socket) 2024-09-12T02:49:08.302 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:49:07 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-12T02:49:08.302 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:49:07 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-12T02:49:08.302 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:49:07 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-12T02:49:08.302 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:49:07 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: s = self.context.wrap_socket( 2024-09-12T02:49:08.302 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:49:07 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-12T02:49:08.302 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:49:07 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: return self.sslsocket_class._create( 2024-09-12T02:49:08.303 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:49:07 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-12T02:49:08.303 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:49:07 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: self.do_handshake() 2024-09-12T02:49:08.303 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:49:07 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-12T02:49:08.303 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:49:07 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: self._sslobj.do_handshake() 2024-09-12T02:49:08.303 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:49:07 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-12T02:49:08.303 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:49:07 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: 2024-09-12T02:49:08.335 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:07.553+0000 7f7c7884d640 1 Processor -- start 2024-09-12T02:49:08.335 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:07.553+0000 7f7c7884d640 1 -- start start 2024-09-12T02:49:08.335 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:07.554+0000 7f7c7884d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c74105fb0 0x7f7c741063b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:08.335 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:07.554+0000 7f7c7884d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7c7407bdb0 con 0x7f7c74105fb0 2024-09-12T02:49:08.335 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:07.554+0000 7f7c7259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c74105fb0 0x7f7c741063b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:08.335 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:07.554+0000 7f7c7259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c74105fb0 0x7f7c741063b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36616/0 (socket says 172.21.15.5:36616) 2024-09-12T02:49:08.336 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:07.554+0000 7f7c7259c640 1 -- 172.21.15.5:0/2295508172 learned_addr learned my addr 172.21.15.5:0/2295508172 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:49:08.336 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:07.554+0000 7f7c7259c640 1 -- 172.21.15.5:0/2295508172 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c741068f0 con 0x7f7c74105fb0 2024-09-12T02:49:08.336 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:07.555+0000 7f7c7259c640 1 --2- 172.21.15.5:0/2295508172 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c74105fb0 0x7f7c741063b0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f7c64009820 tx=0x7f7c6402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b81fee5990357c6d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:08.336 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:07.555+0000 7f7c71d9b640 1 -- 172.21.15.5:0/2295508172 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7c64037070 con 0x7f7c74105fb0 2024-09-12T02:49:08.336 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:07.555+0000 7f7c71d9b640 1 -- 172.21.15.5:0/2295508172 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7c64032900 con 0x7f7c74105fb0 2024-09-12T02:49:08.336 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:07.555+0000 7f7c71d9b640 1 -- 172.21.15.5:0/2295508172 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7c64030850 con 0x7f7c74105fb0 2024-09-12T02:49:08.336 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:07.556+0000 7f7c7884d640 1 -- 172.21.15.5:0/2295508172 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c74105fb0 msgr2=0x7f7c741063b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:08.336 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:07.556+0000 7f7c7884d640 1 --2- 172.21.15.5:0/2295508172 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c74105fb0 0x7f7c741063b0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f7c64009820 tx=0x7f7c6402a270 comp rx=0 tx=0).stop 2024-09-12T02:49:08.336 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:07.556+0000 7f7c7884d640 1 -- 172.21.15.5:0/2295508172 shutdown_connections 2024-09-12T02:49:08.336 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:07.556+0000 7f7c7884d640 1 --2- 172.21.15.5:0/2295508172 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c74105fb0 0x7f7c741063b0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:08.336 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:07.556+0000 7f7c7884d640 1 -- 172.21.15.5:0/2295508172 >> 172.21.15.5:0/2295508172 conn(0x7f7c74101e20 msgr2=0x7f7c74104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:08.336 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:07.556+0000 7f7c7884d640 1 -- 172.21.15.5:0/2295508172 shutdown_connections 2024-09-12T02:49:08.337 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:07.556+0000 7f7c7884d640 1 -- 172.21.15.5:0/2295508172 wait complete. 2024-09-12T02:49:08.337 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:07.557+0000 7f7c7884d640 1 Processor -- start 2024-09-12T02:49:08.337 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:07.557+0000 7f7c7884d640 1 -- start start 2024-09-12T02:49:08.337 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:07.557+0000 7f7c7884d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c74105fb0 0x7f7c7419d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:08.337 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:07.557+0000 7f7c7884d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7c741079f0 con 0x7f7c74105fb0 2024-09-12T02:49:08.337 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:07.557+0000 7f7c7259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c74105fb0 0x7f7c7419d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:08.337 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:07.557+0000 7f7c7259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c74105fb0 0x7f7c7419d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36630/0 (socket says 172.21.15.5:36630) 2024-09-12T02:49:08.337 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:07.557+0000 7f7c7259c640 1 -- 172.21.15.5:0/2993981130 learned_addr learned my addr 172.21.15.5:0/2993981130 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:49:08.337 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:07.557+0000 7f7c7259c640 1 -- 172.21.15.5:0/2993981130 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c7419de10 con 0x7f7c74105fb0 2024-09-12T02:49:08.337 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:07.557+0000 7f7c7259c640 1 --2- 172.21.15.5:0/2993981130 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c74105fb0 0x7f7c7419d8d0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f7c640097f0 tx=0x7f7c64030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:08.337 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:07.558+0000 7f7c6b7fe640 1 -- 172.21.15.5:0/2993981130 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7c64037040 con 0x7f7c74105fb0 2024-09-12T02:49:08.337 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:07.558+0000 7f7c6b7fe640 1 -- 172.21.15.5:0/2993981130 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7c64040070 con 0x7f7c74105fb0 2024-09-12T02:49:08.337 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:07.558+0000 7f7c7884d640 1 -- 172.21.15.5:0/2993981130 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c7419e0a0 con 0x7f7c74105fb0 2024-09-12T02:49:08.338 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:07.558+0000 7f7c6b7fe640 1 -- 172.21.15.5:0/2993981130 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7c6403a390 con 0x7f7c74105fb0 2024-09-12T02:49:08.338 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:07.558+0000 7f7c7884d640 1 -- 172.21.15.5:0/2993981130 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c741a0d90 con 0x7f7c74105fb0 2024-09-12T02:49:08.338 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:07.559+0000 7f7c7884d640 1 -- 172.21.15.5:0/2993981130 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c741063b0 con 0x7f7c74105fb0 2024-09-12T02:49:08.338 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:07.559+0000 7f7c6b7fe640 1 -- 172.21.15.5:0/2993981130 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f7c6403b430 con 0x7f7c74105fb0 2024-09-12T02:49:08.338 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:07.560+0000 7f7c6b7fe640 1 --2- 172.21.15.5:0/2993981130 >> [v2:172.21.15.5:6800/3125821798,v1:172.21.15.5:6801/3125821798] conn(0x7f7c60038fc0 0x7f7c6003b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:08.338 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:07.560+0000 7f7c6b7fe640 1 -- 172.21.15.5:0/2993981130 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f7c6406c870 con 0x7f7c74105fb0 2024-09-12T02:49:08.338 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:07.560+0000 7f7c6bfff640 1 --2- 172.21.15.5:0/2993981130 >> [v2:172.21.15.5:6800/3125821798,v1:172.21.15.5:6801/3125821798] conn(0x7f7c60038fc0 0x7f7c6003b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:08.338 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:07.560+0000 7f7c6bfff640 1 --2- 172.21.15.5:0/2993981130 >> [v2:172.21.15.5:6800/3125821798,v1:172.21.15.5:6801/3125821798] conn(0x7f7c60038fc0 0x7f7c6003b480 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7c5c0098c0 tx=0x7f7c5c006db0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:08.338 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:07.564+0000 7f7c6b7fe640 1 -- 172.21.15.5:0/2993981130 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7c6404a030 con 0x7f7c74105fb0 2024-09-12T02:49:08.339 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:07.757+0000 7f7c7884d640 1 -- 172.21.15.5:0/2993981130 --> [v2:172.21.15.5:6800/3125821798,v1:172.21.15.5:6801/3125821798] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f7c74074c10 con 0x7f7c60038fc0 2024-09-12T02:49:08.339 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:07.767+0000 7f7c6b7fe640 1 -- 172.21.15.5:0/2993981130 <== mgr.14116 v2:172.21.15.5:6800/3125821798 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f7c74074c10 con 0x7f7c60038fc0 2024-09-12T02:49:08.339 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:07.769+0000 7f7c7884d640 1 -- 172.21.15.5:0/2993981130 >> [v2:172.21.15.5:6800/3125821798,v1:172.21.15.5:6801/3125821798] conn(0x7f7c60038fc0 msgr2=0x7f7c6003b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:08.339 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:07.769+0000 7f7c7884d640 1 --2- 172.21.15.5:0/2993981130 >> [v2:172.21.15.5:6800/3125821798,v1:172.21.15.5:6801/3125821798] conn(0x7f7c60038fc0 0x7f7c6003b480 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7c5c0098c0 tx=0x7f7c5c006db0 comp rx=0 tx=0).stop 2024-09-12T02:49:08.339 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:07.769+0000 7f7c7884d640 1 -- 172.21.15.5:0/2993981130 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c74105fb0 msgr2=0x7f7c7419d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:08.339 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:07.769+0000 7f7c7884d640 1 --2- 172.21.15.5:0/2993981130 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c74105fb0 0x7f7c7419d8d0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f7c640097f0 tx=0x7f7c64030d80 comp rx=0 tx=0).stop 2024-09-12T02:49:08.339 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:07.769+0000 7f7c7884d640 1 -- 172.21.15.5:0/2993981130 shutdown_connections 2024-09-12T02:49:08.339 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:07.769+0000 7f7c7884d640 1 --2- 172.21.15.5:0/2993981130 >> [v2:172.21.15.5:6800/3125821798,v1:172.21.15.5:6801/3125821798] conn(0x7f7c60038fc0 0x7f7c6003b480 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:08.339 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:07.769+0000 7f7c7884d640 1 --2- 172.21.15.5:0/2993981130 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c74105fb0 0x7f7c7419d8d0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:08.339 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:07.769+0000 7f7c7884d640 1 -- 172.21.15.5:0/2993981130 >> 172.21.15.5:0/2993981130 conn(0x7f7c74101e20 msgr2=0x7f7c74191c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:08.339 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:07.769+0000 7f7c7884d640 1 -- 172.21.15.5:0/2993981130 shutdown_connections 2024-09-12T02:49:08.339 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:07.769+0000 7f7c7884d640 1 -- 172.21.15.5:0/2993981130 wait complete. 2024-09-12T02:49:09.301 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:09 smithi005 ceph-mon[31197]: [12/Sep/2024:02:49:07] ENGINE Bus STARTING 2024-09-12T02:49:09.301 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:09 smithi005 ceph-mon[31197]: from='client.14128 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:49:09.301 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:09 smithi005 ceph-mon[31197]: [12/Sep/2024:02:49:07] ENGINE Serving on https://172.21.15.5:7150 2024-09-12T02:49:09.301 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:09 smithi005 ceph-mon[31197]: [12/Sep/2024:02:49:07] ENGINE Error in HTTPServer.serve 2024-09-12T02:49:09.301 INFO:journalctl@ceph.mon.a.smithi005.stdout: Traceback (most recent call last): 2024-09-12T02:49:09.301 INFO:journalctl@ceph.mon.a.smithi005.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-12T02:49:09.301 INFO:journalctl@ceph.mon.a.smithi005.stdout: self._connections.run(self.expiration_interval) 2024-09-12T02:49:09.302 INFO:journalctl@ceph.mon.a.smithi005.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-12T02:49:09.302 INFO:journalctl@ceph.mon.a.smithi005.stdout: self._run(expiration_interval) 2024-09-12T02:49:09.302 INFO:journalctl@ceph.mon.a.smithi005.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-12T02:49:09.302 INFO:journalctl@ceph.mon.a.smithi005.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-12T02:49:09.302 INFO:journalctl@ceph.mon.a.smithi005.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-12T02:49:09.302 INFO:journalctl@ceph.mon.a.smithi005.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-12T02:49:09.302 INFO:journalctl@ceph.mon.a.smithi005.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-12T02:49:09.302 INFO:journalctl@ceph.mon.a.smithi005.stdout: s = self.context.wrap_socket( 2024-09-12T02:49:09.302 INFO:journalctl@ceph.mon.a.smithi005.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-12T02:49:09.302 INFO:journalctl@ceph.mon.a.smithi005.stdout: return self.sslsocket_class._create( 2024-09-12T02:49:09.302 INFO:journalctl@ceph.mon.a.smithi005.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-12T02:49:09.302 INFO:journalctl@ceph.mon.a.smithi005.stdout: self.do_handshake() 2024-09-12T02:49:09.302 INFO:journalctl@ceph.mon.a.smithi005.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-12T02:49:09.303 INFO:journalctl@ceph.mon.a.smithi005.stdout: self._sslobj.do_handshake() 2024-09-12T02:49:09.303 INFO:journalctl@ceph.mon.a.smithi005.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-12T02:49:09.303 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:09 smithi005 ceph-mon[31197]: [12/Sep/2024:02:49:07] ENGINE Serving on http://172.21.15.5:8765 2024-09-12T02:49:09.303 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:09 smithi005 ceph-mon[31197]: [12/Sep/2024:02:49:07] ENGINE Bus STARTED 2024-09-12T02:49:09.566 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-12T02:49:09.567 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:08.727+0000 7ff2baf47640 1 Processor -- start 2024-09-12T02:49:09.567 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:08.728+0000 7ff2baf47640 1 -- start start 2024-09-12T02:49:09.567 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:08.728+0000 7ff2baf47640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2b41081d0 0x7ff2b410a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:09.567 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:08.728+0000 7ff2baf47640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b407bdb0 con 0x7ff2b41081d0 2024-09-12T02:49:09.567 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:08.728+0000 7ff2b8ce3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2b41081d0 0x7ff2b410a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:09.567 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:08.728+0000 7ff2b8ce3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2b41081d0 0x7ff2b410a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36634/0 (socket says 172.21.15.5:36634) 2024-09-12T02:49:09.567 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:08.728+0000 7ff2b8ce3640 1 -- 172.21.15.5:0/526465504 learned_addr learned my addr 172.21.15.5:0/526465504 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:49:09.567 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:08.728+0000 7ff2b8ce3640 1 -- 172.21.15.5:0/526465504 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2b410ab00 con 0x7ff2b41081d0 2024-09-12T02:49:09.567 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:08.729+0000 7ff2b8ce3640 1 --2- 172.21.15.5:0/526465504 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2b41081d0 0x7ff2b410a5c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ff2b0009820 tx=0x7ff2b002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e66f53c4140f4129 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:09.567 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:08.729+0000 7ff2ab7fe640 1 -- 172.21.15.5:0/526465504 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff2b0037070 con 0x7ff2b41081d0 2024-09-12T02:49:09.567 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:08.729+0000 7ff2ab7fe640 1 -- 172.21.15.5:0/526465504 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff2b0032900 con 0x7ff2b41081d0 2024-09-12T02:49:09.567 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:08.729+0000 7ff2ab7fe640 1 -- 172.21.15.5:0/526465504 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff2b0030850 con 0x7ff2b41081d0 2024-09-12T02:49:09.567 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:08.730+0000 7ff2baf47640 1 -- 172.21.15.5:0/526465504 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2b41081d0 msgr2=0x7ff2b410a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:09.568 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:08.730+0000 7ff2baf47640 1 --2- 172.21.15.5:0/526465504 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2b41081d0 0x7ff2b410a5c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ff2b0009820 tx=0x7ff2b002a270 comp rx=0 tx=0).stop 2024-09-12T02:49:09.568 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:08.730+0000 7ff2baf47640 1 -- 172.21.15.5:0/526465504 shutdown_connections 2024-09-12T02:49:09.568 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:08.730+0000 7ff2baf47640 1 --2- 172.21.15.5:0/526465504 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2b41081d0 0x7ff2b410a5c0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:09.568 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:08.730+0000 7ff2baf47640 1 -- 172.21.15.5:0/526465504 >> 172.21.15.5:0/526465504 conn(0x7ff2b4101d90 msgr2=0x7ff2b41041b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:09.568 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:08.730+0000 7ff2baf47640 1 -- 172.21.15.5:0/526465504 shutdown_connections 2024-09-12T02:49:09.568 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:08.730+0000 7ff2baf47640 1 -- 172.21.15.5:0/526465504 wait complete. 2024-09-12T02:49:09.568 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:08.730+0000 7ff2baf47640 1 Processor -- start 2024-09-12T02:49:09.568 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:08.730+0000 7ff2baf47640 1 -- start start 2024-09-12T02:49:09.568 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:08.731+0000 7ff2baf47640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2b41081d0 0x7ff2b419b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:09.568 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:08.731+0000 7ff2baf47640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b410bae0 con 0x7ff2b41081d0 2024-09-12T02:49:09.568 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:08.731+0000 7ff2b8ce3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2b41081d0 0x7ff2b419b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:09.568 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:08.731+0000 7ff2b8ce3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2b41081d0 0x7ff2b419b6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36636/0 (socket says 172.21.15.5:36636) 2024-09-12T02:49:09.569 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:08.731+0000 7ff2b8ce3640 1 -- 172.21.15.5:0/1503547807 learned_addr learned my addr 172.21.15.5:0/1503547807 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:49:09.569 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:08.731+0000 7ff2b8ce3640 1 -- 172.21.15.5:0/1503547807 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2b419bc30 con 0x7ff2b41081d0 2024-09-12T02:49:09.569 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:08.731+0000 7ff2b8ce3640 1 --2- 172.21.15.5:0/1503547807 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2b41081d0 0x7ff2b419b6f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ff2b00097f0 tx=0x7ff2b0030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:09.569 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:08.731+0000 7ff2a9ffb640 1 -- 172.21.15.5:0/1503547807 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff2b0037040 con 0x7ff2b41081d0 2024-09-12T02:49:09.569 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:08.731+0000 7ff2a9ffb640 1 -- 172.21.15.5:0/1503547807 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff2b0040070 con 0x7ff2b41081d0 2024-09-12T02:49:09.569 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:08.731+0000 7ff2baf47640 1 -- 172.21.15.5:0/1503547807 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2b419bec0 con 0x7ff2b41081d0 2024-09-12T02:49:09.569 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:08.731+0000 7ff2a9ffb640 1 -- 172.21.15.5:0/1503547807 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff2b003a390 con 0x7ff2b41081d0 2024-09-12T02:49:09.569 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:08.732+0000 7ff2baf47640 1 -- 172.21.15.5:0/1503547807 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2b419ebb0 con 0x7ff2b41081d0 2024-09-12T02:49:09.569 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:08.732+0000 7ff2a9ffb640 1 -- 172.21.15.5:0/1503547807 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7ff2b003a5e0 con 0x7ff2b41081d0 2024-09-12T02:49:09.569 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:08.733+0000 7ff2baf47640 1 -- 172.21.15.5:0/1503547807 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2b4108630 con 0x7ff2b41081d0 2024-09-12T02:49:09.569 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:08.733+0000 7ff2a9ffb640 1 --2- 172.21.15.5:0/1503547807 >> [v2:172.21.15.5:6800/3125821798,v1:172.21.15.5:6801/3125821798] conn(0x7ff29c038fc0 0x7ff29c03b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:09.569 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:08.733+0000 7ff2a9ffb640 1 -- 172.21.15.5:0/1503547807 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ff2b006c690 con 0x7ff2b41081d0 2024-09-12T02:49:09.569 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:08.733+0000 7ff2abfff640 1 --2- 172.21.15.5:0/1503547807 >> [v2:172.21.15.5:6800/3125821798,v1:172.21.15.5:6801/3125821798] conn(0x7ff29c038fc0 0x7ff29c03b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:09.570 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:08.733+0000 7ff2abfff640 1 --2- 172.21.15.5:0/1503547807 >> [v2:172.21.15.5:6800/3125821798,v1:172.21.15.5:6801/3125821798] conn(0x7ff29c038fc0 0x7ff29c03b480 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff2a40098c0 tx=0x7ff2a4006db0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:09.570 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:08.738+0000 7ff2a9ffb640 1 -- 172.21.15.5:0/1503547807 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff2b0043760 con 0x7ff2b41081d0 2024-09-12T02:49:09.570 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:08.916+0000 7ff2baf47640 1 -- 172.21.15.5:0/1503547807 --> [v2:172.21.15.5:6800/3125821798,v1:172.21.15.5:6801/3125821798] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7ff2b4103080 con 0x7ff29c038fc0 2024-09-12T02:49:09.570 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:08.917+0000 7ff2a9ffb640 1 -- 172.21.15.5:0/1503547807 <== mgr.14116 v2:172.21.15.5:6800/3125821798 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7ff2b4103080 con 0x7ff29c038fc0 2024-09-12T02:49:09.570 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:08.919+0000 7ff2baf47640 1 -- 172.21.15.5:0/1503547807 >> [v2:172.21.15.5:6800/3125821798,v1:172.21.15.5:6801/3125821798] conn(0x7ff29c038fc0 msgr2=0x7ff29c03b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:09.570 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:08.919+0000 7ff2baf47640 1 --2- 172.21.15.5:0/1503547807 >> [v2:172.21.15.5:6800/3125821798,v1:172.21.15.5:6801/3125821798] conn(0x7ff29c038fc0 0x7ff29c03b480 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff2a40098c0 tx=0x7ff2a4006db0 comp rx=0 tx=0).stop 2024-09-12T02:49:09.570 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:08.919+0000 7ff2baf47640 1 -- 172.21.15.5:0/1503547807 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2b41081d0 msgr2=0x7ff2b419b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:09.570 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:08.919+0000 7ff2baf47640 1 --2- 172.21.15.5:0/1503547807 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2b41081d0 0x7ff2b419b6f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ff2b00097f0 tx=0x7ff2b0030d80 comp rx=0 tx=0).stop 2024-09-12T02:49:09.570 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:08.919+0000 7ff2baf47640 1 -- 172.21.15.5:0/1503547807 shutdown_connections 2024-09-12T02:49:09.570 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:08.919+0000 7ff2baf47640 1 --2- 172.21.15.5:0/1503547807 >> [v2:172.21.15.5:6800/3125821798,v1:172.21.15.5:6801/3125821798] conn(0x7ff29c038fc0 0x7ff29c03b480 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:09.570 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:08.919+0000 7ff2baf47640 1 --2- 172.21.15.5:0/1503547807 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2b41081d0 0x7ff2b419b6f0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:09.570 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:08.919+0000 7ff2baf47640 1 -- 172.21.15.5:0/1503547807 >> 172.21.15.5:0/1503547807 conn(0x7ff2b4101d90 msgr2=0x7ff2b4102970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:09.570 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:08.919+0000 7ff2baf47640 1 -- 172.21.15.5:0/1503547807 shutdown_connections 2024-09-12T02:49:09.571 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:08.919+0000 7ff2baf47640 1 -- 172.21.15.5:0/1503547807 wait complete. 2024-09-12T02:49:09.571 INFO:teuthology.orchestra.run.smithi005.stdout:Generating ssh key... 2024-09-12T02:49:10.183 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:10 smithi005 ceph-mon[31197]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:49:10.806 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:49:10 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: Generating public/private rsa key pair. 2024-09-12T02:49:10.807 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:49:10 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: Your identification has been saved in /tmp/tmpushqngwc/key 2024-09-12T02:49:10.807 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:49:10 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: Your public key has been saved in /tmp/tmpushqngwc/key.pub 2024-09-12T02:49:10.807 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:49:10 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: The key fingerprint is: 2024-09-12T02:49:10.807 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:49:10 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: SHA256:66RlS0oXp1IoiQGvuT+P+1MKQ2UogPCUOcpciKeGeXM ceph-618305f8-70b1-11ef-bcea-c7b262605968 2024-09-12T02:49:10.807 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:49:10 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: The key's randomart image is: 2024-09-12T02:49:10.807 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:49:10 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: +---[RSA 3072]----+ 2024-09-12T02:49:10.807 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:49:10 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: |O o+ | 2024-09-12T02:49:10.807 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:49:10 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: |+**.o | 2024-09-12T02:49:10.807 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:49:10 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: |+*=+ | 2024-09-12T02:49:10.807 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:49:10 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: |=*+oE. . | 2024-09-12T02:49:10.807 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:49:10 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: |+o.oo . S . | 2024-09-12T02:49:10.808 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:49:10 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: | .o ... = | 2024-09-12T02:49:10.808 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:49:10 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: |. o oo O | 2024-09-12T02:49:10.808 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:49:10 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: | ...o. @ . | 2024-09-12T02:49:10.808 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:49:10 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: | +=o.o o | 2024-09-12T02:49:10.808 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:49:10 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: +----[SHA256]-----+ 2024-09-12T02:49:11.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:11 smithi005 ceph-mon[31197]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:49:11.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:11 smithi005 ceph-mon[31197]: Generating ssh key... 2024-09-12T02:49:11.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:11 smithi005 ceph-mon[31197]: from='mgr.14116 172.21.15.5:0/613993494' entity='mgr.x' 2024-09-12T02:49:11.057 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:11 smithi005 ceph-mon[31197]: from='mgr.14116 172.21.15.5:0/613993494' entity='mgr.x' 2024-09-12T02:49:11.065 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:09.996+0000 7f7e1a34f640 1 Processor -- start 2024-09-12T02:49:11.065 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:09.996+0000 7f7e1a34f640 1 -- start start 2024-09-12T02:49:11.065 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:09.997+0000 7f7e1a34f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e14106bc0 0x7f7e14106fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:11.065 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:09.997+0000 7f7e1a34f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7e14107590 con 0x7f7e14106bc0 2024-09-12T02:49:11.065 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:09.997+0000 7f7e13fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e14106bc0 0x7f7e14106fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:11.065 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:09.997+0000 7f7e13fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e14106bc0 0x7f7e14106fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36650/0 (socket says 172.21.15.5:36650) 2024-09-12T02:49:11.065 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:09.997+0000 7f7e13fff640 1 -- 172.21.15.5:0/2396744192 learned_addr learned my addr 172.21.15.5:0/2396744192 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:49:11.065 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:09.997+0000 7f7e13fff640 1 -- 172.21.15.5:0/2396744192 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e141078e0 con 0x7f7e14106bc0 2024-09-12T02:49:11.065 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:09.998+0000 7f7e13fff640 1 --2- 172.21.15.5:0/2396744192 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e14106bc0 0x7f7e14106fc0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7e04009820 tx=0x7f7e0402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc7e8ad1bda16bd6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:11.065 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:09.998+0000 7f7e137fe640 1 -- 172.21.15.5:0/2396744192 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7e04037070 con 0x7f7e14106bc0 2024-09-12T02:49:11.066 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:09.998+0000 7f7e137fe640 1 -- 172.21.15.5:0/2396744192 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7e04032900 con 0x7f7e14106bc0 2024-09-12T02:49:11.066 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:09.998+0000 7f7e137fe640 1 -- 172.21.15.5:0/2396744192 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7e04030850 con 0x7f7e14106bc0 2024-09-12T02:49:11.066 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:09.999+0000 7f7e1a34f640 1 -- 172.21.15.5:0/2396744192 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e14106bc0 msgr2=0x7f7e14106fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:11.066 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:09.999+0000 7f7e1a34f640 1 --2- 172.21.15.5:0/2396744192 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e14106bc0 0x7f7e14106fc0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7e04009820 tx=0x7f7e0402a270 comp rx=0 tx=0).stop 2024-09-12T02:49:11.066 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:09.999+0000 7f7e1a34f640 1 -- 172.21.15.5:0/2396744192 shutdown_connections 2024-09-12T02:49:11.066 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:09.999+0000 7f7e1a34f640 1 --2- 172.21.15.5:0/2396744192 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e14106bc0 0x7f7e14106fc0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:11.066 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:09.999+0000 7f7e1a34f640 1 -- 172.21.15.5:0/2396744192 >> 172.21.15.5:0/2396744192 conn(0x7f7e141015f0 msgr2=0x7f7e14103a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:11.066 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:09.999+0000 7f7e1a34f640 1 -- 172.21.15.5:0/2396744192 shutdown_connections 2024-09-12T02:49:11.066 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:09.999+0000 7f7e1a34f640 1 -- 172.21.15.5:0/2396744192 wait complete. 2024-09-12T02:49:11.066 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:09.999+0000 7f7e1a34f640 1 Processor -- start 2024-09-12T02:49:11.066 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:10.000+0000 7f7e1a34f640 1 -- start start 2024-09-12T02:49:11.066 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:10.000+0000 7f7e1a34f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e14106bc0 0x7f7e1419b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:11.066 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:10.000+0000 7f7e1a34f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7e1410ac90 con 0x7f7e14106bc0 2024-09-12T02:49:11.067 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:10.000+0000 7f7e13fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e14106bc0 0x7f7e1419b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:11.067 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:10.000+0000 7f7e13fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e14106bc0 0x7f7e1419b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36664/0 (socket says 172.21.15.5:36664) 2024-09-12T02:49:11.067 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:10.000+0000 7f7e13fff640 1 -- 172.21.15.5:0/1066784769 learned_addr learned my addr 172.21.15.5:0/1066784769 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:49:11.067 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:10.000+0000 7f7e13fff640 1 -- 172.21.15.5:0/1066784769 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e1419bc70 con 0x7f7e14106bc0 2024-09-12T02:49:11.067 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:10.000+0000 7f7e13fff640 1 --2- 172.21.15.5:0/1066784769 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e14106bc0 0x7f7e1419b730 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7e040097f0 tx=0x7f7e04030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:11.067 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:10.000+0000 7f7e11ffb640 1 -- 172.21.15.5:0/1066784769 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7e04037040 con 0x7f7e14106bc0 2024-09-12T02:49:11.067 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:10.001+0000 7f7e11ffb640 1 -- 172.21.15.5:0/1066784769 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7e04040070 con 0x7f7e14106bc0 2024-09-12T02:49:11.067 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:10.001+0000 7f7e1a34f640 1 -- 172.21.15.5:0/1066784769 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e1419bf00 con 0x7f7e14106bc0 2024-09-12T02:49:11.067 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:10.001+0000 7f7e11ffb640 1 -- 172.21.15.5:0/1066784769 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7e0403a390 con 0x7f7e14106bc0 2024-09-12T02:49:11.067 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:10.001+0000 7f7e1a34f640 1 -- 172.21.15.5:0/1066784769 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e1419ebf0 con 0x7f7e14106bc0 2024-09-12T02:49:11.068 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:10.002+0000 7f7e11ffb640 1 -- 172.21.15.5:0/1066784769 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f7e04032900 con 0x7f7e14106bc0 2024-09-12T02:49:11.068 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:10.002+0000 7f7e1a34f640 1 -- 172.21.15.5:0/1066784769 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e14107040 con 0x7f7e14106bc0 2024-09-12T02:49:11.068 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:10.002+0000 7f7e11ffb640 1 --2- 172.21.15.5:0/1066784769 >> [v2:172.21.15.5:6800/3125821798,v1:172.21.15.5:6801/3125821798] conn(0x7f7e00038fc0 0x7f7e0003b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:11.068 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:10.002+0000 7f7e11ffb640 1 -- 172.21.15.5:0/1066784769 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f7e0406c4c0 con 0x7f7e14106bc0 2024-09-12T02:49:11.068 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:10.002+0000 7f7e0bfff640 1 --2- 172.21.15.5:0/1066784769 >> [v2:172.21.15.5:6800/3125821798,v1:172.21.15.5:6801/3125821798] conn(0x7f7e00038fc0 0x7f7e0003b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:11.068 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:10.002+0000 7f7e0bfff640 1 --2- 172.21.15.5:0/1066784769 >> [v2:172.21.15.5:6800/3125821798,v1:172.21.15.5:6801/3125821798] conn(0x7f7e00038fc0 0x7f7e0003b480 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc0098c0 tx=0x7f7dfc006db0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:11.068 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:10.007+0000 7f7e11ffb640 1 -- 172.21.15.5:0/1066784769 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7e0406be50 con 0x7f7e14106bc0 2024-09-12T02:49:11.068 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:10.183+0000 7f7e1a34f640 1 -- 172.21.15.5:0/1066784769 --> [v2:172.21.15.5:6800/3125821798,v1:172.21.15.5:6801/3125821798] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f7e1402db70 con 0x7f7e00038fc0 2024-09-12T02:49:11.068 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:10.507+0000 7f7e11ffb640 1 -- 172.21.15.5:0/1066784769 <== mgr.14116 v2:172.21.15.5:6800/3125821798 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f7e1402db70 con 0x7f7e00038fc0 2024-09-12T02:49:11.068 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:10.508+0000 7f7e1a34f640 1 -- 172.21.15.5:0/1066784769 >> [v2:172.21.15.5:6800/3125821798,v1:172.21.15.5:6801/3125821798] conn(0x7f7e00038fc0 msgr2=0x7f7e0003b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:11.068 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:10.508+0000 7f7e1a34f640 1 --2- 172.21.15.5:0/1066784769 >> [v2:172.21.15.5:6800/3125821798,v1:172.21.15.5:6801/3125821798] conn(0x7f7e00038fc0 0x7f7e0003b480 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc0098c0 tx=0x7f7dfc006db0 comp rx=0 tx=0).stop 2024-09-12T02:49:11.068 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:10.508+0000 7f7e1a34f640 1 -- 172.21.15.5:0/1066784769 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e14106bc0 msgr2=0x7f7e1419b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:11.069 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:10.508+0000 7f7e1a34f640 1 --2- 172.21.15.5:0/1066784769 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e14106bc0 0x7f7e1419b730 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7e040097f0 tx=0x7f7e04030d80 comp rx=0 tx=0).stop 2024-09-12T02:49:11.069 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:10.509+0000 7f7e1a34f640 1 -- 172.21.15.5:0/1066784769 shutdown_connections 2024-09-12T02:49:11.069 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:10.509+0000 7f7e1a34f640 1 --2- 172.21.15.5:0/1066784769 >> [v2:172.21.15.5:6800/3125821798,v1:172.21.15.5:6801/3125821798] conn(0x7f7e00038fc0 0x7f7e0003b480 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:11.070 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:10.509+0000 7f7e1a34f640 1 --2- 172.21.15.5:0/1066784769 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e14106bc0 0x7f7e1419b730 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:11.070 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:10.509+0000 7f7e1a34f640 1 -- 172.21.15.5:0/1066784769 >> 172.21.15.5:0/1066784769 conn(0x7f7e141015f0 msgr2=0x7f7e141024e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:11.070 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:10.509+0000 7f7e1a34f640 1 -- 172.21.15.5:0/1066784769 shutdown_connections 2024-09-12T02:49:11.070 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:10.509+0000 7f7e1a34f640 1 -- 172.21.15.5:0/1066784769 wait complete. 2024-09-12T02:49:12.204 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCkO7uq7a+tQ6DHnIxiHhyCltvl6iy/AWOzZGgRZdA2BT0+s8CNG2ITuTQLE4EbGMp8R39OwF6zRuJy9ltDSStD/i3e9DM7kj8RtZe586zqM2rsII3pfUdhYaNvUslnrIw9nZVBaNTvX8UHZ3+SK8ODUOrRJBuRnNHYrdKIRx8iyQEbxetAcxKEJjnr/atSKc1sGd0YgztITWr+J9SKlu0wxQ/Cel+I99Zz/oX4ACpiKeyG5uW1LzRLi3EUtT7ReVp7mCo9OE6Wt0RWefhNBnOnX2ejSwF8gaKVgEsmFaSXBuFeq+TczOninzrVwzXkHNqVHOnKIsNNROzwnAn85oA1Cf0iI2bETVmj76Fcwtuvy0PBmQJbGt76jjNfOXSYN8cVEEMC7v3ybTiKmr2SYWkmsm7YjohjnOlsmSMFwgjlZwWngYv8abwdMG9+vUrPPlAiYqW8UJLgvgsgjoJTS8vGtJkMU4w248YmcGSjLV2jK4aaFlab571vnWwwh33wnjc= ceph-618305f8-70b1-11ef-bcea-c7b262605968 2024-09-12T02:49:12.204 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:11.426+0000 7f9c6d11c640 1 Processor -- start 2024-09-12T02:49:12.204 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:11.426+0000 7f9c6d11c640 1 -- start start 2024-09-12T02:49:12.204 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:11.426+0000 7f9c6d11c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c68107780 0x7f9c68107b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:12.204 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:11.426+0000 7f9c6d11c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9c68108150 con 0x7f9c68107780 2024-09-12T02:49:12.204 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:11.426+0000 7f9c66d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c68107780 0x7f9c68107b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:12.204 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:11.426+0000 7f9c66d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c68107780 0x7f9c68107b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36680/0 (socket says 172.21.15.5:36680) 2024-09-12T02:49:12.205 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:11.426+0000 7f9c66d9d640 1 -- 172.21.15.5:0/639617453 learned_addr learned my addr 172.21.15.5:0/639617453 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:49:12.205 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:11.427+0000 7f9c66d9d640 1 -- 172.21.15.5:0/639617453 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c68108980 con 0x7f9c68107780 2024-09-12T02:49:12.205 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:11.427+0000 7f9c66d9d640 1 --2- 172.21.15.5:0/639617453 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c68107780 0x7f9c68107b80 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9c58009820 tx=0x7f9c5802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b3905300c236d73 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:12.205 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:11.428+0000 7f9c6659c640 1 -- 172.21.15.5:0/639617453 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9c58037070 con 0x7f9c68107780 2024-09-12T02:49:12.205 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:11.428+0000 7f9c6659c640 1 -- 172.21.15.5:0/639617453 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9c58032900 con 0x7f9c68107780 2024-09-12T02:49:12.205 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:11.428+0000 7f9c6659c640 1 -- 172.21.15.5:0/639617453 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9c58030850 con 0x7f9c68107780 2024-09-12T02:49:12.205 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:11.428+0000 7f9c6d11c640 1 -- 172.21.15.5:0/639617453 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c68107780 msgr2=0x7f9c68107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:12.205 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:11.428+0000 7f9c6d11c640 1 --2- 172.21.15.5:0/639617453 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c68107780 0x7f9c68107b80 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9c58009820 tx=0x7f9c5802a270 comp rx=0 tx=0).stop 2024-09-12T02:49:12.205 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:11.428+0000 7f9c6d11c640 1 -- 172.21.15.5:0/639617453 shutdown_connections 2024-09-12T02:49:12.205 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:11.428+0000 7f9c6d11c640 1 --2- 172.21.15.5:0/639617453 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c68107780 0x7f9c68107b80 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:12.206 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:11.428+0000 7f9c6d11c640 1 -- 172.21.15.5:0/639617453 >> 172.21.15.5:0/639617453 conn(0x7f9c68102f30 msgr2=0x7f9c68105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:12.206 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:11.428+0000 7f9c6d11c640 1 -- 172.21.15.5:0/639617453 shutdown_connections 2024-09-12T02:49:12.206 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:11.428+0000 7f9c6d11c640 1 -- 172.21.15.5:0/639617453 wait complete. 2024-09-12T02:49:12.206 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:11.429+0000 7f9c6d11c640 1 Processor -- start 2024-09-12T02:49:12.206 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:11.429+0000 7f9c6d11c640 1 -- start start 2024-09-12T02:49:12.206 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:11.429+0000 7f9c6d11c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c68107780 0x7f9c6807daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:12.206 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:11.429+0000 7f9c6d11c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9c6810bd30 con 0x7f9c68107780 2024-09-12T02:49:12.206 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:11.430+0000 7f9c66d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c68107780 0x7f9c6807daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:12.206 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:11.430+0000 7f9c66d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c68107780 0x7f9c6807daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36694/0 (socket says 172.21.15.5:36694) 2024-09-12T02:49:12.206 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:11.430+0000 7f9c66d9d640 1 -- 172.21.15.5:0/784366339 learned_addr learned my addr 172.21.15.5:0/784366339 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:49:12.206 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:11.430+0000 7f9c66d9d640 1 -- 172.21.15.5:0/784366339 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c6807e030 con 0x7f9c68107780 2024-09-12T02:49:12.206 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:11.430+0000 7f9c66d9d640 1 --2- 172.21.15.5:0/784366339 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c68107780 0x7f9c6807daf0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9c58009940 tx=0x7f9c58032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:12.206 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:11.430+0000 7f9c64d99640 1 -- 172.21.15.5:0/784366339 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9c58042070 con 0x7f9c68107780 2024-09-12T02:49:12.207 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:11.430+0000 7f9c6d11c640 1 -- 172.21.15.5:0/784366339 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c6807c220 con 0x7f9c68107780 2024-09-12T02:49:12.207 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:11.430+0000 7f9c64d99640 1 -- 172.21.15.5:0/784366339 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9c580308c0 con 0x7f9c68107780 2024-09-12T02:49:12.207 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:11.430+0000 7f9c64d99640 1 -- 172.21.15.5:0/784366339 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9c58037040 con 0x7f9c68107780 2024-09-12T02:49:12.207 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:11.430+0000 7f9c6d11c640 1 -- 172.21.15.5:0/784366339 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c6807c760 con 0x7f9c68107780 2024-09-12T02:49:12.207 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:11.431+0000 7f9c64d99640 1 -- 172.21.15.5:0/784366339 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f9c58045070 con 0x7f9c68107780 2024-09-12T02:49:12.207 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:11.431+0000 7f9c64d99640 1 --2- 172.21.15.5:0/784366339 >> [v2:172.21.15.5:6800/3125821798,v1:172.21.15.5:6801/3125821798] conn(0x7f9c54038f70 0x7f9c5403b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:12.207 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:11.431+0000 7f9c6d11c640 1 -- 172.21.15.5:0/784366339 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c68107c00 con 0x7f9c68107780 2024-09-12T02:49:12.207 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:11.431+0000 7f9c64d99640 1 -- 172.21.15.5:0/784366339 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f9c5806c060 con 0x7f9c68107780 2024-09-12T02:49:12.207 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:11.431+0000 7f9c5ffff640 1 --2- 172.21.15.5:0/784366339 >> [v2:172.21.15.5:6800/3125821798,v1:172.21.15.5:6801/3125821798] conn(0x7f9c54038f70 0x7f9c5403b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:12.208 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:11.432+0000 7f9c5ffff640 1 --2- 172.21.15.5:0/784366339 >> [v2:172.21.15.5:6800/3125821798,v1:172.21.15.5:6801/3125821798] conn(0x7f9c54038f70 0x7f9c5403b430 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9c500098c0 tx=0x7f9c50006db0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:12.208 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:11.436+0000 7f9c64d99640 1 -- 172.21.15.5:0/784366339 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9c58045520 con 0x7f9c68107780 2024-09-12T02:49:12.208 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:11.615+0000 7f9c6d11c640 1 -- 172.21.15.5:0/784366339 --> [v2:172.21.15.5:6800/3125821798,v1:172.21.15.5:6801/3125821798] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f9c6807cfc0 con 0x7f9c54038f70 2024-09-12T02:49:12.208 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:11.616+0000 7f9c64d99640 1 -- 172.21.15.5:0/784366339 <== mgr.14116 v2:172.21.15.5:6800/3125821798 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f9c6807cfc0 con 0x7f9c54038f70 2024-09-12T02:49:12.208 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:11.618+0000 7f9c6d11c640 1 -- 172.21.15.5:0/784366339 >> [v2:172.21.15.5:6800/3125821798,v1:172.21.15.5:6801/3125821798] conn(0x7f9c54038f70 msgr2=0x7f9c5403b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:12.208 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:11.618+0000 7f9c6d11c640 1 --2- 172.21.15.5:0/784366339 >> [v2:172.21.15.5:6800/3125821798,v1:172.21.15.5:6801/3125821798] conn(0x7f9c54038f70 0x7f9c5403b430 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9c500098c0 tx=0x7f9c50006db0 comp rx=0 tx=0).stop 2024-09-12T02:49:12.208 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:11.618+0000 7f9c6d11c640 1 -- 172.21.15.5:0/784366339 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c68107780 msgr2=0x7f9c6807daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:12.208 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:11.618+0000 7f9c6d11c640 1 --2- 172.21.15.5:0/784366339 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c68107780 0x7f9c6807daf0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9c58009940 tx=0x7f9c58032ce0 comp rx=0 tx=0).stop 2024-09-12T02:49:12.208 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:11.619+0000 7f9c6d11c640 1 -- 172.21.15.5:0/784366339 shutdown_connections 2024-09-12T02:49:12.208 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:11.619+0000 7f9c6d11c640 1 --2- 172.21.15.5:0/784366339 >> [v2:172.21.15.5:6800/3125821798,v1:172.21.15.5:6801/3125821798] conn(0x7f9c54038f70 0x7f9c5403b430 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:12.208 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:11.619+0000 7f9c6d11c640 1 --2- 172.21.15.5:0/784366339 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9c68107780 0x7f9c6807daf0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:12.208 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:11.619+0000 7f9c6d11c640 1 -- 172.21.15.5:0/784366339 >> 172.21.15.5:0/784366339 conn(0x7f9c68102f30 msgr2=0x7f9c68103aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:12.208 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:11.619+0000 7f9c6d11c640 1 -- 172.21.15.5:0/784366339 shutdown_connections 2024-09-12T02:49:12.209 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:11.619+0000 7f9c6d11c640 1 -- 172.21.15.5:0/784366339 wait complete. 2024-09-12T02:49:12.209 INFO:teuthology.orchestra.run.smithi005.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-12T02:49:12.209 INFO:teuthology.orchestra.run.smithi005.stdout:Adding key to root@localhost authorized_keys... 2024-09-12T02:49:12.211 INFO:teuthology.orchestra.run.smithi005.stdout:Adding host smithi005... 2024-09-12T02:49:12.532 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:12 smithi005 ceph-mon[31197]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:49:13.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:13 smithi005 ceph-mon[31197]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi005", "addr": "172.21.15.5", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:49:14.466 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:14 smithi005 ceph-mon[31197]: Deploying cephadm binary to smithi005 2024-09-12T02:49:17.341 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout Added host 'smithi005' with addr '172.21.15.5' 2024-09-12T02:49:17.341 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:12.617+0000 7fce13e8e640 1 Processor -- start 2024-09-12T02:49:17.341 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:12.617+0000 7fce13e8e640 1 -- start start 2024-09-12T02:49:17.341 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:12.617+0000 7fce13e8e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fce0c107700 0x7fce0c107b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:17.341 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:12.617+0000 7fce13e8e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fce0c1080d0 con 0x7fce0c107700 2024-09-12T02:49:17.341 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:12.617+0000 7fce11c2a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fce0c107700 0x7fce0c107b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:17.342 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:12.617+0000 7fce11c2a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fce0c107700 0x7fce0c107b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36696/0 (socket says 172.21.15.5:36696) 2024-09-12T02:49:17.342 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:12.617+0000 7fce11c2a640 1 -- 172.21.15.5:0/1251520839 learned_addr learned my addr 172.21.15.5:0/1251520839 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:49:17.342 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:12.618+0000 7fce11c2a640 1 -- 172.21.15.5:0/1251520839 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce0c108900 con 0x7fce0c107700 2024-09-12T02:49:17.342 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:12.618+0000 7fce11c2a640 1 --2- 172.21.15.5:0/1251520839 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fce0c107700 0x7fce0c107b00 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fce08009820 tx=0x7fce0802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be47823898519fd9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:17.342 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:12.618+0000 7fce10c28640 1 -- 172.21.15.5:0/1251520839 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fce08037070 con 0x7fce0c107700 2024-09-12T02:49:17.342 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:12.618+0000 7fce10c28640 1 -- 172.21.15.5:0/1251520839 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fce08032900 con 0x7fce0c107700 2024-09-12T02:49:17.342 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:12.618+0000 7fce10c28640 1 -- 172.21.15.5:0/1251520839 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fce08030850 con 0x7fce0c107700 2024-09-12T02:49:17.342 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:12.619+0000 7fce13e8e640 1 -- 172.21.15.5:0/1251520839 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fce0c107700 msgr2=0x7fce0c107b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:17.342 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:12.619+0000 7fce13e8e640 1 --2- 172.21.15.5:0/1251520839 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fce0c107700 0x7fce0c107b00 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fce08009820 tx=0x7fce0802a270 comp rx=0 tx=0).stop 2024-09-12T02:49:17.342 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:12.619+0000 7fce13e8e640 1 -- 172.21.15.5:0/1251520839 shutdown_connections 2024-09-12T02:49:17.342 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:12.619+0000 7fce13e8e640 1 --2- 172.21.15.5:0/1251520839 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fce0c107700 0x7fce0c107b00 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:17.342 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:12.619+0000 7fce13e8e640 1 -- 172.21.15.5:0/1251520839 >> 172.21.15.5:0/1251520839 conn(0x7fce0c102f30 msgr2=0x7fce0c105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:17.342 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:12.619+0000 7fce13e8e640 1 -- 172.21.15.5:0/1251520839 shutdown_connections 2024-09-12T02:49:17.343 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:12.619+0000 7fce13e8e640 1 -- 172.21.15.5:0/1251520839 wait complete. 2024-09-12T02:49:17.343 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:12.620+0000 7fce13e8e640 1 Processor -- start 2024-09-12T02:49:17.343 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:12.620+0000 7fce13e8e640 1 -- start start 2024-09-12T02:49:17.343 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:12.620+0000 7fce13e8e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fce0c107700 0x7fce0c19b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:17.343 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:12.620+0000 7fce13e8e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fce0c10bcb0 con 0x7fce0c107700 2024-09-12T02:49:17.343 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:12.620+0000 7fce11c2a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fce0c107700 0x7fce0c19b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:17.343 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:12.620+0000 7fce11c2a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fce0c107700 0x7fce0c19b6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36706/0 (socket says 172.21.15.5:36706) 2024-09-12T02:49:17.343 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:12.620+0000 7fce11c2a640 1 -- 172.21.15.5:0/2069152300 learned_addr learned my addr 172.21.15.5:0/2069152300 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:49:17.343 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:12.620+0000 7fce11c2a640 1 -- 172.21.15.5:0/2069152300 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce0c19bc30 con 0x7fce0c107700 2024-09-12T02:49:17.344 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:12.621+0000 7fce11c2a640 1 --2- 172.21.15.5:0/2069152300 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fce0c107700 0x7fce0c19b6f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fce08009940 tx=0x7fce08032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:17.344 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:12.621+0000 7fce02ffd640 1 -- 172.21.15.5:0/2069152300 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fce08042070 con 0x7fce0c107700 2024-09-12T02:49:17.344 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:12.621+0000 7fce02ffd640 1 -- 172.21.15.5:0/2069152300 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fce080308c0 con 0x7fce0c107700 2024-09-12T02:49:17.344 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:12.621+0000 7fce13e8e640 1 -- 172.21.15.5:0/2069152300 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce0c19bec0 con 0x7fce0c107700 2024-09-12T02:49:17.344 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:12.621+0000 7fce02ffd640 1 -- 172.21.15.5:0/2069152300 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fce08037040 con 0x7fce0c107700 2024-09-12T02:49:17.344 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:12.621+0000 7fce13e8e640 1 -- 172.21.15.5:0/2069152300 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce0c19ebb0 con 0x7fce0c107700 2024-09-12T02:49:17.344 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:12.622+0000 7fce02ffd640 1 -- 172.21.15.5:0/2069152300 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fce08045070 con 0x7fce0c107700 2024-09-12T02:49:17.344 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:12.622+0000 7fce02ffd640 1 --2- 172.21.15.5:0/2069152300 >> [v2:172.21.15.5:6800/3125821798,v1:172.21.15.5:6801/3125821798] conn(0x7fcdf8038fc0 0x7fcdf803b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:17.344 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:12.622+0000 7fce02ffd640 1 -- 172.21.15.5:0/2069152300 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fce0806c050 con 0x7fce0c107700 2024-09-12T02:49:17.344 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:12.622+0000 7fce11429640 1 --2- 172.21.15.5:0/2069152300 >> [v2:172.21.15.5:6800/3125821798,v1:172.21.15.5:6801/3125821798] conn(0x7fcdf8038fc0 0x7fcdf803b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:17.344 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:12.622+0000 7fce13e8e640 1 -- 172.21.15.5:0/2069152300 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce0c10be30 con 0x7fce0c107700 2024-09-12T02:49:17.344 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:12.622+0000 7fce11429640 1 --2- 172.21.15.5:0/2069152300 >> [v2:172.21.15.5:6800/3125821798,v1:172.21.15.5:6801/3125821798] conn(0x7fcdf8038fc0 0x7fcdf803b480 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc0098c0 tx=0x7fcdfc006db0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:17.344 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:12.627+0000 7fce02ffd640 1 -- 172.21.15.5:0/2069152300 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fce0804b170 con 0x7fce0c107700 2024-09-12T02:49:17.345 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:12.808+0000 7fce13e8e640 1 -- 172.21.15.5:0/2069152300 --> [v2:172.21.15.5:6800/3125821798,v1:172.21.15.5:6801/3125821798] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi005", "addr": "172.21.15.5", "target": ["mon-mgr", ""]}) -- 0x7fce0c104160 con 0x7fcdf8038fc0 2024-09-12T02:49:17.345 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:16.756+0000 7fce02ffd640 1 -- 172.21.15.5:0/2069152300 <== mgr.14116 v2:172.21.15.5:6800/3125821798 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+47 (secure 0 0 0) 0x7fce0c104160 con 0x7fcdf8038fc0 2024-09-12T02:49:17.345 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:16.759+0000 7fce13e8e640 1 -- 172.21.15.5:0/2069152300 >> [v2:172.21.15.5:6800/3125821798,v1:172.21.15.5:6801/3125821798] conn(0x7fcdf8038fc0 msgr2=0x7fcdf803b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:17.345 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:16.759+0000 7fce13e8e640 1 --2- 172.21.15.5:0/2069152300 >> [v2:172.21.15.5:6800/3125821798,v1:172.21.15.5:6801/3125821798] conn(0x7fcdf8038fc0 0x7fcdf803b480 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc0098c0 tx=0x7fcdfc006db0 comp rx=0 tx=0).stop 2024-09-12T02:49:17.345 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:16.759+0000 7fce13e8e640 1 -- 172.21.15.5:0/2069152300 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fce0c107700 msgr2=0x7fce0c19b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:17.345 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:16.759+0000 7fce13e8e640 1 --2- 172.21.15.5:0/2069152300 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fce0c107700 0x7fce0c19b6f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fce08009940 tx=0x7fce08032ce0 comp rx=0 tx=0).stop 2024-09-12T02:49:17.345 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:16.759+0000 7fce13e8e640 1 -- 172.21.15.5:0/2069152300 shutdown_connections 2024-09-12T02:49:17.345 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:16.759+0000 7fce13e8e640 1 --2- 172.21.15.5:0/2069152300 >> [v2:172.21.15.5:6800/3125821798,v1:172.21.15.5:6801/3125821798] conn(0x7fcdf8038fc0 0x7fcdf803b480 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:17.345 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:16.759+0000 7fce13e8e640 1 --2- 172.21.15.5:0/2069152300 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fce0c107700 0x7fce0c19b6f0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:17.345 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:16.759+0000 7fce13e8e640 1 -- 172.21.15.5:0/2069152300 >> 172.21.15.5:0/2069152300 conn(0x7fce0c102f30 msgr2=0x7fce0c103a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:17.345 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:16.759+0000 7fce13e8e640 1 -- 172.21.15.5:0/2069152300 shutdown_connections 2024-09-12T02:49:17.345 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:16.759+0000 7fce13e8e640 1 -- 172.21.15.5:0/2069152300 wait complete. 2024-09-12T02:49:17.345 INFO:teuthology.orchestra.run.smithi005.stdout:Deploying unmanaged mon service... 2024-09-12T02:49:17.929 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:17 smithi005 ceph-mon[31197]: from='mgr.14116 172.21.15.5:0/613993494' entity='mgr.x' 2024-09-12T02:49:17.929 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:17 smithi005 ceph-mon[31197]: Added host smithi005 2024-09-12T02:49:17.929 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:17 smithi005 ceph-mon[31197]: from='mgr.14116 172.21.15.5:0/613993494' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:49:18.494 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-12T02:49:18.495 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:17.714+0000 7f68c9625640 1 Processor -- start 2024-09-12T02:49:18.495 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:17.714+0000 7f68c9625640 1 -- start start 2024-09-12T02:49:18.495 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:17.714+0000 7f68c9625640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68c4107700 0x7f68c4107b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:18.495 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:17.714+0000 7f68c9625640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f68c41080d0 con 0x7f68c4107700 2024-09-12T02:49:18.495 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:17.714+0000 7f68c2ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68c4107700 0x7f68c4107b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:18.495 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:17.714+0000 7f68c2ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68c4107700 0x7f68c4107b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33076/0 (socket says 172.21.15.5:33076) 2024-09-12T02:49:18.495 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:17.714+0000 7f68c2ffd640 1 -- 172.21.15.5:0/1294871502 learned_addr learned my addr 172.21.15.5:0/1294871502 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:49:18.495 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:17.715+0000 7f68c2ffd640 1 -- 172.21.15.5:0/1294871502 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68c4108900 con 0x7f68c4107700 2024-09-12T02:49:18.495 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:17.716+0000 7f68c2ffd640 1 --2- 172.21.15.5:0/1294871502 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68c4107700 0x7f68c4107b00 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f68b4009e80 tx=0x7f68b402a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4379c9b4d36cf77c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:18.495 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:17.716+0000 7f68c1ffb640 1 -- 172.21.15.5:0/1294871502 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f68b4036070 con 0x7f68c4107700 2024-09-12T02:49:18.496 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:17.716+0000 7f68c1ffb640 1 -- 172.21.15.5:0/1294871502 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f68b40318c0 con 0x7f68c4107700 2024-09-12T02:49:18.496 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:17.716+0000 7f68c1ffb640 1 -- 172.21.15.5:0/1294871502 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f68b4007ab0 con 0x7f68c4107700 2024-09-12T02:49:18.496 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:17.716+0000 7f68c9625640 1 -- 172.21.15.5:0/1294871502 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68c4107700 msgr2=0x7f68c4107b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:18.496 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:17.716+0000 7f68c9625640 1 --2- 172.21.15.5:0/1294871502 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68c4107700 0x7f68c4107b00 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f68b4009e80 tx=0x7f68b402a7a0 comp rx=0 tx=0).stop 2024-09-12T02:49:18.496 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:17.716+0000 7f68c9625640 1 -- 172.21.15.5:0/1294871502 shutdown_connections 2024-09-12T02:49:18.496 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:17.716+0000 7f68c9625640 1 --2- 172.21.15.5:0/1294871502 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68c4107700 0x7f68c4107b00 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:18.496 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:17.716+0000 7f68c9625640 1 -- 172.21.15.5:0/1294871502 >> 172.21.15.5:0/1294871502 conn(0x7f68c4102f30 msgr2=0x7f68c4105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:18.496 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:17.717+0000 7f68c9625640 1 -- 172.21.15.5:0/1294871502 shutdown_connections 2024-09-12T02:49:18.496 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:17.717+0000 7f68c9625640 1 -- 172.21.15.5:0/1294871502 wait complete. 2024-09-12T02:49:18.496 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:17.717+0000 7f68c9625640 1 Processor -- start 2024-09-12T02:49:18.496 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:17.717+0000 7f68c9625640 1 -- start start 2024-09-12T02:49:18.496 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:17.717+0000 7f68c9625640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68c4107700 0x7f68c419b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:18.497 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:17.717+0000 7f68c9625640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f68c410bcb0 con 0x7f68c4107700 2024-09-12T02:49:18.497 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:17.718+0000 7f68c2ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68c4107700 0x7f68c419b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:18.497 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:17.718+0000 7f68c2ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68c4107700 0x7f68c419b6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33086/0 (socket says 172.21.15.5:33086) 2024-09-12T02:49:18.497 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:17.718+0000 7f68c2ffd640 1 -- 172.21.15.5:0/2058719502 learned_addr learned my addr 172.21.15.5:0/2058719502 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:49:18.497 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:17.718+0000 7f68c2ffd640 1 -- 172.21.15.5:0/2058719502 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68c419bc30 con 0x7f68c4107700 2024-09-12T02:49:18.497 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:17.718+0000 7f68c2ffd640 1 --2- 172.21.15.5:0/2058719502 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68c4107700 0x7f68c419b6f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f68b402acc0 tx=0x7f68b4031ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:18.497 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:17.718+0000 7f68abfff640 1 -- 172.21.15.5:0/2058719502 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f68b4036050 con 0x7f68c4107700 2024-09-12T02:49:18.497 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:17.718+0000 7f68abfff640 1 -- 172.21.15.5:0/2058719502 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f68b4037040 con 0x7f68c4107700 2024-09-12T02:49:18.497 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:17.718+0000 7f68c9625640 1 -- 172.21.15.5:0/2058719502 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68c419bec0 con 0x7f68c4107700 2024-09-12T02:49:18.497 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:17.718+0000 7f68abfff640 1 -- 172.21.15.5:0/2058719502 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f68b403d8c0 con 0x7f68c4107700 2024-09-12T02:49:18.497 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:17.718+0000 7f68c9625640 1 -- 172.21.15.5:0/2058719502 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68c419ebb0 con 0x7f68c4107700 2024-09-12T02:49:18.497 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:17.719+0000 7f68abfff640 1 -- 172.21.15.5:0/2058719502 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f68b4044020 con 0x7f68c4107700 2024-09-12T02:49:18.498 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:17.719+0000 7f68abfff640 1 --2- 172.21.15.5:0/2058719502 >> [v2:172.21.15.5:6800/3125821798,v1:172.21.15.5:6801/3125821798] conn(0x7f68ac038fc0 0x7f68ac03b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:18.498 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:17.719+0000 7f68abfff640 1 -- 172.21.15.5:0/2058719502 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f68b4042020 con 0x7f68c4107700 2024-09-12T02:49:18.498 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:17.719+0000 7f68c9625640 1 -- 172.21.15.5:0/2058719502 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68c410be30 con 0x7f68c4107700 2024-09-12T02:49:18.498 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:17.719+0000 7f68c27fc640 1 --2- 172.21.15.5:0/2058719502 >> [v2:172.21.15.5:6800/3125821798,v1:172.21.15.5:6801/3125821798] conn(0x7f68ac038fc0 0x7f68ac03b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:18.498 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:17.720+0000 7f68c27fc640 1 --2- 172.21.15.5:0/2058719502 >> [v2:172.21.15.5:6800/3125821798,v1:172.21.15.5:6801/3125821798] conn(0x7f68ac038fc0 0x7f68ac03b480 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f68b80098c0 tx=0x7f68b8006db0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:18.498 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:17.724+0000 7f68abfff640 1 -- 172.21.15.5:0/2058719502 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f68b404b020 con 0x7f68c4107700 2024-09-12T02:49:18.498 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:17.927+0000 7f68c9625640 1 -- 172.21.15.5:0/2058719502 --> [v2:172.21.15.5:6800/3125821798,v1:172.21.15.5:6801/3125821798] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f68c4104190 con 0x7f68ac038fc0 2024-09-12T02:49:18.498 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:17.932+0000 7f68abfff640 1 -- 172.21.15.5:0/2058719502 <== mgr.14116 v2:172.21.15.5:6800/3125821798 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f68c4104190 con 0x7f68ac038fc0 2024-09-12T02:49:18.498 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:17.935+0000 7f68c9625640 1 -- 172.21.15.5:0/2058719502 >> [v2:172.21.15.5:6800/3125821798,v1:172.21.15.5:6801/3125821798] conn(0x7f68ac038fc0 msgr2=0x7f68ac03b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:18.498 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:17.935+0000 7f68c9625640 1 --2- 172.21.15.5:0/2058719502 >> [v2:172.21.15.5:6800/3125821798,v1:172.21.15.5:6801/3125821798] conn(0x7f68ac038fc0 0x7f68ac03b480 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f68b80098c0 tx=0x7f68b8006db0 comp rx=0 tx=0).stop 2024-09-12T02:49:18.498 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:17.935+0000 7f68c9625640 1 -- 172.21.15.5:0/2058719502 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68c4107700 msgr2=0x7f68c419b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:18.498 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:17.935+0000 7f68c9625640 1 --2- 172.21.15.5:0/2058719502 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68c4107700 0x7f68c419b6f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f68b402acc0 tx=0x7f68b4031ca0 comp rx=0 tx=0).stop 2024-09-12T02:49:18.499 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:17.935+0000 7f68c9625640 1 -- 172.21.15.5:0/2058719502 shutdown_connections 2024-09-12T02:49:18.499 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:17.935+0000 7f68c9625640 1 --2- 172.21.15.5:0/2058719502 >> [v2:172.21.15.5:6800/3125821798,v1:172.21.15.5:6801/3125821798] conn(0x7f68ac038fc0 0x7f68ac03b480 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:18.499 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:17.935+0000 7f68c9625640 1 --2- 172.21.15.5:0/2058719502 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68c4107700 0x7f68c419b6f0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:18.499 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:17.935+0000 7f68c9625640 1 -- 172.21.15.5:0/2058719502 >> 172.21.15.5:0/2058719502 conn(0x7f68c4102f30 msgr2=0x7f68c4103a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:18.499 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:17.935+0000 7f68c9625640 1 -- 172.21.15.5:0/2058719502 shutdown_connections 2024-09-12T02:49:18.499 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:17.935+0000 7f68c9625640 1 -- 172.21.15.5:0/2058719502 wait complete. 2024-09-12T02:49:18.499 INFO:teuthology.orchestra.run.smithi005.stdout:Deploying unmanaged mgr service... 2024-09-12T02:49:19.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:18 smithi005 ceph-mon[31197]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:49:19.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:18 smithi005 ceph-mon[31197]: Saving service mon spec with placement count:5 2024-09-12T02:49:19.061 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:18 smithi005 ceph-mon[31197]: from='mgr.14116 172.21.15.5:0/613993494' entity='mgr.x' 2024-09-12T02:49:20.251 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:20 smithi005 ceph-mon[31197]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:49:20.252 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:20 smithi005 ceph-mon[31197]: Saving service mgr spec with placement count:2 2024-09-12T02:49:20.252 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:20 smithi005 ceph-mon[31197]: from='mgr.14116 172.21.15.5:0/613993494' entity='mgr.x' 2024-09-12T02:49:20.501 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-12T02:49:20.501 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:18.880+0000 7fafa4fb0640 1 Processor -- start 2024-09-12T02:49:20.501 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:18.880+0000 7fafa4fb0640 1 -- start start 2024-09-12T02:49:20.501 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:18.880+0000 7fafa4fb0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fafa0108200 0x7fafa010a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:20.501 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:18.880+0000 7fafa4fb0640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fafa007bdb0 con 0x7fafa0108200 2024-09-12T02:49:20.501 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:18.880+0000 7faf9e59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fafa0108200 0x7fafa010a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:20.501 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:18.880+0000 7faf9e59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fafa0108200 0x7fafa010a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33094/0 (socket says 172.21.15.5:33094) 2024-09-12T02:49:20.501 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:18.880+0000 7faf9e59c640 1 -- 172.21.15.5:0/2951551463 learned_addr learned my addr 172.21.15.5:0/2951551463 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:49:20.501 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:18.881+0000 7faf9e59c640 1 -- 172.21.15.5:0/2951551463 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafa010ab30 con 0x7fafa0108200 2024-09-12T02:49:20.501 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:18.881+0000 7faf9e59c640 1 --2- 172.21.15.5:0/2951551463 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fafa0108200 0x7fafa010a5f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7faf90009e80 tx=0x7faf9002a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=864a27c343ca9ca8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:20.501 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:18.881+0000 7faf9d59a640 1 -- 172.21.15.5:0/2951551463 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faf90036070 con 0x7fafa0108200 2024-09-12T02:49:20.502 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:18.881+0000 7faf9d59a640 1 -- 172.21.15.5:0/2951551463 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7faf900318c0 con 0x7fafa0108200 2024-09-12T02:49:20.502 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:18.881+0000 7faf9d59a640 1 -- 172.21.15.5:0/2951551463 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faf90007ab0 con 0x7fafa0108200 2024-09-12T02:49:20.502 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:18.881+0000 7fafa4fb0640 1 -- 172.21.15.5:0/2951551463 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fafa0108200 msgr2=0x7fafa010a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:20.502 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:18.881+0000 7fafa4fb0640 1 --2- 172.21.15.5:0/2951551463 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fafa0108200 0x7fafa010a5f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7faf90009e80 tx=0x7faf9002a7a0 comp rx=0 tx=0).stop 2024-09-12T02:49:20.502 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:18.881+0000 7fafa4fb0640 1 -- 172.21.15.5:0/2951551463 shutdown_connections 2024-09-12T02:49:20.502 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:18.881+0000 7fafa4fb0640 1 --2- 172.21.15.5:0/2951551463 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fafa0108200 0x7fafa010a5f0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:20.502 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:18.881+0000 7fafa4fb0640 1 -- 172.21.15.5:0/2951551463 >> 172.21.15.5:0/2951551463 conn(0x7fafa0101e20 msgr2=0x7fafa0104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:20.502 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:18.881+0000 7fafa4fb0640 1 -- 172.21.15.5:0/2951551463 shutdown_connections 2024-09-12T02:49:20.502 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:18.881+0000 7fafa4fb0640 1 -- 172.21.15.5:0/2951551463 wait complete. 2024-09-12T02:49:20.503 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:18.882+0000 7fafa4fb0640 1 Processor -- start 2024-09-12T02:49:20.503 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:18.882+0000 7fafa4fb0640 1 -- start start 2024-09-12T02:49:20.503 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:18.883+0000 7fafa4fb0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fafa0108200 0x7fafa019d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:20.503 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:18.883+0000 7fafa4fb0640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fafa010bb10 con 0x7fafa0108200 2024-09-12T02:49:20.503 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:18.883+0000 7faf9e59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fafa0108200 0x7fafa019d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:20.503 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:18.883+0000 7faf9e59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fafa0108200 0x7fafa019d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33098/0 (socket says 172.21.15.5:33098) 2024-09-12T02:49:20.503 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:18.883+0000 7faf9e59c640 1 -- 172.21.15.5:0/3573683491 learned_addr learned my addr 172.21.15.5:0/3573683491 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:49:20.503 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:18.883+0000 7faf9e59c640 1 -- 172.21.15.5:0/3573683491 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafa019de50 con 0x7fafa0108200 2024-09-12T02:49:20.503 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:18.883+0000 7faf9e59c640 1 --2- 172.21.15.5:0/3573683491 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fafa0108200 0x7fafa019d910 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7faf9002acc0 tx=0x7faf90031ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:20.503 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:18.883+0000 7faf8f7fe640 1 -- 172.21.15.5:0/3573683491 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faf90036050 con 0x7fafa0108200 2024-09-12T02:49:20.503 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:18.883+0000 7faf8f7fe640 1 -- 172.21.15.5:0/3573683491 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7faf90037040 con 0x7fafa0108200 2024-09-12T02:49:20.503 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:18.883+0000 7faf8f7fe640 1 -- 172.21.15.5:0/3573683491 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faf9003d850 con 0x7fafa0108200 2024-09-12T02:49:20.503 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:18.883+0000 7fafa4fb0640 1 -- 172.21.15.5:0/3573683491 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafa019e0e0 con 0x7fafa0108200 2024-09-12T02:49:20.504 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:18.883+0000 7fafa4fb0640 1 -- 172.21.15.5:0/3573683491 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafa01a0dd0 con 0x7fafa0108200 2024-09-12T02:49:20.504 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:18.885+0000 7faf8f7fe640 1 -- 172.21.15.5:0/3573683491 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7faf90044020 con 0x7fafa0108200 2024-09-12T02:49:20.504 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:18.885+0000 7faf8f7fe640 1 --2- 172.21.15.5:0/3573683491 >> [v2:172.21.15.5:6800/3125821798,v1:172.21.15.5:6801/3125821798] conn(0x7faf88038fc0 0x7faf8803b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:20.504 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:18.885+0000 7fafa4fb0640 1 -- 172.21.15.5:0/3573683491 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafa0108660 con 0x7fafa0108200 2024-09-12T02:49:20.504 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:18.885+0000 7faf8f7fe640 1 -- 172.21.15.5:0/3573683491 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7faf90042020 con 0x7fafa0108200 2024-09-12T02:49:20.504 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:18.885+0000 7faf9dd9b640 1 --2- 172.21.15.5:0/3573683491 >> [v2:172.21.15.5:6800/3125821798,v1:172.21.15.5:6801/3125821798] conn(0x7faf88038fc0 0x7faf8803b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:20.504 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:18.885+0000 7faf9dd9b640 1 --2- 172.21.15.5:0/3573683491 >> [v2:172.21.15.5:6800/3125821798,v1:172.21.15.5:6801/3125821798] conn(0x7faf88038fc0 0x7faf8803b480 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7faf940098c0 tx=0x7faf94006db0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:20.504 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:18.890+0000 7faf8f7fe640 1 -- 172.21.15.5:0/3573683491 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7faf9004b020 con 0x7fafa0108200 2024-09-12T02:49:20.504 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:19.082+0000 7fafa4fb0640 1 -- 172.21.15.5:0/3573683491 --> [v2:172.21.15.5:6800/3125821798,v1:172.21.15.5:6801/3125821798] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fafa0103050 con 0x7faf88038fc0 2024-09-12T02:49:20.504 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:19.086+0000 7faf8f7fe640 1 -- 172.21.15.5:0/3573683491 <== mgr.14116 v2:172.21.15.5:6800/3125821798 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fafa0103050 con 0x7faf88038fc0 2024-09-12T02:49:20.504 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:19.088+0000 7fafa4fb0640 1 -- 172.21.15.5:0/3573683491 >> [v2:172.21.15.5:6800/3125821798,v1:172.21.15.5:6801/3125821798] conn(0x7faf88038fc0 msgr2=0x7faf8803b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:20.504 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:19.088+0000 7fafa4fb0640 1 --2- 172.21.15.5:0/3573683491 >> [v2:172.21.15.5:6800/3125821798,v1:172.21.15.5:6801/3125821798] conn(0x7faf88038fc0 0x7faf8803b480 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7faf940098c0 tx=0x7faf94006db0 comp rx=0 tx=0).stop 2024-09-12T02:49:20.505 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:19.088+0000 7fafa4fb0640 1 -- 172.21.15.5:0/3573683491 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fafa0108200 msgr2=0x7fafa019d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:20.505 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:19.088+0000 7fafa4fb0640 1 --2- 172.21.15.5:0/3573683491 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fafa0108200 0x7fafa019d910 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7faf9002acc0 tx=0x7faf90031ca0 comp rx=0 tx=0).stop 2024-09-12T02:49:20.505 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:19.088+0000 7fafa4fb0640 1 -- 172.21.15.5:0/3573683491 shutdown_connections 2024-09-12T02:49:20.505 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:19.088+0000 7fafa4fb0640 1 --2- 172.21.15.5:0/3573683491 >> [v2:172.21.15.5:6800/3125821798,v1:172.21.15.5:6801/3125821798] conn(0x7faf88038fc0 0x7faf8803b480 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:20.505 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:19.088+0000 7fafa4fb0640 1 --2- 172.21.15.5:0/3573683491 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fafa0108200 0x7fafa019d910 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:20.505 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:19.088+0000 7fafa4fb0640 1 -- 172.21.15.5:0/3573683491 >> 172.21.15.5:0/3573683491 conn(0x7fafa0101e20 msgr2=0x7fafa0102940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:20.505 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:19.088+0000 7fafa4fb0640 1 -- 172.21.15.5:0/3573683491 shutdown_connections 2024-09-12T02:49:20.505 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:19.088+0000 7fafa4fb0640 1 -- 172.21.15.5:0/3573683491 wait complete. 2024-09-12T02:49:21.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:21 smithi005 ceph-mon[31197]: from='mgr.14116 172.21.15.5:0/613993494' entity='mgr.x' 2024-09-12T02:49:21.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:21 smithi005 ceph-mon[31197]: from='mgr.14116 172.21.15.5:0/613993494' entity='mgr.x' 2024-09-12T02:49:21.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:21 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/1209306118' entity='client.admin' 2024-09-12T02:49:21.722 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:20.896+0000 7f3fcd3cf640 1 Processor -- start 2024-09-12T02:49:22.277 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:20.896+0000 7f3fcd3cf640 1 -- start start 2024-09-12T02:49:22.277 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:20.896+0000 7f3fcd3cf640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fc8107730 0x7f3fc8107b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:22.277 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:20.896+0000 7f3fcd3cf640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3fc8108100 con 0x7f3fc8107730 2024-09-12T02:49:22.278 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:20.896+0000 7f3fc6ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fc8107730 0x7f3fc8107b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:22.278 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:20.896+0000 7f3fc6ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fc8107730 0x7f3fc8107b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33104/0 (socket says 172.21.15.5:33104) 2024-09-12T02:49:22.278 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:20.896+0000 7f3fc6ffd640 1 -- 172.21.15.5:0/1516551561 learned_addr learned my addr 172.21.15.5:0/1516551561 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:49:22.278 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:20.897+0000 7f3fc6ffd640 1 -- 172.21.15.5:0/1516551561 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fc8108930 con 0x7f3fc8107730 2024-09-12T02:49:22.278 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:20.897+0000 7f3fc6ffd640 1 --2- 172.21.15.5:0/1516551561 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fc8107730 0x7f3fc8107b30 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3fb8009820 tx=0x7f3fb802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=321ea999ada74994 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:22.278 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:20.898+0000 7f3fc5ffb640 1 -- 172.21.15.5:0/1516551561 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3fb8037070 con 0x7f3fc8107730 2024-09-12T02:49:22.283 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:20.898+0000 7f3fc5ffb640 1 -- 172.21.15.5:0/1516551561 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3fb8032920 con 0x7f3fc8107730 2024-09-12T02:49:22.283 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:20.898+0000 7f3fc5ffb640 1 -- 172.21.15.5:0/1516551561 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3fb8030810 con 0x7f3fc8107730 2024-09-12T02:49:22.283 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:20.898+0000 7f3fcd3cf640 1 -- 172.21.15.5:0/1516551561 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fc8107730 msgr2=0x7f3fc8107b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:22.283 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:20.898+0000 7f3fcd3cf640 1 --2- 172.21.15.5:0/1516551561 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fc8107730 0x7f3fc8107b30 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3fb8009820 tx=0x7f3fb802a270 comp rx=0 tx=0).stop 2024-09-12T02:49:22.283 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:20.898+0000 7f3fcd3cf640 1 -- 172.21.15.5:0/1516551561 shutdown_connections 2024-09-12T02:49:22.283 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:20.898+0000 7f3fcd3cf640 1 --2- 172.21.15.5:0/1516551561 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fc8107730 0x7f3fc8107b30 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:22.283 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:20.898+0000 7f3fcd3cf640 1 -- 172.21.15.5:0/1516551561 >> 172.21.15.5:0/1516551561 conn(0x7f3fc8102ea0 msgr2=0x7f3fc8105300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:22.283 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:20.898+0000 7f3fcd3cf640 1 -- 172.21.15.5:0/1516551561 shutdown_connections 2024-09-12T02:49:22.283 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:20.898+0000 7f3fcd3cf640 1 -- 172.21.15.5:0/1516551561 wait complete. 2024-09-12T02:49:22.284 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:20.899+0000 7f3fcd3cf640 1 Processor -- start 2024-09-12T02:49:22.284 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:20.899+0000 7f3fcd3cf640 1 -- start start 2024-09-12T02:49:22.284 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:20.899+0000 7f3fcd3cf640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fc8107730 0x7f3fc819b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:22.284 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:20.899+0000 7f3fcd3cf640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3fc810bce0 con 0x7f3fc8107730 2024-09-12T02:49:22.284 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:20.900+0000 7f3fc6ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fc8107730 0x7f3fc819b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:22.284 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:20.900+0000 7f3fc6ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fc8107730 0x7f3fc819b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33112/0 (socket says 172.21.15.5:33112) 2024-09-12T02:49:22.284 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:20.900+0000 7f3fc6ffd640 1 -- 172.21.15.5:0/1209306118 learned_addr learned my addr 172.21.15.5:0/1209306118 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:49:22.284 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:20.900+0000 7f3fc6ffd640 1 -- 172.21.15.5:0/1209306118 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fc819bca0 con 0x7f3fc8107730 2024-09-12T02:49:22.284 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:20.900+0000 7f3fc6ffd640 1 --2- 172.21.15.5:0/1209306118 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fc8107730 0x7f3fc819b760 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3fb8030560 tx=0x7f3fb8032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:22.284 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:20.900+0000 7f3fbffff640 1 -- 172.21.15.5:0/1209306118 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3fb8037040 con 0x7f3fc8107730 2024-09-12T02:49:22.284 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:20.900+0000 7f3fbffff640 1 -- 172.21.15.5:0/1209306118 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3fb8040070 con 0x7f3fc8107730 2024-09-12T02:49:22.284 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:20.900+0000 7f3fcd3cf640 1 -- 172.21.15.5:0/1209306118 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fc819bf30 con 0x7f3fc8107730 2024-09-12T02:49:22.285 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:20.900+0000 7f3fbffff640 1 -- 172.21.15.5:0/1209306118 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3fb803a740 con 0x7f3fc8107730 2024-09-12T02:49:22.285 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:20.900+0000 7f3fcd3cf640 1 -- 172.21.15.5:0/1209306118 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fc819ec20 con 0x7f3fc8107730 2024-09-12T02:49:22.285 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:20.901+0000 7f3fbffff640 1 -- 172.21.15.5:0/1209306118 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f3fb803a9b0 con 0x7f3fc8107730 2024-09-12T02:49:22.285 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:20.901+0000 7f3fcd3cf640 1 -- 172.21.15.5:0/1209306118 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fc810be60 con 0x7f3fc8107730 2024-09-12T02:49:22.285 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:20.901+0000 7f3fbffff640 1 --2- 172.21.15.5:0/1209306118 >> [v2:172.21.15.5:6800/3125821798,v1:172.21.15.5:6801/3125821798] conn(0x7f3fb4038f70 0x7f3fb403b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:22.285 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:20.901+0000 7f3fbffff640 1 -- 172.21.15.5:0/1209306118 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f3fb806c590 con 0x7f3fc8107730 2024-09-12T02:49:22.285 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:20.902+0000 7f3fc67fc640 1 --2- 172.21.15.5:0/1209306118 >> [v2:172.21.15.5:6800/3125821798,v1:172.21.15.5:6801/3125821798] conn(0x7f3fb4038f70 0x7f3fb403b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:22.285 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:20.902+0000 7f3fc67fc640 1 --2- 172.21.15.5:0/1209306118 >> [v2:172.21.15.5:6800/3125821798,v1:172.21.15.5:6801/3125821798] conn(0x7f3fb4038f70 0x7f3fb403b430 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f3fb00098c0 tx=0x7f3fb0006db0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:22.285 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:20.906+0000 7f3fbffff640 1 -- 172.21.15.5:0/1209306118 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3fb8041270 con 0x7f3fc8107730 2024-09-12T02:49:22.285 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:21.084+0000 7f3fcd3cf640 1 -- 172.21.15.5:0/1209306118 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f3fc8107bb0 con 0x7f3fc8107730 2024-09-12T02:49:22.285 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:21.086+0000 7f3fbffff640 1 -- 172.21.15.5:0/1209306118 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f3fb803ada0 con 0x7f3fc8107730 2024-09-12T02:49:22.285 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:21.089+0000 7f3fcd3cf640 1 -- 172.21.15.5:0/1209306118 >> [v2:172.21.15.5:6800/3125821798,v1:172.21.15.5:6801/3125821798] conn(0x7f3fb4038f70 msgr2=0x7f3fb403b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:22.285 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:21.089+0000 7f3fcd3cf640 1 --2- 172.21.15.5:0/1209306118 >> [v2:172.21.15.5:6800/3125821798,v1:172.21.15.5:6801/3125821798] conn(0x7f3fb4038f70 0x7f3fb403b430 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f3fb00098c0 tx=0x7f3fb0006db0 comp rx=0 tx=0).stop 2024-09-12T02:49:22.286 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:21.089+0000 7f3fcd3cf640 1 -- 172.21.15.5:0/1209306118 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fc8107730 msgr2=0x7f3fc819b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:22.286 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:21.089+0000 7f3fcd3cf640 1 --2- 172.21.15.5:0/1209306118 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fc8107730 0x7f3fc819b760 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3fb8030560 tx=0x7f3fb8032670 comp rx=0 tx=0).stop 2024-09-12T02:49:22.286 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:21.089+0000 7f3fcd3cf640 1 -- 172.21.15.5:0/1209306118 shutdown_connections 2024-09-12T02:49:22.286 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:21.089+0000 7f3fcd3cf640 1 --2- 172.21.15.5:0/1209306118 >> [v2:172.21.15.5:6800/3125821798,v1:172.21.15.5:6801/3125821798] conn(0x7f3fb4038f70 0x7f3fb403b430 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:22.286 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:21.089+0000 7f3fcd3cf640 1 --2- 172.21.15.5:0/1209306118 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3fc8107730 0x7f3fc819b760 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:22.286 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:21.089+0000 7f3fcd3cf640 1 -- 172.21.15.5:0/1209306118 >> 172.21.15.5:0/1209306118 conn(0x7f3fc8102ea0 msgr2=0x7f3fc8103a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:22.286 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:21.089+0000 7f3fcd3cf640 1 -- 172.21.15.5:0/1209306118 shutdown_connections 2024-09-12T02:49:22.286 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:21.089+0000 7f3fcd3cf640 1 -- 172.21.15.5:0/1209306118 wait complete. 2024-09-12T02:49:23.124 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:22.161+0000 7ff5feafd640 1 Processor -- start 2024-09-12T02:49:23.124 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:22.161+0000 7ff5feafd640 1 -- start start 2024-09-12T02:49:23.124 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:22.161+0000 7ff5feafd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5f8107780 0x7ff5f8107b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:23.125 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:22.161+0000 7ff5feafd640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff5f8108150 con 0x7ff5f8107780 2024-09-12T02:49:23.125 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:22.161+0000 7ff5fc899640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5f8107780 0x7ff5f8107b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:23.125 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:22.161+0000 7ff5fc899640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5f8107780 0x7ff5f8107b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33118/0 (socket says 172.21.15.5:33118) 2024-09-12T02:49:23.125 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:22.161+0000 7ff5fc899640 1 -- 172.21.15.5:0/3730406821 learned_addr learned my addr 172.21.15.5:0/3730406821 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:49:23.125 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:22.161+0000 7ff5fc899640 1 -- 172.21.15.5:0/3730406821 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5f8108980 con 0x7ff5f8107780 2024-09-12T02:49:23.125 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:22.163+0000 7ff5fc899640 1 --2- 172.21.15.5:0/3730406821 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5f8107780 0x7ff5f8107b80 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec009820 tx=0x7ff5ec02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed07c974005a5ccd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:23.125 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:22.163+0000 7ff5f77fe640 1 -- 172.21.15.5:0/3730406821 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff5ec037070 con 0x7ff5f8107780 2024-09-12T02:49:23.125 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:22.163+0000 7ff5f77fe640 1 -- 172.21.15.5:0/3730406821 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5ec032920 con 0x7ff5f8107780 2024-09-12T02:49:23.125 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:22.163+0000 7ff5f77fe640 1 -- 172.21.15.5:0/3730406821 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff5ec030810 con 0x7ff5f8107780 2024-09-12T02:49:23.126 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:22.164+0000 7ff5feafd640 1 -- 172.21.15.5:0/3730406821 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5f8107780 msgr2=0x7ff5f8107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:23.126 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:22.164+0000 7ff5feafd640 1 --2- 172.21.15.5:0/3730406821 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5f8107780 0x7ff5f8107b80 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec009820 tx=0x7ff5ec02a270 comp rx=0 tx=0).stop 2024-09-12T02:49:23.126 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:22.164+0000 7ff5feafd640 1 -- 172.21.15.5:0/3730406821 shutdown_connections 2024-09-12T02:49:23.126 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:22.164+0000 7ff5feafd640 1 --2- 172.21.15.5:0/3730406821 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5f8107780 0x7ff5f8107b80 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:23.126 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:22.164+0000 7ff5feafd640 1 -- 172.21.15.5:0/3730406821 >> 172.21.15.5:0/3730406821 conn(0x7ff5f8102f30 msgr2=0x7ff5f8105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:23.126 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:22.164+0000 7ff5feafd640 1 -- 172.21.15.5:0/3730406821 shutdown_connections 2024-09-12T02:49:23.126 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:22.164+0000 7ff5feafd640 1 -- 172.21.15.5:0/3730406821 wait complete. 2024-09-12T02:49:23.126 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:22.164+0000 7ff5feafd640 1 Processor -- start 2024-09-12T02:49:23.126 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:22.164+0000 7ff5feafd640 1 -- start start 2024-09-12T02:49:23.126 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:22.165+0000 7ff5feafd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5f8107780 0x7ff5f807daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:23.126 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:22.165+0000 7ff5feafd640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff5f810bd30 con 0x7ff5f8107780 2024-09-12T02:49:23.126 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:22.165+0000 7ff5fc899640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5f8107780 0x7ff5f807daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:23.127 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:22.165+0000 7ff5fc899640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5f8107780 0x7ff5f807daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33122/0 (socket says 172.21.15.5:33122) 2024-09-12T02:49:23.127 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:22.165+0000 7ff5fc899640 1 -- 172.21.15.5:0/1788834074 learned_addr learned my addr 172.21.15.5:0/1788834074 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:49:23.127 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:22.165+0000 7ff5fc899640 1 -- 172.21.15.5:0/1788834074 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5f807e030 con 0x7ff5f8107780 2024-09-12T02:49:23.127 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:22.165+0000 7ff5fc899640 1 --2- 172.21.15.5:0/1788834074 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5f8107780 0x7ff5f807daf0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec0097f0 tx=0x7ff5ec0327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:23.127 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:22.165+0000 7ff5f5ffb640 1 -- 172.21.15.5:0/1788834074 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff5ec037040 con 0x7ff5f8107780 2024-09-12T02:49:23.127 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:22.165+0000 7ff5feafd640 1 -- 172.21.15.5:0/1788834074 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5f807c220 con 0x7ff5f8107780 2024-09-12T02:49:23.127 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:22.165+0000 7ff5f5ffb640 1 -- 172.21.15.5:0/1788834074 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5ec030cc0 con 0x7ff5f8107780 2024-09-12T02:49:23.127 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:22.165+0000 7ff5f5ffb640 1 -- 172.21.15.5:0/1788834074 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff5ec03a640 con 0x7ff5f8107780 2024-09-12T02:49:23.127 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:22.166+0000 7ff5feafd640 1 -- 172.21.15.5:0/1788834074 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5f807c760 con 0x7ff5f8107780 2024-09-12T02:49:23.128 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:22.166+0000 7ff5f5ffb640 1 -- 172.21.15.5:0/1788834074 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7ff5ec03a940 con 0x7ff5f8107780 2024-09-12T02:49:23.128 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:22.167+0000 7ff5feafd640 1 -- 172.21.15.5:0/1788834074 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5f8107c00 con 0x7ff5f8107780 2024-09-12T02:49:23.128 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:22.167+0000 7ff5f5ffb640 1 --2- 172.21.15.5:0/1788834074 >> [v2:172.21.15.5:6800/3125821798,v1:172.21.15.5:6801/3125821798] conn(0x7ff5e8038f70 0x7ff5e803b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:23.128 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:22.167+0000 7ff5f5ffb640 1 -- 172.21.15.5:0/1788834074 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ff5ec06c5b0 con 0x7ff5f8107780 2024-09-12T02:49:23.128 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:22.167+0000 7ff5f7fff640 1 --2- 172.21.15.5:0/1788834074 >> [v2:172.21.15.5:6800/3125821798,v1:172.21.15.5:6801/3125821798] conn(0x7ff5e8038f70 0x7ff5e803b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:23.128 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:22.167+0000 7ff5f7fff640 1 --2- 172.21.15.5:0/1788834074 >> [v2:172.21.15.5:6800/3125821798,v1:172.21.15.5:6801/3125821798] conn(0x7ff5e8038f70 0x7ff5e803b430 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff5e40098c0 tx=0x7ff5e4006db0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:23.128 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:22.172+0000 7ff5f5ffb640 1 -- 172.21.15.5:0/1788834074 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff5ec06bf00 con 0x7ff5f8107780 2024-09-12T02:49:23.128 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:22.361+0000 7ff5feafd640 1 -- 172.21.15.5:0/1788834074 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7ff5f807cf70 con 0x7ff5f8107780 2024-09-12T02:49:23.128 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:22.363+0000 7ff5f5ffb640 1 -- 172.21.15.5:0/1788834074 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7ff5ec03ba40 con 0x7ff5f8107780 2024-09-12T02:49:23.128 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:22.366+0000 7ff5feafd640 1 -- 172.21.15.5:0/1788834074 >> [v2:172.21.15.5:6800/3125821798,v1:172.21.15.5:6801/3125821798] conn(0x7ff5e8038f70 msgr2=0x7ff5e803b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:23.128 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:22.366+0000 7ff5feafd640 1 --2- 172.21.15.5:0/1788834074 >> [v2:172.21.15.5:6800/3125821798,v1:172.21.15.5:6801/3125821798] conn(0x7ff5e8038f70 0x7ff5e803b430 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff5e40098c0 tx=0x7ff5e4006db0 comp rx=0 tx=0).stop 2024-09-12T02:49:23.129 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:22.366+0000 7ff5feafd640 1 -- 172.21.15.5:0/1788834074 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5f8107780 msgr2=0x7ff5f807daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:23.129 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:22.366+0000 7ff5feafd640 1 --2- 172.21.15.5:0/1788834074 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5f8107780 0x7ff5f807daf0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec0097f0 tx=0x7ff5ec0327b0 comp rx=0 tx=0).stop 2024-09-12T02:49:23.129 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:22.366+0000 7ff5feafd640 1 -- 172.21.15.5:0/1788834074 shutdown_connections 2024-09-12T02:49:23.129 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:22.366+0000 7ff5feafd640 1 --2- 172.21.15.5:0/1788834074 >> [v2:172.21.15.5:6800/3125821798,v1:172.21.15.5:6801/3125821798] conn(0x7ff5e8038f70 0x7ff5e803b430 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:23.129 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:22.366+0000 7ff5feafd640 1 --2- 172.21.15.5:0/1788834074 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5f8107780 0x7ff5f807daf0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:23.129 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:22.366+0000 7ff5feafd640 1 -- 172.21.15.5:0/1788834074 >> 172.21.15.5:0/1788834074 conn(0x7ff5f8102f30 msgr2=0x7ff5f8103ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:23.129 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:22.366+0000 7ff5feafd640 1 -- 172.21.15.5:0/1788834074 shutdown_connections 2024-09-12T02:49:23.129 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:22.366+0000 7ff5feafd640 1 -- 172.21.15.5:0/1788834074 wait complete. 2024-09-12T02:49:23.129 INFO:teuthology.orchestra.run.smithi005.stdout:Enabling the dashboard module... 2024-09-12T02:49:23.535 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:23 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/1788834074' entity='client.admin' 2024-09-12T02:49:24.524 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:24 smithi005 ceph-mon[31197]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:49:24.524 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:24 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/2385721555' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-09-12T02:49:24.524 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:24 smithi005 ceph-mon[31197]: from='mgr.14116 172.21.15.5:0/613993494' entity='mgr.x' 2024-09-12T02:49:24.524 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:49:24 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: ignoring --setuser ceph since I am not root 2024-09-12T02:49:24.525 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:49:24 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: ignoring --setgroup ceph since I am not root 2024-09-12T02:49:24.525 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:49:24 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: 2024-09-12T02:49:24.523+0000 7ff925705040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-12T02:49:24.809 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:49:24 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: 2024-09-12T02:49:24.583+0000 7ff925705040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-12T02:49:24.809 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:49:24 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: 2024-09-12T02:49:24.740+0000 7ff925705040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-12T02:49:24.809 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:49:24 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: 2024-09-12T02:49:24.808+0000 7ff925705040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-12T02:49:25.027 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:23.537+0000 7f4985ac9640 1 Processor -- start 2024-09-12T02:49:25.028 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:23.537+0000 7f4985ac9640 1 -- start start 2024-09-12T02:49:25.028 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:23.537+0000 7f4985ac9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49801081c0 0x7f498010a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:25.028 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:23.537+0000 7f4985ac9640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f498007bdb0 con 0x7f49801081c0 2024-09-12T02:49:25.028 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:23.538+0000 7f497f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49801081c0 0x7f498010a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:25.028 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:23.538+0000 7f497f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49801081c0 0x7f498010a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45636/0 (socket says 172.21.15.5:45636) 2024-09-12T02:49:25.028 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:23.538+0000 7f497f7fe640 1 -- 172.21.15.5:0/552382874 learned_addr learned my addr 172.21.15.5:0/552382874 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:49:25.028 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:23.538+0000 7f497f7fe640 1 -- 172.21.15.5:0/552382874 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f498010aaf0 con 0x7f49801081c0 2024-09-12T02:49:25.028 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:23.539+0000 7f497f7fe640 1 --2- 172.21.15.5:0/552382874 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49801081c0 0x7f498010a5b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4970009e80 tx=0x7f497002a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2ce3d6a190db2d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:25.028 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:23.539+0000 7f497e7fc640 1 -- 172.21.15.5:0/552382874 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4970036070 con 0x7f49801081c0 2024-09-12T02:49:25.028 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:23.539+0000 7f497e7fc640 1 -- 172.21.15.5:0/552382874 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49700318e0 con 0x7f49801081c0 2024-09-12T02:49:25.028 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:23.539+0000 7f497e7fc640 1 -- 172.21.15.5:0/552382874 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4970007a00 con 0x7f49801081c0 2024-09-12T02:49:25.028 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:23.539+0000 7f4985ac9640 1 -- 172.21.15.5:0/552382874 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49801081c0 msgr2=0x7f498010a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:25.028 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:23.539+0000 7f4985ac9640 1 --2- 172.21.15.5:0/552382874 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49801081c0 0x7f498010a5b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4970009e80 tx=0x7f497002a7a0 comp rx=0 tx=0).stop 2024-09-12T02:49:25.029 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:23.539+0000 7f4985ac9640 1 -- 172.21.15.5:0/552382874 shutdown_connections 2024-09-12T02:49:25.029 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:23.540+0000 7f4985ac9640 1 --2- 172.21.15.5:0/552382874 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49801081c0 0x7f498010a5b0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:25.029 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:23.540+0000 7f4985ac9640 1 -- 172.21.15.5:0/552382874 >> 172.21.15.5:0/552382874 conn(0x7f4980101e20 msgr2=0x7f4980104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:25.029 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:23.540+0000 7f4985ac9640 1 -- 172.21.15.5:0/552382874 shutdown_connections 2024-09-12T02:49:25.029 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:23.540+0000 7f4985ac9640 1 -- 172.21.15.5:0/552382874 wait complete. 2024-09-12T02:49:25.029 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:23.540+0000 7f4985ac9640 1 Processor -- start 2024-09-12T02:49:25.029 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:23.540+0000 7f4985ac9640 1 -- start start 2024-09-12T02:49:25.029 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:23.540+0000 7f4985ac9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49801081c0 0x7f498019d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:25.029 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:23.541+0000 7f4985ac9640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f498010bb10 con 0x7f49801081c0 2024-09-12T02:49:25.029 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:23.541+0000 7f497f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49801081c0 0x7f498019d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:25.029 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:23.541+0000 7f497f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49801081c0 0x7f498019d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45646/0 (socket says 172.21.15.5:45646) 2024-09-12T02:49:25.029 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:23.541+0000 7f497f7fe640 1 -- 172.21.15.5:0/2385721555 learned_addr learned my addr 172.21.15.5:0/2385721555 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:49:25.029 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:23.541+0000 7f497f7fe640 1 -- 172.21.15.5:0/2385721555 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f498019de90 con 0x7f49801081c0 2024-09-12T02:49:25.030 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:23.541+0000 7f497f7fe640 1 --2- 172.21.15.5:0/2385721555 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49801081c0 0x7f498019d950 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f497002acc0 tx=0x7f4970031cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:25.030 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:23.541+0000 7f497cff9640 1 -- 172.21.15.5:0/2385721555 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4970036050 con 0x7f49801081c0 2024-09-12T02:49:25.030 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:23.541+0000 7f497cff9640 1 -- 172.21.15.5:0/2385721555 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4970037040 con 0x7f49801081c0 2024-09-12T02:49:25.030 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:23.541+0000 7f4985ac9640 1 -- 172.21.15.5:0/2385721555 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f498019e120 con 0x7f49801081c0 2024-09-12T02:49:25.030 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:23.541+0000 7f497cff9640 1 -- 172.21.15.5:0/2385721555 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f497003d8e0 con 0x7f49801081c0 2024-09-12T02:49:25.030 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:23.541+0000 7f4985ac9640 1 -- 172.21.15.5:0/2385721555 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49801a0e10 con 0x7f49801081c0 2024-09-12T02:49:25.030 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:23.542+0000 7f497cff9640 1 -- 172.21.15.5:0/2385721555 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f4970044020 con 0x7f49801081c0 2024-09-12T02:49:25.030 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:23.542+0000 7f4985ac9640 1 -- 172.21.15.5:0/2385721555 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4980108620 con 0x7f49801081c0 2024-09-12T02:49:25.030 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:23.542+0000 7f497cff9640 1 --2- 172.21.15.5:0/2385721555 >> [v2:172.21.15.5:6800/3125821798,v1:172.21.15.5:6801/3125821798] conn(0x7f4968038f70 0x7f496803b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:25.030 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:23.543+0000 7f497cff9640 1 -- 172.21.15.5:0/2385721555 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f4970042020 con 0x7f49801081c0 2024-09-12T02:49:25.030 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:23.543+0000 7f497effd640 1 --2- 172.21.15.5:0/2385721555 >> [v2:172.21.15.5:6800/3125821798,v1:172.21.15.5:6801/3125821798] conn(0x7f4968038f70 0x7f496803b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:25.030 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:23.543+0000 7f497effd640 1 --2- 172.21.15.5:0/2385721555 >> [v2:172.21.15.5:6800/3125821798,v1:172.21.15.5:6801/3125821798] conn(0x7f4968038f70 0x7f496803b430 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f49740098c0 tx=0x7f4974006db0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:25.031 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:23.547+0000 7f497cff9640 1 -- 172.21.15.5:0/2385721555 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f497004b020 con 0x7f49801081c0 2024-09-12T02:49:25.031 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:23.774+0000 7f4985ac9640 1 -- 172.21.15.5:0/2385721555 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f49801a1400 con 0x7f49801081c0 2024-09-12T02:49:25.031 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:24.365+0000 7f497cff9640 1 -- 172.21.15.5:0/2385721555 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f4970046410 con 0x7f49801081c0 2024-09-12T02:49:25.031 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:24.367+0000 7f497cff9640 1 -- 172.21.15.5:0/2385721555 <== mon.0 v2:172.21.15.5:3300/0 8 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f497003b610 con 0x7f49801081c0 2024-09-12T02:49:25.031 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:24.368+0000 7f4985ac9640 1 -- 172.21.15.5:0/2385721555 >> [v2:172.21.15.5:6800/3125821798,v1:172.21.15.5:6801/3125821798] conn(0x7f4968038f70 msgr2=0x7f496803b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:25.031 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:24.368+0000 7f4985ac9640 1 --2- 172.21.15.5:0/2385721555 >> [v2:172.21.15.5:6800/3125821798,v1:172.21.15.5:6801/3125821798] conn(0x7f4968038f70 0x7f496803b430 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f49740098c0 tx=0x7f4974006db0 comp rx=0 tx=0).stop 2024-09-12T02:49:25.031 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:24.369+0000 7f4985ac9640 1 -- 172.21.15.5:0/2385721555 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49801081c0 msgr2=0x7f498019d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:25.031 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:24.369+0000 7f4985ac9640 1 --2- 172.21.15.5:0/2385721555 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49801081c0 0x7f498019d950 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f497002acc0 tx=0x7f4970031cc0 comp rx=0 tx=0).stop 2024-09-12T02:49:25.031 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:24.369+0000 7f4985ac9640 1 -- 172.21.15.5:0/2385721555 shutdown_connections 2024-09-12T02:49:25.031 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:24.369+0000 7f4985ac9640 1 --2- 172.21.15.5:0/2385721555 >> [v2:172.21.15.5:6800/3125821798,v1:172.21.15.5:6801/3125821798] conn(0x7f4968038f70 0x7f496803b430 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:25.031 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:24.369+0000 7f4985ac9640 1 --2- 172.21.15.5:0/2385721555 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49801081c0 0x7f498019d950 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:25.031 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:24.369+0000 7f4985ac9640 1 -- 172.21.15.5:0/2385721555 >> 172.21.15.5:0/2385721555 conn(0x7f4980101e20 msgr2=0x7f4980102940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:25.031 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:24.369+0000 7f4985ac9640 1 -- 172.21.15.5:0/2385721555 shutdown_connections 2024-09-12T02:49:25.031 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:24.369+0000 7f4985ac9640 1 -- 172.21.15.5:0/2385721555 wait complete. 2024-09-12T02:49:25.060 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:49:24 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: 2024-09-12T02:49:24.962+0000 7ff925705040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-12T02:49:25.422 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:25 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/2385721555' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-12T02:49:25.422 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:25 smithi005 ceph-mon[31197]: mgrmap e9: x(active, since 21s) 2024-09-12T02:49:25.422 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:49:25 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: 2024-09-12T02:49:25.117+0000 7ff925705040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-12T02:49:25.710 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:49:25 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: 2024-09-12T02:49:25.455+0000 7ff925705040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-12T02:49:25.710 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:49:25 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: 2024-09-12T02:49:25.526+0000 7ff925705040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-12T02:49:26.060 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:49:25 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: 2024-09-12T02:49:25.729+0000 7ff925705040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-12T02:49:26.326 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout { 2024-09-12T02:49:26.326 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-12T02:49:26.326 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-12T02:49:26.326 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-12T02:49:26.326 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-12T02:49:26.327 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout } 2024-09-12T02:49:26.327 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:25.503+0000 7f9fc759e640 1 Processor -- start 2024-09-12T02:49:26.327 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:25.503+0000 7f9fc759e640 1 -- start start 2024-09-12T02:49:26.327 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:25.503+0000 7f9fc759e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9fc8108060 0x7f9fc810a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:26.327 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:25.503+0000 7f9fc759e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc807bd50 con 0x7f9fc8108060 2024-09-12T02:49:26.327 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:25.503+0000 7f9fc659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9fc8108060 0x7f9fc810a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:26.327 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:25.503+0000 7f9fc659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9fc8108060 0x7f9fc810a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45666/0 (socket says 172.21.15.5:45666) 2024-09-12T02:49:26.327 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:25.503+0000 7f9fc659c640 1 -- 172.21.15.5:0/3086277638 learned_addr learned my addr 172.21.15.5:0/3086277638 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:49:26.327 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:25.504+0000 7f9fc659c640 1 -- 172.21.15.5:0/3086277638 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fc810a990 con 0x7f9fc8108060 2024-09-12T02:49:26.327 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:25.504+0000 7f9fc659c640 1 --2- 172.21.15.5:0/3086277638 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9fc8108060 0x7f9fc810a450 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9fb8009820 tx=0x7f9fb802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0e54b95d7f91289 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:26.327 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:25.505+0000 7f9fc559a640 1 -- 172.21.15.5:0/3086277638 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9fb8037070 con 0x7f9fc8108060 2024-09-12T02:49:26.327 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:25.505+0000 7f9fc559a640 1 -- 172.21.15.5:0/3086277638 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9fb8032920 con 0x7f9fc8108060 2024-09-12T02:49:26.327 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:25.505+0000 7f9fc559a640 1 -- 172.21.15.5:0/3086277638 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9fb8030810 con 0x7f9fc8108060 2024-09-12T02:49:26.328 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:25.505+0000 7f9fc759e640 1 -- 172.21.15.5:0/3086277638 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9fc8108060 msgr2=0x7f9fc810a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:26.328 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:25.505+0000 7f9fc759e640 1 --2- 172.21.15.5:0/3086277638 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9fc8108060 0x7f9fc810a450 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9fb8009820 tx=0x7f9fb802a270 comp rx=0 tx=0).stop 2024-09-12T02:49:26.328 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:25.505+0000 7f9fc759e640 1 -- 172.21.15.5:0/3086277638 shutdown_connections 2024-09-12T02:49:26.328 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:25.505+0000 7f9fc759e640 1 --2- 172.21.15.5:0/3086277638 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9fc8108060 0x7f9fc810a450 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:26.328 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:25.505+0000 7f9fc759e640 1 -- 172.21.15.5:0/3086277638 >> 172.21.15.5:0/3086277638 conn(0x7f9fc8101bc0 msgr2=0x7f9fc8103fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:26.328 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:25.505+0000 7f9fc759e640 1 -- 172.21.15.5:0/3086277638 shutdown_connections 2024-09-12T02:49:26.328 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:25.505+0000 7f9fc759e640 1 -- 172.21.15.5:0/3086277638 wait complete. 2024-09-12T02:49:26.328 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:25.506+0000 7f9fc759e640 1 Processor -- start 2024-09-12T02:49:26.328 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:25.506+0000 7f9fc759e640 1 -- start start 2024-09-12T02:49:26.328 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:25.506+0000 7f9fc759e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9fc8108060 0x7f9fc819b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:26.328 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:25.507+0000 7f9fc759e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9fc810b950 con 0x7f9fc8108060 2024-09-12T02:49:26.328 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:25.507+0000 7f9fc659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9fc8108060 0x7f9fc819b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:26.329 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:25.507+0000 7f9fc659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9fc8108060 0x7f9fc819b5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45672/0 (socket says 172.21.15.5:45672) 2024-09-12T02:49:26.329 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:25.507+0000 7f9fc659c640 1 -- 172.21.15.5:0/3769003180 learned_addr learned my addr 172.21.15.5:0/3769003180 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:49:26.329 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:25.507+0000 7f9fc659c640 1 -- 172.21.15.5:0/3769003180 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fc819bb30 con 0x7f9fc8108060 2024-09-12T02:49:26.329 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:25.507+0000 7f9fc659c640 1 --2- 172.21.15.5:0/3769003180 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9fc8108060 0x7f9fc819b5f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9fb8030420 tx=0x7f9fb8032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:26.329 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:25.507+0000 7f9fb77fe640 1 -- 172.21.15.5:0/3769003180 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9fb8037040 con 0x7f9fc8108060 2024-09-12T02:49:26.329 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:25.507+0000 7f9fb77fe640 1 -- 172.21.15.5:0/3769003180 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9fb8040070 con 0x7f9fc8108060 2024-09-12T02:49:26.329 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:25.507+0000 7f9fc759e640 1 -- 172.21.15.5:0/3769003180 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fc819bdc0 con 0x7f9fc8108060 2024-09-12T02:49:26.329 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:25.507+0000 7f9fb77fe640 1 -- 172.21.15.5:0/3769003180 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9fb803a6e0 con 0x7f9fc8108060 2024-09-12T02:49:26.329 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:25.508+0000 7f9fc759e640 1 -- 172.21.15.5:0/3769003180 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fc819eab0 con 0x7f9fc8108060 2024-09-12T02:49:26.330 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:25.508+0000 7f9fb77fe640 1 -- 172.21.15.5:0/3769003180 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f9fb803a9e0 con 0x7f9fc8108060 2024-09-12T02:49:26.330 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:25.509+0000 7f9fb77fe640 1 --2- 172.21.15.5:0/3769003180 >> [v2:172.21.15.5:6800/3125821798,v1:172.21.15.5:6801/3125821798] conn(0x7f9fb0038fc0 0x7f9fb003b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:26.330 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:25.509+0000 7f9fb77fe640 1 -- 172.21.15.5:0/3769003180 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f9fb806c690 con 0x7f9fc8108060 2024-09-12T02:49:26.330 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:25.509+0000 7f9fc5d9b640 1 -- 172.21.15.5:0/3769003180 >> [v2:172.21.15.5:6800/3125821798,v1:172.21.15.5:6801/3125821798] conn(0x7f9fb0038fc0 msgr2=0x7f9fb003b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/3125821798 2024-09-12T02:49:26.330 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:25.509+0000 7f9fc5d9b640 1 --2- 172.21.15.5:0/3769003180 >> [v2:172.21.15.5:6800/3125821798,v1:172.21.15.5:6801/3125821798] conn(0x7f9fb0038fc0 0x7f9fb003b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T02:49:26.330 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:25.509+0000 7f9fc759e640 1 -- 172.21.15.5:0/3769003180 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fc81084c0 con 0x7f9fc8108060 2024-09-12T02:49:26.330 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:25.514+0000 7f9fb77fe640 1 -- 172.21.15.5:0/3769003180 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9fb8041270 con 0x7f9fc8108060 2024-09-12T02:49:26.330 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:25.710+0000 7f9fc5d9b640 1 -- 172.21.15.5:0/3769003180 >> [v2:172.21.15.5:6800/3125821798,v1:172.21.15.5:6801/3125821798] conn(0x7f9fb0038fc0 msgr2=0x7f9fb003b480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/3125821798 2024-09-12T02:49:26.330 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:25.710+0000 7f9fc5d9b640 1 --2- 172.21.15.5:0/3769003180 >> [v2:172.21.15.5:6800/3125821798,v1:172.21.15.5:6801/3125821798] conn(0x7f9fb0038fc0 0x7f9fb003b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-12T02:49:26.330 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:25.756+0000 7f9fc759e640 1 -- 172.21.15.5:0/3769003180 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f9fc819ef70 con 0x7f9fc8108060 2024-09-12T02:49:26.330 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:25.756+0000 7f9fb77fe640 1 -- 172.21.15.5:0/3769003180 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f9fb8043bf0 con 0x7f9fc8108060 2024-09-12T02:49:26.330 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:25.758+0000 7f9fc759e640 1 -- 172.21.15.5:0/3769003180 >> [v2:172.21.15.5:6800/3125821798,v1:172.21.15.5:6801/3125821798] conn(0x7f9fb0038fc0 msgr2=0x7f9fb003b480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T02:49:26.330 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:25.758+0000 7f9fc759e640 1 --2- 172.21.15.5:0/3769003180 >> [v2:172.21.15.5:6800/3125821798,v1:172.21.15.5:6801/3125821798] conn(0x7f9fb0038fc0 0x7f9fb003b480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:26.331 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:25.758+0000 7f9fc759e640 1 -- 172.21.15.5:0/3769003180 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9fc8108060 msgr2=0x7f9fc819b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:26.331 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:25.758+0000 7f9fc759e640 1 --2- 172.21.15.5:0/3769003180 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9fc8108060 0x7f9fc819b5f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9fb8030420 tx=0x7f9fb8032670 comp rx=0 tx=0).stop 2024-09-12T02:49:26.331 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:25.758+0000 7f9fc759e640 1 -- 172.21.15.5:0/3769003180 shutdown_connections 2024-09-12T02:49:26.331 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:25.758+0000 7f9fc759e640 1 --2- 172.21.15.5:0/3769003180 >> [v2:172.21.15.5:6800/3125821798,v1:172.21.15.5:6801/3125821798] conn(0x7f9fb0038fc0 0x7f9fb003b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:26.331 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:25.758+0000 7f9fc759e640 1 --2- 172.21.15.5:0/3769003180 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9fc8108060 0x7f9fc819b5f0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:26.331 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:25.758+0000 7f9fc759e640 1 -- 172.21.15.5:0/3769003180 >> 172.21.15.5:0/3769003180 conn(0x7f9fc8101bc0 msgr2=0x7f9fc8102800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:26.331 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:25.758+0000 7f9fc759e640 1 -- 172.21.15.5:0/3769003180 shutdown_connections 2024-09-12T02:49:26.331 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:25.758+0000 7f9fc759e640 1 -- 172.21.15.5:0/3769003180 wait complete. 2024-09-12T02:49:26.331 INFO:teuthology.orchestra.run.smithi005.stdout:Waiting for the mgr to restart... 2024-09-12T02:49:26.331 INFO:teuthology.orchestra.run.smithi005.stdout:Waiting for mgr epoch 9... 2024-09-12T02:49:26.577 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:26 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/3769003180' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-12T02:49:26.950 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:49:26 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: 2024-09-12T02:49:26.675+0000 7ff925705040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-12T02:49:26.951 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:49:26 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: 2024-09-12T02:49:26.748+0000 7ff925705040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-12T02:49:27.618 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:49:27 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: 2024-09-12T02:49:27.497+0000 7ff925705040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-12T02:49:27.618 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:49:27 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: 2024-09-12T02:49:27.617+0000 7ff925705040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-12T02:49:27.987 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:49:27 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: 2024-09-12T02:49:27.766+0000 7ff925705040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-12T02:49:27.987 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:49:27 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: 2024-09-12T02:49:27.830+0000 7ff925705040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-12T02:49:27.987 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:49:27 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: 2024-09-12T02:49:27.986+0000 7ff925705040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-12T02:49:28.286 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:49:28 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: 2024-09-12T02:49:28.057+0000 7ff925705040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-12T02:49:28.287 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:49:28 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: 2024-09-12T02:49:28.120+0000 7ff925705040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-12T02:49:28.287 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:49:28 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: 2024-09-12T02:49:28.286+0000 7ff925705040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-12T02:49:28.560 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:49:28 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: 2024-09-12T02:49:28.354+0000 7ff925705040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-12T02:49:28.560 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:49:28 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: 2024-09-12T02:49:28.423+0000 7ff925705040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-12T02:49:28.560 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:49:28 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: 2024-09-12T02:49:28.509+0000 7ff925705040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-12T02:49:29.060 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:49:28 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: 2024-09-12T02:49:28.571+0000 7ff925705040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-12T02:49:29.060 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:49:28 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: 2024-09-12T02:49:28.633+0000 7ff925705040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-12T02:49:29.712 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:49:29 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: 2024-09-12T02:49:29.460+0000 7ff925705040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-12T02:49:29.712 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:29 smithi005 ceph-mon[31197]: Active manager daemon x restarted 2024-09-12T02:49:29.712 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:29 smithi005 ceph-mon[31197]: Activating manager daemon x 2024-09-12T02:49:29.712 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:29 smithi005 ceph-mon[31197]: osdmap e3: 0 total, 0 up, 0 in 2024-09-12T02:49:29.712 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:29 smithi005 ceph-mon[31197]: mgrmap e10: x(active, starting, since 0.00314796s) 2024-09-12T02:49:29.712 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:29 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T02:49:29.713 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:29 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-12T02:49:29.713 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:29 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-12T02:49:29.713 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:29 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-12T02:49:29.713 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:29 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-12T02:49:29.713 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:29 smithi005 ceph-mon[31197]: Manager daemon x is now available 2024-09-12T02:49:29.713 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:29 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:49:29.713 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:29 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-12T02:49:29.713 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:29 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-12T02:49:31.194 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout { 2024-09-12T02:49:31.194 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-12T02:49:31.194 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-12T02:49:31.194 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout } 2024-09-12T02:49:31.195 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:26.742+0000 7f0c50f9a640 1 Processor -- start 2024-09-12T02:49:31.195 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:26.742+0000 7f0c50f9a640 1 -- start start 2024-09-12T02:49:31.195 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:26.742+0000 7f0c50f9a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c4c1065e0 0x7f0c4c1069e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:31.195 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:26.742+0000 7f0c50f9a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0c4c106fb0 con 0x7f0c4c1065e0 2024-09-12T02:49:31.195 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:26.743+0000 7f0c4a59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c4c1065e0 0x7f0c4c1069e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:31.195 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:26.743+0000 7f0c4a59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c4c1065e0 0x7f0c4c1069e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45680/0 (socket says 172.21.15.5:45680) 2024-09-12T02:49:31.195 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:26.743+0000 7f0c4a59c640 1 -- 172.21.15.5:0/1866555034 learned_addr learned my addr 172.21.15.5:0/1866555034 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:49:31.195 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:26.743+0000 7f0c4a59c640 1 -- 172.21.15.5:0/1866555034 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c4c1077e0 con 0x7f0c4c1065e0 2024-09-12T02:49:31.195 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:26.744+0000 7f0c4a59c640 1 --2- 172.21.15.5:0/1866555034 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c4c1065e0 0x7f0c4c1069e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c009820 tx=0x7f0c3c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a796037735259e7d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:31.195 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:26.744+0000 7f0c49d9b640 1 -- 172.21.15.5:0/1866555034 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0c3c037070 con 0x7f0c4c1065e0 2024-09-12T02:49:31.195 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:26.744+0000 7f0c49d9b640 1 -- 172.21.15.5:0/1866555034 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c3c032920 con 0x7f0c4c1065e0 2024-09-12T02:49:31.195 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:26.744+0000 7f0c49d9b640 1 -- 172.21.15.5:0/1866555034 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0c3c030810 con 0x7f0c4c1065e0 2024-09-12T02:49:31.196 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:26.744+0000 7f0c50f9a640 1 -- 172.21.15.5:0/1866555034 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c4c1065e0 msgr2=0x7f0c4c1069e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:31.196 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:26.744+0000 7f0c50f9a640 1 --2- 172.21.15.5:0/1866555034 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c4c1065e0 0x7f0c4c1069e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c009820 tx=0x7f0c3c02a270 comp rx=0 tx=0).stop 2024-09-12T02:49:31.196 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:26.745+0000 7f0c50f9a640 1 -- 172.21.15.5:0/1866555034 shutdown_connections 2024-09-12T02:49:31.196 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:26.745+0000 7f0c50f9a640 1 --2- 172.21.15.5:0/1866555034 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c4c1065e0 0x7f0c4c1069e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:31.196 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:26.745+0000 7f0c50f9a640 1 -- 172.21.15.5:0/1866555034 >> 172.21.15.5:0/1866555034 conn(0x7f0c4c101d90 msgr2=0x7f0c4c1041b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:31.196 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:26.745+0000 7f0c50f9a640 1 -- 172.21.15.5:0/1866555034 shutdown_connections 2024-09-12T02:49:31.196 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:26.745+0000 7f0c50f9a640 1 -- 172.21.15.5:0/1866555034 wait complete. 2024-09-12T02:49:31.196 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:26.745+0000 7f0c50f9a640 1 Processor -- start 2024-09-12T02:49:31.196 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:26.745+0000 7f0c50f9a640 1 -- start start 2024-09-12T02:49:31.196 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:26.746+0000 7f0c50f9a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c4c1065e0 0x7f0c4c19b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:31.196 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:26.746+0000 7f0c50f9a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0c4c10ab90 con 0x7f0c4c1065e0 2024-09-12T02:49:31.196 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:26.746+0000 7f0c4a59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c4c1065e0 0x7f0c4c19b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:31.196 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:26.746+0000 7f0c4a59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c4c1065e0 0x7f0c4c19b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45690/0 (socket says 172.21.15.5:45690) 2024-09-12T02:49:31.197 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:26.746+0000 7f0c4a59c640 1 -- 172.21.15.5:0/2322542186 learned_addr learned my addr 172.21.15.5:0/2322542186 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:49:31.197 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:26.746+0000 7f0c4a59c640 1 -- 172.21.15.5:0/2322542186 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c4c19bcf0 con 0x7f0c4c1065e0 2024-09-12T02:49:31.197 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:26.746+0000 7f0c4a59c640 1 --2- 172.21.15.5:0/2322542186 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c4c1065e0 0x7f0c4c19b7b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c030560 tx=0x7f0c3c032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:31.197 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:26.746+0000 7f0c437fe640 1 -- 172.21.15.5:0/2322542186 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0c3c037040 con 0x7f0c4c1065e0 2024-09-12T02:49:31.197 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:26.746+0000 7f0c437fe640 1 -- 172.21.15.5:0/2322542186 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c3c040070 con 0x7f0c4c1065e0 2024-09-12T02:49:31.197 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:26.746+0000 7f0c437fe640 1 -- 172.21.15.5:0/2322542186 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0c3c03a740 con 0x7f0c4c1065e0 2024-09-12T02:49:31.197 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:26.746+0000 7f0c50f9a640 1 -- 172.21.15.5:0/2322542186 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c4c19bf80 con 0x7f0c4c1065e0 2024-09-12T02:49:31.197 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:26.747+0000 7f0c50f9a640 1 -- 172.21.15.5:0/2322542186 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c4c19ec70 con 0x7f0c4c1065e0 2024-09-12T02:49:31.197 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:26.747+0000 7f0c437fe640 1 -- 172.21.15.5:0/2322542186 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f0c3c03a8e0 con 0x7f0c4c1065e0 2024-09-12T02:49:31.197 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:26.748+0000 7f0c437fe640 1 --2- 172.21.15.5:0/2322542186 >> [v2:172.21.15.5:6800/3125821798,v1:172.21.15.5:6801/3125821798] conn(0x7f0c38039020 0x7f0c3803b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:31.198 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:26.748+0000 7f0c437fe640 1 -- 172.21.15.5:0/2322542186 --> [v2:172.21.15.5:6800/3125821798,v1:172.21.15.5:6801/3125821798] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f0c3803bbf0 con 0x7f0c38039020 2024-09-12T02:49:31.198 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:26.748+0000 7f0c437fe640 1 -- 172.21.15.5:0/2322542186 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f0c3c06c6d0 con 0x7f0c4c1065e0 2024-09-12T02:49:31.198 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:26.748+0000 7f0c43fff640 1 -- 172.21.15.5:0/2322542186 >> [v2:172.21.15.5:6800/3125821798,v1:172.21.15.5:6801/3125821798] conn(0x7f0c38039020 msgr2=0x7f0c3803b4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/3125821798 2024-09-12T02:49:31.198 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:26.748+0000 7f0c43fff640 1 --2- 172.21.15.5:0/2322542186 >> [v2:172.21.15.5:6800/3125821798,v1:172.21.15.5:6801/3125821798] conn(0x7f0c38039020 0x7f0c3803b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T02:49:31.198 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:26.949+0000 7f0c43fff640 1 -- 172.21.15.5:0/2322542186 >> [v2:172.21.15.5:6800/3125821798,v1:172.21.15.5:6801/3125821798] conn(0x7f0c38039020 msgr2=0x7f0c3803b4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/3125821798 2024-09-12T02:49:31.198 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:26.949+0000 7f0c43fff640 1 --2- 172.21.15.5:0/2322542186 >> [v2:172.21.15.5:6800/3125821798,v1:172.21.15.5:6801/3125821798] conn(0x7f0c38039020 0x7f0c3803b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-12T02:49:31.198 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:27.350+0000 7f0c43fff640 1 -- 172.21.15.5:0/2322542186 >> [v2:172.21.15.5:6800/3125821798,v1:172.21.15.5:6801/3125821798] conn(0x7f0c38039020 msgr2=0x7f0c3803b4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/3125821798 2024-09-12T02:49:31.198 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:27.350+0000 7f0c43fff640 1 --2- 172.21.15.5:0/2322542186 >> [v2:172.21.15.5:6800/3125821798,v1:172.21.15.5:6801/3125821798] conn(0x7f0c38039020 0x7f0c3803b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-12T02:49:31.198 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:28.151+0000 7f0c43fff640 1 -- 172.21.15.5:0/2322542186 >> [v2:172.21.15.5:6800/3125821798,v1:172.21.15.5:6801/3125821798] conn(0x7f0c38039020 msgr2=0x7f0c3803b4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/3125821798 2024-09-12T02:49:31.198 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:28.151+0000 7f0c43fff640 1 --2- 172.21.15.5:0/2322542186 >> [v2:172.21.15.5:6800/3125821798,v1:172.21.15.5:6801/3125821798] conn(0x7f0c38039020 0x7f0c3803b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-12T02:49:31.199 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:29.463+0000 7f0c437fe640 1 -- 172.21.15.5:0/2322542186 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mgrmap(e 10) ==== 46067+0+0 (secure 0 0 0) 0x7f0c3c06b180 con 0x7f0c4c1065e0 2024-09-12T02:49:31.199 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:29.463+0000 7f0c437fe640 1 -- 172.21.15.5:0/2322542186 >> [v2:172.21.15.5:6800/3125821798,v1:172.21.15.5:6801/3125821798] conn(0x7f0c38039020 msgr2=0x7f0c3803b4e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T02:49:31.199 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:29.463+0000 7f0c437fe640 1 --2- 172.21.15.5:0/2322542186 >> [v2:172.21.15.5:6800/3125821798,v1:172.21.15.5:6801/3125821798] conn(0x7f0c38039020 0x7f0c3803b4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:31.199 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:30.465+0000 7f0c437fe640 1 -- 172.21.15.5:0/2322542186 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7f0c3c03abd0 con 0x7f0c4c1065e0 2024-09-12T02:49:31.199 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:30.465+0000 7f0c437fe640 1 --2- 172.21.15.5:0/2322542186 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f0c3803cae0 0x7f0c3803eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:31.199 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:30.465+0000 7f0c437fe640 1 -- 172.21.15.5:0/2322542186 --> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f0c3803bbf0 con 0x7f0c3803cae0 2024-09-12T02:49:31.199 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:30.465+0000 7f0c43fff640 1 --2- 172.21.15.5:0/2322542186 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f0c3803cae0 0x7f0c3803eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:31.199 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:30.466+0000 7f0c43fff640 1 --2- 172.21.15.5:0/2322542186 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f0c3803cae0 0x7f0c3803eed0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f0c34003980 tx=0x7f0c340092b0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:31.199 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:30.467+0000 7f0c437fe640 1 -- 172.21.15.5:0/2322542186 <== mgr.14148 v2:172.21.15.5:6800/3780631642 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f0c3803bbf0 con 0x7f0c3803cae0 2024-09-12T02:49:31.199 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:30.474+0000 7f0c50f9a640 1 -- 172.21.15.5:0/2322542186 --> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f0c4c10ad10 con 0x7f0c3803cae0 2024-09-12T02:49:31.199 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:30.474+0000 7f0c437fe640 1 -- 172.21.15.5:0/2322542186 <== mgr.14148 v2:172.21.15.5:6800/3780631642 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f0c4c10ad10 con 0x7f0c3803cae0 2024-09-12T02:49:31.200 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:30.474+0000 7f0c50f9a640 1 -- 172.21.15.5:0/2322542186 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f0c3803cae0 msgr2=0x7f0c3803eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:31.200 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:30.474+0000 7f0c50f9a640 1 --2- 172.21.15.5:0/2322542186 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f0c3803cae0 0x7f0c3803eed0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f0c34003980 tx=0x7f0c340092b0 comp rx=0 tx=0).stop 2024-09-12T02:49:31.200 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:30.474+0000 7f0c50f9a640 1 -- 172.21.15.5:0/2322542186 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c4c1065e0 msgr2=0x7f0c4c19b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:31.200 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:30.474+0000 7f0c50f9a640 1 --2- 172.21.15.5:0/2322542186 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c4c1065e0 0x7f0c4c19b7b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c030560 tx=0x7f0c3c032670 comp rx=0 tx=0).stop 2024-09-12T02:49:31.200 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:30.474+0000 7f0c50f9a640 1 -- 172.21.15.5:0/2322542186 shutdown_connections 2024-09-12T02:49:31.200 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:30.474+0000 7f0c50f9a640 1 --2- 172.21.15.5:0/2322542186 >> [v2:172.21.15.5:6800/3125821798,v1:172.21.15.5:6801/3125821798] conn(0x7f0c38039020 0x7f0c3803b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:31.200 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:30.474+0000 7f0c50f9a640 1 --2- 172.21.15.5:0/2322542186 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f0c3803cae0 0x7f0c3803eed0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:31.200 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:30.474+0000 7f0c50f9a640 1 --2- 172.21.15.5:0/2322542186 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c4c1065e0 0x7f0c4c19b7b0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:31.200 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:30.474+0000 7f0c50f9a640 1 -- 172.21.15.5:0/2322542186 >> 172.21.15.5:0/2322542186 conn(0x7f0c4c101d90 msgr2=0x7f0c4c102cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:31.200 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:30.475+0000 7f0c50f9a640 1 -- 172.21.15.5:0/2322542186 shutdown_connections 2024-09-12T02:49:31.200 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:30.475+0000 7f0c50f9a640 1 -- 172.21.15.5:0/2322542186 wait complete. 2024-09-12T02:49:31.200 INFO:teuthology.orchestra.run.smithi005.stdout:mgr epoch 9 is available 2024-09-12T02:49:31.200 INFO:teuthology.orchestra.run.smithi005.stdout:Generating a dashboard self-signed certificate... 2024-09-12T02:49:31.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:31 smithi005 ceph-mon[31197]: from='client.14152 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-12T02:49:31.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:31 smithi005 ceph-mon[31197]: mgrmap e11: x(active, since 1.00628s) 2024-09-12T02:49:31.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:31 smithi005 ceph-mon[31197]: from='client.14152 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-12T02:49:32.469 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:49:32 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: 2024-09-12T02:49:32.124+0000 7ff890859640 -1 log_channel(cephadm) log [ERR] : [12/Sep/2024:02:49:32] ENGINE Error in HTTPServer.serve 2024-09-12T02:49:32.470 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:49:32 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: Traceback (most recent call last): 2024-09-12T02:49:32.470 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:49:32 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-12T02:49:32.470 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:49:32 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: self._connections.run(self.expiration_interval) 2024-09-12T02:49:32.470 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:49:32 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-12T02:49:32.470 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:49:32 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: self._run(expiration_interval) 2024-09-12T02:49:32.470 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:49:32 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-12T02:49:32.470 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:49:32 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: new_conn = self._from_server_socket(self.server.socket) 2024-09-12T02:49:32.470 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:49:32 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-12T02:49:32.470 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:49:32 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-12T02:49:32.470 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:49:32 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-12T02:49:32.470 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:49:32 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: s = self.context.wrap_socket( 2024-09-12T02:49:32.470 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:49:32 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-12T02:49:32.471 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:49:32 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: return self.sslsocket_class._create( 2024-09-12T02:49:32.471 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:49:32 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-12T02:49:32.471 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:49:32 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: self.do_handshake() 2024-09-12T02:49:32.471 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:49:32 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-12T02:49:32.471 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:49:32 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: self._sslobj.do_handshake() 2024-09-12T02:49:32.471 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:49:32 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-12T02:49:32.471 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:49:32 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: 2024-09-12T02:49:32.482 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-09-12T02:49:32.483 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:31.626+0000 7fde865f1640 1 Processor -- start 2024-09-12T02:49:32.483 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:31.627+0000 7fde865f1640 1 -- start start 2024-09-12T02:49:32.483 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:31.627+0000 7fde865f1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde80108140 0x7fde8010a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:32.483 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:31.627+0000 7fde865f1640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fde8007be30 con 0x7fde80108140 2024-09-12T02:49:32.483 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:31.627+0000 7fde7ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde80108140 0x7fde8010a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:32.483 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:31.627+0000 7fde7ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde80108140 0x7fde8010a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45738/0 (socket says 172.21.15.5:45738) 2024-09-12T02:49:32.483 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:31.627+0000 7fde7ffff640 1 -- 172.21.15.5:0/903082709 learned_addr learned my addr 172.21.15.5:0/903082709 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:49:32.483 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:31.627+0000 7fde7ffff640 1 -- 172.21.15.5:0/903082709 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde8010aa70 con 0x7fde80108140 2024-09-12T02:49:32.483 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:31.628+0000 7fde7ffff640 1 --2- 172.21.15.5:0/903082709 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde80108140 0x7fde8010a530 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fde70009820 tx=0x7fde7002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a639b6b19559b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:32.483 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:31.628+0000 7fde7f7fe640 1 -- 172.21.15.5:0/903082709 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fde70037070 con 0x7fde80108140 2024-09-12T02:49:32.484 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:31.628+0000 7fde7f7fe640 1 -- 172.21.15.5:0/903082709 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde70032920 con 0x7fde80108140 2024-09-12T02:49:32.484 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:31.628+0000 7fde7f7fe640 1 -- 172.21.15.5:0/903082709 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fde70030810 con 0x7fde80108140 2024-09-12T02:49:32.484 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:31.629+0000 7fde865f1640 1 -- 172.21.15.5:0/903082709 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde80108140 msgr2=0x7fde8010a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:32.484 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:31.629+0000 7fde865f1640 1 --2- 172.21.15.5:0/903082709 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde80108140 0x7fde8010a530 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fde70009820 tx=0x7fde7002a270 comp rx=0 tx=0).stop 2024-09-12T02:49:32.484 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:31.629+0000 7fde865f1640 1 -- 172.21.15.5:0/903082709 shutdown_connections 2024-09-12T02:49:32.484 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:31.629+0000 7fde865f1640 1 --2- 172.21.15.5:0/903082709 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde80108140 0x7fde8010a530 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:32.484 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:31.629+0000 7fde865f1640 1 -- 172.21.15.5:0/903082709 >> 172.21.15.5:0/903082709 conn(0x7fde80101e20 msgr2=0x7fde80104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:32.484 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:31.629+0000 7fde865f1640 1 -- 172.21.15.5:0/903082709 shutdown_connections 2024-09-12T02:49:32.484 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:31.629+0000 7fde865f1640 1 -- 172.21.15.5:0/903082709 wait complete. 2024-09-12T02:49:32.485 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:31.630+0000 7fde865f1640 1 Processor -- start 2024-09-12T02:49:32.485 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:31.630+0000 7fde865f1640 1 -- start start 2024-09-12T02:49:32.485 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:31.630+0000 7fde865f1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde80108140 0x7fde8019d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:32.485 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:31.630+0000 7fde865f1640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fde8010bb50 con 0x7fde80108140 2024-09-12T02:49:32.485 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:31.630+0000 7fde7ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde80108140 0x7fde8019d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:32.485 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:31.630+0000 7fde7ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde80108140 0x7fde8019d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45748/0 (socket says 172.21.15.5:45748) 2024-09-12T02:49:32.485 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:31.630+0000 7fde7ffff640 1 -- 172.21.15.5:0/1174894990 learned_addr learned my addr 172.21.15.5:0/1174894990 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:49:32.485 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:31.630+0000 7fde7ffff640 1 -- 172.21.15.5:0/1174894990 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde8019de30 con 0x7fde80108140 2024-09-12T02:49:32.485 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:31.630+0000 7fde7ffff640 1 --2- 172.21.15.5:0/1174894990 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde80108140 0x7fde8019d8f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fde70030380 tx=0x7fde70032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:32.485 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:31.631+0000 7fde7dffb640 1 -- 172.21.15.5:0/1174894990 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fde70037040 con 0x7fde80108140 2024-09-12T02:49:32.485 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:31.631+0000 7fde7dffb640 1 -- 172.21.15.5:0/1174894990 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde70040070 con 0x7fde80108140 2024-09-12T02:49:32.485 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:31.631+0000 7fde865f1640 1 -- 172.21.15.5:0/1174894990 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde8019e0c0 con 0x7fde80108140 2024-09-12T02:49:32.485 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:31.631+0000 7fde865f1640 1 -- 172.21.15.5:0/1174894990 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde801a0db0 con 0x7fde80108140 2024-09-12T02:49:32.486 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:31.631+0000 7fde7dffb640 1 -- 172.21.15.5:0/1174894990 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fde7003a7b0 con 0x7fde80108140 2024-09-12T02:49:32.486 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:31.632+0000 7fde7dffb640 1 -- 172.21.15.5:0/1174894990 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7fde7003b420 con 0x7fde80108140 2024-09-12T02:49:32.486 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:31.632+0000 7fde865f1640 1 -- 172.21.15.5:0/1174894990 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde801085a0 con 0x7fde80108140 2024-09-12T02:49:32.486 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:31.632+0000 7fde7dffb640 1 --2- 172.21.15.5:0/1174894990 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fde6c039020 0x7fde6c03b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:32.486 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:31.632+0000 7fde7dffb640 1 -- 172.21.15.5:0/1174894990 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fde7006c6e0 con 0x7fde80108140 2024-09-12T02:49:32.486 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:31.632+0000 7fde77fff640 1 --2- 172.21.15.5:0/1174894990 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fde6c039020 0x7fde6c03b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:32.486 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:31.632+0000 7fde77fff640 1 --2- 172.21.15.5:0/1174894990 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fde6c039020 0x7fde6c03b4e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fde680098c0 tx=0x7fde68006db0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:32.486 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:31.637+0000 7fde7dffb640 1 -- 172.21.15.5:0/1174894990 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fde7004a200 con 0x7fde80108140 2024-09-12T02:49:32.486 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:31.821+0000 7fde865f1640 1 -- 172.21.15.5:0/1174894990 --> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7fde80103070 con 0x7fde6c039020 2024-09-12T02:49:32.486 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:31.868+0000 7fde7dffb640 1 -- 172.21.15.5:0/1174894990 <== mgr.14148 v2:172.21.15.5:6800/3780631642 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7fde80103070 con 0x7fde6c039020 2024-09-12T02:49:32.486 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:31.872+0000 7fde865f1640 1 -- 172.21.15.5:0/1174894990 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fde6c039020 msgr2=0x7fde6c03b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:32.486 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:31.872+0000 7fde865f1640 1 --2- 172.21.15.5:0/1174894990 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fde6c039020 0x7fde6c03b4e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fde680098c0 tx=0x7fde68006db0 comp rx=0 tx=0).stop 2024-09-12T02:49:32.486 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:31.872+0000 7fde865f1640 1 -- 172.21.15.5:0/1174894990 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde80108140 msgr2=0x7fde8019d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:32.487 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:31.872+0000 7fde865f1640 1 --2- 172.21.15.5:0/1174894990 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde80108140 0x7fde8019d8f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fde70030380 tx=0x7fde70032670 comp rx=0 tx=0).stop 2024-09-12T02:49:32.487 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:31.872+0000 7fde865f1640 1 -- 172.21.15.5:0/1174894990 shutdown_connections 2024-09-12T02:49:32.487 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:31.872+0000 7fde865f1640 1 --2- 172.21.15.5:0/1174894990 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fde6c039020 0x7fde6c03b4e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:32.487 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:31.872+0000 7fde865f1640 1 --2- 172.21.15.5:0/1174894990 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fde80108140 0x7fde8019d8f0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:32.487 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:31.872+0000 7fde865f1640 1 -- 172.21.15.5:0/1174894990 >> 172.21.15.5:0/1174894990 conn(0x7fde80101e20 msgr2=0x7fde80102960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:32.487 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:31.872+0000 7fde865f1640 1 -- 172.21.15.5:0/1174894990 shutdown_connections 2024-09-12T02:49:32.487 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:31.872+0000 7fde865f1640 1 -- 172.21.15.5:0/1174894990 wait complete. 2024-09-12T02:49:32.487 INFO:teuthology.orchestra.run.smithi005.stdout:Creating initial admin user... 2024-09-12T02:49:32.723 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:32 smithi005 ceph-mon[31197]: mgrmap e12: x(active, since 2s) 2024-09-12T02:49:32.724 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:32 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:49:32.724 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:32 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:49:32.724 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:32 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:49:32.724 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:32 smithi005 ceph-mon[31197]: [12/Sep/2024:02:49:32] ENGINE Bus STARTING 2024-09-12T02:49:32.724 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:32 smithi005 ceph-mon[31197]: [12/Sep/2024:02:49:32] ENGINE Serving on https://172.21.15.5:7150 2024-09-12T02:49:32.724 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:32 smithi005 ceph-mon[31197]: [12/Sep/2024:02:49:32] ENGINE Error in HTTPServer.serve 2024-09-12T02:49:32.724 INFO:journalctl@ceph.mon.a.smithi005.stdout: Traceback (most recent call last): 2024-09-12T02:49:32.724 INFO:journalctl@ceph.mon.a.smithi005.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-12T02:49:32.724 INFO:journalctl@ceph.mon.a.smithi005.stdout: self._connections.run(self.expiration_interval) 2024-09-12T02:49:32.724 INFO:journalctl@ceph.mon.a.smithi005.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-12T02:49:32.724 INFO:journalctl@ceph.mon.a.smithi005.stdout: self._run(expiration_interval) 2024-09-12T02:49:32.725 INFO:journalctl@ceph.mon.a.smithi005.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-12T02:49:32.725 INFO:journalctl@ceph.mon.a.smithi005.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-12T02:49:32.725 INFO:journalctl@ceph.mon.a.smithi005.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-12T02:49:32.725 INFO:journalctl@ceph.mon.a.smithi005.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-12T02:49:32.725 INFO:journalctl@ceph.mon.a.smithi005.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-12T02:49:32.725 INFO:journalctl@ceph.mon.a.smithi005.stdout: s = self.context.wrap_socket( 2024-09-12T02:49:32.725 INFO:journalctl@ceph.mon.a.smithi005.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-12T02:49:32.725 INFO:journalctl@ceph.mon.a.smithi005.stdout: return self.sslsocket_class._create( 2024-09-12T02:49:32.725 INFO:journalctl@ceph.mon.a.smithi005.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-12T02:49:32.725 INFO:journalctl@ceph.mon.a.smithi005.stdout: self.do_handshake() 2024-09-12T02:49:32.725 INFO:journalctl@ceph.mon.a.smithi005.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-12T02:49:32.725 INFO:journalctl@ceph.mon.a.smithi005.stdout: self._sslobj.do_handshake() 2024-09-12T02:49:32.726 INFO:journalctl@ceph.mon.a.smithi005.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-12T02:49:32.726 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:32 smithi005 ceph-mon[31197]: [12/Sep/2024:02:49:32] ENGINE Serving on http://172.21.15.5:8765 2024-09-12T02:49:32.726 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:32 smithi005 ceph-mon[31197]: [12/Sep/2024:02:49:32] ENGINE Bus STARTED 2024-09-12T02:49:33.744 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:33 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:49:33.745 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:33 smithi005 ceph-mon[31197]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:49:33.745 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:33 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:49:34.143 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$jUpGb4cPqrc/U8Gj6wTc3ukTBbTztCSBNbYR2PZxiqbvYMSR2cECe", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1726109373, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-12T02:49:34.144 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:32.929+0000 7f50af565640 1 Processor -- start 2024-09-12T02:49:34.144 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:32.929+0000 7f50af565640 1 -- start start 2024-09-12T02:49:34.144 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:32.929+0000 7f50af565640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50a8107780 0x7f50a8107b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:34.144 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:32.929+0000 7f50af565640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f50a8108150 con 0x7f50a8107780 2024-09-12T02:49:34.144 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:32.929+0000 7f50ad301640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50a8107780 0x7f50a8107b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:34.144 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:32.929+0000 7f50ad301640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50a8107780 0x7f50a8107b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45752/0 (socket says 172.21.15.5:45752) 2024-09-12T02:49:34.144 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:32.929+0000 7f50ad301640 1 -- 172.21.15.5:0/3742722104 learned_addr learned my addr 172.21.15.5:0/3742722104 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:49:34.144 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:32.930+0000 7f50ad301640 1 -- 172.21.15.5:0/3742722104 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50a8108980 con 0x7f50a8107780 2024-09-12T02:49:34.144 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:32.930+0000 7f50ad301640 1 --2- 172.21.15.5:0/3742722104 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50a8107780 0x7f50a8107b80 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f50a4009820 tx=0x7f50a402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25e2e0f80cb499d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:34.144 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:32.931+0000 7f509ffff640 1 -- 172.21.15.5:0/3742722104 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f50a4037070 con 0x7f50a8107780 2024-09-12T02:49:34.145 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:32.931+0000 7f509ffff640 1 -- 172.21.15.5:0/3742722104 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50a4032920 con 0x7f50a8107780 2024-09-12T02:49:34.145 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:32.931+0000 7f509ffff640 1 -- 172.21.15.5:0/3742722104 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f50a4030810 con 0x7f50a8107780 2024-09-12T02:49:34.145 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:32.931+0000 7f50af565640 1 -- 172.21.15.5:0/3742722104 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50a8107780 msgr2=0x7f50a8107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:34.145 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:32.931+0000 7f50af565640 1 --2- 172.21.15.5:0/3742722104 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50a8107780 0x7f50a8107b80 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f50a4009820 tx=0x7f50a402a270 comp rx=0 tx=0).stop 2024-09-12T02:49:34.145 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:32.931+0000 7f50af565640 1 -- 172.21.15.5:0/3742722104 shutdown_connections 2024-09-12T02:49:34.145 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:32.931+0000 7f50af565640 1 --2- 172.21.15.5:0/3742722104 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50a8107780 0x7f50a8107b80 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:34.145 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:32.931+0000 7f50af565640 1 -- 172.21.15.5:0/3742722104 >> 172.21.15.5:0/3742722104 conn(0x7f50a8102f30 msgr2=0x7f50a8105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:34.145 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:32.931+0000 7f50af565640 1 -- 172.21.15.5:0/3742722104 shutdown_connections 2024-09-12T02:49:34.145 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:32.931+0000 7f50af565640 1 -- 172.21.15.5:0/3742722104 wait complete. 2024-09-12T02:49:34.145 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:32.932+0000 7f50af565640 1 Processor -- start 2024-09-12T02:49:34.145 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:32.932+0000 7f50af565640 1 -- start start 2024-09-12T02:49:34.145 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:32.932+0000 7f50af565640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50a8107780 0x7f50a819b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:34.145 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:32.932+0000 7f50af565640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f50a810bd30 con 0x7f50a8107780 2024-09-12T02:49:34.146 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:32.932+0000 7f50ad301640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50a8107780 0x7f50a819b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:34.146 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:32.932+0000 7f50ad301640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50a8107780 0x7f50a819b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45768/0 (socket says 172.21.15.5:45768) 2024-09-12T02:49:34.146 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:32.932+0000 7f50ad301640 1 -- 172.21.15.5:0/407393762 learned_addr learned my addr 172.21.15.5:0/407393762 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:49:34.146 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:32.933+0000 7f50ad301640 1 -- 172.21.15.5:0/407393762 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50a819bc70 con 0x7f50a8107780 2024-09-12T02:49:34.146 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:32.933+0000 7f50ad301640 1 --2- 172.21.15.5:0/407393762 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50a8107780 0x7f50a819b730 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f50a4030560 tx=0x7f50a4032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:34.146 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:32.933+0000 7f509e7fc640 1 -- 172.21.15.5:0/407393762 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f50a4037040 con 0x7f50a8107780 2024-09-12T02:49:34.146 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:32.933+0000 7f509e7fc640 1 -- 172.21.15.5:0/407393762 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50a4040070 con 0x7f50a8107780 2024-09-12T02:49:34.146 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:32.933+0000 7f50af565640 1 -- 172.21.15.5:0/407393762 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50a819bf00 con 0x7f50a8107780 2024-09-12T02:49:34.146 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:32.933+0000 7f509e7fc640 1 -- 172.21.15.5:0/407393762 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f50a403a740 con 0x7f50a8107780 2024-09-12T02:49:34.146 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:32.933+0000 7f50af565640 1 -- 172.21.15.5:0/407393762 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50a819ebf0 con 0x7f50a8107780 2024-09-12T02:49:34.146 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:32.934+0000 7f509e7fc640 1 -- 172.21.15.5:0/407393762 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f50a403a8e0 con 0x7f50a8107780 2024-09-12T02:49:34.146 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:32.934+0000 7f50af565640 1 -- 172.21.15.5:0/407393762 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50a8107c00 con 0x7f50a8107780 2024-09-12T02:49:34.146 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:32.934+0000 7f509e7fc640 1 --2- 172.21.15.5:0/407393762 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f5094039020 0x7f509403b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:34.147 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:32.934+0000 7f509e7fc640 1 -- 172.21.15.5:0/407393762 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f50a406c930 con 0x7f50a8107780 2024-09-12T02:49:34.147 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:32.934+0000 7f50acb00640 1 --2- 172.21.15.5:0/407393762 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f5094039020 0x7f509403b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:34.147 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:32.935+0000 7f50acb00640 1 --2- 172.21.15.5:0/407393762 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f5094039020 0x7f509403b4e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f50980098c0 tx=0x7f5098006db0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:34.147 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:32.939+0000 7f509e7fc640 1 -- 172.21.15.5:0/407393762 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f50a4044c70 con 0x7f50a8107780 2024-09-12T02:49:34.147 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:33.139+0000 7f50af565640 1 -- 172.21.15.5:0/407393762 --> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f50a819c3e0 con 0x7f5094039020 2024-09-12T02:49:34.147 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:33.390+0000 7f509e7fc640 1 -- 172.21.15.5:0/407393762 <== mgr.14148 v2:172.21.15.5:6800/3780631642 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f50a819c3e0 con 0x7f5094039020 2024-09-12T02:49:34.147 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:33.392+0000 7f50af565640 1 -- 172.21.15.5:0/407393762 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f5094039020 msgr2=0x7f509403b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:34.147 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:33.392+0000 7f50af565640 1 --2- 172.21.15.5:0/407393762 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f5094039020 0x7f509403b4e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f50980098c0 tx=0x7f5098006db0 comp rx=0 tx=0).stop 2024-09-12T02:49:34.147 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:33.392+0000 7f50af565640 1 -- 172.21.15.5:0/407393762 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50a8107780 msgr2=0x7f50a819b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:34.147 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:33.392+0000 7f50af565640 1 --2- 172.21.15.5:0/407393762 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50a8107780 0x7f50a819b730 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f50a4030560 tx=0x7f50a4032670 comp rx=0 tx=0).stop 2024-09-12T02:49:34.147 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:33.392+0000 7f50af565640 1 -- 172.21.15.5:0/407393762 shutdown_connections 2024-09-12T02:49:34.147 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:33.392+0000 7f50af565640 1 --2- 172.21.15.5:0/407393762 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f5094039020 0x7f509403b4e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:34.147 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:33.392+0000 7f50af565640 1 --2- 172.21.15.5:0/407393762 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50a8107780 0x7f50a819b730 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:34.148 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:33.392+0000 7f50af565640 1 -- 172.21.15.5:0/407393762 >> 172.21.15.5:0/407393762 conn(0x7f50a8102f30 msgr2=0x7f50a8103ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:34.148 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:33.392+0000 7f50af565640 1 -- 172.21.15.5:0/407393762 shutdown_connections 2024-09-12T02:49:34.148 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:33.393+0000 7f50af565640 1 -- 172.21.15.5:0/407393762 wait complete. 2024-09-12T02:49:34.148 INFO:teuthology.orchestra.run.smithi005.stdout:Fetching dashboard port number... 2024-09-12T02:49:35.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:34 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/2699430625' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-09-12T02:49:35.645 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 8443 2024-09-12T02:49:35.645 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:34.530+0000 7fcdd2ff7640 1 Processor -- start 2024-09-12T02:49:35.645 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:34.530+0000 7fcdd2ff7640 1 -- start start 2024-09-12T02:49:35.645 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:34.531+0000 7fcdd2ff7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcdcc108180 0x7fcdcc10a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:35.645 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:34.531+0000 7fcdd2ff7640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcdcc07bdf0 con 0x7fcdcc108180 2024-09-12T02:49:35.645 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:34.531+0000 7fcdd0d93640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcdcc108180 0x7fcdcc10a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:35.645 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:34.531+0000 7fcdd0d93640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcdcc108180 0x7fcdcc10a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48204/0 (socket says 172.21.15.5:48204) 2024-09-12T02:49:35.648 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:34.531+0000 7fcdd0d93640 1 -- 172.21.15.5:0/1692838428 learned_addr learned my addr 172.21.15.5:0/1692838428 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:49:35.648 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:34.531+0000 7fcdd0d93640 1 -- 172.21.15.5:0/1692838428 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdcc10aab0 con 0x7fcdcc108180 2024-09-12T02:49:35.648 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:34.532+0000 7fcdd0d93640 1 --2- 172.21.15.5:0/1692838428 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcdcc108180 0x7fcdcc10a570 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fcdc0009820 tx=0x7fcdc002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3bf8fe8bc073e0b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:35.648 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:34.532+0000 7fcdcb7fe640 1 -- 172.21.15.5:0/1692838428 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcdc0037070 con 0x7fcdcc108180 2024-09-12T02:49:35.648 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:34.532+0000 7fcdcb7fe640 1 -- 172.21.15.5:0/1692838428 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcdc0032920 con 0x7fcdcc108180 2024-09-12T02:49:35.648 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:34.532+0000 7fcdcb7fe640 1 -- 172.21.15.5:0/1692838428 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcdc0030810 con 0x7fcdcc108180 2024-09-12T02:49:35.648 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:34.533+0000 7fcdd2ff7640 1 -- 172.21.15.5:0/1692838428 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcdcc108180 msgr2=0x7fcdcc10a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:35.649 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:34.533+0000 7fcdd2ff7640 1 --2- 172.21.15.5:0/1692838428 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcdcc108180 0x7fcdcc10a570 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fcdc0009820 tx=0x7fcdc002a270 comp rx=0 tx=0).stop 2024-09-12T02:49:35.649 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:34.533+0000 7fcdd2ff7640 1 -- 172.21.15.5:0/1692838428 shutdown_connections 2024-09-12T02:49:35.649 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:34.533+0000 7fcdd2ff7640 1 --2- 172.21.15.5:0/1692838428 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcdcc108180 0x7fcdcc10a570 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:35.649 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:34.533+0000 7fcdd2ff7640 1 -- 172.21.15.5:0/1692838428 >> 172.21.15.5:0/1692838428 conn(0x7fcdcc101e20 msgr2=0x7fcdcc104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:35.649 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:34.533+0000 7fcdd2ff7640 1 -- 172.21.15.5:0/1692838428 shutdown_connections 2024-09-12T02:49:35.649 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:34.533+0000 7fcdd2ff7640 1 -- 172.21.15.5:0/1692838428 wait complete. 2024-09-12T02:49:35.649 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:34.534+0000 7fcdd2ff7640 1 Processor -- start 2024-09-12T02:49:35.649 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:34.534+0000 7fcdd2ff7640 1 -- start start 2024-09-12T02:49:35.649 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:34.534+0000 7fcdd2ff7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcdcc108180 0x7fcdcc19d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:35.649 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:34.534+0000 7fcdd2ff7640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcdcc10bb30 con 0x7fcdcc108180 2024-09-12T02:49:35.650 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:34.534+0000 7fcdd0d93640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcdcc108180 0x7fcdcc19d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:35.650 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:34.534+0000 7fcdd0d93640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcdcc108180 0x7fcdcc19d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48208/0 (socket says 172.21.15.5:48208) 2024-09-12T02:49:35.650 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:34.534+0000 7fcdd0d93640 1 -- 172.21.15.5:0/2699430625 learned_addr learned my addr 172.21.15.5:0/2699430625 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:49:35.650 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:34.535+0000 7fcdd0d93640 1 -- 172.21.15.5:0/2699430625 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdcc19dec0 con 0x7fcdcc108180 2024-09-12T02:49:35.650 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:34.535+0000 7fcdd0d93640 1 --2- 172.21.15.5:0/2699430625 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcdcc108180 0x7fcdcc19d980 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fcdc0030380 tx=0x7fcdc0032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:35.650 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:34.535+0000 7fcdc9ffb640 1 -- 172.21.15.5:0/2699430625 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcdc0037040 con 0x7fcdcc108180 2024-09-12T02:49:35.650 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:34.535+0000 7fcdc9ffb640 1 -- 172.21.15.5:0/2699430625 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcdc0040070 con 0x7fcdcc108180 2024-09-12T02:49:35.650 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:34.535+0000 7fcdc9ffb640 1 -- 172.21.15.5:0/2699430625 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcdc003a7b0 con 0x7fcdcc108180 2024-09-12T02:49:35.650 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:34.535+0000 7fcdd2ff7640 1 -- 172.21.15.5:0/2699430625 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdcc19e150 con 0x7fcdcc108180 2024-09-12T02:49:35.650 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:34.535+0000 7fcdd2ff7640 1 -- 172.21.15.5:0/2699430625 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdcc1a0e40 con 0x7fcdcc108180 2024-09-12T02:49:35.650 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:34.537+0000 7fcdc9ffb640 1 -- 172.21.15.5:0/2699430625 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7fcdc003aa30 con 0x7fcdcc108180 2024-09-12T02:49:35.650 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:34.537+0000 7fcdd2ff7640 1 -- 172.21.15.5:0/2699430625 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcdcc1085e0 con 0x7fcdcc108180 2024-09-12T02:49:35.650 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:34.537+0000 7fcdc9ffb640 1 --2- 172.21.15.5:0/2699430625 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fcdbc038fc0 0x7fcdbc03b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:35.651 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:34.537+0000 7fcdc9ffb640 1 -- 172.21.15.5:0/2699430625 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fcdc006c880 con 0x7fcdcc108180 2024-09-12T02:49:35.651 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:34.537+0000 7fcdcbfff640 1 --2- 172.21.15.5:0/2699430625 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fcdbc038fc0 0x7fcdbc03b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:35.651 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:34.537+0000 7fcdcbfff640 1 --2- 172.21.15.5:0/2699430625 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fcdbc038fc0 0x7fcdbc03b480 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fcdb80098c0 tx=0x7fcdb8006db0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:35.651 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:34.546+0000 7fcdc9ffb640 1 -- 172.21.15.5:0/2699430625 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcdc003b420 con 0x7fcdcc108180 2024-09-12T02:49:35.651 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:34.724+0000 7fcdd2ff7640 1 -- 172.21.15.5:0/2699430625 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7fcdcc1a13c0 con 0x7fcdcc108180 2024-09-12T02:49:35.651 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:34.724+0000 7fcdc9ffb640 1 -- 172.21.15.5:0/2699430625 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7fcdc003ad20 con 0x7fcdcc108180 2024-09-12T02:49:35.651 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:34.726+0000 7fcdd2ff7640 1 -- 172.21.15.5:0/2699430625 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fcdbc038fc0 msgr2=0x7fcdbc03b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:35.651 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:34.726+0000 7fcdd2ff7640 1 --2- 172.21.15.5:0/2699430625 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fcdbc038fc0 0x7fcdbc03b480 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fcdb80098c0 tx=0x7fcdb8006db0 comp rx=0 tx=0).stop 2024-09-12T02:49:35.651 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:34.726+0000 7fcdd2ff7640 1 -- 172.21.15.5:0/2699430625 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcdcc108180 msgr2=0x7fcdcc19d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:35.651 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:34.726+0000 7fcdd2ff7640 1 --2- 172.21.15.5:0/2699430625 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcdcc108180 0x7fcdcc19d980 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fcdc0030380 tx=0x7fcdc0032670 comp rx=0 tx=0).stop 2024-09-12T02:49:35.651 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:34.727+0000 7fcdd2ff7640 1 -- 172.21.15.5:0/2699430625 shutdown_connections 2024-09-12T02:49:35.651 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:34.727+0000 7fcdd2ff7640 1 --2- 172.21.15.5:0/2699430625 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fcdbc038fc0 0x7fcdbc03b480 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:35.652 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:34.727+0000 7fcdd2ff7640 1 --2- 172.21.15.5:0/2699430625 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcdcc108180 0x7fcdcc19d980 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:35.652 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:34.727+0000 7fcdd2ff7640 1 -- 172.21.15.5:0/2699430625 >> 172.21.15.5:0/2699430625 conn(0x7fcdcc101e20 msgr2=0x7fcdcc102960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:35.652 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:34.727+0000 7fcdd2ff7640 1 -- 172.21.15.5:0/2699430625 shutdown_connections 2024-09-12T02:49:35.652 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:34.727+0000 7fcdd2ff7640 1 -- 172.21.15.5:0/2699430625 wait complete. 2024-09-12T02:49:35.656 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-12T02:49:35.656 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout disabled 2024-09-12T02:49:35.664 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-12T02:49:35.664 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout inactive 2024-09-12T02:49:35.664 INFO:teuthology.orchestra.run.smithi005.stdout:firewalld.service is not enabled 2024-09-12T02:49:35.664 INFO:teuthology.orchestra.run.smithi005.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-12T02:49:35.666 INFO:teuthology.orchestra.run.smithi005.stdout:Ceph Dashboard is now available at: 2024-09-12T02:49:35.666 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-12T02:49:35.666 INFO:teuthology.orchestra.run.smithi005.stdout: URL: https://smithi005.front.sepia.ceph.com:8443/ 2024-09-12T02:49:35.667 INFO:teuthology.orchestra.run.smithi005.stdout: User: admin 2024-09-12T02:49:35.667 INFO:teuthology.orchestra.run.smithi005.stdout: Password: 1lc3pa4pwi 2024-09-12T02:49:35.667 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-12T02:49:35.667 INFO:teuthology.orchestra.run.smithi005.stdout:Saving cluster configuration to /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/config directory 2024-09-12T02:49:36.851 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:36.061+0000 7f2c8e9c9640 1 Processor -- start 2024-09-12T02:49:36.851 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:36.061+0000 7f2c8e9c9640 1 -- start start 2024-09-12T02:49:36.851 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:36.061+0000 7f2c8e9c9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c88106bc0 0x7f2c88106fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:36.851 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:36.061+0000 7f2c8e9c9640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2c88107590 con 0x7f2c88106bc0 2024-09-12T02:49:36.851 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:36.061+0000 7f2c87fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c88106bc0 0x7f2c88106fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:36.851 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:36.061+0000 7f2c87fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c88106bc0 0x7f2c88106fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48224/0 (socket says 172.21.15.5:48224) 2024-09-12T02:49:36.851 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:36.061+0000 7f2c87fff640 1 -- 172.21.15.5:0/2901476550 learned_addr learned my addr 172.21.15.5:0/2901476550 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:49:36.851 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:36.062+0000 7f2c87fff640 1 -- 172.21.15.5:0/2901476550 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c881078e0 con 0x7f2c88106bc0 2024-09-12T02:49:36.852 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:36.062+0000 7f2c87fff640 1 --2- 172.21.15.5:0/2901476550 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c88106bc0 0x7f2c88106fc0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2c78009820 tx=0x7f2c7802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a39ffca8aa8bf73c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:36.852 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:36.063+0000 7f2c877fe640 1 -- 172.21.15.5:0/2901476550 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2c78037070 con 0x7f2c88106bc0 2024-09-12T02:49:36.852 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:36.063+0000 7f2c877fe640 1 -- 172.21.15.5:0/2901476550 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c78032920 con 0x7f2c88106bc0 2024-09-12T02:49:36.852 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:36.063+0000 7f2c877fe640 1 -- 172.21.15.5:0/2901476550 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2c78030810 con 0x7f2c88106bc0 2024-09-12T02:49:36.852 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:36.063+0000 7f2c8e9c9640 1 -- 172.21.15.5:0/2901476550 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c88106bc0 msgr2=0x7f2c88106fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:36.852 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:36.063+0000 7f2c8e9c9640 1 --2- 172.21.15.5:0/2901476550 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c88106bc0 0x7f2c88106fc0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2c78009820 tx=0x7f2c7802a270 comp rx=0 tx=0).stop 2024-09-12T02:49:36.852 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:36.063+0000 7f2c8e9c9640 1 -- 172.21.15.5:0/2901476550 shutdown_connections 2024-09-12T02:49:36.852 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:36.063+0000 7f2c8e9c9640 1 --2- 172.21.15.5:0/2901476550 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c88106bc0 0x7f2c88106fc0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:36.852 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:36.063+0000 7f2c8e9c9640 1 -- 172.21.15.5:0/2901476550 >> 172.21.15.5:0/2901476550 conn(0x7f2c881015f0 msgr2=0x7f2c88103a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:36.852 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:36.063+0000 7f2c8e9c9640 1 -- 172.21.15.5:0/2901476550 shutdown_connections 2024-09-12T02:49:36.852 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:36.063+0000 7f2c8e9c9640 1 -- 172.21.15.5:0/2901476550 wait complete. 2024-09-12T02:49:36.852 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:36.064+0000 7f2c8e9c9640 1 Processor -- start 2024-09-12T02:49:36.853 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:36.064+0000 7f2c8e9c9640 1 -- start start 2024-09-12T02:49:36.853 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:36.064+0000 7f2c8e9c9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c88106bc0 0x7f2c8819b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:36.853 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:36.064+0000 7f2c8e9c9640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2c8810ac90 con 0x7f2c88106bc0 2024-09-12T02:49:36.853 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:36.064+0000 7f2c87fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c88106bc0 0x7f2c8819b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:36.853 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:36.064+0000 7f2c87fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c88106bc0 0x7f2c8819b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48232/0 (socket says 172.21.15.5:48232) 2024-09-12T02:49:36.853 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:36.064+0000 7f2c87fff640 1 -- 172.21.15.5:0/2062827536 learned_addr learned my addr 172.21.15.5:0/2062827536 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:49:36.853 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:36.065+0000 7f2c87fff640 1 -- 172.21.15.5:0/2062827536 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c8819bcc0 con 0x7f2c88106bc0 2024-09-12T02:49:36.853 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:36.065+0000 7f2c87fff640 1 --2- 172.21.15.5:0/2062827536 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c88106bc0 0x7f2c8819b780 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2c780097f0 tx=0x7f2c780327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:36.853 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:36.065+0000 7f2c85ffb640 1 -- 172.21.15.5:0/2062827536 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2c78037040 con 0x7f2c88106bc0 2024-09-12T02:49:36.853 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:36.065+0000 7f2c85ffb640 1 -- 172.21.15.5:0/2062827536 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c78030cc0 con 0x7f2c88106bc0 2024-09-12T02:49:36.853 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:36.065+0000 7f2c8e9c9640 1 -- 172.21.15.5:0/2062827536 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c8819bf50 con 0x7f2c88106bc0 2024-09-12T02:49:36.853 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:36.065+0000 7f2c85ffb640 1 -- 172.21.15.5:0/2062827536 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2c7803a640 con 0x7f2c88106bc0 2024-09-12T02:49:36.853 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:36.065+0000 7f2c8e9c9640 1 -- 172.21.15.5:0/2062827536 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c8819ec40 con 0x7f2c88106bc0 2024-09-12T02:49:36.853 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:36.066+0000 7f2c85ffb640 1 -- 172.21.15.5:0/2062827536 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f2c7803a940 con 0x7f2c88106bc0 2024-09-12T02:49:36.854 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:36.066+0000 7f2c85ffb640 1 --2- 172.21.15.5:0/2062827536 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f2c74039060 0x7f2c7403b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:36.854 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:36.066+0000 7f2c85ffb640 1 -- 172.21.15.5:0/2062827536 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f2c7806c730 con 0x7f2c88106bc0 2024-09-12T02:49:36.854 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:36.066+0000 7f2c8e9c9640 1 -- 172.21.15.5:0/2062827536 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c88107040 con 0x7f2c88106bc0 2024-09-12T02:49:36.854 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:36.066+0000 7f2c7ffff640 1 --2- 172.21.15.5:0/2062827536 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f2c74039060 0x7f2c7403b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:36.854 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:36.067+0000 7f2c7ffff640 1 --2- 172.21.15.5:0/2062827536 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f2c74039060 0x7f2c7403b520 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2c700098c0 tx=0x7f2c70006db0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:36.854 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:36.071+0000 7f2c85ffb640 1 -- 172.21.15.5:0/2062827536 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2c78042300 con 0x7f2c88106bc0 2024-09-12T02:49:36.854 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:36.336+0000 7f2c8e9c9640 1 -- 172.21.15.5:0/2062827536 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f2c8802d950 con 0x7f2c88106bc0 2024-09-12T02:49:36.854 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:36.338+0000 7f2c85ffb640 1 -- 172.21.15.5:0/2062827536 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v27)=0 set mgr/dashboard/cluster/status v27) ==== 153+0+0 (secure 0 0 0) 0x7f2c7803ba40 con 0x7f2c88106bc0 2024-09-12T02:49:36.854 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-12T02:49:36.854 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:36.340+0000 7f2c8e9c9640 1 -- 172.21.15.5:0/2062827536 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f2c74039060 msgr2=0x7f2c7403b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:36.854 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:36.340+0000 7f2c8e9c9640 1 --2- 172.21.15.5:0/2062827536 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f2c74039060 0x7f2c7403b520 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2c700098c0 tx=0x7f2c70006db0 comp rx=0 tx=0).stop 2024-09-12T02:49:36.854 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:36.340+0000 7f2c8e9c9640 1 -- 172.21.15.5:0/2062827536 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c88106bc0 msgr2=0x7f2c8819b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:36.854 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:36.340+0000 7f2c8e9c9640 1 --2- 172.21.15.5:0/2062827536 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c88106bc0 0x7f2c8819b780 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2c780097f0 tx=0x7f2c780327b0 comp rx=0 tx=0).stop 2024-09-12T02:49:36.855 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:36.341+0000 7f2c8e9c9640 1 -- 172.21.15.5:0/2062827536 shutdown_connections 2024-09-12T02:49:36.855 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:36.341+0000 7f2c8e9c9640 1 --2- 172.21.15.5:0/2062827536 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f2c74039060 0x7f2c7403b520 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:36.855 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:36.341+0000 7f2c8e9c9640 1 --2- 172.21.15.5:0/2062827536 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c88106bc0 0x7f2c8819b780 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:36.855 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:36.341+0000 7f2c8e9c9640 1 -- 172.21.15.5:0/2062827536 >> 172.21.15.5:0/2062827536 conn(0x7f2c881015f0 msgr2=0x7f2c88102510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:36.855 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:36.341+0000 7f2c8e9c9640 1 -- 172.21.15.5:0/2062827536 shutdown_connections 2024-09-12T02:49:36.855 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2024-09-12T02:49:36.341+0000 7f2c8e9c9640 1 -- 172.21.15.5:0/2062827536 wait complete. 2024-09-12T02:49:36.855 INFO:teuthology.orchestra.run.smithi005.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-12T02:49:36.855 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-12T02:49:36.855 INFO:teuthology.orchestra.run.smithi005.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-12T02:49:36.855 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-12T02:49:36.855 INFO:teuthology.orchestra.run.smithi005.stdout:Or, if you are only running a single cluster on this host: 2024-09-12T02:49:36.855 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-12T02:49:36.855 INFO:teuthology.orchestra.run.smithi005.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-12T02:49:36.855 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-12T02:49:36.856 INFO:teuthology.orchestra.run.smithi005.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-12T02:49:36.856 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-12T02:49:36.856 INFO:teuthology.orchestra.run.smithi005.stdout: ceph telemetry on 2024-09-12T02:49:36.856 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-12T02:49:36.856 INFO:teuthology.orchestra.run.smithi005.stdout:For more information see: 2024-09-12T02:49:36.856 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-12T02:49:36.856 INFO:teuthology.orchestra.run.smithi005.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-12T02:49:36.856 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-12T02:49:36.856 INFO:teuthology.orchestra.run.smithi005.stdout:Bootstrap complete. 2024-09-12T02:49:36.891 INFO:tasks.cephadm:Fetching config... 2024-09-12T02:49:36.892 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-09-12T02:49:36.892 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-12T02:49:36.910 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-12T02:49:36.910 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-09-12T02:49:36.910 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-12T02:49:36.967 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-12T02:49:36.968 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-09-12T02:49:36.968 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/keyring of=/dev/stdout 2024-09-12T02:49:37.029 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:36 smithi005 ceph-mon[31197]: mgrmap e13: x(active, since 6s) 2024-09-12T02:49:37.030 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:36 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/2062827536' entity='client.admin' 2024-09-12T02:49:37.033 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-12T02:49:37.034 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-09-12T02:49:37.034 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-12T02:49:37.090 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-12T02:49:37.091 DEBUG:teuthology.orchestra.run.smithi005:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCkO7uq7a+tQ6DHnIxiHhyCltvl6iy/AWOzZGgRZdA2BT0+s8CNG2ITuTQLE4EbGMp8R39OwF6zRuJy9ltDSStD/i3e9DM7kj8RtZe586zqM2rsII3pfUdhYaNvUslnrIw9nZVBaNTvX8UHZ3+SK8ODUOrRJBuRnNHYrdKIRx8iyQEbxetAcxKEJjnr/atSKc1sGd0YgztITWr+J9SKlu0wxQ/Cel+I99Zz/oX4ACpiKeyG5uW1LzRLi3EUtT7ReVp7mCo9OE6Wt0RWefhNBnOnX2ejSwF8gaKVgEsmFaSXBuFeq+TczOninzrVwzXkHNqVHOnKIsNNROzwnAn85oA1Cf0iI2bETVmj76Fcwtuvy0PBmQJbGt76jjNfOXSYN8cVEEMC7v3ybTiKmr2SYWkmsm7YjohjnOlsmSMFwgjlZwWngYv8abwdMG9+vUrPPlAiYqW8UJLgvgsgjoJTS8vGtJkMU4w248YmcGSjLV2jK4aaFlab571vnWwwh33wnjc= ceph-618305f8-70b1-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-12T02:49:37.169 INFO:teuthology.orchestra.run.smithi005.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCkO7uq7a+tQ6DHnIxiHhyCltvl6iy/AWOzZGgRZdA2BT0+s8CNG2ITuTQLE4EbGMp8R39OwF6zRuJy9ltDSStD/i3e9DM7kj8RtZe586zqM2rsII3pfUdhYaNvUslnrIw9nZVBaNTvX8UHZ3+SK8ODUOrRJBuRnNHYrdKIRx8iyQEbxetAcxKEJjnr/atSKc1sGd0YgztITWr+J9SKlu0wxQ/Cel+I99Zz/oX4ACpiKeyG5uW1LzRLi3EUtT7ReVp7mCo9OE6Wt0RWefhNBnOnX2ejSwF8gaKVgEsmFaSXBuFeq+TczOninzrVwzXkHNqVHOnKIsNNROzwnAn85oA1Cf0iI2bETVmj76Fcwtuvy0PBmQJbGt76jjNfOXSYN8cVEEMC7v3ybTiKmr2SYWkmsm7YjohjnOlsmSMFwgjlZwWngYv8abwdMG9+vUrPPlAiYqW8UJLgvgsgjoJTS8vGtJkMU4w248YmcGSjLV2jK4aaFlab571vnWwwh33wnjc= ceph-618305f8-70b1-11ef-bcea-c7b262605968 2024-09-12T02:49:37.194 DEBUG:teuthology.orchestra.run.smithi146:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCkO7uq7a+tQ6DHnIxiHhyCltvl6iy/AWOzZGgRZdA2BT0+s8CNG2ITuTQLE4EbGMp8R39OwF6zRuJy9ltDSStD/i3e9DM7kj8RtZe586zqM2rsII3pfUdhYaNvUslnrIw9nZVBaNTvX8UHZ3+SK8ODUOrRJBuRnNHYrdKIRx8iyQEbxetAcxKEJjnr/atSKc1sGd0YgztITWr+J9SKlu0wxQ/Cel+I99Zz/oX4ACpiKeyG5uW1LzRLi3EUtT7ReVp7mCo9OE6Wt0RWefhNBnOnX2ejSwF8gaKVgEsmFaSXBuFeq+TczOninzrVwzXkHNqVHOnKIsNNROzwnAn85oA1Cf0iI2bETVmj76Fcwtuvy0PBmQJbGt76jjNfOXSYN8cVEEMC7v3ybTiKmr2SYWkmsm7YjohjnOlsmSMFwgjlZwWngYv8abwdMG9+vUrPPlAiYqW8UJLgvgsgjoJTS8vGtJkMU4w248YmcGSjLV2jK4aaFlab571vnWwwh33wnjc= ceph-618305f8-70b1-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-12T02:49:37.237 INFO:teuthology.orchestra.run.smithi146.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCkO7uq7a+tQ6DHnIxiHhyCltvl6iy/AWOzZGgRZdA2BT0+s8CNG2ITuTQLE4EbGMp8R39OwF6zRuJy9ltDSStD/i3e9DM7kj8RtZe586zqM2rsII3pfUdhYaNvUslnrIw9nZVBaNTvX8UHZ3+SK8ODUOrRJBuRnNHYrdKIRx8iyQEbxetAcxKEJjnr/atSKc1sGd0YgztITWr+J9SKlu0wxQ/Cel+I99Zz/oX4ACpiKeyG5uW1LzRLi3EUtT7ReVp7mCo9OE6Wt0RWefhNBnOnX2ejSwF8gaKVgEsmFaSXBuFeq+TczOninzrVwzXkHNqVHOnKIsNNROzwnAn85oA1Cf0iI2bETVmj76Fcwtuvy0PBmQJbGt76jjNfOXSYN8cVEEMC7v3ybTiKmr2SYWkmsm7YjohjnOlsmSMFwgjlZwWngYv8abwdMG9+vUrPPlAiYqW8UJLgvgsgjoJTS8vGtJkMU4w248YmcGSjLV2jK4aaFlab571vnWwwh33wnjc= ceph-618305f8-70b1-11ef-bcea-c7b262605968 2024-09-12T02:49:37.250 DEBUG:teuthology.orchestra.run.smithi148:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCkO7uq7a+tQ6DHnIxiHhyCltvl6iy/AWOzZGgRZdA2BT0+s8CNG2ITuTQLE4EbGMp8R39OwF6zRuJy9ltDSStD/i3e9DM7kj8RtZe586zqM2rsII3pfUdhYaNvUslnrIw9nZVBaNTvX8UHZ3+SK8ODUOrRJBuRnNHYrdKIRx8iyQEbxetAcxKEJjnr/atSKc1sGd0YgztITWr+J9SKlu0wxQ/Cel+I99Zz/oX4ACpiKeyG5uW1LzRLi3EUtT7ReVp7mCo9OE6Wt0RWefhNBnOnX2ejSwF8gaKVgEsmFaSXBuFeq+TczOninzrVwzXkHNqVHOnKIsNNROzwnAn85oA1Cf0iI2bETVmj76Fcwtuvy0PBmQJbGt76jjNfOXSYN8cVEEMC7v3ybTiKmr2SYWkmsm7YjohjnOlsmSMFwgjlZwWngYv8abwdMG9+vUrPPlAiYqW8UJLgvgsgjoJTS8vGtJkMU4w248YmcGSjLV2jK4aaFlab571vnWwwh33wnjc= ceph-618305f8-70b1-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-12T02:49:37.292 INFO:teuthology.orchestra.run.smithi148.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCkO7uq7a+tQ6DHnIxiHhyCltvl6iy/AWOzZGgRZdA2BT0+s8CNG2ITuTQLE4EbGMp8R39OwF6zRuJy9ltDSStD/i3e9DM7kj8RtZe586zqM2rsII3pfUdhYaNvUslnrIw9nZVBaNTvX8UHZ3+SK8ODUOrRJBuRnNHYrdKIRx8iyQEbxetAcxKEJjnr/atSKc1sGd0YgztITWr+J9SKlu0wxQ/Cel+I99Zz/oX4ACpiKeyG5uW1LzRLi3EUtT7ReVp7mCo9OE6Wt0RWefhNBnOnX2ejSwF8gaKVgEsmFaSXBuFeq+TczOninzrVwzXkHNqVHOnKIsNNROzwnAn85oA1Cf0iI2bETVmj76Fcwtuvy0PBmQJbGt76jjNfOXSYN8cVEEMC7v3ybTiKmr2SYWkmsm7YjohjnOlsmSMFwgjlZwWngYv8abwdMG9+vUrPPlAiYqW8UJLgvgsgjoJTS8vGtJkMU4w248YmcGSjLV2jK4aaFlab571vnWwwh33wnjc= ceph-618305f8-70b1-11ef-bcea-c7b262605968 2024-09-12T02:49:37.305 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-12T02:49:37.564 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:49:38.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:38.047+0000 7f9f4911a640 1 -- 172.21.15.5:0/2789568420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9f44100540 msgr2=0x7f9f44100940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:38.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:38.047+0000 7f9f4911a640 1 --2- 172.21.15.5:0/2789568420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9f44100540 0x7f9f44100940 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9f34009c20 tx=0x7f9f3402aa90 comp rx=0 tx=0).stop 2024-09-12T02:49:38.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:38.047+0000 7f9f4911a640 1 -- 172.21.15.5:0/2789568420 shutdown_connections 2024-09-12T02:49:38.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:38.047+0000 7f9f4911a640 1 --2- 172.21.15.5:0/2789568420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9f44100540 0x7f9f44100940 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:38.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:38.047+0000 7f9f4911a640 1 -- 172.21.15.5:0/2789568420 >> 172.21.15.5:0/2789568420 conn(0x7f9f440fbcd0 msgr2=0x7f9f440fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:38.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:38.047+0000 7f9f4911a640 1 -- 172.21.15.5:0/2789568420 shutdown_connections 2024-09-12T02:49:38.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:38.047+0000 7f9f4911a640 1 -- 172.21.15.5:0/2789568420 wait complete. 2024-09-12T02:49:38.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:38.048+0000 7f9f4911a640 1 Processor -- start 2024-09-12T02:49:38.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:38.048+0000 7f9f4911a640 1 -- start start 2024-09-12T02:49:38.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:38.048+0000 7f9f4911a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9f44100540 0x7f9f44196d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:38.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:38.048+0000 7f9f4911a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9f441065f0 con 0x7f9f44100540 2024-09-12T02:49:38.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:38.048+0000 7f9f42d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9f44100540 0x7f9f44196d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:38.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:38.048+0000 7f9f42d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9f44100540 0x7f9f44196d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48254/0 (socket says 172.21.15.5:48254) 2024-09-12T02:49:38.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:38.048+0000 7f9f42d9d640 1 -- 172.21.15.5:0/955866325 learned_addr learned my addr 172.21.15.5:0/955866325 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:49:38.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:38.049+0000 7f9f42d9d640 1 -- 172.21.15.5:0/955866325 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f441972a0 con 0x7f9f44100540 2024-09-12T02:49:38.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:38.049+0000 7f9f42d9d640 1 --2- 172.21.15.5:0/955866325 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9f44100540 0x7f9f44196d60 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9f34005cc0 tx=0x7f9f3402fc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:38.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:38.049+0000 7f9f3bfff640 1 -- 172.21.15.5:0/955866325 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9f34047070 con 0x7f9f44100540 2024-09-12T02:49:38.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:38.049+0000 7f9f3bfff640 1 -- 172.21.15.5:0/955866325 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f34003e50 con 0x7f9f44100540 2024-09-12T02:49:38.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:38.049+0000 7f9f4911a640 1 -- 172.21.15.5:0/955866325 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f44197530 con 0x7f9f44100540 2024-09-12T02:49:38.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:38.049+0000 7f9f3bfff640 1 -- 172.21.15.5:0/955866325 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9f34033050 con 0x7f9f44100540 2024-09-12T02:49:38.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:38.049+0000 7f9f4911a640 1 -- 172.21.15.5:0/955866325 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f4419a220 con 0x7f9f44100540 2024-09-12T02:49:38.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:38.050+0000 7f9f3bfff640 1 -- 172.21.15.5:0/955866325 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f9f3404a070 con 0x7f9f44100540 2024-09-12T02:49:38.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:38.050+0000 7f9f4911a640 1 -- 172.21.15.5:0/955866325 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f44106d70 con 0x7f9f44100540 2024-09-12T02:49:38.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:38.050+0000 7f9f3bfff640 1 --2- 172.21.15.5:0/955866325 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f9f300390c0 0x7f9f3003b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:38.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:38.050+0000 7f9f3bfff640 1 -- 172.21.15.5:0/955866325 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f9f3403e030 con 0x7f9f44100540 2024-09-12T02:49:38.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:38.050+0000 7f9f4259c640 1 --2- 172.21.15.5:0/955866325 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f9f300390c0 0x7f9f3003b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:38.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:38.051+0000 7f9f4259c640 1 --2- 172.21.15.5:0/955866325 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f9f300390c0 0x7f9f3003b580 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c0098c0 tx=0x7f9f2c006db0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:38.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:38.055+0000 7f9f3bfff640 1 -- 172.21.15.5:0/955866325 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9f340373b0 con 0x7f9f44100540 2024-09-12T02:49:38.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:38.230+0000 7f9f4911a640 1 -- 172.21.15.5:0/955866325 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f9f4402d830 con 0x7f9f44100540 2024-09-12T02:49:38.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:38.233+0000 7f9f3bfff640 1 -- 172.21.15.5:0/955866325 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f9f340492c0 con 0x7f9f44100540 2024-09-12T02:49:38.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:38.235+0000 7f9f4911a640 1 -- 172.21.15.5:0/955866325 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f9f300390c0 msgr2=0x7f9f3003b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:38.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:38.235+0000 7f9f4911a640 1 --2- 172.21.15.5:0/955866325 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f9f300390c0 0x7f9f3003b580 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c0098c0 tx=0x7f9f2c006db0 comp rx=0 tx=0).stop 2024-09-12T02:49:38.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:38.235+0000 7f9f4911a640 1 -- 172.21.15.5:0/955866325 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9f44100540 msgr2=0x7f9f44196d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:38.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:38.235+0000 7f9f4911a640 1 --2- 172.21.15.5:0/955866325 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9f44100540 0x7f9f44196d60 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9f34005cc0 tx=0x7f9f3402fc10 comp rx=0 tx=0).stop 2024-09-12T02:49:38.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:38.235+0000 7f9f4911a640 1 -- 172.21.15.5:0/955866325 shutdown_connections 2024-09-12T02:49:38.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:38.235+0000 7f9f4911a640 1 --2- 172.21.15.5:0/955866325 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f9f300390c0 0x7f9f3003b580 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:38.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:38.235+0000 7f9f4911a640 1 --2- 172.21.15.5:0/955866325 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9f44100540 0x7f9f44196d60 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:38.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:38.235+0000 7f9f4911a640 1 -- 172.21.15.5:0/955866325 >> 172.21.15.5:0/955866325 conn(0x7f9f440fbcd0 msgr2=0x7f9f44069b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:38.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:38.236+0000 7f9f4911a640 1 -- 172.21.15.5:0/955866325 shutdown_connections 2024-09-12T02:49:38.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:38.236+0000 7f9f4911a640 1 -- 172.21.15.5:0/955866325 wait complete. 2024-09-12T02:49:38.920 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-12T02:49:38.920 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-12T02:49:39.361 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:49:39.483 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:39 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/955866325' entity='client.admin' 2024-09-12T02:49:39.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:39.863+0000 7fe03b796640 1 -- 172.21.15.5:0/17525343 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe034100510 msgr2=0x7fe034100910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:39.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:39.863+0000 7fe03b796640 1 --2- 172.21.15.5:0/17525343 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe034100510 0x7fe034100910 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe030009880 tx=0x7fe03002a5d0 comp rx=0 tx=0).stop 2024-09-12T02:49:39.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:39.864+0000 7fe03b796640 1 -- 172.21.15.5:0/17525343 shutdown_connections 2024-09-12T02:49:39.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:39.864+0000 7fe03b796640 1 --2- 172.21.15.5:0/17525343 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe034100510 0x7fe034100910 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:39.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:39.864+0000 7fe03b796640 1 -- 172.21.15.5:0/17525343 >> 172.21.15.5:0/17525343 conn(0x7fe0340fbc80 msgr2=0x7fe0340fe0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:39.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:39.864+0000 7fe03b796640 1 -- 172.21.15.5:0/17525343 shutdown_connections 2024-09-12T02:49:39.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:39.864+0000 7fe03b796640 1 -- 172.21.15.5:0/17525343 wait complete. 2024-09-12T02:49:39.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:39.864+0000 7fe03b796640 1 Processor -- start 2024-09-12T02:49:39.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:39.864+0000 7fe03b796640 1 -- start start 2024-09-12T02:49:39.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:39.865+0000 7fe03b796640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe034100510 0x7fe034194ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:39.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:39.865+0000 7fe03b796640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe034105520 con 0x7fe034100510 2024-09-12T02:49:39.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:39.865+0000 7fe039532640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe034100510 0x7fe034194ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:39.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:39.865+0000 7fe039532640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe034100510 0x7fe034194ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48278/0 (socket says 172.21.15.5:48278) 2024-09-12T02:49:39.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:39.865+0000 7fe039532640 1 -- 172.21.15.5:0/4006386222 learned_addr learned my addr 172.21.15.5:0/4006386222 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:49:39.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:39.865+0000 7fe039532640 1 -- 172.21.15.5:0/4006386222 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0341950e0 con 0x7fe034100510 2024-09-12T02:49:39.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:39.865+0000 7fe039532640 1 --2- 172.21.15.5:0/4006386222 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe034100510 0x7fe034194ba0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe030004f60 tx=0x7fe0300048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:39.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:39.865+0000 7fe02a7fc640 1 -- 172.21.15.5:0/4006386222 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe03003f070 con 0x7fe034100510 2024-09-12T02:49:39.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:39.865+0000 7fe02a7fc640 1 -- 172.21.15.5:0/4006386222 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe030004cc0 con 0x7fe034100510 2024-09-12T02:49:39.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:39.865+0000 7fe02a7fc640 1 -- 172.21.15.5:0/4006386222 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe030035030 con 0x7fe034100510 2024-09-12T02:49:39.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:39.865+0000 7fe03b796640 1 -- 172.21.15.5:0/4006386222 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe034195370 con 0x7fe034100510 2024-09-12T02:49:39.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:39.865+0000 7fe03b796640 1 -- 172.21.15.5:0/4006386222 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe034198060 con 0x7fe034100510 2024-09-12T02:49:39.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:39.866+0000 7fe02a7fc640 1 -- 172.21.15.5:0/4006386222 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7fe030042070 con 0x7fe034100510 2024-09-12T02:49:39.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:39.866+0000 7fe03b796640 1 -- 172.21.15.5:0/4006386222 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe034105ca0 con 0x7fe034100510 2024-09-12T02:49:39.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:39.866+0000 7fe02a7fc640 1 --2- 172.21.15.5:0/4006386222 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fe02003b2c0 0x7fe02003d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:39.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:39.866+0000 7fe02a7fc640 1 -- 172.21.15.5:0/4006386222 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fe03006cd30 con 0x7fe034100510 2024-09-12T02:49:39.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:39.867+0000 7fe038d31640 1 --2- 172.21.15.5:0/4006386222 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fe02003b2c0 0x7fe02003d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:39.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:39.867+0000 7fe038d31640 1 --2- 172.21.15.5:0/4006386222 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fe02003b2c0 0x7fe02003d780 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fe0240098c0 tx=0x7fe024006db0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:39.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:39.871+0000 7fe02a7fc640 1 -- 172.21.15.5:0/4006386222 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe030003dd0 con 0x7fe034100510 2024-09-12T02:49:40.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:40.048+0000 7fe03b796640 1 -- 172.21.15.5:0/4006386222 --> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7fe03402d830 con 0x7fe02003b2c0 2024-09-12T02:49:40.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:40.052+0000 7fe02a7fc640 1 -- 172.21.15.5:0/4006386222 <== mgr.14148 v2:172.21.15.5:6800/3780631642 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe03402d830 con 0x7fe02003b2c0 2024-09-12T02:49:40.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:40.054+0000 7fe03b796640 1 -- 172.21.15.5:0/4006386222 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fe02003b2c0 msgr2=0x7fe02003d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:40.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:40.054+0000 7fe03b796640 1 --2- 172.21.15.5:0/4006386222 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fe02003b2c0 0x7fe02003d780 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fe0240098c0 tx=0x7fe024006db0 comp rx=0 tx=0).stop 2024-09-12T02:49:40.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:40.054+0000 7fe03b796640 1 -- 172.21.15.5:0/4006386222 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe034100510 msgr2=0x7fe034194ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:40.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:40.054+0000 7fe03b796640 1 --2- 172.21.15.5:0/4006386222 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe034100510 0x7fe034194ba0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe030004f60 tx=0x7fe0300048f0 comp rx=0 tx=0).stop 2024-09-12T02:49:40.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:40.054+0000 7fe03b796640 1 -- 172.21.15.5:0/4006386222 shutdown_connections 2024-09-12T02:49:40.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:40.054+0000 7fe03b796640 1 --2- 172.21.15.5:0/4006386222 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fe02003b2c0 0x7fe02003d780 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:40.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:40.054+0000 7fe03b796640 1 --2- 172.21.15.5:0/4006386222 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe034100510 0x7fe034194ba0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:40.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:40.054+0000 7fe03b796640 1 -- 172.21.15.5:0/4006386222 >> 172.21.15.5:0/4006386222 conn(0x7fe0340fbc80 msgr2=0x7fe0340fc680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:40.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:40.054+0000 7fe03b796640 1 -- 172.21.15.5:0/4006386222 shutdown_connections 2024-09-12T02:49:40.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:40.054+0000 7fe03b796640 1 -- 172.21.15.5:0/4006386222 wait complete. 2024-09-12T02:49:40.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:40 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:49:40.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:40 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:49:40.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:40 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi005", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:49:40.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:40 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:49:40.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:40 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:49:40.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:40 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:49:40.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:40 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:49:40.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:40 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:49:40.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:40 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:49:40.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:40 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:49:40.780 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi146 2024-09-12T02:49:40.781 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-12T02:49:40.781 DEBUG:teuthology.orchestra.run.smithi146:> dd of=/etc/ceph/ceph.conf 2024-09-12T02:49:40.800 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-12T02:49:40.800 DEBUG:teuthology.orchestra.run.smithi146:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-12T02:49:40.859 INFO:tasks.cephadm:Adding host smithi146 to orchestrator... 2024-09-12T02:49:40.859 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph orch host add smithi146 2024-09-12T02:49:41.113 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:49:41.537 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:41 smithi005 ceph-mon[31197]: from='client.14170 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:49:41.537 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:41 smithi005 ceph-mon[31197]: Updating smithi005:/etc/ceph/ceph.conf 2024-09-12T02:49:41.537 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:41 smithi005 ceph-mon[31197]: Updating smithi005:/var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T02:49:41.537 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:41 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:49:41.537 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:41 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:49:41.537 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:41 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:49:41.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:41.536+0000 7f2677cdd640 1 -- 172.21.15.5:0/1993162817 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2670107650 msgr2=0x7f2670107a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:41.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:41.536+0000 7f2677cdd640 1 --2- 172.21.15.5:0/1993162817 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2670107650 0x7f2670107a30 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f266c009880 tx=0x7f266c02a5d0 comp rx=0 tx=0).stop 2024-09-12T02:49:41.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:41.536+0000 7f2677cdd640 1 -- 172.21.15.5:0/1993162817 shutdown_connections 2024-09-12T02:49:41.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:41.536+0000 7f2677cdd640 1 --2- 172.21.15.5:0/1993162817 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2670107650 0x7f2670107a30 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:41.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:41.536+0000 7f2677cdd640 1 -- 172.21.15.5:0/1993162817 >> 172.21.15.5:0/1993162817 conn(0x7f26700fd260 msgr2=0x7f26700ff680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:41.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:41.536+0000 7f2677cdd640 1 -- 172.21.15.5:0/1993162817 shutdown_connections 2024-09-12T02:49:41.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:41.536+0000 7f2677cdd640 1 -- 172.21.15.5:0/1993162817 wait complete. 2024-09-12T02:49:41.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:41.536+0000 7f2677cdd640 1 Processor -- start 2024-09-12T02:49:41.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:41.537+0000 7f2677cdd640 1 -- start start 2024-09-12T02:49:41.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:41.537+0000 7f2677cdd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2670107650 0x7f26701970b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:41.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:41.537+0000 7f2677cdd640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f267010bc80 con 0x7f2670107650 2024-09-12T02:49:41.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:41.537+0000 7f2675a79640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2670107650 0x7f26701970b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:41.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:41.537+0000 7f2675a79640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2670107650 0x7f26701970b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48304/0 (socket says 172.21.15.5:48304) 2024-09-12T02:49:41.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:41.537+0000 7f2675a79640 1 -- 172.21.15.5:0/1858233459 learned_addr learned my addr 172.21.15.5:0/1858233459 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:49:41.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:41.537+0000 7f2675a79640 1 -- 172.21.15.5:0/1858233459 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26701975f0 con 0x7f2670107650 2024-09-12T02:49:41.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:41.537+0000 7f2675a79640 1 --2- 172.21.15.5:0/1858233459 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2670107650 0x7f26701970b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f266c009230 tx=0x7f266c003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:41.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:41.538+0000 7f2666ffd640 1 -- 172.21.15.5:0/1858233459 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f266c03f070 con 0x7f2670107650 2024-09-12T02:49:41.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:41.538+0000 7f2666ffd640 1 -- 172.21.15.5:0/1858233459 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f266c03bc80 con 0x7f2670107650 2024-09-12T02:49:41.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:41.538+0000 7f2677cdd640 1 -- 172.21.15.5:0/1858233459 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2670197880 con 0x7f2670107650 2024-09-12T02:49:41.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:41.538+0000 7f2666ffd640 1 -- 172.21.15.5:0/1858233459 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f266c035030 con 0x7f2670107650 2024-09-12T02:49:41.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:41.538+0000 7f2677cdd640 1 -- 172.21.15.5:0/1858233459 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f267019b980 con 0x7f2670107650 2024-09-12T02:49:41.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:41.539+0000 7f2666ffd640 1 -- 172.21.15.5:0/1858233459 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f266c042070 con 0x7f2670107650 2024-09-12T02:49:41.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:41.539+0000 7f2677cdd640 1 -- 172.21.15.5:0/1858233459 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2670102cc0 con 0x7f2670107650 2024-09-12T02:49:41.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:41.539+0000 7f2666ffd640 1 --2- 172.21.15.5:0/1858233459 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f265c039060 0x7f265c03b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:41.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:41.539+0000 7f2666ffd640 1 -- 172.21.15.5:0/1858233459 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f266c06cd00 con 0x7f2670107650 2024-09-12T02:49:41.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:41.539+0000 7f2675278640 1 --2- 172.21.15.5:0/1858233459 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f265c039060 0x7f265c03b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:41.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:41.539+0000 7f2675278640 1 --2- 172.21.15.5:0/1858233459 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f265c039060 0x7f265c03b520 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f26600098c0 tx=0x7f2660006db0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:41.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:41.543+0000 7f2666ffd640 1 -- 172.21.15.5:0/1858233459 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f266c031e20 con 0x7f2670107650 2024-09-12T02:49:41.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:41.725+0000 7f2677cdd640 1 -- 172.21.15.5:0/1858233459 --> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi146", "target": ["mon-mgr", ""]}) -- 0x7f2670198410 con 0x7f265c039060 2024-09-12T02:49:42.563 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:42 smithi005 ceph-mon[31197]: Updating smithi005:/etc/ceph/ceph.client.admin.keyring 2024-09-12T02:49:42.563 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:42 smithi005 ceph-mon[31197]: Updating smithi005:/var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T02:49:43.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:43 smithi005 ceph-mon[31197]: from='client.14172 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi146", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:49:44.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:44 smithi005 ceph-mon[31197]: Deploying cephadm binary to smithi146 2024-09-12T02:49:45.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:45.546+0000 7f2666ffd640 1 -- 172.21.15.5:0/1858233459 <== mgr.14148 v2:172.21.15.5:6800/3780631642 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f2670198410 con 0x7f265c039060 2024-09-12T02:49:45.547 INFO:teuthology.orchestra.run.smithi005.stdout:Added host 'smithi146' with addr '172.21.15.146' 2024-09-12T02:49:45.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:45.548+0000 7f2677cdd640 1 -- 172.21.15.5:0/1858233459 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f265c039060 msgr2=0x7f265c03b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:45.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:45.548+0000 7f2677cdd640 1 --2- 172.21.15.5:0/1858233459 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f265c039060 0x7f265c03b520 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f26600098c0 tx=0x7f2660006db0 comp rx=0 tx=0).stop 2024-09-12T02:49:45.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:45.548+0000 7f2677cdd640 1 -- 172.21.15.5:0/1858233459 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2670107650 msgr2=0x7f26701970b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:45.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:45.548+0000 7f2677cdd640 1 --2- 172.21.15.5:0/1858233459 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2670107650 0x7f26701970b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f266c009230 tx=0x7f266c003d40 comp rx=0 tx=0).stop 2024-09-12T02:49:45.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:45.548+0000 7f2677cdd640 1 -- 172.21.15.5:0/1858233459 shutdown_connections 2024-09-12T02:49:45.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:45.548+0000 7f2677cdd640 1 --2- 172.21.15.5:0/1858233459 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f265c039060 0x7f265c03b520 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:45.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:45.548+0000 7f2677cdd640 1 --2- 172.21.15.5:0/1858233459 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2670107650 0x7f26701970b0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:45.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:45.548+0000 7f2677cdd640 1 -- 172.21.15.5:0/1858233459 >> 172.21.15.5:0/1858233459 conn(0x7f26700fd260 msgr2=0x7f26700fd640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:45.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:45.548+0000 7f2677cdd640 1 -- 172.21.15.5:0/1858233459 shutdown_connections 2024-09-12T02:49:45.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:45.548+0000 7f2677cdd640 1 -- 172.21.15.5:0/1858233459 wait complete. 2024-09-12T02:49:46.093 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph orch host ls --format=json 2024-09-12T02:49:46.343 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:49:46.717 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:46 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:49:46.717 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:46 smithi005 ceph-mon[31197]: Added host smithi146 2024-09-12T02:49:46.718 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:46 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:49:46.718 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:46 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:49:46.718 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:46 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:49:46.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:46.799+0000 7fabc3f4a640 1 -- 172.21.15.5:0/2459495476 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fabbc1027d0 msgr2=0x7fabbc102bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:46.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:46.799+0000 7fabc3f4a640 1 --2- 172.21.15.5:0/2459495476 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fabbc1027d0 0x7fabbc102bb0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fabb4009810 tx=0x7fabb402a540 comp rx=0 tx=0).stop 2024-09-12T02:49:46.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:46.799+0000 7fabc3f4a640 1 -- 172.21.15.5:0/2459495476 shutdown_connections 2024-09-12T02:49:46.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:46.799+0000 7fabc3f4a640 1 --2- 172.21.15.5:0/2459495476 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fabbc1027d0 0x7fabbc102bb0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:46.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:46.799+0000 7fabc3f4a640 1 -- 172.21.15.5:0/2459495476 >> 172.21.15.5:0/2459495476 conn(0x7fabbc077310 msgr2=0x7fabbc0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:46.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:46.800+0000 7fabc3f4a640 1 -- 172.21.15.5:0/2459495476 shutdown_connections 2024-09-12T02:49:46.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:46.800+0000 7fabc3f4a640 1 -- 172.21.15.5:0/2459495476 wait complete. 2024-09-12T02:49:46.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:46.800+0000 7fabc3f4a640 1 Processor -- start 2024-09-12T02:49:46.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:46.800+0000 7fabc3f4a640 1 -- start start 2024-09-12T02:49:46.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:46.800+0000 7fabc3f4a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fabbc1027d0 0x7fabbc1971b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:46.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:46.800+0000 7fabc3f4a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fabbc10bc80 con 0x7fabbc1027d0 2024-09-12T02:49:46.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:46.800+0000 7fabc1ce6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fabbc1027d0 0x7fabbc1971b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:46.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:46.801+0000 7fabc1ce6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fabbc1027d0 0x7fabbc1971b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59916/0 (socket says 172.21.15.5:59916) 2024-09-12T02:49:46.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:46.801+0000 7fabc1ce6640 1 -- 172.21.15.5:0/237755807 learned_addr learned my addr 172.21.15.5:0/237755807 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:49:46.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:46.801+0000 7fabc1ce6640 1 -- 172.21.15.5:0/237755807 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabbc1976f0 con 0x7fabbc1027d0 2024-09-12T02:49:46.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:46.801+0000 7fabc1ce6640 1 --2- 172.21.15.5:0/237755807 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fabbc1027d0 0x7fabbc1971b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fabb40045e0 tx=0x7fabb4004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:46.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:46.801+0000 7fabbaffd640 1 -- 172.21.15.5:0/237755807 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fabb4030e70 con 0x7fabbc1027d0 2024-09-12T02:49:46.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:46.801+0000 7fabbaffd640 1 -- 172.21.15.5:0/237755807 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabb40318e0 con 0x7fabbc1027d0 2024-09-12T02:49:46.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:46.801+0000 7fabc3f4a640 1 -- 172.21.15.5:0/237755807 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabbc197980 con 0x7fabbc1027d0 2024-09-12T02:49:46.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:46.801+0000 7fabbaffd640 1 -- 172.21.15.5:0/237755807 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fabb40049c0 con 0x7fabbc1027d0 2024-09-12T02:49:46.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:46.801+0000 7fabc3f4a640 1 -- 172.21.15.5:0/237755807 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabbc197de0 con 0x7fabbc1027d0 2024-09-12T02:49:46.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:46.802+0000 7fabbaffd640 1 -- 172.21.15.5:0/237755807 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7fabb4039040 con 0x7fabbc1027d0 2024-09-12T02:49:46.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:46.802+0000 7fabc3f4a640 1 -- 172.21.15.5:0/237755807 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabbc069ef0 con 0x7fabbc1027d0 2024-09-12T02:49:46.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:46.802+0000 7fabbaffd640 1 --2- 172.21.15.5:0/237755807 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7faba4039060 0x7faba403b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:46.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:46.802+0000 7fabbaffd640 1 -- 172.21.15.5:0/237755807 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fabb406d410 con 0x7fabbc1027d0 2024-09-12T02:49:46.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:46.802+0000 7fabc14e5640 1 --2- 172.21.15.5:0/237755807 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7faba4039060 0x7faba403b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:46.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:46.803+0000 7fabc14e5640 1 --2- 172.21.15.5:0/237755807 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7faba4039060 0x7faba403b520 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fabb00098c0 tx=0x7fabb0006db0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:46.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:46.807+0000 7fabbaffd640 1 -- 172.21.15.5:0/237755807 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fabb4040030 con 0x7fabbc1027d0 2024-09-12T02:49:46.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:46.987+0000 7fabc3f4a640 1 -- 172.21.15.5:0/237755807 --> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fabbc198580 con 0x7faba4039060 2024-09-12T02:49:46.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:46.988+0000 7fabbaffd640 1 -- 172.21.15.5:0/237755807 <== mgr.14148 v2:172.21.15.5:6800/3780631642 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+159 (secure 0 0 0) 0x7fabbc198580 con 0x7faba4039060 2024-09-12T02:49:46.989 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-12T02:49:46.990 INFO:teuthology.orchestra.run.smithi005.stdout:[{"addr": "172.21.15.5", "hostname": "smithi005", "labels": [], "status": ""}, {"addr": "172.21.15.146", "hostname": "smithi146", "labels": [], "status": ""}] 2024-09-12T02:49:46.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:46.990+0000 7fabc3f4a640 1 -- 172.21.15.5:0/237755807 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7faba4039060 msgr2=0x7faba403b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:46.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:46.990+0000 7fabc3f4a640 1 --2- 172.21.15.5:0/237755807 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7faba4039060 0x7faba403b520 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fabb00098c0 tx=0x7fabb0006db0 comp rx=0 tx=0).stop 2024-09-12T02:49:46.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:46.990+0000 7fabc3f4a640 1 -- 172.21.15.5:0/237755807 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fabbc1027d0 msgr2=0x7fabbc1971b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:46.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:46.990+0000 7fabc3f4a640 1 --2- 172.21.15.5:0/237755807 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fabbc1027d0 0x7fabbc1971b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fabb40045e0 tx=0x7fabb4004610 comp rx=0 tx=0).stop 2024-09-12T02:49:46.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:46.990+0000 7fabc3f4a640 1 -- 172.21.15.5:0/237755807 shutdown_connections 2024-09-12T02:49:46.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:46.990+0000 7fabc3f4a640 1 --2- 172.21.15.5:0/237755807 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7faba4039060 0x7faba403b520 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:46.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:46.990+0000 7fabc3f4a640 1 --2- 172.21.15.5:0/237755807 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fabbc1027d0 0x7fabbc1971b0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:46.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:46.990+0000 7fabc3f4a640 1 -- 172.21.15.5:0/237755807 >> 172.21.15.5:0/237755807 conn(0x7fabbc077310 msgr2=0x7fabbc105560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:46.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:46.990+0000 7fabc3f4a640 1 -- 172.21.15.5:0/237755807 shutdown_connections 2024-09-12T02:49:46.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:46.990+0000 7fabc3f4a640 1 -- 172.21.15.5:0/237755807 wait complete. 2024-09-12T02:49:47.570 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi148 2024-09-12T02:49:47.570 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-12T02:49:47.570 DEBUG:teuthology.orchestra.run.smithi148:> dd of=/etc/ceph/ceph.conf 2024-09-12T02:49:47.590 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-12T02:49:47.590 DEBUG:teuthology.orchestra.run.smithi148:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-12T02:49:47.648 INFO:tasks.cephadm:Adding host smithi148 to orchestrator... 2024-09-12T02:49:47.649 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph orch host add smithi148 2024-09-12T02:49:47.904 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:49:48.014 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:47 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:49:48.015 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:47 smithi005 ceph-mon[31197]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:49:48.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:48.329+0000 7f24b7780640 1 -- 172.21.15.5:0/911818132 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f24b0106540 msgr2=0x7f24b0106920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:48.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:48.329+0000 7f24b7780640 1 --2- 172.21.15.5:0/911818132 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f24b0106540 0x7f24b0106920 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f24ac009810 tx=0x7f24ac02a540 comp rx=0 tx=0).stop 2024-09-12T02:49:48.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:48.329+0000 7f24b7780640 1 -- 172.21.15.5:0/911818132 shutdown_connections 2024-09-12T02:49:48.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:48.329+0000 7f24b7780640 1 --2- 172.21.15.5:0/911818132 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f24b0106540 0x7f24b0106920 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:48.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:48.329+0000 7f24b7780640 1 -- 172.21.15.5:0/911818132 >> 172.21.15.5:0/911818132 conn(0x7f24b0077310 msgr2=0x7f24b00fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:48.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:48.329+0000 7f24b7780640 1 -- 172.21.15.5:0/911818132 shutdown_connections 2024-09-12T02:49:48.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:48.329+0000 7f24b7780640 1 -- 172.21.15.5:0/911818132 wait complete. 2024-09-12T02:49:48.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:48.330+0000 7f24b7780640 1 Processor -- start 2024-09-12T02:49:48.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:48.330+0000 7f24b7780640 1 -- start start 2024-09-12T02:49:48.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:48.330+0000 7f24b7780640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f24b0106540 0x7f24b0196060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:48.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:48.330+0000 7f24b7780640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f24b010ab70 con 0x7f24b0106540 2024-09-12T02:49:48.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:48.330+0000 7f24b551c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f24b0106540 0x7f24b0196060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:48.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:48.330+0000 7f24b551c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f24b0106540 0x7f24b0196060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59934/0 (socket says 172.21.15.5:59934) 2024-09-12T02:49:48.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:48.330+0000 7f24b551c640 1 -- 172.21.15.5:0/4084347895 learned_addr learned my addr 172.21.15.5:0/4084347895 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:49:48.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:48.331+0000 7f24b551c640 1 -- 172.21.15.5:0/4084347895 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24b01965a0 con 0x7f24b0106540 2024-09-12T02:49:48.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:48.331+0000 7f24b551c640 1 --2- 172.21.15.5:0/4084347895 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f24b0106540 0x7f24b0196060 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f24ac0045e0 tx=0x7f24ac004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:48.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:48.331+0000 7f24a67fc640 1 -- 172.21.15.5:0/4084347895 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f24ac030e70 con 0x7f24b0106540 2024-09-12T02:49:48.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:48.331+0000 7f24b7780640 1 -- 172.21.15.5:0/4084347895 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24b0196830 con 0x7f24b0106540 2024-09-12T02:49:48.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:48.331+0000 7f24a67fc640 1 -- 172.21.15.5:0/4084347895 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f24ac0318e0 con 0x7f24b0106540 2024-09-12T02:49:48.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:48.331+0000 7f24a67fc640 1 -- 172.21.15.5:0/4084347895 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f24ac0049c0 con 0x7f24b0106540 2024-09-12T02:49:48.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:48.331+0000 7f24b7780640 1 -- 172.21.15.5:0/4084347895 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24b0196c90 con 0x7f24b0106540 2024-09-12T02:49:48.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:48.332+0000 7f24a67fc640 1 -- 172.21.15.5:0/4084347895 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f24ac039040 con 0x7f24b0106540 2024-09-12T02:49:48.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:48.332+0000 7f24b7780640 1 -- 172.21.15.5:0/4084347895 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24b0101bb0 con 0x7f24b0106540 2024-09-12T02:49:48.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:48.332+0000 7f24a67fc640 1 --2- 172.21.15.5:0/4084347895 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f249c039060 0x7f249c03b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:48.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:48.332+0000 7f24a67fc640 1 -- 172.21.15.5:0/4084347895 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f24ac06d300 con 0x7f24b0106540 2024-09-12T02:49:48.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:48.332+0000 7f24b4d1b640 1 --2- 172.21.15.5:0/4084347895 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f249c039060 0x7f249c03b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:48.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:48.333+0000 7f24b4d1b640 1 --2- 172.21.15.5:0/4084347895 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f249c039060 0x7f249c03b520 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f24a00098c0 tx=0x7f24a0006db0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:48.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:48.338+0000 7f24a67fc640 1 -- 172.21.15.5:0/4084347895 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f24ac040030 con 0x7f24b0106540 2024-09-12T02:49:48.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:48.520+0000 7f24b7780640 1 -- 172.21.15.5:0/4084347895 --> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi148", "target": ["mon-mgr", ""]}) -- 0x7f24b0197420 con 0x7f249c039060 2024-09-12T02:49:49.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:49 smithi005 ceph-mon[31197]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi148", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:49:51.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:50 smithi005 ceph-mon[31197]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:49:51.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:50 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:49:51.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:50 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:49:51.061 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:50 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:49:51.061 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:50 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:49:51.061 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:50 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi146", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:49:51.061 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:50 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:49:51.061 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:50 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:49:51.061 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:50 smithi005 ceph-mon[31197]: Updating smithi146:/etc/ceph/ceph.conf 2024-09-12T02:49:51.061 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:50 smithi005 ceph-mon[31197]: Updating smithi146:/var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T02:49:51.061 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:50 smithi005 ceph-mon[31197]: Deploying cephadm binary to smithi148 2024-09-12T02:49:51.061 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:50 smithi005 ceph-mon[31197]: Updating smithi146:/etc/ceph/ceph.client.admin.keyring 2024-09-12T02:49:51.061 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:50 smithi005 ceph-mon[31197]: Updating smithi146:/var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T02:49:51.061 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:50 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:49:51.061 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:50 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:49:51.061 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:50 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:49:52.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:52.508+0000 7f24a67fc640 1 -- 172.21.15.5:0/4084347895 <== mgr.14148 v2:172.21.15.5:6800/3780631642 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f24b0197420 con 0x7f249c039060 2024-09-12T02:49:52.509 INFO:teuthology.orchestra.run.smithi005.stdout:Added host 'smithi148' with addr '172.21.15.148' 2024-09-12T02:49:52.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:52.511+0000 7f24b7780640 1 -- 172.21.15.5:0/4084347895 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f249c039060 msgr2=0x7f249c03b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:52.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:52.511+0000 7f24b7780640 1 --2- 172.21.15.5:0/4084347895 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f249c039060 0x7f249c03b520 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f24a00098c0 tx=0x7f24a0006db0 comp rx=0 tx=0).stop 2024-09-12T02:49:52.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:52.511+0000 7f24b7780640 1 -- 172.21.15.5:0/4084347895 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f24b0106540 msgr2=0x7f24b0196060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:52.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:52.511+0000 7f24b7780640 1 --2- 172.21.15.5:0/4084347895 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f24b0106540 0x7f24b0196060 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f24ac0045e0 tx=0x7f24ac004610 comp rx=0 tx=0).stop 2024-09-12T02:49:52.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:52.511+0000 7f24b7780640 1 -- 172.21.15.5:0/4084347895 shutdown_connections 2024-09-12T02:49:52.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:52.511+0000 7f24b7780640 1 --2- 172.21.15.5:0/4084347895 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f249c039060 0x7f249c03b520 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:52.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:52.511+0000 7f24b7780640 1 --2- 172.21.15.5:0/4084347895 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f24b0106540 0x7f24b0196060 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:52.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:52.511+0000 7f24b7780640 1 -- 172.21.15.5:0/4084347895 >> 172.21.15.5:0/4084347895 conn(0x7f24b0077310 msgr2=0x7f24b00fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:52.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:52.511+0000 7f24b7780640 1 -- 172.21.15.5:0/4084347895 shutdown_connections 2024-09-12T02:49:52.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:52.512+0000 7f24b7780640 1 -- 172.21.15.5:0/4084347895 wait complete. 2024-09-12T02:49:52.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:52 smithi005 ceph-mon[31197]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:49:52.829 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:52 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:49:52.829 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:52 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:49:53.116 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph orch host ls --format=json 2024-09-12T02:49:53.366 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:49:53.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:53.796+0000 7f4cc7dc2640 1 -- 172.21.15.5:0/3368267766 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4cc0106540 msgr2=0x7f4cc0106920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:53.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:53.796+0000 7f4cc7dc2640 1 --2- 172.21.15.5:0/3368267766 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4cc0106540 0x7f4cc0106920 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4cb0009c10 tx=0x7f4cb002aa10 comp rx=0 tx=0).stop 2024-09-12T02:49:53.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:53.796+0000 7f4cc7dc2640 1 -- 172.21.15.5:0/3368267766 shutdown_connections 2024-09-12T02:49:53.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:53.796+0000 7f4cc7dc2640 1 --2- 172.21.15.5:0/3368267766 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4cc0106540 0x7f4cc0106920 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:53.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:53.796+0000 7f4cc7dc2640 1 -- 172.21.15.5:0/3368267766 >> 172.21.15.5:0/3368267766 conn(0x7f4cc0077310 msgr2=0x7f4cc00fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:53.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:53.796+0000 7f4cc7dc2640 1 -- 172.21.15.5:0/3368267766 shutdown_connections 2024-09-12T02:49:53.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:53.796+0000 7f4cc7dc2640 1 -- 172.21.15.5:0/3368267766 wait complete. 2024-09-12T02:49:53.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:53.796+0000 7f4cc7dc2640 1 Processor -- start 2024-09-12T02:49:53.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:53.796+0000 7f4cc7dc2640 1 -- start start 2024-09-12T02:49:53.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:53.796+0000 7f4cc7dc2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4cc0106540 0x7f4cc01970d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:53.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:53.796+0000 7f4cc7dc2640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc010ab70 con 0x7f4cc0106540 2024-09-12T02:49:53.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:53.797+0000 7f4cc5b5e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4cc0106540 0x7f4cc01970d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:53.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:53.797+0000 7f4cc5b5e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4cc0106540 0x7f4cc01970d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44582/0 (socket says 172.21.15.5:44582) 2024-09-12T02:49:53.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:53.797+0000 7f4cc5b5e640 1 -- 172.21.15.5:0/645874645 learned_addr learned my addr 172.21.15.5:0/645874645 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:49:53.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:53.797+0000 7f4cc5b5e640 1 -- 172.21.15.5:0/645874645 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cc0197610 con 0x7f4cc0106540 2024-09-12T02:49:53.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:53.797+0000 7f4cc5b5e640 1 --2- 172.21.15.5:0/645874645 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4cc0106540 0x7f4cc01970d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4cb0005dd0 tx=0x7f4cb0030c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:53.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:53.797+0000 7f4cbeffd640 1 -- 172.21.15.5:0/645874645 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4cb0034070 con 0x7f4cc0106540 2024-09-12T02:49:53.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:53.797+0000 7f4cbeffd640 1 -- 172.21.15.5:0/645874645 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4cb002f8c0 con 0x7f4cc0106540 2024-09-12T02:49:53.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:53.797+0000 7f4cbeffd640 1 -- 172.21.15.5:0/645874645 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4cb0004520 con 0x7f4cc0106540 2024-09-12T02:49:53.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:53.797+0000 7f4cc7dc2640 1 -- 172.21.15.5:0/645874645 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cc01978a0 con 0x7f4cc0106540 2024-09-12T02:49:53.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:53.797+0000 7f4cc7dc2640 1 -- 172.21.15.5:0/645874645 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cc0197cc0 con 0x7f4cc0106540 2024-09-12T02:49:53.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:53.798+0000 7f4cbeffd640 1 -- 172.21.15.5:0/645874645 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f4cb0037020 con 0x7f4cc0106540 2024-09-12T02:49:53.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:53.798+0000 7f4cc7dc2640 1 -- 172.21.15.5:0/645874645 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4cc0101bb0 con 0x7f4cc0106540 2024-09-12T02:49:53.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:53.798+0000 7f4cbeffd640 1 --2- 172.21.15.5:0/645874645 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f4ca80390c0 0x7f4ca803b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:53.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:53.798+0000 7f4cbeffd640 1 -- 172.21.15.5:0/645874645 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f4cb006ceb0 con 0x7f4cc0106540 2024-09-12T02:49:53.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:53.799+0000 7f4cc535d640 1 --2- 172.21.15.5:0/645874645 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f4ca80390c0 0x7f4ca803b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:53.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:53.799+0000 7f4cc535d640 1 --2- 172.21.15.5:0/645874645 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f4ca80390c0 0x7f4ca803b580 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f4cb40098c0 tx=0x7f4cb4006db0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:53.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:53.803+0000 7f4cbeffd640 1 -- 172.21.15.5:0/645874645 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4cb003f070 con 0x7f4cc0106540 2024-09-12T02:49:53.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:53.985+0000 7f4cc7dc2640 1 -- 172.21.15.5:0/645874645 --> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4cc01984d0 con 0x7f4ca80390c0 2024-09-12T02:49:53.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:53.986+0000 7f4cbeffd640 1 -- 172.21.15.5:0/645874645 <== mgr.14148 v2:172.21.15.5:6800/3780631642 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+239 (secure 0 0 0) 0x7f4cc01984d0 con 0x7f4ca80390c0 2024-09-12T02:49:53.986 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-12T02:49:53.986 INFO:teuthology.orchestra.run.smithi005.stdout:[{"addr": "172.21.15.5", "hostname": "smithi005", "labels": [], "status": ""}, {"addr": "172.21.15.146", "hostname": "smithi146", "labels": [], "status": ""}, {"addr": "172.21.15.148", "hostname": "smithi148", "labels": [], "status": ""}] 2024-09-12T02:49:53.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:53.988+0000 7f4cc7dc2640 1 -- 172.21.15.5:0/645874645 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f4ca80390c0 msgr2=0x7f4ca803b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:53.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:53.988+0000 7f4cc7dc2640 1 --2- 172.21.15.5:0/645874645 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f4ca80390c0 0x7f4ca803b580 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f4cb40098c0 tx=0x7f4cb4006db0 comp rx=0 tx=0).stop 2024-09-12T02:49:53.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:53.988+0000 7f4cc7dc2640 1 -- 172.21.15.5:0/645874645 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4cc0106540 msgr2=0x7f4cc01970d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:53.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:53.988+0000 7f4cc7dc2640 1 --2- 172.21.15.5:0/645874645 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4cc0106540 0x7f4cc01970d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4cb0005dd0 tx=0x7f4cb0030c30 comp rx=0 tx=0).stop 2024-09-12T02:49:53.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:53.988+0000 7f4cc7dc2640 1 -- 172.21.15.5:0/645874645 shutdown_connections 2024-09-12T02:49:53.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:53.988+0000 7f4cc7dc2640 1 --2- 172.21.15.5:0/645874645 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f4ca80390c0 0x7f4ca803b580 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:53.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:53.988+0000 7f4cc7dc2640 1 --2- 172.21.15.5:0/645874645 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4cc0106540 0x7f4cc01970d0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:53.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:53.988+0000 7f4cc7dc2640 1 -- 172.21.15.5:0/645874645 >> 172.21.15.5:0/645874645 conn(0x7f4cc0077310 msgr2=0x7f4cc00696b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:53.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:53.988+0000 7f4cc7dc2640 1 -- 172.21.15.5:0/645874645 shutdown_connections 2024-09-12T02:49:53.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:53.988+0000 7f4cc7dc2640 1 -- 172.21.15.5:0/645874645 wait complete. 2024-09-12T02:49:54.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:53 smithi005 ceph-mon[31197]: Added host smithi148 2024-09-12T02:49:54.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:53 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:49:54.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:53 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:49:54.572 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-12T02:49:54.573 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph osd crush tunables default 2024-09-12T02:49:54.823 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:49:55.187 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:54 smithi005 ceph-mon[31197]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:49:55.187 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:54 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:49:55.187 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:54 smithi005 ceph-mon[31197]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:49:55.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:55.270+0000 7fd1c54ea640 1 -- 172.21.15.5:0/3906146091 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd1c0106560 msgr2=0x7fd1c0106940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:55.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:55.270+0000 7fd1c54ea640 1 --2- 172.21.15.5:0/3906146091 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd1c0106560 0x7fd1c0106940 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd1b0009880 tx=0x7fd1b002a5d0 comp rx=0 tx=0).stop 2024-09-12T02:49:55.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:55.270+0000 7fd1c54ea640 1 -- 172.21.15.5:0/3906146091 shutdown_connections 2024-09-12T02:49:55.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:55.270+0000 7fd1c54ea640 1 --2- 172.21.15.5:0/3906146091 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd1c0106560 0x7fd1c0106940 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:55.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:55.270+0000 7fd1c54ea640 1 -- 172.21.15.5:0/3906146091 >> 172.21.15.5:0/3906146091 conn(0x7fd1c00fc1b0 msgr2=0x7fd1c00fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:55.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:55.270+0000 7fd1c54ea640 1 -- 172.21.15.5:0/3906146091 shutdown_connections 2024-09-12T02:49:55.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:55.270+0000 7fd1c54ea640 1 -- 172.21.15.5:0/3906146091 wait complete. 2024-09-12T02:49:55.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:55.271+0000 7fd1c54ea640 1 Processor -- start 2024-09-12T02:49:55.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:55.271+0000 7fd1c54ea640 1 -- start start 2024-09-12T02:49:55.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:55.271+0000 7fd1c54ea640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd1c0106560 0x7fd1c0197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:55.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:55.271+0000 7fd1c54ea640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd1c010ab90 con 0x7fd1c0106560 2024-09-12T02:49:55.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:55.271+0000 7fd1beffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd1c0106560 0x7fd1c0197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:55.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:55.271+0000 7fd1beffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd1c0106560 0x7fd1c0197f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44590/0 (socket says 172.21.15.5:44590) 2024-09-12T02:49:55.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:55.271+0000 7fd1beffd640 1 -- 172.21.15.5:0/1306750870 learned_addr learned my addr 172.21.15.5:0/1306750870 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:49:55.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:55.272+0000 7fd1beffd640 1 -- 172.21.15.5:0/1306750870 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1c0198490 con 0x7fd1c0106560 2024-09-12T02:49:55.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:55.272+0000 7fd1beffd640 1 --2- 172.21.15.5:0/1306750870 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd1c0106560 0x7fd1c0197f50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd1b0009230 tx=0x7fd1b0003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:55.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:55.272+0000 7fd1a7fff640 1 -- 172.21.15.5:0/1306750870 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd1b003f070 con 0x7fd1c0106560 2024-09-12T02:49:55.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:55.272+0000 7fd1a7fff640 1 -- 172.21.15.5:0/1306750870 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1b003bc80 con 0x7fd1c0106560 2024-09-12T02:49:55.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:55.272+0000 7fd1a7fff640 1 -- 172.21.15.5:0/1306750870 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd1b0035030 con 0x7fd1c0106560 2024-09-12T02:49:55.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:55.272+0000 7fd1c54ea640 1 -- 172.21.15.5:0/1306750870 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1c0198720 con 0x7fd1c0106560 2024-09-12T02:49:55.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:55.272+0000 7fd1c54ea640 1 -- 172.21.15.5:0/1306750870 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1c0198b80 con 0x7fd1c0106560 2024-09-12T02:49:55.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:55.273+0000 7fd1a7fff640 1 -- 172.21.15.5:0/1306750870 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7fd1b0042070 con 0x7fd1c0106560 2024-09-12T02:49:55.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:55.273+0000 7fd1a7fff640 1 --2- 172.21.15.5:0/1306750870 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fd1a8039060 0x7fd1a803b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:55.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:55.273+0000 7fd1c54ea640 1 -- 172.21.15.5:0/1306750870 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1c0101bd0 con 0x7fd1c0106560 2024-09-12T02:49:55.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:55.273+0000 7fd1a7fff640 1 -- 172.21.15.5:0/1306750870 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fd1b006cef0 con 0x7fd1c0106560 2024-09-12T02:49:55.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:55.273+0000 7fd1be7fc640 1 --2- 172.21.15.5:0/1306750870 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fd1a8039060 0x7fd1a803b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:55.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:55.274+0000 7fd1be7fc640 1 --2- 172.21.15.5:0/1306750870 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fd1a8039060 0x7fd1a803b520 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fd1b40098c0 tx=0x7fd1b4006db0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:55.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:55.278+0000 7fd1a7fff640 1 -- 172.21.15.5:0/1306750870 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd1b004c350 con 0x7fd1c0106560 2024-09-12T02:49:55.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:55.461+0000 7fd1c54ea640 1 -- 172.21.15.5:0/1306750870 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fd1c01069c0 con 0x7fd1c0106560 2024-09-12T02:49:55.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:55.861+0000 7fd1a7fff640 1 -- 172.21.15.5:0/1306750870 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7fd1b0031e20 con 0x7fd1c0106560 2024-09-12T02:49:55.862 INFO:teuthology.orchestra.run.smithi005.stderr:adjusted tunables profile to default 2024-09-12T02:49:55.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:55.864+0000 7fd1c54ea640 1 -- 172.21.15.5:0/1306750870 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fd1a8039060 msgr2=0x7fd1a803b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:55.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:55.864+0000 7fd1c54ea640 1 --2- 172.21.15.5:0/1306750870 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fd1a8039060 0x7fd1a803b520 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fd1b40098c0 tx=0x7fd1b4006db0 comp rx=0 tx=0).stop 2024-09-12T02:49:55.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:55.864+0000 7fd1c54ea640 1 -- 172.21.15.5:0/1306750870 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd1c0106560 msgr2=0x7fd1c0197f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:55.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:55.864+0000 7fd1c54ea640 1 --2- 172.21.15.5:0/1306750870 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd1c0106560 0x7fd1c0197f50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd1b0009230 tx=0x7fd1b0003d40 comp rx=0 tx=0).stop 2024-09-12T02:49:55.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:55.864+0000 7fd1c54ea640 1 -- 172.21.15.5:0/1306750870 shutdown_connections 2024-09-12T02:49:55.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:55.864+0000 7fd1c54ea640 1 --2- 172.21.15.5:0/1306750870 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fd1a8039060 0x7fd1a803b520 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:55.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:55.864+0000 7fd1c54ea640 1 --2- 172.21.15.5:0/1306750870 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd1c0106560 0x7fd1c0197f50 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:55.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:55.864+0000 7fd1c54ea640 1 -- 172.21.15.5:0/1306750870 >> 172.21.15.5:0/1306750870 conn(0x7fd1c00fc1b0 msgr2=0x7fd1c00fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:55.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:55.864+0000 7fd1c54ea640 1 -- 172.21.15.5:0/1306750870 shutdown_connections 2024-09-12T02:49:55.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:49:55.864+0000 7fd1c54ea640 1 -- 172.21.15.5:0/1306750870 wait complete. 2024-09-12T02:49:56.115 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:55 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/1306750870' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-12T02:49:56.412 INFO:tasks.cephadm:Adding mon.a on smithi005 2024-09-12T02:49:56.412 INFO:tasks.cephadm:Adding mon.b on smithi146 2024-09-12T02:49:56.413 INFO:tasks.cephadm:Adding mon.c on smithi148 2024-09-12T02:49:56.413 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph orch apply mon '3;smithi005:172.21.15.5=a;smithi146:172.21.15.146=b;smithi148:172.21.15.148=c' 2024-09-12T02:49:56.672 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-12T02:49:56.719 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-12T02:49:57.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:49:57.176+0000 7f658b583640 1 -- 172.21.15.148:0/3333057561 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6584100540 msgr2=0x7f6584100940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:57.221 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:49:57.176+0000 7f658b583640 1 --2- 172.21.15.148:0/3333057561 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6584100540 0x7f6584100940 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f657c009880 tx=0x7f657c02a5d0 comp rx=0 tx=0).stop 2024-09-12T02:49:57.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:49:57.176+0000 7f658b583640 1 -- 172.21.15.148:0/3333057561 shutdown_connections 2024-09-12T02:49:57.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:49:57.176+0000 7f658b583640 1 --2- 172.21.15.148:0/3333057561 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6584100540 0x7f6584100940 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:57.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:49:57.176+0000 7f658b583640 1 -- 172.21.15.148:0/3333057561 >> 172.21.15.148:0/3333057561 conn(0x7f65840fbcd0 msgr2=0x7f65840fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:57.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:49:57.176+0000 7f658b583640 1 -- 172.21.15.148:0/3333057561 shutdown_connections 2024-09-12T02:49:57.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:49:57.176+0000 7f658b583640 1 -- 172.21.15.148:0/3333057561 wait complete. 2024-09-12T02:49:57.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:49:57.177+0000 7f658b583640 1 Processor -- start 2024-09-12T02:49:57.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:49:57.177+0000 7f658b583640 1 -- start start 2024-09-12T02:49:57.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:49:57.177+0000 7f658b583640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6584100540 0x7f6584195c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:57.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:49:57.177+0000 7f658b583640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f65841065f0 con 0x7f6584100540 2024-09-12T02:49:57.222 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:49:57.177+0000 7f658931f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6584100540 0x7f6584195c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:57.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:49:57.177+0000 7f658931f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6584100540 0x7f6584195c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.148:42424/0 (socket says 172.21.15.148:42424) 2024-09-12T02:49:57.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:49:57.177+0000 7f658931f640 1 -- 172.21.15.148:0/2591438857 learned_addr learned my addr 172.21.15.148:0/2591438857 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-09-12T02:49:57.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:49:57.177+0000 7f658931f640 1 -- 172.21.15.148:0/2591438857 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65841961a0 con 0x7f6584100540 2024-09-12T02:49:57.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:49:57.177+0000 7f658931f640 1 --2- 172.21.15.148:0/2591438857 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6584100540 0x7f6584195c60 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f657c004f60 tx=0x7f657c0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:57.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:49:57.178+0000 7f65767fc640 1 -- 172.21.15.148:0/2591438857 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f657c03f070 con 0x7f6584100540 2024-09-12T02:49:57.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:49:57.178+0000 7f65767fc640 1 -- 172.21.15.148:0/2591438857 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f657c004d40 con 0x7f6584100540 2024-09-12T02:49:57.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:49:57.178+0000 7f658b583640 1 -- 172.21.15.148:0/2591438857 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6584196430 con 0x7f6584100540 2024-09-12T02:49:57.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:49:57.178+0000 7f65767fc640 1 -- 172.21.15.148:0/2591438857 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f657c035030 con 0x7f6584100540 2024-09-12T02:49:57.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:49:57.178+0000 7f658b583640 1 -- 172.21.15.148:0/2591438857 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6584199120 con 0x7f6584100540 2024-09-12T02:49:57.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:49:57.178+0000 7f65767fc640 1 -- 172.21.15.148:0/2591438857 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f657c042070 con 0x7f6584100540 2024-09-12T02:49:57.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:49:57.179+0000 7f658b583640 1 -- 172.21.15.148:0/2591438857 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6584106d70 con 0x7f6584100540 2024-09-12T02:49:57.223 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:49:57.179+0000 7f65767fc640 1 --2- 172.21.15.148:0/2591438857 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f6570039060 0x7f657003b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:49:57.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:49:57.179+0000 7f65767fc640 1 -- 172.21.15.148:0/2591438857 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f657c06cd90 con 0x7f6584100540 2024-09-12T02:49:57.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:49:57.179+0000 7f6588b1e640 1 --2- 172.21.15.148:0/2591438857 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f6570039060 0x7f657003b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:49:57.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:49:57.179+0000 7f6588b1e640 1 --2- 172.21.15.148:0/2591438857 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f6570039060 0x7f657003b520 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f65780098c0 tx=0x7f6578006db0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:49:57.224 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:49:57.184+0000 7f65767fc640 1 -- 172.21.15.148:0/2591438857 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f657c040260 con 0x7f6584100540 2024-09-12T02:49:57.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:56 smithi005 ceph-mon[31197]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:49:57.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:56 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/1306750870' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-12T02:49:57.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:56 smithi005 ceph-mon[31197]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T02:49:57.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:56 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:49:57.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:56 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:49:57.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:56 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:49:57.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:56 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:49:57.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:56 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi148", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:49:57.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:56 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:49:57.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:56 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:49:57.371 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:49:57.371+0000 7f658b583640 1 -- 172.21.15.148:0/2591438857 --> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi005:172.21.15.5=a;smithi146:172.21.15.146=b;smithi148:172.21.15.148=c", "target": ["mon-mgr", ""]}) -- 0x7f658402d8f0 con 0x7f6570039060 2024-09-12T02:49:57.378 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:49:57.378+0000 7f65767fc640 1 -- 172.21.15.148:0/2591438857 <== mgr.14148 v2:172.21.15.5:6800/3780631642 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f658402d8f0 con 0x7f6570039060 2024-09-12T02:49:57.378 INFO:teuthology.orchestra.run.smithi148.stdout:Scheduled mon update... 2024-09-12T02:49:57.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:49:57.380+0000 7f658b583640 1 -- 172.21.15.148:0/2591438857 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f6570039060 msgr2=0x7f657003b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:57.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:49:57.380+0000 7f658b583640 1 --2- 172.21.15.148:0/2591438857 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f6570039060 0x7f657003b520 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f65780098c0 tx=0x7f6578006db0 comp rx=0 tx=0).stop 2024-09-12T02:49:57.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:49:57.380+0000 7f658b583640 1 -- 172.21.15.148:0/2591438857 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6584100540 msgr2=0x7f6584195c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:49:57.380 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:49:57.380+0000 7f658b583640 1 --2- 172.21.15.148:0/2591438857 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6584100540 0x7f6584195c60 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f657c004f60 tx=0x7f657c0048f0 comp rx=0 tx=0).stop 2024-09-12T02:49:57.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:49:57.380+0000 7f658b583640 1 -- 172.21.15.148:0/2591438857 shutdown_connections 2024-09-12T02:49:57.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:49:57.380+0000 7f658b583640 1 --2- 172.21.15.148:0/2591438857 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f6570039060 0x7f657003b520 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:57.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:49:57.380+0000 7f658b583640 1 --2- 172.21.15.148:0/2591438857 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6584100540 0x7f6584195c60 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:49:57.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:49:57.380+0000 7f658b583640 1 -- 172.21.15.148:0/2591438857 >> 172.21.15.148:0/2591438857 conn(0x7f65840fbcd0 msgr2=0x7f65840fc6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:49:57.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:49:57.380+0000 7f658b583640 1 -- 172.21.15.148:0/2591438857 shutdown_connections 2024-09-12T02:49:57.381 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:49:57.380+0000 7f658b583640 1 -- 172.21.15.148:0/2591438857 wait complete. 2024-09-12T02:49:58.069 DEBUG:teuthology.orchestra.run.smithi146:mon.b> sudo journalctl -f -n 0 -u ceph-618305f8-70b1-11ef-bcea-c7b262605968@mon.b.service 2024-09-12T02:49:58.073 DEBUG:teuthology.orchestra.run.smithi148:mon.c> sudo journalctl -f -n 0 -u ceph-618305f8-70b1-11ef-bcea-c7b262605968@mon.c.service 2024-09-12T02:49:58.075 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-12T02:49:58.075 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-12T02:49:58.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:58 smithi005 ceph-mon[31197]: Updating smithi148:/etc/ceph/ceph.conf 2024-09-12T02:49:58.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:58 smithi005 ceph-mon[31197]: Updating smithi148:/var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T02:49:58.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:58 smithi005 ceph-mon[31197]: Updating smithi148:/etc/ceph/ceph.client.admin.keyring 2024-09-12T02:49:58.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:58 smithi005 ceph-mon[31197]: Updating smithi148:/var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T02:49:58.811 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:58 smithi005 ceph-mon[31197]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi005:172.21.15.5=a;smithi146:172.21.15.146=b;smithi148:172.21.15.148=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:49:58.811 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:58 smithi005 ceph-mon[31197]: Saving service mon spec with placement smithi005:172.21.15.5=a;smithi146:172.21.15.146=b;smithi148:172.21.15.148=c;count:3 2024-09-12T02:49:58.811 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:58 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:49:58.811 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:58 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:49:58.811 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:58 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:49:58.811 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:58 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:49:58.811 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:58 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T02:49:58.811 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:58 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:49:59.095 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T02:49:59.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:59 smithi005 ceph-mon[31197]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:49:59.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:49:59 smithi005 ceph-mon[31197]: Deploying daemon mon.c on smithi148 2024-09-12T02:50:00.035 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:00.031+0000 7f97a58a5640 1 -- 172.21.15.148:0/3633576643 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f97a0106540 msgr2=0x7f97a0106920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:00.035 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:00.031+0000 7f97a58a5640 1 --2- 172.21.15.148:0/3633576643 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f97a0106540 0x7f97a0106920 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9790009880 tx=0x7f979002a5d0 comp rx=0 tx=0).stop 2024-09-12T02:50:00.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:00.031+0000 7f97a58a5640 1 -- 172.21.15.148:0/3633576643 shutdown_connections 2024-09-12T02:50:00.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:00.031+0000 7f97a58a5640 1 --2- 172.21.15.148:0/3633576643 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f97a0106540 0x7f97a0106920 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:00.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:00.031+0000 7f97a58a5640 1 -- 172.21.15.148:0/3633576643 >> 172.21.15.148:0/3633576643 conn(0x7f97a00fc130 msgr2=0x7f97a00fe550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:50:00.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:00.031+0000 7f97a58a5640 1 -- 172.21.15.148:0/3633576643 shutdown_connections 2024-09-12T02:50:00.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:00.031+0000 7f97a58a5640 1 -- 172.21.15.148:0/3633576643 wait complete. 2024-09-12T02:50:00.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:00.032+0000 7f97a58a5640 1 Processor -- start 2024-09-12T02:50:00.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:00.032+0000 7f97a58a5640 1 -- start start 2024-09-12T02:50:00.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:00.032+0000 7f97a58a5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f97a0106540 0x7f97a0195fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:00.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:00.032+0000 7f97a58a5640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f97a010ab70 con 0x7f97a0106540 2024-09-12T02:50:00.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:00.032+0000 7f979effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f97a0106540 0x7f97a0195fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:00.036 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:00.032+0000 7f979effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f97a0106540 0x7f97a0195fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.148:52318/0 (socket says 172.21.15.148:52318) 2024-09-12T02:50:00.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:00.033+0000 7f979effd640 1 -- 172.21.15.148:0/2119777598 learned_addr learned my addr 172.21.15.148:0/2119777598 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-09-12T02:50:00.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:00.033+0000 7f979effd640 1 -- 172.21.15.148:0/2119777598 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97a0196500 con 0x7f97a0106540 2024-09-12T02:50:00.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:00.033+0000 7f979effd640 1 --2- 172.21.15.148:0/2119777598 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f97a0106540 0x7f97a0195fc0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9790009230 tx=0x7f9790003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:50:00.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:00.033+0000 7f97a48a3640 1 -- 172.21.15.148:0/2119777598 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9790046070 con 0x7f97a0106540 2024-09-12T02:50:00.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:00.033+0000 7f97a48a3640 1 -- 172.21.15.148:0/2119777598 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9790038b60 con 0x7f97a0106540 2024-09-12T02:50:00.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:00.033+0000 7f97a58a5640 1 -- 172.21.15.148:0/2119777598 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97a0196790 con 0x7f97a0106540 2024-09-12T02:50:00.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:00.033+0000 7f97a48a3640 1 -- 172.21.15.148:0/2119777598 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9790035070 con 0x7f97a0106540 2024-09-12T02:50:00.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:00.033+0000 7f97a58a5640 1 -- 172.21.15.148:0/2119777598 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97a019a890 con 0x7f97a0106540 2024-09-12T02:50:00.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:00.034+0000 7f97a48a3640 1 -- 172.21.15.148:0/2119777598 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f9790049070 con 0x7f97a0106540 2024-09-12T02:50:00.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:00.034+0000 7f97a58a5640 1 -- 172.21.15.148:0/2119777598 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97a0101bb0 con 0x7f97a0106540 2024-09-12T02:50:00.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:00.035+0000 7f97a48a3640 1 --2- 172.21.15.148:0/2119777598 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f978c039060 0x7f978c03b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:00.037 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:00.035+0000 7f97a48a3640 1 -- 172.21.15.148:0/2119777598 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f979003e070 con 0x7f97a0106540 2024-09-12T02:50:00.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:00.035+0000 7f97967fc640 1 --2- 172.21.15.148:0/2119777598 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f978c039060 0x7f978c03b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:00.038 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:00.035+0000 7f97967fc640 1 --2- 172.21.15.148:0/2119777598 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f978c039060 0x7f978c03b520 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f97880098c0 tx=0x7f9788006db0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:50:00.039 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:00.039+0000 7f97a48a3640 1 -- 172.21.15.148:0/2119777598 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9790047d20 con 0x7f97a0106540 2024-09-12T02:50:00.288 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:00.287+0000 7f97a58a5640 1 -- 172.21.15.148:0/2119777598 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f97a0197320 con 0x7f97a0106540 2024-09-12T02:50:00.289 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:00.287+0000 7f97a48a3640 1 -- 172.21.15.148:0/2119777598 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+741 (secure 0 0 0) 0x7f9790031c90 con 0x7f97a0106540 2024-09-12T02:50:00.289 INFO:teuthology.orchestra.run.smithi148.stderr:dumped monmap epoch 1 2024-09-12T02:50:00.292 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-12T02:50:00.292 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":1,"fsid":"618305f8-70b1-11ef-bcea-c7b262605968","modified":"2024-09-12T02:48:39.314065Z","created":"2024-09-12T02:48:39.314065Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:3300","nonce":0},{"type":"v1","addr":"172.21.15.5:6789","nonce":0}]},"addr":"172.21.15.5:6789/0","public_addr":"172.21.15.5:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-12T02:50:00.293 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:00.290+0000 7f97a58a5640 1 -- 172.21.15.148:0/2119777598 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f978c039060 msgr2=0x7f978c03b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:00.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:00.290+0000 7f97a58a5640 1 --2- 172.21.15.148:0/2119777598 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f978c039060 0x7f978c03b520 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f97880098c0 tx=0x7f9788006db0 comp rx=0 tx=0).stop 2024-09-12T02:50:00.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:00.290+0000 7f97a58a5640 1 -- 172.21.15.148:0/2119777598 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f97a0106540 msgr2=0x7f97a0195fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:00.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:00.290+0000 7f97a58a5640 1 --2- 172.21.15.148:0/2119777598 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f97a0106540 0x7f97a0195fc0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9790009230 tx=0x7f9790003d40 comp rx=0 tx=0).stop 2024-09-12T02:50:00.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:00.290+0000 7f97a58a5640 1 -- 172.21.15.148:0/2119777598 shutdown_connections 2024-09-12T02:50:00.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:00.290+0000 7f97a58a5640 1 --2- 172.21.15.148:0/2119777598 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f978c039060 0x7f978c03b520 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:00.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:00.290+0000 7f97a58a5640 1 --2- 172.21.15.148:0/2119777598 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f97a0106540 0x7f97a0195fc0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:00.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:00.290+0000 7f97a58a5640 1 -- 172.21.15.148:0/2119777598 >> 172.21.15.148:0/2119777598 conn(0x7f97a00fc130 msgr2=0x7f97a00fc510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:50:00.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:00.290+0000 7f97a58a5640 1 -- 172.21.15.148:0/2119777598 shutdown_connections 2024-09-12T02:50:00.294 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:00.290+0000 7f97a58a5640 1 -- 172.21.15.148:0/2119777598 wait complete. 2024-09-12T02:50:00.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:00 smithi005 ceph-mon[31197]: from='client.? 172.21.15.148:0/2119777598' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T02:50:01.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:01 smithi005 ceph-mon[31197]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:50:02.088 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-12T02:50:02.089 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-12T02:50:02.095 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 podman[35070]: 2024-09-12 02:50:02.073525557 +0000 UTC m=+0.357396196 container start faab55de658f2675968742874a556c0ab1a65b38e664778596137e7746fd9011 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-mon-c, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T02:50:02.095 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: set uid:gid to 167:167 (ceph:ceph) 2024-09-12T02:50:02.095 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: ceph version 19.1.1-298-gb53b6d3e (b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc) squid (stable), process ceph-mon, pid 2 2024-09-12T02:50:02.095 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: pidfile_write: ignore empty --pid-file 2024-09-12T02:50:02.354 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T02:50:02.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: load: jerasure load: lrc 2024-09-12T02:50:02.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: RocksDB version: 7.9.2 2024-09-12T02:50:02.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Git sha 0 2024-09-12T02:50:02.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Compile date 2024-09-09 16:23:28 2024-09-12T02:50:02.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: DB SUMMARY 2024-09-12T02:50:02.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: DB Session ID: 9970FC1AQC920VATL4J7 2024-09-12T02:50:02.417 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: CURRENT file: CURRENT 2024-09-12T02:50:02.417 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: IDENTITY file: IDENTITY 2024-09-12T02:50:02.417 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-09-12T02:50:02.417 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-09-12T02:50:02.417 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-09-12T02:50:02.417 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.error_if_exists: 0 2024-09-12T02:50:02.417 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.create_if_missing: 0 2024-09-12T02:50:02.417 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.paranoid_checks: 1 2024-09-12T02:50:02.417 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-12T02:50:02.418 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-12T02:50:02.418 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-12T02:50:02.418 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.env: 0x55b115fbe940 2024-09-12T02:50:02.418 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.fs: PosixFileSystem 2024-09-12T02:50:02.418 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.info_log: 0x55b1177d96e0 2024-09-12T02:50:02.418 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.max_file_opening_threads: 16 2024-09-12T02:50:02.419 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.statistics: (nil) 2024-09-12T02:50:02.419 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.use_fsync: 0 2024-09-12T02:50:02.419 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.max_log_file_size: 0 2024-09-12T02:50:02.419 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-12T02:50:02.419 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-12T02:50:02.420 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.keep_log_file_num: 1000 2024-09-12T02:50:02.420 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.recycle_log_file_num: 0 2024-09-12T02:50:02.420 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.allow_fallocate: 1 2024-09-12T02:50:02.420 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.allow_mmap_reads: 0 2024-09-12T02:50:02.420 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.allow_mmap_writes: 0 2024-09-12T02:50:02.420 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.use_direct_reads: 0 2024-09-12T02:50:02.420 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-12T02:50:02.421 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.create_missing_column_families: 0 2024-09-12T02:50:02.421 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.db_log_dir: 2024-09-12T02:50:02.421 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.wal_dir: 2024-09-12T02:50:02.421 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-12T02:50:02.421 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-12T02:50:02.421 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-12T02:50:02.422 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-12T02:50:02.424 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-12T02:50:02.424 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-12T02:50:02.424 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.advise_random_on_open: 1 2024-09-12T02:50:02.424 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.db_write_buffer_size: 0 2024-09-12T02:50:02.425 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.write_buffer_manager: 0x55b1177e8dc0 2024-09-12T02:50:02.425 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-12T02:50:02.425 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-12T02:50:02.425 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-12T02:50:02.425 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.rate_limiter: (nil) 2024-09-12T02:50:02.425 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-12T02:50:02.425 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.wal_recovery_mode: 2 2024-09-12T02:50:02.425 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.enable_thread_tracking: 0 2024-09-12T02:50:02.426 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.enable_pipelined_write: 0 2024-09-12T02:50:02.426 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.unordered_write: 0 2024-09-12T02:50:02.426 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-12T02:50:02.426 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-12T02:50:02.426 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-12T02:50:02.426 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-12T02:50:02.426 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.row_cache: None 2024-09-12T02:50:02.427 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.wal_filter: None 2024-09-12T02:50:02.427 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-12T02:50:02.427 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.allow_ingest_behind: 0 2024-09-12T02:50:02.427 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.two_write_queues: 0 2024-09-12T02:50:02.427 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.manual_wal_flush: 0 2024-09-12T02:50:02.428 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.wal_compression: 0 2024-09-12T02:50:02.428 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.atomic_flush: 0 2024-09-12T02:50:02.428 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-12T02:50:02.428 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-12T02:50:02.428 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-12T02:50:02.428 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.log_readahead_size: 0 2024-09-12T02:50:02.429 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-12T02:50:02.429 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.best_efforts_recovery: 0 2024-09-12T02:50:02.429 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-12T02:50:02.429 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-12T02:50:02.429 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.allow_data_in_errors: 0 2024-09-12T02:50:02.430 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.db_host_id: __hostname__ 2024-09-12T02:50:02.430 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-12T02:50:02.430 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.max_background_jobs: 2 2024-09-12T02:50:02.430 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.max_background_compactions: -1 2024-09-12T02:50:02.430 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.max_subcompactions: 1 2024-09-12T02:50:02.430 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-12T02:50:02.430 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-12T02:50:02.430 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-12T02:50:02.431 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.max_total_wal_size: 0 2024-09-12T02:50:02.431 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-12T02:50:02.431 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-12T02:50:02.431 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-12T02:50:02.431 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-12T02:50:02.431 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.max_open_files: -1 2024-09-12T02:50:02.431 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.bytes_per_sync: 0 2024-09-12T02:50:02.431 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-12T02:50:02.431 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-12T02:50:02.432 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.compaction_readahead_size: 0 2024-09-12T02:50:02.432 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.max_background_flushes: -1 2024-09-12T02:50:02.432 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Compression algorithms supported: 2024-09-12T02:50:02.432 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: kZSTD supported: 0 2024-09-12T02:50:02.432 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: kXpressCompression supported: 0 2024-09-12T02:50:02.432 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: kBZip2Compression supported: 0 2024-09-12T02:50:02.432 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-12T02:50:02.432 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: kLZ4Compression supported: 1 2024-09-12T02:50:02.433 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: kZlibCompression supported: 1 2024-09-12T02:50:02.433 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: kLZ4HCCompression supported: 1 2024-09-12T02:50:02.433 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: kSnappyCompression supported: 1 2024-09-12T02:50:02.433 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-12T02:50:02.433 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-12T02:50:02.433 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-09-12T02:50:02.433 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-12T02:50:02.434 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-12T02:50:02.434 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.merge_operator: 2024-09-12T02:50:02.434 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.compaction_filter: None 2024-09-12T02:50:02.434 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.compaction_filter_factory: None 2024-09-12T02:50:02.434 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.sst_partitioner_factory: None 2024-09-12T02:50:02.434 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-12T02:50:02.435 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-12T02:50:02.435 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55b1177d9320) 2024-09-12T02:50:02.435 INFO:journalctl@ceph.mon.c.smithi148.stdout: cache_index_and_filter_blocks: 1 2024-09-12T02:50:02.435 INFO:journalctl@ceph.mon.c.smithi148.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-12T02:50:02.435 INFO:journalctl@ceph.mon.c.smithi148.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-12T02:50:02.435 INFO:journalctl@ceph.mon.c.smithi148.stdout: pin_top_level_index_and_filter: 1 2024-09-12T02:50:02.435 INFO:journalctl@ceph.mon.c.smithi148.stdout: index_type: 0 2024-09-12T02:50:02.436 INFO:journalctl@ceph.mon.c.smithi148.stdout: data_block_index_type: 0 2024-09-12T02:50:02.436 INFO:journalctl@ceph.mon.c.smithi148.stdout: index_shortening: 1 2024-09-12T02:50:02.436 INFO:journalctl@ceph.mon.c.smithi148.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-12T02:50:02.436 INFO:journalctl@ceph.mon.c.smithi148.stdout: checksum: 4 2024-09-12T02:50:02.436 INFO:journalctl@ceph.mon.c.smithi148.stdout: no_block_cache: 0 2024-09-12T02:50:02.436 INFO:journalctl@ceph.mon.c.smithi148.stdout: block_cache: 0x55b1177cd350 2024-09-12T02:50:02.436 INFO:journalctl@ceph.mon.c.smithi148.stdout: block_cache_name: BinnedLRUCache 2024-09-12T02:50:02.436 INFO:journalctl@ceph.mon.c.smithi148.stdout: block_cache_options: 2024-09-12T02:50:02.436 INFO:journalctl@ceph.mon.c.smithi148.stdout: capacity : 536870912 2024-09-12T02:50:02.437 INFO:journalctl@ceph.mon.c.smithi148.stdout: num_shard_bits : 4 2024-09-12T02:50:02.437 INFO:journalctl@ceph.mon.c.smithi148.stdout: strict_capacity_limit : 0 2024-09-12T02:50:02.437 INFO:journalctl@ceph.mon.c.smithi148.stdout: high_pri_pool_ratio: 0.000 2024-09-12T02:50:02.437 INFO:journalctl@ceph.mon.c.smithi148.stdout: block_cache_compressed: (nil) 2024-09-12T02:50:02.437 INFO:journalctl@ceph.mon.c.smithi148.stdout: persistent_cache: (nil) 2024-09-12T02:50:02.437 INFO:journalctl@ceph.mon.c.smithi148.stdout: block_size: 4096 2024-09-12T02:50:02.437 INFO:journalctl@ceph.mon.c.smithi148.stdout: block_size_deviation: 10 2024-09-12T02:50:02.437 INFO:journalctl@ceph.mon.c.smithi148.stdout: block_restart_interval: 16 2024-09-12T02:50:02.437 INFO:journalctl@ceph.mon.c.smithi148.stdout: index_block_restart_interval: 1 2024-09-12T02:50:02.437 INFO:journalctl@ceph.mon.c.smithi148.stdout: metadata_block_size: 4096 2024-09-12T02:50:02.437 INFO:journalctl@ceph.mon.c.smithi148.stdout: partition_filters: 0 2024-09-12T02:50:02.437 INFO:journalctl@ceph.mon.c.smithi148.stdout: use_delta_encoding: 1 2024-09-12T02:50:02.438 INFO:journalctl@ceph.mon.c.smithi148.stdout: filter_policy: bloomfilter 2024-09-12T02:50:02.438 INFO:journalctl@ceph.mon.c.smithi148.stdout: whole_key_filtering: 1 2024-09-12T02:50:02.438 INFO:journalctl@ceph.mon.c.smithi148.stdout: verify_compression: 0 2024-09-12T02:50:02.438 INFO:journalctl@ceph.mon.c.smithi148.stdout: read_amp_bytes_per_bit: 0 2024-09-12T02:50:02.438 INFO:journalctl@ceph.mon.c.smithi148.stdout: format_version: 5 2024-09-12T02:50:02.438 INFO:journalctl@ceph.mon.c.smithi148.stdout: enable_index_compression: 1 2024-09-12T02:50:02.438 INFO:journalctl@ceph.mon.c.smithi148.stdout: block_align: 0 2024-09-12T02:50:02.438 INFO:journalctl@ceph.mon.c.smithi148.stdout: max_auto_readahead_size: 262144 2024-09-12T02:50:02.438 INFO:journalctl@ceph.mon.c.smithi148.stdout: prepopulate_block_cache: 0 2024-09-12T02:50:02.438 INFO:journalctl@ceph.mon.c.smithi148.stdout: initial_auto_readahead_size: 8192 2024-09-12T02:50:02.438 INFO:journalctl@ceph.mon.c.smithi148.stdout: num_file_reads_for_auto_readahead: 2 2024-09-12T02:50:02.438 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.write_buffer_size: 33554432 2024-09-12T02:50:02.439 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.max_write_buffer_number: 2 2024-09-12T02:50:02.439 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.compression: NoCompression 2024-09-12T02:50:02.439 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.bottommost_compression: Disabled 2024-09-12T02:50:02.439 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.prefix_extractor: nullptr 2024-09-12T02:50:02.439 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-12T02:50:02.439 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.num_levels: 7 2024-09-12T02:50:02.439 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-12T02:50:02.439 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-12T02:50:02.439 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-12T02:50:02.439 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-12T02:50:02.439 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-12T02:50:02.440 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-12T02:50:02.440 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-12T02:50:02.440 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-12T02:50:02.440 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-12T02:50:02.440 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-12T02:50:02.440 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-12T02:50:02.440 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-12T02:50:02.440 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-12T02:50:02.440 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.compression_opts.level: 32767 2024-09-12T02:50:02.440 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.compression_opts.strategy: 0 2024-09-12T02:50:02.440 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-12T02:50:02.440 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-12T02:50:02.441 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-12T02:50:02.441 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-12T02:50:02.441 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.compression_opts.enabled: false 2024-09-12T02:50:02.441 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-12T02:50:02.441 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-12T02:50:02.441 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-12T02:50:02.441 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-12T02:50:02.441 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.target_file_size_base: 67108864 2024-09-12T02:50:02.441 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-12T02:50:02.441 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-12T02:50:02.441 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-12T02:50:02.442 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-12T02:50:02.442 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-12T02:50:02.442 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-12T02:50:02.442 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-12T02:50:02.442 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-12T02:50:02.442 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-12T02:50:02.442 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-12T02:50:02.442 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-12T02:50:02.442 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-12T02:50:02.443 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-12T02:50:02.443 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-12T02:50:02.443 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.arena_block_size: 1048576 2024-09-12T02:50:02.443 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-12T02:50:02.443 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-12T02:50:02.443 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.disable_auto_compactions: 0 2024-09-12T02:50:02.443 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-12T02:50:02.443 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-12T02:50:02.443 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-12T02:50:02.443 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-12T02:50:02.443 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-12T02:50:02.443 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-12T02:50:02.443 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-12T02:50:02.444 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-12T02:50:02.444 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-12T02:50:02.444 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-12T02:50:02.444 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-12T02:50:02.444 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.inplace_update_support: 0 2024-09-12T02:50:02.444 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-12T02:50:02.444 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-12T02:50:02.444 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-12T02:50:02.444 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-12T02:50:02.444 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.bloom_locality: 0 2024-09-12T02:50:02.444 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.max_successive_merges: 0 2024-09-12T02:50:02.444 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-12T02:50:02.445 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.paranoid_file_checks: 0 2024-09-12T02:50:02.445 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.force_consistency_checks: 1 2024-09-12T02:50:02.445 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.report_bg_io_stats: 0 2024-09-12T02:50:02.445 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.ttl: 2592000 2024-09-12T02:50:02.445 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-12T02:50:02.445 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-12T02:50:02.445 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-12T02:50:02.445 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.enable_blob_files: false 2024-09-12T02:50:02.445 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.min_blob_size: 0 2024-09-12T02:50:02.445 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.blob_file_size: 268435456 2024-09-12T02:50:02.445 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-12T02:50:02.445 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-12T02:50:02.445 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-12T02:50:02.446 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-12T02:50:02.446 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-12T02:50:02.446 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.blob_file_starting_level: 0 2024-09-12T02:50:02.446 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-12T02:50:02.446 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-12T02:50:02.446 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-09-12T02:50:02.447 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: fbe1b4a5-cdad-44c6-9519-e89512b0f104 2024-09-12T02:50:02.447 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726109402101030, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-12T02:50:02.447 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-09-12T02:50:02.447 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726109402101446, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726109402, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "fbe1b4a5-cdad-44c6-9519-e89512b0f104", "db_session_id": "9970FC1AQC920VATL4J7", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-09-12T02:50:02.447 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726109402101545, "job": 1, "event": "recovery_finished"} 2024-09-12T02:50:02.447 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-09-12T02:50:02.448 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-12T02:50:02.448 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55b1177fae00 2024-09-12T02:50:02.448 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: DB pointer 0x55b1178f6000 2024-09-12T02:50:02.448 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-12T02:50:02.448 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-12T02:50:02.448 INFO:journalctl@ceph.mon.c.smithi148.stdout: ** DB Stats ** 2024-09-12T02:50:02.448 INFO:journalctl@ceph.mon.c.smithi148.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-12T02:50:02.448 INFO:journalctl@ceph.mon.c.smithi148.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-12T02:50:02.448 INFO:journalctl@ceph.mon.c.smithi148.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-12T02:50:02.448 INFO:journalctl@ceph.mon.c.smithi148.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-12T02:50:02.448 INFO:journalctl@ceph.mon.c.smithi148.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-12T02:50:02.448 INFO:journalctl@ceph.mon.c.smithi148.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-12T02:50:02.449 INFO:journalctl@ceph.mon.c.smithi148.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-12T02:50:02.449 INFO:journalctl@ceph.mon.c.smithi148.stdout: 2024-09-12T02:50:02.449 INFO:journalctl@ceph.mon.c.smithi148.stdout: ** Compaction Stats [default] ** 2024-09-12T02:50:02.449 INFO:journalctl@ceph.mon.c.smithi148.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-12T02:50:02.449 INFO:journalctl@ceph.mon.c.smithi148.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-12T02:50:02.449 INFO:journalctl@ceph.mon.c.smithi148.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-12T02:50:02.449 INFO:journalctl@ceph.mon.c.smithi148.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-12T02:50:02.449 INFO:journalctl@ceph.mon.c.smithi148.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-12T02:50:02.449 INFO:journalctl@ceph.mon.c.smithi148.stdout: 2024-09-12T02:50:02.449 INFO:journalctl@ceph.mon.c.smithi148.stdout: ** Compaction Stats [default] ** 2024-09-12T02:50:02.449 INFO:journalctl@ceph.mon.c.smithi148.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-12T02:50:02.449 INFO:journalctl@ceph.mon.c.smithi148.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-12T02:50:02.449 INFO:journalctl@ceph.mon.c.smithi148.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-12T02:50:02.450 INFO:journalctl@ceph.mon.c.smithi148.stdout: 2024-09-12T02:50:02.450 INFO:journalctl@ceph.mon.c.smithi148.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-12T02:50:02.450 INFO:journalctl@ceph.mon.c.smithi148.stdout: 2024-09-12T02:50:02.450 INFO:journalctl@ceph.mon.c.smithi148.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-12T02:50:02.450 INFO:journalctl@ceph.mon.c.smithi148.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-12T02:50:02.450 INFO:journalctl@ceph.mon.c.smithi148.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-12T02:50:02.450 INFO:journalctl@ceph.mon.c.smithi148.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-12T02:50:02.450 INFO:journalctl@ceph.mon.c.smithi148.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-12T02:50:02.450 INFO:journalctl@ceph.mon.c.smithi148.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-12T02:50:02.450 INFO:journalctl@ceph.mon.c.smithi148.stdout: Cumulative compaction: 0.00 GB write, 0.53 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-12T02:50:02.450 INFO:journalctl@ceph.mon.c.smithi148.stdout: Interval compaction: 0.00 GB write, 0.53 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-12T02:50:02.450 INFO:journalctl@ceph.mon.c.smithi148.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-12T02:50:02.450 INFO:journalctl@ceph.mon.c.smithi148.stdout: Block cache BinnedLRUCache@0x55b1177cd350#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-09-12T02:50:02.450 INFO:journalctl@ceph.mon.c.smithi148.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-09-12T02:50:02.451 INFO:journalctl@ceph.mon.c.smithi148.stdout: 2024-09-12T02:50:02.451 INFO:journalctl@ceph.mon.c.smithi148.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-12T02:50:02.451 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-09-12T02:50:02.451 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: using public_addr v2:172.21.15.148:0/0 -> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] 2024-09-12T02:50:02.451 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: starting mon.c rank -1 at public addrs [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] at bind addrs [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 618305f8-70b1-11ef-bcea-c7b262605968 2024-09-12T02:50:02.451 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: mon.c@-1(???) e0 preinit fsid 618305f8-70b1-11ef-bcea-c7b262605968 2024-09-12T02:50:02.451 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: mon.c@-1(synchronizing).mds e1 new map 2024-09-12T02:50:02.451 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: mon.c@-1(synchronizing).mds e1 print_map 2024-09-12T02:50:02.451 INFO:journalctl@ceph.mon.c.smithi148.stdout: e1 2024-09-12T02:50:02.451 INFO:journalctl@ceph.mon.c.smithi148.stdout: btime 2024-09-12T02:48:42:351994+0000 2024-09-12T02:50:02.451 INFO:journalctl@ceph.mon.c.smithi148.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-12T02:50:02.451 INFO:journalctl@ceph.mon.c.smithi148.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-12T02:50:02.452 INFO:journalctl@ceph.mon.c.smithi148.stdout: legacy client fscid: -1 2024-09-12T02:50:02.452 INFO:journalctl@ceph.mon.c.smithi148.stdout: 2024-09-12T02:50:02.452 INFO:journalctl@ceph.mon.c.smithi148.stdout: No filesystems configured 2024-09-12T02:50:02.452 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-12T02:50:02.452 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-12T02:50:02.452 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-12T02:50:02.452 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-12T02:50:02.452 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-12T02:50:02.452 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-12T02:50:02.452 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-12T02:50:02.452 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-12T02:50:02.452 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-12T02:50:02.452 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-12T02:50:02.453 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/1306750870' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-12T02:50:02.453 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:50:02.453 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/1306750870' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-12T02:50:02.453 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T02:50:02.453 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:02.453 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:02.453 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:02.453 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:02.453 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi148", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:50:02.453 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:50:02.454 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:50:02.454 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: Updating smithi148:/etc/ceph/ceph.conf 2024-09-12T02:50:02.454 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: Updating smithi148:/var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T02:50:02.454 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: Updating smithi148:/etc/ceph/ceph.client.admin.keyring 2024-09-12T02:50:02.454 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: Updating smithi148:/var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T02:50:02.454 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi005:172.21.15.5=a;smithi146:172.21.15.146=b;smithi148:172.21.15.148=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:50:02.455 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: Saving service mon spec with placement smithi005:172.21.15.5=a;smithi146:172.21.15.146=b;smithi148:172.21.15.148=c;count:3 2024-09-12T02:50:02.455 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:02.455 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:02.455 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:02.455 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:02.455 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T02:50:02.455 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:50:02.456 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:50:02.456 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: Deploying daemon mon.c on smithi148 2024-09-12T02:50:02.456 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: from='client.? 172.21.15.148:0/2119777598' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T02:50:02.456 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:50:02.456 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 ceph-mon[35090]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-12T02:50:02.456 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 bash[35070]: faab55de658f2675968742874a556c0ab1a65b38e664778596137e7746fd9011 2024-09-12T02:50:02.457 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:02 smithi148 systemd[1]: Started Ceph mon.c for 618305f8-70b1-11ef-bcea-c7b262605968. 2024-09-12T02:50:07.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:07.268+0000 7f002a342640 1 -- 172.21.15.148:0/1548745623 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0010003530 msgr2=0x7f00100059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:07.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:07.268+0000 7f002a342640 1 --2- 172.21.15.148:0/1548745623 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0010003530 0x7f00100059c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f00241016c0 tx=0x7f001402c490 comp rx=0 tx=0).stop 2024-09-12T02:50:07.269 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:07.268+0000 7f002a342640 1 -- 172.21.15.148:0/1548745623 shutdown_connections 2024-09-12T02:50:07.270 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:07.268+0000 7f002a342640 1 --2- 172.21.15.148:0/1548745623 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0010003530 0x7f00100059c0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:07.270 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:07.268+0000 7f002a342640 1 -- 172.21.15.148:0/1548745623 >> 172.21.15.148:0/1548745623 conn(0x7f00240fc280 msgr2=0x7f00240fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:50:07.270 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:07.268+0000 7f002a342640 1 -- 172.21.15.148:0/1548745623 shutdown_connections 2024-09-12T02:50:07.270 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:07.268+0000 7f002a342640 1 -- 172.21.15.148:0/1548745623 wait complete. 2024-09-12T02:50:07.270 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:07.269+0000 7f002a342640 1 Processor -- start 2024-09-12T02:50:07.270 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:07.269+0000 7f002a342640 1 -- start start 2024-09-12T02:50:07.270 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:07.269+0000 7f002a342640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0010003530 0x7f0024102210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:07.270 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:07.269+0000 7f002a342640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0024102750 0x7f0024102bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:07.270 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:07.269+0000 7f002a342640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0024108690 con 0x7f0010003530 2024-09-12T02:50:07.270 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:07.269+0000 7f002a342640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f0024108510 con 0x7f0024102750 2024-09-12T02:50:07.272 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:07.269+0000 7f001bfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0024102750 0x7f0024102bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:07.272 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:07.270+0000 7f0023fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0010003530 0x7f0024102210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:07.272 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:07.270+0000 7f001bfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0024102750 0x7f0024102bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:44740/0 (socket says 172.21.15.148:44740) 2024-09-12T02:50:07.272 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:07.270+0000 7f001bfff640 1 -- 172.21.15.148:0/1952082487 learned_addr learned my addr 172.21.15.148:0/1952082487 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-09-12T02:50:07.272 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:07.270+0000 7f001bfff640 1 -- 172.21.15.148:0/1952082487 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0024102750 msgr2=0x7f0024102bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-12T02:50:07.272 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:07.270+0000 7f001bfff640 1 -- 172.21.15.148:0/1952082487 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0024102750 msgr2=0x7f0024102bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-12T02:50:07.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:07.270+0000 7f001bfff640 1 --2- 172.21.15.148:0/1952082487 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0024102750 0x7f0024102bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-12T02:50:07.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:07.270+0000 7f001bfff640 1 --2- 172.21.15.148:0/1952082487 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0024102750 0x7f0024102bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T02:50:07.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:07.270+0000 7f0023fff640 1 -- 172.21.15.148:0/1952082487 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0024102750 msgr2=0x7f0024102bb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T02:50:07.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:07.270+0000 7f0023fff640 1 --2- 172.21.15.148:0/1952082487 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0024102750 0x7f0024102bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:07.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:07.270+0000 7f0023fff640 1 -- 172.21.15.148:0/1952082487 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00241040b0 con 0x7f0010003530 2024-09-12T02:50:07.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:07.270+0000 7f0023fff640 1 --2- 172.21.15.148:0/1952082487 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0010003530 0x7f0024102210 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0014036040 tx=0x7f0014033680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:50:07.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:07.271+0000 7f0021ffb640 1 -- 172.21.15.148:0/1952082487 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f0014032920 con 0x7f0010003530 2024-09-12T02:50:07.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:07.271+0000 7f0021ffb640 1 -- 172.21.15.148:0/1952082487 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0014041070 con 0x7f0010003530 2024-09-12T02:50:07.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:07.271+0000 7f002a342640 1 -- 172.21.15.148:0/1952082487 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00241a3560 con 0x7f0010003530 2024-09-12T02:50:07.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:07.271+0000 7f0021ffb640 1 -- 172.21.15.148:0/1952082487 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f0014004970 con 0x7f0010003530 2024-09-12T02:50:07.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:07.271+0000 7f002a342640 1 -- 172.21.15.148:0/1952082487 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00241a3af0 con 0x7f0010003530 2024-09-12T02:50:07.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:07.272+0000 7f0021ffb640 1 -- 172.21.15.148:0/1952082487 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f0014004c20 con 0x7f0010003530 2024-09-12T02:50:07.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:07.272+0000 7f002a342640 1 -- 172.21.15.148:0/1952082487 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0024101cc0 con 0x7f0010003530 2024-09-12T02:50:07.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:07.272+0000 7f0021ffb640 1 --2- 172.21.15.148:0/1952082487 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f0004038e50 0x7f000403b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:07.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:07.272+0000 7f0021ffb640 1 -- 172.21.15.148:0/1952082487 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f001406e410 con 0x7f0010003530 2024-09-12T02:50:07.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:07.272+0000 7f001bfff640 1 --2- 172.21.15.148:0/1952082487 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f0004038e50 0x7f000403b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:07.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:07.273+0000 7f001bfff640 1 --2- 172.21.15.148:0/1952082487 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f0004038e50 0x7f000403b310 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f000c009100 tx=0x7f000c008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:50:07.277 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:07.277+0000 7f0021ffb640 1 -- 172.21.15.148:0/1952082487 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f001403a2f0 con 0x7f0010003530 2024-09-12T02:50:07.509 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:07.509+0000 7f002a342640 1 -- 172.21.15.148:0/1952082487 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f00241a3da0 con 0x7f0010003530 2024-09-12T02:50:07.510 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:07 smithi148 ceph-mon[35090]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:50:07.510 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:07 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T02:50:07.510 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:07 smithi148 ceph-mon[35090]: mon.a calling monitor election 2024-09-12T02:50:07.510 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:07 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:50:07.510 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:07 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:50:07.511 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:07 smithi148 ceph-mon[35090]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:50:07.511 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:07 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:50:07.511 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:07 smithi148 ceph-mon[35090]: mon.c calling monitor election 2024-09-12T02:50:07.511 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:07 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:50:07.511 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:07 smithi148 ceph-mon[35090]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:50:07.511 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:07 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:50:07.511 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:07 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:50:07.511 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:07 smithi148 ceph-mon[35090]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-12T02:50:07.511 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:07 smithi148 ceph-mon[35090]: monmap epoch 2 2024-09-12T02:50:07.511 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:07 smithi148 ceph-mon[35090]: fsid 618305f8-70b1-11ef-bcea-c7b262605968 2024-09-12T02:50:07.511 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:07 smithi148 ceph-mon[35090]: last_changed 2024-09-12T02:50:02.127801+0000 2024-09-12T02:50:07.511 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:07 smithi148 ceph-mon[35090]: created 2024-09-12T02:48:39.314065+0000 2024-09-12T02:50:07.511 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:07 smithi148 ceph-mon[35090]: min_mon_release 19 (squid) 2024-09-12T02:50:07.511 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:07 smithi148 ceph-mon[35090]: election_strategy: 1 2024-09-12T02:50:07.512 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:07 smithi148 ceph-mon[35090]: 0: [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] mon.a 2024-09-12T02:50:07.512 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:07 smithi148 ceph-mon[35090]: 1: [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] mon.c 2024-09-12T02:50:07.512 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:07 smithi148 ceph-mon[35090]: fsmap 2024-09-12T02:50:07.512 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:07 smithi148 ceph-mon[35090]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T02:50:07.512 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:07 smithi148 ceph-mon[35090]: mgrmap e13: x(active, since 37s) 2024-09-12T02:50:07.512 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:07 smithi148 ceph-mon[35090]: overall HEALTH_OK 2024-09-12T02:50:07.512 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:07 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:07.512 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:07 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:07.512 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:07 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:07.512 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:07 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T02:50:07.512 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:07 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:50:07.513 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:07.510+0000 7f0021ffb640 1 -- 172.21.15.148:0/1952082487 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1011 (secure 0 0 0) 0x7f00140381e0 con 0x7f0010003530 2024-09-12T02:50:07.513 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-12T02:50:07.513 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":2,"fsid":"618305f8-70b1-11ef-bcea-c7b262605968","modified":"2024-09-12T02:50:02.127801Z","created":"2024-09-12T02:48:39.314065Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:3300","nonce":0},{"type":"v1","addr":"172.21.15.5:6789","nonce":0}]},"addr":"172.21.15.5:6789/0","public_addr":"172.21.15.5:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:3300","nonce":0},{"type":"v1","addr":"172.21.15.148:6789","nonce":0}]},"addr":"172.21.15.148:6789/0","public_addr":"172.21.15.148:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-12T02:50:07.514 INFO:teuthology.orchestra.run.smithi148.stderr:dumped monmap epoch 2 2024-09-12T02:50:07.514 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:07.512+0000 7f002a342640 1 -- 172.21.15.148:0/1952082487 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f0004038e50 msgr2=0x7f000403b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:07.514 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:07.512+0000 7f002a342640 1 --2- 172.21.15.148:0/1952082487 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f0004038e50 0x7f000403b310 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f000c009100 tx=0x7f000c008040 comp rx=0 tx=0).stop 2024-09-12T02:50:07.514 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:07.512+0000 7f002a342640 1 -- 172.21.15.148:0/1952082487 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0010003530 msgr2=0x7f0024102210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:07.514 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:07.512+0000 7f002a342640 1 --2- 172.21.15.148:0/1952082487 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0010003530 0x7f0024102210 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0014036040 tx=0x7f0014033680 comp rx=0 tx=0).stop 2024-09-12T02:50:07.514 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:07.512+0000 7f002a342640 1 -- 172.21.15.148:0/1952082487 shutdown_connections 2024-09-12T02:50:07.514 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:07.512+0000 7f002a342640 1 --2- 172.21.15.148:0/1952082487 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0024102750 0x7f0024102bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:07.514 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:07.512+0000 7f002a342640 1 --2- 172.21.15.148:0/1952082487 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f0004038e50 0x7f000403b310 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:07.514 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:07.512+0000 7f002a342640 1 --2- 172.21.15.148:0/1952082487 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0010003530 0x7f0024102210 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:07.515 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:07.512+0000 7f002a342640 1 -- 172.21.15.148:0/1952082487 >> 172.21.15.148:0/1952082487 conn(0x7f00240fc280 msgr2=0x7f002406a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:50:07.515 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:07.512+0000 7f002a342640 1 -- 172.21.15.148:0/1952082487 shutdown_connections 2024-09-12T02:50:07.515 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:07.512+0000 7f002a342640 1 -- 172.21.15.148:0/1952082487 wait complete. 2024-09-12T02:50:07.515 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:07 smithi005 ceph-mon[31197]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:50:07.515 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:07 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T02:50:07.515 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:07 smithi005 ceph-mon[31197]: mon.a calling monitor election 2024-09-12T02:50:07.515 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:07 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:50:07.515 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:07 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:50:07.515 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:07 smithi005 ceph-mon[31197]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:50:07.516 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:07 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:50:07.516 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:07 smithi005 ceph-mon[31197]: mon.c calling monitor election 2024-09-12T02:50:07.516 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:07 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:50:07.516 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:07 smithi005 ceph-mon[31197]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:50:07.516 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:07 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:50:07.516 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:07 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:50:07.516 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:07 smithi005 ceph-mon[31197]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-12T02:50:07.516 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:07 smithi005 ceph-mon[31197]: monmap epoch 2 2024-09-12T02:50:07.516 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:07 smithi005 ceph-mon[31197]: fsid 618305f8-70b1-11ef-bcea-c7b262605968 2024-09-12T02:50:07.516 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:07 smithi005 ceph-mon[31197]: last_changed 2024-09-12T02:50:02.127801+0000 2024-09-12T02:50:07.516 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:07 smithi005 ceph-mon[31197]: created 2024-09-12T02:48:39.314065+0000 2024-09-12T02:50:07.517 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:07 smithi005 ceph-mon[31197]: min_mon_release 19 (squid) 2024-09-12T02:50:07.517 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:07 smithi005 ceph-mon[31197]: election_strategy: 1 2024-09-12T02:50:07.517 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:07 smithi005 ceph-mon[31197]: 0: [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] mon.a 2024-09-12T02:50:07.517 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:07 smithi005 ceph-mon[31197]: 1: [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] mon.c 2024-09-12T02:50:07.517 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:07 smithi005 ceph-mon[31197]: fsmap 2024-09-12T02:50:07.517 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:07 smithi005 ceph-mon[31197]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T02:50:07.517 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:07 smithi005 ceph-mon[31197]: mgrmap e13: x(active, since 37s) 2024-09-12T02:50:07.517 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:07 smithi005 ceph-mon[31197]: overall HEALTH_OK 2024-09-12T02:50:07.517 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:07 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:07.517 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:07 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:07.517 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:07 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:07.518 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:07 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T02:50:07.518 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:07 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:50:08.524 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:08 smithi005 ceph-mon[31197]: Deploying daemon mon.b on smithi146 2024-09-12T02:50:08.524 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:08 smithi005 ceph-mon[31197]: from='client.? 172.21.15.148:0/1952082487' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T02:50:08.524 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:08 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:50:08.665 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:08 smithi148 ceph-mon[35090]: Deploying daemon mon.b on smithi146 2024-09-12T02:50:08.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:08 smithi148 ceph-mon[35090]: from='client.? 172.21.15.148:0/1952082487' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T02:50:08.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:08 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:50:09.089 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-12T02:50:09.089 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-12T02:50:09.346 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T02:50:09.415 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:09 smithi148 ceph-mon[35090]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:50:09.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:09 smithi005 ceph-mon[31197]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:50:09.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:09.777+0000 7f6f98de9640 1 -- 172.21.15.148:0/2826076446 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f94106630 msgr2=0x7f6f94106a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:09.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:09.777+0000 7f6f98de9640 1 --2- 172.21.15.148:0/2826076446 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f94106630 0x7f6f94106a10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f6f84009880 tx=0x7f6f8402a5d0 comp rx=0 tx=0).stop 2024-09-12T02:50:09.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:09.777+0000 7f6f98de9640 1 -- 172.21.15.148:0/2826076446 shutdown_connections 2024-09-12T02:50:09.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:09.777+0000 7f6f98de9640 1 --2- 172.21.15.148:0/2826076446 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f94106630 0x7f6f94106a10 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:09.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:09.777+0000 7f6f98de9640 1 -- 172.21.15.148:0/2826076446 >> 172.21.15.148:0/2826076446 conn(0x7f6f940fc220 msgr2=0x7f6f940fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:50:09.781 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:09.777+0000 7f6f98de9640 1 -- 172.21.15.148:0/2826076446 shutdown_connections 2024-09-12T02:50:09.782 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:09.777+0000 7f6f98de9640 1 -- 172.21.15.148:0/2826076446 wait complete. 2024-09-12T02:50:09.782 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:09.778+0000 7f6f98de9640 1 Processor -- start 2024-09-12T02:50:09.782 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:09.778+0000 7f6f98de9640 1 -- start start 2024-09-12T02:50:09.782 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:09.778+0000 7f6f98de9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f94106630 0x7f6f941962e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:09.782 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:09.778+0000 7f6f98de9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f94196820 0x7f6f9419abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:09.782 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:09.778+0000 7f6f98de9640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6f9410ac60 con 0x7f6f94106630 2024-09-12T02:50:09.782 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:09.778+0000 7f6f98de9640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f6f9410aae0 con 0x7f6f94196820 2024-09-12T02:50:09.782 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:09.779+0000 7f6f91d9b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f94196820 0x7f6f9419abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:09.782 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:09.779+0000 7f6f91d9b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f94196820 0x7f6f9419abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51800/0 (socket says 172.21.15.148:51800) 2024-09-12T02:50:09.782 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:09.779+0000 7f6f91d9b640 1 -- 172.21.15.148:0/2038283923 learned_addr learned my addr 172.21.15.148:0/2038283923 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-09-12T02:50:09.782 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:09.779+0000 7f6f91d9b640 1 -- 172.21.15.148:0/2038283923 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f94196820 msgr2=0x7f6f9419abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-12T02:50:09.782 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:09.779+0000 7f6f91d9b640 1 -- 172.21.15.148:0/2038283923 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f94196820 msgr2=0x7f6f9419abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-12T02:50:09.782 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:09.779+0000 7f6f91d9b640 1 --2- 172.21.15.148:0/2038283923 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f94196820 0x7f6f9419abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-12T02:50:09.783 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:09.779+0000 7f6f91d9b640 1 --2- 172.21.15.148:0/2038283923 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f94196820 0x7f6f9419abb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T02:50:09.783 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:09.779+0000 7f6f9259c640 1 --2- 172.21.15.148:0/2038283923 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f94106630 0x7f6f941962e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:09.783 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:09.779+0000 7f6f9259c640 1 -- 172.21.15.148:0/2038283923 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f94196820 msgr2=0x7f6f9419abb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T02:50:09.783 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:09.779+0000 7f6f9259c640 1 --2- 172.21.15.148:0/2038283923 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f94196820 0x7f6f9419abb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:09.783 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:09.779+0000 7f6f9259c640 1 -- 172.21.15.148:0/2038283923 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f94196e40 con 0x7f6f94106630 2024-09-12T02:50:09.783 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:09.779+0000 7f6f9259c640 1 --2- 172.21.15.148:0/2038283923 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f94106630 0x7f6f941962e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f6f84005ab0 tx=0x7f6f84032c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:50:09.783 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:09.779+0000 7f6f837fe640 1 -- 172.21.15.148:0/2038283923 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f6f84035070 con 0x7f6f94106630 2024-09-12T02:50:09.783 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:09.779+0000 7f6f837fe640 1 -- 172.21.15.148:0/2038283923 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f84032e40 con 0x7f6f94106630 2024-09-12T02:50:09.783 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:09.780+0000 7f6f98de9640 1 -- 172.21.15.148:0/2038283923 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f9419b0f0 con 0x7f6f94106630 2024-09-12T02:50:09.783 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:09.780+0000 7f6f837fe640 1 -- 172.21.15.148:0/2038283923 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f6f8403bc10 con 0x7f6f94106630 2024-09-12T02:50:09.783 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:09.780+0000 7f6f98de9640 1 -- 172.21.15.148:0/2038283923 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f9419b5d0 con 0x7f6f94106630 2024-09-12T02:50:09.783 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:09.780+0000 7f6f837fe640 1 -- 172.21.15.148:0/2038283923 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f6f84042030 con 0x7f6f94106630 2024-09-12T02:50:09.783 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:09.780+0000 7f6f98de9640 1 -- 172.21.15.148:0/2038283923 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f94101ca0 con 0x7f6f94106630 2024-09-12T02:50:09.784 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:09.781+0000 7f6f837fe640 1 --2- 172.21.15.148:0/2038283923 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f6f7c038e50 0x7f6f7c03b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:09.784 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:09.781+0000 7f6f837fe640 1 -- 172.21.15.148:0/2038283923 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f6f84040070 con 0x7f6f94106630 2024-09-12T02:50:09.784 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:09.781+0000 7f6f91d9b640 1 --2- 172.21.15.148:0/2038283923 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f6f7c038e50 0x7f6f7c03b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:09.784 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:09.781+0000 7f6f91d9b640 1 --2- 172.21.15.148:0/2038283923 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f6f7c038e50 0x7f6f7c03b310 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f6f88009100 tx=0x7f6f88008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:50:09.786 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:09.786+0000 7f6f837fe640 1 -- 172.21.15.148:0/2038283923 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6f8406ed00 con 0x7f6f94106630 2024-09-12T02:50:10.016 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:10.016+0000 7f6f98de9640 1 -- 172.21.15.148:0/2038283923 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f6f9419b8b0 con 0x7f6f94106630 2024-09-12T02:50:10.016 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:10.016+0000 7f6f837fe640 1 -- 172.21.15.148:0/2038283923 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1011 (secure 0 0 0) 0x7f6f8403b460 con 0x7f6f94106630 2024-09-12T02:50:10.017 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-12T02:50:10.017 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":2,"fsid":"618305f8-70b1-11ef-bcea-c7b262605968","modified":"2024-09-12T02:50:02.127801Z","created":"2024-09-12T02:48:39.314065Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:3300","nonce":0},{"type":"v1","addr":"172.21.15.5:6789","nonce":0}]},"addr":"172.21.15.5:6789/0","public_addr":"172.21.15.5:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:3300","nonce":0},{"type":"v1","addr":"172.21.15.148:6789","nonce":0}]},"addr":"172.21.15.148:6789/0","public_addr":"172.21.15.148:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-12T02:50:10.017 INFO:teuthology.orchestra.run.smithi148.stderr:dumped monmap epoch 2 2024-09-12T02:50:10.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:10.018+0000 7f6f98de9640 1 -- 172.21.15.148:0/2038283923 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f6f7c038e50 msgr2=0x7f6f7c03b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:10.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:10.018+0000 7f6f98de9640 1 --2- 172.21.15.148:0/2038283923 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f6f7c038e50 0x7f6f7c03b310 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f6f88009100 tx=0x7f6f88008040 comp rx=0 tx=0).stop 2024-09-12T02:50:10.018 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:10.018+0000 7f6f98de9640 1 -- 172.21.15.148:0/2038283923 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f94106630 msgr2=0x7f6f941962e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:10.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:10.018+0000 7f6f98de9640 1 --2- 172.21.15.148:0/2038283923 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f94106630 0x7f6f941962e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f6f84005ab0 tx=0x7f6f84032c50 comp rx=0 tx=0).stop 2024-09-12T02:50:10.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:10.018+0000 7f6f98de9640 1 -- 172.21.15.148:0/2038283923 shutdown_connections 2024-09-12T02:50:10.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:10.018+0000 7f6f98de9640 1 --2- 172.21.15.148:0/2038283923 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6f94196820 0x7f6f9419abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:10.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:10.018+0000 7f6f98de9640 1 --2- 172.21.15.148:0/2038283923 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f6f7c038e50 0x7f6f7c03b310 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:10.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:10.018+0000 7f6f98de9640 1 --2- 172.21.15.148:0/2038283923 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6f94106630 0x7f6f941962e0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:10.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:10.018+0000 7f6f98de9640 1 -- 172.21.15.148:0/2038283923 >> 172.21.15.148:0/2038283923 conn(0x7f6f940fc220 msgr2=0x7f6f940fdcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:50:10.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:10.018+0000 7f6f98de9640 1 -- 172.21.15.148:0/2038283923 shutdown_connections 2024-09-12T02:50:10.019 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:10.018+0000 7f6f98de9640 1 -- 172.21.15.148:0/2038283923 wait complete. 2024-09-12T02:50:10.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:10 smithi148 ceph-mon[35090]: from='client.? 172.21.15.148:0/2038283923' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T02:50:10.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:10 smithi005 ceph-mon[31197]: from='client.? 172.21.15.148:0/2038283923' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T02:50:11.050 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:11 smithi146 ceph-mon[35011]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-12T02:50:11.050 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:11 smithi146 ceph-mon[35011]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-12T02:50:11.050 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:11 smithi146 ceph-mon[35011]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-12T02:50:11.050 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:11 smithi146 ceph-mon[35011]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-12T02:50:11.050 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:11 smithi146 ceph-mon[35011]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-12T02:50:11.051 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:11 smithi146 ceph-mon[35011]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-12T02:50:11.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:11 smithi146 ceph-mon[35011]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-12T02:50:11.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:11 smithi146 ceph-mon[35011]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-12T02:50:11.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:11 smithi146 ceph-mon[35011]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-12T02:50:11.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:11 smithi146 ceph-mon[35011]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-12T02:50:11.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:11 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/1306750870' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-12T02:50:11.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:11 smithi146 ceph-mon[35011]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:50:11.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:11 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/1306750870' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-12T02:50:11.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:11 smithi146 ceph-mon[35011]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T02:50:11.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:11 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:11.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:11 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:11.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:11 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:11.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:11 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:11.458 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:11 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi148", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:50:11.458 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:11 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:50:11.458 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:11 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:50:11.458 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:11 smithi146 ceph-mon[35011]: Updating smithi148:/etc/ceph/ceph.conf 2024-09-12T02:50:11.458 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:11 smithi146 ceph-mon[35011]: Updating smithi148:/var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T02:50:11.458 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:11 smithi146 ceph-mon[35011]: Updating smithi148:/etc/ceph/ceph.client.admin.keyring 2024-09-12T02:50:11.458 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:11 smithi146 ceph-mon[35011]: Updating smithi148:/var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-12T02:50:11.458 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:11 smithi146 ceph-mon[35011]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi005:172.21.15.5=a;smithi146:172.21.15.146=b;smithi148:172.21.15.148=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:50:11.458 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:11 smithi146 ceph-mon[35011]: Saving service mon spec with placement smithi005:172.21.15.5=a;smithi146:172.21.15.146=b;smithi148:172.21.15.148=c;count:3 2024-09-12T02:50:11.458 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:11 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:11.458 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:11 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:11.458 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:11 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:11.458 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:11 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:11.459 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:11 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T02:50:11.459 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:11 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:50:11.459 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:11 smithi146 ceph-mon[35011]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:50:11.459 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:11 smithi146 ceph-mon[35011]: Deploying daemon mon.c on smithi148 2024-09-12T02:50:11.459 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:11 smithi146 ceph-mon[35011]: from='client.? 172.21.15.148:0/2119777598' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T02:50:11.459 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:11 smithi146 ceph-mon[35011]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:50:11.459 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:11 smithi146 ceph-mon[35011]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:50:11.459 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:11 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T02:50:11.459 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:11 smithi146 ceph-mon[35011]: mon.a calling monitor election 2024-09-12T02:50:11.460 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:11 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:50:11.460 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:11 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:50:11.460 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:11 smithi146 ceph-mon[35011]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:50:11.460 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:11 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:50:11.460 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:11 smithi146 ceph-mon[35011]: mon.c calling monitor election 2024-09-12T02:50:11.460 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:11 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:50:11.460 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:11 smithi146 ceph-mon[35011]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:50:11.460 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:11 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:50:11.460 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:11 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:50:11.460 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:11 smithi146 ceph-mon[35011]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-12T02:50:11.460 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:11 smithi146 ceph-mon[35011]: monmap epoch 2 2024-09-12T02:50:11.460 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:11 smithi146 ceph-mon[35011]: fsid 618305f8-70b1-11ef-bcea-c7b262605968 2024-09-12T02:50:11.460 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:11 smithi146 ceph-mon[35011]: last_changed 2024-09-12T02:50:02.127801+0000 2024-09-12T02:50:11.461 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:11 smithi146 ceph-mon[35011]: created 2024-09-12T02:48:39.314065+0000 2024-09-12T02:50:11.461 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:11 smithi146 ceph-mon[35011]: min_mon_release 19 (squid) 2024-09-12T02:50:11.461 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:11 smithi146 ceph-mon[35011]: election_strategy: 1 2024-09-12T02:50:11.461 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:11 smithi146 ceph-mon[35011]: 0: [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] mon.a 2024-09-12T02:50:11.461 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:11 smithi146 ceph-mon[35011]: 1: [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] mon.c 2024-09-12T02:50:11.461 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:11 smithi146 ceph-mon[35011]: fsmap 2024-09-12T02:50:11.461 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:11 smithi146 ceph-mon[35011]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T02:50:11.461 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:11 smithi146 ceph-mon[35011]: mgrmap e13: x(active, since 37s) 2024-09-12T02:50:11.461 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:11 smithi146 ceph-mon[35011]: overall HEALTH_OK 2024-09-12T02:50:11.461 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:11 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:11.461 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:11 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:11.461 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:11 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:11.462 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:11 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T02:50:11.462 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:11 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:50:11.462 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:11 smithi146 ceph-mon[35011]: Deploying daemon mon.b on smithi146 2024-09-12T02:50:11.462 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:11 smithi146 ceph-mon[35011]: from='client.? 172.21.15.148:0/1952082487' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T02:50:11.462 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:11 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:50:11.462 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:11 smithi146 ceph-mon[35011]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:50:11.462 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:11 smithi146 ceph-mon[35011]: from='client.? 172.21.15.148:0/2038283923' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T02:50:11.462 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:11 smithi146 ceph-mon[35011]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-12T02:50:11.462 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:11 smithi146 bash[34992]: 50219fda65c4452d947aba33e6b0e42a99d42fd23b4badf735fc1c2b5357c25d 2024-09-12T02:50:11.462 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:11 smithi146 systemd[1]: Started Ceph mon.b for 618305f8-70b1-11ef-bcea-c7b262605968. 2024-09-12T02:50:11.657 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-12T02:50:11.657 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-12T02:50:11.908 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T02:50:16.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:16 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T02:50:16.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:16 smithi148 ceph-mon[35090]: mon.a calling monitor election 2024-09-12T02:50:16.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:16 smithi148 ceph-mon[35090]: mon.c calling monitor election 2024-09-12T02:50:16.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:16 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:50:16.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:16 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:50:16.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:16 smithi148 ceph-mon[35090]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:50:16.417 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:16 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:50:16.417 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:16 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:50:16.417 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:16 smithi148 ceph-mon[35090]: mon.b calling monitor election 2024-09-12T02:50:16.417 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:16 smithi148 ceph-mon[35090]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:50:16.417 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:16 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:50:16.417 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:16 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:50:16.417 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:16 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:50:16.417 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:16 smithi148 ceph-mon[35090]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-12T02:50:16.417 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:16 smithi148 ceph-mon[35090]: monmap epoch 3 2024-09-12T02:50:16.417 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:16 smithi148 ceph-mon[35090]: fsid 618305f8-70b1-11ef-bcea-c7b262605968 2024-09-12T02:50:16.417 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:16 smithi148 ceph-mon[35090]: last_changed 2024-09-12T02:50:11.054757+0000 2024-09-12T02:50:16.417 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:16 smithi148 ceph-mon[35090]: created 2024-09-12T02:48:39.314065+0000 2024-09-12T02:50:16.418 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:16 smithi148 ceph-mon[35090]: min_mon_release 19 (squid) 2024-09-12T02:50:16.418 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:16 smithi148 ceph-mon[35090]: election_strategy: 1 2024-09-12T02:50:16.418 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:16 smithi148 ceph-mon[35090]: 0: [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] mon.a 2024-09-12T02:50:16.418 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:16 smithi148 ceph-mon[35090]: 1: [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] mon.c 2024-09-12T02:50:16.418 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:16 smithi148 ceph-mon[35090]: 2: [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] mon.b 2024-09-12T02:50:16.418 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:16 smithi148 ceph-mon[35090]: fsmap 2024-09-12T02:50:16.418 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:16 smithi148 ceph-mon[35090]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T02:50:16.418 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:16 smithi148 ceph-mon[35090]: mgrmap e13: x(active, since 46s) 2024-09-12T02:50:16.418 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:16 smithi148 ceph-mon[35090]: overall HEALTH_OK 2024-09-12T02:50:16.418 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:16 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:16.418 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:16 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:16.418 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:16 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:16.418 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:16 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:16.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:16 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T02:50:16.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:16 smithi146 ceph-mon[35011]: mon.a calling monitor election 2024-09-12T02:50:16.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:16 smithi146 ceph-mon[35011]: mon.c calling monitor election 2024-09-12T02:50:16.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:16 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:50:16.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:16 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:50:16.458 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:16 smithi146 ceph-mon[35011]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:50:16.458 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:16 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:50:16.458 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:16 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:50:16.458 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:16 smithi146 ceph-mon[35011]: mon.b calling monitor election 2024-09-12T02:50:16.458 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:16 smithi146 ceph-mon[35011]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:50:16.458 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:16 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:50:16.458 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:16 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:50:16.458 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:16 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:50:16.458 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:16 smithi146 ceph-mon[35011]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-12T02:50:16.458 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:16 smithi146 ceph-mon[35011]: monmap epoch 3 2024-09-12T02:50:16.458 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:16 smithi146 ceph-mon[35011]: fsid 618305f8-70b1-11ef-bcea-c7b262605968 2024-09-12T02:50:16.458 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:16 smithi146 ceph-mon[35011]: last_changed 2024-09-12T02:50:11.054757+0000 2024-09-12T02:50:16.458 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:16 smithi146 ceph-mon[35011]: created 2024-09-12T02:48:39.314065+0000 2024-09-12T02:50:16.459 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:16 smithi146 ceph-mon[35011]: min_mon_release 19 (squid) 2024-09-12T02:50:16.459 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:16 smithi146 ceph-mon[35011]: election_strategy: 1 2024-09-12T02:50:16.459 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:16 smithi146 ceph-mon[35011]: 0: [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] mon.a 2024-09-12T02:50:16.459 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:16 smithi146 ceph-mon[35011]: 1: [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] mon.c 2024-09-12T02:50:16.459 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:16 smithi146 ceph-mon[35011]: 2: [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] mon.b 2024-09-12T02:50:16.459 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:16 smithi146 ceph-mon[35011]: fsmap 2024-09-12T02:50:16.459 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:16 smithi146 ceph-mon[35011]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T02:50:16.459 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:16 smithi146 ceph-mon[35011]: mgrmap e13: x(active, since 46s) 2024-09-12T02:50:16.459 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:16 smithi146 ceph-mon[35011]: overall HEALTH_OK 2024-09-12T02:50:16.459 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:16 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:16.459 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:16 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:16.459 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:16 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:16.459 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:16 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:16.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:16 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T02:50:16.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:16 smithi005 ceph-mon[31197]: mon.a calling monitor election 2024-09-12T02:50:16.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:16 smithi005 ceph-mon[31197]: mon.c calling monitor election 2024-09-12T02:50:16.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:16 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:50:16.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:16 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:50:16.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:16 smithi005 ceph-mon[31197]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:50:16.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:16 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:50:16.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:16 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:50:16.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:16 smithi005 ceph-mon[31197]: mon.b calling monitor election 2024-09-12T02:50:16.562 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:16 smithi005 ceph-mon[31197]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:50:16.562 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:16 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:50:16.562 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:16 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:50:16.562 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:16 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:50:16.562 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:16 smithi005 ceph-mon[31197]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-12T02:50:16.562 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:16 smithi005 ceph-mon[31197]: monmap epoch 3 2024-09-12T02:50:16.562 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:16 smithi005 ceph-mon[31197]: fsid 618305f8-70b1-11ef-bcea-c7b262605968 2024-09-12T02:50:16.562 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:16 smithi005 ceph-mon[31197]: last_changed 2024-09-12T02:50:11.054757+0000 2024-09-12T02:50:16.562 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:16 smithi005 ceph-mon[31197]: created 2024-09-12T02:48:39.314065+0000 2024-09-12T02:50:16.562 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:16 smithi005 ceph-mon[31197]: min_mon_release 19 (squid) 2024-09-12T02:50:16.562 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:16 smithi005 ceph-mon[31197]: election_strategy: 1 2024-09-12T02:50:16.562 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:16 smithi005 ceph-mon[31197]: 0: [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] mon.a 2024-09-12T02:50:16.562 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:16 smithi005 ceph-mon[31197]: 1: [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] mon.c 2024-09-12T02:50:16.562 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:16 smithi005 ceph-mon[31197]: 2: [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] mon.b 2024-09-12T02:50:16.563 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:16 smithi005 ceph-mon[31197]: fsmap 2024-09-12T02:50:16.563 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:16 smithi005 ceph-mon[31197]: osdmap e4: 0 total, 0 up, 0 in 2024-09-12T02:50:16.563 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:16 smithi005 ceph-mon[31197]: mgrmap e13: x(active, since 46s) 2024-09-12T02:50:16.563 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:16 smithi005 ceph-mon[31197]: overall HEALTH_OK 2024-09-12T02:50:16.563 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:16 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:16.563 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:16 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:16.563 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:16 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:16.563 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:16 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:16.695 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:16.695+0000 7f726c1f4640 1 -- 172.21.15.148:0/354792045 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f724c003530 msgr2=0x7f724c0059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:16.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:16.695+0000 7f726c1f4640 1 --2- 172.21.15.148:0/354792045 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f724c003530 0x7f724c0059c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f72641016c0 tx=0x7f725402bfe0 comp rx=0 tx=0).stop 2024-09-12T02:50:16.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:16.695+0000 7f726c1f4640 1 -- 172.21.15.148:0/354792045 shutdown_connections 2024-09-12T02:50:16.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:16.695+0000 7f726c1f4640 1 --2- 172.21.15.148:0/354792045 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f724c003530 0x7f724c0059c0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:16.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:16.695+0000 7f726c1f4640 1 -- 172.21.15.148:0/354792045 >> 172.21.15.148:0/354792045 conn(0x7f72640fc280 msgr2=0x7f72640fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:50:16.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:16.695+0000 7f726c1f4640 1 -- 172.21.15.148:0/354792045 shutdown_connections 2024-09-12T02:50:16.696 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:16.695+0000 7f726c1f4640 1 -- 172.21.15.148:0/354792045 wait complete. 2024-09-12T02:50:16.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:16.696+0000 7f726c1f4640 1 Processor -- start 2024-09-12T02:50:16.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:16.696+0000 7f726c1f4640 1 -- start start 2024-09-12T02:50:16.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:16.696+0000 7f726c1f4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f724c003530 0x7f72641981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:16.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:16.696+0000 7f726c1f4640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f72641986e0 0x7f7264198b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:16.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:16.696+0000 7f726c1f4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f72641a1210 0x7f72641a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:16.698 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:16.696+0000 7f726c1f4640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7264108690 con 0x7f72641a1210 2024-09-12T02:50:16.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:16.696+0000 7f726c1f4640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f7264108510 con 0x7f72641986e0 2024-09-12T02:50:16.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:16.696+0000 7f726c1f4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f7264108810 con 0x7f724c003530 2024-09-12T02:50:16.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:16.696+0000 7f7269f90640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f724c003530 0x7f72641981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:16.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:16.696+0000 7f7269f90640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f724c003530 0x7f72641981a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:51814/0 (socket says 172.21.15.148:51814) 2024-09-12T02:50:16.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:16.696+0000 7f7269f90640 1 -- 172.21.15.148:0/3546763677 learned_addr learned my addr 172.21.15.148:0/3546763677 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-09-12T02:50:16.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:16.696+0000 7f7269f90640 1 -- 172.21.15.148:0/3546763677 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f724c003530 msgr2=0x7f72641981a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-12T02:50:16.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:16.696+0000 7f7269f90640 1 -- 172.21.15.148:0/3546763677 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f724c003530 msgr2=0x7f72641981a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-12T02:50:16.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:16.696+0000 7f7269f90640 1 --2- 172.21.15.148:0/3546763677 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f724c003530 0x7f72641981a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-12T02:50:16.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:16.696+0000 7f7269f90640 1 --2- 172.21.15.148:0/3546763677 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f724c003530 0x7f72641981a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T02:50:16.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:16.697+0000 7f726a791640 1 --2- 172.21.15.148:0/3546763677 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f72641a1210 0x7f72641a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:16.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:16.697+0000 7f726978f640 1 --2- 172.21.15.148:0/3546763677 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f72641986e0 0x7f7264198b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:16.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:16.697+0000 7f726978f640 1 -- 172.21.15.148:0/3546763677 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f72641986e0 msgr2=0x7f7264198b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-12T02:50:16.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:16.697+0000 7f726978f640 1 -- 172.21.15.148:0/3546763677 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f72641986e0 msgr2=0x7f7264198b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-12T02:50:16.699 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:16.697+0000 7f726978f640 1 --2- 172.21.15.148:0/3546763677 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f72641986e0 0x7f7264198b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-12T02:50:16.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:16.697+0000 7f726978f640 1 --2- 172.21.15.148:0/3546763677 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f72641986e0 0x7f7264198b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-12T02:50:16.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:16.697+0000 7f726a791640 1 -- 172.21.15.148:0/3546763677 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f724c003530 msgr2=0x7f72641981a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T02:50:16.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:16.697+0000 7f726a791640 1 --2- 172.21.15.148:0/3546763677 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f724c003530 0x7f72641981a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:16.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:16.697+0000 7f726a791640 1 -- 172.21.15.148:0/3546763677 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f72641986e0 msgr2=0x7f7264198b40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T02:50:16.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:16.697+0000 7f726a791640 1 --2- 172.21.15.148:0/3546763677 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f72641986e0 0x7f7264198b40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:16.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:16.697+0000 7f726a791640 1 -- 172.21.15.148:0/3546763677 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72641a3da0 con 0x7f72641a1210 2024-09-12T02:50:16.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:16.697+0000 7f726a791640 1 --2- 172.21.15.148:0/3546763677 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f72641a1210 0x7f72641a3600 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f725c009fd0 tx=0x7f725c00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:50:16.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:16.697+0000 7f7262ffd640 1 -- 172.21.15.148:0/3546763677 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f725c015020 con 0x7f72641a1210 2024-09-12T02:50:16.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:16.698+0000 7f7262ffd640 1 -- 172.21.15.148:0/3546763677 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f725c011d10 con 0x7f72641a1210 2024-09-12T02:50:16.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:16.698+0000 7f726c1f4640 1 -- 172.21.15.148:0/3546763677 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72641a4090 con 0x7f72641a1210 2024-09-12T02:50:16.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:16.698+0000 7f7262ffd640 1 -- 172.21.15.148:0/3546763677 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f725c00f2e0 con 0x7f72641a1210 2024-09-12T02:50:16.700 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:16.698+0000 7f726c1f4640 1 -- 172.21.15.148:0/3546763677 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72641a4580 con 0x7f72641a1210 2024-09-12T02:50:16.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:16.698+0000 7f7262ffd640 1 -- 172.21.15.148:0/3546763677 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f725c018020 con 0x7f72641a1210 2024-09-12T02:50:16.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:16.699+0000 7f7262ffd640 1 --2- 172.21.15.148:0/3546763677 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f72500394f0 0x7f725003b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:16.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:16.699+0000 7f7262ffd640 1 -- 172.21.15.148:0/3546763677 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f725c04bd10 con 0x7f72641a1210 2024-09-12T02:50:16.701 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:16.699+0000 7f726c1f4640 1 -- 172.21.15.148:0/3546763677 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72641097c0 con 0x7f72641a1210 2024-09-12T02:50:16.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:16.699+0000 7f7269f90640 1 --2- 172.21.15.148:0/3546763677 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f72500394f0 0x7f725003b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:16.702 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:16.700+0000 7f7269f90640 1 --2- 172.21.15.148:0/3546763677 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f72500394f0 0x7f725003b9b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f7254009b30 tx=0x7f7254005b00 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:50:16.704 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:16.704+0000 7f7262ffd640 1 -- 172.21.15.148:0/3546763677 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f725c01f020 con 0x7f72641a1210 2024-09-12T02:50:16.942 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:16.941+0000 7f726c1f4640 1 -- 172.21.15.148:0/3546763677 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f72641a4840 con 0x7f72641a1210 2024-09-12T02:50:16.942 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:16.942+0000 7f7262ffd640 1 -- 172.21.15.148:0/3546763677 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1281 (secure 0 0 0) 0x7f725c018310 con 0x7f72641a1210 2024-09-12T02:50:16.942 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-12T02:50:16.943 INFO:teuthology.orchestra.run.smithi148.stdout:{"epoch":3,"fsid":"618305f8-70b1-11ef-bcea-c7b262605968","modified":"2024-09-12T02:50:11.054757Z","created":"2024-09-12T02:48:39.314065Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:3300","nonce":0},{"type":"v1","addr":"172.21.15.5:6789","nonce":0}]},"addr":"172.21.15.5:6789/0","public_addr":"172.21.15.5:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:3300","nonce":0},{"type":"v1","addr":"172.21.15.148:6789","nonce":0}]},"addr":"172.21.15.148:6789/0","public_addr":"172.21.15.148:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:3300","nonce":0},{"type":"v1","addr":"172.21.15.146:6789","nonce":0}]},"addr":"172.21.15.146:6789/0","public_addr":"172.21.15.146:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-09-12T02:50:16.943 INFO:teuthology.orchestra.run.smithi148.stderr:dumped monmap epoch 3 2024-09-12T02:50:16.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:16.944+0000 7f726c1f4640 1 -- 172.21.15.148:0/3546763677 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f72500394f0 msgr2=0x7f725003b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:16.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:16.944+0000 7f726c1f4640 1 --2- 172.21.15.148:0/3546763677 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f72500394f0 0x7f725003b9b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f7254009b30 tx=0x7f7254005b00 comp rx=0 tx=0).stop 2024-09-12T02:50:16.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:16.944+0000 7f726c1f4640 1 -- 172.21.15.148:0/3546763677 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f72641a1210 msgr2=0x7f72641a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:16.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:16.944+0000 7f726c1f4640 1 --2- 172.21.15.148:0/3546763677 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f72641a1210 0x7f72641a3600 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f725c009fd0 tx=0x7f725c00b870 comp rx=0 tx=0).stop 2024-09-12T02:50:16.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:16.944+0000 7f726c1f4640 1 -- 172.21.15.148:0/3546763677 shutdown_connections 2024-09-12T02:50:16.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:16.944+0000 7f726c1f4640 1 --2- 172.21.15.148:0/3546763677 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f72500394f0 0x7f725003b9b0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:16.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:16.944+0000 7f726c1f4640 1 --2- 172.21.15.148:0/3546763677 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f72641a1210 0x7f72641a3600 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:16.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:16.944+0000 7f726c1f4640 1 --2- 172.21.15.148:0/3546763677 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f72641986e0 0x7f7264198b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:16.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:16.944+0000 7f726c1f4640 1 --2- 172.21.15.148:0/3546763677 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f724c003530 0x7f72641981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:16.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:16.944+0000 7f726c1f4640 1 -- 172.21.15.148:0/3546763677 >> 172.21.15.148:0/3546763677 conn(0x7f72640fc280 msgr2=0x7f72640fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:50:16.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:16.945+0000 7f726c1f4640 1 -- 172.21.15.148:0/3546763677 shutdown_connections 2024-09-12T02:50:16.945 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:16.945+0000 7f726c1f4640 1 -- 172.21.15.148:0/3546763677 wait complete. 2024-09-12T02:50:17.283 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:17 smithi148 ceph-mon[35090]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:50:17.283 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:17 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:50:17.283 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:17 smithi148 ceph-mon[35090]: from='client.? 172.21.15.148:0/3546763677' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T02:50:17.283 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:17 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:50:17.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:17 smithi146 ceph-mon[35011]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:50:17.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:17 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:50:17.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:17 smithi146 ceph-mon[35011]: from='client.? 172.21.15.148:0/3546763677' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T02:50:17.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:17 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:50:17.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:17 smithi005 ceph-mon[31197]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:50:17.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:17 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:50:17.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:17 smithi005 ceph-mon[31197]: from='client.? 172.21.15.148:0/3546763677' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-12T02:50:17.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:17 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:50:17.947 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-12T02:50:17.947 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph config generate-minimal-conf 2024-09-12T02:50:18.255 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:18 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:18.255 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:18 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:18.255 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:18 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:18.255 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:18 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:18.255 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:18 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:50:18.255 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:18 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:50:18.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:18 smithi005 ceph-mon[31197]: Updating smithi005:/etc/ceph/ceph.conf 2024-09-12T02:50:18.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:18 smithi005 ceph-mon[31197]: Updating smithi146:/etc/ceph/ceph.conf 2024-09-12T02:50:18.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:18 smithi005 ceph-mon[31197]: Updating smithi148:/etc/ceph/ceph.conf 2024-09-12T02:50:18.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:18 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:18.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:18 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:18.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:18 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:18.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:18 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:18.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:18 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:18.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:18 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:18.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:18 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:18.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:18 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:18.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:18 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:18.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:18 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:18.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:18 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:18.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:18 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T02:50:18.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:18 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T02:50:18.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:18 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:50:18.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:18 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:18.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:18 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:18.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:18 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:18.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:18 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:18.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:18 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:50:18.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:18 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:50:18.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:18 smithi146 ceph-mon[35011]: Updating smithi005:/etc/ceph/ceph.conf 2024-09-12T02:50:18.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:18 smithi146 ceph-mon[35011]: Updating smithi146:/etc/ceph/ceph.conf 2024-09-12T02:50:18.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:18 smithi146 ceph-mon[35011]: Updating smithi148:/etc/ceph/ceph.conf 2024-09-12T02:50:18.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:18 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:18.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:18 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:18.458 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:18 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:18.458 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:18 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:18.458 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:18 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:18.458 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:18 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:18.458 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:18 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:18.458 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:18 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:18.458 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:18 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:18.458 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:18 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:18.458 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:18 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:18.458 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:18 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T02:50:18.458 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:18 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T02:50:18.458 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:18 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:50:18.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:18 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:18.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:18 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:18.667 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:18 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:18.667 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:18 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:18.667 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:18 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:50:18.667 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:18 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:50:18.667 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:18 smithi148 ceph-mon[35090]: Updating smithi005:/etc/ceph/ceph.conf 2024-09-12T02:50:18.667 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:18 smithi148 ceph-mon[35090]: Updating smithi146:/etc/ceph/ceph.conf 2024-09-12T02:50:18.668 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:18 smithi148 ceph-mon[35090]: Updating smithi148:/etc/ceph/ceph.conf 2024-09-12T02:50:18.668 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:18 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:18.668 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:18 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:18.668 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:18 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:18.668 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:18 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:18.668 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:18 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:18.669 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:18 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:18.669 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:18 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:18.669 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:18 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:18.669 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:18 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:18.669 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:18 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:18.669 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:18 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:18.670 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:18 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T02:50:18.670 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:18 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T02:50:18.670 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:18 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:50:18.872 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:50:19.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:19 smithi146 ceph-mon[35011]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:50:19.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:19 smithi146 ceph-mon[35011]: Updating smithi148:/var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T02:50:19.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:19 smithi146 ceph-mon[35011]: Updating smithi146:/var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T02:50:19.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:19 smithi146 ceph-mon[35011]: Updating smithi005:/var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T02:50:19.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:19 smithi146 ceph-mon[35011]: Reconfiguring mon.a (unknown last config time)... 2024-09-12T02:50:19.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:19 smithi146 ceph-mon[35011]: Reconfiguring daemon mon.a on smithi005 2024-09-12T02:50:19.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:19 smithi005 ceph-mon[31197]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:50:19.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:19 smithi005 ceph-mon[31197]: Updating smithi148:/var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T02:50:19.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:19 smithi005 ceph-mon[31197]: Updating smithi146:/var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T02:50:19.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:19 smithi005 ceph-mon[31197]: Updating smithi005:/var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T02:50:19.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:19 smithi005 ceph-mon[31197]: Reconfiguring mon.a (unknown last config time)... 2024-09-12T02:50:19.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:19 smithi005 ceph-mon[31197]: Reconfiguring daemon mon.a on smithi005 2024-09-12T02:50:19.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:19 smithi148 ceph-mon[35090]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:50:19.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:19 smithi148 ceph-mon[35090]: Updating smithi148:/var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T02:50:19.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:19 smithi148 ceph-mon[35090]: Updating smithi146:/var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T02:50:19.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:19 smithi148 ceph-mon[35090]: Updating smithi005:/var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-12T02:50:19.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:19 smithi148 ceph-mon[35090]: Reconfiguring mon.a (unknown last config time)... 2024-09-12T02:50:19.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:19 smithi148 ceph-mon[35090]: Reconfiguring daemon mon.a on smithi005 2024-09-12T02:50:19.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:50:19.757+0000 7f5e93208640 1 -- 172.21.15.5:0/854421791 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e8c102d40 msgr2=0x7f5e8c103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:19.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:50:19.757+0000 7f5e93208640 1 --2- 172.21.15.5:0/854421791 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e8c102d40 0x7f5e8c103120 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c009900 tx=0x7f5e7c02a480 comp rx=0 tx=0).stop 2024-09-12T02:50:19.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:50:19.757+0000 7f5e93208640 1 -- 172.21.15.5:0/854421791 shutdown_connections 2024-09-12T02:50:19.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:50:19.757+0000 7f5e93208640 1 --2- 172.21.15.5:0/854421791 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5e8c10c170 0x7f5e8c10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:19.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:50:19.757+0000 7f5e93208640 1 --2- 172.21.15.5:0/854421791 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5e8c103660 0x7f5e8c10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:19.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:50:19.757+0000 7f5e93208640 1 --2- 172.21.15.5:0/854421791 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e8c102d40 0x7f5e8c103120 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:19.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:50:19.757+0000 7f5e93208640 1 -- 172.21.15.5:0/854421791 >> 172.21.15.5:0/854421791 conn(0x7f5e8c0fc7a0 msgr2=0x7f5e8c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:50:19.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:50:19.757+0000 7f5e93208640 1 -- 172.21.15.5:0/854421791 shutdown_connections 2024-09-12T02:50:19.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:50:19.757+0000 7f5e93208640 1 -- 172.21.15.5:0/854421791 wait complete. 2024-09-12T02:50:19.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:50:19.758+0000 7f5e93208640 1 Processor -- start 2024-09-12T02:50:19.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:50:19.758+0000 7f5e93208640 1 -- start start 2024-09-12T02:50:19.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:50:19.758+0000 7f5e93208640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5e8c102d40 0x7f5e8c19a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:19.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:50:19.758+0000 7f5e93208640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e8c103660 0x7f5e8c19abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:19.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:50:19.758+0000 7f5e93208640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5e8c10c170 0x7f5e8c19ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:19.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:50:19.758+0000 7f5e93208640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5e8c111f70 con 0x7f5e8c103660 2024-09-12T02:50:19.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:50:19.758+0000 7f5e93208640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f5e8c111df0 con 0x7f5e8c102d40 2024-09-12T02:50:19.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:50:19.758+0000 7f5e93208640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5e8c1120f0 con 0x7f5e8c10c170 2024-09-12T02:50:19.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:50:19.758+0000 7f5e8bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e8c103660 0x7f5e8c19abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:19.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:50:19.758+0000 7f5e8bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e8c103660 0x7f5e8c19abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42432/0 (socket says 172.21.15.5:42432) 2024-09-12T02:50:19.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:50:19.758+0000 7f5e90fa4640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5e8c102d40 0x7f5e8c19a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:19.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:50:19.758+0000 7f5e8bfff640 1 -- 172.21.15.5:0/215903361 learned_addr learned my addr 172.21.15.5:0/215903361 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:50:19.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:50:19.758+0000 7f5e917a5640 1 --2- 172.21.15.5:0/215903361 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5e8c10c170 0x7f5e8c19ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:19.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:50:19.758+0000 7f5e8bfff640 1 -- 172.21.15.5:0/215903361 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5e8c10c170 msgr2=0x7f5e8c19ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:19.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:50:19.758+0000 7f5e8bfff640 1 --2- 172.21.15.5:0/215903361 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5e8c10c170 0x7f5e8c19ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:19.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:50:19.758+0000 7f5e8bfff640 1 -- 172.21.15.5:0/215903361 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5e8c102d40 msgr2=0x7f5e8c19a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:19.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:50:19.758+0000 7f5e8bfff640 1 --2- 172.21.15.5:0/215903361 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5e8c102d40 0x7f5e8c19a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:19.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:50:19.758+0000 7f5e8bfff640 1 -- 172.21.15.5:0/215903361 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e8c19f620 con 0x7f5e8c103660 2024-09-12T02:50:19.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:50:19.758+0000 7f5e90fa4640 1 --2- 172.21.15.5:0/215903361 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5e8c102d40 0x7f5e8c19a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:50:19.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:50:19.759+0000 7f5e8bfff640 1 --2- 172.21.15.5:0/215903361 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e8c103660 0x7f5e8c19abb0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f5e7400e9f0 tx=0x7f5e7400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:50:19.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:50:19.759+0000 7f5e89ffb640 1 -- 172.21.15.5:0/215903361 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e7400ccc0 con 0x7f5e8c103660 2024-09-12T02:50:19.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:50:19.759+0000 7f5e89ffb640 1 -- 172.21.15.5:0/215903361 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e74010910 con 0x7f5e8c103660 2024-09-12T02:50:19.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:50:19.759+0000 7f5e93208640 1 -- 172.21.15.5:0/215903361 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e8c19f910 con 0x7f5e8c103660 2024-09-12T02:50:19.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:50:19.759+0000 7f5e89ffb640 1 -- 172.21.15.5:0/215903361 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e74018920 con 0x7f5e8c103660 2024-09-12T02:50:19.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:50:19.759+0000 7f5e93208640 1 -- 172.21.15.5:0/215903361 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e8c1a71f0 con 0x7f5e8c103660 2024-09-12T02:50:19.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:50:19.760+0000 7f5e89ffb640 1 -- 172.21.15.5:0/215903361 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f5e74010430 con 0x7f5e8c103660 2024-09-12T02:50:19.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:50:19.760+0000 7f5e93208640 1 -- 172.21.15.5:0/215903361 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e8c103120 con 0x7f5e8c103660 2024-09-12T02:50:19.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:50:19.760+0000 7f5e89ffb640 1 --2- 172.21.15.5:0/215903361 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f5e78039290 0x7f5e7803b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:19.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:50:19.760+0000 7f5e89ffb640 1 -- 172.21.15.5:0/215903361 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f5e7404c710 con 0x7f5e8c103660 2024-09-12T02:50:19.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:50:19.760+0000 7f5e90fa4640 1 --2- 172.21.15.5:0/215903361 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f5e78039290 0x7f5e7803b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:19.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:50:19.761+0000 7f5e90fa4640 1 --2- 172.21.15.5:0/215903361 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f5e78039290 0x7f5e7803b750 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c009230 tx=0x7f5e7c0058d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:50:19.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:50:19.765+0000 7f5e89ffb640 1 -- 172.21.15.5:0/215903361 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5e74020360 con 0x7f5e8c103660 2024-09-12T02:50:19.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:50:19.940+0000 7f5e93208640 1 -- 172.21.15.5:0/215903361 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f5e8c1a7520 con 0x7f5e8c103660 2024-09-12T02:50:19.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:50:19.940+0000 7f5e89ffb640 1 -- 172.21.15.5:0/215903361 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+273 (secure 0 0 0) 0x7f5e7401d210 con 0x7f5e8c103660 2024-09-12T02:50:19.941 INFO:teuthology.orchestra.run.smithi005.stdout:# minimal ceph.conf for 618305f8-70b1-11ef-bcea-c7b262605968 2024-09-12T02:50:19.941 INFO:teuthology.orchestra.run.smithi005.stdout:[global] 2024-09-12T02:50:19.941 INFO:teuthology.orchestra.run.smithi005.stdout: fsid = 618305f8-70b1-11ef-bcea-c7b262605968 2024-09-12T02:50:19.941 INFO:teuthology.orchestra.run.smithi005.stdout: mon_host = [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] 2024-09-12T02:50:19.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:50:19.943+0000 7f5e93208640 1 -- 172.21.15.5:0/215903361 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f5e78039290 msgr2=0x7f5e7803b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:19.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:50:19.943+0000 7f5e93208640 1 --2- 172.21.15.5:0/215903361 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f5e78039290 0x7f5e7803b750 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5e7c009230 tx=0x7f5e7c0058d0 comp rx=0 tx=0).stop 2024-09-12T02:50:19.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:50:19.943+0000 7f5e93208640 1 -- 172.21.15.5:0/215903361 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e8c103660 msgr2=0x7f5e8c19abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:19.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:50:19.943+0000 7f5e93208640 1 --2- 172.21.15.5:0/215903361 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e8c103660 0x7f5e8c19abb0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f5e7400e9f0 tx=0x7f5e7400eeb0 comp rx=0 tx=0).stop 2024-09-12T02:50:19.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:50:19.943+0000 7f5e93208640 1 -- 172.21.15.5:0/215903361 shutdown_connections 2024-09-12T02:50:19.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:50:19.943+0000 7f5e93208640 1 --2- 172.21.15.5:0/215903361 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f5e78039290 0x7f5e7803b750 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:19.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:50:19.943+0000 7f5e93208640 1 --2- 172.21.15.5:0/215903361 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e8c103660 0x7f5e8c19abb0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:19.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:50:19.943+0000 7f5e93208640 1 --2- 172.21.15.5:0/215903361 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5e8c10c170 0x7f5e8c19ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:19.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:50:19.943+0000 7f5e93208640 1 --2- 172.21.15.5:0/215903361 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5e8c102d40 0x7f5e8c19a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:19.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:50:19.943+0000 7f5e93208640 1 -- 172.21.15.5:0/215903361 >> 172.21.15.5:0/215903361 conn(0x7f5e8c0fc7a0 msgr2=0x7f5e8c10d490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:50:19.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:50:19.943+0000 7f5e93208640 1 -- 172.21.15.5:0/215903361 shutdown_connections 2024-09-12T02:50:19.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:50:19.943+0000 7f5e93208640 1 -- 172.21.15.5:0/215903361 wait complete. 2024-09-12T02:50:20.545 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-12T02:50:20.545 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-09-12T02:50:20.546 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/etc/ceph/ceph.conf 2024-09-12T02:50:20.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:20 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:20.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:20 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:20.557 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:20 smithi005 ceph-mon[31197]: Reconfiguring mon.b (monmap changed)... 2024-09-12T02:50:20.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:20 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T02:50:20.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:20 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T02:50:20.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:20 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:50:20.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:20 smithi005 ceph-mon[31197]: Reconfiguring daemon mon.b on smithi146 2024-09-12T02:50:20.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:20 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:20.558 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:20 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/215903361' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:50:20.558 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:20 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:20.558 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:20 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:20.558 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:20 smithi146 ceph-mon[35011]: Reconfiguring mon.b (monmap changed)... 2024-09-12T02:50:20.559 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:20 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T02:50:20.559 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:20 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T02:50:20.559 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:20 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:50:20.559 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:20 smithi146 ceph-mon[35011]: Reconfiguring daemon mon.b on smithi146 2024-09-12T02:50:20.559 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:20 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:20.559 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:20 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/215903361' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:50:20.578 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-09-12T02:50:20.578 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-12T02:50:20.646 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-12T02:50:20.646 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd of=/etc/ceph/ceph.conf 2024-09-12T02:50:20.665 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:20 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:20.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:20 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:20.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:20 smithi148 ceph-mon[35090]: Reconfiguring mon.b (monmap changed)... 2024-09-12T02:50:20.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:20 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T02:50:20.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:20 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T02:50:20.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:20 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:50:20.667 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:20 smithi148 ceph-mon[35090]: Reconfiguring daemon mon.b on smithi146 2024-09-12T02:50:20.667 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:20 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:20.667 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:20 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/215903361' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:50:20.675 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-12T02:50:20.676 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-12T02:50:20.742 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-12T02:50:20.742 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/etc/ceph/ceph.conf 2024-09-12T02:50:20.769 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-12T02:50:20.769 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-12T02:50:20.836 INFO:tasks.cephadm:Adding mgr.x on smithi005 2024-09-12T02:50:20.836 INFO:tasks.cephadm:Adding mgr.y on smithi146 2024-09-12T02:50:20.836 INFO:tasks.cephadm:Adding mgr.z on smithi148 2024-09-12T02:50:20.837 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph orch apply mgr '3;smithi005=x;smithi146=y;smithi148=z' 2024-09-12T02:50:21.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:21 smithi005 ceph-mon[31197]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:50:21.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:21 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:21.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:21 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:21.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:21 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T02:50:21.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:21 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T02:50:21.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:21 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:50:21.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:21 smithi148 ceph-mon[35090]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:50:21.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:21 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:21.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:21 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:21.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:21 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T02:50:21.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:21 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T02:50:21.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:21 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:50:21.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:21 smithi146 ceph-mon[35011]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:50:21.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:21 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:21.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:21 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:21.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:21 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-12T02:50:21.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:21 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-12T02:50:21.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:21 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:50:21.959 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T02:50:22.403 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:22.384+0000 7f817540c640 1 -- 172.21.15.148:0/4130277555 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8170101e50 msgr2=0x7f81701086e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:22.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:22.384+0000 7f817540c640 1 --2- 172.21.15.148:0/4130277555 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8170101e50 0x7f81701086e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8168009ca0 tx=0x7f816802a8c0 comp rx=0 tx=0).stop 2024-09-12T02:50:22.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:22.384+0000 7f817540c640 1 -- 172.21.15.148:0/4130277555 shutdown_connections 2024-09-12T02:50:22.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:22.384+0000 7f817540c640 1 --2- 172.21.15.148:0/4130277555 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8170101e50 0x7f81701086e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:22.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:22.384+0000 7f817540c640 1 --2- 172.21.15.148:0/4130277555 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8170101490 0x7f8170101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:22.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:22.384+0000 7f817540c640 1 --2- 172.21.15.148:0/4130277555 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8170100290 0x7f8170100690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:22.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:22.384+0000 7f817540c640 1 -- 172.21.15.148:0/4130277555 >> 172.21.15.148:0/4130277555 conn(0x7f81700fba40 msgr2=0x7f81700fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:50:22.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:22.384+0000 7f817540c640 1 -- 172.21.15.148:0/4130277555 shutdown_connections 2024-09-12T02:50:22.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:22.384+0000 7f817540c640 1 -- 172.21.15.148:0/4130277555 wait complete. 2024-09-12T02:50:22.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:22.385+0000 7f817540c640 1 Processor -- start 2024-09-12T02:50:22.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:22.385+0000 7f817540c640 1 -- start start 2024-09-12T02:50:22.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:22.385+0000 7f817540c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8170100290 0x7f8170197480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:22.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:22.385+0000 7f817540c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8170101490 0x7f8170195ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:22.404 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:22.385+0000 7f817540c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8170101e50 0x7f8170196010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:22.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:22.385+0000 7f817540c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f817010b670 con 0x7f8170101490 2024-09-12T02:50:22.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:22.385+0000 7f817540c640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f817010b4f0 con 0x7f8170101e50 2024-09-12T02:50:22.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:22.385+0000 7f817540c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f817010b7f0 con 0x7f8170100290 2024-09-12T02:50:22.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:22.385+0000 7f816effd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8170100290 0x7f8170197480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:22.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:22.385+0000 7f816effd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8170100290 0x7f8170197480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:58374/0 (socket says 172.21.15.148:58374) 2024-09-12T02:50:22.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:22.385+0000 7f816effd640 1 -- 172.21.15.148:0/835436696 learned_addr learned my addr 172.21.15.148:0/835436696 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-09-12T02:50:22.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:22.385+0000 7f8167fff640 1 --2- 172.21.15.148:0/835436696 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8170101490 0x7f8170195ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:22.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:22.385+0000 7f816f7fe640 1 --2- 172.21.15.148:0/835436696 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8170101e50 0x7f8170196010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:22.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:22.385+0000 7f816effd640 1 -- 172.21.15.148:0/835436696 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8170101e50 msgr2=0x7f8170196010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:22.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:22.385+0000 7f816effd640 1 --2- 172.21.15.148:0/835436696 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8170101e50 0x7f8170196010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:22.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:22.385+0000 7f816effd640 1 -- 172.21.15.148:0/835436696 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8170101490 msgr2=0x7f8170195ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:22.405 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:22.385+0000 7f816effd640 1 --2- 172.21.15.148:0/835436696 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8170101490 0x7f8170195ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:22.406 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:22.385+0000 7f816effd640 1 -- 172.21.15.148:0/835436696 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81701968d0 con 0x7f8170100290 2024-09-12T02:50:22.406 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:22.385+0000 7f8167fff640 1 --2- 172.21.15.148:0/835436696 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8170101490 0x7f8170195ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:50:22.406 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:22.385+0000 7f816effd640 1 --2- 172.21.15.148:0/835436696 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8170100290 0x7f8170197480 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f816000ddc0 tx=0x7f816000a620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:50:22.406 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:22.385+0000 7f816cff9640 1 -- 172.21.15.148:0/835436696 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f816001b070 con 0x7f8170100290 2024-09-12T02:50:22.406 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:22.386+0000 7f817540c640 1 -- 172.21.15.148:0/835436696 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81701a2080 con 0x7f8170100290 2024-09-12T02:50:22.406 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:22.386+0000 7f816cff9640 1 -- 172.21.15.148:0/835436696 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f816000be40 con 0x7f8170100290 2024-09-12T02:50:22.406 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:22.386+0000 7f816cff9640 1 -- 172.21.15.148:0/835436696 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8160016bb0 con 0x7f8170100290 2024-09-12T02:50:22.406 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:22.386+0000 7f817540c640 1 -- 172.21.15.148:0/835436696 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81701a25e0 con 0x7f8170100290 2024-09-12T02:50:22.406 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:22.386+0000 7f816cff9640 1 -- 172.21.15.148:0/835436696 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7f816001e070 con 0x7f8170100290 2024-09-12T02:50:22.406 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:22.386+0000 7f817540c640 1 -- 172.21.15.148:0/835436696 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8170107690 con 0x7f8170100290 2024-09-12T02:50:22.406 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:22.387+0000 7f816cff9640 1 --2- 172.21.15.148:0/835436696 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f815c039290 0x7f815c03b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:22.406 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:22.387+0000 7f816cff9640 1 -- 172.21.15.148:0/835436696 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f816004c3e0 con 0x7f8170100290 2024-09-12T02:50:22.406 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:22.387+0000 7f8167fff640 1 --2- 172.21.15.148:0/835436696 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f815c039290 0x7f815c03b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:22.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:22.387+0000 7f8167fff640 1 --2- 172.21.15.148:0/835436696 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f815c039290 0x7f815c03b750 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8158005d90 tx=0x7f8158005d20 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:50:22.407 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:22.392+0000 7f816cff9640 1 -- 172.21.15.148:0/835436696 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8160024170 con 0x7f8170100290 2024-09-12T02:50:22.570 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:22.568+0000 7f817540c640 1 -- 172.21.15.148:0/835436696 --> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi005=x;smithi146=y;smithi148=z", "target": ["mon-mgr", ""]}) -- 0x7f81700626c0 con 0x7f815c039290 2024-09-12T02:50:22.570 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:22 smithi148 ceph-mon[35090]: Reconfiguring mon.c (monmap changed)... 2024-09-12T02:50:22.570 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:22 smithi148 ceph-mon[35090]: Reconfiguring daemon mon.c on smithi148 2024-09-12T02:50:22.570 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:22 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:22.571 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:22 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:22.571 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:22 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:50:22.571 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:22 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:50:22.571 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:22 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:50:22.571 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:22 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:22.575 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:22.574+0000 7f816cff9640 1 -- 172.21.15.148:0/835436696 <== mgr.14148 v2:172.21.15.5:6800/3780631642 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f81700626c0 con 0x7f815c039290 2024-09-12T02:50:22.575 INFO:teuthology.orchestra.run.smithi148.stdout:Scheduled mgr update... 2024-09-12T02:50:22.578 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:22.576+0000 7f817540c640 1 -- 172.21.15.148:0/835436696 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f815c039290 msgr2=0x7f815c03b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:22.578 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:22.576+0000 7f817540c640 1 --2- 172.21.15.148:0/835436696 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f815c039290 0x7f815c03b750 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8158005d90 tx=0x7f8158005d20 comp rx=0 tx=0).stop 2024-09-12T02:50:22.578 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:22.576+0000 7f817540c640 1 -- 172.21.15.148:0/835436696 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8170100290 msgr2=0x7f8170197480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:22.578 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:22.576+0000 7f817540c640 1 --2- 172.21.15.148:0/835436696 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8170100290 0x7f8170197480 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f816000ddc0 tx=0x7f816000a620 comp rx=0 tx=0).stop 2024-09-12T02:50:22.578 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:22.577+0000 7f817540c640 1 -- 172.21.15.148:0/835436696 shutdown_connections 2024-09-12T02:50:22.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:22.577+0000 7f817540c640 1 --2- 172.21.15.148:0/835436696 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f815c039290 0x7f815c03b750 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:22.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:22.577+0000 7f817540c640 1 --2- 172.21.15.148:0/835436696 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8170101490 0x7f8170195ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:22.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:22.577+0000 7f817540c640 1 --2- 172.21.15.148:0/835436696 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8170101e50 0x7f8170196010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:22.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:22.577+0000 7f817540c640 1 --2- 172.21.15.148:0/835436696 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8170100290 0x7f8170197480 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:22.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:22.577+0000 7f817540c640 1 -- 172.21.15.148:0/835436696 >> 172.21.15.148:0/835436696 conn(0x7f81700fba40 msgr2=0x7f8170104860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:50:22.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:22.577+0000 7f817540c640 1 -- 172.21.15.148:0/835436696 shutdown_connections 2024-09-12T02:50:22.579 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:50:22.577+0000 7f817540c640 1 -- 172.21.15.148:0/835436696 wait complete. 2024-09-12T02:50:22.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:22 smithi146 ceph-mon[35011]: Reconfiguring mon.c (monmap changed)... 2024-09-12T02:50:22.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:22 smithi146 ceph-mon[35011]: Reconfiguring daemon mon.c on smithi148 2024-09-12T02:50:22.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:22 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:22.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:22 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:22.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:22 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:50:22.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:22 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:50:22.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:22 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:50:22.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:22 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:22.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:22 smithi005 ceph-mon[31197]: Reconfiguring mon.c (monmap changed)... 2024-09-12T02:50:22.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:22 smithi005 ceph-mon[31197]: Reconfiguring daemon mon.c on smithi148 2024-09-12T02:50:22.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:22 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:22.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:22 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:22.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:22 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:50:22.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:22 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:50:22.811 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:22 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:50:22.811 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:22 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:23.174 DEBUG:teuthology.orchestra.run.smithi146:mgr.y> sudo journalctl -f -n 0 -u ceph-618305f8-70b1-11ef-bcea-c7b262605968@mgr.y.service 2024-09-12T02:50:23.177 DEBUG:teuthology.orchestra.run.smithi148:mgr.z> sudo journalctl -f -n 0 -u ceph-618305f8-70b1-11ef-bcea-c7b262605968@mgr.z.service 2024-09-12T02:50:23.179 INFO:tasks.cephadm:Deploying OSDs... 2024-09-12T02:50:23.179 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-09-12T02:50:23.179 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/scratch_devs of=/dev/stdout 2024-09-12T02:50:23.200 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-12T02:50:23.200 DEBUG:teuthology.orchestra.run.smithi005:> stat /dev/vg_nvme/lv_1 2024-09-12T02:50:23.260 INFO:teuthology.orchestra.run.smithi005.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-12T02:50:23.260 INFO:teuthology.orchestra.run.smithi005.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T02:50:23.260 INFO:teuthology.orchestra.run.smithi005.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-09-12T02:50:23.260 INFO:teuthology.orchestra.run.smithi005.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T02:50:23.260 INFO:teuthology.orchestra.run.smithi005.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T02:50:23.260 INFO:teuthology.orchestra.run.smithi005.stdout:Access: 2024-09-12 02:49:37.756512127 +0000 2024-09-12T02:50:23.261 INFO:teuthology.orchestra.run.smithi005.stdout:Modify: 2024-09-12 02:45:18.745337075 +0000 2024-09-12T02:50:23.261 INFO:teuthology.orchestra.run.smithi005.stdout:Change: 2024-09-12 02:45:18.745337075 +0000 2024-09-12T02:50:23.261 INFO:teuthology.orchestra.run.smithi005.stdout: Birth: 2024-09-12 02:45:18.745337075 +0000 2024-09-12T02:50:23.261 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-12T02:50:23.330 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records in 2024-09-12T02:50:23.330 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records out 2024-09-12T02:50:23.330 INFO:teuthology.orchestra.run.smithi005.stderr:512 bytes copied, 0.00012977 s, 3.9 MB/s 2024-09-12T02:50:23.331 DEBUG:teuthology.orchestra.run.smithi005:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-12T02:50:23.392 DEBUG:teuthology.orchestra.run.smithi005:> stat /dev/vg_nvme/lv_2 2024-09-12T02:50:23.451 INFO:teuthology.orchestra.run.smithi005.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-12T02:50:23.451 INFO:teuthology.orchestra.run.smithi005.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T02:50:23.451 INFO:teuthology.orchestra.run.smithi005.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-12T02:50:23.451 INFO:teuthology.orchestra.run.smithi005.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T02:50:23.451 INFO:teuthology.orchestra.run.smithi005.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T02:50:23.451 INFO:teuthology.orchestra.run.smithi005.stdout:Access: 2024-09-12 02:49:37.791511399 +0000 2024-09-12T02:50:23.451 INFO:teuthology.orchestra.run.smithi005.stdout:Modify: 2024-09-12 02:45:19.052330714 +0000 2024-09-12T02:50:23.451 INFO:teuthology.orchestra.run.smithi005.stdout:Change: 2024-09-12 02:45:19.052330714 +0000 2024-09-12T02:50:23.451 INFO:teuthology.orchestra.run.smithi005.stdout: Birth: 2024-09-12 02:45:19.052330714 +0000 2024-09-12T02:50:23.452 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-12T02:50:23.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:23 smithi146 ceph-mon[35011]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:50:23.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:23 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:23.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:23 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:50:23.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:23 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:50:23.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:23 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:50:23.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:23 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:23.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:23 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T02:50:23.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:23 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-12T02:50:23.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:23 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T02:50:23.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:23 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:50:23.519 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records in 2024-09-12T02:50:23.519 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records out 2024-09-12T02:50:23.519 INFO:teuthology.orchestra.run.smithi005.stderr:512 bytes copied, 0.000131388 s, 3.9 MB/s 2024-09-12T02:50:23.520 DEBUG:teuthology.orchestra.run.smithi005:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-12T02:50:23.579 DEBUG:teuthology.orchestra.run.smithi005:> stat /dev/vg_nvme/lv_3 2024-09-12T02:50:23.637 INFO:teuthology.orchestra.run.smithi005.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-12T02:50:23.637 INFO:teuthology.orchestra.run.smithi005.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T02:50:23.637 INFO:teuthology.orchestra.run.smithi005.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-09-12T02:50:23.637 INFO:teuthology.orchestra.run.smithi005.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T02:50:23.637 INFO:teuthology.orchestra.run.smithi005.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T02:50:23.637 INFO:teuthology.orchestra.run.smithi005.stdout:Access: 2024-09-12 02:49:37.829510609 +0000 2024-09-12T02:50:23.637 INFO:teuthology.orchestra.run.smithi005.stdout:Modify: 2024-09-12 02:45:19.357324394 +0000 2024-09-12T02:50:23.637 INFO:teuthology.orchestra.run.smithi005.stdout:Change: 2024-09-12 02:45:19.357324394 +0000 2024-09-12T02:50:23.637 INFO:teuthology.orchestra.run.smithi005.stdout: Birth: 2024-09-12 02:45:19.357324394 +0000 2024-09-12T02:50:23.638 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-12T02:50:23.643 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:23 smithi148 ceph-mon[35090]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:50:23.643 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:23 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:23.643 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:23 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:50:23.643 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:23 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:50:23.643 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:23 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:50:23.643 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:23 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:23.643 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:23 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T02:50:23.644 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:23 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-12T02:50:23.644 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:23 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T02:50:23.644 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:23 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:50:23.701 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:23 smithi005 ceph-mon[31197]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:50:23.701 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:23 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:23.701 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:23 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:50:23.702 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:23 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:50:23.702 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:23 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:50:23.702 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:23 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:23.702 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:23 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T02:50:23.702 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:23 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-12T02:50:23.702 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:23 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T02:50:23.702 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:23 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:50:23.703 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records in 2024-09-12T02:50:23.704 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records out 2024-09-12T02:50:23.704 INFO:teuthology.orchestra.run.smithi005.stderr:512 bytes copied, 0.000111707 s, 4.6 MB/s 2024-09-12T02:50:23.705 DEBUG:teuthology.orchestra.run.smithi005:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-12T02:50:23.763 DEBUG:teuthology.orchestra.run.smithi005:> stat /dev/vg_nvme/lv_4 2024-09-12T02:50:23.819 INFO:teuthology.orchestra.run.smithi005.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-12T02:50:23.820 INFO:teuthology.orchestra.run.smithi005.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T02:50:23.820 INFO:teuthology.orchestra.run.smithi005.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-09-12T02:50:23.820 INFO:teuthology.orchestra.run.smithi005.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T02:50:23.820 INFO:teuthology.orchestra.run.smithi005.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T02:50:23.820 INFO:teuthology.orchestra.run.smithi005.stdout:Access: 2024-09-12 02:49:37.864509881 +0000 2024-09-12T02:50:23.820 INFO:teuthology.orchestra.run.smithi005.stdout:Modify: 2024-09-12 02:45:19.675317805 +0000 2024-09-12T02:50:23.820 INFO:teuthology.orchestra.run.smithi005.stdout:Change: 2024-09-12 02:45:19.675317805 +0000 2024-09-12T02:50:23.820 INFO:teuthology.orchestra.run.smithi005.stdout: Birth: 2024-09-12 02:45:19.675317805 +0000 2024-09-12T02:50:23.820 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-12T02:50:23.885 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records in 2024-09-12T02:50:23.885 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records out 2024-09-12T02:50:23.885 INFO:teuthology.orchestra.run.smithi005.stderr:512 bytes copied, 0.000115574 s, 4.4 MB/s 2024-09-12T02:50:23.887 DEBUG:teuthology.orchestra.run.smithi005:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-12T02:50:23.944 DEBUG:teuthology.orchestra.run.smithi146:> set -ex 2024-09-12T02:50:23.963 DEBUG:teuthology.orchestra.run.smithi146:> dd if=/scratch_devs of=/dev/stdout 2024-09-12T02:50:23.983 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-12T02:50:23.983 DEBUG:teuthology.orchestra.run.smithi146:> stat /dev/vg_nvme/lv_1 2024-09-12T02:50:24.043 INFO:teuthology.orchestra.run.smithi146.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-12T02:50:24.043 INFO:teuthology.orchestra.run.smithi146.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T02:50:24.043 INFO:teuthology.orchestra.run.smithi146.stdout:Device: 5h/5d Inode: 663 Links: 1 2024-09-12T02:50:24.043 INFO:teuthology.orchestra.run.smithi146.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T02:50:24.043 INFO:teuthology.orchestra.run.smithi146.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T02:50:24.043 INFO:teuthology.orchestra.run.smithi146.stdout:Access: 2024-09-12 02:50:10.501922533 +0000 2024-09-12T02:50:24.043 INFO:teuthology.orchestra.run.smithi146.stdout:Modify: 2024-09-12 02:45:38.559275680 +0000 2024-09-12T02:50:24.043 INFO:teuthology.orchestra.run.smithi146.stdout:Change: 2024-09-12 02:45:38.559275680 +0000 2024-09-12T02:50:24.043 INFO:teuthology.orchestra.run.smithi146.stdout: Birth: 2024-09-12 02:45:38.559275680 +0000 2024-09-12T02:50:24.044 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-12T02:50:24.111 INFO:teuthology.orchestra.run.smithi146.stderr:1+0 records in 2024-09-12T02:50:24.111 INFO:teuthology.orchestra.run.smithi146.stderr:1+0 records out 2024-09-12T02:50:24.111 INFO:teuthology.orchestra.run.smithi146.stderr:512 bytes copied, 0.000365176 s, 1.4 MB/s 2024-09-12T02:50:24.112 DEBUG:teuthology.orchestra.run.smithi146:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-12T02:50:24.171 DEBUG:teuthology.orchestra.run.smithi146:> stat /dev/vg_nvme/lv_2 2024-09-12T02:50:24.229 INFO:teuthology.orchestra.run.smithi146.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-12T02:50:24.229 INFO:teuthology.orchestra.run.smithi146.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T02:50:24.230 INFO:teuthology.orchestra.run.smithi146.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-12T02:50:24.230 INFO:teuthology.orchestra.run.smithi146.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T02:50:24.230 INFO:teuthology.orchestra.run.smithi146.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T02:50:24.230 INFO:teuthology.orchestra.run.smithi146.stdout:Access: 2024-09-12 02:50:10.501922533 +0000 2024-09-12T02:50:24.230 INFO:teuthology.orchestra.run.smithi146.stdout:Modify: 2024-09-12 02:45:38.888270692 +0000 2024-09-12T02:50:24.230 INFO:teuthology.orchestra.run.smithi146.stdout:Change: 2024-09-12 02:45:38.888270692 +0000 2024-09-12T02:50:24.230 INFO:teuthology.orchestra.run.smithi146.stdout: Birth: 2024-09-12 02:45:38.888270692 +0000 2024-09-12T02:50:24.230 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-12T02:50:24.296 INFO:teuthology.orchestra.run.smithi146.stderr:1+0 records in 2024-09-12T02:50:24.296 INFO:teuthology.orchestra.run.smithi146.stderr:1+0 records out 2024-09-12T02:50:24.296 INFO:teuthology.orchestra.run.smithi146.stderr:512 bytes copied, 0.000238521 s, 2.1 MB/s 2024-09-12T02:50:24.298 DEBUG:teuthology.orchestra.run.smithi146:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-12T02:50:24.355 DEBUG:teuthology.orchestra.run.smithi146:> stat /dev/vg_nvme/lv_3 2024-09-12T02:50:24.412 INFO:teuthology.orchestra.run.smithi146.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-12T02:50:24.412 INFO:teuthology.orchestra.run.smithi146.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T02:50:24.412 INFO:teuthology.orchestra.run.smithi146.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-09-12T02:50:24.413 INFO:teuthology.orchestra.run.smithi146.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T02:50:24.413 INFO:teuthology.orchestra.run.smithi146.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T02:50:24.413 INFO:teuthology.orchestra.run.smithi146.stdout:Access: 2024-09-12 02:50:10.502922518 +0000 2024-09-12T02:50:24.413 INFO:teuthology.orchestra.run.smithi146.stdout:Modify: 2024-09-12 02:45:39.223265613 +0000 2024-09-12T02:50:24.413 INFO:teuthology.orchestra.run.smithi146.stdout:Change: 2024-09-12 02:45:39.223265613 +0000 2024-09-12T02:50:24.413 INFO:teuthology.orchestra.run.smithi146.stdout: Birth: 2024-09-12 02:45:39.223265613 +0000 2024-09-12T02:50:24.413 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-12T02:50:24.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:24 smithi146 ceph-mon[35011]: from='client.24109 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi005=x;smithi146=y;smithi148=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:50:24.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:24 smithi146 ceph-mon[35011]: Saving service mgr spec with placement smithi005=x;smithi146=y;smithi148=z;count:3 2024-09-12T02:50:24.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:24 smithi146 ceph-mon[35011]: Deploying daemon mgr.z on smithi148 2024-09-12T02:50:24.476 INFO:teuthology.orchestra.run.smithi146.stderr:1+0 records in 2024-09-12T02:50:24.476 INFO:teuthology.orchestra.run.smithi146.stderr:1+0 records out 2024-09-12T02:50:24.476 INFO:teuthology.orchestra.run.smithi146.stderr:512 bytes copied, 0.000177118 s, 2.9 MB/s 2024-09-12T02:50:24.477 DEBUG:teuthology.orchestra.run.smithi146:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-12T02:50:24.534 DEBUG:teuthology.orchestra.run.smithi146:> stat /dev/vg_nvme/lv_4 2024-09-12T02:50:24.589 INFO:teuthology.orchestra.run.smithi146.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-12T02:50:24.590 INFO:teuthology.orchestra.run.smithi146.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T02:50:24.590 INFO:teuthology.orchestra.run.smithi146.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-09-12T02:50:24.590 INFO:teuthology.orchestra.run.smithi146.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T02:50:24.590 INFO:teuthology.orchestra.run.smithi146.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T02:50:24.590 INFO:teuthology.orchestra.run.smithi146.stdout:Access: 2024-09-12 02:50:10.502922518 +0000 2024-09-12T02:50:24.590 INFO:teuthology.orchestra.run.smithi146.stdout:Modify: 2024-09-12 02:45:39.547260701 +0000 2024-09-12T02:50:24.590 INFO:teuthology.orchestra.run.smithi146.stdout:Change: 2024-09-12 02:45:39.547260701 +0000 2024-09-12T02:50:24.590 INFO:teuthology.orchestra.run.smithi146.stdout: Birth: 2024-09-12 02:45:39.547260701 +0000 2024-09-12T02:50:24.590 DEBUG:teuthology.orchestra.run.smithi146:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-12T02:50:24.653 INFO:teuthology.orchestra.run.smithi146.stderr:1+0 records in 2024-09-12T02:50:24.653 INFO:teuthology.orchestra.run.smithi146.stderr:1+0 records out 2024-09-12T02:50:24.654 INFO:teuthology.orchestra.run.smithi146.stderr:512 bytes copied, 0.000202547 s, 2.5 MB/s 2024-09-12T02:50:24.654 DEBUG:teuthology.orchestra.run.smithi146:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-12T02:50:24.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:24 smithi148 ceph-mon[35090]: from='client.24109 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi005=x;smithi146=y;smithi148=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:50:24.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:24 smithi148 ceph-mon[35090]: Saving service mgr spec with placement smithi005=x;smithi146=y;smithi148=z;count:3 2024-09-12T02:50:24.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:24 smithi148 ceph-mon[35090]: Deploying daemon mgr.z on smithi148 2024-09-12T02:50:24.666 INFO:journalctl@ceph.mgr.z.smithi148.stdout:Sep 12 02:50:24 smithi148 systemd[1]: Starting Ceph mgr.z for 618305f8-70b1-11ef-bcea-c7b262605968... 2024-09-12T02:50:24.711 DEBUG:teuthology.orchestra.run.smithi148:> set -ex 2024-09-12T02:50:24.711 DEBUG:teuthology.orchestra.run.smithi148:> dd if=/scratch_devs of=/dev/stdout 2024-09-12T02:50:24.729 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-12T02:50:24.729 DEBUG:teuthology.orchestra.run.smithi148:> stat /dev/vg_nvme/lv_1 2024-09-12T02:50:24.785 INFO:teuthology.orchestra.run.smithi148.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-12T02:50:24.785 INFO:teuthology.orchestra.run.smithi148.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T02:50:24.785 INFO:teuthology.orchestra.run.smithi148.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-09-12T02:50:24.785 INFO:teuthology.orchestra.run.smithi148.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T02:50:24.785 INFO:teuthology.orchestra.run.smithi148.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T02:50:24.785 INFO:teuthology.orchestra.run.smithi148.stdout:Access: 2024-09-12 02:50:24.439833740 +0000 2024-09-12T02:50:24.785 INFO:teuthology.orchestra.run.smithi148.stdout:Modify: 2024-09-12 02:45:36.060519980 +0000 2024-09-12T02:50:24.785 INFO:teuthology.orchestra.run.smithi148.stdout:Change: 2024-09-12 02:45:36.060519980 +0000 2024-09-12T02:50:24.785 INFO:teuthology.orchestra.run.smithi148.stdout: Birth: 2024-09-12 02:45:36.060519980 +0000 2024-09-12T02:50:24.786 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-12T02:50:24.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:24 smithi005 ceph-mon[31197]: from='client.24109 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi005=x;smithi146=y;smithi148=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:50:24.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:24 smithi005 ceph-mon[31197]: Saving service mgr spec with placement smithi005=x;smithi146=y;smithi148=z;count:3 2024-09-12T02:50:24.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:24 smithi005 ceph-mon[31197]: Deploying daemon mgr.z on smithi148 2024-09-12T02:50:24.855 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-09-12T02:50:24.855 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-09-12T02:50:24.856 INFO:teuthology.orchestra.run.smithi148.stderr:512 bytes copied, 0.00414388 s, 124 kB/s 2024-09-12T02:50:24.856 DEBUG:teuthology.orchestra.run.smithi148:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-12T02:50:24.913 DEBUG:teuthology.orchestra.run.smithi148:> stat /dev/vg_nvme/lv_2 2024-09-12T02:50:24.970 INFO:teuthology.orchestra.run.smithi148.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-12T02:50:24.970 INFO:teuthology.orchestra.run.smithi148.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T02:50:24.971 INFO:teuthology.orchestra.run.smithi148.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-09-12T02:50:24.971 INFO:teuthology.orchestra.run.smithi148.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T02:50:24.971 INFO:teuthology.orchestra.run.smithi148.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T02:50:24.971 INFO:teuthology.orchestra.run.smithi148.stdout:Access: 2024-09-12 02:50:24.439833740 +0000 2024-09-12T02:50:24.971 INFO:teuthology.orchestra.run.smithi148.stdout:Modify: 2024-09-12 02:45:36.389511681 +0000 2024-09-12T02:50:24.971 INFO:teuthology.orchestra.run.smithi148.stdout:Change: 2024-09-12 02:45:36.389511681 +0000 2024-09-12T02:50:24.971 INFO:teuthology.orchestra.run.smithi148.stdout: Birth: 2024-09-12 02:45:36.389511681 +0000 2024-09-12T02:50:24.971 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-12T02:50:24.972 INFO:journalctl@ceph.mgr.z.smithi148.stdout:Sep 12 02:50:24 smithi148 podman[36688]: 2024-09-12 02:50:24.715100414 +0000 UTC m=+0.112112939 container create c05a997c2d4fe74dc6427091f5e8e13875beedc85e2b12334da9c4a4a0fbf65c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-z, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T02:50:24.972 INFO:journalctl@ceph.mgr.z.smithi148.stdout:Sep 12 02:50:24 smithi148 podman[36688]: 2024-09-12 02:50:24.619244515 +0000 UTC m=+0.016257046 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T02:50:24.972 INFO:journalctl@ceph.mgr.z.smithi148.stdout:Sep 12 02:50:24 smithi148 podman[36688]: 2024-09-12 02:50:24.942881885 +0000 UTC m=+0.339894409 container init c05a997c2d4fe74dc6427091f5e8e13875beedc85e2b12334da9c4a4a0fbf65c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-z, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T02:50:24.972 INFO:journalctl@ceph.mgr.z.smithi148.stdout:Sep 12 02:50:24 smithi148 podman[36688]: 2024-09-12 02:50:24.945976195 +0000 UTC m=+0.342988724 container start c05a997c2d4fe74dc6427091f5e8e13875beedc85e2b12334da9c4a4a0fbf65c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-z, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-12T02:50:24.972 INFO:journalctl@ceph.mgr.z.smithi148.stdout:Sep 12 02:50:24 smithi148 ceph-mgr[36776]: -- 172.21.15.148:0/3119462089 <== mon.1 v2:172.21.15.148:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560a31c630e0 con 0x560a31c40800 2024-09-12T02:50:25.036 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-09-12T02:50:25.036 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-09-12T02:50:25.036 INFO:teuthology.orchestra.run.smithi148.stderr:512 bytes copied, 0.000198746 s, 2.6 MB/s 2024-09-12T02:50:25.037 DEBUG:teuthology.orchestra.run.smithi148:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-12T02:50:25.094 DEBUG:teuthology.orchestra.run.smithi148:> stat /dev/vg_nvme/lv_3 2024-09-12T02:50:25.150 INFO:teuthology.orchestra.run.smithi148.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-12T02:50:25.151 INFO:teuthology.orchestra.run.smithi148.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T02:50:25.151 INFO:teuthology.orchestra.run.smithi148.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-09-12T02:50:25.151 INFO:teuthology.orchestra.run.smithi148.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T02:50:25.151 INFO:teuthology.orchestra.run.smithi148.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T02:50:25.151 INFO:teuthology.orchestra.run.smithi148.stdout:Access: 2024-09-12 02:50:24.439833740 +0000 2024-09-12T02:50:25.151 INFO:teuthology.orchestra.run.smithi148.stdout:Modify: 2024-09-12 02:45:36.722503279 +0000 2024-09-12T02:50:25.151 INFO:teuthology.orchestra.run.smithi148.stdout:Change: 2024-09-12 02:45:36.722503279 +0000 2024-09-12T02:50:25.151 INFO:teuthology.orchestra.run.smithi148.stdout: Birth: 2024-09-12 02:45:36.722503279 +0000 2024-09-12T02:50:25.151 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-12T02:50:25.215 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-09-12T02:50:25.215 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-09-12T02:50:25.215 INFO:teuthology.orchestra.run.smithi148.stderr:512 bytes copied, 0.000186387 s, 2.7 MB/s 2024-09-12T02:50:25.216 DEBUG:teuthology.orchestra.run.smithi148:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-12T02:50:25.274 DEBUG:teuthology.orchestra.run.smithi148:> stat /dev/vg_nvme/lv_4 2024-09-12T02:50:25.289 INFO:journalctl@ceph.mgr.z.smithi148.stdout:Sep 12 02:50:25 smithi148 bash[36688]: c05a997c2d4fe74dc6427091f5e8e13875beedc85e2b12334da9c4a4a0fbf65c 2024-09-12T02:50:25.289 INFO:journalctl@ceph.mgr.z.smithi148.stdout:Sep 12 02:50:25 smithi148 systemd[1]: Started Ceph mgr.z for 618305f8-70b1-11ef-bcea-c7b262605968. 2024-09-12T02:50:25.289 INFO:journalctl@ceph.mgr.z.smithi148.stdout:Sep 12 02:50:25 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-z[36739]: 2024-09-12T02:50:25.070+0000 7f77c0b4c040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-12T02:50:25.289 INFO:journalctl@ceph.mgr.z.smithi148.stdout:Sep 12 02:50:25 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-z[36739]: 2024-09-12T02:50:25.132+0000 7f77c0b4c040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-12T02:50:25.289 INFO:journalctl@ceph.mgr.z.smithi148.stdout:Sep 12 02:50:25 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-z[36739]: 2024-09-12T02:50:25.288+0000 7f77c0b4c040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-12T02:50:25.305 INFO:teuthology.orchestra.run.smithi148.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-12T02:50:25.305 INFO:teuthology.orchestra.run.smithi148.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-12T02:50:25.305 INFO:teuthology.orchestra.run.smithi148.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-09-12T02:50:25.305 INFO:teuthology.orchestra.run.smithi148.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T02:50:25.305 INFO:teuthology.orchestra.run.smithi148.stdout:Context: system_u:object_r:device_t:s0 2024-09-12T02:50:25.305 INFO:teuthology.orchestra.run.smithi148.stdout:Access: 2024-09-12 02:50:24.440833715 +0000 2024-09-12T02:50:25.305 INFO:teuthology.orchestra.run.smithi148.stdout:Modify: 2024-09-12 02:45:37.065494625 +0000 2024-09-12T02:50:25.305 INFO:teuthology.orchestra.run.smithi148.stdout:Change: 2024-09-12 02:45:37.065494625 +0000 2024-09-12T02:50:25.306 INFO:teuthology.orchestra.run.smithi148.stdout: Birth: 2024-09-12 02:45:37.065494625 +0000 2024-09-12T02:50:25.306 DEBUG:teuthology.orchestra.run.smithi148:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-12T02:50:25.369 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records in 2024-09-12T02:50:25.369 INFO:teuthology.orchestra.run.smithi148.stderr:1+0 records out 2024-09-12T02:50:25.369 INFO:teuthology.orchestra.run.smithi148.stderr:512 bytes copied, 0.000208295 s, 2.5 MB/s 2024-09-12T02:50:25.370 DEBUG:teuthology.orchestra.run.smithi148:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-12T02:50:25.427 INFO:tasks.cephadm:Deploying osd.0 on smithi005 with /dev/vg_nvme/lv_4... 2024-09-12T02:50:25.427 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-12T02:50:25.638 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:25 smithi146 ceph-mon[35011]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:50:25.638 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:25 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:25.639 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:25 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:25.639 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:25 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:25.639 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:25 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T02:50:25.639 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:25 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-12T02:50:25.639 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:25 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T02:50:25.639 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:25 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:50:25.648 INFO:journalctl@ceph.mgr.z.smithi148.stdout:Sep 12 02:50:25 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-z[36739]: 2024-09-12T02:50:25.357+0000 7f77c0b4c040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-12T02:50:25.648 INFO:journalctl@ceph.mgr.z.smithi148.stdout:Sep 12 02:50:25 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-z[36739]: 2024-09-12T02:50:25.509+0000 7f77c0b4c040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-12T02:50:25.648 INFO:journalctl@ceph.mgr.z.smithi148.stdout:Sep 12 02:50:25 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-z[36739]: 2024-09-12T02:50:25.647+0000 7f77c0b4c040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-12T02:50:25.649 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:25 smithi148 ceph-mon[35090]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:50:25.649 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:25 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:25.649 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:25 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:25.649 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:25 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:25.649 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:25 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T02:50:25.649 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:25 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-12T02:50:25.649 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:25 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T02:50:25.649 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:25 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:50:25.691 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:50:25.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:25 smithi005 ceph-mon[31197]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:50:25.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:25 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:25.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:25 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:25.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:25 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:25.811 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:25 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T02:50:25.811 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:25 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-12T02:50:25.811 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:25 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T02:50:25.811 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:25 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:50:26.322 INFO:journalctl@ceph.mgr.z.smithi148.stdout:Sep 12 02:50:25 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-z[36739]: 2024-09-12T02:50:25.961+0000 7f77c0b4c040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-12T02:50:26.322 INFO:journalctl@ceph.mgr.z.smithi148.stdout:Sep 12 02:50:26 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-z[36739]: 2024-09-12T02:50:26.021+0000 7f77c0b4c040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-12T02:50:26.322 INFO:journalctl@ceph.mgr.z.smithi148.stdout:Sep 12 02:50:26 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-z[36739]: 2024-09-12T02:50:26.203+0000 7f77c0b4c040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-12T02:50:26.455 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:26 smithi146 ceph-mon[35011]: Deploying daemon mgr.y on smithi146 2024-09-12T02:50:26.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:26 smithi005 ceph-mon[31197]: Deploying daemon mgr.y on smithi146 2024-09-12T02:50:26.665 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:26 smithi148 ceph-mon[35090]: Deploying daemon mgr.y on smithi146 2024-09-12T02:50:27.139 INFO:journalctl@ceph.mgr.y.smithi146.stdout:Sep 12 02:50:26 smithi146 systemd[1]: Starting Ceph mgr.y for 618305f8-70b1-11ef-bcea-c7b262605968... 2024-09-12T02:50:27.401 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:27 smithi146 ceph-mon[35011]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:50:27.401 INFO:journalctl@ceph.mgr.y.smithi146.stdout:Sep 12 02:50:27 smithi146 podman[36398]: 2024-09-12 02:50:27.042200533 +0000 UTC m=+0.016132977 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T02:50:27.402 INFO:journalctl@ceph.mgr.y.smithi146.stdout:Sep 12 02:50:27 smithi146 podman[36398]: 2024-09-12 02:50:27.141140883 +0000 UTC m=+0.115073314 container create 44071781cc3203a9c7001dd091a183c375a76de4586c000a7f655e6c8beb0c7b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-y, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3) 2024-09-12T02:50:27.402 INFO:journalctl@ceph.mgr.y.smithi146.stdout:Sep 12 02:50:27 smithi146 podman[36398]: 2024-09-12 02:50:27.301980712 +0000 UTC m=+0.275913152 container init 44071781cc3203a9c7001dd091a183c375a76de4586c000a7f655e6c8beb0c7b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-y, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-12T02:50:27.402 INFO:journalctl@ceph.mgr.y.smithi146.stdout:Sep 12 02:50:27 smithi146 podman[36398]: 2024-09-12 02:50:27.304941393 +0000 UTC m=+0.278873836 container start 44071781cc3203a9c7001dd091a183c375a76de4586c000a7f655e6c8beb0c7b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-y, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=) 2024-09-12T02:50:27.402 INFO:journalctl@ceph.mgr.y.smithi146.stdout:Sep 12 02:50:27 smithi146 ceph-mgr[36417]: -- 172.21.15.146:0/650446643 <== mon.2 v2:172.21.15.146:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558d8b02f0e0 con 0x558d8b00d000 2024-09-12T02:50:27.402 INFO:journalctl@ceph.mgr.y.smithi146.stdout:Sep 12 02:50:27 smithi146 bash[36398]: 44071781cc3203a9c7001dd091a183c375a76de4586c000a7f655e6c8beb0c7b 2024-09-12T02:50:27.415 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:27 smithi148 ceph-mon[35090]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:50:27.416 INFO:journalctl@ceph.mgr.z.smithi148.stdout:Sep 12 02:50:27 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-z[36739]: 2024-09-12T02:50:27.082+0000 7f77c0b4c040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-12T02:50:27.416 INFO:journalctl@ceph.mgr.z.smithi148.stdout:Sep 12 02:50:27 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-z[36739]: 2024-09-12T02:50:27.149+0000 7f77c0b4c040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-12T02:50:27.657 INFO:journalctl@ceph.mgr.y.smithi146.stdout:Sep 12 02:50:27 smithi146 systemd[1]: Started Ceph mgr.y for 618305f8-70b1-11ef-bcea-c7b262605968. 2024-09-12T02:50:27.657 INFO:journalctl@ceph.mgr.y.smithi146.stdout:Sep 12 02:50:27 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-y[36413]: 2024-09-12T02:50:27.426+0000 7ff123050040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-12T02:50:27.657 INFO:journalctl@ceph.mgr.y.smithi146.stdout:Sep 12 02:50:27 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-y[36413]: 2024-09-12T02:50:27.501+0000 7ff123050040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-12T02:50:27.657 INFO:journalctl@ceph.mgr.y.smithi146.stdout:Sep 12 02:50:27 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-y[36413]: 2024-09-12T02:50:27.656+0000 7ff123050040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-12T02:50:27.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:27 smithi005 ceph-mon[31197]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:50:27.956 INFO:journalctl@ceph.mgr.y.smithi146.stdout:Sep 12 02:50:27 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-y[36413]: 2024-09-12T02:50:27.726+0000 7ff123050040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-12T02:50:27.956 INFO:journalctl@ceph.mgr.y.smithi146.stdout:Sep 12 02:50:27 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-y[36413]: 2024-09-12T02:50:27.883+0000 7ff123050040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-12T02:50:28.096 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-12T02:50:28.123 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi005:vg_nvme/lv_4 2024-09-12T02:50:28.166 INFO:journalctl@ceph.mgr.z.smithi148.stdout:Sep 12 02:50:27 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-z[36739]: 2024-09-12T02:50:27.903+0000 7f77c0b4c040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-12T02:50:28.166 INFO:journalctl@ceph.mgr.z.smithi148.stdout:Sep 12 02:50:28 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-z[36739]: 2024-09-12T02:50:28.036+0000 7f77c0b4c040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-12T02:50:28.372 INFO:journalctl@ceph.mgr.y.smithi146.stdout:Sep 12 02:50:28 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-y[36413]: 2024-09-12T02:50:28.044+0000 7ff123050040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-12T02:50:28.372 INFO:journalctl@ceph.mgr.y.smithi146.stdout:Sep 12 02:50:28 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-y[36413]: 2024-09-12T02:50:28.371+0000 7ff123050040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-12T02:50:28.376 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:50:28.472 INFO:journalctl@ceph.mgr.z.smithi148.stdout:Sep 12 02:50:28 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-z[36739]: 2024-09-12T02:50:28.181+0000 7f77c0b4c040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-12T02:50:28.472 INFO:journalctl@ceph.mgr.z.smithi148.stdout:Sep 12 02:50:28 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-z[36739]: 2024-09-12T02:50:28.246+0000 7f77c0b4c040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-12T02:50:28.472 INFO:journalctl@ceph.mgr.z.smithi148.stdout:Sep 12 02:50:28 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-z[36739]: 2024-09-12T02:50:28.400+0000 7f77c0b4c040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-12T02:50:28.472 INFO:journalctl@ceph.mgr.z.smithi148.stdout:Sep 12 02:50:28 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-z[36739]: 2024-09-12T02:50:28.470+0000 7f77c0b4c040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-12T02:50:28.628 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:28 smithi146 ceph-mon[35011]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:50:28.628 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:28 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:28.628 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:28 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:28.628 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:28 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:28.628 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:28 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:28.628 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:28 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:50:28.628 INFO:journalctl@ceph.mgr.y.smithi146.stdout:Sep 12 02:50:28 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-y[36413]: 2024-09-12T02:50:28.433+0000 7ff123050040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-12T02:50:28.629 INFO:journalctl@ceph.mgr.y.smithi146.stdout:Sep 12 02:50:28 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-y[36413]: 2024-09-12T02:50:28.627+0000 7ff123050040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-12T02:50:28.737 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:28 smithi005 ceph-mon[31197]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:50:28.737 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:28 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:28.737 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:28 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:28.737 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:28 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:28.737 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:28 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:28.737 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:28 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:50:28.771 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:28 smithi148 ceph-mon[35090]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:50:28.771 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:28 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:28.771 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:28 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:28.771 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:28 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:28.771 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:28 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:28.771 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:28 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:50:28.771 INFO:journalctl@ceph.mgr.z.smithi148.stdout:Sep 12 02:50:28 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-z[36739]: 2024-09-12T02:50:28.532+0000 7f77c0b4c040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-12T02:50:28.771 INFO:journalctl@ceph.mgr.z.smithi148.stdout:Sep 12 02:50:28 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-z[36739]: 2024-09-12T02:50:28.701+0000 7f77c0b4c040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-12T02:50:28.772 INFO:journalctl@ceph.mgr.z.smithi148.stdout:Sep 12 02:50:28 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-z[36739]: 2024-09-12T02:50:28.769+0000 7f77c0b4c040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-12T02:50:28.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:50:28.822+0000 7fa25fbab640 1 -- 172.21.15.5:0/2997784413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa258107940 msgr2=0x7fa258109d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:28.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:50:28.822+0000 7fa25fbab640 1 --2- 172.21.15.5:0/2997784413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa258107940 0x7fa258109d30 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa250009ca0 tx=0x7fa25002a8c0 comp rx=0 tx=0).stop 2024-09-12T02:50:28.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:50:28.823+0000 7fa25fbab640 1 -- 172.21.15.5:0/2997784413 shutdown_connections 2024-09-12T02:50:28.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:50:28.823+0000 7fa25fbab640 1 --2- 172.21.15.5:0/2997784413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa258107940 0x7fa258109d30 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:28.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:50:28.823+0000 7fa25fbab640 1 --2- 172.21.15.5:0/2997784413 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa258105010 0x7fa258107400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:28.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:50:28.823+0000 7fa25fbab640 1 --2- 172.21.15.5:0/2997784413 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2581026e0 0x7fa258104ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:28.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:50:28.823+0000 7fa25fbab640 1 -- 172.21.15.5:0/2997784413 >> 172.21.15.5:0/2997784413 conn(0x7fa2580fc280 msgr2=0x7fa2580fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:50:28.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:50:28.823+0000 7fa25fbab640 1 -- 172.21.15.5:0/2997784413 shutdown_connections 2024-09-12T02:50:28.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:50:28.823+0000 7fa25fbab640 1 -- 172.21.15.5:0/2997784413 wait complete. 2024-09-12T02:50:28.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:50:28.823+0000 7fa25fbab640 1 Processor -- start 2024-09-12T02:50:28.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:50:28.823+0000 7fa25fbab640 1 -- start start 2024-09-12T02:50:28.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:50:28.824+0000 7fa25fbab640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2581026e0 0x7fa258197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:28.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:50:28.824+0000 7fa25fbab640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa258105010 0x7fa258198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:28.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:50:28.824+0000 7fa25fbab640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa258107940 0x7fa25819f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:28.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:50:28.824+0000 7fa25fbab640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa25810d880 con 0x7fa258107940 2024-09-12T02:50:28.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:50:28.824+0000 7fa25fbab640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fa25810d700 con 0x7fa258105010 2024-09-12T02:50:28.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:50:28.824+0000 7fa25fbab640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa25810da00 con 0x7fa2581026e0 2024-09-12T02:50:28.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:50:28.824+0000 7fa25e148640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa258107940 0x7fa25819f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:28.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:50:28.824+0000 7fa25d146640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa258105010 0x7fa258198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:28.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:50:28.824+0000 7fa25d146640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa258105010 0x7fa258198450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.5:43346/0 (socket says 172.21.15.5:43346) 2024-09-12T02:50:28.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:50:28.824+0000 7fa25d947640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2581026e0 0x7fa258197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:28.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:50:28.824+0000 7fa25d146640 1 -- 172.21.15.5:0/3650851777 learned_addr learned my addr 172.21.15.5:0/3650851777 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:50:28.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:50:28.824+0000 7fa25e148640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa258107940 0x7fa25819f4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40818/0 (socket says 172.21.15.5:40818) 2024-09-12T02:50:28.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:50:28.824+0000 7fa25d146640 1 -- 172.21.15.5:0/3650851777 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2581026e0 msgr2=0x7fa258197f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:28.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:50:28.824+0000 7fa25d146640 1 --2- 172.21.15.5:0/3650851777 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2581026e0 0x7fa258197f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:28.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:50:28.824+0000 7fa25d146640 1 -- 172.21.15.5:0/3650851777 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa258107940 msgr2=0x7fa25819f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:28.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:50:28.824+0000 7fa25d146640 1 --2- 172.21.15.5:0/3650851777 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa258107940 0x7fa25819f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:28.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:50:28.824+0000 7fa25d146640 1 -- 172.21.15.5:0/3650851777 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa25819fb40 con 0x7fa258105010 2024-09-12T02:50:28.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:50:28.824+0000 7fa25e148640 1 --2- 172.21.15.5:0/3650851777 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa258107940 0x7fa25819f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:50:28.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:50:28.824+0000 7fa25d146640 1 --2- 172.21.15.5:0/3650851777 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa258105010 0x7fa258198450 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa24800e9c0 tx=0x7fa24800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:50:28.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:50:28.824+0000 7fa24effd640 1 -- 172.21.15.5:0/3650851777 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa24800cd80 con 0x7fa258105010 2024-09-12T02:50:28.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:50:28.824+0000 7fa24effd640 1 -- 172.21.15.5:0/3650851777 <== mon.2 v2:172.21.15.146:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa248005500 con 0x7fa258105010 2024-09-12T02:50:28.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:50:28.825+0000 7fa25fbab640 1 -- 172.21.15.5:0/3650851777 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa25819fe30 con 0x7fa258105010 2024-09-12T02:50:28.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:50:28.825+0000 7fa24effd640 1 -- 172.21.15.5:0/3650851777 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa248010690 con 0x7fa258105010 2024-09-12T02:50:28.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:50:28.825+0000 7fa25fbab640 1 -- 172.21.15.5:0/3650851777 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2581a02e0 con 0x7fa258105010 2024-09-12T02:50:28.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:50:28.825+0000 7fa24effd640 1 -- 172.21.15.5:0/3650851777 <== mon.2 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 13) ==== 46342+0+0 (secure 0 0 0) 0x7fa248010830 con 0x7fa258105010 2024-09-12T02:50:28.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:50:28.826+0000 7fa24effd640 1 --2- 172.21.15.5:0/3650851777 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fa2440392c0 0x7fa24403b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:28.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:50:28.826+0000 7fa24effd640 1 -- 172.21.15.5:0/3650851777 <== mon.2 v2:172.21.15.146:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fa24804cad0 con 0x7fa258105010 2024-09-12T02:50:28.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:50:28.826+0000 7fa25d947640 1 --2- 172.21.15.5:0/3650851777 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fa2440392c0 0x7fa24403b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:28.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:50:28.826+0000 7fa25fbab640 1 -- 172.21.15.5:0/3650851777 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2580fdd70 con 0x7fa258105010 2024-09-12T02:50:28.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:50:28.826+0000 7fa25d947640 1 --2- 172.21.15.5:0/3650851777 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fa2440392c0 0x7fa24403b780 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fa254005d00 tx=0x7fa25400b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:50:28.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:50:28.831+0000 7fa24effd640 1 -- 172.21.15.5:0/3650851777 <== mon.2 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa2480047a0 con 0x7fa258105010 2024-09-12T02:50:29.019 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:50:29.019+0000 7fa25fbab640 1 -- 172.21.15.5:0/3650851777 --> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fa258062630 con 0x7fa2440392c0 2024-09-12T02:50:29.039 INFO:journalctl@ceph.mgr.z.smithi148.stdout:Sep 12 02:50:28 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-z[36739]: 2024-09-12T02:50:28.837+0000 7f77c0b4c040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-12T02:50:29.039 INFO:journalctl@ceph.mgr.z.smithi148.stdout:Sep 12 02:50:28 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-z[36739]: 2024-09-12T02:50:28.919+0000 7f77c0b4c040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-12T02:50:29.039 INFO:journalctl@ceph.mgr.z.smithi148.stdout:Sep 12 02:50:28 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-z[36739]: 2024-09-12T02:50:28.979+0000 7f77c0b4c040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-12T02:50:29.039 INFO:journalctl@ceph.mgr.z.smithi148.stdout:Sep 12 02:50:29 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-z[36739]: 2024-09-12T02:50:29.037+0000 7f77c0b4c040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-12T02:50:29.916 INFO:journalctl@ceph.mgr.z.smithi148.stdout:Sep 12 02:50:29 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-z[36739]: 2024-09-12T02:50:29.823+0000 7f77c0b4c040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-12T02:50:29.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:29 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:29.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:29 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:29.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:29 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:29.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:29 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:29.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:29 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:50:29.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:29 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:50:29.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:29 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:29.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:29 smithi148 ceph-mon[35090]: Reconfiguring mgr.x (unknown last config time)... 2024-09-12T02:50:29.917 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:29 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T02:50:29.917 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:29 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T02:50:29.917 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:29 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:50:29.917 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:29 smithi148 ceph-mon[35090]: Reconfiguring daemon mgr.x on smithi005 2024-09-12T02:50:29.917 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:29 smithi148 ceph-mon[35090]: from='client.24125 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:50:29.917 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:29 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:50:29.917 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:29 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:50:29.917 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:29 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:50:29.917 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:29 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:29.956 INFO:journalctl@ceph.mgr.y.smithi146.stdout:Sep 12 02:50:29 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-y[36413]: 2024-09-12T02:50:29.513+0000 7ff123050040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-12T02:50:29.956 INFO:journalctl@ceph.mgr.y.smithi146.stdout:Sep 12 02:50:29 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-y[36413]: 2024-09-12T02:50:29.580+0000 7ff123050040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-12T02:50:29.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:29 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:29.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:29 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:29.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:29 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:29.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:29 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:29.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:29 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:50:29.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:29 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:50:29.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:29 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:29.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:29 smithi146 ceph-mon[35011]: Reconfiguring mgr.x (unknown last config time)... 2024-09-12T02:50:29.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:29 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T02:50:29.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:29 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T02:50:29.958 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:29 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:50:29.958 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:29 smithi146 ceph-mon[35011]: Reconfiguring daemon mgr.x on smithi005 2024-09-12T02:50:29.958 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:29 smithi146 ceph-mon[35011]: from='client.24125 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:50:29.958 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:29 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:50:29.958 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:29 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:50:29.958 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:29 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:50:29.958 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:29 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:29.977 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:29 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:29.977 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:29 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:29.977 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:29 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:29.977 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:29 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:29.977 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:29 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:50:29.977 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:29 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:50:29.977 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:29 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:29.978 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:29 smithi005 ceph-mon[31197]: Reconfiguring mgr.x (unknown last config time)... 2024-09-12T02:50:29.978 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:29 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-12T02:50:29.978 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:29 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-12T02:50:29.978 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:29 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:50:29.978 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:29 smithi005 ceph-mon[31197]: Reconfiguring daemon mgr.x on smithi005 2024-09-12T02:50:29.978 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:29 smithi005 ceph-mon[31197]: from='client.24125 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:50:29.978 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:29 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:50:29.978 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:29 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:50:29.978 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:29 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:50:29.978 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:29 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:30.561 INFO:journalctl@ceph.mgr.y.smithi146.stdout:Sep 12 02:50:30 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-y[36413]: 2024-09-12T02:50:30.304+0000 7ff123050040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-12T02:50:30.562 INFO:journalctl@ceph.mgr.y.smithi146.stdout:Sep 12 02:50:30 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-y[36413]: 2024-09-12T02:50:30.419+0000 7ff123050040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-12T02:50:30.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:50:30.657+0000 7fa24effd640 1 -- 172.21.15.5:0/3650851777 <== mon.2 v2:172.21.15.146:3300/0 7 ==== mgrmap(e 14) ==== 92059+0+0 (secure 0 0 0) 0x7fa248021960 con 0x7fa258105010 2024-09-12T02:50:30.840 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:30 smithi146 ceph-mon[35011]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:50:30.840 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:30 smithi146 ceph-mon[35011]: Standby manager daemon z started 2024-09-12T02:50:30.840 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:30 smithi146 ceph-mon[35011]: from='mgr.? 172.21.15.148:0/3794115166' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-12T02:50:30.840 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:30 smithi146 ceph-mon[35011]: from='mgr.? 172.21.15.148:0/3794115166' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-12T02:50:30.840 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:30 smithi146 ceph-mon[35011]: from='mgr.? 172.21.15.148:0/3794115166' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-12T02:50:30.840 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:30 smithi146 ceph-mon[35011]: from='mgr.? 172.21.15.148:0/3794115166' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-12T02:50:30.840 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:30 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:30.841 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:30 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:30.841 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:30 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:50:30.841 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:30 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:50:30.841 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:30 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:50:30.841 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:30 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:30.841 INFO:journalctl@ceph.mgr.y.smithi146.stdout:Sep 12 02:50:30 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-y[36413]: 2024-09-12T02:50:30.561+0000 7ff123050040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-12T02:50:30.841 INFO:journalctl@ceph.mgr.y.smithi146.stdout:Sep 12 02:50:30 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-y[36413]: 2024-09-12T02:50:30.622+0000 7ff123050040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-12T02:50:30.841 INFO:journalctl@ceph.mgr.y.smithi146.stdout:Sep 12 02:50:30 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-y[36413]: 2024-09-12T02:50:30.772+0000 7ff123050040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-12T02:50:30.841 INFO:journalctl@ceph.mgr.y.smithi146.stdout:Sep 12 02:50:30 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-y[36413]: 2024-09-12T02:50:30.839+0000 7ff123050040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-12T02:50:30.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:30 smithi148 ceph-mon[35090]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:50:30.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:30 smithi148 ceph-mon[35090]: Standby manager daemon z started 2024-09-12T02:50:30.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:30 smithi148 ceph-mon[35090]: from='mgr.? 172.21.15.148:0/3794115166' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-12T02:50:30.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:30 smithi148 ceph-mon[35090]: from='mgr.? 172.21.15.148:0/3794115166' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-12T02:50:30.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:30 smithi148 ceph-mon[35090]: from='mgr.? 172.21.15.148:0/3794115166' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-12T02:50:30.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:30 smithi148 ceph-mon[35090]: from='mgr.? 172.21.15.148:0/3794115166' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-12T02:50:30.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:30 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:30.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:30 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:30.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:30 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:50:30.917 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:30 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:50:30.917 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:30 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:50:30.917 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:30 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:30.917 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:30 smithi005 ceph-mon[31197]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:50:30.917 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:30 smithi005 ceph-mon[31197]: Standby manager daemon z started 2024-09-12T02:50:30.917 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:30 smithi005 ceph-mon[31197]: from='mgr.? 172.21.15.148:0/3794115166' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-12T02:50:30.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:30 smithi005 ceph-mon[31197]: from='mgr.? 172.21.15.148:0/3794115166' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-12T02:50:30.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:30 smithi005 ceph-mon[31197]: from='mgr.? 172.21.15.148:0/3794115166' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-12T02:50:30.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:30 smithi005 ceph-mon[31197]: from='mgr.? 172.21.15.148:0/3794115166' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-12T02:50:30.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:30 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:30.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:30 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:30.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:30 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:50:30.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:30 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:50:30.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:30 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:50:30.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:30 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:31.123 INFO:journalctl@ceph.mgr.y.smithi146.stdout:Sep 12 02:50:30 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-y[36413]: 2024-09-12T02:50:30.899+0000 7ff123050040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-12T02:50:31.123 INFO:journalctl@ceph.mgr.y.smithi146.stdout:Sep 12 02:50:31 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-y[36413]: 2024-09-12T02:50:31.057+0000 7ff123050040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-12T02:50:31.123 INFO:journalctl@ceph.mgr.y.smithi146.stdout:Sep 12 02:50:31 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-y[36413]: 2024-09-12T02:50:31.123+0000 7ff123050040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-12T02:50:31.390 INFO:journalctl@ceph.mgr.y.smithi146.stdout:Sep 12 02:50:31 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-y[36413]: 2024-09-12T02:50:31.189+0000 7ff123050040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-12T02:50:31.390 INFO:journalctl@ceph.mgr.y.smithi146.stdout:Sep 12 02:50:31 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-y[36413]: 2024-09-12T02:50:31.271+0000 7ff123050040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-12T02:50:31.390 INFO:journalctl@ceph.mgr.y.smithi146.stdout:Sep 12 02:50:31 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-y[36413]: 2024-09-12T02:50:31.330+0000 7ff123050040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-12T02:50:31.391 INFO:journalctl@ceph.mgr.y.smithi146.stdout:Sep 12 02:50:31 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-y[36413]: 2024-09-12T02:50:31.390+0000 7ff123050040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-12T02:50:31.793 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:31 smithi005 ceph-mon[31197]: mgrmap e14: x(active, since 61s), standbys: z 2024-09-12T02:50:31.794 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:31 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-12T02:50:31.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:31 smithi148 ceph-mon[35090]: mgrmap e14: x(active, since 61s), standbys: z 2024-09-12T02:50:31.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:31 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-12T02:50:31.956 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:31 smithi146 ceph-mon[35011]: mgrmap e14: x(active, since 61s), standbys: z 2024-09-12T02:50:31.956 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:31 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-12T02:50:32.456 INFO:journalctl@ceph.mgr.y.smithi146.stdout:Sep 12 02:50:32 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-y[36413]: 2024-09-12T02:50:32.180+0000 7ff123050040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-12T02:50:32.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:32 smithi148 ceph-mon[35090]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:50:32.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:32 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/2980123906' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "17da560a-6aa2-4f2c-a95f-67481dce4d3a"}]: dispatch 2024-09-12T02:50:32.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:32 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/2980123906' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "17da560a-6aa2-4f2c-a95f-67481dce4d3a"}]': finished 2024-09-12T02:50:32.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:32 smithi148 ceph-mon[35090]: osdmap e5: 1 total, 0 up, 1 in 2024-09-12T02:50:32.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:32 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T02:50:32.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:32 smithi148 ceph-mon[35090]: Standby manager daemon y started 2024-09-12T02:50:32.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:32 smithi148 ceph-mon[35090]: from='mgr.? 172.21.15.146:0/1023632756' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-12T02:50:32.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:32 smithi148 ceph-mon[35090]: from='mgr.? 172.21.15.146:0/1023632756' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-12T02:50:32.917 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:32 smithi148 ceph-mon[35090]: from='mgr.? 172.21.15.146:0/1023632756' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-12T02:50:32.917 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:32 smithi148 ceph-mon[35090]: from='mgr.? 172.21.15.146:0/1023632756' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-12T02:50:32.956 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:32 smithi146 ceph-mon[35011]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:50:32.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:32 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/2980123906' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "17da560a-6aa2-4f2c-a95f-67481dce4d3a"}]: dispatch 2024-09-12T02:50:32.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:32 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/2980123906' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "17da560a-6aa2-4f2c-a95f-67481dce4d3a"}]': finished 2024-09-12T02:50:32.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:32 smithi146 ceph-mon[35011]: osdmap e5: 1 total, 0 up, 1 in 2024-09-12T02:50:32.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:32 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T02:50:32.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:32 smithi146 ceph-mon[35011]: Standby manager daemon y started 2024-09-12T02:50:32.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:32 smithi146 ceph-mon[35011]: from='mgr.? 172.21.15.146:0/1023632756' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-12T02:50:32.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:32 smithi146 ceph-mon[35011]: from='mgr.? 172.21.15.146:0/1023632756' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-12T02:50:32.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:32 smithi146 ceph-mon[35011]: from='mgr.? 172.21.15.146:0/1023632756' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-12T02:50:32.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:32 smithi146 ceph-mon[35011]: from='mgr.? 172.21.15.146:0/1023632756' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-12T02:50:33.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:32 smithi005 ceph-mon[31197]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:50:33.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:32 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/2980123906' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "17da560a-6aa2-4f2c-a95f-67481dce4d3a"}]: dispatch 2024-09-12T02:50:33.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:32 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/2980123906' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "17da560a-6aa2-4f2c-a95f-67481dce4d3a"}]': finished 2024-09-12T02:50:33.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:32 smithi005 ceph-mon[31197]: osdmap e5: 1 total, 0 up, 1 in 2024-09-12T02:50:33.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:32 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T02:50:33.061 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:32 smithi005 ceph-mon[31197]: Standby manager daemon y started 2024-09-12T02:50:33.061 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:32 smithi005 ceph-mon[31197]: from='mgr.? 172.21.15.146:0/1023632756' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-12T02:50:33.061 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:32 smithi005 ceph-mon[31197]: from='mgr.? 172.21.15.146:0/1023632756' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-12T02:50:33.061 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:32 smithi005 ceph-mon[31197]: from='mgr.? 172.21.15.146:0/1023632756' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-12T02:50:33.061 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:32 smithi005 ceph-mon[31197]: from='mgr.? 172.21.15.146:0/1023632756' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-12T02:50:33.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:50:33.150+0000 7fa24effd640 1 -- 172.21.15.5:0/3650851777 <== mon.2 v2:172.21.15.146:3300/0 8 ==== mgrmap(e 15) ==== 137776+0+0 (secure 0 0 0) 0x7fa248021440 con 0x7fa258105010 2024-09-12T02:50:33.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:33 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/1965761030' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:50:33.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:33 smithi148 ceph-mon[35090]: mgrmap e15: x(active, since 63s), standbys: z, y 2024-09-12T02:50:33.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:33 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-12T02:50:33.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:33 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/1965761030' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:50:33.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:33 smithi146 ceph-mon[35011]: mgrmap e15: x(active, since 63s), standbys: z, y 2024-09-12T02:50:33.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:33 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-12T02:50:34.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:33 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/1965761030' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:50:34.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:33 smithi005 ceph-mon[31197]: mgrmap e15: x(active, since 63s), standbys: z, y 2024-09-12T02:50:34.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:33 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-12T02:50:35.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:35 smithi148 ceph-mon[35090]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:50:35.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:35 smithi146 ceph-mon[35011]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:50:35.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:35 smithi005 ceph-mon[31197]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:50:37.286 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:37 smithi005 ceph-mon[31197]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:50:37.415 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:37 smithi148 ceph-mon[35090]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:50:37.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:37 smithi146 ceph-mon[35011]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:50:39.247 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:39 smithi005 ceph-mon[31197]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:50:39.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:39 smithi148 ceph-mon[35090]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:50:39.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:39 smithi146 ceph-mon[35011]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:50:41.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:41 smithi148 ceph-mon[35090]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:50:41.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:41 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-12T02:50:41.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:41 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:50:41.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:41 smithi146 ceph-mon[35011]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:50:41.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:41 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-12T02:50:41.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:41 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:50:41.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:41 smithi005 ceph-mon[31197]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:50:41.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:41 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-12T02:50:41.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:41 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:50:42.326 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:42 smithi005 ceph-mon[31197]: Deploying daemon osd.0 on smithi005 2024-09-12T02:50:42.415 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:42 smithi148 ceph-mon[35090]: Deploying daemon osd.0 on smithi005 2024-09-12T02:50:42.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:42 smithi146 ceph-mon[35011]: Deploying daemon osd.0 on smithi005 2024-09-12T02:50:43.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:43 smithi148 ceph-mon[35090]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:50:43.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:43 smithi146 ceph-mon[35011]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:50:43.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:43 smithi005 ceph-mon[31197]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:50:45.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:45 smithi146 ceph-mon[35011]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:50:45.504 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:45 smithi005 ceph-mon[31197]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:50:45.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:45 smithi148 ceph-mon[35090]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:50:47.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:47 smithi146 ceph-mon[35011]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:50:47.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:47 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:50:47.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:47 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:47.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:47 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:47.505 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:47 smithi005 ceph-mon[31197]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:50:47.505 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:47 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:50:47.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:47 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:47.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:47 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:47.665 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:47 smithi148 ceph-mon[35090]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:50:47.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:47 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:50:47.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:47 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:47.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:47 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:49.255 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:49 smithi005 ceph-mon[31197]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:50:49.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:49 smithi146 ceph-mon[35011]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:50:49.665 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:49 smithi148 ceph-mon[35090]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:50:50.877 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:50 smithi005 ceph-mon[31197]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:50:50.877 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:50 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:50.877 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:50 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:50.877 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:50 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:50:50.877 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:50 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:50:50.877 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:50 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:50.956 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:50 smithi146 ceph-mon[35011]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:50:50.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:50 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:50.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:50 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:50.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:50 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:50:50.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:50 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:50:50.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:50 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:51.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:50 smithi148 ceph-mon[35090]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:50:51.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:50 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:51.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:50 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:51.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:50 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:50:51.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:50 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:50:51.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:50 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:51.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:50:51.280+0000 7fa24effd640 1 -- 172.21.15.5:0/3650851777 <== mgr.14148 v2:172.21.15.5:6800/3780631642 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa258062630 con 0x7fa2440392c0 2024-09-12T02:50:51.282 INFO:teuthology.orchestra.run.smithi005.stdout:Created osd(s) 0 on host 'smithi005' 2024-09-12T02:50:51.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:50:51.282+0000 7fa25fbab640 1 -- 172.21.15.5:0/3650851777 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fa2440392c0 msgr2=0x7fa24403b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:51.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:50:51.282+0000 7fa25fbab640 1 --2- 172.21.15.5:0/3650851777 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fa2440392c0 0x7fa24403b780 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fa254005d00 tx=0x7fa25400b410 comp rx=0 tx=0).stop 2024-09-12T02:50:51.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:50:51.282+0000 7fa25fbab640 1 -- 172.21.15.5:0/3650851777 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa258105010 msgr2=0x7fa258198450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:51.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:50:51.282+0000 7fa25fbab640 1 --2- 172.21.15.5:0/3650851777 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa258105010 0x7fa258198450 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa24800e9c0 tx=0x7fa24800ee80 comp rx=0 tx=0).stop 2024-09-12T02:50:51.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:50:51.282+0000 7fa25fbab640 1 -- 172.21.15.5:0/3650851777 shutdown_connections 2024-09-12T02:50:51.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:50:51.282+0000 7fa25fbab640 1 --2- 172.21.15.5:0/3650851777 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fa2440392c0 0x7fa24403b780 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:51.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:50:51.282+0000 7fa25fbab640 1 --2- 172.21.15.5:0/3650851777 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa258107940 0x7fa25819f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:51.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:50:51.282+0000 7fa25fbab640 1 --2- 172.21.15.5:0/3650851777 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa258105010 0x7fa258198450 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:51.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:50:51.282+0000 7fa25fbab640 1 --2- 172.21.15.5:0/3650851777 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2581026e0 0x7fa258197f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:51.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:50:51.282+0000 7fa25fbab640 1 -- 172.21.15.5:0/3650851777 >> 172.21.15.5:0/3650851777 conn(0x7fa2580fc280 msgr2=0x7fa2581030a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:50:51.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:50:51.282+0000 7fa25fbab640 1 -- 172.21.15.5:0/3650851777 shutdown_connections 2024-09-12T02:50:51.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:50:51.282+0000 7fa25fbab640 1 -- 172.21.15.5:0/3650851777 wait complete. 2024-09-12T02:50:51.876 DEBUG:teuthology.orchestra.run.smithi005:osd.0> sudo journalctl -f -n 0 -u ceph-618305f8-70b1-11ef-bcea-c7b262605968@osd.0.service 2024-09-12T02:50:51.879 INFO:tasks.cephadm:Deploying osd.1 on smithi146 with /dev/vg_nvme/lv_4... 2024-09-12T02:50:51.879 DEBUG:teuthology.orchestra.run.smithi146:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-12T02:50:51.956 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:51 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:50:51.956 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:51 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:51.956 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:51 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:52.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:51 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:50:52.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:51 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:52.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:51 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:52.142 INFO:teuthology.orchestra.run.smithi146.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T02:50:52.165 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:51 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:50:52.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:51 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:52.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:51 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:53.224 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 12 02:50:53 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-0[42657]: 2024-09-12T02:50:53.106+0000 7f63ede80640 -1 osd.0 0 log_to_monitors true 2024-09-12T02:50:53.521 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:53 smithi146 ceph-mon[35011]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:50:53.521 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:53 smithi146 ceph-mon[35011]: from='osd.0 [v2:172.21.15.5:6802/4248507573,v1:172.21.15.5:6803/4248507573]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-12T02:50:53.532 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:53 smithi005 ceph-mon[31197]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:50:53.533 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:53 smithi005 ceph-mon[31197]: from='osd.0 [v2:172.21.15.5:6802/4248507573,v1:172.21.15.5:6803/4248507573]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-12T02:50:53.665 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:53 smithi148 ceph-mon[35090]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:50:53.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:53 smithi148 ceph-mon[35090]: from='osd.0 [v2:172.21.15.5:6802/4248507573,v1:172.21.15.5:6803/4248507573]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-12T02:50:54.318 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-12T02:50:54.344 DEBUG:teuthology.orchestra.run.smithi146:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi146:vg_nvme/lv_4 2024-09-12T02:50:54.427 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 12 02:50:54 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-0[42657]: 2024-09-12T02:50:54.287+0000 7f63e95e0640 -1 osd.0 0 waiting for initial osdmap 2024-09-12T02:50:54.428 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 12 02:50:54 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-0[42657]: 2024-09-12T02:50:54.294+0000 7f63e5c1a640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T02:50:54.428 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:54 smithi005 ceph-mon[31197]: from='osd.0 [v2:172.21.15.5:6802/4248507573,v1:172.21.15.5:6803/4248507573]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-12T02:50:54.428 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:54 smithi005 ceph-mon[31197]: osdmap e6: 1 total, 0 up, 1 in 2024-09-12T02:50:54.428 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:54 smithi005 ceph-mon[31197]: from='osd.0 [v2:172.21.15.5:6802/4248507573,v1:172.21.15.5:6803/4248507573]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2024-09-12T02:50:54.429 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:54 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T02:50:54.429 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:54 smithi005 ceph-mon[31197]: from='osd.0 [v2:172.21.15.5:6802/4248507573,v1:172.21.15.5:6803/4248507573]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2024-09-12T02:50:54.429 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:54 smithi005 ceph-mon[31197]: osdmap e7: 1 total, 0 up, 1 in 2024-09-12T02:50:54.429 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:54 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T02:50:54.594 INFO:teuthology.orchestra.run.smithi146.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T02:50:54.620 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:54 smithi146 ceph-mon[35011]: from='osd.0 [v2:172.21.15.5:6802/4248507573,v1:172.21.15.5:6803/4248507573]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-12T02:50:54.621 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:54 smithi146 ceph-mon[35011]: osdmap e6: 1 total, 0 up, 1 in 2024-09-12T02:50:54.621 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:54 smithi146 ceph-mon[35011]: from='osd.0 [v2:172.21.15.5:6802/4248507573,v1:172.21.15.5:6803/4248507573]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2024-09-12T02:50:54.621 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:54 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T02:50:54.621 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:54 smithi146 ceph-mon[35011]: from='osd.0 [v2:172.21.15.5:6802/4248507573,v1:172.21.15.5:6803/4248507573]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2024-09-12T02:50:54.621 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:54 smithi146 ceph-mon[35011]: osdmap e7: 1 total, 0 up, 1 in 2024-09-12T02:50:54.621 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:54 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T02:50:54.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:54 smithi148 ceph-mon[35090]: from='osd.0 [v2:172.21.15.5:6802/4248507573,v1:172.21.15.5:6803/4248507573]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-12T02:50:54.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:54 smithi148 ceph-mon[35090]: osdmap e6: 1 total, 0 up, 1 in 2024-09-12T02:50:54.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:54 smithi148 ceph-mon[35090]: from='osd.0 [v2:172.21.15.5:6802/4248507573,v1:172.21.15.5:6803/4248507573]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2024-09-12T02:50:54.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:54 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T02:50:54.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:54 smithi148 ceph-mon[35090]: from='osd.0 [v2:172.21.15.5:6802/4248507573,v1:172.21.15.5:6803/4248507573]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2024-09-12T02:50:54.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:54 smithi148 ceph-mon[35090]: osdmap e7: 1 total, 0 up, 1 in 2024-09-12T02:50:54.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:54 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T02:50:55.056 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:50:55.055+0000 7f03dc93b640 1 -- 172.21.15.146:0/1361354857 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f03d80fff10 msgr2=0x7f03d8100310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:55.056 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:50:55.055+0000 7f03dc93b640 1 --2- 172.21.15.146:0/1361354857 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f03d80fff10 0x7f03d8100310 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f03c8009900 tx=0x7f03c802a410 comp rx=0 tx=0).stop 2024-09-12T02:50:55.056 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:50:55.055+0000 7f03dc93b640 1 -- 172.21.15.146:0/1361354857 shutdown_connections 2024-09-12T02:50:55.056 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:50:55.055+0000 7f03dc93b640 1 --2- 172.21.15.146:0/1361354857 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03d81038b0 0x7f03d810a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:55.056 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:50:55.055+0000 7f03dc93b640 1 --2- 172.21.15.146:0/1361354857 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03d8102ef0 0x7f03d8103370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:55.056 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:50:55.055+0000 7f03dc93b640 1 --2- 172.21.15.146:0/1361354857 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f03d80fff10 0x7f03d8100310 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:55.057 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:50:55.055+0000 7f03dc93b640 1 -- 172.21.15.146:0/1361354857 >> 172.21.15.146:0/1361354857 conn(0x7f03d80fba80 msgr2=0x7f03d80fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:50:55.057 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:50:55.055+0000 7f03dc93b640 1 -- 172.21.15.146:0/1361354857 shutdown_connections 2024-09-12T02:50:55.057 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:50:55.055+0000 7f03dc93b640 1 -- 172.21.15.146:0/1361354857 wait complete. 2024-09-12T02:50:55.058 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:50:55.056+0000 7f03dc93b640 1 Processor -- start 2024-09-12T02:50:55.058 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:50:55.056+0000 7f03dc93b640 1 -- start start 2024-09-12T02:50:55.058 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:50:55.056+0000 7f03dc93b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03d80fff10 0x7f03d8197e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:55.059 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:50:55.056+0000 7f03dc93b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03d8102ef0 0x7f03d8198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:55.059 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:50:55.056+0000 7f03dc93b640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f03d81038b0 0x7f03d819f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:55.059 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:50:55.056+0000 7f03dc93b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f03d810d860 con 0x7f03d80fff10 2024-09-12T02:50:55.059 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:50:55.056+0000 7f03dc93b640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f03d810d6e0 con 0x7f03d81038b0 2024-09-12T02:50:55.059 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:50:55.056+0000 7f03dc93b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f03d810d9e0 con 0x7f03d8102ef0 2024-09-12T02:50:55.059 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:50:55.056+0000 7f03d6d9d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f03d81038b0 0x7f03d819f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:55.059 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:50:55.056+0000 7f03d6d9d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f03d81038b0 0x7f03d819f3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:34836/0 (socket says 172.21.15.146:34836) 2024-09-12T02:50:55.059 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:50:55.056+0000 7f03d6d9d640 1 -- 172.21.15.146:0/1809872066 learned_addr learned my addr 172.21.15.146:0/1809872066 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-12T02:50:55.059 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:50:55.056+0000 7f03d659c640 1 --2- 172.21.15.146:0/1809872066 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03d80fff10 0x7f03d8197e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:55.059 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:50:55.056+0000 7f03cffff640 1 --2- 172.21.15.146:0/1809872066 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03d8102ef0 0x7f03d8198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:55.059 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:50:55.056+0000 7f03d6d9d640 1 -- 172.21.15.146:0/1809872066 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03d8102ef0 msgr2=0x7f03d8198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:55.060 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:50:55.056+0000 7f03d6d9d640 1 --2- 172.21.15.146:0/1809872066 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03d8102ef0 0x7f03d8198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:55.060 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:50:55.056+0000 7f03d6d9d640 1 -- 172.21.15.146:0/1809872066 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03d80fff10 msgr2=0x7f03d8197e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:50:55.060 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:50:55.056+0000 7f03d6d9d640 1 --2- 172.21.15.146:0/1809872066 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03d80fff10 0x7f03d8197e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:50:55.060 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:50:55.056+0000 7f03d6d9d640 1 -- 172.21.15.146:0/1809872066 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03d819fa30 con 0x7f03d81038b0 2024-09-12T02:50:55.060 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:50:55.057+0000 7f03d6d9d640 1 --2- 172.21.15.146:0/1809872066 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f03d81038b0 0x7f03d819f3c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f03d000cce0 tx=0x7f03d000a620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:50:55.060 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:50:55.057+0000 7f03cf7fe640 1 -- 172.21.15.146:0/1809872066 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d000f450 con 0x7f03d81038b0 2024-09-12T02:50:55.061 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:50:55.057+0000 7f03dc93b640 1 -- 172.21.15.146:0/1809872066 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03d819fd20 con 0x7f03d81038b0 2024-09-12T02:50:55.061 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:50:55.057+0000 7f03cf7fe640 1 -- 172.21.15.146:0/1809872066 <== mon.2 v2:172.21.15.146:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03d000fad0 con 0x7f03d81038b0 2024-09-12T02:50:55.061 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:50:55.057+0000 7f03cf7fe640 1 -- 172.21.15.146:0/1809872066 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d0018c10 con 0x7f03d81038b0 2024-09-12T02:50:55.062 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:50:55.057+0000 7f03dc93b640 1 -- 172.21.15.146:0/1809872066 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03d81a0200 con 0x7f03d81038b0 2024-09-12T02:50:55.062 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:50:55.058+0000 7f03dc93b640 1 -- 172.21.15.146:0/1809872066 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03d8109020 con 0x7f03d81038b0 2024-09-12T02:50:55.062 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:50:55.059+0000 7f03cf7fe640 1 -- 172.21.15.146:0/1809872066 <== mon.2 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 15) ==== 137776+0+0 (secure 0 0 0) 0x7f03d00213f0 con 0x7f03d81038b0 2024-09-12T02:50:55.062 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:50:55.059+0000 7f03cf7fe640 1 --2- 172.21.15.146:0/1809872066 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f03c40a2b20 0x7f03c40a4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:50:55.062 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:50:55.060+0000 7f03cf7fe640 1 -- 172.21.15.146:0/1809872066 <== mon.2 v2:172.21.15.146:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1487+0+0 (secure 0 0 0) 0x7f03d00cdae0 con 0x7f03d81038b0 2024-09-12T02:50:55.062 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:50:55.060+0000 7f03d659c640 1 --2- 172.21.15.146:0/1809872066 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f03c40a2b20 0x7f03c40a4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:50:55.062 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:50:55.060+0000 7f03d659c640 1 --2- 172.21.15.146:0/1809872066 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f03c40a2b20 0x7f03c40a4fe0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f03c8031470 tx=0x7f03c8031540 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:50:55.064 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:50:55.064+0000 7f03cf7fe640 1 -- 172.21.15.146:0/1809872066 <== mon.2 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f03d0014030 con 0x7f03d81038b0 2024-09-12T02:50:55.247 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:50:55.246+0000 7f03dc93b640 1 -- 172.21.15.146:0/1809872066 --> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi146:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f03d8062630 con 0x7f03c40a2b20 2024-09-12T02:50:55.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:55 smithi005 ceph-mon[31197]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:50:55.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:55 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T02:50:55.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:55 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:55.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:55 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:55.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:55 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:50:55.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:55 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:55.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:55 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:50:55.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:55 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:50:55.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:55 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:55.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:55 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:50:55.562 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:55 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:50:55.562 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:55 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:50:55.616 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:55 smithi146 ceph-mon[35011]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:50:55.616 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:55 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T02:50:55.616 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:55 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:55.616 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:55 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:55.616 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:55 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:50:55.616 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:55 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:55.616 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:55 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:50:55.617 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:55 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:50:55.617 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:55 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:55.617 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:55 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:50:55.617 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:55 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:50:55.617 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:55 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:50:55.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:55 smithi148 ceph-mon[35090]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:50:55.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:55 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T02:50:55.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:55 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:55.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:55 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:55.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:55 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:50:55.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:55 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:55.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:55 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:50:55.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:55 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:50:55.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:55 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:50:55.667 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:55 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:50:55.667 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:55 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:50:55.667 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:55 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:50:56.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:56 smithi005 ceph-mon[31197]: purged_snaps scrub starts 2024-09-12T02:50:56.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:56 smithi005 ceph-mon[31197]: purged_snaps scrub ok 2024-09-12T02:50:56.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:56 smithi005 ceph-mon[31197]: Detected new or changed devices on smithi005 2024-09-12T02:50:56.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:56 smithi005 ceph-mon[31197]: Adjusting osd_memory_target on smithi005 to 17092M 2024-09-12T02:50:56.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:56 smithi005 ceph-mon[31197]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi146:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:50:56.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:56 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T02:50:56.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:56 smithi005 ceph-mon[31197]: osd.0 [v2:172.21.15.5:6802/4248507573,v1:172.21.15.5:6803/4248507573] boot 2024-09-12T02:50:56.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:56 smithi005 ceph-mon[31197]: osdmap e8: 1 total, 1 up, 1 in 2024-09-12T02:50:56.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:56 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T02:50:56.574 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:56 smithi146 ceph-mon[35011]: purged_snaps scrub starts 2024-09-12T02:50:56.575 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:56 smithi146 ceph-mon[35011]: purged_snaps scrub ok 2024-09-12T02:50:56.575 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:56 smithi146 ceph-mon[35011]: Detected new or changed devices on smithi005 2024-09-12T02:50:56.575 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:56 smithi146 ceph-mon[35011]: Adjusting osd_memory_target on smithi005 to 17092M 2024-09-12T02:50:56.575 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:56 smithi146 ceph-mon[35011]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi146:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:50:56.575 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:56 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T02:50:56.575 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:56 smithi146 ceph-mon[35011]: osd.0 [v2:172.21.15.5:6802/4248507573,v1:172.21.15.5:6803/4248507573] boot 2024-09-12T02:50:56.575 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:56 smithi146 ceph-mon[35011]: osdmap e8: 1 total, 1 up, 1 in 2024-09-12T02:50:56.575 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:56 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T02:50:56.665 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:56 smithi148 ceph-mon[35090]: purged_snaps scrub starts 2024-09-12T02:50:56.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:56 smithi148 ceph-mon[35090]: purged_snaps scrub ok 2024-09-12T02:50:56.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:56 smithi148 ceph-mon[35090]: Detected new or changed devices on smithi005 2024-09-12T02:50:56.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:56 smithi148 ceph-mon[35090]: Adjusting osd_memory_target on smithi005 to 17092M 2024-09-12T02:50:56.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:56 smithi148 ceph-mon[35090]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi146:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:50:56.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:56 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T02:50:56.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:56 smithi148 ceph-mon[35090]: osd.0 [v2:172.21.15.5:6802/4248507573,v1:172.21.15.5:6803/4248507573] boot 2024-09-12T02:50:56.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:56 smithi148 ceph-mon[35090]: osdmap e8: 1 total, 1 up, 1 in 2024-09-12T02:50:56.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:56 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-12T02:50:57.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:57 smithi146 ceph-mon[35011]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:50:57.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:57 smithi146 ceph-mon[35011]: osdmap e9: 1 total, 1 up, 1 in 2024-09-12T02:50:57.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:57 smithi005 ceph-mon[31197]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:50:57.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:57 smithi005 ceph-mon[31197]: osdmap e9: 1 total, 1 up, 1 in 2024-09-12T02:50:57.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:57 smithi148 ceph-mon[35090]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-12T02:50:57.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:57 smithi148 ceph-mon[35090]: osdmap e9: 1 total, 1 up, 1 in 2024-09-12T02:50:58.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:58 smithi146 ceph-mon[35011]: from='client.? 172.21.15.146:0/3882104924' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a3237006-99af-4fc1-860c-d4917032647f"}]: dispatch 2024-09-12T02:50:58.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:58 smithi146 ceph-mon[35011]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a3237006-99af-4fc1-860c-d4917032647f"}]: dispatch 2024-09-12T02:50:58.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:58 smithi146 ceph-mon[35011]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a3237006-99af-4fc1-860c-d4917032647f"}]': finished 2024-09-12T02:50:58.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:58 smithi146 ceph-mon[35011]: osdmap e10: 2 total, 1 up, 2 in 2024-09-12T02:50:58.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:58 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T02:50:58.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:58 smithi146 ceph-mon[35011]: from='client.? 172.21.15.146:0/1936929703' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:50:58.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:58 smithi005 ceph-mon[31197]: from='client.? 172.21.15.146:0/3882104924' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a3237006-99af-4fc1-860c-d4917032647f"}]: dispatch 2024-09-12T02:50:58.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:58 smithi005 ceph-mon[31197]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a3237006-99af-4fc1-860c-d4917032647f"}]: dispatch 2024-09-12T02:50:58.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:58 smithi005 ceph-mon[31197]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a3237006-99af-4fc1-860c-d4917032647f"}]': finished 2024-09-12T02:50:58.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:58 smithi005 ceph-mon[31197]: osdmap e10: 2 total, 1 up, 2 in 2024-09-12T02:50:58.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:58 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T02:50:58.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:58 smithi005 ceph-mon[31197]: from='client.? 172.21.15.146:0/1936929703' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:50:58.665 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:58 smithi148 ceph-mon[35090]: from='client.? 172.21.15.146:0/3882104924' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a3237006-99af-4fc1-860c-d4917032647f"}]: dispatch 2024-09-12T02:50:58.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:58 smithi148 ceph-mon[35090]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a3237006-99af-4fc1-860c-d4917032647f"}]: dispatch 2024-09-12T02:50:58.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:58 smithi148 ceph-mon[35090]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a3237006-99af-4fc1-860c-d4917032647f"}]': finished 2024-09-12T02:50:58.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:58 smithi148 ceph-mon[35090]: osdmap e10: 2 total, 1 up, 2 in 2024-09-12T02:50:58.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:58 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T02:50:58.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:58 smithi148 ceph-mon[35090]: from='client.? 172.21.15.146:0/1936929703' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:50:59.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:50:59 smithi005 ceph-mon[31197]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:50:59.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:50:59 smithi148 ceph-mon[35090]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:50:59.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:50:59 smithi146 ceph-mon[35011]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:51:01.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:01 smithi005 ceph-mon[31197]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:51:01.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:01 smithi148 ceph-mon[35090]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:51:01.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:01 smithi146 ceph-mon[35011]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:51:03.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:03 smithi148 ceph-mon[35090]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:51:03.673 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:03 smithi146 ceph-mon[35011]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:51:03.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:03 smithi005 ceph-mon[31197]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:51:05.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:05 smithi146 ceph-mon[35011]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:51:05.665 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:05 smithi148 ceph-mon[35090]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:51:05.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:05 smithi005 ceph-mon[31197]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:51:06.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:06 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-12T02:51:06.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:06 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:51:06.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:06 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-12T02:51:06.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:06 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:51:06.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:06 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-12T02:51:06.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:06 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:51:07.419 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:07 smithi146 ceph-mon[35011]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:51:07.419 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:07 smithi146 ceph-mon[35011]: Deploying daemon osd.1 on smithi146 2024-09-12T02:51:07.665 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:07 smithi148 ceph-mon[35090]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:51:07.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:07 smithi148 ceph-mon[35090]: Deploying daemon osd.1 on smithi146 2024-09-12T02:51:07.712 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:07 smithi005 ceph-mon[31197]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:51:07.712 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:07 smithi005 ceph-mon[31197]: Deploying daemon osd.1 on smithi146 2024-09-12T02:51:09.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:09 smithi148 ceph-mon[35090]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:51:09.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:09 smithi146 ceph-mon[35011]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:51:09.811 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:09 smithi005 ceph-mon[31197]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:51:11.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:11 smithi146 ceph-mon[35011]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:51:11.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:11 smithi148 ceph-mon[35090]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:51:11.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:11 smithi005 ceph-mon[31197]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:51:12.527 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:12 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:51:12.528 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:12 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:51:12.528 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:12 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:51:12.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:12 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:51:12.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:12 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:51:12.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:12 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:51:12.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:12 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:51:12.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:12 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:51:12.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:12 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:51:13.665 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:13 smithi148 ceph-mon[35090]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:51:13.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:13 smithi146 ceph-mon[35011]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:51:13.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:13 smithi005 ceph-mon[31197]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:51:15.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:14 smithi148 ceph-mon[35090]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:51:15.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:14 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:51:15.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:14 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:51:15.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:14 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:51:15.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:14 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:51:15.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:14 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:51:15.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:14 smithi146 ceph-mon[35011]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:51:15.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:14 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:51:15.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:14 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:51:15.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:14 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:51:15.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:14 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:51:15.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:14 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:51:15.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:14 smithi005 ceph-mon[31197]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:51:15.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:14 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:51:15.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:14 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:51:15.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:14 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:51:15.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:14 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:51:15.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:14 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:51:15.840 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:51:15.836+0000 7f03cf7fe640 1 -- 172.21.15.146:0/1809872066 <== mgr.14148 v2:172.21.15.5:6800/3780631642 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f03d8062630 con 0x7f03c40a2b20 2024-09-12T02:51:15.840 INFO:teuthology.orchestra.run.smithi146.stdout:Created osd(s) 1 on host 'smithi146' 2024-09-12T02:51:15.841 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:51:15.838+0000 7f03dc93b640 1 -- 172.21.15.146:0/1809872066 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f03c40a2b20 msgr2=0x7f03c40a4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:51:15.841 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:51:15.838+0000 7f03dc93b640 1 --2- 172.21.15.146:0/1809872066 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f03c40a2b20 0x7f03c40a4fe0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f03c8031470 tx=0x7f03c8031540 comp rx=0 tx=0).stop 2024-09-12T02:51:15.841 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:51:15.838+0000 7f03dc93b640 1 -- 172.21.15.146:0/1809872066 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f03d81038b0 msgr2=0x7f03d819f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:51:15.841 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:51:15.838+0000 7f03dc93b640 1 --2- 172.21.15.146:0/1809872066 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f03d81038b0 0x7f03d819f3c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f03d000cce0 tx=0x7f03d000a620 comp rx=0 tx=0).stop 2024-09-12T02:51:15.841 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:51:15.838+0000 7f03dc93b640 1 -- 172.21.15.146:0/1809872066 shutdown_connections 2024-09-12T02:51:15.841 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:51:15.838+0000 7f03dc93b640 1 --2- 172.21.15.146:0/1809872066 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f03d81038b0 0x7f03d819f3c0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:15.841 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:51:15.838+0000 7f03dc93b640 1 --2- 172.21.15.146:0/1809872066 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f03d8102ef0 0x7f03d8198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:15.841 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:51:15.838+0000 7f03dc93b640 1 --2- 172.21.15.146:0/1809872066 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f03c40a2b20 0x7f03c40a4fe0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:15.841 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:51:15.838+0000 7f03dc93b640 1 --2- 172.21.15.146:0/1809872066 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03d80fff10 0x7f03d8197e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:15.842 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:51:15.838+0000 7f03dc93b640 1 -- 172.21.15.146:0/1809872066 >> 172.21.15.146:0/1809872066 conn(0x7f03d80fba80 msgr2=0x7f03d81061f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:51:15.842 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:51:15.838+0000 7f03dc93b640 1 -- 172.21.15.146:0/1809872066 shutdown_connections 2024-09-12T02:51:15.842 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:51:15.838+0000 7f03dc93b640 1 -- 172.21.15.146:0/1809872066 wait complete. 2024-09-12T02:51:16.520 DEBUG:teuthology.orchestra.run.smithi146:osd.1> sudo journalctl -f -n 0 -u ceph-618305f8-70b1-11ef-bcea-c7b262605968@osd.1.service 2024-09-12T02:51:16.522 INFO:tasks.cephadm:Deploying osd.2 on smithi148 with /dev/vg_nvme/lv_4... 2024-09-12T02:51:16.523 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-12T02:51:16.777 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T02:51:17.120 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:16 smithi148 ceph-mon[35090]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:51:17.120 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:16 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:51:17.120 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:16 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:51:17.120 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:16 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:51:17.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:16 smithi146 ceph-mon[35011]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:51:17.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:16 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:51:17.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:16 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:51:17.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:16 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:51:17.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:16 smithi005 ceph-mon[31197]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:51:17.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:16 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:51:17.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:16 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:51:17.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:16 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:51:18.457 INFO:journalctl@ceph.osd.1.smithi146.stdout:Sep 12 02:51:18 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-1[39529]: 2024-09-12T02:51:18.072+0000 7f281e7c6640 -1 osd.1 0 log_to_monitors true 2024-09-12T02:51:18.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:18 smithi148 ceph-mon[35090]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:51:18.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:18 smithi148 ceph-mon[35090]: from='osd.1 [v2:172.21.15.146:6800/1105990287,v1:172.21.15.146:6801/1105990287]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-12T02:51:18.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:18 smithi148 ceph-mon[35090]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-12T02:51:19.039 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:18 smithi146 ceph-mon[35011]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:51:19.039 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:18 smithi146 ceph-mon[35011]: from='osd.1 [v2:172.21.15.146:6800/1105990287,v1:172.21.15.146:6801/1105990287]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-12T02:51:19.039 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:18 smithi146 ceph-mon[35011]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-12T02:51:19.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:18 smithi005 ceph-mon[31197]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:51:19.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:18 smithi005 ceph-mon[31197]: from='osd.1 [v2:172.21.15.146:6800/1105990287,v1:172.21.15.146:6801/1105990287]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-12T02:51:19.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:18 smithi005 ceph-mon[31197]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-12T02:51:19.417 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-12T02:51:19.448 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi148:vg_nvme/lv_4 2024-09-12T02:51:19.696 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T02:51:19.957 INFO:journalctl@ceph.osd.1.smithi146.stdout:Sep 12 02:51:19 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-1[39529]: 2024-09-12T02:51:19.833+0000 7f2819f26640 -1 osd.1 0 waiting for initial osdmap 2024-09-12T02:51:19.957 INFO:journalctl@ceph.osd.1.smithi146.stdout:Sep 12 02:51:19 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-1[39529]: 2024-09-12T02:51:19.837+0000 7f2816560640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T02:51:19.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:19 smithi146 ceph-mon[35011]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-12T02:51:19.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:19 smithi146 ceph-mon[35011]: osdmap e11: 2 total, 1 up, 2 in 2024-09-12T02:51:19.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:19 smithi146 ceph-mon[35011]: from='osd.1 [v2:172.21.15.146:6800/1105990287,v1:172.21.15.146:6801/1105990287]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2024-09-12T02:51:19.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:19 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T02:51:19.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:19 smithi146 ceph-mon[35011]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2024-09-12T02:51:19.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:19 smithi146 ceph-mon[35011]: Detected new or changed devices on smithi146 2024-09-12T02:51:19.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:19 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:51:19.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:19 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:51:19.958 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:19 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:51:19.958 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:19 smithi146 ceph-mon[35011]: Adjusting osd_memory_target on smithi146 to 17088M 2024-09-12T02:51:19.958 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:19 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:51:19.958 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:19 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:51:19.958 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:19 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:51:19.958 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:19 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:51:19.958 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:19 smithi146 ceph-mon[35011]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]': finished 2024-09-12T02:51:20.125 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:19 smithi148 ceph-mon[35090]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-12T02:51:20.125 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:19 smithi148 ceph-mon[35090]: osdmap e11: 2 total, 1 up, 2 in 2024-09-12T02:51:20.125 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:19 smithi148 ceph-mon[35090]: from='osd.1 [v2:172.21.15.146:6800/1105990287,v1:172.21.15.146:6801/1105990287]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2024-09-12T02:51:20.126 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:19 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T02:51:20.126 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:19 smithi148 ceph-mon[35090]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2024-09-12T02:51:20.126 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:19 smithi148 ceph-mon[35090]: Detected new or changed devices on smithi146 2024-09-12T02:51:20.126 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:19 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:51:20.126 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:19 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:51:20.126 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:19 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:51:20.126 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:19 smithi148 ceph-mon[35090]: Adjusting osd_memory_target on smithi146 to 17088M 2024-09-12T02:51:20.126 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:19 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:51:20.126 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:19 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:51:20.126 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:19 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:51:20.126 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:19 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:51:20.127 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:19 smithi148 ceph-mon[35090]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]': finished 2024-09-12T02:51:20.137 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:51:20.134+0000 7f1df3d0c640 1 -- 172.21.15.148:0/3345657853 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1dec101c00 msgr2=0x7f1dec102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:51:20.137 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:51:20.134+0000 7f1df3d0c640 1 --2- 172.21.15.148:0/3345657853 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1dec101c00 0x7f1dec102000 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1de8009900 tx=0x7f1de802a410 comp rx=0 tx=0).stop 2024-09-12T02:51:20.137 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:51:20.134+0000 7f1df3d0c640 1 -- 172.21.15.148:0/3345657853 shutdown_connections 2024-09-12T02:51:20.137 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:51:20.134+0000 7f1df3d0c640 1 --2- 172.21.15.148:0/3345657853 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1dec102e00 0x7f1dec103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:20.137 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:51:20.134+0000 7f1df3d0c640 1 --2- 172.21.15.148:0/3345657853 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1dec1037c0 0x7f1dec10a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:20.137 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:51:20.134+0000 7f1df3d0c640 1 --2- 172.21.15.148:0/3345657853 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1dec101c00 0x7f1dec102000 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:20.137 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:51:20.134+0000 7f1df3d0c640 1 -- 172.21.15.148:0/3345657853 >> 172.21.15.148:0/3345657853 conn(0x7f1dec0fd390 msgr2=0x7f1dec0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:51:20.137 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:51:20.134+0000 7f1df3d0c640 1 -- 172.21.15.148:0/3345657853 shutdown_connections 2024-09-12T02:51:20.137 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:51:20.134+0000 7f1df3d0c640 1 -- 172.21.15.148:0/3345657853 wait complete. 2024-09-12T02:51:20.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:51:20.135+0000 7f1df3d0c640 1 Processor -- start 2024-09-12T02:51:20.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:51:20.135+0000 7f1df3d0c640 1 -- start start 2024-09-12T02:51:20.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:51:20.135+0000 7f1df3d0c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1dec101c00 0x7f1dec075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:51:20.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:51:20.135+0000 7f1df3d0c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1dec102e00 0x7f1dec0745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:51:20.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:51:20.135+0000 7f1df3d0c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1dec1037c0 0x7f1dec074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:51:20.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:51:20.135+0000 7f1df3d0c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1dec10d890 con 0x7f1dec1037c0 2024-09-12T02:51:20.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:51:20.135+0000 7f1df3d0c640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f1dec10d710 con 0x7f1dec101c00 2024-09-12T02:51:20.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:51:20.135+0000 7f1df3d0c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1dec10da10 con 0x7f1dec102e00 2024-09-12T02:51:20.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:51:20.135+0000 7f1df12a7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1dec102e00 0x7f1dec0745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:51:20.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:51:20.135+0000 7f1df12a7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1dec102e00 0x7f1dec0745e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:39846/0 (socket says 172.21.15.148:39846) 2024-09-12T02:51:20.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:51:20.135+0000 7f1df12a7640 1 -- 172.21.15.148:0/1366504584 learned_addr learned my addr 172.21.15.148:0/1366504584 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-09-12T02:51:20.138 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:51:20.135+0000 7f1df22a9640 1 --2- 172.21.15.148:0/1366504584 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1dec1037c0 0x7f1dec074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:51:20.139 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:51:20.136+0000 7f1df1aa8640 1 --2- 172.21.15.148:0/1366504584 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1dec101c00 0x7f1dec075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:51:20.139 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:51:20.136+0000 7f1df12a7640 1 -- 172.21.15.148:0/1366504584 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1dec101c00 msgr2=0x7f1dec075f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:51:20.139 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:51:20.136+0000 7f1df12a7640 1 --2- 172.21.15.148:0/1366504584 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1dec101c00 0x7f1dec075f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:20.139 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:51:20.136+0000 7f1df12a7640 1 -- 172.21.15.148:0/1366504584 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1dec1037c0 msgr2=0x7f1dec074b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:51:20.139 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:51:20.136+0000 7f1df12a7640 1 --2- 172.21.15.148:0/1366504584 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1dec1037c0 0x7f1dec074b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:20.139 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:51:20.136+0000 7f1df12a7640 1 -- 172.21.15.148:0/1366504584 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dec0753e0 con 0x7f1dec102e00 2024-09-12T02:51:20.139 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:51:20.136+0000 7f1df12a7640 1 --2- 172.21.15.148:0/1366504584 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1dec102e00 0x7f1dec0745e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1ddc00d950 tx=0x7f1ddc00de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:51:20.139 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:51:20.136+0000 7f1de2ffd640 1 -- 172.21.15.148:0/1366504584 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ddc017070 con 0x7f1dec102e00 2024-09-12T02:51:20.139 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:51:20.136+0000 7f1df3d0c640 1 -- 172.21.15.148:0/1366504584 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1dec1a2100 con 0x7f1dec102e00 2024-09-12T02:51:20.139 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:51:20.136+0000 7f1de2ffd640 1 -- 172.21.15.148:0/1366504584 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ddc009c30 con 0x7f1dec102e00 2024-09-12T02:51:20.139 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:51:20.136+0000 7f1de2ffd640 1 -- 172.21.15.148:0/1366504584 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ddc00f6b0 con 0x7f1dec102e00 2024-09-12T02:51:20.139 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:51:20.136+0000 7f1df3d0c640 1 -- 172.21.15.148:0/1366504584 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1dec1a2660 con 0x7f1dec102e00 2024-09-12T02:51:20.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:51:20.137+0000 7f1df3d0c640 1 -- 172.21.15.148:0/1366504584 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1dec109040 con 0x7f1dec102e00 2024-09-12T02:51:20.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:51:20.138+0000 7f1de2ffd640 1 -- 172.21.15.148:0/1366504584 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 15) ==== 137776+0+0 (secure 0 0 0) 0x7f1ddc0109d0 con 0x7f1dec102e00 2024-09-12T02:51:20.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:51:20.139+0000 7f1de2ffd640 1 --2- 172.21.15.148:0/1366504584 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f1dd80a2af0 0x7f1dd80a4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:51:20.140 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:51:20.139+0000 7f1de2ffd640 1 -- 172.21.15.148:0/1366504584 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2029+0+0 (secure 0 0 0) 0x7f1ddc0cd190 con 0x7f1dec102e00 2024-09-12T02:51:20.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:51:20.139+0000 7f1df1aa8640 1 --2- 172.21.15.148:0/1366504584 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f1dd80a2af0 0x7f1dd80a4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:51:20.141 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:51:20.140+0000 7f1df1aa8640 1 --2- 172.21.15.148:0/1366504584 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f1dd80a2af0 0x7f1dd80a4fb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1de8005ab0 tx=0x7f1de8000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:51:20.146 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:51:20.146+0000 7f1de2ffd640 1 -- 172.21.15.148:0/1366504584 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1ddc009de0 con 0x7f1dec102e00 2024-09-12T02:51:20.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:19 smithi005 ceph-mon[31197]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-12T02:51:20.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:19 smithi005 ceph-mon[31197]: osdmap e11: 2 total, 1 up, 2 in 2024-09-12T02:51:20.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:19 smithi005 ceph-mon[31197]: from='osd.1 [v2:172.21.15.146:6800/1105990287,v1:172.21.15.146:6801/1105990287]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2024-09-12T02:51:20.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:19 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T02:51:20.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:19 smithi005 ceph-mon[31197]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2024-09-12T02:51:20.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:19 smithi005 ceph-mon[31197]: Detected new or changed devices on smithi146 2024-09-12T02:51:20.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:19 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:51:20.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:19 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:51:20.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:19 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:51:20.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:19 smithi005 ceph-mon[31197]: Adjusting osd_memory_target on smithi146 to 17088M 2024-09-12T02:51:20.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:19 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:51:20.312 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:19 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:51:20.312 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:19 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:51:20.312 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:19 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:51:20.312 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:19 smithi005 ceph-mon[31197]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]': finished 2024-09-12T02:51:20.322 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:51:20.322+0000 7f1df3d0c640 1 -- 172.21.15.148:0/1366504584 --> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f1dec062630 con 0x7f1dd80a2af0 2024-09-12T02:51:21.002 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:20 smithi148 ceph-mon[35090]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:51:21.003 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:20 smithi148 ceph-mon[35090]: osdmap e12: 2 total, 1 up, 2 in 2024-09-12T02:51:21.003 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:20 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T02:51:21.003 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:20 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T02:51:21.003 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:20 smithi148 ceph-mon[35090]: from='client.24172 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:51:21.003 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:20 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:51:21.003 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:20 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:51:21.003 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:20 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:51:21.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:20 smithi146 ceph-mon[35011]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:51:21.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:20 smithi146 ceph-mon[35011]: osdmap e12: 2 total, 1 up, 2 in 2024-09-12T02:51:21.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:20 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T02:51:21.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:20 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T02:51:21.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:20 smithi146 ceph-mon[35011]: from='client.24172 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:51:21.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:20 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:51:21.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:20 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:51:21.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:20 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:51:21.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:20 smithi005 ceph-mon[31197]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:51:21.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:20 smithi005 ceph-mon[31197]: osdmap e12: 2 total, 1 up, 2 in 2024-09-12T02:51:21.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:20 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T02:51:21.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:20 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T02:51:21.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:20 smithi005 ceph-mon[31197]: from='client.24172 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:51:21.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:20 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:51:21.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:20 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:51:21.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:20 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:51:22.025 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:21 smithi148 ceph-mon[35090]: purged_snaps scrub starts 2024-09-12T02:51:22.025 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:21 smithi148 ceph-mon[35090]: purged_snaps scrub ok 2024-09-12T02:51:22.025 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:21 smithi148 ceph-mon[35090]: osd.1 [v2:172.21.15.146:6800/1105990287,v1:172.21.15.146:6801/1105990287] boot 2024-09-12T02:51:22.025 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:21 smithi148 ceph-mon[35090]: osdmap e13: 2 total, 2 up, 2 in 2024-09-12T02:51:22.025 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:21 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T02:51:22.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:21 smithi146 ceph-mon[35011]: purged_snaps scrub starts 2024-09-12T02:51:22.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:21 smithi146 ceph-mon[35011]: purged_snaps scrub ok 2024-09-12T02:51:22.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:21 smithi146 ceph-mon[35011]: osd.1 [v2:172.21.15.146:6800/1105990287,v1:172.21.15.146:6801/1105990287] boot 2024-09-12T02:51:22.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:21 smithi146 ceph-mon[35011]: osdmap e13: 2 total, 2 up, 2 in 2024-09-12T02:51:22.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:21 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T02:51:22.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:21 smithi005 ceph-mon[31197]: purged_snaps scrub starts 2024-09-12T02:51:22.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:21 smithi005 ceph-mon[31197]: purged_snaps scrub ok 2024-09-12T02:51:22.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:21 smithi005 ceph-mon[31197]: osd.1 [v2:172.21.15.146:6800/1105990287,v1:172.21.15.146:6801/1105990287] boot 2024-09-12T02:51:22.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:21 smithi005 ceph-mon[31197]: osdmap e13: 2 total, 2 up, 2 in 2024-09-12T02:51:22.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:21 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-12T02:51:23.071 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:22 smithi148 ceph-mon[35090]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:51:23.071 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:22 smithi148 ceph-mon[35090]: osdmap e14: 2 total, 2 up, 2 in 2024-09-12T02:51:23.071 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:22 smithi148 ceph-mon[35090]: from='client.? 172.21.15.148:0/1699607800' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ea3ab84a-7739-4975-8244-31ddd5aad80d"}]: dispatch 2024-09-12T02:51:23.071 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:22 smithi148 ceph-mon[35090]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ea3ab84a-7739-4975-8244-31ddd5aad80d"}]: dispatch 2024-09-12T02:51:23.072 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:22 smithi148 ceph-mon[35090]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ea3ab84a-7739-4975-8244-31ddd5aad80d"}]': finished 2024-09-12T02:51:23.072 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:22 smithi148 ceph-mon[35090]: osdmap e15: 3 total, 2 up, 3 in 2024-09-12T02:51:23.072 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:22 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T02:51:23.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:22 smithi146 ceph-mon[35011]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:51:23.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:22 smithi146 ceph-mon[35011]: osdmap e14: 2 total, 2 up, 2 in 2024-09-12T02:51:23.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:22 smithi146 ceph-mon[35011]: from='client.? 172.21.15.148:0/1699607800' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ea3ab84a-7739-4975-8244-31ddd5aad80d"}]: dispatch 2024-09-12T02:51:23.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:22 smithi146 ceph-mon[35011]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ea3ab84a-7739-4975-8244-31ddd5aad80d"}]: dispatch 2024-09-12T02:51:23.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:22 smithi146 ceph-mon[35011]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ea3ab84a-7739-4975-8244-31ddd5aad80d"}]': finished 2024-09-12T02:51:23.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:22 smithi146 ceph-mon[35011]: osdmap e15: 3 total, 2 up, 3 in 2024-09-12T02:51:23.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:22 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T02:51:23.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:22 smithi005 ceph-mon[31197]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-12T02:51:23.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:22 smithi005 ceph-mon[31197]: osdmap e14: 2 total, 2 up, 2 in 2024-09-12T02:51:23.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:22 smithi005 ceph-mon[31197]: from='client.? 172.21.15.148:0/1699607800' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ea3ab84a-7739-4975-8244-31ddd5aad80d"}]: dispatch 2024-09-12T02:51:23.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:22 smithi005 ceph-mon[31197]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ea3ab84a-7739-4975-8244-31ddd5aad80d"}]: dispatch 2024-09-12T02:51:23.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:22 smithi005 ceph-mon[31197]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ea3ab84a-7739-4975-8244-31ddd5aad80d"}]': finished 2024-09-12T02:51:23.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:22 smithi005 ceph-mon[31197]: osdmap e15: 3 total, 2 up, 3 in 2024-09-12T02:51:23.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:22 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T02:51:24.165 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:23 smithi148 ceph-mon[35090]: from='client.? 172.21.15.148:0/3401460891' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:51:24.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:23 smithi146 ceph-mon[35011]: from='client.? 172.21.15.148:0/3401460891' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:51:24.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:23 smithi005 ceph-mon[31197]: from='client.? 172.21.15.148:0/3401460891' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:51:25.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:24 smithi148 ceph-mon[35090]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:51:25.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:24 smithi146 ceph-mon[35011]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:51:25.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:24 smithi005 ceph-mon[31197]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:51:27.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:26 smithi148 ceph-mon[35090]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:51:27.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:26 smithi146 ceph-mon[35011]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:51:27.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:26 smithi005 ceph-mon[31197]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:51:29.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:28 smithi148 ceph-mon[35090]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:51:29.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:28 smithi146 ceph-mon[35011]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:51:29.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:28 smithi005 ceph-mon[31197]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:51:31.167 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:30 smithi148 ceph-mon[35090]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:51:31.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:30 smithi146 ceph-mon[35011]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:51:31.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:30 smithi005 ceph-mon[31197]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:51:32.170 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:31 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-12T02:51:32.171 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:31 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:51:32.171 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:31 smithi148 ceph-mon[35090]: Deploying daemon osd.2 on smithi148 2024-09-12T02:51:32.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:31 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-12T02:51:32.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:31 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:51:32.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:31 smithi146 ceph-mon[35011]: Deploying daemon osd.2 on smithi148 2024-09-12T02:51:32.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:31 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-12T02:51:32.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:31 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:51:32.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:31 smithi005 ceph-mon[31197]: Deploying daemon osd.2 on smithi148 2024-09-12T02:51:33.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:32 smithi148 ceph-mon[35090]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:51:33.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:32 smithi146 ceph-mon[35011]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:51:33.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:32 smithi005 ceph-mon[31197]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:51:35.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:34 smithi148 ceph-mon[35090]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:51:35.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:34 smithi146 ceph-mon[35011]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:51:35.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:34 smithi005 ceph-mon[31197]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:51:37.130 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:36 smithi005 ceph-mon[31197]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:51:37.165 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:36 smithi148 ceph-mon[35090]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:51:37.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:36 smithi146 ceph-mon[35011]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:51:38.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:37 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:51:38.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:37 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:51:38.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:37 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:51:38.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:37 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:51:38.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:37 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:51:38.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:37 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:51:38.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:37 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:51:38.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:37 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:51:38.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:37 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:51:39.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:39 smithi148 ceph-mon[35090]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:51:39.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:39 smithi005 ceph-mon[31197]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:51:39.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:39 smithi146 ceph-mon[35011]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:51:40.869 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:40 smithi148 ceph-mon[35090]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:51:40.869 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:40 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:51:40.869 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:40 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:51:40.869 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:40 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:51:40.869 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:40 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:51:40.869 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:40 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:51:40.956 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:40 smithi146 ceph-mon[35011]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:51:40.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:40 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:51:40.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:40 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:51:40.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:40 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:51:40.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:40 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:51:40.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:40 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:51:41.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:40 smithi005 ceph-mon[31197]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:51:41.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:40 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:51:41.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:40 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:51:41.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:40 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:51:41.061 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:40 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:51:41.061 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:40 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:51:41.121 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:51:41.120+0000 7f1de2ffd640 1 -- 172.21.15.148:0/1366504584 <== mgr.14148 v2:172.21.15.5:6800/3780631642 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f1dec062630 con 0x7f1dd80a2af0 2024-09-12T02:51:41.122 INFO:teuthology.orchestra.run.smithi148.stdout:Created osd(s) 2 on host 'smithi148' 2024-09-12T02:51:41.124 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:51:41.123+0000 7f1df3d0c640 1 -- 172.21.15.148:0/1366504584 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f1dd80a2af0 msgr2=0x7f1dd80a4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:51:41.124 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:51:41.123+0000 7f1df3d0c640 1 --2- 172.21.15.148:0/1366504584 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f1dd80a2af0 0x7f1dd80a4fb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1de8005ab0 tx=0x7f1de8000bc0 comp rx=0 tx=0).stop 2024-09-12T02:51:41.124 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:51:41.123+0000 7f1df3d0c640 1 -- 172.21.15.148:0/1366504584 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1dec102e00 msgr2=0x7f1dec0745e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:51:41.124 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:51:41.123+0000 7f1df3d0c640 1 --2- 172.21.15.148:0/1366504584 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1dec102e00 0x7f1dec0745e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1ddc00d950 tx=0x7f1ddc00de10 comp rx=0 tx=0).stop 2024-09-12T02:51:41.124 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:51:41.123+0000 7f1df3d0c640 1 -- 172.21.15.148:0/1366504584 shutdown_connections 2024-09-12T02:51:41.124 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:51:41.123+0000 7f1df3d0c640 1 --2- 172.21.15.148:0/1366504584 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f1dd80a2af0 0x7f1dd80a4fb0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:41.125 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:51:41.123+0000 7f1df3d0c640 1 --2- 172.21.15.148:0/1366504584 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1dec1037c0 0x7f1dec074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:41.125 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:51:41.123+0000 7f1df3d0c640 1 --2- 172.21.15.148:0/1366504584 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1dec102e00 0x7f1dec0745e0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:41.125 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:51:41.123+0000 7f1df3d0c640 1 --2- 172.21.15.148:0/1366504584 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1dec101c00 0x7f1dec075f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:41.125 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:51:41.123+0000 7f1df3d0c640 1 -- 172.21.15.148:0/1366504584 >> 172.21.15.148:0/1366504584 conn(0x7f1dec0fd390 msgr2=0x7f1dec106210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:51:41.125 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:51:41.123+0000 7f1df3d0c640 1 -- 172.21.15.148:0/1366504584 shutdown_connections 2024-09-12T02:51:41.125 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:51:41.123+0000 7f1df3d0c640 1 -- 172.21.15.148:0/1366504584 wait complete. 2024-09-12T02:51:41.688 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:41 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:51:41.688 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:41 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:51:41.688 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:41 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:51:41.713 DEBUG:teuthology.orchestra.run.smithi148:osd.2> sudo journalctl -f -n 0 -u ceph-618305f8-70b1-11ef-bcea-c7b262605968@osd.2.service 2024-09-12T02:51:41.715 INFO:tasks.cephadm:Deploying osd.3 on smithi005 with /dev/vg_nvme/lv_3... 2024-09-12T02:51:41.715 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-12T02:51:41.956 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:41 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:51:41.956 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:41 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:51:41.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:41 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:51:41.987 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:51:42.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:41 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:51:42.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:41 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:51:42.061 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:41 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:51:43.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:43 smithi005 ceph-mon[31197]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:51:43.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:43 smithi148 ceph-mon[35090]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:51:43.416 INFO:journalctl@ceph.osd.2.smithi148.stdout:Sep 12 02:51:43 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-2[40096]: 2024-09-12T02:51:43.140+0000 7fb2a71a8640 -1 osd.2 0 log_to_monitors true 2024-09-12T02:51:43.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:43 smithi146 ceph-mon[35011]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:51:44.388 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:44 smithi005 ceph-mon[31197]: from='osd.2 [v2:172.21.15.148:6800/3378174313,v1:172.21.15.148:6801/3378174313]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-12T02:51:44.389 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:44 smithi005 ceph-mon[31197]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-12T02:51:44.392 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-12T02:51:44.415 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:44 smithi148 ceph-mon[35090]: from='osd.2 [v2:172.21.15.148:6800/3378174313,v1:172.21.15.148:6801/3378174313]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-12T02:51:44.415 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:44 smithi148 ceph-mon[35090]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-12T02:51:44.419 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi005:vg_nvme/lv_3 2024-09-12T02:51:44.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:44 smithi146 ceph-mon[35011]: from='osd.2 [v2:172.21.15.148:6800/3378174313,v1:172.21.15.148:6801/3378174313]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-12T02:51:44.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:44 smithi146 ceph-mon[35011]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-12T02:51:44.683 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:51:45.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:51:45.078+0000 7f5056aeb640 1 -- 172.21.15.5:0/2591929163 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50501026a0 msgr2=0x7f5050104a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:51:45.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:51:45.078+0000 7f5056aeb640 1 --2- 172.21.15.5:0/2591929163 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50501026a0 0x7f5050104a90 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5044009900 tx=0x7f504402a3f0 comp rx=0 tx=0).stop 2024-09-12T02:51:45.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:51:45.078+0000 7f5056aeb640 1 -- 172.21.15.5:0/2591929163 shutdown_connections 2024-09-12T02:51:45.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:51:45.078+0000 7f5056aeb640 1 --2- 172.21.15.5:0/2591929163 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5050107900 0x7f5050109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:45.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:51:45.078+0000 7f5056aeb640 1 --2- 172.21.15.5:0/2591929163 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5050104fd0 0x7f50501073c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:45.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:51:45.078+0000 7f5056aeb640 1 --2- 172.21.15.5:0/2591929163 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50501026a0 0x7f5050104a90 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:45.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:51:45.078+0000 7f5056aeb640 1 -- 172.21.15.5:0/2591929163 >> 172.21.15.5:0/2591929163 conn(0x7f50500fc2a0 msgr2=0x7f50500fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:51:45.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:51:45.078+0000 7f5056aeb640 1 -- 172.21.15.5:0/2591929163 shutdown_connections 2024-09-12T02:51:45.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:51:45.078+0000 7f5056aeb640 1 -- 172.21.15.5:0/2591929163 wait complete. 2024-09-12T02:51:45.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:51:45.079+0000 7f5056aeb640 1 Processor -- start 2024-09-12T02:51:45.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:51:45.079+0000 7f5056aeb640 1 -- start start 2024-09-12T02:51:45.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:51:45.079+0000 7f5056aeb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50501026a0 0x7f5050102000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:51:45.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:51:45.079+0000 7f5056aeb640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5050104fd0 0x7f5050100650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:51:45.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:51:45.079+0000 7f5056aeb640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5050107900 0x7f5050100b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:51:45.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:51:45.079+0000 7f5056aeb640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f505010d8b0 con 0x7f50501026a0 2024-09-12T02:51:45.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:51:45.079+0000 7f5056aeb640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f505010d730 con 0x7f5050104fd0 2024-09-12T02:51:45.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:51:45.079+0000 7f5056aeb640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f505010da30 con 0x7f5050107900 2024-09-12T02:51:45.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:51:45.079+0000 7f5054887640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50501026a0 0x7f5050102000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:51:45.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:51:45.079+0000 7f5054887640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50501026a0 0x7f5050102000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35052/0 (socket says 172.21.15.5:35052) 2024-09-12T02:51:45.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:51:45.079+0000 7f5054887640 1 -- 172.21.15.5:0/3777991906 learned_addr learned my addr 172.21.15.5:0/3777991906 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:51:45.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:51:45.079+0000 7f5055088640 1 --2- 172.21.15.5:0/3777991906 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5050107900 0x7f5050100b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:51:45.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:51:45.079+0000 7f504ffff640 1 --2- 172.21.15.5:0/3777991906 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5050104fd0 0x7f5050100650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:51:45.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:51:45.080+0000 7f5054887640 1 -- 172.21.15.5:0/3777991906 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5050107900 msgr2=0x7f5050100b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:51:45.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:51:45.080+0000 7f5054887640 1 --2- 172.21.15.5:0/3777991906 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5050107900 0x7f5050100b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:45.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:51:45.080+0000 7f5054887640 1 -- 172.21.15.5:0/3777991906 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5050104fd0 msgr2=0x7f5050100650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:51:45.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:51:45.080+0000 7f5054887640 1 --2- 172.21.15.5:0/3777991906 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5050104fd0 0x7f5050100650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:51:45.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:51:45.080+0000 7f5054887640 1 -- 172.21.15.5:0/3777991906 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5050101450 con 0x7f50501026a0 2024-09-12T02:51:45.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:51:45.080+0000 7f5054887640 1 --2- 172.21.15.5:0/3777991906 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50501026a0 0x7f5050102000 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f5044009230 tx=0x7f5044004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:51:45.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:51:45.080+0000 7f504dffb640 1 -- 172.21.15.5:0/3777991906 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f504403f030 con 0x7f50501026a0 2024-09-12T02:51:45.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:51:45.080+0000 7f5056aeb640 1 -- 172.21.15.5:0/3777991906 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50501a2010 con 0x7f50501026a0 2024-09-12T02:51:45.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:51:45.080+0000 7f504dffb640 1 -- 172.21.15.5:0/3777991906 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5044003ce0 con 0x7f50501026a0 2024-09-12T02:51:45.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:51:45.080+0000 7f504dffb640 1 -- 172.21.15.5:0/3777991906 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5044035070 con 0x7f50501026a0 2024-09-12T02:51:45.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:51:45.080+0000 7f5056aeb640 1 -- 172.21.15.5:0/3777991906 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50501a2510 con 0x7f50501026a0 2024-09-12T02:51:45.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:51:45.081+0000 7f5056aeb640 1 -- 172.21.15.5:0/3777991906 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50500fdd90 con 0x7f50501026a0 2024-09-12T02:51:45.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:51:45.082+0000 7f504dffb640 1 -- 172.21.15.5:0/3777991906 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 15) ==== 137776+0+0 (secure 0 0 0) 0x7f5044042030 con 0x7f50501026a0 2024-09-12T02:51:45.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:51:45.083+0000 7f504dffb640 1 --2- 172.21.15.5:0/3777991906 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f50400a2af0 0x7f50400a4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:51:45.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:51:45.083+0000 7f504ffff640 1 --2- 172.21.15.5:0/3777991906 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f50400a2af0 0x7f50400a4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:51:45.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:51:45.083+0000 7f504dffb640 1 -- 172.21.15.5:0/3777991906 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 2445+0+0 (secure 0 0 0) 0x7f50440ee120 con 0x7f50501026a0 2024-09-12T02:51:45.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:51:45.084+0000 7f504ffff640 1 --2- 172.21.15.5:0/3777991906 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f50400a2af0 0x7f50400a4fb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5050101d80 tx=0x7f503c00b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:51:45.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:51:45.087+0000 7f504dffb640 1 -- 172.21.15.5:0/3777991906 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f50440bc200 con 0x7f50501026a0 2024-09-12T02:51:45.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:51:45.274+0000 7f5056aeb640 1 -- 172.21.15.5:0/3777991906 --> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f5050062630 con 0x7f50400a2af0 2024-09-12T02:51:45.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:45 smithi148 ceph-mon[35090]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:51:45.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:45 smithi148 ceph-mon[35090]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-12T02:51:45.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:45 smithi148 ceph-mon[35090]: osdmap e16: 3 total, 2 up, 3 in 2024-09-12T02:51:45.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:45 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T02:51:45.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:45 smithi148 ceph-mon[35090]: from='osd.2 [v2:172.21.15.148:6800/3378174313,v1:172.21.15.148:6801/3378174313]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-09-12T02:51:45.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:45 smithi148 ceph-mon[35090]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-09-12T02:51:45.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:45 smithi148 ceph-mon[35090]: Detected new or changed devices on smithi148 2024-09-12T02:51:45.417 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:45 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:51:45.417 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:45 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:51:45.417 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:45 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:51:45.417 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:45 smithi148 ceph-mon[35090]: Adjusting osd_memory_target on smithi148 to 17088M 2024-09-12T02:51:45.417 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:45 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:51:45.417 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:45 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:51:45.417 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:45 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:51:45.417 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:45 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:51:45.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:45 smithi146 ceph-mon[35011]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:51:45.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:45 smithi146 ceph-mon[35011]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-12T02:51:45.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:45 smithi146 ceph-mon[35011]: osdmap e16: 3 total, 2 up, 3 in 2024-09-12T02:51:45.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:45 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T02:51:45.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:45 smithi146 ceph-mon[35011]: from='osd.2 [v2:172.21.15.148:6800/3378174313,v1:172.21.15.148:6801/3378174313]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-09-12T02:51:45.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:45 smithi146 ceph-mon[35011]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-09-12T02:51:45.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:45 smithi146 ceph-mon[35011]: Detected new or changed devices on smithi148 2024-09-12T02:51:45.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:45 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:51:45.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:45 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:51:45.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:45 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:51:45.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:45 smithi146 ceph-mon[35011]: Adjusting osd_memory_target on smithi148 to 17088M 2024-09-12T02:51:45.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:45 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:51:45.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:45 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:51:45.458 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:45 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:51:45.458 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:45 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:51:45.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:45 smithi005 ceph-mon[31197]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:51:45.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:45 smithi005 ceph-mon[31197]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-12T02:51:45.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:45 smithi005 ceph-mon[31197]: osdmap e16: 3 total, 2 up, 3 in 2024-09-12T02:51:45.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:45 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T02:51:45.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:45 smithi005 ceph-mon[31197]: from='osd.2 [v2:172.21.15.148:6800/3378174313,v1:172.21.15.148:6801/3378174313]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-09-12T02:51:45.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:45 smithi005 ceph-mon[31197]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-09-12T02:51:45.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:45 smithi005 ceph-mon[31197]: Detected new or changed devices on smithi148 2024-09-12T02:51:45.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:45 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:51:45.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:45 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:51:45.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:45 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:51:45.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:45 smithi005 ceph-mon[31197]: Adjusting osd_memory_target on smithi148 to 17088M 2024-09-12T02:51:45.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:45 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:51:45.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:45 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:51:45.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:45 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:51:45.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:45 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:51:46.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:46 smithi005 ceph-mon[31197]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2024-09-12T02:51:46.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:46 smithi005 ceph-mon[31197]: osdmap e17: 3 total, 2 up, 3 in 2024-09-12T02:51:46.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:46 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T02:51:46.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:46 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T02:51:46.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:46 smithi005 ceph-mon[31197]: from='client.14277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:51:46.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:46 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:51:46.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:46 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:51:46.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:46 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:51:46.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:46 smithi005 ceph-mon[31197]: from='osd.2 ' entity='osd.2' 2024-09-12T02:51:46.416 INFO:journalctl@ceph.osd.2.smithi148.stdout:Sep 12 02:51:45 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-2[40096]: 2024-09-12T02:51:45.962+0000 7fb2a2908640 -1 osd.2 0 waiting for initial osdmap 2024-09-12T02:51:46.416 INFO:journalctl@ceph.osd.2.smithi148.stdout:Sep 12 02:51:45 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-2[40096]: 2024-09-12T02:51:45.969+0000 7fb29af3a640 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T02:51:46.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:46 smithi148 ceph-mon[35090]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2024-09-12T02:51:46.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:46 smithi148 ceph-mon[35090]: osdmap e17: 3 total, 2 up, 3 in 2024-09-12T02:51:46.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:46 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T02:51:46.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:46 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T02:51:46.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:46 smithi148 ceph-mon[35090]: from='client.14277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:51:46.417 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:46 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:51:46.417 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:46 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:51:46.417 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:46 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:51:46.417 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:46 smithi148 ceph-mon[35090]: from='osd.2 ' entity='osd.2' 2024-09-12T02:51:46.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:46 smithi146 ceph-mon[35011]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2024-09-12T02:51:46.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:46 smithi146 ceph-mon[35011]: osdmap e17: 3 total, 2 up, 3 in 2024-09-12T02:51:46.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:46 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T02:51:46.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:46 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T02:51:46.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:46 smithi146 ceph-mon[35011]: from='client.14277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:51:46.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:46 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:51:46.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:46 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:51:46.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:46 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:51:46.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:46 smithi146 ceph-mon[35011]: from='osd.2 ' entity='osd.2' 2024-09-12T02:51:47.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:47 smithi005 ceph-mon[31197]: purged_snaps scrub starts 2024-09-12T02:51:47.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:47 smithi005 ceph-mon[31197]: purged_snaps scrub ok 2024-09-12T02:51:47.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:47 smithi005 ceph-mon[31197]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:51:47.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:47 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T02:51:47.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:47 smithi005 ceph-mon[31197]: osd.2 [v2:172.21.15.148:6800/3378174313,v1:172.21.15.148:6801/3378174313] boot 2024-09-12T02:51:47.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:47 smithi005 ceph-mon[31197]: osdmap e18: 3 total, 3 up, 3 in 2024-09-12T02:51:47.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:47 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T02:51:47.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:47 smithi148 ceph-mon[35090]: purged_snaps scrub starts 2024-09-12T02:51:47.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:47 smithi148 ceph-mon[35090]: purged_snaps scrub ok 2024-09-12T02:51:47.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:47 smithi148 ceph-mon[35090]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:51:47.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:47 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T02:51:47.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:47 smithi148 ceph-mon[35090]: osd.2 [v2:172.21.15.148:6800/3378174313,v1:172.21.15.148:6801/3378174313] boot 2024-09-12T02:51:47.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:47 smithi148 ceph-mon[35090]: osdmap e18: 3 total, 3 up, 3 in 2024-09-12T02:51:47.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:47 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T02:51:47.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:47 smithi146 ceph-mon[35011]: purged_snaps scrub starts 2024-09-12T02:51:47.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:47 smithi146 ceph-mon[35011]: purged_snaps scrub ok 2024-09-12T02:51:47.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:47 smithi146 ceph-mon[35011]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-12T02:51:47.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:47 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T02:51:47.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:47 smithi146 ceph-mon[35011]: osd.2 [v2:172.21.15.148:6800/3378174313,v1:172.21.15.148:6801/3378174313] boot 2024-09-12T02:51:47.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:47 smithi146 ceph-mon[35011]: osdmap e18: 3 total, 3 up, 3 in 2024-09-12T02:51:47.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:47 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-12T02:51:48.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:48 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/3930084648' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4396a9ee-f3f3-410b-9fb1-124f1e33a1f1"}]: dispatch 2024-09-12T02:51:48.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:48 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/3930084648' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4396a9ee-f3f3-410b-9fb1-124f1e33a1f1"}]': finished 2024-09-12T02:51:48.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:48 smithi148 ceph-mon[35090]: osdmap e19: 4 total, 3 up, 4 in 2024-09-12T02:51:48.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:48 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T02:51:48.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:48 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-12T02:51:48.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:48 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/4046431632' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:51:48.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:48 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/3930084648' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4396a9ee-f3f3-410b-9fb1-124f1e33a1f1"}]: dispatch 2024-09-12T02:51:48.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:48 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/3930084648' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4396a9ee-f3f3-410b-9fb1-124f1e33a1f1"}]': finished 2024-09-12T02:51:48.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:48 smithi146 ceph-mon[35011]: osdmap e19: 4 total, 3 up, 4 in 2024-09-12T02:51:48.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:48 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T02:51:48.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:48 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-12T02:51:48.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:48 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/4046431632' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:51:48.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:48 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/3930084648' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4396a9ee-f3f3-410b-9fb1-124f1e33a1f1"}]: dispatch 2024-09-12T02:51:48.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:48 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/3930084648' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4396a9ee-f3f3-410b-9fb1-124f1e33a1f1"}]': finished 2024-09-12T02:51:48.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:48 smithi005 ceph-mon[31197]: osdmap e19: 4 total, 3 up, 4 in 2024-09-12T02:51:48.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:48 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T02:51:48.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:48 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-12T02:51:48.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:48 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/4046431632' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:51:49.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:49 smithi148 ceph-mon[35090]: pgmap v77: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-12T02:51:49.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:49 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-12T02:51:49.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:49 smithi148 ceph-mon[35090]: osdmap e20: 4 total, 3 up, 4 in 2024-09-12T02:51:49.417 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:49 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T02:51:49.417 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:49 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-12T02:51:49.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:49 smithi146 ceph-mon[35011]: pgmap v77: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-12T02:51:49.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:49 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-12T02:51:49.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:49 smithi146 ceph-mon[35011]: osdmap e20: 4 total, 3 up, 4 in 2024-09-12T02:51:49.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:49 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T02:51:49.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:49 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-12T02:51:49.562 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:49 smithi005 ceph-mon[31197]: pgmap v77: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-12T02:51:49.563 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:49 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-12T02:51:49.563 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:49 smithi005 ceph-mon[31197]: osdmap e20: 4 total, 3 up, 4 in 2024-09-12T02:51:49.563 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:49 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T02:51:49.563 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:49 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-12T02:51:50.521 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 12 02:51:50 smithi005 sudo[47956]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-12T02:51:50.521 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 12 02:51:50 smithi005 sudo[47956]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T02:51:50.521 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 12 02:51:50 smithi005 sudo[47956]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T02:51:50.521 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 12 02:51:50 smithi005 sudo[47956]: pam_unix(sudo:session): session closed for user root 2024-09-12T02:51:50.792 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:50 smithi146 ceph-mon[35011]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-12T02:51:50.792 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:50 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-12T02:51:50.792 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:50 smithi146 ceph-mon[35011]: osdmap e21: 4 total, 3 up, 4 in 2024-09-12T02:51:50.793 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:50 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T02:51:50.793 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:50 smithi146 ceph-mon[35011]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-12T02:51:50.810 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 12 02:51:50 smithi005 sudo[48148]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-12T02:51:50.810 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 12 02:51:50 smithi005 sudo[48148]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T02:51:50.810 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 12 02:51:50 smithi005 sudo[48148]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T02:51:50.810 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 12 02:51:50 smithi005 sudo[48148]: pam_unix(sudo:session): session closed for user root 2024-09-12T02:51:50.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:50 smithi005 ceph-mon[31197]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-12T02:51:50.811 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:50 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-12T02:51:50.811 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:50 smithi005 ceph-mon[31197]: osdmap e21: 4 total, 3 up, 4 in 2024-09-12T02:51:50.811 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:50 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T02:51:50.811 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:50 smithi005 ceph-mon[31197]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-12T02:51:50.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:50 smithi148 ceph-mon[35090]: pgmap v80: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-12T02:51:50.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:50 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-12T02:51:50.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:50 smithi148 ceph-mon[35090]: osdmap e21: 4 total, 3 up, 4 in 2024-09-12T02:51:50.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:50 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T02:51:50.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:50 smithi148 ceph-mon[35090]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-12T02:51:51.045 INFO:journalctl@ceph.osd.1.smithi146.stdout:Sep 12 02:51:50 smithi146 sudo[43382]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-12T02:51:51.045 INFO:journalctl@ceph.osd.1.smithi146.stdout:Sep 12 02:51:50 smithi146 sudo[43382]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T02:51:51.046 INFO:journalctl@ceph.osd.1.smithi146.stdout:Sep 12 02:51:50 smithi146 sudo[43382]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T02:51:51.046 INFO:journalctl@ceph.osd.1.smithi146.stdout:Sep 12 02:51:50 smithi146 sudo[43382]: pam_unix(sudo:session): session closed for user root 2024-09-12T02:51:51.456 INFO:journalctl@ceph.osd.1.smithi146.stdout:Sep 12 02:51:51 smithi146 sudo[43385]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-12T02:51:51.457 INFO:journalctl@ceph.osd.1.smithi146.stdout:Sep 12 02:51:51 smithi146 sudo[43385]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T02:51:51.457 INFO:journalctl@ceph.osd.1.smithi146.stdout:Sep 12 02:51:51 smithi146 sudo[43385]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T02:51:51.457 INFO:journalctl@ceph.osd.1.smithi146.stdout:Sep 12 02:51:51 smithi146 sudo[43385]: pam_unix(sudo:session): session closed for user root 2024-09-12T02:51:51.571 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:51 smithi148 ceph-mon[35090]: osdmap e22: 4 total, 3 up, 4 in 2024-09-12T02:51:51.572 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:51 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T02:51:51.572 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:51 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:51:51.572 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:51 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:51:51.572 INFO:journalctl@ceph.osd.2.smithi148.stdout:Sep 12 02:51:51 smithi148 sudo[43924]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-12T02:51:51.572 INFO:journalctl@ceph.osd.2.smithi148.stdout:Sep 12 02:51:51 smithi148 sudo[43924]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T02:51:51.572 INFO:journalctl@ceph.osd.2.smithi148.stdout:Sep 12 02:51:51 smithi148 sudo[43924]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T02:51:51.572 INFO:journalctl@ceph.osd.2.smithi148.stdout:Sep 12 02:51:51 smithi148 sudo[43924]: pam_unix(sudo:session): session closed for user root 2024-09-12T02:51:51.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:51 smithi005 ceph-mon[31197]: osdmap e22: 4 total, 3 up, 4 in 2024-09-12T02:51:51.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:51 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T02:51:51.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:51 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:51:51.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:51 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:51:51.916 INFO:journalctl@ceph.osd.2.smithi148.stdout:Sep 12 02:51:51 smithi148 sudo[43927]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-12T02:51:51.916 INFO:journalctl@ceph.osd.2.smithi148.stdout:Sep 12 02:51:51 smithi148 sudo[43927]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T02:51:51.916 INFO:journalctl@ceph.osd.2.smithi148.stdout:Sep 12 02:51:51 smithi148 sudo[43927]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T02:51:51.916 INFO:journalctl@ceph.osd.2.smithi148.stdout:Sep 12 02:51:51 smithi148 sudo[43927]: pam_unix(sudo:session): session closed for user root 2024-09-12T02:51:51.956 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:51 smithi146 ceph-mon[35011]: osdmap e22: 4 total, 3 up, 4 in 2024-09-12T02:51:51.956 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:51 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T02:51:51.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:51 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:51:51.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:51 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:51:52.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:52 smithi005 sudo[48712]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-12T02:51:52.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:52 smithi005 sudo[48712]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T02:51:52.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:52 smithi005 sudo[48712]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T02:51:52.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:52 smithi005 sudo[48712]: pam_unix(sudo:session): session closed for user root 2024-09-12T02:51:52.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:51:52.512+0000 7f504dffb640 1 -- 172.21.15.5:0/3777991906 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f50440b8870 con 0x7f50501026a0 2024-09-12T02:51:52.651 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:52 smithi005 sudo[48805]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-12T02:51:52.651 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:52 smithi005 sudo[48805]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T02:51:52.651 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:52 smithi005 sudo[48805]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T02:51:52.651 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:52 smithi005 sudo[48805]: pam_unix(sudo:session): session closed for user root 2024-09-12T02:51:52.997 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:52 smithi148 sudo[43930]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-12T02:51:52.998 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:52 smithi148 sudo[43930]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T02:51:52.998 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:52 smithi148 sudo[43930]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T02:51:52.998 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:52 smithi148 sudo[43930]: pam_unix(sudo:session): session closed for user root 2024-09-12T02:51:53.242 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:53 smithi005 ceph-mon[31197]: pgmap v83: 1 pgs: 1 unknown; 0 B data, 85 MiB used, 268 GiB / 268 GiB avail 2024-09-12T02:51:53.242 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:53 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:51:53.242 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:53 smithi005 ceph-mon[31197]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T02:51:53.242 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:53 smithi005 ceph-mon[31197]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T02:51:53.242 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:53 smithi005 ceph-mon[31197]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-12T02:51:53.242 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:53 smithi005 ceph-mon[31197]: Cluster is now healthy 2024-09-12T02:51:53.243 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:53 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T02:51:53.243 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:53 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:51:53.243 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:53 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:51:53.243 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:53 smithi005 ceph-mon[31197]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-12T02:51:53.243 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:53 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T02:51:53.243 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:53 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:51:53.243 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:53 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:51:53.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:52 smithi148 sudo[43933]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-12T02:51:53.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:52 smithi148 sudo[43933]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T02:51:53.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:52 smithi148 sudo[43933]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T02:51:53.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:53 smithi148 sudo[43933]: pam_unix(sudo:session): session closed for user root 2024-09-12T02:51:53.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:53 smithi148 ceph-mon[35090]: pgmap v83: 1 pgs: 1 unknown; 0 B data, 85 MiB used, 268 GiB / 268 GiB avail 2024-09-12T02:51:53.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:53 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:51:53.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:53 smithi148 ceph-mon[35090]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T02:51:53.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:53 smithi148 ceph-mon[35090]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T02:51:53.417 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:53 smithi148 ceph-mon[35090]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-12T02:51:53.417 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:53 smithi148 ceph-mon[35090]: Cluster is now healthy 2024-09-12T02:51:53.417 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:53 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T02:51:53.417 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:53 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:51:53.417 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:53 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:51:53.417 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:53 smithi148 ceph-mon[35090]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-12T02:51:53.417 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:53 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T02:51:53.417 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:53 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:51:53.417 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:53 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:51:53.497 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:53 smithi146 sudo[43388]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-12T02:51:53.497 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:53 smithi146 sudo[43388]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T02:51:53.497 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:53 smithi146 sudo[43388]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T02:51:53.497 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:53 smithi146 sudo[43388]: pam_unix(sudo:session): session closed for user root 2024-09-12T02:51:53.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:53 smithi146 sudo[43391]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-12T02:51:53.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:53 smithi146 sudo[43391]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-12T02:51:53.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:53 smithi146 sudo[43391]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-12T02:51:53.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:53 smithi146 sudo[43391]: pam_unix(sudo:session): session closed for user root 2024-09-12T02:51:53.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:53 smithi146 ceph-mon[35011]: pgmap v83: 1 pgs: 1 unknown; 0 B data, 85 MiB used, 268 GiB / 268 GiB avail 2024-09-12T02:51:53.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:53 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:51:53.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:53 smithi146 ceph-mon[35011]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T02:51:53.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:53 smithi146 ceph-mon[35011]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T02:51:53.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:53 smithi146 ceph-mon[35011]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-12T02:51:53.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:53 smithi146 ceph-mon[35011]: Cluster is now healthy 2024-09-12T02:51:53.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:53 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T02:51:53.958 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:53 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:51:53.958 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:53 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:51:53.958 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:53 smithi146 ceph-mon[35011]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-12T02:51:53.958 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:53 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T02:51:53.958 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:53 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:51:53.958 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:53 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:51:54.238 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:54 smithi005 ceph-mon[31197]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T02:51:54.239 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:54 smithi005 ceph-mon[31197]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T02:51:54.239 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:54 smithi005 ceph-mon[31197]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T02:51:54.239 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:54 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T02:51:54.239 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:54 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:51:54.239 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:54 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:51:54.239 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:54 smithi005 ceph-mon[31197]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T02:51:54.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:54 smithi148 ceph-mon[35090]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T02:51:54.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:54 smithi148 ceph-mon[35090]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T02:51:54.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:54 smithi148 ceph-mon[35090]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T02:51:54.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:54 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T02:51:54.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:54 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:51:54.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:54 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:51:54.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:54 smithi148 ceph-mon[35090]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T02:51:54.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:54 smithi146 ceph-mon[35011]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T02:51:54.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:54 smithi146 ceph-mon[35011]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T02:51:54.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:54 smithi146 ceph-mon[35011]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-12T02:51:54.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:54 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-12T02:51:54.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:54 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-12T02:51:54.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:54 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-12T02:51:54.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:54 smithi146 ceph-mon[35011]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-12T02:51:55.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:55 smithi005 ceph-mon[31197]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-09-12T02:51:55.415 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:55 smithi148 ceph-mon[35090]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-09-12T02:51:55.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:55 smithi146 ceph-mon[35011]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-09-12T02:51:56.271 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:56 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-12T02:51:56.271 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:56 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:51:56.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:56 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-12T02:51:56.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:56 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:51:56.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:56 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-12T02:51:56.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:56 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:51:57.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:57 smithi005 ceph-mon[31197]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-09-12T02:51:57.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:57 smithi005 ceph-mon[31197]: Deploying daemon osd.3 on smithi005 2024-09-12T02:51:57.415 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:57 smithi148 ceph-mon[35090]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-09-12T02:51:57.428 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:57 smithi148 ceph-mon[35090]: Deploying daemon osd.3 on smithi005 2024-09-12T02:51:57.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:57 smithi146 ceph-mon[35011]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-09-12T02:51:57.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:57 smithi146 ceph-mon[35011]: Deploying daemon osd.3 on smithi005 2024-09-12T02:51:59.280 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:51:59 smithi005 ceph-mon[31197]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 65 KiB/s wr, 5 op/s 2024-09-12T02:51:59.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:51:59 smithi148 ceph-mon[35090]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 65 KiB/s wr, 5 op/s 2024-09-12T02:51:59.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:51:59 smithi146 ceph-mon[35011]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 65 KiB/s wr, 5 op/s 2024-09-12T02:52:01.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:01 smithi005 ceph-mon[31197]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-09-12T02:52:01.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:01 smithi148 ceph-mon[35090]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-09-12T02:52:01.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:01 smithi146 ceph-mon[35011]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-09-12T02:52:02.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:02 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:52:02.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:02 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:52:02.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:02 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:52:02.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:02 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:52:02.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:02 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:52:02.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:02 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:52:02.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:02 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:52:02.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:02 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:52:02.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:02 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:52:03.255 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:03 smithi005 ceph-mon[31197]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-09-12T02:52:03.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:03 smithi148 ceph-mon[35090]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-09-12T02:52:03.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:03 smithi146 ceph-mon[35011]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-09-12T02:52:05.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:05 smithi148 ceph-mon[35090]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-09-12T02:52:05.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:05 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:52:05.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:05 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:52:05.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:05 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:52:05.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:05 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:52:05.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:05 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:52:05.444 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:05 smithi005 ceph-mon[31197]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-09-12T02:52:05.444 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:05 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:52:05.445 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:05 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:52:05.445 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:05 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:52:05.445 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:05 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:52:05.445 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:05 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:52:05.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:05 smithi146 ceph-mon[35011]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-09-12T02:52:05.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:05 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:52:05.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:05 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:52:05.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:05 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:52:05.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:05 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:52:05.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:05 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:52:05.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:52:05.488+0000 7f504dffb640 1 -- 172.21.15.5:0/3777991906 <== mgr.14148 v2:172.21.15.5:6800/3780631642 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f5050062630 con 0x7f50400a2af0 2024-09-12T02:52:05.488 INFO:teuthology.orchestra.run.smithi005.stdout:Created osd(s) 3 on host 'smithi005' 2024-09-12T02:52:05.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:52:05.490+0000 7f5056aeb640 1 -- 172.21.15.5:0/3777991906 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f50400a2af0 msgr2=0x7f50400a4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:52:05.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:52:05.490+0000 7f5056aeb640 1 --2- 172.21.15.5:0/3777991906 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f50400a2af0 0x7f50400a4fb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5050101d80 tx=0x7f503c00b410 comp rx=0 tx=0).stop 2024-09-12T02:52:05.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:52:05.490+0000 7f5056aeb640 1 -- 172.21.15.5:0/3777991906 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50501026a0 msgr2=0x7f5050102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:52:05.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:52:05.490+0000 7f5056aeb640 1 --2- 172.21.15.5:0/3777991906 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50501026a0 0x7f5050102000 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f5044009230 tx=0x7f5044004360 comp rx=0 tx=0).stop 2024-09-12T02:52:05.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:52:05.490+0000 7f5056aeb640 1 -- 172.21.15.5:0/3777991906 shutdown_connections 2024-09-12T02:52:05.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:52:05.490+0000 7f5056aeb640 1 --2- 172.21.15.5:0/3777991906 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5050107900 0x7f5050100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:05.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:52:05.490+0000 7f5056aeb640 1 --2- 172.21.15.5:0/3777991906 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5050104fd0 0x7f5050100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:05.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:52:05.490+0000 7f5056aeb640 1 --2- 172.21.15.5:0/3777991906 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f50400a2af0 0x7f50400a4fb0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:05.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:52:05.490+0000 7f5056aeb640 1 --2- 172.21.15.5:0/3777991906 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50501026a0 0x7f5050102000 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:05.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:52:05.490+0000 7f5056aeb640 1 -- 172.21.15.5:0/3777991906 >> 172.21.15.5:0/3777991906 conn(0x7f50500fc2a0 msgr2=0x7f5050102f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:52:05.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:52:05.490+0000 7f5056aeb640 1 -- 172.21.15.5:0/3777991906 shutdown_connections 2024-09-12T02:52:05.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:52:05.490+0000 7f5056aeb640 1 -- 172.21.15.5:0/3777991906 wait complete. 2024-09-12T02:52:06.103 DEBUG:teuthology.orchestra.run.smithi005:osd.3> sudo journalctl -f -n 0 -u ceph-618305f8-70b1-11ef-bcea-c7b262605968@osd.3.service 2024-09-12T02:52:06.105 INFO:tasks.cephadm:Deploying osd.4 on smithi146 with /dev/vg_nvme/lv_3... 2024-09-12T02:52:06.105 DEBUG:teuthology.orchestra.run.smithi146:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-12T02:52:06.337 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:06 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:52:06.337 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:06 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:52:06.337 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:06 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:52:06.359 INFO:teuthology.orchestra.run.smithi146.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T02:52:06.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:06 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:52:06.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:06 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:52:06.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:06 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:52:06.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:06 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:52:06.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:06 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:52:06.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:06 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:52:07.197 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:07 smithi005 ceph-mon[31197]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-12T02:52:07.408 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:07 smithi146 ceph-mon[35011]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-12T02:52:07.415 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:07 smithi148 ceph-mon[35090]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-12T02:52:08.561 INFO:journalctl@ceph.osd.3.smithi005.stdout:Sep 12 02:52:08 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-3[49554]: 2024-09-12T02:52:08.133+0000 7f971eb9f640 -1 osd.3 0 log_to_monitors true 2024-09-12T02:52:08.734 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-12T02:52:08.760 DEBUG:teuthology.orchestra.run.smithi146:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi146:vg_nvme/lv_3 2024-09-12T02:52:09.010 INFO:teuthology.orchestra.run.smithi146.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T02:52:09.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:09 smithi005 ceph-mon[31197]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-12T02:52:09.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:09 smithi005 ceph-mon[31197]: from='osd.3 [v2:172.21.15.5:6810/11741938,v1:172.21.15.5:6811/11741938]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-12T02:52:09.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:09 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:52:09.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:09 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:52:09.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:09 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:52:09.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:09 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:52:09.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:09 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:52:09.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:09 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:52:09.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:09 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:52:09.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:09 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:52:09.378 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:09 smithi146 ceph-mon[35011]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-12T02:52:09.378 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:09 smithi146 ceph-mon[35011]: from='osd.3 [v2:172.21.15.5:6810/11741938,v1:172.21.15.5:6811/11741938]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-12T02:52:09.378 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:09 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:52:09.378 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:09 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:52:09.378 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:09 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:52:09.378 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:09 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:52:09.378 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:09 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:52:09.378 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:09 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:52:09.378 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:09 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:52:09.378 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:09 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:52:09.415 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:09 smithi148 ceph-mon[35090]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-12T02:52:09.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:09 smithi148 ceph-mon[35090]: from='osd.3 [v2:172.21.15.5:6810/11741938,v1:172.21.15.5:6811/11741938]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-12T02:52:09.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:09 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:52:09.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:09 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:52:09.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:09 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:52:09.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:09 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:52:09.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:09 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:52:09.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:09 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:52:09.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:09 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:52:09.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:09 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:52:09.468 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:52:09.435+0000 7f0ce02d9640 1 -- 172.21.15.146:0/1639472118 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0cd81026d0 msgr2=0x7f0cd8108f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:52:09.468 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:52:09.435+0000 7f0ce02d9640 1 --2- 172.21.15.146:0/1639472118 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0cd81026d0 0x7f0cd8108f60 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0cd0009ca0 tx=0x7f0cd002a8c0 comp rx=0 tx=0).stop 2024-09-12T02:52:09.468 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:52:09.436+0000 7f0ce02d9640 1 -- 172.21.15.146:0/1639472118 shutdown_connections 2024-09-12T02:52:09.468 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:52:09.436+0000 7f0ce02d9640 1 --2- 172.21.15.146:0/1639472118 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0cd81026d0 0x7f0cd8108f60 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:09.468 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:52:09.436+0000 7f0ce02d9640 1 --2- 172.21.15.146:0/1639472118 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0cd8101d10 0x7f0cd8102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:09.468 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:52:09.436+0000 7f0ce02d9640 1 --2- 172.21.15.146:0/1639472118 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0cd8100b10 0x7f0cd8100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:09.468 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:52:09.436+0000 7f0ce02d9640 1 -- 172.21.15.146:0/1639472118 >> 172.21.15.146:0/1639472118 conn(0x7f0cd80fc2a0 msgr2=0x7f0cd80fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:52:09.468 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:52:09.436+0000 7f0ce02d9640 1 -- 172.21.15.146:0/1639472118 shutdown_connections 2024-09-12T02:52:09.468 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:52:09.436+0000 7f0ce02d9640 1 -- 172.21.15.146:0/1639472118 wait complete. 2024-09-12T02:52:09.469 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:52:09.436+0000 7f0ce02d9640 1 Processor -- start 2024-09-12T02:52:09.469 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:52:09.436+0000 7f0ce02d9640 1 -- start start 2024-09-12T02:52:09.469 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:52:09.437+0000 7f0ce02d9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0cd8100b10 0x7f0cd819a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:52:09.469 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:52:09.437+0000 7f0ce02d9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0cd8101d10 0x7f0cd819a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:52:09.469 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:52:09.437+0000 7f0ce02d9640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0cd81026d0 0x7f0cd81a1680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:52:09.469 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:52:09.437+0000 7f0ce02d9640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0cd810c770 con 0x7f0cd8101d10 2024-09-12T02:52:09.469 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:52:09.437+0000 7f0ce02d9640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f0cd810c5f0 con 0x7f0cd81026d0 2024-09-12T02:52:09.469 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:52:09.437+0000 7f0ce02d9640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f0cd810c8f0 con 0x7f0cd8100b10 2024-09-12T02:52:09.469 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:52:09.437+0000 7f0cde876640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0cd81026d0 0x7f0cd81a1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:52:09.469 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:52:09.437+0000 7f0cde075640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0cd8100b10 0x7f0cd819a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:52:09.469 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:52:09.437+0000 7f0cde075640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0cd8100b10 0x7f0cd819a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.146:45322/0 (socket says 172.21.15.146:45322) 2024-09-12T02:52:09.470 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:52:09.437+0000 7f0cdd874640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0cd8101d10 0x7f0cd819a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:52:09.470 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:52:09.437+0000 7f0cde075640 1 -- 172.21.15.146:0/3092275285 learned_addr learned my addr 172.21.15.146:0/3092275285 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-12T02:52:09.470 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:52:09.437+0000 7f0cdd874640 1 -- 172.21.15.146:0/3092275285 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0cd8100b10 msgr2=0x7f0cd819a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:52:09.470 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:52:09.437+0000 7f0cdd874640 1 --2- 172.21.15.146:0/3092275285 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0cd8100b10 0x7f0cd819a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:09.470 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:52:09.437+0000 7f0cdd874640 1 -- 172.21.15.146:0/3092275285 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0cd81026d0 msgr2=0x7f0cd81a1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:52:09.470 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:52:09.437+0000 7f0cdd874640 1 --2- 172.21.15.146:0/3092275285 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0cd81026d0 0x7f0cd81a1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:09.470 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:52:09.437+0000 7f0cdd874640 1 -- 172.21.15.146:0/3092275285 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cd81a1cf0 con 0x7f0cd8101d10 2024-09-12T02:52:09.470 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:52:09.437+0000 7f0cde075640 1 --2- 172.21.15.146:0/3092275285 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0cd8100b10 0x7f0cd819a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:52:09.470 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:52:09.437+0000 7f0cdd874640 1 --2- 172.21.15.146:0/3092275285 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0cd8101d10 0x7f0cd819a600 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0cd400e9f0 tx=0x7f0cd400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:52:09.470 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:52:09.437+0000 7f0ccf7fe640 1 -- 172.21.15.146:0/3092275285 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cd400ccc0 con 0x7f0cd8101d10 2024-09-12T02:52:09.470 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:52:09.437+0000 7f0ccf7fe640 1 -- 172.21.15.146:0/3092275285 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0cd4010910 con 0x7f0cd8101d10 2024-09-12T02:52:09.470 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:52:09.437+0000 7f0ce02d9640 1 -- 172.21.15.146:0/3092275285 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cd81a1fe0 con 0x7f0cd8101d10 2024-09-12T02:52:09.470 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:52:09.437+0000 7f0ccf7fe640 1 -- 172.21.15.146:0/3092275285 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cd4018960 con 0x7f0cd8101d10 2024-09-12T02:52:09.471 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:52:09.437+0000 7f0ce02d9640 1 -- 172.21.15.146:0/3092275285 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cd81a2490 con 0x7f0cd8101d10 2024-09-12T02:52:09.471 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:52:09.438+0000 7f0ce02d9640 1 -- 172.21.15.146:0/3092275285 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cd81a26c0 con 0x7f0cd8101d10 2024-09-12T02:52:09.471 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:52:09.440+0000 7f0ccf7fe640 1 -- 172.21.15.146:0/3092275285 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f0cd4010ac0 con 0x7f0cd8101d10 2024-09-12T02:52:09.471 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:52:09.440+0000 7f0ccf7fe640 1 --2- 172.21.15.146:0/3092275285 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f0cc40a2bb0 0x7f0cc40a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:52:09.471 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:52:09.440+0000 7f0ccf7fe640 1 -- 172.21.15.146:0/3092275285 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 3373+0+0 (secure 0 0 0) 0x7f0cd40ce410 con 0x7f0cd8101d10 2024-09-12T02:52:09.471 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:52:09.440+0000 7f0cde075640 1 --2- 172.21.15.146:0/3092275285 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f0cc40a2bb0 0x7f0cc40a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:52:09.471 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:52:09.441+0000 7f0cde075640 1 --2- 172.21.15.146:0/3092275285 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f0cc40a2bb0 0x7f0cc40a5070 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0cc80095f0 tx=0x7f0cc8005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:52:09.471 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:52:09.444+0000 7f0ccf7fe640 1 -- 172.21.15.146:0/3092275285 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0cd409bfd0 con 0x7f0cd8101d10 2024-09-12T02:52:09.622 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:52:09.622+0000 7f0ce02d9640 1 -- 172.21.15.146:0/3092275285 --> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi146:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f0cd806aea0 con 0x7f0cc40a2bb0 2024-09-12T02:52:10.337 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:10 smithi146 ceph-mon[35011]: Detected new or changed devices on smithi005 2024-09-12T02:52:10.337 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:10 smithi146 ceph-mon[35011]: Adjusting osd_memory_target on smithi005 to 8546M 2024-09-12T02:52:10.337 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:10 smithi146 ceph-mon[35011]: from='osd.3 [v2:172.21.15.5:6810/11741938,v1:172.21.15.5:6811/11741938]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-12T02:52:10.337 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:10 smithi146 ceph-mon[35011]: osdmap e23: 4 total, 3 up, 4 in 2024-09-12T02:52:10.337 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:10 smithi146 ceph-mon[35011]: from='osd.3 [v2:172.21.15.5:6810/11741938,v1:172.21.15.5:6811/11741938]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2024-09-12T02:52:10.337 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:10 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T02:52:10.337 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:10 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:52:10.337 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:10 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:52:10.338 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:10 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:52:10.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:10 smithi148 ceph-mon[35090]: Detected new or changed devices on smithi005 2024-09-12T02:52:10.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:10 smithi148 ceph-mon[35090]: Adjusting osd_memory_target on smithi005 to 8546M 2024-09-12T02:52:10.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:10 smithi148 ceph-mon[35090]: from='osd.3 [v2:172.21.15.5:6810/11741938,v1:172.21.15.5:6811/11741938]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-12T02:52:10.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:10 smithi148 ceph-mon[35090]: osdmap e23: 4 total, 3 up, 4 in 2024-09-12T02:52:10.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:10 smithi148 ceph-mon[35090]: from='osd.3 [v2:172.21.15.5:6810/11741938,v1:172.21.15.5:6811/11741938]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2024-09-12T02:52:10.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:10 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T02:52:10.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:10 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:52:10.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:10 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:52:10.417 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:10 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:52:10.560 INFO:journalctl@ceph.osd.3.smithi005.stdout:Sep 12 02:52:10 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-3[49554]: 2024-09-12T02:52:10.131+0000 7f971a2ff640 -1 osd.3 0 waiting for initial osdmap 2024-09-12T02:52:10.560 INFO:journalctl@ceph.osd.3.smithi005.stdout:Sep 12 02:52:10 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-3[49554]: 2024-09-12T02:52:10.136+0000 7f9716939640 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T02:52:10.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:10 smithi005 ceph-mon[31197]: Detected new or changed devices on smithi005 2024-09-12T02:52:10.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:10 smithi005 ceph-mon[31197]: Adjusting osd_memory_target on smithi005 to 8546M 2024-09-12T02:52:10.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:10 smithi005 ceph-mon[31197]: from='osd.3 [v2:172.21.15.5:6810/11741938,v1:172.21.15.5:6811/11741938]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-12T02:52:10.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:10 smithi005 ceph-mon[31197]: osdmap e23: 4 total, 3 up, 4 in 2024-09-12T02:52:10.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:10 smithi005 ceph-mon[31197]: from='osd.3 [v2:172.21.15.5:6810/11741938,v1:172.21.15.5:6811/11741938]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2024-09-12T02:52:10.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:10 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T02:52:10.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:10 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:52:10.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:10 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:52:10.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:10 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:52:11.378 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:11 smithi146 ceph-mon[35011]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-09-12T02:52:11.378 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:11 smithi146 ceph-mon[35011]: from='client.14313 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi146:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:52:11.378 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:11 smithi146 ceph-mon[35011]: from='osd.3 [v2:172.21.15.5:6810/11741938,v1:172.21.15.5:6811/11741938]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2024-09-12T02:52:11.378 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:11 smithi146 ceph-mon[35011]: osdmap e24: 4 total, 3 up, 4 in 2024-09-12T02:52:11.378 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:11 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T02:52:11.378 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:11 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T02:52:11.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:11 smithi148 ceph-mon[35090]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-09-12T02:52:11.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:11 smithi148 ceph-mon[35090]: from='client.14313 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi146:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:52:11.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:11 smithi148 ceph-mon[35090]: from='osd.3 [v2:172.21.15.5:6810/11741938,v1:172.21.15.5:6811/11741938]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2024-09-12T02:52:11.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:11 smithi148 ceph-mon[35090]: osdmap e24: 4 total, 3 up, 4 in 2024-09-12T02:52:11.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:11 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T02:52:11.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:11 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T02:52:11.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:11 smithi005 ceph-mon[31197]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-09-12T02:52:11.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:11 smithi005 ceph-mon[31197]: from='client.14313 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi146:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:52:11.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:11 smithi005 ceph-mon[31197]: from='osd.3 [v2:172.21.15.5:6810/11741938,v1:172.21.15.5:6811/11741938]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2024-09-12T02:52:11.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:11 smithi005 ceph-mon[31197]: osdmap e24: 4 total, 3 up, 4 in 2024-09-12T02:52:11.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:11 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T02:52:11.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:11 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T02:52:12.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:12 smithi148 ceph-mon[35090]: purged_snaps scrub starts 2024-09-12T02:52:12.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:12 smithi148 ceph-mon[35090]: purged_snaps scrub ok 2024-09-12T02:52:12.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:12 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T02:52:12.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:12 smithi148 ceph-mon[35090]: osd.3 [v2:172.21.15.5:6810/11741938,v1:172.21.15.5:6811/11741938] boot 2024-09-12T02:52:12.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:12 smithi148 ceph-mon[35090]: osdmap e25: 4 total, 4 up, 4 in 2024-09-12T02:52:12.417 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:12 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T02:52:12.417 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:12 smithi148 ceph-mon[35090]: from='client.? 172.21.15.146:0/47244454' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "989d76e7-c742-49d0-994a-c58f2771b1d7"}]: dispatch 2024-09-12T02:52:12.417 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:12 smithi148 ceph-mon[35090]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "989d76e7-c742-49d0-994a-c58f2771b1d7"}]: dispatch 2024-09-12T02:52:12.417 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:12 smithi148 ceph-mon[35090]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "989d76e7-c742-49d0-994a-c58f2771b1d7"}]': finished 2024-09-12T02:52:12.417 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:12 smithi148 ceph-mon[35090]: osdmap e26: 5 total, 4 up, 5 in 2024-09-12T02:52:12.417 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:12 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T02:52:12.450 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:12 smithi146 ceph-mon[35011]: purged_snaps scrub starts 2024-09-12T02:52:12.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:12 smithi146 ceph-mon[35011]: purged_snaps scrub ok 2024-09-12T02:52:12.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:12 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T02:52:12.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:12 smithi146 ceph-mon[35011]: osd.3 [v2:172.21.15.5:6810/11741938,v1:172.21.15.5:6811/11741938] boot 2024-09-12T02:52:12.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:12 smithi146 ceph-mon[35011]: osdmap e25: 4 total, 4 up, 4 in 2024-09-12T02:52:12.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:12 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T02:52:12.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:12 smithi146 ceph-mon[35011]: from='client.? 172.21.15.146:0/47244454' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "989d76e7-c742-49d0-994a-c58f2771b1d7"}]: dispatch 2024-09-12T02:52:12.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:12 smithi146 ceph-mon[35011]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "989d76e7-c742-49d0-994a-c58f2771b1d7"}]: dispatch 2024-09-12T02:52:12.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:12 smithi146 ceph-mon[35011]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "989d76e7-c742-49d0-994a-c58f2771b1d7"}]': finished 2024-09-12T02:52:12.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:12 smithi146 ceph-mon[35011]: osdmap e26: 5 total, 4 up, 5 in 2024-09-12T02:52:12.451 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:12 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T02:52:12.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:12 smithi005 ceph-mon[31197]: purged_snaps scrub starts 2024-09-12T02:52:12.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:12 smithi005 ceph-mon[31197]: purged_snaps scrub ok 2024-09-12T02:52:12.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:12 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T02:52:12.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:12 smithi005 ceph-mon[31197]: osd.3 [v2:172.21.15.5:6810/11741938,v1:172.21.15.5:6811/11741938] boot 2024-09-12T02:52:12.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:12 smithi005 ceph-mon[31197]: osdmap e25: 4 total, 4 up, 4 in 2024-09-12T02:52:12.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:12 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-12T02:52:12.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:12 smithi005 ceph-mon[31197]: from='client.? 172.21.15.146:0/47244454' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "989d76e7-c742-49d0-994a-c58f2771b1d7"}]: dispatch 2024-09-12T02:52:12.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:12 smithi005 ceph-mon[31197]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "989d76e7-c742-49d0-994a-c58f2771b1d7"}]: dispatch 2024-09-12T02:52:12.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:12 smithi005 ceph-mon[31197]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "989d76e7-c742-49d0-994a-c58f2771b1d7"}]': finished 2024-09-12T02:52:12.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:12 smithi005 ceph-mon[31197]: osdmap e26: 5 total, 4 up, 5 in 2024-09-12T02:52:12.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:12 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T02:52:13.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:13 smithi148 ceph-mon[35090]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-09-12T02:52:13.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:13 smithi148 ceph-mon[35090]: from='client.? 172.21.15.146:0/2738348577' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:52:13.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:13 smithi148 ceph-mon[35090]: osdmap e27: 5 total, 4 up, 5 in 2024-09-12T02:52:13.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:13 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T02:52:13.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:13 smithi146 ceph-mon[35011]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-09-12T02:52:13.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:13 smithi146 ceph-mon[35011]: from='client.? 172.21.15.146:0/2738348577' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:52:13.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:13 smithi146 ceph-mon[35011]: osdmap e27: 5 total, 4 up, 5 in 2024-09-12T02:52:13.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:13 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T02:52:13.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:13 smithi005 ceph-mon[31197]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-09-12T02:52:13.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:13 smithi005 ceph-mon[31197]: from='client.? 172.21.15.146:0/2738348577' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:52:13.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:13 smithi005 ceph-mon[31197]: osdmap e27: 5 total, 4 up, 5 in 2024-09-12T02:52:13.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:13 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T02:52:15.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:14 smithi148 ceph-mon[35090]: pgmap v99: 1 pgs: 1 remapped+peering; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-12T02:52:15.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:14 smithi148 ceph-mon[35090]: osdmap e28: 5 total, 4 up, 5 in 2024-09-12T02:52:15.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:14 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T02:52:15.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:14 smithi146 ceph-mon[35011]: pgmap v99: 1 pgs: 1 remapped+peering; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-12T02:52:15.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:14 smithi146 ceph-mon[35011]: osdmap e28: 5 total, 4 up, 5 in 2024-09-12T02:52:15.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:14 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T02:52:15.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:14 smithi005 ceph-mon[31197]: pgmap v99: 1 pgs: 1 remapped+peering; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-12T02:52:15.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:14 smithi005 ceph-mon[31197]: osdmap e28: 5 total, 4 up, 5 in 2024-09-12T02:52:15.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:14 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T02:52:16.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:15 smithi148 ceph-mon[35090]: osdmap e29: 5 total, 4 up, 5 in 2024-09-12T02:52:16.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:15 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T02:52:16.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:15 smithi146 ceph-mon[35011]: osdmap e29: 5 total, 4 up, 5 in 2024-09-12T02:52:16.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:15 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T02:52:16.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:15 smithi005 ceph-mon[31197]: osdmap e29: 5 total, 4 up, 5 in 2024-09-12T02:52:16.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:15 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T02:52:17.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:16 smithi148 ceph-mon[35090]: pgmap v102: 1 pgs: 1 remapped+peering; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-12T02:52:17.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:16 smithi146 ceph-mon[35011]: pgmap v102: 1 pgs: 1 remapped+peering; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-12T02:52:17.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:16 smithi005 ceph-mon[31197]: pgmap v102: 1 pgs: 1 remapped+peering; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-12T02:52:19.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:18 smithi148 ceph-mon[35090]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-09-12T02:52:19.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:18 smithi146 ceph-mon[35011]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-09-12T02:52:19.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:18 smithi005 ceph-mon[31197]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-09-12T02:52:21.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:20 smithi148 ceph-mon[35090]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-09-12T02:52:21.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:20 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-12T02:52:21.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:20 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:52:21.181 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:20 smithi146 ceph-mon[35011]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-09-12T02:52:21.181 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:20 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-12T02:52:21.182 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:20 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:52:21.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:20 smithi005 ceph-mon[31197]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-09-12T02:52:21.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:20 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-12T02:52:21.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:20 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:52:22.071 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:21 smithi146 ceph-mon[35011]: Deploying daemon osd.4 on smithi146 2024-09-12T02:52:22.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:21 smithi148 ceph-mon[35090]: Deploying daemon osd.4 on smithi146 2024-09-12T02:52:22.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:21 smithi005 ceph-mon[31197]: Deploying daemon osd.4 on smithi146 2024-09-12T02:52:23.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:22 smithi148 ceph-mon[35090]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-12T02:52:23.184 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:22 smithi146 ceph-mon[35011]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-12T02:52:23.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:22 smithi005 ceph-mon[31197]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-12T02:52:25.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:24 smithi148 ceph-mon[35090]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-09-12T02:52:25.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:24 smithi146 ceph-mon[35011]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-09-12T02:52:25.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:24 smithi005 ceph-mon[31197]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-09-12T02:52:27.138 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:26 smithi146 ceph-mon[35011]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-09-12T02:52:27.138 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:26 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:52:27.138 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:26 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:52:27.138 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:26 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:52:27.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:26 smithi005 ceph-mon[31197]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-09-12T02:52:27.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:26 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:52:27.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:26 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:52:27.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:26 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:52:27.415 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:26 smithi148 ceph-mon[35090]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-09-12T02:52:27.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:26 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:52:27.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:26 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:52:27.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:26 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:52:28.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:28 smithi146 ceph-mon[35011]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-12T02:52:28.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:28 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:52:28.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:28 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:52:28.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:28 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:52:28.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:28 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:52:28.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:28 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:52:29.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:28 smithi005 ceph-mon[31197]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-12T02:52:29.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:28 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:52:29.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:28 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:52:29.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:28 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:52:29.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:28 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:52:29.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:28 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:52:29.415 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:28 smithi148 ceph-mon[35090]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-12T02:52:29.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:28 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:52:29.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:28 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:52:29.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:28 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:52:29.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:28 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:52:29.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:28 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:52:30.120 INFO:teuthology.orchestra.run.smithi146.stdout:Created osd(s) 4 on host 'smithi146' 2024-09-12T02:52:30.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:52:30.118+0000 7f0ccf7fe640 1 -- 172.21.15.146:0/3092275285 <== mgr.14148 v2:172.21.15.5:6800/3780631642 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f0cd806aea0 con 0x7f0cc40a2bb0 2024-09-12T02:52:30.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:52:30.121+0000 7f0ce02d9640 1 -- 172.21.15.146:0/3092275285 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f0cc40a2bb0 msgr2=0x7f0cc40a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:52:30.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:52:30.121+0000 7f0ce02d9640 1 --2- 172.21.15.146:0/3092275285 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f0cc40a2bb0 0x7f0cc40a5070 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0cc80095f0 tx=0x7f0cc8005f90 comp rx=0 tx=0).stop 2024-09-12T02:52:30.121 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:52:30.121+0000 7f0ce02d9640 1 -- 172.21.15.146:0/3092275285 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0cd8101d10 msgr2=0x7f0cd819a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:52:30.122 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:52:30.121+0000 7f0ce02d9640 1 --2- 172.21.15.146:0/3092275285 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0cd8101d10 0x7f0cd819a600 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0cd400e9f0 tx=0x7f0cd400eeb0 comp rx=0 tx=0).stop 2024-09-12T02:52:30.122 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:52:30.121+0000 7f0ce02d9640 1 -- 172.21.15.146:0/3092275285 shutdown_connections 2024-09-12T02:52:30.122 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:52:30.121+0000 7f0ce02d9640 1 --2- 172.21.15.146:0/3092275285 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f0cc40a2bb0 0x7f0cc40a5070 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:30.122 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:52:30.121+0000 7f0ce02d9640 1 --2- 172.21.15.146:0/3092275285 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0cd8101d10 0x7f0cd819a600 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:30.122 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:52:30.121+0000 7f0ce02d9640 1 --2- 172.21.15.146:0/3092275285 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0cd81026d0 0x7f0cd81a1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:30.122 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:52:30.121+0000 7f0ce02d9640 1 --2- 172.21.15.146:0/3092275285 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0cd8100b10 0x7f0cd819a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:30.122 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:52:30.121+0000 7f0ce02d9640 1 -- 172.21.15.146:0/3092275285 >> 172.21.15.146:0/3092275285 conn(0x7f0cd80fc2a0 msgr2=0x7f0cd806a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:52:30.122 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:52:30.121+0000 7f0ce02d9640 1 -- 172.21.15.146:0/3092275285 shutdown_connections 2024-09-12T02:52:30.122 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:52:30.121+0000 7f0ce02d9640 1 -- 172.21.15.146:0/3092275285 wait complete. 2024-09-12T02:52:30.749 DEBUG:teuthology.orchestra.run.smithi146:osd.4> sudo journalctl -f -n 0 -u ceph-618305f8-70b1-11ef-bcea-c7b262605968@osd.4.service 2024-09-12T02:52:30.751 INFO:tasks.cephadm:Deploying osd.5 on smithi148 with /dev/vg_nvme/lv_3... 2024-09-12T02:52:30.751 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-12T02:52:30.996 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:30 smithi146 ceph-mon[35011]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-12T02:52:30.997 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:30 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:52:30.997 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:30 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:52:30.997 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:30 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:52:31.013 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T02:52:31.134 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:30 smithi148 ceph-mon[35090]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-12T02:52:31.134 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:30 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:52:31.134 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:30 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:52:31.134 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:30 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:52:31.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:30 smithi005 ceph-mon[31197]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-12T02:52:31.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:30 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:52:31.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:30 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:52:31.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:30 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:52:33.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:32 smithi148 ceph-mon[35090]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-12T02:52:33.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:32 smithi148 ceph-mon[35090]: from='osd.4 [v2:172.21.15.146:6808/3274923851,v1:172.21.15.146:6809/3274923851]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-12T02:52:33.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:32 smithi148 ceph-mon[35090]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-12T02:52:33.175 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:32 smithi146 ceph-mon[35011]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-12T02:52:33.175 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:32 smithi146 ceph-mon[35011]: from='osd.4 [v2:172.21.15.146:6808/3274923851,v1:172.21.15.146:6809/3274923851]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-12T02:52:33.175 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:32 smithi146 ceph-mon[35011]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-12T02:52:33.175 INFO:journalctl@ceph.osd.4.smithi146.stdout:Sep 12 02:52:32 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-4[46161]: 2024-09-12T02:52:32.740+0000 7fd5f8b1d640 -1 osd.4 0 log_to_monitors true 2024-09-12T02:52:33.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:32 smithi005 ceph-mon[31197]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-12T02:52:33.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:32 smithi005 ceph-mon[31197]: from='osd.4 [v2:172.21.15.146:6808/3274923851,v1:172.21.15.146:6809/3274923851]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-12T02:52:33.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:32 smithi005 ceph-mon[31197]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-12T02:52:33.586 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-12T02:52:33.622 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi148:vg_nvme/lv_3 2024-09-12T02:52:33.880 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T02:52:34.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:33 smithi146 ceph-mon[35011]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-12T02:52:34.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:33 smithi146 ceph-mon[35011]: osdmap e30: 5 total, 4 up, 5 in 2024-09-12T02:52:34.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:33 smithi146 ceph-mon[35011]: from='osd.4 [v2:172.21.15.146:6808/3274923851,v1:172.21.15.146:6809/3274923851]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2024-09-12T02:52:34.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:33 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T02:52:34.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:33 smithi146 ceph-mon[35011]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2024-09-12T02:52:34.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:33 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:52:34.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:33 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:52:34.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:33 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:52:34.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:33 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:52:34.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:33 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:52:34.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:33 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:52:34.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:33 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:52:34.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:33 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:52:34.208 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:33 smithi146 ceph-mon[35011]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]': finished 2024-09-12T02:52:34.244 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:33 smithi148 ceph-mon[35090]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-12T02:52:34.244 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:33 smithi148 ceph-mon[35090]: osdmap e30: 5 total, 4 up, 5 in 2024-09-12T02:52:34.245 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:33 smithi148 ceph-mon[35090]: from='osd.4 [v2:172.21.15.146:6808/3274923851,v1:172.21.15.146:6809/3274923851]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2024-09-12T02:52:34.245 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:33 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T02:52:34.245 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:33 smithi148 ceph-mon[35090]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2024-09-12T02:52:34.245 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:33 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:52:34.245 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:33 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:52:34.245 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:33 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:52:34.245 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:33 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:52:34.245 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:33 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:52:34.245 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:33 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:52:34.245 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:33 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:52:34.245 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:33 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:52:34.245 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:33 smithi148 ceph-mon[35090]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]': finished 2024-09-12T02:52:34.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:33 smithi005 ceph-mon[31197]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-12T02:52:34.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:33 smithi005 ceph-mon[31197]: osdmap e30: 5 total, 4 up, 5 in 2024-09-12T02:52:34.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:33 smithi005 ceph-mon[31197]: from='osd.4 [v2:172.21.15.146:6808/3274923851,v1:172.21.15.146:6809/3274923851]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2024-09-12T02:52:34.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:33 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T02:52:34.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:33 smithi005 ceph-mon[31197]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2024-09-12T02:52:34.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:33 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:52:34.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:33 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:52:34.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:33 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:52:34.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:33 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:52:34.312 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:33 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:52:34.312 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:33 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:52:34.312 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:33 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:52:34.312 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:33 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:52:34.312 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:33 smithi005 ceph-mon[31197]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]': finished 2024-09-12T02:52:34.344 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:52:34.330+0000 7f310b8e3640 1 -- 172.21.15.148:0/330828329 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f31041037c0 msgr2=0x7f310410a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:52:34.344 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:52:34.330+0000 7f310b8e3640 1 --2- 172.21.15.148:0/330828329 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f31041037c0 0x7f310410a050 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f30fc009900 tx=0x7f30fc02a410 comp rx=0 tx=0).stop 2024-09-12T02:52:34.344 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:52:34.330+0000 7f310b8e3640 1 -- 172.21.15.148:0/330828329 shutdown_connections 2024-09-12T02:52:34.345 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:52:34.330+0000 7f310b8e3640 1 --2- 172.21.15.148:0/330828329 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f31041037c0 0x7f310410a050 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:34.345 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:52:34.330+0000 7f310b8e3640 1 --2- 172.21.15.148:0/330828329 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3104102e00 0x7f3104103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:34.345 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:52:34.330+0000 7f310b8e3640 1 --2- 172.21.15.148:0/330828329 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3104101c00 0x7f3104102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:34.345 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:52:34.330+0000 7f310b8e3640 1 -- 172.21.15.148:0/330828329 >> 172.21.15.148:0/330828329 conn(0x7f31040fd390 msgr2=0x7f31040ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:52:34.345 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:52:34.330+0000 7f310b8e3640 1 -- 172.21.15.148:0/330828329 shutdown_connections 2024-09-12T02:52:34.345 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:52:34.330+0000 7f310b8e3640 1 -- 172.21.15.148:0/330828329 wait complete. 2024-09-12T02:52:34.345 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:52:34.331+0000 7f310b8e3640 1 Processor -- start 2024-09-12T02:52:34.345 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:52:34.331+0000 7f310b8e3640 1 -- start start 2024-09-12T02:52:34.345 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:52:34.331+0000 7f310b8e3640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3104101c00 0x7f3104197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:52:34.345 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:52:34.331+0000 7f310b8e3640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3104102e00 0x7f31041983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:52:34.345 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:52:34.331+0000 7f310b8e3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31041037c0 0x7f310419f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:52:34.345 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:52:34.331+0000 7f310b8e3640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f310410d880 con 0x7f31041037c0 2024-09-12T02:52:34.345 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:52:34.331+0000 7f310b8e3640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f310410d700 con 0x7f3104101c00 2024-09-12T02:52:34.346 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:52:34.331+0000 7f310b8e3640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f310410da00 con 0x7f3104102e00 2024-09-12T02:52:34.346 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:52:34.331+0000 7f3108e7e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3104102e00 0x7f31041983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:52:34.346 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:52:34.331+0000 7f3108e7e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3104102e00 0x7f31041983f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:42736/0 (socket says 172.21.15.148:42736) 2024-09-12T02:52:34.346 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:52:34.331+0000 7f3108e7e640 1 -- 172.21.15.148:0/649442590 learned_addr learned my addr 172.21.15.148:0/649442590 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-09-12T02:52:34.346 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:52:34.331+0000 7f310967f640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3104101c00 0x7f3104197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:52:34.346 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:52:34.331+0000 7f3108e7e640 1 -- 172.21.15.148:0/649442590 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3104101c00 msgr2=0x7f3104197eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:52:34.346 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:52:34.331+0000 7f3108e7e640 1 --2- 172.21.15.148:0/649442590 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3104101c00 0x7f3104197eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:34.346 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:52:34.331+0000 7f3108e7e640 1 -- 172.21.15.148:0/649442590 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31041037c0 msgr2=0x7f310419f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:52:34.346 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:52:34.331+0000 7f3108e7e640 1 --2- 172.21.15.148:0/649442590 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31041037c0 0x7f310419f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:34.346 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:52:34.331+0000 7f3108e7e640 1 -- 172.21.15.148:0/649442590 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f310419fae0 con 0x7f3104102e00 2024-09-12T02:52:34.346 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:52:34.331+0000 7f3108e7e640 1 --2- 172.21.15.148:0/649442590 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3104102e00 0x7f31041983f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f30f800e9f0 tx=0x7f30f800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:52:34.346 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:52:34.332+0000 7f31027fc640 1 -- 172.21.15.148:0/649442590 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f800cd80 con 0x7f3104102e00 2024-09-12T02:52:34.346 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:52:34.332+0000 7f31027fc640 1 -- 172.21.15.148:0/649442590 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30f8005500 con 0x7f3104102e00 2024-09-12T02:52:34.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:52:34.332+0000 7f310b8e3640 1 -- 172.21.15.148:0/649442590 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f310419fdd0 con 0x7f3104102e00 2024-09-12T02:52:34.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:52:34.332+0000 7f31027fc640 1 -- 172.21.15.148:0/649442590 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30f8010690 con 0x7f3104102e00 2024-09-12T02:52:34.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:52:34.332+0000 7f310b8e3640 1 -- 172.21.15.148:0/649442590 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31041a0280 con 0x7f3104102e00 2024-09-12T02:52:34.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:52:34.333+0000 7f310b8e3640 1 -- 172.21.15.148:0/649442590 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3104109000 con 0x7f3104102e00 2024-09-12T02:52:34.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:52:34.334+0000 7f31027fc640 1 -- 172.21.15.148:0/649442590 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f30f80046c0 con 0x7f3104102e00 2024-09-12T02:52:34.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:52:34.334+0000 7f31027fc640 1 --2- 172.21.15.148:0/649442590 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f30ec0a2c30 0x7f30ec0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:52:34.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:52:34.334+0000 7f31027fc640 1 -- 172.21.15.148:0/649442590 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 3821+0+0 (secure 0 0 0) 0x7f30f80ce2a0 con 0x7f3104102e00 2024-09-12T02:52:34.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:52:34.335+0000 7f310967f640 1 --2- 172.21.15.148:0/649442590 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f30ec0a2c30 0x7f30ec0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:52:34.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:52:34.335+0000 7f310967f640 1 --2- 172.21.15.148:0/649442590 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f30ec0a2c30 0x7f30ec0a50f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3104102c60 tx=0x7f30f400a590 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:52:34.347 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:52:34.339+0000 7f31027fc640 1 -- 172.21.15.148:0/649442590 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f30f809c390 con 0x7f3104102e00 2024-09-12T02:52:34.515 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:52:34.515+0000 7f310b8e3640 1 -- 172.21.15.148:0/649442590 --> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f3104062630 con 0x7f30ec0a2c30 2024-09-12T02:52:35.206 INFO:journalctl@ceph.osd.4.smithi146.stdout:Sep 12 02:52:34 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-4[46161]: 2024-09-12T02:52:34.738+0000 7fd5f427d640 -1 osd.4 0 waiting for initial osdmap 2024-09-12T02:52:35.206 INFO:journalctl@ceph.osd.4.smithi146.stdout:Sep 12 02:52:34 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-4[46161]: 2024-09-12T02:52:34.747+0000 7fd5ec8af640 -1 osd.4 31 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T02:52:35.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:34 smithi146 ceph-mon[35011]: Detected new or changed devices on smithi146 2024-09-12T02:52:35.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:34 smithi146 ceph-mon[35011]: Adjusting osd_memory_target on smithi146 to 8544M 2024-09-12T02:52:35.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:34 smithi146 ceph-mon[35011]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-12T02:52:35.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:34 smithi146 ceph-mon[35011]: osdmap e31: 5 total, 4 up, 5 in 2024-09-12T02:52:35.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:34 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T02:52:35.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:34 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T02:52:35.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:34 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:52:35.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:34 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:52:35.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:34 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:52:35.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:34 smithi146 ceph-mon[35011]: from='osd.4 ' entity='osd.4' 2024-09-12T02:52:35.248 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:34 smithi148 ceph-mon[35090]: Detected new or changed devices on smithi146 2024-09-12T02:52:35.248 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:34 smithi148 ceph-mon[35090]: Adjusting osd_memory_target on smithi146 to 8544M 2024-09-12T02:52:35.248 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:34 smithi148 ceph-mon[35090]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-12T02:52:35.248 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:34 smithi148 ceph-mon[35090]: osdmap e31: 5 total, 4 up, 5 in 2024-09-12T02:52:35.248 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:34 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T02:52:35.248 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:34 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T02:52:35.248 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:34 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:52:35.248 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:34 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:52:35.248 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:34 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:52:35.248 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:34 smithi148 ceph-mon[35090]: from='osd.4 ' entity='osd.4' 2024-09-12T02:52:35.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:34 smithi005 ceph-mon[31197]: Detected new or changed devices on smithi146 2024-09-12T02:52:35.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:34 smithi005 ceph-mon[31197]: Adjusting osd_memory_target on smithi146 to 8544M 2024-09-12T02:52:35.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:34 smithi005 ceph-mon[31197]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-12T02:52:35.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:34 smithi005 ceph-mon[31197]: osdmap e31: 5 total, 4 up, 5 in 2024-09-12T02:52:35.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:34 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T02:52:35.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:34 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T02:52:35.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:34 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:52:35.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:34 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:52:35.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:34 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:52:35.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:34 smithi005 ceph-mon[31197]: from='osd.4 ' entity='osd.4' 2024-09-12T02:52:36.187 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:35 smithi148 ceph-mon[35090]: purged_snaps scrub starts 2024-09-12T02:52:36.187 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:35 smithi148 ceph-mon[35090]: purged_snaps scrub ok 2024-09-12T02:52:36.187 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:35 smithi148 ceph-mon[35090]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:52:36.187 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:35 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T02:52:36.188 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:35 smithi148 ceph-mon[35090]: osd.4 [v2:172.21.15.146:6808/3274923851,v1:172.21.15.146:6809/3274923851] boot 2024-09-12T02:52:36.188 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:35 smithi148 ceph-mon[35090]: osdmap e32: 5 total, 5 up, 5 in 2024-09-12T02:52:36.188 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:35 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T02:52:36.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:35 smithi146 ceph-mon[35011]: purged_snaps scrub starts 2024-09-12T02:52:36.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:35 smithi146 ceph-mon[35011]: purged_snaps scrub ok 2024-09-12T02:52:36.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:35 smithi146 ceph-mon[35011]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:52:36.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:35 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T02:52:36.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:35 smithi146 ceph-mon[35011]: osd.4 [v2:172.21.15.146:6808/3274923851,v1:172.21.15.146:6809/3274923851] boot 2024-09-12T02:52:36.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:35 smithi146 ceph-mon[35011]: osdmap e32: 5 total, 5 up, 5 in 2024-09-12T02:52:36.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:35 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T02:52:36.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:35 smithi005 ceph-mon[31197]: purged_snaps scrub starts 2024-09-12T02:52:36.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:35 smithi005 ceph-mon[31197]: purged_snaps scrub ok 2024-09-12T02:52:36.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:35 smithi005 ceph-mon[31197]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:52:36.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:35 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T02:52:36.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:35 smithi005 ceph-mon[31197]: osd.4 [v2:172.21.15.146:6808/3274923851,v1:172.21.15.146:6809/3274923851] boot 2024-09-12T02:52:36.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:35 smithi005 ceph-mon[31197]: osdmap e32: 5 total, 5 up, 5 in 2024-09-12T02:52:36.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:35 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-12T02:52:37.205 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:36 smithi148 ceph-mon[35090]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-12T02:52:37.205 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:36 smithi148 ceph-mon[35090]: from='client.? 172.21.15.148:0/1155107960' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "64ada7d4-70a0-4b83-9af8-032be544608d"}]: dispatch 2024-09-12T02:52:37.205 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:36 smithi148 ceph-mon[35090]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "64ada7d4-70a0-4b83-9af8-032be544608d"}]: dispatch 2024-09-12T02:52:37.205 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:36 smithi148 ceph-mon[35090]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "64ada7d4-70a0-4b83-9af8-032be544608d"}]': finished 2024-09-12T02:52:37.205 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:36 smithi148 ceph-mon[35090]: osdmap e33: 6 total, 5 up, 6 in 2024-09-12T02:52:37.205 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:36 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T02:52:37.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:36 smithi146 ceph-mon[35011]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-12T02:52:37.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:36 smithi146 ceph-mon[35011]: from='client.? 172.21.15.148:0/1155107960' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "64ada7d4-70a0-4b83-9af8-032be544608d"}]: dispatch 2024-09-12T02:52:37.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:36 smithi146 ceph-mon[35011]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "64ada7d4-70a0-4b83-9af8-032be544608d"}]: dispatch 2024-09-12T02:52:37.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:36 smithi146 ceph-mon[35011]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "64ada7d4-70a0-4b83-9af8-032be544608d"}]': finished 2024-09-12T02:52:37.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:36 smithi146 ceph-mon[35011]: osdmap e33: 6 total, 5 up, 6 in 2024-09-12T02:52:37.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:36 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T02:52:37.211 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:36 smithi005 ceph-mon[31197]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 112 MiB used, 357 GiB / 358 GiB avail 2024-09-12T02:52:37.211 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:36 smithi005 ceph-mon[31197]: from='client.? 172.21.15.148:0/1155107960' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "64ada7d4-70a0-4b83-9af8-032be544608d"}]: dispatch 2024-09-12T02:52:37.211 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:36 smithi005 ceph-mon[31197]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "64ada7d4-70a0-4b83-9af8-032be544608d"}]: dispatch 2024-09-12T02:52:37.212 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:36 smithi005 ceph-mon[31197]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "64ada7d4-70a0-4b83-9af8-032be544608d"}]': finished 2024-09-12T02:52:37.212 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:36 smithi005 ceph-mon[31197]: osdmap e33: 6 total, 5 up, 6 in 2024-09-12T02:52:37.212 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:36 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T02:52:38.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:37 smithi005 ceph-mon[31197]: from='client.? 172.21.15.148:0/3580497567' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:52:38.376 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:37 smithi146 ceph-mon[35011]: from='client.? 172.21.15.148:0/3580497567' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:52:38.415 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:38 smithi148 ceph-mon[35090]: from='client.? 172.21.15.148:0/3580497567' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:52:39.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:38 smithi005 ceph-mon[31197]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:52:39.332 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:38 smithi146 ceph-mon[35011]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:52:39.415 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:38 smithi148 ceph-mon[35090]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:52:41.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:40 smithi005 ceph-mon[31197]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:52:41.415 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:40 smithi148 ceph-mon[35090]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:52:41.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:40 smithi146 ceph-mon[35011]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:52:43.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:42 smithi005 ceph-mon[31197]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:52:43.410 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:42 smithi148 ceph-mon[35090]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:52:43.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:42 smithi146 ceph-mon[35011]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:52:45.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:45 smithi148 ceph-mon[35090]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:52:45.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:45 smithi005 ceph-mon[31197]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:52:45.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:45 smithi146 ceph-mon[35011]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:52:46.207 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:46 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-12T02:52:46.207 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:46 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:52:46.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:46 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-12T02:52:46.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:46 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:52:46.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:46 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-12T02:52:46.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:46 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:52:47.221 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:47 smithi148 ceph-mon[35090]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:52:47.221 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:47 smithi148 ceph-mon[35090]: Deploying daemon osd.5 on smithi148 2024-09-12T02:52:47.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:47 smithi005 ceph-mon[31197]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:52:47.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:47 smithi005 ceph-mon[31197]: Deploying daemon osd.5 on smithi148 2024-09-12T02:52:47.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:47 smithi146 ceph-mon[35011]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:52:47.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:47 smithi146 ceph-mon[35011]: Deploying daemon osd.5 on smithi148 2024-09-12T02:52:49.122 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:49 smithi148 ceph-mon[35090]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:52:49.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:49 smithi146 ceph-mon[35011]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:52:49.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:49 smithi005 ceph-mon[31197]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:52:51.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:51 smithi148 ceph-mon[35090]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:52:51.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:51 smithi005 ceph-mon[31197]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:52:51.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:51 smithi146 ceph-mon[35011]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:52:52.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:52 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:52:52.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:52 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:52:52.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:52 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:52:52.331 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:52 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:52:52.331 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:52 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:52:52.331 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:52 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:52:52.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:52 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:52:52.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:52 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:52:52.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:52 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:52:53.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:53 smithi148 ceph-mon[35090]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:52:53.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:53 smithi005 ceph-mon[31197]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:52:53.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:53 smithi146 ceph-mon[35011]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:52:55.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:54 smithi005 ceph-mon[31197]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:52:55.061 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:54 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:52:55.061 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:54 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:52:55.061 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:54 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:52:55.061 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:54 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:52:55.061 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:54 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:52:55.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:54 smithi148 ceph-mon[35090]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:52:55.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:54 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:52:55.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:54 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:52:55.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:54 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:52:55.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:54 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:52:55.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:54 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:52:55.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:54 smithi146 ceph-mon[35011]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:52:55.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:54 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:52:55.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:54 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:52:55.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:54 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:52:55.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:54 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:52:55.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:54 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:52:55.501 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:52:55.500+0000 7f31027fc640 1 -- 172.21.15.148:0/649442590 <== mgr.14148 v2:172.21.15.5:6800/3780631642 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f3104062630 con 0x7f30ec0a2c30 2024-09-12T02:52:55.501 INFO:teuthology.orchestra.run.smithi148.stdout:Created osd(s) 5 on host 'smithi148' 2024-09-12T02:52:55.503 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:52:55.503+0000 7f310b8e3640 1 -- 172.21.15.148:0/649442590 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f30ec0a2c30 msgr2=0x7f30ec0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:52:55.503 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:52:55.503+0000 7f310b8e3640 1 --2- 172.21.15.148:0/649442590 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f30ec0a2c30 0x7f30ec0a50f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3104102c60 tx=0x7f30f400a590 comp rx=0 tx=0).stop 2024-09-12T02:52:55.503 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:52:55.503+0000 7f310b8e3640 1 -- 172.21.15.148:0/649442590 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3104102e00 msgr2=0x7f31041983f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:52:55.503 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:52:55.503+0000 7f310b8e3640 1 --2- 172.21.15.148:0/649442590 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3104102e00 0x7f31041983f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f30f800e9f0 tx=0x7f30f800edf0 comp rx=0 tx=0).stop 2024-09-12T02:52:55.503 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:52:55.503+0000 7f310b8e3640 1 -- 172.21.15.148:0/649442590 shutdown_connections 2024-09-12T02:52:55.503 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:52:55.503+0000 7f310b8e3640 1 --2- 172.21.15.148:0/649442590 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f30ec0a2c30 0x7f30ec0a50f0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:55.504 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:52:55.503+0000 7f310b8e3640 1 --2- 172.21.15.148:0/649442590 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f31041037c0 0x7f310419f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:55.504 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:52:55.503+0000 7f310b8e3640 1 --2- 172.21.15.148:0/649442590 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3104102e00 0x7f31041983f0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:55.504 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:52:55.503+0000 7f310b8e3640 1 --2- 172.21.15.148:0/649442590 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3104101c00 0x7f3104197eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:55.504 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:52:55.503+0000 7f310b8e3640 1 -- 172.21.15.148:0/649442590 >> 172.21.15.148:0/649442590 conn(0x7f31040fd390 msgr2=0x7f31041061d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:52:55.504 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:52:55.503+0000 7f310b8e3640 1 -- 172.21.15.148:0/649442590 shutdown_connections 2024-09-12T02:52:55.504 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:52:55.503+0000 7f310b8e3640 1 -- 172.21.15.148:0/649442590 wait complete. 2024-09-12T02:52:56.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:55 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:52:56.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:55 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:52:56.061 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:55 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:52:56.105 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:55 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:52:56.106 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:55 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:52:56.106 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:55 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:52:56.131 DEBUG:teuthology.orchestra.run.smithi148:osd.5> sudo journalctl -f -n 0 -u ceph-618305f8-70b1-11ef-bcea-c7b262605968@osd.5.service 2024-09-12T02:52:56.134 INFO:tasks.cephadm:Deploying osd.6 on smithi005 with /dev/vg_nvme/lv_2... 2024-09-12T02:52:56.134 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-12T02:52:56.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:55 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:52:56.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:55 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:52:56.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:55 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:52:56.406 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:52:56.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:56 smithi005 ceph-mon[31197]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:52:57.078 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:56 smithi148 ceph-mon[35090]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:52:57.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:56 smithi146 ceph-mon[35011]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:52:58.132 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 12 02:52:57 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-5[46768]: 2024-09-12T02:52:57.867+0000 7fcd13310640 -1 osd.5 0 log_to_monitors true 2024-09-12T02:52:58.843 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-12T02:52:58.870 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi005:vg_nvme/lv_2 2024-09-12T02:52:59.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:58 smithi005 ceph-mon[31197]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:52:59.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:58 smithi005 ceph-mon[31197]: from='osd.5 [v2:172.21.15.148:6808/2771670476,v1:172.21.15.148:6809/2771670476]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-12T02:52:59.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:58 smithi005 ceph-mon[31197]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-12T02:52:59.061 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:58 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:52:59.061 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:58 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:52:59.061 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:58 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:52:59.061 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:58 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:52:59.061 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:58 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:52:59.061 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:58 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:52:59.061 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:58 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:52:59.125 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:52:59.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:58 smithi148 ceph-mon[35090]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:52:59.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:58 smithi148 ceph-mon[35090]: from='osd.5 [v2:172.21.15.148:6808/2771670476,v1:172.21.15.148:6809/2771670476]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-12T02:52:59.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:58 smithi148 ceph-mon[35090]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-12T02:52:59.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:58 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:52:59.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:58 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:52:59.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:58 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:52:59.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:58 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:52:59.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:58 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:52:59.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:58 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:52:59.167 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:58 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:52:59.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:58 smithi146 ceph-mon[35011]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:52:59.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:58 smithi146 ceph-mon[35011]: from='osd.5 [v2:172.21.15.148:6808/2771670476,v1:172.21.15.148:6809/2771670476]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-12T02:52:59.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:58 smithi146 ceph-mon[35011]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-12T02:52:59.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:58 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:52:59.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:58 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:52:59.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:58 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:52:59.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:58 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:52:59.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:58 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:52:59.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:58 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:52:59.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:58 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:52:59.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:52:59.554+0000 7f01d827f640 1 -- 172.21.15.5:0/2968533380 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01d0102660 msgr2=0x7f01d0104a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:52:59.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:52:59.554+0000 7f01d827f640 1 --2- 172.21.15.5:0/2968533380 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01d0102660 0x7f01d0104a50 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f01cc009900 tx=0x7f01cc02a410 comp rx=0 tx=0).stop 2024-09-12T02:52:59.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:52:59.554+0000 7f01d827f640 1 -- 172.21.15.5:0/2968533380 shutdown_connections 2024-09-12T02:52:59.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:52:59.554+0000 7f01d827f640 1 --2- 172.21.15.5:0/2968533380 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01d01078c0 0x7f01d0109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:59.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:52:59.554+0000 7f01d827f640 1 --2- 172.21.15.5:0/2968533380 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f01d0104f90 0x7f01d0107380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:59.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:52:59.554+0000 7f01d827f640 1 --2- 172.21.15.5:0/2968533380 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01d0102660 0x7f01d0104a50 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:59.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:52:59.554+0000 7f01d827f640 1 -- 172.21.15.5:0/2968533380 >> 172.21.15.5:0/2968533380 conn(0x7f01d00fc280 msgr2=0x7f01d00fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:52:59.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:52:59.554+0000 7f01d827f640 1 -- 172.21.15.5:0/2968533380 shutdown_connections 2024-09-12T02:52:59.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:52:59.554+0000 7f01d827f640 1 -- 172.21.15.5:0/2968533380 wait complete. 2024-09-12T02:52:59.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:52:59.555+0000 7f01d827f640 1 Processor -- start 2024-09-12T02:52:59.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:52:59.555+0000 7f01d827f640 1 -- start start 2024-09-12T02:52:59.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:52:59.555+0000 7f01d827f640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f01d0102660 0x7f01d0197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:52:59.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:52:59.555+0000 7f01d827f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01d0104f90 0x7f01d01984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:52:59.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:52:59.555+0000 7f01d827f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01d01078c0 0x7f01d019f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:52:59.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:52:59.555+0000 7f01d827f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f01d010d900 con 0x7f01d01078c0 2024-09-12T02:52:59.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:52:59.555+0000 7f01d827f640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f01d010d780 con 0x7f01d0102660 2024-09-12T02:52:59.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:52:59.555+0000 7f01d827f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f01d010da80 con 0x7f01d0104f90 2024-09-12T02:52:59.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:52:59.555+0000 7f01d681c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01d01078c0 0x7f01d019f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:52:59.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:52:59.555+0000 7f01d681c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01d01078c0 0x7f01d019f540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50488/0 (socket says 172.21.15.5:50488) 2024-09-12T02:52:59.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:52:59.555+0000 7f01d681c640 1 -- 172.21.15.5:0/4288027557 learned_addr learned my addr 172.21.15.5:0/4288027557 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:52:59.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:52:59.555+0000 7f01d601b640 1 --2- 172.21.15.5:0/4288027557 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f01d0102660 0x7f01d0197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:52:59.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:52:59.555+0000 7f01d581a640 1 --2- 172.21.15.5:0/4288027557 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01d0104f90 0x7f01d01984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:52:59.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:52:59.555+0000 7f01d681c640 1 -- 172.21.15.5:0/4288027557 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01d0104f90 msgr2=0x7f01d01984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:52:59.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:52:59.555+0000 7f01d681c640 1 --2- 172.21.15.5:0/4288027557 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01d0104f90 0x7f01d01984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:59.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:52:59.555+0000 7f01d681c640 1 -- 172.21.15.5:0/4288027557 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f01d0102660 msgr2=0x7f01d0197f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:52:59.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:52:59.555+0000 7f01d681c640 1 --2- 172.21.15.5:0/4288027557 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f01d0102660 0x7f01d0197f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:52:59.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:52:59.555+0000 7f01d681c640 1 -- 172.21.15.5:0/4288027557 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01d019fbb0 con 0x7f01d01078c0 2024-09-12T02:52:59.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:52:59.556+0000 7f01d681c640 1 --2- 172.21.15.5:0/4288027557 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01d01078c0 0x7f01d019f540 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f01c800eaa0 tx=0x7f01c800eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:52:59.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:52:59.556+0000 7f01c77fe640 1 -- 172.21.15.5:0/4288027557 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01c8004d20 con 0x7f01d01078c0 2024-09-12T02:52:59.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:52:59.556+0000 7f01c77fe640 1 -- 172.21.15.5:0/4288027557 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01c8005020 con 0x7f01d01078c0 2024-09-12T02:52:59.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:52:59.556+0000 7f01d827f640 1 -- 172.21.15.5:0/4288027557 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01d019fea0 con 0x7f01d01078c0 2024-09-12T02:52:59.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:52:59.556+0000 7f01c77fe640 1 -- 172.21.15.5:0/4288027557 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01c80116d0 con 0x7f01d01078c0 2024-09-12T02:52:59.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:52:59.556+0000 7f01d827f640 1 -- 172.21.15.5:0/4288027557 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01d01a0420 con 0x7f01d01078c0 2024-09-12T02:52:59.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:52:59.557+0000 7f01d827f640 1 -- 172.21.15.5:0/4288027557 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01d00fdd70 con 0x7f01d01078c0 2024-09-12T02:52:59.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:52:59.558+0000 7f01c77fe640 1 -- 172.21.15.5:0/4288027557 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f01c80051d0 con 0x7f01d01078c0 2024-09-12T02:52:59.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:52:59.559+0000 7f01c77fe640 1 --2- 172.21.15.5:0/4288027557 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f01bc0a2c30 0x7f01bc0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:52:59.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:52:59.559+0000 7f01c77fe640 1 -- 172.21.15.5:0/4288027557 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 4237+0+0 (secure 0 0 0) 0x7f01c80ce460 con 0x7f01d01078c0 2024-09-12T02:52:59.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:52:59.559+0000 7f01d601b640 1 --2- 172.21.15.5:0/4288027557 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f01bc0a2c30 0x7f01bc0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:52:59.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:52:59.559+0000 7f01d601b640 1 --2- 172.21.15.5:0/4288027557 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f01bc0a2c30 0x7f01bc0a50f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f01cc0314e0 tx=0x7f01cc000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:52:59.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:52:59.563+0000 7f01c77fe640 1 -- 172.21.15.5:0/4288027557 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f01c8016030 con 0x7f01d01078c0 2024-09-12T02:52:59.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:52:59.747+0000 7f01d827f640 1 -- 172.21.15.5:0/4288027557 --> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f01d0062630 con 0x7f01bc0a2c30 2024-09-12T02:52:59.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:59 smithi005 ceph-mon[31197]: Detected new or changed devices on smithi148 2024-09-12T02:52:59.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:59 smithi005 ceph-mon[31197]: Adjusting osd_memory_target on smithi148 to 8544M 2024-09-12T02:52:59.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:59 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:52:59.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:59 smithi005 ceph-mon[31197]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-12T02:52:59.811 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:59 smithi005 ceph-mon[31197]: from='osd.5 [v2:172.21.15.148:6808/2771670476,v1:172.21.15.148:6809/2771670476]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-09-12T02:52:59.811 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:59 smithi005 ceph-mon[31197]: osdmap e34: 6 total, 5 up, 6 in 2024-09-12T02:52:59.811 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:59 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T02:52:59.811 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:59 smithi005 ceph-mon[31197]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-09-12T02:52:59.811 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:59 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:52:59.811 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:59 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:52:59.811 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:52:59 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:53:00.166 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 12 02:52:59 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-5[46768]: 2024-09-12T02:52:59.778+0000 7fcd0ea70640 -1 osd.5 0 waiting for initial osdmap 2024-09-12T02:53:00.166 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 12 02:52:59 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-5[46768]: 2024-09-12T02:52:59.785+0000 7fcd0b0aa640 -1 osd.5 35 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T02:53:00.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:59 smithi148 ceph-mon[35090]: Detected new or changed devices on smithi148 2024-09-12T02:53:00.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:59 smithi148 ceph-mon[35090]: Adjusting osd_memory_target on smithi148 to 8544M 2024-09-12T02:53:00.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:59 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:53:00.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:59 smithi148 ceph-mon[35090]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-12T02:53:00.167 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:59 smithi148 ceph-mon[35090]: from='osd.5 [v2:172.21.15.148:6808/2771670476,v1:172.21.15.148:6809/2771670476]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-09-12T02:53:00.167 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:59 smithi148 ceph-mon[35090]: osdmap e34: 6 total, 5 up, 6 in 2024-09-12T02:53:00.167 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:59 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T02:53:00.167 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:59 smithi148 ceph-mon[35090]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-09-12T02:53:00.167 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:59 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:53:00.167 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:59 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:53:00.167 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:52:59 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:53:00.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:59 smithi146 ceph-mon[35011]: Detected new or changed devices on smithi148 2024-09-12T02:53:00.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:59 smithi146 ceph-mon[35011]: Adjusting osd_memory_target on smithi148 to 8544M 2024-09-12T02:53:00.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:59 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:53:00.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:59 smithi146 ceph-mon[35011]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-12T02:53:00.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:59 smithi146 ceph-mon[35011]: from='osd.5 [v2:172.21.15.148:6808/2771670476,v1:172.21.15.148:6809/2771670476]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-09-12T02:53:00.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:59 smithi146 ceph-mon[35011]: osdmap e34: 6 total, 5 up, 6 in 2024-09-12T02:53:00.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:59 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T02:53:00.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:59 smithi146 ceph-mon[35011]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-09-12T02:53:00.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:59 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:53:00.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:59 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:53:00.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:52:59 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:53:01.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:00 smithi005 ceph-mon[31197]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:53:01.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:00 smithi005 ceph-mon[31197]: from='client.14349 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:53:01.061 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:00 smithi005 ceph-mon[31197]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2024-09-12T02:53:01.061 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:00 smithi005 ceph-mon[31197]: osdmap e35: 6 total, 5 up, 6 in 2024-09-12T02:53:01.061 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:00 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T02:53:01.061 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:00 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T02:53:01.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:00 smithi148 ceph-mon[35090]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:53:01.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:00 smithi148 ceph-mon[35090]: from='client.14349 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:53:01.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:00 smithi148 ceph-mon[35090]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2024-09-12T02:53:01.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:00 smithi148 ceph-mon[35090]: osdmap e35: 6 total, 5 up, 6 in 2024-09-12T02:53:01.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:00 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T02:53:01.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:00 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T02:53:01.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:00 smithi146 ceph-mon[35011]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:53:01.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:00 smithi146 ceph-mon[35011]: from='client.14349 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:53:01.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:00 smithi146 ceph-mon[35011]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2024-09-12T02:53:01.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:00 smithi146 ceph-mon[35011]: osdmap e35: 6 total, 5 up, 6 in 2024-09-12T02:53:01.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:00 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T02:53:01.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:00 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T02:53:02.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:01 smithi005 ceph-mon[31197]: purged_snaps scrub starts 2024-09-12T02:53:02.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:01 smithi005 ceph-mon[31197]: purged_snaps scrub ok 2024-09-12T02:53:02.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:01 smithi005 ceph-mon[31197]: osd.5 [v2:172.21.15.148:6808/2771670476,v1:172.21.15.148:6809/2771670476] boot 2024-09-12T02:53:02.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:01 smithi005 ceph-mon[31197]: osdmap e36: 6 total, 6 up, 6 in 2024-09-12T02:53:02.061 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:01 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T02:53:02.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:01 smithi148 ceph-mon[35090]: purged_snaps scrub starts 2024-09-12T02:53:02.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:01 smithi148 ceph-mon[35090]: purged_snaps scrub ok 2024-09-12T02:53:02.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:01 smithi148 ceph-mon[35090]: osd.5 [v2:172.21.15.148:6808/2771670476,v1:172.21.15.148:6809/2771670476] boot 2024-09-12T02:53:02.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:01 smithi148 ceph-mon[35090]: osdmap e36: 6 total, 6 up, 6 in 2024-09-12T02:53:02.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:01 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T02:53:02.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:01 smithi146 ceph-mon[35011]: purged_snaps scrub starts 2024-09-12T02:53:02.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:01 smithi146 ceph-mon[35011]: purged_snaps scrub ok 2024-09-12T02:53:02.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:01 smithi146 ceph-mon[35011]: osd.5 [v2:172.21.15.148:6808/2771670476,v1:172.21.15.148:6809/2771670476] boot 2024-09-12T02:53:02.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:01 smithi146 ceph-mon[35011]: osdmap e36: 6 total, 6 up, 6 in 2024-09-12T02:53:02.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:01 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-12T02:53:03.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:02 smithi005 ceph-mon[31197]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:53:03.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:02 smithi005 ceph-mon[31197]: osdmap e37: 6 total, 6 up, 6 in 2024-09-12T02:53:03.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:02 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/2681609990' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a70a88fd-2e4b-480c-bb98-62636cf70cc6"}]: dispatch 2024-09-12T02:53:03.061 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:02 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/2681609990' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a70a88fd-2e4b-480c-bb98-62636cf70cc6"}]': finished 2024-09-12T02:53:03.061 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:02 smithi005 ceph-mon[31197]: osdmap e38: 7 total, 6 up, 7 in 2024-09-12T02:53:03.061 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:02 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T02:53:03.061 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:02 smithi005 ceph-mon[31197]: osdmap e39: 7 total, 6 up, 7 in 2024-09-12T02:53:03.061 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:02 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T02:53:03.061 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:02 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/532500478' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:53:03.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:02 smithi148 ceph-mon[35090]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:53:03.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:02 smithi148 ceph-mon[35090]: osdmap e37: 6 total, 6 up, 6 in 2024-09-12T02:53:03.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:02 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/2681609990' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a70a88fd-2e4b-480c-bb98-62636cf70cc6"}]: dispatch 2024-09-12T02:53:03.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:02 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/2681609990' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a70a88fd-2e4b-480c-bb98-62636cf70cc6"}]': finished 2024-09-12T02:53:03.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:02 smithi148 ceph-mon[35090]: osdmap e38: 7 total, 6 up, 7 in 2024-09-12T02:53:03.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:02 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T02:53:03.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:02 smithi148 ceph-mon[35090]: osdmap e39: 7 total, 6 up, 7 in 2024-09-12T02:53:03.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:02 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T02:53:03.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:02 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/532500478' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:53:03.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:02 smithi146 ceph-mon[35011]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-12T02:53:03.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:02 smithi146 ceph-mon[35011]: osdmap e37: 6 total, 6 up, 6 in 2024-09-12T02:53:03.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:02 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/2681609990' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a70a88fd-2e4b-480c-bb98-62636cf70cc6"}]: dispatch 2024-09-12T02:53:03.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:02 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/2681609990' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a70a88fd-2e4b-480c-bb98-62636cf70cc6"}]': finished 2024-09-12T02:53:03.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:02 smithi146 ceph-mon[35011]: osdmap e38: 7 total, 6 up, 7 in 2024-09-12T02:53:03.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:02 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T02:53:03.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:02 smithi146 ceph-mon[35011]: osdmap e39: 7 total, 6 up, 7 in 2024-09-12T02:53:03.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:02 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T02:53:03.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:02 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/532500478' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:53:04.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:04 smithi005 ceph-mon[31197]: pgmap v136: 1 pgs: 1 remapped+peering; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-12T02:53:04.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:04 smithi005 ceph-mon[31197]: osdmap e40: 7 total, 6 up, 7 in 2024-09-12T02:53:04.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:04 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T02:53:04.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:04 smithi148 ceph-mon[35090]: pgmap v136: 1 pgs: 1 remapped+peering; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-12T02:53:04.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:04 smithi148 ceph-mon[35090]: osdmap e40: 7 total, 6 up, 7 in 2024-09-12T02:53:04.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:04 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T02:53:04.956 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:04 smithi146 ceph-mon[35011]: pgmap v136: 1 pgs: 1 remapped+peering; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-12T02:53:04.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:04 smithi146 ceph-mon[35011]: osdmap e40: 7 total, 6 up, 7 in 2024-09-12T02:53:04.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:04 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T02:53:06.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:06 smithi005 ceph-mon[31197]: pgmap v138: 1 pgs: 1 remapped+peering; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-12T02:53:06.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:06 smithi148 ceph-mon[35090]: pgmap v138: 1 pgs: 1 remapped+peering; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-12T02:53:06.956 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:06 smithi146 ceph-mon[35011]: pgmap v138: 1 pgs: 1 remapped+peering; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-12T02:53:08.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:08 smithi005 ceph-mon[31197]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-09-12T02:53:08.915 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:08 smithi148 ceph-mon[35090]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-09-12T02:53:08.956 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:08 smithi146 ceph-mon[35011]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-09-12T02:53:10.783 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:10 smithi005 ceph-mon[31197]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-09-12T02:53:10.783 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:10 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-12T02:53:10.783 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:10 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:53:10.783 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:10 smithi005 ceph-mon[31197]: Deploying daemon osd.6 on smithi005 2024-09-12T02:53:10.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:10 smithi148 ceph-mon[35090]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-09-12T02:53:10.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:10 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-12T02:53:10.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:10 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:53:10.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:10 smithi148 ceph-mon[35090]: Deploying daemon osd.6 on smithi005 2024-09-12T02:53:10.956 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:10 smithi146 ceph-mon[35011]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-09-12T02:53:10.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:10 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-12T02:53:10.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:10 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:53:10.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:10 smithi146 ceph-mon[35011]: Deploying daemon osd.6 on smithi005 2024-09-12T02:53:12.796 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:12 smithi005 ceph-mon[31197]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-09-12T02:53:12.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:12 smithi148 ceph-mon[35090]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-09-12T02:53:12.956 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:12 smithi146 ceph-mon[35011]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-09-12T02:53:14.599 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:14 smithi005 ceph-mon[31197]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-12T02:53:14.915 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:14 smithi148 ceph-mon[35090]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-12T02:53:14.956 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:14 smithi146 ceph-mon[35011]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-12T02:53:16.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:16 smithi005 ceph-mon[31197]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-12T02:53:16.811 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:16 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:53:16.811 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:16 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:53:16.811 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:16 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:53:16.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:16 smithi148 ceph-mon[35090]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-12T02:53:16.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:16 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:53:16.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:16 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:53:16.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:16 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:53:16.956 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:16 smithi146 ceph-mon[35011]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-12T02:53:16.956 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:16 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:53:16.956 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:16 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:53:16.956 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:16 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:53:19.265 INFO:teuthology.orchestra.run.smithi005.stdout:Created osd(s) 6 on host 'smithi005' 2024-09-12T02:53:19.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:53:19.265+0000 7f01c77fe640 1 -- 172.21.15.5:0/4288027557 <== mgr.14148 v2:172.21.15.5:6800/3780631642 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f01d0062630 con 0x7f01bc0a2c30 2024-09-12T02:53:19.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:53:19.267+0000 7f01d827f640 1 -- 172.21.15.5:0/4288027557 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f01bc0a2c30 msgr2=0x7f01bc0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:53:19.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:53:19.267+0000 7f01d827f640 1 --2- 172.21.15.5:0/4288027557 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f01bc0a2c30 0x7f01bc0a50f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f01cc0314e0 tx=0x7f01cc000bc0 comp rx=0 tx=0).stop 2024-09-12T02:53:19.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:53:19.267+0000 7f01d827f640 1 -- 172.21.15.5:0/4288027557 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01d01078c0 msgr2=0x7f01d019f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:53:19.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:53:19.267+0000 7f01d827f640 1 --2- 172.21.15.5:0/4288027557 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01d01078c0 0x7f01d019f540 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f01c800eaa0 tx=0x7f01c800eea0 comp rx=0 tx=0).stop 2024-09-12T02:53:19.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:53:19.267+0000 7f01d827f640 1 -- 172.21.15.5:0/4288027557 shutdown_connections 2024-09-12T02:53:19.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:53:19.267+0000 7f01d827f640 1 --2- 172.21.15.5:0/4288027557 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f01bc0a2c30 0x7f01bc0a50f0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:19.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:53:19.267+0000 7f01d827f640 1 --2- 172.21.15.5:0/4288027557 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01d01078c0 0x7f01d019f540 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:19.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:53:19.267+0000 7f01d827f640 1 --2- 172.21.15.5:0/4288027557 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f01d0104f90 0x7f01d01984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:19.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:53:19.267+0000 7f01d827f640 1 --2- 172.21.15.5:0/4288027557 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f01d0102660 0x7f01d0197f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:19.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:53:19.267+0000 7f01d827f640 1 -- 172.21.15.5:0/4288027557 >> 172.21.15.5:0/4288027557 conn(0x7f01d00fc280 msgr2=0x7f01d01030a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:53:19.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:53:19.267+0000 7f01d827f640 1 -- 172.21.15.5:0/4288027557 shutdown_connections 2024-09-12T02:53:19.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:53:19.267+0000 7f01d827f640 1 -- 172.21.15.5:0/4288027557 wait complete. 2024-09-12T02:53:19.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:19 smithi005 ceph-mon[31197]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-12T02:53:19.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:19 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:53:19.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:19 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:53:19.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:19 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:53:19.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:19 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:53:19.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:19 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:53:19.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:19 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:53:19.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:19 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:53:19.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:19 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:53:19.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:19 smithi148 ceph-mon[35090]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-12T02:53:19.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:19 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:53:19.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:19 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:53:19.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:19 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:53:19.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:19 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:53:19.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:19 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:53:19.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:19 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:53:19.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:19 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:53:19.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:19 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:53:19.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:19 smithi146 ceph-mon[35011]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-12T02:53:19.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:19 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:53:19.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:19 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:53:19.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:19 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:53:19.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:19 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:53:19.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:19 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:53:19.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:19 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:53:19.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:19 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:53:19.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:19 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:53:19.927 DEBUG:teuthology.orchestra.run.smithi005:osd.6> sudo journalctl -f -n 0 -u ceph-618305f8-70b1-11ef-bcea-c7b262605968@osd.6.service 2024-09-12T02:53:19.930 INFO:tasks.cephadm:Deploying osd.7 on smithi146 with /dev/vg_nvme/lv_2... 2024-09-12T02:53:19.930 DEBUG:teuthology.orchestra.run.smithi146:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-12T02:53:20.186 INFO:teuthology.orchestra.run.smithi146.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T02:53:21.418 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:21 smithi005 ceph-mon[31197]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-12T02:53:21.559 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:21 smithi146 ceph-mon[35011]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-12T02:53:21.665 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:21 smithi148 ceph-mon[35090]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-12T02:53:22.444 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:22 smithi005 ceph-mon[31197]: from='osd.6 [v2:172.21.15.5:6818/3176073169,v1:172.21.15.5:6819/3176073169]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-12T02:53:22.444 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 12 02:53:22 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-6[56400]: 2024-09-12T02:53:22.106+0000 7f8e50db0640 -1 osd.6 0 log_to_monitors true 2024-09-12T02:53:22.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:22 smithi146 ceph-mon[35011]: from='osd.6 [v2:172.21.15.5:6818/3176073169,v1:172.21.15.5:6819/3176073169]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-12T02:53:22.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:22 smithi148 ceph-mon[35090]: from='osd.6 [v2:172.21.15.5:6818/3176073169,v1:172.21.15.5:6819/3176073169]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-12T02:53:22.748 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-12T02:53:22.775 DEBUG:teuthology.orchestra.run.smithi146:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi146:vg_nvme/lv_2 2024-09-12T02:53:23.028 INFO:teuthology.orchestra.run.smithi146.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T02:53:23.431 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:23 smithi146 ceph-mon[35011]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-12T02:53:23.431 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:23 smithi146 ceph-mon[35011]: from='osd.6 [v2:172.21.15.5:6818/3176073169,v1:172.21.15.5:6819/3176073169]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-12T02:53:23.431 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:23 smithi146 ceph-mon[35011]: osdmap e41: 7 total, 6 up, 7 in 2024-09-12T02:53:23.431 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:23 smithi146 ceph-mon[35011]: from='osd.6 [v2:172.21.15.5:6818/3176073169,v1:172.21.15.5:6819/3176073169]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2024-09-12T02:53:23.432 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:23 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T02:53:23.432 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:23 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:53:23.432 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:23 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:53:23.432 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:23 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:53:23.432 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:23 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:53:23.432 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:23 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:53:23.432 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:23 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:53:23.432 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:23 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:53:23.432 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:23 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:53:23.432 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:23 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:53:23.531 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:53:23.514+0000 7efd91e81640 1 -- 172.21.15.146:0/727936986 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efd8c0ffed0 msgr2=0x7efd8c1002d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:53:23.531 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:53:23.514+0000 7efd91e81640 1 --2- 172.21.15.146:0/727936986 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efd8c0ffed0 0x7efd8c1002d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7efd7c009900 tx=0x7efd7c02a410 comp rx=0 tx=0).stop 2024-09-12T02:53:23.531 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:53:23.514+0000 7efd91e81640 1 -- 172.21.15.146:0/727936986 shutdown_connections 2024-09-12T02:53:23.531 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:53:23.514+0000 7efd91e81640 1 --2- 172.21.15.146:0/727936986 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd8c102eb0 0x7efd8c103330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:23.532 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:53:23.514+0000 7efd91e81640 1 --2- 172.21.15.146:0/727936986 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efd8c103870 0x7efd8c10a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:23.532 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:53:23.514+0000 7efd91e81640 1 --2- 172.21.15.146:0/727936986 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efd8c0ffed0 0x7efd8c1002d0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:23.532 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:53:23.514+0000 7efd91e81640 1 -- 172.21.15.146:0/727936986 >> 172.21.15.146:0/727936986 conn(0x7efd8c0fba40 msgr2=0x7efd8c0fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:53:23.532 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:53:23.514+0000 7efd91e81640 1 -- 172.21.15.146:0/727936986 shutdown_connections 2024-09-12T02:53:23.532 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:53:23.514+0000 7efd91e81640 1 -- 172.21.15.146:0/727936986 wait complete. 2024-09-12T02:53:23.532 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:53:23.515+0000 7efd91e81640 1 Processor -- start 2024-09-12T02:53:23.532 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:53:23.515+0000 7efd91e81640 1 -- start start 2024-09-12T02:53:23.532 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:53:23.515+0000 7efd91e81640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efd8c0ffed0 0x7efd8c102810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:23.532 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:53:23.515+0000 7efd91e81640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efd8c102eb0 0x7efd8c100e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:23.532 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:53:23.515+0000 7efd91e81640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd8c103870 0x7efd8c1013a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:23.532 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:53:23.515+0000 7efd91e81640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efd8c10d890 con 0x7efd8c103870 2024-09-12T02:53:23.532 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:53:23.515+0000 7efd91e81640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7efd8c10d710 con 0x7efd8c0ffed0 2024-09-12T02:53:23.533 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:53:23.515+0000 7efd91e81640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7efd8c10da10 con 0x7efd8c102eb0 2024-09-12T02:53:23.533 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:53:23.515+0000 7efd8b7fe640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efd8c0ffed0 0x7efd8c102810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:23.533 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:53:23.515+0000 7efd8b7fe640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efd8c0ffed0 0x7efd8c102810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:36562/0 (socket says 172.21.15.146:36562) 2024-09-12T02:53:23.533 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:53:23.515+0000 7efd8b7fe640 1 -- 172.21.15.146:0/2718149263 learned_addr learned my addr 172.21.15.146:0/2718149263 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-12T02:53:23.533 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:53:23.516+0000 7efd8affd640 1 --2- 172.21.15.146:0/2718149263 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efd8c102eb0 0x7efd8c100e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:23.533 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:53:23.516+0000 7efd8bfff640 1 --2- 172.21.15.146:0/2718149263 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd8c103870 0x7efd8c1013a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:23.533 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:53:23.516+0000 7efd8b7fe640 1 -- 172.21.15.146:0/2718149263 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efd8c102eb0 msgr2=0x7efd8c100e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:53:23.533 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:53:23.516+0000 7efd8b7fe640 1 --2- 172.21.15.146:0/2718149263 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efd8c102eb0 0x7efd8c100e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:23.533 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:53:23.516+0000 7efd8b7fe640 1 -- 172.21.15.146:0/2718149263 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd8c103870 msgr2=0x7efd8c1013a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:53:23.533 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:53:23.516+0000 7efd8b7fe640 1 --2- 172.21.15.146:0/2718149263 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd8c103870 0x7efd8c1013a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:23.533 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:53:23.516+0000 7efd8b7fe640 1 -- 172.21.15.146:0/2718149263 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd8c101c60 con 0x7efd8c0ffed0 2024-09-12T02:53:23.533 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:53:23.516+0000 7efd8bfff640 1 --2- 172.21.15.146:0/2718149263 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd8c103870 0x7efd8c1013a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:53:23.534 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:53:23.516+0000 7efd8b7fe640 1 --2- 172.21.15.146:0/2718149263 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efd8c0ffed0 0x7efd8c102810 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7efd7c006010 tx=0x7efd7c032c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:53:23.534 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:53:23.516+0000 7efd88ff9640 1 -- 172.21.15.146:0/2718149263 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd7c03f030 con 0x7efd8c0ffed0 2024-09-12T02:53:23.534 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:53:23.516+0000 7efd88ff9640 1 -- 172.21.15.146:0/2718149263 <== mon.2 v2:172.21.15.146:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd7c003ce0 con 0x7efd8c0ffed0 2024-09-12T02:53:23.534 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:53:23.516+0000 7efd88ff9640 1 -- 172.21.15.146:0/2718149263 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd7c035070 con 0x7efd8c0ffed0 2024-09-12T02:53:23.534 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:53:23.516+0000 7efd91e81640 1 -- 172.21.15.146:0/2718149263 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd8c1a2050 con 0x7efd8c0ffed0 2024-09-12T02:53:23.534 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:53:23.516+0000 7efd91e81640 1 -- 172.21.15.146:0/2718149263 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd8c1a2500 con 0x7efd8c0ffed0 2024-09-12T02:53:23.534 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:53:23.517+0000 7efd91e81640 1 -- 172.21.15.146:0/2718149263 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd8c1090f0 con 0x7efd8c0ffed0 2024-09-12T02:53:23.534 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:53:23.518+0000 7efd88ff9640 1 -- 172.21.15.146:0/2718149263 <== mon.2 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7efd7c042030 con 0x7efd8c0ffed0 2024-09-12T02:53:23.534 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:53:23.519+0000 7efd88ff9640 1 --2- 172.21.15.146:0/2718149263 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7efd740a2b80 0x7efd740a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:23.534 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:53:23.519+0000 7efd88ff9640 1 -- 172.21.15.146:0/2718149263 <== mon.2 v2:172.21.15.146:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 4685+0+0 (secure 0 0 0) 0x7efd7c0ee900 con 0x7efd8c0ffed0 2024-09-12T02:53:23.534 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:53:23.519+0000 7efd8affd640 1 --2- 172.21.15.146:0/2718149263 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7efd740a2b80 0x7efd740a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:23.534 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:53:23.520+0000 7efd8affd640 1 --2- 172.21.15.146:0/2718149263 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7efd740a2b80 0x7efd740a5040 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7efd80005e10 tx=0x7efd80005da0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:53:23.534 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:53:23.523+0000 7efd88ff9640 1 -- 172.21.15.146:0/2718149263 <== mon.2 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efd7c0bc690 con 0x7efd8c0ffed0 2024-09-12T02:53:23.560 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 12 02:53:23 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-6[56400]: 2024-09-12T02:53:23.280+0000 7f8e4c510640 -1 osd.6 0 waiting for initial osdmap 2024-09-12T02:53:23.560 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 12 02:53:23 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-6[56400]: 2024-09-12T02:53:23.293+0000 7f8e48b4a640 -1 osd.6 42 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T02:53:23.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:23 smithi005 ceph-mon[31197]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-12T02:53:23.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:23 smithi005 ceph-mon[31197]: from='osd.6 [v2:172.21.15.5:6818/3176073169,v1:172.21.15.5:6819/3176073169]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-12T02:53:23.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:23 smithi005 ceph-mon[31197]: osdmap e41: 7 total, 6 up, 7 in 2024-09-12T02:53:23.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:23 smithi005 ceph-mon[31197]: from='osd.6 [v2:172.21.15.5:6818/3176073169,v1:172.21.15.5:6819/3176073169]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2024-09-12T02:53:23.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:23 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T02:53:23.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:23 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:53:23.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:23 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:53:23.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:23 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:53:23.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:23 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:53:23.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:23 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:53:23.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:23 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:53:23.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:23 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:53:23.562 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:23 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:53:23.562 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:23 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:53:23.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:23 smithi148 ceph-mon[35090]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-12T02:53:23.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:23 smithi148 ceph-mon[35090]: from='osd.6 [v2:172.21.15.5:6818/3176073169,v1:172.21.15.5:6819/3176073169]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-12T02:53:23.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:23 smithi148 ceph-mon[35090]: osdmap e41: 7 total, 6 up, 7 in 2024-09-12T02:53:23.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:23 smithi148 ceph-mon[35090]: from='osd.6 [v2:172.21.15.5:6818/3176073169,v1:172.21.15.5:6819/3176073169]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2024-09-12T02:53:23.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:23 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T02:53:23.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:23 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:53:23.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:23 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:53:23.667 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:23 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:53:23.667 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:23 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:53:23.667 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:23 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:53:23.667 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:23 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:53:23.667 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:23 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:53:23.667 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:23 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:53:23.667 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:23 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:53:23.701 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:53:23.700+0000 7efd91e81640 1 -- 172.21.15.146:0/2718149263 --> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi146:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7efd8c062630 con 0x7efd740a2b80 2024-09-12T02:53:24.409 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:24 smithi146 ceph-mon[35011]: Detected new or changed devices on smithi005 2024-09-12T02:53:24.409 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:24 smithi146 ceph-mon[35011]: Adjusting osd_memory_target on smithi005 to 5697M 2024-09-12T02:53:24.409 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:24 smithi146 ceph-mon[35011]: from='osd.6 [v2:172.21.15.5:6818/3176073169,v1:172.21.15.5:6819/3176073169]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2024-09-12T02:53:24.410 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:24 smithi146 ceph-mon[35011]: osdmap e42: 7 total, 6 up, 7 in 2024-09-12T02:53:24.410 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:24 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T02:53:24.410 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:24 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T02:53:24.410 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:24 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:53:24.410 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:24 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:53:24.410 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:24 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:53:24.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:24 smithi005 ceph-mon[31197]: Detected new or changed devices on smithi005 2024-09-12T02:53:24.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:24 smithi005 ceph-mon[31197]: Adjusting osd_memory_target on smithi005 to 5697M 2024-09-12T02:53:24.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:24 smithi005 ceph-mon[31197]: from='osd.6 [v2:172.21.15.5:6818/3176073169,v1:172.21.15.5:6819/3176073169]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2024-09-12T02:53:24.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:24 smithi005 ceph-mon[31197]: osdmap e42: 7 total, 6 up, 7 in 2024-09-12T02:53:24.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:24 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T02:53:24.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:24 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T02:53:24.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:24 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:53:24.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:24 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:53:24.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:24 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:53:24.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:24 smithi148 ceph-mon[35090]: Detected new or changed devices on smithi005 2024-09-12T02:53:24.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:24 smithi148 ceph-mon[35090]: Adjusting osd_memory_target on smithi005 to 5697M 2024-09-12T02:53:24.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:24 smithi148 ceph-mon[35090]: from='osd.6 [v2:172.21.15.5:6818/3176073169,v1:172.21.15.5:6819/3176073169]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2024-09-12T02:53:24.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:24 smithi148 ceph-mon[35090]: osdmap e42: 7 total, 6 up, 7 in 2024-09-12T02:53:24.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:24 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T02:53:24.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:24 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T02:53:24.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:24 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:53:24.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:24 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:53:24.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:24 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:53:25.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:25 smithi005 ceph-mon[31197]: purged_snaps scrub starts 2024-09-12T02:53:25.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:25 smithi005 ceph-mon[31197]: purged_snaps scrub ok 2024-09-12T02:53:25.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:25 smithi005 ceph-mon[31197]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-12T02:53:25.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:25 smithi005 ceph-mon[31197]: from='client.24263 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi146:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:53:25.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:25 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T02:53:25.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:25 smithi005 ceph-mon[31197]: osd.6 [v2:172.21.15.5:6818/3176073169,v1:172.21.15.5:6819/3176073169] boot 2024-09-12T02:53:25.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:25 smithi005 ceph-mon[31197]: osdmap e43: 7 total, 7 up, 7 in 2024-09-12T02:53:25.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:25 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T02:53:25.646 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:25 smithi146 ceph-mon[35011]: purged_snaps scrub starts 2024-09-12T02:53:25.646 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:25 smithi146 ceph-mon[35011]: purged_snaps scrub ok 2024-09-12T02:53:25.646 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:25 smithi146 ceph-mon[35011]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-12T02:53:25.647 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:25 smithi146 ceph-mon[35011]: from='client.24263 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi146:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:53:25.647 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:25 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T02:53:25.647 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:25 smithi146 ceph-mon[35011]: osd.6 [v2:172.21.15.5:6818/3176073169,v1:172.21.15.5:6819/3176073169] boot 2024-09-12T02:53:25.647 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:25 smithi146 ceph-mon[35011]: osdmap e43: 7 total, 7 up, 7 in 2024-09-12T02:53:25.647 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:25 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T02:53:25.665 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:25 smithi148 ceph-mon[35090]: purged_snaps scrub starts 2024-09-12T02:53:25.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:25 smithi148 ceph-mon[35090]: purged_snaps scrub ok 2024-09-12T02:53:25.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:25 smithi148 ceph-mon[35090]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-12T02:53:25.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:25 smithi148 ceph-mon[35090]: from='client.24263 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi146:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:53:25.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:25 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T02:53:25.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:25 smithi148 ceph-mon[35090]: osd.6 [v2:172.21.15.5:6818/3176073169,v1:172.21.15.5:6819/3176073169] boot 2024-09-12T02:53:25.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:25 smithi148 ceph-mon[35090]: osdmap e43: 7 total, 7 up, 7 in 2024-09-12T02:53:25.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:25 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-12T02:53:26.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:26 smithi005 ceph-mon[31197]: osdmap e44: 7 total, 7 up, 7 in 2024-09-12T02:53:26.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:26 smithi005 ceph-mon[31197]: from='client.? 172.21.15.146:0/1574440708' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "49451e8d-c741-4ad5-b8f5-89bad8d4d129"}]: dispatch 2024-09-12T02:53:26.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:26 smithi005 ceph-mon[31197]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "49451e8d-c741-4ad5-b8f5-89bad8d4d129"}]: dispatch 2024-09-12T02:53:26.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:26 smithi005 ceph-mon[31197]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "49451e8d-c741-4ad5-b8f5-89bad8d4d129"}]': finished 2024-09-12T02:53:26.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:26 smithi005 ceph-mon[31197]: osdmap e45: 8 total, 7 up, 8 in 2024-09-12T02:53:26.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:26 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T02:53:26.582 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:26 smithi146 ceph-mon[35011]: osdmap e44: 7 total, 7 up, 7 in 2024-09-12T02:53:26.582 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:26 smithi146 ceph-mon[35011]: from='client.? 172.21.15.146:0/1574440708' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "49451e8d-c741-4ad5-b8f5-89bad8d4d129"}]: dispatch 2024-09-12T02:53:26.582 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:26 smithi146 ceph-mon[35011]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "49451e8d-c741-4ad5-b8f5-89bad8d4d129"}]: dispatch 2024-09-12T02:53:26.582 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:26 smithi146 ceph-mon[35011]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "49451e8d-c741-4ad5-b8f5-89bad8d4d129"}]': finished 2024-09-12T02:53:26.582 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:26 smithi146 ceph-mon[35011]: osdmap e45: 8 total, 7 up, 8 in 2024-09-12T02:53:26.582 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:26 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T02:53:26.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:26 smithi148 ceph-mon[35090]: osdmap e44: 7 total, 7 up, 7 in 2024-09-12T02:53:26.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:26 smithi148 ceph-mon[35090]: from='client.? 172.21.15.146:0/1574440708' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "49451e8d-c741-4ad5-b8f5-89bad8d4d129"}]: dispatch 2024-09-12T02:53:26.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:26 smithi148 ceph-mon[35090]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "49451e8d-c741-4ad5-b8f5-89bad8d4d129"}]: dispatch 2024-09-12T02:53:26.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:26 smithi148 ceph-mon[35090]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "49451e8d-c741-4ad5-b8f5-89bad8d4d129"}]': finished 2024-09-12T02:53:26.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:26 smithi148 ceph-mon[35090]: osdmap e45: 8 total, 7 up, 8 in 2024-09-12T02:53:26.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:26 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T02:53:27.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:27 smithi005 ceph-mon[31197]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:53:27.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:27 smithi005 ceph-mon[31197]: from='client.? 172.21.15.146:0/3809415047' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:53:27.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:27 smithi148 ceph-mon[35090]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:53:27.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:27 smithi148 ceph-mon[35090]: from='client.? 172.21.15.146:0/3809415047' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:53:27.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:27 smithi146 ceph-mon[35011]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:53:27.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:27 smithi146 ceph-mon[35011]: from='client.? 172.21.15.146:0/3809415047' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:53:29.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:29 smithi005 ceph-mon[31197]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:53:29.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:29 smithi148 ceph-mon[35090]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:53:29.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:29 smithi146 ceph-mon[35011]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:53:31.404 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:31 smithi146 ceph-mon[35011]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:53:31.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:31 smithi005 ceph-mon[31197]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:53:31.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:31 smithi148 ceph-mon[35090]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:53:33.359 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:33 smithi146 ceph-mon[35011]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:53:33.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:33 smithi148 ceph-mon[35090]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:53:33.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:33 smithi005 ceph-mon[31197]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:53:35.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:35 smithi148 ceph-mon[35090]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:53:35.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:35 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-12T02:53:35.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:35 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:53:35.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:35 smithi146 ceph-mon[35011]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:53:35.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:35 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-12T02:53:35.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:35 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:53:35.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:35 smithi005 ceph-mon[31197]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:53:35.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:35 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-12T02:53:35.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:35 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:53:36.359 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:36 smithi146 ceph-mon[35011]: Deploying daemon osd.7 on smithi146 2024-09-12T02:53:36.665 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:36 smithi148 ceph-mon[35090]: Deploying daemon osd.7 on smithi146 2024-09-12T02:53:36.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:36 smithi005 ceph-mon[31197]: Deploying daemon osd.7 on smithi146 2024-09-12T02:53:37.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:37 smithi146 ceph-mon[35011]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:53:37.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:37 smithi148 ceph-mon[35090]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:53:37.711 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:37 smithi005 ceph-mon[31197]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:53:39.580 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:39 smithi146 ceph-mon[35011]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:53:39.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:39 smithi148 ceph-mon[35090]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:53:39.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:39 smithi005 ceph-mon[31197]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:53:40.605 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:40 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:53:40.606 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:40 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:53:40.606 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:40 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:53:40.665 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:40 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:53:40.668 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:40 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:53:40.668 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:40 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:53:40.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:40 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:53:40.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:40 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:53:40.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:40 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:53:41.636 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:41 smithi146 ceph-mon[35011]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:53:41.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:41 smithi148 ceph-mon[35090]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:53:41.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:41 smithi005 ceph-mon[31197]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:53:43.564 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:43 smithi146 ceph-mon[35011]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:53:43.564 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:43 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:53:43.564 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:43 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:53:43.564 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:43 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:53:43.564 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:43 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:53:43.564 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:43 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:53:43.564 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:53:43.563+0000 7efd88ff9640 1 -- 172.21.15.146:0/2718149263 <== mgr.14148 v2:172.21.15.5:6800/3780631642 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7efd8c062630 con 0x7efd740a2b80 2024-09-12T02:53:43.565 INFO:teuthology.orchestra.run.smithi146.stdout:Created osd(s) 7 on host 'smithi146' 2024-09-12T02:53:43.566 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:53:43.565+0000 7efd91e81640 1 -- 172.21.15.146:0/2718149263 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7efd740a2b80 msgr2=0x7efd740a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:53:43.566 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:53:43.565+0000 7efd91e81640 1 --2- 172.21.15.146:0/2718149263 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7efd740a2b80 0x7efd740a5040 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7efd80005e10 tx=0x7efd80005da0 comp rx=0 tx=0).stop 2024-09-12T02:53:43.566 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:53:43.565+0000 7efd91e81640 1 -- 172.21.15.146:0/2718149263 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efd8c0ffed0 msgr2=0x7efd8c102810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:53:43.566 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:53:43.565+0000 7efd91e81640 1 --2- 172.21.15.146:0/2718149263 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efd8c0ffed0 0x7efd8c102810 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7efd7c006010 tx=0x7efd7c032c70 comp rx=0 tx=0).stop 2024-09-12T02:53:43.566 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:53:43.565+0000 7efd91e81640 1 -- 172.21.15.146:0/2718149263 shutdown_connections 2024-09-12T02:53:43.566 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:53:43.565+0000 7efd91e81640 1 --2- 172.21.15.146:0/2718149263 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7efd740a2b80 0x7efd740a5040 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:43.566 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:53:43.565+0000 7efd91e81640 1 --2- 172.21.15.146:0/2718149263 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd8c103870 0x7efd8c1013a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:43.566 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:53:43.565+0000 7efd91e81640 1 --2- 172.21.15.146:0/2718149263 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efd8c102eb0 0x7efd8c100e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:43.566 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:53:43.565+0000 7efd91e81640 1 --2- 172.21.15.146:0/2718149263 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efd8c0ffed0 0x7efd8c102810 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:43.566 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:53:43.565+0000 7efd91e81640 1 -- 172.21.15.146:0/2718149263 >> 172.21.15.146:0/2718149263 conn(0x7efd8c0fba40 msgr2=0x7efd8c1062c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:53:43.566 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:53:43.565+0000 7efd91e81640 1 -- 172.21.15.146:0/2718149263 shutdown_connections 2024-09-12T02:53:43.566 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:53:43.565+0000 7efd91e81640 1 -- 172.21.15.146:0/2718149263 wait complete. 2024-09-12T02:53:43.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:43 smithi148 ceph-mon[35090]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:53:43.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:43 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:53:43.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:43 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:53:43.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:43 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:53:43.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:43 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:53:43.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:43 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:53:43.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:43 smithi005 ceph-mon[31197]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:53:43.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:43 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:53:43.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:43 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:53:43.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:43 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:53:43.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:43 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:53:43.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:43 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:53:44.156 DEBUG:teuthology.orchestra.run.smithi146:osd.7> sudo journalctl -f -n 0 -u ceph-618305f8-70b1-11ef-bcea-c7b262605968@osd.7.service 2024-09-12T02:53:44.159 INFO:tasks.cephadm:Deploying osd.8 on smithi148 with /dev/vg_nvme/lv_2... 2024-09-12T02:53:44.159 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-12T02:53:44.403 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:44 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:53:44.403 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:44 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:53:44.404 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:44 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:53:44.416 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T02:53:44.537 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:44 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:53:44.537 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:44 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:53:44.537 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:44 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:53:44.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:44 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:53:44.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:44 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:53:44.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:44 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:53:45.530 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:45 smithi148 ceph-mon[35090]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:53:45.599 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:45 smithi146 ceph-mon[35011]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:53:45.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:45 smithi005 ceph-mon[31197]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:53:46.664 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 12 02:53:46 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-7[52718]: 2024-09-12T02:53:46.476+0000 7f3322423640 -1 osd.7 0 log_to_monitors true 2024-09-12T02:53:46.889 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-12T02:53:46.915 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi148:vg_nvme/lv_2 2024-09-12T02:53:47.166 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T02:53:47.566 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:47 smithi148 ceph-mon[35090]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:53:47.566 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:47 smithi148 ceph-mon[35090]: from='osd.7 [v2:172.21.15.146:6816/2209508838,v1:172.21.15.146:6817/2209508838]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-12T02:53:47.566 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:47 smithi148 ceph-mon[35090]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-12T02:53:47.566 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:47 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:53:47.566 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:47 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:53:47.566 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:47 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:53:47.566 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:47 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:53:47.566 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:47 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:53:47.566 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:47 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:53:47.566 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:47 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:53:47.567 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:47 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:53:47.567 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:47 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:53:47.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:53:47.647+0000 7f3bb0a42640 1 -- 172.21.15.148:0/1271864665 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3bac102e00 msgr2=0x7f3bac103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:53:47.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:53:47.647+0000 7f3bb0a42640 1 --2- 172.21.15.148:0/1271864665 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3bac102e00 0x7f3bac103280 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f3ba0009900 tx=0x7f3ba002a410 comp rx=0 tx=0).stop 2024-09-12T02:53:47.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:53:47.647+0000 7f3bb0a42640 1 -- 172.21.15.148:0/1271864665 shutdown_connections 2024-09-12T02:53:47.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:53:47.647+0000 7f3bb0a42640 1 --2- 172.21.15.148:0/1271864665 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3bac1037c0 0x7f3bac10a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:47.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:53:47.647+0000 7f3bb0a42640 1 --2- 172.21.15.148:0/1271864665 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3bac102e00 0x7f3bac103280 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:47.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:53:47.647+0000 7f3bb0a42640 1 --2- 172.21.15.148:0/1271864665 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3bac101c00 0x7f3bac102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:47.657 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:53:47.647+0000 7f3bb0a42640 1 -- 172.21.15.148:0/1271864665 >> 172.21.15.148:0/1271864665 conn(0x7f3bac0fd390 msgr2=0x7f3bac0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:53:47.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:53:47.647+0000 7f3bb0a42640 1 -- 172.21.15.148:0/1271864665 shutdown_connections 2024-09-12T02:53:47.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:53:47.647+0000 7f3bb0a42640 1 -- 172.21.15.148:0/1271864665 wait complete. 2024-09-12T02:53:47.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:53:47.648+0000 7f3bb0a42640 1 Processor -- start 2024-09-12T02:53:47.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:53:47.648+0000 7f3bb0a42640 1 -- start start 2024-09-12T02:53:47.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:53:47.648+0000 7f3bb0a42640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3bac101c00 0x7f3bac197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:47.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:53:47.648+0000 7f3bb0a42640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3bac102e00 0x7f3bac198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:47.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:53:47.648+0000 7f3bb0a42640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3bac1037c0 0x7f3bac19f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:47.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:53:47.648+0000 7f3bb0a42640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3bac10d8d0 con 0x7f3bac101c00 2024-09-12T02:53:47.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:53:47.648+0000 7f3bb0a42640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f3bac10d750 con 0x7f3bac1037c0 2024-09-12T02:53:47.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:53:47.648+0000 7f3bb0a42640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3bac10da50 con 0x7f3bac102e00 2024-09-12T02:53:47.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:53:47.648+0000 7f3ba9d9b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3bac102e00 0x7f3bac198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:47.658 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:53:47.648+0000 7f3ba9d9b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3bac102e00 0x7f3bac198460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:33144/0 (socket says 172.21.15.148:33144) 2024-09-12T02:53:47.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:53:47.649+0000 7f3ba9d9b640 1 -- 172.21.15.148:0/349897710 learned_addr learned my addr 172.21.15.148:0/349897710 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-09-12T02:53:47.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:53:47.649+0000 7f3baad9d640 1 --2- 172.21.15.148:0/349897710 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3bac1037c0 0x7f3bac19f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:47.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:53:47.649+0000 7f3baa59c640 1 --2- 172.21.15.148:0/349897710 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3bac101c00 0x7f3bac197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:47.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:53:47.649+0000 7f3ba9d9b640 1 -- 172.21.15.148:0/349897710 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3bac1037c0 msgr2=0x7f3bac19f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:53:47.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:53:47.649+0000 7f3ba9d9b640 1 --2- 172.21.15.148:0/349897710 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3bac1037c0 0x7f3bac19f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:47.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:53:47.649+0000 7f3ba9d9b640 1 -- 172.21.15.148:0/349897710 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3bac101c00 msgr2=0x7f3bac197f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:53:47.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:53:47.649+0000 7f3ba9d9b640 1 --2- 172.21.15.148:0/349897710 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3bac101c00 0x7f3bac197f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:53:47.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:53:47.649+0000 7f3ba9d9b640 1 -- 172.21.15.148:0/349897710 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bac19fb50 con 0x7f3bac102e00 2024-09-12T02:53:47.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:53:47.649+0000 7f3ba9d9b640 1 --2- 172.21.15.148:0/349897710 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3bac102e00 0x7f3bac198460 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f3ba0031440 tx=0x7f3ba0032e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:53:47.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:53:47.649+0000 7f3b9b7fe640 1 -- 172.21.15.148:0/349897710 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba003f030 con 0x7f3bac102e00 2024-09-12T02:53:47.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:53:47.649+0000 7f3b9b7fe640 1 -- 172.21.15.148:0/349897710 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ba0003ce0 con 0x7f3bac102e00 2024-09-12T02:53:47.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:53:47.649+0000 7f3bb0a42640 1 -- 172.21.15.148:0/349897710 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bac19fde0 con 0x7f3bac102e00 2024-09-12T02:53:47.659 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:53:47.649+0000 7f3b9b7fe640 1 -- 172.21.15.148:0/349897710 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba0035070 con 0x7f3bac102e00 2024-09-12T02:53:47.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:53:47.649+0000 7f3bb0a42640 1 -- 172.21.15.148:0/349897710 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bac1a0240 con 0x7f3bac102e00 2024-09-12T02:53:47.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:53:47.650+0000 7f3bb0a42640 1 -- 172.21.15.148:0/349897710 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bac109000 con 0x7f3bac102e00 2024-09-12T02:53:47.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:53:47.651+0000 7f3b9b7fe640 1 -- 172.21.15.148:0/349897710 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f3ba0042030 con 0x7f3bac102e00 2024-09-12T02:53:47.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:53:47.652+0000 7f3b9b7fe640 1 --2- 172.21.15.148:0/349897710 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f3b940a2c30 0x7f3b940a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:53:47.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:53:47.652+0000 7f3b9b7fe640 1 -- 172.21.15.148:0/349897710 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5101+0+0 (secure 0 0 0) 0x7f3ba00ee890 con 0x7f3bac102e00 2024-09-12T02:53:47.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:53:47.653+0000 7f3baa59c640 1 --2- 172.21.15.148:0/349897710 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f3b940a2c30 0x7f3b940a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:53:47.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:53:47.653+0000 7f3baa59c640 1 --2- 172.21.15.148:0/349897710 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f3b940a2c30 0x7f3b940a50f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c005fd0 tx=0x7f3b9c00b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:53:47.660 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:53:47.657+0000 7f3b9b7fe640 1 -- 172.21.15.148:0/349897710 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3ba00bbe30 con 0x7f3bac102e00 2024-09-12T02:53:47.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:47 smithi146 ceph-mon[35011]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:53:47.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:47 smithi146 ceph-mon[35011]: from='osd.7 [v2:172.21.15.146:6816/2209508838,v1:172.21.15.146:6817/2209508838]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-12T02:53:47.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:47 smithi146 ceph-mon[35011]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-12T02:53:47.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:47 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:53:47.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:47 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:53:47.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:47 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:53:47.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:47 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:53:47.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:47 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:53:47.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:47 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:53:47.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:47 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:53:47.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:47 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:53:47.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:47 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:53:47.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:47 smithi005 ceph-mon[31197]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:53:47.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:47 smithi005 ceph-mon[31197]: from='osd.7 [v2:172.21.15.146:6816/2209508838,v1:172.21.15.146:6817/2209508838]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-12T02:53:47.811 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:47 smithi005 ceph-mon[31197]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-12T02:53:47.811 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:47 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:53:47.811 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:47 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:53:47.811 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:47 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:53:47.811 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:47 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:53:47.811 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:47 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:53:47.811 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:47 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:53:47.811 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:47 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:53:47.811 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:47 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:53:47.811 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:47 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:53:47.832 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:53:47.831+0000 7f3bb0a42640 1 -- 172.21.15.148:0/349897710 --> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f3bac062630 con 0x7f3b940a2c30 2024-09-12T02:53:48.568 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:48 smithi148 ceph-mon[35090]: Detected new or changed devices on smithi146 2024-09-12T02:53:48.568 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:48 smithi148 ceph-mon[35090]: Adjusting osd_memory_target on smithi146 to 5696M 2024-09-12T02:53:48.568 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:48 smithi148 ceph-mon[35090]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-12T02:53:48.568 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:48 smithi148 ceph-mon[35090]: from='osd.7 [v2:172.21.15.146:6816/2209508838,v1:172.21.15.146:6817/2209508838]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2024-09-12T02:53:48.568 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:48 smithi148 ceph-mon[35090]: osdmap e46: 8 total, 7 up, 8 in 2024-09-12T02:53:48.568 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:48 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T02:53:48.568 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:48 smithi148 ceph-mon[35090]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2024-09-12T02:53:48.568 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:48 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:53:48.568 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:48 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:53:48.568 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:48 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:53:48.569 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:48 smithi148 ceph-mon[35090]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]': finished 2024-09-12T02:53:48.569 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:48 smithi148 ceph-mon[35090]: osdmap e47: 8 total, 7 up, 8 in 2024-09-12T02:53:48.569 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:48 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T02:53:48.706 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 12 02:53:48 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-7[52718]: 2024-09-12T02:53:48.350+0000 7f331db83640 -1 osd.7 0 waiting for initial osdmap 2024-09-12T02:53:48.707 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 12 02:53:48 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-7[52718]: 2024-09-12T02:53:48.362+0000 7f331a1bd640 -1 osd.7 47 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T02:53:48.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:48 smithi146 ceph-mon[35011]: Detected new or changed devices on smithi146 2024-09-12T02:53:48.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:48 smithi146 ceph-mon[35011]: Adjusting osd_memory_target on smithi146 to 5696M 2024-09-12T02:53:48.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:48 smithi146 ceph-mon[35011]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-12T02:53:48.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:48 smithi146 ceph-mon[35011]: from='osd.7 [v2:172.21.15.146:6816/2209508838,v1:172.21.15.146:6817/2209508838]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2024-09-12T02:53:48.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:48 smithi146 ceph-mon[35011]: osdmap e46: 8 total, 7 up, 8 in 2024-09-12T02:53:48.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:48 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T02:53:48.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:48 smithi146 ceph-mon[35011]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2024-09-12T02:53:48.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:48 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:53:48.708 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:48 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:53:48.708 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:48 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:53:48.708 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:48 smithi146 ceph-mon[35011]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]': finished 2024-09-12T02:53:48.708 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:48 smithi146 ceph-mon[35011]: osdmap e47: 8 total, 7 up, 8 in 2024-09-12T02:53:48.708 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:48 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T02:53:48.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:48 smithi005 ceph-mon[31197]: Detected new or changed devices on smithi146 2024-09-12T02:53:48.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:48 smithi005 ceph-mon[31197]: Adjusting osd_memory_target on smithi146 to 5696M 2024-09-12T02:53:48.811 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:48 smithi005 ceph-mon[31197]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-12T02:53:48.811 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:48 smithi005 ceph-mon[31197]: from='osd.7 [v2:172.21.15.146:6816/2209508838,v1:172.21.15.146:6817/2209508838]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2024-09-12T02:53:48.811 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:48 smithi005 ceph-mon[31197]: osdmap e46: 8 total, 7 up, 8 in 2024-09-12T02:53:48.811 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:48 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T02:53:48.811 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:48 smithi005 ceph-mon[31197]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2024-09-12T02:53:48.811 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:48 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:53:48.811 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:48 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:53:48.811 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:48 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:53:48.811 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:48 smithi005 ceph-mon[31197]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]': finished 2024-09-12T02:53:48.811 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:48 smithi005 ceph-mon[31197]: osdmap e47: 8 total, 7 up, 8 in 2024-09-12T02:53:48.811 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:48 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T02:53:49.558 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:49 smithi148 ceph-mon[35090]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:53:49.558 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:49 smithi148 ceph-mon[35090]: from='client.24313 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:53:49.558 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:49 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T02:53:49.558 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:49 smithi148 ceph-mon[35090]: osd.7 [v2:172.21.15.146:6816/2209508838,v1:172.21.15.146:6817/2209508838] boot 2024-09-12T02:53:49.559 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:49 smithi148 ceph-mon[35090]: osdmap e48: 8 total, 8 up, 8 in 2024-09-12T02:53:49.559 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:49 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T02:53:49.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:49 smithi146 ceph-mon[35011]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:53:49.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:49 smithi146 ceph-mon[35011]: from='client.24313 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:53:49.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:49 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T02:53:49.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:49 smithi146 ceph-mon[35011]: osd.7 [v2:172.21.15.146:6816/2209508838,v1:172.21.15.146:6817/2209508838] boot 2024-09-12T02:53:49.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:49 smithi146 ceph-mon[35011]: osdmap e48: 8 total, 8 up, 8 in 2024-09-12T02:53:49.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:49 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T02:53:49.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:49 smithi005 ceph-mon[31197]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-09-12T02:53:49.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:49 smithi005 ceph-mon[31197]: from='client.24313 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:53:49.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:49 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T02:53:49.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:49 smithi005 ceph-mon[31197]: osd.7 [v2:172.21.15.146:6816/2209508838,v1:172.21.15.146:6817/2209508838] boot 2024-09-12T02:53:49.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:49 smithi005 ceph-mon[31197]: osdmap e48: 8 total, 8 up, 8 in 2024-09-12T02:53:49.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:49 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-12T02:53:50.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:50 smithi148 ceph-mon[35090]: purged_snaps scrub starts 2024-09-12T02:53:50.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:50 smithi148 ceph-mon[35090]: purged_snaps scrub ok 2024-09-12T02:53:50.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:50 smithi148 ceph-mon[35090]: from='client.? 172.21.15.148:0/2972663880' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "57bfc121-a75e-4a7f-b1dc-64074b178b03"}]: dispatch 2024-09-12T02:53:50.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:50 smithi148 ceph-mon[35090]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "57bfc121-a75e-4a7f-b1dc-64074b178b03"}]: dispatch 2024-09-12T02:53:50.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:50 smithi148 ceph-mon[35090]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "57bfc121-a75e-4a7f-b1dc-64074b178b03"}]': finished 2024-09-12T02:53:50.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:50 smithi148 ceph-mon[35090]: osdmap e49: 9 total, 8 up, 9 in 2024-09-12T02:53:50.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:50 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T02:53:50.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:50 smithi146 ceph-mon[35011]: purged_snaps scrub starts 2024-09-12T02:53:50.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:50 smithi146 ceph-mon[35011]: purged_snaps scrub ok 2024-09-12T02:53:50.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:50 smithi146 ceph-mon[35011]: from='client.? 172.21.15.148:0/2972663880' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "57bfc121-a75e-4a7f-b1dc-64074b178b03"}]: dispatch 2024-09-12T02:53:50.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:50 smithi146 ceph-mon[35011]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "57bfc121-a75e-4a7f-b1dc-64074b178b03"}]: dispatch 2024-09-12T02:53:50.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:50 smithi146 ceph-mon[35011]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "57bfc121-a75e-4a7f-b1dc-64074b178b03"}]': finished 2024-09-12T02:53:50.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:50 smithi146 ceph-mon[35011]: osdmap e49: 9 total, 8 up, 9 in 2024-09-12T02:53:50.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:50 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T02:53:50.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:50 smithi005 ceph-mon[31197]: purged_snaps scrub starts 2024-09-12T02:53:50.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:50 smithi005 ceph-mon[31197]: purged_snaps scrub ok 2024-09-12T02:53:50.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:50 smithi005 ceph-mon[31197]: from='client.? 172.21.15.148:0/2972663880' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "57bfc121-a75e-4a7f-b1dc-64074b178b03"}]: dispatch 2024-09-12T02:53:50.811 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:50 smithi005 ceph-mon[31197]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "57bfc121-a75e-4a7f-b1dc-64074b178b03"}]: dispatch 2024-09-12T02:53:50.811 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:50 smithi005 ceph-mon[31197]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "57bfc121-a75e-4a7f-b1dc-64074b178b03"}]': finished 2024-09-12T02:53:50.811 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:50 smithi005 ceph-mon[31197]: osdmap e49: 9 total, 8 up, 9 in 2024-09-12T02:53:50.811 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:50 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T02:53:51.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:51 smithi146 ceph-mon[35011]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:53:51.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:51 smithi146 ceph-mon[35011]: from='client.? 172.21.15.148:0/877252079' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:53:51.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:51 smithi146 ceph-mon[35011]: osdmap e50: 9 total, 8 up, 9 in 2024-09-12T02:53:51.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:51 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T02:53:51.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:51 smithi005 ceph-mon[31197]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:53:51.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:51 smithi005 ceph-mon[31197]: from='client.? 172.21.15.148:0/877252079' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:53:51.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:51 smithi005 ceph-mon[31197]: osdmap e50: 9 total, 8 up, 9 in 2024-09-12T02:53:51.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:51 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T02:53:51.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:51 smithi148 ceph-mon[35090]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:53:51.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:51 smithi148 ceph-mon[35090]: from='client.? 172.21.15.148:0/877252079' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:53:51.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:51 smithi148 ceph-mon[35090]: osdmap e50: 9 total, 8 up, 9 in 2024-09-12T02:53:51.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:51 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T02:53:53.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:53 smithi148 ceph-mon[35090]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:53:53.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:53 smithi148 ceph-mon[35090]: osdmap e51: 9 total, 8 up, 9 in 2024-09-12T02:53:53.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:53 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T02:53:53.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:53 smithi146 ceph-mon[35011]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:53:53.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:53 smithi146 ceph-mon[35011]: osdmap e51: 9 total, 8 up, 9 in 2024-09-12T02:53:53.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:53 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T02:53:53.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:53 smithi005 ceph-mon[31197]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:53:53.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:53 smithi005 ceph-mon[31197]: osdmap e51: 9 total, 8 up, 9 in 2024-09-12T02:53:53.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:53 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T02:53:54.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:54 smithi148 ceph-mon[35090]: osdmap e52: 9 total, 8 up, 9 in 2024-09-12T02:53:54.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:54 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T02:53:54.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:54 smithi146 ceph-mon[35011]: osdmap e52: 9 total, 8 up, 9 in 2024-09-12T02:53:54.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:54 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T02:53:54.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:54 smithi005 ceph-mon[31197]: osdmap e52: 9 total, 8 up, 9 in 2024-09-12T02:53:54.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:54 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T02:53:55.653 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:55 smithi148 ceph-mon[35090]: pgmap v174: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:53:55.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:55 smithi146 ceph-mon[35011]: pgmap v174: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:53:55.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:55 smithi005 ceph-mon[31197]: pgmap v174: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:53:57.652 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:57 smithi148 ceph-mon[35090]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:53:57.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:57 smithi146 ceph-mon[35011]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:53:57.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:57 smithi005 ceph-mon[31197]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:53:59.595 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:59 smithi148 ceph-mon[35090]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:53:59.595 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:59 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-12T02:53:59.595 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:53:59 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:53:59.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:59 smithi146 ceph-mon[35011]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:53:59.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:59 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-12T02:53:59.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:53:59 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:53:59.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:59 smithi005 ceph-mon[31197]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:53:59.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:59 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-12T02:53:59.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:53:59 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:54:00.526 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:00 smithi148 ceph-mon[35090]: Deploying daemon osd.8 on smithi148 2024-09-12T02:54:00.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:00 smithi146 ceph-mon[35011]: Deploying daemon osd.8 on smithi148 2024-09-12T02:54:00.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:00 smithi005 ceph-mon[31197]: Deploying daemon osd.8 on smithi148 2024-09-12T02:54:01.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:01 smithi148 ceph-mon[35090]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:54:01.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:01 smithi146 ceph-mon[35011]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:54:01.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:01 smithi005 ceph-mon[31197]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:54:03.510 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:03 smithi148 ceph-mon[35090]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:54:03.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:03 smithi146 ceph-mon[35011]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:54:03.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:03 smithi005 ceph-mon[31197]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:54:05.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:05 smithi148 ceph-mon[35090]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:54:05.708 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:05 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:54:05.709 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:05 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:54:05.709 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:05 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:54:05.709 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:05 smithi146 ceph-mon[35011]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:54:05.709 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:05 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:54:05.709 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:05 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:54:05.709 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:05 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:54:05.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:05 smithi005 ceph-mon[31197]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:54:05.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:05 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:54:05.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:05 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:54:05.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:05 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:54:07.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:07 smithi146 ceph-mon[35011]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:54:07.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:07 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:54:07.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:07 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:54:07.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:07 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:54:07.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:07 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:54:07.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:07 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:54:07.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:07 smithi005 ceph-mon[31197]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:54:07.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:07 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:54:07.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:07 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:54:07.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:07 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:54:07.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:07 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:54:07.811 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:07 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:54:07.915 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:07 smithi148 ceph-mon[35090]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:54:07.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:07 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:54:07.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:07 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:54:07.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:07 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:54:07.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:07 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:54:07.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:07 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:54:08.271 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:54:08.271+0000 7f3b9b7fe640 1 -- 172.21.15.148:0/349897710 <== mgr.14148 v2:172.21.15.5:6800/3780631642 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f3bac062630 con 0x7f3b940a2c30 2024-09-12T02:54:08.271 INFO:teuthology.orchestra.run.smithi148.stdout:Created osd(s) 8 on host 'smithi148' 2024-09-12T02:54:08.273 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:54:08.273+0000 7f3bb0a42640 1 -- 172.21.15.148:0/349897710 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f3b940a2c30 msgr2=0x7f3b940a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:08.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:54:08.273+0000 7f3bb0a42640 1 --2- 172.21.15.148:0/349897710 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f3b940a2c30 0x7f3b940a50f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c005fd0 tx=0x7f3b9c00b390 comp rx=0 tx=0).stop 2024-09-12T02:54:08.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:54:08.273+0000 7f3bb0a42640 1 -- 172.21.15.148:0/349897710 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3bac102e00 msgr2=0x7f3bac198460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:08.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:54:08.273+0000 7f3bb0a42640 1 --2- 172.21.15.148:0/349897710 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3bac102e00 0x7f3bac198460 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f3ba0031440 tx=0x7f3ba0032e20 comp rx=0 tx=0).stop 2024-09-12T02:54:08.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:54:08.273+0000 7f3bb0a42640 1 -- 172.21.15.148:0/349897710 shutdown_connections 2024-09-12T02:54:08.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:54:08.274+0000 7f3bb0a42640 1 --2- 172.21.15.148:0/349897710 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3bac1037c0 0x7f3bac19f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:08.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:54:08.274+0000 7f3bb0a42640 1 --2- 172.21.15.148:0/349897710 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3bac102e00 0x7f3bac198460 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:08.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:54:08.274+0000 7f3bb0a42640 1 --2- 172.21.15.148:0/349897710 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f3b940a2c30 0x7f3b940a50f0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:08.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:54:08.274+0000 7f3bb0a42640 1 --2- 172.21.15.148:0/349897710 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3bac101c00 0x7f3bac197f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:08.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:54:08.274+0000 7f3bb0a42640 1 -- 172.21.15.148:0/349897710 >> 172.21.15.148:0/349897710 conn(0x7f3bac0fd390 msgr2=0x7f3bac1061d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:54:08.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:54:08.274+0000 7f3bb0a42640 1 -- 172.21.15.148:0/349897710 shutdown_connections 2024-09-12T02:54:08.274 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:54:08.274+0000 7f3bb0a42640 1 -- 172.21.15.148:0/349897710 wait complete. 2024-09-12T02:54:08.601 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:08 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:54:08.601 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:08 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:54:08.601 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:08 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:54:08.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:08 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:54:08.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:08 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:54:08.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:08 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:54:08.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:08 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:54:08.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:08 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:54:08.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:08 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:54:09.003 DEBUG:teuthology.orchestra.run.smithi148:osd.8> sudo journalctl -f -n 0 -u ceph-618305f8-70b1-11ef-bcea-c7b262605968@osd.8.service 2024-09-12T02:54:09.005 INFO:tasks.cephadm:Deploying osd.9 on smithi005 with /dev/vg_nvme/lv_1... 2024-09-12T02:54:09.005 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-12T02:54:09.275 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:54:09.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:09 smithi148 ceph-mon[35090]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:54:09.671 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:09 smithi005 ceph-mon[31197]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:54:09.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:09 smithi146 ceph-mon[35011]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:54:11.066 INFO:journalctl@ceph.osd.8.smithi148.stdout:Sep 12 02:54:10 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-8[53762]: 2024-09-12T02:54:10.965+0000 7f7e1837c640 -1 osd.8 0 log_to_monitors true 2024-09-12T02:54:11.611 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:11 smithi005 ceph-mon[31197]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:54:11.611 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:11 smithi005 ceph-mon[31197]: from='osd.8 [v2:172.21.15.148:6816/651045756,v1:172.21.15.148:6817/651045756]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-12T02:54:11.611 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:11 smithi005 ceph-mon[31197]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-12T02:54:11.665 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:11 smithi148 ceph-mon[35090]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:54:11.665 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:11 smithi148 ceph-mon[35090]: from='osd.8 [v2:172.21.15.148:6816/651045756,v1:172.21.15.148:6817/651045756]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-12T02:54:11.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:11 smithi148 ceph-mon[35090]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-12T02:54:11.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:11 smithi146 ceph-mon[35011]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:54:11.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:11 smithi146 ceph-mon[35011]: from='osd.8 [v2:172.21.15.148:6816/651045756,v1:172.21.15.148:6817/651045756]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-12T02:54:11.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:11 smithi146 ceph-mon[35011]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-12T02:54:12.186 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-12T02:54:12.216 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi005:vg_nvme/lv_1 2024-09-12T02:54:12.474 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:54:12.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:12 smithi146 ceph-mon[35011]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-12T02:54:12.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:12 smithi146 ceph-mon[35011]: osdmap e53: 9 total, 8 up, 9 in 2024-09-12T02:54:12.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:12 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T02:54:12.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:12 smithi146 ceph-mon[35011]: from='osd.8 [v2:172.21.15.148:6816/651045756,v1:172.21.15.148:6817/651045756]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-09-12T02:54:12.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:12 smithi146 ceph-mon[35011]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-09-12T02:54:12.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:12 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:54:12.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:12 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:54:12.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:12 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:54:12.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:12 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:54:12.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:12 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:54:12.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:12 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:54:12.708 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:12 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:54:12.708 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:12 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:54:12.708 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:12 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:54:12.711 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:12 smithi005 ceph-mon[31197]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-12T02:54:12.711 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:12 smithi005 ceph-mon[31197]: osdmap e53: 9 total, 8 up, 9 in 2024-09-12T02:54:12.711 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:12 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T02:54:12.712 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:12 smithi005 ceph-mon[31197]: from='osd.8 [v2:172.21.15.148:6816/651045756,v1:172.21.15.148:6817/651045756]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-09-12T02:54:12.712 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:12 smithi005 ceph-mon[31197]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-09-12T02:54:12.712 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:12 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:54:12.712 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:12 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:54:12.712 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:12 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:54:12.712 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:12 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:54:12.712 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:12 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:54:12.712 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:12 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:54:12.712 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:12 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:54:12.712 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:12 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:54:12.712 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:12 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:54:12.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:54:12.914+0000 7f25ede38640 1 -- 172.21.15.5:0/3234049032 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25e8103870 msgr2=0x7f25e810a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:12.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:54:12.914+0000 7f25ede38640 1 --2- 172.21.15.5:0/3234049032 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25e8103870 0x7f25e810a100 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f25e0009930 tx=0x7f25e002a420 comp rx=0 tx=0).stop 2024-09-12T02:54:12.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:54:12.914+0000 7f25ede38640 1 -- 172.21.15.5:0/3234049032 shutdown_connections 2024-09-12T02:54:12.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:54:12.914+0000 7f25ede38640 1 --2- 172.21.15.5:0/3234049032 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25e8103870 0x7f25e810a100 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:12.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:54:12.914+0000 7f25ede38640 1 --2- 172.21.15.5:0/3234049032 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f25e8102eb0 0x7f25e8103330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:12.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:54:12.914+0000 7f25ede38640 1 --2- 172.21.15.5:0/3234049032 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25e80ffed0 0x7f25e81002d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:12.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:54:12.914+0000 7f25ede38640 1 -- 172.21.15.5:0/3234049032 >> 172.21.15.5:0/3234049032 conn(0x7f25e80fba40 msgr2=0x7f25e80fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:54:12.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:54:12.914+0000 7f25ede38640 1 -- 172.21.15.5:0/3234049032 shutdown_connections 2024-09-12T02:54:12.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:54:12.914+0000 7f25ede38640 1 -- 172.21.15.5:0/3234049032 wait complete. 2024-09-12T02:54:12.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:54:12.914+0000 7f25ede38640 1 Processor -- start 2024-09-12T02:54:12.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:54:12.915+0000 7f25ede38640 1 -- start start 2024-09-12T02:54:12.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:54:12.915+0000 7f25ede38640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25e80ffed0 0x7f25e8102810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:12.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:54:12.915+0000 7f25ede38640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f25e8102eb0 0x7f25e8100e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:12.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:54:12.915+0000 7f25ede38640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25e8103870 0x7f25e81013a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:12.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:54:12.915+0000 7f25ede38640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f25e810d890 con 0x7f25e8103870 2024-09-12T02:54:12.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:54:12.915+0000 7f25ede38640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f25e810d710 con 0x7f25e8102eb0 2024-09-12T02:54:12.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:54:12.915+0000 7f25ede38640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f25e810da10 con 0x7f25e80ffed0 2024-09-12T02:54:12.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:54:12.915+0000 7f25e77fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25e80ffed0 0x7f25e8102810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:12.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:54:12.915+0000 7f25e7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25e8103870 0x7f25e81013a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:12.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:54:12.915+0000 7f25e77fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25e80ffed0 0x7f25e8102810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.5:51212/0 (socket says 172.21.15.5:51212) 2024-09-12T02:54:12.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:54:12.915+0000 7f25e77fe640 1 -- 172.21.15.5:0/3284915698 learned_addr learned my addr 172.21.15.5:0/3284915698 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:54:12.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:54:12.915+0000 7f25e7fff640 1 --2- 172.21.15.5:0/3284915698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25e8103870 0x7f25e81013a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52128/0 (socket says 172.21.15.5:52128) 2024-09-12T02:54:12.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:54:12.915+0000 7f25e6ffd640 1 --2- 172.21.15.5:0/3284915698 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f25e8102eb0 0x7f25e8100e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:12.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:54:12.915+0000 7f25e7fff640 1 -- 172.21.15.5:0/3284915698 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25e80ffed0 msgr2=0x7f25e8102810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:12.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:54:12.915+0000 7f25e7fff640 1 --2- 172.21.15.5:0/3284915698 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25e80ffed0 0x7f25e8102810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:12.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:54:12.915+0000 7f25e7fff640 1 -- 172.21.15.5:0/3284915698 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f25e8102eb0 msgr2=0x7f25e8100e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:12.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:54:12.915+0000 7f25e7fff640 1 --2- 172.21.15.5:0/3284915698 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f25e8102eb0 0x7f25e8100e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:12.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:54:12.915+0000 7f25e7fff640 1 -- 172.21.15.5:0/3284915698 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25e8101c60 con 0x7f25e8103870 2024-09-12T02:54:12.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:54:12.915+0000 7f25e6ffd640 1 --2- 172.21.15.5:0/3284915698 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f25e8102eb0 0x7f25e8100e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:54:12.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:54:12.915+0000 7f25e7fff640 1 --2- 172.21.15.5:0/3284915698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25e8103870 0x7f25e81013a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f25e0009900 tx=0x7f25e00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:54:12.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:54:12.916+0000 7f25e4ff9640 1 -- 172.21.15.5:0/3284915698 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25e003f030 con 0x7f25e8103870 2024-09-12T02:54:12.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:54:12.916+0000 7f25ede38640 1 -- 172.21.15.5:0/3284915698 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25e81a2060 con 0x7f25e8103870 2024-09-12T02:54:12.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:54:12.916+0000 7f25e4ff9640 1 -- 172.21.15.5:0/3284915698 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25e0003ce0 con 0x7f25e8103870 2024-09-12T02:54:12.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:54:12.916+0000 7f25e4ff9640 1 -- 172.21.15.5:0/3284915698 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25e0035070 con 0x7f25e8103870 2024-09-12T02:54:12.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:54:12.916+0000 7f25ede38640 1 -- 172.21.15.5:0/3284915698 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25e81a2560 con 0x7f25e8103870 2024-09-12T02:54:12.919 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:12 smithi148 ceph-mon[35090]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-12T02:54:12.919 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:12 smithi148 ceph-mon[35090]: osdmap e53: 9 total, 8 up, 9 in 2024-09-12T02:54:12.919 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:12 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T02:54:12.919 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:12 smithi148 ceph-mon[35090]: from='osd.8 [v2:172.21.15.148:6816/651045756,v1:172.21.15.148:6817/651045756]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-09-12T02:54:12.920 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:12 smithi148 ceph-mon[35090]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-09-12T02:54:12.920 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:12 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:54:12.920 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:12 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:54:12.920 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:12 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:54:12.920 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:12 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:54:12.920 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:12 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:54:12.920 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:12 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:54:12.920 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:12 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:54:12.920 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:12 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:54:12.920 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:12 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:54:12.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:54:12.917+0000 7f25ede38640 1 -- 172.21.15.5:0/3284915698 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25e81090b0 con 0x7f25e8103870 2024-09-12T02:54:12.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:54:12.918+0000 7f25e4ff9640 1 -- 172.21.15.5:0/3284915698 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f25e0042030 con 0x7f25e8103870 2024-09-12T02:54:12.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:54:12.919+0000 7f25e4ff9640 1 --2- 172.21.15.5:0/3284915698 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f25d40a2b80 0x7f25d40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:12.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:54:12.919+0000 7f25e77fe640 1 --2- 172.21.15.5:0/3284915698 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f25d40a2b80 0x7f25d40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:12.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:54:12.919+0000 7f25e4ff9640 1 -- 172.21.15.5:0/3284915698 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5582+0+0 (secure 0 0 0) 0x7f25e00eecc0 con 0x7f25e8103870 2024-09-12T02:54:12.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:54:12.919+0000 7f25e77fe640 1 --2- 172.21.15.5:0/3284915698 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f25d40a2b80 0x7f25d40a5040 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f25d8011440 tx=0x7f25d8009500 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:54:12.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:54:12.922+0000 7f25e4ff9640 1 -- 172.21.15.5:0/3284915698 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f25e00bc630 con 0x7f25e8103870 2024-09-12T02:54:13.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:54:13.107+0000 7f25ede38640 1 -- 172.21.15.5:0/3284915698 --> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f25e8062630 con 0x7f25d40a2b80 2024-09-12T02:54:13.666 INFO:journalctl@ceph.osd.8.smithi148.stdout:Sep 12 02:54:13 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-8[53762]: 2024-09-12T02:54:13.289+0000 7f7e13adc640 -1 osd.8 0 waiting for initial osdmap 2024-09-12T02:54:13.666 INFO:journalctl@ceph.osd.8.smithi148.stdout:Sep 12 02:54:13 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-8[53762]: 2024-09-12T02:54:13.306+0000 7f7e0c10e640 -1 osd.8 55 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T02:54:13.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:13 smithi148 ceph-mon[35090]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:54:13.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:13 smithi148 ceph-mon[35090]: Detected new or changed devices on smithi148 2024-09-12T02:54:13.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:13 smithi148 ceph-mon[35090]: Adjusting osd_memory_target on smithi148 to 5696M 2024-09-12T02:54:13.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:13 smithi148 ceph-mon[35090]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2024-09-12T02:54:13.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:13 smithi148 ceph-mon[35090]: osdmap e54: 9 total, 8 up, 9 in 2024-09-12T02:54:13.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:13 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T02:54:13.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:13 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T02:54:13.667 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:13 smithi148 ceph-mon[35090]: osdmap e55: 9 total, 8 up, 9 in 2024-09-12T02:54:13.667 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:13 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T02:54:13.667 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:13 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:54:13.667 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:13 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:54:13.667 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:13 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:54:13.667 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:13 smithi148 ceph-mon[35090]: from='osd.8 ' entity='osd.8' 2024-09-12T02:54:13.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:13 smithi146 ceph-mon[35011]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:54:13.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:13 smithi146 ceph-mon[35011]: Detected new or changed devices on smithi148 2024-09-12T02:54:13.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:13 smithi146 ceph-mon[35011]: Adjusting osd_memory_target on smithi148 to 5696M 2024-09-12T02:54:13.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:13 smithi146 ceph-mon[35011]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2024-09-12T02:54:13.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:13 smithi146 ceph-mon[35011]: osdmap e54: 9 total, 8 up, 9 in 2024-09-12T02:54:13.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:13 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T02:54:13.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:13 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T02:54:13.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:13 smithi146 ceph-mon[35011]: osdmap e55: 9 total, 8 up, 9 in 2024-09-12T02:54:13.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:13 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T02:54:13.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:13 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:54:13.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:13 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:54:13.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:13 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:54:13.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:13 smithi146 ceph-mon[35011]: from='osd.8 ' entity='osd.8' 2024-09-12T02:54:13.781 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:13 smithi005 ceph-mon[31197]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:54:13.781 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:13 smithi005 ceph-mon[31197]: Detected new or changed devices on smithi148 2024-09-12T02:54:13.781 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:13 smithi005 ceph-mon[31197]: Adjusting osd_memory_target on smithi148 to 5696M 2024-09-12T02:54:13.781 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:13 smithi005 ceph-mon[31197]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2024-09-12T02:54:13.782 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:13 smithi005 ceph-mon[31197]: osdmap e54: 9 total, 8 up, 9 in 2024-09-12T02:54:13.782 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:13 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T02:54:13.782 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:13 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T02:54:13.782 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:13 smithi005 ceph-mon[31197]: osdmap e55: 9 total, 8 up, 9 in 2024-09-12T02:54:13.782 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:13 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T02:54:13.782 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:13 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:54:13.782 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:13 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:54:13.782 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:13 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:54:13.782 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:13 smithi005 ceph-mon[31197]: from='osd.8 ' entity='osd.8' 2024-09-12T02:54:14.503 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:14 smithi005 ceph-mon[31197]: purged_snaps scrub starts 2024-09-12T02:54:14.504 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:14 smithi005 ceph-mon[31197]: purged_snaps scrub ok 2024-09-12T02:54:14.504 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:14 smithi005 ceph-mon[31197]: from='client.14415 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:54:14.504 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:14 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T02:54:14.504 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:14 smithi005 ceph-mon[31197]: osd.8 [v2:172.21.15.148:6816/651045756,v1:172.21.15.148:6817/651045756] boot 2024-09-12T02:54:14.504 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:14 smithi005 ceph-mon[31197]: osdmap e56: 9 total, 9 up, 9 in 2024-09-12T02:54:14.504 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:14 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T02:54:14.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:14 smithi148 ceph-mon[35090]: purged_snaps scrub starts 2024-09-12T02:54:14.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:14 smithi148 ceph-mon[35090]: purged_snaps scrub ok 2024-09-12T02:54:14.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:14 smithi148 ceph-mon[35090]: from='client.14415 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:54:14.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:14 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T02:54:14.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:14 smithi148 ceph-mon[35090]: osd.8 [v2:172.21.15.148:6816/651045756,v1:172.21.15.148:6817/651045756] boot 2024-09-12T02:54:14.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:14 smithi148 ceph-mon[35090]: osdmap e56: 9 total, 9 up, 9 in 2024-09-12T02:54:14.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:14 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T02:54:14.956 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:14 smithi146 ceph-mon[35011]: purged_snaps scrub starts 2024-09-12T02:54:14.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:14 smithi146 ceph-mon[35011]: purged_snaps scrub ok 2024-09-12T02:54:14.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:14 smithi146 ceph-mon[35011]: from='client.14415 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:54:14.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:14 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T02:54:14.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:14 smithi146 ceph-mon[35011]: osd.8 [v2:172.21.15.148:6816/651045756,v1:172.21.15.148:6817/651045756] boot 2024-09-12T02:54:14.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:14 smithi146 ceph-mon[35011]: osdmap e56: 9 total, 9 up, 9 in 2024-09-12T02:54:14.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:14 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-12T02:54:15.709 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:15 smithi005 ceph-mon[31197]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:54:15.709 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:15 smithi005 ceph-mon[31197]: osdmap e57: 9 total, 9 up, 9 in 2024-09-12T02:54:15.915 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:15 smithi148 ceph-mon[35090]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:54:15.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:15 smithi148 ceph-mon[35090]: osdmap e57: 9 total, 9 up, 9 in 2024-09-12T02:54:15.956 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:15 smithi146 ceph-mon[35011]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-12T02:54:15.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:15 smithi146 ceph-mon[35011]: osdmap e57: 9 total, 9 up, 9 in 2024-09-12T02:54:16.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:16 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/3364297180' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d9726168-7494-4bf8-b433-350e85bfd461"}]: dispatch 2024-09-12T02:54:16.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:16 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/3364297180' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d9726168-7494-4bf8-b433-350e85bfd461"}]': finished 2024-09-12T02:54:16.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:16 smithi005 ceph-mon[31197]: osdmap e58: 10 total, 9 up, 10 in 2024-09-12T02:54:16.811 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:16 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T02:54:16.811 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:16 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/3957569449' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:54:16.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:16 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/3364297180' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d9726168-7494-4bf8-b433-350e85bfd461"}]: dispatch 2024-09-12T02:54:16.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:16 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/3364297180' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d9726168-7494-4bf8-b433-350e85bfd461"}]': finished 2024-09-12T02:54:16.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:16 smithi148 ceph-mon[35090]: osdmap e58: 10 total, 9 up, 10 in 2024-09-12T02:54:16.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:16 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T02:54:16.917 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:16 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/3957569449' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:54:16.956 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:16 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/3364297180' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d9726168-7494-4bf8-b433-350e85bfd461"}]: dispatch 2024-09-12T02:54:16.956 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:16 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/3364297180' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d9726168-7494-4bf8-b433-350e85bfd461"}]': finished 2024-09-12T02:54:16.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:16 smithi146 ceph-mon[35011]: osdmap e58: 10 total, 9 up, 10 in 2024-09-12T02:54:16.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:16 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T02:54:16.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:16 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/3957569449' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:54:17.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:17 smithi005 ceph-mon[31197]: pgmap v191: 1 pgs: 1 remapped+peering; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-09-12T02:54:17.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:17 smithi005 ceph-mon[31197]: osdmap e59: 10 total, 9 up, 10 in 2024-09-12T02:54:17.811 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:17 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T02:54:17.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:17 smithi148 ceph-mon[35090]: pgmap v191: 1 pgs: 1 remapped+peering; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-09-12T02:54:17.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:17 smithi148 ceph-mon[35090]: osdmap e59: 10 total, 9 up, 10 in 2024-09-12T02:54:17.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:17 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T02:54:17.956 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:17 smithi146 ceph-mon[35011]: pgmap v191: 1 pgs: 1 remapped+peering; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-09-12T02:54:17.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:17 smithi146 ceph-mon[35011]: osdmap e59: 10 total, 9 up, 10 in 2024-09-12T02:54:17.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:17 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T02:54:18.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:18 smithi005 ceph-mon[31197]: osdmap e60: 10 total, 9 up, 10 in 2024-09-12T02:54:18.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:18 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T02:54:18.915 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:18 smithi148 ceph-mon[35090]: osdmap e60: 10 total, 9 up, 10 in 2024-09-12T02:54:18.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:18 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T02:54:18.956 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:18 smithi146 ceph-mon[35011]: osdmap e60: 10 total, 9 up, 10 in 2024-09-12T02:54:18.956 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:18 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T02:54:19.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:19 smithi146 ceph-mon[35011]: pgmap v194: 1 pgs: 1 remapped+peering; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-09-12T02:54:19.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:19 smithi005 ceph-mon[31197]: pgmap v194: 1 pgs: 1 remapped+peering; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-09-12T02:54:19.915 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:19 smithi148 ceph-mon[35090]: pgmap v194: 1 pgs: 1 remapped+peering; 577 KiB data, 256 MiB used, 804 GiB / 805 GiB avail 2024-09-12T02:54:20.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:20 smithi005 ceph-mon[31197]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 116 KiB/s, 0 objects/s recovering 2024-09-12T02:54:20.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:20 smithi148 ceph-mon[35090]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 116 KiB/s, 0 objects/s recovering 2024-09-12T02:54:20.956 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:20 smithi146 ceph-mon[35011]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 116 KiB/s, 0 objects/s recovering 2024-09-12T02:54:22.803 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:22 smithi005 ceph-mon[31197]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-09-12T02:54:22.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:22 smithi148 ceph-mon[35090]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-09-12T02:54:22.956 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:22 smithi146 ceph-mon[35011]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 95 KiB/s, 0 objects/s recovering 2024-09-12T02:54:24.687 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:24 smithi005 ceph-mon[31197]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-12T02:54:24.688 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:24 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-12T02:54:24.688 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:24 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:54:24.688 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:24 smithi005 ceph-mon[31197]: Deploying daemon osd.9 on smithi005 2024-09-12T02:54:24.915 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:24 smithi148 ceph-mon[35090]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-12T02:54:24.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:24 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-12T02:54:24.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:24 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:54:24.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:24 smithi148 ceph-mon[35090]: Deploying daemon osd.9 on smithi005 2024-09-12T02:54:24.956 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:24 smithi146 ceph-mon[35011]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-12T02:54:24.956 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:24 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-12T02:54:24.956 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:24 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:54:24.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:24 smithi146 ceph-mon[35011]: Deploying daemon osd.9 on smithi005 2024-09-12T02:54:26.781 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:26 smithi005 ceph-mon[31197]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-09-12T02:54:26.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:26 smithi148 ceph-mon[35090]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-09-12T02:54:26.956 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:26 smithi146 ceph-mon[35011]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-09-12T02:54:28.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:28 smithi005 ceph-mon[31197]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-09-12T02:54:28.915 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:28 smithi148 ceph-mon[35090]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-09-12T02:54:28.956 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:28 smithi146 ceph-mon[35011]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-09-12T02:54:30.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:30 smithi005 ceph-mon[31197]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-12T02:54:30.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:30 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:54:30.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:30 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:54:30.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:30 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:54:30.915 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:30 smithi148 ceph-mon[35090]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-12T02:54:30.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:30 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:54:30.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:30 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:54:30.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:30 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:54:30.956 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:30 smithi146 ceph-mon[35011]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-12T02:54:30.956 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:30 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:54:30.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:30 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:54:30.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:30 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:54:33.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:54:33.083+0000 7f25e4ff9640 1 -- 172.21.15.5:0/3284915698 <== mgr.14148 v2:172.21.15.5:6800/3780631642 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f25e8062630 con 0x7f25d40a2b80 2024-09-12T02:54:33.083 INFO:teuthology.orchestra.run.smithi005.stdout:Created osd(s) 9 on host 'smithi005' 2024-09-12T02:54:33.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:54:33.085+0000 7f25ede38640 1 -- 172.21.15.5:0/3284915698 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f25d40a2b80 msgr2=0x7f25d40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:33.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:54:33.085+0000 7f25ede38640 1 --2- 172.21.15.5:0/3284915698 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f25d40a2b80 0x7f25d40a5040 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f25d8011440 tx=0x7f25d8009500 comp rx=0 tx=0).stop 2024-09-12T02:54:33.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:54:33.085+0000 7f25ede38640 1 -- 172.21.15.5:0/3284915698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25e8103870 msgr2=0x7f25e81013a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:33.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:54:33.085+0000 7f25ede38640 1 --2- 172.21.15.5:0/3284915698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25e8103870 0x7f25e81013a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f25e0009900 tx=0x7f25e00042f0 comp rx=0 tx=0).stop 2024-09-12T02:54:33.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:54:33.085+0000 7f25ede38640 1 -- 172.21.15.5:0/3284915698 shutdown_connections 2024-09-12T02:54:33.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:54:33.085+0000 7f25ede38640 1 --2- 172.21.15.5:0/3284915698 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f25d40a2b80 0x7f25d40a5040 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:33.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:54:33.085+0000 7f25ede38640 1 --2- 172.21.15.5:0/3284915698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25e8103870 0x7f25e81013a0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:33.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:54:33.085+0000 7f25ede38640 1 --2- 172.21.15.5:0/3284915698 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f25e8102eb0 0x7f25e8100e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:33.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:54:33.085+0000 7f25ede38640 1 --2- 172.21.15.5:0/3284915698 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f25e80ffed0 0x7f25e8102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:33.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:54:33.085+0000 7f25ede38640 1 -- 172.21.15.5:0/3284915698 >> 172.21.15.5:0/3284915698 conn(0x7f25e80fba40 msgr2=0x7f25e8106280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:54:33.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:54:33.085+0000 7f25ede38640 1 -- 172.21.15.5:0/3284915698 shutdown_connections 2024-09-12T02:54:33.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:54:33.085+0000 7f25ede38640 1 -- 172.21.15.5:0/3284915698 wait complete. 2024-09-12T02:54:33.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:33 smithi005 ceph-mon[31197]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-09-12T02:54:33.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:33 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:54:33.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:33 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:54:33.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:33 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:54:33.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:33 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:54:33.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:33 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:54:33.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:33 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:54:33.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:33 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:54:33.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:33 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:54:33.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:33 smithi148 ceph-mon[35090]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-09-12T02:54:33.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:33 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:54:33.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:33 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:54:33.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:33 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:54:33.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:33 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:54:33.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:33 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:54:33.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:33 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:54:33.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:33 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:54:33.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:33 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:54:33.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:33 smithi146 ceph-mon[35011]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-09-12T02:54:33.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:33 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:54:33.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:33 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:54:33.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:33 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:54:33.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:33 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:54:33.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:33 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:54:33.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:33 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:54:33.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:33 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:54:33.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:33 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:54:33.761 DEBUG:teuthology.orchestra.run.smithi005:osd.9> sudo journalctl -f -n 0 -u ceph-618305f8-70b1-11ef-bcea-c7b262605968@osd.9.service 2024-09-12T02:54:33.764 INFO:tasks.cephadm:Deploying osd.10 on smithi146 with /dev/vg_nvme/lv_1... 2024-09-12T02:54:33.764 DEBUG:teuthology.orchestra.run.smithi146:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-12T02:54:34.020 INFO:teuthology.orchestra.run.smithi146.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T02:54:35.336 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:35 smithi146 ceph-mon[35011]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-09-12T02:54:35.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:35 smithi005 ceph-mon[31197]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-09-12T02:54:35.665 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:35 smithi148 ceph-mon[35090]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-09-12T02:54:36.061 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 02:54:35 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T02:54:35.746+0000 7fe45192b640 -1 osd.9 0 log_to_monitors true 2024-09-12T02:54:36.336 INFO:teuthology.orchestra.run.smithi146.stdout: 2024-09-12T02:54:36.361 DEBUG:teuthology.orchestra.run.smithi146:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi146:vg_nvme/lv_1 2024-09-12T02:54:36.440 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:36 smithi146 ceph-mon[35011]: from='osd.9 [v2:172.21.15.5:6826/1961857360,v1:172.21.15.5:6827/1961857360]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-12T02:54:36.497 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:36 smithi005 ceph-mon[31197]: from='osd.9 [v2:172.21.15.5:6826/1961857360,v1:172.21.15.5:6827/1961857360]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-12T02:54:36.665 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:36 smithi148 ceph-mon[35090]: from='osd.9 [v2:172.21.15.5:6826/1961857360,v1:172.21.15.5:6827/1961857360]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-12T02:54:36.679 INFO:teuthology.orchestra.run.smithi146.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T02:54:37.191 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:54:37.189+0000 7ffa435c6640 1 -- 172.21.15.146:0/3469526657 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffa3c102e00 msgr2=0x7ffa3c103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:37.191 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:54:37.189+0000 7ffa435c6640 1 --2- 172.21.15.146:0/3469526657 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffa3c102e00 0x7ffa3c103280 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c009900 tx=0x7ffa2c02a410 comp rx=0 tx=0).stop 2024-09-12T02:54:37.191 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:54:37.189+0000 7ffa435c6640 1 -- 172.21.15.146:0/3469526657 shutdown_connections 2024-09-12T02:54:37.191 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:54:37.189+0000 7ffa435c6640 1 --2- 172.21.15.146:0/3469526657 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa3c1037c0 0x7ffa3c10a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:37.192 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:54:37.189+0000 7ffa435c6640 1 --2- 172.21.15.146:0/3469526657 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffa3c102e00 0x7ffa3c103280 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:37.192 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:54:37.189+0000 7ffa435c6640 1 --2- 172.21.15.146:0/3469526657 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa3c101c00 0x7ffa3c102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:37.192 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:54:37.189+0000 7ffa435c6640 1 -- 172.21.15.146:0/3469526657 >> 172.21.15.146:0/3469526657 conn(0x7ffa3c0fd3b0 msgr2=0x7ffa3c0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:54:37.192 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:54:37.189+0000 7ffa435c6640 1 -- 172.21.15.146:0/3469526657 shutdown_connections 2024-09-12T02:54:37.192 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:54:37.189+0000 7ffa435c6640 1 -- 172.21.15.146:0/3469526657 wait complete. 2024-09-12T02:54:37.192 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:54:37.190+0000 7ffa435c6640 1 Processor -- start 2024-09-12T02:54:37.192 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:54:37.190+0000 7ffa435c6640 1 -- start start 2024-09-12T02:54:37.192 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:54:37.190+0000 7ffa435c6640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffa3c101c00 0x7ffa3c075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:37.192 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:54:37.190+0000 7ffa435c6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa3c102e00 0x7ffa3c0745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:37.192 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:54:37.190+0000 7ffa435c6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa3c1037c0 0x7ffa3c074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:37.192 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:54:37.190+0000 7ffa435c6640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ffa3c10d900 con 0x7ffa3c102e00 2024-09-12T02:54:37.193 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:54:37.190+0000 7ffa435c6640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ffa3c10d780 con 0x7ffa3c101c00 2024-09-12T02:54:37.193 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:54:37.190+0000 7ffa435c6640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ffa3c10da80 con 0x7ffa3c1037c0 2024-09-12T02:54:37.193 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:54:37.190+0000 7ffa41362640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffa3c101c00 0x7ffa3c075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:37.193 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:54:37.190+0000 7ffa41362640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffa3c101c00 0x7ffa3c075f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.146:46984/0 (socket says 172.21.15.146:46984) 2024-09-12T02:54:37.193 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:54:37.190+0000 7ffa41362640 1 -- 172.21.15.146:0/986720782 learned_addr learned my addr 172.21.15.146:0/986720782 (peer_addr_for_me v2:172.21.15.146:0/0) 2024-09-12T02:54:37.193 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:54:37.190+0000 7ffa40b61640 1 --2- 172.21.15.146:0/986720782 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa3c102e00 0x7ffa3c0745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:37.193 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:54:37.190+0000 7ffa41b63640 1 --2- 172.21.15.146:0/986720782 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa3c1037c0 0x7ffa3c074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:37.193 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:54:37.190+0000 7ffa41362640 1 -- 172.21.15.146:0/986720782 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa3c1037c0 msgr2=0x7ffa3c074b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:37.193 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:54:37.190+0000 7ffa41362640 1 --2- 172.21.15.146:0/986720782 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa3c1037c0 0x7ffa3c074b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:37.193 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:54:37.190+0000 7ffa41362640 1 -- 172.21.15.146:0/986720782 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa3c102e00 msgr2=0x7ffa3c0745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:37.193 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:54:37.190+0000 7ffa41362640 1 --2- 172.21.15.146:0/986720782 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa3c102e00 0x7ffa3c0745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:37.193 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:54:37.190+0000 7ffa41362640 1 -- 172.21.15.146:0/986720782 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa3c0753e0 con 0x7ffa3c101c00 2024-09-12T02:54:37.193 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:54:37.190+0000 7ffa40b61640 1 --2- 172.21.15.146:0/986720782 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa3c102e00 0x7ffa3c0745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:54:37.194 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:54:37.191+0000 7ffa41362640 1 --2- 172.21.15.146:0/986720782 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffa3c101c00 0x7ffa3c075f90 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ffa3800b450 tx=0x7ffa3800b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:54:37.194 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:54:37.191+0000 7ffa327fc640 1 -- 172.21.15.146:0/986720782 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa38015020 con 0x7ffa3c101c00 2024-09-12T02:54:37.194 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:54:37.191+0000 7ffa327fc640 1 -- 172.21.15.146:0/986720782 <== mon.2 v2:172.21.15.146:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa38011d70 con 0x7ffa3c101c00 2024-09-12T02:54:37.194 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:54:37.191+0000 7ffa327fc640 1 -- 172.21.15.146:0/986720782 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa3800f340 con 0x7ffa3c101c00 2024-09-12T02:54:37.194 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:54:37.191+0000 7ffa435c6640 1 -- 172.21.15.146:0/986720782 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa3c1a2120 con 0x7ffa3c101c00 2024-09-12T02:54:37.194 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:54:37.191+0000 7ffa435c6640 1 -- 172.21.15.146:0/986720782 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa3c1a2700 con 0x7ffa3c101c00 2024-09-12T02:54:37.195 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:54:37.192+0000 7ffa435c6640 1 -- 172.21.15.146:0/986720782 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa3c109070 con 0x7ffa3c101c00 2024-09-12T02:54:37.195 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:54:37.193+0000 7ffa327fc640 1 -- 172.21.15.146:0/986720782 <== mon.2 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7ffa38018020 con 0x7ffa3c101c00 2024-09-12T02:54:37.195 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:54:37.194+0000 7ffa327fc640 1 --2- 172.21.15.146:0/986720782 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7ffa280a2c00 0x7ffa280a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:54:37.195 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:54:37.194+0000 7ffa327fc640 1 -- 172.21.15.146:0/986720782 <== mon.2 v2:172.21.15.146:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5965+0+0 (secure 0 0 0) 0x7ffa380ce6b0 con 0x7ffa3c101c00 2024-09-12T02:54:37.195 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:54:37.194+0000 7ffa40b61640 1 --2- 172.21.15.146:0/986720782 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7ffa280a2c00 0x7ffa280a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:54:37.195 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:54:37.195+0000 7ffa40b61640 1 --2- 172.21.15.146:0/986720782 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7ffa280a2c00 0x7ffa280a50c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c000c00 tx=0x7ffa2c0315d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:54:37.197 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:54:37.197+0000 7ffa327fc640 1 -- 172.21.15.146:0/986720782 <== mon.2 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ffa3801f020 con 0x7ffa3c101c00 2024-09-12T02:54:37.380 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:54:37.380+0000 7ffa435c6640 1 -- 172.21.15.146:0/986720782 --> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi146:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7ffa3c062630 con 0x7ffa280a2c00 2024-09-12T02:54:37.380 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:37 smithi146 ceph-mon[35011]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-09-12T02:54:37.381 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:37 smithi146 ceph-mon[35011]: from='osd.9 [v2:172.21.15.5:6826/1961857360,v1:172.21.15.5:6827/1961857360]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-12T02:54:37.381 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:37 smithi146 ceph-mon[35011]: osdmap e61: 10 total, 9 up, 10 in 2024-09-12T02:54:37.381 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:37 smithi146 ceph-mon[35011]: from='osd.9 [v2:172.21.15.5:6826/1961857360,v1:172.21.15.5:6827/1961857360]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2024-09-12T02:54:37.381 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:37 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T02:54:37.381 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:37 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:54:37.381 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:37 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:54:37.381 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:37 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:54:37.381 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:37 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:54:37.381 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:37 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:54:37.381 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:37 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:54:37.381 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:37 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:54:37.381 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:37 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:54:37.381 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:37 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:54:37.381 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:37 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:54:37.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:37 smithi148 ceph-mon[35090]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-09-12T02:54:37.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:37 smithi148 ceph-mon[35090]: from='osd.9 [v2:172.21.15.5:6826/1961857360,v1:172.21.15.5:6827/1961857360]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-12T02:54:37.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:37 smithi148 ceph-mon[35090]: osdmap e61: 10 total, 9 up, 10 in 2024-09-12T02:54:37.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:37 smithi148 ceph-mon[35090]: from='osd.9 [v2:172.21.15.5:6826/1961857360,v1:172.21.15.5:6827/1961857360]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2024-09-12T02:54:37.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:37 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T02:54:37.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:37 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:54:37.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:37 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:54:37.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:37 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:54:37.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:37 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:54:37.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:37 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:54:37.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:37 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:54:37.667 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:37 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:54:37.667 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:37 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:54:37.667 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:37 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:54:37.667 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:37 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:54:37.711 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:37 smithi005 ceph-mon[31197]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-09-12T02:54:37.712 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:37 smithi005 ceph-mon[31197]: from='osd.9 [v2:172.21.15.5:6826/1961857360,v1:172.21.15.5:6827/1961857360]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-12T02:54:37.712 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:37 smithi005 ceph-mon[31197]: osdmap e61: 10 total, 9 up, 10 in 2024-09-12T02:54:37.712 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:37 smithi005 ceph-mon[31197]: from='osd.9 [v2:172.21.15.5:6826/1961857360,v1:172.21.15.5:6827/1961857360]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2024-09-12T02:54:37.712 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:37 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T02:54:37.712 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:37 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:54:37.712 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:37 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:54:37.712 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:37 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:54:37.712 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:37 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:54:37.712 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:37 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:54:37.712 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:37 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:54:37.712 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:37 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:54:37.713 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:37 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:54:37.713 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:37 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:54:37.713 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:37 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:54:38.310 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 02:54:38 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T02:54:38.063+0000 7fe44d08b640 -1 osd.9 0 waiting for initial osdmap 2024-09-12T02:54:38.311 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 02:54:38 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T02:54:38.076+0000 7fe4456bd640 -1 osd.9 62 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T02:54:38.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:38 smithi148 ceph-mon[35090]: Detected new or changed devices on smithi005 2024-09-12T02:54:38.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:38 smithi148 ceph-mon[35090]: Adjusting osd_memory_target on smithi005 to 4273M 2024-09-12T02:54:38.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:38 smithi148 ceph-mon[35090]: from='osd.9 [v2:172.21.15.5:6826/1961857360,v1:172.21.15.5:6827/1961857360]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2024-09-12T02:54:38.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:38 smithi148 ceph-mon[35090]: osdmap e62: 10 total, 9 up, 10 in 2024-09-12T02:54:38.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:38 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T02:54:38.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:38 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T02:54:38.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:38 smithi148 ceph-mon[35090]: from='client.24326 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi146:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:54:38.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:38 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:54:38.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:38 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:54:38.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:38 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:54:38.667 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:38 smithi148 ceph-mon[35090]: from='osd.9 [v2:172.21.15.5:6826/1961857360,v1:172.21.15.5:6827/1961857360]' entity='osd.9' 2024-09-12T02:54:38.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:38 smithi146 ceph-mon[35011]: Detected new or changed devices on smithi005 2024-09-12T02:54:38.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:38 smithi146 ceph-mon[35011]: Adjusting osd_memory_target on smithi005 to 4273M 2024-09-12T02:54:38.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:38 smithi146 ceph-mon[35011]: from='osd.9 [v2:172.21.15.5:6826/1961857360,v1:172.21.15.5:6827/1961857360]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2024-09-12T02:54:38.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:38 smithi146 ceph-mon[35011]: osdmap e62: 10 total, 9 up, 10 in 2024-09-12T02:54:38.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:38 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T02:54:38.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:38 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T02:54:38.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:38 smithi146 ceph-mon[35011]: from='client.24326 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi146:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:54:38.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:38 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:54:38.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:38 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:54:38.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:38 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:54:38.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:38 smithi146 ceph-mon[35011]: from='osd.9 [v2:172.21.15.5:6826/1961857360,v1:172.21.15.5:6827/1961857360]' entity='osd.9' 2024-09-12T02:54:38.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:38 smithi005 ceph-mon[31197]: Detected new or changed devices on smithi005 2024-09-12T02:54:38.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:38 smithi005 ceph-mon[31197]: Adjusting osd_memory_target on smithi005 to 4273M 2024-09-12T02:54:38.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:38 smithi005 ceph-mon[31197]: from='osd.9 [v2:172.21.15.5:6826/1961857360,v1:172.21.15.5:6827/1961857360]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2024-09-12T02:54:38.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:38 smithi005 ceph-mon[31197]: osdmap e62: 10 total, 9 up, 10 in 2024-09-12T02:54:38.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:38 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T02:54:38.811 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:38 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T02:54:38.811 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:38 smithi005 ceph-mon[31197]: from='client.24326 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi146:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:54:38.811 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:38 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:54:38.811 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:38 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:54:38.811 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:38 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:54:38.811 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:38 smithi005 ceph-mon[31197]: from='osd.9 [v2:172.21.15.5:6826/1961857360,v1:172.21.15.5:6827/1961857360]' entity='osd.9' 2024-09-12T02:54:39.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:39 smithi005 ceph-mon[31197]: purged_snaps scrub starts 2024-09-12T02:54:39.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:39 smithi005 ceph-mon[31197]: purged_snaps scrub ok 2024-09-12T02:54:39.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:39 smithi005 ceph-mon[31197]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-09-12T02:54:39.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:39 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T02:54:39.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:39 smithi005 ceph-mon[31197]: osd.9 [v2:172.21.15.5:6826/1961857360,v1:172.21.15.5:6827/1961857360] boot 2024-09-12T02:54:39.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:39 smithi005 ceph-mon[31197]: osdmap e63: 10 total, 10 up, 10 in 2024-09-12T02:54:39.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:39 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T02:54:39.590 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:39 smithi148 ceph-mon[35090]: purged_snaps scrub starts 2024-09-12T02:54:39.590 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:39 smithi148 ceph-mon[35090]: purged_snaps scrub ok 2024-09-12T02:54:39.590 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:39 smithi148 ceph-mon[35090]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-09-12T02:54:39.590 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:39 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T02:54:39.591 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:39 smithi148 ceph-mon[35090]: osd.9 [v2:172.21.15.5:6826/1961857360,v1:172.21.15.5:6827/1961857360] boot 2024-09-12T02:54:39.591 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:39 smithi148 ceph-mon[35090]: osdmap e63: 10 total, 10 up, 10 in 2024-09-12T02:54:39.591 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:39 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T02:54:39.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:39 smithi146 ceph-mon[35011]: purged_snaps scrub starts 2024-09-12T02:54:39.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:39 smithi146 ceph-mon[35011]: purged_snaps scrub ok 2024-09-12T02:54:39.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:39 smithi146 ceph-mon[35011]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 257 MiB used, 804 GiB / 805 GiB avail 2024-09-12T02:54:39.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:39 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T02:54:39.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:39 smithi146 ceph-mon[35011]: osd.9 [v2:172.21.15.5:6826/1961857360,v1:172.21.15.5:6827/1961857360] boot 2024-09-12T02:54:39.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:39 smithi146 ceph-mon[35011]: osdmap e63: 10 total, 10 up, 10 in 2024-09-12T02:54:39.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:39 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-12T02:54:40.665 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:40 smithi148 ceph-mon[35090]: from='client.? 172.21.15.146:0/4235197988' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "519ec60f-3606-48ac-9043-6f59a798766d"}]: dispatch 2024-09-12T02:54:40.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:40 smithi148 ceph-mon[35090]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "519ec60f-3606-48ac-9043-6f59a798766d"}]: dispatch 2024-09-12T02:54:40.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:40 smithi148 ceph-mon[35090]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "519ec60f-3606-48ac-9043-6f59a798766d"}]': finished 2024-09-12T02:54:40.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:40 smithi148 ceph-mon[35090]: osdmap e64: 11 total, 10 up, 11 in 2024-09-12T02:54:40.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:40 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T02:54:40.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:40 smithi148 ceph-mon[35090]: from='client.? 172.21.15.146:0/181712717' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:54:40.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:40 smithi146 ceph-mon[35011]: from='client.? 172.21.15.146:0/4235197988' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "519ec60f-3606-48ac-9043-6f59a798766d"}]: dispatch 2024-09-12T02:54:40.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:40 smithi146 ceph-mon[35011]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "519ec60f-3606-48ac-9043-6f59a798766d"}]: dispatch 2024-09-12T02:54:40.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:40 smithi146 ceph-mon[35011]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "519ec60f-3606-48ac-9043-6f59a798766d"}]': finished 2024-09-12T02:54:40.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:40 smithi146 ceph-mon[35011]: osdmap e64: 11 total, 10 up, 11 in 2024-09-12T02:54:40.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:40 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T02:54:40.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:40 smithi146 ceph-mon[35011]: from='client.? 172.21.15.146:0/181712717' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:54:40.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:40 smithi005 ceph-mon[31197]: from='client.? 172.21.15.146:0/4235197988' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "519ec60f-3606-48ac-9043-6f59a798766d"}]: dispatch 2024-09-12T02:54:40.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:40 smithi005 ceph-mon[31197]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "519ec60f-3606-48ac-9043-6f59a798766d"}]: dispatch 2024-09-12T02:54:40.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:40 smithi005 ceph-mon[31197]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "519ec60f-3606-48ac-9043-6f59a798766d"}]': finished 2024-09-12T02:54:40.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:40 smithi005 ceph-mon[31197]: osdmap e64: 11 total, 10 up, 11 in 2024-09-12T02:54:40.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:40 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T02:54:40.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:40 smithi005 ceph-mon[31197]: from='client.? 172.21.15.146:0/181712717' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:54:41.668 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:41 smithi148 ceph-mon[35090]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:54:41.668 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:41 smithi148 ceph-mon[35090]: osdmap e65: 11 total, 10 up, 11 in 2024-09-12T02:54:41.668 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:41 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T02:54:41.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:41 smithi146 ceph-mon[35011]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:54:41.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:41 smithi146 ceph-mon[35011]: osdmap e65: 11 total, 10 up, 11 in 2024-09-12T02:54:41.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:41 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T02:54:41.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:41 smithi005 ceph-mon[31197]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:54:41.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:41 smithi005 ceph-mon[31197]: osdmap e65: 11 total, 10 up, 11 in 2024-09-12T02:54:41.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:41 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T02:54:43.665 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:43 smithi148 ceph-mon[35090]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:54:43.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:43 smithi146 ceph-mon[35011]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:54:43.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:43 smithi005 ceph-mon[31197]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:54:45.442 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:45 smithi146 ceph-mon[35011]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:54:45.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:45 smithi148 ceph-mon[35090]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:54:45.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:45 smithi005 ceph-mon[31197]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:54:47.349 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:47 smithi146 ceph-mon[35011]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:54:47.665 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:47 smithi148 ceph-mon[35090]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:54:47.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:47 smithi005 ceph-mon[31197]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:54:49.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:49 smithi146 ceph-mon[35011]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:54:49.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:49 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-12T02:54:49.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:49 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:54:49.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:49 smithi146 ceph-mon[35011]: Deploying daemon osd.10 on smithi146 2024-09-12T02:54:49.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:49 smithi005 ceph-mon[31197]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:54:49.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:49 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-12T02:54:49.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:49 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:54:49.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:49 smithi005 ceph-mon[31197]: Deploying daemon osd.10 on smithi146 2024-09-12T02:54:49.665 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:49 smithi148 ceph-mon[35090]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:54:49.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:49 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-12T02:54:49.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:49 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:54:49.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:49 smithi148 ceph-mon[35090]: Deploying daemon osd.10 on smithi146 2024-09-12T02:54:51.624 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:51 smithi146 ceph-mon[35011]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:54:51.665 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:51 smithi148 ceph-mon[35090]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:54:51.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:51 smithi005 ceph-mon[31197]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:54:53.403 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:53 smithi146 ceph-mon[35011]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:54:53.665 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:53 smithi148 ceph-mon[35090]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:54:53.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:53 smithi005 ceph-mon[31197]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:54:54.651 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:54 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:54:54.651 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:54 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:54:54.651 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:54 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:54:54.665 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:54 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:54:54.665 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:54 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:54:54.665 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:54 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:54:54.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:54 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:54:54.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:54 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:54:54.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:54 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:54:55.489 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:55 smithi146 ceph-mon[35011]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:54:55.665 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:55 smithi148 ceph-mon[35090]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:54:55.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:55 smithi005 ceph-mon[31197]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:54:56.929 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:54:56.927+0000 7ffa327fc640 1 -- 172.21.15.146:0/986720782 <== mgr.14148 v2:172.21.15.5:6800/3780631642 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7ffa3c062630 con 0x7ffa280a2c00 2024-09-12T02:54:56.929 INFO:teuthology.orchestra.run.smithi146.stdout:Created osd(s) 10 on host 'smithi146' 2024-09-12T02:54:56.930 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:54:56.930+0000 7ffa435c6640 1 -- 172.21.15.146:0/986720782 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7ffa280a2c00 msgr2=0x7ffa280a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:56.930 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:54:56.930+0000 7ffa435c6640 1 --2- 172.21.15.146:0/986720782 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7ffa280a2c00 0x7ffa280a50c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c000c00 tx=0x7ffa2c0315d0 comp rx=0 tx=0).stop 2024-09-12T02:54:56.930 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:54:56.930+0000 7ffa435c6640 1 -- 172.21.15.146:0/986720782 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffa3c101c00 msgr2=0x7ffa3c075f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:54:56.930 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:54:56.930+0000 7ffa435c6640 1 --2- 172.21.15.146:0/986720782 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffa3c101c00 0x7ffa3c075f90 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ffa3800b450 tx=0x7ffa3800b910 comp rx=0 tx=0).stop 2024-09-12T02:54:56.930 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:54:56.930+0000 7ffa435c6640 1 -- 172.21.15.146:0/986720782 shutdown_connections 2024-09-12T02:54:56.930 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:54:56.930+0000 7ffa435c6640 1 --2- 172.21.15.146:0/986720782 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7ffa280a2c00 0x7ffa280a50c0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:56.930 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:54:56.930+0000 7ffa435c6640 1 --2- 172.21.15.146:0/986720782 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa3c102e00 0x7ffa3c0745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:56.930 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:54:56.930+0000 7ffa435c6640 1 --2- 172.21.15.146:0/986720782 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa3c1037c0 0x7ffa3c074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:56.931 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:54:56.930+0000 7ffa435c6640 1 --2- 172.21.15.146:0/986720782 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffa3c101c00 0x7ffa3c075f90 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:54:56.931 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:54:56.930+0000 7ffa435c6640 1 -- 172.21.15.146:0/986720782 >> 172.21.15.146:0/986720782 conn(0x7ffa3c0fd3b0 msgr2=0x7ffa3c106240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:54:56.931 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:54:56.930+0000 7ffa435c6640 1 -- 172.21.15.146:0/986720782 shutdown_connections 2024-09-12T02:54:56.931 INFO:teuthology.orchestra.run.smithi146.stderr:2024-09-12T02:54:56.930+0000 7ffa435c6640 1 -- 172.21.15.146:0/986720782 wait complete. 2024-09-12T02:54:57.497 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:57 smithi146 ceph-mon[35011]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:54:57.497 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:57 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:54:57.497 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:57 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:54:57.499 DEBUG:teuthology.orchestra.run.smithi146:osd.10> sudo journalctl -f -n 0 -u ceph-618305f8-70b1-11ef-bcea-c7b262605968@osd.10.service 2024-09-12T02:54:57.542 INFO:tasks.cephadm:Deploying osd.11 on smithi148 with /dev/vg_nvme/lv_1... 2024-09-12T02:54:57.543 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-12T02:54:57.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:57 smithi148 ceph-mon[35090]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:54:57.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:57 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:54:57.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:57 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:54:57.800 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T02:54:57.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:57 smithi005 ceph-mon[31197]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:54:57.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:57 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:54:57.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:57 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:54:58.885 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:58 smithi148 ceph-mon[35090]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:54:58.885 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:58 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:54:58.885 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:54:58 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:54:58.956 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:58 smithi146 ceph-mon[35011]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:54:58.956 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:58 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:54:58.956 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:54:58 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:54:59.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:58 smithi005 ceph-mon[31197]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:54:59.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:58 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:54:59.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:54:58 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:00.414 INFO:teuthology.orchestra.run.smithi148.stdout: 2024-09-12T02:55:00.439 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi148:vg_nvme/lv_1 2024-09-12T02:55:00.565 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 02:55:00 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T02:55:00.313+0000 7f8b64ed5640 -1 osd.10 0 log_to_monitors true 2024-09-12T02:55:00.697 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T02:55:00.956 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:00 smithi146 ceph-mon[35011]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:55:00.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:00 smithi146 ceph-mon[35011]: from='osd.10 [v2:172.21.15.146:6824/2041733103,v1:172.21.15.146:6825/2041733103]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-12T02:55:00.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:00 smithi146 ceph-mon[35011]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-12T02:55:00.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:00 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:00.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:00 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:00.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:00 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:55:00.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:00 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:55:00.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:00 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:55:00.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:00 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:55:00.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:00 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:00.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:00 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:55:00.958 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:00 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:55:00.958 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:00 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:00.960 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:00 smithi148 ceph-mon[35090]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:55:00.960 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:00 smithi148 ceph-mon[35090]: from='osd.10 [v2:172.21.15.146:6824/2041733103,v1:172.21.15.146:6825/2041733103]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-12T02:55:00.960 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:00 smithi148 ceph-mon[35090]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-12T02:55:00.960 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:00 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:00.960 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:00 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:00.960 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:00 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:55:00.961 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:00 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:55:00.961 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:00 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:55:00.961 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:00 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:55:00.961 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:00 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:00.961 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:00 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:55:00.961 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:00 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:55:00.961 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:00 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:01.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:00 smithi005 ceph-mon[31197]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:55:01.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:00 smithi005 ceph-mon[31197]: from='osd.10 [v2:172.21.15.146:6824/2041733103,v1:172.21.15.146:6825/2041733103]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-12T02:55:01.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:00 smithi005 ceph-mon[31197]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-12T02:55:01.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:00 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:01.061 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:00 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:01.061 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:00 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:55:01.061 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:00 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:55:01.061 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:00 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:55:01.061 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:00 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:55:01.061 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:00 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:01.061 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:00 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:55:01.061 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:00 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:55:01.061 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:00 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:01.156 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:55:01.149+0000 7f71eca63640 1 -- 172.21.15.148:0/2499207953 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f71e81026c0 msgr2=0x7f71e8108f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:01.156 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:55:01.149+0000 7f71eca63640 1 --2- 172.21.15.148:0/2499207953 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f71e81026c0 0x7f71e8108f50 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f71e0009ca0 tx=0x7f71e002a8c0 comp rx=0 tx=0).stop 2024-09-12T02:55:01.157 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:55:01.149+0000 7f71eca63640 1 -- 172.21.15.148:0/2499207953 shutdown_connections 2024-09-12T02:55:01.157 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:55:01.149+0000 7f71eca63640 1 --2- 172.21.15.148:0/2499207953 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71e8101d00 0x7f71e8102180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:01.157 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:55:01.149+0000 7f71eca63640 1 --2- 172.21.15.148:0/2499207953 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f71e81026c0 0x7f71e8108f50 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:01.157 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:55:01.149+0000 7f71eca63640 1 --2- 172.21.15.148:0/2499207953 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f71e8100b00 0x7f71e8100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:01.157 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:55:01.149+0000 7f71eca63640 1 -- 172.21.15.148:0/2499207953 >> 172.21.15.148:0/2499207953 conn(0x7f71e80fc270 msgr2=0x7f71e80fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:55:01.157 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:55:01.149+0000 7f71eca63640 1 -- 172.21.15.148:0/2499207953 shutdown_connections 2024-09-12T02:55:01.157 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:55:01.149+0000 7f71eca63640 1 -- 172.21.15.148:0/2499207953 wait complete. 2024-09-12T02:55:01.157 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:55:01.150+0000 7f71eca63640 1 Processor -- start 2024-09-12T02:55:01.157 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:55:01.150+0000 7f71eca63640 1 -- start start 2024-09-12T02:55:01.157 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:55:01.150+0000 7f71eca63640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f71e8100b00 0x7f71e8197e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:01.157 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:55:01.150+0000 7f71eca63640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71e8101d00 0x7f71e81983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:01.157 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:55:01.150+0000 7f71eca63640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f71e81026c0 0x7f71e819f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:01.158 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:55:01.150+0000 7f71eca63640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f71e810c780 con 0x7f71e8101d00 2024-09-12T02:55:01.158 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:55:01.150+0000 7f71eca63640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f71e810c600 con 0x7f71e8100b00 2024-09-12T02:55:01.158 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:55:01.150+0000 7f71eca63640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f71e810c900 con 0x7f71e81026c0 2024-09-12T02:55:01.158 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:55:01.150+0000 7f71e6d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f71e81026c0 0x7f71e819f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:01.158 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:55:01.150+0000 7f71e6d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f71e81026c0 0x7f71e819f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:37122/0 (socket says 172.21.15.148:37122) 2024-09-12T02:55:01.158 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:55:01.150+0000 7f71e6d9d640 1 -- 172.21.15.148:0/2250012714 learned_addr learned my addr 172.21.15.148:0/2250012714 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-09-12T02:55:01.158 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:55:01.150+0000 7f71e5d9b640 1 --2- 172.21.15.148:0/2250012714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71e8101d00 0x7f71e81983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:01.158 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:55:01.150+0000 7f71e659c640 1 --2- 172.21.15.148:0/2250012714 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f71e8100b00 0x7f71e8197e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:01.158 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:55:01.150+0000 7f71e6d9d640 1 -- 172.21.15.148:0/2250012714 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f71e8100b00 msgr2=0x7f71e8197e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:01.158 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:55:01.150+0000 7f71e6d9d640 1 --2- 172.21.15.148:0/2250012714 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f71e8100b00 0x7f71e8197e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:01.159 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:55:01.150+0000 7f71e6d9d640 1 -- 172.21.15.148:0/2250012714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71e8101d00 msgr2=0x7f71e81983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:01.159 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:55:01.150+0000 7f71e6d9d640 1 --2- 172.21.15.148:0/2250012714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71e8101d00 0x7f71e81983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:01.159 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:55:01.150+0000 7f71e6d9d640 1 -- 172.21.15.148:0/2250012714 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71e819fac0 con 0x7f71e81026c0 2024-09-12T02:55:01.159 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:55:01.151+0000 7f71e6d9d640 1 --2- 172.21.15.148:0/2250012714 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f71e81026c0 0x7f71e819f450 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f71e0005e30 tx=0x7f71e0004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:01.159 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:55:01.151+0000 7f71d77fe640 1 -- 172.21.15.148:0/2250012714 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e0003b70 con 0x7f71e81026c0 2024-09-12T02:55:01.159 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:55:01.151+0000 7f71eca63640 1 -- 172.21.15.148:0/2250012714 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71e819fd50 con 0x7f71e81026c0 2024-09-12T02:55:01.159 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:55:01.151+0000 7f71d77fe640 1 -- 172.21.15.148:0/2250012714 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71e0003d10 con 0x7f71e81026c0 2024-09-12T02:55:01.159 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:55:01.151+0000 7f71d77fe640 1 -- 172.21.15.148:0/2250012714 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e002fb10 con 0x7f71e81026c0 2024-09-12T02:55:01.159 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:55:01.151+0000 7f71eca63640 1 -- 172.21.15.148:0/2250012714 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71e81a0230 con 0x7f71e81026c0 2024-09-12T02:55:01.159 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:55:01.152+0000 7f71eca63640 1 -- 172.21.15.148:0/2250012714 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71e81a0460 con 0x7f71e81026c0 2024-09-12T02:55:01.159 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:55:01.154+0000 7f71d77fe640 1 -- 172.21.15.148:0/2250012714 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f71e00045c0 con 0x7f71e81026c0 2024-09-12T02:55:01.159 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:55:01.155+0000 7f71d77fe640 1 --2- 172.21.15.148:0/2250012714 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f71d00a2bb0 0x7f71d00a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:01.159 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:55:01.155+0000 7f71d77fe640 1 -- 172.21.15.148:0/2250012714 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6398+0+0 (secure 0 0 0) 0x7f71e00efab0 con 0x7f71e81026c0 2024-09-12T02:55:01.160 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:55:01.155+0000 7f71e659c640 1 --2- 172.21.15.148:0/2250012714 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f71d00a2bb0 0x7f71d00a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:01.160 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:55:01.156+0000 7f71e659c640 1 --2- 172.21.15.148:0/2250012714 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f71d00a2bb0 0x7f71d00a5070 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f71e8101b60 tx=0x7f71dc005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:01.160 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:55:01.158+0000 7f71d77fe640 1 -- 172.21.15.148:0/2250012714 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f71e00bd220 con 0x7f71e81026c0 2024-09-12T02:55:01.342 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:55:01.340+0000 7f71eca63640 1 -- 172.21.15.148:0/2250012714 --> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f71e81a0630 con 0x7f71d00a2bb0 2024-09-12T02:55:01.956 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:01 smithi146 ceph-mon[35011]: Detected new or changed devices on smithi146 2024-09-12T02:55:01.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:01 smithi146 ceph-mon[35011]: Adjusting osd_memory_target on smithi146 to 4272M 2024-09-12T02:55:01.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:01 smithi146 ceph-mon[35011]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-12T02:55:01.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:01 smithi146 ceph-mon[35011]: osdmap e66: 11 total, 10 up, 11 in 2024-09-12T02:55:01.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:01 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T02:55:01.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:01 smithi146 ceph-mon[35011]: from='osd.10 [v2:172.21.15.146:6824/2041733103,v1:172.21.15.146:6825/2041733103]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2024-09-12T02:55:01.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:01 smithi146 ceph-mon[35011]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2024-09-12T02:55:01.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:01 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:55:01.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:01 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:55:01.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:01 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:55:01.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:01 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:01.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:01 smithi146 ceph-mon[35011]: from='client.24376 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:55:01.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:01 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:55:01.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:01 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:55:01.958 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:01 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:55:02.010 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:01 smithi148 ceph-mon[35090]: Detected new or changed devices on smithi146 2024-09-12T02:55:02.010 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:01 smithi148 ceph-mon[35090]: Adjusting osd_memory_target on smithi146 to 4272M 2024-09-12T02:55:02.010 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:01 smithi148 ceph-mon[35090]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-12T02:55:02.010 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:01 smithi148 ceph-mon[35090]: osdmap e66: 11 total, 10 up, 11 in 2024-09-12T02:55:02.010 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:01 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T02:55:02.010 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:01 smithi148 ceph-mon[35090]: from='osd.10 [v2:172.21.15.146:6824/2041733103,v1:172.21.15.146:6825/2041733103]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2024-09-12T02:55:02.010 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:01 smithi148 ceph-mon[35090]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2024-09-12T02:55:02.011 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:01 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:55:02.011 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:01 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:55:02.011 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:01 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:55:02.011 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:01 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:02.011 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:01 smithi148 ceph-mon[35090]: from='client.24376 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:55:02.011 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:01 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:55:02.011 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:01 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:55:02.011 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:01 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:55:02.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:01 smithi005 ceph-mon[31197]: Detected new or changed devices on smithi146 2024-09-12T02:55:02.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:01 smithi005 ceph-mon[31197]: Adjusting osd_memory_target on smithi146 to 4272M 2024-09-12T02:55:02.061 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:01 smithi005 ceph-mon[31197]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-12T02:55:02.061 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:01 smithi005 ceph-mon[31197]: osdmap e66: 11 total, 10 up, 11 in 2024-09-12T02:55:02.061 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:01 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T02:55:02.061 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:01 smithi005 ceph-mon[31197]: from='osd.10 [v2:172.21.15.146:6824/2041733103,v1:172.21.15.146:6825/2041733103]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2024-09-12T02:55:02.061 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:01 smithi005 ceph-mon[31197]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]: dispatch 2024-09-12T02:55:02.061 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:01 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:55:02.061 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:01 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:55:02.061 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:01 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:55:02.061 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:01 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:02.061 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:01 smithi005 ceph-mon[31197]: from='client.24376 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi148:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:55:02.061 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:01 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-12T02:55:02.061 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:01 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-12T02:55:02.062 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:01 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:55:02.956 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 02:55:02 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T02:55:02.471+0000 7f8b60635640 -1 osd.10 0 waiting for initial osdmap 2024-09-12T02:55:02.957 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 02:55:02 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T02:55:02.488+0000 7f8b58c67640 -1 osd.10 67 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T02:55:02.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:02 smithi146 ceph-mon[35011]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:55:02.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:02 smithi146 ceph-mon[35011]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]': finished 2024-09-12T02:55:02.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:02 smithi146 ceph-mon[35011]: osdmap e67: 11 total, 10 up, 11 in 2024-09-12T02:55:02.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:02 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T02:55:02.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:02 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T02:55:02.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:02 smithi146 ceph-mon[35011]: from='osd.10 ' entity='osd.10' 2024-09-12T02:55:02.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:02 smithi146 ceph-mon[35011]: osd.10 [v2:172.21.15.146:6824/2041733103,v1:172.21.15.146:6825/2041733103] boot 2024-09-12T02:55:02.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:02 smithi146 ceph-mon[35011]: osdmap e68: 11 total, 11 up, 11 in 2024-09-12T02:55:02.958 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:02 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T02:55:02.993 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:02 smithi148 ceph-mon[35090]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:55:02.993 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:02 smithi148 ceph-mon[35090]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]': finished 2024-09-12T02:55:02.993 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:02 smithi148 ceph-mon[35090]: osdmap e67: 11 total, 10 up, 11 in 2024-09-12T02:55:02.993 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:02 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T02:55:02.994 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:02 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T02:55:02.994 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:02 smithi148 ceph-mon[35090]: from='osd.10 ' entity='osd.10' 2024-09-12T02:55:02.994 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:02 smithi148 ceph-mon[35090]: osd.10 [v2:172.21.15.146:6824/2041733103,v1:172.21.15.146:6825/2041733103] boot 2024-09-12T02:55:02.994 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:02 smithi148 ceph-mon[35090]: osdmap e68: 11 total, 11 up, 11 in 2024-09-12T02:55:02.994 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:02 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T02:55:03.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:02 smithi005 ceph-mon[31197]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-09-12T02:55:03.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:02 smithi005 ceph-mon[31197]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi146", "root=default"]}]': finished 2024-09-12T02:55:03.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:02 smithi005 ceph-mon[31197]: osdmap e67: 11 total, 10 up, 11 in 2024-09-12T02:55:03.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:02 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T02:55:03.061 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:02 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T02:55:03.061 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:02 smithi005 ceph-mon[31197]: from='osd.10 ' entity='osd.10' 2024-09-12T02:55:03.061 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:02 smithi005 ceph-mon[31197]: osd.10 [v2:172.21.15.146:6824/2041733103,v1:172.21.15.146:6825/2041733103] boot 2024-09-12T02:55:03.061 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:02 smithi005 ceph-mon[31197]: osdmap e68: 11 total, 11 up, 11 in 2024-09-12T02:55:03.061 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:02 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-12T02:55:03.976 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:03 smithi148 ceph-mon[35090]: purged_snaps scrub starts 2024-09-12T02:55:03.977 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:03 smithi148 ceph-mon[35090]: purged_snaps scrub ok 2024-09-12T02:55:03.977 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:03 smithi148 ceph-mon[35090]: osdmap e69: 11 total, 11 up, 11 in 2024-09-12T02:55:04.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:03 smithi005 ceph-mon[31197]: purged_snaps scrub starts 2024-09-12T02:55:04.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:03 smithi005 ceph-mon[31197]: purged_snaps scrub ok 2024-09-12T02:55:04.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:03 smithi005 ceph-mon[31197]: osdmap e69: 11 total, 11 up, 11 in 2024-09-12T02:55:04.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:03 smithi146 ceph-mon[35011]: purged_snaps scrub starts 2024-09-12T02:55:04.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:03 smithi146 ceph-mon[35011]: purged_snaps scrub ok 2024-09-12T02:55:04.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:03 smithi146 ceph-mon[35011]: osdmap e69: 11 total, 11 up, 11 in 2024-09-12T02:55:05.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:04 smithi005 ceph-mon[31197]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 727 MiB used, 983 GiB / 983 GiB avail 2024-09-12T02:55:05.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:04 smithi005 ceph-mon[31197]: from='client.? 172.21.15.148:0/500398950' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0f92d0cb-ba54-4bc3-a6d0-3c54d910556a"}]: dispatch 2024-09-12T02:55:05.061 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:04 smithi005 ceph-mon[31197]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0f92d0cb-ba54-4bc3-a6d0-3c54d910556a"}]: dispatch 2024-09-12T02:55:05.061 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:04 smithi005 ceph-mon[31197]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0f92d0cb-ba54-4bc3-a6d0-3c54d910556a"}]': finished 2024-09-12T02:55:05.061 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:04 smithi005 ceph-mon[31197]: osdmap e70: 12 total, 11 up, 12 in 2024-09-12T02:55:05.061 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:04 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T02:55:05.061 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:04 smithi005 ceph-mon[31197]: from='client.? 172.21.15.148:0/2028451273' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:55:05.165 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:04 smithi148 ceph-mon[35090]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 727 MiB used, 983 GiB / 983 GiB avail 2024-09-12T02:55:05.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:04 smithi148 ceph-mon[35090]: from='client.? 172.21.15.148:0/500398950' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0f92d0cb-ba54-4bc3-a6d0-3c54d910556a"}]: dispatch 2024-09-12T02:55:05.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:04 smithi148 ceph-mon[35090]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0f92d0cb-ba54-4bc3-a6d0-3c54d910556a"}]: dispatch 2024-09-12T02:55:05.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:04 smithi148 ceph-mon[35090]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0f92d0cb-ba54-4bc3-a6d0-3c54d910556a"}]': finished 2024-09-12T02:55:05.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:04 smithi148 ceph-mon[35090]: osdmap e70: 12 total, 11 up, 12 in 2024-09-12T02:55:05.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:04 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T02:55:05.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:04 smithi148 ceph-mon[35090]: from='client.? 172.21.15.148:0/2028451273' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:55:05.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:04 smithi146 ceph-mon[35011]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 727 MiB used, 983 GiB / 983 GiB avail 2024-09-12T02:55:05.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:04 smithi146 ceph-mon[35011]: from='client.? 172.21.15.148:0/500398950' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0f92d0cb-ba54-4bc3-a6d0-3c54d910556a"}]: dispatch 2024-09-12T02:55:05.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:04 smithi146 ceph-mon[35011]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0f92d0cb-ba54-4bc3-a6d0-3c54d910556a"}]: dispatch 2024-09-12T02:55:05.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:04 smithi146 ceph-mon[35011]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0f92d0cb-ba54-4bc3-a6d0-3c54d910556a"}]': finished 2024-09-12T02:55:05.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:04 smithi146 ceph-mon[35011]: osdmap e70: 12 total, 11 up, 12 in 2024-09-12T02:55:05.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:04 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T02:55:05.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:04 smithi146 ceph-mon[35011]: from='client.? 172.21.15.148:0/2028451273' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-12T02:55:06.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:05 smithi005 ceph-mon[31197]: osdmap e71: 12 total, 11 up, 12 in 2024-09-12T02:55:06.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:05 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T02:55:06.165 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:05 smithi148 ceph-mon[35090]: osdmap e71: 12 total, 11 up, 12 in 2024-09-12T02:55:06.165 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:05 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T02:55:06.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:05 smithi146 ceph-mon[35011]: osdmap e71: 12 total, 11 up, 12 in 2024-09-12T02:55:06.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:05 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T02:55:07.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:06 smithi005 ceph-mon[31197]: pgmap v229: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 727 MiB used, 983 GiB / 983 GiB avail; 151 KiB/s, 0 objects/s recovering 2024-09-12T02:55:07.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:06 smithi005 ceph-mon[31197]: osdmap e72: 12 total, 11 up, 12 in 2024-09-12T02:55:07.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:06 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T02:55:07.165 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:06 smithi148 ceph-mon[35090]: pgmap v229: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 727 MiB used, 983 GiB / 983 GiB avail; 151 KiB/s, 0 objects/s recovering 2024-09-12T02:55:07.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:06 smithi148 ceph-mon[35090]: osdmap e72: 12 total, 11 up, 12 in 2024-09-12T02:55:07.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:06 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T02:55:07.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:06 smithi146 ceph-mon[35011]: pgmap v229: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 727 MiB used, 983 GiB / 983 GiB avail; 151 KiB/s, 0 objects/s recovering 2024-09-12T02:55:07.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:06 smithi146 ceph-mon[35011]: osdmap e72: 12 total, 11 up, 12 in 2024-09-12T02:55:07.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:06 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T02:55:09.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:08 smithi005 ceph-mon[31197]: pgmap v231: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 727 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-12T02:55:09.101 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:08 smithi148 ceph-mon[35090]: pgmap v231: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 727 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-12T02:55:09.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:08 smithi146 ceph-mon[35011]: pgmap v231: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 727 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-12T02:55:10.915 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:10 smithi148 ceph-mon[35090]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-12T02:55:11.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:10 smithi005 ceph-mon[31197]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-12T02:55:11.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:10 smithi146 ceph-mon[35011]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-12T02:55:13.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:12 smithi005 ceph-mon[31197]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-09-12T02:55:13.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:12 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-12T02:55:13.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:12 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:55:13.063 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:12 smithi148 ceph-mon[35090]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-09-12T02:55:13.063 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:12 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-12T02:55:13.063 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:12 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:55:13.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:12 smithi146 ceph-mon[35011]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-09-12T02:55:13.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:12 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-12T02:55:13.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:12 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:55:13.915 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:13 smithi148 ceph-mon[35090]: Deploying daemon osd.11 on smithi148 2024-09-12T02:55:14.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:13 smithi146 ceph-mon[35011]: Deploying daemon osd.11 on smithi148 2024-09-12T02:55:14.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:13 smithi005 ceph-mon[31197]: Deploying daemon osd.11 on smithi148 2024-09-12T02:55:15.090 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:14 smithi148 ceph-mon[35090]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-12T02:55:15.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:14 smithi146 ceph-mon[35011]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-12T02:55:15.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:14 smithi005 ceph-mon[31197]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-12T02:55:16.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:16 smithi148 ceph-mon[35090]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-12T02:55:17.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:16 smithi146 ceph-mon[35011]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-12T02:55:17.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:16 smithi005 ceph-mon[31197]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-12T02:55:19.005 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:18 smithi148 ceph-mon[35090]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-12T02:55:19.005 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:18 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:55:19.005 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:18 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:19.006 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:18 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:19.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:18 smithi146 ceph-mon[35011]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-12T02:55:19.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:18 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:55:19.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:18 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:19.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:18 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:19.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:18 smithi005 ceph-mon[31197]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-12T02:55:19.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:18 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:55:19.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:18 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:19.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:18 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:21.072 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:20 smithi148 ceph-mon[35090]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-12T02:55:21.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:20 smithi146 ceph-mon[35011]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-12T02:55:21.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:20 smithi005 ceph-mon[31197]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-12T02:55:21.877 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:55:21.876+0000 7f71d77fe640 1 -- 172.21.15.148:0/2250012714 <== mgr.14148 v2:172.21.15.5:6800/3780631642 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f71e81a0630 con 0x7f71d00a2bb0 2024-09-12T02:55:21.877 INFO:teuthology.orchestra.run.smithi148.stdout:Created osd(s) 11 on host 'smithi148' 2024-09-12T02:55:21.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:55:21.879+0000 7f71eca63640 1 -- 172.21.15.148:0/2250012714 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f71d00a2bb0 msgr2=0x7f71d00a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:21.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:55:21.879+0000 7f71eca63640 1 --2- 172.21.15.148:0/2250012714 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f71d00a2bb0 0x7f71d00a5070 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f71e8101b60 tx=0x7f71dc005f90 comp rx=0 tx=0).stop 2024-09-12T02:55:21.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:55:21.879+0000 7f71eca63640 1 -- 172.21.15.148:0/2250012714 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f71e81026c0 msgr2=0x7f71e819f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:21.879 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:55:21.879+0000 7f71eca63640 1 --2- 172.21.15.148:0/2250012714 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f71e81026c0 0x7f71e819f450 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f71e0005e30 tx=0x7f71e0004ab0 comp rx=0 tx=0).stop 2024-09-12T02:55:21.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:55:21.879+0000 7f71eca63640 1 -- 172.21.15.148:0/2250012714 shutdown_connections 2024-09-12T02:55:21.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:55:21.879+0000 7f71eca63640 1 --2- 172.21.15.148:0/2250012714 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f71d00a2bb0 0x7f71d00a5070 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:21.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:55:21.879+0000 7f71eca63640 1 --2- 172.21.15.148:0/2250012714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71e8101d00 0x7f71e81983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:21.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:55:21.879+0000 7f71eca63640 1 --2- 172.21.15.148:0/2250012714 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f71e81026c0 0x7f71e819f450 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:21.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:55:21.879+0000 7f71eca63640 1 --2- 172.21.15.148:0/2250012714 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f71e8100b00 0x7f71e8197e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:21.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:55:21.879+0000 7f71eca63640 1 -- 172.21.15.148:0/2250012714 >> 172.21.15.148:0/2250012714 conn(0x7f71e80fc270 msgr2=0x7f71e806a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:55:21.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:55:21.879+0000 7f71eca63640 1 -- 172.21.15.148:0/2250012714 shutdown_connections 2024-09-12T02:55:21.880 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:55:21.879+0000 7f71eca63640 1 -- 172.21.15.148:0/2250012714 wait complete. 2024-09-12T02:55:22.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:22 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:22.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:22 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:22.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:22 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:55:22.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:22 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:55:22.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:22 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:22.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:22 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:55:22.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:22 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:22.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:22 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:22.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:22 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:22.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:22 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:22.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:22 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:55:22.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:22 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:55:22.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:22 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:22.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:22 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:55:22.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:22 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:22.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:22 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:22.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:22 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:22.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:22 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:22.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:22 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:55:22.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:22 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:55:22.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:22 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:22.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:22 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:55:22.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:22 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:22.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:22 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:22.574 DEBUG:teuthology.orchestra.run.smithi148:osd.11> sudo journalctl -f -n 0 -u ceph-618305f8-70b1-11ef-bcea-c7b262605968@osd.11.service 2024-09-12T02:55:22.577 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-09-12T02:55:22.577 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-12T02:55:22.850 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:55:23.165 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:23 smithi148 ceph-mon[35090]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-12T02:55:23.222 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:23 smithi005 ceph-mon[31197]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-12T02:55:23.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:23.305+0000 7ff4a2f48640 1 -- 172.21.15.5:0/4228877736 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff49c101c00 msgr2=0x7ff49c102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:23.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:23.305+0000 7ff4a2f48640 1 --2- 172.21.15.5:0/4228877736 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff49c101c00 0x7ff49c102000 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff490009900 tx=0x7ff49002a410 comp rx=0 tx=0).stop 2024-09-12T02:55:23.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:23.305+0000 7ff4a2f48640 1 -- 172.21.15.5:0/4228877736 shutdown_connections 2024-09-12T02:55:23.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:23.306+0000 7ff4a2f48640 1 --2- 172.21.15.5:0/4228877736 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff49c1037c0 0x7ff49c10a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:23.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:23.306+0000 7ff4a2f48640 1 --2- 172.21.15.5:0/4228877736 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff49c102e00 0x7ff49c103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:23.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:23.306+0000 7ff4a2f48640 1 --2- 172.21.15.5:0/4228877736 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff49c101c00 0x7ff49c102000 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:23.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:23.306+0000 7ff4a2f48640 1 -- 172.21.15.5:0/4228877736 >> 172.21.15.5:0/4228877736 conn(0x7ff49c0fd390 msgr2=0x7ff49c0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:55:23.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:23.306+0000 7ff4a2f48640 1 -- 172.21.15.5:0/4228877736 shutdown_connections 2024-09-12T02:55:23.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:23.306+0000 7ff4a2f48640 1 -- 172.21.15.5:0/4228877736 wait complete. 2024-09-12T02:55:23.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:23.306+0000 7ff4a2f48640 1 Processor -- start 2024-09-12T02:55:23.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:23.306+0000 7ff4a2f48640 1 -- start start 2024-09-12T02:55:23.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:23.306+0000 7ff4a2f48640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff49c101c00 0x7ff49c197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:23.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:23.306+0000 7ff4a2f48640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff49c102e00 0x7ff49c198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:23.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:23.306+0000 7ff4a2f48640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff49c1037c0 0x7ff49c19f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:23.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:23.306+0000 7ff4a2f48640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff49c10d860 con 0x7ff49c101c00 2024-09-12T02:55:23.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:23.306+0000 7ff4a2f48640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ff49c10d6e0 con 0x7ff49c1037c0 2024-09-12T02:55:23.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:23.306+0000 7ff4a2f48640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff49c10d9e0 con 0x7ff49c102e00 2024-09-12T02:55:23.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:23.307+0000 7ff4a0ce4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff49c101c00 0x7ff49c197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:23.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:23.307+0000 7ff4a0ce4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff49c101c00 0x7ff49c197f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35504/0 (socket says 172.21.15.5:35504) 2024-09-12T02:55:23.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:23.307+0000 7ff4a0ce4640 1 -- 172.21.15.5:0/2028181074 learned_addr learned my addr 172.21.15.5:0/2028181074 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:55:23.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:23.307+0000 7ff49bfff640 1 --2- 172.21.15.5:0/2028181074 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff49c102e00 0x7ff49c198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:23.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:23.307+0000 7ff4a14e5640 1 --2- 172.21.15.5:0/2028181074 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff49c1037c0 0x7ff49c19f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:23.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:23.307+0000 7ff4a0ce4640 1 -- 172.21.15.5:0/2028181074 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff49c102e00 msgr2=0x7ff49c198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:23.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:23.307+0000 7ff4a0ce4640 1 --2- 172.21.15.5:0/2028181074 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff49c102e00 0x7ff49c198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:23.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:23.307+0000 7ff4a0ce4640 1 -- 172.21.15.5:0/2028181074 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff49c1037c0 msgr2=0x7ff49c19f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:23.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:23.307+0000 7ff4a0ce4640 1 --2- 172.21.15.5:0/2028181074 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff49c1037c0 0x7ff49c19f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:23.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:23.307+0000 7ff4a0ce4640 1 -- 172.21.15.5:0/2028181074 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff49c19fb30 con 0x7ff49c101c00 2024-09-12T02:55:23.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:23.307+0000 7ff4a0ce4640 1 --2- 172.21.15.5:0/2028181074 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff49c101c00 0x7ff49c197f00 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ff490031610 tx=0x7ff490004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:23.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:23.307+0000 7ff499ffb640 1 -- 172.21.15.5:0/2028181074 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff49003f030 con 0x7ff49c101c00 2024-09-12T02:55:23.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:23.307+0000 7ff4a2f48640 1 -- 172.21.15.5:0/2028181074 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff49c19fdc0 con 0x7ff49c101c00 2024-09-12T02:55:23.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:23.307+0000 7ff499ffb640 1 -- 172.21.15.5:0/2028181074 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff490003ad0 con 0x7ff49c101c00 2024-09-12T02:55:23.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:23.307+0000 7ff499ffb640 1 -- 172.21.15.5:0/2028181074 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff490035070 con 0x7ff49c101c00 2024-09-12T02:55:23.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:23.308+0000 7ff4a2f48640 1 -- 172.21.15.5:0/2028181074 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff49c1a0210 con 0x7ff49c101c00 2024-09-12T02:55:23.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:23.309+0000 7ff4a2f48640 1 -- 172.21.15.5:0/2028181074 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff49c109000 con 0x7ff49c101c00 2024-09-12T02:55:23.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:23.310+0000 7ff499ffb640 1 -- 172.21.15.5:0/2028181074 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7ff490042030 con 0x7ff49c101c00 2024-09-12T02:55:23.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:23.311+0000 7ff499ffb640 1 --2- 172.21.15.5:0/2028181074 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7ff48c0a2bb0 0x7ff48c0a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:23.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:23.311+0000 7ff49bfff640 1 --2- 172.21.15.5:0/2028181074 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7ff48c0a2bb0 0x7ff48c0a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:23.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:23.311+0000 7ff499ffb640 1 -- 172.21.15.5:0/2028181074 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 6809+0+0 (secure 0 0 0) 0x7ff4900ef2d0 con 0x7ff49c101c00 2024-09-12T02:55:23.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:23.311+0000 7ff49bfff640 1 --2- 172.21.15.5:0/2028181074 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7ff48c0a2bb0 0x7ff48c0a5070 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff49c199420 tx=0x7ff488005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:23.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:23.314+0000 7ff499ffb640 1 -- 172.21.15.5:0/2028181074 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff4900bc120 con 0x7ff49c101c00 2024-09-12T02:55:23.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:23 smithi146 ceph-mon[35011]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-12T02:55:23.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:23.487+0000 7ff4a2f48640 1 -- 172.21.15.5:0/2028181074 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7ff49c02d9c0 con 0x7ff49c101c00 2024-09-12T02:55:23.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:23.487+0000 7ff499ffb640 1 -- 172.21.15.5:0/2028181074 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v72) ==== 74+0+133 (secure 0 0 0) 0x7ff4900c0420 con 0x7ff49c101c00 2024-09-12T02:55:23.487 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-12T02:55:23.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:23.490+0000 7ff4a2f48640 1 -- 172.21.15.5:0/2028181074 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7ff48c0a2bb0 msgr2=0x7ff48c0a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:23.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:23.490+0000 7ff4a2f48640 1 --2- 172.21.15.5:0/2028181074 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7ff48c0a2bb0 0x7ff48c0a5070 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff49c199420 tx=0x7ff488005f90 comp rx=0 tx=0).stop 2024-09-12T02:55:23.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:23.490+0000 7ff4a2f48640 1 -- 172.21.15.5:0/2028181074 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff49c101c00 msgr2=0x7ff49c197f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:23.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:23.490+0000 7ff4a2f48640 1 --2- 172.21.15.5:0/2028181074 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff49c101c00 0x7ff49c197f00 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ff490031610 tx=0x7ff490004270 comp rx=0 tx=0).stop 2024-09-12T02:55:23.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:23.490+0000 7ff4a2f48640 1 -- 172.21.15.5:0/2028181074 shutdown_connections 2024-09-12T02:55:23.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:23.490+0000 7ff4a2f48640 1 --2- 172.21.15.5:0/2028181074 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff49c1037c0 0x7ff49c19f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:23.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:23.490+0000 7ff4a2f48640 1 --2- 172.21.15.5:0/2028181074 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff49c102e00 0x7ff49c198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:23.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:23.490+0000 7ff4a2f48640 1 --2- 172.21.15.5:0/2028181074 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7ff48c0a2bb0 0x7ff48c0a5070 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:23.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:23.490+0000 7ff4a2f48640 1 --2- 172.21.15.5:0/2028181074 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff49c101c00 0x7ff49c197f00 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:23.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:23.490+0000 7ff4a2f48640 1 -- 172.21.15.5:0/2028181074 >> 172.21.15.5:0/2028181074 conn(0x7ff49c0fd390 msgr2=0x7ff49c1061d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:55:23.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:23.490+0000 7ff4a2f48640 1 -- 172.21.15.5:0/2028181074 shutdown_connections 2024-09-12T02:55:23.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:23.490+0000 7ff4a2f48640 1 -- 172.21.15.5:0/2028181074 wait complete. 2024-09-12T02:55:24.301 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:24 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/2028181074' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-12T02:55:24.331 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":72,"num_osds":12,"num_up_osds":11,"osd_up_since":1726109702,"num_in_osds":12,"osd_in_since":1726109703,"num_remapped_pgs":0} 2024-09-12T02:55:24.348 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:24 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/2028181074' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-12T02:55:24.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:24 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/2028181074' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-12T02:55:24.666 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 02:55:24 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T02:55:24.347+0000 7fb89e559640 -1 osd.11 0 log_to_monitors true 2024-09-12T02:55:25.286 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:25 smithi148 ceph-mon[35090]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-12T02:55:25.286 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:25 smithi148 ceph-mon[35090]: from='osd.11 [v2:172.21.15.148:6824/839290773,v1:172.21.15.148:6825/839290773]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-12T02:55:25.286 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:25 smithi148 ceph-mon[35090]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-12T02:55:25.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:25 smithi005 ceph-mon[31197]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-12T02:55:25.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:25 smithi005 ceph-mon[31197]: from='osd.11 [v2:172.21.15.148:6824/839290773,v1:172.21.15.148:6825/839290773]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-12T02:55:25.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:25 smithi005 ceph-mon[31197]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-12T02:55:25.332 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-12T02:55:25.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:25 smithi146 ceph-mon[35011]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-12T02:55:25.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:25 smithi146 ceph-mon[35011]: from='osd.11 [v2:172.21.15.148:6824/839290773,v1:172.21.15.148:6825/839290773]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-12T02:55:25.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:25 smithi146 ceph-mon[35011]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-12T02:55:25.597 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:55:26.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:26.002+0000 7f606123c640 1 -- 172.21.15.5:0/2404299503 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f605c102e00 msgr2=0x7f605c103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:26.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:26.002+0000 7f606123c640 1 --2- 172.21.15.5:0/2404299503 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f605c102e00 0x7f605c103280 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f6050009900 tx=0x7f605002a410 comp rx=0 tx=0).stop 2024-09-12T02:55:26.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:26.002+0000 7f606123c640 1 -- 172.21.15.5:0/2404299503 shutdown_connections 2024-09-12T02:55:26.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:26.002+0000 7f606123c640 1 --2- 172.21.15.5:0/2404299503 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f605c102e00 0x7f605c103280 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:26.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:26.002+0000 7f606123c640 1 --2- 172.21.15.5:0/2404299503 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f605c1037c0 0x7f605c10a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:26.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:26.002+0000 7f606123c640 1 --2- 172.21.15.5:0/2404299503 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f605c101c00 0x7f605c102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:26.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:26.002+0000 7f606123c640 1 -- 172.21.15.5:0/2404299503 >> 172.21.15.5:0/2404299503 conn(0x7f605c0fd390 msgr2=0x7f605c0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:55:26.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:26.002+0000 7f606123c640 1 -- 172.21.15.5:0/2404299503 shutdown_connections 2024-09-12T02:55:26.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:26.002+0000 7f606123c640 1 -- 172.21.15.5:0/2404299503 wait complete. 2024-09-12T02:55:26.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:26.003+0000 7f606123c640 1 Processor -- start 2024-09-12T02:55:26.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:26.003+0000 7f606123c640 1 -- start start 2024-09-12T02:55:26.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:26.003+0000 7f606123c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f605c101c00 0x7f605c197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:26.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:26.003+0000 7f606123c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f605c102e00 0x7f605c198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:26.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:26.003+0000 7f606123c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f605c1037c0 0x7f605c19f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:26.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:26.003+0000 7f606123c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f605c10d880 con 0x7f605c102e00 2024-09-12T02:55:26.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:26.003+0000 7f606123c640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f605c10d700 con 0x7f605c1037c0 2024-09-12T02:55:26.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:26.003+0000 7f606123c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f605c10da00 con 0x7f605c101c00 2024-09-12T02:55:26.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:26.004+0000 7f605ad9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f605c101c00 0x7f605c197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:26.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:26.004+0000 7f605ad9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f605c101c00 0x7f605c197f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.5:37778/0 (socket says 172.21.15.5:37778) 2024-09-12T02:55:26.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:26.004+0000 7f605ad9d640 1 -- 172.21.15.5:0/1993204216 learned_addr learned my addr 172.21.15.5:0/1993204216 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:55:26.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:26.004+0000 7f605a59c640 1 --2- 172.21.15.5:0/1993204216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f605c102e00 0x7f605c198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:26.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:26.004+0000 7f605b59e640 1 --2- 172.21.15.5:0/1993204216 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f605c1037c0 0x7f605c19f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:26.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:26.004+0000 7f605a59c640 1 -- 172.21.15.5:0/1993204216 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f605c101c00 msgr2=0x7f605c197f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:26.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:26.004+0000 7f605a59c640 1 --2- 172.21.15.5:0/1993204216 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f605c101c00 0x7f605c197f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:26.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:26.004+0000 7f605a59c640 1 -- 172.21.15.5:0/1993204216 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f605c1037c0 msgr2=0x7f605c19f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:26.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:26.004+0000 7f605a59c640 1 --2- 172.21.15.5:0/1993204216 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f605c1037c0 0x7f605c19f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:26.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:26.004+0000 7f605a59c640 1 -- 172.21.15.5:0/1993204216 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f605c19fb30 con 0x7f605c102e00 2024-09-12T02:55:26.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:26.004+0000 7f605b59e640 1 --2- 172.21.15.5:0/1993204216 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f605c1037c0 0x7f605c19f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:55:26.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:26.004+0000 7f605ad9d640 1 --2- 172.21.15.5:0/1993204216 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f605c101c00 0x7f605c197f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:55:26.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:26.004+0000 7f605a59c640 1 --2- 172.21.15.5:0/1993204216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f605c102e00 0x7f605c198440 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6050031440 tx=0x7f6050031b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:26.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:26.005+0000 7f6043fff640 1 -- 172.21.15.5:0/1993204216 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f605003f070 con 0x7f605c102e00 2024-09-12T02:55:26.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:26.005+0000 7f606123c640 1 -- 172.21.15.5:0/1993204216 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f605c19fdc0 con 0x7f605c102e00 2024-09-12T02:55:26.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:26.005+0000 7f6043fff640 1 -- 172.21.15.5:0/1993204216 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6050003cd0 con 0x7f605c102e00 2024-09-12T02:55:26.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:26.005+0000 7f6043fff640 1 -- 172.21.15.5:0/1993204216 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6050035070 con 0x7f605c102e00 2024-09-12T02:55:26.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:26.005+0000 7f606123c640 1 -- 172.21.15.5:0/1993204216 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f605c1a0260 con 0x7f605c102e00 2024-09-12T02:55:26.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:26.006+0000 7f606123c640 1 -- 172.21.15.5:0/1993204216 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f605c108fa0 con 0x7f605c102e00 2024-09-12T02:55:26.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:26.007+0000 7f6043fff640 1 -- 172.21.15.5:0/1993204216 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f605003b560 con 0x7f605c102e00 2024-09-12T02:55:26.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:26.008+0000 7f6043fff640 1 --2- 172.21.15.5:0/1993204216 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f60440a2bb0 0x7f60440a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:26.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:26.008+0000 7f6043fff640 1 -- 172.21.15.5:0/1993204216 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 6831+0+0 (secure 0 0 0) 0x7f60500eed30 con 0x7f605c102e00 2024-09-12T02:55:26.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:26.008+0000 7f605ad9d640 1 --2- 172.21.15.5:0/1993204216 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f60440a2bb0 0x7f60440a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:26.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:26.008+0000 7f605ad9d640 1 --2- 172.21.15.5:0/1993204216 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f60440a2bb0 0x7f60440a5070 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f604c0058c0 tx=0x7f604c005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:26.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:26.012+0000 7f6043fff640 1 -- 172.21.15.5:0/1993204216 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f60500bc260 con 0x7f605c102e00 2024-09-12T02:55:26.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:26.190+0000 7f606123c640 1 -- 172.21.15.5:0/1993204216 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f605c02d990 con 0x7f605c102e00 2024-09-12T02:55:26.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:26.190+0000 7f6043fff640 1 -- 172.21.15.5:0/1993204216 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v74) ==== 74+0+133 (secure 0 0 0) 0x7f60500bbb60 con 0x7f605c102e00 2024-09-12T02:55:26.191 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-12T02:55:26.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:26.193+0000 7f606123c640 1 -- 172.21.15.5:0/1993204216 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f60440a2bb0 msgr2=0x7f60440a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:26.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:26.193+0000 7f606123c640 1 --2- 172.21.15.5:0/1993204216 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f60440a2bb0 0x7f60440a5070 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f604c0058c0 tx=0x7f604c005f90 comp rx=0 tx=0).stop 2024-09-12T02:55:26.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:26.193+0000 7f606123c640 1 -- 172.21.15.5:0/1993204216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f605c102e00 msgr2=0x7f605c198440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:26.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:26.193+0000 7f606123c640 1 --2- 172.21.15.5:0/1993204216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f605c102e00 0x7f605c198440 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6050031440 tx=0x7f6050031b60 comp rx=0 tx=0).stop 2024-09-12T02:55:26.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:26.193+0000 7f606123c640 1 -- 172.21.15.5:0/1993204216 shutdown_connections 2024-09-12T02:55:26.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:26.193+0000 7f606123c640 1 --2- 172.21.15.5:0/1993204216 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f60440a2bb0 0x7f60440a5070 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:26.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:26.193+0000 7f606123c640 1 --2- 172.21.15.5:0/1993204216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f605c102e00 0x7f605c198440 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:26.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:26.193+0000 7f606123c640 1 --2- 172.21.15.5:0/1993204216 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f605c1037c0 0x7f605c19f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:26.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:26.193+0000 7f606123c640 1 --2- 172.21.15.5:0/1993204216 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f605c101c00 0x7f605c197f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:26.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:26.193+0000 7f606123c640 1 -- 172.21.15.5:0/1993204216 >> 172.21.15.5:0/1993204216 conn(0x7f605c0fd390 msgr2=0x7f605c106170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:55:26.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:26.193+0000 7f606123c640 1 -- 172.21.15.5:0/1993204216 shutdown_connections 2024-09-12T02:55:26.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:26.193+0000 7f606123c640 1 -- 172.21.15.5:0/1993204216 wait complete. 2024-09-12T02:55:26.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:26 smithi005 ceph-mon[31197]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-12T02:55:26.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:26 smithi005 ceph-mon[31197]: osdmap e73: 12 total, 11 up, 12 in 2024-09-12T02:55:26.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:26 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T02:55:26.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:26 smithi005 ceph-mon[31197]: from='osd.11 [v2:172.21.15.148:6824/839290773,v1:172.21.15.148:6825/839290773]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-09-12T02:55:26.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:26 smithi005 ceph-mon[31197]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-09-12T02:55:26.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:26 smithi005 ceph-mon[31197]: Detected new or changed devices on smithi148 2024-09-12T02:55:26.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:26 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:26.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:26 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:26.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:26 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:55:26.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:26 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:55:26.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:26 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:55:26.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:26 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:55:26.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:26 smithi005 ceph-mon[31197]: Adjusting osd_memory_target on smithi148 to 4272M 2024-09-12T02:55:26.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:26 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:26.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:26 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:55:26.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:26 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:55:26.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:26 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:26.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:26 smithi148 ceph-mon[35090]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-12T02:55:26.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:26 smithi148 ceph-mon[35090]: osdmap e73: 12 total, 11 up, 12 in 2024-09-12T02:55:26.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:26 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T02:55:26.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:26 smithi148 ceph-mon[35090]: from='osd.11 [v2:172.21.15.148:6824/839290773,v1:172.21.15.148:6825/839290773]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-09-12T02:55:26.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:26 smithi148 ceph-mon[35090]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-09-12T02:55:26.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:26 smithi148 ceph-mon[35090]: Detected new or changed devices on smithi148 2024-09-12T02:55:26.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:26 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:26.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:26 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:26.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:26 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:55:26.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:26 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:55:26.417 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:26 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:55:26.417 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:26 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:55:26.417 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:26 smithi148 ceph-mon[35090]: Adjusting osd_memory_target on smithi148 to 4272M 2024-09-12T02:55:26.417 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:26 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:26.417 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:26 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:55:26.417 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:26 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:55:26.417 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:26 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:26.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:26 smithi146 ceph-mon[35011]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-12T02:55:26.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:26 smithi146 ceph-mon[35011]: osdmap e73: 12 total, 11 up, 12 in 2024-09-12T02:55:26.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:26 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T02:55:26.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:26 smithi146 ceph-mon[35011]: from='osd.11 [v2:172.21.15.148:6824/839290773,v1:172.21.15.148:6825/839290773]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-09-12T02:55:26.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:26 smithi146 ceph-mon[35011]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]: dispatch 2024-09-12T02:55:26.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:26 smithi146 ceph-mon[35011]: Detected new or changed devices on smithi148 2024-09-12T02:55:26.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:26 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:26.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:26 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:26.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:26 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:55:26.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:26 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:55:26.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:26 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:55:26.458 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:26 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-12T02:55:26.458 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:26 smithi146 ceph-mon[35011]: Adjusting osd_memory_target on smithi148 to 4272M 2024-09-12T02:55:26.458 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:26 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:26.458 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:26 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:55:26.458 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:26 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:55:26.458 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:26 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:26.862 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":74,"num_osds":12,"num_up_osds":11,"osd_up_since":1726109702,"num_in_osds":12,"osd_in_since":1726109703,"num_remapped_pgs":0} 2024-09-12T02:55:27.165 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 02:55:26 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T02:55:26.814+0000 7fb899cb9640 -1 osd.11 0 waiting for initial osdmap 2024-09-12T02:55:27.166 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 02:55:26 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T02:55:26.829+0000 7fb8922eb640 -1 osd.11 74 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-12T02:55:27.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:27 smithi148 ceph-mon[35090]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-12T02:55:27.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:27 smithi148 ceph-mon[35090]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2024-09-12T02:55:27.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:27 smithi148 ceph-mon[35090]: osdmap e74: 12 total, 11 up, 12 in 2024-09-12T02:55:27.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:27 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T02:55:27.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:27 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T02:55:27.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:27 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/1993204216' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-12T02:55:27.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:27 smithi148 ceph-mon[35090]: from='osd.11 ' entity='osd.11' 2024-09-12T02:55:27.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:27 smithi005 ceph-mon[31197]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-12T02:55:27.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:27 smithi005 ceph-mon[31197]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2024-09-12T02:55:27.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:27 smithi005 ceph-mon[31197]: osdmap e74: 12 total, 11 up, 12 in 2024-09-12T02:55:27.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:27 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T02:55:27.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:27 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T02:55:27.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:27 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/1993204216' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-12T02:55:27.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:27 smithi005 ceph-mon[31197]: from='osd.11 ' entity='osd.11' 2024-09-12T02:55:27.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:27 smithi146 ceph-mon[35011]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 324 MiB used, 983 GiB / 983 GiB avail 2024-09-12T02:55:27.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:27 smithi146 ceph-mon[35011]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi148", "root=default"]}]': finished 2024-09-12T02:55:27.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:27 smithi146 ceph-mon[35011]: osdmap e74: 12 total, 11 up, 12 in 2024-09-12T02:55:27.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:27 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T02:55:27.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:27 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T02:55:27.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:27 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/1993204216' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-12T02:55:27.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:27 smithi146 ceph-mon[35011]: from='osd.11 ' entity='osd.11' 2024-09-12T02:55:27.863 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-12T02:55:28.132 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:55:28.254 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:28 smithi005 ceph-mon[31197]: purged_snaps scrub starts 2024-09-12T02:55:28.254 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:28 smithi005 ceph-mon[31197]: purged_snaps scrub ok 2024-09-12T02:55:28.254 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:28 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T02:55:28.254 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:28 smithi005 ceph-mon[31197]: osd.11 [v2:172.21.15.148:6824/839290773,v1:172.21.15.148:6825/839290773] boot 2024-09-12T02:55:28.255 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:28 smithi005 ceph-mon[31197]: osdmap e75: 12 total, 12 up, 12 in 2024-09-12T02:55:28.255 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:28 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T02:55:28.415 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:28 smithi148 ceph-mon[35090]: purged_snaps scrub starts 2024-09-12T02:55:28.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:28 smithi148 ceph-mon[35090]: purged_snaps scrub ok 2024-09-12T02:55:28.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:28 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T02:55:28.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:28 smithi148 ceph-mon[35090]: osd.11 [v2:172.21.15.148:6824/839290773,v1:172.21.15.148:6825/839290773] boot 2024-09-12T02:55:28.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:28 smithi148 ceph-mon[35090]: osdmap e75: 12 total, 12 up, 12 in 2024-09-12T02:55:28.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:28 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T02:55:28.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:28 smithi146 ceph-mon[35011]: purged_snaps scrub starts 2024-09-12T02:55:28.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:28 smithi146 ceph-mon[35011]: purged_snaps scrub ok 2024-09-12T02:55:28.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:28 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T02:55:28.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:28 smithi146 ceph-mon[35011]: osd.11 [v2:172.21.15.148:6824/839290773,v1:172.21.15.148:6825/839290773] boot 2024-09-12T02:55:28.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:28 smithi146 ceph-mon[35011]: osdmap e75: 12 total, 12 up, 12 in 2024-09-12T02:55:28.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:28 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-12T02:55:28.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:28.611+0000 7f3d26ddb640 1 -- 172.21.15.5:0/2396104507 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d20100ad0 msgr2=0x7f3d20100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:28.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:28.611+0000 7f3d26ddb640 1 --2- 172.21.15.5:0/2396104507 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d20100ad0 0x7f3d20100ed0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f3d14009900 tx=0x7f3d1402a410 comp rx=0 tx=0).stop 2024-09-12T02:55:28.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:28.612+0000 7f3d26ddb640 1 -- 172.21.15.5:0/2396104507 shutdown_connections 2024-09-12T02:55:28.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:28.612+0000 7f3d26ddb640 1 --2- 172.21.15.5:0/2396104507 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d20102690 0x7f3d20108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:28.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:28.612+0000 7f3d26ddb640 1 --2- 172.21.15.5:0/2396104507 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3d20101cd0 0x7f3d20102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:28.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:28.612+0000 7f3d26ddb640 1 --2- 172.21.15.5:0/2396104507 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d20100ad0 0x7f3d20100ed0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:28.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:28.612+0000 7f3d26ddb640 1 -- 172.21.15.5:0/2396104507 >> 172.21.15.5:0/2396104507 conn(0x7f3d200fc280 msgr2=0x7f3d200fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:55:28.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:28.612+0000 7f3d26ddb640 1 -- 172.21.15.5:0/2396104507 shutdown_connections 2024-09-12T02:55:28.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:28.612+0000 7f3d26ddb640 1 -- 172.21.15.5:0/2396104507 wait complete. 2024-09-12T02:55:28.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:28.612+0000 7f3d26ddb640 1 Processor -- start 2024-09-12T02:55:28.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:28.612+0000 7f3d26ddb640 1 -- start start 2024-09-12T02:55:28.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:28.612+0000 7f3d26ddb640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3d20100ad0 0x7f3d20104bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:28.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:28.612+0000 7f3d26ddb640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d20101cd0 0x7f3d20103270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:28.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:28.612+0000 7f3d26ddb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d20102690 0x7f3d201037b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:28.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:28.612+0000 7f3d26ddb640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3d2010c760 con 0x7f3d20102690 2024-09-12T02:55:28.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:28.612+0000 7f3d26ddb640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f3d2010c5e0 con 0x7f3d20100ad0 2024-09-12T02:55:28.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:28.612+0000 7f3d26ddb640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3d2010c8e0 con 0x7f3d20101cd0 2024-09-12T02:55:28.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:28.612+0000 7f3d25378640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d20102690 0x7f3d201037b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:28.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:28.613+0000 7f3d25378640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d20102690 0x7f3d201037b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52370/0 (socket says 172.21.15.5:52370) 2024-09-12T02:55:28.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:28.613+0000 7f3d25378640 1 -- 172.21.15.5:0/3044198332 learned_addr learned my addr 172.21.15.5:0/3044198332 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:55:28.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:28.613+0000 7f3d1ffff640 1 --2- 172.21.15.5:0/3044198332 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d20101cd0 0x7f3d20103270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:28.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:28.613+0000 7f3d24b77640 1 --2- 172.21.15.5:0/3044198332 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3d20100ad0 0x7f3d20104bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:28.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:28.613+0000 7f3d25378640 1 -- 172.21.15.5:0/3044198332 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d20101cd0 msgr2=0x7f3d20103270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:28.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:28.613+0000 7f3d25378640 1 --2- 172.21.15.5:0/3044198332 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d20101cd0 0x7f3d20103270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:28.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:28.613+0000 7f3d25378640 1 -- 172.21.15.5:0/3044198332 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3d20100ad0 msgr2=0x7f3d20104bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:28.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:28.613+0000 7f3d25378640 1 --2- 172.21.15.5:0/3044198332 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3d20100ad0 0x7f3d20104bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:28.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:28.613+0000 7f3d25378640 1 -- 172.21.15.5:0/3044198332 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d20103fe0 con 0x7f3d20102690 2024-09-12T02:55:28.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:28.613+0000 7f3d25378640 1 --2- 172.21.15.5:0/3044198332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d20102690 0x7f3d201037b0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3d1800eb10 tx=0x7f3d1800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:28.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:28.613+0000 7f3d1dffb640 1 -- 172.21.15.5:0/3044198332 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d18004b20 con 0x7f3d20102690 2024-09-12T02:55:28.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:28.613+0000 7f3d1dffb640 1 -- 172.21.15.5:0/3044198332 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d18004cc0 con 0x7f3d20102690 2024-09-12T02:55:28.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:28.613+0000 7f3d26ddb640 1 -- 172.21.15.5:0/3044198332 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d201a1ff0 con 0x7f3d20102690 2024-09-12T02:55:28.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:28.613+0000 7f3d1dffb640 1 -- 172.21.15.5:0/3044198332 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d18010760 con 0x7f3d20102690 2024-09-12T02:55:28.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:28.613+0000 7f3d26ddb640 1 -- 172.21.15.5:0/3044198332 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d201a24a0 con 0x7f3d20102690 2024-09-12T02:55:28.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:28.615+0000 7f3d26ddb640 1 -- 172.21.15.5:0/3044198332 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d201a2790 con 0x7f3d20102690 2024-09-12T02:55:28.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:28.616+0000 7f3d1dffb640 1 -- 172.21.15.5:0/3044198332 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f3d18005020 con 0x7f3d20102690 2024-09-12T02:55:28.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:28.616+0000 7f3d1dffb640 1 --2- 172.21.15.5:0/3044198332 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f3d100a2c00 0x7f3d100a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:28.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:28.616+0000 7f3d1dffb640 1 -- 172.21.15.5:0/3044198332 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7127+0+0 (secure 0 0 0) 0x7f3d180cf310 con 0x7f3d20102690 2024-09-12T02:55:28.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:28.616+0000 7f3d24b77640 1 --2- 172.21.15.5:0/3044198332 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f3d100a2c00 0x7f3d100a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:28.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:28.617+0000 7f3d24b77640 1 --2- 172.21.15.5:0/3044198332 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f3d100a2c00 0x7f3d100a50c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3d14006010 tx=0x7f3d14000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:28.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:28.620+0000 7f3d1dffb640 1 -- 172.21.15.5:0/3044198332 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3d1809c710 con 0x7f3d20102690 2024-09-12T02:55:28.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:28.796+0000 7f3d26ddb640 1 -- 172.21.15.5:0/3044198332 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f3d201a2b20 con 0x7f3d20102690 2024-09-12T02:55:28.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:28.796+0000 7f3d1dffb640 1 -- 172.21.15.5:0/3044198332 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v75) ==== 74+0+133 (secure 0 0 0) 0x7f3d180bc0d0 con 0x7f3d20102690 2024-09-12T02:55:28.797 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-12T02:55:28.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:28.799+0000 7f3d26ddb640 1 -- 172.21.15.5:0/3044198332 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f3d100a2c00 msgr2=0x7f3d100a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:28.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:28.799+0000 7f3d26ddb640 1 --2- 172.21.15.5:0/3044198332 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f3d100a2c00 0x7f3d100a50c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f3d14006010 tx=0x7f3d14000bc0 comp rx=0 tx=0).stop 2024-09-12T02:55:28.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:28.799+0000 7f3d26ddb640 1 -- 172.21.15.5:0/3044198332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d20102690 msgr2=0x7f3d201037b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:28.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:28.799+0000 7f3d26ddb640 1 --2- 172.21.15.5:0/3044198332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d20102690 0x7f3d201037b0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3d1800eb10 tx=0x7f3d1800c2d0 comp rx=0 tx=0).stop 2024-09-12T02:55:28.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:28.799+0000 7f3d26ddb640 1 -- 172.21.15.5:0/3044198332 shutdown_connections 2024-09-12T02:55:28.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:28.799+0000 7f3d26ddb640 1 --2- 172.21.15.5:0/3044198332 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f3d100a2c00 0x7f3d100a50c0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:28.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:28.799+0000 7f3d26ddb640 1 --2- 172.21.15.5:0/3044198332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d20102690 0x7f3d201037b0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:28.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:28.799+0000 7f3d26ddb640 1 --2- 172.21.15.5:0/3044198332 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d20101cd0 0x7f3d20103270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:28.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:28.799+0000 7f3d26ddb640 1 --2- 172.21.15.5:0/3044198332 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3d20100ad0 0x7f3d20104bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:28.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:28.799+0000 7f3d26ddb640 1 -- 172.21.15.5:0/3044198332 >> 172.21.15.5:0/3044198332 conn(0x7f3d200fc280 msgr2=0x7f3d20069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:55:28.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:28.799+0000 7f3d26ddb640 1 -- 172.21.15.5:0/3044198332 shutdown_connections 2024-09-12T02:55:28.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:28.799+0000 7f3d26ddb640 1 -- 172.21.15.5:0/3044198332 wait complete. 2024-09-12T02:55:29.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:29 smithi005 ceph-mon[31197]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-09-12T02:55:29.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:29 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/3044198332' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-12T02:55:29.415 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:29 smithi148 ceph-mon[35090]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-09-12T02:55:29.415 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:29 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/3044198332' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-12T02:55:29.444 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":75,"num_osds":12,"num_up_osds":12,"osd_up_since":1726109727,"num_in_osds":12,"osd_in_since":1726109703,"num_remapped_pgs":0} 2024-09-12T02:55:29.445 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T02:55:29.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:29 smithi146 ceph-mon[35011]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 325 MiB used, 983 GiB / 983 GiB avail 2024-09-12T02:55:29.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:29 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/3044198332' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-12T02:55:29.709 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:55:30.152 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:30 smithi005 ceph-mon[31197]: osdmap e76: 12 total, 12 up, 12 in 2024-09-12T02:55:30.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:30.151+0000 7f4c28ac4640 1 -- 172.21.15.5:0/311139889 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c24106f10 msgr2=0x7f4c2410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:30.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:30.151+0000 7f4c28ac4640 1 --2- 172.21.15.5:0/311139889 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c24106f10 0x7f4c2410ea90 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c009ca0 tx=0x7f4c1c02a8c0 comp rx=0 tx=0).stop 2024-09-12T02:55:30.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:30.152+0000 7f4c28ac4640 1 -- 172.21.15.5:0/311139889 shutdown_connections 2024-09-12T02:55:30.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:30.152+0000 7f4c28ac4640 1 --2- 172.21.15.5:0/311139889 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c24106f10 0x7f4c2410ea90 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:30.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:30.152+0000 7f4c28ac4640 1 --2- 172.21.15.5:0/311139889 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4c24102790 0x7f4c241067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:30.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:30.152+0000 7f4c28ac4640 1 --2- 172.21.15.5:0/311139889 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4c24101de0 0x7f4c241021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:30.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:30.152+0000 7f4c28ac4640 1 -- 172.21.15.5:0/311139889 >> 172.21.15.5:0/311139889 conn(0x7f4c240fd8c0 msgr2=0x7f4c240ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:55:30.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:30.152+0000 7f4c28ac4640 1 -- 172.21.15.5:0/311139889 shutdown_connections 2024-09-12T02:55:30.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:30.152+0000 7f4c28ac4640 1 -- 172.21.15.5:0/311139889 wait complete. 2024-09-12T02:55:30.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:30.152+0000 7f4c28ac4640 1 Processor -- start 2024-09-12T02:55:30.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:30.153+0000 7f4c28ac4640 1 -- start start 2024-09-12T02:55:30.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:30.153+0000 7f4c28ac4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c24101de0 0x7f4c241a4650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:30.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:30.153+0000 7f4c28ac4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4c24102790 0x7f4c241a4b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:30.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:30.153+0000 7f4c28ac4640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4c24106f10 0x7f4c2410db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:30.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:30.153+0000 7f4c28ac4640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4c24111f90 con 0x7f4c24101de0 2024-09-12T02:55:30.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:30.153+0000 7f4c28ac4640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f4c24111e10 con 0x7f4c24106f10 2024-09-12T02:55:30.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:30.153+0000 7f4c28ac4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4c24112110 con 0x7f4c24102790 2024-09-12T02:55:30.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:30.153+0000 7f4c2259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c24101de0 0x7f4c241a4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:30.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:30.153+0000 7f4c2259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c24101de0 0x7f4c241a4650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52398/0 (socket says 172.21.15.5:52398) 2024-09-12T02:55:30.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:30.153+0000 7f4c2259c640 1 -- 172.21.15.5:0/3263141466 learned_addr learned my addr 172.21.15.5:0/3263141466 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:55:30.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:30.153+0000 7f4c21d9b640 1 --2- 172.21.15.5:0/3263141466 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4c24102790 0x7f4c241a4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:30.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:30.153+0000 7f4c22d9d640 1 --2- 172.21.15.5:0/3263141466 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4c24106f10 0x7f4c2410db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:30.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:30.153+0000 7f4c2259c640 1 -- 172.21.15.5:0/3263141466 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4c24102790 msgr2=0x7f4c241a4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:30.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:30.153+0000 7f4c2259c640 1 --2- 172.21.15.5:0/3263141466 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4c24102790 0x7f4c241a4b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:30.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:30.153+0000 7f4c2259c640 1 -- 172.21.15.5:0/3263141466 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4c24106f10 msgr2=0x7f4c2410db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:30.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:30.153+0000 7f4c2259c640 1 --2- 172.21.15.5:0/3263141466 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4c24106f10 0x7f4c2410db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:30.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:30.153+0000 7f4c2259c640 1 -- 172.21.15.5:0/3263141466 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c2410e2d0 con 0x7f4c24101de0 2024-09-12T02:55:30.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:30.154+0000 7f4c2259c640 1 --2- 172.21.15.5:0/3263141466 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c24101de0 0x7f4c241a4650 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4c1400c920 tx=0x7f4c1400cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:30.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:30.154+0000 7f4c137fe640 1 -- 172.21.15.5:0/3263141466 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c14007bb0 con 0x7f4c24101de0 2024-09-12T02:55:30.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:30.154+0000 7f4c137fe640 1 -- 172.21.15.5:0/3263141466 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c14007d50 con 0x7f4c24101de0 2024-09-12T02:55:30.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:30.154+0000 7f4c28ac4640 1 -- 172.21.15.5:0/3263141466 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c2410e5c0 con 0x7f4c24101de0 2024-09-12T02:55:30.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:30.154+0000 7f4c137fe640 1 -- 172.21.15.5:0/3263141466 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c140187d0 con 0x7f4c24101de0 2024-09-12T02:55:30.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:30.154+0000 7f4c28ac4640 1 -- 172.21.15.5:0/3263141466 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c241ab4b0 con 0x7f4c24101de0 2024-09-12T02:55:30.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:30.155+0000 7f4c28ac4640 1 -- 172.21.15.5:0/3263141466 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c04005290 con 0x7f4c24101de0 2024-09-12T02:55:30.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:30.156+0000 7f4c137fe640 1 -- 172.21.15.5:0/3263141466 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f4c1400f450 con 0x7f4c24101de0 2024-09-12T02:55:30.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:30.157+0000 7f4c137fe640 1 --2- 172.21.15.5:0/3263141466 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f4c0c0a2b80 0x7f4c0c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:30.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:30.157+0000 7f4c137fe640 1 -- 172.21.15.5:0/3263141466 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f4c140ceb90 con 0x7f4c24101de0 2024-09-12T02:55:30.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:30.157+0000 7f4c21d9b640 1 --2- 172.21.15.5:0/3263141466 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f4c0c0a2b80 0x7f4c0c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:30.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:30.158+0000 7f4c21d9b640 1 --2- 172.21.15.5:0/3263141466 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f4c0c0a2b80 0x7f4c0c0a5040 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4c240ff650 tx=0x7f4c18009110 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:30.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:30.161+0000 7f4c137fe640 1 -- 172.21.15.5:0/3263141466 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4c14014030 con 0x7f4c24101de0 2024-09-12T02:55:30.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:30.340+0000 7f4c28ac4640 1 -- 172.21.15.5:0/3263141466 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4c04005580 con 0x7f4c24101de0 2024-09-12T02:55:30.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:30.341+0000 7f4c137fe640 1 -- 172.21.15.5:0/3263141466 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) ==== 74+0+18412 (secure 0 0 0) 0x7f4c1409bf30 con 0x7f4c24101de0 2024-09-12T02:55:30.341 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-12T02:55:30.342 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":76,"fsid":"618305f8-70b1-11ef-bcea-c7b262605968","created":"2024-09-12T02:48:42.352146+0000","modified":"2024-09-12T02:55:29.041307+0000","last_up_change":"2024-09-12T02:55:27.817161+0000","last_in_change":"2024-09-12T02:55:03.745281+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T02:51:47.970639+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17da560a-6aa2-4f2c-a95f-67481dce4d3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":4248507573},{"type":"v1","addr":"172.21.15.5:6803","nonce":4248507573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":4248507573},{"type":"v1","addr":"172.21.15.5:6805","nonce":4248507573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":4248507573},{"type":"v1","addr":"172.21.15.5:6809","nonce":4248507573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":4248507573},{"type":"v1","addr":"172.21.15.5:6807","nonce":4248507573}]},"public_addr":"172.21.15.5:6803/4248507573","cluster_addr":"172.21.15.5:6805/4248507573","heartbeat_back_addr":"172.21.15.5:6809/4248507573","heartbeat_front_addr":"172.21.15.5:6807/4248507573","state":["exists","up"]},{"osd":1,"uuid":"a3237006-99af-4fc1-860c-d4917032647f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":1105990287},{"type":"v1","addr":"172.21.15.146:6801","nonce":1105990287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":1105990287},{"type":"v1","addr":"172.21.15.146:6803","nonce":1105990287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":1105990287},{"type":"v1","addr":"172.21.15.146:6807","nonce":1105990287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":1105990287},{"type":"v1","addr":"172.21.15.146:6805","nonce":1105990287}]},"public_addr":"172.21.15.146:6801/1105990287","cluster_addr":"172.21.15.146:6803/1105990287","heartbeat_back_addr":"172.21.15.146:6807/1105990287","heartbeat_front_addr":"172.21.15.146:6805/1105990287","state":["exists","up"]},{"osd":2,"uuid":"ea3ab84a-7739-4975-8244-31ddd5aad80d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3378174313},{"type":"v1","addr":"172.21.15.148:6801","nonce":3378174313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3378174313},{"type":"v1","addr":"172.21.15.148:6803","nonce":3378174313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3378174313},{"type":"v1","addr":"172.21.15.148:6807","nonce":3378174313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3378174313},{"type":"v1","addr":"172.21.15.148:6805","nonce":3378174313}]},"public_addr":"172.21.15.148:6801/3378174313","cluster_addr":"172.21.15.148:6803/3378174313","heartbeat_back_addr":"172.21.15.148:6807/3378174313","heartbeat_front_addr":"172.21.15.148:6805/3378174313","state":["exists","up"]},{"osd":3,"uuid":"4396a9ee-f3f3-410b-9fb1-124f1e33a1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":11741938},{"type":"v1","addr":"172.21.15.5:6811","nonce":11741938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":11741938},{"type":"v1","addr":"172.21.15.5:6813","nonce":11741938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":11741938},{"type":"v1","addr":"172.21.15.5:6817","nonce":11741938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":11741938},{"type":"v1","addr":"172.21.15.5:6815","nonce":11741938}]},"public_addr":"172.21.15.5:6811/11741938","cluster_addr":"172.21.15.5:6813/11741938","heartbeat_back_addr":"172.21.15.5:6817/11741938","heartbeat_front_addr":"172.21.15.5:6815/11741938","state":["exists","up"]},{"osd":4,"uuid":"989d76e7-c742-49d0-994a-c58f2771b1d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":3274923851},{"type":"v1","addr":"172.21.15.146:6809","nonce":3274923851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":3274923851},{"type":"v1","addr":"172.21.15.146:6811","nonce":3274923851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3274923851},{"type":"v1","addr":"172.21.15.146:6815","nonce":3274923851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":3274923851},{"type":"v1","addr":"172.21.15.146:6813","nonce":3274923851}]},"public_addr":"172.21.15.146:6809/3274923851","cluster_addr":"172.21.15.146:6811/3274923851","heartbeat_back_addr":"172.21.15.146:6815/3274923851","heartbeat_front_addr":"172.21.15.146:6813/3274923851","state":["exists","up"]},{"osd":5,"uuid":"64ada7d4-70a0-4b83-9af8-032be544608d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":2771670476},{"type":"v1","addr":"172.21.15.148:6809","nonce":2771670476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":2771670476},{"type":"v1","addr":"172.21.15.148:6811","nonce":2771670476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":2771670476},{"type":"v1","addr":"172.21.15.148:6815","nonce":2771670476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":2771670476},{"type":"v1","addr":"172.21.15.148:6813","nonce":2771670476}]},"public_addr":"172.21.15.148:6809/2771670476","cluster_addr":"172.21.15.148:6811/2771670476","heartbeat_back_addr":"172.21.15.148:6815/2771670476","heartbeat_front_addr":"172.21.15.148:6813/2771670476","state":["exists","up"]},{"osd":6,"uuid":"a70a88fd-2e4b-480c-bb98-62636cf70cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3176073169},{"type":"v1","addr":"172.21.15.5:6819","nonce":3176073169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":3176073169},{"type":"v1","addr":"172.21.15.5:6821","nonce":3176073169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":3176073169},{"type":"v1","addr":"172.21.15.5:6825","nonce":3176073169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3176073169},{"type":"v1","addr":"172.21.15.5:6823","nonce":3176073169}]},"public_addr":"172.21.15.5:6819/3176073169","cluster_addr":"172.21.15.5:6821/3176073169","heartbeat_back_addr":"172.21.15.5:6825/3176073169","heartbeat_front_addr":"172.21.15.5:6823/3176073169","state":["exists","up"]},{"osd":7,"uuid":"49451e8d-c741-4ad5-b8f5-89bad8d4d129","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2209508838},{"type":"v1","addr":"172.21.15.146:6817","nonce":2209508838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2209508838},{"type":"v1","addr":"172.21.15.146:6819","nonce":2209508838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2209508838},{"type":"v1","addr":"172.21.15.146:6823","nonce":2209508838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2209508838},{"type":"v1","addr":"172.21.15.146:6821","nonce":2209508838}]},"public_addr":"172.21.15.146:6817/2209508838","cluster_addr":"172.21.15.146:6819/2209508838","heartbeat_back_addr":"172.21.15.146:6823/2209508838","heartbeat_front_addr":"172.21.15.146:6821/2209508838","state":["exists","up"]},{"osd":8,"uuid":"57bfc121-a75e-4a7f-b1dc-64074b178b03","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":651045756},{"type":"v1","addr":"172.21.15.148:6817","nonce":651045756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":651045756},{"type":"v1","addr":"172.21.15.148:6819","nonce":651045756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":651045756},{"type":"v1","addr":"172.21.15.148:6823","nonce":651045756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":651045756},{"type":"v1","addr":"172.21.15.148:6821","nonce":651045756}]},"public_addr":"172.21.15.148:6817/651045756","cluster_addr":"172.21.15.148:6819/651045756","heartbeat_back_addr":"172.21.15.148:6823/651045756","heartbeat_front_addr":"172.21.15.148:6821/651045756","state":["exists","up"]},{"osd":9,"uuid":"d9726168-7494-4bf8-b433-350e85bfd461","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1961857360},{"type":"v1","addr":"172.21.15.5:6827","nonce":1961857360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1961857360},{"type":"v1","addr":"172.21.15.5:6829","nonce":1961857360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":1961857360},{"type":"v1","addr":"172.21.15.5:6833","nonce":1961857360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1961857360},{"type":"v1","addr":"172.21.15.5:6831","nonce":1961857360}]},"public_addr":"172.21.15.5:6827/1961857360","cluster_addr":"172.21.15.5:6829/1961857360","heartbeat_back_addr":"172.21.15.5:6833/1961857360","heartbeat_front_addr":"172.21.15.5:6831/1961857360","state":["exists","up"]},{"osd":10,"uuid":"519ec60f-3606-48ac-9043-6f59a798766d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":2041733103},{"type":"v1","addr":"172.21.15.146:6825","nonce":2041733103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":2041733103},{"type":"v1","addr":"172.21.15.146:6827","nonce":2041733103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":2041733103},{"type":"v1","addr":"172.21.15.146:6831","nonce":2041733103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":2041733103},{"type":"v1","addr":"172.21.15.146:6829","nonce":2041733103}]},"public_addr":"172.21.15.146:6825/2041733103","cluster_addr":"172.21.15.146:6827/2041733103","heartbeat_back_addr":"172.21.15.146:6831/2041733103","heartbeat_front_addr":"172.21.15.146:6829/2041733103","state":["exists","up"]},{"osd":11,"uuid":"0f92d0cb-ba54-4bc3-a6d0-3c54d910556a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":839290773},{"type":"v1","addr":"172.21.15.148:6825","nonce":839290773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":839290773},{"type":"v1","addr":"172.21.15.148:6827","nonce":839290773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":839290773},{"type":"v1","addr":"172.21.15.148:6831","nonce":839290773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":839290773},{"type":"v1","addr":"172.21.15.148:6829","nonce":839290773}]},"public_addr":"172.21.15.148:6825/839290773","cluster_addr":"172.21.15.148:6827/839290773","heartbeat_back_addr":"172.21.15.148:6831/839290773","heartbeat_front_addr":"172.21.15.148:6829/839290773","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:50:54.101755+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:51:19.037385+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:51:44.142837+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:52:09.150887+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:52:33.705079+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:52:58.845233+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:53:23.068339+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:53:47.520398+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:54:11.965370+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:54:36.776075+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:55:01.292236+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:55:25.346589+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:6800/3125821798":"2024-09-13T02:49:29.462216+0000","172.21.15.5:0/3479911451":"2024-09-13T02:49:29.462216+0000","172.21.15.5:0/1547753207":"2024-09-13T02:49:03.300872+0000","172.21.15.5:0/1905762627":"2024-09-13T02:49:03.300872+0000","172.21.15.5:0/4142340280":"2024-09-13T02:49:29.462216+0000","172.21.15.5:6801/3125821798":"2024-09-13T02:49:29.462216+0000","172.21.15.5:6801/2595781680":"2024-09-13T02:49:03.300872+0000","172.21.15.5:0/2273691311":"2024-09-13T02:49:29.462216+0000","172.21.15.5:6800/2595781680":"2024-09-13T02:49:03.300872+0000","172.21.15.5:0/1196201217":"2024-09-13T02:49:03.300872+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T02:55:30.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:30.343+0000 7f4c28ac4640 1 -- 172.21.15.5:0/3263141466 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f4c0c0a2b80 msgr2=0x7f4c0c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:30.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:30.343+0000 7f4c28ac4640 1 --2- 172.21.15.5:0/3263141466 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f4c0c0a2b80 0x7f4c0c0a5040 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4c240ff650 tx=0x7f4c18009110 comp rx=0 tx=0).stop 2024-09-12T02:55:30.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:30.343+0000 7f4c28ac4640 1 -- 172.21.15.5:0/3263141466 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c24101de0 msgr2=0x7f4c241a4650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:30.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:30.343+0000 7f4c28ac4640 1 --2- 172.21.15.5:0/3263141466 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c24101de0 0x7f4c241a4650 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4c1400c920 tx=0x7f4c1400cd20 comp rx=0 tx=0).stop 2024-09-12T02:55:30.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:30.343+0000 7f4c28ac4640 1 -- 172.21.15.5:0/3263141466 shutdown_connections 2024-09-12T02:55:30.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:30.343+0000 7f4c28ac4640 1 --2- 172.21.15.5:0/3263141466 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4c24106f10 0x7f4c2410db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:30.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:30.343+0000 7f4c28ac4640 1 --2- 172.21.15.5:0/3263141466 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4c24102790 0x7f4c241a4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:30.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:30.343+0000 7f4c28ac4640 1 --2- 172.21.15.5:0/3263141466 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f4c0c0a2b80 0x7f4c0c0a5040 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:30.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:30.343+0000 7f4c28ac4640 1 --2- 172.21.15.5:0/3263141466 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c24101de0 0x7f4c241a4650 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:30.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:30.343+0000 7f4c28ac4640 1 -- 172.21.15.5:0/3263141466 >> 172.21.15.5:0/3263141466 conn(0x7f4c240fd8c0 msgr2=0x7f4c240fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:55:30.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:30.343+0000 7f4c28ac4640 1 -- 172.21.15.5:0/3263141466 shutdown_connections 2024-09-12T02:55:30.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:30.343+0000 7f4c28ac4640 1 -- 172.21.15.5:0/3263141466 wait complete. 2024-09-12T02:55:30.415 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:30 smithi148 ceph-mon[35090]: osdmap e76: 12 total, 12 up, 12 in 2024-09-12T02:55:30.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:30 smithi146 ceph-mon[35011]: osdmap e76: 12 total, 12 up, 12 in 2024-09-12T02:55:31.014 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-12T02:51:47.970639+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-12T02:55:31.015 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-09-12T02:55:31.271 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:55:31.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:31 smithi005 ceph-mon[31197]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:55:31.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:31 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/3263141466' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:55:31.415 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:31 smithi148 ceph-mon[35090]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:55:31.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:31 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/3263141466' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:55:31.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:31 smithi146 ceph-mon[35011]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:55:31.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:31 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/3263141466' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:55:31.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:31.728+0000 7f295f94d640 1 -- 172.21.15.5:0/3924604933 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2958102ef0 msgr2=0x7f295810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:31.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:31.728+0000 7f295f94d640 1 --2- 172.21.15.5:0/3924604933 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2958102ef0 0x7f295810da40 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f2948009ca0 tx=0x7f294802a950 comp rx=0 tx=0).stop 2024-09-12T02:55:31.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:31.729+0000 7f295f94d640 1 -- 172.21.15.5:0/3924604933 shutdown_connections 2024-09-12T02:55:31.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:31.729+0000 7f295f94d640 1 --2- 172.21.15.5:0/3924604933 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2958102ef0 0x7f295810da40 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:31.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:31.729+0000 7f295f94d640 1 --2- 172.21.15.5:0/3924604933 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2958102570 0x7f29581029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:31.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:31.729+0000 7f295f94d640 1 --2- 172.21.15.5:0/3924604933 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2958100e20 0x7f2958101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:31.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:31.729+0000 7f295f94d640 1 -- 172.21.15.5:0/3924604933 >> 172.21.15.5:0/3924604933 conn(0x7f29580fc7d0 msgr2=0x7f29580febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:55:31.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:31.729+0000 7f295f94d640 1 -- 172.21.15.5:0/3924604933 shutdown_connections 2024-09-12T02:55:31.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:31.729+0000 7f295f94d640 1 -- 172.21.15.5:0/3924604933 wait complete. 2024-09-12T02:55:31.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:31.729+0000 7f295f94d640 1 Processor -- start 2024-09-12T02:55:31.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:31.729+0000 7f295f94d640 1 -- start start 2024-09-12T02:55:31.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:31.730+0000 7f295f94d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2958100e20 0x7f29581983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:31.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:31.730+0000 7f295f94d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2958102570 0x7f2958198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:31.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:31.730+0000 7f295f94d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2958102ef0 0x7f295819cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:31.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:31.730+0000 7f295f94d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2958110f80 con 0x7f2958102ef0 2024-09-12T02:55:31.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:31.730+0000 7f295f94d640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f2958110e00 con 0x7f2958102570 2024-09-12T02:55:31.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:31.730+0000 7f295f94d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f2958111100 con 0x7f2958100e20 2024-09-12T02:55:31.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:31.730+0000 7f295deea640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2958102ef0 0x7f295819cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:31.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:31.730+0000 7f295deea640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2958102ef0 0x7f295819cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52416/0 (socket says 172.21.15.5:52416) 2024-09-12T02:55:31.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:31.730+0000 7f295deea640 1 -- 172.21.15.5:0/2056242497 learned_addr learned my addr 172.21.15.5:0/2056242497 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:55:31.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:31.730+0000 7f295d6e9640 1 --2- 172.21.15.5:0/2056242497 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2958100e20 0x7f29581983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:31.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:31.730+0000 7f295cee8640 1 --2- 172.21.15.5:0/2056242497 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2958102570 0x7f2958198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:31.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:31.730+0000 7f295deea640 1 -- 172.21.15.5:0/2056242497 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2958100e20 msgr2=0x7f29581983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:31.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:31.730+0000 7f295deea640 1 --2- 172.21.15.5:0/2056242497 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2958100e20 0x7f29581983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:31.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:31.730+0000 7f295deea640 1 -- 172.21.15.5:0/2056242497 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2958102570 msgr2=0x7f2958198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:31.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:31.730+0000 7f295deea640 1 --2- 172.21.15.5:0/2056242497 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2958102570 0x7f2958198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:31.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:31.730+0000 7f295deea640 1 -- 172.21.15.5:0/2056242497 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f295819d380 con 0x7f2958102ef0 2024-09-12T02:55:31.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:31.730+0000 7f295deea640 1 --2- 172.21.15.5:0/2056242497 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2958102ef0 0x7f295819cca0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2948006010 tx=0x7f294802f6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:31.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:31.730+0000 7f29567fc640 1 -- 172.21.15.5:0/2056242497 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2948034040 con 0x7f2958102ef0 2024-09-12T02:55:31.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:31.730+0000 7f29567fc640 1 -- 172.21.15.5:0/2056242497 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2948003e00 con 0x7f2958102ef0 2024-09-12T02:55:31.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:31.730+0000 7f29567fc640 1 -- 172.21.15.5:0/2056242497 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2948045480 con 0x7f2958102ef0 2024-09-12T02:55:31.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:31.730+0000 7f295f94d640 1 -- 172.21.15.5:0/2056242497 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f295819d610 con 0x7f2958102ef0 2024-09-12T02:55:31.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:31.731+0000 7f295f94d640 1 -- 172.21.15.5:0/2056242497 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29581a4f50 con 0x7f2958102ef0 2024-09-12T02:55:31.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:31.731+0000 7f295f94d640 1 -- 172.21.15.5:0/2056242497 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2958101200 con 0x7f2958102ef0 2024-09-12T02:55:31.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:31.733+0000 7f29567fc640 1 -- 172.21.15.5:0/2056242497 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f2948037020 con 0x7f2958102ef0 2024-09-12T02:55:31.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:31.734+0000 7f29567fc640 1 --2- 172.21.15.5:0/2056242497 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f29400a2c00 0x7f29400a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:31.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:31.734+0000 7f295d6e9640 1 --2- 172.21.15.5:0/2056242497 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f29400a2c00 0x7f29400a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:31.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:31.734+0000 7f29567fc640 1 -- 172.21.15.5:0/2056242497 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f29480efcc0 con 0x7f2958102ef0 2024-09-12T02:55:31.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:31.734+0000 7f295d6e9640 1 --2- 172.21.15.5:0/2056242497 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f29400a2c00 0x7f29400a50c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f295000abe0 tx=0x7f295000a2d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:31.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:31.737+0000 7f29567fc640 1 -- 172.21.15.5:0/2056242497 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f29480bd0b0 con 0x7f2958102ef0 2024-09-12T02:55:31.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:31.910+0000 7f295f94d640 1 -- 172.21.15.5:0/2056242497 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f295810a0a0 con 0x7f2958102ef0 2024-09-12T02:55:31.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:31.911+0000 7f29567fc640 1 -- 172.21.15.5:0/2056242497 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v76) ==== 93+0+10 (secure 0 0 0) 0x7f29480bc9b0 con 0x7f2958102ef0 2024-09-12T02:55:31.911 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-09-12T02:55:31.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:31.913+0000 7f295f94d640 1 -- 172.21.15.5:0/2056242497 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f29400a2c00 msgr2=0x7f29400a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:31.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:31.913+0000 7f295f94d640 1 --2- 172.21.15.5:0/2056242497 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f29400a2c00 0x7f29400a50c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f295000abe0 tx=0x7f295000a2d0 comp rx=0 tx=0).stop 2024-09-12T02:55:31.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:31.913+0000 7f295f94d640 1 -- 172.21.15.5:0/2056242497 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2958102ef0 msgr2=0x7f295819cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:31.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:31.913+0000 7f295f94d640 1 --2- 172.21.15.5:0/2056242497 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2958102ef0 0x7f295819cca0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2948006010 tx=0x7f294802f6f0 comp rx=0 tx=0).stop 2024-09-12T02:55:31.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:31.913+0000 7f295f94d640 1 -- 172.21.15.5:0/2056242497 shutdown_connections 2024-09-12T02:55:31.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:31.913+0000 7f295f94d640 1 --2- 172.21.15.5:0/2056242497 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f29400a2c00 0x7f29400a50c0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:31.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:31.913+0000 7f295f94d640 1 --2- 172.21.15.5:0/2056242497 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2958102ef0 0x7f295819cca0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:31.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:31.913+0000 7f295f94d640 1 --2- 172.21.15.5:0/2056242497 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2958102570 0x7f2958198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:31.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:31.913+0000 7f295f94d640 1 --2- 172.21.15.5:0/2056242497 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2958100e20 0x7f29581983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:31.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:31.913+0000 7f295f94d640 1 -- 172.21.15.5:0/2056242497 >> 172.21.15.5:0/2056242497 conn(0x7f29580fc7d0 msgr2=0x7f295806a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:55:31.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:31.913+0000 7f295f94d640 1 -- 172.21.15.5:0/2056242497 shutdown_connections 2024-09-12T02:55:31.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:31.913+0000 7f295f94d640 1 -- 172.21.15.5:0/2056242497 wait complete. 2024-09-12T02:55:32.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:32 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/2056242497' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-12T02:55:32.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:32 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/2056242497' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-12T02:55:32.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:32 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/2056242497' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-12T02:55:32.569 INFO:tasks.cephadm:Adding mds.a on smithi005 2024-09-12T02:55:32.570 INFO:tasks.cephadm:Adding mds.d on smithi005 2024-09-12T02:55:32.570 INFO:tasks.cephadm:Adding mds.g on smithi005 2024-09-12T02:55:32.570 INFO:tasks.cephadm:Adding mds.j on smithi005 2024-09-12T02:55:32.570 INFO:tasks.cephadm:Adding mds.b on smithi146 2024-09-12T02:55:32.570 INFO:tasks.cephadm:Adding mds.e on smithi146 2024-09-12T02:55:32.570 INFO:tasks.cephadm:Adding mds.h on smithi146 2024-09-12T02:55:32.570 INFO:tasks.cephadm:Adding mds.k on smithi146 2024-09-12T02:55:32.570 INFO:tasks.cephadm:Adding mds.c on smithi148 2024-09-12T02:55:32.570 INFO:tasks.cephadm:Adding mds.f on smithi148 2024-09-12T02:55:32.570 INFO:tasks.cephadm:Adding mds.i on smithi148 2024-09-12T02:55:32.570 INFO:tasks.cephadm:Adding mds.l on smithi148 2024-09-12T02:55:32.571 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph orch apply mds all '12;smithi005=a;smithi005=d;smithi005=g;smithi005=j;smithi146=b;smithi146=e;smithi146=h;smithi146=k;smithi148=c;smithi148=f;smithi148=i;smithi148=l' 2024-09-12T02:55:32.831 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T02:55:33.256 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:33 smithi148 ceph-mon[35090]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:55:33.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:55:33.256+0000 7ffa6d7c4640 1 -- 172.21.15.148:0/632087358 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa68100ad0 msgr2=0x7ffa68100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:33.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:55:33.256+0000 7ffa6d7c4640 1 --2- 172.21.15.148:0/632087358 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa68100ad0 0x7ffa68100ed0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ffa58009900 tx=0x7ffa5802a410 comp rx=0 tx=0).stop 2024-09-12T02:55:33.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:55:33.256+0000 7ffa6d7c4640 1 -- 172.21.15.148:0/632087358 shutdown_connections 2024-09-12T02:55:33.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:55:33.256+0000 7ffa6d7c4640 1 --2- 172.21.15.148:0/632087358 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa68101cd0 0x7ffa68102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:33.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:55:33.256+0000 7ffa6d7c4640 1 --2- 172.21.15.148:0/632087358 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffa68102690 0x7ffa68108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:33.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:55:33.256+0000 7ffa6d7c4640 1 --2- 172.21.15.148:0/632087358 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa68100ad0 0x7ffa68100ed0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:33.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:55:33.256+0000 7ffa6d7c4640 1 -- 172.21.15.148:0/632087358 >> 172.21.15.148:0/632087358 conn(0x7ffa680fc280 msgr2=0x7ffa680fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:55:33.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:55:33.256+0000 7ffa6d7c4640 1 -- 172.21.15.148:0/632087358 shutdown_connections 2024-09-12T02:55:33.275 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:55:33.256+0000 7ffa6d7c4640 1 -- 172.21.15.148:0/632087358 wait complete. 2024-09-12T02:55:33.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:55:33.256+0000 7ffa6d7c4640 1 Processor -- start 2024-09-12T02:55:33.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:55:33.257+0000 7ffa6d7c4640 1 -- start start 2024-09-12T02:55:33.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:55:33.257+0000 7ffa6d7c4640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffa68100ad0 0x7ffa68104a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:33.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:55:33.257+0000 7ffa6d7c4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa68101cd0 0x7ffa681030b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:33.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:55:33.257+0000 7ffa6d7c4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa68102690 0x7ffa681035f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:33.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:55:33.257+0000 7ffa6d7c4640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ffa6810c760 con 0x7ffa68101cd0 2024-09-12T02:55:33.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:55:33.257+0000 7ffa6d7c4640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ffa6810c5e0 con 0x7ffa68100ad0 2024-09-12T02:55:33.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:55:33.257+0000 7ffa6d7c4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ffa6810c8e0 con 0x7ffa68102690 2024-09-12T02:55:33.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:55:33.257+0000 7ffa677fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa68102690 0x7ffa681035f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:33.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:55:33.257+0000 7ffa677fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa68102690 0x7ffa681035f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.148:39678/0 (socket says 172.21.15.148:39678) 2024-09-12T02:55:33.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:55:33.257+0000 7ffa677fe640 1 -- 172.21.15.148:0/2188222923 learned_addr learned my addr 172.21.15.148:0/2188222923 (peer_addr_for_me v2:172.21.15.148:0/0) 2024-09-12T02:55:33.276 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:55:33.257+0000 7ffa66ffd640 1 --2- 172.21.15.148:0/2188222923 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffa68100ad0 0x7ffa68104a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:33.277 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:55:33.257+0000 7ffa5e7fc640 1 --2- 172.21.15.148:0/2188222923 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa68101cd0 0x7ffa681030b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:33.277 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:55:33.257+0000 7ffa677fe640 1 -- 172.21.15.148:0/2188222923 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffa68100ad0 msgr2=0x7ffa68104a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:33.277 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:55:33.257+0000 7ffa677fe640 1 --2- 172.21.15.148:0/2188222923 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffa68100ad0 0x7ffa68104a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:33.277 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:55:33.257+0000 7ffa677fe640 1 -- 172.21.15.148:0/2188222923 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa68101cd0 msgr2=0x7ffa681030b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:33.277 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:55:33.257+0000 7ffa677fe640 1 --2- 172.21.15.148:0/2188222923 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa68101cd0 0x7ffa681030b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:33.277 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:55:33.257+0000 7ffa677fe640 1 -- 172.21.15.148:0/2188222923 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa68103eb0 con 0x7ffa68102690 2024-09-12T02:55:33.277 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:55:33.257+0000 7ffa66ffd640 1 --2- 172.21.15.148:0/2188222923 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffa68100ad0 0x7ffa68104a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:55:33.277 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:55:33.257+0000 7ffa677fe640 1 --2- 172.21.15.148:0/2188222923 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa68102690 0x7ffa681035f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ffa6000cad0 tx=0x7ffa6000cf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:33.277 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:55:33.258+0000 7ffa64ff9640 1 -- 172.21.15.148:0/2188222923 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa6000ae20 con 0x7ffa68102690 2024-09-12T02:55:33.277 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:55:33.258+0000 7ffa6d7c4640 1 -- 172.21.15.148:0/2188222923 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa681a20d0 con 0x7ffa68102690 2024-09-12T02:55:33.277 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:55:33.258+0000 7ffa64ff9640 1 -- 172.21.15.148:0/2188222923 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa60005020 con 0x7ffa68102690 2024-09-12T02:55:33.277 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:55:33.258+0000 7ffa64ff9640 1 -- 172.21.15.148:0/2188222923 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa6000f6b0 con 0x7ffa68102690 2024-09-12T02:55:33.277 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:55:33.259+0000 7ffa6d7c4640 1 -- 172.21.15.148:0/2188222923 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa681a2630 con 0x7ffa68102690 2024-09-12T02:55:33.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:55:33.260+0000 7ffa6d7c4640 1 -- 172.21.15.148:0/2188222923 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa681a28c0 con 0x7ffa68102690 2024-09-12T02:55:33.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:55:33.260+0000 7ffa64ff9640 1 -- 172.21.15.148:0/2188222923 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7ffa600051d0 con 0x7ffa68102690 2024-09-12T02:55:33.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:55:33.261+0000 7ffa64ff9640 1 --2- 172.21.15.148:0/2188222923 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7ffa540a2b80 0x7ffa540a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:33.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:55:33.261+0000 7ffa64ff9640 1 -- 172.21.15.148:0/2188222923 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7ffa600cef80 con 0x7ffa68102690 2024-09-12T02:55:33.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:55:33.261+0000 7ffa66ffd640 1 --2- 172.21.15.148:0/2188222923 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7ffa540a2b80 0x7ffa540a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:33.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:55:33.261+0000 7ffa66ffd640 1 --2- 172.21.15.148:0/2188222923 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7ffa540a2b80 0x7ffa540a5040 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ffa58006010 tx=0x7ffa58032dd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:33.278 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:55:33.265+0000 7ffa64ff9640 1 -- 172.21.15.148:0/2188222923 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ffa60014030 con 0x7ffa68102690 2024-09-12T02:55:33.441 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:55:33.441+0000 7ffa6d7c4640 1 -- 172.21.15.148:0/2188222923 --> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi005=a;smithi005=d;smithi005=g;smithi005=j;smithi146=b;smithi146=e;smithi146=h;smithi146=k;smithi148=c;smithi148=f;smithi148=i;smithi148=l", "target": ["mon-mgr", ""]}) -- 0x7ffa6806a6d0 con 0x7ffa540a2b80 2024-09-12T02:55:33.449 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:55:33.449+0000 7ffa64ff9640 1 -- 172.21.15.148:0/2188222923 <== mgr.14148 v2:172.21.15.5:6800/3780631642 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7ffa6806a6d0 con 0x7ffa540a2b80 2024-09-12T02:55:33.449 INFO:teuthology.orchestra.run.smithi148.stdout:Scheduled mds.all update... 2024-09-12T02:55:33.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:55:33.451+0000 7ffa6d7c4640 1 -- 172.21.15.148:0/2188222923 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7ffa540a2b80 msgr2=0x7ffa540a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:33.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:55:33.451+0000 7ffa6d7c4640 1 --2- 172.21.15.148:0/2188222923 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7ffa540a2b80 0x7ffa540a5040 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ffa58006010 tx=0x7ffa58032dd0 comp rx=0 tx=0).stop 2024-09-12T02:55:33.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:55:33.451+0000 7ffa6d7c4640 1 -- 172.21.15.148:0/2188222923 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa68102690 msgr2=0x7ffa681035f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:33.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:55:33.451+0000 7ffa6d7c4640 1 --2- 172.21.15.148:0/2188222923 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa68102690 0x7ffa681035f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ffa6000cad0 tx=0x7ffa6000cf90 comp rx=0 tx=0).stop 2024-09-12T02:55:33.451 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:55:33.451+0000 7ffa6d7c4640 1 -- 172.21.15.148:0/2188222923 shutdown_connections 2024-09-12T02:55:33.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:55:33.451+0000 7ffa6d7c4640 1 --2- 172.21.15.148:0/2188222923 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7ffa540a2b80 0x7ffa540a5040 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:33.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:55:33.451+0000 7ffa6d7c4640 1 --2- 172.21.15.148:0/2188222923 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa68101cd0 0x7ffa681030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:33.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:55:33.451+0000 7ffa6d7c4640 1 --2- 172.21.15.148:0/2188222923 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffa68102690 0x7ffa681035f0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:33.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:55:33.451+0000 7ffa6d7c4640 1 --2- 172.21.15.148:0/2188222923 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffa68100ad0 0x7ffa68104a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:33.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:55:33.451+0000 7ffa6d7c4640 1 -- 172.21.15.148:0/2188222923 >> 172.21.15.148:0/2188222923 conn(0x7ffa680fc280 msgr2=0x7ffa68069fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:55:33.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:55:33.451+0000 7ffa6d7c4640 1 -- 172.21.15.148:0/2188222923 shutdown_connections 2024-09-12T02:55:33.452 INFO:teuthology.orchestra.run.smithi148.stderr:2024-09-12T02:55:33.451+0000 7ffa6d7c4640 1 -- 172.21.15.148:0/2188222923 wait complete. 2024-09-12T02:55:33.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:33 smithi146 ceph-mon[35011]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:55:33.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:33 smithi005 ceph-mon[31197]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:55:34.115 DEBUG:teuthology.orchestra.run.smithi005:mds.a> sudo journalctl -f -n 0 -u ceph-618305f8-70b1-11ef-bcea-c7b262605968@mds.a.service 2024-09-12T02:55:34.117 DEBUG:teuthology.orchestra.run.smithi005:mds.d> sudo journalctl -f -n 0 -u ceph-618305f8-70b1-11ef-bcea-c7b262605968@mds.d.service 2024-09-12T02:55:34.160 DEBUG:teuthology.orchestra.run.smithi005:mds.g> sudo journalctl -f -n 0 -u ceph-618305f8-70b1-11ef-bcea-c7b262605968@mds.g.service 2024-09-12T02:55:34.203 DEBUG:teuthology.orchestra.run.smithi005:mds.j> sudo journalctl -f -n 0 -u ceph-618305f8-70b1-11ef-bcea-c7b262605968@mds.j.service 2024-09-12T02:55:34.247 DEBUG:teuthology.orchestra.run.smithi146:mds.b> sudo journalctl -f -n 0 -u ceph-618305f8-70b1-11ef-bcea-c7b262605968@mds.b.service 2024-09-12T02:55:34.249 DEBUG:teuthology.orchestra.run.smithi146:mds.e> sudo journalctl -f -n 0 -u ceph-618305f8-70b1-11ef-bcea-c7b262605968@mds.e.service 2024-09-12T02:55:34.295 DEBUG:teuthology.orchestra.run.smithi146:mds.h> sudo journalctl -f -n 0 -u ceph-618305f8-70b1-11ef-bcea-c7b262605968@mds.h.service 2024-09-12T02:55:34.338 DEBUG:teuthology.orchestra.run.smithi146:mds.k> sudo journalctl -f -n 0 -u ceph-618305f8-70b1-11ef-bcea-c7b262605968@mds.k.service 2024-09-12T02:55:34.381 DEBUG:teuthology.orchestra.run.smithi148:mds.c> sudo journalctl -f -n 0 -u ceph-618305f8-70b1-11ef-bcea-c7b262605968@mds.c.service 2024-09-12T02:55:34.383 DEBUG:teuthology.orchestra.run.smithi148:mds.f> sudo journalctl -f -n 0 -u ceph-618305f8-70b1-11ef-bcea-c7b262605968@mds.f.service 2024-09-12T02:55:34.427 DEBUG:teuthology.orchestra.run.smithi148:mds.i> sudo journalctl -f -n 0 -u ceph-618305f8-70b1-11ef-bcea-c7b262605968@mds.i.service 2024-09-12T02:55:34.470 DEBUG:teuthology.orchestra.run.smithi148:mds.l> sudo journalctl -f -n 0 -u ceph-618305f8-70b1-11ef-bcea-c7b262605968@mds.l.service 2024-09-12T02:55:34.513 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-09-12T02:55:34.513 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:55:34.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:34 smithi148 ceph-mon[35090]: from='client.24427 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi005=a;smithi005=d;smithi005=g;smithi005=j;smithi146=b;smithi146=e;smithi146=h;smithi146=k;smithi148=c;smithi148=f;smithi148=i;smithi148=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:55:34.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:34 smithi148 ceph-mon[35090]: Saving service mds.all spec with placement smithi005=a;smithi005=d;smithi005=g;smithi005=j;smithi146=b;smithi146=e;smithi146=h;smithi146=k;smithi148=c;smithi148=f;smithi148=i;smithi148=l;count:12 2024-09-12T02:55:34.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:34 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:34.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:34 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:55:34.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:34 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:55:34.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:34 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:55:34.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:34 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:34.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:34 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T02:55:34.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:34 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T02:55:34.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:34 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:55:34.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:34 smithi146 ceph-mon[35011]: from='client.24427 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi005=a;smithi005=d;smithi005=g;smithi005=j;smithi146=b;smithi146=e;smithi146=h;smithi146=k;smithi148=c;smithi148=f;smithi148=i;smithi148=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:55:34.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:34 smithi146 ceph-mon[35011]: Saving service mds.all spec with placement smithi005=a;smithi005=d;smithi005=g;smithi005=j;smithi146=b;smithi146=e;smithi146=h;smithi146=k;smithi148=c;smithi148=f;smithi148=i;smithi148=l;count:12 2024-09-12T02:55:34.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:34 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:34.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:34 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:55:34.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:34 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:55:34.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:34 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:55:34.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:34 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:34.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:34 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T02:55:34.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:34 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T02:55:34.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:34 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:55:34.765 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:55:34.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:34 smithi005 ceph-mon[31197]: from='client.24427 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi005=a;smithi005=d;smithi005=g;smithi005=j;smithi146=b;smithi146=e;smithi146=h;smithi146=k;smithi148=c;smithi148=f;smithi148=i;smithi148=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:55:34.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:34 smithi005 ceph-mon[31197]: Saving service mds.all spec with placement smithi005=a;smithi005=d;smithi005=g;smithi005=j;smithi146=b;smithi146=e;smithi146=h;smithi146=k;smithi148=c;smithi148=f;smithi148=i;smithi148=l;count:12 2024-09-12T02:55:34.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:34 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:34.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:34 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:55:34.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:34 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:55:34.811 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:34 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:55:34.811 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:34 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:34.811 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:34 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T02:55:34.811 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:34 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T02:55:34.811 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:34 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:55:35.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:35.226+0000 7fbbc9fa0640 1 -- 172.21.15.5:0/2845055959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbc4102370 msgr2=0x7fbbc410e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:35.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:35.226+0000 7fbbc9fa0640 1 --2- 172.21.15.5:0/2845055959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbc4102370 0x7fbbc410e8a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc009930 tx=0x7fbbbc02a420 comp rx=0 tx=0).stop 2024-09-12T02:55:35.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:35.226+0000 7fbbc9fa0640 1 -- 172.21.15.5:0/2845055959 shutdown_connections 2024-09-12T02:55:35.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:35.226+0000 7fbbc9fa0640 1 --2- 172.21.15.5:0/2845055959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbc4102370 0x7fbbc410e8a0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:35.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:35.226+0000 7fbbc9fa0640 1 --2- 172.21.15.5:0/2845055959 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbbc41019d0 0x7fbbc4101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:35.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:35.226+0000 7fbbc9fa0640 1 --2- 172.21.15.5:0/2845055959 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbbc41079d0 0x7fbbc4107db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:35.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:35.226+0000 7fbbc9fa0640 1 -- 172.21.15.5:0/2845055959 >> 172.21.15.5:0/2845055959 conn(0x7fbbc40fd710 msgr2=0x7fbbc40ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:55:35.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:35.226+0000 7fbbc9fa0640 1 -- 172.21.15.5:0/2845055959 shutdown_connections 2024-09-12T02:55:35.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:35.226+0000 7fbbc9fa0640 1 -- 172.21.15.5:0/2845055959 wait complete. 2024-09-12T02:55:35.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:35.227+0000 7fbbc9fa0640 1 Processor -- start 2024-09-12T02:55:35.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:35.227+0000 7fbbc9fa0640 1 -- start start 2024-09-12T02:55:35.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:35.227+0000 7fbbc9fa0640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbbc41019d0 0x7fbbc4198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:35.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:35.227+0000 7fbbc9fa0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbc4102370 0x7fbbc41987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:35.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:35.227+0000 7fbbc9fa0640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbbc41079d0 0x7fbbc419cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:35.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:35.227+0000 7fbbc9fa0640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc4111dd0 con 0x7fbbc4102370 2024-09-12T02:55:35.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:35.227+0000 7fbbc9fa0640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc4111c50 con 0x7fbbc41079d0 2024-09-12T02:55:35.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:35.227+0000 7fbbc9fa0640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc4111f50 con 0x7fbbc41019d0 2024-09-12T02:55:35.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:35.227+0000 7fbbc3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbc4102370 0x7fbbc41987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:35.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:35.227+0000 7fbbc3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbc4102370 0x7fbbc41987b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59524/0 (socket says 172.21.15.5:59524) 2024-09-12T02:55:35.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:35.227+0000 7fbbc3fff640 1 -- 172.21.15.5:0/1501134394 learned_addr learned my addr 172.21.15.5:0/1501134394 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:55:35.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:35.227+0000 7fbbc979f640 1 --2- 172.21.15.5:0/1501134394 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbbc41079d0 0x7fbbc419cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:35.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:35.227+0000 7fbbc8f9e640 1 --2- 172.21.15.5:0/1501134394 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbbc41019d0 0x7fbbc4198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:35.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:35.227+0000 7fbbc3fff640 1 -- 172.21.15.5:0/1501134394 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbbc41019d0 msgr2=0x7fbbc4198270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:35.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:35.227+0000 7fbbc3fff640 1 --2- 172.21.15.5:0/1501134394 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbbc41019d0 0x7fbbc4198270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:35.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:35.227+0000 7fbbc3fff640 1 -- 172.21.15.5:0/1501134394 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbbc41079d0 msgr2=0x7fbbc419cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:35.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:35.227+0000 7fbbc3fff640 1 --2- 172.21.15.5:0/1501134394 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbbc41079d0 0x7fbbc419cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:35.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:35.227+0000 7fbbc3fff640 1 -- 172.21.15.5:0/1501134394 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbc419d220 con 0x7fbbc4102370 2024-09-12T02:55:35.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:35.228+0000 7fbbc3fff640 1 --2- 172.21.15.5:0/1501134394 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbc4102370 0x7fbbc41987b0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fbbac00d770 tx=0x7fbbac00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:35.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:35.228+0000 7fbbc1ffb640 1 -- 172.21.15.5:0/1501134394 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbac017070 con 0x7fbbc4102370 2024-09-12T02:55:35.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:35.228+0000 7fbbc9fa0640 1 -- 172.21.15.5:0/1501134394 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbc419d510 con 0x7fbbc4102370 2024-09-12T02:55:35.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:35.228+0000 7fbbc1ffb640 1 -- 172.21.15.5:0/1501134394 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbbac005500 con 0x7fbbc4102370 2024-09-12T02:55:35.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:35.228+0000 7fbbc1ffb640 1 -- 172.21.15.5:0/1501134394 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbac004e00 con 0x7fbbc4102370 2024-09-12T02:55:35.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:35.228+0000 7fbbc9fa0640 1 -- 172.21.15.5:0/1501134394 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbc41a4df0 con 0x7fbbc4102370 2024-09-12T02:55:35.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:35.229+0000 7fbbc9fa0640 1 -- 172.21.15.5:0/1501134394 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbbc4101e30 con 0x7fbbc4102370 2024-09-12T02:55:35.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:35.230+0000 7fbbc1ffb640 1 -- 172.21.15.5:0/1501134394 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fbbac004fa0 con 0x7fbbc4102370 2024-09-12T02:55:35.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:35.231+0000 7fbbc1ffb640 1 --2- 172.21.15.5:0/1501134394 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fbbb00a2b80 0x7fbbb00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:35.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:35.231+0000 7fbbc8f9e640 1 --2- 172.21.15.5:0/1501134394 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fbbb00a2b80 0x7fbbb00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:35.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:35.231+0000 7fbbc1ffb640 1 -- 172.21.15.5:0/1501134394 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7fbbac0ceaf0 con 0x7fbbc4102370 2024-09-12T02:55:35.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:35.231+0000 7fbbc8f9e640 1 --2- 172.21.15.5:0/1501134394 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fbbb00a2b80 0x7fbbb00a5040 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fbbb4005fd0 tx=0x7fbbb4009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:35.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:35.234+0000 7fbbc1ffb640 1 -- 172.21.15.5:0/1501134394 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbbac09be50 con 0x7fbbc4102370 2024-09-12T02:55:35.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:35.446+0000 7fbbc9fa0640 1 -- 172.21.15.5:0/1501134394 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbbc41995d0 con 0x7fbbc4102370 2024-09-12T02:55:35.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:35.446+0000 7fbbc1ffb640 1 -- 172.21.15.5:0/1501134394 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+611 (secure 0 0 0) 0x7fbbc41995d0 con 0x7fbbc4102370 2024-09-12T02:55:35.447 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-12T02:55:35.447 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":1,"btime":"2024-09-12T02:48:42:351994+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-09-12T02:55:35.447 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 1 2024-09-12T02:55:35.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:35.448+0000 7fbbc9fa0640 1 -- 172.21.15.5:0/1501134394 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fbbb00a2b80 msgr2=0x7fbbb00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:35.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:35.448+0000 7fbbc9fa0640 1 --2- 172.21.15.5:0/1501134394 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fbbb00a2b80 0x7fbbb00a5040 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fbbb4005fd0 tx=0x7fbbb4009260 comp rx=0 tx=0).stop 2024-09-12T02:55:35.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:35.448+0000 7fbbc9fa0640 1 -- 172.21.15.5:0/1501134394 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbc4102370 msgr2=0x7fbbc41987b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:35.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:35.448+0000 7fbbc9fa0640 1 --2- 172.21.15.5:0/1501134394 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbc4102370 0x7fbbc41987b0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fbbac00d770 tx=0x7fbbac00dc30 comp rx=0 tx=0).stop 2024-09-12T02:55:35.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:35.448+0000 7fbbc9fa0640 1 -- 172.21.15.5:0/1501134394 shutdown_connections 2024-09-12T02:55:35.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:35.448+0000 7fbbc9fa0640 1 --2- 172.21.15.5:0/1501134394 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fbbb00a2b80 0x7fbbb00a5040 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:35.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:35.448+0000 7fbbc9fa0640 1 --2- 172.21.15.5:0/1501134394 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbc4102370 0x7fbbc41987b0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:35.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:35.448+0000 7fbbc9fa0640 1 --2- 172.21.15.5:0/1501134394 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbbc41079d0 0x7fbbc419cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:35.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:35.448+0000 7fbbc9fa0640 1 --2- 172.21.15.5:0/1501134394 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbbc41019d0 0x7fbbc4198270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:35.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:35.448+0000 7fbbc9fa0640 1 -- 172.21.15.5:0/1501134394 >> 172.21.15.5:0/1501134394 conn(0x7fbbc40fd710 msgr2=0x7fbbc40fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:55:35.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:35.449+0000 7fbbc9fa0640 1 -- 172.21.15.5:0/1501134394 shutdown_connections 2024-09-12T02:55:35.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:35.449+0000 7fbbc9fa0640 1 -- 172.21.15.5:0/1501134394 wait complete. 2024-09-12T02:55:35.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:35 smithi146 ceph-mon[35011]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:55:35.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:35 smithi146 ceph-mon[35011]: Deploying daemon mds.l on smithi148 2024-09-12T02:55:35.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:35 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/1501134394' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:55:35.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:35 smithi005 ceph-mon[31197]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:55:35.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:35 smithi005 ceph-mon[31197]: Deploying daemon mds.l on smithi148 2024-09-12T02:55:35.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:35 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/1501134394' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:55:35.885 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:35 smithi148 ceph-mon[35090]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:55:35.885 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:35 smithi148 ceph-mon[35090]: Deploying daemon mds.l on smithi148 2024-09-12T02:55:35.885 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:35 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/1501134394' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:55:35.885 INFO:journalctl@ceph.mds.l.smithi148.stdout:Sep 12 02:55:35 smithi148 systemd[1]: Starting Ceph mds.l for 618305f8-70b1-11ef-bcea-c7b262605968... 2024-09-12T02:55:36.137 INFO:journalctl@ceph.mds.l.smithi148.stdout:Sep 12 02:55:35 smithi148 podman[64633]: 2024-09-12 02:55:35.885006493 +0000 UTC m=+0.110291920 container create d3975d960a316984d08fff93587579a89f37aa124d7cc4c1116f469efcd4654c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-l, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T02:55:36.137 INFO:journalctl@ceph.mds.l.smithi148.stdout:Sep 12 02:55:35 smithi148 podman[64633]: 2024-09-12 02:55:35.790900891 +0000 UTC m=+0.016186320 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T02:55:36.137 INFO:journalctl@ceph.mds.l.smithi148.stdout:Sep 12 02:55:36 smithi148 podman[64633]: 2024-09-12 02:55:36.079051464 +0000 UTC m=+0.304336893 container init d3975d960a316984d08fff93587579a89f37aa124d7cc4c1116f469efcd4654c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-l, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-12T02:55:36.138 INFO:journalctl@ceph.mds.l.smithi148.stdout:Sep 12 02:55:36 smithi148 podman[64633]: 2024-09-12 02:55:36.081814929 +0000 UTC m=+0.307100358 container start d3975d960a316984d08fff93587579a89f37aa124d7cc4c1116f469efcd4654c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-l, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731) 2024-09-12T02:55:36.138 INFO:journalctl@ceph.mds.l.smithi148.stdout:Sep 12 02:55:36 smithi148 ceph-mds[64653]: -- 172.21.15.148:0/3851255482 <== mon.1 v2:172.21.15.148:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e227deb860 con 0x55e227dc8800 2024-09-12T02:55:36.138 INFO:journalctl@ceph.mds.l.smithi148.stdout:Sep 12 02:55:36 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-l[64649]: starting mds.l at 2024-09-12T02:55:36.138 INFO:journalctl@ceph.mds.l.smithi148.stdout:Sep 12 02:55:36 smithi148 bash[64633]: d3975d960a316984d08fff93587579a89f37aa124d7cc4c1116f469efcd4654c 2024-09-12T02:55:36.415 INFO:journalctl@ceph.mds.l.smithi148.stdout:Sep 12 02:55:36 smithi148 systemd[1]: Started Ceph mds.l for 618305f8-70b1-11ef-bcea-c7b262605968. 2024-09-12T02:55:37.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:37 smithi146 ceph-mon[35011]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:55:37.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:37 smithi146 ceph-mon[35011]: mds.? [v2:172.21.15.148:6832/568512209,v1:172.21.15.148:6833/568512209] up:boot 2024-09-12T02:55:37.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:37 smithi146 ceph-mon[35011]: fsmap 1 up:standby 2024-09-12T02:55:37.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:37 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-12T02:55:37.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:37 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:37.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:37 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:37.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:37 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:37.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:37 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T02:55:37.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:37 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T02:55:37.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:37 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:55:37.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:37 smithi146 ceph-mon[35011]: Deploying daemon mds.k on smithi146 2024-09-12T02:55:37.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:37 smithi148 ceph-mon[35090]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:55:37.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:37 smithi148 ceph-mon[35090]: mds.? [v2:172.21.15.148:6832/568512209,v1:172.21.15.148:6833/568512209] up:boot 2024-09-12T02:55:37.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:37 smithi148 ceph-mon[35090]: fsmap 1 up:standby 2024-09-12T02:55:37.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:37 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-12T02:55:37.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:37 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:37.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:37 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:37.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:37 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:37.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:37 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T02:55:37.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:37 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T02:55:37.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:37 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:55:37.417 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:37 smithi148 ceph-mon[35090]: Deploying daemon mds.k on smithi146 2024-09-12T02:55:37.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:37 smithi005 ceph-mon[31197]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:55:37.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:37 smithi005 ceph-mon[31197]: mds.? [v2:172.21.15.148:6832/568512209,v1:172.21.15.148:6833/568512209] up:boot 2024-09-12T02:55:37.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:37 smithi005 ceph-mon[31197]: fsmap 1 up:standby 2024-09-12T02:55:37.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:37 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-12T02:55:37.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:37 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:37.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:37 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:37.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:37 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:37.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:37 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T02:55:37.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:37 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T02:55:37.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:37 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:55:37.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:37 smithi005 ceph-mon[31197]: Deploying daemon mds.k on smithi146 2024-09-12T02:55:38.088 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:55:38.357 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:55:38.360 INFO:journalctl@ceph.mds.k.smithi146.stdout:Sep 12 02:55:38 smithi146 systemd[1]: Starting Ceph mds.k for 618305f8-70b1-11ef-bcea-c7b262605968... 2024-09-12T02:55:38.674 INFO:journalctl@ceph.mds.k.smithi146.stdout:Sep 12 02:55:38 smithi146 podman[64485]: 2024-09-12 02:55:38.263590664 +0000 UTC m=+0.016391813 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T02:55:38.674 INFO:journalctl@ceph.mds.k.smithi146.stdout:Sep 12 02:55:38 smithi146 podman[64485]: 2024-09-12 02:55:38.388547332 +0000 UTC m=+0.141348474 container create 889f13ae82d140d2d4ffadb8fe1751d38a7eed5a018267a9133d2877e6df5242 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-k, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, ceph=True, GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T02:55:38.674 INFO:journalctl@ceph.mds.k.smithi146.stdout:Sep 12 02:55:38 smithi146 podman[64485]: 2024-09-12 02:55:38.582594448 +0000 UTC m=+0.335395613 container init 889f13ae82d140d2d4ffadb8fe1751d38a7eed5a018267a9133d2877e6df5242 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-k, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, GIT_CLEAN=True, ceph=True) 2024-09-12T02:55:38.674 INFO:journalctl@ceph.mds.k.smithi146.stdout:Sep 12 02:55:38 smithi146 podman[64485]: 2024-09-12 02:55:38.586216969 +0000 UTC m=+0.339018121 container start 889f13ae82d140d2d4ffadb8fe1751d38a7eed5a018267a9133d2877e6df5242 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-k, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T02:55:38.674 INFO:journalctl@ceph.mds.k.smithi146.stdout:Sep 12 02:55:38 smithi146 ceph-mds[64504]: -- 172.21.15.146:0/1294739383 <== mon.2 v2:172.21.15.146:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556d66e7f860 con 0x556d66e5c800 2024-09-12T02:55:38.674 INFO:journalctl@ceph.mds.k.smithi146.stdout:Sep 12 02:55:38 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-k[64500]: starting mds.k at 2024-09-12T02:55:38.674 INFO:journalctl@ceph.mds.k.smithi146.stdout:Sep 12 02:55:38 smithi146 bash[64485]: 889f13ae82d140d2d4ffadb8fe1751d38a7eed5a018267a9133d2877e6df5242 2024-09-12T02:55:38.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:38.803+0000 7fa0dfe1b640 1 -- 172.21.15.5:0/1056591159 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0d8102ef0 msgr2=0x7fa0d810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:38.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:38.803+0000 7fa0dfe1b640 1 --2- 172.21.15.5:0/1056591159 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0d8102ef0 0x7fa0d810da40 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa0d0009ca0 tx=0x7fa0d002a950 comp rx=0 tx=0).stop 2024-09-12T02:55:38.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:38.804+0000 7fa0dfe1b640 1 -- 172.21.15.5:0/1056591159 shutdown_connections 2024-09-12T02:55:38.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:38.804+0000 7fa0dfe1b640 1 --2- 172.21.15.5:0/1056591159 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0d8102ef0 0x7fa0d810da40 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:38.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:38.804+0000 7fa0dfe1b640 1 --2- 172.21.15.5:0/1056591159 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa0d8102570 0x7fa0d81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:38.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:38.804+0000 7fa0dfe1b640 1 --2- 172.21.15.5:0/1056591159 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa0d8100e20 0x7fa0d8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:38.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:38.804+0000 7fa0dfe1b640 1 -- 172.21.15.5:0/1056591159 >> 172.21.15.5:0/1056591159 conn(0x7fa0d80fc7d0 msgr2=0x7fa0d80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:55:38.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:38.804+0000 7fa0dfe1b640 1 -- 172.21.15.5:0/1056591159 shutdown_connections 2024-09-12T02:55:38.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:38.804+0000 7fa0dfe1b640 1 -- 172.21.15.5:0/1056591159 wait complete. 2024-09-12T02:55:38.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:38.804+0000 7fa0dfe1b640 1 Processor -- start 2024-09-12T02:55:38.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:38.805+0000 7fa0dfe1b640 1 -- start start 2024-09-12T02:55:38.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:38.805+0000 7fa0dfe1b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0d8100e20 0x7fa0d80fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:38.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:38.805+0000 7fa0dfe1b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa0d8102570 0x7fa0d80fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:38.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:38.805+0000 7fa0dfe1b640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa0d8102ef0 0x7fa0d80fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:38.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:38.805+0000 7fa0dfe1b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa0d8110f80 con 0x7fa0d8100e20 2024-09-12T02:55:38.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:38.805+0000 7fa0dfe1b640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fa0d8110e00 con 0x7fa0d8102ef0 2024-09-12T02:55:38.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:38.805+0000 7fa0dfe1b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa0d8111100 con 0x7fa0d8102570 2024-09-12T02:55:38.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:38.805+0000 7fa0ddbb7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0d8100e20 0x7fa0d80fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:38.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:38.805+0000 7fa0ddbb7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0d8100e20 0x7fa0d80fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59528/0 (socket says 172.21.15.5:59528) 2024-09-12T02:55:38.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:38.805+0000 7fa0ddbb7640 1 -- 172.21.15.5:0/1842117091 learned_addr learned my addr 172.21.15.5:0/1842117091 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:55:38.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:38.805+0000 7fa0de3b8640 1 --2- 172.21.15.5:0/1842117091 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa0d8102ef0 0x7fa0d80fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:38.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:38.805+0000 7fa0dd3b6640 1 --2- 172.21.15.5:0/1842117091 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa0d8102570 0x7fa0d80fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:38.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:38.805+0000 7fa0ddbb7640 1 -- 172.21.15.5:0/1842117091 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa0d8102570 msgr2=0x7fa0d80fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:38.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:38.805+0000 7fa0ddbb7640 1 --2- 172.21.15.5:0/1842117091 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa0d8102570 0x7fa0d80fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:38.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:38.805+0000 7fa0ddbb7640 1 -- 172.21.15.5:0/1842117091 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa0d8102ef0 msgr2=0x7fa0d80fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:38.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:38.805+0000 7fa0ddbb7640 1 --2- 172.21.15.5:0/1842117091 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa0d8102ef0 0x7fa0d80fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:38.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:38.805+0000 7fa0ddbb7640 1 -- 172.21.15.5:0/1842117091 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0d81a69c0 con 0x7fa0d8100e20 2024-09-12T02:55:38.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:38.806+0000 7fa0ddbb7640 1 --2- 172.21.15.5:0/1842117091 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0d8100e20 0x7fa0d80fcc80 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa0d400d950 tx=0x7fa0d400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:38.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:38.806+0000 7fa0ceffd640 1 -- 172.21.15.5:0/1842117091 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0d4017070 con 0x7fa0d8100e20 2024-09-12T02:55:38.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:38.806+0000 7fa0ceffd640 1 -- 172.21.15.5:0/1842117091 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0d4009c30 con 0x7fa0d8100e20 2024-09-12T02:55:38.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:38.806+0000 7fa0dfe1b640 1 -- 172.21.15.5:0/1842117091 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0d81a6cb0 con 0x7fa0d8100e20 2024-09-12T02:55:38.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:38.806+0000 7fa0ceffd640 1 -- 172.21.15.5:0/1842117091 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0d400f6b0 con 0x7fa0d8100e20 2024-09-12T02:55:38.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:38.806+0000 7fa0dfe1b640 1 -- 172.21.15.5:0/1842117091 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0d81a71e0 con 0x7fa0d8100e20 2024-09-12T02:55:38.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:38.807+0000 7fa0dfe1b640 1 -- 172.21.15.5:0/1842117091 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0d8101200 con 0x7fa0d8100e20 2024-09-12T02:55:38.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:38.808+0000 7fa0ceffd640 1 -- 172.21.15.5:0/1842117091 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fa0d40109d0 con 0x7fa0d8100e20 2024-09-12T02:55:38.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:38.809+0000 7fa0ceffd640 1 --2- 172.21.15.5:0/1842117091 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fa0c40a2c00 0x7fa0c40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:38.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:38.809+0000 7fa0dd3b6640 1 --2- 172.21.15.5:0/1842117091 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fa0c40a2c00 0x7fa0c40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:38.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:38.809+0000 7fa0ceffd640 1 -- 172.21.15.5:0/1842117091 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7fa0d40cea40 con 0x7fa0d8100e20 2024-09-12T02:55:38.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:38.810+0000 7fa0dd3b6640 1 --2- 172.21.15.5:0/1842117091 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fa0c40a2c00 0x7fa0c40a50c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa0d80fe440 tx=0x7fa0c800b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:38.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:38.814+0000 7fa0ceffd640 1 -- 172.21.15.5:0/1842117091 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa0d4009de0 con 0x7fa0d8100e20 2024-09-12T02:55:38.956 INFO:journalctl@ceph.mds.k.smithi146.stdout:Sep 12 02:55:38 smithi146 systemd[1]: Started Ceph mds.k for 618305f8-70b1-11ef-bcea-c7b262605968. 2024-09-12T02:55:39.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:39.039+0000 7fa0dfe1b640 1 -- 172.21.15.5:0/1842117091 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa0d80fe1b0 con 0x7fa0d8100e20 2024-09-12T02:55:39.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:39.040+0000 7fa0ceffd640 1 -- 172.21.15.5:0/1842117091 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1435 (secure 0 0 0) 0x7fa0d409b7d0 con 0x7fa0d8100e20 2024-09-12T02:55:39.041 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-12T02:55:39.041 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":2,"btime":"2024-09-12T02:55:36:110153+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/568512209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":568512209},{"type":"v1","addr":"172.21.15.148:6833","nonce":568512209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-12T02:55:39.042 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 2 2024-09-12T02:55:39.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:39.043+0000 7fa0dfe1b640 1 -- 172.21.15.5:0/1842117091 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fa0c40a2c00 msgr2=0x7fa0c40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:39.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:39.043+0000 7fa0dfe1b640 1 --2- 172.21.15.5:0/1842117091 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fa0c40a2c00 0x7fa0c40a50c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa0d80fe440 tx=0x7fa0c800b390 comp rx=0 tx=0).stop 2024-09-12T02:55:39.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:39.043+0000 7fa0dfe1b640 1 -- 172.21.15.5:0/1842117091 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0d8100e20 msgr2=0x7fa0d80fcc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:39.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:39.043+0000 7fa0dfe1b640 1 --2- 172.21.15.5:0/1842117091 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0d8100e20 0x7fa0d80fcc80 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa0d400d950 tx=0x7fa0d400de10 comp rx=0 tx=0).stop 2024-09-12T02:55:39.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:39.043+0000 7fa0dfe1b640 1 -- 172.21.15.5:0/1842117091 shutdown_connections 2024-09-12T02:55:39.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:39.043+0000 7fa0dfe1b640 1 --2- 172.21.15.5:0/1842117091 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa0d8102ef0 0x7fa0d80fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:39.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:39.044+0000 7fa0dfe1b640 1 --2- 172.21.15.5:0/1842117091 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa0d8102570 0x7fa0d80fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:39.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:39.044+0000 7fa0dfe1b640 1 --2- 172.21.15.5:0/1842117091 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fa0c40a2c00 0x7fa0c40a50c0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:39.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:39.044+0000 7fa0dfe1b640 1 --2- 172.21.15.5:0/1842117091 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0d8100e20 0x7fa0d80fcc80 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:39.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:39.044+0000 7fa0dfe1b640 1 -- 172.21.15.5:0/1842117091 >> 172.21.15.5:0/1842117091 conn(0x7fa0d80fc7d0 msgr2=0x7fa0d806a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:55:39.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:39.044+0000 7fa0dfe1b640 1 -- 172.21.15.5:0/1842117091 shutdown_connections 2024-09-12T02:55:39.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:39.044+0000 7fa0dfe1b640 1 -- 172.21.15.5:0/1842117091 wait complete. 2024-09-12T02:55:39.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:39 smithi005 ceph-mon[31197]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:55:39.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:39 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:39.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:39 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:39.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:39 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:39.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:39 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T02:55:39.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:39 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T02:55:39.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:39 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:55:39.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:39 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/1842117091' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:55:39.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:39 smithi148 ceph-mon[35090]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:55:39.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:39 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:39.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:39 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:39.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:39 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:39.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:39 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T02:55:39.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:39 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T02:55:39.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:39 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:55:39.417 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:39 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/1842117091' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:55:39.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:39 smithi146 ceph-mon[35011]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:55:39.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:39 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:39.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:39 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:39.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:39 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:39.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:39 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T02:55:39.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:39 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T02:55:39.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:39 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:55:39.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:39 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/1842117091' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:55:40.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:40 smithi005 ceph-mon[31197]: Deploying daemon mds.j on smithi005 2024-09-12T02:55:40.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:40 smithi005 ceph-mon[31197]: mds.? [v2:172.21.15.146:6832/3686970137,v1:172.21.15.146:6833/3686970137] up:boot 2024-09-12T02:55:40.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:40 smithi005 ceph-mon[31197]: fsmap 2 up:standby 2024-09-12T02:55:40.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:40 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-12T02:55:40.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:40 smithi148 ceph-mon[35090]: Deploying daemon mds.j on smithi005 2024-09-12T02:55:40.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:40 smithi148 ceph-mon[35090]: mds.? [v2:172.21.15.146:6832/3686970137,v1:172.21.15.146:6833/3686970137] up:boot 2024-09-12T02:55:40.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:40 smithi148 ceph-mon[35090]: fsmap 2 up:standby 2024-09-12T02:55:40.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:40 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-12T02:55:40.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:40 smithi146 ceph-mon[35011]: Deploying daemon mds.j on smithi005 2024-09-12T02:55:40.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:40 smithi146 ceph-mon[35011]: mds.? [v2:172.21.15.146:6832/3686970137,v1:172.21.15.146:6833/3686970137] up:boot 2024-09-12T02:55:40.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:40 smithi146 ceph-mon[35011]: fsmap 2 up:standby 2024-09-12T02:55:40.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:40 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-12T02:55:41.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:41 smithi148 ceph-mon[35090]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:55:41.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:41 smithi146 ceph-mon[35011]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:55:41.549 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:41 smithi005 ceph-mon[31197]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:55:41.549 INFO:journalctl@ceph.mds.j.smithi005.stdout:Sep 12 02:55:41 smithi005 systemd[1]: Starting Ceph mds.j for 618305f8-70b1-11ef-bcea-c7b262605968... 2024-09-12T02:55:41.753 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:55:41.810 INFO:journalctl@ceph.mds.j.smithi005.stdout:Sep 12 02:55:41 smithi005 podman[68356]: 2024-09-12 02:55:41.549040508 +0000 UTC m=+0.106860058 container create 6307c2f6d51ab2a63267ac61623dbc66702827b4b88ea1a1882930c4d0fda744 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3) 2024-09-12T02:55:41.810 INFO:journalctl@ceph.mds.j.smithi005.stdout:Sep 12 02:55:41 smithi005 podman[68356]: 2024-09-12 02:55:41.458630656 +0000 UTC m=+0.016450219 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T02:55:41.810 INFO:journalctl@ceph.mds.j.smithi005.stdout:Sep 12 02:55:41 smithi005 podman[68356]: 2024-09-12 02:55:41.793633051 +0000 UTC m=+0.351452601 container init 6307c2f6d51ab2a63267ac61623dbc66702827b4b88ea1a1882930c4d0fda744 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-j, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, ceph=True) 2024-09-12T02:55:41.811 INFO:journalctl@ceph.mds.j.smithi005.stdout:Sep 12 02:55:41 smithi005 podman[68356]: 2024-09-12 02:55:41.797697549 +0000 UTC m=+0.355517100 container start 6307c2f6d51ab2a63267ac61623dbc66702827b4b88ea1a1882930c4d0fda744 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-j, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, ceph=True, org.label-schema.vendor=CentOS) 2024-09-12T02:55:42.028 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:55:42.150 INFO:journalctl@ceph.mds.j.smithi005.stdout:Sep 12 02:55:41 smithi005 ceph-mds[68394]: -- 172.21.15.5:0/2244340397 <== mon.0 v2:172.21.15.5:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5594fb037860 con 0x5594fb014800 2024-09-12T02:55:42.150 INFO:journalctl@ceph.mds.j.smithi005.stdout:Sep 12 02:55:41 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-j[68372]: starting mds.j at 2024-09-12T02:55:42.150 INFO:journalctl@ceph.mds.j.smithi005.stdout:Sep 12 02:55:41 smithi005 bash[68356]: 6307c2f6d51ab2a63267ac61623dbc66702827b4b88ea1a1882930c4d0fda744 2024-09-12T02:55:42.150 INFO:journalctl@ceph.mds.j.smithi005.stdout:Sep 12 02:55:41 smithi005 systemd[1]: Started Ceph mds.j for 618305f8-70b1-11ef-bcea-c7b262605968. 2024-09-12T02:55:42.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:42.471+0000 7fe74e829640 1 -- 172.21.15.5:0/2524238957 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe748106f10 msgr2=0x7fe74810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:42.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:42.471+0000 7fe74e829640 1 --2- 172.21.15.5:0/2524238957 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe748106f10 0x7fe74810ea90 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe740009ca0 tx=0x7fe74002a8c0 comp rx=0 tx=0).stop 2024-09-12T02:55:42.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:42.471+0000 7fe74e829640 1 -- 172.21.15.5:0/2524238957 shutdown_connections 2024-09-12T02:55:42.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:42.471+0000 7fe74e829640 1 --2- 172.21.15.5:0/2524238957 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe748106f10 0x7fe74810ea90 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:42.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:42.471+0000 7fe74e829640 1 --2- 172.21.15.5:0/2524238957 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe748102790 0x7fe7481067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:42.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:42.471+0000 7fe74e829640 1 --2- 172.21.15.5:0/2524238957 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe748101de0 0x7fe7481021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:42.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:42.471+0000 7fe74e829640 1 -- 172.21.15.5:0/2524238957 >> 172.21.15.5:0/2524238957 conn(0x7fe7480fd8c0 msgr2=0x7fe7480ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:55:42.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:42.471+0000 7fe74e829640 1 -- 172.21.15.5:0/2524238957 shutdown_connections 2024-09-12T02:55:42.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:42.471+0000 7fe74e829640 1 -- 172.21.15.5:0/2524238957 wait complete. 2024-09-12T02:55:42.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:42.472+0000 7fe74e829640 1 Processor -- start 2024-09-12T02:55:42.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:42.472+0000 7fe74e829640 1 -- start start 2024-09-12T02:55:42.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:42.472+0000 7fe74e829640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe748101de0 0x7fe7481983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:42.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:42.472+0000 7fe74e829640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe748102790 0x7fe748198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:42.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:42.472+0000 7fe74e829640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe748106f10 0x7fe74819ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:42.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:42.472+0000 7fe74e829640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe748111f70 con 0x7fe748101de0 2024-09-12T02:55:42.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:42.472+0000 7fe74e829640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fe748111df0 con 0x7fe748106f10 2024-09-12T02:55:42.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:42.472+0000 7fe74e829640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe7481120f0 con 0x7fe748102790 2024-09-12T02:55:42.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:42.472+0000 7fe747fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe748101de0 0x7fe7481983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:42.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:42.472+0000 7fe747fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe748101de0 0x7fe7481983f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59566/0 (socket says 172.21.15.5:59566) 2024-09-12T02:55:42.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:42.472+0000 7fe747fff640 1 -- 172.21.15.5:0/288672611 learned_addr learned my addr 172.21.15.5:0/288672611 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:55:42.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:42.473+0000 7fe74cdc6640 1 --2- 172.21.15.5:0/288672611 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe748106f10 0x7fe74819ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:42.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:42.473+0000 7fe7477fe640 1 --2- 172.21.15.5:0/288672611 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe748102790 0x7fe748198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:42.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:42.473+0000 7fe747fff640 1 -- 172.21.15.5:0/288672611 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe748102790 msgr2=0x7fe748198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:42.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:42.473+0000 7fe747fff640 1 --2- 172.21.15.5:0/288672611 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe748102790 0x7fe748198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:42.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:42.473+0000 7fe747fff640 1 -- 172.21.15.5:0/288672611 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe748106f10 msgr2=0x7fe74819ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:42.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:42.473+0000 7fe747fff640 1 --2- 172.21.15.5:0/288672611 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe748106f10 0x7fe74819ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:42.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:42.473+0000 7fe747fff640 1 -- 172.21.15.5:0/288672611 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe74819d3a0 con 0x7fe748101de0 2024-09-12T02:55:42.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:42.473+0000 7fe747fff640 1 --2- 172.21.15.5:0/288672611 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe748101de0 0x7fe7481983f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe73800d770 tx=0x7fe73800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:42.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:42.473+0000 7fe7457fa640 1 -- 172.21.15.5:0/288672611 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe738017070 con 0x7fe748101de0 2024-09-12T02:55:42.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:42.473+0000 7fe74e829640 1 -- 172.21.15.5:0/288672611 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe74819d690 con 0x7fe748101de0 2024-09-12T02:55:42.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:42.473+0000 7fe7457fa640 1 -- 172.21.15.5:0/288672611 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe738005500 con 0x7fe748101de0 2024-09-12T02:55:42.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:42.473+0000 7fe7457fa640 1 -- 172.21.15.5:0/288672611 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe738004e00 con 0x7fe748101de0 2024-09-12T02:55:42.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:42.473+0000 7fe74e829640 1 -- 172.21.15.5:0/288672611 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7481a4f70 con 0x7fe748101de0 2024-09-12T02:55:42.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:42.474+0000 7fe74e829640 1 -- 172.21.15.5:0/288672611 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7481021c0 con 0x7fe748101de0 2024-09-12T02:55:42.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:42.476+0000 7fe7457fa640 1 -- 172.21.15.5:0/288672611 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fe738004fa0 con 0x7fe748101de0 2024-09-12T02:55:42.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:42.476+0000 7fe7457fa640 1 --2- 172.21.15.5:0/288672611 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fe7300a2b80 0x7fe7300a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:42.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:42.476+0000 7fe7477fe640 1 --2- 172.21.15.5:0/288672611 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fe7300a2b80 0x7fe7300a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:42.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:42.476+0000 7fe7457fa640 1 -- 172.21.15.5:0/288672611 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7fe7380ce920 con 0x7fe748101de0 2024-09-12T02:55:42.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:42.477+0000 7fe7477fe640 1 --2- 172.21.15.5:0/288672611 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fe7300a2b80 0x7fe7300a5040 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe7480fe0c0 tx=0x7fe73c00b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:42.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:42.479+0000 7fe7457fa640 1 -- 172.21.15.5:0/288672611 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe73809b6d0 con 0x7fe748101de0 2024-09-12T02:55:42.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:42.700+0000 7fe74e829640 1 -- 172.21.15.5:0/288672611 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe748199750 con 0x7fe748101de0 2024-09-12T02:55:42.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:42.701+0000 7fe7457fa640 1 -- 172.21.15.5:0/288672611 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3085 (secure 0 0 0) 0x7fe73809f9d0 con 0x7fe748101de0 2024-09-12T02:55:42.702 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-12T02:55:42.702 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":4,"btime":"2024-09-12T02:55:42:127560+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/4135232521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":4135232521},{"type":"v1","addr":"172.21.15.5:6835","nonce":4135232521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/3686970137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3686970137},{"type":"v1","addr":"172.21.15.146:6833","nonce":3686970137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/568512209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":568512209},{"type":"v1","addr":"172.21.15.148:6833","nonce":568512209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-12T02:55:42.702 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 4 2024-09-12T02:55:42.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:42.703+0000 7fe74e829640 1 -- 172.21.15.5:0/288672611 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fe7300a2b80 msgr2=0x7fe7300a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:42.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:42.703+0000 7fe74e829640 1 --2- 172.21.15.5:0/288672611 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fe7300a2b80 0x7fe7300a5040 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe7480fe0c0 tx=0x7fe73c00b390 comp rx=0 tx=0).stop 2024-09-12T02:55:42.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:42.703+0000 7fe74e829640 1 -- 172.21.15.5:0/288672611 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe748101de0 msgr2=0x7fe7481983f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:42.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:42.703+0000 7fe74e829640 1 --2- 172.21.15.5:0/288672611 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe748101de0 0x7fe7481983f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe73800d770 tx=0x7fe73800dc30 comp rx=0 tx=0).stop 2024-09-12T02:55:42.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:42.703+0000 7fe74e829640 1 -- 172.21.15.5:0/288672611 shutdown_connections 2024-09-12T02:55:42.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:42.703+0000 7fe74e829640 1 --2- 172.21.15.5:0/288672611 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe748106f10 0x7fe74819ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:42.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:42.703+0000 7fe74e829640 1 --2- 172.21.15.5:0/288672611 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe748102790 0x7fe748198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:42.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:42.703+0000 7fe74e829640 1 --2- 172.21.15.5:0/288672611 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fe7300a2b80 0x7fe7300a5040 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:42.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:42.703+0000 7fe74e829640 1 --2- 172.21.15.5:0/288672611 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe748101de0 0x7fe7481983f0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:42.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:42.703+0000 7fe74e829640 1 -- 172.21.15.5:0/288672611 >> 172.21.15.5:0/288672611 conn(0x7fe7480fd8c0 msgr2=0x7fe7480fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:55:42.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:42.703+0000 7fe74e829640 1 -- 172.21.15.5:0/288672611 shutdown_connections 2024-09-12T02:55:42.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:42.703+0000 7fe74e829640 1 -- 172.21.15.5:0/288672611 wait complete. 2024-09-12T02:55:43.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:42 smithi005 ceph-mon[31197]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:55:43.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:42 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:43.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:42 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:43.061 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:42 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:43.061 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:42 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T02:55:43.061 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:42 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T02:55:43.061 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:42 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:55:43.061 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:42 smithi005 ceph-mon[31197]: Deploying daemon mds.d on smithi005 2024-09-12T02:55:43.061 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:42 smithi005 ceph-mon[31197]: mds.? [v2:172.21.15.5:6834/4135232521,v1:172.21.15.5:6835/4135232521] up:boot 2024-09-12T02:55:43.061 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:42 smithi005 ceph-mon[31197]: fsmap 3 up:standby 2024-09-12T02:55:43.061 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:42 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-12T02:55:43.061 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:42 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/288672611' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:55:43.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:42 smithi146 ceph-mon[35011]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:55:43.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:42 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:43.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:42 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:43.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:42 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:43.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:42 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T02:55:43.208 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:42 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T02:55:43.208 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:42 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:55:43.208 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:42 smithi146 ceph-mon[35011]: Deploying daemon mds.d on smithi005 2024-09-12T02:55:43.208 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:42 smithi146 ceph-mon[35011]: mds.? [v2:172.21.15.5:6834/4135232521,v1:172.21.15.5:6835/4135232521] up:boot 2024-09-12T02:55:43.208 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:42 smithi146 ceph-mon[35011]: fsmap 3 up:standby 2024-09-12T02:55:43.208 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:42 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-12T02:55:43.208 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:42 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/288672611' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:55:43.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:42 smithi148 ceph-mon[35090]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:55:43.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:42 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:43.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:42 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:43.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:42 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:43.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:42 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T02:55:43.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:42 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T02:55:43.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:42 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:55:43.417 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:42 smithi148 ceph-mon[35090]: Deploying daemon mds.d on smithi005 2024-09-12T02:55:43.417 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:42 smithi148 ceph-mon[35090]: mds.? [v2:172.21.15.5:6834/4135232521,v1:172.21.15.5:6835/4135232521] up:boot 2024-09-12T02:55:43.417 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:42 smithi148 ceph-mon[35090]: fsmap 3 up:standby 2024-09-12T02:55:43.417 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:42 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-12T02:55:43.417 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:42 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/288672611' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:55:44.950 INFO:journalctl@ceph.mds.d.smithi005.stdout:Sep 12 02:55:44 smithi005 systemd[1]: Starting Ceph mds.d for 618305f8-70b1-11ef-bcea-c7b262605968... 2024-09-12T02:55:45.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:44 smithi146 ceph-mon[35011]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:55:45.241 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:44 smithi005 ceph-mon[31197]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:55:45.241 INFO:journalctl@ceph.mds.d.smithi005.stdout:Sep 12 02:55:45 smithi005 podman[68770]: 2024-09-12 02:55:44.991500055 +0000 UTC m=+0.016247260 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T02:55:45.241 INFO:journalctl@ceph.mds.d.smithi005.stdout:Sep 12 02:55:45 smithi005 podman[68770]: 2024-09-12 02:55:45.110949709 +0000 UTC m=+0.135696918 container create eaf2f38083f94da41b0fc877099d5d01d3241eca608a29e3ea6fe088949a8004 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-d, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-09-12T02:55:45.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:44 smithi148 ceph-mon[35090]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:55:45.513 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:55:45.538 INFO:journalctl@ceph.mds.d.smithi005.stdout:Sep 12 02:55:45 smithi005 podman[68770]: 2024-09-12 02:55:45.347272142 +0000 UTC m=+0.372019349 container init eaf2f38083f94da41b0fc877099d5d01d3241eca608a29e3ea6fe088949a8004 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-d, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T02:55:45.538 INFO:journalctl@ceph.mds.d.smithi005.stdout:Sep 12 02:55:45 smithi005 podman[68770]: 2024-09-12 02:55:45.350109246 +0000 UTC m=+0.374856455 container start eaf2f38083f94da41b0fc877099d5d01d3241eca608a29e3ea6fe088949a8004 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-d, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, ceph=True, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T02:55:45.538 INFO:journalctl@ceph.mds.d.smithi005.stdout:Sep 12 02:55:45 smithi005 ceph-mds[68789]: -- 172.21.15.5:0/2691416489 <== mon.0 v2:172.21.15.5:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563c41ffb860 con 0x563c41fd9400 2024-09-12T02:55:45.538 INFO:journalctl@ceph.mds.d.smithi005.stdout:Sep 12 02:55:45 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-d[68785]: starting mds.d at 2024-09-12T02:55:45.538 INFO:journalctl@ceph.mds.d.smithi005.stdout:Sep 12 02:55:45 smithi005 bash[68770]: eaf2f38083f94da41b0fc877099d5d01d3241eca608a29e3ea6fe088949a8004 2024-09-12T02:55:45.539 INFO:journalctl@ceph.mds.d.smithi005.stdout:Sep 12 02:55:45 smithi005 systemd[1]: Started Ceph mds.d for 618305f8-70b1-11ef-bcea-c7b262605968. 2024-09-12T02:55:45.774 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:55:46.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:46.241+0000 7fc4bad4a640 1 -- 172.21.15.5:0/3849148198 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4b4106f10 msgr2=0x7fc4b410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:46.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:46.241+0000 7fc4bad4a640 1 --2- 172.21.15.5:0/3849148198 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4b4106f10 0x7fc4b410ea90 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac009ca0 tx=0x7fc4ac02a8c0 comp rx=0 tx=0).stop 2024-09-12T02:55:46.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:46.241+0000 7fc4bad4a640 1 -- 172.21.15.5:0/3849148198 shutdown_connections 2024-09-12T02:55:46.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:46.241+0000 7fc4bad4a640 1 --2- 172.21.15.5:0/3849148198 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4b4106f10 0x7fc4b410ea90 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:46.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:46.241+0000 7fc4bad4a640 1 --2- 172.21.15.5:0/3849148198 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc4b4102790 0x7fc4b41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:46.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:46.241+0000 7fc4bad4a640 1 --2- 172.21.15.5:0/3849148198 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc4b4101de0 0x7fc4b41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:46.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:46.241+0000 7fc4bad4a640 1 -- 172.21.15.5:0/3849148198 >> 172.21.15.5:0/3849148198 conn(0x7fc4b40fd8c0 msgr2=0x7fc4b40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:55:46.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:46.241+0000 7fc4bad4a640 1 -- 172.21.15.5:0/3849148198 shutdown_connections 2024-09-12T02:55:46.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:46.241+0000 7fc4bad4a640 1 -- 172.21.15.5:0/3849148198 wait complete. 2024-09-12T02:55:46.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:46.242+0000 7fc4bad4a640 1 Processor -- start 2024-09-12T02:55:46.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:46.242+0000 7fc4bad4a640 1 -- start start 2024-09-12T02:55:46.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:46.242+0000 7fc4bad4a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4b4101de0 0x7fc4b40744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:46.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:46.242+0000 7fc4bad4a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc4b4102790 0x7fc4b4074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:46.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:46.242+0000 7fc4bad4a640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc4b4106f10 0x7fc4b4074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:46.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:46.242+0000 7fc4bad4a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc4b4111f70 con 0x7fc4b4101de0 2024-09-12T02:55:46.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:46.242+0000 7fc4bad4a640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fc4b4111df0 con 0x7fc4b4106f10 2024-09-12T02:55:46.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:46.242+0000 7fc4bad4a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc4b41120f0 con 0x7fc4b4102790 2024-09-12T02:55:46.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:46.242+0000 7fc4b8ae6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4b4101de0 0x7fc4b40744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:46.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:46.242+0000 7fc4b8ae6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4b4101de0 0x7fc4b40744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44880/0 (socket says 172.21.15.5:44880) 2024-09-12T02:55:46.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:46.242+0000 7fc4b8ae6640 1 -- 172.21.15.5:0/652406098 learned_addr learned my addr 172.21.15.5:0/652406098 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:55:46.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:46.242+0000 7fc4b92e7640 1 --2- 172.21.15.5:0/652406098 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc4b4106f10 0x7fc4b4074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:46.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:46.243+0000 7fc4b3fff640 1 --2- 172.21.15.5:0/652406098 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc4b4102790 0x7fc4b4074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:46.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:46.243+0000 7fc4b8ae6640 1 -- 172.21.15.5:0/652406098 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc4b4102790 msgr2=0x7fc4b4074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:46.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:46.243+0000 7fc4b8ae6640 1 --2- 172.21.15.5:0/652406098 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc4b4102790 0x7fc4b4074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:46.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:46.243+0000 7fc4b8ae6640 1 -- 172.21.15.5:0/652406098 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc4b4106f10 msgr2=0x7fc4b4074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:46.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:46.243+0000 7fc4b8ae6640 1 --2- 172.21.15.5:0/652406098 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc4b4106f10 0x7fc4b4074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:46.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:46.243+0000 7fc4b8ae6640 1 -- 172.21.15.5:0/652406098 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4b41a6800 con 0x7fc4b4101de0 2024-09-12T02:55:46.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:46.243+0000 7fc4b8ae6640 1 --2- 172.21.15.5:0/652406098 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4b4101de0 0x7fc4b40744f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc49c00d950 tx=0x7fc49c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:46.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:46.243+0000 7fc4b1ffb640 1 -- 172.21.15.5:0/652406098 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc49c017070 con 0x7fc4b4101de0 2024-09-12T02:55:46.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:46.243+0000 7fc4bad4a640 1 -- 172.21.15.5:0/652406098 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4b41a6af0 con 0x7fc4b4101de0 2024-09-12T02:55:46.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:46.243+0000 7fc4b1ffb640 1 -- 172.21.15.5:0/652406098 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc49c009c30 con 0x7fc4b4101de0 2024-09-12T02:55:46.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:46.243+0000 7fc4b1ffb640 1 -- 172.21.15.5:0/652406098 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc49c00f6b0 con 0x7fc4b4101de0 2024-09-12T02:55:46.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:46.243+0000 7fc4bad4a640 1 -- 172.21.15.5:0/652406098 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4b41a7030 con 0x7fc4b4101de0 2024-09-12T02:55:46.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:46.244+0000 7fc4bad4a640 1 -- 172.21.15.5:0/652406098 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4b41021c0 con 0x7fc4b4101de0 2024-09-12T02:55:46.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:46.245+0000 7fc4b1ffb640 1 -- 172.21.15.5:0/652406098 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fc49c0109d0 con 0x7fc4b4101de0 2024-09-12T02:55:46.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:46.246+0000 7fc4b1ffb640 1 --2- 172.21.15.5:0/652406098 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fc4a00a2b80 0x7fc4a00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:46.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:46.246+0000 7fc4b3fff640 1 --2- 172.21.15.5:0/652406098 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fc4a00a2b80 0x7fc4a00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:46.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:46.246+0000 7fc4b1ffb640 1 -- 172.21.15.5:0/652406098 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7fc49c0ce7b0 con 0x7fc4b4101de0 2024-09-12T02:55:46.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:46.247+0000 7fc4b3fff640 1 --2- 172.21.15.5:0/652406098 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fc4a00a2b80 0x7fc4a00a5040 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc4b4075f80 tx=0x7fc4a400c4a0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:46.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:46.250+0000 7fc4b1ffb640 1 -- 172.21.15.5:0/652406098 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc49c009de0 con 0x7fc4b4101de0 2024-09-12T02:55:46.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:46.461+0000 7fc4bad4a640 1 -- 172.21.15.5:0/652406098 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc4b4075a20 con 0x7fc4b4101de0 2024-09-12T02:55:46.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:46.462+0000 7fc4b1ffb640 1 -- 172.21.15.5:0/652406098 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3907 (secure 0 0 0) 0x7fc49c09bbb0 con 0x7fc4b4101de0 2024-09-12T02:55:46.462 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-12T02:55:46.462 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":5,"btime":"2024-09-12T02:55:45:949971+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/4135232521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":4135232521},{"type":"v1","addr":"172.21.15.5:6835","nonce":4135232521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/4290221940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":4290221940},{"type":"v1","addr":"172.21.15.5:6837","nonce":4290221940}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/3686970137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3686970137},{"type":"v1","addr":"172.21.15.146:6833","nonce":3686970137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/568512209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":568512209},{"type":"v1","addr":"172.21.15.148:6833","nonce":568512209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-12T02:55:46.463 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 5 2024-09-12T02:55:46.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:46.464+0000 7fc4bad4a640 1 -- 172.21.15.5:0/652406098 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fc4a00a2b80 msgr2=0x7fc4a00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:46.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:46.464+0000 7fc4bad4a640 1 --2- 172.21.15.5:0/652406098 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fc4a00a2b80 0x7fc4a00a5040 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc4b4075f80 tx=0x7fc4a400c4a0 comp rx=0 tx=0).stop 2024-09-12T02:55:46.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:46.464+0000 7fc4bad4a640 1 -- 172.21.15.5:0/652406098 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4b4101de0 msgr2=0x7fc4b40744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:46.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:46.464+0000 7fc4bad4a640 1 --2- 172.21.15.5:0/652406098 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4b4101de0 0x7fc4b40744f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc49c00d950 tx=0x7fc49c00de10 comp rx=0 tx=0).stop 2024-09-12T02:55:46.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:46.464+0000 7fc4bad4a640 1 -- 172.21.15.5:0/652406098 shutdown_connections 2024-09-12T02:55:46.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:46.464+0000 7fc4bad4a640 1 --2- 172.21.15.5:0/652406098 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc4b4106f10 0x7fc4b4074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:46.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:46.464+0000 7fc4bad4a640 1 --2- 172.21.15.5:0/652406098 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc4b4102790 0x7fc4b4074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:46.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:46.464+0000 7fc4bad4a640 1 --2- 172.21.15.5:0/652406098 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fc4a00a2b80 0x7fc4a00a5040 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:46.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:46.464+0000 7fc4bad4a640 1 --2- 172.21.15.5:0/652406098 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4b4101de0 0x7fc4b40744f0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:46.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:46.464+0000 7fc4bad4a640 1 -- 172.21.15.5:0/652406098 >> 172.21.15.5:0/652406098 conn(0x7fc4b40fd8c0 msgr2=0x7fc4b40febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:55:46.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:46.464+0000 7fc4bad4a640 1 -- 172.21.15.5:0/652406098 shutdown_connections 2024-09-12T02:55:46.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:46.464+0000 7fc4bad4a640 1 -- 172.21.15.5:0/652406098 wait complete. 2024-09-12T02:55:46.761 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:46 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:46.762 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:46 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:46.762 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:46 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:46.762 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:46 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T02:55:46.762 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:46 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T02:55:46.762 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:46 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:55:46.762 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:46 smithi005 ceph-mon[31197]: mds.? [v2:172.21.15.5:6836/4290221940,v1:172.21.15.5:6837/4290221940] up:boot 2024-09-12T02:55:46.762 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:46 smithi005 ceph-mon[31197]: fsmap 4 up:standby 2024-09-12T02:55:46.762 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:46 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-12T02:55:46.763 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:46 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/652406098' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:55:46.813 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:46 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:46.814 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:46 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:46.814 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:46 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:46.814 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:46 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T02:55:46.814 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:46 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T02:55:46.814 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:46 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:55:46.814 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:46 smithi146 ceph-mon[35011]: mds.? [v2:172.21.15.5:6836/4290221940,v1:172.21.15.5:6837/4290221940] up:boot 2024-09-12T02:55:46.814 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:46 smithi146 ceph-mon[35011]: fsmap 4 up:standby 2024-09-12T02:55:46.814 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:46 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-12T02:55:46.814 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:46 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/652406098' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:55:46.915 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:46 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:46.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:46 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:46.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:46 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:46.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:46 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T02:55:46.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:46 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T02:55:46.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:46 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:55:46.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:46 smithi148 ceph-mon[35090]: mds.? [v2:172.21.15.5:6836/4290221940,v1:172.21.15.5:6837/4290221940] up:boot 2024-09-12T02:55:46.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:46 smithi148 ceph-mon[35090]: fsmap 4 up:standby 2024-09-12T02:55:46.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:46 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-12T02:55:46.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:46 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/652406098' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:55:47.457 INFO:journalctl@ceph.mds.e.smithi146.stdout:Sep 12 02:55:47 smithi146 systemd[1]: Starting Ceph mds.e for 618305f8-70b1-11ef-bcea-c7b262605968... 2024-09-12T02:55:47.752 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:47 smithi146 ceph-mon[35011]: Deploying daemon mds.e on smithi146 2024-09-12T02:55:47.752 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:47 smithi146 ceph-mon[35011]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:55:47.753 INFO:journalctl@ceph.mds.e.smithi146.stdout:Sep 12 02:55:47 smithi146 podman[64804]: 2024-09-12 02:55:47.406218948 +0000 UTC m=+0.016433636 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T02:55:47.753 INFO:journalctl@ceph.mds.e.smithi146.stdout:Sep 12 02:55:47 smithi146 podman[64804]: 2024-09-12 02:55:47.515444476 +0000 UTC m=+0.125659154 container create c6bdd085a6c147772cfabf791383e536d0b2b29a41da50c7b7199d39284bffe1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-e, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS) 2024-09-12T02:55:47.753 INFO:journalctl@ceph.mds.e.smithi146.stdout:Sep 12 02:55:47 smithi146 podman[64804]: 2024-09-12 02:55:47.726284601 +0000 UTC m=+0.336499285 container init c6bdd085a6c147772cfabf791383e536d0b2b29a41da50c7b7199d39284bffe1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-e, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T02:55:47.753 INFO:journalctl@ceph.mds.e.smithi146.stdout:Sep 12 02:55:47 smithi146 podman[64804]: 2024-09-12 02:55:47.72922498 +0000 UTC m=+0.339439667 container start c6bdd085a6c147772cfabf791383e536d0b2b29a41da50c7b7199d39284bffe1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-e, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3) 2024-09-12T02:55:47.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:47 smithi005 ceph-mon[31197]: Deploying daemon mds.e on smithi146 2024-09-12T02:55:47.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:47 smithi005 ceph-mon[31197]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:55:47.915 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:47 smithi148 ceph-mon[35090]: Deploying daemon mds.e on smithi146 2024-09-12T02:55:47.915 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:47 smithi148 ceph-mon[35090]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:55:48.206 INFO:journalctl@ceph.mds.e.smithi146.stdout:Sep 12 02:55:47 smithi146 ceph-mds[64823]: -- 172.21.15.146:0/3347163175 <== mon.2 v2:172.21.15.146:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e7b1081860 con 0x55e7b105f000 2024-09-12T02:55:48.206 INFO:journalctl@ceph.mds.e.smithi146.stdout:Sep 12 02:55:47 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-e[64819]: starting mds.e at 2024-09-12T02:55:48.206 INFO:journalctl@ceph.mds.e.smithi146.stdout:Sep 12 02:55:47 smithi146 bash[64804]: c6bdd085a6c147772cfabf791383e536d0b2b29a41da50c7b7199d39284bffe1 2024-09-12T02:55:48.207 INFO:journalctl@ceph.mds.e.smithi146.stdout:Sep 12 02:55:47 smithi146 systemd[1]: Started Ceph mds.e for 618305f8-70b1-11ef-bcea-c7b262605968. 2024-09-12T02:55:48.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:48 smithi148 ceph-mon[35090]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:55:48.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:48 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:48.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:48 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:48.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:48 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:48.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:48 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T02:55:48.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:48 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T02:55:48.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:48 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:55:48.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:48 smithi148 ceph-mon[35090]: Deploying daemon mds.f on smithi148 2024-09-12T02:55:48.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:48 smithi148 ceph-mon[35090]: mds.? [v2:172.21.15.146:6834/2540740420,v1:172.21.15.146:6835/2540740420] up:boot 2024-09-12T02:55:48.917 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:48 smithi148 ceph-mon[35090]: fsmap 5 up:standby 2024-09-12T02:55:48.917 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:48 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-12T02:55:49.191 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:55:49.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:48 smithi146 ceph-mon[35011]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:55:49.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:48 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:49.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:48 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:49.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:48 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:49.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:48 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T02:55:49.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:48 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T02:55:49.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:48 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:55:49.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:48 smithi146 ceph-mon[35011]: Deploying daemon mds.f on smithi148 2024-09-12T02:55:49.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:48 smithi146 ceph-mon[35011]: mds.? [v2:172.21.15.146:6834/2540740420,v1:172.21.15.146:6835/2540740420] up:boot 2024-09-12T02:55:49.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:48 smithi146 ceph-mon[35011]: fsmap 5 up:standby 2024-09-12T02:55:49.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:48 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-12T02:55:49.220 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:48 smithi005 ceph-mon[31197]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:55:49.220 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:48 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:49.220 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:48 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:49.221 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:48 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:49.221 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:48 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T02:55:49.221 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:48 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T02:55:49.221 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:48 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:55:49.221 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:48 smithi005 ceph-mon[31197]: Deploying daemon mds.f on smithi148 2024-09-12T02:55:49.221 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:48 smithi005 ceph-mon[31197]: mds.? [v2:172.21.15.146:6834/2540740420,v1:172.21.15.146:6835/2540740420] up:boot 2024-09-12T02:55:49.221 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:48 smithi005 ceph-mon[31197]: fsmap 5 up:standby 2024-09-12T02:55:49.221 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:48 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-12T02:55:49.465 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:55:49.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:49.898+0000 7fcc5db9f640 1 -- 172.21.15.5:0/3995043811 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc5810c170 msgr2=0x7fcc5810e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:49.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:49.898+0000 7fcc5db9f640 1 --2- 172.21.15.5:0/3995043811 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc5810c170 0x7fcc5810e610 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fcc50009ca0 tx=0x7fcc5002a950 comp rx=0 tx=0).stop 2024-09-12T02:55:49.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:49.898+0000 7fcc5db9f640 1 -- 172.21.15.5:0/3995043811 shutdown_connections 2024-09-12T02:55:49.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:49.898+0000 7fcc5db9f640 1 --2- 172.21.15.5:0/3995043811 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc5810c170 0x7fcc5810e610 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:49.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:49.898+0000 7fcc5db9f640 1 --2- 172.21.15.5:0/3995043811 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcc58103660 0x7fcc5810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:49.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:49.898+0000 7fcc5db9f640 1 --2- 172.21.15.5:0/3995043811 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc58102d40 0x7fcc58103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:49.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:49.898+0000 7fcc5db9f640 1 -- 172.21.15.5:0/3995043811 >> 172.21.15.5:0/3995043811 conn(0x7fcc580fc7a0 msgr2=0x7fcc580febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:55:49.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:49.898+0000 7fcc5db9f640 1 -- 172.21.15.5:0/3995043811 shutdown_connections 2024-09-12T02:55:49.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:49.899+0000 7fcc5db9f640 1 -- 172.21.15.5:0/3995043811 wait complete. 2024-09-12T02:55:49.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:49.899+0000 7fcc5db9f640 1 Processor -- start 2024-09-12T02:55:49.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:49.899+0000 7fcc5db9f640 1 -- start start 2024-09-12T02:55:49.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:49.900+0000 7fcc5db9f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc58102d40 0x7fcc58198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:49.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:49.900+0000 7fcc5db9f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc58103660 0x7fcc581989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:49.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:49.900+0000 7fcc5db9f640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcc5810c170 0x7fcc5819cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:49.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:49.900+0000 7fcc5db9f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcc58111fb0 con 0x7fcc58103660 2024-09-12T02:55:49.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:49.900+0000 7fcc5db9f640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fcc58111e30 con 0x7fcc5810c170 2024-09-12T02:55:49.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:49.900+0000 7fcc5db9f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fcc58112130 con 0x7fcc58102d40 2024-09-12T02:55:49.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:49.900+0000 7fcc56ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc58103660 0x7fcc581989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:49.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:49.900+0000 7fcc577fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc58102d40 0x7fcc58198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:49.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:49.900+0000 7fcc56ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc58103660 0x7fcc581989b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44894/0 (socket says 172.21.15.5:44894) 2024-09-12T02:55:49.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:49.900+0000 7fcc56ffd640 1 -- 172.21.15.5:0/1989996998 learned_addr learned my addr 172.21.15.5:0/1989996998 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:55:49.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:49.900+0000 7fcc57fff640 1 --2- 172.21.15.5:0/1989996998 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcc5810c170 0x7fcc5819cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:49.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:49.900+0000 7fcc56ffd640 1 -- 172.21.15.5:0/1989996998 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc58102d40 msgr2=0x7fcc58198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:49.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:49.900+0000 7fcc56ffd640 1 --2- 172.21.15.5:0/1989996998 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc58102d40 0x7fcc58198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:49.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:49.900+0000 7fcc56ffd640 1 -- 172.21.15.5:0/1989996998 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcc5810c170 msgr2=0x7fcc5819cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:49.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:49.900+0000 7fcc56ffd640 1 --2- 172.21.15.5:0/1989996998 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcc5810c170 0x7fcc5819cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:49.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:49.900+0000 7fcc56ffd640 1 -- 172.21.15.5:0/1989996998 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc5819d420 con 0x7fcc58103660 2024-09-12T02:55:49.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:49.900+0000 7fcc57fff640 1 --2- 172.21.15.5:0/1989996998 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcc5810c170 0x7fcc5819cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:55:49.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:49.901+0000 7fcc56ffd640 1 --2- 172.21.15.5:0/1989996998 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc58103660 0x7fcc581989b0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c00b3f0 tx=0x7fcc4c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:49.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:49.901+0000 7fcc54ff9640 1 -- 172.21.15.5:0/1989996998 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc4c015020 con 0x7fcc58103660 2024-09-12T02:55:49.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:49.901+0000 7fcc5db9f640 1 -- 172.21.15.5:0/1989996998 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc5819d710 con 0x7fcc58103660 2024-09-12T02:55:49.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:49.901+0000 7fcc54ff9640 1 -- 172.21.15.5:0/1989996998 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc4c011d10 con 0x7fcc58103660 2024-09-12T02:55:49.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:49.901+0000 7fcc54ff9640 1 -- 172.21.15.5:0/1989996998 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc4c00f2e0 con 0x7fcc58103660 2024-09-12T02:55:49.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:49.901+0000 7fcc5db9f640 1 -- 172.21.15.5:0/1989996998 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc581a4ff0 con 0x7fcc58103660 2024-09-12T02:55:49.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:49.902+0000 7fcc5db9f640 1 -- 172.21.15.5:0/1989996998 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc58103120 con 0x7fcc58103660 2024-09-12T02:55:49.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:49.903+0000 7fcc54ff9640 1 -- 172.21.15.5:0/1989996998 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fcc4c018020 con 0x7fcc58103660 2024-09-12T02:55:49.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:49.904+0000 7fcc54ff9640 1 --2- 172.21.15.5:0/1989996998 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fcc400a2b80 0x7fcc400a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:49.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:49.904+0000 7fcc577fe640 1 --2- 172.21.15.5:0/1989996998 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fcc400a2b80 0x7fcc400a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:49.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:49.904+0000 7fcc54ff9640 1 -- 172.21.15.5:0/1989996998 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7fcc4c0ce470 con 0x7fcc58103660 2024-09-12T02:55:49.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:49.904+0000 7fcc577fe640 1 --2- 172.21.15.5:0/1989996998 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fcc400a2b80 0x7fcc400a5040 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fcc4800bbe0 tx=0x7fcc4800b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:49.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:49.908+0000 7fcc54ff9640 1 -- 172.21.15.5:0/1989996998 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcc4c01f020 con 0x7fcc58103660 2024-09-12T02:55:49.916 INFO:journalctl@ceph.mds.f.smithi148.stdout:Sep 12 02:55:49 smithi148 systemd[1]: Starting Ceph mds.f for 618305f8-70b1-11ef-bcea-c7b262605968... 2024-09-12T02:55:50.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:50.120+0000 7fcc5db9f640 1 -- 172.21.15.5:0/1989996998 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcc581a5380 con 0x7fcc58103660 2024-09-12T02:55:50.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:50.120+0000 7fcc54ff9640 1 -- 172.21.15.5:0/1989996998 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4735 (secure 0 0 0) 0x7fcc4c09b7d0 con 0x7fcc58103660 2024-09-12T02:55:50.121 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-12T02:55:50.121 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":6,"btime":"2024-09-12T02:55:48:498110+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/4135232521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":4135232521},{"type":"v1","addr":"172.21.15.5:6835","nonce":4135232521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/4290221940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":4290221940},{"type":"v1","addr":"172.21.15.5:6837","nonce":4290221940}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/3686970137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3686970137},{"type":"v1","addr":"172.21.15.146:6833","nonce":3686970137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/568512209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":568512209},{"type":"v1","addr":"172.21.15.148:6833","nonce":568512209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/2540740420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2540740420},{"type":"v1","addr":"172.21.15.146:6835","nonce":2540740420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-09-12T02:55:50.121 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 6 2024-09-12T02:55:50.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:50.122+0000 7fcc5db9f640 1 -- 172.21.15.5:0/1989996998 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fcc400a2b80 msgr2=0x7fcc400a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:50.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:50.122+0000 7fcc5db9f640 1 --2- 172.21.15.5:0/1989996998 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fcc400a2b80 0x7fcc400a5040 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fcc4800bbe0 tx=0x7fcc4800b390 comp rx=0 tx=0).stop 2024-09-12T02:55:50.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:50.122+0000 7fcc5db9f640 1 -- 172.21.15.5:0/1989996998 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc58103660 msgr2=0x7fcc581989b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:50.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:50.122+0000 7fcc5db9f640 1 --2- 172.21.15.5:0/1989996998 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc58103660 0x7fcc581989b0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c00b3f0 tx=0x7fcc4c00b8b0 comp rx=0 tx=0).stop 2024-09-12T02:55:50.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:50.122+0000 7fcc5db9f640 1 -- 172.21.15.5:0/1989996998 shutdown_connections 2024-09-12T02:55:50.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:50.122+0000 7fcc5db9f640 1 --2- 172.21.15.5:0/1989996998 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fcc400a2b80 0x7fcc400a5040 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:50.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:50.122+0000 7fcc5db9f640 1 --2- 172.21.15.5:0/1989996998 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc58103660 0x7fcc581989b0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:50.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:50.122+0000 7fcc5db9f640 1 --2- 172.21.15.5:0/1989996998 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcc5810c170 0x7fcc5819cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:50.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:50.122+0000 7fcc5db9f640 1 --2- 172.21.15.5:0/1989996998 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcc58102d40 0x7fcc58198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:50.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:50.122+0000 7fcc5db9f640 1 -- 172.21.15.5:0/1989996998 >> 172.21.15.5:0/1989996998 conn(0x7fcc580fc7a0 msgr2=0x7fcc580fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:55:50.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:50.123+0000 7fcc5db9f640 1 -- 172.21.15.5:0/1989996998 shutdown_connections 2024-09-12T02:55:50.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:50.123+0000 7fcc5db9f640 1 -- 172.21.15.5:0/1989996998 wait complete. 2024-09-12T02:55:50.298 INFO:journalctl@ceph.mds.f.smithi148.stdout:Sep 12 02:55:49 smithi148 podman[64934]: 2024-09-12 02:55:49.883421585 +0000 UTC m=+0.016018164 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T02:55:50.298 INFO:journalctl@ceph.mds.f.smithi148.stdout:Sep 12 02:55:49 smithi148 podman[64934]: 2024-09-12 02:55:49.987524282 +0000 UTC m=+0.120120883 container create 3cb01b1b23a3bdaf7aee42a146972760fb89f5d0c4c1a0035de2bf7d949dc385 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-f, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-12T02:55:50.298 INFO:journalctl@ceph.mds.f.smithi148.stdout:Sep 12 02:55:50 smithi148 podman[64934]: 2024-09-12 02:55:50.198224396 +0000 UTC m=+0.330820954 container init 3cb01b1b23a3bdaf7aee42a146972760fb89f5d0c4c1a0035de2bf7d949dc385 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-f, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T02:55:50.298 INFO:journalctl@ceph.mds.f.smithi148.stdout:Sep 12 02:55:50 smithi148 podman[64934]: 2024-09-12 02:55:50.200901673 +0000 UTC m=+0.333498243 container start 3cb01b1b23a3bdaf7aee42a146972760fb89f5d0c4c1a0035de2bf7d949dc385 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-12T02:55:50.298 INFO:journalctl@ceph.mds.f.smithi148.stdout:Sep 12 02:55:50 smithi148 ceph-mds[64954]: -- 172.21.15.148:0/3508942274 <== mon.1 v2:172.21.15.148:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55587d0d1860 con 0x55587d0ae800 2024-09-12T02:55:50.298 INFO:journalctl@ceph.mds.f.smithi148.stdout:Sep 12 02:55:50 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-f[64950]: starting mds.f at 2024-09-12T02:55:50.298 INFO:journalctl@ceph.mds.f.smithi148.stdout:Sep 12 02:55:50 smithi148 bash[64934]: 3cb01b1b23a3bdaf7aee42a146972760fb89f5d0c4c1a0035de2bf7d949dc385 2024-09-12T02:55:50.665 INFO:journalctl@ceph.mds.f.smithi148.stdout:Sep 12 02:55:50 smithi148 systemd[1]: Started Ceph mds.f for 618305f8-70b1-11ef-bcea-c7b262605968. 2024-09-12T02:55:51.092 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:50 smithi148 ceph-mon[35090]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:55:51.092 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:50 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/1989996998' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:55:51.093 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:50 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:51.093 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:50 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:51.093 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:50 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:51.093 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:50 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T02:55:51.093 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:50 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T02:55:51.093 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:50 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:55:51.093 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:50 smithi148 ceph-mon[35090]: Deploying daemon mds.c on smithi148 2024-09-12T02:55:51.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:50 smithi146 ceph-mon[35011]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:55:51.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:50 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/1989996998' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:55:51.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:50 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:51.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:50 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:51.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:50 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:51.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:50 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T02:55:51.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:50 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T02:55:51.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:50 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:55:51.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:50 smithi146 ceph-mon[35011]: Deploying daemon mds.c on smithi148 2024-09-12T02:55:51.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:50 smithi005 ceph-mon[31197]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:55:51.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:50 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/1989996998' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:55:51.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:50 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:51.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:50 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:51.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:50 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:51.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:50 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T02:55:51.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:50 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T02:55:51.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:50 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:55:51.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:50 smithi005 ceph-mon[31197]: Deploying daemon mds.c on smithi148 2024-09-12T02:55:52.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:51 smithi148 ceph-mon[35090]: mds.? [v2:172.21.15.148:6834/4195086427,v1:172.21.15.148:6835/4195086427] up:boot 2024-09-12T02:55:52.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:51 smithi148 ceph-mon[35090]: fsmap 6 up:standby 2024-09-12T02:55:52.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:51 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T02:55:52.166 INFO:journalctl@ceph.mds.c.smithi148.stdout:Sep 12 02:55:52 smithi148 systemd[1]: Starting Ceph mds.c for 618305f8-70b1-11ef-bcea-c7b262605968... 2024-09-12T02:55:52.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:51 smithi146 ceph-mon[35011]: mds.? [v2:172.21.15.148:6834/4195086427,v1:172.21.15.148:6835/4195086427] up:boot 2024-09-12T02:55:52.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:51 smithi146 ceph-mon[35011]: fsmap 6 up:standby 2024-09-12T02:55:52.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:51 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T02:55:52.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:51 smithi005 ceph-mon[31197]: mds.? [v2:172.21.15.148:6834/4195086427,v1:172.21.15.148:6835/4195086427] up:boot 2024-09-12T02:55:52.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:51 smithi005 ceph-mon[31197]: fsmap 6 up:standby 2024-09-12T02:55:52.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:51 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T02:55:52.570 INFO:journalctl@ceph.mds.c.smithi148.stdout:Sep 12 02:55:52 smithi148 podman[65236]: 2024-09-12 02:55:52.206921608 +0000 UTC m=+0.016467864 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T02:55:52.570 INFO:journalctl@ceph.mds.c.smithi148.stdout:Sep 12 02:55:52 smithi148 podman[65236]: 2024-09-12 02:55:52.323435002 +0000 UTC m=+0.132981245 container create 3d621945b9f5be9ecb22d3441728918a2fa1feda51217d12a96cd2b4a18c9c77 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-c, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-12T02:55:52.570 INFO:journalctl@ceph.mds.c.smithi148.stdout:Sep 12 02:55:52 smithi148 podman[65236]: 2024-09-12 02:55:52.542592588 +0000 UTC m=+0.352138832 container init 3d621945b9f5be9ecb22d3441728918a2fa1feda51217d12a96cd2b4a18c9c77 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-c, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-12T02:55:52.570 INFO:journalctl@ceph.mds.c.smithi148.stdout:Sep 12 02:55:52 smithi148 podman[65236]: 2024-09-12 02:55:52.545313186 +0000 UTC m=+0.354859430 container start 3d621945b9f5be9ecb22d3441728918a2fa1feda51217d12a96cd2b4a18c9c77 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-c, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T02:55:52.887 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:55:52.899 INFO:journalctl@ceph.mds.c.smithi148.stdout:Sep 12 02:55:52 smithi148 ceph-mds[65255]: -- 172.21.15.148:0/3399152073 <== mon.1 v2:172.21.15.148:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560644eb7860 con 0x560644e94800 2024-09-12T02:55:52.899 INFO:journalctl@ceph.mds.c.smithi148.stdout:Sep 12 02:55:52 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-c[65251]: starting mds.c at 2024-09-12T02:55:52.899 INFO:journalctl@ceph.mds.c.smithi148.stdout:Sep 12 02:55:52 smithi148 bash[65236]: 3d621945b9f5be9ecb22d3441728918a2fa1feda51217d12a96cd2b4a18c9c77 2024-09-12T02:55:52.899 INFO:journalctl@ceph.mds.c.smithi148.stdout:Sep 12 02:55:52 smithi148 systemd[1]: Started Ceph mds.c for 618305f8-70b1-11ef-bcea-c7b262605968. 2024-09-12T02:55:53.160 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:55:53.160 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:52 smithi146 ceph-mon[35011]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:55:53.160 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:52 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:53.160 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:52 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:53.160 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:52 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:53.161 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:52 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T02:55:53.161 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:52 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T02:55:53.161 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:52 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:55:53.165 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:52 smithi148 ceph-mon[35090]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:55:53.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:52 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:53.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:52 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:53.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:52 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:53.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:52 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T02:55:53.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:52 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T02:55:53.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:52 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:55:53.282 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:52 smithi005 ceph-mon[31197]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:55:53.282 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:52 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:53.282 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:52 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:53.282 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:52 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:53.282 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:52 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T02:55:53.282 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:52 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T02:55:53.282 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:52 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:55:53.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:53.639+0000 7f6b2e0c9640 1 -- 172.21.15.5:0/2785072702 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b28101de0 msgr2=0x7f6b281021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:53.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:53.639+0000 7f6b2e0c9640 1 --2- 172.21.15.5:0/2785072702 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b28101de0 0x7f6b281021c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6b18009ca0 tx=0x7f6b1802a8c0 comp rx=0 tx=0).stop 2024-09-12T02:55:53.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:53.639+0000 7f6b2e0c9640 1 -- 172.21.15.5:0/2785072702 shutdown_connections 2024-09-12T02:55:53.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:53.639+0000 7f6b2e0c9640 1 --2- 172.21.15.5:0/2785072702 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6b28106f10 0x7f6b2810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:53.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:53.639+0000 7f6b2e0c9640 1 --2- 172.21.15.5:0/2785072702 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b28102790 0x7f6b281067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:53.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:53.639+0000 7f6b2e0c9640 1 --2- 172.21.15.5:0/2785072702 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b28101de0 0x7f6b281021c0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:53.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:53.639+0000 7f6b2e0c9640 1 -- 172.21.15.5:0/2785072702 >> 172.21.15.5:0/2785072702 conn(0x7f6b280fd8c0 msgr2=0x7f6b280ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:55:53.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:53.639+0000 7f6b2e0c9640 1 -- 172.21.15.5:0/2785072702 shutdown_connections 2024-09-12T02:55:53.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:53.639+0000 7f6b2e0c9640 1 -- 172.21.15.5:0/2785072702 wait complete. 2024-09-12T02:55:53.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:53.639+0000 7f6b2e0c9640 1 Processor -- start 2024-09-12T02:55:53.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:53.640+0000 7f6b2e0c9640 1 -- start start 2024-09-12T02:55:53.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:53.640+0000 7f6b2e0c9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b28101de0 0x7f6b280744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:53.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:53.640+0000 7f6b2e0c9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b28102790 0x7f6b28074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:53.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:53.640+0000 7f6b2e0c9640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6b28106f10 0x7f6b28074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:53.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:53.640+0000 7f6b2e0c9640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6b28111f70 con 0x7f6b28102790 2024-09-12T02:55:53.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:53.640+0000 7f6b2e0c9640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f6b28111df0 con 0x7f6b28106f10 2024-09-12T02:55:53.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:53.640+0000 7f6b2e0c9640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f6b281120f0 con 0x7f6b28101de0 2024-09-12T02:55:53.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:53.640+0000 7f6b26ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b28102790 0x7f6b28074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:53.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:53.640+0000 7f6b26ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b28102790 0x7f6b28074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:32898/0 (socket says 172.21.15.5:32898) 2024-09-12T02:55:53.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:53.640+0000 7f6b26ffd640 1 -- 172.21.15.5:0/4289001441 learned_addr learned my addr 172.21.15.5:0/4289001441 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:55:53.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:53.640+0000 7f6b27fff640 1 --2- 172.21.15.5:0/4289001441 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6b28106f10 0x7f6b28074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:53.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:53.640+0000 7f6b277fe640 1 --2- 172.21.15.5:0/4289001441 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b28101de0 0x7f6b280744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:53.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:53.640+0000 7f6b26ffd640 1 -- 172.21.15.5:0/4289001441 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b28101de0 msgr2=0x7f6b280744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:53.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:53.640+0000 7f6b26ffd640 1 --2- 172.21.15.5:0/4289001441 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b28101de0 0x7f6b280744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:53.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:53.640+0000 7f6b26ffd640 1 -- 172.21.15.5:0/4289001441 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6b28106f10 msgr2=0x7f6b28074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:53.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:53.640+0000 7f6b26ffd640 1 --2- 172.21.15.5:0/4289001441 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6b28106f10 0x7f6b28074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:53.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:53.640+0000 7f6b26ffd640 1 -- 172.21.15.5:0/4289001441 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b281a6970 con 0x7f6b28102790 2024-09-12T02:55:53.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:53.640+0000 7f6b277fe640 1 --2- 172.21.15.5:0/4289001441 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b28101de0 0x7f6b280744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:55:53.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:53.641+0000 7f6b26ffd640 1 --2- 172.21.15.5:0/4289001441 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b28102790 0x7f6b28074a30 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c00c970 tx=0x7f6b1c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:53.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:53.641+0000 7f6b24ff9640 1 -- 172.21.15.5:0/4289001441 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b1c007bb0 con 0x7f6b28102790 2024-09-12T02:55:53.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:53.641+0000 7f6b2e0c9640 1 -- 172.21.15.5:0/4289001441 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b281a6c60 con 0x7f6b28102790 2024-09-12T02:55:53.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:53.641+0000 7f6b24ff9640 1 -- 172.21.15.5:0/4289001441 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b1c007d50 con 0x7f6b28102790 2024-09-12T02:55:53.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:53.641+0000 7f6b24ff9640 1 -- 172.21.15.5:0/4289001441 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b1c00f450 con 0x7f6b28102790 2024-09-12T02:55:53.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:53.641+0000 7f6b2e0c9640 1 -- 172.21.15.5:0/4289001441 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b281a71a0 con 0x7f6b28102790 2024-09-12T02:55:53.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:53.642+0000 7f6b2e0c9640 1 -- 172.21.15.5:0/4289001441 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b281021c0 con 0x7f6b28102790 2024-09-12T02:55:53.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:53.643+0000 7f6b24ff9640 1 -- 172.21.15.5:0/4289001441 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f6b1c004fe0 con 0x7f6b28102790 2024-09-12T02:55:53.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:53.644+0000 7f6b24ff9640 1 --2- 172.21.15.5:0/4289001441 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f6b100a2b80 0x7f6b100a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:53.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:53.644+0000 7f6b277fe640 1 --2- 172.21.15.5:0/4289001441 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f6b100a2b80 0x7f6b100a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:53.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:53.644+0000 7f6b24ff9640 1 -- 172.21.15.5:0/4289001441 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f6b1c0ced20 con 0x7f6b28102790 2024-09-12T02:55:53.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:53.644+0000 7f6b277fe640 1 --2- 172.21.15.5:0/4289001441 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f6b100a2b80 0x7f6b100a5040 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f6b18006010 tx=0x7f6b180093f0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:53.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:53.647+0000 7f6b24ff9640 1 -- 172.21.15.5:0/4289001441 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6b1c014030 con 0x7f6b28102790 2024-09-12T02:55:53.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:53.858+0000 7f6b2e0c9640 1 -- 172.21.15.5:0/4289001441 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6b2810a0f0 con 0x7f6b28102790 2024-09-12T02:55:53.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:53.859+0000 7f6b24ff9640 1 -- 172.21.15.5:0/4289001441 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6391 (secure 0 0 0) 0x7f6b1c09c120 con 0x7f6b28102790 2024-09-12T02:55:53.859 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-12T02:55:53.859 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":8,"btime":"2024-09-12T02:55:52:898316+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/4135232521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":4135232521},{"type":"v1","addr":"172.21.15.5:6835","nonce":4135232521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/4290221940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":4290221940},{"type":"v1","addr":"172.21.15.5:6837","nonce":4290221940}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/3686970137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3686970137},{"type":"v1","addr":"172.21.15.146:6833","nonce":3686970137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/568512209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":568512209},{"type":"v1","addr":"172.21.15.148:6833","nonce":568512209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/2540740420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2540740420},{"type":"v1","addr":"172.21.15.146:6835","nonce":2540740420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24481,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/4195086427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":4195086427},{"type":"v1","addr":"172.21.15.148:6835","nonce":4195086427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24487,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/2606873922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2606873922},{"type":"v1","addr":"172.21.15.148:6837","nonce":2606873922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-09-12T02:55:53.860 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 8 2024-09-12T02:55:53.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:53.861+0000 7f6b2e0c9640 1 -- 172.21.15.5:0/4289001441 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f6b100a2b80 msgr2=0x7f6b100a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:53.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:53.861+0000 7f6b2e0c9640 1 --2- 172.21.15.5:0/4289001441 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f6b100a2b80 0x7f6b100a5040 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f6b18006010 tx=0x7f6b180093f0 comp rx=0 tx=0).stop 2024-09-12T02:55:53.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:53.861+0000 7f6b2e0c9640 1 -- 172.21.15.5:0/4289001441 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b28102790 msgr2=0x7f6b28074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:53.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:53.861+0000 7f6b2e0c9640 1 --2- 172.21.15.5:0/4289001441 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b28102790 0x7f6b28074a30 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c00c970 tx=0x7f6b1c00ce30 comp rx=0 tx=0).stop 2024-09-12T02:55:53.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:53.861+0000 7f6b2e0c9640 1 -- 172.21.15.5:0/4289001441 shutdown_connections 2024-09-12T02:55:53.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:53.861+0000 7f6b2e0c9640 1 --2- 172.21.15.5:0/4289001441 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f6b100a2b80 0x7f6b100a5040 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:53.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:53.861+0000 7f6b2e0c9640 1 --2- 172.21.15.5:0/4289001441 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b28102790 0x7f6b28074a30 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:53.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:53.861+0000 7f6b2e0c9640 1 --2- 172.21.15.5:0/4289001441 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6b28106f10 0x7f6b28074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:53.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:53.861+0000 7f6b2e0c9640 1 --2- 172.21.15.5:0/4289001441 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6b28101de0 0x7f6b280744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:53.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:53.861+0000 7f6b2e0c9640 1 -- 172.21.15.5:0/4289001441 >> 172.21.15.5:0/4289001441 conn(0x7f6b280fd8c0 msgr2=0x7f6b280fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:55:53.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:53.861+0000 7f6b2e0c9640 1 -- 172.21.15.5:0/4289001441 shutdown_connections 2024-09-12T02:55:53.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:53.861+0000 7f6b2e0c9640 1 -- 172.21.15.5:0/4289001441 wait complete. 2024-09-12T02:55:54.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:53 smithi005 ceph-mon[31197]: Deploying daemon mds.h on smithi146 2024-09-12T02:55:54.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:53 smithi005 ceph-mon[31197]: mds.? [v2:172.21.15.148:6836/2606873922,v1:172.21.15.148:6837/2606873922] up:boot 2024-09-12T02:55:54.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:53 smithi005 ceph-mon[31197]: fsmap 7 up:standby 2024-09-12T02:55:54.156 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:53 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-12T02:55:54.157 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:53 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/4289001441' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:55:54.163 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:53 smithi146 ceph-mon[35011]: Deploying daemon mds.h on smithi146 2024-09-12T02:55:54.163 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:53 smithi146 ceph-mon[35011]: mds.? [v2:172.21.15.148:6836/2606873922,v1:172.21.15.148:6837/2606873922] up:boot 2024-09-12T02:55:54.164 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:53 smithi146 ceph-mon[35011]: fsmap 7 up:standby 2024-09-12T02:55:54.164 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:53 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-12T02:55:54.164 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:53 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/4289001441' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:55:54.165 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:53 smithi148 ceph-mon[35090]: Deploying daemon mds.h on smithi146 2024-09-12T02:55:54.165 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:53 smithi148 ceph-mon[35090]: mds.? [v2:172.21.15.148:6836/2606873922,v1:172.21.15.148:6837/2606873922] up:boot 2024-09-12T02:55:54.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:53 smithi148 ceph-mon[35090]: fsmap 7 up:standby 2024-09-12T02:55:54.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:53 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-12T02:55:54.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:53 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/4289001441' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:55:54.911 INFO:journalctl@ceph.mds.h.smithi146.stdout:Sep 12 02:55:54 smithi146 systemd[1]: Starting Ceph mds.h for 618305f8-70b1-11ef-bcea-c7b262605968... 2024-09-12T02:55:55.165 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:54 smithi148 ceph-mon[35090]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:55:55.180 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:54 smithi005 ceph-mon[31197]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:55:55.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:54 smithi146 ceph-mon[35011]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:55:55.206 INFO:journalctl@ceph.mds.h.smithi146.stdout:Sep 12 02:55:54 smithi146 podman[65103]: 2024-09-12 02:55:54.823128336 +0000 UTC m=+0.016597170 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T02:55:55.206 INFO:journalctl@ceph.mds.h.smithi146.stdout:Sep 12 02:55:54 smithi146 podman[65103]: 2024-09-12 02:55:54.927211283 +0000 UTC m=+0.120680102 container create f42ddd7f5459b111551df6b5ece3a3733b65634a19260ea36e67af718728a8c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-h, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T02:55:55.206 INFO:journalctl@ceph.mds.h.smithi146.stdout:Sep 12 02:55:55 smithi146 podman[65103]: 2024-09-12 02:55:55.129853144 +0000 UTC m=+0.323321966 container init f42ddd7f5459b111551df6b5ece3a3733b65634a19260ea36e67af718728a8c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-h, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True) 2024-09-12T02:55:55.207 INFO:journalctl@ceph.mds.h.smithi146.stdout:Sep 12 02:55:55 smithi146 podman[65103]: 2024-09-12 02:55:55.132621648 +0000 UTC m=+0.326090482 container start f42ddd7f5459b111551df6b5ece3a3733b65634a19260ea36e67af718728a8c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-h, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-12T02:55:55.207 INFO:journalctl@ceph.mds.h.smithi146.stdout:Sep 12 02:55:55 smithi146 ceph-mds[65122]: -- 172.21.15.146:0/677177899 <== mon.2 v2:172.21.15.146:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56199c297860 con 0x56199c274800 2024-09-12T02:55:55.207 INFO:journalctl@ceph.mds.h.smithi146.stdout:Sep 12 02:55:55 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-h[65118]: starting mds.h at 2024-09-12T02:55:55.706 INFO:journalctl@ceph.mds.h.smithi146.stdout:Sep 12 02:55:55 smithi146 bash[65103]: f42ddd7f5459b111551df6b5ece3a3733b65634a19260ea36e67af718728a8c8 2024-09-12T02:55:55.706 INFO:journalctl@ceph.mds.h.smithi146.stdout:Sep 12 02:55:55 smithi146 systemd[1]: Started Ceph mds.h for 618305f8-70b1-11ef-bcea-c7b262605968. 2024-09-12T02:55:56.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:56 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:56.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:56 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:56.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:56 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:56.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:56 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T02:55:56.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:56 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T02:55:56.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:56 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:55:56.562 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:56 smithi005 ceph-mon[31197]: Deploying daemon mds.i on smithi148 2024-09-12T02:55:56.562 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:56 smithi005 ceph-mon[31197]: mds.? [v2:172.21.15.146:6836/1673370937,v1:172.21.15.146:6837/1673370937] up:boot 2024-09-12T02:55:56.562 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:56 smithi005 ceph-mon[31197]: fsmap 8 up:standby 2024-09-12T02:55:56.562 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:56 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-12T02:55:56.601 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:55:56.657 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:56 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:56.657 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:56 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:56.657 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:56 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:56.657 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:56 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T02:55:56.657 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:56 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T02:55:56.657 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:56 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:55:56.657 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:56 smithi148 ceph-mon[35090]: Deploying daemon mds.i on smithi148 2024-09-12T02:55:56.657 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:56 smithi148 ceph-mon[35090]: mds.? [v2:172.21.15.146:6836/1673370937,v1:172.21.15.146:6837/1673370937] up:boot 2024-09-12T02:55:56.658 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:56 smithi148 ceph-mon[35090]: fsmap 8 up:standby 2024-09-12T02:55:56.658 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:56 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-12T02:55:56.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:56 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:56.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:56 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:56.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:56 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:56.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:56 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T02:55:56.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:56 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T02:55:56.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:56 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:55:56.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:56 smithi146 ceph-mon[35011]: Deploying daemon mds.i on smithi148 2024-09-12T02:55:56.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:56 smithi146 ceph-mon[35011]: mds.? [v2:172.21.15.146:6836/1673370937,v1:172.21.15.146:6837/1673370937] up:boot 2024-09-12T02:55:56.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:56 smithi146 ceph-mon[35011]: fsmap 8 up:standby 2024-09-12T02:55:56.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:56 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-12T02:55:56.872 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:55:57.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:57.281+0000 7f3a6b333640 1 -- 172.21.15.5:0/3038761838 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a6410e480 msgr2=0x7f3a64110870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:57.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:57.281+0000 7f3a6b333640 1 --2- 172.21.15.5:0/3038761838 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a6410e480 0x7f3a64110870 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c009ca0 tx=0x7f3a5c02a8c0 comp rx=0 tx=0).stop 2024-09-12T02:55:57.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:57.281+0000 7f3a6b333640 1 -- 172.21.15.5:0/3038761838 shutdown_connections 2024-09-12T02:55:57.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:57.281+0000 7f3a6b333640 1 --2- 172.21.15.5:0/3038761838 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a6410e480 0x7f3a64110870 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:57.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:57.281+0000 7f3a6b333640 1 --2- 172.21.15.5:0/3038761838 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3a64101280 0x7f3a6410df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:57.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:57.281+0000 7f3a6b333640 1 --2- 172.21.15.5:0/3038761838 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3a641008d0 0x7f3a64100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:57.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:57.281+0000 7f3a6b333640 1 -- 172.21.15.5:0/3038761838 >> 172.21.15.5:0/3038761838 conn(0x7f3a640fc7a0 msgr2=0x7f3a640febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:55:57.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:57.282+0000 7f3a6b333640 1 -- 172.21.15.5:0/3038761838 shutdown_connections 2024-09-12T02:55:57.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:57.282+0000 7f3a6b333640 1 -- 172.21.15.5:0/3038761838 wait complete. 2024-09-12T02:55:57.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:57.282+0000 7f3a6b333640 1 Processor -- start 2024-09-12T02:55:57.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:57.282+0000 7f3a6b333640 1 -- start start 2024-09-12T02:55:57.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:57.282+0000 7f3a6b333640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3a641008d0 0x7f3a6419a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:57.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:57.282+0000 7f3a6b333640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a64101280 0x7f3a6419ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:57.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:57.282+0000 7f3a6b333640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3a6410e480 0x7f3a6419ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:57.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:57.282+0000 7f3a6b333640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3a64114160 con 0x7f3a64101280 2024-09-12T02:55:57.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:57.282+0000 7f3a6b333640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f3a64113fe0 con 0x7f3a6410e480 2024-09-12T02:55:57.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:57.282+0000 7f3a6b333640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3a641142e0 con 0x7f3a641008d0 2024-09-12T02:55:57.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:57.283+0000 7f3a688ce640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a64101280 0x7f3a6419ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:57.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:57.283+0000 7f3a688ce640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a64101280 0x7f3a6419ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:32918/0 (socket says 172.21.15.5:32918) 2024-09-12T02:55:57.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:57.283+0000 7f3a688ce640 1 -- 172.21.15.5:0/3715720566 learned_addr learned my addr 172.21.15.5:0/3715720566 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:55:57.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:57.283+0000 7f3a690cf640 1 --2- 172.21.15.5:0/3715720566 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3a641008d0 0x7f3a6419a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:57.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:57.283+0000 7f3a698d0640 1 --2- 172.21.15.5:0/3715720566 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3a6410e480 0x7f3a6419ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:57.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:57.283+0000 7f3a688ce640 1 -- 172.21.15.5:0/3715720566 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3a641008d0 msgr2=0x7f3a6419a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:57.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:57.283+0000 7f3a688ce640 1 --2- 172.21.15.5:0/3715720566 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3a641008d0 0x7f3a6419a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:57.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:57.283+0000 7f3a688ce640 1 -- 172.21.15.5:0/3715720566 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3a6410e480 msgr2=0x7f3a6419ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:57.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:57.283+0000 7f3a688ce640 1 --2- 172.21.15.5:0/3715720566 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3a6410e480 0x7f3a6419ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:57.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:57.283+0000 7f3a688ce640 1 -- 172.21.15.5:0/3715720566 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a6419f5f0 con 0x7f3a64101280 2024-09-12T02:55:57.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:57.283+0000 7f3a698d0640 1 --2- 172.21.15.5:0/3715720566 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3a6410e480 0x7f3a6419ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:55:57.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:57.283+0000 7f3a688ce640 1 --2- 172.21.15.5:0/3715720566 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a64101280 0x7f3a6419ab80 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f3a5400d950 tx=0x7f3a5400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:57.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:57.283+0000 7f3a5a7fc640 1 -- 172.21.15.5:0/3715720566 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a54017070 con 0x7f3a64101280 2024-09-12T02:55:57.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:57.283+0000 7f3a5a7fc640 1 -- 172.21.15.5:0/3715720566 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a54009c30 con 0x7f3a64101280 2024-09-12T02:55:57.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:57.283+0000 7f3a6b333640 1 -- 172.21.15.5:0/3715720566 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a6419f8e0 con 0x7f3a64101280 2024-09-12T02:55:57.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:57.283+0000 7f3a5a7fc640 1 -- 172.21.15.5:0/3715720566 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a5400f6b0 con 0x7f3a64101280 2024-09-12T02:55:57.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:57.283+0000 7f3a6b333640 1 -- 172.21.15.5:0/3715720566 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a641a71c0 con 0x7f3a64101280 2024-09-12T02:55:57.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:57.284+0000 7f3a6b333640 1 -- 172.21.15.5:0/3715720566 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a64100cb0 con 0x7f3a64101280 2024-09-12T02:55:57.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:57.285+0000 7f3a5a7fc640 1 -- 172.21.15.5:0/3715720566 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f3a540109d0 con 0x7f3a64101280 2024-09-12T02:55:57.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:57.286+0000 7f3a5a7fc640 1 --2- 172.21.15.5:0/3715720566 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f3a500a2c00 0x7f3a500a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:55:57.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:57.287+0000 7f3a690cf640 1 --2- 172.21.15.5:0/3715720566 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f3a500a2c00 0x7f3a500a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:55:57.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:57.287+0000 7f3a5a7fc640 1 -- 172.21.15.5:0/3715720566 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f3a540ce6f0 con 0x7f3a64101280 2024-09-12T02:55:57.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:57.287+0000 7f3a690cf640 1 --2- 172.21.15.5:0/3715720566 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f3a500a2c00 0x7f3a500a50c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f3a600058c0 tx=0x7f3a6000a2d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:55:57.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:57.291+0000 7f3a5a7fc640 1 -- 172.21.15.5:0/3715720566 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3a54009de0 con 0x7f3a64101280 2024-09-12T02:55:57.304 INFO:journalctl@ceph.mds.i.smithi148.stdout:Sep 12 02:55:57 smithi148 systemd[1]: Starting Ceph mds.i for 618305f8-70b1-11ef-bcea-c7b262605968... 2024-09-12T02:55:57.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:57.505+0000 7f3a6b333640 1 -- 172.21.15.5:0/3715720566 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3a64109610 con 0x7f3a64101280 2024-09-12T02:55:57.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:57.505+0000 7f3a5a7fc640 1 -- 172.21.15.5:0/3715720566 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7219 (secure 0 0 0) 0x7f3a5409baf0 con 0x7f3a64101280 2024-09-12T02:55:57.506 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-12T02:55:57.506 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":9,"btime":"2024-09-12T02:55:55:910491+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/4135232521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":4135232521},{"type":"v1","addr":"172.21.15.5:6835","nonce":4135232521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/4290221940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":4290221940},{"type":"v1","addr":"172.21.15.5:6837","nonce":4290221940}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/3686970137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3686970137},{"type":"v1","addr":"172.21.15.146:6833","nonce":3686970137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/568512209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":568512209},{"type":"v1","addr":"172.21.15.148:6833","nonce":568512209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/2540740420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2540740420},{"type":"v1","addr":"172.21.15.146:6835","nonce":2540740420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24458,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/1673370937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":1673370937},{"type":"v1","addr":"172.21.15.146:6837","nonce":1673370937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24481,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/4195086427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":4195086427},{"type":"v1","addr":"172.21.15.148:6835","nonce":4195086427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24487,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/2606873922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2606873922},{"type":"v1","addr":"172.21.15.148:6837","nonce":2606873922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-09-12T02:55:57.506 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 9 2024-09-12T02:55:57.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:57.507+0000 7f3a6b333640 1 -- 172.21.15.5:0/3715720566 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f3a500a2c00 msgr2=0x7f3a500a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:57.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:57.507+0000 7f3a6b333640 1 --2- 172.21.15.5:0/3715720566 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f3a500a2c00 0x7f3a500a50c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f3a600058c0 tx=0x7f3a6000a2d0 comp rx=0 tx=0).stop 2024-09-12T02:55:57.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:57.507+0000 7f3a6b333640 1 -- 172.21.15.5:0/3715720566 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a64101280 msgr2=0x7f3a6419ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:55:57.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:57.507+0000 7f3a6b333640 1 --2- 172.21.15.5:0/3715720566 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a64101280 0x7f3a6419ab80 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f3a5400d950 tx=0x7f3a5400de10 comp rx=0 tx=0).stop 2024-09-12T02:55:57.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:57.507+0000 7f3a6b333640 1 -- 172.21.15.5:0/3715720566 shutdown_connections 2024-09-12T02:55:57.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:57.507+0000 7f3a6b333640 1 --2- 172.21.15.5:0/3715720566 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f3a500a2c00 0x7f3a500a50c0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:57.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:57.507+0000 7f3a6b333640 1 --2- 172.21.15.5:0/3715720566 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a64101280 0x7f3a6419ab80 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:57.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:57.507+0000 7f3a6b333640 1 --2- 172.21.15.5:0/3715720566 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3a6410e480 0x7f3a6419ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:57.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:57.507+0000 7f3a6b333640 1 --2- 172.21.15.5:0/3715720566 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3a641008d0 0x7f3a6419a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:55:57.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:57.507+0000 7f3a6b333640 1 -- 172.21.15.5:0/3715720566 >> 172.21.15.5:0/3715720566 conn(0x7f3a640fc7a0 msgr2=0x7f3a6410f830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:55:57.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:57.507+0000 7f3a6b333640 1 -- 172.21.15.5:0/3715720566 shutdown_connections 2024-09-12T02:55:57.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:55:57.507+0000 7f3a6b333640 1 -- 172.21.15.5:0/3715720566 wait complete. 2024-09-12T02:55:57.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:57 smithi005 ceph-mon[31197]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:55:57.636 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:57 smithi148 ceph-mon[35090]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:55:57.636 INFO:journalctl@ceph.mds.i.smithi148.stdout:Sep 12 02:55:57 smithi148 podman[65536]: 2024-09-12 02:55:57.251763105 +0000 UTC m=+0.016198315 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T02:55:57.637 INFO:journalctl@ceph.mds.i.smithi148.stdout:Sep 12 02:55:57 smithi148 podman[65536]: 2024-09-12 02:55:57.384402399 +0000 UTC m=+0.148837589 container create 6c536fc6aff28c9b9923d921b3efad4e43b4a170a735b3b1221c884fcb39c0db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-i, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS) 2024-09-12T02:55:57.637 INFO:journalctl@ceph.mds.i.smithi148.stdout:Sep 12 02:55:57 smithi148 podman[65536]: 2024-09-12 02:55:57.636948874 +0000 UTC m=+0.401384064 container init 6c536fc6aff28c9b9923d921b3efad4e43b4a170a735b3b1221c884fcb39c0db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-i, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-09-12T02:55:57.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:57 smithi146 ceph-mon[35011]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:55:57.916 INFO:journalctl@ceph.mds.i.smithi148.stdout:Sep 12 02:55:57 smithi148 podman[65536]: 2024-09-12 02:55:57.639992003 +0000 UTC m=+0.404427193 container start 6c536fc6aff28c9b9923d921b3efad4e43b4a170a735b3b1221c884fcb39c0db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-i, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2) 2024-09-12T02:55:57.916 INFO:journalctl@ceph.mds.i.smithi148.stdout:Sep 12 02:55:57 smithi148 ceph-mds[65556]: -- 172.21.15.148:0/2612320065 <== mon.1 v2:172.21.15.148:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c06e1f1860 con 0x55c06e1ce800 2024-09-12T02:55:57.916 INFO:journalctl@ceph.mds.i.smithi148.stdout:Sep 12 02:55:57 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-i[65552]: starting mds.i at 2024-09-12T02:55:57.916 INFO:journalctl@ceph.mds.i.smithi148.stdout:Sep 12 02:55:57 smithi148 bash[65536]: 6c536fc6aff28c9b9923d921b3efad4e43b4a170a735b3b1221c884fcb39c0db 2024-09-12T02:55:57.916 INFO:journalctl@ceph.mds.i.smithi148.stdout:Sep 12 02:55:57 smithi148 systemd[1]: Started Ceph mds.i for 618305f8-70b1-11ef-bcea-c7b262605968. 2024-09-12T02:55:58.623 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:58 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/3715720566' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:55:58.623 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:58 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:58.623 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:58 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:58.624 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:58 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:58.624 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:58 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T02:55:58.624 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:58 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T02:55:58.624 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:58 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:55:58.665 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:58 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/3715720566' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:55:58.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:58 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:58.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:58 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:58.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:58 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:58.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:58 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T02:55:58.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:58 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T02:55:58.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:58 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:55:58.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:58 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/3715720566' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:55:58.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:58 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:58.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:58 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:58.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:58 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:55:58.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:58 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T02:55:58.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:58 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T02:55:58.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:58 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:55:59.665 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:59 smithi148 ceph-mon[35090]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:55:59.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:59 smithi148 ceph-mon[35090]: Deploying daemon mds.g on smithi005 2024-09-12T02:55:59.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:59 smithi148 ceph-mon[35090]: mds.? [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] up:boot 2024-09-12T02:55:59.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:59 smithi148 ceph-mon[35090]: fsmap 9 up:standby 2024-09-12T02:55:59.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:55:59 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-12T02:55:59.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:59 smithi146 ceph-mon[35011]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:55:59.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:59 smithi146 ceph-mon[35011]: Deploying daemon mds.g on smithi005 2024-09-12T02:55:59.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:59 smithi146 ceph-mon[35011]: mds.? [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] up:boot 2024-09-12T02:55:59.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:59 smithi146 ceph-mon[35011]: fsmap 9 up:standby 2024-09-12T02:55:59.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:55:59 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-12T02:55:59.724 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:59 smithi005 ceph-mon[31197]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:55:59.724 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:59 smithi005 ceph-mon[31197]: Deploying daemon mds.g on smithi005 2024-09-12T02:55:59.724 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:59 smithi005 ceph-mon[31197]: mds.? [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] up:boot 2024-09-12T02:55:59.724 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:59 smithi005 ceph-mon[31197]: fsmap 9 up:standby 2024-09-12T02:55:59.725 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:55:59 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-12T02:55:59.996 INFO:journalctl@ceph.mds.g.smithi005.stdout:Sep 12 02:55:59 smithi005 podman[69545]: 2024-09-12 02:55:59.996301893 +0000 UTC m=+0.170166695 container create f37068bb74e3ae7bfbbd6738d4d33707d2ad5f23fa2da53d9af3a8ebd4f5d212 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-g, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, ceph=True, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-12T02:56:00.133 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:56:00.421 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:56:00.551 INFO:journalctl@ceph.mds.g.smithi005.stdout:Sep 12 02:56:00 smithi005 podman[69545]: 2024-09-12 02:56:00.248966904 +0000 UTC m=+0.422831661 container init f37068bb74e3ae7bfbbd6738d4d33707d2ad5f23fa2da53d9af3a8ebd4f5d212 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-g, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T02:56:00.551 INFO:journalctl@ceph.mds.g.smithi005.stdout:Sep 12 02:56:00 smithi005 podman[69545]: 2024-09-12 02:56:00.252548946 +0000 UTC m=+0.426413711 container start f37068bb74e3ae7bfbbd6738d4d33707d2ad5f23fa2da53d9af3a8ebd4f5d212 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-g, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True) 2024-09-12T02:56:00.551 INFO:journalctl@ceph.mds.g.smithi005.stdout:Sep 12 02:56:00 smithi005 ceph-mds[69583]: -- 172.21.15.5:0/2352323936 <== mon.0 v2:172.21.15.5:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f5061e1860 con 0x55f5061be800 2024-09-12T02:56:00.551 INFO:journalctl@ceph.mds.g.smithi005.stdout:Sep 12 02:56:00 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-g[69561]: starting mds.g at 2024-09-12T02:56:00.551 INFO:journalctl@ceph.mds.g.smithi005.stdout:Sep 12 02:56:00 smithi005 bash[69545]: f37068bb74e3ae7bfbbd6738d4d33707d2ad5f23fa2da53d9af3a8ebd4f5d212 2024-09-12T02:56:00.551 INFO:journalctl@ceph.mds.g.smithi005.stdout:Sep 12 02:56:00 smithi005 systemd[1]: Started Ceph mds.g for 618305f8-70b1-11ef-bcea-c7b262605968. 2024-09-12T02:56:00.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:00.884+0000 7f7ba5835640 1 -- 172.21.15.5:0/2735403540 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ba0106a80 msgr2=0x7f7ba0106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:00.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:00.884+0000 7f7ba5835640 1 --2- 172.21.15.5:0/2735403540 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ba0106a80 0x7f7ba0106e60 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f7b90009930 tx=0x7f7b9002a420 comp rx=0 tx=0).stop 2024-09-12T02:56:00.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:00.884+0000 7f7ba5835640 1 -- 172.21.15.5:0/2735403540 shutdown_connections 2024-09-12T02:56:00.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:00.884+0000 7f7ba5835640 1 --2- 172.21.15.5:0/2735403540 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7ba0101420 0x7f7ba010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:00.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:00.884+0000 7f7ba5835640 1 --2- 172.21.15.5:0/2735403540 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ba0100a80 0x7f7ba0100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:00.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:00.884+0000 7f7ba5835640 1 --2- 172.21.15.5:0/2735403540 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ba0106a80 0x7f7ba0106e60 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:00.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:00.884+0000 7f7ba5835640 1 -- 172.21.15.5:0/2735403540 >> 172.21.15.5:0/2735403540 conn(0x7f7ba00fc7a0 msgr2=0x7f7ba00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:00.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:00.884+0000 7f7ba5835640 1 -- 172.21.15.5:0/2735403540 shutdown_connections 2024-09-12T02:56:00.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:00.884+0000 7f7ba5835640 1 -- 172.21.15.5:0/2735403540 wait complete. 2024-09-12T02:56:00.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:00.885+0000 7f7ba5835640 1 Processor -- start 2024-09-12T02:56:00.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:00.885+0000 7f7ba5835640 1 -- start start 2024-09-12T02:56:00.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:00.885+0000 7f7ba5835640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ba0100a80 0x7f7ba01984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:00.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:00.885+0000 7f7ba5835640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ba0101420 0x7f7ba0198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:00.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:00.885+0000 7f7ba5835640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7ba0106a80 0x7f7ba019cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:00.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:00.885+0000 7f7ba5835640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7ba0110ec0 con 0x7f7ba0100a80 2024-09-12T02:56:00.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:00.885+0000 7f7ba5835640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f7ba0110d40 con 0x7f7ba0106a80 2024-09-12T02:56:00.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:00.885+0000 7f7ba5835640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f7ba0111040 con 0x7f7ba0101420 2024-09-12T02:56:00.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:00.885+0000 7f7b9effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ba0100a80 0x7f7ba01984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:00.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:00.885+0000 7f7b9effd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ba0100a80 0x7f7ba01984f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:32950/0 (socket says 172.21.15.5:32950) 2024-09-12T02:56:00.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:00.886+0000 7f7b9effd640 1 -- 172.21.15.5:0/1806151073 learned_addr learned my addr 172.21.15.5:0/1806151073 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:56:00.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:00.886+0000 7f7b9e7fc640 1 --2- 172.21.15.5:0/1806151073 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ba0101420 0x7f7ba0198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:00.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:00.886+0000 7f7b9f7fe640 1 --2- 172.21.15.5:0/1806151073 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7ba0106a80 0x7f7ba019cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:00.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:00.886+0000 7f7b9effd640 1 -- 172.21.15.5:0/1806151073 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ba0101420 msgr2=0x7f7ba0198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:00.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:00.886+0000 7f7b9effd640 1 --2- 172.21.15.5:0/1806151073 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ba0101420 0x7f7ba0198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:00.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:00.886+0000 7f7b9effd640 1 -- 172.21.15.5:0/1806151073 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7ba0106a80 msgr2=0x7f7ba019cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:00.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:00.886+0000 7f7b9effd640 1 --2- 172.21.15.5:0/1806151073 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7ba0106a80 0x7f7ba019cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:00.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:00.886+0000 7f7b9effd640 1 -- 172.21.15.5:0/1806151073 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ba019d4a0 con 0x7f7ba0100a80 2024-09-12T02:56:00.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:00.886+0000 7f7b9effd640 1 --2- 172.21.15.5:0/1806151073 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ba0100a80 0x7f7ba01984f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7b90009a50 tx=0x7f7b90004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:00.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:00.886+0000 7f7ba4833640 1 -- 172.21.15.5:0/1806151073 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b9003f030 con 0x7f7ba0100a80 2024-09-12T02:56:00.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:00.886+0000 7f7ba4833640 1 -- 172.21.15.5:0/1806151073 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b90003ce0 con 0x7f7ba0100a80 2024-09-12T02:56:00.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:00.886+0000 7f7ba5835640 1 -- 172.21.15.5:0/1806151073 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ba019d730 con 0x7f7ba0100a80 2024-09-12T02:56:00.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:00.886+0000 7f7ba4833640 1 -- 172.21.15.5:0/1806151073 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b90035070 con 0x7f7ba0100a80 2024-09-12T02:56:00.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:00.887+0000 7f7ba5835640 1 -- 172.21.15.5:0/1806151073 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ba01a5070 con 0x7f7ba0100a80 2024-09-12T02:56:00.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:00.887+0000 7f7ba5835640 1 -- 172.21.15.5:0/1806151073 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ba0100ee0 con 0x7f7ba0100a80 2024-09-12T02:56:00.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:00.888+0000 7f7ba4833640 1 -- 172.21.15.5:0/1806151073 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f7b90042030 con 0x7f7ba0100a80 2024-09-12T02:56:00.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:00.889+0000 7f7ba4833640 1 --2- 172.21.15.5:0/1806151073 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f7b880a2c00 0x7f7b880a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:00.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:00.889+0000 7f7ba4833640 1 -- 172.21.15.5:0/1806151073 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f7b900ef010 con 0x7f7ba0100a80 2024-09-12T02:56:00.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:00.889+0000 7f7b9e7fc640 1 --2- 172.21.15.5:0/1806151073 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f7b880a2c00 0x7f7b880a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:00.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:00.890+0000 7f7b9e7fc640 1 --2- 172.21.15.5:0/1806151073 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f7b880a2c00 0x7f7b880a50c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7ba006b7e0 tx=0x7f7b9400a250 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:00.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:00.893+0000 7f7ba4833640 1 -- 172.21.15.5:0/1806151073 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7b900bbdc0 con 0x7f7ba0100a80 2024-09-12T02:56:01.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:01.118+0000 7f7ba5835640 1 -- 172.21.15.5:0/1806151073 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7ba01a5240 con 0x7f7ba0100a80 2024-09-12T02:56:01.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:01.119+0000 7f7ba4833640 1 -- 172.21.15.5:0/1806151073 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8866 (secure 0 0 0) 0x7f7b900c00c0 con 0x7f7ba0100a80 2024-09-12T02:56:01.120 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-12T02:56:01.120 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":11,"btime":"2024-09-12T02:56:00:312788+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/4135232521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":4135232521},{"type":"v1","addr":"172.21.15.5:6835","nonce":4135232521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/4290221940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":4290221940},{"type":"v1","addr":"172.21.15.5:6837","nonce":4290221940}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/26447341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":26447341},{"type":"v1","addr":"172.21.15.5:6839","nonce":26447341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/3686970137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3686970137},{"type":"v1","addr":"172.21.15.146:6833","nonce":3686970137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/568512209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":568512209},{"type":"v1","addr":"172.21.15.148:6833","nonce":568512209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/2540740420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2540740420},{"type":"v1","addr":"172.21.15.146:6835","nonce":2540740420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24458,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/1673370937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":1673370937},{"type":"v1","addr":"172.21.15.146:6837","nonce":1673370937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24481,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/4195086427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":4195086427},{"type":"v1","addr":"172.21.15.148:6835","nonce":4195086427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24487,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/2606873922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2606873922},{"type":"v1","addr":"172.21.15.148:6837","nonce":2606873922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24502,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6839/2317600000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":2317600000},{"type":"v1","addr":"172.21.15.148:6839","nonce":2317600000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-12T02:56:01.120 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 11 2024-09-12T02:56:01.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:01.121+0000 7f7ba5835640 1 -- 172.21.15.5:0/1806151073 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f7b880a2c00 msgr2=0x7f7b880a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:01.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:01.121+0000 7f7ba5835640 1 --2- 172.21.15.5:0/1806151073 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f7b880a2c00 0x7f7b880a50c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7ba006b7e0 tx=0x7f7b9400a250 comp rx=0 tx=0).stop 2024-09-12T02:56:01.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:01.121+0000 7f7ba5835640 1 -- 172.21.15.5:0/1806151073 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ba0100a80 msgr2=0x7f7ba01984f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:01.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:01.121+0000 7f7ba5835640 1 --2- 172.21.15.5:0/1806151073 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ba0100a80 0x7f7ba01984f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7b90009a50 tx=0x7f7b90004270 comp rx=0 tx=0).stop 2024-09-12T02:56:01.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:01.121+0000 7f7ba5835640 1 -- 172.21.15.5:0/1806151073 shutdown_connections 2024-09-12T02:56:01.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:01.121+0000 7f7ba5835640 1 --2- 172.21.15.5:0/1806151073 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7ba0106a80 0x7f7ba019cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:01.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:01.121+0000 7f7ba5835640 1 --2- 172.21.15.5:0/1806151073 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ba0101420 0x7f7ba0198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:01.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:01.121+0000 7f7ba5835640 1 --2- 172.21.15.5:0/1806151073 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f7b880a2c00 0x7f7b880a50c0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:01.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:01.121+0000 7f7ba5835640 1 --2- 172.21.15.5:0/1806151073 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ba0100a80 0x7f7ba01984f0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:01.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:01.121+0000 7f7ba5835640 1 -- 172.21.15.5:0/1806151073 >> 172.21.15.5:0/1806151073 conn(0x7f7ba00fc7a0 msgr2=0x7f7ba01048d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:01.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:01.122+0000 7f7ba5835640 1 -- 172.21.15.5:0/1806151073 shutdown_connections 2024-09-12T02:56:01.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:01.122+0000 7f7ba5835640 1 -- 172.21.15.5:0/1806151073 wait complete. 2024-09-12T02:56:01.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:01 smithi148 ceph-mon[35090]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:56:01.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:01 smithi148 ceph-mon[35090]: mds.? [v2:172.21.15.5:6838/26447341,v1:172.21.15.5:6839/26447341] up:boot 2024-09-12T02:56:01.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:01 smithi148 ceph-mon[35090]: fsmap 10 up:standby 2024-09-12T02:56:01.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:01 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-12T02:56:01.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:01 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:56:01.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:01 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:56:01.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:01 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:56:01.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:01 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T02:56:01.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:01 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T02:56:01.667 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:01 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:56:01.667 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:01 smithi148 ceph-mon[35090]: Deploying daemon mds.a on smithi005 2024-09-12T02:56:01.667 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:01 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/1806151073' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:56:01.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:01 smithi146 ceph-mon[35011]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:56:01.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:01 smithi146 ceph-mon[35011]: mds.? [v2:172.21.15.5:6838/26447341,v1:172.21.15.5:6839/26447341] up:boot 2024-09-12T02:56:01.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:01 smithi146 ceph-mon[35011]: fsmap 10 up:standby 2024-09-12T02:56:01.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:01 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-12T02:56:01.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:01 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:56:01.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:01 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:56:01.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:01 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:56:01.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:01 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T02:56:01.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:01 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T02:56:01.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:01 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:56:01.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:01 smithi146 ceph-mon[35011]: Deploying daemon mds.a on smithi005 2024-09-12T02:56:01.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:01 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/1806151073' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:56:01.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:01 smithi005 ceph-mon[31197]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:56:01.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:01 smithi005 ceph-mon[31197]: mds.? [v2:172.21.15.5:6838/26447341,v1:172.21.15.5:6839/26447341] up:boot 2024-09-12T02:56:01.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:01 smithi005 ceph-mon[31197]: fsmap 10 up:standby 2024-09-12T02:56:01.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:01 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-12T02:56:01.811 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:01 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:56:01.811 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:01 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:56:01.811 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:01 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:56:01.811 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:01 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T02:56:01.811 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:01 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T02:56:01.811 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:01 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:56:01.811 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:01 smithi005 ceph-mon[31197]: Deploying daemon mds.a on smithi005 2024-09-12T02:56:01.811 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:01 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/1806151073' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:56:03.060 INFO:journalctl@ceph.mds.a.smithi005.stdout:Sep 12 02:56:03 smithi005 systemd[1]: Starting Ceph mds.a for 618305f8-70b1-11ef-bcea-c7b262605968... 2024-09-12T02:56:03.478 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:03 smithi005 ceph-mon[31197]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:56:03.479 INFO:journalctl@ceph.mds.a.smithi005.stdout:Sep 12 02:56:03 smithi005 podman[69966]: 2024-09-12 02:56:03.215601175 +0000 UTC m=+0.113125713 container create be62dcbdf775072357f2fc3ef613231786d44dff6d2f9686dcd9296e7f68670d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, ceph=True) 2024-09-12T02:56:03.479 INFO:journalctl@ceph.mds.a.smithi005.stdout:Sep 12 02:56:03 smithi005 podman[69966]: 2024-09-12 02:56:03.119444199 +0000 UTC m=+0.016968754 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T02:56:03.479 INFO:journalctl@ceph.mds.a.smithi005.stdout:Sep 12 02:56:03 smithi005 podman[69966]: 2024-09-12 02:56:03.451478001 +0000 UTC m=+0.349002556 container init be62dcbdf775072357f2fc3ef613231786d44dff6d2f9686dcd9296e7f68670d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-a, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_BRANCH=HEAD) 2024-09-12T02:56:03.479 INFO:journalctl@ceph.mds.a.smithi005.stdout:Sep 12 02:56:03 smithi005 podman[69966]: 2024-09-12 02:56:03.454427477 +0000 UTC m=+0.351952022 container start be62dcbdf775072357f2fc3ef613231786d44dff6d2f9686dcd9296e7f68670d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-12T02:56:03.479 INFO:journalctl@ceph.mds.a.smithi005.stdout:Sep 12 02:56:03 smithi005 ceph-mds[69985]: -- 172.21.15.5:0/3857801830 <== mon.0 v2:172.21.15.5:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556c189ff860 con 0x556c189dd400 2024-09-12T02:56:03.665 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:03 smithi148 ceph-mon[35090]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:56:03.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:03 smithi146 ceph-mon[35011]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:56:03.810 INFO:journalctl@ceph.mds.a.smithi005.stdout:Sep 12 02:56:03 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-a[69981]: starting mds.a at 2024-09-12T02:56:03.810 INFO:journalctl@ceph.mds.a.smithi005.stdout:Sep 12 02:56:03 smithi005 bash[69966]: be62dcbdf775072357f2fc3ef613231786d44dff6d2f9686dcd9296e7f68670d 2024-09-12T02:56:03.810 INFO:journalctl@ceph.mds.a.smithi005.stdout:Sep 12 02:56:03 smithi005 systemd[1]: Started Ceph mds.a for 618305f8-70b1-11ef-bcea-c7b262605968. 2024-09-12T02:56:04.103 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:56:04.377 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:56:04.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:04 smithi146 ceph-mon[35011]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:56:04.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:04 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:56:04.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:04 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:56:04.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:04 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:56:04.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:04 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T02:56:04.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:04 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T02:56:04.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:04 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:56:04.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:04 smithi146 ceph-mon[35011]: Deploying daemon mds.b on smithi146 2024-09-12T02:56:04.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:04 smithi146 ceph-mon[35011]: mds.? [v2:172.21.15.5:6840/2393443295,v1:172.21.15.5:6841/2393443295] up:boot 2024-09-12T02:56:04.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:04 smithi146 ceph-mon[35011]: fsmap 11 up:standby 2024-09-12T02:56:04.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:04 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-12T02:56:04.805 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:04 smithi005 ceph-mon[31197]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:56:04.805 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:04 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:56:04.805 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:04 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:56:04.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:04 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:56:04.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:04 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T02:56:04.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:04 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T02:56:04.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:04 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:56:04.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:04 smithi005 ceph-mon[31197]: Deploying daemon mds.b on smithi146 2024-09-12T02:56:04.806 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:04 smithi005 ceph-mon[31197]: mds.? [v2:172.21.15.5:6840/2393443295,v1:172.21.15.5:6841/2393443295] up:boot 2024-09-12T02:56:04.807 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:04 smithi005 ceph-mon[31197]: fsmap 11 up:standby 2024-09-12T02:56:04.807 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:04 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-12T02:56:04.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:04.888+0000 7fd36a9ef640 1 -- 172.21.15.5:0/2509981632 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd364102790 msgr2=0x7fd3641067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:04.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:04.888+0000 7fd36a9ef640 1 --2- 172.21.15.5:0/2509981632 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd364102790 0x7fd3641067e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd358009900 tx=0x7fd35802a3f0 comp rx=0 tx=0).stop 2024-09-12T02:56:04.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:04.888+0000 7fd36a9ef640 1 -- 172.21.15.5:0/2509981632 shutdown_connections 2024-09-12T02:56:04.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:04.888+0000 7fd36a9ef640 1 --2- 172.21.15.5:0/2509981632 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd364102790 0x7fd3641067e0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:04.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:04.888+0000 7fd36a9ef640 1 --2- 172.21.15.5:0/2509981632 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd364106f10 0x7fd36410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:04.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:04.888+0000 7fd36a9ef640 1 --2- 172.21.15.5:0/2509981632 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd364101de0 0x7fd3641021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:04.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:04.888+0000 7fd36a9ef640 1 -- 172.21.15.5:0/2509981632 >> 172.21.15.5:0/2509981632 conn(0x7fd3640fd8c0 msgr2=0x7fd3640ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:04.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:04.888+0000 7fd36a9ef640 1 -- 172.21.15.5:0/2509981632 shutdown_connections 2024-09-12T02:56:04.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:04.888+0000 7fd36a9ef640 1 -- 172.21.15.5:0/2509981632 wait complete. 2024-09-12T02:56:04.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:04.889+0000 7fd36a9ef640 1 Processor -- start 2024-09-12T02:56:04.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:04.889+0000 7fd36a9ef640 1 -- start start 2024-09-12T02:56:04.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:04.889+0000 7fd36a9ef640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd364101de0 0x7fd3640744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:04.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:04.889+0000 7fd36a9ef640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd364102790 0x7fd364074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:04.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:04.889+0000 7fd36a9ef640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd364106f10 0x7fd364074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:04.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:04.889+0000 7fd36a9ef640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd364111f90 con 0x7fd364102790 2024-09-12T02:56:04.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:04.889+0000 7fd36a9ef640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fd364111e10 con 0x7fd364106f10 2024-09-12T02:56:04.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:04.889+0000 7fd36a9ef640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd364112110 con 0x7fd364101de0 2024-09-12T02:56:04.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:04.889+0000 7fd3637fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd364102790 0x7fd364074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:04.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:04.889+0000 7fd3637fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd364102790 0x7fd364074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47026/0 (socket says 172.21.15.5:47026) 2024-09-12T02:56:04.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:04.889+0000 7fd3637fe640 1 -- 172.21.15.5:0/2662374386 learned_addr learned my addr 172.21.15.5:0/2662374386 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:56:04.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:04.889+0000 7fd363fff640 1 --2- 172.21.15.5:0/2662374386 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd364101de0 0x7fd3640744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:04.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:04.889+0000 7fd368f8c640 1 --2- 172.21.15.5:0/2662374386 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd364106f10 0x7fd364074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:04.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:04.890+0000 7fd3637fe640 1 -- 172.21.15.5:0/2662374386 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd364101de0 msgr2=0x7fd3640744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:04.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:04.890+0000 7fd3637fe640 1 --2- 172.21.15.5:0/2662374386 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd364101de0 0x7fd3640744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:04.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:04.890+0000 7fd3637fe640 1 -- 172.21.15.5:0/2662374386 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd364106f10 msgr2=0x7fd364074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:04.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:04.890+0000 7fd3637fe640 1 --2- 172.21.15.5:0/2662374386 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd364106f10 0x7fd364074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:04.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:04.890+0000 7fd3637fe640 1 -- 172.21.15.5:0/2662374386 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3641a68d0 con 0x7fd364102790 2024-09-12T02:56:04.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:04.890+0000 7fd3637fe640 1 --2- 172.21.15.5:0/2662374386 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd364102790 0x7fd364074a30 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd358005ab0 tx=0x7fd3580042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:04.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:04.890+0000 7fd3617fa640 1 -- 172.21.15.5:0/2662374386 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd35803f030 con 0x7fd364102790 2024-09-12T02:56:04.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:04.890+0000 7fd36a9ef640 1 -- 172.21.15.5:0/2662374386 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3641a6b60 con 0x7fd364102790 2024-09-12T02:56:04.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:04.890+0000 7fd3617fa640 1 -- 172.21.15.5:0/2662374386 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd358003ce0 con 0x7fd364102790 2024-09-12T02:56:04.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:04.890+0000 7fd3617fa640 1 -- 172.21.15.5:0/2662374386 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd358035070 con 0x7fd364102790 2024-09-12T02:56:04.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:04.890+0000 7fd36a9ef640 1 -- 172.21.15.5:0/2662374386 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3641a7090 con 0x7fd364102790 2024-09-12T02:56:04.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:04.891+0000 7fd36a9ef640 1 -- 172.21.15.5:0/2662374386 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3641021c0 con 0x7fd364102790 2024-09-12T02:56:04.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:04.892+0000 7fd3617fa640 1 -- 172.21.15.5:0/2662374386 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fd358042030 con 0x7fd364102790 2024-09-12T02:56:04.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:04.893+0000 7fd3617fa640 1 --2- 172.21.15.5:0/2662374386 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fd34c0a2b80 0x7fd34c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:04.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:04.893+0000 7fd3617fa640 1 -- 172.21.15.5:0/2662374386 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7fd3580ef160 con 0x7fd364102790 2024-09-12T02:56:04.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:04.893+0000 7fd363fff640 1 --2- 172.21.15.5:0/2662374386 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fd34c0a2b80 0x7fd34c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:04.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:04.894+0000 7fd363fff640 1 --2- 172.21.15.5:0/2662374386 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fd34c0a2b80 0x7fd34c0a5040 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd354009990 tx=0x7fd354009110 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:04.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:04.897+0000 7fd3617fa640 1 -- 172.21.15.5:0/2662374386 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd3580bc4c0 con 0x7fd364102790 2024-09-12T02:56:04.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:04 smithi148 ceph-mon[35090]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:56:04.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:04 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:56:04.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:04 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:56:04.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:04 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:56:04.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:04 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-12T02:56:04.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:04 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-12T02:56:04.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:04 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:56:04.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:04 smithi148 ceph-mon[35090]: Deploying daemon mds.b on smithi146 2024-09-12T02:56:04.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:04 smithi148 ceph-mon[35090]: mds.? [v2:172.21.15.5:6840/2393443295,v1:172.21.15.5:6841/2393443295] up:boot 2024-09-12T02:56:04.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:04 smithi148 ceph-mon[35090]: fsmap 11 up:standby 2024-09-12T02:56:04.917 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:04 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-12T02:56:05.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:05.109+0000 7fd36a9ef640 1 -- 172.21.15.5:0/2662374386 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd36410a0f0 con 0x7fd364102790 2024-09-12T02:56:05.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:05.109+0000 7fd3617fa640 1 -- 172.21.15.5:0/2662374386 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9689 (secure 0 0 0) 0x7fd3580bbdc0 con 0x7fd364102790 2024-09-12T02:56:05.110 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-12T02:56:05.110 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":12,"btime":"2024-09-12T02:56:04:322589+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/4135232521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":4135232521},{"type":"v1","addr":"172.21.15.5:6835","nonce":4135232521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/4290221940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":4290221940},{"type":"v1","addr":"172.21.15.5:6837","nonce":4290221940}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/26447341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":26447341},{"type":"v1","addr":"172.21.15.5:6839","nonce":26447341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/2393443295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2393443295},{"type":"v1","addr":"172.21.15.5:6841","nonce":2393443295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/3686970137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3686970137},{"type":"v1","addr":"172.21.15.146:6833","nonce":3686970137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/568512209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":568512209},{"type":"v1","addr":"172.21.15.148:6833","nonce":568512209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/2540740420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2540740420},{"type":"v1","addr":"172.21.15.146:6835","nonce":2540740420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24458,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/1673370937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":1673370937},{"type":"v1","addr":"172.21.15.146:6837","nonce":1673370937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24481,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/4195086427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":4195086427},{"type":"v1","addr":"172.21.15.148:6835","nonce":4195086427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24487,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/2606873922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2606873922},{"type":"v1","addr":"172.21.15.148:6837","nonce":2606873922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24502,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6839/2317600000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":2317600000},{"type":"v1","addr":"172.21.15.148:6839","nonce":2317600000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-12T02:56:05.110 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 12 2024-09-12T02:56:05.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:05.111+0000 7fd36a9ef640 1 -- 172.21.15.5:0/2662374386 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fd34c0a2b80 msgr2=0x7fd34c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:05.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:05.111+0000 7fd36a9ef640 1 --2- 172.21.15.5:0/2662374386 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fd34c0a2b80 0x7fd34c0a5040 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd354009990 tx=0x7fd354009110 comp rx=0 tx=0).stop 2024-09-12T02:56:05.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:05.111+0000 7fd36a9ef640 1 -- 172.21.15.5:0/2662374386 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd364102790 msgr2=0x7fd364074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:05.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:05.111+0000 7fd36a9ef640 1 --2- 172.21.15.5:0/2662374386 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd364102790 0x7fd364074a30 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd358005ab0 tx=0x7fd3580042f0 comp rx=0 tx=0).stop 2024-09-12T02:56:05.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:05.111+0000 7fd36a9ef640 1 -- 172.21.15.5:0/2662374386 shutdown_connections 2024-09-12T02:56:05.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:05.111+0000 7fd36a9ef640 1 --2- 172.21.15.5:0/2662374386 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fd34c0a2b80 0x7fd34c0a5040 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:05.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:05.111+0000 7fd36a9ef640 1 --2- 172.21.15.5:0/2662374386 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd364102790 0x7fd364074a30 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:05.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:05.111+0000 7fd36a9ef640 1 --2- 172.21.15.5:0/2662374386 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd364106f10 0x7fd364074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:05.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:05.111+0000 7fd36a9ef640 1 --2- 172.21.15.5:0/2662374386 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd364101de0 0x7fd3640744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:05.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:05.112+0000 7fd36a9ef640 1 -- 172.21.15.5:0/2662374386 >> 172.21.15.5:0/2662374386 conn(0x7fd3640fd8c0 msgr2=0x7fd3640fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:05.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:05.112+0000 7fd36a9ef640 1 -- 172.21.15.5:0/2662374386 shutdown_connections 2024-09-12T02:56:05.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:05.112+0000 7fd36a9ef640 1 -- 172.21.15.5:0/2662374386 wait complete. 2024-09-12T02:56:05.645 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:05 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/2662374386' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:56:05.646 INFO:journalctl@ceph.mds.b.smithi146.stdout:Sep 12 02:56:05 smithi146 systemd[1]: Starting Ceph mds.b for 618305f8-70b1-11ef-bcea-c7b262605968... 2024-09-12T02:56:05.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:05 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/2662374386' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:56:05.915 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:05 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/2662374386' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:56:05.957 INFO:journalctl@ceph.mds.b.smithi146.stdout:Sep 12 02:56:05 smithi146 podman[65407]: 2024-09-12 02:56:05.761153404 +0000 UTC m=+0.016440754 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T02:56:05.957 INFO:journalctl@ceph.mds.b.smithi146.stdout:Sep 12 02:56:05 smithi146 podman[65407]: 2024-09-12 02:56:05.872202035 +0000 UTC m=+0.127489386 container create 4dd5146b28909eb32e1a24aa1f94938941a23fdf93fede8fbc2f721f733284d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-b, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T02:56:06.456 INFO:journalctl@ceph.mds.b.smithi146.stdout:Sep 12 02:56:06 smithi146 podman[65407]: 2024-09-12 02:56:06.074528238 +0000 UTC m=+0.329815588 container init 4dd5146b28909eb32e1a24aa1f94938941a23fdf93fede8fbc2f721f733284d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-b, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-b53b6d3) 2024-09-12T02:56:06.456 INFO:journalctl@ceph.mds.b.smithi146.stdout:Sep 12 02:56:06 smithi146 podman[65407]: 2024-09-12 02:56:06.077430869 +0000 UTC m=+0.332718228 container start 4dd5146b28909eb32e1a24aa1f94938941a23fdf93fede8fbc2f721f733284d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-b, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T02:56:06.457 INFO:journalctl@ceph.mds.b.smithi146.stdout:Sep 12 02:56:06 smithi146 ceph-mds[65426]: -- 172.21.15.146:0/1864122326 <== mon.2 v2:172.21.15.146:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e243173860 con 0x55e243150800 2024-09-12T02:56:06.457 INFO:journalctl@ceph.mds.b.smithi146.stdout:Sep 12 02:56:06 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-b[65422]: starting mds.b at 2024-09-12T02:56:06.457 INFO:journalctl@ceph.mds.b.smithi146.stdout:Sep 12 02:56:06 smithi146 bash[65407]: 4dd5146b28909eb32e1a24aa1f94938941a23fdf93fede8fbc2f721f733284d3 2024-09-12T02:56:06.457 INFO:journalctl@ceph.mds.b.smithi146.stdout:Sep 12 02:56:06 smithi146 systemd[1]: Started Ceph mds.b for 618305f8-70b1-11ef-bcea-c7b262605968. 2024-09-12T02:56:07.381 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:07 smithi148 ceph-mon[35090]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:56:07.381 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:07 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:56:07.381 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:07 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:56:07.381 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:07 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:56:07.381 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:07 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:56:07.381 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:07 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:56:07.381 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:07 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:56:07.381 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:07 smithi148 ceph-mon[35090]: mds.? [v2:172.21.15.146:6838/3036767228,v1:172.21.15.146:6839/3036767228] up:boot 2024-09-12T02:56:07.381 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:07 smithi148 ceph-mon[35090]: fsmap 12 up:standby 2024-09-12T02:56:07.382 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:07 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-12T02:56:07.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:07 smithi005 ceph-mon[31197]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:56:07.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:07 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:56:07.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:07 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:56:07.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:07 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:56:07.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:07 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:56:07.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:07 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:56:07.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:07 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:56:07.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:07 smithi005 ceph-mon[31197]: mds.? [v2:172.21.15.146:6838/3036767228,v1:172.21.15.146:6839/3036767228] up:boot 2024-09-12T02:56:07.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:07 smithi005 ceph-mon[31197]: fsmap 12 up:standby 2024-09-12T02:56:07.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:07 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-12T02:56:07.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:07 smithi146 ceph-mon[35011]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:56:07.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:07 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:56:07.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:07 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:56:07.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:07 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:56:07.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:07 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:56:07.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:07 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:56:07.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:07 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:56:07.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:07 smithi146 ceph-mon[35011]: mds.? [v2:172.21.15.146:6838/3036767228,v1:172.21.15.146:6839/3036767228] up:boot 2024-09-12T02:56:07.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:07 smithi146 ceph-mon[35011]: fsmap 12 up:standby 2024-09-12T02:56:07.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:07 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-12T02:56:07.791 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:56:08.049 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:56:08.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:08.513+0000 7fe3913d1640 1 -- 172.21.15.5:0/2181918998 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe38c108150 msgr2=0x7fe38c10ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:08.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:08.513+0000 7fe3913d1640 1 --2- 172.21.15.5:0/2181918998 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe38c108150 0x7fe38c10ea80 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe384009ca0 tx=0x7fe38402a8c0 comp rx=0 tx=0).stop 2024-09-12T02:56:08.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:08.513+0000 7fe3913d1640 1 -- 172.21.15.5:0/2181918998 shutdown_connections 2024-09-12T02:56:08.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:08.513+0000 7fe3913d1640 1 --2- 172.21.15.5:0/2181918998 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe38c108150 0x7fe38c10ea80 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:08.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:08.513+0000 7fe3913d1640 1 --2- 172.21.15.5:0/2181918998 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe38c1012d0 0x7fe38c107a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:08.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:08.513+0000 7fe3913d1640 1 --2- 172.21.15.5:0/2181918998 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe38c100920 0x7fe38c100d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:08.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:08.513+0000 7fe3913d1640 1 -- 172.21.15.5:0/2181918998 >> 172.21.15.5:0/2181918998 conn(0x7fe38c075f70 msgr2=0x7fe38c076380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:08.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:08.513+0000 7fe3913d1640 1 -- 172.21.15.5:0/2181918998 shutdown_connections 2024-09-12T02:56:08.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:08.513+0000 7fe3913d1640 1 -- 172.21.15.5:0/2181918998 wait complete. 2024-09-12T02:56:08.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:08.514+0000 7fe3913d1640 1 Processor -- start 2024-09-12T02:56:08.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:08.514+0000 7fe3913d1640 1 -- start start 2024-09-12T02:56:08.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:08.514+0000 7fe3913d1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe38c100920 0x7fe38c19ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:08.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:08.514+0000 7fe3913d1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe38c1012d0 0x7fe38c19f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:08.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:08.514+0000 7fe3913d1640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe38c108150 0x7fe38c10dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:08.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:08.514+0000 7fe3913d1640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe38c111f60 con 0x7fe38c1012d0 2024-09-12T02:56:08.559 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:08.514+0000 7fe3913d1640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fe38c111de0 con 0x7fe38c108150 2024-09-12T02:56:08.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:08.514+0000 7fe3913d1640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe38c1120e0 con 0x7fe38c100920 2024-09-12T02:56:08.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:08.515+0000 7fe38a7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe38c1012d0 0x7fe38c19f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:08.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:08.515+0000 7fe38a7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe38c1012d0 0x7fe38c19f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47046/0 (socket says 172.21.15.5:47046) 2024-09-12T02:56:08.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:08.515+0000 7fe38a7fc640 1 -- 172.21.15.5:0/779208130 learned_addr learned my addr 172.21.15.5:0/779208130 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:56:08.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:08.515+0000 7fe38affd640 1 --2- 172.21.15.5:0/779208130 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe38c100920 0x7fe38c19ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:08.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:08.515+0000 7fe38b7fe640 1 --2- 172.21.15.5:0/779208130 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe38c108150 0x7fe38c10dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:08.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:08.515+0000 7fe38b7fe640 1 -- 172.21.15.5:0/779208130 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe38c100920 msgr2=0x7fe38c19ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:08.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:08.515+0000 7fe38b7fe640 1 --2- 172.21.15.5:0/779208130 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe38c100920 0x7fe38c19ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:08.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:08.515+0000 7fe38b7fe640 1 -- 172.21.15.5:0/779208130 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe38c1012d0 msgr2=0x7fe38c19f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:08.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:08.515+0000 7fe38b7fe640 1 --2- 172.21.15.5:0/779208130 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe38c1012d0 0x7fe38c19f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:08.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:08.515+0000 7fe38b7fe640 1 -- 172.21.15.5:0/779208130 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe38c10e4b0 con 0x7fe38c108150 2024-09-12T02:56:08.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:08.515+0000 7fe38a7fc640 1 --2- 172.21.15.5:0/779208130 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe38c1012d0 0x7fe38c19f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:56:08.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:08.515+0000 7fe38affd640 1 --2- 172.21.15.5:0/779208130 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe38c100920 0x7fe38c19ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:56:08.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:08.515+0000 7fe38b7fe640 1 --2- 172.21.15.5:0/779208130 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe38c108150 0x7fe38c10dbf0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe38402ade0 tx=0x7fe3840319f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:08.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:08.516+0000 7fe373fff640 1 -- 172.21.15.5:0/779208130 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe384003b70 con 0x7fe38c108150 2024-09-12T02:56:08.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:08.516+0000 7fe373fff640 1 -- 172.21.15.5:0/779208130 <== mon.2 v2:172.21.15.146:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe384003d10 con 0x7fe38c108150 2024-09-12T02:56:08.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:08.516+0000 7fe3913d1640 1 -- 172.21.15.5:0/779208130 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe38c10e740 con 0x7fe38c108150 2024-09-12T02:56:08.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:08.516+0000 7fe373fff640 1 -- 172.21.15.5:0/779208130 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe38402fcb0 con 0x7fe38c108150 2024-09-12T02:56:08.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:08.516+0000 7fe3913d1640 1 -- 172.21.15.5:0/779208130 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe38c19ac80 con 0x7fe38c108150 2024-09-12T02:56:08.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:08.517+0000 7fe3913d1640 1 -- 172.21.15.5:0/779208130 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe38c06a110 con 0x7fe38c108150 2024-09-12T02:56:08.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:08.518+0000 7fe373fff640 1 -- 172.21.15.5:0/779208130 <== mon.2 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fe384004050 con 0x7fe38c108150 2024-09-12T02:56:08.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:08.518+0000 7fe373fff640 1 --2- 172.21.15.5:0/779208130 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fe3740a2b80 0x7fe3740a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:08.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:08.519+0000 7fe38affd640 1 --2- 172.21.15.5:0/779208130 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fe3740a2b80 0x7fe3740a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:08.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:08.519+0000 7fe373fff640 1 -- 172.21.15.5:0/779208130 <== mon.2 v2:172.21.15.146:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7fe3840efd50 con 0x7fe38c108150 2024-09-12T02:56:08.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:08.519+0000 7fe38affd640 1 --2- 172.21.15.5:0/779208130 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fe3740a2b80 0x7fe3740a5040 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe37c009fc0 tx=0x7fe37c005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:08.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:08.523+0000 7fe373fff640 1 -- 172.21.15.5:0/779208130 <== mon.2 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe3840bd0f0 con 0x7fe38c108150 2024-09-12T02:56:08.700 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:08 smithi005 ceph-mon[31197]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:56:08.700 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:08 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:56:08.700 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:08 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:56:08.700 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:08 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:56:08.700 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:08 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:56:08.700 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:08 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:56:08.700 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:08 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:56:08.700 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:08 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:56:08.700 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:08 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:56:08.700 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:08 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:56:08.701 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:08 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:56:08.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:08.773+0000 7fe3913d1640 1 -- 172.21.15.5:0/779208130 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe38c19af30 con 0x7fe38c108150 2024-09-12T02:56:08.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:08.774+0000 7fe373fff640 1 -- 172.21.15.5:0/779208130 <== mon.2 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10518 (secure 0 0 0) 0x7fe3840bc9f0 con 0x7fe38c108150 2024-09-12T02:56:08.774 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-12T02:56:08.775 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":13,"btime":"2024-09-12T02:56:06:590317+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/4135232521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":4135232521},{"type":"v1","addr":"172.21.15.5:6835","nonce":4135232521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/4290221940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":4290221940},{"type":"v1","addr":"172.21.15.5:6837","nonce":4290221940}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14601,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/26447341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":26447341},{"type":"v1","addr":"172.21.15.5:6839","nonce":26447341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14613,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/2393443295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2393443295},{"type":"v1","addr":"172.21.15.5:6841","nonce":2393443295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6833/3686970137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3686970137},{"type":"v1","addr":"172.21.15.146:6833","nonce":3686970137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24439,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6833/568512209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":568512209},{"type":"v1","addr":"172.21.15.148:6833","nonce":568512209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6835/2540740420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2540740420},{"type":"v1","addr":"172.21.15.146:6835","nonce":2540740420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24458,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6837/1673370937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":1673370937},{"type":"v1","addr":"172.21.15.146:6837","nonce":1673370937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24479,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.146:6839/3036767228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":3036767228},{"type":"v1","addr":"172.21.15.146:6839","nonce":3036767228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":24481,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6835/4195086427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":4195086427},{"type":"v1","addr":"172.21.15.148:6835","nonce":4195086427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24487,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6837/2606873922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2606873922},{"type":"v1","addr":"172.21.15.148:6837","nonce":2606873922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24502,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.148:6839/2317600000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":2317600000},{"type":"v1","addr":"172.21.15.148:6839","nonce":2317600000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-09-12T02:56:08.775 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 13 2024-09-12T02:56:08.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:08.776+0000 7fe3913d1640 1 -- 172.21.15.5:0/779208130 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fe3740a2b80 msgr2=0x7fe3740a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:08.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:08.776+0000 7fe3913d1640 1 --2- 172.21.15.5:0/779208130 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fe3740a2b80 0x7fe3740a5040 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe37c009fc0 tx=0x7fe37c005f90 comp rx=0 tx=0).stop 2024-09-12T02:56:08.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:08.776+0000 7fe3913d1640 1 -- 172.21.15.5:0/779208130 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe38c108150 msgr2=0x7fe38c10dbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:08.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:08.776+0000 7fe3913d1640 1 --2- 172.21.15.5:0/779208130 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe38c108150 0x7fe38c10dbf0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe38402ade0 tx=0x7fe3840319f0 comp rx=0 tx=0).stop 2024-09-12T02:56:08.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:08.776+0000 7fe3913d1640 1 -- 172.21.15.5:0/779208130 shutdown_connections 2024-09-12T02:56:08.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:08.776+0000 7fe3913d1640 1 --2- 172.21.15.5:0/779208130 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fe3740a2b80 0x7fe3740a5040 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:08.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:08.776+0000 7fe3913d1640 1 --2- 172.21.15.5:0/779208130 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe38c1012d0 0x7fe38c19f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:08.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:08.776+0000 7fe3913d1640 1 --2- 172.21.15.5:0/779208130 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe38c108150 0x7fe38c10dbf0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:08.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:08.776+0000 7fe3913d1640 1 --2- 172.21.15.5:0/779208130 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe38c100920 0x7fe38c19ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:08.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:08.776+0000 7fe3913d1640 1 -- 172.21.15.5:0/779208130 >> 172.21.15.5:0/779208130 conn(0x7fe38c075f70 msgr2=0x7fe38c0ff680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:08.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:08.776+0000 7fe3913d1640 1 -- 172.21.15.5:0/779208130 shutdown_connections 2024-09-12T02:56:08.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:08.776+0000 7fe3913d1640 1 -- 172.21.15.5:0/779208130 wait complete. 2024-09-12T02:56:08.956 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:08 smithi146 ceph-mon[35011]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:56:08.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:08 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:56:08.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:08 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:56:08.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:08 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:56:08.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:08 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:56:08.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:08 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:56:08.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:08 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:56:08.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:08 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:56:08.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:08 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:56:08.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:08 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:56:08.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:08 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:56:09.025 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:08 smithi148 ceph-mon[35090]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:56:09.025 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:08 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:56:09.025 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:08 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:56:09.025 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:08 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:56:09.025 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:08 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:56:09.025 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:08 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:56:09.025 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:08 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:56:09.025 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:08 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:56:09.026 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:08 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:56:09.026 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:08 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:56:09.026 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:08 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:56:09.398 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-09-12T02:56:09.762 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:09 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/779208130' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:56:09.762 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:09 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:56:10.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:09 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/779208130' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:56:10.105 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:09 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:56:10.130 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:56:10.165 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:09 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/779208130' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:56:10.165 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:09 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:56:10.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:10.568+0000 7f78d3163640 1 -- 172.21.15.5:0/2617790869 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78cc102ef0 msgr2=0x7f78cc10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:10.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:10.568+0000 7f78d3163640 1 --2- 172.21.15.5:0/2617790869 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78cc102ef0 0x7f78cc10da40 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f78c4009900 tx=0x7f78c402a480 comp rx=0 tx=0).stop 2024-09-12T02:56:10.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:10.568+0000 7f78d3163640 1 -- 172.21.15.5:0/2617790869 shutdown_connections 2024-09-12T02:56:10.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:10.568+0000 7f78d3163640 1 --2- 172.21.15.5:0/2617790869 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78cc102ef0 0x7f78cc10da40 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:10.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:10.568+0000 7f78d3163640 1 --2- 172.21.15.5:0/2617790869 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f78cc102570 0x7f78cc1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:10.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:10.568+0000 7f78d3163640 1 --2- 172.21.15.5:0/2617790869 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f78cc100e20 0x7f78cc101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:10.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:10.568+0000 7f78d3163640 1 -- 172.21.15.5:0/2617790869 >> 172.21.15.5:0/2617790869 conn(0x7f78cc0fc7d0 msgr2=0x7f78cc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:10.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:10.568+0000 7f78d3163640 1 -- 172.21.15.5:0/2617790869 shutdown_connections 2024-09-12T02:56:10.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:10.568+0000 7f78d3163640 1 -- 172.21.15.5:0/2617790869 wait complete. 2024-09-12T02:56:10.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:10.569+0000 7f78d3163640 1 Processor -- start 2024-09-12T02:56:10.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:10.569+0000 7f78d3163640 1 -- start start 2024-09-12T02:56:10.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:10.569+0000 7f78d3163640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f78cc100e20 0x7f78cc10e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:10.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:10.569+0000 7f78d3163640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78cc102570 0x7f78cc10e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:10.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:10.569+0000 7f78d3163640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f78cc102ef0 0x7f78cc10edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:10.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:10.569+0000 7f78d3163640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f78cc112090 con 0x7f78cc102570 2024-09-12T02:56:10.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:10.569+0000 7f78d3163640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f78cc111f10 con 0x7f78cc100e20 2024-09-12T02:56:10.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:10.569+0000 7f78d3163640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f78cc112210 con 0x7f78cc102ef0 2024-09-12T02:56:10.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:10.569+0000 7f78cbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78cc102570 0x7f78cc10e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:10.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:10.569+0000 7f78cbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78cc102570 0x7f78cc10e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47078/0 (socket says 172.21.15.5:47078) 2024-09-12T02:56:10.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:10.569+0000 7f78cbfff640 1 -- 172.21.15.5:0/1499593153 learned_addr learned my addr 172.21.15.5:0/1499593153 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:56:10.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:10.569+0000 7f78d1700640 1 --2- 172.21.15.5:0/1499593153 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f78cc102ef0 0x7f78cc10edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:10.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:10.569+0000 7f78d0eff640 1 --2- 172.21.15.5:0/1499593153 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f78cc100e20 0x7f78cc10e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:10.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:10.569+0000 7f78cbfff640 1 -- 172.21.15.5:0/1499593153 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f78cc102ef0 msgr2=0x7f78cc10edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:10.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:10.569+0000 7f78cbfff640 1 --2- 172.21.15.5:0/1499593153 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f78cc102ef0 0x7f78cc10edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:10.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:10.569+0000 7f78cbfff640 1 -- 172.21.15.5:0/1499593153 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f78cc100e20 msgr2=0x7f78cc10e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:10.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:10.569+0000 7f78cbfff640 1 --2- 172.21.15.5:0/1499593153 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f78cc100e20 0x7f78cc10e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:10.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:10.569+0000 7f78cbfff640 1 -- 172.21.15.5:0/1499593153 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78cc1a69e0 con 0x7f78cc102570 2024-09-12T02:56:10.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:10.569+0000 7f78d1700640 1 --2- 172.21.15.5:0/1499593153 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f78cc102ef0 0x7f78cc10edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:56:10.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:10.570+0000 7f78cbfff640 1 --2- 172.21.15.5:0/1499593153 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78cc102570 0x7f78cc10e890 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f78c000ebc0 tx=0x7f78c000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:10.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:10.570+0000 7f78c9ffb640 1 -- 172.21.15.5:0/1499593153 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78c0010630 con 0x7f78cc102570 2024-09-12T02:56:10.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:10.570+0000 7f78d3163640 1 -- 172.21.15.5:0/1499593153 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78cc1a6cd0 con 0x7f78cc102570 2024-09-12T02:56:10.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:10.570+0000 7f78c9ffb640 1 -- 172.21.15.5:0/1499593153 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f78c0010cb0 con 0x7f78cc102570 2024-09-12T02:56:10.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:10.570+0000 7f78c9ffb640 1 -- 172.21.15.5:0/1499593153 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78c0018bd0 con 0x7f78cc102570 2024-09-12T02:56:10.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:10.570+0000 7f78d3163640 1 -- 172.21.15.5:0/1499593153 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78cc1a71b0 con 0x7f78cc102570 2024-09-12T02:56:10.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:10.571+0000 7f78d3163640 1 -- 172.21.15.5:0/1499593153 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78cc101200 con 0x7f78cc102570 2024-09-12T02:56:10.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:10.572+0000 7f78c9ffb640 1 -- 172.21.15.5:0/1499593153 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f78c00107d0 con 0x7f78cc102570 2024-09-12T02:56:10.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:10.573+0000 7f78c9ffb640 1 --2- 172.21.15.5:0/1499593153 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f78b40a2b80 0x7f78b40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:10.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:10.573+0000 7f78d0eff640 1 --2- 172.21.15.5:0/1499593153 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f78b40a2b80 0x7f78b40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:10.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:10.573+0000 7f78c9ffb640 1 -- 172.21.15.5:0/1499593153 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f78c00cf1a0 con 0x7f78cc102570 2024-09-12T02:56:10.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:10.573+0000 7f78d0eff640 1 --2- 172.21.15.5:0/1499593153 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f78b40a2b80 0x7f78b40a5040 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f78bc009bd0 tx=0x7f78bc005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:10.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:10.577+0000 7f78c9ffb640 1 -- 172.21.15.5:0/1499593153 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f78c009beb0 con 0x7f78cc102570 2024-09-12T02:56:10.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:10.799+0000 7f78d3163640 1 -- 172.21.15.5:0/1499593153 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f78cc109090 con 0x7f78cc102570 2024-09-12T02:56:10.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:10.799+0000 7f78c9ffb640 1 -- 172.21.15.5:0/1499593153 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v76) ==== 84+0+65 (secure 0 0 0) 0x7f78c00a01b0 con 0x7f78cc102570 2024-09-12T02:56:10.800 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-12T02:56:10.800 INFO:teuthology.orchestra.run.smithi005.stdout:[ 2024-09-12T02:56:10.800 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-12T02:56:10.800 INFO:teuthology.orchestra.run.smithi005.stdout: "poolnum": 1, 2024-09-12T02:56:10.801 INFO:teuthology.orchestra.run.smithi005.stdout: "poolname": ".mgr" 2024-09-12T02:56:10.801 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T02:56:10.801 INFO:teuthology.orchestra.run.smithi005.stdout:] 2024-09-12T02:56:10.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:10.802+0000 7f78d3163640 1 -- 172.21.15.5:0/1499593153 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f78b40a2b80 msgr2=0x7f78b40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:10.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:10.802+0000 7f78d3163640 1 --2- 172.21.15.5:0/1499593153 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f78b40a2b80 0x7f78b40a5040 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f78bc009bd0 tx=0x7f78bc005f90 comp rx=0 tx=0).stop 2024-09-12T02:56:10.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:10.802+0000 7f78d3163640 1 -- 172.21.15.5:0/1499593153 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78cc102570 msgr2=0x7f78cc10e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:10.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:10.802+0000 7f78d3163640 1 --2- 172.21.15.5:0/1499593153 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78cc102570 0x7f78cc10e890 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f78c000ebc0 tx=0x7f78c000c6a0 comp rx=0 tx=0).stop 2024-09-12T02:56:10.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:10.803+0000 7f78d3163640 1 -- 172.21.15.5:0/1499593153 shutdown_connections 2024-09-12T02:56:10.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:10.803+0000 7f78d3163640 1 --2- 172.21.15.5:0/1499593153 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f78b40a2b80 0x7f78b40a5040 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:10.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:10.803+0000 7f78d3163640 1 --2- 172.21.15.5:0/1499593153 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f78cc102570 0x7f78cc10e890 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:10.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:10.803+0000 7f78d3163640 1 --2- 172.21.15.5:0/1499593153 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f78cc102ef0 0x7f78cc10edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:10.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:10.803+0000 7f78d3163640 1 --2- 172.21.15.5:0/1499593153 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f78cc100e20 0x7f78cc10e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:10.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:10.803+0000 7f78d3163640 1 -- 172.21.15.5:0/1499593153 >> 172.21.15.5:0/1499593153 conn(0x7f78cc0fc7d0 msgr2=0x7f78cc1068d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:10.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:10.803+0000 7f78d3163640 1 -- 172.21.15.5:0/1499593153 shutdown_connections 2024-09-12T02:56:10.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:10.803+0000 7f78d3163640 1 -- 172.21.15.5:0/1499593153 wait complete. 2024-09-12T02:56:11.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:10 smithi148 ceph-mon[35090]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:56:11.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:10 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:56:11.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:10 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:56:11.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:10 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:56:11.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:10 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:56:11.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:10 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/1499593153' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-12T02:56:11.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:10 smithi146 ceph-mon[35011]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:56:11.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:10 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:56:11.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:10 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:56:11.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:10 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:56:11.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:10 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:56:11.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:10 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/1499593153' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-12T02:56:11.260 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:10 smithi005 ceph-mon[31197]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:56:11.260 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:10 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:56:11.261 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:10 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:56:11.261 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:10 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:56:11.261 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:10 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:56:11.261 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:10 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/1499593153' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-12T02:56:11.438 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-12T02:56:11.439 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-12T02:56:11.738 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:56:12.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:12.214+0000 7fb37d42b640 1 -- 172.21.15.5:0/940861652 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb378106f10 msgr2=0x7fb37810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:12.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:12.214+0000 7fb37d42b640 1 --2- 172.21.15.5:0/940861652 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb378106f10 0x7fb37810ea90 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb370009ca0 tx=0x7fb37002a8c0 comp rx=0 tx=0).stop 2024-09-12T02:56:12.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:12.214+0000 7fb37d42b640 1 -- 172.21.15.5:0/940861652 shutdown_connections 2024-09-12T02:56:12.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:12.214+0000 7fb37d42b640 1 --2- 172.21.15.5:0/940861652 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb378106f10 0x7fb37810ea90 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:12.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:12.214+0000 7fb37d42b640 1 --2- 172.21.15.5:0/940861652 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb378102790 0x7fb3781067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:12.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:12.214+0000 7fb37d42b640 1 --2- 172.21.15.5:0/940861652 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb378101de0 0x7fb3781021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:12.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:12.215+0000 7fb37d42b640 1 -- 172.21.15.5:0/940861652 >> 172.21.15.5:0/940861652 conn(0x7fb3780fd8c0 msgr2=0x7fb3780ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:12.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:12.215+0000 7fb37d42b640 1 -- 172.21.15.5:0/940861652 shutdown_connections 2024-09-12T02:56:12.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:12.215+0000 7fb37d42b640 1 -- 172.21.15.5:0/940861652 wait complete. 2024-09-12T02:56:12.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:12.215+0000 7fb37d42b640 1 Processor -- start 2024-09-12T02:56:12.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:12.215+0000 7fb37d42b640 1 -- start start 2024-09-12T02:56:12.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:12.216+0000 7fb37d42b640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb378101de0 0x7fb378198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:12.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:12.216+0000 7fb37d42b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb378102790 0x7fb378198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:12.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:12.216+0000 7fb37d42b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb378106f10 0x7fb37819cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:12.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:12.216+0000 7fb37d42b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb378111f90 con 0x7fb378102790 2024-09-12T02:56:12.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:12.216+0000 7fb37d42b640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fb378111e10 con 0x7fb378101de0 2024-09-12T02:56:12.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:12.216+0000 7fb37d42b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb378112110 con 0x7fb378106f10 2024-09-12T02:56:12.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:12.216+0000 7fb3767fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb378102790 0x7fb378198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:12.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:12.216+0000 7fb3767fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb378102790 0x7fb378198950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47106/0 (socket says 172.21.15.5:47106) 2024-09-12T02:56:12.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:12.216+0000 7fb376ffd640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb378101de0 0x7fb378198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:12.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:12.216+0000 7fb3767fc640 1 -- 172.21.15.5:0/2409340765 learned_addr learned my addr 172.21.15.5:0/2409340765 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:56:12.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:12.216+0000 7fb3767fc640 1 -- 172.21.15.5:0/2409340765 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb378106f10 msgr2=0x7fb37819cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:12.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:12.216+0000 7fb3777fe640 1 --2- 172.21.15.5:0/2409340765 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb378106f10 0x7fb37819cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:12.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:12.216+0000 7fb3767fc640 1 --2- 172.21.15.5:0/2409340765 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb378106f10 0x7fb37819cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:12.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:12.216+0000 7fb3767fc640 1 -- 172.21.15.5:0/2409340765 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb378101de0 msgr2=0x7fb378198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:12.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:12.216+0000 7fb3767fc640 1 --2- 172.21.15.5:0/2409340765 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb378101de0 0x7fb378198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:12.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:12.216+0000 7fb3767fc640 1 -- 172.21.15.5:0/2409340765 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb37819d3c0 con 0x7fb378102790 2024-09-12T02:56:12.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:12.216+0000 7fb3777fe640 1 --2- 172.21.15.5:0/2409340765 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb378106f10 0x7fb37819cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:56:12.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:12.216+0000 7fb376ffd640 1 --2- 172.21.15.5:0/2409340765 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb378101de0 0x7fb378198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:56:12.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:12.216+0000 7fb3767fc640 1 --2- 172.21.15.5:0/2409340765 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb378102790 0x7fb378198950 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fb36c00b4a0 tx=0x7fb36c00b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:12.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:12.217+0000 7fb35ffff640 1 -- 172.21.15.5:0/2409340765 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb36c015020 con 0x7fb378102790 2024-09-12T02:56:12.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:12.217+0000 7fb35ffff640 1 -- 172.21.15.5:0/2409340765 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb36c011d10 con 0x7fb378102790 2024-09-12T02:56:12.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:12.217+0000 7fb35ffff640 1 -- 172.21.15.5:0/2409340765 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb36c00f2e0 con 0x7fb378102790 2024-09-12T02:56:12.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:12.217+0000 7fb37d42b640 1 -- 172.21.15.5:0/2409340765 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb37819d6b0 con 0x7fb378102790 2024-09-12T02:56:12.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:12.217+0000 7fb37d42b640 1 -- 172.21.15.5:0/2409340765 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3781a4f90 con 0x7fb378102790 2024-09-12T02:56:12.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:12.218+0000 7fb37d42b640 1 -- 172.21.15.5:0/2409340765 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3781021c0 con 0x7fb378102790 2024-09-12T02:56:12.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:12.219+0000 7fb35ffff640 1 -- 172.21.15.5:0/2409340765 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fb36c018020 con 0x7fb378102790 2024-09-12T02:56:12.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:12.220+0000 7fb35ffff640 1 --2- 172.21.15.5:0/2409340765 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fb3600a2b80 0x7fb3600a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:12.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:12.220+0000 7fb376ffd640 1 --2- 172.21.15.5:0/2409340765 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fb3600a2b80 0x7fb3600a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:12.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:12.220+0000 7fb35ffff640 1 -- 172.21.15.5:0/2409340765 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7fb36c0ce510 con 0x7fb378102790 2024-09-12T02:56:12.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:12.220+0000 7fb376ffd640 1 --2- 172.21.15.5:0/2409340765 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fb3600a2b80 0x7fb3600a5040 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb36800afb0 tx=0x7fb36800a2d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:12.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:12.224+0000 7fb35ffff640 1 -- 172.21.15.5:0/2409340765 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb36c01f020 con 0x7fb378102790 2024-09-12T02:56:12.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:12.402+0000 7fb37d42b640 1 -- 172.21.15.5:0/2409340765 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fb3781a5160 con 0x7fb378102790 2024-09-12T02:56:12.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:12.907+0000 7fb35ffff640 1 -- 172.21.15.5:0/2409340765 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v77) ==== 138+0+0 (secure 0 0 0) 0x7fb36c09b870 con 0x7fb378102790 2024-09-12T02:56:12.908 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_metadata' created 2024-09-12T02:56:12.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:12.910+0000 7fb37d42b640 1 -- 172.21.15.5:0/2409340765 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fb3600a2b80 msgr2=0x7fb3600a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:12.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:12.910+0000 7fb37d42b640 1 --2- 172.21.15.5:0/2409340765 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fb3600a2b80 0x7fb3600a5040 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb36800afb0 tx=0x7fb36800a2d0 comp rx=0 tx=0).stop 2024-09-12T02:56:12.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:12.910+0000 7fb37d42b640 1 -- 172.21.15.5:0/2409340765 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb378102790 msgr2=0x7fb378198950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:12.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:12.910+0000 7fb37d42b640 1 --2- 172.21.15.5:0/2409340765 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb378102790 0x7fb378198950 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fb36c00b4a0 tx=0x7fb36c00b8a0 comp rx=0 tx=0).stop 2024-09-12T02:56:12.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:12.910+0000 7fb37d42b640 1 -- 172.21.15.5:0/2409340765 shutdown_connections 2024-09-12T02:56:12.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:12.910+0000 7fb37d42b640 1 --2- 172.21.15.5:0/2409340765 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fb3600a2b80 0x7fb3600a5040 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:12.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:12.910+0000 7fb37d42b640 1 --2- 172.21.15.5:0/2409340765 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb378102790 0x7fb378198950 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:12.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:12.910+0000 7fb37d42b640 1 --2- 172.21.15.5:0/2409340765 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb378106f10 0x7fb37819cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:12.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:12.910+0000 7fb37d42b640 1 --2- 172.21.15.5:0/2409340765 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb378101de0 0x7fb378198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:12.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:12.910+0000 7fb37d42b640 1 -- 172.21.15.5:0/2409340765 >> 172.21.15.5:0/2409340765 conn(0x7fb3780fd8c0 msgr2=0x7fb3780ffcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:12.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:12.910+0000 7fb37d42b640 1 -- 172.21.15.5:0/2409340765 shutdown_connections 2024-09-12T02:56:12.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:12.910+0000 7fb37d42b640 1 -- 172.21.15.5:0/2409340765 wait complete. 2024-09-12T02:56:13.165 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:12 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:56:13.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:12 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:56:13.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:12 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:56:13.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:12 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:56:13.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:12 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:56:13.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:12 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/2409340765' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-12T02:56:13.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:12 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:56:13.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:12 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:56:13.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:12 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:56:13.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:12 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:56:13.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:12 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:56:13.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:12 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/2409340765' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-12T02:56:13.274 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:12 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:56:13.274 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:12 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:56:13.274 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:12 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:56:13.274 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:12 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:56:13.274 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:12 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:56:13.274 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:12 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/2409340765' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-12T02:56:13.604 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-12T02:56:13.863 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:56:14.102 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:13 smithi005 ceph-mon[31197]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:56:14.102 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:13 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/2409340765' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-12T02:56:14.102 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:13 smithi005 ceph-mon[31197]: osdmap e77: 12 total, 12 up, 12 in 2024-09-12T02:56:14.165 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:13 smithi148 ceph-mon[35090]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:56:14.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:13 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/2409340765' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-12T02:56:14.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:13 smithi148 ceph-mon[35090]: osdmap e77: 12 total, 12 up, 12 in 2024-09-12T02:56:14.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:13 smithi146 ceph-mon[35011]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:56:14.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:13 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/2409340765' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-12T02:56:14.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:13 smithi146 ceph-mon[35011]: osdmap e77: 12 total, 12 up, 12 in 2024-09-12T02:56:14.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:14.335+0000 7f7914f3e640 1 -- 172.21.15.5:0/3514988077 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7910100a80 msgr2=0x7f7910100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:14.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:14.335+0000 7f7914f3e640 1 --2- 172.21.15.5:0/3514988077 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7910100a80 0x7f7910100ee0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7904009930 tx=0x7f790402a420 comp rx=0 tx=0).stop 2024-09-12T02:56:14.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:14.336+0000 7f7914f3e640 1 -- 172.21.15.5:0/3514988077 shutdown_connections 2024-09-12T02:56:14.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:14.336+0000 7f7914f3e640 1 --2- 172.21.15.5:0/3514988077 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7910100a80 0x7f7910100ee0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:14.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:14.336+0000 7f7914f3e640 1 --2- 172.21.15.5:0/3514988077 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7910101420 0x7f791010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:14.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:14.336+0000 7f7914f3e640 1 --2- 172.21.15.5:0/3514988077 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7910106a80 0x7f7910106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:14.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:14.336+0000 7f7914f3e640 1 -- 172.21.15.5:0/3514988077 >> 172.21.15.5:0/3514988077 conn(0x7f79100fc7a0 msgr2=0x7f79100febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:14.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:14.336+0000 7f7914f3e640 1 -- 172.21.15.5:0/3514988077 shutdown_connections 2024-09-12T02:56:14.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:14.336+0000 7f7914f3e640 1 -- 172.21.15.5:0/3514988077 wait complete. 2024-09-12T02:56:14.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:14.336+0000 7f7914f3e640 1 Processor -- start 2024-09-12T02:56:14.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:14.336+0000 7f7914f3e640 1 -- start start 2024-09-12T02:56:14.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:14.337+0000 7f7914f3e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7910100a80 0x7f79100fccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:14.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:14.337+0000 7f7914f3e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7910101420 0x7f79100fd230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:14.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:14.337+0000 7f7914f3e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7910106a80 0x7f79100fd770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:14.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:14.337+0000 7f7914f3e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7910110ef0 con 0x7f7910106a80 2024-09-12T02:56:14.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:14.337+0000 7f7914f3e640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f7910110d70 con 0x7f7910100a80 2024-09-12T02:56:14.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:14.337+0000 7f7914f3e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f7910111070 con 0x7f7910101420 2024-09-12T02:56:14.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:14.337+0000 7f790ed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7910106a80 0x7f79100fd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:14.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:14.337+0000 7f790ed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7910106a80 0x7f79100fd770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58376/0 (socket says 172.21.15.5:58376) 2024-09-12T02:56:14.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:14.337+0000 7f790ed9d640 1 -- 172.21.15.5:0/786024977 learned_addr learned my addr 172.21.15.5:0/786024977 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:56:14.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:14.337+0000 7f790e59c640 1 --2- 172.21.15.5:0/786024977 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7910100a80 0x7f79100fccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:14.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:14.337+0000 7f790dd9b640 1 --2- 172.21.15.5:0/786024977 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7910101420 0x7f79100fd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:14.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:14.337+0000 7f790ed9d640 1 -- 172.21.15.5:0/786024977 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7910101420 msgr2=0x7f79100fd230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:14.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:14.337+0000 7f790ed9d640 1 --2- 172.21.15.5:0/786024977 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7910101420 0x7f79100fd230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:14.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:14.337+0000 7f790ed9d640 1 -- 172.21.15.5:0/786024977 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7910100a80 msgr2=0x7f79100fccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:14.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:14.337+0000 7f790ed9d640 1 --2- 172.21.15.5:0/786024977 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7910100a80 0x7f79100fccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:14.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:14.337+0000 7f790ed9d640 1 -- 172.21.15.5:0/786024977 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79101a69d0 con 0x7f7910106a80 2024-09-12T02:56:14.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:14.337+0000 7f790dd9b640 1 --2- 172.21.15.5:0/786024977 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7910101420 0x7f79100fd230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:56:14.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:14.337+0000 7f790ed9d640 1 --2- 172.21.15.5:0/786024977 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7910106a80 0x7f79100fd770 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f790800cce0 tx=0x7f790800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:14.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:14.338+0000 7f78ff7fe640 1 -- 172.21.15.5:0/786024977 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f790800f450 con 0x7f7910106a80 2024-09-12T02:56:14.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:14.338+0000 7f7914f3e640 1 -- 172.21.15.5:0/786024977 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79101a6cc0 con 0x7f7910106a80 2024-09-12T02:56:14.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:14.338+0000 7f78ff7fe640 1 -- 172.21.15.5:0/786024977 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f790800fad0 con 0x7f7910106a80 2024-09-12T02:56:14.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:14.338+0000 7f78ff7fe640 1 -- 172.21.15.5:0/786024977 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7908018b60 con 0x7f7910106a80 2024-09-12T02:56:14.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:14.338+0000 7f7914f3e640 1 -- 172.21.15.5:0/786024977 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79101a7180 con 0x7f7910106a80 2024-09-12T02:56:14.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:14.339+0000 7f7914f3e640 1 -- 172.21.15.5:0/786024977 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7910100ee0 con 0x7f7910106a80 2024-09-12T02:56:14.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:14.340+0000 7f78ff7fe640 1 -- 172.21.15.5:0/786024977 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f790800fc80 con 0x7f7910106a80 2024-09-12T02:56:14.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:14.341+0000 7f78ff7fe640 1 --2- 172.21.15.5:0/786024977 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f78f80a2b80 0x7f78f80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:14.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:14.341+0000 7f790e59c640 1 --2- 172.21.15.5:0/786024977 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f78f80a2b80 0x7f78f80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:14.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:14.341+0000 7f78ff7fe640 1 -- 172.21.15.5:0/786024977 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7497+0+0 (secure 0 0 0) 0x7f79080cf6d0 con 0x7f7910106a80 2024-09-12T02:56:14.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:14.341+0000 7f790e59c640 1 --2- 172.21.15.5:0/786024977 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f78f80a2b80 0x7f78f80a5040 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7900009990 tx=0x7f7900009110 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:14.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:14.344+0000 7f78ff7fe640 1 -- 172.21.15.5:0/786024977 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7908014030 con 0x7f7910106a80 2024-09-12T02:56:14.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:14.519+0000 7f7914f3e640 1 -- 172.21.15.5:0/786024977 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f79100fe220 con 0x7f7910106a80 2024-09-12T02:56:14.957 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:14.956+0000 7f78ff7fe640 1 -- 172.21.15.5:0/786024977 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v79) ==== 170+0+0 (secure 0 0 0) 0x7f79100fe220 con 0x7f7910106a80 2024-09-12T02:56:14.957 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_data' created 2024-09-12T02:56:14.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:14.959+0000 7f7914f3e640 1 -- 172.21.15.5:0/786024977 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f78f80a2b80 msgr2=0x7f78f80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:14.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:14.959+0000 7f7914f3e640 1 --2- 172.21.15.5:0/786024977 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f78f80a2b80 0x7f78f80a5040 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7900009990 tx=0x7f7900009110 comp rx=0 tx=0).stop 2024-09-12T02:56:14.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:14.959+0000 7f7914f3e640 1 -- 172.21.15.5:0/786024977 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7910106a80 msgr2=0x7f79100fd770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:14.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:14.959+0000 7f7914f3e640 1 --2- 172.21.15.5:0/786024977 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7910106a80 0x7f79100fd770 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f790800cce0 tx=0x7f790800a620 comp rx=0 tx=0).stop 2024-09-12T02:56:14.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:14.959+0000 7f7914f3e640 1 -- 172.21.15.5:0/786024977 shutdown_connections 2024-09-12T02:56:14.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:14.959+0000 7f7914f3e640 1 --2- 172.21.15.5:0/786024977 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f78f80a2b80 0x7f78f80a5040 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:14.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:14.959+0000 7f7914f3e640 1 --2- 172.21.15.5:0/786024977 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7910106a80 0x7f79100fd770 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:14.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:14.959+0000 7f7914f3e640 1 --2- 172.21.15.5:0/786024977 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7910101420 0x7f79100fd230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:14.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:14.959+0000 7f7914f3e640 1 --2- 172.21.15.5:0/786024977 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7910100a80 0x7f79100fccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:14.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:14.959+0000 7f7914f3e640 1 -- 172.21.15.5:0/786024977 >> 172.21.15.5:0/786024977 conn(0x7f79100fc7a0 msgr2=0x7f791006aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:14.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:14.959+0000 7f7914f3e640 1 -- 172.21.15.5:0/786024977 shutdown_connections 2024-09-12T02:56:14.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:14.959+0000 7f7914f3e640 1 -- 172.21.15.5:0/786024977 wait complete. 2024-09-12T02:56:15.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:14 smithi146 ceph-mon[35011]: pgmap v269: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:56:15.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:14 smithi146 ceph-mon[35011]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-12T02:56:15.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:14 smithi146 ceph-mon[35011]: osdmap e78: 12 total, 12 up, 12 in 2024-09-12T02:56:15.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:14 smithi005 ceph-mon[31197]: pgmap v269: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:56:15.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:14 smithi005 ceph-mon[31197]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-12T02:56:15.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:14 smithi005 ceph-mon[31197]: osdmap e78: 12 total, 12 up, 12 in 2024-09-12T02:56:15.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:14 smithi148 ceph-mon[35090]: pgmap v269: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:56:15.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:14 smithi148 ceph-mon[35090]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-12T02:56:15.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:14 smithi148 ceph-mon[35090]: osdmap e78: 12 total, 12 up, 12 in 2024-09-12T02:56:15.622 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-12T02:56:15.882 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:56:16.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:15 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/786024977' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-12T02:56:16.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:15 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/786024977' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-12T02:56:16.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:15 smithi146 ceph-mon[35011]: osdmap e79: 12 total, 12 up, 12 in 2024-09-12T02:56:16.286 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:15 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/786024977' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-12T02:56:16.286 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:15 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/786024977' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-12T02:56:16.286 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:15 smithi005 ceph-mon[31197]: osdmap e79: 12 total, 12 up, 12 in 2024-09-12T02:56:16.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:16.371+0000 7fa5aa700640 1 -- 172.21.15.5:0/3998259155 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5a4101280 msgr2=0x7fa5a410df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:16.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:16.371+0000 7fa5aa700640 1 --2- 172.21.15.5:0/3998259155 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5a4101280 0x7fa5a410df40 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa598009900 tx=0x7fa59802a410 comp rx=0 tx=0).stop 2024-09-12T02:56:16.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:16.371+0000 7fa5aa700640 1 -- 172.21.15.5:0/3998259155 shutdown_connections 2024-09-12T02:56:16.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:16.371+0000 7fa5aa700640 1 --2- 172.21.15.5:0/3998259155 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5a4101280 0x7fa5a410df40 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:16.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:16.371+0000 7fa5aa700640 1 --2- 172.21.15.5:0/3998259155 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5a410e480 0x7fa5a4110870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:16.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:16.371+0000 7fa5aa700640 1 --2- 172.21.15.5:0/3998259155 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa5a41008d0 0x7fa5a4100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:16.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:16.371+0000 7fa5aa700640 1 -- 172.21.15.5:0/3998259155 >> 172.21.15.5:0/3998259155 conn(0x7fa5a40fc7a0 msgr2=0x7fa5a40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:16.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:16.371+0000 7fa5aa700640 1 -- 172.21.15.5:0/3998259155 shutdown_connections 2024-09-12T02:56:16.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:16.371+0000 7fa5aa700640 1 -- 172.21.15.5:0/3998259155 wait complete. 2024-09-12T02:56:16.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:16.372+0000 7fa5aa700640 1 Processor -- start 2024-09-12T02:56:16.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:16.372+0000 7fa5aa700640 1 -- start start 2024-09-12T02:56:16.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:16.373+0000 7fa5aa700640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5a41008d0 0x7fa5a419a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:16.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:16.373+0000 7fa5aa700640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa5a4101280 0x7fa5a419ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:16.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:16.373+0000 7fa5aa700640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5a410e480 0x7fa5a419efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:16.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:16.373+0000 7fa5aa700640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a4114230 con 0x7fa5a41008d0 2024-09-12T02:56:16.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:16.373+0000 7fa5aa700640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a41140b0 con 0x7fa5a4101280 2024-09-12T02:56:16.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:16.373+0000 7fa5aa700640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a41143b0 con 0x7fa5a410e480 2024-09-12T02:56:16.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:16.373+0000 7fa5a3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5a41008d0 0x7fa5a419a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:16.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:16.373+0000 7fa5a3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5a41008d0 0x7fa5a419a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58392/0 (socket says 172.21.15.5:58392) 2024-09-12T02:56:16.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:16.373+0000 7fa5a3fff640 1 -- 172.21.15.5:0/1500954631 learned_addr learned my addr 172.21.15.5:0/1500954631 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:56:16.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:16.373+0000 7fa5a37fe640 1 --2- 172.21.15.5:0/1500954631 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa5a4101280 0x7fa5a419ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:16.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:16.373+0000 7fa5a3fff640 1 -- 172.21.15.5:0/1500954631 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5a410e480 msgr2=0x7fa5a419efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:16.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:16.373+0000 7fa5a3fff640 1 --2- 172.21.15.5:0/1500954631 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5a410e480 0x7fa5a419efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:16.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:16.373+0000 7fa5a3fff640 1 -- 172.21.15.5:0/1500954631 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa5a4101280 msgr2=0x7fa5a419ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:16.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:16.373+0000 7fa5a3fff640 1 --2- 172.21.15.5:0/1500954631 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa5a4101280 0x7fa5a419ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:16.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:16.373+0000 7fa5a3fff640 1 -- 172.21.15.5:0/1500954631 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5a419f6c0 con 0x7fa5a41008d0 2024-09-12T02:56:16.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:16.373+0000 7fa5a3fff640 1 --2- 172.21.15.5:0/1500954631 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5a41008d0 0x7fa5a419a710 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa59400d950 tx=0x7fa59400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:16.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:16.374+0000 7fa5a17fa640 1 -- 172.21.15.5:0/1500954631 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa594017070 con 0x7fa5a41008d0 2024-09-12T02:56:16.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:16.374+0000 7fa5aa700640 1 -- 172.21.15.5:0/1500954631 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5a419f9b0 con 0x7fa5a41008d0 2024-09-12T02:56:16.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:16.374+0000 7fa5a17fa640 1 -- 172.21.15.5:0/1500954631 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa594009c30 con 0x7fa5a41008d0 2024-09-12T02:56:16.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:16.374+0000 7fa5a17fa640 1 -- 172.21.15.5:0/1500954631 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa59400f6b0 con 0x7fa5a41008d0 2024-09-12T02:56:16.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:16.374+0000 7fa5aa700640 1 -- 172.21.15.5:0/1500954631 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5a41a7290 con 0x7fa5a41008d0 2024-09-12T02:56:16.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:16.375+0000 7fa5aa700640 1 -- 172.21.15.5:0/1500954631 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5a4100cb0 con 0x7fa5a41008d0 2024-09-12T02:56:16.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:16.376+0000 7fa5a17fa640 1 -- 172.21.15.5:0/1500954631 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fa5940109d0 con 0x7fa5a41008d0 2024-09-12T02:56:16.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:16.377+0000 7fa5a17fa640 1 --2- 172.21.15.5:0/1500954631 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fa58c0a2b80 0x7fa58c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:16.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:16.377+0000 7fa5a37fe640 1 --2- 172.21.15.5:0/1500954631 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fa58c0a2b80 0x7fa58c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:16.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:16.377+0000 7fa5a17fa640 1 -- 172.21.15.5:0/1500954631 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7879+0+0 (secure 0 0 0) 0x7fa5940ceac0 con 0x7fa5a41008d0 2024-09-12T02:56:16.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:16.377+0000 7fa5a37fe640 1 --2- 172.21.15.5:0/1500954631 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fa58c0a2b80 0x7fa58c0a5040 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa598006010 tx=0x7fa598005a40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:16.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:16.380+0000 7fa5a17fa640 1 -- 172.21.15.5:0/1500954631 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa594009de0 con 0x7fa5a41008d0 2024-09-12T02:56:16.415 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:15 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/786024977' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-12T02:56:16.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:15 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/786024977' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-12T02:56:16.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:15 smithi148 ceph-mon[35090]: osdmap e79: 12 total, 12 up, 12 in 2024-09-12T02:56:16.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:16.594+0000 7fa5aa700640 1 -- 172.21.15.5:0/1500954631 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fa5a402d7c0 con 0x7fa5a41008d0 2024-09-12T02:56:16.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:16.969+0000 7fa5a17fa640 1 -- 172.21.15.5:0/1500954631 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7fa59409bbd0 con 0x7fa5a41008d0 2024-09-12T02:56:16.970 INFO:teuthology.orchestra.run.smithi005.stderr:new fs with metadata pool 2 and data pool 3 2024-09-12T02:56:16.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:16.972+0000 7fa5aa700640 1 -- 172.21.15.5:0/1500954631 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fa58c0a2b80 msgr2=0x7fa58c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:16.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:16.972+0000 7fa5aa700640 1 --2- 172.21.15.5:0/1500954631 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fa58c0a2b80 0x7fa58c0a5040 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa598006010 tx=0x7fa598005a40 comp rx=0 tx=0).stop 2024-09-12T02:56:16.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:16.972+0000 7fa5aa700640 1 -- 172.21.15.5:0/1500954631 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5a41008d0 msgr2=0x7fa5a419a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:16.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:16.972+0000 7fa5aa700640 1 --2- 172.21.15.5:0/1500954631 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5a41008d0 0x7fa5a419a710 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fa59400d950 tx=0x7fa59400de10 comp rx=0 tx=0).stop 2024-09-12T02:56:16.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:16.972+0000 7fa5aa700640 1 -- 172.21.15.5:0/1500954631 shutdown_connections 2024-09-12T02:56:16.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:16.972+0000 7fa5aa700640 1 --2- 172.21.15.5:0/1500954631 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5a410e480 0x7fa5a419efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:16.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:16.972+0000 7fa5aa700640 1 --2- 172.21.15.5:0/1500954631 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa5a4101280 0x7fa5a419ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:16.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:16.972+0000 7fa5aa700640 1 --2- 172.21.15.5:0/1500954631 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fa58c0a2b80 0x7fa58c0a5040 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:16.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:16.972+0000 7fa5aa700640 1 --2- 172.21.15.5:0/1500954631 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5a41008d0 0x7fa5a419a710 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:16.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:16.972+0000 7fa5aa700640 1 -- 172.21.15.5:0/1500954631 >> 172.21.15.5:0/1500954631 conn(0x7fa5a40fc7a0 msgr2=0x7fa5a40feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:16.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:16.972+0000 7fa5aa700640 1 -- 172.21.15.5:0/1500954631 shutdown_connections 2024-09-12T02:56:16.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:16.972+0000 7fa5aa700640 1 -- 172.21.15.5:0/1500954631 wait complete. 2024-09-12T02:56:17.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:16 smithi005 ceph-mon[31197]: pgmap v272: 97 pgs: 6 creating+peering, 16 active+clean, 75 unknown; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:56:17.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:16 smithi005 ceph-mon[31197]: osdmap e80: 12 total, 12 up, 12 in 2024-09-12T02:56:17.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:16 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/1500954631' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-12T02:56:17.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:16 smithi005 ceph-mon[31197]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-09-12T02:56:17.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:16 smithi148 ceph-mon[35090]: pgmap v272: 97 pgs: 6 creating+peering, 16 active+clean, 75 unknown; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:56:17.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:16 smithi148 ceph-mon[35090]: osdmap e80: 12 total, 12 up, 12 in 2024-09-12T02:56:17.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:16 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/1500954631' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-12T02:56:17.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:16 smithi148 ceph-mon[35090]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-09-12T02:56:17.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:16 smithi146 ceph-mon[35011]: pgmap v272: 97 pgs: 6 creating+peering, 16 active+clean, 75 unknown; 577 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T02:56:17.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:16 smithi146 ceph-mon[35011]: osdmap e80: 12 total, 12 up, 12 in 2024-09-12T02:56:17.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:16 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/1500954631' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-12T02:56:17.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:16 smithi146 ceph-mon[35011]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-09-12T02:56:17.521 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T02:56:17.778 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:56:18.160 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:17 smithi005 ceph-mon[31197]: osdmap e81: 12 total, 12 up, 12 in 2024-09-12T02:56:18.161 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:17 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/1500954631' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-12T02:56:18.161 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:17 smithi005 ceph-mon[31197]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-09-12T02:56:18.161 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:17 smithi005 ceph-mon[31197]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-09-12T02:56:18.161 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:17 smithi005 ceph-mon[31197]: mds.? [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] up:active 2024-09-12T02:56:18.161 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:17 smithi005 ceph-mon[31197]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-12T02:56:18.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:18.242+0000 7f10d4251640 1 -- 172.21.15.5:0/1731252976 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10cc106ba0 msgr2=0x7f10cc106f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:18.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:18.242+0000 7f10d4251640 1 --2- 172.21.15.5:0/1731252976 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10cc106ba0 0x7f10cc106f80 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f10c8009ca0 tx=0x7f10c802a8c0 comp rx=0 tx=0).stop 2024-09-12T02:56:18.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:18.242+0000 7f10d4251640 1 -- 172.21.15.5:0/1731252976 shutdown_connections 2024-09-12T02:56:18.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:18.242+0000 7f10d4251640 1 --2- 172.21.15.5:0/1731252976 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10cc100a70 0x7f10cc10ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:18.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:18.242+0000 7f10d4251640 1 --2- 172.21.15.5:0/1731252976 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f10cc1000d0 0x7f10cc100530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:18.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:18.242+0000 7f10d4251640 1 --2- 172.21.15.5:0/1731252976 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10cc106ba0 0x7f10cc106f80 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:18.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:18.242+0000 7f10d4251640 1 -- 172.21.15.5:0/1731252976 >> 172.21.15.5:0/1731252976 conn(0x7f10cc077210 msgr2=0x7f10cc0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:18.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:18.242+0000 7f10d4251640 1 -- 172.21.15.5:0/1731252976 shutdown_connections 2024-09-12T02:56:18.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:18.242+0000 7f10d4251640 1 -- 172.21.15.5:0/1731252976 wait complete. 2024-09-12T02:56:18.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:18.243+0000 7f10d4251640 1 Processor -- start 2024-09-12T02:56:18.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:18.243+0000 7f10d4251640 1 -- start start 2024-09-12T02:56:18.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:18.243+0000 7f10d4251640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10cc1000d0 0x7f10cc1984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:18.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:18.243+0000 7f10d4251640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10cc100a70 0x7f10cc198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:18.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:18.243+0000 7f10d4251640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f10cc106ba0 0x7f10cc19cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:18.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:18.243+0000 7f10d4251640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f10cc112090 con 0x7f10cc1000d0 2024-09-12T02:56:18.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:18.243+0000 7f10d4251640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f10cc111f10 con 0x7f10cc106ba0 2024-09-12T02:56:18.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:18.243+0000 7f10d4251640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f10cc112210 con 0x7f10cc100a70 2024-09-12T02:56:18.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:18.243+0000 7f10d1fed640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10cc1000d0 0x7f10cc1984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:18.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:18.243+0000 7f10d1fed640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10cc1000d0 0x7f10cc1984d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58408/0 (socket says 172.21.15.5:58408) 2024-09-12T02:56:18.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:18.243+0000 7f10d1fed640 1 -- 172.21.15.5:0/2778001905 learned_addr learned my addr 172.21.15.5:0/2778001905 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:56:18.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:18.243+0000 7f10d27ee640 1 --2- 172.21.15.5:0/2778001905 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f10cc106ba0 0x7f10cc19cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:18.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:18.244+0000 7f10d1fed640 1 -- 172.21.15.5:0/2778001905 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10cc100a70 msgr2=0x7f10cc198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:18.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:18.244+0000 7f10d17ec640 1 --2- 172.21.15.5:0/2778001905 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10cc100a70 0x7f10cc198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:18.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:18.244+0000 7f10d1fed640 1 --2- 172.21.15.5:0/2778001905 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10cc100a70 0x7f10cc198a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:18.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:18.244+0000 7f10d1fed640 1 -- 172.21.15.5:0/2778001905 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f10cc106ba0 msgr2=0x7f10cc19cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:18.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:18.244+0000 7f10d1fed640 1 --2- 172.21.15.5:0/2778001905 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f10cc106ba0 0x7f10cc19cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:18.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:18.244+0000 7f10d1fed640 1 -- 172.21.15.5:0/2778001905 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10cc19d480 con 0x7f10cc1000d0 2024-09-12T02:56:18.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:18.244+0000 7f10d27ee640 1 --2- 172.21.15.5:0/2778001905 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f10cc106ba0 0x7f10cc19cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:56:18.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:18.244+0000 7f10d1fed640 1 --2- 172.21.15.5:0/2778001905 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10cc1000d0 0x7f10cc1984d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f10c8006010 tx=0x7f10c8003f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:18.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:18.244+0000 7f10beffd640 1 -- 172.21.15.5:0/2778001905 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c80317d0 con 0x7f10cc1000d0 2024-09-12T02:56:18.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:18.244+0000 7f10beffd640 1 -- 172.21.15.5:0/2778001905 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10c8031970 con 0x7f10cc1000d0 2024-09-12T02:56:18.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:18.244+0000 7f10beffd640 1 -- 172.21.15.5:0/2778001905 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10c8030d70 con 0x7f10cc1000d0 2024-09-12T02:56:18.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:18.244+0000 7f10d4251640 1 -- 172.21.15.5:0/2778001905 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10cc19d710 con 0x7f10cc1000d0 2024-09-12T02:56:18.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:18.244+0000 7f10d4251640 1 -- 172.21.15.5:0/2778001905 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10cc1a5050 con 0x7f10cc1000d0 2024-09-12T02:56:18.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:18.245+0000 7f10d4251640 1 -- 172.21.15.5:0/2778001905 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10cc100530 con 0x7f10cc1000d0 2024-09-12T02:56:18.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:18.247+0000 7f10beffd640 1 -- 172.21.15.5:0/2778001905 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f10c8004b40 con 0x7f10cc1000d0 2024-09-12T02:56:18.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:18.248+0000 7f10beffd640 1 --2- 172.21.15.5:0/2778001905 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f10b80a2c00 0x7f10b80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:18.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:18.248+0000 7f10d17ec640 1 --2- 172.21.15.5:0/2778001905 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f10b80a2c00 0x7f10b80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:18.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:18.248+0000 7f10beffd640 1 -- 172.21.15.5:0/2778001905 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f10c80f06a0 con 0x7f10cc1000d0 2024-09-12T02:56:18.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:18.248+0000 7f10d17ec640 1 --2- 172.21.15.5:0/2778001905 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f10b80a2c00 0x7f10b80a50c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f10cc0ff7a0 tx=0x7f10c0005ca0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:18.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:18.251+0000 7f10beffd640 1 -- 172.21.15.5:0/2778001905 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f10c80bd750 con 0x7f10cc1000d0 2024-09-12T02:56:18.319 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:17 smithi148 ceph-mon[35090]: osdmap e81: 12 total, 12 up, 12 in 2024-09-12T02:56:18.319 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:17 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/1500954631' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-12T02:56:18.319 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:17 smithi148 ceph-mon[35090]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-09-12T02:56:18.319 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:17 smithi148 ceph-mon[35090]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-09-12T02:56:18.319 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:17 smithi148 ceph-mon[35090]: mds.? [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] up:active 2024-09-12T02:56:18.320 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:17 smithi148 ceph-mon[35090]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-12T02:56:18.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:18.427+0000 7f10d4251640 1 -- 172.21.15.5:0/2778001905 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f10cc10a090 con 0x7f10cc1000d0 2024-09-12T02:56:18.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:18.428+0000 7f10beffd640 1 -- 172.21.15.5:0/2778001905 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+22223 (secure 0 0 0) 0x7f10c80bd050 con 0x7f10cc1000d0 2024-09-12T02:56:18.429 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-12T02:56:18.430 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":81,"fsid":"618305f8-70b1-11ef-bcea-c7b262605968","created":"2024-09-12T02:48:42.352146+0000","modified":"2024-09-12T02:56:16.595365+0000","last_up_change":"2024-09-12T02:55:27.817161+0000","last_in_change":"2024-09-12T02:55:03.745281+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T02:51:47.970639+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T02:56:12.834271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T02:56:14.951437+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17da560a-6aa2-4f2c-a95f-67481dce4d3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":4248507573},{"type":"v1","addr":"172.21.15.5:6803","nonce":4248507573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":4248507573},{"type":"v1","addr":"172.21.15.5:6805","nonce":4248507573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":4248507573},{"type":"v1","addr":"172.21.15.5:6809","nonce":4248507573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":4248507573},{"type":"v1","addr":"172.21.15.5:6807","nonce":4248507573}]},"public_addr":"172.21.15.5:6803/4248507573","cluster_addr":"172.21.15.5:6805/4248507573","heartbeat_back_addr":"172.21.15.5:6809/4248507573","heartbeat_front_addr":"172.21.15.5:6807/4248507573","state":["exists","up"]},{"osd":1,"uuid":"a3237006-99af-4fc1-860c-d4917032647f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":1105990287},{"type":"v1","addr":"172.21.15.146:6801","nonce":1105990287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":1105990287},{"type":"v1","addr":"172.21.15.146:6803","nonce":1105990287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":1105990287},{"type":"v1","addr":"172.21.15.146:6807","nonce":1105990287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":1105990287},{"type":"v1","addr":"172.21.15.146:6805","nonce":1105990287}]},"public_addr":"172.21.15.146:6801/1105990287","cluster_addr":"172.21.15.146:6803/1105990287","heartbeat_back_addr":"172.21.15.146:6807/1105990287","heartbeat_front_addr":"172.21.15.146:6805/1105990287","state":["exists","up"]},{"osd":2,"uuid":"ea3ab84a-7739-4975-8244-31ddd5aad80d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3378174313},{"type":"v1","addr":"172.21.15.148:6801","nonce":3378174313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3378174313},{"type":"v1","addr":"172.21.15.148:6803","nonce":3378174313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3378174313},{"type":"v1","addr":"172.21.15.148:6807","nonce":3378174313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3378174313},{"type":"v1","addr":"172.21.15.148:6805","nonce":3378174313}]},"public_addr":"172.21.15.148:6801/3378174313","cluster_addr":"172.21.15.148:6803/3378174313","heartbeat_back_addr":"172.21.15.148:6807/3378174313","heartbeat_front_addr":"172.21.15.148:6805/3378174313","state":["exists","up"]},{"osd":3,"uuid":"4396a9ee-f3f3-410b-9fb1-124f1e33a1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":11741938},{"type":"v1","addr":"172.21.15.5:6811","nonce":11741938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":11741938},{"type":"v1","addr":"172.21.15.5:6813","nonce":11741938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":11741938},{"type":"v1","addr":"172.21.15.5:6817","nonce":11741938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":11741938},{"type":"v1","addr":"172.21.15.5:6815","nonce":11741938}]},"public_addr":"172.21.15.5:6811/11741938","cluster_addr":"172.21.15.5:6813/11741938","heartbeat_back_addr":"172.21.15.5:6817/11741938","heartbeat_front_addr":"172.21.15.5:6815/11741938","state":["exists","up"]},{"osd":4,"uuid":"989d76e7-c742-49d0-994a-c58f2771b1d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":3274923851},{"type":"v1","addr":"172.21.15.146:6809","nonce":3274923851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":3274923851},{"type":"v1","addr":"172.21.15.146:6811","nonce":3274923851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3274923851},{"type":"v1","addr":"172.21.15.146:6815","nonce":3274923851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":3274923851},{"type":"v1","addr":"172.21.15.146:6813","nonce":3274923851}]},"public_addr":"172.21.15.146:6809/3274923851","cluster_addr":"172.21.15.146:6811/3274923851","heartbeat_back_addr":"172.21.15.146:6815/3274923851","heartbeat_front_addr":"172.21.15.146:6813/3274923851","state":["exists","up"]},{"osd":5,"uuid":"64ada7d4-70a0-4b83-9af8-032be544608d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":2771670476},{"type":"v1","addr":"172.21.15.148:6809","nonce":2771670476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":2771670476},{"type":"v1","addr":"172.21.15.148:6811","nonce":2771670476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":2771670476},{"type":"v1","addr":"172.21.15.148:6815","nonce":2771670476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":2771670476},{"type":"v1","addr":"172.21.15.148:6813","nonce":2771670476}]},"public_addr":"172.21.15.148:6809/2771670476","cluster_addr":"172.21.15.148:6811/2771670476","heartbeat_back_addr":"172.21.15.148:6815/2771670476","heartbeat_front_addr":"172.21.15.148:6813/2771670476","state":["exists","up"]},{"osd":6,"uuid":"a70a88fd-2e4b-480c-bb98-62636cf70cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3176073169},{"type":"v1","addr":"172.21.15.5:6819","nonce":3176073169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":3176073169},{"type":"v1","addr":"172.21.15.5:6821","nonce":3176073169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":3176073169},{"type":"v1","addr":"172.21.15.5:6825","nonce":3176073169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3176073169},{"type":"v1","addr":"172.21.15.5:6823","nonce":3176073169}]},"public_addr":"172.21.15.5:6819/3176073169","cluster_addr":"172.21.15.5:6821/3176073169","heartbeat_back_addr":"172.21.15.5:6825/3176073169","heartbeat_front_addr":"172.21.15.5:6823/3176073169","state":["exists","up"]},{"osd":7,"uuid":"49451e8d-c741-4ad5-b8f5-89bad8d4d129","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2209508838},{"type":"v1","addr":"172.21.15.146:6817","nonce":2209508838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2209508838},{"type":"v1","addr":"172.21.15.146:6819","nonce":2209508838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2209508838},{"type":"v1","addr":"172.21.15.146:6823","nonce":2209508838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2209508838},{"type":"v1","addr":"172.21.15.146:6821","nonce":2209508838}]},"public_addr":"172.21.15.146:6817/2209508838","cluster_addr":"172.21.15.146:6819/2209508838","heartbeat_back_addr":"172.21.15.146:6823/2209508838","heartbeat_front_addr":"172.21.15.146:6821/2209508838","state":["exists","up"]},{"osd":8,"uuid":"57bfc121-a75e-4a7f-b1dc-64074b178b03","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":651045756},{"type":"v1","addr":"172.21.15.148:6817","nonce":651045756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":651045756},{"type":"v1","addr":"172.21.15.148:6819","nonce":651045756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":651045756},{"type":"v1","addr":"172.21.15.148:6823","nonce":651045756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":651045756},{"type":"v1","addr":"172.21.15.148:6821","nonce":651045756}]},"public_addr":"172.21.15.148:6817/651045756","cluster_addr":"172.21.15.148:6819/651045756","heartbeat_back_addr":"172.21.15.148:6823/651045756","heartbeat_front_addr":"172.21.15.148:6821/651045756","state":["exists","up"]},{"osd":9,"uuid":"d9726168-7494-4bf8-b433-350e85bfd461","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1961857360},{"type":"v1","addr":"172.21.15.5:6827","nonce":1961857360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1961857360},{"type":"v1","addr":"172.21.15.5:6829","nonce":1961857360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":1961857360},{"type":"v1","addr":"172.21.15.5:6833","nonce":1961857360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1961857360},{"type":"v1","addr":"172.21.15.5:6831","nonce":1961857360}]},"public_addr":"172.21.15.5:6827/1961857360","cluster_addr":"172.21.15.5:6829/1961857360","heartbeat_back_addr":"172.21.15.5:6833/1961857360","heartbeat_front_addr":"172.21.15.5:6831/1961857360","state":["exists","up"]},{"osd":10,"uuid":"519ec60f-3606-48ac-9043-6f59a798766d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":2041733103},{"type":"v1","addr":"172.21.15.146:6825","nonce":2041733103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":2041733103},{"type":"v1","addr":"172.21.15.146:6827","nonce":2041733103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":2041733103},{"type":"v1","addr":"172.21.15.146:6831","nonce":2041733103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":2041733103},{"type":"v1","addr":"172.21.15.146:6829","nonce":2041733103}]},"public_addr":"172.21.15.146:6825/2041733103","cluster_addr":"172.21.15.146:6827/2041733103","heartbeat_back_addr":"172.21.15.146:6831/2041733103","heartbeat_front_addr":"172.21.15.146:6829/2041733103","state":["exists","up"]},{"osd":11,"uuid":"0f92d0cb-ba54-4bc3-a6d0-3c54d910556a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":839290773},{"type":"v1","addr":"172.21.15.148:6825","nonce":839290773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":839290773},{"type":"v1","addr":"172.21.15.148:6827","nonce":839290773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":839290773},{"type":"v1","addr":"172.21.15.148:6831","nonce":839290773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":839290773},{"type":"v1","addr":"172.21.15.148:6829","nonce":839290773}]},"public_addr":"172.21.15.148:6825/839290773","cluster_addr":"172.21.15.148:6827/839290773","heartbeat_back_addr":"172.21.15.148:6831/839290773","heartbeat_front_addr":"172.21.15.148:6829/839290773","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:50:54.101755+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:51:19.037385+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:51:44.142837+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:52:09.150887+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:52:33.705079+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:52:58.845233+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:53:23.068339+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:53:47.520398+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:54:11.965370+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:54:36.776075+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:55:01.292236+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:55:25.346589+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:6800/3125821798":"2024-09-13T02:49:29.462216+0000","172.21.15.5:0/3479911451":"2024-09-13T02:49:29.462216+0000","172.21.15.5:0/1547753207":"2024-09-13T02:49:03.300872+0000","172.21.15.5:0/1905762627":"2024-09-13T02:49:03.300872+0000","172.21.15.5:0/4142340280":"2024-09-13T02:49:29.462216+0000","172.21.15.5:6801/3125821798":"2024-09-13T02:49:29.462216+0000","172.21.15.5:6801/2595781680":"2024-09-13T02:49:03.300872+0000","172.21.15.5:0/2273691311":"2024-09-13T02:49:29.462216+0000","172.21.15.5:6800/2595781680":"2024-09-13T02:49:03.300872+0000","172.21.15.5:0/1196201217":"2024-09-13T02:49:03.300872+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T02:56:18.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:18.431+0000 7f10d4251640 1 -- 172.21.15.5:0/2778001905 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f10b80a2c00 msgr2=0x7f10b80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:18.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:18.431+0000 7f10d4251640 1 --2- 172.21.15.5:0/2778001905 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f10b80a2c00 0x7f10b80a50c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f10cc0ff7a0 tx=0x7f10c0005ca0 comp rx=0 tx=0).stop 2024-09-12T02:56:18.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:18.431+0000 7f10d4251640 1 -- 172.21.15.5:0/2778001905 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10cc1000d0 msgr2=0x7f10cc1984d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:18.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:18.431+0000 7f10d4251640 1 --2- 172.21.15.5:0/2778001905 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10cc1000d0 0x7f10cc1984d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f10c8006010 tx=0x7f10c8003f80 comp rx=0 tx=0).stop 2024-09-12T02:56:18.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:18.431+0000 7f10d4251640 1 -- 172.21.15.5:0/2778001905 shutdown_connections 2024-09-12T02:56:18.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:18.431+0000 7f10d4251640 1 --2- 172.21.15.5:0/2778001905 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f10cc106ba0 0x7f10cc19cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:18.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:18.431+0000 7f10d4251640 1 --2- 172.21.15.5:0/2778001905 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f10cc100a70 0x7f10cc198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:18.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:18.431+0000 7f10d4251640 1 --2- 172.21.15.5:0/2778001905 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f10b80a2c00 0x7f10b80a50c0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:18.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:18.431+0000 7f10d4251640 1 --2- 172.21.15.5:0/2778001905 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10cc1000d0 0x7f10cc1984d0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:18.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:18.431+0000 7f10d4251640 1 -- 172.21.15.5:0/2778001905 >> 172.21.15.5:0/2778001905 conn(0x7f10cc077210 msgr2=0x7f10cc104c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:18.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:18.431+0000 7f10d4251640 1 -- 172.21.15.5:0/2778001905 shutdown_connections 2024-09-12T02:56:18.432 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:18.431+0000 7f10d4251640 1 -- 172.21.15.5:0/2778001905 wait complete. 2024-09-12T02:56:18.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:17 smithi146 ceph-mon[35011]: osdmap e81: 12 total, 12 up, 12 in 2024-09-12T02:56:18.458 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:17 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/1500954631' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-12T02:56:18.458 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:17 smithi146 ceph-mon[35011]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-09-12T02:56:18.458 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:17 smithi146 ceph-mon[35011]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-09-12T02:56:18.458 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:17 smithi146 ceph-mon[35011]: mds.? [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] up:active 2024-09-12T02:56:18.458 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:17 smithi146 ceph-mon[35011]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-12T02:56:18.665 INFO:journalctl@ceph.mds.i.smithi148.stdout:Sep 12 02:56:18 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-i[65552]: 2024-09-12T02:56:18.317+0000 7f3d98deb640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-12T02:56:19.010 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T02:56:19.266 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:56:19.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:18 smithi005 ceph-mon[31197]: pgmap v275: 97 pgs: 8 creating+peering, 61 active+clean, 28 unknown; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 0 op/s 2024-09-12T02:56:19.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:18 smithi005 ceph-mon[31197]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-12T02:56:19.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:18 smithi005 ceph-mon[31197]: Cluster is now healthy 2024-09-12T02:56:19.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:18 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/2778001905' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:56:19.415 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:18 smithi148 ceph-mon[35090]: pgmap v275: 97 pgs: 8 creating+peering, 61 active+clean, 28 unknown; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 0 op/s 2024-09-12T02:56:19.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:18 smithi148 ceph-mon[35090]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-12T02:56:19.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:18 smithi148 ceph-mon[35090]: Cluster is now healthy 2024-09-12T02:56:19.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:18 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/2778001905' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:56:19.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:18 smithi146 ceph-mon[35011]: pgmap v275: 97 pgs: 8 creating+peering, 61 active+clean, 28 unknown; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 0 op/s 2024-09-12T02:56:19.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:18 smithi146 ceph-mon[35011]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-12T02:56:19.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:18 smithi146 ceph-mon[35011]: Cluster is now healthy 2024-09-12T02:56:19.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:18 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/2778001905' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:56:19.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:19.725+0000 7fbf7227f640 1 -- 172.21.15.5:0/1935087645 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf6c100e20 msgr2=0x7fbf6c101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:19.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:19.725+0000 7fbf7227f640 1 --2- 172.21.15.5:0/1935087645 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf6c100e20 0x7fbf6c101200 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c009900 tx=0x7fbf5c02a480 comp rx=0 tx=0).stop 2024-09-12T02:56:19.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:19.725+0000 7fbf7227f640 1 -- 172.21.15.5:0/1935087645 shutdown_connections 2024-09-12T02:56:19.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:19.725+0000 7fbf7227f640 1 --2- 172.21.15.5:0/1935087645 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbf6c102ef0 0x7fbf6c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:19.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:19.725+0000 7fbf7227f640 1 --2- 172.21.15.5:0/1935087645 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf6c102570 0x7fbf6c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:19.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:19.725+0000 7fbf7227f640 1 --2- 172.21.15.5:0/1935087645 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf6c100e20 0x7fbf6c101200 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:19.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:19.725+0000 7fbf7227f640 1 -- 172.21.15.5:0/1935087645 >> 172.21.15.5:0/1935087645 conn(0x7fbf6c0fc7d0 msgr2=0x7fbf6c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:19.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:19.725+0000 7fbf7227f640 1 -- 172.21.15.5:0/1935087645 shutdown_connections 2024-09-12T02:56:19.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:19.725+0000 7fbf7227f640 1 -- 172.21.15.5:0/1935087645 wait complete. 2024-09-12T02:56:19.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:19.725+0000 7fbf7227f640 1 Processor -- start 2024-09-12T02:56:19.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:19.725+0000 7fbf7227f640 1 -- start start 2024-09-12T02:56:19.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:19.726+0000 7fbf7227f640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbf6c100e20 0x7fbf6c1986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:19.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:19.726+0000 7fbf7227f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf6c102570 0x7fbf6c198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:19.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:19.726+0000 7fbf7227f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf6c102ef0 0x7fbf6c19cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:19.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:19.726+0000 7fbf7227f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbf6c111070 con 0x7fbf6c102570 2024-09-12T02:56:19.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:19.726+0000 7fbf7227f640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fbf6c110ef0 con 0x7fbf6c100e20 2024-09-12T02:56:19.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:19.726+0000 7fbf7227f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fbf6c1111f0 con 0x7fbf6c102ef0 2024-09-12T02:56:19.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:19.726+0000 7fbf6b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf6c102570 0x7fbf6c198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:19.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:19.726+0000 7fbf6b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf6c102570 0x7fbf6c198c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58432/0 (socket says 172.21.15.5:58432) 2024-09-12T02:56:19.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:19.726+0000 7fbf6b7fe640 1 -- 172.21.15.5:0/4219037511 learned_addr learned my addr 172.21.15.5:0/4219037511 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:56:19.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:19.726+0000 7fbf6bfff640 1 --2- 172.21.15.5:0/4219037511 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbf6c100e20 0x7fbf6c1986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:19.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:19.726+0000 7fbf7081c640 1 --2- 172.21.15.5:0/4219037511 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf6c102ef0 0x7fbf6c19cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:19.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:19.726+0000 7fbf6b7fe640 1 -- 172.21.15.5:0/4219037511 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf6c102ef0 msgr2=0x7fbf6c19cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:19.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:19.726+0000 7fbf6b7fe640 1 --2- 172.21.15.5:0/4219037511 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf6c102ef0 0x7fbf6c19cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:19.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:19.726+0000 7fbf6b7fe640 1 -- 172.21.15.5:0/4219037511 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbf6c100e20 msgr2=0x7fbf6c1986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:19.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:19.726+0000 7fbf6b7fe640 1 --2- 172.21.15.5:0/4219037511 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbf6c100e20 0x7fbf6c1986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:19.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:19.726+0000 7fbf6b7fe640 1 -- 172.21.15.5:0/4219037511 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf6c19d690 con 0x7fbf6c102570 2024-09-12T02:56:19.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:19.726+0000 7fbf6b7fe640 1 --2- 172.21.15.5:0/4219037511 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf6c102570 0x7fbf6c198c20 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fbf6000b4a0 tx=0x7fbf6000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:19.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:19.727+0000 7fbf697fa640 1 -- 172.21.15.5:0/4219037511 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf60014070 con 0x7fbf6c102570 2024-09-12T02:56:19.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:19.727+0000 7fbf7227f640 1 -- 172.21.15.5:0/4219037511 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf6c19d980 con 0x7fbf6c102570 2024-09-12T02:56:19.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:19.727+0000 7fbf7227f640 1 -- 172.21.15.5:0/4219037511 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf6c1a5260 con 0x7fbf6c102570 2024-09-12T02:56:19.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:19.727+0000 7fbf697fa640 1 -- 172.21.15.5:0/4219037511 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbf60010af0 con 0x7fbf6c102570 2024-09-12T02:56:19.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:19.727+0000 7fbf697fa640 1 -- 172.21.15.5:0/4219037511 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf60017d10 con 0x7fbf6c102570 2024-09-12T02:56:19.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:19.728+0000 7fbf7227f640 1 -- 172.21.15.5:0/4219037511 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf6c101280 con 0x7fbf6c102570 2024-09-12T02:56:19.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:19.729+0000 7fbf697fa640 1 -- 172.21.15.5:0/4219037511 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fbf60017070 con 0x7fbf6c102570 2024-09-12T02:56:19.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:19.729+0000 7fbf697fa640 1 --2- 172.21.15.5:0/4219037511 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fbf540a2c00 0x7fbf540a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:19.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:19.730+0000 7fbf6bfff640 1 --2- 172.21.15.5:0/4219037511 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fbf540a2c00 0x7fbf540a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:19.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:19.730+0000 7fbf697fa640 1 -- 172.21.15.5:0/4219037511 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7fbf600ce330 con 0x7fbf6c102570 2024-09-12T02:56:19.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:19.730+0000 7fbf6bfff640 1 --2- 172.21.15.5:0/4219037511 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fbf540a2c00 0x7fbf540a50c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c000c00 tx=0x7fbf5c005a20 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:19.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:19.734+0000 7fbf697fa640 1 -- 172.21.15.5:0/4219037511 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbf6009b340 con 0x7fbf6c102570 2024-09-12T02:56:19.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:19.911+0000 7fbf7227f640 1 -- 172.21.15.5:0/4219037511 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbf6c10a0a0 con 0x7fbf6c102570 2024-09-12T02:56:19.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:19.913+0000 7fbf697fa640 1 -- 172.21.15.5:0/4219037511 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+22223 (secure 0 0 0) 0x7fbf6009ac40 con 0x7fbf6c102570 2024-09-12T02:56:19.913 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-12T02:56:19.913 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":81,"fsid":"618305f8-70b1-11ef-bcea-c7b262605968","created":"2024-09-12T02:48:42.352146+0000","modified":"2024-09-12T02:56:16.595365+0000","last_up_change":"2024-09-12T02:55:27.817161+0000","last_in_change":"2024-09-12T02:55:03.745281+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T02:51:47.970639+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T02:56:12.834271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T02:56:14.951437+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17da560a-6aa2-4f2c-a95f-67481dce4d3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":4248507573},{"type":"v1","addr":"172.21.15.5:6803","nonce":4248507573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":4248507573},{"type":"v1","addr":"172.21.15.5:6805","nonce":4248507573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":4248507573},{"type":"v1","addr":"172.21.15.5:6809","nonce":4248507573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":4248507573},{"type":"v1","addr":"172.21.15.5:6807","nonce":4248507573}]},"public_addr":"172.21.15.5:6803/4248507573","cluster_addr":"172.21.15.5:6805/4248507573","heartbeat_back_addr":"172.21.15.5:6809/4248507573","heartbeat_front_addr":"172.21.15.5:6807/4248507573","state":["exists","up"]},{"osd":1,"uuid":"a3237006-99af-4fc1-860c-d4917032647f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":1105990287},{"type":"v1","addr":"172.21.15.146:6801","nonce":1105990287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":1105990287},{"type":"v1","addr":"172.21.15.146:6803","nonce":1105990287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":1105990287},{"type":"v1","addr":"172.21.15.146:6807","nonce":1105990287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":1105990287},{"type":"v1","addr":"172.21.15.146:6805","nonce":1105990287}]},"public_addr":"172.21.15.146:6801/1105990287","cluster_addr":"172.21.15.146:6803/1105990287","heartbeat_back_addr":"172.21.15.146:6807/1105990287","heartbeat_front_addr":"172.21.15.146:6805/1105990287","state":["exists","up"]},{"osd":2,"uuid":"ea3ab84a-7739-4975-8244-31ddd5aad80d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3378174313},{"type":"v1","addr":"172.21.15.148:6801","nonce":3378174313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3378174313},{"type":"v1","addr":"172.21.15.148:6803","nonce":3378174313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3378174313},{"type":"v1","addr":"172.21.15.148:6807","nonce":3378174313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3378174313},{"type":"v1","addr":"172.21.15.148:6805","nonce":3378174313}]},"public_addr":"172.21.15.148:6801/3378174313","cluster_addr":"172.21.15.148:6803/3378174313","heartbeat_back_addr":"172.21.15.148:6807/3378174313","heartbeat_front_addr":"172.21.15.148:6805/3378174313","state":["exists","up"]},{"osd":3,"uuid":"4396a9ee-f3f3-410b-9fb1-124f1e33a1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":11741938},{"type":"v1","addr":"172.21.15.5:6811","nonce":11741938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":11741938},{"type":"v1","addr":"172.21.15.5:6813","nonce":11741938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":11741938},{"type":"v1","addr":"172.21.15.5:6817","nonce":11741938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":11741938},{"type":"v1","addr":"172.21.15.5:6815","nonce":11741938}]},"public_addr":"172.21.15.5:6811/11741938","cluster_addr":"172.21.15.5:6813/11741938","heartbeat_back_addr":"172.21.15.5:6817/11741938","heartbeat_front_addr":"172.21.15.5:6815/11741938","state":["exists","up"]},{"osd":4,"uuid":"989d76e7-c742-49d0-994a-c58f2771b1d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":3274923851},{"type":"v1","addr":"172.21.15.146:6809","nonce":3274923851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":3274923851},{"type":"v1","addr":"172.21.15.146:6811","nonce":3274923851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3274923851},{"type":"v1","addr":"172.21.15.146:6815","nonce":3274923851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":3274923851},{"type":"v1","addr":"172.21.15.146:6813","nonce":3274923851}]},"public_addr":"172.21.15.146:6809/3274923851","cluster_addr":"172.21.15.146:6811/3274923851","heartbeat_back_addr":"172.21.15.146:6815/3274923851","heartbeat_front_addr":"172.21.15.146:6813/3274923851","state":["exists","up"]},{"osd":5,"uuid":"64ada7d4-70a0-4b83-9af8-032be544608d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":2771670476},{"type":"v1","addr":"172.21.15.148:6809","nonce":2771670476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":2771670476},{"type":"v1","addr":"172.21.15.148:6811","nonce":2771670476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":2771670476},{"type":"v1","addr":"172.21.15.148:6815","nonce":2771670476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":2771670476},{"type":"v1","addr":"172.21.15.148:6813","nonce":2771670476}]},"public_addr":"172.21.15.148:6809/2771670476","cluster_addr":"172.21.15.148:6811/2771670476","heartbeat_back_addr":"172.21.15.148:6815/2771670476","heartbeat_front_addr":"172.21.15.148:6813/2771670476","state":["exists","up"]},{"osd":6,"uuid":"a70a88fd-2e4b-480c-bb98-62636cf70cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3176073169},{"type":"v1","addr":"172.21.15.5:6819","nonce":3176073169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":3176073169},{"type":"v1","addr":"172.21.15.5:6821","nonce":3176073169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":3176073169},{"type":"v1","addr":"172.21.15.5:6825","nonce":3176073169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3176073169},{"type":"v1","addr":"172.21.15.5:6823","nonce":3176073169}]},"public_addr":"172.21.15.5:6819/3176073169","cluster_addr":"172.21.15.5:6821/3176073169","heartbeat_back_addr":"172.21.15.5:6825/3176073169","heartbeat_front_addr":"172.21.15.5:6823/3176073169","state":["exists","up"]},{"osd":7,"uuid":"49451e8d-c741-4ad5-b8f5-89bad8d4d129","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2209508838},{"type":"v1","addr":"172.21.15.146:6817","nonce":2209508838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2209508838},{"type":"v1","addr":"172.21.15.146:6819","nonce":2209508838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2209508838},{"type":"v1","addr":"172.21.15.146:6823","nonce":2209508838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2209508838},{"type":"v1","addr":"172.21.15.146:6821","nonce":2209508838}]},"public_addr":"172.21.15.146:6817/2209508838","cluster_addr":"172.21.15.146:6819/2209508838","heartbeat_back_addr":"172.21.15.146:6823/2209508838","heartbeat_front_addr":"172.21.15.146:6821/2209508838","state":["exists","up"]},{"osd":8,"uuid":"57bfc121-a75e-4a7f-b1dc-64074b178b03","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":651045756},{"type":"v1","addr":"172.21.15.148:6817","nonce":651045756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":651045756},{"type":"v1","addr":"172.21.15.148:6819","nonce":651045756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":651045756},{"type":"v1","addr":"172.21.15.148:6823","nonce":651045756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":651045756},{"type":"v1","addr":"172.21.15.148:6821","nonce":651045756}]},"public_addr":"172.21.15.148:6817/651045756","cluster_addr":"172.21.15.148:6819/651045756","heartbeat_back_addr":"172.21.15.148:6823/651045756","heartbeat_front_addr":"172.21.15.148:6821/651045756","state":["exists","up"]},{"osd":9,"uuid":"d9726168-7494-4bf8-b433-350e85bfd461","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1961857360},{"type":"v1","addr":"172.21.15.5:6827","nonce":1961857360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1961857360},{"type":"v1","addr":"172.21.15.5:6829","nonce":1961857360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":1961857360},{"type":"v1","addr":"172.21.15.5:6833","nonce":1961857360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1961857360},{"type":"v1","addr":"172.21.15.5:6831","nonce":1961857360}]},"public_addr":"172.21.15.5:6827/1961857360","cluster_addr":"172.21.15.5:6829/1961857360","heartbeat_back_addr":"172.21.15.5:6833/1961857360","heartbeat_front_addr":"172.21.15.5:6831/1961857360","state":["exists","up"]},{"osd":10,"uuid":"519ec60f-3606-48ac-9043-6f59a798766d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":2041733103},{"type":"v1","addr":"172.21.15.146:6825","nonce":2041733103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":2041733103},{"type":"v1","addr":"172.21.15.146:6827","nonce":2041733103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":2041733103},{"type":"v1","addr":"172.21.15.146:6831","nonce":2041733103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":2041733103},{"type":"v1","addr":"172.21.15.146:6829","nonce":2041733103}]},"public_addr":"172.21.15.146:6825/2041733103","cluster_addr":"172.21.15.146:6827/2041733103","heartbeat_back_addr":"172.21.15.146:6831/2041733103","heartbeat_front_addr":"172.21.15.146:6829/2041733103","state":["exists","up"]},{"osd":11,"uuid":"0f92d0cb-ba54-4bc3-a6d0-3c54d910556a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":839290773},{"type":"v1","addr":"172.21.15.148:6825","nonce":839290773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":839290773},{"type":"v1","addr":"172.21.15.148:6827","nonce":839290773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":839290773},{"type":"v1","addr":"172.21.15.148:6831","nonce":839290773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":839290773},{"type":"v1","addr":"172.21.15.148:6829","nonce":839290773}]},"public_addr":"172.21.15.148:6825/839290773","cluster_addr":"172.21.15.148:6827/839290773","heartbeat_back_addr":"172.21.15.148:6831/839290773","heartbeat_front_addr":"172.21.15.148:6829/839290773","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:50:54.101755+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:51:19.037385+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:51:44.142837+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:52:09.150887+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:52:33.705079+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:52:58.845233+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:53:23.068339+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:53:47.520398+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:54:11.965370+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:54:36.776075+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:55:01.292236+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:55:25.346589+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:6800/3125821798":"2024-09-13T02:49:29.462216+0000","172.21.15.5:0/3479911451":"2024-09-13T02:49:29.462216+0000","172.21.15.5:0/1547753207":"2024-09-13T02:49:03.300872+0000","172.21.15.5:0/1905762627":"2024-09-13T02:49:03.300872+0000","172.21.15.5:0/4142340280":"2024-09-13T02:49:29.462216+0000","172.21.15.5:6801/3125821798":"2024-09-13T02:49:29.462216+0000","172.21.15.5:6801/2595781680":"2024-09-13T02:49:03.300872+0000","172.21.15.5:0/2273691311":"2024-09-13T02:49:29.462216+0000","172.21.15.5:6800/2595781680":"2024-09-13T02:49:03.300872+0000","172.21.15.5:0/1196201217":"2024-09-13T02:49:03.300872+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T02:56:19.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:19.915+0000 7fbf7227f640 1 -- 172.21.15.5:0/4219037511 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fbf540a2c00 msgr2=0x7fbf540a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:19.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:19.915+0000 7fbf7227f640 1 --2- 172.21.15.5:0/4219037511 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fbf540a2c00 0x7fbf540a50c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c000c00 tx=0x7fbf5c005a20 comp rx=0 tx=0).stop 2024-09-12T02:56:19.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:19.915+0000 7fbf7227f640 1 -- 172.21.15.5:0/4219037511 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf6c102570 msgr2=0x7fbf6c198c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:19.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:19.915+0000 7fbf7227f640 1 --2- 172.21.15.5:0/4219037511 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf6c102570 0x7fbf6c198c20 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fbf6000b4a0 tx=0x7fbf6000b960 comp rx=0 tx=0).stop 2024-09-12T02:56:19.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:19.915+0000 7fbf7227f640 1 -- 172.21.15.5:0/4219037511 shutdown_connections 2024-09-12T02:56:19.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:19.915+0000 7fbf7227f640 1 --2- 172.21.15.5:0/4219037511 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fbf540a2c00 0x7fbf540a50c0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:19.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:19.915+0000 7fbf7227f640 1 --2- 172.21.15.5:0/4219037511 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf6c102570 0x7fbf6c198c20 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:19.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:19.915+0000 7fbf7227f640 1 --2- 172.21.15.5:0/4219037511 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbf6c102ef0 0x7fbf6c19cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:19.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:19.915+0000 7fbf7227f640 1 --2- 172.21.15.5:0/4219037511 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbf6c100e20 0x7fbf6c1986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:19.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:19.915+0000 7fbf7227f640 1 -- 172.21.15.5:0/4219037511 >> 172.21.15.5:0/4219037511 conn(0x7fbf6c0fc7d0 msgr2=0x7fbf6c06a270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:19.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:19.915+0000 7fbf7227f640 1 -- 172.21.15.5:0/4219037511 shutdown_connections 2024-09-12T02:56:19.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:19.915+0000 7fbf7227f640 1 -- 172.21.15.5:0/4219037511 wait complete. 2024-09-12T02:56:20.165 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:19 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/4219037511' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:56:20.197 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:19 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/4219037511' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:56:20.246 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:19 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/4219037511' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:56:20.540 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-09-12T02:56:20.801 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:56:21.185 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:20 smithi005 ceph-mon[31197]: pgmap v276: 97 pgs: 8 creating+peering, 89 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 6 op/s 2024-09-12T02:56:21.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:20 smithi146 ceph-mon[35011]: pgmap v276: 97 pgs: 8 creating+peering, 89 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 6 op/s 2024-09-12T02:56:21.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:21.270+0000 7ff25230b640 1 -- 172.21.15.5:0/2620771723 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff24c106f10 msgr2=0x7ff24c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:21.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:21.270+0000 7ff25230b640 1 --2- 172.21.15.5:0/2620771723 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff24c106f10 0x7ff24c10ea90 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff244009ca0 tx=0x7ff24402a8c0 comp rx=0 tx=0).stop 2024-09-12T02:56:21.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:21.270+0000 7ff25230b640 1 -- 172.21.15.5:0/2620771723 shutdown_connections 2024-09-12T02:56:21.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:21.270+0000 7ff25230b640 1 --2- 172.21.15.5:0/2620771723 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff24c106f10 0x7ff24c10ea90 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:21.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:21.270+0000 7ff25230b640 1 --2- 172.21.15.5:0/2620771723 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff24c102790 0x7ff24c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:21.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:21.270+0000 7ff25230b640 1 --2- 172.21.15.5:0/2620771723 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff24c101de0 0x7ff24c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:21.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:21.270+0000 7ff25230b640 1 -- 172.21.15.5:0/2620771723 >> 172.21.15.5:0/2620771723 conn(0x7ff24c0fd8c0 msgr2=0x7ff24c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:21.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:21.270+0000 7ff25230b640 1 -- 172.21.15.5:0/2620771723 shutdown_connections 2024-09-12T02:56:21.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:21.270+0000 7ff25230b640 1 -- 172.21.15.5:0/2620771723 wait complete. 2024-09-12T02:56:21.277 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:21.271+0000 7ff25230b640 1 Processor -- start 2024-09-12T02:56:21.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:21.271+0000 7ff25230b640 1 -- start start 2024-09-12T02:56:21.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:21.271+0000 7ff25230b640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff24c101de0 0x7ff24c198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:21.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:21.271+0000 7ff25230b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff24c102790 0x7ff24c198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:21.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:21.271+0000 7ff25230b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff24c106f10 0x7ff24c19cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:21.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:21.271+0000 7ff25230b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff24c111f90 con 0x7ff24c106f10 2024-09-12T02:56:21.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:21.271+0000 7ff25230b640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ff24c111e10 con 0x7ff24c101de0 2024-09-12T02:56:21.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:21.271+0000 7ff25230b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff24c112110 con 0x7ff24c102790 2024-09-12T02:56:21.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:21.271+0000 7ff2508a8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff24c106f10 0x7ff24c19cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:21.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:21.271+0000 7ff2508a8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff24c106f10 0x7ff24c19cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58452/0 (socket says 172.21.15.5:58452) 2024-09-12T02:56:21.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:21.271+0000 7ff2508a8640 1 -- 172.21.15.5:0/1863608050 learned_addr learned my addr 172.21.15.5:0/1863608050 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:56:21.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:21.271+0000 7ff24bfff640 1 --2- 172.21.15.5:0/1863608050 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff24c101de0 0x7ff24c198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:21.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:21.271+0000 7ff24b7fe640 1 --2- 172.21.15.5:0/1863608050 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff24c102790 0x7ff24c198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:21.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:21.272+0000 7ff2508a8640 1 -- 172.21.15.5:0/1863608050 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff24c102790 msgr2=0x7ff24c198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:21.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:21.272+0000 7ff2508a8640 1 --2- 172.21.15.5:0/1863608050 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff24c102790 0x7ff24c198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:21.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:21.272+0000 7ff2508a8640 1 -- 172.21.15.5:0/1863608050 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff24c101de0 msgr2=0x7ff24c198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:21.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:21.272+0000 7ff2508a8640 1 --2- 172.21.15.5:0/1863608050 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff24c101de0 0x7ff24c198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:21.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:21.272+0000 7ff2508a8640 1 -- 172.21.15.5:0/1863608050 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff24c19d3c0 con 0x7ff24c106f10 2024-09-12T02:56:21.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:21.272+0000 7ff2508a8640 1 --2- 172.21.15.5:0/1863608050 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff24c106f10 0x7ff24c19cce0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff244031a60 tx=0x7ff244031a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:21.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:21.272+0000 7ff2497fa640 1 -- 172.21.15.5:0/1863608050 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff244003b70 con 0x7ff24c106f10 2024-09-12T02:56:21.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:21.272+0000 7ff2497fa640 1 -- 172.21.15.5:0/1863608050 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff244003d10 con 0x7ff24c106f10 2024-09-12T02:56:21.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:21.272+0000 7ff2497fa640 1 -- 172.21.15.5:0/1863608050 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff244004770 con 0x7ff24c106f10 2024-09-12T02:56:21.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:21.272+0000 7ff25230b640 1 -- 172.21.15.5:0/1863608050 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff24c19d650 con 0x7ff24c106f10 2024-09-12T02:56:21.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:21.272+0000 7ff25230b640 1 -- 172.21.15.5:0/1863608050 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff24c1a4f90 con 0x7ff24c106f10 2024-09-12T02:56:21.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:21.273+0000 7ff25230b640 1 -- 172.21.15.5:0/1863608050 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff24c1021c0 con 0x7ff24c106f10 2024-09-12T02:56:21.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:21.274+0000 7ff2497fa640 1 -- 172.21.15.5:0/1863608050 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7ff24402f790 con 0x7ff24c106f10 2024-09-12T02:56:21.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:21.275+0000 7ff2497fa640 1 --2- 172.21.15.5:0/1863608050 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7ff2340a2c00 0x7ff2340a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:21.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:21.275+0000 7ff24bfff640 1 --2- 172.21.15.5:0/1863608050 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7ff2340a2c00 0x7ff2340a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:21.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:21.275+0000 7ff2497fa640 1 -- 172.21.15.5:0/1863608050 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7ff2440f0180 con 0x7ff24c106f10 2024-09-12T02:56:21.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:21.276+0000 7ff24bfff640 1 --2- 172.21.15.5:0/1863608050 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7ff2340a2c00 0x7ff2340a50c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff23c009990 tx=0x7ff23c009110 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:21.280 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:21.280+0000 7ff2497fa640 1 -- 172.21.15.5:0/1863608050 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff2440bd190 con 0x7ff24c106f10 2024-09-12T02:56:21.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:20 smithi148 ceph-mon[35090]: pgmap v276: 97 pgs: 8 creating+peering, 89 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 6 op/s 2024-09-12T02:56:21.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:21.491+0000 7ff25230b640 1 -- 172.21.15.5:0/1863608050 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7ff24c1a5190 con 0x7ff24c106f10 2024-09-12T02:56:21.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:21.973+0000 7ff2497fa640 1 -- 172.21.15.5:0/1863608050 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) ==== 118+0+0 (secure 0 0 0) 0x7ff2440bca90 con 0x7ff24c106f10 2024-09-12T02:56:21.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:21.976+0000 7ff25230b640 1 -- 172.21.15.5:0/1863608050 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7ff2340a2c00 msgr2=0x7ff2340a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:21.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:21.976+0000 7ff25230b640 1 --2- 172.21.15.5:0/1863608050 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7ff2340a2c00 0x7ff2340a50c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff23c009990 tx=0x7ff23c009110 comp rx=0 tx=0).stop 2024-09-12T02:56:21.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:21.976+0000 7ff25230b640 1 -- 172.21.15.5:0/1863608050 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff24c106f10 msgr2=0x7ff24c19cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:21.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:21.976+0000 7ff25230b640 1 --2- 172.21.15.5:0/1863608050 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff24c106f10 0x7ff24c19cce0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff244031a60 tx=0x7ff244031a90 comp rx=0 tx=0).stop 2024-09-12T02:56:21.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:21.976+0000 7ff25230b640 1 -- 172.21.15.5:0/1863608050 shutdown_connections 2024-09-12T02:56:21.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:21.976+0000 7ff25230b640 1 --2- 172.21.15.5:0/1863608050 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7ff2340a2c00 0x7ff2340a50c0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:21.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:21.976+0000 7ff25230b640 1 --2- 172.21.15.5:0/1863608050 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff24c106f10 0x7ff24c19cce0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:21.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:21.976+0000 7ff25230b640 1 --2- 172.21.15.5:0/1863608050 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff24c102790 0x7ff24c198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:21.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:21.976+0000 7ff25230b640 1 --2- 172.21.15.5:0/1863608050 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff24c101de0 0x7ff24c198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:21.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:21.976+0000 7ff25230b640 1 -- 172.21.15.5:0/1863608050 >> 172.21.15.5:0/1863608050 conn(0x7ff24c0fd8c0 msgr2=0x7ff24c0ffcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:21.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:21.977+0000 7ff25230b640 1 -- 172.21.15.5:0/1863608050 shutdown_connections 2024-09-12T02:56:21.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:21.977+0000 7ff25230b640 1 -- 172.21.15.5:0/1863608050 wait complete. 2024-09-12T02:56:22.297 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:21 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/1863608050' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-12T02:56:22.415 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:21 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/1863608050' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-12T02:56:22.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:21 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/1863608050' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-12T02:56:22.584 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-09-12T02:56:22.585 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph fs set cephfs max_mds 5 2024-09-12T02:56:22.851 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:56:23.243 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:22 smithi005 ceph-mon[31197]: pgmap v277: 97 pgs: 97 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 6 op/s 2024-09-12T02:56:23.243 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:22 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/1863608050' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-12T02:56:23.243 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:22 smithi005 ceph-mon[31197]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-12T02:56:23.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:23.328+0000 7fe84519f640 1 -- 172.21.15.5:0/3466279325 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe840101270 msgr2=0x7fe84010d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:23.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:23.328+0000 7fe84519f640 1 --2- 172.21.15.5:0/3466279325 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe840101270 0x7fe84010d7a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe838009cd0 tx=0x7fe83802a8e0 comp rx=0 tx=0).stop 2024-09-12T02:56:23.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:23.329+0000 7fe84519f640 1 -- 172.21.15.5:0/3466279325 shutdown_connections 2024-09-12T02:56:23.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:23.329+0000 7fe84519f640 1 --2- 172.21.15.5:0/3466279325 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe840101270 0x7fe84010d7a0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:23.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:23.329+0000 7fe84519f640 1 --2- 172.21.15.5:0/3466279325 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe8401008d0 0x7fe840100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:23.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:23.329+0000 7fe84519f640 1 --2- 172.21.15.5:0/3466279325 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe8401068d0 0x7fe840106cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:23.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:23.329+0000 7fe84519f640 1 -- 172.21.15.5:0/3466279325 >> 172.21.15.5:0/3466279325 conn(0x7fe8400fc610 msgr2=0x7fe8400fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:23.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:23.329+0000 7fe84519f640 1 -- 172.21.15.5:0/3466279325 shutdown_connections 2024-09-12T02:56:23.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:23.329+0000 7fe84519f640 1 -- 172.21.15.5:0/3466279325 wait complete. 2024-09-12T02:56:23.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:23.329+0000 7fe84519f640 1 Processor -- start 2024-09-12T02:56:23.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:23.329+0000 7fe84519f640 1 -- start start 2024-09-12T02:56:23.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:23.330+0000 7fe84519f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe8401008d0 0x7fe8401981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:23.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:23.330+0000 7fe84519f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe840101270 0x7fe840198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:23.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:23.330+0000 7fe84519f640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe8401068d0 0x7fe84019cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:23.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:23.330+0000 7fe84519f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe840110cd0 con 0x7fe840101270 2024-09-12T02:56:23.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:23.330+0000 7fe84519f640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fe840110b50 con 0x7fe8401068d0 2024-09-12T02:56:23.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:23.330+0000 7fe84519f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe840110e50 con 0x7fe8401008d0 2024-09-12T02:56:23.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:23.330+0000 7fe837fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe840101270 0x7fe840198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:23.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:23.330+0000 7fe837fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe840101270 0x7fe840198730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58466/0 (socket says 172.21.15.5:58466) 2024-09-12T02:56:23.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:23.330+0000 7fe837fff640 1 -- 172.21.15.5:0/4070276172 learned_addr learned my addr 172.21.15.5:0/4070276172 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:56:23.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:23.330+0000 7fe84499e640 1 --2- 172.21.15.5:0/4070276172 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe8401068d0 0x7fe84019cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:23.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:23.330+0000 7fe83ffff640 1 --2- 172.21.15.5:0/4070276172 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe8401008d0 0x7fe8401981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:23.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:23.330+0000 7fe837fff640 1 -- 172.21.15.5:0/4070276172 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe8401008d0 msgr2=0x7fe8401981f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:23.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:23.330+0000 7fe837fff640 1 --2- 172.21.15.5:0/4070276172 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe8401008d0 0x7fe8401981f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:23.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:23.330+0000 7fe837fff640 1 -- 172.21.15.5:0/4070276172 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe8401068d0 msgr2=0x7fe84019cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:23.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:23.330+0000 7fe837fff640 1 --2- 172.21.15.5:0/4070276172 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe8401068d0 0x7fe84019cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:23.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:23.330+0000 7fe837fff640 1 -- 172.21.15.5:0/4070276172 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe84019d1a0 con 0x7fe840101270 2024-09-12T02:56:23.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:23.330+0000 7fe837fff640 1 --2- 172.21.15.5:0/4070276172 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe840101270 0x7fe840198730 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe82800c920 tx=0x7fe82800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:23.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:23.330+0000 7fe83dffb640 1 -- 172.21.15.5:0/4070276172 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe828007c70 con 0x7fe840101270 2024-09-12T02:56:23.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:23.330+0000 7fe83dffb640 1 -- 172.21.15.5:0/4070276172 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe828007e10 con 0x7fe840101270 2024-09-12T02:56:23.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:23.330+0000 7fe84519f640 1 -- 172.21.15.5:0/4070276172 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe84019d490 con 0x7fe840101270 2024-09-12T02:56:23.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:23.331+0000 7fe83dffb640 1 -- 172.21.15.5:0/4070276172 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe828018890 con 0x7fe840101270 2024-09-12T02:56:23.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:23.331+0000 7fe84519f640 1 -- 172.21.15.5:0/4070276172 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8401a4d70 con 0x7fe840101270 2024-09-12T02:56:23.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:23.331+0000 7fe84519f640 1 -- 172.21.15.5:0/4070276172 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe840100d30 con 0x7fe840101270 2024-09-12T02:56:23.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:23.333+0000 7fe83dffb640 1 -- 172.21.15.5:0/4070276172 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7fe828018a50 con 0x7fe840101270 2024-09-12T02:56:23.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:23.333+0000 7fe83dffb640 1 --2- 172.21.15.5:0/4070276172 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fe82c0a2c00 0x7fe82c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:23.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:23.334+0000 7fe83dffb640 1 -- 172.21.15.5:0/4070276172 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7fe8280cf1c0 con 0x7fe840101270 2024-09-12T02:56:23.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:23.334+0000 7fe83ffff640 1 --2- 172.21.15.5:0/4070276172 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fe82c0a2c00 0x7fe82c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:23.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:23.334+0000 7fe83ffff640 1 --2- 172.21.15.5:0/4070276172 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fe82c0a2c00 0x7fe82c0a50c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe83000bb60 tx=0x7fe83000b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:23.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:23.337+0000 7fe83dffb640 1 -- 172.21.15.5:0/4070276172 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe828014030 con 0x7fe840101270 2024-09-12T02:56:23.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:22 smithi148 ceph-mon[35090]: pgmap v277: 97 pgs: 97 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 6 op/s 2024-09-12T02:56:23.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:22 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/1863608050' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-12T02:56:23.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:22 smithi148 ceph-mon[35090]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-12T02:56:23.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:22 smithi146 ceph-mon[35011]: pgmap v277: 97 pgs: 97 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 6 op/s 2024-09-12T02:56:23.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:22 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/1863608050' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-12T02:56:23.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:22 smithi146 ceph-mon[35011]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-12T02:56:23.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:23.548+0000 7fe84519f640 1 -- 172.21.15.5:0/4070276172 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7fe84002d7c0 con 0x7fe840101270 2024-09-12T02:56:23.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:23.981+0000 7fe83dffb640 1 -- 172.21.15.5:0/4070276172 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v17) ==== 105+0+0 (secure 0 0 0) 0x7fe82809c380 con 0x7fe840101270 2024-09-12T02:56:23.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:23.984+0000 7fe84519f640 1 -- 172.21.15.5:0/4070276172 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fe82c0a2c00 msgr2=0x7fe82c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:23.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:23.984+0000 7fe84519f640 1 --2- 172.21.15.5:0/4070276172 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fe82c0a2c00 0x7fe82c0a50c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe83000bb60 tx=0x7fe83000b390 comp rx=0 tx=0).stop 2024-09-12T02:56:23.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:23.984+0000 7fe84519f640 1 -- 172.21.15.5:0/4070276172 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe840101270 msgr2=0x7fe840198730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:23.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:23.984+0000 7fe84519f640 1 --2- 172.21.15.5:0/4070276172 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe840101270 0x7fe840198730 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe82800c920 tx=0x7fe82800cde0 comp rx=0 tx=0).stop 2024-09-12T02:56:23.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:23.984+0000 7fe84519f640 1 -- 172.21.15.5:0/4070276172 shutdown_connections 2024-09-12T02:56:23.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:23.984+0000 7fe84519f640 1 --2- 172.21.15.5:0/4070276172 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fe82c0a2c00 0x7fe82c0a50c0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:23.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:23.984+0000 7fe84519f640 1 --2- 172.21.15.5:0/4070276172 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe840101270 0x7fe840198730 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:23.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:23.984+0000 7fe84519f640 1 --2- 172.21.15.5:0/4070276172 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe8401068d0 0x7fe84019cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:23.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:23.984+0000 7fe84519f640 1 --2- 172.21.15.5:0/4070276172 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe8401008d0 0x7fe8401981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:23.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:23.984+0000 7fe84519f640 1 -- 172.21.15.5:0/4070276172 >> 172.21.15.5:0/4070276172 conn(0x7fe8400fc610 msgr2=0x7fe84006a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:23.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:23.984+0000 7fe84519f640 1 -- 172.21.15.5:0/4070276172 shutdown_connections 2024-09-12T02:56:23.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:23.985+0000 7fe84519f640 1 -- 172.21.15.5:0/4070276172 wait complete. 2024-09-12T02:56:24.288 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:23 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/4070276172' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-09-12T02:56:24.415 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:23 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/4070276172' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-09-12T02:56:24.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:23 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/4070276172' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-09-12T02:56:24.626 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-09-12T02:56:24.887 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:56:25.276 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:24 smithi005 ceph-mon[31197]: pgmap v278: 97 pgs: 97 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-09-12T02:56:25.276 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:24 smithi005 ceph-mon[31197]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-12T02:56:25.276 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:24 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/4070276172' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-12T02:56:25.276 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:24 smithi005 ceph-mon[31197]: daemon mds.c assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-12T02:56:25.276 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:24 smithi005 ceph-mon[31197]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-12T02:56:25.276 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:24 smithi005 ceph-mon[31197]: fsmap cephfs:2 {0=i=up:active,1=c=up:creating} 10 up:standby 2024-09-12T02:56:25.276 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:24 smithi005 ceph-mon[31197]: daemon mds.c is now active in filesystem cephfs as rank 1 2024-09-12T02:56:25.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:25.362+0000 7f597abbb640 1 -- 172.21.15.5:0/101022498 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5974102d30 msgr2=0x7f5974103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:25.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:25.362+0000 7f597abbb640 1 --2- 172.21.15.5:0/101022498 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5974102d30 0x7f5974103110 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f595c009930 tx=0x7f595c02a420 comp rx=0 tx=0).stop 2024-09-12T02:56:25.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:25.362+0000 7f597abbb640 1 -- 172.21.15.5:0/101022498 shutdown_connections 2024-09-12T02:56:25.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:25.362+0000 7f597abbb640 1 --2- 172.21.15.5:0/101022498 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f59741082a0 0x7f597410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:25.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:25.362+0000 7f597abbb640 1 --2- 172.21.15.5:0/101022498 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59741036e0 0x7f5974107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:25.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:25.362+0000 7f597abbb640 1 --2- 172.21.15.5:0/101022498 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5974102d30 0x7f5974103110 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:25.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:25.362+0000 7f597abbb640 1 -- 172.21.15.5:0/101022498 >> 172.21.15.5:0/101022498 conn(0x7f59740fc7d0 msgr2=0x7f59740febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:25.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:25.362+0000 7f597abbb640 1 -- 172.21.15.5:0/101022498 shutdown_connections 2024-09-12T02:56:25.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:25.362+0000 7f597abbb640 1 -- 172.21.15.5:0/101022498 wait complete. 2024-09-12T02:56:25.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:25.363+0000 7f597abbb640 1 Processor -- start 2024-09-12T02:56:25.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:25.363+0000 7f597abbb640 1 -- start start 2024-09-12T02:56:25.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:25.363+0000 7f597abbb640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5974102d30 0x7f5974100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:25.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:25.363+0000 7f597abbb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59741036e0 0x7f5974100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:25.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:25.363+0000 7f597abbb640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59741082a0 0x7f5974101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:25.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:25.363+0000 7f597abbb640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f59741120a0 con 0x7f59741036e0 2024-09-12T02:56:25.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:25.363+0000 7f597abbb640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f5974111f20 con 0x7f5974102d30 2024-09-12T02:56:25.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:25.363+0000 7f597abbb640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5974112220 con 0x7f59741082a0 2024-09-12T02:56:25.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:25.363+0000 7f5973fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59741036e0 0x7f5974100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:25.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:25.363+0000 7f5973fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59741036e0 0x7f5974100ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58324/0 (socket says 172.21.15.5:58324) 2024-09-12T02:56:25.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:25.363+0000 7f5973fff640 1 -- 172.21.15.5:0/3423483812 learned_addr learned my addr 172.21.15.5:0/3423483812 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:56:25.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:25.363+0000 7f5978957640 1 --2- 172.21.15.5:0/3423483812 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5974102d30 0x7f5974100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:25.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:25.364+0000 7f5979158640 1 --2- 172.21.15.5:0/3423483812 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59741082a0 0x7f5974101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:25.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:25.364+0000 7f5973fff640 1 -- 172.21.15.5:0/3423483812 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59741082a0 msgr2=0x7f5974101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:25.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:25.364+0000 7f5973fff640 1 --2- 172.21.15.5:0/3423483812 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59741082a0 0x7f5974101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:25.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:25.364+0000 7f5973fff640 1 -- 172.21.15.5:0/3423483812 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5974102d30 msgr2=0x7f5974100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:25.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:25.364+0000 7f5973fff640 1 --2- 172.21.15.5:0/3423483812 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5974102d30 0x7f5974100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:25.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:25.364+0000 7f5973fff640 1 -- 172.21.15.5:0/3423483812 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59741a6a40 con 0x7f59741036e0 2024-09-12T02:56:25.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:25.364+0000 7f5973fff640 1 --2- 172.21.15.5:0/3423483812 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59741036e0 0x7f5974100ed0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f596400ea50 tx=0x7f596400ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:25.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:25.364+0000 7f5971ffb640 1 -- 172.21.15.5:0/3423483812 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f596400cdc0 con 0x7f59741036e0 2024-09-12T02:56:25.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:25.364+0000 7f597abbb640 1 -- 172.21.15.5:0/3423483812 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59741a6d30 con 0x7f59741036e0 2024-09-12T02:56:25.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:25.364+0000 7f5971ffb640 1 -- 172.21.15.5:0/3423483812 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5964011950 con 0x7f59741036e0 2024-09-12T02:56:25.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:25.364+0000 7f5971ffb640 1 -- 172.21.15.5:0/3423483812 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f596401aaf0 con 0x7f59741036e0 2024-09-12T02:56:25.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:25.364+0000 7f597abbb640 1 -- 172.21.15.5:0/3423483812 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59741a7210 con 0x7f59741036e0 2024-09-12T02:56:25.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:25.365+0000 7f597abbb640 1 -- 172.21.15.5:0/3423483812 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5974103110 con 0x7f59741036e0 2024-09-12T02:56:25.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:25.366+0000 7f5971ffb640 1 -- 172.21.15.5:0/3423483812 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f596401ac90 con 0x7f59741036e0 2024-09-12T02:56:25.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:25.367+0000 7f5971ffb640 1 --2- 172.21.15.5:0/3423483812 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f59600a2b80 0x7f59600a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:25.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:25.367+0000 7f5971ffb640 1 -- 172.21.15.5:0/3423483812 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f59640cf4e0 con 0x7f59741036e0 2024-09-12T02:56:25.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:25.367+0000 7f5978957640 1 --2- 172.21.15.5:0/3423483812 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f59600a2b80 0x7f59600a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:25.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:25.368+0000 7f5978957640 1 --2- 172.21.15.5:0/3423483812 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f59600a2b80 0x7f59600a5040 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f595c009900 tx=0x7f595c0058d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:25.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:25.371+0000 7f5971ffb640 1 -- 172.21.15.5:0/3423483812 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5964016030 con 0x7f59741036e0 2024-09-12T02:56:25.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:24 smithi148 ceph-mon[35090]: pgmap v278: 97 pgs: 97 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-09-12T02:56:25.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:24 smithi148 ceph-mon[35090]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-12T02:56:25.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:24 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/4070276172' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-12T02:56:25.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:24 smithi148 ceph-mon[35090]: daemon mds.c assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-12T02:56:25.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:24 smithi148 ceph-mon[35090]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-12T02:56:25.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:24 smithi148 ceph-mon[35090]: fsmap cephfs:2 {0=i=up:active,1=c=up:creating} 10 up:standby 2024-09-12T02:56:25.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:24 smithi148 ceph-mon[35090]: daemon mds.c is now active in filesystem cephfs as rank 1 2024-09-12T02:56:25.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:24 smithi146 ceph-mon[35011]: pgmap v278: 97 pgs: 97 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-09-12T02:56:25.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:24 smithi146 ceph-mon[35011]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-12T02:56:25.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:24 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/4070276172' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-09-12T02:56:25.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:24 smithi146 ceph-mon[35011]: daemon mds.c assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-12T02:56:25.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:24 smithi146 ceph-mon[35011]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-09-12T02:56:25.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:24 smithi146 ceph-mon[35011]: fsmap cephfs:2 {0=i=up:active,1=c=up:creating} 10 up:standby 2024-09-12T02:56:25.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:24 smithi146 ceph-mon[35011]: daemon mds.c is now active in filesystem cephfs as rank 1 2024-09-12T02:56:25.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:25.585+0000 7f597abbb640 1 -- 172.21.15.5:0/3423483812 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f597410a2b0 con 0x7f59741036e0 2024-09-12T02:56:26.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:26.003+0000 7f5971ffb640 1 -- 172.21.15.5:0/3423483812 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v21) ==== 121+0+0 (secure 0 0 0) 0x7f596409c590 con 0x7f59741036e0 2024-09-12T02:56:26.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:26.006+0000 7f597abbb640 1 -- 172.21.15.5:0/3423483812 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f59600a2b80 msgr2=0x7f59600a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:26.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:26.006+0000 7f597abbb640 1 --2- 172.21.15.5:0/3423483812 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f59600a2b80 0x7f59600a5040 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f595c009900 tx=0x7f595c0058d0 comp rx=0 tx=0).stop 2024-09-12T02:56:26.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:26.006+0000 7f597abbb640 1 -- 172.21.15.5:0/3423483812 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59741036e0 msgr2=0x7f5974100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:26.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:26.006+0000 7f597abbb640 1 --2- 172.21.15.5:0/3423483812 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59741036e0 0x7f5974100ed0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f596400ea50 tx=0x7f596400ef10 comp rx=0 tx=0).stop 2024-09-12T02:56:26.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:26.006+0000 7f597abbb640 1 -- 172.21.15.5:0/3423483812 shutdown_connections 2024-09-12T02:56:26.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:26.006+0000 7f597abbb640 1 --2- 172.21.15.5:0/3423483812 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f59600a2b80 0x7f59600a5040 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:26.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:26.006+0000 7f597abbb640 1 --2- 172.21.15.5:0/3423483812 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59741036e0 0x7f5974100ed0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:26.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:26.006+0000 7f597abbb640 1 --2- 172.21.15.5:0/3423483812 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f59741082a0 0x7f5974101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:26.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:26.006+0000 7f597abbb640 1 --2- 172.21.15.5:0/3423483812 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5974102d30 0x7f5974100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:26.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:26.006+0000 7f597abbb640 1 -- 172.21.15.5:0/3423483812 >> 172.21.15.5:0/3423483812 conn(0x7f59740fc7d0 msgr2=0x7f5974105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:26.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:26.006+0000 7f597abbb640 1 -- 172.21.15.5:0/3423483812 shutdown_connections 2024-09-12T02:56:26.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:26.006+0000 7f597abbb640 1 -- 172.21.15.5:0/3423483812 wait complete. 2024-09-12T02:56:26.305 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:25 smithi005 ceph-mon[31197]: mds.? [v2:172.21.15.148:6836/2606873922,v1:172.21.15.148:6837/2606873922] up:active 2024-09-12T02:56:26.305 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:25 smithi005 ceph-mon[31197]: daemon mds.f assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-12T02:56:26.305 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:25 smithi005 ceph-mon[31197]: fsmap cephfs:2 {0=i=up:active,1=c=up:active} 10 up:standby 2024-09-12T02:56:26.305 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:25 smithi005 ceph-mon[31197]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=f=up:creating} 9 up:standby 2024-09-12T02:56:26.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:25 smithi005 ceph-mon[31197]: daemon mds.f is now active in filesystem cephfs as rank 2 2024-09-12T02:56:26.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:25 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/3423483812' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-12T02:56:26.415 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:25 smithi148 ceph-mon[35090]: mds.? [v2:172.21.15.148:6836/2606873922,v1:172.21.15.148:6837/2606873922] up:active 2024-09-12T02:56:26.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:25 smithi148 ceph-mon[35090]: daemon mds.f assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-12T02:56:26.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:25 smithi148 ceph-mon[35090]: fsmap cephfs:2 {0=i=up:active,1=c=up:active} 10 up:standby 2024-09-12T02:56:26.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:25 smithi148 ceph-mon[35090]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=f=up:creating} 9 up:standby 2024-09-12T02:56:26.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:25 smithi148 ceph-mon[35090]: daemon mds.f is now active in filesystem cephfs as rank 2 2024-09-12T02:56:26.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:25 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/3423483812' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-12T02:56:26.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:25 smithi146 ceph-mon[35011]: mds.? [v2:172.21.15.148:6836/2606873922,v1:172.21.15.148:6837/2606873922] up:active 2024-09-12T02:56:26.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:25 smithi146 ceph-mon[35011]: daemon mds.f assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-12T02:56:26.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:25 smithi146 ceph-mon[35011]: fsmap cephfs:2 {0=i=up:active,1=c=up:active} 10 up:standby 2024-09-12T02:56:26.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:25 smithi146 ceph-mon[35011]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=f=up:creating} 9 up:standby 2024-09-12T02:56:26.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:25 smithi146 ceph-mon[35011]: daemon mds.f is now active in filesystem cephfs as rank 2 2024-09-12T02:56:26.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:25 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/3423483812' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-12T02:56:26.579 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-09-12T02:56:26.841 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:56:27.251 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:27 smithi005 ceph-mon[31197]: pgmap v279: 97 pgs: 97 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 6 op/s 2024-09-12T02:56:27.251 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:27 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/3423483812' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-12T02:56:27.251 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:27 smithi005 ceph-mon[31197]: mds.? [v2:172.21.15.148:6834/4195086427,v1:172.21.15.148:6835/4195086427] up:active 2024-09-12T02:56:27.251 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:27 smithi005 ceph-mon[31197]: daemon mds.b assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-12T02:56:27.251 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:27 smithi005 ceph-mon[31197]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=f=up:active} 9 up:standby 2024-09-12T02:56:27.251 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:27 smithi005 ceph-mon[31197]: fsmap cephfs:4 {0=i=up:active,1=c=up:active,2=f=up:active,3=b=up:creating} 8 up:standby 2024-09-12T02:56:27.251 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:27 smithi005 ceph-mon[31197]: daemon mds.b is now active in filesystem cephfs as rank 3 2024-09-12T02:56:27.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:27.277+0000 7f39f69c8640 1 -- 172.21.15.5:0/519944590 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39f01082a0 msgr2=0x7f39f010ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:27.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:27.277+0000 7f39f69c8640 1 --2- 172.21.15.5:0/519944590 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39f01082a0 0x7f39f010ec40 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f39e8009cd0 tx=0x7f39e802a8e0 comp rx=0 tx=0).stop 2024-09-12T02:56:27.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:27.277+0000 7f39f69c8640 1 -- 172.21.15.5:0/519944590 shutdown_connections 2024-09-12T02:56:27.278 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:27.277+0000 7f39f69c8640 1 --2- 172.21.15.5:0/519944590 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39f01082a0 0x7f39f010ec40 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:27.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:27.277+0000 7f39f69c8640 1 --2- 172.21.15.5:0/519944590 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f39f01036e0 0x7f39f0107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:27.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:27.277+0000 7f39f69c8640 1 --2- 172.21.15.5:0/519944590 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39f0102d30 0x7f39f0103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:27.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:27.277+0000 7f39f69c8640 1 -- 172.21.15.5:0/519944590 >> 172.21.15.5:0/519944590 conn(0x7f39f00fc7d0 msgr2=0x7f39f00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:27.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:27.277+0000 7f39f69c8640 1 -- 172.21.15.5:0/519944590 shutdown_connections 2024-09-12T02:56:27.279 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:27.277+0000 7f39f69c8640 1 -- 172.21.15.5:0/519944590 wait complete. 2024-09-12T02:56:27.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:27.278+0000 7f39f69c8640 1 Processor -- start 2024-09-12T02:56:27.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:27.278+0000 7f39f69c8640 1 -- start start 2024-09-12T02:56:27.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:27.278+0000 7f39f69c8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39f0102d30 0x7f39f0198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:27.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:27.278+0000 7f39f69c8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39f01036e0 0x7f39f0198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:27.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:27.278+0000 7f39f69c8640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f39f01082a0 0x7f39f019ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:27.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:27.278+0000 7f39f69c8640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f39f0112100 con 0x7f39f0102d30 2024-09-12T02:56:27.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:27.278+0000 7f39f69c8640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f39f0111f80 con 0x7f39f01082a0 2024-09-12T02:56:27.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:27.278+0000 7f39f69c8640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f39f0112280 con 0x7f39f01036e0 2024-09-12T02:56:27.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:27.278+0000 7f39effff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39f0102d30 0x7f39f0198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:27.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:27.278+0000 7f39effff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39f0102d30 0x7f39f0198590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58342/0 (socket says 172.21.15.5:58342) 2024-09-12T02:56:27.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:27.278+0000 7f39effff640 1 -- 172.21.15.5:0/1920840878 learned_addr learned my addr 172.21.15.5:0/1920840878 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:56:27.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:27.278+0000 7f39f4f65640 1 --2- 172.21.15.5:0/1920840878 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f39f01082a0 0x7f39f019ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:27.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:27.278+0000 7f39e7fff640 1 --2- 172.21.15.5:0/1920840878 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39f01036e0 0x7f39f0198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:27.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:27.278+0000 7f39effff640 1 -- 172.21.15.5:0/1920840878 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39f01036e0 msgr2=0x7f39f0198ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:27.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:27.278+0000 7f39effff640 1 --2- 172.21.15.5:0/1920840878 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39f01036e0 0x7f39f0198ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:27.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:27.278+0000 7f39effff640 1 -- 172.21.15.5:0/1920840878 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f39f01082a0 msgr2=0x7f39f019ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:27.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:27.278+0000 7f39effff640 1 --2- 172.21.15.5:0/1920840878 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f39f01082a0 0x7f39f019ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:27.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:27.278+0000 7f39effff640 1 -- 172.21.15.5:0/1920840878 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39f019d540 con 0x7f39f0102d30 2024-09-12T02:56:27.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:27.278+0000 7f39f4f65640 1 --2- 172.21.15.5:0/1920840878 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f39f01082a0 0x7f39f019ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:56:27.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:27.278+0000 7f39e7fff640 1 --2- 172.21.15.5:0/1920840878 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39f01036e0 0x7f39f0198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:56:27.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:27.279+0000 7f39effff640 1 --2- 172.21.15.5:0/1920840878 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39f0102d30 0x7f39f0198590 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f39e000b4a0 tx=0x7f39e000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:27.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:27.279+0000 7f39edffb640 1 -- 172.21.15.5:0/1920840878 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39e0014070 con 0x7f39f0102d30 2024-09-12T02:56:27.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:27.279+0000 7f39edffb640 1 -- 172.21.15.5:0/1920840878 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39e0010af0 con 0x7f39f0102d30 2024-09-12T02:56:27.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:27.279+0000 7f39edffb640 1 -- 172.21.15.5:0/1920840878 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39e0017d10 con 0x7f39f0102d30 2024-09-12T02:56:27.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:27.279+0000 7f39f69c8640 1 -- 172.21.15.5:0/1920840878 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39f019d830 con 0x7f39f0102d30 2024-09-12T02:56:27.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:27.279+0000 7f39f69c8640 1 -- 172.21.15.5:0/1920840878 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39f01a5110 con 0x7f39f0102d30 2024-09-12T02:56:27.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:27.280+0000 7f39f69c8640 1 -- 172.21.15.5:0/1920840878 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39f006a850 con 0x7f39f0102d30 2024-09-12T02:56:27.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:27.281+0000 7f39edffb640 1 -- 172.21.15.5:0/1920840878 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f39e0017070 con 0x7f39f0102d30 2024-09-12T02:56:27.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:27.282+0000 7f39edffb640 1 --2- 172.21.15.5:0/1920840878 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f39dc0a2c00 0x7f39dc0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:27.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:27.282+0000 7f39e7fff640 1 --2- 172.21.15.5:0/1920840878 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f39dc0a2c00 0x7f39dc0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:27.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:27.282+0000 7f39edffb640 1 -- 172.21.15.5:0/1920840878 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f39e00ce300 con 0x7f39f0102d30 2024-09-12T02:56:27.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:27.282+0000 7f39e7fff640 1 --2- 172.21.15.5:0/1920840878 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f39dc0a2c00 0x7f39dc0a50c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f39d80058c0 tx=0x7f39d8005850 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:27.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:27.286+0000 7f39edffb640 1 -- 172.21.15.5:0/1920840878 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f39e009b3b0 con 0x7f39f0102d30 2024-09-12T02:56:27.415 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:27 smithi148 ceph-mon[35090]: pgmap v279: 97 pgs: 97 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 6 op/s 2024-09-12T02:56:27.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:27 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/3423483812' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-12T02:56:27.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:27 smithi148 ceph-mon[35090]: mds.? [v2:172.21.15.148:6834/4195086427,v1:172.21.15.148:6835/4195086427] up:active 2024-09-12T02:56:27.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:27 smithi148 ceph-mon[35090]: daemon mds.b assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-12T02:56:27.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:27 smithi148 ceph-mon[35090]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=f=up:active} 9 up:standby 2024-09-12T02:56:27.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:27 smithi148 ceph-mon[35090]: fsmap cephfs:4 {0=i=up:active,1=c=up:active,2=f=up:active,3=b=up:creating} 8 up:standby 2024-09-12T02:56:27.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:27 smithi148 ceph-mon[35090]: daemon mds.b is now active in filesystem cephfs as rank 3 2024-09-12T02:56:27.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:27 smithi146 ceph-mon[35011]: pgmap v279: 97 pgs: 97 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 6 op/s 2024-09-12T02:56:27.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:27 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/3423483812' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-12T02:56:27.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:27 smithi146 ceph-mon[35011]: mds.? [v2:172.21.15.148:6834/4195086427,v1:172.21.15.148:6835/4195086427] up:active 2024-09-12T02:56:27.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:27 smithi146 ceph-mon[35011]: daemon mds.b assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-09-12T02:56:27.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:27 smithi146 ceph-mon[35011]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=f=up:active} 9 up:standby 2024-09-12T02:56:27.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:27 smithi146 ceph-mon[35011]: fsmap cephfs:4 {0=i=up:active,1=c=up:active,2=f=up:active,3=b=up:creating} 8 up:standby 2024-09-12T02:56:27.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:27 smithi146 ceph-mon[35011]: daemon mds.b is now active in filesystem cephfs as rank 3 2024-09-12T02:56:27.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:27.500+0000 7f39f69c8640 1 -- 172.21.15.5:0/1920840878 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f39f010a300 con 0x7f39f0102d30 2024-09-12T02:56:27.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:27.839+0000 7f39edffb640 1 -- 172.21.15.5:0/1920840878 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v25) ==== 115+0+0 (secure 0 0 0) 0x7f39f010a300 con 0x7f39f0102d30 2024-09-12T02:56:27.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:27.842+0000 7f39f69c8640 1 -- 172.21.15.5:0/1920840878 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f39dc0a2c00 msgr2=0x7f39dc0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:27.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:27.842+0000 7f39f69c8640 1 --2- 172.21.15.5:0/1920840878 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f39dc0a2c00 0x7f39dc0a50c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f39d80058c0 tx=0x7f39d8005850 comp rx=0 tx=0).stop 2024-09-12T02:56:27.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:27.842+0000 7f39f69c8640 1 -- 172.21.15.5:0/1920840878 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39f0102d30 msgr2=0x7f39f0198590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:27.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:27.842+0000 7f39f69c8640 1 --2- 172.21.15.5:0/1920840878 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39f0102d30 0x7f39f0198590 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f39e000b4a0 tx=0x7f39e000b960 comp rx=0 tx=0).stop 2024-09-12T02:56:27.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:27.842+0000 7f39f69c8640 1 -- 172.21.15.5:0/1920840878 shutdown_connections 2024-09-12T02:56:27.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:27.842+0000 7f39f69c8640 1 --2- 172.21.15.5:0/1920840878 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f39f01082a0 0x7f39f019ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:27.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:27.842+0000 7f39f69c8640 1 --2- 172.21.15.5:0/1920840878 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f39f01036e0 0x7f39f0198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:27.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:27.842+0000 7f39f69c8640 1 --2- 172.21.15.5:0/1920840878 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f39dc0a2c00 0x7f39dc0a50c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:27.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:27.842+0000 7f39f69c8640 1 --2- 172.21.15.5:0/1920840878 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39f0102d30 0x7f39f0198590 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:27.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:27.842+0000 7f39f69c8640 1 -- 172.21.15.5:0/1920840878 >> 172.21.15.5:0/1920840878 conn(0x7f39f00fc7d0 msgr2=0x7f39f0105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:27.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:27.843+0000 7f39f69c8640 1 -- 172.21.15.5:0/1920840878 shutdown_connections 2024-09-12T02:56:27.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:27.843+0000 7f39f69c8640 1 -- 172.21.15.5:0/1920840878 wait complete. 2024-09-12T02:56:28.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:28 smithi005 ceph-mon[31197]: mds.? [v2:172.21.15.146:6838/3036767228,v1:172.21.15.146:6839/3036767228] up:active 2024-09-12T02:56:28.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:28 smithi005 ceph-mon[31197]: daemon mds.h assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-12T02:56:28.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:28 smithi005 ceph-mon[31197]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-12T02:56:28.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:28 smithi005 ceph-mon[31197]: Cluster is now healthy 2024-09-12T02:56:28.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:28 smithi005 ceph-mon[31197]: fsmap cephfs:4 {0=i=up:active,1=c=up:active,2=f=up:active,3=b=up:active} 8 up:standby 2024-09-12T02:56:28.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:28 smithi005 ceph-mon[31197]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-09-12T02:56:28.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:28 smithi005 ceph-mon[31197]: daemon mds.h is now active in filesystem cephfs as rank 4 2024-09-12T02:56:28.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:28 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/1920840878' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-12T02:56:28.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:28 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/1920840878' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-12T02:56:28.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:28 smithi005 ceph-mon[31197]: mds.? [v2:172.21.15.146:6836/1673370937,v1:172.21.15.146:6837/1673370937] up:active 2024-09-12T02:56:28.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:28 smithi005 ceph-mon[31197]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T02:56:28.318 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:28 smithi148 ceph-mon[35090]: mds.? [v2:172.21.15.146:6838/3036767228,v1:172.21.15.146:6839/3036767228] up:active 2024-09-12T02:56:28.319 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:28 smithi148 ceph-mon[35090]: daemon mds.h assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-12T02:56:28.319 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:28 smithi148 ceph-mon[35090]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-12T02:56:28.319 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:28 smithi148 ceph-mon[35090]: Cluster is now healthy 2024-09-12T02:56:28.319 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:28 smithi148 ceph-mon[35090]: fsmap cephfs:4 {0=i=up:active,1=c=up:active,2=f=up:active,3=b=up:active} 8 up:standby 2024-09-12T02:56:28.319 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:28 smithi148 ceph-mon[35090]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-09-12T02:56:28.319 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:28 smithi148 ceph-mon[35090]: daemon mds.h is now active in filesystem cephfs as rank 4 2024-09-12T02:56:28.319 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:28 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/1920840878' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-12T02:56:28.319 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:28 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/1920840878' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-12T02:56:28.319 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:28 smithi148 ceph-mon[35090]: mds.? [v2:172.21.15.146:6836/1673370937,v1:172.21.15.146:6837/1673370937] up:active 2024-09-12T02:56:28.319 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:28 smithi148 ceph-mon[35090]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T02:56:28.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:28 smithi146 ceph-mon[35011]: mds.? [v2:172.21.15.146:6838/3036767228,v1:172.21.15.146:6839/3036767228] up:active 2024-09-12T02:56:28.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:28 smithi146 ceph-mon[35011]: daemon mds.h assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-09-12T02:56:28.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:28 smithi146 ceph-mon[35011]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-12T02:56:28.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:28 smithi146 ceph-mon[35011]: Cluster is now healthy 2024-09-12T02:56:28.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:28 smithi146 ceph-mon[35011]: fsmap cephfs:4 {0=i=up:active,1=c=up:active,2=f=up:active,3=b=up:active} 8 up:standby 2024-09-12T02:56:28.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:28 smithi146 ceph-mon[35011]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-09-12T02:56:28.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:28 smithi146 ceph-mon[35011]: daemon mds.h is now active in filesystem cephfs as rank 4 2024-09-12T02:56:28.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:28 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/1920840878' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-12T02:56:28.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:28 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/1920840878' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-12T02:56:28.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:28 smithi146 ceph-mon[35011]: mds.? [v2:172.21.15.146:6836/1673370937,v1:172.21.15.146:6837/1673370937] up:active 2024-09-12T02:56:28.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:28 smithi146 ceph-mon[35011]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T02:56:28.473 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated'} subvols for filesystem 'cephfs' 2024-09-12T02:56:28.474 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-09-12T02:56:28.665 INFO:journalctl@ceph.mds.i.smithi148.stdout:Sep 12 02:56:28 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-i[65552]: 2024-09-12T02:56:28.317+0000 7f3d98deb640 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2024-09-12T02:56:28.739 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:56:29.141 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:29 smithi005 ceph-mon[31197]: pgmap v280: 97 pgs: 97 active+clean; 583 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s wr, 10 op/s 2024-09-12T02:56:29.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:29.142+0000 7f8a639c0640 1 -- 172.21.15.5:0/1023506456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a5c109120 msgr2=0x7f8a5c110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:29.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:29.142+0000 7f8a639c0640 1 --2- 172.21.15.5:0/1023506456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a5c109120 0x7f8a5c110ca0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8a54009ca0 tx=0x7f8a5402a8c0 comp rx=0 tx=0).stop 2024-09-12T02:56:29.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:29.142+0000 7f8a639c0640 1 -- 172.21.15.5:0/1023506456 shutdown_connections 2024-09-12T02:56:29.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:29.142+0000 7f8a639c0640 1 --2- 172.21.15.5:0/1023506456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a5c109120 0x7f8a5c110ca0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:29.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:29.142+0000 7f8a639c0640 1 --2- 172.21.15.5:0/1023506456 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a5c1049a0 0x7f8a5c1089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:29.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:29.142+0000 7f8a639c0640 1 --2- 172.21.15.5:0/1023506456 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8a5c103ff0 0x7f8a5c1043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:29.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:29.142+0000 7f8a639c0640 1 -- 172.21.15.5:0/1023506456 >> 172.21.15.5:0/1023506456 conn(0x7f8a5c0ffab0 msgr2=0x7f8a5c101ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:29.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:29.142+0000 7f8a639c0640 1 -- 172.21.15.5:0/1023506456 shutdown_connections 2024-09-12T02:56:29.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:29.142+0000 7f8a639c0640 1 -- 172.21.15.5:0/1023506456 wait complete. 2024-09-12T02:56:29.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:29.142+0000 7f8a639c0640 1 Processor -- start 2024-09-12T02:56:29.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:29.143+0000 7f8a639c0640 1 -- start start 2024-09-12T02:56:29.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:29.143+0000 7f8a639c0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a5c103ff0 0x7f8a5c19a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:29.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:29.143+0000 7f8a639c0640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a5c1049a0 0x7f8a5c19aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:29.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:29.143+0000 7f8a639c0640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8a5c109120 0x7f8a5c19ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:29.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:29.143+0000 7f8a639c0640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8a5c114150 con 0x7f8a5c103ff0 2024-09-12T02:56:29.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:29.143+0000 7f8a639c0640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f8a5c113fd0 con 0x7f8a5c109120 2024-09-12T02:56:29.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:29.143+0000 7f8a639c0640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8a5c1142d0 con 0x7f8a5c1049a0 2024-09-12T02:56:29.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:29.143+0000 7f8a6175c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a5c103ff0 0x7f8a5c19a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:29.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:29.143+0000 7f8a6175c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a5c103ff0 0x7f8a5c19a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58348/0 (socket says 172.21.15.5:58348) 2024-09-12T02:56:29.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:29.143+0000 7f8a6175c640 1 -- 172.21.15.5:0/1881414914 learned_addr learned my addr 172.21.15.5:0/1881414914 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:56:29.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:29.143+0000 7f8a60f5b640 1 --2- 172.21.15.5:0/1881414914 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a5c1049a0 0x7f8a5c19aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:29.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:29.143+0000 7f8a61f5d640 1 --2- 172.21.15.5:0/1881414914 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8a5c109120 0x7f8a5c19ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:29.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:29.143+0000 7f8a6175c640 1 -- 172.21.15.5:0/1881414914 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a5c1049a0 msgr2=0x7f8a5c19aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:29.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:29.143+0000 7f8a6175c640 1 --2- 172.21.15.5:0/1881414914 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a5c1049a0 0x7f8a5c19aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:29.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:29.143+0000 7f8a6175c640 1 -- 172.21.15.5:0/1881414914 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8a5c109120 msgr2=0x7f8a5c19ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:29.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:29.143+0000 7f8a6175c640 1 --2- 172.21.15.5:0/1881414914 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8a5c109120 0x7f8a5c19ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:29.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:29.143+0000 7f8a6175c640 1 -- 172.21.15.5:0/1881414914 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a5c19f540 con 0x7f8a5c103ff0 2024-09-12T02:56:29.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:29.143+0000 7f8a6175c640 1 --2- 172.21.15.5:0/1881414914 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a5c103ff0 0x7f8a5c19a590 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8a5800c920 tx=0x7f8a5800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:29.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:29.144+0000 7f8a527fc640 1 -- 172.21.15.5:0/1881414914 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a58007c70 con 0x7f8a5c103ff0 2024-09-12T02:56:29.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:29.144+0000 7f8a527fc640 1 -- 172.21.15.5:0/1881414914 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a58007e10 con 0x7f8a5c103ff0 2024-09-12T02:56:29.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:29.144+0000 7f8a639c0640 1 -- 172.21.15.5:0/1881414914 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a5c19f830 con 0x7f8a5c103ff0 2024-09-12T02:56:29.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:29.144+0000 7f8a527fc640 1 -- 172.21.15.5:0/1881414914 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a58018890 con 0x7f8a5c103ff0 2024-09-12T02:56:29.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:29.144+0000 7f8a639c0640 1 -- 172.21.15.5:0/1881414914 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a5c1a7110 con 0x7f8a5c103ff0 2024-09-12T02:56:29.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:29.145+0000 7f8a639c0640 1 -- 172.21.15.5:0/1881414914 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a5c1043d0 con 0x7f8a5c103ff0 2024-09-12T02:56:29.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:29.146+0000 7f8a527fc640 1 -- 172.21.15.5:0/1881414914 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) ==== 137832+0+0 (secure 0 0 0) 0x7f8a58018a70 con 0x7f8a5c103ff0 2024-09-12T02:56:29.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:29.146+0000 7f8a527fc640 1 --2- 172.21.15.5:0/1881414914 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f8a480a2c00 0x7f8a480a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:29.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:29.147+0000 7f8a60f5b640 1 --2- 172.21.15.5:0/1881414914 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f8a480a2c00 0x7f8a480a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:29.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:29.147+0000 7f8a527fc640 1 -- 172.21.15.5:0/1881414914 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f8a580cf030 con 0x7f8a5c103ff0 2024-09-12T02:56:29.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:29.147+0000 7f8a60f5b640 1 --2- 172.21.15.5:0/1881414914 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f8a480a2c00 0x7f8a480a50c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c10aab0 tx=0x7f8a4c00b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:29.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:29.151+0000 7f8a527fc640 1 -- 172.21.15.5:0/1881414914 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8a58014030 con 0x7f8a5c103ff0 2024-09-12T02:56:29.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:29.351+0000 7f8a639c0640 1 -- 172.21.15.5:0/1881414914 --> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f8a5c101df0 con 0x7f8a480a2c00 2024-09-12T02:56:29.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:29.378+0000 7f8a527fc640 1 -- 172.21.15.5:0/1881414914 <== mgr.14148 v2:172.21.15.5:6800/3780631642 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8a5c101df0 con 0x7f8a480a2c00 2024-09-12T02:56:29.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:29.380+0000 7f8a639c0640 1 -- 172.21.15.5:0/1881414914 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f8a480a2c00 msgr2=0x7f8a480a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:29.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:29.380+0000 7f8a639c0640 1 --2- 172.21.15.5:0/1881414914 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f8a480a2c00 0x7f8a480a50c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c10aab0 tx=0x7f8a4c00b390 comp rx=0 tx=0).stop 2024-09-12T02:56:29.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:29.380+0000 7f8a639c0640 1 -- 172.21.15.5:0/1881414914 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a5c103ff0 msgr2=0x7f8a5c19a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:29.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:29.380+0000 7f8a639c0640 1 --2- 172.21.15.5:0/1881414914 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a5c103ff0 0x7f8a5c19a590 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8a5800c920 tx=0x7f8a5800cde0 comp rx=0 tx=0).stop 2024-09-12T02:56:29.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:29.380+0000 7f8a639c0640 1 -- 172.21.15.5:0/1881414914 shutdown_connections 2024-09-12T02:56:29.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:29.380+0000 7f8a639c0640 1 --2- 172.21.15.5:0/1881414914 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8a5c109120 0x7f8a5c19ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:29.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:29.380+0000 7f8a639c0640 1 --2- 172.21.15.5:0/1881414914 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8a5c1049a0 0x7f8a5c19aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:29.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:29.380+0000 7f8a639c0640 1 --2- 172.21.15.5:0/1881414914 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f8a480a2c00 0x7f8a480a50c0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:29.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:29.380+0000 7f8a639c0640 1 --2- 172.21.15.5:0/1881414914 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a5c103ff0 0x7f8a5c19a590 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:29.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:29.380+0000 7f8a639c0640 1 -- 172.21.15.5:0/1881414914 >> 172.21.15.5:0/1881414914 conn(0x7f8a5c0ffab0 msgr2=0x7f8a5c100e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:29.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:29.380+0000 7f8a639c0640 1 -- 172.21.15.5:0/1881414914 shutdown_connections 2024-09-12T02:56:29.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:29.380+0000 7f8a639c0640 1 -- 172.21.15.5:0/1881414914 wait complete. 2024-09-12T02:56:29.415 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:29 smithi148 ceph-mon[35090]: pgmap v280: 97 pgs: 97 active+clean; 583 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s wr, 10 op/s 2024-09-12T02:56:29.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:29 smithi146 ceph-mon[35011]: pgmap v280: 97 pgs: 97 active+clean; 583 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s wr, 10 op/s 2024-09-12T02:56:29.560 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:56:29 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: 2024-09-12T02:56:29.355+0000 7ff89129a640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T02:56:29.560 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:56:29 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: 2024-09-12T02:56:29.355+0000 7ff89129a640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T02:56:29.560 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:56:29 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: 2024-09-12T02:56:29.355+0000 7ff89129a640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T02:56:29.560 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:56:29 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: 2024-09-12T02:56:29.355+0000 7ff89129a640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T02:56:29.561 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:56:29 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: 2024-09-12T02:56:29.355+0000 7ff89129a640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T02:56:29.958 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-09-12T02:56:30.223 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:56:30.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:30 smithi005 ceph-mon[31197]: from='client.14691 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:56:30.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:30 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-09-12T02:56:30.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:30 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]: dispatch 2024-09-12T02:56:30.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:30 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2024-09-12T02:56:30.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:30 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]: dispatch 2024-09-12T02:56:30.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:30 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-09-12T02:56:30.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:30 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]: dispatch 2024-09-12T02:56:30.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:30 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]: dispatch 2024-09-12T02:56:30.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:30 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-09-12T02:56:30.415 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:30 smithi148 ceph-mon[35090]: from='client.14691 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:56:30.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:30 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-09-12T02:56:30.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:30 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]: dispatch 2024-09-12T02:56:30.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:30 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2024-09-12T02:56:30.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:30 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]: dispatch 2024-09-12T02:56:30.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:30 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-09-12T02:56:30.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:30 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]: dispatch 2024-09-12T02:56:30.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:30 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]: dispatch 2024-09-12T02:56:30.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:30 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-09-12T02:56:30.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:30 smithi146 ceph-mon[35011]: from='client.14691 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:56:30.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:30 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-09-12T02:56:30.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:30 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]: dispatch 2024-09-12T02:56:30.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:30 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2024-09-12T02:56:30.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:30 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]: dispatch 2024-09-12T02:56:30.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:30 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-09-12T02:56:30.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:30 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]: dispatch 2024-09-12T02:56:30.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:30 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]: dispatch 2024-09-12T02:56:30.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:30 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-09-12T02:56:30.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:30.688+0000 7fe62053d640 1 -- 172.21.15.5:0/4003225523 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe618106f10 msgr2=0x7fe61810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:30.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:30.688+0000 7fe62053d640 1 --2- 172.21.15.5:0/4003225523 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe618106f10 0x7fe61810ea90 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe610009ca0 tx=0x7fe61002a8c0 comp rx=0 tx=0).stop 2024-09-12T02:56:30.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:30.688+0000 7fe62053d640 1 -- 172.21.15.5:0/4003225523 shutdown_connections 2024-09-12T02:56:30.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:30.688+0000 7fe62053d640 1 --2- 172.21.15.5:0/4003225523 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe618106f10 0x7fe61810ea90 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:30.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:30.688+0000 7fe62053d640 1 --2- 172.21.15.5:0/4003225523 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe618102790 0x7fe6181067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:30.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:30.688+0000 7fe62053d640 1 --2- 172.21.15.5:0/4003225523 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe618101de0 0x7fe6181021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:30.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:30.688+0000 7fe62053d640 1 -- 172.21.15.5:0/4003225523 >> 172.21.15.5:0/4003225523 conn(0x7fe6180fd8c0 msgr2=0x7fe6180ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:30.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:30.688+0000 7fe62053d640 1 -- 172.21.15.5:0/4003225523 shutdown_connections 2024-09-12T02:56:30.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:30.688+0000 7fe62053d640 1 -- 172.21.15.5:0/4003225523 wait complete. 2024-09-12T02:56:30.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:30.688+0000 7fe62053d640 1 Processor -- start 2024-09-12T02:56:30.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:30.689+0000 7fe62053d640 1 -- start start 2024-09-12T02:56:30.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:30.689+0000 7fe62053d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe618101de0 0x7fe6180744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:30.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:30.689+0000 7fe62053d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe618102790 0x7fe618074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:30.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:30.689+0000 7fe62053d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe618106f10 0x7fe618074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:30.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:30.689+0000 7fe62053d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe618111f70 con 0x7fe618101de0 2024-09-12T02:56:30.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:30.689+0000 7fe62053d640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fe618111df0 con 0x7fe618102790 2024-09-12T02:56:30.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:30.689+0000 7fe62053d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe6181120f0 con 0x7fe618106f10 2024-09-12T02:56:30.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:30.689+0000 7fe61e2d9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe618101de0 0x7fe6180744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:30.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:30.689+0000 7fe61e2d9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe618101de0 0x7fe6180744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58390/0 (socket says 172.21.15.5:58390) 2024-09-12T02:56:30.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:30.689+0000 7fe61e2d9640 1 -- 172.21.15.5:0/3792792611 learned_addr learned my addr 172.21.15.5:0/3792792611 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:56:30.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:30.689+0000 7fe61dad8640 1 --2- 172.21.15.5:0/3792792611 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe618102790 0x7fe618074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:30.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:30.689+0000 7fe61eada640 1 --2- 172.21.15.5:0/3792792611 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe618106f10 0x7fe618074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:30.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:30.689+0000 7fe61e2d9640 1 -- 172.21.15.5:0/3792792611 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe618106f10 msgr2=0x7fe618074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:30.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:30.689+0000 7fe61e2d9640 1 --2- 172.21.15.5:0/3792792611 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe618106f10 0x7fe618074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:30.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:30.689+0000 7fe61e2d9640 1 -- 172.21.15.5:0/3792792611 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe618102790 msgr2=0x7fe618074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:30.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:30.689+0000 7fe61e2d9640 1 --2- 172.21.15.5:0/3792792611 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe618102790 0x7fe618074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:30.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:30.689+0000 7fe61e2d9640 1 -- 172.21.15.5:0/3792792611 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6181a68b0 con 0x7fe618101de0 2024-09-12T02:56:30.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:30.689+0000 7fe61dad8640 1 --2- 172.21.15.5:0/3792792611 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe618102790 0x7fe618074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:56:30.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:30.689+0000 7fe61e2d9640 1 --2- 172.21.15.5:0/3792792611 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe618101de0 0x7fe6180744f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe61400c890 tx=0x7fe61400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:30.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:30.690+0000 7fe60f7fe640 1 -- 172.21.15.5:0/3792792611 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe614004220 con 0x7fe618101de0 2024-09-12T02:56:30.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:30.690+0000 7fe62053d640 1 -- 172.21.15.5:0/3792792611 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6181a6ba0 con 0x7fe618101de0 2024-09-12T02:56:30.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:30.690+0000 7fe60f7fe640 1 -- 172.21.15.5:0/3792792611 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6140043c0 con 0x7fe618101de0 2024-09-12T02:56:30.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:30.690+0000 7fe60f7fe640 1 -- 172.21.15.5:0/3792792611 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe614004a00 con 0x7fe618101de0 2024-09-12T02:56:30.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:30.690+0000 7fe62053d640 1 -- 172.21.15.5:0/3792792611 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6181a70e0 con 0x7fe618101de0 2024-09-12T02:56:30.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:30.691+0000 7fe62053d640 1 -- 172.21.15.5:0/3792792611 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6181021c0 con 0x7fe618101de0 2024-09-12T02:56:30.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:30.692+0000 7fe60f7fe640 1 -- 172.21.15.5:0/3792792611 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fe6140079b0 con 0x7fe618101de0 2024-09-12T02:56:30.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:30.692+0000 7fe60f7fe640 1 --2- 172.21.15.5:0/3792792611 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fe6040a2c10 0x7fe6040a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:30.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:30.693+0000 7fe61dad8640 1 --2- 172.21.15.5:0/3792792611 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fe6040a2c10 0x7fe6040a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:30.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:30.693+0000 7fe60f7fe640 1 -- 172.21.15.5:0/3792792611 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8211+0+0 (secure 0 0 0) 0x7fe6140cef70 con 0x7fe618101de0 2024-09-12T02:56:30.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:30.693+0000 7fe61dad8640 1 --2- 172.21.15.5:0/3792792611 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fe6040a2c10 0x7fe6040a50d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe608005d70 tx=0x7fe608005d00 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:30.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:30.697+0000 7fe60f7fe640 1 -- 172.21.15.5:0/3792792611 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe614014030 con 0x7fe618101de0 2024-09-12T02:56:30.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:30.898+0000 7fe62053d640 1 -- 172.21.15.5:0/3792792611 --> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fe618062630 con 0x7fe6040a2c10 2024-09-12T02:56:30.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:30.922+0000 7fe60f7fe640 1 -- 172.21.15.5:0/3792792611 <== mgr.14148 v2:172.21.15.5:6800/3780631642 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe618062630 con 0x7fe6040a2c10 2024-09-12T02:56:30.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:30.924+0000 7fe62053d640 1 -- 172.21.15.5:0/3792792611 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fe6040a2c10 msgr2=0x7fe6040a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:30.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:30.924+0000 7fe62053d640 1 --2- 172.21.15.5:0/3792792611 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fe6040a2c10 0x7fe6040a50d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe608005d70 tx=0x7fe608005d00 comp rx=0 tx=0).stop 2024-09-12T02:56:30.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:30.924+0000 7fe62053d640 1 -- 172.21.15.5:0/3792792611 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe618101de0 msgr2=0x7fe6180744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:30.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:30.924+0000 7fe62053d640 1 --2- 172.21.15.5:0/3792792611 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe618101de0 0x7fe6180744f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe61400c890 tx=0x7fe61400cd50 comp rx=0 tx=0).stop 2024-09-12T02:56:30.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:30.924+0000 7fe62053d640 1 -- 172.21.15.5:0/3792792611 shutdown_connections 2024-09-12T02:56:30.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:30.924+0000 7fe62053d640 1 --2- 172.21.15.5:0/3792792611 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe618106f10 0x7fe618074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:30.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:30.924+0000 7fe62053d640 1 --2- 172.21.15.5:0/3792792611 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe618102790 0x7fe618074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:30.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:30.924+0000 7fe62053d640 1 --2- 172.21.15.5:0/3792792611 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fe6040a2c10 0x7fe6040a50d0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:30.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:30.924+0000 7fe62053d640 1 --2- 172.21.15.5:0/3792792611 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe618101de0 0x7fe6180744f0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:30.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:30.924+0000 7fe62053d640 1 -- 172.21.15.5:0/3792792611 >> 172.21.15.5:0/3792792611 conn(0x7fe6180fd8c0 msgr2=0x7fe6180febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:30.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:30.924+0000 7fe62053d640 1 -- 172.21.15.5:0/3792792611 shutdown_connections 2024-09-12T02:56:30.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:30.924+0000 7fe62053d640 1 -- 172.21.15.5:0/3792792611 wait complete. 2024-09-12T02:56:31.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:31 smithi005 ceph-mon[31197]: pgmap v281: 97 pgs: 97 active+clean; 584 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 3.8 KiB/s wr, 16 op/s 2024-09-12T02:56:31.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:31 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-09-12T02:56:31.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:31 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]': finished 2024-09-12T02:56:31.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:31 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-09-12T02:56:31.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:31 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]': finished 2024-09-12T02:56:31.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:31 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-09-12T02:56:31.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:31 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]': finished 2024-09-12T02:56:31.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:31 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]': finished 2024-09-12T02:56:31.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:31 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-09-12T02:56:31.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:31 smithi005 ceph-mon[31197]: osdmap e82: 12 total, 12 up, 12 in 2024-09-12T02:56:31.311 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:31 smithi005 ceph-mon[31197]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-12T02:56:31.415 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:31 smithi148 ceph-mon[35090]: pgmap v281: 97 pgs: 97 active+clean; 584 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 3.8 KiB/s wr, 16 op/s 2024-09-12T02:56:31.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:31 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-09-12T02:56:31.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:31 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]': finished 2024-09-12T02:56:31.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:31 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-09-12T02:56:31.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:31 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]': finished 2024-09-12T02:56:31.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:31 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-09-12T02:56:31.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:31 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]': finished 2024-09-12T02:56:31.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:31 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]': finished 2024-09-12T02:56:31.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:31 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-09-12T02:56:31.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:31 smithi148 ceph-mon[35090]: osdmap e82: 12 total, 12 up, 12 in 2024-09-12T02:56:31.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:31 smithi148 ceph-mon[35090]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-12T02:56:31.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:31 smithi146 ceph-mon[35011]: pgmap v281: 97 pgs: 97 active+clean; 584 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 3.8 KiB/s wr, 16 op/s 2024-09-12T02:56:31.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:31 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-09-12T02:56:31.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:31 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]': finished 2024-09-12T02:56:31.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:31 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-09-12T02:56:31.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:31 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]': finished 2024-09-12T02:56:31.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:31 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-09-12T02:56:31.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:31 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]': finished 2024-09-12T02:56:31.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:31 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]': finished 2024-09-12T02:56:31.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:31 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-09-12T02:56:31.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:31 smithi146 ceph-mon[35011]: osdmap e82: 12 total, 12 up, 12 in 2024-09-12T02:56:31.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:31 smithi146 ceph-mon[35011]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-12T02:56:31.687 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-09-12T02:56:31.948 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:56:32.299 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:32 smithi005 ceph-mon[31197]: from='client.14703 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:56:32.299 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:32 smithi005 ceph-mon[31197]: osdmap e83: 12 total, 12 up, 12 in 2024-09-12T02:56:32.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:32.386+0000 7f48b547a640 1 -- 172.21.15.5:0/2571664650 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48b0100a80 msgr2=0x7f48b0100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:32.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:32.386+0000 7f48b547a640 1 --2- 172.21.15.5:0/2571664650 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48b0100a80 0x7f48b0100ee0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f4898009930 tx=0x7f489802a420 comp rx=0 tx=0).stop 2024-09-12T02:56:32.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:32.386+0000 7f48b547a640 1 -- 172.21.15.5:0/2571664650 shutdown_connections 2024-09-12T02:56:32.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:32.386+0000 7f48b547a640 1 --2- 172.21.15.5:0/2571664650 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48b0100a80 0x7f48b0100ee0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:32.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:32.386+0000 7f48b547a640 1 --2- 172.21.15.5:0/2571664650 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48b0101420 0x7f48b010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:32.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:32.386+0000 7f48b547a640 1 --2- 172.21.15.5:0/2571664650 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f48b0106a80 0x7f48b0106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:32.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:32.386+0000 7f48b547a640 1 -- 172.21.15.5:0/2571664650 >> 172.21.15.5:0/2571664650 conn(0x7f48b00fc7a0 msgr2=0x7f48b00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:32.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:32.386+0000 7f48b547a640 1 -- 172.21.15.5:0/2571664650 shutdown_connections 2024-09-12T02:56:32.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:32.386+0000 7f48b547a640 1 -- 172.21.15.5:0/2571664650 wait complete. 2024-09-12T02:56:32.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:32.387+0000 7f48b547a640 1 Processor -- start 2024-09-12T02:56:32.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:32.387+0000 7f48b547a640 1 -- start start 2024-09-12T02:56:32.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:32.387+0000 7f48b547a640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f48b0100a80 0x7f48b01974c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:32.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:32.387+0000 7f48b547a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48b0101420 0x7f48b0197a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:32.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:32.387+0000 7f48b547a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48b0106a80 0x7f48b019bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:32.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:32.387+0000 7f48b547a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f48b0110f20 con 0x7f48b0101420 2024-09-12T02:56:32.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:32.387+0000 7f48b547a640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f48b0110da0 con 0x7f48b0100a80 2024-09-12T02:56:32.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:32.387+0000 7f48b547a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f48b01110a0 con 0x7f48b0106a80 2024-09-12T02:56:32.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:32.387+0000 7f48a67fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48b0101420 0x7f48b0197a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:32.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:32.387+0000 7f48a67fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48b0101420 0x7f48b0197a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58410/0 (socket says 172.21.15.5:58410) 2024-09-12T02:56:32.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:32.387+0000 7f48a67fc640 1 -- 172.21.15.5:0/3731953395 learned_addr learned my addr 172.21.15.5:0/3731953395 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:56:32.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:32.387+0000 7f48aeffd640 1 --2- 172.21.15.5:0/3731953395 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f48b0100a80 0x7f48b01974c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:32.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:32.387+0000 7f48a67fc640 1 -- 172.21.15.5:0/3731953395 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48b0106a80 msgr2=0x7f48b019bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:32.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:32.387+0000 7f48a67fc640 1 --2- 172.21.15.5:0/3731953395 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48b0106a80 0x7f48b019bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:32.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:32.387+0000 7f48a67fc640 1 -- 172.21.15.5:0/3731953395 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f48b0100a80 msgr2=0x7f48b01974c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:32.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:32.387+0000 7f48a67fc640 1 --2- 172.21.15.5:0/3731953395 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f48b0100a80 0x7f48b01974c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:32.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:32.387+0000 7f48a67fc640 1 -- 172.21.15.5:0/3731953395 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48b019c470 con 0x7f48b0101420 2024-09-12T02:56:32.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:32.387+0000 7f48aeffd640 1 --2- 172.21.15.5:0/3731953395 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f48b0100a80 0x7f48b01974c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:56:32.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:32.388+0000 7f48a67fc640 1 --2- 172.21.15.5:0/3731953395 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48b0101420 0x7f48b0197a00 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f4898009a50 tx=0x7f4898004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:32.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:32.388+0000 7f48acff9640 1 -- 172.21.15.5:0/3731953395 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f489803f030 con 0x7f48b0101420 2024-09-12T02:56:32.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:32.388+0000 7f48b547a640 1 -- 172.21.15.5:0/3731953395 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48b019c700 con 0x7f48b0101420 2024-09-12T02:56:32.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:32.388+0000 7f48acff9640 1 -- 172.21.15.5:0/3731953395 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4898003ce0 con 0x7f48b0101420 2024-09-12T02:56:32.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:32.388+0000 7f48acff9640 1 -- 172.21.15.5:0/3731953395 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4898035070 con 0x7f48b0101420 2024-09-12T02:56:32.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:32.388+0000 7f48b547a640 1 -- 172.21.15.5:0/3731953395 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48b01a4040 con 0x7f48b0101420 2024-09-12T02:56:32.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:32.389+0000 7f48b547a640 1 -- 172.21.15.5:0/3731953395 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48b0100f60 con 0x7f48b0101420 2024-09-12T02:56:32.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:32.390+0000 7f48acff9640 1 -- 172.21.15.5:0/3731953395 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f4898042030 con 0x7f48b0101420 2024-09-12T02:56:32.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:32.391+0000 7f48acff9640 1 --2- 172.21.15.5:0/3731953395 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f489c0a2c10 0x7f489c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:32.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:32.391+0000 7f48aeffd640 1 --2- 172.21.15.5:0/3731953395 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f489c0a2c10 0x7f489c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:32.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:32.391+0000 7f48acff9640 1 -- 172.21.15.5:0/3731953395 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f48980efcb0 con 0x7f48b0101420 2024-09-12T02:56:32.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:32.391+0000 7f48aeffd640 1 --2- 172.21.15.5:0/3731953395 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f489c0a2c10 0x7f489c0a50d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f48a0006fd0 tx=0x7f48a0008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:32.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:32.394+0000 7f48acff9640 1 -- 172.21.15.5:0/3731953395 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f48980bcac0 con 0x7f48b0101420 2024-09-12T02:56:32.415 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:32 smithi148 ceph-mon[35090]: from='client.14703 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:56:32.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:32 smithi148 ceph-mon[35090]: osdmap e83: 12 total, 12 up, 12 in 2024-09-12T02:56:32.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:32 smithi146 ceph-mon[35011]: from='client.14703 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:56:32.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:32 smithi146 ceph-mon[35011]: osdmap e83: 12 total, 12 up, 12 in 2024-09-12T02:56:32.598 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:32.598+0000 7f48b547a640 1 -- 172.21.15.5:0/3731953395 --> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f48b0062630 con 0x7f489c0a2c10 2024-09-12T02:56:32.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:32.602+0000 7f48acff9640 1 -- 172.21.15.5:0/3731953395 <== mgr.14148 v2:172.21.15.5:6800/3780631642 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f48b0062630 con 0x7f489c0a2c10 2024-09-12T02:56:32.602 INFO:teuthology.orchestra.run.smithi005.stdout:/volumes/qa/sv_0/9ddc0410-23b7-4188-8efb-320602019e37 2024-09-12T02:56:32.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:32.604+0000 7f48b547a640 1 -- 172.21.15.5:0/3731953395 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f489c0a2c10 msgr2=0x7f489c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:32.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:32.605+0000 7f48b547a640 1 --2- 172.21.15.5:0/3731953395 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f489c0a2c10 0x7f489c0a50d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f48a0006fd0 tx=0x7f48a0008040 comp rx=0 tx=0).stop 2024-09-12T02:56:32.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:32.605+0000 7f48b547a640 1 -- 172.21.15.5:0/3731953395 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48b0101420 msgr2=0x7f48b0197a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:32.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:32.605+0000 7f48b547a640 1 --2- 172.21.15.5:0/3731953395 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48b0101420 0x7f48b0197a00 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f4898009a50 tx=0x7f4898004270 comp rx=0 tx=0).stop 2024-09-12T02:56:32.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:32.605+0000 7f48b547a640 1 -- 172.21.15.5:0/3731953395 shutdown_connections 2024-09-12T02:56:32.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:32.605+0000 7f48b547a640 1 --2- 172.21.15.5:0/3731953395 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f489c0a2c10 0x7f489c0a50d0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:32.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:32.605+0000 7f48b547a640 1 --2- 172.21.15.5:0/3731953395 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48b0101420 0x7f48b0197a00 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:32.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:32.605+0000 7f48b547a640 1 --2- 172.21.15.5:0/3731953395 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f48b0106a80 0x7f48b019bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:32.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:32.605+0000 7f48b547a640 1 --2- 172.21.15.5:0/3731953395 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f48b0100a80 0x7f48b01974c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:32.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:32.605+0000 7f48b547a640 1 -- 172.21.15.5:0/3731953395 >> 172.21.15.5:0/3731953395 conn(0x7f48b00fc7a0 msgr2=0x7f48b00fd020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:32.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:32.605+0000 7f48b547a640 1 -- 172.21.15.5:0/3731953395 shutdown_connections 2024-09-12T02:56:32.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:32.605+0000 7f48b547a640 1 -- 172.21.15.5:0/3731953395 wait complete. 2024-09-12T02:56:33.170 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:33 smithi005 ceph-mon[31197]: pgmap v284: 97 pgs: 1 peering, 96 active+clean; 585 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6.1 KiB/s wr, 23 op/s 2024-09-12T02:56:33.201 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-09-12T02:56:33.415 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:33 smithi148 ceph-mon[35090]: pgmap v284: 97 pgs: 1 peering, 96 active+clean; 585 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6.1 KiB/s wr, 23 op/s 2024-09-12T02:56:33.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:33 smithi146 ceph-mon[35011]: pgmap v284: 97 pgs: 1 peering, 96 active+clean; 585 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6.1 KiB/s wr, 23 op/s 2024-09-12T02:56:33.457 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:56:33.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:33.944+0000 7f7f2a489640 1 -- 172.21.15.5:0/537751029 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f24103660 msgr2=0x7f7f2410bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:33.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:33.944+0000 7f7f2a489640 1 --2- 172.21.15.5:0/537751029 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f24103660 0x7f7f2410bb50 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7f18009900 tx=0x7f7f1802a480 comp rx=0 tx=0).stop 2024-09-12T02:56:33.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:33.944+0000 7f7f2a489640 1 -- 172.21.15.5:0/537751029 shutdown_connections 2024-09-12T02:56:33.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:33.944+0000 7f7f2a489640 1 --2- 172.21.15.5:0/537751029 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f24103660 0x7f7f2410bb50 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:33.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:33.944+0000 7f7f2a489640 1 --2- 172.21.15.5:0/537751029 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7f2410c170 0x7f7f2410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:33.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:33.944+0000 7f7f2a489640 1 --2- 172.21.15.5:0/537751029 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7f24102d40 0x7f7f24103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:33.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:33.944+0000 7f7f2a489640 1 -- 172.21.15.5:0/537751029 >> 172.21.15.5:0/537751029 conn(0x7f7f240fc7a0 msgr2=0x7f7f240febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:33.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:33.944+0000 7f7f2a489640 1 -- 172.21.15.5:0/537751029 shutdown_connections 2024-09-12T02:56:33.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:33.944+0000 7f7f2a489640 1 -- 172.21.15.5:0/537751029 wait complete. 2024-09-12T02:56:33.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:33.945+0000 7f7f2a489640 1 Processor -- start 2024-09-12T02:56:33.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:33.945+0000 7f7f2a489640 1 -- start start 2024-09-12T02:56:33.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:33.945+0000 7f7f2a489640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f24102d40 0x7f7f24198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:33.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:33.945+0000 7f7f2a489640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7f24103660 0x7f7f24198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:33.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:33.945+0000 7f7f2a489640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7f2410c170 0x7f7f2419cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:33.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:33.945+0000 7f7f2a489640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7f24112030 con 0x7f7f24102d40 2024-09-12T02:56:33.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:33.945+0000 7f7f2a489640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f7f24111eb0 con 0x7f7f2410c170 2024-09-12T02:56:33.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:33.945+0000 7f7f2a489640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f7f241121b0 con 0x7f7f24103660 2024-09-12T02:56:33.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:33.946+0000 7f7f23fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f24102d40 0x7f7f24198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:33.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:33.946+0000 7f7f23fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f24102d40 0x7f7f24198690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49540/0 (socket says 172.21.15.5:49540) 2024-09-12T02:56:33.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:33.946+0000 7f7f23fff640 1 -- 172.21.15.5:0/3806737612 learned_addr learned my addr 172.21.15.5:0/3806737612 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:56:33.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:33.946+0000 7f7f237fe640 1 --2- 172.21.15.5:0/3806737612 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7f24103660 0x7f7f24198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:33.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:33.946+0000 7f7f28a26640 1 --2- 172.21.15.5:0/3806737612 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7f2410c170 0x7f7f2419cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:33.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:33.946+0000 7f7f23fff640 1 -- 172.21.15.5:0/3806737612 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7f24103660 msgr2=0x7f7f24198bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:33.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:33.946+0000 7f7f23fff640 1 --2- 172.21.15.5:0/3806737612 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7f24103660 0x7f7f24198bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:33.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:33.946+0000 7f7f23fff640 1 -- 172.21.15.5:0/3806737612 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7f2410c170 msgr2=0x7f7f2419cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:33.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:33.946+0000 7f7f23fff640 1 --2- 172.21.15.5:0/3806737612 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7f2410c170 0x7f7f2419cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:33.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:33.946+0000 7f7f23fff640 1 -- 172.21.15.5:0/3806737612 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f2419d640 con 0x7f7f24102d40 2024-09-12T02:56:33.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:33.946+0000 7f7f237fe640 1 --2- 172.21.15.5:0/3806737612 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7f24103660 0x7f7f24198bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:56:33.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:33.946+0000 7f7f23fff640 1 --2- 172.21.15.5:0/3806737612 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f24102d40 0x7f7f24198690 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f7f1400b4a0 tx=0x7f7f1400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:33.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:33.946+0000 7f7f217fa640 1 -- 172.21.15.5:0/3806737612 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f14014070 con 0x7f7f24102d40 2024-09-12T02:56:33.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:33.946+0000 7f7f217fa640 1 -- 172.21.15.5:0/3806737612 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f14010af0 con 0x7f7f24102d40 2024-09-12T02:56:33.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:33.946+0000 7f7f2a489640 1 -- 172.21.15.5:0/3806737612 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f2419d930 con 0x7f7f24102d40 2024-09-12T02:56:33.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:33.946+0000 7f7f217fa640 1 -- 172.21.15.5:0/3806737612 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f14017d10 con 0x7f7f24102d40 2024-09-12T02:56:33.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:33.946+0000 7f7f2a489640 1 -- 172.21.15.5:0/3806737612 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f241a5210 con 0x7f7f24102d40 2024-09-12T02:56:33.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:33.947+0000 7f7f2a489640 1 -- 172.21.15.5:0/3806737612 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f241031a0 con 0x7f7f24102d40 2024-09-12T02:56:33.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:33.949+0000 7f7f217fa640 1 -- 172.21.15.5:0/3806737612 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f7f14017070 con 0x7f7f24102d40 2024-09-12T02:56:33.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:33.949+0000 7f7f217fa640 1 --2- 172.21.15.5:0/3806737612 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f7f0c0a2c10 0x7f7f0c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:33.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:33.949+0000 7f7f237fe640 1 --2- 172.21.15.5:0/3806737612 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f7f0c0a2c10 0x7f7f0c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:33.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:33.949+0000 7f7f217fa640 1 -- 172.21.15.5:0/3806737612 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f7f140ce8c0 con 0x7f7f24102d40 2024-09-12T02:56:33.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:33.950+0000 7f7f237fe640 1 --2- 172.21.15.5:0/3806737612 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f7f0c0a2c10 0x7f7f0c0a50d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7f240fee70 tx=0x7f7f18005a20 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:33.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:33.952+0000 7f7f217fa640 1 -- 172.21.15.5:0/3806737612 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7f1409b7e0 con 0x7f7f24102d40 2024-09-12T02:56:34.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:34.154+0000 7f7f2a489640 1 -- 172.21.15.5:0/3806737612 --> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f7f24062630 con 0x7f7f0c0a2c10 2024-09-12T02:56:34.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:34 smithi005 ceph-mon[31197]: from='client.14709 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:56:34.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:34.179+0000 7f7f217fa640 1 -- 172.21.15.5:0/3806737612 <== mgr.14148 v2:172.21.15.5:6800/3780631642 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f7f24062630 con 0x7f7f0c0a2c10 2024-09-12T02:56:34.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:34.181+0000 7f7f2a489640 1 -- 172.21.15.5:0/3806737612 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f7f0c0a2c10 msgr2=0x7f7f0c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:34.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:34.181+0000 7f7f2a489640 1 --2- 172.21.15.5:0/3806737612 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f7f0c0a2c10 0x7f7f0c0a50d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7f240fee70 tx=0x7f7f18005a20 comp rx=0 tx=0).stop 2024-09-12T02:56:34.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:34.181+0000 7f7f2a489640 1 -- 172.21.15.5:0/3806737612 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f24102d40 msgr2=0x7f7f24198690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:34.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:34.181+0000 7f7f2a489640 1 --2- 172.21.15.5:0/3806737612 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f24102d40 0x7f7f24198690 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f7f1400b4a0 tx=0x7f7f1400b960 comp rx=0 tx=0).stop 2024-09-12T02:56:34.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:34.181+0000 7f7f2a489640 1 -- 172.21.15.5:0/3806737612 shutdown_connections 2024-09-12T02:56:34.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:34.181+0000 7f7f2a489640 1 --2- 172.21.15.5:0/3806737612 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7f2410c170 0x7f7f2419cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:34.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:34.181+0000 7f7f2a489640 1 --2- 172.21.15.5:0/3806737612 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7f24103660 0x7f7f24198bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:34.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:34.181+0000 7f7f2a489640 1 --2- 172.21.15.5:0/3806737612 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f7f0c0a2c10 0x7f7f0c0a50d0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:34.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:34.181+0000 7f7f2a489640 1 --2- 172.21.15.5:0/3806737612 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7f24102d40 0x7f7f24198690 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:34.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:34.181+0000 7f7f2a489640 1 -- 172.21.15.5:0/3806737612 >> 172.21.15.5:0/3806737612 conn(0x7f7f240fc7a0 msgr2=0x7f7f240fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:34.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:34.181+0000 7f7f2a489640 1 -- 172.21.15.5:0/3806737612 shutdown_connections 2024-09-12T02:56:34.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:34.181+0000 7f7f2a489640 1 -- 172.21.15.5:0/3806737612 wait complete. 2024-09-12T02:56:34.415 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:34 smithi148 ceph-mon[35090]: from='client.14709 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:56:34.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:34 smithi146 ceph-mon[35011]: from='client.14709 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:56:34.833 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-09-12T02:56:35.094 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:56:35.306 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:35 smithi005 ceph-mon[31197]: pgmap v285: 97 pgs: 1 peering, 96 active+clean; 610 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 9.6 KiB/s wr, 28 op/s; 91 B/s, 1 objects/s recovering 2024-09-12T02:56:35.307 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:35 smithi005 ceph-mon[31197]: from='client.14715 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:56:35.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:35 smithi148 ceph-mon[35090]: pgmap v285: 97 pgs: 1 peering, 96 active+clean; 610 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 9.6 KiB/s wr, 28 op/s; 91 B/s, 1 objects/s recovering 2024-09-12T02:56:35.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:35 smithi148 ceph-mon[35090]: from='client.14715 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:56:35.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:35 smithi146 ceph-mon[35011]: pgmap v285: 97 pgs: 1 peering, 96 active+clean; 610 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 9.6 KiB/s wr, 28 op/s; 91 B/s, 1 objects/s recovering 2024-09-12T02:56:35.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:35 smithi146 ceph-mon[35011]: from='client.14715 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:56:35.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:35.537+0000 7fb98144d640 1 -- 172.21.15.5:0/3596287900 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb97c102d90 msgr2=0x7fb97c103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:35.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:35.537+0000 7fb98144d640 1 --2- 172.21.15.5:0/3596287900 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb97c102d90 0x7fb97c103170 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb96c009cd0 tx=0x7fb96c02a8e0 comp rx=0 tx=0).stop 2024-09-12T02:56:35.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:35.537+0000 7fb98144d640 1 -- 172.21.15.5:0/3596287900 shutdown_connections 2024-09-12T02:56:35.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:35.537+0000 7fb98144d640 1 --2- 172.21.15.5:0/3596287900 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb97c108290 0x7fb97c10ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:35.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:35.537+0000 7fb98144d640 1 --2- 172.21.15.5:0/3596287900 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb97c103740 0x7fb97c107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:35.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:35.537+0000 7fb98144d640 1 --2- 172.21.15.5:0/3596287900 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb97c102d90 0x7fb97c103170 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:35.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:35.537+0000 7fb98144d640 1 -- 172.21.15.5:0/3596287900 >> 172.21.15.5:0/3596287900 conn(0x7fb97c0fc7d0 msgr2=0x7fb97c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:35.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:35.537+0000 7fb98144d640 1 -- 172.21.15.5:0/3596287900 shutdown_connections 2024-09-12T02:56:35.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:35.537+0000 7fb98144d640 1 -- 172.21.15.5:0/3596287900 wait complete. 2024-09-12T02:56:35.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:35.538+0000 7fb98144d640 1 Processor -- start 2024-09-12T02:56:35.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:35.538+0000 7fb98144d640 1 -- start start 2024-09-12T02:56:35.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:35.538+0000 7fb98144d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb97c102d90 0x7fb97c1985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:35.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:35.538+0000 7fb98144d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb97c103740 0x7fb97c198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:35.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:35.538+0000 7fb98144d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb97c108290 0x7fb97c19cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:35.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:35.538+0000 7fb98144d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb97c112090 con 0x7fb97c103740 2024-09-12T02:56:35.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:35.538+0000 7fb98144d640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fb97c111f10 con 0x7fb97c108290 2024-09-12T02:56:35.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:35.538+0000 7fb98144d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb97c112210 con 0x7fb97c102d90 2024-09-12T02:56:35.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:35.539+0000 7fb97a7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb97c103740 0x7fb97c198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:35.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:35.539+0000 7fb97a7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb97c103740 0x7fb97c198b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49556/0 (socket says 172.21.15.5:49556) 2024-09-12T02:56:35.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:35.539+0000 7fb97a7fc640 1 -- 172.21.15.5:0/2402569214 learned_addr learned my addr 172.21.15.5:0/2402569214 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:56:35.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:35.539+0000 7fb97affd640 1 --2- 172.21.15.5:0/2402569214 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb97c102d90 0x7fb97c1985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:35.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:35.539+0000 7fb97b7fe640 1 --2- 172.21.15.5:0/2402569214 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb97c108290 0x7fb97c19cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:35.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:35.539+0000 7fb97a7fc640 1 -- 172.21.15.5:0/2402569214 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb97c102d90 msgr2=0x7fb97c1985f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:35.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:35.539+0000 7fb97a7fc640 1 --2- 172.21.15.5:0/2402569214 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb97c102d90 0x7fb97c1985f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:35.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:35.539+0000 7fb97a7fc640 1 -- 172.21.15.5:0/2402569214 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb97c108290 msgr2=0x7fb97c19cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:35.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:35.539+0000 7fb97a7fc640 1 --2- 172.21.15.5:0/2402569214 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb97c108290 0x7fb97c19cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:35.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:35.539+0000 7fb97a7fc640 1 -- 172.21.15.5:0/2402569214 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb97c19d5a0 con 0x7fb97c103740 2024-09-12T02:56:35.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:35.539+0000 7fb97b7fe640 1 --2- 172.21.15.5:0/2402569214 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb97c108290 0x7fb97c19cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:56:35.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:35.539+0000 7fb97affd640 1 --2- 172.21.15.5:0/2402569214 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb97c102d90 0x7fb97c1985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:56:35.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:35.539+0000 7fb97a7fc640 1 --2- 172.21.15.5:0/2402569214 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb97c103740 0x7fb97c198b30 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb97000e9f0 tx=0x7fb97000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:35.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:35.539+0000 7fb963fff640 1 -- 172.21.15.5:0/2402569214 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb97000ccc0 con 0x7fb97c103740 2024-09-12T02:56:35.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:35.539+0000 7fb98144d640 1 -- 172.21.15.5:0/2402569214 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb97c19d890 con 0x7fb97c103740 2024-09-12T02:56:35.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:35.539+0000 7fb963fff640 1 -- 172.21.15.5:0/2402569214 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb970005500 con 0x7fb97c103740 2024-09-12T02:56:35.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:35.539+0000 7fb963fff640 1 -- 172.21.15.5:0/2402569214 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb970010690 con 0x7fb97c103740 2024-09-12T02:56:35.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:35.539+0000 7fb98144d640 1 -- 172.21.15.5:0/2402569214 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb97c1a5170 con 0x7fb97c103740 2024-09-12T02:56:35.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:35.540+0000 7fb98144d640 1 -- 172.21.15.5:0/2402569214 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb97c06a7f0 con 0x7fb97c103740 2024-09-12T02:56:35.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:35.542+0000 7fb963fff640 1 -- 172.21.15.5:0/2402569214 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fb9700046c0 con 0x7fb97c103740 2024-09-12T02:56:35.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:35.542+0000 7fb963fff640 1 --2- 172.21.15.5:0/2402569214 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fb9640a2c10 0x7fb9640a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:35.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:35.542+0000 7fb97affd640 1 --2- 172.21.15.5:0/2402569214 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fb9640a2c10 0x7fb9640a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:35.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:35.543+0000 7fb963fff640 1 -- 172.21.15.5:0/2402569214 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fb9700cf970 con 0x7fb97c103740 2024-09-12T02:56:35.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:35.543+0000 7fb97affd640 1 --2- 172.21.15.5:0/2402569214 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fb9640a2c10 0x7fb9640a50d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb96c000f80 tx=0x7fb96c0093f0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:35.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:35.545+0000 7fb963fff640 1 -- 172.21.15.5:0/2402569214 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb97001f340 con 0x7fb97c103740 2024-09-12T02:56:35.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:35.749+0000 7fb98144d640 1 -- 172.21.15.5:0/2402569214 --> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7fb97c062630 con 0x7fb9640a2c10 2024-09-12T02:56:35.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:35.754+0000 7fb963fff640 1 -- 172.21.15.5:0/2402569214 <== mgr.14148 v2:172.21.15.5:6800/3780631642 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fb97c062630 con 0x7fb9640a2c10 2024-09-12T02:56:35.754 INFO:teuthology.orchestra.run.smithi005.stdout:/volumes/qa/sv_1/e6595afd-b733-4dfd-b518-50fa51993b54 2024-09-12T02:56:35.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:35.756+0000 7fb98144d640 1 -- 172.21.15.5:0/2402569214 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fb9640a2c10 msgr2=0x7fb9640a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:35.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:35.756+0000 7fb98144d640 1 --2- 172.21.15.5:0/2402569214 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fb9640a2c10 0x7fb9640a50d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb96c000f80 tx=0x7fb96c0093f0 comp rx=0 tx=0).stop 2024-09-12T02:56:35.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:35.756+0000 7fb98144d640 1 -- 172.21.15.5:0/2402569214 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb97c103740 msgr2=0x7fb97c198b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:35.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:35.756+0000 7fb98144d640 1 --2- 172.21.15.5:0/2402569214 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb97c103740 0x7fb97c198b30 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb97000e9f0 tx=0x7fb97000eeb0 comp rx=0 tx=0).stop 2024-09-12T02:56:35.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:35.756+0000 7fb98144d640 1 -- 172.21.15.5:0/2402569214 shutdown_connections 2024-09-12T02:56:35.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:35.756+0000 7fb98144d640 1 --2- 172.21.15.5:0/2402569214 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fb9640a2c10 0x7fb9640a50d0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:35.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:35.756+0000 7fb98144d640 1 --2- 172.21.15.5:0/2402569214 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb97c103740 0x7fb97c198b30 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:35.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:35.756+0000 7fb98144d640 1 --2- 172.21.15.5:0/2402569214 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb97c108290 0x7fb97c19cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:35.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:35.756+0000 7fb98144d640 1 --2- 172.21.15.5:0/2402569214 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb97c102d90 0x7fb97c1985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:35.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:35.756+0000 7fb98144d640 1 -- 172.21.15.5:0/2402569214 >> 172.21.15.5:0/2402569214 conn(0x7fb97c0fc7d0 msgr2=0x7fb97c105e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:35.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:35.756+0000 7fb98144d640 1 -- 172.21.15.5:0/2402569214 shutdown_connections 2024-09-12T02:56:35.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:35.756+0000 7fb98144d640 1 -- 172.21.15.5:0/2402569214 wait complete. 2024-09-12T02:56:36.400 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:56:36.657 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:56:37.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:37.072+0000 7f0ce1dab640 1 -- 172.21.15.5:0/2429731559 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0cdc102790 msgr2=0x7f0cdc1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:37.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:37.072+0000 7f0ce1dab640 1 --2- 172.21.15.5:0/2429731559 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0cdc102790 0x7f0cdc1067e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f0cd0009900 tx=0x7f0cd002a410 comp rx=0 tx=0).stop 2024-09-12T02:56:37.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:37.072+0000 7f0ce1dab640 1 -- 172.21.15.5:0/2429731559 shutdown_connections 2024-09-12T02:56:37.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:37.072+0000 7f0ce1dab640 1 --2- 172.21.15.5:0/2429731559 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0cdc102790 0x7f0cdc1067e0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:37.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:37.072+0000 7f0ce1dab640 1 --2- 172.21.15.5:0/2429731559 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0cdc106f10 0x7f0cdc10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:37.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:37.072+0000 7f0ce1dab640 1 --2- 172.21.15.5:0/2429731559 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0cdc101de0 0x7f0cdc1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:37.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:37.072+0000 7f0ce1dab640 1 -- 172.21.15.5:0/2429731559 >> 172.21.15.5:0/2429731559 conn(0x7f0cdc0fd8c0 msgr2=0x7f0cdc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:37.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:37.072+0000 7f0ce1dab640 1 -- 172.21.15.5:0/2429731559 shutdown_connections 2024-09-12T02:56:37.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:37.072+0000 7f0ce1dab640 1 -- 172.21.15.5:0/2429731559 wait complete. 2024-09-12T02:56:37.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:37.073+0000 7f0ce1dab640 1 Processor -- start 2024-09-12T02:56:37.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:37.073+0000 7f0ce1dab640 1 -- start start 2024-09-12T02:56:37.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:37.073+0000 7f0ce1dab640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0cdc101de0 0x7f0cdc198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:37.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:37.073+0000 7f0ce1dab640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0cdc102790 0x7f0cdc1988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:37.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:37.073+0000 7f0ce1dab640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0cdc106f10 0x7f0cdc19cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:37.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:37.073+0000 7f0ce1dab640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0cdc111f30 con 0x7f0cdc101de0 2024-09-12T02:56:37.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:37.073+0000 7f0ce1dab640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f0cdc111db0 con 0x7f0cdc102790 2024-09-12T02:56:37.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:37.073+0000 7f0ce1dab640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f0cdc1120b0 con 0x7f0cdc106f10 2024-09-12T02:56:37.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:37.073+0000 7f0cdb7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0cdc101de0 0x7f0cdc198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:37.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:37.073+0000 7f0cdb7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0cdc101de0 0x7f0cdc198380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49570/0 (socket says 172.21.15.5:49570) 2024-09-12T02:56:37.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:37.073+0000 7f0cdb7fe640 1 -- 172.21.15.5:0/2092561202 learned_addr learned my addr 172.21.15.5:0/2092561202 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:56:37.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:37.073+0000 7f0cdaffd640 1 --2- 172.21.15.5:0/2092561202 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0cdc102790 0x7f0cdc1988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:37.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:37.074+0000 7f0cdbfff640 1 --2- 172.21.15.5:0/2092561202 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0cdc106f10 0x7f0cdc19cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:37.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:37.074+0000 7f0cdb7fe640 1 -- 172.21.15.5:0/2092561202 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0cdc106f10 msgr2=0x7f0cdc19cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:37.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:37.074+0000 7f0cdb7fe640 1 --2- 172.21.15.5:0/2092561202 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0cdc106f10 0x7f0cdc19cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:37.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:37.074+0000 7f0cdb7fe640 1 -- 172.21.15.5:0/2092561202 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0cdc102790 msgr2=0x7f0cdc1988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:37.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:37.074+0000 7f0cdb7fe640 1 --2- 172.21.15.5:0/2092561202 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0cdc102790 0x7f0cdc1988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:37.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:37.074+0000 7f0cdb7fe640 1 -- 172.21.15.5:0/2092561202 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cdc19d330 con 0x7f0cdc101de0 2024-09-12T02:56:37.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:37.074+0000 7f0cdb7fe640 1 --2- 172.21.15.5:0/2092561202 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0cdc101de0 0x7f0cdc198380 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc00cce0 tx=0x7f0ccc00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:37.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:37.074+0000 7f0cd8ff9640 1 -- 172.21.15.5:0/2092561202 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ccc00f650 con 0x7f0cdc101de0 2024-09-12T02:56:37.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:37.074+0000 7f0ce1dab640 1 -- 172.21.15.5:0/2092561202 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cdc19d620 con 0x7f0cdc101de0 2024-09-12T02:56:37.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:37.074+0000 7f0cd8ff9640 1 -- 172.21.15.5:0/2092561202 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ccc00fcd0 con 0x7f0cdc101de0 2024-09-12T02:56:37.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:37.074+0000 7f0cd8ff9640 1 -- 172.21.15.5:0/2092561202 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ccc018c70 con 0x7f0cdc101de0 2024-09-12T02:56:37.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:37.074+0000 7f0ce1dab640 1 -- 172.21.15.5:0/2092561202 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cdc1a4f00 con 0x7f0cdc101de0 2024-09-12T02:56:37.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:37.075+0000 7f0ce1dab640 1 -- 172.21.15.5:0/2092561202 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cdc1021c0 con 0x7f0cdc101de0 2024-09-12T02:56:37.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:37.076+0000 7f0cd8ff9640 1 -- 172.21.15.5:0/2092561202 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f0ccc00f7f0 con 0x7f0cdc101de0 2024-09-12T02:56:37.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:37.077+0000 7f0cd8ff9640 1 --2- 172.21.15.5:0/2092561202 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f0cc40a2c10 0x7f0cc40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:37.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:37.077+0000 7f0cdaffd640 1 --2- 172.21.15.5:0/2092561202 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f0cc40a2c10 0x7f0cc40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:37.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:37.077+0000 7f0cd8ff9640 1 -- 172.21.15.5:0/2092561202 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f0ccc0cf790 con 0x7f0cdc101de0 2024-09-12T02:56:37.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:37.078+0000 7f0cdaffd640 1 --2- 172.21.15.5:0/2092561202 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f0cc40a2c10 0x7f0cc40a50d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc0fdce0 tx=0x7f0cd0005a20 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:37.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:37.081+0000 7f0cd8ff9640 1 -- 172.21.15.5:0/2092561202 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0ccc014030 con 0x7f0cdc101de0 2024-09-12T02:56:37.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:37.292+0000 7f0ce1dab640 1 -- 172.21.15.5:0/2092561202 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0cdc02d7c0 con 0x7f0cdc101de0 2024-09-12T02:56:37.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:37.293+0000 7f0cd8ff9640 1 -- 172.21.15.5:0/2092561202 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11966 (secure 0 0 0) 0x7f0ccc09c540 con 0x7f0cdc101de0 2024-09-12T02:56:37.294 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-12T02:56:37.294 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":25,"btime":"2024-09-12T02:56:27:840212+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/4135232521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":4135232521},{"type":"v1","addr":"172.21.15.5:6835","nonce":4135232521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/4290221940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":4290221940},{"type":"v1","addr":"172.21.15.5:6837","nonce":4290221940}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T02:56:16.595562+0000","modified":"2024-09-12T02:56:27.840207+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24502,"mds_1":24487,"mds_2":24481,"mds_3":24479,"mds_4":24458},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/26447341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":26447341},{"type":"v1","addr":"172.21.15.5:6839","nonce":26447341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2393443295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2393443295},{"type":"v1","addr":"172.21.15.5:6841","nonce":2393443295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6833/3686970137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3686970137},{"type":"v1","addr":"172.21.15.146:6833","nonce":3686970137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6833/568512209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":568512209},{"type":"v1","addr":"172.21.15.148:6833","nonce":568512209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6835/2540740420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2540740420},{"type":"v1","addr":"172.21.15.146:6835","nonce":2540740420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.146:6837/1673370937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":1673370937},{"type":"v1","addr":"172.21.15.146:6837","nonce":1673370937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":3,"incarnation":22,"state":"up:active","state_seq":6,"addr":"172.21.15.146:6839/3036767228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":3036767228},{"type":"v1","addr":"172.21.15.146:6839","nonce":3036767228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":2,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.148:6835/4195086427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":4195086427},{"type":"v1","addr":"172.21.15.148:6835","nonce":4195086427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.148:6837/2606873922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2606873922},{"type":"v1","addr":"172.21.15.148:6837","nonce":2606873922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6839/2317600000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":2317600000},{"type":"v1","addr":"172.21.15.148:6839","nonce":2317600000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24502,"qdb_cluster":[24458,24479,24481,24487,24502]},"id":1}]} 2024-09-12T02:56:37.294 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 25 2024-09-12T02:56:37.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:37.295+0000 7f0ce1dab640 1 -- 172.21.15.5:0/2092561202 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f0cc40a2c10 msgr2=0x7f0cc40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:37.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:37.295+0000 7f0ce1dab640 1 --2- 172.21.15.5:0/2092561202 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f0cc40a2c10 0x7f0cc40a50d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc0fdce0 tx=0x7f0cd0005a20 comp rx=0 tx=0).stop 2024-09-12T02:56:37.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:37.295+0000 7f0ce1dab640 1 -- 172.21.15.5:0/2092561202 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0cdc101de0 msgr2=0x7f0cdc198380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:37.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:37.295+0000 7f0ce1dab640 1 --2- 172.21.15.5:0/2092561202 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0cdc101de0 0x7f0cdc198380 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0ccc00cce0 tx=0x7f0ccc00a620 comp rx=0 tx=0).stop 2024-09-12T02:56:37.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:37.295+0000 7f0ce1dab640 1 -- 172.21.15.5:0/2092561202 shutdown_connections 2024-09-12T02:56:37.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:37.295+0000 7f0ce1dab640 1 --2- 172.21.15.5:0/2092561202 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0cdc106f10 0x7f0cdc19cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:37.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:37.295+0000 7f0ce1dab640 1 --2- 172.21.15.5:0/2092561202 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0cdc102790 0x7f0cdc1988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:37.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:37.295+0000 7f0ce1dab640 1 --2- 172.21.15.5:0/2092561202 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f0cc40a2c10 0x7f0cc40a50d0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:37.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:37.295+0000 7f0ce1dab640 1 --2- 172.21.15.5:0/2092561202 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0cdc101de0 0x7f0cdc198380 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:37.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:37.295+0000 7f0ce1dab640 1 -- 172.21.15.5:0/2092561202 >> 172.21.15.5:0/2092561202 conn(0x7f0cdc0fd8c0 msgr2=0x7f0cdc0fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:37.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:37.296+0000 7f0ce1dab640 1 -- 172.21.15.5:0/2092561202 shutdown_connections 2024-09-12T02:56:37.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:37.296+0000 7f0ce1dab640 1 -- 172.21.15.5:0/2092561202 wait complete. 2024-09-12T02:56:37.305 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:37 smithi005 ceph-mon[31197]: pgmap v286: 97 pgs: 97 active+clean; 610 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.2 KiB/s wr, 25 op/s; 91 B/s, 1 objects/s recovering 2024-09-12T02:56:37.305 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:37 smithi005 ceph-mon[31197]: from='client.14721 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:56:37.415 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:37 smithi148 ceph-mon[35090]: pgmap v286: 97 pgs: 97 active+clean; 610 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.2 KiB/s wr, 25 op/s; 91 B/s, 1 objects/s recovering 2024-09-12T02:56:37.415 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:37 smithi148 ceph-mon[35090]: from='client.14721 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:56:37.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:37 smithi146 ceph-mon[35011]: pgmap v286: 97 pgs: 97 active+clean; 610 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.2 KiB/s wr, 25 op/s; 91 B/s, 1 objects/s recovering 2024-09-12T02:56:37.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:37 smithi146 ceph-mon[35011]: from='client.14721 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:56:37.884 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T02:56:38.146 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:56:38.267 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:38 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/2092561202' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:56:38.415 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:38 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/2092561202' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:56:38.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:38 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/2092561202' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:56:38.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:38.688+0000 7f109f0f4640 1 -- 172.21.15.5:0/1183660094 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1098106a80 msgr2=0x7f1098106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:38.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:38.688+0000 7f109f0f4640 1 --2- 172.21.15.5:0/1183660094 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1098106a80 0x7f1098106e60 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f1094009930 tx=0x7f109402a420 comp rx=0 tx=0).stop 2024-09-12T02:56:38.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:38.688+0000 7f109f0f4640 1 -- 172.21.15.5:0/1183660094 shutdown_connections 2024-09-12T02:56:38.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:38.688+0000 7f109f0f4640 1 --2- 172.21.15.5:0/1183660094 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1098101420 0x7f109810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:38.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:38.688+0000 7f109f0f4640 1 --2- 172.21.15.5:0/1183660094 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1098100a80 0x7f1098100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:38.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:38.688+0000 7f109f0f4640 1 --2- 172.21.15.5:0/1183660094 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1098106a80 0x7f1098106e60 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:38.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:38.688+0000 7f109f0f4640 1 -- 172.21.15.5:0/1183660094 >> 172.21.15.5:0/1183660094 conn(0x7f10980fc7a0 msgr2=0x7f10980febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:38.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:38.688+0000 7f109f0f4640 1 -- 172.21.15.5:0/1183660094 shutdown_connections 2024-09-12T02:56:38.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:38.688+0000 7f109f0f4640 1 -- 172.21.15.5:0/1183660094 wait complete. 2024-09-12T02:56:38.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:38.689+0000 7f109f0f4640 1 Processor -- start 2024-09-12T02:56:38.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:38.689+0000 7f109f0f4640 1 -- start start 2024-09-12T02:56:38.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:38.689+0000 7f109f0f4640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1098100a80 0x7f1098197300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:38.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:38.689+0000 7f109f0f4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1098101420 0x7f1098197840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:38.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:38.689+0000 7f109f0f4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1098106a80 0x7f109819bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:38.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:38.689+0000 7f109f0f4640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1098110e80 con 0x7f1098106a80 2024-09-12T02:56:38.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:38.689+0000 7f109f0f4640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f1098110d00 con 0x7f1098100a80 2024-09-12T02:56:38.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:38.689+0000 7f109f0f4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1098111000 con 0x7f1098101420 2024-09-12T02:56:38.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:38.689+0000 7f109d691640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1098106a80 0x7f109819bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:38.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:38.689+0000 7f109d691640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1098106a80 0x7f109819bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49580/0 (socket says 172.21.15.5:49580) 2024-09-12T02:56:38.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:38.689+0000 7f109d691640 1 -- 172.21.15.5:0/3069736288 learned_addr learned my addr 172.21.15.5:0/3069736288 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:56:38.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:38.689+0000 7f109ce90640 1 --2- 172.21.15.5:0/3069736288 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1098100a80 0x7f1098197300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:38.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:38.689+0000 7f108ffff640 1 --2- 172.21.15.5:0/3069736288 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1098101420 0x7f1098197840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:38.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:38.689+0000 7f109d691640 1 -- 172.21.15.5:0/3069736288 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1098101420 msgr2=0x7f1098197840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:38.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:38.689+0000 7f109d691640 1 --2- 172.21.15.5:0/3069736288 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1098101420 0x7f1098197840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:38.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:38.689+0000 7f109d691640 1 -- 172.21.15.5:0/3069736288 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1098100a80 msgr2=0x7f1098197300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:38.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:38.689+0000 7f109d691640 1 --2- 172.21.15.5:0/3069736288 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1098100a80 0x7f1098197300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:38.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:38.689+0000 7f109d691640 1 -- 172.21.15.5:0/3069736288 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f109819c2b0 con 0x7f1098106a80 2024-09-12T02:56:38.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:38.690+0000 7f109d691640 1 --2- 172.21.15.5:0/3069736288 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1098106a80 0x7f109819bbd0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f109000eaa0 tx=0x7f109000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:38.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:38.690+0000 7f108dffb640 1 -- 172.21.15.5:0/3069736288 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f109000cdb0 con 0x7f1098106a80 2024-09-12T02:56:38.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:38.690+0000 7f109f0f4640 1 -- 172.21.15.5:0/3069736288 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f109819c5a0 con 0x7f1098106a80 2024-09-12T02:56:38.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:38.690+0000 7f108dffb640 1 -- 172.21.15.5:0/3069736288 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1090005020 con 0x7f1098106a80 2024-09-12T02:56:38.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:38.690+0000 7f108dffb640 1 -- 172.21.15.5:0/3069736288 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10900116d0 con 0x7f1098106a80 2024-09-12T02:56:38.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:38.690+0000 7f109f0f4640 1 -- 172.21.15.5:0/3069736288 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10981a3e80 con 0x7f1098106a80 2024-09-12T02:56:38.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:38.691+0000 7f109f0f4640 1 -- 172.21.15.5:0/3069736288 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1098100ee0 con 0x7f1098106a80 2024-09-12T02:56:38.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:38.692+0000 7f108dffb640 1 -- 172.21.15.5:0/3069736288 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f10900054d0 con 0x7f1098106a80 2024-09-12T02:56:38.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:38.693+0000 7f108dffb640 1 --2- 172.21.15.5:0/3069736288 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f10800a2c10 0x7f10800a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:38.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:38.693+0000 7f109ce90640 1 --2- 172.21.15.5:0/3069736288 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f10800a2c10 0x7f10800a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:38.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:38.693+0000 7f108dffb640 1 -- 172.21.15.5:0/3069736288 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f10900cf2f0 con 0x7f1098106a80 2024-09-12T02:56:38.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:38.694+0000 7f109ce90640 1 --2- 172.21.15.5:0/3069736288 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f10800a2c10 0x7f10800a50d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f1094009900 tx=0x7f10940058d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:38.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:38.698+0000 7f108dffb640 1 -- 172.21.15.5:0/3069736288 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1090016030 con 0x7f1098106a80 2024-09-12T02:56:38.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:38.876+0000 7f109f0f4640 1 -- 172.21.15.5:0/3069736288 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1098108a60 con 0x7f1098106a80 2024-09-12T02:56:38.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:38.879+0000 7f108dffb640 1 -- 172.21.15.5:0/3069736288 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+22656 (secure 0 0 0) 0x7f109009bab0 con 0x7f1098106a80 2024-09-12T02:56:38.879 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-12T02:56:38.879 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":83,"fsid":"618305f8-70b1-11ef-bcea-c7b262605968","created":"2024-09-12T02:48:42.352146+0000","modified":"2024-09-12T02:56:31.026079+0000","last_up_change":"2024-09-12T02:55:27.817161+0000","last_in_change":"2024-09-12T02:55:03.745281+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T02:51:47.970639+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T02:56:12.834271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T02:56:14.951437+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17da560a-6aa2-4f2c-a95f-67481dce4d3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":4248507573},{"type":"v1","addr":"172.21.15.5:6803","nonce":4248507573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":4248507573},{"type":"v1","addr":"172.21.15.5:6805","nonce":4248507573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":4248507573},{"type":"v1","addr":"172.21.15.5:6809","nonce":4248507573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":4248507573},{"type":"v1","addr":"172.21.15.5:6807","nonce":4248507573}]},"public_addr":"172.21.15.5:6803/4248507573","cluster_addr":"172.21.15.5:6805/4248507573","heartbeat_back_addr":"172.21.15.5:6809/4248507573","heartbeat_front_addr":"172.21.15.5:6807/4248507573","state":["exists","up"]},{"osd":1,"uuid":"a3237006-99af-4fc1-860c-d4917032647f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":1105990287},{"type":"v1","addr":"172.21.15.146:6801","nonce":1105990287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":1105990287},{"type":"v1","addr":"172.21.15.146:6803","nonce":1105990287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":1105990287},{"type":"v1","addr":"172.21.15.146:6807","nonce":1105990287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":1105990287},{"type":"v1","addr":"172.21.15.146:6805","nonce":1105990287}]},"public_addr":"172.21.15.146:6801/1105990287","cluster_addr":"172.21.15.146:6803/1105990287","heartbeat_back_addr":"172.21.15.146:6807/1105990287","heartbeat_front_addr":"172.21.15.146:6805/1105990287","state":["exists","up"]},{"osd":2,"uuid":"ea3ab84a-7739-4975-8244-31ddd5aad80d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3378174313},{"type":"v1","addr":"172.21.15.148:6801","nonce":3378174313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3378174313},{"type":"v1","addr":"172.21.15.148:6803","nonce":3378174313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3378174313},{"type":"v1","addr":"172.21.15.148:6807","nonce":3378174313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3378174313},{"type":"v1","addr":"172.21.15.148:6805","nonce":3378174313}]},"public_addr":"172.21.15.148:6801/3378174313","cluster_addr":"172.21.15.148:6803/3378174313","heartbeat_back_addr":"172.21.15.148:6807/3378174313","heartbeat_front_addr":"172.21.15.148:6805/3378174313","state":["exists","up"]},{"osd":3,"uuid":"4396a9ee-f3f3-410b-9fb1-124f1e33a1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":11741938},{"type":"v1","addr":"172.21.15.5:6811","nonce":11741938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":11741938},{"type":"v1","addr":"172.21.15.5:6813","nonce":11741938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":11741938},{"type":"v1","addr":"172.21.15.5:6817","nonce":11741938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":11741938},{"type":"v1","addr":"172.21.15.5:6815","nonce":11741938}]},"public_addr":"172.21.15.5:6811/11741938","cluster_addr":"172.21.15.5:6813/11741938","heartbeat_back_addr":"172.21.15.5:6817/11741938","heartbeat_front_addr":"172.21.15.5:6815/11741938","state":["exists","up"]},{"osd":4,"uuid":"989d76e7-c742-49d0-994a-c58f2771b1d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":3274923851},{"type":"v1","addr":"172.21.15.146:6809","nonce":3274923851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":3274923851},{"type":"v1","addr":"172.21.15.146:6811","nonce":3274923851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3274923851},{"type":"v1","addr":"172.21.15.146:6815","nonce":3274923851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":3274923851},{"type":"v1","addr":"172.21.15.146:6813","nonce":3274923851}]},"public_addr":"172.21.15.146:6809/3274923851","cluster_addr":"172.21.15.146:6811/3274923851","heartbeat_back_addr":"172.21.15.146:6815/3274923851","heartbeat_front_addr":"172.21.15.146:6813/3274923851","state":["exists","up"]},{"osd":5,"uuid":"64ada7d4-70a0-4b83-9af8-032be544608d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":2771670476},{"type":"v1","addr":"172.21.15.148:6809","nonce":2771670476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":2771670476},{"type":"v1","addr":"172.21.15.148:6811","nonce":2771670476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":2771670476},{"type":"v1","addr":"172.21.15.148:6815","nonce":2771670476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":2771670476},{"type":"v1","addr":"172.21.15.148:6813","nonce":2771670476}]},"public_addr":"172.21.15.148:6809/2771670476","cluster_addr":"172.21.15.148:6811/2771670476","heartbeat_back_addr":"172.21.15.148:6815/2771670476","heartbeat_front_addr":"172.21.15.148:6813/2771670476","state":["exists","up"]},{"osd":6,"uuid":"a70a88fd-2e4b-480c-bb98-62636cf70cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3176073169},{"type":"v1","addr":"172.21.15.5:6819","nonce":3176073169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":3176073169},{"type":"v1","addr":"172.21.15.5:6821","nonce":3176073169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":3176073169},{"type":"v1","addr":"172.21.15.5:6825","nonce":3176073169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3176073169},{"type":"v1","addr":"172.21.15.5:6823","nonce":3176073169}]},"public_addr":"172.21.15.5:6819/3176073169","cluster_addr":"172.21.15.5:6821/3176073169","heartbeat_back_addr":"172.21.15.5:6825/3176073169","heartbeat_front_addr":"172.21.15.5:6823/3176073169","state":["exists","up"]},{"osd":7,"uuid":"49451e8d-c741-4ad5-b8f5-89bad8d4d129","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2209508838},{"type":"v1","addr":"172.21.15.146:6817","nonce":2209508838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2209508838},{"type":"v1","addr":"172.21.15.146:6819","nonce":2209508838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2209508838},{"type":"v1","addr":"172.21.15.146:6823","nonce":2209508838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2209508838},{"type":"v1","addr":"172.21.15.146:6821","nonce":2209508838}]},"public_addr":"172.21.15.146:6817/2209508838","cluster_addr":"172.21.15.146:6819/2209508838","heartbeat_back_addr":"172.21.15.146:6823/2209508838","heartbeat_front_addr":"172.21.15.146:6821/2209508838","state":["exists","up"]},{"osd":8,"uuid":"57bfc121-a75e-4a7f-b1dc-64074b178b03","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":651045756},{"type":"v1","addr":"172.21.15.148:6817","nonce":651045756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":651045756},{"type":"v1","addr":"172.21.15.148:6819","nonce":651045756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":651045756},{"type":"v1","addr":"172.21.15.148:6823","nonce":651045756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":651045756},{"type":"v1","addr":"172.21.15.148:6821","nonce":651045756}]},"public_addr":"172.21.15.148:6817/651045756","cluster_addr":"172.21.15.148:6819/651045756","heartbeat_back_addr":"172.21.15.148:6823/651045756","heartbeat_front_addr":"172.21.15.148:6821/651045756","state":["exists","up"]},{"osd":9,"uuid":"d9726168-7494-4bf8-b433-350e85bfd461","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1961857360},{"type":"v1","addr":"172.21.15.5:6827","nonce":1961857360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1961857360},{"type":"v1","addr":"172.21.15.5:6829","nonce":1961857360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":1961857360},{"type":"v1","addr":"172.21.15.5:6833","nonce":1961857360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1961857360},{"type":"v1","addr":"172.21.15.5:6831","nonce":1961857360}]},"public_addr":"172.21.15.5:6827/1961857360","cluster_addr":"172.21.15.5:6829/1961857360","heartbeat_back_addr":"172.21.15.5:6833/1961857360","heartbeat_front_addr":"172.21.15.5:6831/1961857360","state":["exists","up"]},{"osd":10,"uuid":"519ec60f-3606-48ac-9043-6f59a798766d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":2041733103},{"type":"v1","addr":"172.21.15.146:6825","nonce":2041733103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":2041733103},{"type":"v1","addr":"172.21.15.146:6827","nonce":2041733103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":2041733103},{"type":"v1","addr":"172.21.15.146:6831","nonce":2041733103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":2041733103},{"type":"v1","addr":"172.21.15.146:6829","nonce":2041733103}]},"public_addr":"172.21.15.146:6825/2041733103","cluster_addr":"172.21.15.146:6827/2041733103","heartbeat_back_addr":"172.21.15.146:6831/2041733103","heartbeat_front_addr":"172.21.15.146:6829/2041733103","state":["exists","up"]},{"osd":11,"uuid":"0f92d0cb-ba54-4bc3-a6d0-3c54d910556a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":839290773},{"type":"v1","addr":"172.21.15.148:6825","nonce":839290773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":839290773},{"type":"v1","addr":"172.21.15.148:6827","nonce":839290773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":839290773},{"type":"v1","addr":"172.21.15.148:6831","nonce":839290773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":839290773},{"type":"v1","addr":"172.21.15.148:6829","nonce":839290773}]},"public_addr":"172.21.15.148:6825/839290773","cluster_addr":"172.21.15.148:6827/839290773","heartbeat_back_addr":"172.21.15.148:6831/839290773","heartbeat_front_addr":"172.21.15.148:6829/839290773","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:50:54.101755+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:51:19.037385+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:51:44.142837+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:52:09.150887+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:52:33.705079+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:52:58.845233+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:53:23.068339+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:53:47.520398+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:54:11.965370+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:54:36.776075+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:55:01.292236+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:55:25.346589+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:6800/3125821798":"2024-09-13T02:49:29.462216+0000","172.21.15.5:0/3479911451":"2024-09-13T02:49:29.462216+0000","172.21.15.5:0/1547753207":"2024-09-13T02:49:03.300872+0000","172.21.15.5:0/1905762627":"2024-09-13T02:49:03.300872+0000","172.21.15.5:0/4142340280":"2024-09-13T02:49:29.462216+0000","172.21.15.5:6801/3125821798":"2024-09-13T02:49:29.462216+0000","172.21.15.5:6801/2595781680":"2024-09-13T02:49:03.300872+0000","172.21.15.5:0/2273691311":"2024-09-13T02:49:29.462216+0000","172.21.15.5:6800/2595781680":"2024-09-13T02:49:03.300872+0000","172.21.15.5:0/1196201217":"2024-09-13T02:49:03.300872+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T02:56:38.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:38.881+0000 7f109f0f4640 1 -- 172.21.15.5:0/3069736288 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f10800a2c10 msgr2=0x7f10800a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:38.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:38.881+0000 7f109f0f4640 1 --2- 172.21.15.5:0/3069736288 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f10800a2c10 0x7f10800a50d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f1094009900 tx=0x7f10940058d0 comp rx=0 tx=0).stop 2024-09-12T02:56:38.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:38.881+0000 7f109f0f4640 1 -- 172.21.15.5:0/3069736288 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1098106a80 msgr2=0x7f109819bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:38.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:38.881+0000 7f109f0f4640 1 --2- 172.21.15.5:0/3069736288 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1098106a80 0x7f109819bbd0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f109000eaa0 tx=0x7f109000ef60 comp rx=0 tx=0).stop 2024-09-12T02:56:38.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:38.881+0000 7f109f0f4640 1 -- 172.21.15.5:0/3069736288 shutdown_connections 2024-09-12T02:56:38.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:38.881+0000 7f109f0f4640 1 --2- 172.21.15.5:0/3069736288 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f10800a2c10 0x7f10800a50d0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:38.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:38.881+0000 7f109f0f4640 1 --2- 172.21.15.5:0/3069736288 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1098106a80 0x7f109819bbd0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:38.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:38.881+0000 7f109f0f4640 1 --2- 172.21.15.5:0/3069736288 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1098101420 0x7f1098197840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:38.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:38.881+0000 7f109f0f4640 1 --2- 172.21.15.5:0/3069736288 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1098100a80 0x7f1098197300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:38.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:38.881+0000 7f109f0f4640 1 -- 172.21.15.5:0/3069736288 >> 172.21.15.5:0/3069736288 conn(0x7f10980fc7a0 msgr2=0x7f10980fe020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:38.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:38.881+0000 7f109f0f4640 1 -- 172.21.15.5:0/3069736288 shutdown_connections 2024-09-12T02:56:38.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:38.881+0000 7f109f0f4640 1 -- 172.21.15.5:0/3069736288 wait complete. 2024-09-12T02:56:39.193 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:39 smithi005 ceph-mon[31197]: pgmap v287: 97 pgs: 97 active+clean; 610 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 5.5 KiB/s wr, 15 op/s; 91 B/s, 1 objects/s recovering 2024-09-12T02:56:39.193 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:39 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/3069736288' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:56:39.317 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:39 smithi148 ceph-mon[35090]: pgmap v287: 97 pgs: 97 active+clean; 610 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 5.5 KiB/s wr, 15 op/s; 91 B/s, 1 objects/s recovering 2024-09-12T02:56:39.317 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:39 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/3069736288' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:56:39.358 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:39 smithi146 ceph-mon[35011]: pgmap v287: 97 pgs: 97 active+clean; 610 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 5.5 KiB/s wr, 15 op/s; 91 B/s, 1 objects/s recovering 2024-09-12T02:56:39.358 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:39 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/3069736288' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:56:39.500 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-12T02:56:39.501 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-12T02:56:39.770 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:56:40.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:40.250+0000 7f04301af640 1 -- 172.21.15.5:0/1586885357 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0428100e20 msgr2=0x7f0428101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:40.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:40.250+0000 7f04301af640 1 --2- 172.21.15.5:0/1586885357 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0428100e20 0x7f0428101200 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f041c009890 tx=0x7f041c02a400 comp rx=0 tx=0).stop 2024-09-12T02:56:40.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:40.250+0000 7f04301af640 1 -- 172.21.15.5:0/1586885357 shutdown_connections 2024-09-12T02:56:40.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:40.250+0000 7f04301af640 1 --2- 172.21.15.5:0/1586885357 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0428102570 0x7f04281029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:40.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:40.250+0000 7f04301af640 1 --2- 172.21.15.5:0/1586885357 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0428102ef0 0x7f042810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:40.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:40.250+0000 7f04301af640 1 --2- 172.21.15.5:0/1586885357 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0428100e20 0x7f0428101200 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:40.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:40.250+0000 7f04301af640 1 -- 172.21.15.5:0/1586885357 >> 172.21.15.5:0/1586885357 conn(0x7f04280fc7d0 msgr2=0x7f04280febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:40.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:40.250+0000 7f04301af640 1 -- 172.21.15.5:0/1586885357 shutdown_connections 2024-09-12T02:56:40.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:40.250+0000 7f04301af640 1 -- 172.21.15.5:0/1586885357 wait complete. 2024-09-12T02:56:40.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:40.250+0000 7f04301af640 1 Processor -- start 2024-09-12T02:56:40.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:40.250+0000 7f04301af640 1 -- start start 2024-09-12T02:56:40.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:40.251+0000 7f04301af640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0428100e20 0x7f04281a03f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:40.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:40.251+0000 7f04301af640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0428102570 0x7f04281a0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:40.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:40.251+0000 7f04301af640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0428102ef0 0x7f042819a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:40.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:40.251+0000 7f04301af640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0428110fa0 con 0x7f0428102ef0 2024-09-12T02:56:40.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:40.251+0000 7f04301af640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f0428110e20 con 0x7f0428102570 2024-09-12T02:56:40.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:40.251+0000 7f04301af640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f0428111120 con 0x7f0428100e20 2024-09-12T02:56:40.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:40.251+0000 7f042e74c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0428102ef0 0x7f042819a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:40.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:40.251+0000 7f042df4b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0428100e20 0x7f04281a03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:40.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:40.251+0000 7f042e74c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0428102ef0 0x7f042819a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49594/0 (socket says 172.21.15.5:49594) 2024-09-12T02:56:40.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:40.251+0000 7f042e74c640 1 -- 172.21.15.5:0/2043018637 learned_addr learned my addr 172.21.15.5:0/2043018637 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:56:40.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:40.251+0000 7f042d74a640 1 --2- 172.21.15.5:0/2043018637 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0428102570 0x7f04281a0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:40.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:40.251+0000 7f042e74c640 1 -- 172.21.15.5:0/2043018637 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0428100e20 msgr2=0x7f04281a03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:40.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:40.251+0000 7f042e74c640 1 --2- 172.21.15.5:0/2043018637 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0428100e20 0x7f04281a03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:40.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:40.251+0000 7f042e74c640 1 -- 172.21.15.5:0/2043018637 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0428102570 msgr2=0x7f04281a0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:40.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:40.251+0000 7f042e74c640 1 --2- 172.21.15.5:0/2043018637 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0428102570 0x7f04281a0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:40.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:40.251+0000 7f042e74c640 1 -- 172.21.15.5:0/2043018637 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f042819ace0 con 0x7f0428102ef0 2024-09-12T02:56:40.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:40.251+0000 7f042df4b640 1 --2- 172.21.15.5:0/2043018637 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0428100e20 0x7f04281a03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:56:40.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:40.251+0000 7f042e74c640 1 --2- 172.21.15.5:0/2043018637 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0428102ef0 0x7f042819a5b0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f042000b970 tx=0x7f042000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:40.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:40.252+0000 7f041affd640 1 -- 172.21.15.5:0/2043018637 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0420019070 con 0x7f0428102ef0 2024-09-12T02:56:40.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:40.252+0000 7f04301af640 1 -- 172.21.15.5:0/2043018637 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f042819afd0 con 0x7f0428102ef0 2024-09-12T02:56:40.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:40.252+0000 7f041affd640 1 -- 172.21.15.5:0/2043018637 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f042000cb40 con 0x7f0428102ef0 2024-09-12T02:56:40.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:40.252+0000 7f041affd640 1 -- 172.21.15.5:0/2043018637 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0420007650 con 0x7f0428102ef0 2024-09-12T02:56:40.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:40.252+0000 7f04301af640 1 -- 172.21.15.5:0/2043018637 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04281a72c0 con 0x7f0428102ef0 2024-09-12T02:56:40.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:40.253+0000 7f04301af640 1 -- 172.21.15.5:0/2043018637 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0428101200 con 0x7f0428102ef0 2024-09-12T02:56:40.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:40.254+0000 7f041affd640 1 -- 172.21.15.5:0/2043018637 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f042001c070 con 0x7f0428102ef0 2024-09-12T02:56:40.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:40.255+0000 7f041affd640 1 --2- 172.21.15.5:0/2043018637 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f04140a2c10 0x7f04140a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:40.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:40.255+0000 7f042df4b640 1 --2- 172.21.15.5:0/2043018637 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f04140a2c10 0x7f04140a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:40.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:40.255+0000 7f041affd640 1 -- 172.21.15.5:0/2043018637 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f04200ce6a0 con 0x7f0428102ef0 2024-09-12T02:56:40.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:40.255+0000 7f042df4b640 1 --2- 172.21.15.5:0/2043018637 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f04140a2c10 0x7f04140a50d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f041c000c00 tx=0x7f041c005890 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:40.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:40.263+0000 7f041affd640 1 -- 172.21.15.5:0/2043018637 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f042009af70 con 0x7f0428102ef0 2024-09-12T02:56:40.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:40.437+0000 7f04301af640 1 -- 172.21.15.5:0/2043018637 --> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f04280fdd00 con 0x7f04140a2c10 2024-09-12T02:56:40.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:40.445+0000 7f041affd640 1 -- 172.21.15.5:0/2043018637 <== mgr.14148 v2:172.21.15.5:6800/3780631642 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258767 (secure 0 0 0) 0x7f04280fdd00 con 0x7f04140a2c10 2024-09-12T02:56:40.447 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-12T02:56:40.449 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-09-12T02:56:40.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:40.448+0000 7f04301af640 1 -- 172.21.15.5:0/2043018637 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f04140a2c10 msgr2=0x7f04140a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:40.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:40.448+0000 7f04301af640 1 --2- 172.21.15.5:0/2043018637 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f04140a2c10 0x7f04140a50d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f041c000c00 tx=0x7f041c005890 comp rx=0 tx=0).stop 2024-09-12T02:56:40.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:40.448+0000 7f04301af640 1 -- 172.21.15.5:0/2043018637 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0428102ef0 msgr2=0x7f042819a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:40.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:40.448+0000 7f04301af640 1 --2- 172.21.15.5:0/2043018637 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0428102ef0 0x7f042819a5b0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f042000b970 tx=0x7f042000be30 comp rx=0 tx=0).stop 2024-09-12T02:56:40.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:40.448+0000 7f04301af640 1 -- 172.21.15.5:0/2043018637 shutdown_connections 2024-09-12T02:56:40.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:40.448+0000 7f04301af640 1 --2- 172.21.15.5:0/2043018637 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f04140a2c10 0x7f04140a50d0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:40.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:40.448+0000 7f04301af640 1 --2- 172.21.15.5:0/2043018637 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0428102ef0 0x7f042819a5b0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:40.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:40.448+0000 7f04301af640 1 --2- 172.21.15.5:0/2043018637 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0428102570 0x7f04281a0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:40.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:40.448+0000 7f04301af640 1 --2- 172.21.15.5:0/2043018637 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0428100e20 0x7f04281a03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:40.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:40.448+0000 7f04301af640 1 -- 172.21.15.5:0/2043018637 >> 172.21.15.5:0/2043018637 conn(0x7f04280fc7d0 msgr2=0x7f042806a470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:40.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:40.448+0000 7f04301af640 1 -- 172.21.15.5:0/2043018637 shutdown_connections 2024-09-12T02:56:40.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:40.448+0000 7f04301af640 1 -- 172.21.15.5:0/2043018637 wait complete. 2024-09-12T02:56:41.073 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":288,"stamp":"2024-09-12T02:56:39.547171+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":649910,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":263,"num_read_kb":232,"num_write":422,"num_write_kb":3064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":1181470,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":345,"ondisk_log_size":345,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":374528,"kb_used_data":12768,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124567296,"statfs":{"total":1151940427776,"available":1151556911104,"internally_reserved":0,"allocated":13074432,"data_stored":7184541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":370193723},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":51586,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":123,"num_read_kb":111,"num_write":54,"num_write_kb":73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":734,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.520204"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.033421+0000","last_change":"2024-09-12T02:56:15.988690+0000","last_active":"2024-09-12T02:56:31.033421+0000","last_peered":"2024-09-12T02:56:31.033421+0000","last_clean":"2024-09-12T02:56:31.033421+0000","last_became_active":"2024-09-12T02:56:15.988320+0000","last_became_peered":"2024-09-12T02:56:15.988320+0000","last_unstale":"2024-09-12T02:56:31.033421+0000","last_undegraded":"2024-09-12T02:56:31.033421+0000","last_fullsized":"2024-09-12T02:56:31.033421+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:31:14.157622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.032544+0000","last_change":"2024-09-12T02:56:15.978368+0000","last_active":"2024-09-12T02:56:31.032544+0000","last_peered":"2024-09-12T02:56:31.032544+0000","last_clean":"2024-09-12T02:56:31.032544+0000","last_became_active":"2024-09-12T02:56:15.978092+0000","last_became_peered":"2024-09-12T02:56:15.978092+0000","last_unstale":"2024-09-12T02:56:31.032544+0000","last_undegraded":"2024-09-12T02:56:31.032544+0000","last_fullsized":"2024-09-12T02:56:31.032544+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:32:48.399005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.032403+0000","last_change":"2024-09-12T02:56:15.987249+0000","last_active":"2024-09-12T02:56:31.032403+0000","last_peered":"2024-09-12T02:56:31.032403+0000","last_clean":"2024-09-12T02:56:31.032403+0000","last_became_active":"2024-09-12T02:56:15.986921+0000","last_became_peered":"2024-09-12T02:56:15.986921+0000","last_unstale":"2024-09-12T02:56:31.032403+0000","last_undegraded":"2024-09-12T02:56:31.032403+0000","last_fullsized":"2024-09-12T02:56:31.032403+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:42:13.427871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.031295+0000","last_change":"2024-09-12T02:56:15.987673+0000","last_active":"2024-09-12T02:56:31.031295+0000","last_peered":"2024-09-12T02:56:31.031295+0000","last_clean":"2024-09-12T02:56:31.031295+0000","last_became_active":"2024-09-12T02:56:15.986964+0000","last_became_peered":"2024-09-12T02:56:15.986964+0000","last_unstale":"2024-09-12T02:56:31.031295+0000","last_undegraded":"2024-09-12T02:56:31.031295+0000","last_fullsized":"2024-09-12T02:56:31.031295+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:07:12.406974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.034804+0000","last_change":"2024-09-12T02:56:15.974473+0000","last_active":"2024-09-12T02:56:31.034804+0000","last_peered":"2024-09-12T02:56:31.034804+0000","last_clean":"2024-09-12T02:56:31.034804+0000","last_became_active":"2024-09-12T02:56:15.974124+0000","last_became_peered":"2024-09-12T02:56:15.974124+0000","last_unstale":"2024-09-12T02:56:31.034804+0000","last_undegraded":"2024-09-12T02:56:31.034804+0000","last_fullsized":"2024-09-12T02:56:31.034804+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:44:29.054841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.032976+0000","last_change":"2024-09-12T02:56:15.980242+0000","last_active":"2024-09-12T02:56:31.032976+0000","last_peered":"2024-09-12T02:56:31.032976+0000","last_clean":"2024-09-12T02:56:31.032976+0000","last_became_active":"2024-09-12T02:56:15.979937+0000","last_became_peered":"2024-09-12T02:56:15.979937+0000","last_unstale":"2024-09-12T02:56:31.032976+0000","last_undegraded":"2024-09-12T02:56:31.032976+0000","last_fullsized":"2024-09-12T02:56:31.032976+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:44:13.406341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.035162+0000","last_change":"2024-09-12T02:56:15.978746+0000","last_active":"2024-09-12T02:56:31.035162+0000","last_peered":"2024-09-12T02:56:31.035162+0000","last_clean":"2024-09-12T02:56:31.035162+0000","last_became_active":"2024-09-12T02:56:15.978304+0000","last_became_peered":"2024-09-12T02:56:15.978304+0000","last_unstale":"2024-09-12T02:56:31.035162+0000","last_undegraded":"2024-09-12T02:56:31.035162+0000","last_fullsized":"2024-09-12T02:56:31.035162+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:34:52.115630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.034360+0000","last_change":"2024-09-12T02:56:15.980083+0000","last_active":"2024-09-12T02:56:31.034360+0000","last_peered":"2024-09-12T02:56:31.034360+0000","last_clean":"2024-09-12T02:56:31.034360+0000","last_became_active":"2024-09-12T02:56:15.979771+0000","last_became_peered":"2024-09-12T02:56:15.979771+0000","last_unstale":"2024-09-12T02:56:31.034360+0000","last_undegraded":"2024-09-12T02:56:31.034360+0000","last_fullsized":"2024-09-12T02:56:31.034360+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:59:42.351970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.306732+0000","last_change":"2024-09-12T02:56:15.988091+0000","last_active":"2024-09-12T02:56:31.306732+0000","last_peered":"2024-09-12T02:56:31.306732+0000","last_clean":"2024-09-12T02:56:31.306732+0000","last_became_active":"2024-09-12T02:56:15.986295+0000","last_became_peered":"2024-09-12T02:56:15.986295+0000","last_unstale":"2024-09-12T02:56:31.306732+0000","last_undegraded":"2024-09-12T02:56:31.306732+0000","last_fullsized":"2024-09-12T02:56:31.306732+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:45:40.331805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.033168+0000","last_change":"2024-09-12T02:56:15.987349+0000","last_active":"2024-09-12T02:56:31.033168+0000","last_peered":"2024-09-12T02:56:31.033168+0000","last_clean":"2024-09-12T02:56:31.033168+0000","last_became_active":"2024-09-12T02:56:15.987018+0000","last_became_peered":"2024-09-12T02:56:15.987018+0000","last_unstale":"2024-09-12T02:56:31.033168+0000","last_undegraded":"2024-09-12T02:56:31.033168+0000","last_fullsized":"2024-09-12T02:56:31.033168+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:34:19.652348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.035420+0000","last_change":"2024-09-12T02:56:15.985164+0000","last_active":"2024-09-12T02:56:31.035420+0000","last_peered":"2024-09-12T02:56:31.035420+0000","last_clean":"2024-09-12T02:56:31.035420+0000","last_became_active":"2024-09-12T02:56:15.984787+0000","last_became_peered":"2024-09-12T02:56:15.984787+0000","last_unstale":"2024-09-12T02:56:31.035420+0000","last_undegraded":"2024-09-12T02:56:31.035420+0000","last_fullsized":"2024-09-12T02:56:31.035420+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:36:29.727442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.031157+0000","last_change":"2024-09-12T02:56:15.979963+0000","last_active":"2024-09-12T02:56:31.031157+0000","last_peered":"2024-09-12T02:56:31.031157+0000","last_clean":"2024-09-12T02:56:31.031157+0000","last_became_active":"2024-09-12T02:56:15.979639+0000","last_became_peered":"2024-09-12T02:56:15.979639+0000","last_unstale":"2024-09-12T02:56:31.031157+0000","last_undegraded":"2024-09-12T02:56:31.031157+0000","last_fullsized":"2024-09-12T02:56:31.031157+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:34:18.273245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.031973+0000","last_change":"2024-09-12T02:56:15.980415+0000","last_active":"2024-09-12T02:56:31.031973+0000","last_peered":"2024-09-12T02:56:31.031973+0000","last_clean":"2024-09-12T02:56:31.031973+0000","last_became_active":"2024-09-12T02:56:15.980059+0000","last_became_peered":"2024-09-12T02:56:15.980059+0000","last_unstale":"2024-09-12T02:56:31.031973+0000","last_undegraded":"2024-09-12T02:56:31.031973+0000","last_fullsized":"2024-09-12T02:56:31.031973+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:11:42.689081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.032089+0000","last_change":"2024-09-12T02:56:15.981423+0000","last_active":"2024-09-12T02:56:31.032089+0000","last_peered":"2024-09-12T02:56:31.032089+0000","last_clean":"2024-09-12T02:56:31.032089+0000","last_became_active":"2024-09-12T02:56:15.981186+0000","last_became_peered":"2024-09-12T02:56:15.981186+0000","last_unstale":"2024-09-12T02:56:31.032089+0000","last_undegraded":"2024-09-12T02:56:31.032089+0000","last_fullsized":"2024-09-12T02:56:31.032089+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:36:57.406907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.032139+0000","last_change":"2024-09-12T02:56:15.982587+0000","last_active":"2024-09-12T02:56:31.032139+0000","last_peered":"2024-09-12T02:56:31.032139+0000","last_clean":"2024-09-12T02:56:31.032139+0000","last_became_active":"2024-09-12T02:56:15.982315+0000","last_became_peered":"2024-09-12T02:56:15.982315+0000","last_unstale":"2024-09-12T02:56:31.032139+0000","last_undegraded":"2024-09-12T02:56:31.032139+0000","last_fullsized":"2024-09-12T02:56:31.032139+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:42:17.736999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.306084+0000","last_change":"2024-09-12T02:56:15.979482+0000","last_active":"2024-09-12T02:56:31.306084+0000","last_peered":"2024-09-12T02:56:31.306084+0000","last_clean":"2024-09-12T02:56:31.306084+0000","last_became_active":"2024-09-12T02:56:15.979122+0000","last_became_peered":"2024-09-12T02:56:15.979122+0000","last_unstale":"2024-09-12T02:56:31.306084+0000","last_undegraded":"2024-09-12T02:56:31.306084+0000","last_fullsized":"2024-09-12T02:56:31.306084+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:18:05.642487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.033518+0000","last_change":"2024-09-12T02:56:15.989908+0000","last_active":"2024-09-12T02:56:31.033518+0000","last_peered":"2024-09-12T02:56:31.033518+0000","last_clean":"2024-09-12T02:56:31.033518+0000","last_became_active":"2024-09-12T02:56:15.989515+0000","last_became_peered":"2024-09-12T02:56:15.989515+0000","last_unstale":"2024-09-12T02:56:31.033518+0000","last_undegraded":"2024-09-12T02:56:31.033518+0000","last_fullsized":"2024-09-12T02:56:31.033518+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:59:57.141607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.029078+0000","last_change":"2024-09-12T02:56:15.980323+0000","last_active":"2024-09-12T02:56:31.029078+0000","last_peered":"2024-09-12T02:56:31.029078+0000","last_clean":"2024-09-12T02:56:31.029078+0000","last_became_active":"2024-09-12T02:56:15.979989+0000","last_became_peered":"2024-09-12T02:56:15.979989+0000","last_unstale":"2024-09-12T02:56:31.029078+0000","last_undegraded":"2024-09-12T02:56:31.029078+0000","last_fullsized":"2024-09-12T02:56:31.029078+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:55:56.064125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.030503+0000","last_change":"2024-09-12T02:56:15.986003+0000","last_active":"2024-09-12T02:56:31.030503+0000","last_peered":"2024-09-12T02:56:31.030503+0000","last_clean":"2024-09-12T02:56:31.030503+0000","last_became_active":"2024-09-12T02:56:15.985335+0000","last_became_peered":"2024-09-12T02:56:15.985335+0000","last_unstale":"2024-09-12T02:56:31.030503+0000","last_undegraded":"2024-09-12T02:56:31.030503+0000","last_fullsized":"2024-09-12T02:56:31.030503+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:51:47.944530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.036870+0000","last_change":"2024-09-12T02:56:31.036870+0000","last_active":"2024-09-12T02:56:31.036870+0000","last_peered":"2024-09-12T02:56:31.036870+0000","last_clean":"2024-09-12T02:56:31.036870+0000","last_became_active":"2024-09-12T02:56:31.036443+0000","last_became_peered":"2024-09-12T02:56:31.036443+0000","last_unstale":"2024-09-12T02:56:31.036870+0000","last_undegraded":"2024-09-12T02:56:31.036870+0000","last_fullsized":"2024-09-12T02:56:31.036870+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:49:29.326184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.032066+0000","last_change":"2024-09-12T02:56:15.974547+0000","last_active":"2024-09-12T02:56:31.032066+0000","last_peered":"2024-09-12T02:56:31.032066+0000","last_clean":"2024-09-12T02:56:31.032066+0000","last_became_active":"2024-09-12T02:56:15.974174+0000","last_became_peered":"2024-09-12T02:56:15.974174+0000","last_unstale":"2024-09-12T02:56:31.032066+0000","last_undegraded":"2024-09-12T02:56:31.032066+0000","last_fullsized":"2024-09-12T02:56:31.032066+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:22:16.954309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.306220+0000","last_change":"2024-09-12T02:56:15.981771+0000","last_active":"2024-09-12T02:56:31.306220+0000","last_peered":"2024-09-12T02:56:31.306220+0000","last_clean":"2024-09-12T02:56:31.306220+0000","last_became_active":"2024-09-12T02:56:15.980948+0000","last_became_peered":"2024-09-12T02:56:15.980948+0000","last_unstale":"2024-09-12T02:56:31.306220+0000","last_undegraded":"2024-09-12T02:56:31.306220+0000","last_fullsized":"2024-09-12T02:56:31.306220+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:34:35.961348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.034487+0000","last_change":"2024-09-12T02:56:15.982318+0000","last_active":"2024-09-12T02:56:31.034487+0000","last_peered":"2024-09-12T02:56:31.034487+0000","last_clean":"2024-09-12T02:56:31.034487+0000","last_became_active":"2024-09-12T02:56:15.982013+0000","last_became_peered":"2024-09-12T02:56:15.982013+0000","last_unstale":"2024-09-12T02:56:31.034487+0000","last_undegraded":"2024-09-12T02:56:31.034487+0000","last_fullsized":"2024-09-12T02:56:31.034487+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:10:50.895579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.032248+0000","last_change":"2024-09-12T02:56:15.969992+0000","last_active":"2024-09-12T02:56:31.032248+0000","last_peered":"2024-09-12T02:56:31.032248+0000","last_clean":"2024-09-12T02:56:31.032248+0000","last_became_active":"2024-09-12T02:56:15.969621+0000","last_became_peered":"2024-09-12T02:56:15.969621+0000","last_unstale":"2024-09-12T02:56:31.032248+0000","last_undegraded":"2024-09-12T02:56:31.032248+0000","last_fullsized":"2024-09-12T02:56:31.032248+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:47:40.206042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.034306+0000","last_change":"2024-09-12T02:56:15.982315+0000","last_active":"2024-09-12T02:56:31.034306+0000","last_peered":"2024-09-12T02:56:31.034306+0000","last_clean":"2024-09-12T02:56:31.034306+0000","last_became_active":"2024-09-12T02:56:15.981992+0000","last_became_peered":"2024-09-12T02:56:15.981992+0000","last_unstale":"2024-09-12T02:56:31.034306+0000","last_undegraded":"2024-09-12T02:56:31.034306+0000","last_fullsized":"2024-09-12T02:56:31.034306+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:54:04.188418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.030767+0000","last_change":"2024-09-12T02:56:15.980712+0000","last_active":"2024-09-12T02:56:31.030767+0000","last_peered":"2024-09-12T02:56:31.030767+0000","last_clean":"2024-09-12T02:56:31.030767+0000","last_became_active":"2024-09-12T02:56:15.980410+0000","last_became_peered":"2024-09-12T02:56:15.980410+0000","last_unstale":"2024-09-12T02:56:31.030767+0000","last_undegraded":"2024-09-12T02:56:31.030767+0000","last_fullsized":"2024-09-12T02:56:31.030767+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:13:53.844490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.033126+0000","last_change":"2024-09-12T02:56:15.981675+0000","last_active":"2024-09-12T02:56:31.033126+0000","last_peered":"2024-09-12T02:56:31.033126+0000","last_clean":"2024-09-12T02:56:31.033126+0000","last_became_active":"2024-09-12T02:56:15.981372+0000","last_became_peered":"2024-09-12T02:56:15.981372+0000","last_unstale":"2024-09-12T02:56:31.033126+0000","last_undegraded":"2024-09-12T02:56:31.033126+0000","last_fullsized":"2024-09-12T02:56:31.033126+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:39:17.399199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.032248+0000","last_change":"2024-09-12T02:56:15.972750+0000","last_active":"2024-09-12T02:56:31.032248+0000","last_peered":"2024-09-12T02:56:31.032248+0000","last_clean":"2024-09-12T02:56:31.032248+0000","last_became_active":"2024-09-12T02:56:15.972434+0000","last_became_peered":"2024-09-12T02:56:15.972434+0000","last_unstale":"2024-09-12T02:56:31.032248+0000","last_undegraded":"2024-09-12T02:56:31.032248+0000","last_fullsized":"2024-09-12T02:56:31.032248+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:59:22.784908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.032502+0000","last_change":"2024-09-12T02:56:15.973567+0000","last_active":"2024-09-12T02:56:31.032502+0000","last_peered":"2024-09-12T02:56:31.032502+0000","last_clean":"2024-09-12T02:56:31.032502+0000","last_became_active":"2024-09-12T02:56:15.973251+0000","last_became_peered":"2024-09-12T02:56:15.973251+0000","last_unstale":"2024-09-12T02:56:31.032502+0000","last_undegraded":"2024-09-12T02:56:31.032502+0000","last_fullsized":"2024-09-12T02:56:31.032502+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:16:39.536817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.030130+0000","last_change":"2024-09-12T02:56:15.986108+0000","last_active":"2024-09-12T02:56:31.030130+0000","last_peered":"2024-09-12T02:56:31.030130+0000","last_clean":"2024-09-12T02:56:31.030130+0000","last_became_active":"2024-09-12T02:56:15.985701+0000","last_became_peered":"2024-09-12T02:56:15.985701+0000","last_unstale":"2024-09-12T02:56:31.030130+0000","last_undegraded":"2024-09-12T02:56:31.030130+0000","last_fullsized":"2024-09-12T02:56:31.030130+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:59.377473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.030220+0000","last_change":"2024-09-12T02:56:15.980078+0000","last_active":"2024-09-12T02:56:31.030220+0000","last_peered":"2024-09-12T02:56:31.030220+0000","last_clean":"2024-09-12T02:56:31.030220+0000","last_became_active":"2024-09-12T02:56:15.979785+0000","last_became_peered":"2024-09-12T02:56:15.979785+0000","last_unstale":"2024-09-12T02:56:31.030220+0000","last_undegraded":"2024-09-12T02:56:31.030220+0000","last_fullsized":"2024-09-12T02:56:31.030220+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:00:19.252398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.034505+0000","last_change":"2024-09-12T02:56:15.984170+0000","last_active":"2024-09-12T02:56:31.034505+0000","last_peered":"2024-09-12T02:56:31.034505+0000","last_clean":"2024-09-12T02:56:31.034505+0000","last_became_active":"2024-09-12T02:56:15.983416+0000","last_became_peered":"2024-09-12T02:56:15.983416+0000","last_unstale":"2024-09-12T02:56:31.034505+0000","last_undegraded":"2024-09-12T02:56:31.034505+0000","last_fullsized":"2024-09-12T02:56:31.034505+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:13:24.723923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.035795+0000","last_change":"2024-09-12T02:56:15.980987+0000","last_active":"2024-09-12T02:56:31.035795+0000","last_peered":"2024-09-12T02:56:31.035795+0000","last_clean":"2024-09-12T02:56:31.035795+0000","last_became_active":"2024-09-12T02:56:15.980661+0000","last_became_peered":"2024-09-12T02:56:15.980661+0000","last_unstale":"2024-09-12T02:56:31.035795+0000","last_undegraded":"2024-09-12T02:56:31.035795+0000","last_fullsized":"2024-09-12T02:56:31.035795+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:19:49.294698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"81'5","reported_seq":50,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:35.046435+0000","last_change":"2024-09-12T02:56:31.042914+0000","last_active":"2024-09-12T02:56:35.046435+0000","last_peered":"2024-09-12T02:56:35.046435+0000","last_clean":"2024-09-12T02:56:35.046435+0000","last_became_active":"2024-09-12T02:56:31.037356+0000","last_became_peered":"2024-09-12T02:56:31.037356+0000","last_unstale":"2024-09-12T02:56:35.046435+0000","last_undegraded":"2024-09-12T02:56:35.046435+0000","last_fullsized":"2024-09-12T02:56:35.046435+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:17:48.582725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":622,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.036619+0000","last_change":"2024-09-12T02:56:31.036170+0000","last_active":"2024-09-12T02:56:31.036619+0000","last_peered":"2024-09-12T02:56:31.036619+0000","last_clean":"2024-09-12T02:56:31.036619+0000","last_became_active":"2024-09-12T02:56:31.035754+0000","last_became_peered":"2024-09-12T02:56:31.035754+0000","last_unstale":"2024-09-12T02:56:31.036619+0000","last_undegraded":"2024-09-12T02:56:31.036619+0000","last_fullsized":"2024-09-12T02:56:31.036619+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:58:06.464818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1a","version":"81'7","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.032813+0000","last_change":"2024-09-12T02:56:13.931202+0000","last_active":"2024-09-12T02:56:31.032813+0000","last_peered":"2024-09-12T02:56:31.032813+0000","last_clean":"2024-09-12T02:56:31.032813+0000","last_became_active":"2024-09-12T02:56:13.930776+0000","last_became_peered":"2024-09-12T02:56:13.930776+0000","last_unstale":"2024-09-12T02:56:31.032813+0000","last_undegraded":"2024-09-12T02:56:31.032813+0000","last_fullsized":"2024-09-12T02:56:31.032813+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:22:16.919253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"81'3","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.031662+0000","last_change":"2024-09-12T02:56:13.924660+0000","last_active":"2024-09-12T02:56:31.031662+0000","last_peered":"2024-09-12T02:56:31.031662+0000","last_clean":"2024-09-12T02:56:31.031662+0000","last_became_active":"2024-09-12T02:56:13.924342+0000","last_became_peered":"2024-09-12T02:56:13.924342+0000","last_unstale":"2024-09-12T02:56:31.031662+0000","last_undegraded":"2024-09-12T02:56:31.031662+0000","last_fullsized":"2024-09-12T02:56:31.031662+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:48:53.907044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.034409+0000","last_change":"2024-09-12T02:56:15.978723+0000","last_active":"2024-09-12T02:56:31.034409+0000","last_peered":"2024-09-12T02:56:31.034409+0000","last_clean":"2024-09-12T02:56:31.034409+0000","last_became_active":"2024-09-12T02:56:15.978344+0000","last_became_peered":"2024-09-12T02:56:15.978344+0000","last_unstale":"2024-09-12T02:56:31.034409+0000","last_undegraded":"2024-09-12T02:56:31.034409+0000","last_fullsized":"2024-09-12T02:56:31.034409+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:25:04.559395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"81'3","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.032936+0000","last_change":"2024-09-12T02:56:13.933065+0000","last_active":"2024-09-12T02:56:31.032936+0000","last_peered":"2024-09-12T02:56:31.032936+0000","last_clean":"2024-09-12T02:56:31.032936+0000","last_became_active":"2024-09-12T02:56:13.932597+0000","last_became_peered":"2024-09-12T02:56:13.932597+0000","last_unstale":"2024-09-12T02:56:31.032936+0000","last_undegraded":"2024-09-12T02:56:31.032936+0000","last_fullsized":"2024-09-12T02:56:31.032936+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:57:26.563392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.028878+0000","last_change":"2024-09-12T02:56:15.975722+0000","last_active":"2024-09-12T02:56:31.028878+0000","last_peered":"2024-09-12T02:56:31.028878+0000","last_clean":"2024-09-12T02:56:31.028878+0000","last_became_active":"2024-09-12T02:56:15.975342+0000","last_became_peered":"2024-09-12T02:56:15.975342+0000","last_unstale":"2024-09-12T02:56:31.028878+0000","last_undegraded":"2024-09-12T02:56:31.028878+0000","last_fullsized":"2024-09-12T02:56:31.028878+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:33:17.613738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"81'4","reported_seq":46,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:36.863668+0000","last_change":"2024-09-12T02:56:13.929700+0000","last_active":"2024-09-12T02:56:36.863668+0000","last_peered":"2024-09-12T02:56:36.863668+0000","last_clean":"2024-09-12T02:56:36.863668+0000","last_became_active":"2024-09-12T02:56:13.929317+0000","last_became_peered":"2024-09-12T02:56:13.929317+0000","last_unstale":"2024-09-12T02:56:36.863668+0000","last_undegraded":"2024-09-12T02:56:36.863668+0000","last_fullsized":"2024-09-12T02:56:36.863668+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:46:32.172674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.306705+0000","last_change":"2024-09-12T02:56:15.985754+0000","last_active":"2024-09-12T02:56:31.306705+0000","last_peered":"2024-09-12T02:56:31.306705+0000","last_clean":"2024-09-12T02:56:31.306705+0000","last_became_active":"2024-09-12T02:56:15.985402+0000","last_became_peered":"2024-09-12T02:56:15.985402+0000","last_unstale":"2024-09-12T02:56:31.306705+0000","last_undegraded":"2024-09-12T02:56:31.306705+0000","last_fullsized":"2024-09-12T02:56:31.306705+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:05:29.979961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"81'4","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.035428+0000","last_change":"2024-09-12T02:56:13.933329+0000","last_active":"2024-09-12T02:56:31.035428+0000","last_peered":"2024-09-12T02:56:31.035428+0000","last_clean":"2024-09-12T02:56:31.035428+0000","last_became_active":"2024-09-12T02:56:13.932960+0000","last_became_peered":"2024-09-12T02:56:13.932960+0000","last_unstale":"2024-09-12T02:56:31.035428+0000","last_undegraded":"2024-09-12T02:56:31.035428+0000","last_fullsized":"2024-09-12T02:56:31.035428+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:02:57.564202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.037340+0000","last_change":"2024-09-12T02:56:31.037340+0000","last_active":"2024-09-12T02:56:31.037340+0000","last_peered":"2024-09-12T02:56:31.037340+0000","last_clean":"2024-09-12T02:56:31.037340+0000","last_became_active":"2024-09-12T02:56:31.036993+0000","last_became_peered":"2024-09-12T02:56:31.036993+0000","last_unstale":"2024-09-12T02:56:31.037340+0000","last_undegraded":"2024-09-12T02:56:31.037340+0000","last_fullsized":"2024-09-12T02:56:31.037340+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:44:27.788791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"81'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.033005+0000","last_change":"2024-09-12T02:56:14.077202+0000","last_active":"2024-09-12T02:56:31.033005+0000","last_peered":"2024-09-12T02:56:31.033005+0000","last_clean":"2024-09-12T02:56:31.033005+0000","last_became_active":"2024-09-12T02:56:14.076610+0000","last_became_peered":"2024-09-12T02:56:14.076610+0000","last_unstale":"2024-09-12T02:56:31.033005+0000","last_undegraded":"2024-09-12T02:56:31.033005+0000","last_fullsized":"2024-09-12T02:56:31.033005+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:37:50.333152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.306307+0000","last_change":"2024-09-12T02:56:15.977161+0000","last_active":"2024-09-12T02:56:31.306307+0000","last_peered":"2024-09-12T02:56:31.306307+0000","last_clean":"2024-09-12T02:56:31.306307+0000","last_became_active":"2024-09-12T02:56:15.976683+0000","last_became_peered":"2024-09-12T02:56:15.976683+0000","last_unstale":"2024-09-12T02:56:31.306307+0000","last_undegraded":"2024-09-12T02:56:31.306307+0000","last_fullsized":"2024-09-12T02:56:31.306307+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:34:07.526890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'28","reported_seq":76,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:37.674313+0000","last_change":"2024-09-12T02:56:13.931680+0000","last_active":"2024-09-12T02:56:37.674313+0000","last_peered":"2024-09-12T02:56:37.674313+0000","last_clean":"2024-09-12T02:56:37.674313+0000","last_became_active":"2024-09-12T02:56:13.931214+0000","last_became_peered":"2024-09-12T02:56:13.931214+0000","last_unstale":"2024-09-12T02:56:37.674313+0000","last_undegraded":"2024-09-12T02:56:37.674313+0000","last_fullsized":"2024-09-12T02:56:37.674313+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:38:34.151933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":52100,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":26,"num_read_kb":52,"num_write":30,"num_write_kb":58,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.035329+0000","last_change":"2024-09-12T02:56:15.977720+0000","last_active":"2024-09-12T02:56:31.035329+0000","last_peered":"2024-09-12T02:56:31.035329+0000","last_clean":"2024-09-12T02:56:31.035329+0000","last_became_active":"2024-09-12T02:56:15.977380+0000","last_became_peered":"2024-09-12T02:56:15.977380+0000","last_unstale":"2024-09-12T02:56:31.035329+0000","last_undegraded":"2024-09-12T02:56:31.035329+0000","last_fullsized":"2024-09-12T02:56:31.035329+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:52:16.841909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"81'9","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.030362+0000","last_change":"2024-09-12T02:56:13.929496+0000","last_active":"2024-09-12T02:56:31.030362+0000","last_peered":"2024-09-12T02:56:31.030362+0000","last_clean":"2024-09-12T02:56:31.030362+0000","last_became_active":"2024-09-12T02:56:13.929117+0000","last_became_peered":"2024-09-12T02:56:13.929117+0000","last_unstale":"2024-09-12T02:56:31.030362+0000","last_undegraded":"2024-09-12T02:56:31.030362+0000","last_fullsized":"2024-09-12T02:56:31.030362+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:59:38.399135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.031759+0000","last_change":"2024-09-12T02:56:15.974323+0000","last_active":"2024-09-12T02:56:31.031759+0000","last_peered":"2024-09-12T02:56:31.031759+0000","last_clean":"2024-09-12T02:56:31.031759+0000","last_became_active":"2024-09-12T02:56:15.973819+0000","last_became_peered":"2024-09-12T02:56:15.973819+0000","last_unstale":"2024-09-12T02:56:31.031759+0000","last_undegraded":"2024-09-12T02:56:31.031759+0000","last_fullsized":"2024-09-12T02:56:31.031759+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:53:13.130695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"81'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.032473+0000","last_change":"2024-09-12T02:56:13.935263+0000","last_active":"2024-09-12T02:56:31.032473+0000","last_peered":"2024-09-12T02:56:31.032473+0000","last_clean":"2024-09-12T02:56:31.032473+0000","last_became_active":"2024-09-12T02:56:13.934805+0000","last_became_peered":"2024-09-12T02:56:13.934805+0000","last_unstale":"2024-09-12T02:56:31.032473+0000","last_undegraded":"2024-09-12T02:56:31.032473+0000","last_fullsized":"2024-09-12T02:56:31.032473+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:39:42.882031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.035564+0000","last_change":"2024-09-12T02:56:15.976630+0000","last_active":"2024-09-12T02:56:31.035564+0000","last_peered":"2024-09-12T02:56:31.035564+0000","last_clean":"2024-09-12T02:56:31.035564+0000","last_became_active":"2024-09-12T02:56:15.976304+0000","last_became_peered":"2024-09-12T02:56:15.976304+0000","last_unstale":"2024-09-12T02:56:31.035564+0000","last_undegraded":"2024-09-12T02:56:31.035564+0000","last_fullsized":"2024-09-12T02:56:31.035564+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:46:03.366740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"81'6","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:34.861091+0000","last_change":"2024-09-12T02:56:13.934224+0000","last_active":"2024-09-12T02:56:34.861091+0000","last_peered":"2024-09-12T02:56:34.861091+0000","last_clean":"2024-09-12T02:56:34.861091+0000","last_became_active":"2024-09-12T02:56:13.933838+0000","last_became_peered":"2024-09-12T02:56:13.933838+0000","last_unstale":"2024-09-12T02:56:34.861091+0000","last_undegraded":"2024-09-12T02:56:34.861091+0000","last_fullsized":"2024-09-12T02:56:34.861091+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:34:34.206051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":13,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.035229+0000","last_change":"2024-09-12T02:56:15.980364+0000","last_active":"2024-09-12T02:56:31.035229+0000","last_peered":"2024-09-12T02:56:31.035229+0000","last_clean":"2024-09-12T02:56:31.035229+0000","last_became_active":"2024-09-12T02:56:15.979437+0000","last_became_peered":"2024-09-12T02:56:15.979437+0000","last_unstale":"2024-09-12T02:56:31.035229+0000","last_undegraded":"2024-09-12T02:56:31.035229+0000","last_fullsized":"2024-09-12T02:56:31.035229+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:43:33.424867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"81'17","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.050509+0000","last_change":"2024-09-12T02:56:31.050046+0000","last_active":"2024-09-12T02:56:31.050509+0000","last_peered":"2024-09-12T02:56:31.050509+0000","last_clean":"2024-09-12T02:56:31.050509+0000","last_became_active":"2024-09-12T02:56:31.036990+0000","last_became_peered":"2024-09-12T02:56:31.036990+0000","last_unstale":"2024-09-12T02:56:31.050509+0000","last_undegraded":"2024-09-12T02:56:31.050509+0000","last_fullsized":"2024-09-12T02:56:31.050509+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:05:53.336691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.032226+0000","last_change":"2024-09-12T02:56:15.986434+0000","last_active":"2024-09-12T02:56:31.032226+0000","last_peered":"2024-09-12T02:56:31.032226+0000","last_clean":"2024-09-12T02:56:31.032226+0000","last_became_active":"2024-09-12T02:56:15.986101+0000","last_became_peered":"2024-09-12T02:56:15.986101+0000","last_unstale":"2024-09-12T02:56:31.032226+0000","last_undegraded":"2024-09-12T02:56:31.032226+0000","last_fullsized":"2024-09-12T02:56:31.032226+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:51:27.840951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"81'2","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.035974+0000","last_change":"2024-09-12T02:56:13.925068+0000","last_active":"2024-09-12T02:56:31.035974+0000","last_peered":"2024-09-12T02:56:31.035974+0000","last_clean":"2024-09-12T02:56:31.035974+0000","last_became_active":"2024-09-12T02:56:13.924535+0000","last_became_peered":"2024-09-12T02:56:13.924535+0000","last_unstale":"2024-09-12T02:56:31.035974+0000","last_undegraded":"2024-09-12T02:56:31.035974+0000","last_fullsized":"2024-09-12T02:56:31.035974+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:45:32.478632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.033033+0000","last_change":"2024-09-12T02:56:15.983850+0000","last_active":"2024-09-12T02:56:31.033033+0000","last_peered":"2024-09-12T02:56:31.033033+0000","last_clean":"2024-09-12T02:56:31.033033+0000","last_became_active":"2024-09-12T02:56:15.983480+0000","last_became_peered":"2024-09-12T02:56:15.983480+0000","last_unstale":"2024-09-12T02:56:31.033033+0000","last_undegraded":"2024-09-12T02:56:31.033033+0000","last_fullsized":"2024-09-12T02:56:31.033033+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:09:42.185237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"81'5","reported_seq":39,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.312450+0000","last_change":"2024-09-12T02:56:31.312450+0000","last_active":"2024-09-12T02:56:31.312450+0000","last_peered":"2024-09-12T02:56:31.312450+0000","last_clean":"2024-09-12T02:56:31.312450+0000","last_became_active":"2024-09-12T02:56:31.308030+0000","last_became_peered":"2024-09-12T02:56:31.308030+0000","last_unstale":"2024-09-12T02:56:31.312450+0000","last_undegraded":"2024-09-12T02:56:31.312450+0000","last_fullsized":"2024-09-12T02:56:31.312450+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:15:14.692292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.035489+0000","last_change":"2024-09-12T02:56:15.978996+0000","last_active":"2024-09-12T02:56:31.035489+0000","last_peered":"2024-09-12T02:56:31.035489+0000","last_clean":"2024-09-12T02:56:31.035489+0000","last_became_active":"2024-09-12T02:56:15.978636+0000","last_became_peered":"2024-09-12T02:56:15.978636+0000","last_unstale":"2024-09-12T02:56:31.035489+0000","last_undegraded":"2024-09-12T02:56:31.035489+0000","last_fullsized":"2024-09-12T02:56:31.035489+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:29:20.401664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"81'3","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.033119+0000","last_change":"2024-09-12T02:56:13.929388+0000","last_active":"2024-09-12T02:56:31.033119+0000","last_peered":"2024-09-12T02:56:31.033119+0000","last_clean":"2024-09-12T02:56:31.033119+0000","last_became_active":"2024-09-12T02:56:13.928998+0000","last_became_peered":"2024-09-12T02:56:13.928998+0000","last_unstale":"2024-09-12T02:56:31.033119+0000","last_undegraded":"2024-09-12T02:56:31.033119+0000","last_fullsized":"2024-09-12T02:56:31.033119+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:37:57.368050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'1","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.035355+0000","last_change":"2024-09-12T02:56:15.977491+0000","last_active":"2024-09-12T02:56:31.035355+0000","last_peered":"2024-09-12T02:56:31.035355+0000","last_clean":"2024-09-12T02:56:31.035355+0000","last_became_active":"2024-09-12T02:56:15.977160+0000","last_became_peered":"2024-09-12T02:56:15.977160+0000","last_unstale":"2024-09-12T02:56:31.035355+0000","last_undegraded":"2024-09-12T02:56:31.035355+0000","last_fullsized":"2024-09-12T02:56:31.035355+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:05:40.577342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"81'6","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:35.862996+0000","last_change":"2024-09-12T02:56:14.327147+0000","last_active":"2024-09-12T02:56:35.862996+0000","last_peered":"2024-09-12T02:56:35.862996+0000","last_clean":"2024-09-12T02:56:35.862996+0000","last_became_active":"2024-09-12T02:56:14.326737+0000","last_became_peered":"2024-09-12T02:56:14.326737+0000","last_unstale":"2024-09-12T02:56:35.862996+0000","last_undegraded":"2024-09-12T02:56:35.862996+0000","last_fullsized":"2024-09-12T02:56:35.862996+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:55:33.510599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1563,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":13,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.035591+0000","last_change":"2024-09-12T02:56:15.984363+0000","last_active":"2024-09-12T02:56:31.035591+0000","last_peered":"2024-09-12T02:56:31.035591+0000","last_clean":"2024-09-12T02:56:31.035591+0000","last_became_active":"2024-09-12T02:56:15.983822+0000","last_became_peered":"2024-09-12T02:56:15.983822+0000","last_unstale":"2024-09-12T02:56:31.035591+0000","last_undegraded":"2024-09-12T02:56:31.035591+0000","last_fullsized":"2024-09-12T02:56:31.035591+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:41:46.082716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":332,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.035507+0000","last_change":"2024-09-12T02:55:05.800210+0000","last_active":"2024-09-12T02:56:31.035507+0000","last_peered":"2024-09-12T02:56:31.035507+0000","last_clean":"2024-09-12T02:56:31.035507+0000","last_became_active":"2024-09-12T02:55:05.799791+0000","last_became_peered":"2024-09-12T02:55:05.799791+0000","last_unstale":"2024-09-12T02:56:31.035507+0000","last_undegraded":"2024-09-12T02:56:31.035507+0000","last_fullsized":"2024-09-12T02:56:31.035507+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:51:48.530016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:51:48.530016+0000","last_clean_scrub_stamp":"2024-09-12T02:51:48.530016+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:05:47.416697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"81'6","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.031169+0000","last_change":"2024-09-12T02:56:13.930269+0000","last_active":"2024-09-12T02:56:31.031169+0000","last_peered":"2024-09-12T02:56:31.031169+0000","last_clean":"2024-09-12T02:56:31.031169+0000","last_became_active":"2024-09-12T02:56:13.929831+0000","last_became_peered":"2024-09-12T02:56:13.929831+0000","last_unstale":"2024-09-12T02:56:31.031169+0000","last_undegraded":"2024-09-12T02:56:31.031169+0000","last_fullsized":"2024-09-12T02:56:31.031169+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:38:17.878398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.034774+0000","last_change":"2024-09-12T02:56:15.969744+0000","last_active":"2024-09-12T02:56:31.034774+0000","last_peered":"2024-09-12T02:56:31.034774+0000","last_clean":"2024-09-12T02:56:31.034774+0000","last_became_active":"2024-09-12T02:56:15.969354+0000","last_became_peered":"2024-09-12T02:56:15.969354+0000","last_unstale":"2024-09-12T02:56:31.034774+0000","last_undegraded":"2024-09-12T02:56:31.034774+0000","last_fullsized":"2024-09-12T02:56:31.034774+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:26:02.409510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"81'4","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.035385+0000","last_change":"2024-09-12T02:56:13.929231+0000","last_active":"2024-09-12T02:56:31.035385+0000","last_peered":"2024-09-12T02:56:31.035385+0000","last_clean":"2024-09-12T02:56:31.035385+0000","last_became_active":"2024-09-12T02:56:13.928843+0000","last_became_peered":"2024-09-12T02:56:13.928843+0000","last_unstale":"2024-09-12T02:56:31.035385+0000","last_undegraded":"2024-09-12T02:56:31.035385+0000","last_fullsized":"2024-09-12T02:56:31.035385+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:26:33.619755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.031654+0000","last_change":"2024-09-12T02:56:15.982503+0000","last_active":"2024-09-12T02:56:31.031654+0000","last_peered":"2024-09-12T02:56:31.031654+0000","last_clean":"2024-09-12T02:56:31.031654+0000","last_became_active":"2024-09-12T02:56:15.982192+0000","last_became_peered":"2024-09-12T02:56:15.982192+0000","last_unstale":"2024-09-12T02:56:31.031654+0000","last_undegraded":"2024-09-12T02:56:31.031654+0000","last_fullsized":"2024-09-12T02:56:31.031654+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:55:18.370967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"81'5","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:33.859471+0000","last_change":"2024-09-12T02:56:14.075486+0000","last_active":"2024-09-12T02:56:33.859471+0000","last_peered":"2024-09-12T02:56:33.859471+0000","last_clean":"2024-09-12T02:56:33.859471+0000","last_became_active":"2024-09-12T02:56:14.075152+0000","last_became_peered":"2024-09-12T02:56:14.075152+0000","last_unstale":"2024-09-12T02:56:33.859471+0000","last_undegraded":"2024-09-12T02:56:33.859471+0000","last_fullsized":"2024-09-12T02:56:33.859471+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:59:40.950191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":9,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.032420+0000","last_change":"2024-09-12T02:56:15.981584+0000","last_active":"2024-09-12T02:56:31.032420+0000","last_peered":"2024-09-12T02:56:31.032420+0000","last_clean":"2024-09-12T02:56:31.032420+0000","last_became_active":"2024-09-12T02:56:15.981240+0000","last_became_peered":"2024-09-12T02:56:15.981240+0000","last_unstale":"2024-09-12T02:56:31.032420+0000","last_undegraded":"2024-09-12T02:56:31.032420+0000","last_fullsized":"2024-09-12T02:56:31.032420+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:49:04.036830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"81'5","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:37.864430+0000","last_change":"2024-09-12T02:56:13.935143+0000","last_active":"2024-09-12T02:56:37.864430+0000","last_peered":"2024-09-12T02:56:37.864430+0000","last_clean":"2024-09-12T02:56:37.864430+0000","last_became_active":"2024-09-12T02:56:13.934792+0000","last_became_peered":"2024-09-12T02:56:13.934792+0000","last_unstale":"2024-09-12T02:56:37.864430+0000","last_undegraded":"2024-09-12T02:56:37.864430+0000","last_fullsized":"2024-09-12T02:56:37.864430+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:00:26.973612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.034825+0000","last_change":"2024-09-12T02:56:31.034825+0000","last_active":"2024-09-12T02:56:31.034825+0000","last_peered":"2024-09-12T02:56:31.034825+0000","last_clean":"2024-09-12T02:56:31.034825+0000","last_became_active":"2024-09-12T02:56:31.034516+0000","last_became_peered":"2024-09-12T02:56:31.034516+0000","last_unstale":"2024-09-12T02:56:31.034825+0000","last_undegraded":"2024-09-12T02:56:31.034825+0000","last_fullsized":"2024-09-12T02:56:31.034825+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:10:42.161023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"81'1","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.029027+0000","last_change":"2024-09-12T02:56:14.076762+0000","last_active":"2024-09-12T02:56:31.029027+0000","last_peered":"2024-09-12T02:56:31.029027+0000","last_clean":"2024-09-12T02:56:31.029027+0000","last_became_active":"2024-09-12T02:56:14.076421+0000","last_became_peered":"2024-09-12T02:56:14.076421+0000","last_unstale":"2024-09-12T02:56:31.029027+0000","last_undegraded":"2024-09-12T02:56:31.029027+0000","last_fullsized":"2024-09-12T02:56:31.029027+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:56:56.422910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'1","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.033103+0000","last_change":"2024-09-12T02:56:15.978810+0000","last_active":"2024-09-12T02:56:31.033103+0000","last_peered":"2024-09-12T02:56:31.033103+0000","last_clean":"2024-09-12T02:56:31.033103+0000","last_became_active":"2024-09-12T02:56:15.978038+0000","last_became_peered":"2024-09-12T02:56:15.978038+0000","last_unstale":"2024-09-12T02:56:31.033103+0000","last_undegraded":"2024-09-12T02:56:31.033103+0000","last_fullsized":"2024-09-12T02:56:31.033103+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:04:52.215784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"81'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.035440+0000","last_change":"2024-09-12T02:56:14.075987+0000","last_active":"2024-09-12T02:56:31.035440+0000","last_peered":"2024-09-12T02:56:31.035440+0000","last_clean":"2024-09-12T02:56:31.035440+0000","last_became_active":"2024-09-12T02:56:14.075596+0000","last_became_peered":"2024-09-12T02:56:14.075596+0000","last_unstale":"2024-09-12T02:56:31.035440+0000","last_undegraded":"2024-09-12T02:56:31.035440+0000","last_fullsized":"2024-09-12T02:56:31.035440+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:18:48.713387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.032345+0000","last_change":"2024-09-12T02:56:15.987842+0000","last_active":"2024-09-12T02:56:31.032345+0000","last_peered":"2024-09-12T02:56:31.032345+0000","last_clean":"2024-09-12T02:56:31.032345+0000","last_became_active":"2024-09-12T02:56:15.987507+0000","last_became_peered":"2024-09-12T02:56:15.987507+0000","last_unstale":"2024-09-12T02:56:31.032345+0000","last_undegraded":"2024-09-12T02:56:31.032345+0000","last_fullsized":"2024-09-12T02:56:31.032345+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:52:23.000549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'14","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:37.979903+0000","last_change":"2024-09-12T02:56:13.924372+0000","last_active":"2024-09-12T02:56:37.979903+0000","last_peered":"2024-09-12T02:56:37.979903+0000","last_clean":"2024-09-12T02:56:37.979903+0000","last_became_active":"2024-09-12T02:56:13.923955+0000","last_became_peered":"2024-09-12T02:56:13.923955+0000","last_unstale":"2024-09-12T02:56:37.979903+0000","last_undegraded":"2024-09-12T02:56:37.979903+0000","last_fullsized":"2024-09-12T02:56:37.979903+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:52:47.452410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.034382+0000","last_change":"2024-09-12T02:56:15.978262+0000","last_active":"2024-09-12T02:56:31.034382+0000","last_peered":"2024-09-12T02:56:31.034382+0000","last_clean":"2024-09-12T02:56:31.034382+0000","last_became_active":"2024-09-12T02:56:15.977913+0000","last_became_peered":"2024-09-12T02:56:15.977913+0000","last_unstale":"2024-09-12T02:56:31.034382+0000","last_undegraded":"2024-09-12T02:56:31.034382+0000","last_fullsized":"2024-09-12T02:56:31.034382+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:35:27.104635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"81'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.031488+0000","last_change":"2024-09-12T02:56:14.327417+0000","last_active":"2024-09-12T02:56:31.031488+0000","last_peered":"2024-09-12T02:56:31.031488+0000","last_clean":"2024-09-12T02:56:31.031488+0000","last_became_active":"2024-09-12T02:56:14.327001+0000","last_became_peered":"2024-09-12T02:56:14.327001+0000","last_unstale":"2024-09-12T02:56:31.031488+0000","last_undegraded":"2024-09-12T02:56:31.031488+0000","last_fullsized":"2024-09-12T02:56:31.031488+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:15:39.019252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.032658+0000","last_change":"2024-09-12T02:56:15.977286+0000","last_active":"2024-09-12T02:56:31.032658+0000","last_peered":"2024-09-12T02:56:31.032658+0000","last_clean":"2024-09-12T02:56:31.032658+0000","last_became_active":"2024-09-12T02:56:15.976944+0000","last_became_peered":"2024-09-12T02:56:15.976944+0000","last_unstale":"2024-09-12T02:56:31.032658+0000","last_undegraded":"2024-09-12T02:56:31.032658+0000","last_fullsized":"2024-09-12T02:56:31.032658+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:22:57.056781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.306212+0000","last_change":"2024-09-12T02:56:14.075701+0000","last_active":"2024-09-12T02:56:31.306212+0000","last_peered":"2024-09-12T02:56:31.306212+0000","last_clean":"2024-09-12T02:56:31.306212+0000","last_became_active":"2024-09-12T02:56:14.075340+0000","last_became_peered":"2024-09-12T02:56:14.075340+0000","last_unstale":"2024-09-12T02:56:31.306212+0000","last_undegraded":"2024-09-12T02:56:31.306212+0000","last_fullsized":"2024-09-12T02:56:31.306212+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:17:42.255058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.034874+0000","last_change":"2024-09-12T02:56:15.979783+0000","last_active":"2024-09-12T02:56:31.034874+0000","last_peered":"2024-09-12T02:56:31.034874+0000","last_clean":"2024-09-12T02:56:31.034874+0000","last_became_active":"2024-09-12T02:56:15.979424+0000","last_became_peered":"2024-09-12T02:56:15.979424+0000","last_unstale":"2024-09-12T02:56:31.034874+0000","last_undegraded":"2024-09-12T02:56:31.034874+0000","last_fullsized":"2024-09-12T02:56:31.034874+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:59:11.568138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"81'7","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.034660+0000","last_change":"2024-09-12T02:56:13.928996+0000","last_active":"2024-09-12T02:56:31.034660+0000","last_peered":"2024-09-12T02:56:31.034660+0000","last_clean":"2024-09-12T02:56:31.034660+0000","last_became_active":"2024-09-12T02:56:13.928664+0000","last_became_peered":"2024-09-12T02:56:13.928664+0000","last_unstale":"2024-09-12T02:56:31.034660+0000","last_undegraded":"2024-09-12T02:56:31.034660+0000","last_fullsized":"2024-09-12T02:56:31.034660+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:37:22.067930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.034581+0000","last_change":"2024-09-12T02:56:15.984096+0000","last_active":"2024-09-12T02:56:31.034581+0000","last_peered":"2024-09-12T02:56:31.034581+0000","last_clean":"2024-09-12T02:56:31.034581+0000","last_became_active":"2024-09-12T02:56:15.983737+0000","last_became_peered":"2024-09-12T02:56:15.983737+0000","last_unstale":"2024-09-12T02:56:31.034581+0000","last_undegraded":"2024-09-12T02:56:31.034581+0000","last_fullsized":"2024-09-12T02:56:31.034581+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:02:17.892318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"81'3","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.053785+0000","last_change":"2024-09-12T02:56:31.053343+0000","last_active":"2024-09-12T02:56:31.053785+0000","last_peered":"2024-09-12T02:56:31.053785+0000","last_clean":"2024-09-12T02:56:31.053785+0000","last_became_active":"2024-09-12T02:56:31.037918+0000","last_became_peered":"2024-09-12T02:56:31.037918+0000","last_unstale":"2024-09-12T02:56:31.053785+0000","last_undegraded":"2024-09-12T02:56:31.053785+0000","last_fullsized":"2024-09-12T02:56:31.053785+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:39:54.990724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.034343+0000","last_change":"2024-09-12T02:56:15.975640+0000","last_active":"2024-09-12T02:56:31.034343+0000","last_peered":"2024-09-12T02:56:31.034343+0000","last_clean":"2024-09-12T02:56:31.034343+0000","last_became_active":"2024-09-12T02:56:15.975342+0000","last_became_peered":"2024-09-12T02:56:15.975342+0000","last_unstale":"2024-09-12T02:56:31.034343+0000","last_undegraded":"2024-09-12T02:56:31.034343+0000","last_fullsized":"2024-09-12T02:56:31.034343+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:26:23.694876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.034922+0000","last_change":"2024-09-12T02:56:13.934149+0000","last_active":"2024-09-12T02:56:31.034922+0000","last_peered":"2024-09-12T02:56:31.034922+0000","last_clean":"2024-09-12T02:56:31.034922+0000","last_became_active":"2024-09-12T02:56:13.933743+0000","last_became_peered":"2024-09-12T02:56:13.933743+0000","last_unstale":"2024-09-12T02:56:31.034922+0000","last_undegraded":"2024-09-12T02:56:31.034922+0000","last_fullsized":"2024-09-12T02:56:31.034922+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:16:17.854457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.035435+0000","last_change":"2024-09-12T02:56:15.984782+0000","last_active":"2024-09-12T02:56:31.035435+0000","last_peered":"2024-09-12T02:56:31.035435+0000","last_clean":"2024-09-12T02:56:31.035435+0000","last_became_active":"2024-09-12T02:56:15.984336+0000","last_became_peered":"2024-09-12T02:56:15.984336+0000","last_unstale":"2024-09-12T02:56:31.035435+0000","last_undegraded":"2024-09-12T02:56:31.035435+0000","last_fullsized":"2024-09-12T02:56:31.035435+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:03:39.832897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.032443+0000","last_change":"2024-09-12T02:56:15.985486+0000","last_active":"2024-09-12T02:56:31.032443+0000","last_peered":"2024-09-12T02:56:31.032443+0000","last_clean":"2024-09-12T02:56:31.032443+0000","last_became_active":"2024-09-12T02:56:15.985167+0000","last_became_peered":"2024-09-12T02:56:15.985167+0000","last_unstale":"2024-09-12T02:56:31.032443+0000","last_undegraded":"2024-09-12T02:56:31.032443+0000","last_fullsized":"2024-09-12T02:56:31.032443+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:46:20.569453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"81'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.035106+0000","last_change":"2024-09-12T02:56:13.929324+0000","last_active":"2024-09-12T02:56:31.035106+0000","last_peered":"2024-09-12T02:56:31.035106+0000","last_clean":"2024-09-12T02:56:31.035106+0000","last_became_active":"2024-09-12T02:56:13.928994+0000","last_became_peered":"2024-09-12T02:56:13.928994+0000","last_unstale":"2024-09-12T02:56:31.035106+0000","last_undegraded":"2024-09-12T02:56:31.035106+0000","last_fullsized":"2024-09-12T02:56:31.035106+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:30:27.290328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.306230+0000","last_change":"2024-09-12T02:56:15.985889+0000","last_active":"2024-09-12T02:56:31.306230+0000","last_peered":"2024-09-12T02:56:31.306230+0000","last_clean":"2024-09-12T02:56:31.306230+0000","last_became_active":"2024-09-12T02:56:15.985509+0000","last_became_peered":"2024-09-12T02:56:15.985509+0000","last_unstale":"2024-09-12T02:56:31.306230+0000","last_undegraded":"2024-09-12T02:56:31.306230+0000","last_fullsized":"2024-09-12T02:56:31.306230+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:30:43.120718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"81'12","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.035386+0000","last_change":"2024-09-12T02:56:13.934192+0000","last_active":"2024-09-12T02:56:31.035386+0000","last_peered":"2024-09-12T02:56:31.035386+0000","last_clean":"2024-09-12T02:56:31.035386+0000","last_became_active":"2024-09-12T02:56:13.933789+0000","last_became_peered":"2024-09-12T02:56:13.933789+0000","last_unstale":"2024-09-12T02:56:31.035386+0000","last_undegraded":"2024-09-12T02:56:31.035386+0000","last_fullsized":"2024-09-12T02:56:31.035386+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:32:47.771085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.032496+0000","last_change":"2024-09-12T02:56:15.983531+0000","last_active":"2024-09-12T02:56:31.032496+0000","last_peered":"2024-09-12T02:56:31.032496+0000","last_clean":"2024-09-12T02:56:31.032496+0000","last_became_active":"2024-09-12T02:56:15.983192+0000","last_became_peered":"2024-09-12T02:56:15.983192+0000","last_unstale":"2024-09-12T02:56:31.032496+0000","last_undegraded":"2024-09-12T02:56:31.032496+0000","last_fullsized":"2024-09-12T02:56:31.032496+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:16:59.923379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.034752+0000","last_change":"2024-09-12T02:56:14.075396+0000","last_active":"2024-09-12T02:56:31.034752+0000","last_peered":"2024-09-12T02:56:31.034752+0000","last_clean":"2024-09-12T02:56:31.034752+0000","last_became_active":"2024-09-12T02:56:14.075023+0000","last_became_peered":"2024-09-12T02:56:14.075023+0000","last_unstale":"2024-09-12T02:56:31.034752+0000","last_undegraded":"2024-09-12T02:56:31.034752+0000","last_fullsized":"2024-09-12T02:56:31.034752+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:41:45.760689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.034442+0000","last_change":"2024-09-12T02:56:15.987276+0000","last_active":"2024-09-12T02:56:31.034442+0000","last_peered":"2024-09-12T02:56:31.034442+0000","last_clean":"2024-09-12T02:56:31.034442+0000","last_became_active":"2024-09-12T02:56:15.986949+0000","last_became_peered":"2024-09-12T02:56:15.986949+0000","last_unstale":"2024-09-12T02:56:31.034442+0000","last_undegraded":"2024-09-12T02:56:31.034442+0000","last_fullsized":"2024-09-12T02:56:31.034442+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:49:45.627093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"81'13","reported_seq":48,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:37.864001+0000","last_change":"2024-09-12T02:56:13.932365+0000","last_active":"2024-09-12T02:56:37.864001+0000","last_peered":"2024-09-12T02:56:37.864001+0000","last_clean":"2024-09-12T02:56:37.864001+0000","last_became_active":"2024-09-12T02:56:13.932062+0000","last_became_peered":"2024-09-12T02:56:13.932062+0000","last_unstale":"2024-09-12T02:56:37.864001+0000","last_undegraded":"2024-09-12T02:56:37.864001+0000","last_fullsized":"2024-09-12T02:56:37.864001+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:17:08.451839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":13,"num_read_kb":13,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":59308,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":137,"num_read_kb":123,"num_write":239,"num_write_kb":124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":734,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":516096,"data_stored":155854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":206,"ondisk_log_size":206,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547217,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":917504,"data_stored":440690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776149,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32736,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712416,"statfs":{"total":95995035648,"available":95961513984,"internally_reserved":0,"allocated":1589248,"data_stored":1072672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939675,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40416,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704736,"statfs":{"total":95995035648,"available":95953649664,"internally_reserved":0,"allocated":933888,"data_stored":440115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":40434112},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168608,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":921600,"data_stored":440381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430244,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28516,"kb_used_data":1492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716636,"statfs":{"total":95995035648,"available":95965835264,"internally_reserved":0,"allocated":1527808,"data_stored":1033013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593769,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":917504,"data_stored":439832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822701,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28528,"kb_used_data":1504,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716624,"statfs":{"total":95995035648,"available":95965822976,"internally_reserved":0,"allocated":1540096,"data_stored":1032298,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953523,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32096,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713056,"statfs":{"total":95995035648,"available":95962169344,"internally_reserved":0,"allocated":925696,"data_stored":440708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182455,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":933888,"data_stored":439657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411389,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36292,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708860,"statfs":{"total":95995035648,"available":95957872640,"internally_reserved":0,"allocated":970752,"data_stored":481801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574913,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":958464,"data_stored":481810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738439,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":937984,"data_stored":441564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":42817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":42808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":54229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T02:56:41.078 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-12T02:56:41.335 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:56:41.415 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:41 smithi148 ceph-mon[35090]: pgmap v288: 97 pgs: 97 active+clean; 635 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.7 KiB/s wr, 18 op/s; 77 B/s, 1 objects/s recovering 2024-09-12T02:56:41.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:41 smithi148 ceph-mon[35090]: from='client.14739 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:56:41.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:41 smithi146 ceph-mon[35011]: pgmap v288: 97 pgs: 97 active+clean; 635 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.7 KiB/s wr, 18 op/s; 77 B/s, 1 objects/s recovering 2024-09-12T02:56:41.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:41 smithi146 ceph-mon[35011]: from='client.14739 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:56:41.457 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:41 smithi005 ceph-mon[31197]: pgmap v288: 97 pgs: 97 active+clean; 635 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.7 KiB/s wr, 18 op/s; 77 B/s, 1 objects/s recovering 2024-09-12T02:56:41.457 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:41 smithi005 ceph-mon[31197]: from='client.14739 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:56:41.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:41.763+0000 7f1e74152640 1 -- 172.21.15.5:0/278542767 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e6c106a70 msgr2=0x7f1e6c106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:41.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:41.763+0000 7f1e74152640 1 --2- 172.21.15.5:0/278542767 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e6c106a70 0x7f1e6c106e50 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f1e68009930 tx=0x7f1e6802a420 comp rx=0 tx=0).stop 2024-09-12T02:56:41.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:41.763+0000 7f1e74152640 1 -- 172.21.15.5:0/278542767 shutdown_connections 2024-09-12T02:56:41.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:41.763+0000 7f1e74152640 1 --2- 172.21.15.5:0/278542767 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1e6c101410 0x7f1e6c10d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:41.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:41.763+0000 7f1e74152640 1 --2- 172.21.15.5:0/278542767 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1e6c100a70 0x7f1e6c100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:41.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:41.763+0000 7f1e74152640 1 --2- 172.21.15.5:0/278542767 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e6c106a70 0x7f1e6c106e50 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:41.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:41.763+0000 7f1e74152640 1 -- 172.21.15.5:0/278542767 >> 172.21.15.5:0/278542767 conn(0x7f1e6c0fc790 msgr2=0x7f1e6c0febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:41.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:41.763+0000 7f1e74152640 1 -- 172.21.15.5:0/278542767 shutdown_connections 2024-09-12T02:56:41.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:41.763+0000 7f1e74152640 1 -- 172.21.15.5:0/278542767 wait complete. 2024-09-12T02:56:41.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:41.764+0000 7f1e74152640 1 Processor -- start 2024-09-12T02:56:41.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:41.764+0000 7f1e74152640 1 -- start start 2024-09-12T02:56:41.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:41.764+0000 7f1e74152640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1e6c100a70 0x7f1e6c102640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:41.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:41.764+0000 7f1e74152640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1e6c101410 0x7f1e6c102b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:41.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:41.764+0000 7f1e74152640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e6c106a70 0x7f1e6c1030c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:41.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:41.764+0000 7f1e74152640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1e6c110e70 con 0x7f1e6c106a70 2024-09-12T02:56:41.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:41.764+0000 7f1e74152640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f1e6c110cf0 con 0x7f1e6c100a70 2024-09-12T02:56:41.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:41.764+0000 7f1e74152640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1e6c110ff0 con 0x7f1e6c101410 2024-09-12T02:56:41.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:41.764+0000 7f1e726ef640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e6c106a70 0x7f1e6c1030c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:41.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:41.764+0000 7f1e726ef640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e6c106a70 0x7f1e6c1030c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49602/0 (socket says 172.21.15.5:49602) 2024-09-12T02:56:41.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:41.764+0000 7f1e726ef640 1 -- 172.21.15.5:0/3491070885 learned_addr learned my addr 172.21.15.5:0/3491070885 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:56:41.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:41.764+0000 7f1e71eee640 1 --2- 172.21.15.5:0/3491070885 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1e6c100a70 0x7f1e6c102640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:41.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:41.764+0000 7f1e716ed640 1 --2- 172.21.15.5:0/3491070885 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1e6c101410 0x7f1e6c102b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:41.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:41.764+0000 7f1e726ef640 1 -- 172.21.15.5:0/3491070885 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1e6c101410 msgr2=0x7f1e6c102b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:41.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:41.764+0000 7f1e726ef640 1 --2- 172.21.15.5:0/3491070885 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1e6c101410 0x7f1e6c102b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:41.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:41.764+0000 7f1e726ef640 1 -- 172.21.15.5:0/3491070885 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1e6c100a70 msgr2=0x7f1e6c102640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:41.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:41.764+0000 7f1e726ef640 1 --2- 172.21.15.5:0/3491070885 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1e6c100a70 0x7f1e6c102640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:41.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:41.764+0000 7f1e726ef640 1 -- 172.21.15.5:0/3491070885 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e6c1a68c0 con 0x7f1e6c106a70 2024-09-12T02:56:41.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:41.765+0000 7f1e726ef640 1 --2- 172.21.15.5:0/3491070885 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e6c106a70 0x7f1e6c1030c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f1e6400eaa0 tx=0x7f1e6400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:41.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:41.765+0000 7f1e62ffd640 1 -- 172.21.15.5:0/3491070885 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e6400cdb0 con 0x7f1e6c106a70 2024-09-12T02:56:41.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:41.765+0000 7f1e62ffd640 1 -- 172.21.15.5:0/3491070885 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e64005020 con 0x7f1e6c106a70 2024-09-12T02:56:41.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:41.765+0000 7f1e74152640 1 -- 172.21.15.5:0/3491070885 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e6c1a6bb0 con 0x7f1e6c106a70 2024-09-12T02:56:41.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:41.765+0000 7f1e62ffd640 1 -- 172.21.15.5:0/3491070885 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e640116b0 con 0x7f1e6c106a70 2024-09-12T02:56:41.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:41.765+0000 7f1e74152640 1 -- 172.21.15.5:0/3491070885 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e6c1a70f0 con 0x7f1e6c106a70 2024-09-12T02:56:41.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:41.766+0000 7f1e74152640 1 -- 172.21.15.5:0/3491070885 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e6c100ed0 con 0x7f1e6c106a70 2024-09-12T02:56:41.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:41.767+0000 7f1e62ffd640 1 -- 172.21.15.5:0/3491070885 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f1e64005560 con 0x7f1e6c106a70 2024-09-12T02:56:41.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:41.768+0000 7f1e62ffd640 1 --2- 172.21.15.5:0/3491070885 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f1e580a2c90 0x7f1e580a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:41.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:41.768+0000 7f1e71eee640 1 --2- 172.21.15.5:0/3491070885 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f1e580a2c90 0x7f1e580a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:41.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:41.768+0000 7f1e62ffd640 1 -- 172.21.15.5:0/3491070885 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f1e640cf6d0 con 0x7f1e6c106a70 2024-09-12T02:56:41.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:41.768+0000 7f1e71eee640 1 --2- 172.21.15.5:0/3491070885 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f1e580a2c90 0x7f1e580a5150 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1e68009900 tx=0x7f1e680058d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:41.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:41.771+0000 7f1e62ffd640 1 -- 172.21.15.5:0/3491070885 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1e64016030 con 0x7f1e6c106a70 2024-09-12T02:56:41.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:41.944+0000 7f1e74152640 1 -- 172.21.15.5:0/3491070885 --> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1e6c06af80 con 0x7f1e580a2c90 2024-09-12T02:56:41.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:41.950+0000 7f1e62ffd640 1 -- 172.21.15.5:0/3491070885 <== mgr.14148 v2:172.21.15.5:6800/3780631642 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258767 (secure 0 0 0) 0x7f1e6c06af80 con 0x7f1e580a2c90 2024-09-12T02:56:41.952 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-12T02:56:41.953 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-09-12T02:56:41.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:41.953+0000 7f1e74152640 1 -- 172.21.15.5:0/3491070885 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f1e580a2c90 msgr2=0x7f1e580a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:41.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:41.953+0000 7f1e74152640 1 --2- 172.21.15.5:0/3491070885 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f1e580a2c90 0x7f1e580a5150 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1e68009900 tx=0x7f1e680058d0 comp rx=0 tx=0).stop 2024-09-12T02:56:41.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:41.953+0000 7f1e74152640 1 -- 172.21.15.5:0/3491070885 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e6c106a70 msgr2=0x7f1e6c1030c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:41.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:41.953+0000 7f1e74152640 1 --2- 172.21.15.5:0/3491070885 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e6c106a70 0x7f1e6c1030c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f1e6400eaa0 tx=0x7f1e6400ef60 comp rx=0 tx=0).stop 2024-09-12T02:56:41.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:41.953+0000 7f1e74152640 1 -- 172.21.15.5:0/3491070885 shutdown_connections 2024-09-12T02:56:41.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:41.953+0000 7f1e74152640 1 --2- 172.21.15.5:0/3491070885 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f1e580a2c90 0x7f1e580a5150 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:41.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:41.953+0000 7f1e74152640 1 --2- 172.21.15.5:0/3491070885 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1e6c106a70 0x7f1e6c1030c0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:41.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:41.953+0000 7f1e74152640 1 --2- 172.21.15.5:0/3491070885 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1e6c101410 0x7f1e6c102b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:41.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:41.953+0000 7f1e74152640 1 --2- 172.21.15.5:0/3491070885 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1e6c100a70 0x7f1e6c102640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:41.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:41.953+0000 7f1e74152640 1 -- 172.21.15.5:0/3491070885 >> 172.21.15.5:0/3491070885 conn(0x7f1e6c0fc790 msgr2=0x7f1e6c06a870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:41.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:41.953+0000 7f1e74152640 1 -- 172.21.15.5:0/3491070885 shutdown_connections 2024-09-12T02:56:41.956 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:41.953+0000 7f1e74152640 1 -- 172.21.15.5:0/3491070885 wait complete. 2024-09-12T02:56:42.682 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":289,"stamp":"2024-09-12T02:56:41.547612+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":649910,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":273,"num_read_kb":237,"num_write":422,"num_write_kb":3064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":1181470,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":345,"ondisk_log_size":345,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":374528,"kb_used_data":12768,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124567296,"statfs":{"total":1151940427776,"available":1151556911104,"internally_reserved":0,"allocated":13074432,"data_stored":7184541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":370193723},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":51115,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":116,"num_read_kb":103,"num_write":36,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":734,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.519646"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.033421+0000","last_change":"2024-09-12T02:56:15.988690+0000","last_active":"2024-09-12T02:56:31.033421+0000","last_peered":"2024-09-12T02:56:31.033421+0000","last_clean":"2024-09-12T02:56:31.033421+0000","last_became_active":"2024-09-12T02:56:15.988320+0000","last_became_peered":"2024-09-12T02:56:15.988320+0000","last_unstale":"2024-09-12T02:56:31.033421+0000","last_undegraded":"2024-09-12T02:56:31.033421+0000","last_fullsized":"2024-09-12T02:56:31.033421+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:31:14.157622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.032544+0000","last_change":"2024-09-12T02:56:15.978368+0000","last_active":"2024-09-12T02:56:31.032544+0000","last_peered":"2024-09-12T02:56:31.032544+0000","last_clean":"2024-09-12T02:56:31.032544+0000","last_became_active":"2024-09-12T02:56:15.978092+0000","last_became_peered":"2024-09-12T02:56:15.978092+0000","last_unstale":"2024-09-12T02:56:31.032544+0000","last_undegraded":"2024-09-12T02:56:31.032544+0000","last_fullsized":"2024-09-12T02:56:31.032544+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:32:48.399005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.032403+0000","last_change":"2024-09-12T02:56:15.987249+0000","last_active":"2024-09-12T02:56:31.032403+0000","last_peered":"2024-09-12T02:56:31.032403+0000","last_clean":"2024-09-12T02:56:31.032403+0000","last_became_active":"2024-09-12T02:56:15.986921+0000","last_became_peered":"2024-09-12T02:56:15.986921+0000","last_unstale":"2024-09-12T02:56:31.032403+0000","last_undegraded":"2024-09-12T02:56:31.032403+0000","last_fullsized":"2024-09-12T02:56:31.032403+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:42:13.427871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.031295+0000","last_change":"2024-09-12T02:56:15.987673+0000","last_active":"2024-09-12T02:56:31.031295+0000","last_peered":"2024-09-12T02:56:31.031295+0000","last_clean":"2024-09-12T02:56:31.031295+0000","last_became_active":"2024-09-12T02:56:15.986964+0000","last_became_peered":"2024-09-12T02:56:15.986964+0000","last_unstale":"2024-09-12T02:56:31.031295+0000","last_undegraded":"2024-09-12T02:56:31.031295+0000","last_fullsized":"2024-09-12T02:56:31.031295+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:07:12.406974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.034804+0000","last_change":"2024-09-12T02:56:15.974473+0000","last_active":"2024-09-12T02:56:31.034804+0000","last_peered":"2024-09-12T02:56:31.034804+0000","last_clean":"2024-09-12T02:56:31.034804+0000","last_became_active":"2024-09-12T02:56:15.974124+0000","last_became_peered":"2024-09-12T02:56:15.974124+0000","last_unstale":"2024-09-12T02:56:31.034804+0000","last_undegraded":"2024-09-12T02:56:31.034804+0000","last_fullsized":"2024-09-12T02:56:31.034804+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:44:29.054841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.032976+0000","last_change":"2024-09-12T02:56:15.980242+0000","last_active":"2024-09-12T02:56:31.032976+0000","last_peered":"2024-09-12T02:56:31.032976+0000","last_clean":"2024-09-12T02:56:31.032976+0000","last_became_active":"2024-09-12T02:56:15.979937+0000","last_became_peered":"2024-09-12T02:56:15.979937+0000","last_unstale":"2024-09-12T02:56:31.032976+0000","last_undegraded":"2024-09-12T02:56:31.032976+0000","last_fullsized":"2024-09-12T02:56:31.032976+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:44:13.406341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.035162+0000","last_change":"2024-09-12T02:56:15.978746+0000","last_active":"2024-09-12T02:56:31.035162+0000","last_peered":"2024-09-12T02:56:31.035162+0000","last_clean":"2024-09-12T02:56:31.035162+0000","last_became_active":"2024-09-12T02:56:15.978304+0000","last_became_peered":"2024-09-12T02:56:15.978304+0000","last_unstale":"2024-09-12T02:56:31.035162+0000","last_undegraded":"2024-09-12T02:56:31.035162+0000","last_fullsized":"2024-09-12T02:56:31.035162+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:34:52.115630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.034360+0000","last_change":"2024-09-12T02:56:15.980083+0000","last_active":"2024-09-12T02:56:31.034360+0000","last_peered":"2024-09-12T02:56:31.034360+0000","last_clean":"2024-09-12T02:56:31.034360+0000","last_became_active":"2024-09-12T02:56:15.979771+0000","last_became_peered":"2024-09-12T02:56:15.979771+0000","last_unstale":"2024-09-12T02:56:31.034360+0000","last_undegraded":"2024-09-12T02:56:31.034360+0000","last_fullsized":"2024-09-12T02:56:31.034360+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:59:42.351970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.306732+0000","last_change":"2024-09-12T02:56:15.988091+0000","last_active":"2024-09-12T02:56:31.306732+0000","last_peered":"2024-09-12T02:56:31.306732+0000","last_clean":"2024-09-12T02:56:31.306732+0000","last_became_active":"2024-09-12T02:56:15.986295+0000","last_became_peered":"2024-09-12T02:56:15.986295+0000","last_unstale":"2024-09-12T02:56:31.306732+0000","last_undegraded":"2024-09-12T02:56:31.306732+0000","last_fullsized":"2024-09-12T02:56:31.306732+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:45:40.331805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.033168+0000","last_change":"2024-09-12T02:56:15.987349+0000","last_active":"2024-09-12T02:56:31.033168+0000","last_peered":"2024-09-12T02:56:31.033168+0000","last_clean":"2024-09-12T02:56:31.033168+0000","last_became_active":"2024-09-12T02:56:15.987018+0000","last_became_peered":"2024-09-12T02:56:15.987018+0000","last_unstale":"2024-09-12T02:56:31.033168+0000","last_undegraded":"2024-09-12T02:56:31.033168+0000","last_fullsized":"2024-09-12T02:56:31.033168+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:34:19.652348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.035420+0000","last_change":"2024-09-12T02:56:15.985164+0000","last_active":"2024-09-12T02:56:31.035420+0000","last_peered":"2024-09-12T02:56:31.035420+0000","last_clean":"2024-09-12T02:56:31.035420+0000","last_became_active":"2024-09-12T02:56:15.984787+0000","last_became_peered":"2024-09-12T02:56:15.984787+0000","last_unstale":"2024-09-12T02:56:31.035420+0000","last_undegraded":"2024-09-12T02:56:31.035420+0000","last_fullsized":"2024-09-12T02:56:31.035420+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:36:29.727442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.031157+0000","last_change":"2024-09-12T02:56:15.979963+0000","last_active":"2024-09-12T02:56:31.031157+0000","last_peered":"2024-09-12T02:56:31.031157+0000","last_clean":"2024-09-12T02:56:31.031157+0000","last_became_active":"2024-09-12T02:56:15.979639+0000","last_became_peered":"2024-09-12T02:56:15.979639+0000","last_unstale":"2024-09-12T02:56:31.031157+0000","last_undegraded":"2024-09-12T02:56:31.031157+0000","last_fullsized":"2024-09-12T02:56:31.031157+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:34:18.273245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.031973+0000","last_change":"2024-09-12T02:56:15.980415+0000","last_active":"2024-09-12T02:56:31.031973+0000","last_peered":"2024-09-12T02:56:31.031973+0000","last_clean":"2024-09-12T02:56:31.031973+0000","last_became_active":"2024-09-12T02:56:15.980059+0000","last_became_peered":"2024-09-12T02:56:15.980059+0000","last_unstale":"2024-09-12T02:56:31.031973+0000","last_undegraded":"2024-09-12T02:56:31.031973+0000","last_fullsized":"2024-09-12T02:56:31.031973+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:11:42.689081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.032089+0000","last_change":"2024-09-12T02:56:15.981423+0000","last_active":"2024-09-12T02:56:31.032089+0000","last_peered":"2024-09-12T02:56:31.032089+0000","last_clean":"2024-09-12T02:56:31.032089+0000","last_became_active":"2024-09-12T02:56:15.981186+0000","last_became_peered":"2024-09-12T02:56:15.981186+0000","last_unstale":"2024-09-12T02:56:31.032089+0000","last_undegraded":"2024-09-12T02:56:31.032089+0000","last_fullsized":"2024-09-12T02:56:31.032089+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:36:57.406907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.032139+0000","last_change":"2024-09-12T02:56:15.982587+0000","last_active":"2024-09-12T02:56:31.032139+0000","last_peered":"2024-09-12T02:56:31.032139+0000","last_clean":"2024-09-12T02:56:31.032139+0000","last_became_active":"2024-09-12T02:56:15.982315+0000","last_became_peered":"2024-09-12T02:56:15.982315+0000","last_unstale":"2024-09-12T02:56:31.032139+0000","last_undegraded":"2024-09-12T02:56:31.032139+0000","last_fullsized":"2024-09-12T02:56:31.032139+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:42:17.736999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.306084+0000","last_change":"2024-09-12T02:56:15.979482+0000","last_active":"2024-09-12T02:56:31.306084+0000","last_peered":"2024-09-12T02:56:31.306084+0000","last_clean":"2024-09-12T02:56:31.306084+0000","last_became_active":"2024-09-12T02:56:15.979122+0000","last_became_peered":"2024-09-12T02:56:15.979122+0000","last_unstale":"2024-09-12T02:56:31.306084+0000","last_undegraded":"2024-09-12T02:56:31.306084+0000","last_fullsized":"2024-09-12T02:56:31.306084+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:18:05.642487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.033518+0000","last_change":"2024-09-12T02:56:15.989908+0000","last_active":"2024-09-12T02:56:31.033518+0000","last_peered":"2024-09-12T02:56:31.033518+0000","last_clean":"2024-09-12T02:56:31.033518+0000","last_became_active":"2024-09-12T02:56:15.989515+0000","last_became_peered":"2024-09-12T02:56:15.989515+0000","last_unstale":"2024-09-12T02:56:31.033518+0000","last_undegraded":"2024-09-12T02:56:31.033518+0000","last_fullsized":"2024-09-12T02:56:31.033518+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:59:57.141607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.029078+0000","last_change":"2024-09-12T02:56:15.980323+0000","last_active":"2024-09-12T02:56:31.029078+0000","last_peered":"2024-09-12T02:56:31.029078+0000","last_clean":"2024-09-12T02:56:31.029078+0000","last_became_active":"2024-09-12T02:56:15.979989+0000","last_became_peered":"2024-09-12T02:56:15.979989+0000","last_unstale":"2024-09-12T02:56:31.029078+0000","last_undegraded":"2024-09-12T02:56:31.029078+0000","last_fullsized":"2024-09-12T02:56:31.029078+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:55:56.064125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.030503+0000","last_change":"2024-09-12T02:56:15.986003+0000","last_active":"2024-09-12T02:56:31.030503+0000","last_peered":"2024-09-12T02:56:31.030503+0000","last_clean":"2024-09-12T02:56:31.030503+0000","last_became_active":"2024-09-12T02:56:15.985335+0000","last_became_peered":"2024-09-12T02:56:15.985335+0000","last_unstale":"2024-09-12T02:56:31.030503+0000","last_undegraded":"2024-09-12T02:56:31.030503+0000","last_fullsized":"2024-09-12T02:56:31.030503+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:51:47.944530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.036870+0000","last_change":"2024-09-12T02:56:31.036870+0000","last_active":"2024-09-12T02:56:31.036870+0000","last_peered":"2024-09-12T02:56:31.036870+0000","last_clean":"2024-09-12T02:56:31.036870+0000","last_became_active":"2024-09-12T02:56:31.036443+0000","last_became_peered":"2024-09-12T02:56:31.036443+0000","last_unstale":"2024-09-12T02:56:31.036870+0000","last_undegraded":"2024-09-12T02:56:31.036870+0000","last_fullsized":"2024-09-12T02:56:31.036870+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:49:29.326184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.032066+0000","last_change":"2024-09-12T02:56:15.974547+0000","last_active":"2024-09-12T02:56:31.032066+0000","last_peered":"2024-09-12T02:56:31.032066+0000","last_clean":"2024-09-12T02:56:31.032066+0000","last_became_active":"2024-09-12T02:56:15.974174+0000","last_became_peered":"2024-09-12T02:56:15.974174+0000","last_unstale":"2024-09-12T02:56:31.032066+0000","last_undegraded":"2024-09-12T02:56:31.032066+0000","last_fullsized":"2024-09-12T02:56:31.032066+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:22:16.954309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.306220+0000","last_change":"2024-09-12T02:56:15.981771+0000","last_active":"2024-09-12T02:56:31.306220+0000","last_peered":"2024-09-12T02:56:31.306220+0000","last_clean":"2024-09-12T02:56:31.306220+0000","last_became_active":"2024-09-12T02:56:15.980948+0000","last_became_peered":"2024-09-12T02:56:15.980948+0000","last_unstale":"2024-09-12T02:56:31.306220+0000","last_undegraded":"2024-09-12T02:56:31.306220+0000","last_fullsized":"2024-09-12T02:56:31.306220+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:34:35.961348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.034487+0000","last_change":"2024-09-12T02:56:15.982318+0000","last_active":"2024-09-12T02:56:31.034487+0000","last_peered":"2024-09-12T02:56:31.034487+0000","last_clean":"2024-09-12T02:56:31.034487+0000","last_became_active":"2024-09-12T02:56:15.982013+0000","last_became_peered":"2024-09-12T02:56:15.982013+0000","last_unstale":"2024-09-12T02:56:31.034487+0000","last_undegraded":"2024-09-12T02:56:31.034487+0000","last_fullsized":"2024-09-12T02:56:31.034487+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:10:50.895579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.032248+0000","last_change":"2024-09-12T02:56:15.969992+0000","last_active":"2024-09-12T02:56:31.032248+0000","last_peered":"2024-09-12T02:56:31.032248+0000","last_clean":"2024-09-12T02:56:31.032248+0000","last_became_active":"2024-09-12T02:56:15.969621+0000","last_became_peered":"2024-09-12T02:56:15.969621+0000","last_unstale":"2024-09-12T02:56:31.032248+0000","last_undegraded":"2024-09-12T02:56:31.032248+0000","last_fullsized":"2024-09-12T02:56:31.032248+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:47:40.206042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.034306+0000","last_change":"2024-09-12T02:56:15.982315+0000","last_active":"2024-09-12T02:56:31.034306+0000","last_peered":"2024-09-12T02:56:31.034306+0000","last_clean":"2024-09-12T02:56:31.034306+0000","last_became_active":"2024-09-12T02:56:15.981992+0000","last_became_peered":"2024-09-12T02:56:15.981992+0000","last_unstale":"2024-09-12T02:56:31.034306+0000","last_undegraded":"2024-09-12T02:56:31.034306+0000","last_fullsized":"2024-09-12T02:56:31.034306+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:54:04.188418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.030767+0000","last_change":"2024-09-12T02:56:15.980712+0000","last_active":"2024-09-12T02:56:31.030767+0000","last_peered":"2024-09-12T02:56:31.030767+0000","last_clean":"2024-09-12T02:56:31.030767+0000","last_became_active":"2024-09-12T02:56:15.980410+0000","last_became_peered":"2024-09-12T02:56:15.980410+0000","last_unstale":"2024-09-12T02:56:31.030767+0000","last_undegraded":"2024-09-12T02:56:31.030767+0000","last_fullsized":"2024-09-12T02:56:31.030767+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:13:53.844490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.033126+0000","last_change":"2024-09-12T02:56:15.981675+0000","last_active":"2024-09-12T02:56:31.033126+0000","last_peered":"2024-09-12T02:56:31.033126+0000","last_clean":"2024-09-12T02:56:31.033126+0000","last_became_active":"2024-09-12T02:56:15.981372+0000","last_became_peered":"2024-09-12T02:56:15.981372+0000","last_unstale":"2024-09-12T02:56:31.033126+0000","last_undegraded":"2024-09-12T02:56:31.033126+0000","last_fullsized":"2024-09-12T02:56:31.033126+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:39:17.399199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.032248+0000","last_change":"2024-09-12T02:56:15.972750+0000","last_active":"2024-09-12T02:56:31.032248+0000","last_peered":"2024-09-12T02:56:31.032248+0000","last_clean":"2024-09-12T02:56:31.032248+0000","last_became_active":"2024-09-12T02:56:15.972434+0000","last_became_peered":"2024-09-12T02:56:15.972434+0000","last_unstale":"2024-09-12T02:56:31.032248+0000","last_undegraded":"2024-09-12T02:56:31.032248+0000","last_fullsized":"2024-09-12T02:56:31.032248+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:59:22.784908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.032502+0000","last_change":"2024-09-12T02:56:15.973567+0000","last_active":"2024-09-12T02:56:31.032502+0000","last_peered":"2024-09-12T02:56:31.032502+0000","last_clean":"2024-09-12T02:56:31.032502+0000","last_became_active":"2024-09-12T02:56:15.973251+0000","last_became_peered":"2024-09-12T02:56:15.973251+0000","last_unstale":"2024-09-12T02:56:31.032502+0000","last_undegraded":"2024-09-12T02:56:31.032502+0000","last_fullsized":"2024-09-12T02:56:31.032502+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:16:39.536817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.030130+0000","last_change":"2024-09-12T02:56:15.986108+0000","last_active":"2024-09-12T02:56:31.030130+0000","last_peered":"2024-09-12T02:56:31.030130+0000","last_clean":"2024-09-12T02:56:31.030130+0000","last_became_active":"2024-09-12T02:56:15.985701+0000","last_became_peered":"2024-09-12T02:56:15.985701+0000","last_unstale":"2024-09-12T02:56:31.030130+0000","last_undegraded":"2024-09-12T02:56:31.030130+0000","last_fullsized":"2024-09-12T02:56:31.030130+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:59.377473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.030220+0000","last_change":"2024-09-12T02:56:15.980078+0000","last_active":"2024-09-12T02:56:31.030220+0000","last_peered":"2024-09-12T02:56:31.030220+0000","last_clean":"2024-09-12T02:56:31.030220+0000","last_became_active":"2024-09-12T02:56:15.979785+0000","last_became_peered":"2024-09-12T02:56:15.979785+0000","last_unstale":"2024-09-12T02:56:31.030220+0000","last_undegraded":"2024-09-12T02:56:31.030220+0000","last_fullsized":"2024-09-12T02:56:31.030220+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:00:19.252398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.034505+0000","last_change":"2024-09-12T02:56:15.984170+0000","last_active":"2024-09-12T02:56:31.034505+0000","last_peered":"2024-09-12T02:56:31.034505+0000","last_clean":"2024-09-12T02:56:31.034505+0000","last_became_active":"2024-09-12T02:56:15.983416+0000","last_became_peered":"2024-09-12T02:56:15.983416+0000","last_unstale":"2024-09-12T02:56:31.034505+0000","last_undegraded":"2024-09-12T02:56:31.034505+0000","last_fullsized":"2024-09-12T02:56:31.034505+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:13:24.723923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.035795+0000","last_change":"2024-09-12T02:56:15.980987+0000","last_active":"2024-09-12T02:56:31.035795+0000","last_peered":"2024-09-12T02:56:31.035795+0000","last_clean":"2024-09-12T02:56:31.035795+0000","last_became_active":"2024-09-12T02:56:15.980661+0000","last_became_peered":"2024-09-12T02:56:15.980661+0000","last_unstale":"2024-09-12T02:56:31.035795+0000","last_undegraded":"2024-09-12T02:56:31.035795+0000","last_fullsized":"2024-09-12T02:56:31.035795+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:19:49.294698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"81'5","reported_seq":50,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:35.046435+0000","last_change":"2024-09-12T02:56:31.042914+0000","last_active":"2024-09-12T02:56:35.046435+0000","last_peered":"2024-09-12T02:56:35.046435+0000","last_clean":"2024-09-12T02:56:35.046435+0000","last_became_active":"2024-09-12T02:56:31.037356+0000","last_became_peered":"2024-09-12T02:56:31.037356+0000","last_unstale":"2024-09-12T02:56:35.046435+0000","last_undegraded":"2024-09-12T02:56:35.046435+0000","last_fullsized":"2024-09-12T02:56:35.046435+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:17:48.582725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":622,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.036619+0000","last_change":"2024-09-12T02:56:31.036170+0000","last_active":"2024-09-12T02:56:31.036619+0000","last_peered":"2024-09-12T02:56:31.036619+0000","last_clean":"2024-09-12T02:56:31.036619+0000","last_became_active":"2024-09-12T02:56:31.035754+0000","last_became_peered":"2024-09-12T02:56:31.035754+0000","last_unstale":"2024-09-12T02:56:31.036619+0000","last_undegraded":"2024-09-12T02:56:31.036619+0000","last_fullsized":"2024-09-12T02:56:31.036619+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:58:06.464818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1a","version":"81'7","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.032813+0000","last_change":"2024-09-12T02:56:13.931202+0000","last_active":"2024-09-12T02:56:31.032813+0000","last_peered":"2024-09-12T02:56:31.032813+0000","last_clean":"2024-09-12T02:56:31.032813+0000","last_became_active":"2024-09-12T02:56:13.930776+0000","last_became_peered":"2024-09-12T02:56:13.930776+0000","last_unstale":"2024-09-12T02:56:31.032813+0000","last_undegraded":"2024-09-12T02:56:31.032813+0000","last_fullsized":"2024-09-12T02:56:31.032813+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:22:16.919253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"81'3","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.031662+0000","last_change":"2024-09-12T02:56:13.924660+0000","last_active":"2024-09-12T02:56:31.031662+0000","last_peered":"2024-09-12T02:56:31.031662+0000","last_clean":"2024-09-12T02:56:31.031662+0000","last_became_active":"2024-09-12T02:56:13.924342+0000","last_became_peered":"2024-09-12T02:56:13.924342+0000","last_unstale":"2024-09-12T02:56:31.031662+0000","last_undegraded":"2024-09-12T02:56:31.031662+0000","last_fullsized":"2024-09-12T02:56:31.031662+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:48:53.907044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.034409+0000","last_change":"2024-09-12T02:56:15.978723+0000","last_active":"2024-09-12T02:56:31.034409+0000","last_peered":"2024-09-12T02:56:31.034409+0000","last_clean":"2024-09-12T02:56:31.034409+0000","last_became_active":"2024-09-12T02:56:15.978344+0000","last_became_peered":"2024-09-12T02:56:15.978344+0000","last_unstale":"2024-09-12T02:56:31.034409+0000","last_undegraded":"2024-09-12T02:56:31.034409+0000","last_fullsized":"2024-09-12T02:56:31.034409+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:25:04.559395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"81'3","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.032936+0000","last_change":"2024-09-12T02:56:13.933065+0000","last_active":"2024-09-12T02:56:31.032936+0000","last_peered":"2024-09-12T02:56:31.032936+0000","last_clean":"2024-09-12T02:56:31.032936+0000","last_became_active":"2024-09-12T02:56:13.932597+0000","last_became_peered":"2024-09-12T02:56:13.932597+0000","last_unstale":"2024-09-12T02:56:31.032936+0000","last_undegraded":"2024-09-12T02:56:31.032936+0000","last_fullsized":"2024-09-12T02:56:31.032936+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:57:26.563392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.028878+0000","last_change":"2024-09-12T02:56:15.975722+0000","last_active":"2024-09-12T02:56:31.028878+0000","last_peered":"2024-09-12T02:56:31.028878+0000","last_clean":"2024-09-12T02:56:31.028878+0000","last_became_active":"2024-09-12T02:56:15.975342+0000","last_became_peered":"2024-09-12T02:56:15.975342+0000","last_unstale":"2024-09-12T02:56:31.028878+0000","last_undegraded":"2024-09-12T02:56:31.028878+0000","last_fullsized":"2024-09-12T02:56:31.028878+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:33:17.613738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"81'4","reported_seq":46,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:36.863668+0000","last_change":"2024-09-12T02:56:13.929700+0000","last_active":"2024-09-12T02:56:36.863668+0000","last_peered":"2024-09-12T02:56:36.863668+0000","last_clean":"2024-09-12T02:56:36.863668+0000","last_became_active":"2024-09-12T02:56:13.929317+0000","last_became_peered":"2024-09-12T02:56:13.929317+0000","last_unstale":"2024-09-12T02:56:36.863668+0000","last_undegraded":"2024-09-12T02:56:36.863668+0000","last_fullsized":"2024-09-12T02:56:36.863668+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:46:32.172674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.306705+0000","last_change":"2024-09-12T02:56:15.985754+0000","last_active":"2024-09-12T02:56:31.306705+0000","last_peered":"2024-09-12T02:56:31.306705+0000","last_clean":"2024-09-12T02:56:31.306705+0000","last_became_active":"2024-09-12T02:56:15.985402+0000","last_became_peered":"2024-09-12T02:56:15.985402+0000","last_unstale":"2024-09-12T02:56:31.306705+0000","last_undegraded":"2024-09-12T02:56:31.306705+0000","last_fullsized":"2024-09-12T02:56:31.306705+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:05:29.979961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"81'4","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.035428+0000","last_change":"2024-09-12T02:56:13.933329+0000","last_active":"2024-09-12T02:56:31.035428+0000","last_peered":"2024-09-12T02:56:31.035428+0000","last_clean":"2024-09-12T02:56:31.035428+0000","last_became_active":"2024-09-12T02:56:13.932960+0000","last_became_peered":"2024-09-12T02:56:13.932960+0000","last_unstale":"2024-09-12T02:56:31.035428+0000","last_undegraded":"2024-09-12T02:56:31.035428+0000","last_fullsized":"2024-09-12T02:56:31.035428+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:02:57.564202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.037340+0000","last_change":"2024-09-12T02:56:31.037340+0000","last_active":"2024-09-12T02:56:31.037340+0000","last_peered":"2024-09-12T02:56:31.037340+0000","last_clean":"2024-09-12T02:56:31.037340+0000","last_became_active":"2024-09-12T02:56:31.036993+0000","last_became_peered":"2024-09-12T02:56:31.036993+0000","last_unstale":"2024-09-12T02:56:31.037340+0000","last_undegraded":"2024-09-12T02:56:31.037340+0000","last_fullsized":"2024-09-12T02:56:31.037340+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:44:27.788791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"81'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.033005+0000","last_change":"2024-09-12T02:56:14.077202+0000","last_active":"2024-09-12T02:56:31.033005+0000","last_peered":"2024-09-12T02:56:31.033005+0000","last_clean":"2024-09-12T02:56:31.033005+0000","last_became_active":"2024-09-12T02:56:14.076610+0000","last_became_peered":"2024-09-12T02:56:14.076610+0000","last_unstale":"2024-09-12T02:56:31.033005+0000","last_undegraded":"2024-09-12T02:56:31.033005+0000","last_fullsized":"2024-09-12T02:56:31.033005+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:37:50.333152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.306307+0000","last_change":"2024-09-12T02:56:15.977161+0000","last_active":"2024-09-12T02:56:31.306307+0000","last_peered":"2024-09-12T02:56:31.306307+0000","last_clean":"2024-09-12T02:56:31.306307+0000","last_became_active":"2024-09-12T02:56:15.976683+0000","last_became_peered":"2024-09-12T02:56:15.976683+0000","last_unstale":"2024-09-12T02:56:31.306307+0000","last_undegraded":"2024-09-12T02:56:31.306307+0000","last_fullsized":"2024-09-12T02:56:31.306307+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:34:07.526890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'28","reported_seq":76,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:37.674313+0000","last_change":"2024-09-12T02:56:13.931680+0000","last_active":"2024-09-12T02:56:37.674313+0000","last_peered":"2024-09-12T02:56:37.674313+0000","last_clean":"2024-09-12T02:56:37.674313+0000","last_became_active":"2024-09-12T02:56:13.931214+0000","last_became_peered":"2024-09-12T02:56:13.931214+0000","last_unstale":"2024-09-12T02:56:37.674313+0000","last_undegraded":"2024-09-12T02:56:37.674313+0000","last_fullsized":"2024-09-12T02:56:37.674313+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:38:34.151933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":52100,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":26,"num_read_kb":52,"num_write":30,"num_write_kb":58,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.035329+0000","last_change":"2024-09-12T02:56:15.977720+0000","last_active":"2024-09-12T02:56:31.035329+0000","last_peered":"2024-09-12T02:56:31.035329+0000","last_clean":"2024-09-12T02:56:31.035329+0000","last_became_active":"2024-09-12T02:56:15.977380+0000","last_became_peered":"2024-09-12T02:56:15.977380+0000","last_unstale":"2024-09-12T02:56:31.035329+0000","last_undegraded":"2024-09-12T02:56:31.035329+0000","last_fullsized":"2024-09-12T02:56:31.035329+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:52:16.841909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"81'9","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.030362+0000","last_change":"2024-09-12T02:56:13.929496+0000","last_active":"2024-09-12T02:56:31.030362+0000","last_peered":"2024-09-12T02:56:31.030362+0000","last_clean":"2024-09-12T02:56:31.030362+0000","last_became_active":"2024-09-12T02:56:13.929117+0000","last_became_peered":"2024-09-12T02:56:13.929117+0000","last_unstale":"2024-09-12T02:56:31.030362+0000","last_undegraded":"2024-09-12T02:56:31.030362+0000","last_fullsized":"2024-09-12T02:56:31.030362+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:59:38.399135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.031759+0000","last_change":"2024-09-12T02:56:15.974323+0000","last_active":"2024-09-12T02:56:31.031759+0000","last_peered":"2024-09-12T02:56:31.031759+0000","last_clean":"2024-09-12T02:56:31.031759+0000","last_became_active":"2024-09-12T02:56:15.973819+0000","last_became_peered":"2024-09-12T02:56:15.973819+0000","last_unstale":"2024-09-12T02:56:31.031759+0000","last_undegraded":"2024-09-12T02:56:31.031759+0000","last_fullsized":"2024-09-12T02:56:31.031759+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:53:13.130695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"81'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.032473+0000","last_change":"2024-09-12T02:56:13.935263+0000","last_active":"2024-09-12T02:56:31.032473+0000","last_peered":"2024-09-12T02:56:31.032473+0000","last_clean":"2024-09-12T02:56:31.032473+0000","last_became_active":"2024-09-12T02:56:13.934805+0000","last_became_peered":"2024-09-12T02:56:13.934805+0000","last_unstale":"2024-09-12T02:56:31.032473+0000","last_undegraded":"2024-09-12T02:56:31.032473+0000","last_fullsized":"2024-09-12T02:56:31.032473+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:39:42.882031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.035564+0000","last_change":"2024-09-12T02:56:15.976630+0000","last_active":"2024-09-12T02:56:31.035564+0000","last_peered":"2024-09-12T02:56:31.035564+0000","last_clean":"2024-09-12T02:56:31.035564+0000","last_became_active":"2024-09-12T02:56:15.976304+0000","last_became_peered":"2024-09-12T02:56:15.976304+0000","last_unstale":"2024-09-12T02:56:31.035564+0000","last_undegraded":"2024-09-12T02:56:31.035564+0000","last_fullsized":"2024-09-12T02:56:31.035564+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:46:03.366740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"81'6","reported_seq":45,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:39.867086+0000","last_change":"2024-09-12T02:56:13.934224+0000","last_active":"2024-09-12T02:56:39.867086+0000","last_peered":"2024-09-12T02:56:39.867086+0000","last_clean":"2024-09-12T02:56:39.867086+0000","last_became_active":"2024-09-12T02:56:13.933838+0000","last_became_peered":"2024-09-12T02:56:13.933838+0000","last_unstale":"2024-09-12T02:56:39.867086+0000","last_undegraded":"2024-09-12T02:56:39.867086+0000","last_fullsized":"2024-09-12T02:56:39.867086+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:34:34.206051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.035229+0000","last_change":"2024-09-12T02:56:15.980364+0000","last_active":"2024-09-12T02:56:31.035229+0000","last_peered":"2024-09-12T02:56:31.035229+0000","last_clean":"2024-09-12T02:56:31.035229+0000","last_became_active":"2024-09-12T02:56:15.979437+0000","last_became_peered":"2024-09-12T02:56:15.979437+0000","last_unstale":"2024-09-12T02:56:31.035229+0000","last_undegraded":"2024-09-12T02:56:31.035229+0000","last_fullsized":"2024-09-12T02:56:31.035229+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:43:33.424867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"81'17","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.050509+0000","last_change":"2024-09-12T02:56:31.050046+0000","last_active":"2024-09-12T02:56:31.050509+0000","last_peered":"2024-09-12T02:56:31.050509+0000","last_clean":"2024-09-12T02:56:31.050509+0000","last_became_active":"2024-09-12T02:56:31.036990+0000","last_became_peered":"2024-09-12T02:56:31.036990+0000","last_unstale":"2024-09-12T02:56:31.050509+0000","last_undegraded":"2024-09-12T02:56:31.050509+0000","last_fullsized":"2024-09-12T02:56:31.050509+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:05:53.336691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.032226+0000","last_change":"2024-09-12T02:56:15.986434+0000","last_active":"2024-09-12T02:56:31.032226+0000","last_peered":"2024-09-12T02:56:31.032226+0000","last_clean":"2024-09-12T02:56:31.032226+0000","last_became_active":"2024-09-12T02:56:15.986101+0000","last_became_peered":"2024-09-12T02:56:15.986101+0000","last_unstale":"2024-09-12T02:56:31.032226+0000","last_undegraded":"2024-09-12T02:56:31.032226+0000","last_fullsized":"2024-09-12T02:56:31.032226+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:51:27.840951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"81'2","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.035974+0000","last_change":"2024-09-12T02:56:13.925068+0000","last_active":"2024-09-12T02:56:31.035974+0000","last_peered":"2024-09-12T02:56:31.035974+0000","last_clean":"2024-09-12T02:56:31.035974+0000","last_became_active":"2024-09-12T02:56:13.924535+0000","last_became_peered":"2024-09-12T02:56:13.924535+0000","last_unstale":"2024-09-12T02:56:31.035974+0000","last_undegraded":"2024-09-12T02:56:31.035974+0000","last_fullsized":"2024-09-12T02:56:31.035974+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:45:32.478632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.033033+0000","last_change":"2024-09-12T02:56:15.983850+0000","last_active":"2024-09-12T02:56:31.033033+0000","last_peered":"2024-09-12T02:56:31.033033+0000","last_clean":"2024-09-12T02:56:31.033033+0000","last_became_active":"2024-09-12T02:56:15.983480+0000","last_became_peered":"2024-09-12T02:56:15.983480+0000","last_unstale":"2024-09-12T02:56:31.033033+0000","last_undegraded":"2024-09-12T02:56:31.033033+0000","last_fullsized":"2024-09-12T02:56:31.033033+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:09:42.185237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"81'5","reported_seq":39,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.312450+0000","last_change":"2024-09-12T02:56:31.312450+0000","last_active":"2024-09-12T02:56:31.312450+0000","last_peered":"2024-09-12T02:56:31.312450+0000","last_clean":"2024-09-12T02:56:31.312450+0000","last_became_active":"2024-09-12T02:56:31.308030+0000","last_became_peered":"2024-09-12T02:56:31.308030+0000","last_unstale":"2024-09-12T02:56:31.312450+0000","last_undegraded":"2024-09-12T02:56:31.312450+0000","last_fullsized":"2024-09-12T02:56:31.312450+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:15:14.692292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.035489+0000","last_change":"2024-09-12T02:56:15.978996+0000","last_active":"2024-09-12T02:56:31.035489+0000","last_peered":"2024-09-12T02:56:31.035489+0000","last_clean":"2024-09-12T02:56:31.035489+0000","last_became_active":"2024-09-12T02:56:15.978636+0000","last_became_peered":"2024-09-12T02:56:15.978636+0000","last_unstale":"2024-09-12T02:56:31.035489+0000","last_undegraded":"2024-09-12T02:56:31.035489+0000","last_fullsized":"2024-09-12T02:56:31.035489+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:29:20.401664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"81'3","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.033119+0000","last_change":"2024-09-12T02:56:13.929388+0000","last_active":"2024-09-12T02:56:31.033119+0000","last_peered":"2024-09-12T02:56:31.033119+0000","last_clean":"2024-09-12T02:56:31.033119+0000","last_became_active":"2024-09-12T02:56:13.928998+0000","last_became_peered":"2024-09-12T02:56:13.928998+0000","last_unstale":"2024-09-12T02:56:31.033119+0000","last_undegraded":"2024-09-12T02:56:31.033119+0000","last_fullsized":"2024-09-12T02:56:31.033119+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:37:57.368050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'1","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.035355+0000","last_change":"2024-09-12T02:56:15.977491+0000","last_active":"2024-09-12T02:56:31.035355+0000","last_peered":"2024-09-12T02:56:31.035355+0000","last_clean":"2024-09-12T02:56:31.035355+0000","last_became_active":"2024-09-12T02:56:15.977160+0000","last_became_peered":"2024-09-12T02:56:15.977160+0000","last_unstale":"2024-09-12T02:56:31.035355+0000","last_undegraded":"2024-09-12T02:56:31.035355+0000","last_fullsized":"2024-09-12T02:56:31.035355+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:05:40.577342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"81'6","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:35.862996+0000","last_change":"2024-09-12T02:56:14.327147+0000","last_active":"2024-09-12T02:56:35.862996+0000","last_peered":"2024-09-12T02:56:35.862996+0000","last_clean":"2024-09-12T02:56:35.862996+0000","last_became_active":"2024-09-12T02:56:14.326737+0000","last_became_peered":"2024-09-12T02:56:14.326737+0000","last_unstale":"2024-09-12T02:56:35.862996+0000","last_undegraded":"2024-09-12T02:56:35.862996+0000","last_fullsized":"2024-09-12T02:56:35.862996+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:55:33.510599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1563,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":13,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.035591+0000","last_change":"2024-09-12T02:56:15.984363+0000","last_active":"2024-09-12T02:56:31.035591+0000","last_peered":"2024-09-12T02:56:31.035591+0000","last_clean":"2024-09-12T02:56:31.035591+0000","last_became_active":"2024-09-12T02:56:15.983822+0000","last_became_peered":"2024-09-12T02:56:15.983822+0000","last_unstale":"2024-09-12T02:56:31.035591+0000","last_undegraded":"2024-09-12T02:56:31.035591+0000","last_fullsized":"2024-09-12T02:56:31.035591+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:41:46.082716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":332,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.035507+0000","last_change":"2024-09-12T02:55:05.800210+0000","last_active":"2024-09-12T02:56:31.035507+0000","last_peered":"2024-09-12T02:56:31.035507+0000","last_clean":"2024-09-12T02:56:31.035507+0000","last_became_active":"2024-09-12T02:55:05.799791+0000","last_became_peered":"2024-09-12T02:55:05.799791+0000","last_unstale":"2024-09-12T02:56:31.035507+0000","last_undegraded":"2024-09-12T02:56:31.035507+0000","last_fullsized":"2024-09-12T02:56:31.035507+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:51:48.530016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:51:48.530016+0000","last_clean_scrub_stamp":"2024-09-12T02:51:48.530016+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:05:47.416697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"81'6","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.031169+0000","last_change":"2024-09-12T02:56:13.930269+0000","last_active":"2024-09-12T02:56:31.031169+0000","last_peered":"2024-09-12T02:56:31.031169+0000","last_clean":"2024-09-12T02:56:31.031169+0000","last_became_active":"2024-09-12T02:56:13.929831+0000","last_became_peered":"2024-09-12T02:56:13.929831+0000","last_unstale":"2024-09-12T02:56:31.031169+0000","last_undegraded":"2024-09-12T02:56:31.031169+0000","last_fullsized":"2024-09-12T02:56:31.031169+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:38:17.878398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.034774+0000","last_change":"2024-09-12T02:56:15.969744+0000","last_active":"2024-09-12T02:56:31.034774+0000","last_peered":"2024-09-12T02:56:31.034774+0000","last_clean":"2024-09-12T02:56:31.034774+0000","last_became_active":"2024-09-12T02:56:15.969354+0000","last_became_peered":"2024-09-12T02:56:15.969354+0000","last_unstale":"2024-09-12T02:56:31.034774+0000","last_undegraded":"2024-09-12T02:56:31.034774+0000","last_fullsized":"2024-09-12T02:56:31.034774+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:26:02.409510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"81'4","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.035385+0000","last_change":"2024-09-12T02:56:13.929231+0000","last_active":"2024-09-12T02:56:31.035385+0000","last_peered":"2024-09-12T02:56:31.035385+0000","last_clean":"2024-09-12T02:56:31.035385+0000","last_became_active":"2024-09-12T02:56:13.928843+0000","last_became_peered":"2024-09-12T02:56:13.928843+0000","last_unstale":"2024-09-12T02:56:31.035385+0000","last_undegraded":"2024-09-12T02:56:31.035385+0000","last_fullsized":"2024-09-12T02:56:31.035385+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:26:33.619755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.031654+0000","last_change":"2024-09-12T02:56:15.982503+0000","last_active":"2024-09-12T02:56:31.031654+0000","last_peered":"2024-09-12T02:56:31.031654+0000","last_clean":"2024-09-12T02:56:31.031654+0000","last_became_active":"2024-09-12T02:56:15.982192+0000","last_became_peered":"2024-09-12T02:56:15.982192+0000","last_unstale":"2024-09-12T02:56:31.031654+0000","last_undegraded":"2024-09-12T02:56:31.031654+0000","last_fullsized":"2024-09-12T02:56:31.031654+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:55:18.370967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"81'5","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:38.865606+0000","last_change":"2024-09-12T02:56:14.075486+0000","last_active":"2024-09-12T02:56:38.865606+0000","last_peered":"2024-09-12T02:56:38.865606+0000","last_clean":"2024-09-12T02:56:38.865606+0000","last_became_active":"2024-09-12T02:56:14.075152+0000","last_became_peered":"2024-09-12T02:56:14.075152+0000","last_unstale":"2024-09-12T02:56:38.865606+0000","last_undegraded":"2024-09-12T02:56:38.865606+0000","last_fullsized":"2024-09-12T02:56:38.865606+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:59:40.950191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.032420+0000","last_change":"2024-09-12T02:56:15.981584+0000","last_active":"2024-09-12T02:56:31.032420+0000","last_peered":"2024-09-12T02:56:31.032420+0000","last_clean":"2024-09-12T02:56:31.032420+0000","last_became_active":"2024-09-12T02:56:15.981240+0000","last_became_peered":"2024-09-12T02:56:15.981240+0000","last_unstale":"2024-09-12T02:56:31.032420+0000","last_undegraded":"2024-09-12T02:56:31.032420+0000","last_fullsized":"2024-09-12T02:56:31.032420+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:49:04.036830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"81'5","reported_seq":40,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:37.864430+0000","last_change":"2024-09-12T02:56:13.935143+0000","last_active":"2024-09-12T02:56:37.864430+0000","last_peered":"2024-09-12T02:56:37.864430+0000","last_clean":"2024-09-12T02:56:37.864430+0000","last_became_active":"2024-09-12T02:56:13.934792+0000","last_became_peered":"2024-09-12T02:56:13.934792+0000","last_unstale":"2024-09-12T02:56:37.864430+0000","last_undegraded":"2024-09-12T02:56:37.864430+0000","last_fullsized":"2024-09-12T02:56:37.864430+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:00:26.973612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.034825+0000","last_change":"2024-09-12T02:56:31.034825+0000","last_active":"2024-09-12T02:56:31.034825+0000","last_peered":"2024-09-12T02:56:31.034825+0000","last_clean":"2024-09-12T02:56:31.034825+0000","last_became_active":"2024-09-12T02:56:31.034516+0000","last_became_peered":"2024-09-12T02:56:31.034516+0000","last_unstale":"2024-09-12T02:56:31.034825+0000","last_undegraded":"2024-09-12T02:56:31.034825+0000","last_fullsized":"2024-09-12T02:56:31.034825+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:10:42.161023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"81'1","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.029027+0000","last_change":"2024-09-12T02:56:14.076762+0000","last_active":"2024-09-12T02:56:31.029027+0000","last_peered":"2024-09-12T02:56:31.029027+0000","last_clean":"2024-09-12T02:56:31.029027+0000","last_became_active":"2024-09-12T02:56:14.076421+0000","last_became_peered":"2024-09-12T02:56:14.076421+0000","last_unstale":"2024-09-12T02:56:31.029027+0000","last_undegraded":"2024-09-12T02:56:31.029027+0000","last_fullsized":"2024-09-12T02:56:31.029027+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:56:56.422910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'1","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.033103+0000","last_change":"2024-09-12T02:56:15.978810+0000","last_active":"2024-09-12T02:56:31.033103+0000","last_peered":"2024-09-12T02:56:31.033103+0000","last_clean":"2024-09-12T02:56:31.033103+0000","last_became_active":"2024-09-12T02:56:15.978038+0000","last_became_peered":"2024-09-12T02:56:15.978038+0000","last_unstale":"2024-09-12T02:56:31.033103+0000","last_undegraded":"2024-09-12T02:56:31.033103+0000","last_fullsized":"2024-09-12T02:56:31.033103+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:04:52.215784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"81'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.035440+0000","last_change":"2024-09-12T02:56:14.075987+0000","last_active":"2024-09-12T02:56:31.035440+0000","last_peered":"2024-09-12T02:56:31.035440+0000","last_clean":"2024-09-12T02:56:31.035440+0000","last_became_active":"2024-09-12T02:56:14.075596+0000","last_became_peered":"2024-09-12T02:56:14.075596+0000","last_unstale":"2024-09-12T02:56:31.035440+0000","last_undegraded":"2024-09-12T02:56:31.035440+0000","last_fullsized":"2024-09-12T02:56:31.035440+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:18:48.713387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.032345+0000","last_change":"2024-09-12T02:56:15.987842+0000","last_active":"2024-09-12T02:56:31.032345+0000","last_peered":"2024-09-12T02:56:31.032345+0000","last_clean":"2024-09-12T02:56:31.032345+0000","last_became_active":"2024-09-12T02:56:15.987507+0000","last_became_peered":"2024-09-12T02:56:15.987507+0000","last_unstale":"2024-09-12T02:56:31.032345+0000","last_undegraded":"2024-09-12T02:56:31.032345+0000","last_fullsized":"2024-09-12T02:56:31.032345+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:52:23.000549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'14","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:37.979903+0000","last_change":"2024-09-12T02:56:13.924372+0000","last_active":"2024-09-12T02:56:37.979903+0000","last_peered":"2024-09-12T02:56:37.979903+0000","last_clean":"2024-09-12T02:56:37.979903+0000","last_became_active":"2024-09-12T02:56:13.923955+0000","last_became_peered":"2024-09-12T02:56:13.923955+0000","last_unstale":"2024-09-12T02:56:37.979903+0000","last_undegraded":"2024-09-12T02:56:37.979903+0000","last_fullsized":"2024-09-12T02:56:37.979903+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:52:47.452410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.034382+0000","last_change":"2024-09-12T02:56:15.978262+0000","last_active":"2024-09-12T02:56:31.034382+0000","last_peered":"2024-09-12T02:56:31.034382+0000","last_clean":"2024-09-12T02:56:31.034382+0000","last_became_active":"2024-09-12T02:56:15.977913+0000","last_became_peered":"2024-09-12T02:56:15.977913+0000","last_unstale":"2024-09-12T02:56:31.034382+0000","last_undegraded":"2024-09-12T02:56:31.034382+0000","last_fullsized":"2024-09-12T02:56:31.034382+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:35:27.104635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"81'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.031488+0000","last_change":"2024-09-12T02:56:14.327417+0000","last_active":"2024-09-12T02:56:31.031488+0000","last_peered":"2024-09-12T02:56:31.031488+0000","last_clean":"2024-09-12T02:56:31.031488+0000","last_became_active":"2024-09-12T02:56:14.327001+0000","last_became_peered":"2024-09-12T02:56:14.327001+0000","last_unstale":"2024-09-12T02:56:31.031488+0000","last_undegraded":"2024-09-12T02:56:31.031488+0000","last_fullsized":"2024-09-12T02:56:31.031488+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:15:39.019252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.032658+0000","last_change":"2024-09-12T02:56:15.977286+0000","last_active":"2024-09-12T02:56:31.032658+0000","last_peered":"2024-09-12T02:56:31.032658+0000","last_clean":"2024-09-12T02:56:31.032658+0000","last_became_active":"2024-09-12T02:56:15.976944+0000","last_became_peered":"2024-09-12T02:56:15.976944+0000","last_unstale":"2024-09-12T02:56:31.032658+0000","last_undegraded":"2024-09-12T02:56:31.032658+0000","last_fullsized":"2024-09-12T02:56:31.032658+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:22:57.056781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.306212+0000","last_change":"2024-09-12T02:56:14.075701+0000","last_active":"2024-09-12T02:56:31.306212+0000","last_peered":"2024-09-12T02:56:31.306212+0000","last_clean":"2024-09-12T02:56:31.306212+0000","last_became_active":"2024-09-12T02:56:14.075340+0000","last_became_peered":"2024-09-12T02:56:14.075340+0000","last_unstale":"2024-09-12T02:56:31.306212+0000","last_undegraded":"2024-09-12T02:56:31.306212+0000","last_fullsized":"2024-09-12T02:56:31.306212+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:17:42.255058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.034874+0000","last_change":"2024-09-12T02:56:15.979783+0000","last_active":"2024-09-12T02:56:31.034874+0000","last_peered":"2024-09-12T02:56:31.034874+0000","last_clean":"2024-09-12T02:56:31.034874+0000","last_became_active":"2024-09-12T02:56:15.979424+0000","last_became_peered":"2024-09-12T02:56:15.979424+0000","last_unstale":"2024-09-12T02:56:31.034874+0000","last_undegraded":"2024-09-12T02:56:31.034874+0000","last_fullsized":"2024-09-12T02:56:31.034874+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:59:11.568138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"81'7","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.034660+0000","last_change":"2024-09-12T02:56:13.928996+0000","last_active":"2024-09-12T02:56:31.034660+0000","last_peered":"2024-09-12T02:56:31.034660+0000","last_clean":"2024-09-12T02:56:31.034660+0000","last_became_active":"2024-09-12T02:56:13.928664+0000","last_became_peered":"2024-09-12T02:56:13.928664+0000","last_unstale":"2024-09-12T02:56:31.034660+0000","last_undegraded":"2024-09-12T02:56:31.034660+0000","last_fullsized":"2024-09-12T02:56:31.034660+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:37:22.067930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.034581+0000","last_change":"2024-09-12T02:56:15.984096+0000","last_active":"2024-09-12T02:56:31.034581+0000","last_peered":"2024-09-12T02:56:31.034581+0000","last_clean":"2024-09-12T02:56:31.034581+0000","last_became_active":"2024-09-12T02:56:15.983737+0000","last_became_peered":"2024-09-12T02:56:15.983737+0000","last_unstale":"2024-09-12T02:56:31.034581+0000","last_undegraded":"2024-09-12T02:56:31.034581+0000","last_fullsized":"2024-09-12T02:56:31.034581+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:02:17.892318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"81'3","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.053785+0000","last_change":"2024-09-12T02:56:31.053343+0000","last_active":"2024-09-12T02:56:31.053785+0000","last_peered":"2024-09-12T02:56:31.053785+0000","last_clean":"2024-09-12T02:56:31.053785+0000","last_became_active":"2024-09-12T02:56:31.037918+0000","last_became_peered":"2024-09-12T02:56:31.037918+0000","last_unstale":"2024-09-12T02:56:31.053785+0000","last_undegraded":"2024-09-12T02:56:31.053785+0000","last_fullsized":"2024-09-12T02:56:31.053785+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:39:54.990724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.034343+0000","last_change":"2024-09-12T02:56:15.975640+0000","last_active":"2024-09-12T02:56:31.034343+0000","last_peered":"2024-09-12T02:56:31.034343+0000","last_clean":"2024-09-12T02:56:31.034343+0000","last_became_active":"2024-09-12T02:56:15.975342+0000","last_became_peered":"2024-09-12T02:56:15.975342+0000","last_unstale":"2024-09-12T02:56:31.034343+0000","last_undegraded":"2024-09-12T02:56:31.034343+0000","last_fullsized":"2024-09-12T02:56:31.034343+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:26:23.694876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.034922+0000","last_change":"2024-09-12T02:56:13.934149+0000","last_active":"2024-09-12T02:56:31.034922+0000","last_peered":"2024-09-12T02:56:31.034922+0000","last_clean":"2024-09-12T02:56:31.034922+0000","last_became_active":"2024-09-12T02:56:13.933743+0000","last_became_peered":"2024-09-12T02:56:13.933743+0000","last_unstale":"2024-09-12T02:56:31.034922+0000","last_undegraded":"2024-09-12T02:56:31.034922+0000","last_fullsized":"2024-09-12T02:56:31.034922+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:16:17.854457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.035435+0000","last_change":"2024-09-12T02:56:15.984782+0000","last_active":"2024-09-12T02:56:31.035435+0000","last_peered":"2024-09-12T02:56:31.035435+0000","last_clean":"2024-09-12T02:56:31.035435+0000","last_became_active":"2024-09-12T02:56:15.984336+0000","last_became_peered":"2024-09-12T02:56:15.984336+0000","last_unstale":"2024-09-12T02:56:31.035435+0000","last_undegraded":"2024-09-12T02:56:31.035435+0000","last_fullsized":"2024-09-12T02:56:31.035435+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:03:39.832897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.032443+0000","last_change":"2024-09-12T02:56:15.985486+0000","last_active":"2024-09-12T02:56:31.032443+0000","last_peered":"2024-09-12T02:56:31.032443+0000","last_clean":"2024-09-12T02:56:31.032443+0000","last_became_active":"2024-09-12T02:56:15.985167+0000","last_became_peered":"2024-09-12T02:56:15.985167+0000","last_unstale":"2024-09-12T02:56:31.032443+0000","last_undegraded":"2024-09-12T02:56:31.032443+0000","last_fullsized":"2024-09-12T02:56:31.032443+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:46:20.569453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"81'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.035106+0000","last_change":"2024-09-12T02:56:13.929324+0000","last_active":"2024-09-12T02:56:31.035106+0000","last_peered":"2024-09-12T02:56:31.035106+0000","last_clean":"2024-09-12T02:56:31.035106+0000","last_became_active":"2024-09-12T02:56:13.928994+0000","last_became_peered":"2024-09-12T02:56:13.928994+0000","last_unstale":"2024-09-12T02:56:31.035106+0000","last_undegraded":"2024-09-12T02:56:31.035106+0000","last_fullsized":"2024-09-12T02:56:31.035106+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:30:27.290328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.306230+0000","last_change":"2024-09-12T02:56:15.985889+0000","last_active":"2024-09-12T02:56:31.306230+0000","last_peered":"2024-09-12T02:56:31.306230+0000","last_clean":"2024-09-12T02:56:31.306230+0000","last_became_active":"2024-09-12T02:56:15.985509+0000","last_became_peered":"2024-09-12T02:56:15.985509+0000","last_unstale":"2024-09-12T02:56:31.306230+0000","last_undegraded":"2024-09-12T02:56:31.306230+0000","last_fullsized":"2024-09-12T02:56:31.306230+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:30:43.120718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"81'12","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.035386+0000","last_change":"2024-09-12T02:56:13.934192+0000","last_active":"2024-09-12T02:56:31.035386+0000","last_peered":"2024-09-12T02:56:31.035386+0000","last_clean":"2024-09-12T02:56:31.035386+0000","last_became_active":"2024-09-12T02:56:13.933789+0000","last_became_peered":"2024-09-12T02:56:13.933789+0000","last_unstale":"2024-09-12T02:56:31.035386+0000","last_undegraded":"2024-09-12T02:56:31.035386+0000","last_fullsized":"2024-09-12T02:56:31.035386+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:32:47.771085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.032496+0000","last_change":"2024-09-12T02:56:15.983531+0000","last_active":"2024-09-12T02:56:31.032496+0000","last_peered":"2024-09-12T02:56:31.032496+0000","last_clean":"2024-09-12T02:56:31.032496+0000","last_became_active":"2024-09-12T02:56:15.983192+0000","last_became_peered":"2024-09-12T02:56:15.983192+0000","last_unstale":"2024-09-12T02:56:31.032496+0000","last_undegraded":"2024-09-12T02:56:31.032496+0000","last_fullsized":"2024-09-12T02:56:31.032496+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:16:59.923379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.034752+0000","last_change":"2024-09-12T02:56:14.075396+0000","last_active":"2024-09-12T02:56:31.034752+0000","last_peered":"2024-09-12T02:56:31.034752+0000","last_clean":"2024-09-12T02:56:31.034752+0000","last_became_active":"2024-09-12T02:56:14.075023+0000","last_became_peered":"2024-09-12T02:56:14.075023+0000","last_unstale":"2024-09-12T02:56:31.034752+0000","last_undegraded":"2024-09-12T02:56:31.034752+0000","last_fullsized":"2024-09-12T02:56:31.034752+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:41:45.760689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.034442+0000","last_change":"2024-09-12T02:56:15.987276+0000","last_active":"2024-09-12T02:56:31.034442+0000","last_peered":"2024-09-12T02:56:31.034442+0000","last_clean":"2024-09-12T02:56:31.034442+0000","last_became_active":"2024-09-12T02:56:15.986949+0000","last_became_peered":"2024-09-12T02:56:15.986949+0000","last_unstale":"2024-09-12T02:56:31.034442+0000","last_undegraded":"2024-09-12T02:56:31.034442+0000","last_fullsized":"2024-09-12T02:56:31.034442+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:49:45.627093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"81'13","reported_seq":48,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:37.864001+0000","last_change":"2024-09-12T02:56:13.932365+0000","last_active":"2024-09-12T02:56:37.864001+0000","last_peered":"2024-09-12T02:56:37.864001+0000","last_clean":"2024-09-12T02:56:37.864001+0000","last_became_active":"2024-09-12T02:56:13.932062+0000","last_became_peered":"2024-09-12T02:56:13.932062+0000","last_unstale":"2024-09-12T02:56:37.864001+0000","last_undegraded":"2024-09-12T02:56:37.864001+0000","last_fullsized":"2024-09-12T02:56:37.864001+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:17:08.451839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":13,"num_read_kb":13,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":59308,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":147,"num_read_kb":128,"num_write":239,"num_write_kb":124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":734,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":516096,"data_stored":155854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":206,"ondisk_log_size":206,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547217,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":917504,"data_stored":440690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776149,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32736,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712416,"statfs":{"total":95995035648,"available":95961513984,"internally_reserved":0,"allocated":1589248,"data_stored":1072672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939675,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40416,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704736,"statfs":{"total":95995035648,"available":95953649664,"internally_reserved":0,"allocated":933888,"data_stored":440115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":40434112},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168608,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":921600,"data_stored":440381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430244,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28516,"kb_used_data":1492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716636,"statfs":{"total":95995035648,"available":95965835264,"internally_reserved":0,"allocated":1527808,"data_stored":1033013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593770,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":917504,"data_stored":439832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822701,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28528,"kb_used_data":1504,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716624,"statfs":{"total":95995035648,"available":95965822976,"internally_reserved":0,"allocated":1540096,"data_stored":1032298,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953524,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32096,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713056,"statfs":{"total":95995035648,"available":95962169344,"internally_reserved":0,"allocated":925696,"data_stored":440708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182455,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":933888,"data_stored":439657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411389,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36292,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708860,"statfs":{"total":95995035648,"available":95957872640,"internally_reserved":0,"allocated":970752,"data_stored":481801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574913,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":958464,"data_stored":481810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738440,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":937984,"data_stored":441564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":42817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":42808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":54229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T02:56:42.686 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-12T02:56:42.687 INFO:tasks.cephadm:Setting up client nodes... 2024-09-12T02:56:42.687 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-12T02:56:42.947 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:56:43.330 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:43 smithi005 ceph-mon[31197]: pgmap v289: 97 pgs: 97 active+clean; 635 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 6.0 KiB/s wr, 14 op/s; 69 B/s, 1 objects/s recovering 2024-09-12T02:56:43.330 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:43 smithi005 ceph-mon[31197]: from='client.14745 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:56:43.415 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:43 smithi148 ceph-mon[35090]: pgmap v289: 97 pgs: 97 active+clean; 635 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 6.0 KiB/s wr, 14 op/s; 69 B/s, 1 objects/s recovering 2024-09-12T02:56:43.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:43 smithi148 ceph-mon[35090]: from='client.14745 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:56:43.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:43.421+0000 7ff421e38640 1 -- 172.21.15.5:0/2712828093 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff41c102e00 msgr2=0x7ff41c103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:43.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:43.421+0000 7ff421e38640 1 --2- 172.21.15.5:0/2712828093 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff41c102e00 0x7ff41c103280 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff410009900 tx=0x7ff41002a3f0 comp rx=0 tx=0).stop 2024-09-12T02:56:43.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:43.421+0000 7ff421e38640 1 -- 172.21.15.5:0/2712828093 shutdown_connections 2024-09-12T02:56:43.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:43.421+0000 7ff421e38640 1 --2- 172.21.15.5:0/2712828093 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff41c102e00 0x7ff41c103280 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:43.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:43.421+0000 7ff421e38640 1 --2- 172.21.15.5:0/2712828093 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff41c1037c0 0x7ff41c10a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:43.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:43.421+0000 7ff421e38640 1 --2- 172.21.15.5:0/2712828093 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff41c101c00 0x7ff41c102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:43.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:43.421+0000 7ff421e38640 1 -- 172.21.15.5:0/2712828093 >> 172.21.15.5:0/2712828093 conn(0x7ff41c0fd390 msgr2=0x7ff41c0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:43.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:43.421+0000 7ff421e38640 1 -- 172.21.15.5:0/2712828093 shutdown_connections 2024-09-12T02:56:43.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:43.421+0000 7ff421e38640 1 -- 172.21.15.5:0/2712828093 wait complete. 2024-09-12T02:56:43.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:43.422+0000 7ff421e38640 1 Processor -- start 2024-09-12T02:56:43.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:43.422+0000 7ff421e38640 1 -- start start 2024-09-12T02:56:43.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:43.422+0000 7ff421e38640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff41c101c00 0x7ff41c19a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:43.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:43.422+0000 7ff421e38640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff41c102e00 0x7ff41c19a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:43.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:43.422+0000 7ff421e38640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff41c1037c0 0x7ff41c1a1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:43.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:43.422+0000 7ff421e38640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff41c10d860 con 0x7ff41c101c00 2024-09-12T02:56:43.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:43.422+0000 7ff421e38640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ff41c10d6e0 con 0x7ff41c102e00 2024-09-12T02:56:43.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:43.422+0000 7ff421e38640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff41c10d9e0 con 0x7ff41c1037c0 2024-09-12T02:56:43.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:43.422+0000 7ff41b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff41c101c00 0x7ff41c19a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:43.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:43.422+0000 7ff41b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff41c101c00 0x7ff41c19a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49618/0 (socket says 172.21.15.5:49618) 2024-09-12T02:56:43.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:43.422+0000 7ff41b7fe640 1 -- 172.21.15.5:0/3779124371 learned_addr learned my addr 172.21.15.5:0/3779124371 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:56:43.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:43.422+0000 7ff41affd640 1 --2- 172.21.15.5:0/3779124371 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff41c102e00 0x7ff41c19a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:43.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:43.422+0000 7ff41bfff640 1 --2- 172.21.15.5:0/3779124371 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff41c1037c0 0x7ff41c1a1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:43.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:43.423+0000 7ff41b7fe640 1 -- 172.21.15.5:0/3779124371 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff41c1037c0 msgr2=0x7ff41c1a1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:43.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:43.423+0000 7ff41b7fe640 1 --2- 172.21.15.5:0/3779124371 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff41c1037c0 0x7ff41c1a1690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:43.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:43.423+0000 7ff41b7fe640 1 -- 172.21.15.5:0/3779124371 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff41c102e00 msgr2=0x7ff41c19a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:43.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:43.423+0000 7ff41b7fe640 1 --2- 172.21.15.5:0/3779124371 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff41c102e00 0x7ff41c19a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:43.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:43.423+0000 7ff41b7fe640 1 -- 172.21.15.5:0/3779124371 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff41c1a1d00 con 0x7ff41c101c00 2024-09-12T02:56:43.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:43.423+0000 7ff41b7fe640 1 --2- 172.21.15.5:0/3779124371 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff41c101c00 0x7ff41c19a0d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ff40c00c970 tx=0x7ff40c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:43.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:43.423+0000 7ff418ff9640 1 -- 172.21.15.5:0/3779124371 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff40c004220 con 0x7ff41c101c00 2024-09-12T02:56:43.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:43.423+0000 7ff418ff9640 1 -- 172.21.15.5:0/3779124371 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff40c0043c0 con 0x7ff41c101c00 2024-09-12T02:56:43.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:43.423+0000 7ff418ff9640 1 -- 172.21.15.5:0/3779124371 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff40c00f450 con 0x7ff41c101c00 2024-09-12T02:56:43.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:43.423+0000 7ff421e38640 1 -- 172.21.15.5:0/3779124371 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff41c1a1ff0 con 0x7ff41c101c00 2024-09-12T02:56:43.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:43.423+0000 7ff421e38640 1 -- 172.21.15.5:0/3779124371 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff41c1a2440 con 0x7ff41c101c00 2024-09-12T02:56:43.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:43.424+0000 7ff421e38640 1 -- 172.21.15.5:0/3779124371 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff41c1a2700 con 0x7ff41c101c00 2024-09-12T02:56:43.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:43.425+0000 7ff418ff9640 1 -- 172.21.15.5:0/3779124371 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7ff40c0079b0 con 0x7ff41c101c00 2024-09-12T02:56:43.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:43.426+0000 7ff418ff9640 1 --2- 172.21.15.5:0/3779124371 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7ff4040a2c40 0x7ff4040a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:43.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:43.426+0000 7ff41affd640 1 --2- 172.21.15.5:0/3779124371 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7ff4040a2c40 0x7ff4040a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:43.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:43.426+0000 7ff418ff9640 1 -- 172.21.15.5:0/3779124371 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7ff40c0ceff0 con 0x7ff41c101c00 2024-09-12T02:56:43.427 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:43.426+0000 7ff41affd640 1 --2- 172.21.15.5:0/3779124371 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7ff4040a2c40 0x7ff4040a5100 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff410031610 tx=0x7ff410000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:43.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:43.430+0000 7ff418ff9640 1 -- 172.21.15.5:0/3779124371 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff40c014030 con 0x7ff41c101c00 2024-09-12T02:56:43.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:43 smithi146 ceph-mon[35011]: pgmap v289: 97 pgs: 97 active+clean; 635 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 6.0 KiB/s wr, 14 op/s; 69 B/s, 1 objects/s recovering 2024-09-12T02:56:43.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:43 smithi146 ceph-mon[35011]: from='client.14745 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:56:43.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:43.669+0000 7ff421e38640 1 -- 172.21.15.5:0/3779124371 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7ff41c1a2ba0 con 0x7ff41c101c00 2024-09-12T02:56:43.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:43.671+0000 7ff418ff9640 1 -- 172.21.15.5:0/3779124371 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7ff40c09be00 con 0x7ff41c101c00 2024-09-12T02:56:43.672 INFO:teuthology.orchestra.run.smithi005.stdout:[client.0] 2024-09-12T02:56:43.672 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQBrWOJmjej+JxAAXS8ylsaTvouaS3o2/a7TUg== 2024-09-12T02:56:43.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:43.673+0000 7ff421e38640 1 -- 172.21.15.5:0/3779124371 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7ff4040a2c40 msgr2=0x7ff4040a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:43.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:43.673+0000 7ff421e38640 1 --2- 172.21.15.5:0/3779124371 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7ff4040a2c40 0x7ff4040a5100 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff410031610 tx=0x7ff410000bc0 comp rx=0 tx=0).stop 2024-09-12T02:56:43.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:43.673+0000 7ff421e38640 1 -- 172.21.15.5:0/3779124371 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff41c101c00 msgr2=0x7ff41c19a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:43.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:43.673+0000 7ff421e38640 1 --2- 172.21.15.5:0/3779124371 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff41c101c00 0x7ff41c19a0d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ff40c00c970 tx=0x7ff40c00ce30 comp rx=0 tx=0).stop 2024-09-12T02:56:43.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:43.674+0000 7ff421e38640 1 -- 172.21.15.5:0/3779124371 shutdown_connections 2024-09-12T02:56:43.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:43.674+0000 7ff421e38640 1 --2- 172.21.15.5:0/3779124371 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff41c1037c0 0x7ff41c1a1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:43.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:43.674+0000 7ff421e38640 1 --2- 172.21.15.5:0/3779124371 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff41c102e00 0x7ff41c19a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:43.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:43.674+0000 7ff421e38640 1 --2- 172.21.15.5:0/3779124371 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7ff4040a2c40 0x7ff4040a5100 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:43.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:43.674+0000 7ff421e38640 1 --2- 172.21.15.5:0/3779124371 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff41c101c00 0x7ff41c19a0d0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:43.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:43.674+0000 7ff421e38640 1 -- 172.21.15.5:0/3779124371 >> 172.21.15.5:0/3779124371 conn(0x7ff41c0fd390 msgr2=0x7ff41c074420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:43.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:43.674+0000 7ff421e38640 1 -- 172.21.15.5:0/3779124371 shutdown_connections 2024-09-12T02:56:43.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:43.674+0000 7ff421e38640 1 -- 172.21.15.5:0/3779124371 wait complete. 2024-09-12T02:56:44.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:44 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/3779124371' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-12T02:56:44.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:44 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/3779124371' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-12T02:56:44.348 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-09-12T02:56:44.349 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-12T02:56:44.349 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-12T02:56:44.388 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-12T02:56:44.388 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-12T02:56:44.388 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph mgr dump --format=json 2024-09-12T02:56:44.415 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:44 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/3779124371' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-12T02:56:44.415 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:44 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/3779124371' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-12T02:56:44.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:44 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/3779124371' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-12T02:56:44.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:44 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/3779124371' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-12T02:56:44.682 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:56:45.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:45.138+0000 7f0c2a49b640 1 -- 172.21.15.5:0/2865880380 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c24101de0 msgr2=0x7f0c241021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:45.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:45.138+0000 7f0c2a49b640 1 --2- 172.21.15.5:0/2865880380 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c24101de0 0x7f0c241021c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0c14009900 tx=0x7f0c1402a410 comp rx=0 tx=0).stop 2024-09-12T02:56:45.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:45.138+0000 7f0c2a49b640 1 -- 172.21.15.5:0/2865880380 shutdown_connections 2024-09-12T02:56:45.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:45.138+0000 7f0c2a49b640 1 --2- 172.21.15.5:0/2865880380 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0c24106f10 0x7f0c2410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:45.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:45.138+0000 7f0c2a49b640 1 --2- 172.21.15.5:0/2865880380 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c24102790 0x7f0c241067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:45.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:45.138+0000 7f0c2a49b640 1 --2- 172.21.15.5:0/2865880380 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c24101de0 0x7f0c241021c0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:45.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:45.138+0000 7f0c2a49b640 1 -- 172.21.15.5:0/2865880380 >> 172.21.15.5:0/2865880380 conn(0x7f0c240fd8c0 msgr2=0x7f0c240ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:45.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:45.138+0000 7f0c2a49b640 1 -- 172.21.15.5:0/2865880380 shutdown_connections 2024-09-12T02:56:45.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:45.138+0000 7f0c2a49b640 1 -- 172.21.15.5:0/2865880380 wait complete. 2024-09-12T02:56:45.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:45.139+0000 7f0c2a49b640 1 Processor -- start 2024-09-12T02:56:45.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:45.139+0000 7f0c2a49b640 1 -- start start 2024-09-12T02:56:45.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:45.139+0000 7f0c2a49b640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0c24101de0 0x7f0c241985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:45.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:45.139+0000 7f0c2a49b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c24102790 0x7f0c24198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:45.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:45.139+0000 7f0c2a49b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c24106f10 0x7f0c2419ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:45.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:45.139+0000 7f0c2a49b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0c24112010 con 0x7f0c24102790 2024-09-12T02:56:45.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:45.139+0000 7f0c2a49b640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f0c24111e90 con 0x7f0c24101de0 2024-09-12T02:56:45.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:45.139+0000 7f0c2a49b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f0c24112190 con 0x7f0c24106f10 2024-09-12T02:56:45.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:45.139+0000 7f0c237fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c24102790 0x7f0c24198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:45.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:45.139+0000 7f0c237fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c24102790 0x7f0c24198af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41212/0 (socket says 172.21.15.5:41212) 2024-09-12T02:56:45.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:45.139+0000 7f0c237fe640 1 -- 172.21.15.5:0/3367779276 learned_addr learned my addr 172.21.15.5:0/3367779276 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:56:45.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:45.139+0000 7f0c28a38640 1 --2- 172.21.15.5:0/3367779276 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c24106f10 0x7f0c2419ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:45.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:45.140+0000 7f0c23fff640 1 --2- 172.21.15.5:0/3367779276 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0c24101de0 0x7f0c241985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:45.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:45.140+0000 7f0c237fe640 1 -- 172.21.15.5:0/3367779276 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c24106f10 msgr2=0x7f0c2419ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:45.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:45.140+0000 7f0c237fe640 1 --2- 172.21.15.5:0/3367779276 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c24106f10 0x7f0c2419ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:45.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:45.140+0000 7f0c237fe640 1 -- 172.21.15.5:0/3367779276 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0c24101de0 msgr2=0x7f0c241985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:45.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:45.140+0000 7f0c237fe640 1 --2- 172.21.15.5:0/3367779276 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0c24101de0 0x7f0c241985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:45.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:45.140+0000 7f0c237fe640 1 -- 172.21.15.5:0/3367779276 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c2419d560 con 0x7f0c24102790 2024-09-12T02:56:45.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:45.140+0000 7f0c23fff640 1 --2- 172.21.15.5:0/3367779276 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0c24101de0 0x7f0c241985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:56:45.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:45.140+0000 7f0c237fe640 1 --2- 172.21.15.5:0/3367779276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c24102790 0x7f0c24198af0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f0c1800b4a0 tx=0x7f0c1800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:45.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:45.140+0000 7f0c217fa640 1 -- 172.21.15.5:0/3367779276 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c18014070 con 0x7f0c24102790 2024-09-12T02:56:45.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:45.140+0000 7f0c2a49b640 1 -- 172.21.15.5:0/3367779276 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c2419d850 con 0x7f0c24102790 2024-09-12T02:56:45.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:45.140+0000 7f0c217fa640 1 -- 172.21.15.5:0/3367779276 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c18010af0 con 0x7f0c24102790 2024-09-12T02:56:45.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:45.140+0000 7f0c217fa640 1 -- 172.21.15.5:0/3367779276 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c18017d10 con 0x7f0c24102790 2024-09-12T02:56:45.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:45.140+0000 7f0c2a49b640 1 -- 172.21.15.5:0/3367779276 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c241a5130 con 0x7f0c24102790 2024-09-12T02:56:45.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:45.141+0000 7f0c2a49b640 1 -- 172.21.15.5:0/3367779276 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c24102240 con 0x7f0c24102790 2024-09-12T02:56:45.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:45.142+0000 7f0c217fa640 1 -- 172.21.15.5:0/3367779276 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f0c18017070 con 0x7f0c24102790 2024-09-12T02:56:45.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:45.143+0000 7f0c217fa640 1 --2- 172.21.15.5:0/3367779276 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f0c0c0a2c10 0x7f0c0c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:45.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:45.143+0000 7f0c23fff640 1 --2- 172.21.15.5:0/3367779276 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f0c0c0a2c10 0x7f0c0c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:45.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:45.143+0000 7f0c217fa640 1 -- 172.21.15.5:0/3367779276 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f0c180ce600 con 0x7f0c24102790 2024-09-12T02:56:45.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:45.143+0000 7f0c23fff640 1 --2- 172.21.15.5:0/3367779276 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f0c0c0a2c10 0x7f0c0c0a50d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0c14000c00 tx=0x7f0c14032d50 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:45.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:45.147+0000 7f0c217fa640 1 -- 172.21.15.5:0/3367779276 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0c1809b4c0 con 0x7f0c24102790 2024-09-12T02:56:45.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:45 smithi005 ceph-mon[31197]: pgmap v290: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.9 KiB/s wr, 15 op/s; 61 B/s, 1 objects/s recovering 2024-09-12T02:56:45.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:45.368+0000 7f0c2a49b640 1 -- 172.21.15.5:0/3367779276 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f0c24109b60 con 0x7f0c24102790 2024-09-12T02:56:45.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:45.373+0000 7f0c217fa640 1 -- 172.21.15.5:0/3367779276 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+263191 (secure 0 0 0) 0x7f0c1809adc0 con 0x7f0c24102790 2024-09-12T02:56:45.379 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-12T02:56:45.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:45.376+0000 7f0c2a49b640 1 -- 172.21.15.5:0/3367779276 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f0c0c0a2c10 msgr2=0x7f0c0c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:45.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:45.376+0000 7f0c2a49b640 1 --2- 172.21.15.5:0/3367779276 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f0c0c0a2c10 0x7f0c0c0a50d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0c14000c00 tx=0x7f0c14032d50 comp rx=0 tx=0).stop 2024-09-12T02:56:45.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:45.376+0000 7f0c2a49b640 1 -- 172.21.15.5:0/3367779276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c24102790 msgr2=0x7f0c24198af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:45.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:45.376+0000 7f0c2a49b640 1 --2- 172.21.15.5:0/3367779276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c24102790 0x7f0c24198af0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f0c1800b4a0 tx=0x7f0c1800b960 comp rx=0 tx=0).stop 2024-09-12T02:56:45.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:45.376+0000 7f0c2a49b640 1 -- 172.21.15.5:0/3367779276 shutdown_connections 2024-09-12T02:56:45.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:45.376+0000 7f0c2a49b640 1 --2- 172.21.15.5:0/3367779276 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f0c0c0a2c10 0x7f0c0c0a50d0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:45.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:45.376+0000 7f0c2a49b640 1 --2- 172.21.15.5:0/3367779276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c24102790 0x7f0c24198af0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:45.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:45.376+0000 7f0c2a49b640 1 --2- 172.21.15.5:0/3367779276 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0c24106f10 0x7f0c2419ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:45.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:45.376+0000 7f0c2a49b640 1 --2- 172.21.15.5:0/3367779276 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0c24101de0 0x7f0c241985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:45.384 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:45.376+0000 7f0c2a49b640 1 -- 172.21.15.5:0/3367779276 >> 172.21.15.5:0/3367779276 conn(0x7f0c240fd8c0 msgr2=0x7f0c240fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:45.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:45.376+0000 7f0c2a49b640 1 -- 172.21.15.5:0/3367779276 shutdown_connections 2024-09-12T02:56:45.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:45.376+0000 7f0c2a49b640 1 -- 172.21.15.5:0/3367779276 wait complete. 2024-09-12T02:56:45.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:45 smithi148 ceph-mon[35090]: pgmap v290: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.9 KiB/s wr, 15 op/s; 61 B/s, 1 objects/s recovering 2024-09-12T02:56:45.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:45 smithi146 ceph-mon[35011]: pgmap v290: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.9 KiB/s wr, 15 op/s; 61 B/s, 1 objects/s recovering 2024-09-12T02:56:46.106 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":17,"flags":0,"active_gid":14148,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":3780631642},{"type":"v1","addr":"172.21.15.5:6801","nonce":3780631642}]},"active_addr":"172.21.15.5:6801/3780631642","active_change":"2024-09-12T02:49:29.462328+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24115,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24119,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.5:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.5:0","nonce":2840186524}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.5:0","nonce":2937547841}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.5:0","nonce":3087412097}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.5:0","nonce":411874225}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.5:0","nonce":3415439104}]}]} 2024-09-12T02:56:46.112 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-12T02:56:46.112 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-12T02:56:46.112 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T02:56:46.369 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:56:46.415 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:46 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/3367779276' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-12T02:56:46.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:46 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/3367779276' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-12T02:56:46.491 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:46 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/3367779276' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-12T02:56:46.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:46.839+0000 7f0ec1f7d640 1 -- 172.21.15.5:0/2966675346 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ebc101420 msgr2=0x7f0ebc10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:46.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:46.839+0000 7f0ec1f7d640 1 --2- 172.21.15.5:0/2966675346 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ebc101420 0x7f0ebc10d950 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f0eb4009cd0 tx=0x7f0eb402a8e0 comp rx=0 tx=0).stop 2024-09-12T02:56:46.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:46.839+0000 7f0ec1f7d640 1 -- 172.21.15.5:0/2966675346 shutdown_connections 2024-09-12T02:56:46.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:46.839+0000 7f0ec1f7d640 1 --2- 172.21.15.5:0/2966675346 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ebc101420 0x7f0ebc10d950 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:46.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:46.839+0000 7f0ec1f7d640 1 --2- 172.21.15.5:0/2966675346 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0ebc100a80 0x7f0ebc100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:46.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:46.839+0000 7f0ec1f7d640 1 --2- 172.21.15.5:0/2966675346 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0ebc106a80 0x7f0ebc106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:46.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:46.839+0000 7f0ec1f7d640 1 -- 172.21.15.5:0/2966675346 >> 172.21.15.5:0/2966675346 conn(0x7f0ebc0fc7a0 msgr2=0x7f0ebc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:46.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:46.839+0000 7f0ec1f7d640 1 -- 172.21.15.5:0/2966675346 shutdown_connections 2024-09-12T02:56:46.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:46.839+0000 7f0ec1f7d640 1 -- 172.21.15.5:0/2966675346 wait complete. 2024-09-12T02:56:46.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:46.840+0000 7f0ec1f7d640 1 Processor -- start 2024-09-12T02:56:46.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:46.840+0000 7f0ec1f7d640 1 -- start start 2024-09-12T02:56:46.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:46.840+0000 7f0ec1f7d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ebc100a80 0x7f0ebc0fcd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:46.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:46.840+0000 7f0ec1f7d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0ebc101420 0x7f0ebc0fd280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:46.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:46.840+0000 7f0ec1f7d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0ebc106a80 0x7f0ebc0fd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:46.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:46.840+0000 7f0ec1f7d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0ebc110e80 con 0x7f0ebc100a80 2024-09-12T02:56:46.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:46.840+0000 7f0ec1f7d640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f0ebc110d00 con 0x7f0ebc106a80 2024-09-12T02:56:46.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:46.840+0000 7f0ec1f7d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f0ebc111000 con 0x7f0ebc101420 2024-09-12T02:56:46.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:46.840+0000 7f0ebb7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ebc100a80 0x7f0ebc0fcd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:46.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:46.840+0000 7f0ebb7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ebc100a80 0x7f0ebc0fcd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41220/0 (socket says 172.21.15.5:41220) 2024-09-12T02:56:46.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:46.840+0000 7f0ebb7fe640 1 -- 172.21.15.5:0/2472142460 learned_addr learned my addr 172.21.15.5:0/2472142460 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:56:46.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:46.840+0000 7f0ebaffd640 1 --2- 172.21.15.5:0/2472142460 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0ebc101420 0x7f0ebc0fd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:46.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:46.840+0000 7f0ebbfff640 1 --2- 172.21.15.5:0/2472142460 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0ebc106a80 0x7f0ebc0fd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:46.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:46.840+0000 7f0ebb7fe640 1 -- 172.21.15.5:0/2472142460 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0ebc101420 msgr2=0x7f0ebc0fd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:46.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:46.840+0000 7f0ebb7fe640 1 --2- 172.21.15.5:0/2472142460 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0ebc101420 0x7f0ebc0fd280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:46.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:46.841+0000 7f0ebb7fe640 1 -- 172.21.15.5:0/2472142460 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0ebc106a80 msgr2=0x7f0ebc0fd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:46.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:46.841+0000 7f0ebb7fe640 1 --2- 172.21.15.5:0/2472142460 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0ebc106a80 0x7f0ebc0fd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:46.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:46.841+0000 7f0ebb7fe640 1 -- 172.21.15.5:0/2472142460 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ebc1a68d0 con 0x7f0ebc100a80 2024-09-12T02:56:46.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:46.841+0000 7f0ebbfff640 1 --2- 172.21.15.5:0/2472142460 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0ebc106a80 0x7f0ebc0fd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:56:46.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:46.841+0000 7f0ebb7fe640 1 --2- 172.21.15.5:0/2472142460 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ebc100a80 0x7f0ebc0fcd40 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0eac00d950 tx=0x7f0eac00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:46.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:46.841+0000 7f0eb8ff9640 1 -- 172.21.15.5:0/2472142460 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eac017070 con 0x7f0ebc100a80 2024-09-12T02:56:46.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:46.841+0000 7f0eb8ff9640 1 -- 172.21.15.5:0/2472142460 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0eac009c30 con 0x7f0ebc100a80 2024-09-12T02:56:46.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:46.841+0000 7f0ec1f7d640 1 -- 172.21.15.5:0/2472142460 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ebc1a6bc0 con 0x7f0ebc100a80 2024-09-12T02:56:46.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:46.841+0000 7f0eb8ff9640 1 -- 172.21.15.5:0/2472142460 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eac00f6b0 con 0x7f0ebc100a80 2024-09-12T02:56:46.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:46.841+0000 7f0ec1f7d640 1 -- 172.21.15.5:0/2472142460 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ebc1a7100 con 0x7f0ebc100a80 2024-09-12T02:56:46.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:46.842+0000 7f0ec1f7d640 1 -- 172.21.15.5:0/2472142460 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ebc100ee0 con 0x7f0ebc100a80 2024-09-12T02:56:46.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:46.843+0000 7f0eb8ff9640 1 -- 172.21.15.5:0/2472142460 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f0eac0109d0 con 0x7f0ebc100a80 2024-09-12T02:56:46.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:46.844+0000 7f0eb8ff9640 1 --2- 172.21.15.5:0/2472142460 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f0ea40a2c10 0x7f0ea40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:46.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:46.844+0000 7f0ebaffd640 1 --2- 172.21.15.5:0/2472142460 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f0ea40a2c10 0x7f0ea40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:46.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:46.844+0000 7f0eb8ff9640 1 -- 172.21.15.5:0/2472142460 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f0eac0ced60 con 0x7f0ebc100a80 2024-09-12T02:56:46.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:46.844+0000 7f0ebaffd640 1 --2- 172.21.15.5:0/2472142460 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f0ea40a2c10 0x7f0ea40a50d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc0fe7d0 tx=0x7f0eb000a250 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:46.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:46.848+0000 7f0eb8ff9640 1 -- 172.21.15.5:0/2472142460 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0eac009de0 con 0x7f0ebc100a80 2024-09-12T02:56:47.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:47.025+0000 7f0ec1f7d640 1 -- 172.21.15.5:0/2472142460 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0ebc0fe270 con 0x7f0ebc100a80 2024-09-12T02:56:47.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:47.026+0000 7f0eb8ff9640 1 -- 172.21.15.5:0/2472142460 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+22656 (secure 0 0 0) 0x7f0eac09bb70 con 0x7f0ebc100a80 2024-09-12T02:56:47.026 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-12T02:56:47.027 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":83,"fsid":"618305f8-70b1-11ef-bcea-c7b262605968","created":"2024-09-12T02:48:42.352146+0000","modified":"2024-09-12T02:56:31.026079+0000","last_up_change":"2024-09-12T02:55:27.817161+0000","last_in_change":"2024-09-12T02:55:03.745281+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T02:51:47.970639+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T02:56:12.834271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T02:56:14.951437+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17da560a-6aa2-4f2c-a95f-67481dce4d3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":4248507573},{"type":"v1","addr":"172.21.15.5:6803","nonce":4248507573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":4248507573},{"type":"v1","addr":"172.21.15.5:6805","nonce":4248507573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":4248507573},{"type":"v1","addr":"172.21.15.5:6809","nonce":4248507573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":4248507573},{"type":"v1","addr":"172.21.15.5:6807","nonce":4248507573}]},"public_addr":"172.21.15.5:6803/4248507573","cluster_addr":"172.21.15.5:6805/4248507573","heartbeat_back_addr":"172.21.15.5:6809/4248507573","heartbeat_front_addr":"172.21.15.5:6807/4248507573","state":["exists","up"]},{"osd":1,"uuid":"a3237006-99af-4fc1-860c-d4917032647f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":1105990287},{"type":"v1","addr":"172.21.15.146:6801","nonce":1105990287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":1105990287},{"type":"v1","addr":"172.21.15.146:6803","nonce":1105990287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":1105990287},{"type":"v1","addr":"172.21.15.146:6807","nonce":1105990287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":1105990287},{"type":"v1","addr":"172.21.15.146:6805","nonce":1105990287}]},"public_addr":"172.21.15.146:6801/1105990287","cluster_addr":"172.21.15.146:6803/1105990287","heartbeat_back_addr":"172.21.15.146:6807/1105990287","heartbeat_front_addr":"172.21.15.146:6805/1105990287","state":["exists","up"]},{"osd":2,"uuid":"ea3ab84a-7739-4975-8244-31ddd5aad80d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3378174313},{"type":"v1","addr":"172.21.15.148:6801","nonce":3378174313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3378174313},{"type":"v1","addr":"172.21.15.148:6803","nonce":3378174313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3378174313},{"type":"v1","addr":"172.21.15.148:6807","nonce":3378174313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3378174313},{"type":"v1","addr":"172.21.15.148:6805","nonce":3378174313}]},"public_addr":"172.21.15.148:6801/3378174313","cluster_addr":"172.21.15.148:6803/3378174313","heartbeat_back_addr":"172.21.15.148:6807/3378174313","heartbeat_front_addr":"172.21.15.148:6805/3378174313","state":["exists","up"]},{"osd":3,"uuid":"4396a9ee-f3f3-410b-9fb1-124f1e33a1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":11741938},{"type":"v1","addr":"172.21.15.5:6811","nonce":11741938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":11741938},{"type":"v1","addr":"172.21.15.5:6813","nonce":11741938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":11741938},{"type":"v1","addr":"172.21.15.5:6817","nonce":11741938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":11741938},{"type":"v1","addr":"172.21.15.5:6815","nonce":11741938}]},"public_addr":"172.21.15.5:6811/11741938","cluster_addr":"172.21.15.5:6813/11741938","heartbeat_back_addr":"172.21.15.5:6817/11741938","heartbeat_front_addr":"172.21.15.5:6815/11741938","state":["exists","up"]},{"osd":4,"uuid":"989d76e7-c742-49d0-994a-c58f2771b1d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":3274923851},{"type":"v1","addr":"172.21.15.146:6809","nonce":3274923851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":3274923851},{"type":"v1","addr":"172.21.15.146:6811","nonce":3274923851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3274923851},{"type":"v1","addr":"172.21.15.146:6815","nonce":3274923851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":3274923851},{"type":"v1","addr":"172.21.15.146:6813","nonce":3274923851}]},"public_addr":"172.21.15.146:6809/3274923851","cluster_addr":"172.21.15.146:6811/3274923851","heartbeat_back_addr":"172.21.15.146:6815/3274923851","heartbeat_front_addr":"172.21.15.146:6813/3274923851","state":["exists","up"]},{"osd":5,"uuid":"64ada7d4-70a0-4b83-9af8-032be544608d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":2771670476},{"type":"v1","addr":"172.21.15.148:6809","nonce":2771670476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":2771670476},{"type":"v1","addr":"172.21.15.148:6811","nonce":2771670476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":2771670476},{"type":"v1","addr":"172.21.15.148:6815","nonce":2771670476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":2771670476},{"type":"v1","addr":"172.21.15.148:6813","nonce":2771670476}]},"public_addr":"172.21.15.148:6809/2771670476","cluster_addr":"172.21.15.148:6811/2771670476","heartbeat_back_addr":"172.21.15.148:6815/2771670476","heartbeat_front_addr":"172.21.15.148:6813/2771670476","state":["exists","up"]},{"osd":6,"uuid":"a70a88fd-2e4b-480c-bb98-62636cf70cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3176073169},{"type":"v1","addr":"172.21.15.5:6819","nonce":3176073169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":3176073169},{"type":"v1","addr":"172.21.15.5:6821","nonce":3176073169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":3176073169},{"type":"v1","addr":"172.21.15.5:6825","nonce":3176073169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3176073169},{"type":"v1","addr":"172.21.15.5:6823","nonce":3176073169}]},"public_addr":"172.21.15.5:6819/3176073169","cluster_addr":"172.21.15.5:6821/3176073169","heartbeat_back_addr":"172.21.15.5:6825/3176073169","heartbeat_front_addr":"172.21.15.5:6823/3176073169","state":["exists","up"]},{"osd":7,"uuid":"49451e8d-c741-4ad5-b8f5-89bad8d4d129","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2209508838},{"type":"v1","addr":"172.21.15.146:6817","nonce":2209508838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2209508838},{"type":"v1","addr":"172.21.15.146:6819","nonce":2209508838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2209508838},{"type":"v1","addr":"172.21.15.146:6823","nonce":2209508838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2209508838},{"type":"v1","addr":"172.21.15.146:6821","nonce":2209508838}]},"public_addr":"172.21.15.146:6817/2209508838","cluster_addr":"172.21.15.146:6819/2209508838","heartbeat_back_addr":"172.21.15.146:6823/2209508838","heartbeat_front_addr":"172.21.15.146:6821/2209508838","state":["exists","up"]},{"osd":8,"uuid":"57bfc121-a75e-4a7f-b1dc-64074b178b03","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":651045756},{"type":"v1","addr":"172.21.15.148:6817","nonce":651045756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":651045756},{"type":"v1","addr":"172.21.15.148:6819","nonce":651045756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":651045756},{"type":"v1","addr":"172.21.15.148:6823","nonce":651045756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":651045756},{"type":"v1","addr":"172.21.15.148:6821","nonce":651045756}]},"public_addr":"172.21.15.148:6817/651045756","cluster_addr":"172.21.15.148:6819/651045756","heartbeat_back_addr":"172.21.15.148:6823/651045756","heartbeat_front_addr":"172.21.15.148:6821/651045756","state":["exists","up"]},{"osd":9,"uuid":"d9726168-7494-4bf8-b433-350e85bfd461","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1961857360},{"type":"v1","addr":"172.21.15.5:6827","nonce":1961857360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1961857360},{"type":"v1","addr":"172.21.15.5:6829","nonce":1961857360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":1961857360},{"type":"v1","addr":"172.21.15.5:6833","nonce":1961857360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1961857360},{"type":"v1","addr":"172.21.15.5:6831","nonce":1961857360}]},"public_addr":"172.21.15.5:6827/1961857360","cluster_addr":"172.21.15.5:6829/1961857360","heartbeat_back_addr":"172.21.15.5:6833/1961857360","heartbeat_front_addr":"172.21.15.5:6831/1961857360","state":["exists","up"]},{"osd":10,"uuid":"519ec60f-3606-48ac-9043-6f59a798766d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":2041733103},{"type":"v1","addr":"172.21.15.146:6825","nonce":2041733103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":2041733103},{"type":"v1","addr":"172.21.15.146:6827","nonce":2041733103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":2041733103},{"type":"v1","addr":"172.21.15.146:6831","nonce":2041733103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":2041733103},{"type":"v1","addr":"172.21.15.146:6829","nonce":2041733103}]},"public_addr":"172.21.15.146:6825/2041733103","cluster_addr":"172.21.15.146:6827/2041733103","heartbeat_back_addr":"172.21.15.146:6831/2041733103","heartbeat_front_addr":"172.21.15.146:6829/2041733103","state":["exists","up"]},{"osd":11,"uuid":"0f92d0cb-ba54-4bc3-a6d0-3c54d910556a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":839290773},{"type":"v1","addr":"172.21.15.148:6825","nonce":839290773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":839290773},{"type":"v1","addr":"172.21.15.148:6827","nonce":839290773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":839290773},{"type":"v1","addr":"172.21.15.148:6831","nonce":839290773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":839290773},{"type":"v1","addr":"172.21.15.148:6829","nonce":839290773}]},"public_addr":"172.21.15.148:6825/839290773","cluster_addr":"172.21.15.148:6827/839290773","heartbeat_back_addr":"172.21.15.148:6831/839290773","heartbeat_front_addr":"172.21.15.148:6829/839290773","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:50:54.101755+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:51:19.037385+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:51:44.142837+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:52:09.150887+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:52:33.705079+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:52:58.845233+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:53:23.068339+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:53:47.520398+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:54:11.965370+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:54:36.776075+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:55:01.292236+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:55:25.346589+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:6800/3125821798":"2024-09-13T02:49:29.462216+0000","172.21.15.5:0/3479911451":"2024-09-13T02:49:29.462216+0000","172.21.15.5:0/1547753207":"2024-09-13T02:49:03.300872+0000","172.21.15.5:0/1905762627":"2024-09-13T02:49:03.300872+0000","172.21.15.5:0/4142340280":"2024-09-13T02:49:29.462216+0000","172.21.15.5:6801/3125821798":"2024-09-13T02:49:29.462216+0000","172.21.15.5:6801/2595781680":"2024-09-13T02:49:03.300872+0000","172.21.15.5:0/2273691311":"2024-09-13T02:49:29.462216+0000","172.21.15.5:6800/2595781680":"2024-09-13T02:49:03.300872+0000","172.21.15.5:0/1196201217":"2024-09-13T02:49:03.300872+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T02:56:47.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:47.028+0000 7f0ec1f7d640 1 -- 172.21.15.5:0/2472142460 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f0ea40a2c10 msgr2=0x7f0ea40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:47.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:47.028+0000 7f0ec1f7d640 1 --2- 172.21.15.5:0/2472142460 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f0ea40a2c10 0x7f0ea40a50d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc0fe7d0 tx=0x7f0eb000a250 comp rx=0 tx=0).stop 2024-09-12T02:56:47.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:47.028+0000 7f0ec1f7d640 1 -- 172.21.15.5:0/2472142460 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ebc100a80 msgr2=0x7f0ebc0fcd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:47.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:47.028+0000 7f0ec1f7d640 1 --2- 172.21.15.5:0/2472142460 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ebc100a80 0x7f0ebc0fcd40 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0eac00d950 tx=0x7f0eac00de10 comp rx=0 tx=0).stop 2024-09-12T02:56:47.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:47.028+0000 7f0ec1f7d640 1 -- 172.21.15.5:0/2472142460 shutdown_connections 2024-09-12T02:56:47.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:47.028+0000 7f0ec1f7d640 1 --2- 172.21.15.5:0/2472142460 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0ebc106a80 0x7f0ebc0fd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:47.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:47.028+0000 7f0ec1f7d640 1 --2- 172.21.15.5:0/2472142460 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0ebc101420 0x7f0ebc0fd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:47.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:47.028+0000 7f0ec1f7d640 1 --2- 172.21.15.5:0/2472142460 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f0ea40a2c10 0x7f0ea40a50d0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:47.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:47.028+0000 7f0ec1f7d640 1 --2- 172.21.15.5:0/2472142460 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ebc100a80 0x7f0ebc0fcd40 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:47.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:47.028+0000 7f0ec1f7d640 1 -- 172.21.15.5:0/2472142460 >> 172.21.15.5:0/2472142460 conn(0x7f0ebc0fc7a0 msgr2=0x7f0ebc06a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:47.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:47.028+0000 7f0ec1f7d640 1 -- 172.21.15.5:0/2472142460 shutdown_connections 2024-09-12T02:56:47.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:47.028+0000 7f0ec1f7d640 1 -- 172.21.15.5:0/2472142460 wait complete. 2024-09-12T02:56:47.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:47 smithi005 ceph-mon[31197]: pgmap v291: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 3.4 KiB/s wr, 12 op/s; 0 B/s, 0 objects/s recovering 2024-09-12T02:56:47.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:47 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/2472142460' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:56:47.415 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:47 smithi148 ceph-mon[35090]: pgmap v291: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 3.4 KiB/s wr, 12 op/s; 0 B/s, 0 objects/s recovering 2024-09-12T02:56:47.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:47 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/2472142460' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:56:47.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:47 smithi146 ceph-mon[35011]: pgmap v291: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 3.4 KiB/s wr, 12 op/s; 0 B/s, 0 objects/s recovering 2024-09-12T02:56:47.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:47 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/2472142460' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:56:47.605 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-12T02:56:47.605 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T02:56:47.864 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:56:48.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:48.304+0000 7f8b0c947640 1 -- 172.21.15.5:0/2478440015 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b08101420 msgr2=0x7f8b0810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:48.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:48.304+0000 7f8b0c947640 1 --2- 172.21.15.5:0/2478440015 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b08101420 0x7f8b0810d950 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f8b00009cd0 tx=0x7f8b0002a8e0 comp rx=0 tx=0).stop 2024-09-12T02:56:48.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:48.304+0000 7f8b0c947640 1 -- 172.21.15.5:0/2478440015 shutdown_connections 2024-09-12T02:56:48.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:48.304+0000 7f8b0c947640 1 --2- 172.21.15.5:0/2478440015 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b08101420 0x7f8b0810d950 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:48.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:48.304+0000 7f8b0c947640 1 --2- 172.21.15.5:0/2478440015 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b08100a80 0x7f8b08100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:48.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:48.304+0000 7f8b0c947640 1 --2- 172.21.15.5:0/2478440015 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8b08106a80 0x7f8b08106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:48.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:48.304+0000 7f8b0c947640 1 -- 172.21.15.5:0/2478440015 >> 172.21.15.5:0/2478440015 conn(0x7f8b080fc7a0 msgr2=0x7f8b080febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:48.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:48.304+0000 7f8b0c947640 1 -- 172.21.15.5:0/2478440015 shutdown_connections 2024-09-12T02:56:48.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:48.304+0000 7f8b0c947640 1 -- 172.21.15.5:0/2478440015 wait complete. 2024-09-12T02:56:48.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:48.305+0000 7f8b0c947640 1 Processor -- start 2024-09-12T02:56:48.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:48.305+0000 7f8b0c947640 1 -- start start 2024-09-12T02:56:48.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:48.305+0000 7f8b0c947640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b08100a80 0x7f8b0819a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:48.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:48.305+0000 7f8b0c947640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b08101420 0x7f8b0819ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:48.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:48.305+0000 7f8b0c947640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8b08106a80 0x7f8b0819eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:48.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:48.305+0000 7f8b0c947640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8b08110ea0 con 0x7f8b08101420 2024-09-12T02:56:48.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:48.305+0000 7f8b0c947640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f8b08110d20 con 0x7f8b08106a80 2024-09-12T02:56:48.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:48.305+0000 7f8b0c947640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8b08111020 con 0x7f8b08100a80 2024-09-12T02:56:48.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:48.306+0000 7f8b05d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b08101420 0x7f8b0819ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:48.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:48.306+0000 7f8b05d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b08101420 0x7f8b0819ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41238/0 (socket says 172.21.15.5:41238) 2024-09-12T02:56:48.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:48.306+0000 7f8b05d9b640 1 -- 172.21.15.5:0/3835192618 learned_addr learned my addr 172.21.15.5:0/3835192618 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:56:48.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:48.306+0000 7f8b0659c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b08100a80 0x7f8b0819a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:48.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:48.306+0000 7f8b06d9d640 1 --2- 172.21.15.5:0/3835192618 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8b08106a80 0x7f8b0819eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:48.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:48.306+0000 7f8b05d9b640 1 -- 172.21.15.5:0/3835192618 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b08100a80 msgr2=0x7f8b0819a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:48.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:48.306+0000 7f8b05d9b640 1 --2- 172.21.15.5:0/3835192618 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b08100a80 0x7f8b0819a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:48.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:48.306+0000 7f8b05d9b640 1 -- 172.21.15.5:0/3835192618 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8b08106a80 msgr2=0x7f8b0819eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:48.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:48.306+0000 7f8b05d9b640 1 --2- 172.21.15.5:0/3835192618 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8b08106a80 0x7f8b0819eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:48.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:48.306+0000 7f8b05d9b640 1 -- 172.21.15.5:0/3835192618 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b0819f5c0 con 0x7f8b08101420 2024-09-12T02:56:48.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:48.306+0000 7f8b05d9b640 1 --2- 172.21.15.5:0/3835192618 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b08101420 0x7f8b0819ab50 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f8afc00b4a0 tx=0x7f8afc00b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:48.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:48.306+0000 7f8af77fe640 1 -- 172.21.15.5:0/3835192618 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8afc015020 con 0x7f8b08101420 2024-09-12T02:56:48.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:48.306+0000 7f8af77fe640 1 -- 172.21.15.5:0/3835192618 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8afc011d10 con 0x7f8b08101420 2024-09-12T02:56:48.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:48.306+0000 7f8b0c947640 1 -- 172.21.15.5:0/3835192618 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b0819f8b0 con 0x7f8b08101420 2024-09-12T02:56:48.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:48.306+0000 7f8af77fe640 1 -- 172.21.15.5:0/3835192618 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8afc00f2e0 con 0x7f8b08101420 2024-09-12T02:56:48.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:48.306+0000 7f8b0c947640 1 -- 172.21.15.5:0/3835192618 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b081a7190 con 0x7f8b08101420 2024-09-12T02:56:48.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:48.307+0000 7f8b0c947640 1 -- 172.21.15.5:0/3835192618 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b08100ee0 con 0x7f8b08101420 2024-09-12T02:56:48.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:48.309+0000 7f8af77fe640 1 -- 172.21.15.5:0/3835192618 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f8afc018020 con 0x7f8b08101420 2024-09-12T02:56:48.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:48.309+0000 7f8af77fe640 1 --2- 172.21.15.5:0/3835192618 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f8af00a2c10 0x7f8af00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:48.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:48.309+0000 7f8b0659c640 1 --2- 172.21.15.5:0/3835192618 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f8af00a2c10 0x7f8af00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:48.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:48.309+0000 7f8af77fe640 1 -- 172.21.15.5:0/3835192618 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f8afc0ce980 con 0x7f8b08101420 2024-09-12T02:56:48.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:48.310+0000 7f8b0659c640 1 --2- 172.21.15.5:0/3835192618 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f8af00a2c10 0x7f8af00a50d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8af800abe0 tx=0x7f8af800a2d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:48.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:48.313+0000 7f8af77fe640 1 -- 172.21.15.5:0/3835192618 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8afc01f020 con 0x7f8b08101420 2024-09-12T02:56:48.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:48.491+0000 7f8b0c947640 1 -- 172.21.15.5:0/3835192618 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8b081a74c0 con 0x7f8b08101420 2024-09-12T02:56:48.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:48.493+0000 7f8af77fe640 1 -- 172.21.15.5:0/3835192618 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+22656 (secure 0 0 0) 0x7f8afc09b8a0 con 0x7f8b08101420 2024-09-12T02:56:48.493 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-12T02:56:48.493 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":83,"fsid":"618305f8-70b1-11ef-bcea-c7b262605968","created":"2024-09-12T02:48:42.352146+0000","modified":"2024-09-12T02:56:31.026079+0000","last_up_change":"2024-09-12T02:55:27.817161+0000","last_in_change":"2024-09-12T02:55:03.745281+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T02:51:47.970639+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T02:56:12.834271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T02:56:14.951437+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17da560a-6aa2-4f2c-a95f-67481dce4d3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":4248507573},{"type":"v1","addr":"172.21.15.5:6803","nonce":4248507573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":4248507573},{"type":"v1","addr":"172.21.15.5:6805","nonce":4248507573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":4248507573},{"type":"v1","addr":"172.21.15.5:6809","nonce":4248507573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":4248507573},{"type":"v1","addr":"172.21.15.5:6807","nonce":4248507573}]},"public_addr":"172.21.15.5:6803/4248507573","cluster_addr":"172.21.15.5:6805/4248507573","heartbeat_back_addr":"172.21.15.5:6809/4248507573","heartbeat_front_addr":"172.21.15.5:6807/4248507573","state":["exists","up"]},{"osd":1,"uuid":"a3237006-99af-4fc1-860c-d4917032647f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":1105990287},{"type":"v1","addr":"172.21.15.146:6801","nonce":1105990287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":1105990287},{"type":"v1","addr":"172.21.15.146:6803","nonce":1105990287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":1105990287},{"type":"v1","addr":"172.21.15.146:6807","nonce":1105990287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":1105990287},{"type":"v1","addr":"172.21.15.146:6805","nonce":1105990287}]},"public_addr":"172.21.15.146:6801/1105990287","cluster_addr":"172.21.15.146:6803/1105990287","heartbeat_back_addr":"172.21.15.146:6807/1105990287","heartbeat_front_addr":"172.21.15.146:6805/1105990287","state":["exists","up"]},{"osd":2,"uuid":"ea3ab84a-7739-4975-8244-31ddd5aad80d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3378174313},{"type":"v1","addr":"172.21.15.148:6801","nonce":3378174313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3378174313},{"type":"v1","addr":"172.21.15.148:6803","nonce":3378174313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3378174313},{"type":"v1","addr":"172.21.15.148:6807","nonce":3378174313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3378174313},{"type":"v1","addr":"172.21.15.148:6805","nonce":3378174313}]},"public_addr":"172.21.15.148:6801/3378174313","cluster_addr":"172.21.15.148:6803/3378174313","heartbeat_back_addr":"172.21.15.148:6807/3378174313","heartbeat_front_addr":"172.21.15.148:6805/3378174313","state":["exists","up"]},{"osd":3,"uuid":"4396a9ee-f3f3-410b-9fb1-124f1e33a1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":11741938},{"type":"v1","addr":"172.21.15.5:6811","nonce":11741938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":11741938},{"type":"v1","addr":"172.21.15.5:6813","nonce":11741938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":11741938},{"type":"v1","addr":"172.21.15.5:6817","nonce":11741938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":11741938},{"type":"v1","addr":"172.21.15.5:6815","nonce":11741938}]},"public_addr":"172.21.15.5:6811/11741938","cluster_addr":"172.21.15.5:6813/11741938","heartbeat_back_addr":"172.21.15.5:6817/11741938","heartbeat_front_addr":"172.21.15.5:6815/11741938","state":["exists","up"]},{"osd":4,"uuid":"989d76e7-c742-49d0-994a-c58f2771b1d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":3274923851},{"type":"v1","addr":"172.21.15.146:6809","nonce":3274923851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":3274923851},{"type":"v1","addr":"172.21.15.146:6811","nonce":3274923851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3274923851},{"type":"v1","addr":"172.21.15.146:6815","nonce":3274923851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":3274923851},{"type":"v1","addr":"172.21.15.146:6813","nonce":3274923851}]},"public_addr":"172.21.15.146:6809/3274923851","cluster_addr":"172.21.15.146:6811/3274923851","heartbeat_back_addr":"172.21.15.146:6815/3274923851","heartbeat_front_addr":"172.21.15.146:6813/3274923851","state":["exists","up"]},{"osd":5,"uuid":"64ada7d4-70a0-4b83-9af8-032be544608d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":2771670476},{"type":"v1","addr":"172.21.15.148:6809","nonce":2771670476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":2771670476},{"type":"v1","addr":"172.21.15.148:6811","nonce":2771670476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":2771670476},{"type":"v1","addr":"172.21.15.148:6815","nonce":2771670476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":2771670476},{"type":"v1","addr":"172.21.15.148:6813","nonce":2771670476}]},"public_addr":"172.21.15.148:6809/2771670476","cluster_addr":"172.21.15.148:6811/2771670476","heartbeat_back_addr":"172.21.15.148:6815/2771670476","heartbeat_front_addr":"172.21.15.148:6813/2771670476","state":["exists","up"]},{"osd":6,"uuid":"a70a88fd-2e4b-480c-bb98-62636cf70cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3176073169},{"type":"v1","addr":"172.21.15.5:6819","nonce":3176073169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":3176073169},{"type":"v1","addr":"172.21.15.5:6821","nonce":3176073169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":3176073169},{"type":"v1","addr":"172.21.15.5:6825","nonce":3176073169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3176073169},{"type":"v1","addr":"172.21.15.5:6823","nonce":3176073169}]},"public_addr":"172.21.15.5:6819/3176073169","cluster_addr":"172.21.15.5:6821/3176073169","heartbeat_back_addr":"172.21.15.5:6825/3176073169","heartbeat_front_addr":"172.21.15.5:6823/3176073169","state":["exists","up"]},{"osd":7,"uuid":"49451e8d-c741-4ad5-b8f5-89bad8d4d129","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2209508838},{"type":"v1","addr":"172.21.15.146:6817","nonce":2209508838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2209508838},{"type":"v1","addr":"172.21.15.146:6819","nonce":2209508838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2209508838},{"type":"v1","addr":"172.21.15.146:6823","nonce":2209508838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2209508838},{"type":"v1","addr":"172.21.15.146:6821","nonce":2209508838}]},"public_addr":"172.21.15.146:6817/2209508838","cluster_addr":"172.21.15.146:6819/2209508838","heartbeat_back_addr":"172.21.15.146:6823/2209508838","heartbeat_front_addr":"172.21.15.146:6821/2209508838","state":["exists","up"]},{"osd":8,"uuid":"57bfc121-a75e-4a7f-b1dc-64074b178b03","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":651045756},{"type":"v1","addr":"172.21.15.148:6817","nonce":651045756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":651045756},{"type":"v1","addr":"172.21.15.148:6819","nonce":651045756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":651045756},{"type":"v1","addr":"172.21.15.148:6823","nonce":651045756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":651045756},{"type":"v1","addr":"172.21.15.148:6821","nonce":651045756}]},"public_addr":"172.21.15.148:6817/651045756","cluster_addr":"172.21.15.148:6819/651045756","heartbeat_back_addr":"172.21.15.148:6823/651045756","heartbeat_front_addr":"172.21.15.148:6821/651045756","state":["exists","up"]},{"osd":9,"uuid":"d9726168-7494-4bf8-b433-350e85bfd461","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1961857360},{"type":"v1","addr":"172.21.15.5:6827","nonce":1961857360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1961857360},{"type":"v1","addr":"172.21.15.5:6829","nonce":1961857360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":1961857360},{"type":"v1","addr":"172.21.15.5:6833","nonce":1961857360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1961857360},{"type":"v1","addr":"172.21.15.5:6831","nonce":1961857360}]},"public_addr":"172.21.15.5:6827/1961857360","cluster_addr":"172.21.15.5:6829/1961857360","heartbeat_back_addr":"172.21.15.5:6833/1961857360","heartbeat_front_addr":"172.21.15.5:6831/1961857360","state":["exists","up"]},{"osd":10,"uuid":"519ec60f-3606-48ac-9043-6f59a798766d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":2041733103},{"type":"v1","addr":"172.21.15.146:6825","nonce":2041733103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":2041733103},{"type":"v1","addr":"172.21.15.146:6827","nonce":2041733103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":2041733103},{"type":"v1","addr":"172.21.15.146:6831","nonce":2041733103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":2041733103},{"type":"v1","addr":"172.21.15.146:6829","nonce":2041733103}]},"public_addr":"172.21.15.146:6825/2041733103","cluster_addr":"172.21.15.146:6827/2041733103","heartbeat_back_addr":"172.21.15.146:6831/2041733103","heartbeat_front_addr":"172.21.15.146:6829/2041733103","state":["exists","up"]},{"osd":11,"uuid":"0f92d0cb-ba54-4bc3-a6d0-3c54d910556a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":839290773},{"type":"v1","addr":"172.21.15.148:6825","nonce":839290773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":839290773},{"type":"v1","addr":"172.21.15.148:6827","nonce":839290773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":839290773},{"type":"v1","addr":"172.21.15.148:6831","nonce":839290773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":839290773},{"type":"v1","addr":"172.21.15.148:6829","nonce":839290773}]},"public_addr":"172.21.15.148:6825/839290773","cluster_addr":"172.21.15.148:6827/839290773","heartbeat_back_addr":"172.21.15.148:6831/839290773","heartbeat_front_addr":"172.21.15.148:6829/839290773","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:50:54.101755+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:51:19.037385+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:51:44.142837+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:52:09.150887+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:52:33.705079+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:52:58.845233+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:53:23.068339+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:53:47.520398+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:54:11.965370+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:54:36.776075+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:55:01.292236+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:55:25.346589+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:6800/3125821798":"2024-09-13T02:49:29.462216+0000","172.21.15.5:0/3479911451":"2024-09-13T02:49:29.462216+0000","172.21.15.5:0/1547753207":"2024-09-13T02:49:03.300872+0000","172.21.15.5:0/1905762627":"2024-09-13T02:49:03.300872+0000","172.21.15.5:0/4142340280":"2024-09-13T02:49:29.462216+0000","172.21.15.5:6801/3125821798":"2024-09-13T02:49:29.462216+0000","172.21.15.5:6801/2595781680":"2024-09-13T02:49:03.300872+0000","172.21.15.5:0/2273691311":"2024-09-13T02:49:29.462216+0000","172.21.15.5:6800/2595781680":"2024-09-13T02:49:03.300872+0000","172.21.15.5:0/1196201217":"2024-09-13T02:49:03.300872+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T02:56:48.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:48.495+0000 7f8b0c947640 1 -- 172.21.15.5:0/3835192618 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f8af00a2c10 msgr2=0x7f8af00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:48.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:48.495+0000 7f8b0c947640 1 --2- 172.21.15.5:0/3835192618 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f8af00a2c10 0x7f8af00a50d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8af800abe0 tx=0x7f8af800a2d0 comp rx=0 tx=0).stop 2024-09-12T02:56:48.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:48.495+0000 7f8b0c947640 1 -- 172.21.15.5:0/3835192618 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b08101420 msgr2=0x7f8b0819ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:48.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:48.495+0000 7f8b0c947640 1 --2- 172.21.15.5:0/3835192618 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b08101420 0x7f8b0819ab50 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f8afc00b4a0 tx=0x7f8afc00b8a0 comp rx=0 tx=0).stop 2024-09-12T02:56:48.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:48.495+0000 7f8b0c947640 1 -- 172.21.15.5:0/3835192618 shutdown_connections 2024-09-12T02:56:48.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:48.495+0000 7f8b0c947640 1 --2- 172.21.15.5:0/3835192618 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f8af00a2c10 0x7f8af00a50d0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:48.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:48.495+0000 7f8b0c947640 1 --2- 172.21.15.5:0/3835192618 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b08101420 0x7f8b0819ab50 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:48.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:48.495+0000 7f8b0c947640 1 --2- 172.21.15.5:0/3835192618 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8b08106a80 0x7f8b0819eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:48.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:48.495+0000 7f8b0c947640 1 --2- 172.21.15.5:0/3835192618 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b08100a80 0x7f8b0819a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:48.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:48.495+0000 7f8b0c947640 1 -- 172.21.15.5:0/3835192618 >> 172.21.15.5:0/3835192618 conn(0x7f8b080fc7a0 msgr2=0x7f8b0810c670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:48.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:48.495+0000 7f8b0c947640 1 -- 172.21.15.5:0/3835192618 shutdown_connections 2024-09-12T02:56:48.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:48.495+0000 7f8b0c947640 1 -- 172.21.15.5:0/3835192618 wait complete. 2024-09-12T02:56:49.073 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-12T02:56:49.073 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-12T02:56:49.074 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-12T02:56:49.074 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-09-12T02:56:49.075 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-09-12T02:56:49.075 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-09-12T02:56:49.075 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-09-12T02:56:49.076 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-09-12T02:56:49.076 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-09-12T02:56:49.077 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-09-12T02:56:49.077 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-09-12T02:56:49.078 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-09-12T02:56:49.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:49 smithi005 ceph-mon[31197]: pgmap v292: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.1 KiB/s wr, 11 op/s 2024-09-12T02:56:49.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:49 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/3835192618' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:56:49.415 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:49 smithi148 ceph-mon[35090]: pgmap v292: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.1 KiB/s wr, 11 op/s 2024-09-12T02:56:49.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:49 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/3835192618' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:56:49.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:49 smithi146 ceph-mon[35011]: pgmap v292: 97 pgs: 97 active+clean; 642 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.1 KiB/s wr, 11 op/s 2024-09-12T02:56:49.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:49 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/3835192618' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:56:49.759 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:56:49.768 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:56:49.769 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:56:49.769 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:56:50.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:50.510+0000 7fe690cda640 1 -- 172.21.15.5:0/880494504 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe68c103740 msgr2=0x7fe68c107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:50.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:50.510+0000 7fe690cda640 1 --2- 172.21.15.5:0/880494504 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe68c103740 0x7fe68c107b80 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe674009930 tx=0x7fe67402a420 comp rx=0 tx=0).stop 2024-09-12T02:56:50.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:50.510+0000 7fe690cda640 1 -- 172.21.15.5:0/880494504 shutdown_connections 2024-09-12T02:56:50.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:50.510+0000 7fe690cda640 1 --2- 172.21.15.5:0/880494504 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe68c103740 0x7fe68c107b80 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:50.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:50.510+0000 7fe690cda640 1 --2- 172.21.15.5:0/880494504 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe68c108290 0x7fe68c10ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:50.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:50.510+0000 7fe690cda640 1 --2- 172.21.15.5:0/880494504 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe68c102d90 0x7fe68c103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:50.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:50.510+0000 7fe690cda640 1 -- 172.21.15.5:0/880494504 >> 172.21.15.5:0/880494504 conn(0x7fe68c0fc7d0 msgr2=0x7fe68c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:50.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:50.510+0000 7fe690cda640 1 -- 172.21.15.5:0/880494504 shutdown_connections 2024-09-12T02:56:50.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:50.510+0000 7fe690cda640 1 -- 172.21.15.5:0/880494504 wait complete. 2024-09-12T02:56:50.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:50.511+0000 7fe690cda640 1 Processor -- start 2024-09-12T02:56:50.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:50.511+0000 7fe690cda640 1 -- start start 2024-09-12T02:56:50.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:50.511+0000 7fe690cda640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe68c102d90 0x7fe68c1a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:50.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:50.511+0000 7fe690cda640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe68c103740 0x7fe68c1a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:50.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:50.511+0000 7fe690cda640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe68c108290 0x7fe68c19a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:50.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:50.511+0000 7fe690cda640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe68c112130 con 0x7fe68c108290 2024-09-12T02:56:50.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:50.511+0000 7fe690cda640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fe68c111fb0 con 0x7fe68c102d90 2024-09-12T02:56:50.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:50.511+0000 7fe690cda640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe68c1122b0 con 0x7fe68c103740 2024-09-12T02:56:50.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:50.512+0000 7fe68ad9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe68c108290 0x7fe68c19a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:50.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:50.512+0000 7fe68ad9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe68c108290 0x7fe68c19a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41270/0 (socket says 172.21.15.5:41270) 2024-09-12T02:56:50.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:50.512+0000 7fe68ad9d640 1 -- 172.21.15.5:0/207530078 learned_addr learned my addr 172.21.15.5:0/207530078 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:56:50.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:50.512+0000 7fe683fff640 1 --2- 172.21.15.5:0/207530078 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe68c103740 0x7fe68c1a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:50.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:50.512+0000 7fe68a59c640 1 --2- 172.21.15.5:0/207530078 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe68c102d90 0x7fe68c1a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:50.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:50.512+0000 7fe68ad9d640 1 -- 172.21.15.5:0/207530078 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe68c103740 msgr2=0x7fe68c1a0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:50.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:50.512+0000 7fe68ad9d640 1 --2- 172.21.15.5:0/207530078 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe68c103740 0x7fe68c1a0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:50.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:50.512+0000 7fe68ad9d640 1 -- 172.21.15.5:0/207530078 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe68c102d90 msgr2=0x7fe68c1a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:50.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:50.512+0000 7fe68ad9d640 1 --2- 172.21.15.5:0/207530078 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe68c102d90 0x7fe68c1a0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:50.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:50.512+0000 7fe68ad9d640 1 -- 172.21.15.5:0/207530078 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe68c19af40 con 0x7fe68c108290 2024-09-12T02:56:50.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:50.512+0000 7fe68ad9d640 1 --2- 172.21.15.5:0/207530078 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe68c108290 0x7fe68c19a7a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe68400b970 tx=0x7fe68400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:50.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:50.512+0000 7fe6837fe640 1 -- 172.21.15.5:0/207530078 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe684019070 con 0x7fe68c108290 2024-09-12T02:56:50.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:50.512+0000 7fe6837fe640 1 -- 172.21.15.5:0/207530078 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe68400cb40 con 0x7fe68c108290 2024-09-12T02:56:50.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:50.512+0000 7fe690cda640 1 -- 172.21.15.5:0/207530078 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe68c19b230 con 0x7fe68c108290 2024-09-12T02:56:50.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:50.512+0000 7fe6837fe640 1 -- 172.21.15.5:0/207530078 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe684007650 con 0x7fe68c108290 2024-09-12T02:56:50.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:50.512+0000 7fe690cda640 1 -- 172.21.15.5:0/207530078 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe68c1a7480 con 0x7fe68c108290 2024-09-12T02:56:50.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:50.513+0000 7fe690cda640 1 -- 172.21.15.5:0/207530078 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe68c10e990 con 0x7fe68c108290 2024-09-12T02:56:50.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:50.515+0000 7fe6837fe640 1 -- 172.21.15.5:0/207530078 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fe68401c070 con 0x7fe68c108290 2024-09-12T02:56:50.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:50.515+0000 7fe6837fe640 1 --2- 172.21.15.5:0/207530078 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fe6780a2c10 0x7fe6780a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:50.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:50.515+0000 7fe68a59c640 1 --2- 172.21.15.5:0/207530078 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fe6780a2c10 0x7fe6780a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:50.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:50.515+0000 7fe6837fe640 1 -- 172.21.15.5:0/207530078 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fe6840ce610 con 0x7fe68c108290 2024-09-12T02:56:50.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:50.515+0000 7fe6837fe640 1 --2- 172.21.15.5:0/207530078 >> [v2:172.21.15.148:6808/2771670476,v1:172.21.15.148:6809/2771670476] conn(0x7fe6780ac7c0 0x7fe6780aec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:50.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:50.515+0000 7fe6837fe640 1 -- 172.21.15.5:0/207530078 --> [v2:172.21.15.148:6808/2771670476,v1:172.21.15.148:6809/2771670476] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe6780af330 con 0x7fe6780ac7c0 2024-09-12T02:56:50.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:50.516+0000 7fe6837fe640 1 -- 172.21.15.5:0/207530078 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7fe6840d5050 con 0x7fe68c108290 2024-09-12T02:56:50.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:50.516+0000 7fe68a59c640 1 --2- 172.21.15.5:0/207530078 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fe6780a2c10 0x7fe6780a50d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe67c009890 tx=0x7fe67c008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:50.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:50.516+0000 7fe683fff640 1 --2- 172.21.15.5:0/207530078 >> [v2:172.21.15.148:6808/2771670476,v1:172.21.15.148:6809/2771670476] conn(0x7fe6780ac7c0 0x7fe6780aec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:50.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:50.516+0000 7fe683fff640 1 --2- 172.21.15.5:0/207530078 >> [v2:172.21.15.148:6808/2771670476,v1:172.21.15.148:6809/2771670476] conn(0x7fe6780ac7c0 0x7fe6780aec60 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:50.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:50.517+0000 7fe6837fe640 1 -- 172.21.15.5:0/207530078 <== osd.5 v2:172.21.15.148:6808/2771670476 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe6780af330 con 0x7fe6780ac7c0 2024-09-12T02:56:50.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:50.531+0000 7fe690cda640 1 -- 172.21.15.5:0/207530078 --> [v2:172.21.15.148:6808/2771670476,v1:172.21.15.148:6809/2771670476] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe68c06a7f0 con 0x7fe6780ac7c0 2024-09-12T02:56:50.531 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:50.531+0000 7fe6837fe640 1 -- 172.21.15.5:0/207530078 <== osd.5 v2:172.21.15.148:6808/2771670476 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fe68c06a7f0 con 0x7fe6780ac7c0 2024-09-12T02:56:50.532 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:50.532+0000 7fe690cda640 1 -- 172.21.15.5:0/207530078 >> [v2:172.21.15.148:6808/2771670476,v1:172.21.15.148:6809/2771670476] conn(0x7fe6780ac7c0 msgr2=0x7fe6780aec60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:50.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:50.532+0000 7fe690cda640 1 --2- 172.21.15.5:0/207530078 >> [v2:172.21.15.148:6808/2771670476,v1:172.21.15.148:6809/2771670476] conn(0x7fe6780ac7c0 0x7fe6780aec60 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:50.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:50.532+0000 7fe690cda640 1 -- 172.21.15.5:0/207530078 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fe6780a2c10 msgr2=0x7fe6780a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:50.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:50.532+0000 7fe690cda640 1 --2- 172.21.15.5:0/207530078 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fe6780a2c10 0x7fe6780a50d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe67c009890 tx=0x7fe67c008040 comp rx=0 tx=0).stop 2024-09-12T02:56:50.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:50.532+0000 7fe690cda640 1 -- 172.21.15.5:0/207530078 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe68c108290 msgr2=0x7fe68c19a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:50.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:50.532+0000 7fe690cda640 1 --2- 172.21.15.5:0/207530078 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe68c108290 0x7fe68c19a7a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe68400b970 tx=0x7fe68400be30 comp rx=0 tx=0).stop 2024-09-12T02:56:50.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:50.532+0000 7fe68ad9d640 1 -- 172.21.15.5:0/207530078 reap_dead start 2024-09-12T02:56:50.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:50.532+0000 7fe690cda640 1 -- 172.21.15.5:0/207530078 shutdown_connections 2024-09-12T02:56:50.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:50.532+0000 7fe690cda640 1 -- 172.21.15.5:0/207530078 >> 172.21.15.5:0/207530078 conn(0x7fe68c0fc7d0 msgr2=0x7fe68c10cfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:50.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:50.532+0000 7fe690cda640 1 -- 172.21.15.5:0/207530078 shutdown_connections 2024-09-12T02:56:50.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:50.532+0000 7fe690cda640 1 -- 172.21.15.5:0/207530078 wait complete. 2024-09-12T02:56:50.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:50.922+0000 7fd486586640 1 -- 172.21.15.5:0/1670573295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4801036e0 msgr2=0x7fd480107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:50.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:50.922+0000 7fd486586640 1 --2- 172.21.15.5:0/1670573295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4801036e0 0x7fd480107b90 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd474009930 tx=0x7fd47402a420 comp rx=0 tx=0).stop 2024-09-12T02:56:50.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:50.922+0000 7fd486586640 1 -- 172.21.15.5:0/1670573295 shutdown_connections 2024-09-12T02:56:50.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:50.922+0000 7fd486586640 1 --2- 172.21.15.5:0/1670573295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4801036e0 0x7fd480107b90 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:50.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:50.922+0000 7fd486586640 1 --2- 172.21.15.5:0/1670573295 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd4801082a0 0x7fd48010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:50.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:50.922+0000 7fd486586640 1 --2- 172.21.15.5:0/1670573295 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd480102d30 0x7fd480103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:50.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:50.922+0000 7fd486586640 1 -- 172.21.15.5:0/1670573295 >> 172.21.15.5:0/1670573295 conn(0x7fd4800fc7d0 msgr2=0x7fd4800febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:50.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:50.922+0000 7fd486586640 1 -- 172.21.15.5:0/1670573295 shutdown_connections 2024-09-12T02:56:50.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:50.922+0000 7fd486586640 1 -- 172.21.15.5:0/1670573295 wait complete. 2024-09-12T02:56:50.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:50.922+0000 7fd486586640 1 Processor -- start 2024-09-12T02:56:50.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:50.922+0000 7fd486586640 1 -- start start 2024-09-12T02:56:50.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:50.923+0000 7fd486586640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd480102d30 0x7fd480198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:50.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:50.923+0000 7fd486586640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd4801036e0 0x7fd480198b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:50.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:50.923+0000 7fd486586640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd4801082a0 0x7fd48019cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:50.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:50.923+0000 7fd486586640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd4801120a0 con 0x7fd480102d30 2024-09-12T02:56:50.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:50.923+0000 7fd486586640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fd480111f20 con 0x7fd4801082a0 2024-09-12T02:56:50.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:50.923+0000 7fd486586640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd480112220 con 0x7fd4801036e0 2024-09-12T02:56:50.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:50.923+0000 7fd47ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd480102d30 0x7fd480198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:50.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:50.923+0000 7fd47ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd480102d30 0x7fd480198620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41292/0 (socket says 172.21.15.5:41292) 2024-09-12T02:56:50.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:50.923+0000 7fd47ffff640 1 -- 172.21.15.5:0/883163820 learned_addr learned my addr 172.21.15.5:0/883163820 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:56:50.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:50.923+0000 7fd47f7fe640 1 --2- 172.21.15.5:0/883163820 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd4801036e0 0x7fd480198b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:50.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:50.923+0000 7fd484b23640 1 --2- 172.21.15.5:0/883163820 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd4801082a0 0x7fd48019cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:50.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:50.923+0000 7fd47ffff640 1 -- 172.21.15.5:0/883163820 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd4801036e0 msgr2=0x7fd480198b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:50.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:50.923+0000 7fd47ffff640 1 --2- 172.21.15.5:0/883163820 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd4801036e0 0x7fd480198b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:50.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:50.923+0000 7fd47ffff640 1 -- 172.21.15.5:0/883163820 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd4801082a0 msgr2=0x7fd48019cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:50.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:50.923+0000 7fd47ffff640 1 --2- 172.21.15.5:0/883163820 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd4801082a0 0x7fd48019cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:50.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:50.923+0000 7fd47ffff640 1 -- 172.21.15.5:0/883163820 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd48019d5d0 con 0x7fd480102d30 2024-09-12T02:56:50.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:50.923+0000 7fd484b23640 1 --2- 172.21.15.5:0/883163820 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd4801082a0 0x7fd48019cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:56:50.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:50.923+0000 7fd47ffff640 1 --2- 172.21.15.5:0/883163820 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd480102d30 0x7fd480198620 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd47000d770 tx=0x7fd47000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:50.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:50.923+0000 7fd47d7fa640 1 -- 172.21.15.5:0/883163820 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd470017070 con 0x7fd480102d30 2024-09-12T02:56:50.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:50.924+0000 7fd486586640 1 -- 172.21.15.5:0/883163820 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd48019d8c0 con 0x7fd480102d30 2024-09-12T02:56:50.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:50.924+0000 7fd486586640 1 -- 172.21.15.5:0/883163820 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4801a51a0 con 0x7fd480102d30 2024-09-12T02:56:50.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:50.924+0000 7fd47d7fa640 1 -- 172.21.15.5:0/883163820 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd470005500 con 0x7fd480102d30 2024-09-12T02:56:50.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:50.924+0000 7fd47d7fa640 1 -- 172.21.15.5:0/883163820 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd470020060 con 0x7fd480102d30 2024-09-12T02:56:50.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:50.925+0000 7fd486586640 1 -- 172.21.15.5:0/883163820 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd46c001030 con 0x7fd480102d30 2024-09-12T02:56:50.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:50.926+0000 7fd47d7fa640 1 -- 172.21.15.5:0/883163820 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fd470009c30 con 0x7fd480102d30 2024-09-12T02:56:50.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:50.927+0000 7fd47d7fa640 1 --2- 172.21.15.5:0/883163820 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fd4680a2c10 0x7fd4680a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:50.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:50.927+0000 7fd47f7fe640 1 --2- 172.21.15.5:0/883163820 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fd4680a2c10 0x7fd4680a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:50.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:50.927+0000 7fd47d7fa640 1 -- 172.21.15.5:0/883163820 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fd4700cf070 con 0x7fd480102d30 2024-09-12T02:56:50.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:50.927+0000 7fd47d7fa640 1 --2- 172.21.15.5:0/883163820 >> [v2:172.21.15.5:6802/4248507573,v1:172.21.15.5:6803/4248507573] conn(0x7fd4680ac7c0 0x7fd4680aec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:50.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:50.927+0000 7fd47d7fa640 1 -- 172.21.15.5:0/883163820 --> [v2:172.21.15.5:6802/4248507573,v1:172.21.15.5:6803/4248507573] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd470022e50 con 0x7fd4680ac7c0 2024-09-12T02:56:50.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:50.927+0000 7fd47d7fa640 1 -- 172.21.15.5:0/883163820 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7fd4700cf420 con 0x7fd480102d30 2024-09-12T02:56:50.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:50.927+0000 7fd484b23640 1 --2- 172.21.15.5:0/883163820 >> [v2:172.21.15.5:6802/4248507573,v1:172.21.15.5:6803/4248507573] conn(0x7fd4680ac7c0 0x7fd4680aec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:50.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:50.927+0000 7fd47f7fe640 1 --2- 172.21.15.5:0/883163820 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fd4680a2c10 0x7fd4680a50d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd474009230 tx=0x7fd474000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:50.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:50.927+0000 7fd484b23640 1 --2- 172.21.15.5:0/883163820 >> [v2:172.21.15.5:6802/4248507573,v1:172.21.15.5:6803/4248507573] conn(0x7fd4680ac7c0 0x7fd4680aec60 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:50.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:50.928+0000 7fd47d7fa640 1 -- 172.21.15.5:0/883163820 <== osd.0 v2:172.21.15.5:6802/4248507573 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd470022e50 con 0x7fd4680ac7c0 2024-09-12T02:56:50.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:50.944+0000 7fd486586640 1 -- 172.21.15.5:0/883163820 --> [v2:172.21.15.5:6802/4248507573,v1:172.21.15.5:6803/4248507573] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd46c002dd0 con 0x7fd4680ac7c0 2024-09-12T02:56:50.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:50.944+0000 7fd47d7fa640 1 -- 172.21.15.5:0/883163820 <== osd.0 v2:172.21.15.5:6802/4248507573 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fd46c002dd0 con 0x7fd4680ac7c0 2024-09-12T02:56:50.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:50.945+0000 7fd486586640 1 -- 172.21.15.5:0/883163820 >> [v2:172.21.15.5:6802/4248507573,v1:172.21.15.5:6803/4248507573] conn(0x7fd4680ac7c0 msgr2=0x7fd4680aec60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:50.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:50.945+0000 7fd486586640 1 --2- 172.21.15.5:0/883163820 >> [v2:172.21.15.5:6802/4248507573,v1:172.21.15.5:6803/4248507573] conn(0x7fd4680ac7c0 0x7fd4680aec60 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:50.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:50.945+0000 7fd486586640 1 -- 172.21.15.5:0/883163820 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fd4680a2c10 msgr2=0x7fd4680a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:50.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:50.945+0000 7fd486586640 1 --2- 172.21.15.5:0/883163820 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fd4680a2c10 0x7fd4680a50d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd474009230 tx=0x7fd474000bc0 comp rx=0 tx=0).stop 2024-09-12T02:56:50.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:50.945+0000 7fd486586640 1 -- 172.21.15.5:0/883163820 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd480102d30 msgr2=0x7fd480198620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:50.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:50.945+0000 7fd486586640 1 --2- 172.21.15.5:0/883163820 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd480102d30 0x7fd480198620 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd47000d770 tx=0x7fd47000dc30 comp rx=0 tx=0).stop 2024-09-12T02:56:50.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:50.945+0000 7fd484b23640 1 -- 172.21.15.5:0/883163820 reap_dead start 2024-09-12T02:56:50.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:50.945+0000 7fd486586640 1 -- 172.21.15.5:0/883163820 shutdown_connections 2024-09-12T02:56:50.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:50.945+0000 7fd486586640 1 -- 172.21.15.5:0/883163820 >> 172.21.15.5:0/883163820 conn(0x7fd4800fc7d0 msgr2=0x7fd480105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:50.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:50.945+0000 7fd486586640 1 -- 172.21.15.5:0/883163820 shutdown_connections 2024-09-12T02:56:50.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:50.945+0000 7fd486586640 1 -- 172.21.15.5:0/883163820 wait complete. 2024-09-12T02:56:50.954 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:56:50.954 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:56:50.954 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:56:50.955 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:56:50.955 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:56:50.956 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:56:51.235 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:51 smithi005 ceph-mon[31197]: pgmap v293: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 3.3 KiB/s wr, 13 op/s 2024-09-12T02:56:51.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:51.234+0000 7fe2d437b640 1 -- 172.21.15.5:0/1534287070 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2cc100e20 msgr2=0x7fe2cc101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:51.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:51.234+0000 7fe2d437b640 1 --2- 172.21.15.5:0/1534287070 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2cc100e20 0x7fe2cc101200 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fe2bc009900 tx=0x7fe2bc02a480 comp rx=0 tx=0).stop 2024-09-12T02:56:51.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:51.235+0000 7fe2d437b640 1 -- 172.21.15.5:0/1534287070 shutdown_connections 2024-09-12T02:56:51.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:51.235+0000 7fe2d437b640 1 --2- 172.21.15.5:0/1534287070 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe2cc102ef0 0x7fe2cc10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:51.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:51.235+0000 7fe2d437b640 1 --2- 172.21.15.5:0/1534287070 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe2cc102570 0x7fe2cc1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:51.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:51.235+0000 7fe2d437b640 1 --2- 172.21.15.5:0/1534287070 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2cc100e20 0x7fe2cc101200 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:51.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:51.235+0000 7fe2d437b640 1 -- 172.21.15.5:0/1534287070 >> 172.21.15.5:0/1534287070 conn(0x7fe2cc0fc7d0 msgr2=0x7fe2cc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:51.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:51.235+0000 7fe2d437b640 1 -- 172.21.15.5:0/1534287070 shutdown_connections 2024-09-12T02:56:51.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:51.235+0000 7fe2d437b640 1 -- 172.21.15.5:0/1534287070 wait complete. 2024-09-12T02:56:51.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:51.235+0000 7fe2d437b640 1 Processor -- start 2024-09-12T02:56:51.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:51.235+0000 7fe2d437b640 1 -- start start 2024-09-12T02:56:51.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:51.236+0000 7fe2d437b640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe2cc100e20 0x7fe2cc1984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:51.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:51.236+0000 7fe2d437b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2cc102570 0x7fe2cc1989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:51.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:51.236+0000 7fe2d437b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe2cc102ef0 0x7fe2cc19cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:51.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:51.236+0000 7fe2d437b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe2cc110f80 con 0x7fe2cc102570 2024-09-12T02:56:51.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:51.236+0000 7fe2d437b640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fe2cc110e00 con 0x7fe2cc100e20 2024-09-12T02:56:51.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:51.236+0000 7fe2d437b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe2cc111100 con 0x7fe2cc102ef0 2024-09-12T02:56:51.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:51.236+0000 7fe2d1916640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2cc102570 0x7fe2cc1989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:51.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:51.236+0000 7fe2d1916640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2cc102570 0x7fe2cc1989f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41296/0 (socket says 172.21.15.5:41296) 2024-09-12T02:56:51.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:51.236+0000 7fe2d1916640 1 -- 172.21.15.5:0/3340120808 learned_addr learned my addr 172.21.15.5:0/3340120808 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:56:51.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:51.236+0000 7fe2d2117640 1 --2- 172.21.15.5:0/3340120808 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe2cc100e20 0x7fe2cc1984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:51.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:51.236+0000 7fe2d2918640 1 --2- 172.21.15.5:0/3340120808 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe2cc102ef0 0x7fe2cc19cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:51.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:51.236+0000 7fe2d1916640 1 -- 172.21.15.5:0/3340120808 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe2cc102ef0 msgr2=0x7fe2cc19cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:51.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:51.236+0000 7fe2d1916640 1 --2- 172.21.15.5:0/3340120808 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe2cc102ef0 0x7fe2cc19cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:51.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:51.236+0000 7fe2d1916640 1 -- 172.21.15.5:0/3340120808 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe2cc100e20 msgr2=0x7fe2cc1984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:51.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:51.236+0000 7fe2d1916640 1 --2- 172.21.15.5:0/3340120808 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe2cc100e20 0x7fe2cc1984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:51.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:51.236+0000 7fe2d1916640 1 -- 172.21.15.5:0/3340120808 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2cc19d460 con 0x7fe2cc102570 2024-09-12T02:56:51.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:51.236+0000 7fe2d1916640 1 --2- 172.21.15.5:0/3340120808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2cc102570 0x7fe2cc1989f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe2c000cce0 tx=0x7fe2c000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:51.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:51.236+0000 7fe2cb7fe640 1 -- 172.21.15.5:0/3340120808 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2c0004b20 con 0x7fe2cc102570 2024-09-12T02:56:51.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:51.236+0000 7fe2cb7fe640 1 -- 172.21.15.5:0/3340120808 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe2c0004cc0 con 0x7fe2cc102570 2024-09-12T02:56:51.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:51.236+0000 7fe2d437b640 1 -- 172.21.15.5:0/3340120808 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2cc19d750 con 0x7fe2cc102570 2024-09-12T02:56:51.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:51.236+0000 7fe2cb7fe640 1 -- 172.21.15.5:0/3340120808 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2c000f7d0 con 0x7fe2cc102570 2024-09-12T02:56:51.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:51.237+0000 7fe2d437b640 1 -- 172.21.15.5:0/3340120808 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2cc1a5030 con 0x7fe2cc102570 2024-09-12T02:56:51.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:51.237+0000 7fe2d437b640 1 -- 172.21.15.5:0/3340120808 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe2cc106bf0 con 0x7fe2cc102570 2024-09-12T02:56:51.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:51.239+0000 7fe2cb7fe640 1 -- 172.21.15.5:0/3340120808 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fe2c0004e70 con 0x7fe2cc102570 2024-09-12T02:56:51.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:51.239+0000 7fe2cb7fe640 1 --2- 172.21.15.5:0/3340120808 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fe2b40a2c90 0x7fe2b40a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:51.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:51.239+0000 7fe2d2117640 1 --2- 172.21.15.5:0/3340120808 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fe2b40a2c90 0x7fe2b40a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:51.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:51.239+0000 7fe2cb7fe640 1 -- 172.21.15.5:0/3340120808 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fe2c00cf5b0 con 0x7fe2cc102570 2024-09-12T02:56:51.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:51.240+0000 7fe2cb7fe640 1 --2- 172.21.15.5:0/3340120808 >> [v2:172.21.15.146:6808/3274923851,v1:172.21.15.146:6809/3274923851] conn(0x7fe2b40ac830 0x7fe2b40aecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:51.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:51.240+0000 7fe2cb7fe640 1 -- 172.21.15.5:0/3340120808 --> [v2:172.21.15.146:6808/3274923851,v1:172.21.15.146:6809/3274923851] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe2b40af3a0 con 0x7fe2b40ac830 2024-09-12T02:56:51.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:51.240+0000 7fe2cb7fe640 1 -- 172.21.15.5:0/3340120808 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7fe2c00cfa00 con 0x7fe2cc102570 2024-09-12T02:56:51.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:51.240+0000 7fe2d2117640 1 --2- 172.21.15.5:0/3340120808 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fe2b40a2c90 0x7fe2b40a5150 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe2bc006010 tx=0x7fe2bc0058d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:51.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:51.240+0000 7fe2d2918640 1 --2- 172.21.15.5:0/3340120808 >> [v2:172.21.15.146:6808/3274923851,v1:172.21.15.146:6809/3274923851] conn(0x7fe2b40ac830 0x7fe2b40aecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:51.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:51.240+0000 7fe2d2918640 1 --2- 172.21.15.5:0/3340120808 >> [v2:172.21.15.146:6808/3274923851,v1:172.21.15.146:6809/3274923851] conn(0x7fe2b40ac830 0x7fe2b40aecd0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:51.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:51.241+0000 7fe2cb7fe640 1 -- 172.21.15.5:0/3340120808 <== osd.4 v2:172.21.15.146:6808/3274923851 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe2b40af3a0 con 0x7fe2b40ac830 2024-09-12T02:56:51.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:51.255+0000 7fe2d437b640 1 -- 172.21.15.5:0/3340120808 --> [v2:172.21.15.146:6808/3274923851,v1:172.21.15.146:6809/3274923851] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe2cc1021b0 con 0x7fe2b40ac830 2024-09-12T02:56:51.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:51.255+0000 7fe2cb7fe640 1 -- 172.21.15.5:0/3340120808 <== osd.4 v2:172.21.15.146:6808/3274923851 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fe2cc1021b0 con 0x7fe2b40ac830 2024-09-12T02:56:51.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:51.256+0000 7fe2d437b640 1 -- 172.21.15.5:0/3340120808 >> [v2:172.21.15.146:6808/3274923851,v1:172.21.15.146:6809/3274923851] conn(0x7fe2b40ac830 msgr2=0x7fe2b40aecd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:51.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:51.256+0000 7fe2d437b640 1 --2- 172.21.15.5:0/3340120808 >> [v2:172.21.15.146:6808/3274923851,v1:172.21.15.146:6809/3274923851] conn(0x7fe2b40ac830 0x7fe2b40aecd0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:51.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:51.256+0000 7fe2d437b640 1 -- 172.21.15.5:0/3340120808 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fe2b40a2c90 msgr2=0x7fe2b40a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:51.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:51.256+0000 7fe2d437b640 1 --2- 172.21.15.5:0/3340120808 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fe2b40a2c90 0x7fe2b40a5150 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe2bc006010 tx=0x7fe2bc0058d0 comp rx=0 tx=0).stop 2024-09-12T02:56:51.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:51.256+0000 7fe2d437b640 1 -- 172.21.15.5:0/3340120808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2cc102570 msgr2=0x7fe2cc1989f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:51.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:51.256+0000 7fe2d437b640 1 --2- 172.21.15.5:0/3340120808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2cc102570 0x7fe2cc1989f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe2c000cce0 tx=0x7fe2c000a620 comp rx=0 tx=0).stop 2024-09-12T02:56:51.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:51.256+0000 7fe2d2918640 1 -- 172.21.15.5:0/3340120808 reap_dead start 2024-09-12T02:56:51.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:51.256+0000 7fe2d437b640 1 -- 172.21.15.5:0/3340120808 shutdown_connections 2024-09-12T02:56:51.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:51.256+0000 7fe2d437b640 1 -- 172.21.15.5:0/3340120808 >> 172.21.15.5:0/3340120808 conn(0x7fe2cc0fc7d0 msgr2=0x7fe2cc06a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:51.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:51.256+0000 7fe2d437b640 1 -- 172.21.15.5:0/3340120808 shutdown_connections 2024-09-12T02:56:51.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:51.256+0000 7fe2d437b640 1 -- 172.21.15.5:0/3340120808 wait complete. 2024-09-12T02:56:51.415 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:51 smithi148 ceph-mon[35090]: pgmap v293: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 3.3 KiB/s wr, 13 op/s 2024-09-12T02:56:51.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:51 smithi146 ceph-mon[35011]: pgmap v293: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 3.3 KiB/s wr, 13 op/s 2024-09-12T02:56:52.026 INFO:teuthology.orchestra.run.smithi005.stdout:154618822704 2024-09-12T02:56:52.026 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-09-12T02:56:52.478 INFO:teuthology.orchestra.run.smithi005.stdout:137438953527 2024-09-12T02:56:52.478 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-09-12T02:56:53.369 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:53 smithi005 ceph-mon[31197]: pgmap v294: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 938 B/s wr, 10 op/s 2024-09-12T02:56:53.415 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:53 smithi148 ceph-mon[35090]: pgmap v294: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 938 B/s wr, 10 op/s 2024-09-12T02:56:53.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:53 smithi146 ceph-mon[35011]: pgmap v294: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 938 B/s wr, 10 op/s 2024-09-12T02:56:53.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:53.495+0000 7fbe0ad1e640 1 -- 172.21.15.5:0/1716322943 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe04106f10 msgr2=0x7fbe0410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:53.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:53.495+0000 7fbe0ad1e640 1 --2- 172.21.15.5:0/1716322943 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe04106f10 0x7fbe0410ea90 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fbdfc009ca0 tx=0x7fbdfc02a8c0 comp rx=0 tx=0).stop 2024-09-12T02:56:53.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:53.495+0000 7fbe0ad1e640 1 -- 172.21.15.5:0/1716322943 shutdown_connections 2024-09-12T02:56:53.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:53.495+0000 7fbe0ad1e640 1 --2- 172.21.15.5:0/1716322943 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe04106f10 0x7fbe0410ea90 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:53.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:53.495+0000 7fbe0ad1e640 1 --2- 172.21.15.5:0/1716322943 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbe04102790 0x7fbe041067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:53.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:53.495+0000 7fbe0ad1e640 1 --2- 172.21.15.5:0/1716322943 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbe04101de0 0x7fbe041021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:53.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:53.495+0000 7fbe0ad1e640 1 -- 172.21.15.5:0/1716322943 >> 172.21.15.5:0/1716322943 conn(0x7fbe040fd8c0 msgr2=0x7fbe040ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:53.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:53.495+0000 7fbe0ad1e640 1 -- 172.21.15.5:0/1716322943 shutdown_connections 2024-09-12T02:56:53.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:53.495+0000 7fbe0ad1e640 1 -- 172.21.15.5:0/1716322943 wait complete. 2024-09-12T02:56:53.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:53.495+0000 7fbe0ad1e640 1 Processor -- start 2024-09-12T02:56:53.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:53.495+0000 7fbe0ad1e640 1 -- start start 2024-09-12T02:56:53.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:53.496+0000 7fbe0ad1e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbe04101de0 0x7fbe040744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:53.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:53.496+0000 7fbe0ad1e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe04102790 0x7fbe04074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:53.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:53.496+0000 7fbe0ad1e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbe04106f10 0x7fbe04074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:53.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:53.496+0000 7fbe0ad1e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbe04111f70 con 0x7fbe04102790 2024-09-12T02:56:53.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:53.496+0000 7fbe0ad1e640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fbe04111df0 con 0x7fbe04101de0 2024-09-12T02:56:53.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:53.496+0000 7fbe0ad1e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fbe041120f0 con 0x7fbe04106f10 2024-09-12T02:56:53.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:53.496+0000 7fbe03fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe04102790 0x7fbe04074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:53.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:53.496+0000 7fbe03fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe04102790 0x7fbe04074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58166/0 (socket says 172.21.15.5:58166) 2024-09-12T02:56:53.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:53.496+0000 7fbe03fff640 1 -- 172.21.15.5:0/3919226388 learned_addr learned my addr 172.21.15.5:0/3919226388 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:56:53.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:53.496+0000 7fbe08aba640 1 --2- 172.21.15.5:0/3919226388 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbe04101de0 0x7fbe040744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:53.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:53.496+0000 7fbe092bb640 1 --2- 172.21.15.5:0/3919226388 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbe04106f10 0x7fbe04074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:53.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:53.496+0000 7fbe03fff640 1 -- 172.21.15.5:0/3919226388 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbe04106f10 msgr2=0x7fbe04074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:53.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:53.496+0000 7fbe03fff640 1 --2- 172.21.15.5:0/3919226388 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbe04106f10 0x7fbe04074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:53.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:53.496+0000 7fbe03fff640 1 -- 172.21.15.5:0/3919226388 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbe04101de0 msgr2=0x7fbe040744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:53.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:53.496+0000 7fbe03fff640 1 --2- 172.21.15.5:0/3919226388 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbe04101de0 0x7fbe040744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:53.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:53.496+0000 7fbe03fff640 1 -- 172.21.15.5:0/3919226388 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe041a6860 con 0x7fbe04102790 2024-09-12T02:56:53.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:53.496+0000 7fbe03fff640 1 --2- 172.21.15.5:0/3919226388 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe04102790 0x7fbe04074a30 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fbdf000d950 tx=0x7fbdf000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:53.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:53.497+0000 7fbe01ffb640 1 -- 172.21.15.5:0/3919226388 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdf0017070 con 0x7fbe04102790 2024-09-12T02:56:53.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:53.497+0000 7fbe01ffb640 1 -- 172.21.15.5:0/3919226388 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbdf0009c30 con 0x7fbe04102790 2024-09-12T02:56:53.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:53.497+0000 7fbe0ad1e640 1 -- 172.21.15.5:0/3919226388 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe041a6b50 con 0x7fbe04102790 2024-09-12T02:56:53.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:53.497+0000 7fbe01ffb640 1 -- 172.21.15.5:0/3919226388 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdf000f6b0 con 0x7fbe04102790 2024-09-12T02:56:53.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:53.497+0000 7fbe0ad1e640 1 -- 172.21.15.5:0/3919226388 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe041a7090 con 0x7fbe04102790 2024-09-12T02:56:53.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:53.498+0000 7fbe0ad1e640 1 -- 172.21.15.5:0/3919226388 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbe04062900 con 0x7fbe04102790 2024-09-12T02:56:53.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:53.499+0000 7fbe01ffb640 1 -- 172.21.15.5:0/3919226388 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fbdf00109d0 con 0x7fbe04102790 2024-09-12T02:56:53.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:53.500+0000 7fbe01ffb640 1 --2- 172.21.15.5:0/3919226388 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fbdf40a2c10 0x7fbdf40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:53.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:53.500+0000 7fbe08aba640 1 --2- 172.21.15.5:0/3919226388 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fbdf40a2c10 0x7fbdf40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:53.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:53.500+0000 7fbe01ffb640 1 -- 172.21.15.5:0/3919226388 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fbdf00cf1d0 con 0x7fbe04102790 2024-09-12T02:56:53.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:53.500+0000 7fbe01ffb640 1 --2- 172.21.15.5:0/3919226388 >> [v2:172.21.15.148:6824/839290773,v1:172.21.15.148:6825/839290773] conn(0x7fbdf40ac7c0 0x7fbdf40aec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:53.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:53.500+0000 7fbe01ffb640 1 -- 172.21.15.5:0/3919226388 --> [v2:172.21.15.148:6824/839290773,v1:172.21.15.148:6825/839290773] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbdf40af330 con 0x7fbdf40ac7c0 2024-09-12T02:56:53.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:53.500+0000 7fbe01ffb640 1 -- 172.21.15.5:0/3919226388 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7fbdf00cf580 con 0x7fbe04102790 2024-09-12T02:56:53.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:53.500+0000 7fbe08aba640 1 --2- 172.21.15.5:0/3919226388 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fbdf40a2c10 0x7fbdf40a50d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fbdf8005fd0 tx=0x7fbdf800b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:53.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:53.500+0000 7fbe092bb640 1 --2- 172.21.15.5:0/3919226388 >> [v2:172.21.15.148:6824/839290773,v1:172.21.15.148:6825/839290773] conn(0x7fbdf40ac7c0 0x7fbdf40aec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:53.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:53.501+0000 7fbe092bb640 1 --2- 172.21.15.5:0/3919226388 >> [v2:172.21.15.148:6824/839290773,v1:172.21.15.148:6825/839290773] conn(0x7fbdf40ac7c0 0x7fbdf40aec60 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:53.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:53.502+0000 7fbe01ffb640 1 -- 172.21.15.5:0/3919226388 <== osd.11 v2:172.21.15.148:6824/839290773 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbdf40af330 con 0x7fbdf40ac7c0 2024-09-12T02:56:53.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:53.516+0000 7fbe0ad1e640 1 -- 172.21.15.5:0/3919226388 --> [v2:172.21.15.148:6824/839290773,v1:172.21.15.148:6825/839290773] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fbe04062d80 con 0x7fbdf40ac7c0 2024-09-12T02:56:53.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:53.516+0000 7fbe01ffb640 1 -- 172.21.15.5:0/3919226388 <== osd.11 v2:172.21.15.148:6824/839290773 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fbe04062d80 con 0x7fbdf40ac7c0 2024-09-12T02:56:53.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:53.517+0000 7fbe0ad1e640 1 -- 172.21.15.5:0/3919226388 >> [v2:172.21.15.148:6824/839290773,v1:172.21.15.148:6825/839290773] conn(0x7fbdf40ac7c0 msgr2=0x7fbdf40aec60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:53.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:53.517+0000 7fbe0ad1e640 1 --2- 172.21.15.5:0/3919226388 >> [v2:172.21.15.148:6824/839290773,v1:172.21.15.148:6825/839290773] conn(0x7fbdf40ac7c0 0x7fbdf40aec60 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:53.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:53.517+0000 7fbe0ad1e640 1 -- 172.21.15.5:0/3919226388 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fbdf40a2c10 msgr2=0x7fbdf40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:53.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:53.517+0000 7fbe0ad1e640 1 --2- 172.21.15.5:0/3919226388 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fbdf40a2c10 0x7fbdf40a50d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fbdf8005fd0 tx=0x7fbdf800b390 comp rx=0 tx=0).stop 2024-09-12T02:56:53.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:53.517+0000 7fbe0ad1e640 1 -- 172.21.15.5:0/3919226388 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe04102790 msgr2=0x7fbe04074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:53.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:53.517+0000 7fbe0ad1e640 1 --2- 172.21.15.5:0/3919226388 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe04102790 0x7fbe04074a30 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fbdf000d950 tx=0x7fbdf000de10 comp rx=0 tx=0).stop 2024-09-12T02:56:53.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:53.517+0000 7fbe092bb640 1 -- 172.21.15.5:0/3919226388 reap_dead start 2024-09-12T02:56:53.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:53.517+0000 7fbe0ad1e640 1 -- 172.21.15.5:0/3919226388 shutdown_connections 2024-09-12T02:56:53.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:53.517+0000 7fbe0ad1e640 1 -- 172.21.15.5:0/3919226388 >> 172.21.15.5:0/3919226388 conn(0x7fbe040fd8c0 msgr2=0x7fbe040fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:53.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:53.517+0000 7fbe0ad1e640 1 -- 172.21.15.5:0/3919226388 shutdown_connections 2024-09-12T02:56:53.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:53.517+0000 7fbe0ad1e640 1 -- 172.21.15.5:0/3919226388 wait complete. 2024-09-12T02:56:54.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:53.619+0000 7f08418c5640 1 -- 172.21.15.5:0/2327864523 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f083c100960 msgr2=0x7f083c100d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:54.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:53.619+0000 7f08418c5640 1 --2- 172.21.15.5:0/2327864523 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f083c100960 0x7f083c100d40 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0824009900 tx=0x7f082402a410 comp rx=0 tx=0).stop 2024-09-12T02:56:54.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:53.619+0000 7f08418c5640 1 -- 172.21.15.5:0/2327864523 shutdown_connections 2024-09-12T02:56:54.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:53.619+0000 7f08418c5640 1 --2- 172.21.15.5:0/2327864523 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f083c10e510 0x7f083c110900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:54.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:53.619+0000 7f08418c5640 1 --2- 172.21.15.5:0/2327864523 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f083c101310 0x7f083c10dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:54.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:53.619+0000 7f08418c5640 1 --2- 172.21.15.5:0/2327864523 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f083c100960 0x7f083c100d40 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:54.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:53.619+0000 7f08418c5640 1 -- 172.21.15.5:0/2327864523 >> 172.21.15.5:0/2327864523 conn(0x7f083c0fc7d0 msgr2=0x7f083c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:54.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:53.620+0000 7f08418c5640 1 -- 172.21.15.5:0/2327864523 shutdown_connections 2024-09-12T02:56:54.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:53.620+0000 7f08418c5640 1 -- 172.21.15.5:0/2327864523 wait complete. 2024-09-12T02:56:54.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:53.620+0000 7f08418c5640 1 Processor -- start 2024-09-12T02:56:54.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:53.620+0000 7f08418c5640 1 -- start start 2024-09-12T02:56:54.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:53.621+0000 7f08418c5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f083c100960 0x7f083c19a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:54.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:53.621+0000 7f08418c5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f083c101310 0x7f083c19ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:54.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:53.621+0000 7f08418c5640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f083c10e510 0x7f083c19f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:54.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:53.621+0000 7f08418c5640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f083c114230 con 0x7f083c101310 2024-09-12T02:56:54.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:53.621+0000 7f08418c5640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f083c1140b0 con 0x7f083c10e510 2024-09-12T02:56:54.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:53.621+0000 7f08418c5640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f083c1143b0 con 0x7f083c100960 2024-09-12T02:56:54.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:53.621+0000 7f083a7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f083c101310 0x7f083c19ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:54.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:53.621+0000 7f083a7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f083c101310 0x7f083c19ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58182/0 (socket says 172.21.15.5:58182) 2024-09-12T02:56:54.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:53.621+0000 7f083a7fc640 1 -- 172.21.15.5:0/1419457426 learned_addr learned my addr 172.21.15.5:0/1419457426 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:56:54.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:53.621+0000 7f083b7fe640 1 --2- 172.21.15.5:0/1419457426 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f083c10e510 0x7f083c19f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:54.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:53.621+0000 7f083affd640 1 --2- 172.21.15.5:0/1419457426 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f083c100960 0x7f083c19a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:54.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:53.621+0000 7f083a7fc640 1 -- 172.21.15.5:0/1419457426 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f083c100960 msgr2=0x7f083c19a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:54.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:53.621+0000 7f083a7fc640 1 --2- 172.21.15.5:0/1419457426 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f083c100960 0x7f083c19a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:54.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:53.621+0000 7f083a7fc640 1 -- 172.21.15.5:0/1419457426 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f083c10e510 msgr2=0x7f083c19f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:54.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:53.621+0000 7f083a7fc640 1 --2- 172.21.15.5:0/1419457426 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f083c10e510 0x7f083c19f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:54.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:53.621+0000 7f083a7fc640 1 -- 172.21.15.5:0/1419457426 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f083c19f6e0 con 0x7f083c101310 2024-09-12T02:56:54.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:53.621+0000 7f083a7fc640 1 --2- 172.21.15.5:0/1419457426 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f083c101310 0x7f083c19ac70 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f082c00b450 tx=0x7f082c00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:54.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:53.622+0000 7f08408c3640 1 -- 172.21.15.5:0/1419457426 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f082c015020 con 0x7f083c101310 2024-09-12T02:56:54.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:53.622+0000 7f08408c3640 1 -- 172.21.15.5:0/1419457426 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f082c011d70 con 0x7f083c101310 2024-09-12T02:56:54.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:53.622+0000 7f08418c5640 1 -- 172.21.15.5:0/1419457426 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f083c19f9d0 con 0x7f083c101310 2024-09-12T02:56:54.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:53.622+0000 7f08408c3640 1 -- 172.21.15.5:0/1419457426 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f082c00f340 con 0x7f083c101310 2024-09-12T02:56:54.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:53.622+0000 7f08418c5640 1 -- 172.21.15.5:0/1419457426 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f083c1a72b0 con 0x7f083c101310 2024-09-12T02:56:54.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:53.623+0000 7f08418c5640 1 -- 172.21.15.5:0/1419457426 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f083c062980 con 0x7f083c101310 2024-09-12T02:56:54.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:53.624+0000 7f08408c3640 1 -- 172.21.15.5:0/1419457426 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f082c018020 con 0x7f083c101310 2024-09-12T02:56:54.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:53.625+0000 7f08408c3640 1 --2- 172.21.15.5:0/1419457426 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f08280a2c90 0x7f08280a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:54.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:53.625+0000 7f083affd640 1 --2- 172.21.15.5:0/1419457426 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f08280a2c90 0x7f08280a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:54.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:53.625+0000 7f08408c3640 1 -- 172.21.15.5:0/1419457426 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f082c0cf150 con 0x7f083c101310 2024-09-12T02:56:54.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:53.626+0000 7f08408c3640 1 --2- 172.21.15.5:0/1419457426 >> [v2:172.21.15.146:6800/1105990287,v1:172.21.15.146:6801/1105990287] conn(0x7f08280ac840 0x7f08280aece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:54.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:53.626+0000 7f08408c3640 1 -- 172.21.15.5:0/1419457426 --> [v2:172.21.15.146:6800/1105990287,v1:172.21.15.146:6801/1105990287] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f08280af3b0 con 0x7f08280ac840 2024-09-12T02:56:54.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:53.626+0000 7f08408c3640 1 -- 172.21.15.5:0/1419457426 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f082c0cf5a0 con 0x7f083c101310 2024-09-12T02:56:54.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:53.626+0000 7f083affd640 1 --2- 172.21.15.5:0/1419457426 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f08280a2c90 0x7f08280a5150 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0824005940 tx=0x7f08240058b0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:54.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:53.626+0000 7f083b7fe640 1 --2- 172.21.15.5:0/1419457426 >> [v2:172.21.15.146:6800/1105990287,v1:172.21.15.146:6801/1105990287] conn(0x7f08280ac840 0x7f08280aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:54.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:53.626+0000 7f083b7fe640 1 --2- 172.21.15.5:0/1419457426 >> [v2:172.21.15.146:6800/1105990287,v1:172.21.15.146:6801/1105990287] conn(0x7f08280ac840 0x7f08280aece0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:54.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:53.627+0000 7f08408c3640 1 -- 172.21.15.5:0/1419457426 <== osd.1 v2:172.21.15.146:6800/1105990287 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f08280af3b0 con 0x7f08280ac840 2024-09-12T02:56:54.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:53.641+0000 7f08418c5640 1 -- 172.21.15.5:0/1419457426 --> [v2:172.21.15.146:6800/1105990287,v1:172.21.15.146:6801/1105990287] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f083c062d80 con 0x7f08280ac840 2024-09-12T02:56:54.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:53.642+0000 7f08408c3640 1 -- 172.21.15.5:0/1419457426 <== osd.1 v2:172.21.15.146:6800/1105990287 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f083c062d80 con 0x7f08280ac840 2024-09-12T02:56:54.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:53.642+0000 7f08418c5640 1 -- 172.21.15.5:0/1419457426 >> [v2:172.21.15.146:6800/1105990287,v1:172.21.15.146:6801/1105990287] conn(0x7f08280ac840 msgr2=0x7f08280aece0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:54.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:53.642+0000 7f08418c5640 1 --2- 172.21.15.5:0/1419457426 >> [v2:172.21.15.146:6800/1105990287,v1:172.21.15.146:6801/1105990287] conn(0x7f08280ac840 0x7f08280aece0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:54.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:53.642+0000 7f08418c5640 1 -- 172.21.15.5:0/1419457426 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f08280a2c90 msgr2=0x7f08280a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:54.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:53.642+0000 7f08418c5640 1 --2- 172.21.15.5:0/1419457426 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f08280a2c90 0x7f08280a5150 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0824005940 tx=0x7f08240058b0 comp rx=0 tx=0).stop 2024-09-12T02:56:54.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:53.642+0000 7f08418c5640 1 -- 172.21.15.5:0/1419457426 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f083c101310 msgr2=0x7f083c19ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:54.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:53.642+0000 7f08418c5640 1 --2- 172.21.15.5:0/1419457426 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f083c101310 0x7f083c19ac70 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f082c00b450 tx=0x7f082c00b910 comp rx=0 tx=0).stop 2024-09-12T02:56:54.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:53.642+0000 7f083b7fe640 1 -- 172.21.15.5:0/1419457426 reap_dead start 2024-09-12T02:56:54.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:53.642+0000 7f08418c5640 1 -- 172.21.15.5:0/1419457426 shutdown_connections 2024-09-12T02:56:54.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:53.642+0000 7f08418c5640 1 -- 172.21.15.5:0/1419457426 >> 172.21.15.5:0/1419457426 conn(0x7f083c0fc7d0 msgr2=0x7f083c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:54.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:53.642+0000 7f08418c5640 1 -- 172.21.15.5:0/1419457426 shutdown_connections 2024-09-12T02:56:54.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:53.642+0000 7f08418c5640 1 -- 172.21.15.5:0/1419457426 wait complete. 2024-09-12T02:56:54.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:54.558+0000 7f698f7e1640 1 -- 172.21.15.5:0/1683563213 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6988103660 msgr2=0x7f698810bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:54.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:54.558+0000 7f698f7e1640 1 --2- 172.21.15.5:0/1683563213 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6988103660 0x7f698810bb50 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6978009900 tx=0x7f697802a480 comp rx=0 tx=0).stop 2024-09-12T02:56:54.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:54.558+0000 7f698f7e1640 1 -- 172.21.15.5:0/1683563213 shutdown_connections 2024-09-12T02:56:54.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:54.558+0000 7f698f7e1640 1 --2- 172.21.15.5:0/1683563213 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6988103660 0x7f698810bb50 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:54.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:54.558+0000 7f698f7e1640 1 --2- 172.21.15.5:0/1683563213 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f698810c170 0x7f698810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:54.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:54.558+0000 7f698f7e1640 1 --2- 172.21.15.5:0/1683563213 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6988102d40 0x7f6988103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:54.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:54.558+0000 7f698f7e1640 1 -- 172.21.15.5:0/1683563213 >> 172.21.15.5:0/1683563213 conn(0x7f69880fc7a0 msgr2=0x7f69880febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:54.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:54.559+0000 7f698f7e1640 1 -- 172.21.15.5:0/1683563213 shutdown_connections 2024-09-12T02:56:54.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:54.559+0000 7f698f7e1640 1 -- 172.21.15.5:0/1683563213 wait complete. 2024-09-12T02:56:54.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:54.559+0000 7f698f7e1640 1 Processor -- start 2024-09-12T02:56:54.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:54.559+0000 7f698f7e1640 1 -- start start 2024-09-12T02:56:54.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:54.559+0000 7f698f7e1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6988102d40 0x7f69881009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:54.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:54.559+0000 7f698f7e1640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6988103660 0x7f6988100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:54.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:54.559+0000 7f698f7e1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f698810c170 0x7f6988101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:54.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:54.559+0000 7f698f7e1640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6988111fb0 con 0x7f698810c170 2024-09-12T02:56:54.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:54.559+0000 7f698f7e1640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f6988111e30 con 0x7f6988103660 2024-09-12T02:56:54.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:54.559+0000 7f698f7e1640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f6988112130 con 0x7f6988102d40 2024-09-12T02:56:54.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:54.560+0000 7f698dd7e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f698810c170 0x7f6988101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:54.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:54.560+0000 7f698d57d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6988102d40 0x7f69881009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:54.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:54.560+0000 7f698dd7e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f698810c170 0x7f6988101420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58200/0 (socket says 172.21.15.5:58200) 2024-09-12T02:56:54.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:54.560+0000 7f698dd7e640 1 -- 172.21.15.5:0/941708368 learned_addr learned my addr 172.21.15.5:0/941708368 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:56:54.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:54.560+0000 7f698cd7c640 1 --2- 172.21.15.5:0/941708368 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6988103660 0x7f6988100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:54.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:54.560+0000 7f698dd7e640 1 -- 172.21.15.5:0/941708368 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6988102d40 msgr2=0x7f69881009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:54.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:54.560+0000 7f698dd7e640 1 --2- 172.21.15.5:0/941708368 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6988102d40 0x7f69881009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:54.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:54.560+0000 7f698dd7e640 1 -- 172.21.15.5:0/941708368 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6988103660 msgr2=0x7f6988100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:54.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:54.560+0000 7f698dd7e640 1 --2- 172.21.15.5:0/941708368 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6988103660 0x7f6988100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:54.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:54.560+0000 7f698dd7e640 1 -- 172.21.15.5:0/941708368 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69881a6930 con 0x7f698810c170 2024-09-12T02:56:54.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:54.560+0000 7f698d57d640 1 --2- 172.21.15.5:0/941708368 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6988102d40 0x7f69881009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:56:54.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:54.560+0000 7f698dd7e640 1 --2- 172.21.15.5:0/941708368 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f698810c170 0x7f6988101420 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f698000b970 tx=0x7f698000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:54.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:54.560+0000 7f697e7fc640 1 -- 172.21.15.5:0/941708368 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6980019070 con 0x7f698810c170 2024-09-12T02:56:54.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:54.560+0000 7f698f7e1640 1 -- 172.21.15.5:0/941708368 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69881a6c20 con 0x7f698810c170 2024-09-12T02:56:54.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:54.560+0000 7f697e7fc640 1 -- 172.21.15.5:0/941708368 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f698000cb40 con 0x7f698810c170 2024-09-12T02:56:54.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:54.560+0000 7f697e7fc640 1 -- 172.21.15.5:0/941708368 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6980007650 con 0x7f698810c170 2024-09-12T02:56:54.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:54.560+0000 7f698f7e1640 1 -- 172.21.15.5:0/941708368 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69881a7130 con 0x7f698810c170 2024-09-12T02:56:54.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:54.562+0000 7f698f7e1640 1 -- 172.21.15.5:0/941708368 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f69880628c0 con 0x7f698810c170 2024-09-12T02:56:54.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:54.564+0000 7f697e7fc640 1 -- 172.21.15.5:0/941708368 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f698001c070 con 0x7f698810c170 2024-09-12T02:56:54.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:54.564+0000 7f697e7fc640 1 --2- 172.21.15.5:0/941708368 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f69740a2c10 0x7f69740a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:54.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:54.565+0000 7f698d57d640 1 --2- 172.21.15.5:0/941708368 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f69740a2c10 0x7f69740a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:54.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:54.565+0000 7f697e7fc640 1 -- 172.21.15.5:0/941708368 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f69800cee80 con 0x7f698810c170 2024-09-12T02:56:54.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:54.565+0000 7f697e7fc640 1 --2- 172.21.15.5:0/941708368 >> [v2:172.21.15.5:6818/3176073169,v1:172.21.15.5:6819/3176073169] conn(0x7f69740ac7c0 0x7f69740aec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:54.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:54.565+0000 7f697e7fc640 1 -- 172.21.15.5:0/941708368 --> [v2:172.21.15.5:6818/3176073169,v1:172.21.15.5:6819/3176073169] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f69800047d0 con 0x7f69740ac7c0 2024-09-12T02:56:54.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:54.565+0000 7f697e7fc640 1 -- 172.21.15.5:0/941708368 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f69800cf2d0 con 0x7f698810c170 2024-09-12T02:56:54.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:54.565+0000 7f698cd7c640 1 --2- 172.21.15.5:0/941708368 >> [v2:172.21.15.5:6818/3176073169,v1:172.21.15.5:6819/3176073169] conn(0x7f69740ac7c0 0x7f69740aec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:54.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:54.565+0000 7f698d57d640 1 --2- 172.21.15.5:0/941708368 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f69740a2c10 0x7f69740a50d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f698400bfd0 tx=0x7f698400b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:54.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:54.565+0000 7f698cd7c640 1 --2- 172.21.15.5:0/941708368 >> [v2:172.21.15.5:6818/3176073169,v1:172.21.15.5:6819/3176073169] conn(0x7f69740ac7c0 0x7f69740aec60 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:54.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:54.566+0000 7f697e7fc640 1 -- 172.21.15.5:0/941708368 <== osd.6 v2:172.21.15.5:6818/3176073169 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f69800047d0 con 0x7f69740ac7c0 2024-09-12T02:56:54.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:54.580+0000 7f698f7e1640 1 -- 172.21.15.5:0/941708368 --> [v2:172.21.15.5:6818/3176073169,v1:172.21.15.5:6819/3176073169] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6988062d40 con 0x7f69740ac7c0 2024-09-12T02:56:54.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:54.580+0000 7f697e7fc640 1 -- 172.21.15.5:0/941708368 <== osd.6 v2:172.21.15.5:6818/3176073169 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f6988062d40 con 0x7f69740ac7c0 2024-09-12T02:56:54.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:54.580+0000 7f698f7e1640 1 -- 172.21.15.5:0/941708368 >> [v2:172.21.15.5:6818/3176073169,v1:172.21.15.5:6819/3176073169] conn(0x7f69740ac7c0 msgr2=0x7f69740aec60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:54.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:54.580+0000 7f698f7e1640 1 --2- 172.21.15.5:0/941708368 >> [v2:172.21.15.5:6818/3176073169,v1:172.21.15.5:6819/3176073169] conn(0x7f69740ac7c0 0x7f69740aec60 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:54.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:54.580+0000 7f698f7e1640 1 -- 172.21.15.5:0/941708368 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f69740a2c10 msgr2=0x7f69740a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:54.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:54.580+0000 7f698f7e1640 1 --2- 172.21.15.5:0/941708368 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f69740a2c10 0x7f69740a50d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f698400bfd0 tx=0x7f698400b390 comp rx=0 tx=0).stop 2024-09-12T02:56:54.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:54.581+0000 7f698f7e1640 1 -- 172.21.15.5:0/941708368 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f698810c170 msgr2=0x7f6988101420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:54.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:54.581+0000 7f698f7e1640 1 --2- 172.21.15.5:0/941708368 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f698810c170 0x7f6988101420 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f698000b970 tx=0x7f698000be30 comp rx=0 tx=0).stop 2024-09-12T02:56:54.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:54.581+0000 7f698dd7e640 1 -- 172.21.15.5:0/941708368 reap_dead start 2024-09-12T02:56:54.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:54.581+0000 7f698f7e1640 1 -- 172.21.15.5:0/941708368 shutdown_connections 2024-09-12T02:56:54.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:54.581+0000 7f698f7e1640 1 -- 172.21.15.5:0/941708368 >> 172.21.15.5:0/941708368 conn(0x7f69880fc7a0 msgr2=0x7f69880fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:54.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:54.581+0000 7f698f7e1640 1 -- 172.21.15.5:0/941708368 shutdown_connections 2024-09-12T02:56:54.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:54.581+0000 7f698f7e1640 1 -- 172.21.15.5:0/941708368 wait complete. 2024-09-12T02:56:55.415 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:55 smithi148 ceph-mon[35090]: pgmap v295: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-12T02:56:55.416 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:55 smithi005 ceph-mon[31197]: pgmap v295: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-12T02:56:55.425 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:56:55.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:55 smithi146 ceph-mon[35011]: pgmap v295: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 938 B/s wr, 11 op/s 2024-09-12T02:56:55.462 INFO:teuthology.orchestra.run.smithi005.stdout:34359738442 2024-09-12T02:56:55.462 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-12T02:56:55.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:55.613+0000 7f2eebf02640 1 -- 172.21.15.5:0/2769184245 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ee4100a90 msgr2=0x7f2ee4100ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:55.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:55.613+0000 7f2eebf02640 1 --2- 172.21.15.5:0/2769184245 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ee4100a90 0x7f2ee4100ef0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f2ed4009930 tx=0x7f2ed402a420 comp rx=0 tx=0).stop 2024-09-12T02:56:55.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:55.613+0000 7f2eebf02640 1 -- 172.21.15.5:0/2769184245 shutdown_connections 2024-09-12T02:56:55.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:55.613+0000 7f2eebf02640 1 --2- 172.21.15.5:0/2769184245 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ee4100a90 0x7f2ee4100ef0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:55.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:55.613+0000 7f2eebf02640 1 --2- 172.21.15.5:0/2769184245 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ee4101430 0x7f2ee410d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:55.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:55.613+0000 7f2eebf02640 1 --2- 172.21.15.5:0/2769184245 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ee4106a90 0x7f2ee4106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:55.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:55.613+0000 7f2eebf02640 1 -- 172.21.15.5:0/2769184245 >> 172.21.15.5:0/2769184245 conn(0x7f2ee40fc7d0 msgr2=0x7f2ee40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:55.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:55.613+0000 7f2eebf02640 1 -- 172.21.15.5:0/2769184245 shutdown_connections 2024-09-12T02:56:55.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:55.613+0000 7f2eebf02640 1 -- 172.21.15.5:0/2769184245 wait complete. 2024-09-12T02:56:55.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:55.614+0000 7f2eebf02640 1 Processor -- start 2024-09-12T02:56:55.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:55.614+0000 7f2eebf02640 1 -- start start 2024-09-12T02:56:55.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:55.614+0000 7f2eebf02640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ee4100a90 0x7f2ee40fcdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:55.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:55.614+0000 7f2eebf02640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ee4101430 0x7f2ee40fd310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:55.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:55.614+0000 7f2eebf02640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ee4106a90 0x7f2ee40fd850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:55.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:55.614+0000 7f2eebf02640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee4110eb0 con 0x7f2ee4101430 2024-09-12T02:56:55.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:55.614+0000 7f2eebf02640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee4110d30 con 0x7f2ee4100a90 2024-09-12T02:56:55.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:55.614+0000 7f2eebf02640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee4111030 con 0x7f2ee4106a90 2024-09-12T02:56:55.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:55.614+0000 7f2ee949d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ee4101430 0x7f2ee40fd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:55.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:55.614+0000 7f2ee949d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ee4101430 0x7f2ee40fd310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58210/0 (socket says 172.21.15.5:58210) 2024-09-12T02:56:55.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:55.615+0000 7f2ee949d640 1 -- 172.21.15.5:0/2945053877 learned_addr learned my addr 172.21.15.5:0/2945053877 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:56:55.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:55.615+0000 7f2eea49f640 1 --2- 172.21.15.5:0/2945053877 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ee4106a90 0x7f2ee40fd850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:55.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:55.615+0000 7f2ee9c9e640 1 --2- 172.21.15.5:0/2945053877 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ee4100a90 0x7f2ee40fcdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:55.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:55.615+0000 7f2ee949d640 1 -- 172.21.15.5:0/2945053877 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ee4106a90 msgr2=0x7f2ee40fd850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:55.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:55.615+0000 7f2ee949d640 1 --2- 172.21.15.5:0/2945053877 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ee4106a90 0x7f2ee40fd850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:55.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:55.615+0000 7f2ee949d640 1 -- 172.21.15.5:0/2945053877 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ee4100a90 msgr2=0x7f2ee40fcdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:55.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:55.615+0000 7f2ee949d640 1 --2- 172.21.15.5:0/2945053877 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ee4100a90 0x7f2ee40fcdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:55.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:55.615+0000 7f2ee949d640 1 -- 172.21.15.5:0/2945053877 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ee41a6910 con 0x7f2ee4101430 2024-09-12T02:56:55.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:55.615+0000 7f2ee9c9e640 1 --2- 172.21.15.5:0/2945053877 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ee4100a90 0x7f2ee40fcdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:56:55.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:55.615+0000 7f2ee949d640 1 --2- 172.21.15.5:0/2945053877 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ee4101430 0x7f2ee40fd310 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f2ed4009a50 tx=0x7f2ed4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:55.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:55.615+0000 7f2edaffd640 1 -- 172.21.15.5:0/2945053877 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed403f030 con 0x7f2ee4101430 2024-09-12T02:56:55.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:55.615+0000 7f2eebf02640 1 -- 172.21.15.5:0/2945053877 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ee41a6ba0 con 0x7f2ee4101430 2024-09-12T02:56:55.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:55.615+0000 7f2edaffd640 1 -- 172.21.15.5:0/2945053877 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ed4003ce0 con 0x7f2ee4101430 2024-09-12T02:56:55.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:55.616+0000 7f2edaffd640 1 -- 172.21.15.5:0/2945053877 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed4035070 con 0x7f2ee4101430 2024-09-12T02:56:55.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:55.616+0000 7f2eebf02640 1 -- 172.21.15.5:0/2945053877 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ee41a70d0 con 0x7f2ee4101430 2024-09-12T02:56:55.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:55.616+0000 7f2eebf02640 1 -- 172.21.15.5:0/2945053877 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2ee40628c0 con 0x7f2ee4101430 2024-09-12T02:56:55.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:55.618+0000 7f2edaffd640 1 -- 172.21.15.5:0/2945053877 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f2ed4042030 con 0x7f2ee4101430 2024-09-12T02:56:55.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:55.618+0000 7f2edaffd640 1 --2- 172.21.15.5:0/2945053877 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f2ed00a2c10 0x7f2ed00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:55.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:55.619+0000 7f2edaffd640 1 -- 172.21.15.5:0/2945053877 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f2ed40ef830 con 0x7f2ee4101430 2024-09-12T02:56:55.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:55.619+0000 7f2ee9c9e640 1 --2- 172.21.15.5:0/2945053877 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f2ed00a2c10 0x7f2ed00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:55.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:55.619+0000 7f2edaffd640 1 --2- 172.21.15.5:0/2945053877 >> [v2:172.21.15.146:6816/2209508838,v1:172.21.15.146:6817/2209508838] conn(0x7f2ed00ac690 0x7f2ed00aeb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:55.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:55.619+0000 7f2edaffd640 1 -- 172.21.15.5:0/2945053877 --> [v2:172.21.15.146:6816/2209508838,v1:172.21.15.146:6817/2209508838] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2ed400b0b0 con 0x7f2ed00ac690 2024-09-12T02:56:55.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:55.619+0000 7f2edaffd640 1 -- 172.21.15.5:0/2945053877 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f2ed40efc80 con 0x7f2ee4101430 2024-09-12T02:56:55.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:55.619+0000 7f2ee9c9e640 1 --2- 172.21.15.5:0/2945053877 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f2ed00a2c10 0x7f2ed00a50d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2ee000abe0 tx=0x7f2ee000a5c0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:55.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:55.619+0000 7f2eea49f640 1 --2- 172.21.15.5:0/2945053877 >> [v2:172.21.15.146:6816/2209508838,v1:172.21.15.146:6817/2209508838] conn(0x7f2ed00ac690 0x7f2ed00aeb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:55.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:55.620+0000 7f2eea49f640 1 --2- 172.21.15.5:0/2945053877 >> [v2:172.21.15.146:6816/2209508838,v1:172.21.15.146:6817/2209508838] conn(0x7f2ed00ac690 0x7f2ed00aeb30 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:55.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:55.621+0000 7f2edaffd640 1 -- 172.21.15.5:0/2945053877 <== osd.7 v2:172.21.15.146:6816/2209508838 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2ed400b0b0 con 0x7f2ed00ac690 2024-09-12T02:56:55.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:55.635+0000 7f2eebf02640 1 -- 172.21.15.5:0/2945053877 --> [v2:172.21.15.146:6816/2209508838,v1:172.21.15.146:6817/2209508838] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2ee4062cf0 con 0x7f2ed00ac690 2024-09-12T02:56:55.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:55.635+0000 7f2edaffd640 1 -- 172.21.15.5:0/2945053877 <== osd.7 v2:172.21.15.146:6816/2209508838 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f2ee4062cf0 con 0x7f2ed00ac690 2024-09-12T02:56:55.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:55.636+0000 7f2eebf02640 1 -- 172.21.15.5:0/2945053877 >> [v2:172.21.15.146:6816/2209508838,v1:172.21.15.146:6817/2209508838] conn(0x7f2ed00ac690 msgr2=0x7f2ed00aeb30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:55.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:55.636+0000 7f2eebf02640 1 --2- 172.21.15.5:0/2945053877 >> [v2:172.21.15.146:6816/2209508838,v1:172.21.15.146:6817/2209508838] conn(0x7f2ed00ac690 0x7f2ed00aeb30 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:55.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:55.636+0000 7f2eebf02640 1 -- 172.21.15.5:0/2945053877 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f2ed00a2c10 msgr2=0x7f2ed00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:55.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:55.636+0000 7f2eebf02640 1 --2- 172.21.15.5:0/2945053877 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f2ed00a2c10 0x7f2ed00a50d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2ee000abe0 tx=0x7f2ee000a5c0 comp rx=0 tx=0).stop 2024-09-12T02:56:55.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:55.636+0000 7f2eebf02640 1 -- 172.21.15.5:0/2945053877 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ee4101430 msgr2=0x7f2ee40fd310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:55.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:55.636+0000 7f2eebf02640 1 --2- 172.21.15.5:0/2945053877 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ee4101430 0x7f2ee40fd310 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f2ed4009a50 tx=0x7f2ed4004270 comp rx=0 tx=0).stop 2024-09-12T02:56:55.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:55.636+0000 7f2eea49f640 1 -- 172.21.15.5:0/2945053877 reap_dead start 2024-09-12T02:56:55.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:55.636+0000 7f2eebf02640 1 -- 172.21.15.5:0/2945053877 shutdown_connections 2024-09-12T02:56:55.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:55.636+0000 7f2eebf02640 1 -- 172.21.15.5:0/2945053877 >> 172.21.15.5:0/2945053877 conn(0x7f2ee40fc7d0 msgr2=0x7f2ee406aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:55.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:55.636+0000 7f2eebf02640 1 -- 172.21.15.5:0/2945053877 shutdown_connections 2024-09-12T02:56:55.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:55.636+0000 7f2eebf02640 1 -- 172.21.15.5:0/2945053877 wait complete. 2024-09-12T02:56:56.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:56.706+0000 7f2023799640 1 -- 172.21.15.5:0/2029747428 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f201c10c1a0 msgr2=0x7f201c10e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:56.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:56.706+0000 7f2023799640 1 --2- 172.21.15.5:0/2029747428 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f201c10c1a0 0x7f201c10e610 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f2014009ca0 tx=0x7f201402a950 comp rx=0 tx=0).stop 2024-09-12T02:56:56.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:56.706+0000 7f2023799640 1 -- 172.21.15.5:0/2029747428 shutdown_connections 2024-09-12T02:56:56.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:56.706+0000 7f2023799640 1 --2- 172.21.15.5:0/2029747428 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f201c10c1a0 0x7f201c10e610 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:56.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:56.706+0000 7f2023799640 1 --2- 172.21.15.5:0/2029747428 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f201c103720 0x7f201c10bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:56.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:56.706+0000 7f2023799640 1 --2- 172.21.15.5:0/2029747428 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f201c102e00 0x7f201c1031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:56.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:56.706+0000 7f2023799640 1 -- 172.21.15.5:0/2029747428 >> 172.21.15.5:0/2029747428 conn(0x7f201c0fc7a0 msgr2=0x7f201c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:56.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:56.706+0000 7f2023799640 1 -- 172.21.15.5:0/2029747428 shutdown_connections 2024-09-12T02:56:56.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:56.706+0000 7f2023799640 1 -- 172.21.15.5:0/2029747428 wait complete. 2024-09-12T02:56:56.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:56.707+0000 7f2023799640 1 Processor -- start 2024-09-12T02:56:56.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:56.707+0000 7f2023799640 1 -- start start 2024-09-12T02:56:56.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:56.707+0000 7f2023799640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f201c102e00 0x7f201c19a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:56.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:56.707+0000 7f2023799640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f201c103720 0x7f201c19abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:56.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:56.707+0000 7f2023799640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f201c10c1a0 0x7f201c19ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:56.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:56.707+0000 7f2023799640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f201c111fb0 con 0x7f201c102e00 2024-09-12T02:56:56.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:56.707+0000 7f2023799640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f201c111e30 con 0x7f201c103720 2024-09-12T02:56:56.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:56.707+0000 7f2023799640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f201c112130 con 0x7f201c10c1a0 2024-09-12T02:56:56.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:56.707+0000 7f2021535640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f201c102e00 0x7f201c19a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:56.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:56.707+0000 7f2021535640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f201c102e00 0x7f201c19a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58246/0 (socket says 172.21.15.5:58246) 2024-09-12T02:56:56.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:56.707+0000 7f2021535640 1 -- 172.21.15.5:0/3811457219 learned_addr learned my addr 172.21.15.5:0/3811457219 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:56:56.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:56.707+0000 7f2020d34640 1 --2- 172.21.15.5:0/3811457219 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f201c103720 0x7f201c19abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:56.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:56.708+0000 7f2021d36640 1 --2- 172.21.15.5:0/3811457219 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f201c10c1a0 0x7f201c19ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:56.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:56.708+0000 7f2021535640 1 -- 172.21.15.5:0/3811457219 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f201c10c1a0 msgr2=0x7f201c19ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:56.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:56.708+0000 7f2021535640 1 --2- 172.21.15.5:0/3811457219 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f201c10c1a0 0x7f201c19ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:56.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:56.708+0000 7f2021535640 1 -- 172.21.15.5:0/3811457219 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f201c103720 msgr2=0x7f201c19abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:56.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:56.708+0000 7f2021535640 1 --2- 172.21.15.5:0/3811457219 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f201c103720 0x7f201c19abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:56.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:56.708+0000 7f2021535640 1 -- 172.21.15.5:0/3811457219 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f201c19f660 con 0x7f201c102e00 2024-09-12T02:56:56.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:56.708+0000 7f2021535640 1 --2- 172.21.15.5:0/3811457219 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f201c102e00 0x7f201c19a6b0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f201800d950 tx=0x7f201800dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:56.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:56.708+0000 7f20127fc640 1 -- 172.21.15.5:0/3811457219 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2018017070 con 0x7f201c102e00 2024-09-12T02:56:56.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:56.708+0000 7f2023799640 1 -- 172.21.15.5:0/3811457219 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f201c19f950 con 0x7f201c102e00 2024-09-12T02:56:56.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:56.708+0000 7f20127fc640 1 -- 172.21.15.5:0/3811457219 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f201800de80 con 0x7f201c102e00 2024-09-12T02:56:56.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:56.708+0000 7f20127fc640 1 -- 172.21.15.5:0/3811457219 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f201800f620 con 0x7f201c102e00 2024-09-12T02:56:56.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:56.708+0000 7f2023799640 1 -- 172.21.15.5:0/3811457219 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f201c1a7230 con 0x7f201c102e00 2024-09-12T02:56:56.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:56.709+0000 7f2023799640 1 -- 172.21.15.5:0/3811457219 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f201c102380 con 0x7f201c102e00 2024-09-12T02:56:56.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:56.710+0000 7f20127fc640 1 -- 172.21.15.5:0/3811457219 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f2018009c30 con 0x7f201c102e00 2024-09-12T02:56:56.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:56.711+0000 7f20127fc640 1 --2- 172.21.15.5:0/3811457219 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f20080a2c10 0x7f20080a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:56.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:56.711+0000 7f2020d34640 1 --2- 172.21.15.5:0/3811457219 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f20080a2c10 0x7f20080a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:56.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:56.711+0000 7f20127fc640 1 -- 172.21.15.5:0/3811457219 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f20180cf2c0 con 0x7f201c102e00 2024-09-12T02:56:56.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:56.711+0000 7f20127fc640 1 --2- 172.21.15.5:0/3811457219 >> [v2:172.21.15.146:6824/2041733103,v1:172.21.15.146:6825/2041733103] conn(0x7f20080ac7c0 0x7f20080aec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:56.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:56.711+0000 7f20127fc640 1 -- 172.21.15.5:0/3811457219 --> [v2:172.21.15.146:6824/2041733103,v1:172.21.15.146:6825/2041733103] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f20080af330 con 0x7f20080ac7c0 2024-09-12T02:56:56.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:56.711+0000 7f20127fc640 1 -- 172.21.15.5:0/3811457219 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f20180cf710 con 0x7f201c102e00 2024-09-12T02:56:56.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:56.712+0000 7f2020d34640 1 --2- 172.21.15.5:0/3811457219 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f20080a2c10 0x7f20080a50d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f201c10e9f0 tx=0x7f200c009110 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:56.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:56.712+0000 7f2021d36640 1 --2- 172.21.15.5:0/3811457219 >> [v2:172.21.15.146:6824/2041733103,v1:172.21.15.146:6825/2041733103] conn(0x7f20080ac7c0 0x7f20080aec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:56.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:56.712+0000 7f2021d36640 1 --2- 172.21.15.5:0/3811457219 >> [v2:172.21.15.146:6824/2041733103,v1:172.21.15.146:6825/2041733103] conn(0x7f20080ac7c0 0x7f20080aec60 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:56.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:56.713+0000 7f20127fc640 1 -- 172.21.15.5:0/3811457219 <== osd.10 v2:172.21.15.146:6824/2041733103 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f20080af330 con 0x7f20080ac7c0 2024-09-12T02:56:56.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:56.731+0000 7f2023799640 1 -- 172.21.15.5:0/3811457219 --> [v2:172.21.15.146:6824/2041733103,v1:172.21.15.146:6825/2041733103] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f201c06afb0 con 0x7f20080ac7c0 2024-09-12T02:56:56.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:56.732+0000 7f20127fc640 1 -- 172.21.15.5:0/3811457219 <== osd.10 v2:172.21.15.146:6824/2041733103 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f201c06afb0 con 0x7f20080ac7c0 2024-09-12T02:56:56.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:56.733+0000 7f2023799640 1 -- 172.21.15.5:0/3811457219 >> [v2:172.21.15.146:6824/2041733103,v1:172.21.15.146:6825/2041733103] conn(0x7f20080ac7c0 msgr2=0x7f20080aec60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:56.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:56.733+0000 7f2023799640 1 --2- 172.21.15.5:0/3811457219 >> [v2:172.21.15.146:6824/2041733103,v1:172.21.15.146:6825/2041733103] conn(0x7f20080ac7c0 0x7f20080aec60 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:56.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:56.733+0000 7f2023799640 1 -- 172.21.15.5:0/3811457219 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f20080a2c10 msgr2=0x7f20080a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:56.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:56.733+0000 7f2023799640 1 --2- 172.21.15.5:0/3811457219 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f20080a2c10 0x7f20080a50d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f201c10e9f0 tx=0x7f200c009110 comp rx=0 tx=0).stop 2024-09-12T02:56:56.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:56.733+0000 7f2023799640 1 -- 172.21.15.5:0/3811457219 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f201c102e00 msgr2=0x7f201c19a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:56.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:56.733+0000 7f2023799640 1 --2- 172.21.15.5:0/3811457219 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f201c102e00 0x7f201c19a6b0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f201800d950 tx=0x7f201800dd50 comp rx=0 tx=0).stop 2024-09-12T02:56:56.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:56.733+0000 7f2021d36640 1 -- 172.21.15.5:0/3811457219 reap_dead start 2024-09-12T02:56:56.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:56.733+0000 7f2023799640 1 -- 172.21.15.5:0/3811457219 shutdown_connections 2024-09-12T02:56:56.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:56.733+0000 7f2023799640 1 -- 172.21.15.5:0/3811457219 >> 172.21.15.5:0/3811457219 conn(0x7f201c0fc7a0 msgr2=0x7f201c100990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:56.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:56.733+0000 7f2023799640 1 -- 172.21.15.5:0/3811457219 shutdown_connections 2024-09-12T02:56:56.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:56.733+0000 7f2023799640 1 -- 172.21.15.5:0/3811457219 wait complete. 2024-09-12T02:56:56.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:56.627+0000 7fe930f2b640 1 -- 172.21.15.5:0/1194731607 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe92c1036e0 msgr2=0x7fe92c107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:56.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:56.627+0000 7fe930f2b640 1 --2- 172.21.15.5:0/1194731607 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe92c1036e0 0x7fe92c107b90 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe914009930 tx=0x7fe91402a420 comp rx=0 tx=0).stop 2024-09-12T02:56:56.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:56.628+0000 7fe930f2b640 1 -- 172.21.15.5:0/1194731607 shutdown_connections 2024-09-12T02:56:56.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:56.628+0000 7fe930f2b640 1 --2- 172.21.15.5:0/1194731607 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe92c1036e0 0x7fe92c107b90 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:56.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:56.628+0000 7fe930f2b640 1 --2- 172.21.15.5:0/1194731607 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe92c1082a0 0x7fe92c10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:56.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:56.628+0000 7fe930f2b640 1 --2- 172.21.15.5:0/1194731607 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe92c102d30 0x7fe92c103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:56.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:56.628+0000 7fe930f2b640 1 -- 172.21.15.5:0/1194731607 >> 172.21.15.5:0/1194731607 conn(0x7fe92c0fc7d0 msgr2=0x7fe92c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:56.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:56.628+0000 7fe930f2b640 1 -- 172.21.15.5:0/1194731607 shutdown_connections 2024-09-12T02:56:56.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:56.628+0000 7fe930f2b640 1 -- 172.21.15.5:0/1194731607 wait complete. 2024-09-12T02:56:56.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:56.628+0000 7fe930f2b640 1 Processor -- start 2024-09-12T02:56:56.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:56.628+0000 7fe930f2b640 1 -- start start 2024-09-12T02:56:56.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:56.629+0000 7fe930f2b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe92c102d30 0x7fe92c198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:56.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:56.629+0000 7fe930f2b640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe92c1036e0 0x7fe92c198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:56.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:56.629+0000 7fe930f2b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe92c1082a0 0x7fe92c19cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:56.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:56.629+0000 7fe930f2b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe92c1120a0 con 0x7fe92c102d30 2024-09-12T02:56:56.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:56.629+0000 7fe930f2b640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fe92c111f20 con 0x7fe92c1036e0 2024-09-12T02:56:56.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:56.629+0000 7fe930f2b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe92c112220 con 0x7fe92c1082a0 2024-09-12T02:56:56.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:56.629+0000 7fe92a59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe92c102d30 0x7fe92c198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:56.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:56.629+0000 7fe92a59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe92c102d30 0x7fe92c198640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58228/0 (socket says 172.21.15.5:58228) 2024-09-12T02:56:56.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:56.629+0000 7fe92a59c640 1 -- 172.21.15.5:0/2106726115 learned_addr learned my addr 172.21.15.5:0/2106726115 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:56:56.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:56.629+0000 7fe92ad9d640 1 --2- 172.21.15.5:0/2106726115 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe92c1082a0 0x7fe92c19cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:56.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:56.629+0000 7fe923fff640 1 --2- 172.21.15.5:0/2106726115 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe92c1036e0 0x7fe92c198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:56.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:56.629+0000 7fe92a59c640 1 -- 172.21.15.5:0/2106726115 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe92c1082a0 msgr2=0x7fe92c19cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:56.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:56.629+0000 7fe92a59c640 1 --2- 172.21.15.5:0/2106726115 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe92c1082a0 0x7fe92c19cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:56.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:56.629+0000 7fe92a59c640 1 -- 172.21.15.5:0/2106726115 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe92c1036e0 msgr2=0x7fe92c198b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:56.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:56.629+0000 7fe92a59c640 1 --2- 172.21.15.5:0/2106726115 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe92c1036e0 0x7fe92c198b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:56.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:56.629+0000 7fe92a59c640 1 -- 172.21.15.5:0/2106726115 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe92c19d5f0 con 0x7fe92c102d30 2024-09-12T02:56:56.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:56.629+0000 7fe92a59c640 1 --2- 172.21.15.5:0/2106726115 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe92c102d30 0x7fe92c198640 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe91c00d950 tx=0x7fe91c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:56.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:56.629+0000 7fe9237fe640 1 -- 172.21.15.5:0/2106726115 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe91c017070 con 0x7fe92c102d30 2024-09-12T02:56:56.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:56.630+0000 7fe930f2b640 1 -- 172.21.15.5:0/2106726115 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe92c19d8e0 con 0x7fe92c102d30 2024-09-12T02:56:56.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:56.630+0000 7fe9237fe640 1 -- 172.21.15.5:0/2106726115 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe91c009c30 con 0x7fe92c102d30 2024-09-12T02:56:56.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:56.630+0000 7fe9237fe640 1 -- 172.21.15.5:0/2106726115 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe91c00f6b0 con 0x7fe92c102d30 2024-09-12T02:56:56.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:56.630+0000 7fe930f2b640 1 -- 172.21.15.5:0/2106726115 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe92c1a51c0 con 0x7fe92c102d30 2024-09-12T02:56:56.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:56.631+0000 7fe930f2b640 1 -- 172.21.15.5:0/2106726115 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe92c062900 con 0x7fe92c102d30 2024-09-12T02:56:56.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:56.633+0000 7fe9237fe640 1 -- 172.21.15.5:0/2106726115 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fe91c0109d0 con 0x7fe92c102d30 2024-09-12T02:56:56.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:56.633+0000 7fe9237fe640 1 --2- 172.21.15.5:0/2106726115 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fe9180a2c10 0x7fe9180a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:56.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:56.633+0000 7fe923fff640 1 --2- 172.21.15.5:0/2106726115 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fe9180a2c10 0x7fe9180a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:56.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:56.634+0000 7fe9237fe640 1 -- 172.21.15.5:0/2106726115 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fe91c0cf0f0 con 0x7fe92c102d30 2024-09-12T02:56:56.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:56.634+0000 7fe9237fe640 1 --2- 172.21.15.5:0/2106726115 >> [v2:172.21.15.148:6800/3378174313,v1:172.21.15.148:6801/3378174313] conn(0x7fe9180ac7c0 0x7fe9180aec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:56.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:56.634+0000 7fe9237fe640 1 -- 172.21.15.5:0/2106726115 --> [v2:172.21.15.148:6800/3378174313,v1:172.21.15.148:6801/3378174313] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe9180af330 con 0x7fe9180ac7c0 2024-09-12T02:56:56.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:56.634+0000 7fe9237fe640 1 -- 172.21.15.5:0/2106726115 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7fe91c0cf540 con 0x7fe92c102d30 2024-09-12T02:56:56.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:56.634+0000 7fe923fff640 1 --2- 172.21.15.5:0/2106726115 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fe9180a2c10 0x7fe9180a50d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe914005940 tx=0x7fe9140058d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:56.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:56.634+0000 7fe92ad9d640 1 --2- 172.21.15.5:0/2106726115 >> [v2:172.21.15.148:6800/3378174313,v1:172.21.15.148:6801/3378174313] conn(0x7fe9180ac7c0 0x7fe9180aec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:56.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:56.634+0000 7fe92ad9d640 1 --2- 172.21.15.5:0/2106726115 >> [v2:172.21.15.148:6800/3378174313,v1:172.21.15.148:6801/3378174313] conn(0x7fe9180ac7c0 0x7fe9180aec60 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:56.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:56.635+0000 7fe9237fe640 1 -- 172.21.15.5:0/2106726115 <== osd.2 v2:172.21.15.148:6800/3378174313 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe9180af330 con 0x7fe9180ac7c0 2024-09-12T02:56:56.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:56.649+0000 7fe930f2b640 1 -- 172.21.15.5:0/2106726115 --> [v2:172.21.15.148:6800/3378174313,v1:172.21.15.148:6801/3378174313] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe92c062d00 con 0x7fe9180ac7c0 2024-09-12T02:56:56.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:56.649+0000 7fe9237fe640 1 -- 172.21.15.5:0/2106726115 <== osd.2 v2:172.21.15.148:6800/3378174313 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fe92c062d00 con 0x7fe9180ac7c0 2024-09-12T02:56:56.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:56.650+0000 7fe930f2b640 1 -- 172.21.15.5:0/2106726115 >> [v2:172.21.15.148:6800/3378174313,v1:172.21.15.148:6801/3378174313] conn(0x7fe9180ac7c0 msgr2=0x7fe9180aec60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:56.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:56.650+0000 7fe930f2b640 1 --2- 172.21.15.5:0/2106726115 >> [v2:172.21.15.148:6800/3378174313,v1:172.21.15.148:6801/3378174313] conn(0x7fe9180ac7c0 0x7fe9180aec60 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:56.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:56.650+0000 7fe930f2b640 1 -- 172.21.15.5:0/2106726115 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fe9180a2c10 msgr2=0x7fe9180a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:56.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:56.650+0000 7fe930f2b640 1 --2- 172.21.15.5:0/2106726115 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fe9180a2c10 0x7fe9180a50d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe914005940 tx=0x7fe9140058d0 comp rx=0 tx=0).stop 2024-09-12T02:56:56.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:56.650+0000 7fe930f2b640 1 -- 172.21.15.5:0/2106726115 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe92c102d30 msgr2=0x7fe92c198640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:56.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:56.650+0000 7fe930f2b640 1 --2- 172.21.15.5:0/2106726115 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe92c102d30 0x7fe92c198640 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe91c00d950 tx=0x7fe91c00de10 comp rx=0 tx=0).stop 2024-09-12T02:56:56.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:56.650+0000 7fe92ad9d640 1 -- 172.21.15.5:0/2106726115 reap_dead start 2024-09-12T02:56:56.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:56.650+0000 7fe930f2b640 1 -- 172.21.15.5:0/2106726115 shutdown_connections 2024-09-12T02:56:56.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:56.650+0000 7fe930f2b640 1 -- 172.21.15.5:0/2106726115 >> 172.21.15.5:0/2106726115 conn(0x7fe92c0fc7d0 msgr2=0x7fe92c106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:56.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:56.650+0000 7fe930f2b640 1 -- 172.21.15.5:0/2106726115 shutdown_connections 2024-09-12T02:56:56.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:56.650+0000 7fe930f2b640 1 -- 172.21.15.5:0/2106726115 wait complete. 2024-09-12T02:56:57.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:57 smithi148 ceph-mon[35090]: pgmap v296: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-12T02:56:57.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:57 smithi146 ceph-mon[35011]: pgmap v296: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-12T02:56:57.476 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:57 smithi005 ceph-mon[31197]: pgmap v296: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 255 B/s wr, 10 op/s 2024-09-12T02:56:57.507 INFO:teuthology.orchestra.run.smithi005.stdout:184683593773 2024-09-12T02:56:57.508 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-09-12T02:56:57.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:57.767+0000 7f9f15054640 1 -- 172.21.15.5:0/3440968579 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9f10102570 msgr2=0x7f9f101029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:57.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:57.767+0000 7f9f15054640 1 --2- 172.21.15.5:0/3440968579 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9f10102570 0x7f9f101029b0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f9ef8009900 tx=0x7f9ef802a480 comp rx=0 tx=0).stop 2024-09-12T02:56:57.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:57.767+0000 7f9f15054640 1 -- 172.21.15.5:0/3440968579 shutdown_connections 2024-09-12T02:56:57.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:57.767+0000 7f9f15054640 1 --2- 172.21.15.5:0/3440968579 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9f10102570 0x7f9f101029b0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:57.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:57.767+0000 7f9f15054640 1 --2- 172.21.15.5:0/3440968579 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f10102ef0 0x7f9f1010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:57.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:57.767+0000 7f9f15054640 1 --2- 172.21.15.5:0/3440968579 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9f10100e20 0x7f9f10101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:57.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:57.767+0000 7f9f15054640 1 -- 172.21.15.5:0/3440968579 >> 172.21.15.5:0/3440968579 conn(0x7f9f100fc7d0 msgr2=0x7f9f100febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:57.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:57.767+0000 7f9f15054640 1 -- 172.21.15.5:0/3440968579 shutdown_connections 2024-09-12T02:56:57.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:57.767+0000 7f9f15054640 1 -- 172.21.15.5:0/3440968579 wait complete. 2024-09-12T02:56:57.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:57.768+0000 7f9f15054640 1 Processor -- start 2024-09-12T02:56:57.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:57.768+0000 7f9f15054640 1 -- start start 2024-09-12T02:56:57.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:57.769+0000 7f9f15054640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f10100e20 0x7f9f1019a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:57.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:57.769+0000 7f9f15054640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9f10102570 0x7f9f1019ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:57.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:57.769+0000 7f9f15054640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9f10102ef0 0x7f9f1019efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:57.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:57.769+0000 7f9f15054640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9f10110fa0 con 0x7f9f10102570 2024-09-12T02:56:57.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:57.769+0000 7f9f15054640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f9f10110e20 con 0x7f9f10102ef0 2024-09-12T02:56:57.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:57.769+0000 7f9f15054640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f9f10111120 con 0x7f9f10100e20 2024-09-12T02:56:57.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:57.769+0000 7f9f0e59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9f10102570 0x7f9f1019ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:57.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:57.769+0000 7f9f0e59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9f10102570 0x7f9f1019ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58258/0 (socket says 172.21.15.5:58258) 2024-09-12T02:56:57.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:57.769+0000 7f9f0e59c640 1 -- 172.21.15.5:0/996863658 learned_addr learned my addr 172.21.15.5:0/996863658 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:56:57.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:57.769+0000 7f9f0f59e640 1 --2- 172.21.15.5:0/996863658 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9f10102ef0 0x7f9f1019efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:57.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:57.769+0000 7f9f0ed9d640 1 --2- 172.21.15.5:0/996863658 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f10100e20 0x7f9f1019a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:57.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:57.769+0000 7f9f0e59c640 1 -- 172.21.15.5:0/996863658 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f10100e20 msgr2=0x7f9f1019a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:57.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:57.769+0000 7f9f0e59c640 1 --2- 172.21.15.5:0/996863658 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f10100e20 0x7f9f1019a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:57.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:57.769+0000 7f9f0e59c640 1 -- 172.21.15.5:0/996863658 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9f10102ef0 msgr2=0x7f9f1019efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:57.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:57.769+0000 7f9f0e59c640 1 --2- 172.21.15.5:0/996863658 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9f10102ef0 0x7f9f1019efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:57.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:57.769+0000 7f9f0e59c640 1 -- 172.21.15.5:0/996863658 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f1019f6c0 con 0x7f9f10102570 2024-09-12T02:56:57.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:57.769+0000 7f9f0ed9d640 1 --2- 172.21.15.5:0/996863658 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9f10100e20 0x7f9f1019a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:56:57.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:57.769+0000 7f9f0e59c640 1 --2- 172.21.15.5:0/996863658 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9f10102570 0x7f9f1019ac50 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f9ef8006010 tx=0x7f9ef8003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:57.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:57.769+0000 7f9f07fff640 1 -- 172.21.15.5:0/996863658 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ef803f070 con 0x7f9f10102570 2024-09-12T02:56:57.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:57.770+0000 7f9f15054640 1 -- 172.21.15.5:0/996863658 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f1019f950 con 0x7f9f10102570 2024-09-12T02:56:57.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:57.770+0000 7f9f07fff640 1 -- 172.21.15.5:0/996863658 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ef8031e30 con 0x7f9f10102570 2024-09-12T02:56:57.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:57.770+0000 7f9f07fff640 1 -- 172.21.15.5:0/996863658 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ef8035030 con 0x7f9f10102570 2024-09-12T02:56:57.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:57.770+0000 7f9f15054640 1 -- 172.21.15.5:0/996863658 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f101a7290 con 0x7f9f10102570 2024-09-12T02:56:57.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:57.770+0000 7f9f15054640 1 -- 172.21.15.5:0/996863658 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9f1010d0f0 con 0x7f9f10102570 2024-09-12T02:56:57.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:57.772+0000 7f9f07fff640 1 -- 172.21.15.5:0/996863658 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f9ef8042070 con 0x7f9f10102570 2024-09-12T02:56:57.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:57.772+0000 7f9f07fff640 1 --2- 172.21.15.5:0/996863658 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f9efc0a2c10 0x7f9efc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:57.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:57.773+0000 7f9f0ed9d640 1 --2- 172.21.15.5:0/996863658 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f9efc0a2c10 0x7f9efc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:57.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:57.773+0000 7f9f07fff640 1 -- 172.21.15.5:0/996863658 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f9ef80f0110 con 0x7f9f10102570 2024-09-12T02:56:57.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:57.773+0000 7f9f07fff640 1 --2- 172.21.15.5:0/996863658 >> [v2:172.21.15.148:6816/651045756,v1:172.21.15.148:6817/651045756] conn(0x7f9efc0ac690 0x7f9efc0aeb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:57.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:57.773+0000 7f9f07fff640 1 -- 172.21.15.5:0/996863658 --> [v2:172.21.15.148:6816/651045756,v1:172.21.15.148:6817/651045756] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9ef8038e90 con 0x7f9efc0ac690 2024-09-12T02:56:57.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:57.773+0000 7f9f07fff640 1 -- 172.21.15.5:0/996863658 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f9ef80f0560 con 0x7f9f10102570 2024-09-12T02:56:57.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:57.773+0000 7f9f0ed9d640 1 --2- 172.21.15.5:0/996863658 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f9efc0a2c10 0x7f9efc0a50d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9f080058c0 tx=0x7f9f08005ec0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:57.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:57.773+0000 7f9f0f59e640 1 --2- 172.21.15.5:0/996863658 >> [v2:172.21.15.148:6816/651045756,v1:172.21.15.148:6817/651045756] conn(0x7f9efc0ac690 0x7f9efc0aeb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:57.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:57.774+0000 7f9f0f59e640 1 --2- 172.21.15.5:0/996863658 >> [v2:172.21.15.148:6816/651045756,v1:172.21.15.148:6817/651045756] conn(0x7f9efc0ac690 0x7f9efc0aeb30 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:57.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:57.775+0000 7f9f07fff640 1 -- 172.21.15.5:0/996863658 <== osd.8 v2:172.21.15.148:6816/651045756 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9ef8038e90 con 0x7f9efc0ac690 2024-09-12T02:56:57.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:57.788+0000 7f9f15054640 1 -- 172.21.15.5:0/996863658 --> [v2:172.21.15.148:6816/651045756,v1:172.21.15.148:6817/651045756] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9f101021b0 con 0x7f9efc0ac690 2024-09-12T02:56:57.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:57.789+0000 7f9f07fff640 1 -- 172.21.15.5:0/996863658 <== osd.8 v2:172.21.15.148:6816/651045756 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f9f101021b0 con 0x7f9efc0ac690 2024-09-12T02:56:57.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:57.789+0000 7f9f15054640 1 -- 172.21.15.5:0/996863658 >> [v2:172.21.15.148:6816/651045756,v1:172.21.15.148:6817/651045756] conn(0x7f9efc0ac690 msgr2=0x7f9efc0aeb30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:57.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:57.789+0000 7f9f15054640 1 --2- 172.21.15.5:0/996863658 >> [v2:172.21.15.148:6816/651045756,v1:172.21.15.148:6817/651045756] conn(0x7f9efc0ac690 0x7f9efc0aeb30 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:57.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:57.789+0000 7f9f15054640 1 -- 172.21.15.5:0/996863658 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f9efc0a2c10 msgr2=0x7f9efc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:57.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:57.789+0000 7f9f15054640 1 --2- 172.21.15.5:0/996863658 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f9efc0a2c10 0x7f9efc0a50d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9f080058c0 tx=0x7f9f08005ec0 comp rx=0 tx=0).stop 2024-09-12T02:56:57.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:57.789+0000 7f9f15054640 1 -- 172.21.15.5:0/996863658 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9f10102570 msgr2=0x7f9f1019ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:57.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:57.789+0000 7f9f15054640 1 --2- 172.21.15.5:0/996863658 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9f10102570 0x7f9f1019ac50 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f9ef8006010 tx=0x7f9ef8003c80 comp rx=0 tx=0).stop 2024-09-12T02:56:57.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:57.789+0000 7f9f0f59e640 1 -- 172.21.15.5:0/996863658 reap_dead start 2024-09-12T02:56:57.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:57.789+0000 7f9f15054640 1 -- 172.21.15.5:0/996863658 shutdown_connections 2024-09-12T02:56:57.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:57.789+0000 7f9f15054640 1 -- 172.21.15.5:0/996863658 >> 172.21.15.5:0/996863658 conn(0x7f9f100fc7d0 msgr2=0x7f9f1006a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:57.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:57.789+0000 7f9f15054640 1 -- 172.21.15.5:0/996863658 shutdown_connections 2024-09-12T02:56:57.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:57.789+0000 7f9f15054640 1 -- 172.21.15.5:0/996863658 wait complete. 2024-09-12T02:56:58.327 INFO:teuthology.orchestra.run.smithi005.stdout:240518168612 2024-09-12T02:56:58.328 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-09-12T02:56:58.841 INFO:teuthology.orchestra.run.smithi005.stdout:55834574917 2024-09-12T02:56:58.841 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-12T02:56:59.365 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:56:59 smithi005 ceph-mon[31197]: pgmap v297: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-12T02:56:59.400 INFO:teuthology.orchestra.run.smithi005.stdout:206158430248 2024-09-12T02:56:59.401 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-09-12T02:56:59.415 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:56:59 smithi148 ceph-mon[35090]: pgmap v297: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-12T02:56:59.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:56:59 smithi146 ceph-mon[35011]: pgmap v297: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-12T02:56:59.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:59.674+0000 7f1fab46f640 1 -- 172.21.15.5:0/55418548 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1fa4101f80 msgr2=0x7f1fa410d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:59.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:59.674+0000 7f1fab46f640 1 --2- 172.21.15.5:0/55418548 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1fa4101f80 0x7f1fa410d9b0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c009ca0 tx=0x7f1f9c02a8c0 comp rx=0 tx=0).stop 2024-09-12T02:56:59.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:59.674+0000 7f1fab46f640 1 -- 172.21.15.5:0/55418548 shutdown_connections 2024-09-12T02:56:59.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:59.674+0000 7f1fab46f640 1 --2- 172.21.15.5:0/55418548 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1fa4101f80 0x7f1fa410d9b0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:59.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:59.674+0000 7f1fab46f640 1 --2- 172.21.15.5:0/55418548 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1fa41015e0 0x7f1fa4101a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:59.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:59.674+0000 7f1fab46f640 1 --2- 172.21.15.5:0/55418548 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1fa4106d00 0x7f1fa41070e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:59.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:59.674+0000 7f1fab46f640 1 -- 172.21.15.5:0/55418548 >> 172.21.15.5:0/55418548 conn(0x7f1fa40fba90 msgr2=0x7f1fa40fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:59.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:59.674+0000 7f1fab46f640 1 -- 172.21.15.5:0/55418548 shutdown_connections 2024-09-12T02:56:59.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:59.674+0000 7f1fab46f640 1 -- 172.21.15.5:0/55418548 wait complete. 2024-09-12T02:56:59.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:59.675+0000 7f1fab46f640 1 Processor -- start 2024-09-12T02:56:59.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:59.675+0000 7f1fab46f640 1 -- start start 2024-09-12T02:56:59.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:59.675+0000 7f1fab46f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1fa41015e0 0x7f1fa41984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:59.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:59.675+0000 7f1fab46f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1fa4101f80 0x7f1fa41989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:59.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:59.675+0000 7f1fab46f640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1fa4106d00 0x7f1fa419cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:59.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:59.675+0000 7f1fab46f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1fa4110f50 con 0x7f1fa4101f80 2024-09-12T02:56:59.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:59.675+0000 7f1fab46f640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f1fa4110dd0 con 0x7f1fa4106d00 2024-09-12T02:56:59.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:59.675+0000 7f1fab46f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1fa41110d0 con 0x7f1fa41015e0 2024-09-12T02:56:59.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:59.675+0000 7f1fa8a0a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1fa4101f80 0x7f1fa41989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:59.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:59.675+0000 7f1fa8a0a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1fa4101f80 0x7f1fa41989e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58278/0 (socket says 172.21.15.5:58278) 2024-09-12T02:56:59.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:59.675+0000 7f1fa8a0a640 1 -- 172.21.15.5:0/35805040 learned_addr learned my addr 172.21.15.5:0/35805040 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:56:59.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:59.676+0000 7f1fa920b640 1 --2- 172.21.15.5:0/35805040 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1fa41015e0 0x7f1fa41984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:59.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:59.676+0000 7f1fa9a0c640 1 --2- 172.21.15.5:0/35805040 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1fa4106d00 0x7f1fa419cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:59.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:59.676+0000 7f1fa8a0a640 1 -- 172.21.15.5:0/35805040 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1fa41015e0 msgr2=0x7f1fa41984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:59.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:59.676+0000 7f1fa8a0a640 1 --2- 172.21.15.5:0/35805040 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1fa41015e0 0x7f1fa41984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:59.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:59.676+0000 7f1fa8a0a640 1 -- 172.21.15.5:0/35805040 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1fa4106d00 msgr2=0x7f1fa419cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:59.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:59.676+0000 7f1fa8a0a640 1 --2- 172.21.15.5:0/35805040 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1fa4106d00 0x7f1fa419cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:59.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:59.676+0000 7f1fa8a0a640 1 -- 172.21.15.5:0/35805040 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fa419d450 con 0x7f1fa4101f80 2024-09-12T02:56:59.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:59.676+0000 7f1fa9a0c640 1 --2- 172.21.15.5:0/35805040 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1fa4106d00 0x7f1fa419cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:56:59.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:59.676+0000 7f1fa8a0a640 1 --2- 172.21.15.5:0/35805040 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1fa4101f80 0x7f1fa41989e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f1f9400d950 tx=0x7f1f9400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:59.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:59.676+0000 7f1f9a7fc640 1 -- 172.21.15.5:0/35805040 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f94017070 con 0x7f1fa4101f80 2024-09-12T02:56:59.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:59.676+0000 7f1fab46f640 1 -- 172.21.15.5:0/35805040 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fa419d740 con 0x7f1fa4101f80 2024-09-12T02:56:59.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:59.676+0000 7f1f9a7fc640 1 -- 172.21.15.5:0/35805040 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f94009c30 con 0x7f1fa4101f80 2024-09-12T02:56:59.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:59.676+0000 7f1f9a7fc640 1 -- 172.21.15.5:0/35805040 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f9400f6b0 con 0x7f1fa4101f80 2024-09-12T02:56:59.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:59.676+0000 7f1fab46f640 1 -- 172.21.15.5:0/35805040 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fa41a5020 con 0x7f1fa4101f80 2024-09-12T02:56:59.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:59.677+0000 7f1fab46f640 1 -- 172.21.15.5:0/35805040 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1fa4100f70 con 0x7f1fa4101f80 2024-09-12T02:56:59.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:59.678+0000 7f1f9a7fc640 1 -- 172.21.15.5:0/35805040 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f1f940109d0 con 0x7f1fa4101f80 2024-09-12T02:56:59.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:59.679+0000 7f1f9a7fc640 1 --2- 172.21.15.5:0/35805040 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f1f900a2c10 0x7f1f900a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:59.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:59.679+0000 7f1fa920b640 1 --2- 172.21.15.5:0/35805040 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f1f900a2c10 0x7f1f900a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:59.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:59.679+0000 7f1f9a7fc640 1 -- 172.21.15.5:0/35805040 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f1f940cf1a0 con 0x7f1fa4101f80 2024-09-12T02:56:59.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:59.679+0000 7f1f9a7fc640 1 --2- 172.21.15.5:0/35805040 >> [v2:172.21.15.5:6810/11741938,v1:172.21.15.5:6811/11741938] conn(0x7f1f900ac7c0 0x7f1f900aec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:56:59.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:59.679+0000 7f1f9a7fc640 1 -- 172.21.15.5:0/35805040 --> [v2:172.21.15.5:6810/11741938,v1:172.21.15.5:6811/11741938] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1f900af330 con 0x7f1f900ac7c0 2024-09-12T02:56:59.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:59.679+0000 7f1f9a7fc640 1 -- 172.21.15.5:0/35805040 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f1f940cf5f0 con 0x7f1fa4101f80 2024-09-12T02:56:59.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:59.680+0000 7f1fa9a0c640 1 --2- 172.21.15.5:0/35805040 >> [v2:172.21.15.5:6810/11741938,v1:172.21.15.5:6811/11741938] conn(0x7f1f900ac7c0 0x7f1f900aec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:56:59.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:59.680+0000 7f1fa920b640 1 --2- 172.21.15.5:0/35805040 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f1f900a2c10 0x7f1f900a50d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f1fa000bbe0 tx=0x7f1fa000b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:59.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:59.680+0000 7f1fa9a0c640 1 --2- 172.21.15.5:0/35805040 >> [v2:172.21.15.5:6810/11741938,v1:172.21.15.5:6811/11741938] conn(0x7f1f900ac7c0 0x7f1f900aec60 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:56:59.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:59.681+0000 7f1f9a7fc640 1 -- 172.21.15.5:0/35805040 <== osd.3 v2:172.21.15.5:6810/11741938 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1f900af330 con 0x7f1f900ac7c0 2024-09-12T02:56:59.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:59.694+0000 7f1fab46f640 1 -- 172.21.15.5:0/35805040 --> [v2:172.21.15.5:6810/11741938,v1:172.21.15.5:6811/11741938] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1fa4100070 con 0x7f1f900ac7c0 2024-09-12T02:56:59.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:59.695+0000 7f1f9a7fc640 1 -- 172.21.15.5:0/35805040 <== osd.3 v2:172.21.15.5:6810/11741938 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f1fa4100070 con 0x7f1f900ac7c0 2024-09-12T02:56:59.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:59.695+0000 7f1fab46f640 1 -- 172.21.15.5:0/35805040 >> [v2:172.21.15.5:6810/11741938,v1:172.21.15.5:6811/11741938] conn(0x7f1f900ac7c0 msgr2=0x7f1f900aec60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:59.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:59.695+0000 7f1fab46f640 1 --2- 172.21.15.5:0/35805040 >> [v2:172.21.15.5:6810/11741938,v1:172.21.15.5:6811/11741938] conn(0x7f1f900ac7c0 0x7f1f900aec60 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:56:59.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:59.695+0000 7f1fab46f640 1 -- 172.21.15.5:0/35805040 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f1f900a2c10 msgr2=0x7f1f900a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:59.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:59.695+0000 7f1fab46f640 1 --2- 172.21.15.5:0/35805040 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f1f900a2c10 0x7f1f900a50d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f1fa000bbe0 tx=0x7f1fa000b390 comp rx=0 tx=0).stop 2024-09-12T02:56:59.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:59.695+0000 7f1fab46f640 1 -- 172.21.15.5:0/35805040 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1fa4101f80 msgr2=0x7f1fa41989e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:56:59.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:59.695+0000 7f1fab46f640 1 --2- 172.21.15.5:0/35805040 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1fa4101f80 0x7f1fa41989e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f1f9400d950 tx=0x7f1f9400de10 comp rx=0 tx=0).stop 2024-09-12T02:56:59.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:59.695+0000 7f1fa9a0c640 1 -- 172.21.15.5:0/35805040 reap_dead start 2024-09-12T02:56:59.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:59.695+0000 7f1fab46f640 1 -- 172.21.15.5:0/35805040 shutdown_connections 2024-09-12T02:56:59.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:59.695+0000 7f1fab46f640 1 -- 172.21.15.5:0/35805040 >> 172.21.15.5:0/35805040 conn(0x7f1fa40fba90 msgr2=0x7f1fa4100590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:56:59.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:59.695+0000 7f1fab46f640 1 -- 172.21.15.5:0/35805040 shutdown_connections 2024-09-12T02:56:59.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:56:59.695+0000 7f1fab46f640 1 -- 172.21.15.5:0/35805040 wait complete. 2024-09-12T02:57:00.442 INFO:teuthology.orchestra.run.smithi005.stdout:107374182460 2024-09-12T02:57:00.443 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-09-12T02:57:00.751 INFO:teuthology.orchestra.run.smithi005.stdout:322122547221 2024-09-12T02:57:00.751 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-09-12T02:57:01.151 INFO:teuthology.orchestra.run.smithi005.stdout:77309411393 2024-09-12T02:57:01.151 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-12T02:57:01.178 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:57:01.178 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:57:01.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:57:01 smithi148 ceph-mon[35090]: pgmap v298: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-12T02:57:01.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:57:01 smithi146 ceph-mon[35011]: pgmap v298: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-12T02:57:01.463 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:57:01 smithi005 ceph-mon[31197]: pgmap v298: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 255 B/s wr, 11 op/s 2024-09-12T02:57:01.791 INFO:teuthology.orchestra.run.smithi005.stdout:292057776153 2024-09-12T02:57:01.792 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-09-12T02:57:02.320 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:57:02.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:02.334+0000 7f2eec3a2640 1 -- 172.21.15.5:0/167448982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ee41027a0 msgr2=0x7f2ee41067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:02.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:02.334+0000 7f2eec3a2640 1 --2- 172.21.15.5:0/167448982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ee41027a0 0x7f2ee41067f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f2ed4009900 tx=0x7f2ed402a410 comp rx=0 tx=0).stop 2024-09-12T02:57:02.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:02.334+0000 7f2eec3a2640 1 -- 172.21.15.5:0/167448982 shutdown_connections 2024-09-12T02:57:02.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:02.334+0000 7f2eec3a2640 1 --2- 172.21.15.5:0/167448982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ee41027a0 0x7f2ee41067f0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:02.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:02.334+0000 7f2eec3a2640 1 --2- 172.21.15.5:0/167448982 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ee4106f20 0x7f2ee410eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:02.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:02.334+0000 7f2eec3a2640 1 --2- 172.21.15.5:0/167448982 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ee4101df0 0x7f2ee41021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:02.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:02.334+0000 7f2eec3a2640 1 -- 172.21.15.5:0/167448982 >> 172.21.15.5:0/167448982 conn(0x7f2ee40fd8f0 msgr2=0x7f2ee40ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:02.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:02.334+0000 7f2eec3a2640 1 -- 172.21.15.5:0/167448982 shutdown_connections 2024-09-12T02:57:02.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:02.334+0000 7f2eec3a2640 1 -- 172.21.15.5:0/167448982 wait complete. 2024-09-12T02:57:02.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:02.335+0000 7f2eec3a2640 1 Processor -- start 2024-09-12T02:57:02.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:02.335+0000 7f2eec3a2640 1 -- start start 2024-09-12T02:57:02.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:02.335+0000 7f2eec3a2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ee4101df0 0x7f2ee40744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:02.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:02.335+0000 7f2eec3a2640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ee41027a0 0x7f2ee4074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:02.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:02.335+0000 7f2eec3a2640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ee4106f20 0x7f2ee4074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:02.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:02.335+0000 7f2eec3a2640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee4111f80 con 0x7f2ee4101df0 2024-09-12T02:57:02.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:02.335+0000 7f2eec3a2640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee4111e00 con 0x7f2ee4106f20 2024-09-12T02:57:02.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:02.335+0000 7f2eec3a2640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee4112100 con 0x7f2ee41027a0 2024-09-12T02:57:02.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:02.335+0000 7f2eea13e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ee4101df0 0x7f2ee40744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:02.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:02.335+0000 7f2eea13e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ee4101df0 0x7f2ee40744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58286/0 (socket says 172.21.15.5:58286) 2024-09-12T02:57:02.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:02.335+0000 7f2eea13e640 1 -- 172.21.15.5:0/3042696165 learned_addr learned my addr 172.21.15.5:0/3042696165 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:57:02.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:02.335+0000 7f2ee993d640 1 --2- 172.21.15.5:0/3042696165 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ee41027a0 0x7f2ee4074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:02.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:02.336+0000 7f2eea93f640 1 --2- 172.21.15.5:0/3042696165 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ee4106f20 0x7f2ee4074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:02.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:02.336+0000 7f2ee993d640 1 -- 172.21.15.5:0/3042696165 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ee4106f20 msgr2=0x7f2ee4074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:02.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:02.336+0000 7f2ee993d640 1 --2- 172.21.15.5:0/3042696165 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ee4106f20 0x7f2ee4074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:02.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:02.336+0000 7f2ee993d640 1 -- 172.21.15.5:0/3042696165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ee4101df0 msgr2=0x7f2ee40744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:02.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:02.336+0000 7f2ee993d640 1 --2- 172.21.15.5:0/3042696165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ee4101df0 0x7f2ee40744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:02.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:02.336+0000 7f2ee993d640 1 -- 172.21.15.5:0/3042696165 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ee41a68e0 con 0x7f2ee41027a0 2024-09-12T02:57:02.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:02.336+0000 7f2ee993d640 1 --2- 172.21.15.5:0/3042696165 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ee41027a0 0x7f2ee4074a30 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2ed4005940 tx=0x7f2ed4004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:02.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:02.336+0000 7f2edb7fe640 1 -- 172.21.15.5:0/3042696165 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed403f030 con 0x7f2ee41027a0 2024-09-12T02:57:02.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:02.337+0000 7f2eec3a2640 1 -- 172.21.15.5:0/3042696165 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ee41a6b70 con 0x7f2ee41027a0 2024-09-12T02:57:02.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:02.337+0000 7f2edb7fe640 1 -- 172.21.15.5:0/3042696165 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ed4003ad0 con 0x7f2ee41027a0 2024-09-12T02:57:02.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:02.337+0000 7f2edb7fe640 1 -- 172.21.15.5:0/3042696165 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed4035070 con 0x7f2ee41027a0 2024-09-12T02:57:02.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:02.337+0000 7f2eec3a2640 1 -- 172.21.15.5:0/3042696165 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ee41a7050 con 0x7f2ee41027a0 2024-09-12T02:57:02.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:02.338+0000 7f2eec3a2640 1 -- 172.21.15.5:0/3042696165 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ee41021d0 con 0x7f2ee41027a0 2024-09-12T02:57:02.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:02.340+0000 7f2edb7fe640 1 -- 172.21.15.5:0/3042696165 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f2ed4042030 con 0x7f2ee41027a0 2024-09-12T02:57:02.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:02.341+0000 7f2edb7fe640 1 --2- 172.21.15.5:0/3042696165 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f2ed00a2c90 0x7f2ed00a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:02.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:02.341+0000 7f2edb7fe640 1 -- 172.21.15.5:0/3042696165 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f2ed40ef9f0 con 0x7f2ee41027a0 2024-09-12T02:57:02.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:02.341+0000 7f2eea13e640 1 --2- 172.21.15.5:0/3042696165 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f2ed00a2c90 0x7f2ed00a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:02.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:02.341+0000 7f2eea13e640 1 --2- 172.21.15.5:0/3042696165 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f2ed00a2c90 0x7f2ed00a5150 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2ee0009c90 tx=0x7f2ee0005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:02.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:02.343+0000 7f2edb7fe640 1 -- 172.21.15.5:0/3042696165 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2ed40bc150 con 0x7f2ee41027a0 2024-09-12T02:57:02.362 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:57:02.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:02.524+0000 7f2eec3a2640 1 -- 172.21.15.5:0/3042696165 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f2ee4109bd0 con 0x7f2ee41027a0 2024-09-12T02:57:02.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:02.524+0000 7f2edb7fe640 1 -- 172.21.15.5:0/3042696165 <== mon.1 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f2ed40c0450 con 0x7f2ee41027a0 2024-09-12T02:57:02.525 INFO:teuthology.orchestra.run.smithi005.stdout:137438953529 2024-09-12T02:57:02.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:02.526+0000 7f2eec3a2640 1 -- 172.21.15.5:0/3042696165 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f2ed00a2c90 msgr2=0x7f2ed00a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:02.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:02.526+0000 7f2eec3a2640 1 --2- 172.21.15.5:0/3042696165 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f2ed00a2c90 0x7f2ed00a5150 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2ee0009c90 tx=0x7f2ee0005f90 comp rx=0 tx=0).stop 2024-09-12T02:57:02.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:02.526+0000 7f2eec3a2640 1 -- 172.21.15.5:0/3042696165 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ee41027a0 msgr2=0x7f2ee4074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:02.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:02.526+0000 7f2eec3a2640 1 --2- 172.21.15.5:0/3042696165 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ee41027a0 0x7f2ee4074a30 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2ed4005940 tx=0x7f2ed4004270 comp rx=0 tx=0).stop 2024-09-12T02:57:02.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:02.526+0000 7f2eec3a2640 1 -- 172.21.15.5:0/3042696165 shutdown_connections 2024-09-12T02:57:02.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:02.526+0000 7f2eec3a2640 1 --2- 172.21.15.5:0/3042696165 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ee4106f20 0x7f2ee4074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:02.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:02.526+0000 7f2eec3a2640 1 --2- 172.21.15.5:0/3042696165 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ee41027a0 0x7f2ee4074a30 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:02.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:02.526+0000 7f2eec3a2640 1 --2- 172.21.15.5:0/3042696165 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f2ed00a2c90 0x7f2ed00a5150 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:02.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:02.526+0000 7f2eec3a2640 1 --2- 172.21.15.5:0/3042696165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ee4101df0 0x7f2ee40744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:02.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:02.526+0000 7f2eec3a2640 1 -- 172.21.15.5:0/3042696165 >> 172.21.15.5:0/3042696165 conn(0x7f2ee40fd8f0 msgr2=0x7f2ee40fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:02.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:02.527+0000 7f2eec3a2640 1 -- 172.21.15.5:0/3042696165 shutdown_connections 2024-09-12T02:57:02.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:02.527+0000 7f2eec3a2640 1 -- 172.21.15.5:0/3042696165 wait complete. 2024-09-12T02:57:03.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:03.077+0000 7ff99d268640 1 -- 172.21.15.5:0/2158602634 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff998106f10 msgr2=0x7ff99810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:03.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:03.077+0000 7ff99d268640 1 --2- 172.21.15.5:0/2158602634 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff998106f10 0x7ff99810ea90 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7ff990009ca0 tx=0x7ff99002a8c0 comp rx=0 tx=0).stop 2024-09-12T02:57:03.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:03.077+0000 7ff99d268640 1 -- 172.21.15.5:0/2158602634 shutdown_connections 2024-09-12T02:57:03.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:03.077+0000 7ff99d268640 1 --2- 172.21.15.5:0/2158602634 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff998106f10 0x7ff99810ea90 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:03.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:03.077+0000 7ff99d268640 1 --2- 172.21.15.5:0/2158602634 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff998102790 0x7ff9981067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:03.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:03.077+0000 7ff99d268640 1 --2- 172.21.15.5:0/2158602634 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff998101de0 0x7ff9981021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:03.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:03.077+0000 7ff99d268640 1 -- 172.21.15.5:0/2158602634 >> 172.21.15.5:0/2158602634 conn(0x7ff9980fd8c0 msgr2=0x7ff9980ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:03.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:03.077+0000 7ff99d268640 1 -- 172.21.15.5:0/2158602634 shutdown_connections 2024-09-12T02:57:03.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:03.077+0000 7ff99d268640 1 -- 172.21.15.5:0/2158602634 wait complete. 2024-09-12T02:57:03.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:03.078+0000 7ff99d268640 1 Processor -- start 2024-09-12T02:57:03.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:03.078+0000 7ff99d268640 1 -- start start 2024-09-12T02:57:03.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:03.078+0000 7ff99d268640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff998101de0 0x7ff998198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:03.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:03.078+0000 7ff99d268640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff998102790 0x7ff998198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:03.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:03.078+0000 7ff99d268640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff998106f10 0x7ff99819cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:03.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:03.078+0000 7ff99d268640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff998111f90 con 0x7ff998102790 2024-09-12T02:57:03.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:03.078+0000 7ff99d268640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ff998111e10 con 0x7ff998101de0 2024-09-12T02:57:03.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:03.078+0000 7ff99d268640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff998112110 con 0x7ff998106f10 2024-09-12T02:57:03.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:03.078+0000 7ff9967fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff998102790 0x7ff998198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:03.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:03.078+0000 7ff9967fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff998102790 0x7ff998198950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58316/0 (socket says 172.21.15.5:58316) 2024-09-12T02:57:03.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:03.078+0000 7ff9967fc640 1 -- 172.21.15.5:0/2981802763 learned_addr learned my addr 172.21.15.5:0/2981802763 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:57:03.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:03.079+0000 7ff996ffd640 1 --2- 172.21.15.5:0/2981802763 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff998101de0 0x7ff998198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:03.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:03.079+0000 7ff9977fe640 1 --2- 172.21.15.5:0/2981802763 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff998106f10 0x7ff99819cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:03.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:03.079+0000 7ff9967fc640 1 -- 172.21.15.5:0/2981802763 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff998106f10 msgr2=0x7ff99819cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:03.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:03.079+0000 7ff9967fc640 1 --2- 172.21.15.5:0/2981802763 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff998106f10 0x7ff99819cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:03.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:03.079+0000 7ff9967fc640 1 -- 172.21.15.5:0/2981802763 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff998101de0 msgr2=0x7ff998198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:03.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:03.079+0000 7ff9967fc640 1 --2- 172.21.15.5:0/2981802763 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff998101de0 0x7ff998198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:03.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:03.079+0000 7ff9967fc640 1 -- 172.21.15.5:0/2981802763 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff99819d3c0 con 0x7ff998102790 2024-09-12T02:57:03.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:03.079+0000 7ff9967fc640 1 --2- 172.21.15.5:0/2981802763 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff998102790 0x7ff998198950 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff98c00d950 tx=0x7ff98c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:03.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:03.079+0000 7ff97ffff640 1 -- 172.21.15.5:0/2981802763 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff98c017070 con 0x7ff998102790 2024-09-12T02:57:03.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:03.079+0000 7ff99d268640 1 -- 172.21.15.5:0/2981802763 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff99819d6b0 con 0x7ff998102790 2024-09-12T02:57:03.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:03.079+0000 7ff97ffff640 1 -- 172.21.15.5:0/2981802763 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff98c009c30 con 0x7ff998102790 2024-09-12T02:57:03.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:03.079+0000 7ff97ffff640 1 -- 172.21.15.5:0/2981802763 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff98c00f6b0 con 0x7ff998102790 2024-09-12T02:57:03.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:03.079+0000 7ff99d268640 1 -- 172.21.15.5:0/2981802763 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9981a4f90 con 0x7ff998102790 2024-09-12T02:57:03.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:03.080+0000 7ff99d268640 1 -- 172.21.15.5:0/2981802763 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9981021c0 con 0x7ff998102790 2024-09-12T02:57:03.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:03.081+0000 7ff97ffff640 1 -- 172.21.15.5:0/2981802763 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7ff98c0109d0 con 0x7ff998102790 2024-09-12T02:57:03.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:03.082+0000 7ff97ffff640 1 --2- 172.21.15.5:0/2981802763 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7ff9800a2c10 0x7ff9800a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:03.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:03.082+0000 7ff996ffd640 1 --2- 172.21.15.5:0/2981802763 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7ff9800a2c10 0x7ff9800a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:03.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:03.082+0000 7ff97ffff640 1 -- 172.21.15.5:0/2981802763 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7ff98c0cef40 con 0x7ff998102790 2024-09-12T02:57:03.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:03.083+0000 7ff996ffd640 1 --2- 172.21.15.5:0/2981802763 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7ff9800a2c10 0x7ff9800a50d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff988009890 tx=0x7ff988008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:03.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:03.088+0000 7ff97ffff640 1 -- 172.21.15.5:0/2981802763 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff98c009de0 con 0x7ff998102790 2024-09-12T02:57:03.273 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:57:03 smithi005 ceph-mon[31197]: pgmap v299: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-12T02:57:03.273 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:57:03 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/3042696165' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-12T02:57:03.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:03.272+0000 7ff99d268640 1 -- 172.21.15.5:0/2981802763 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7ff99810a150 con 0x7ff998102790 2024-09-12T02:57:03.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:03.272+0000 7ff97ffff640 1 -- 172.21.15.5:0/2981802763 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7ff98c09bcf0 con 0x7ff998102790 2024-09-12T02:57:03.274 INFO:teuthology.orchestra.run.smithi005.stdout:154618822706 2024-09-12T02:57:03.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:03.275+0000 7ff99d268640 1 -- 172.21.15.5:0/2981802763 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7ff9800a2c10 msgr2=0x7ff9800a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:03.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:03.275+0000 7ff99d268640 1 --2- 172.21.15.5:0/2981802763 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7ff9800a2c10 0x7ff9800a50d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff988009890 tx=0x7ff988008040 comp rx=0 tx=0).stop 2024-09-12T02:57:03.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:03.275+0000 7ff99d268640 1 -- 172.21.15.5:0/2981802763 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff998102790 msgr2=0x7ff998198950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:03.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:03.275+0000 7ff99d268640 1 --2- 172.21.15.5:0/2981802763 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff998102790 0x7ff998198950 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff98c00d950 tx=0x7ff98c00de10 comp rx=0 tx=0).stop 2024-09-12T02:57:03.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:03.275+0000 7ff99d268640 1 -- 172.21.15.5:0/2981802763 shutdown_connections 2024-09-12T02:57:03.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:03.275+0000 7ff99d268640 1 --2- 172.21.15.5:0/2981802763 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7ff9800a2c10 0x7ff9800a50d0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:03.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:03.275+0000 7ff99d268640 1 --2- 172.21.15.5:0/2981802763 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff998102790 0x7ff998198950 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:03.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:03.275+0000 7ff99d268640 1 --2- 172.21.15.5:0/2981802763 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff998106f10 0x7ff99819cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:03.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:03.275+0000 7ff99d268640 1 --2- 172.21.15.5:0/2981802763 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff998101de0 0x7ff998198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:03.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:03.275+0000 7ff99d268640 1 -- 172.21.15.5:0/2981802763 >> 172.21.15.5:0/2981802763 conn(0x7ff9980fd8c0 msgr2=0x7ff9980fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:03.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:03.275+0000 7ff99d268640 1 -- 172.21.15.5:0/2981802763 shutdown_connections 2024-09-12T02:57:03.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:03.275+0000 7ff99d268640 1 -- 172.21.15.5:0/2981802763 wait complete. 2024-09-12T02:57:03.415 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:57:03 smithi148 ceph-mon[35090]: pgmap v299: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-12T02:57:03.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:57:03 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/3042696165' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-12T02:57:03.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:57:03 smithi146 ceph-mon[35011]: pgmap v299: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-09-12T02:57:03.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:57:03 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/3042696165' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-12T02:57:03.639 INFO:tasks.cephadm.ceph_manager.ceph:need seq 137438953527 got 137438953529 for osd.4 2024-09-12T02:57:03.639 DEBUG:teuthology.parallel:result is None 2024-09-12T02:57:04.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:57:04 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/2981802763' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-12T02:57:04.415 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:57:04 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/2981802763' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-12T02:57:04.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:57:04 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/2981802763' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-12T02:57:04.457 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:57:04.492 INFO:tasks.cephadm.ceph_manager.ceph:need seq 154618822704 got 154618822706 for osd.5 2024-09-12T02:57:04.492 DEBUG:teuthology.parallel:result is None 2024-09-12T02:57:04.940 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:57:04.941 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:57:04.941 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:57:04.944 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:57:04.945 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:57:04.946 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:57:05.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:05.170+0000 7f9bdafe7640 1 -- 172.21.15.5:0/2173321173 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9bd4100e20 msgr2=0x7f9bd4101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:05.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:05.170+0000 7f9bdafe7640 1 --2- 172.21.15.5:0/2173321173 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9bd4100e20 0x7f9bd4101200 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f9bc4009900 tx=0x7f9bc402a480 comp rx=0 tx=0).stop 2024-09-12T02:57:05.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:05.170+0000 7f9bdafe7640 1 -- 172.21.15.5:0/2173321173 shutdown_connections 2024-09-12T02:57:05.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:05.170+0000 7f9bdafe7640 1 --2- 172.21.15.5:0/2173321173 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9bd4102ef0 0x7f9bd410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:05.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:05.170+0000 7f9bdafe7640 1 --2- 172.21.15.5:0/2173321173 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9bd4102570 0x7f9bd41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:05.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:05.170+0000 7f9bdafe7640 1 --2- 172.21.15.5:0/2173321173 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9bd4100e20 0x7f9bd4101200 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:05.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:05.170+0000 7f9bdafe7640 1 -- 172.21.15.5:0/2173321173 >> 172.21.15.5:0/2173321173 conn(0x7f9bd40fc7d0 msgr2=0x7f9bd40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:05.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:05.171+0000 7f9bdafe7640 1 -- 172.21.15.5:0/2173321173 shutdown_connections 2024-09-12T02:57:05.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:05.171+0000 7f9bdafe7640 1 -- 172.21.15.5:0/2173321173 wait complete. 2024-09-12T02:57:05.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:05.171+0000 7f9bdafe7640 1 Processor -- start 2024-09-12T02:57:05.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:05.171+0000 7f9bdafe7640 1 -- start start 2024-09-12T02:57:05.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:05.171+0000 7f9bdafe7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9bd4100e20 0x7f9bd4199460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:05.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:05.171+0000 7f9bdafe7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9bd4102570 0x7f9bd41999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:05.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:05.171+0000 7f9bdafe7640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9bd4102ef0 0x7f9bd4199f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:05.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:05.171+0000 7f9bdafe7640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd4110f50 con 0x7f9bd4100e20 2024-09-12T02:57:05.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:05.171+0000 7f9bdafe7640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd4110dd0 con 0x7f9bd4102ef0 2024-09-12T02:57:05.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:05.171+0000 7f9bdafe7640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f9bd41110d0 con 0x7f9bd4102570 2024-09-12T02:57:05.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:05.172+0000 7f9bd8d83640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9bd4100e20 0x7f9bd4199460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:05.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:05.172+0000 7f9bd8d83640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9bd4100e20 0x7f9bd4199460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42892/0 (socket says 172.21.15.5:42892) 2024-09-12T02:57:05.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:05.172+0000 7f9bd8d83640 1 -- 172.21.15.5:0/4013035535 learned_addr learned my addr 172.21.15.5:0/4013035535 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:57:05.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:05.172+0000 7f9bd3fff640 1 --2- 172.21.15.5:0/4013035535 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9bd4102570 0x7f9bd41999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:05.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:05.172+0000 7f9bd9584640 1 --2- 172.21.15.5:0/4013035535 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9bd4102ef0 0x7f9bd4199f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:05.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:05.172+0000 7f9bd8d83640 1 -- 172.21.15.5:0/4013035535 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9bd4102570 msgr2=0x7f9bd41999a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:05.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:05.172+0000 7f9bd8d83640 1 --2- 172.21.15.5:0/4013035535 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9bd4102570 0x7f9bd41999a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:05.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:05.172+0000 7f9bd8d83640 1 -- 172.21.15.5:0/4013035535 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9bd4102ef0 msgr2=0x7f9bd4199f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:05.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:05.172+0000 7f9bd8d83640 1 --2- 172.21.15.5:0/4013035535 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9bd4102ef0 0x7f9bd4199f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:05.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:05.172+0000 7f9bd8d83640 1 -- 172.21.15.5:0/4013035535 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bd41a4860 con 0x7f9bd4100e20 2024-09-12T02:57:05.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:05.172+0000 7f9bd9584640 1 --2- 172.21.15.5:0/4013035535 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9bd4102ef0 0x7f9bd4199f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:57:05.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:05.172+0000 7f9bd8d83640 1 --2- 172.21.15.5:0/4013035535 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9bd4100e20 0x7f9bd4199460 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9bc4006010 tx=0x7f9bc4003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:05.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:05.172+0000 7f9bd1ffb640 1 -- 172.21.15.5:0/4013035535 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc403f070 con 0x7f9bd4100e20 2024-09-12T02:57:05.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:05.172+0000 7f9bdafe7640 1 -- 172.21.15.5:0/4013035535 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bd41a4af0 con 0x7f9bd4100e20 2024-09-12T02:57:05.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:05.172+0000 7f9bd1ffb640 1 -- 172.21.15.5:0/4013035535 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9bc4031c30 con 0x7f9bd4100e20 2024-09-12T02:57:05.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:05.172+0000 7f9bd1ffb640 1 -- 172.21.15.5:0/4013035535 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc4035030 con 0x7f9bd4100e20 2024-09-12T02:57:05.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:05.173+0000 7f9bdafe7640 1 -- 172.21.15.5:0/4013035535 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bd41a5020 con 0x7f9bd4100e20 2024-09-12T02:57:05.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:05.174+0000 7f9bdafe7640 1 -- 172.21.15.5:0/4013035535 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9bd4062880 con 0x7f9bd4100e20 2024-09-12T02:57:05.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:05.175+0000 7f9bd1ffb640 1 -- 172.21.15.5:0/4013035535 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f9bc4042070 con 0x7f9bd4100e20 2024-09-12T02:57:05.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:05.175+0000 7f9bd1ffb640 1 --2- 172.21.15.5:0/4013035535 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f9bbc0a2c10 0x7f9bbc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:05.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:05.175+0000 7f9bd3fff640 1 --2- 172.21.15.5:0/4013035535 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f9bbc0a2c10 0x7f9bbc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:05.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:05.175+0000 7f9bd1ffb640 1 -- 172.21.15.5:0/4013035535 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f9bc40efa80 con 0x7f9bd4100e20 2024-09-12T02:57:05.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:05.176+0000 7f9bd1ffb640 1 --2- 172.21.15.5:0/4013035535 >> [v2:172.21.15.5:6826/1961857360,v1:172.21.15.5:6827/1961857360] conn(0x7f9bbc0ac7c0 0x7f9bbc0aec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:05.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:05.176+0000 7f9bd1ffb640 1 -- 172.21.15.5:0/4013035535 --> [v2:172.21.15.5:6826/1961857360,v1:172.21.15.5:6827/1961857360] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9bc403fe70 con 0x7f9bbc0ac7c0 2024-09-12T02:57:05.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:05.176+0000 7f9bd1ffb640 1 -- 172.21.15.5:0/4013035535 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=83) ==== 24+0+0 (secure 0 0 0) 0x7f9bc40f42a0 con 0x7f9bd4100e20 2024-09-12T02:57:05.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:05.176+0000 7f9bd9584640 1 --2- 172.21.15.5:0/4013035535 >> [v2:172.21.15.5:6826/1961857360,v1:172.21.15.5:6827/1961857360] conn(0x7f9bbc0ac7c0 0x7f9bbc0aec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:05.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:05.176+0000 7f9bd3fff640 1 --2- 172.21.15.5:0/4013035535 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f9bbc0a2c10 0x7f9bbc0a50d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9bd40ffe90 tx=0x7f9bc800a2d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:05.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:05.176+0000 7f9bd9584640 1 --2- 172.21.15.5:0/4013035535 >> [v2:172.21.15.5:6826/1961857360,v1:172.21.15.5:6827/1961857360] conn(0x7f9bbc0ac7c0 0x7f9bbc0aec60 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:05.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:05.177+0000 7f9bd1ffb640 1 -- 172.21.15.5:0/4013035535 <== osd.9 v2:172.21.15.5:6826/1961857360 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9bc403fe70 con 0x7f9bbc0ac7c0 2024-09-12T02:57:05.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:05.190+0000 7f9bdafe7640 1 -- 172.21.15.5:0/4013035535 --> [v2:172.21.15.5:6826/1961857360,v1:172.21.15.5:6827/1961857360] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9bd4062cb0 con 0x7f9bbc0ac7c0 2024-09-12T02:57:05.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:05.191+0000 7f9bd1ffb640 1 -- 172.21.15.5:0/4013035535 <== osd.9 v2:172.21.15.5:6826/1961857360 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f9bd4062cb0 con 0x7f9bbc0ac7c0 2024-09-12T02:57:05.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:05.191+0000 7f9bdafe7640 1 -- 172.21.15.5:0/4013035535 >> [v2:172.21.15.5:6826/1961857360,v1:172.21.15.5:6827/1961857360] conn(0x7f9bbc0ac7c0 msgr2=0x7f9bbc0aec60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:05.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:05.191+0000 7f9bdafe7640 1 --2- 172.21.15.5:0/4013035535 >> [v2:172.21.15.5:6826/1961857360,v1:172.21.15.5:6827/1961857360] conn(0x7f9bbc0ac7c0 0x7f9bbc0aec60 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:05.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:05.191+0000 7f9bdafe7640 1 -- 172.21.15.5:0/4013035535 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f9bbc0a2c10 msgr2=0x7f9bbc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:05.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:05.191+0000 7f9bdafe7640 1 --2- 172.21.15.5:0/4013035535 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f9bbc0a2c10 0x7f9bbc0a50d0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9bd40ffe90 tx=0x7f9bc800a2d0 comp rx=0 tx=0).stop 2024-09-12T02:57:05.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:05.191+0000 7f9bdafe7640 1 -- 172.21.15.5:0/4013035535 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9bd4100e20 msgr2=0x7f9bd4199460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:05.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:05.191+0000 7f9bdafe7640 1 --2- 172.21.15.5:0/4013035535 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9bd4100e20 0x7f9bd4199460 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9bc4006010 tx=0x7f9bc4003c10 comp rx=0 tx=0).stop 2024-09-12T02:57:05.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:05.191+0000 7f9bd9584640 1 -- 172.21.15.5:0/4013035535 reap_dead start 2024-09-12T02:57:05.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:05.191+0000 7f9bdafe7640 1 -- 172.21.15.5:0/4013035535 shutdown_connections 2024-09-12T02:57:05.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:05.191+0000 7f9bdafe7640 1 -- 172.21.15.5:0/4013035535 >> 172.21.15.5:0/4013035535 conn(0x7f9bd40fc7d0 msgr2=0x7f9bd40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:05.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:05.191+0000 7f9bdafe7640 1 -- 172.21.15.5:0/4013035535 shutdown_connections 2024-09-12T02:57:05.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:05.191+0000 7f9bdafe7640 1 -- 172.21.15.5:0/4013035535 wait complete. 2024-09-12T02:57:05.356 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:57:05 smithi005 ceph-mon[31197]: pgmap v300: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-09-12T02:57:05.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:05.355+0000 7fc4f6921640 1 -- 172.21.15.5:0/2434689610 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4f010c170 msgr2=0x7fc4f010e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:05.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:05.355+0000 7fc4f6921640 1 --2- 172.21.15.5:0/2434689610 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4f010c170 0x7fc4f010e610 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fc4e8009ca0 tx=0x7fc4e802a950 comp rx=0 tx=0).stop 2024-09-12T02:57:05.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:05.355+0000 7fc4f6921640 1 -- 172.21.15.5:0/2434689610 shutdown_connections 2024-09-12T02:57:05.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:05.355+0000 7fc4f6921640 1 --2- 172.21.15.5:0/2434689610 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4f010c170 0x7fc4f010e610 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:05.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:05.355+0000 7fc4f6921640 1 --2- 172.21.15.5:0/2434689610 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc4f0103660 0x7fc4f010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:05.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:05.355+0000 7fc4f6921640 1 --2- 172.21.15.5:0/2434689610 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc4f0102d40 0x7fc4f0103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:05.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:05.355+0000 7fc4f6921640 1 -- 172.21.15.5:0/2434689610 >> 172.21.15.5:0/2434689610 conn(0x7fc4f00fc7a0 msgr2=0x7fc4f00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:05.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:05.355+0000 7fc4f6921640 1 -- 172.21.15.5:0/2434689610 shutdown_connections 2024-09-12T02:57:05.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:05.355+0000 7fc4f6921640 1 -- 172.21.15.5:0/2434689610 wait complete. 2024-09-12T02:57:05.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:05.356+0000 7fc4f6921640 1 Processor -- start 2024-09-12T02:57:05.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:05.356+0000 7fc4f6921640 1 -- start start 2024-09-12T02:57:05.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:05.356+0000 7fc4f6921640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc4f0102d40 0x7fc4f01009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:05.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:05.356+0000 7fc4f6921640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc4f0103660 0x7fc4f0100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:05.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:05.357+0000 7fc4f6921640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4f010c170 0x7fc4f0101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:05.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:05.357+0000 7fc4f6921640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f0111f70 con 0x7fc4f010c170 2024-09-12T02:57:05.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:05.357+0000 7fc4f6921640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f0111df0 con 0x7fc4f0103660 2024-09-12T02:57:05.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:05.357+0000 7fc4f6921640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f01120f0 con 0x7fc4f0102d40 2024-09-12T02:57:05.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:05.357+0000 7fc4f4ebe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4f010c170 0x7fc4f0101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:05.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:05.357+0000 7fc4effff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc4f0102d40 0x7fc4f01009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:05.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:05.357+0000 7fc4f4ebe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4f010c170 0x7fc4f0101420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42898/0 (socket says 172.21.15.5:42898) 2024-09-12T02:57:05.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:05.357+0000 7fc4f4ebe640 1 -- 172.21.15.5:0/1450234068 learned_addr learned my addr 172.21.15.5:0/1450234068 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:57:05.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:05.357+0000 7fc4ef7fe640 1 --2- 172.21.15.5:0/1450234068 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc4f0103660 0x7fc4f0100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:05.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:05.357+0000 7fc4f4ebe640 1 -- 172.21.15.5:0/1450234068 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc4f0102d40 msgr2=0x7fc4f01009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:05.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:05.357+0000 7fc4f4ebe640 1 --2- 172.21.15.5:0/1450234068 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc4f0102d40 0x7fc4f01009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:05.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:05.357+0000 7fc4f4ebe640 1 -- 172.21.15.5:0/1450234068 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc4f0103660 msgr2=0x7fc4f0100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:05.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:05.357+0000 7fc4f4ebe640 1 --2- 172.21.15.5:0/1450234068 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc4f0103660 0x7fc4f0100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:05.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:05.357+0000 7fc4f4ebe640 1 -- 172.21.15.5:0/1450234068 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4f01a68d0 con 0x7fc4f010c170 2024-09-12T02:57:05.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:05.357+0000 7fc4effff640 1 --2- 172.21.15.5:0/1450234068 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc4f0102d40 0x7fc4f01009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:57:05.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:05.357+0000 7fc4ef7fe640 1 --2- 172.21.15.5:0/1450234068 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc4f0103660 0x7fc4f0100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:57:05.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:05.357+0000 7fc4f4ebe640 1 --2- 172.21.15.5:0/1450234068 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4f010c170 0x7fc4f0101420 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc4e802ae70 tx=0x7fc4e802f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:05.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:05.357+0000 7fc4ed7fa640 1 -- 172.21.15.5:0/1450234068 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e8034040 con 0x7fc4f010c170 2024-09-12T02:57:05.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:05.357+0000 7fc4f6921640 1 -- 172.21.15.5:0/1450234068 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4f01a6b60 con 0x7fc4f010c170 2024-09-12T02:57:05.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:05.357+0000 7fc4ed7fa640 1 -- 172.21.15.5:0/1450234068 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4e8030920 con 0x7fc4f010c170 2024-09-12T02:57:05.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:05.358+0000 7fc4ed7fa640 1 -- 172.21.15.5:0/1450234068 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e8045480 con 0x7fc4f010c170 2024-09-12T02:57:05.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:05.358+0000 7fc4f6921640 1 -- 172.21.15.5:0/1450234068 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4f01a7090 con 0x7fc4f010c170 2024-09-12T02:57:05.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:05.359+0000 7fc4f6921640 1 -- 172.21.15.5:0/1450234068 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4f0103120 con 0x7fc4f010c170 2024-09-12T02:57:05.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:05.360+0000 7fc4ed7fa640 1 -- 172.21.15.5:0/1450234068 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fc4e8037020 con 0x7fc4f010c170 2024-09-12T02:57:05.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:05.361+0000 7fc4ed7fa640 1 --2- 172.21.15.5:0/1450234068 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fc4d80a2c10 0x7fc4d80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:05.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:05.361+0000 7fc4effff640 1 --2- 172.21.15.5:0/1450234068 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fc4d80a2c10 0x7fc4d80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:05.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:05.361+0000 7fc4ed7fa640 1 -- 172.21.15.5:0/1450234068 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fc4e80f0040 con 0x7fc4f010c170 2024-09-12T02:57:05.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:05.361+0000 7fc4effff640 1 --2- 172.21.15.5:0/1450234068 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fc4d80a2c10 0x7fc4d80a50d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc4e0005fd0 tx=0x7fc4e000b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:05.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:05.365+0000 7fc4ed7fa640 1 -- 172.21.15.5:0/1450234068 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc4e80bcf00 con 0x7fc4f010c170 2024-09-12T02:57:05.371 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:57:05.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:57:05 smithi148 ceph-mon[35090]: pgmap v300: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-09-12T02:57:05.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:57:05 smithi146 ceph-mon[35011]: pgmap v300: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 10 op/s 2024-09-12T02:57:05.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:05.547+0000 7fc4f6921640 1 -- 172.21.15.5:0/1450234068 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7fc4f01a7320 con 0x7fc4f010c170 2024-09-12T02:57:05.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:05.547+0000 7fc4ed7fa640 1 -- 172.21.15.5:0/1450234068 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fc4e80bc800 con 0x7fc4f010c170 2024-09-12T02:57:05.548 INFO:teuthology.orchestra.run.smithi005.stdout:292057776155 2024-09-12T02:57:05.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:05.549+0000 7fc4f6921640 1 -- 172.21.15.5:0/1450234068 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fc4d80a2c10 msgr2=0x7fc4d80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:05.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:05.549+0000 7fc4f6921640 1 --2- 172.21.15.5:0/1450234068 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fc4d80a2c10 0x7fc4d80a50d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc4e0005fd0 tx=0x7fc4e000b390 comp rx=0 tx=0).stop 2024-09-12T02:57:05.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:05.549+0000 7fc4f6921640 1 -- 172.21.15.5:0/1450234068 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4f010c170 msgr2=0x7fc4f0101420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:05.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:05.549+0000 7fc4f6921640 1 --2- 172.21.15.5:0/1450234068 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4f010c170 0x7fc4f0101420 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc4e802ae70 tx=0x7fc4e802f5b0 comp rx=0 tx=0).stop 2024-09-12T02:57:05.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:05.550+0000 7fc4f6921640 1 -- 172.21.15.5:0/1450234068 shutdown_connections 2024-09-12T02:57:05.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:05.550+0000 7fc4f6921640 1 --2- 172.21.15.5:0/1450234068 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fc4d80a2c10 0x7fc4d80a50d0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:05.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:05.550+0000 7fc4f6921640 1 --2- 172.21.15.5:0/1450234068 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4f010c170 0x7fc4f0101420 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:05.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:05.550+0000 7fc4f6921640 1 --2- 172.21.15.5:0/1450234068 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc4f0103660 0x7fc4f0100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:05.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:05.550+0000 7fc4f6921640 1 --2- 172.21.15.5:0/1450234068 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc4f0102d40 0x7fc4f01009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:05.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:05.550+0000 7fc4f6921640 1 -- 172.21.15.5:0/1450234068 >> 172.21.15.5:0/1450234068 conn(0x7fc4f00fc7a0 msgr2=0x7fc4f00fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:05.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:05.550+0000 7fc4f6921640 1 -- 172.21.15.5:0/1450234068 shutdown_connections 2024-09-12T02:57:05.551 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:05.550+0000 7fc4f6921640 1 -- 172.21.15.5:0/1450234068 wait complete. 2024-09-12T02:57:06.258 INFO:tasks.cephadm.ceph_manager.ceph:need seq 292057776153 got 292057776155 for osd.10 2024-09-12T02:57:06.258 DEBUG:teuthology.parallel:result is None 2024-09-12T02:57:06.406 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:57:06 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/1450234068' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-12T02:57:06.415 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:57:06 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/1450234068' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-12T02:57:06.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:57:06 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/1450234068' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-12T02:57:07.219 INFO:teuthology.orchestra.run.smithi005.stdout:270582939681 2024-09-12T02:57:07.219 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-09-12T02:57:07.415 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:57:07 smithi148 ceph-mon[35090]: pgmap v301: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 9 op/s 2024-09-12T02:57:07.451 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:57:07 smithi005 ceph-mon[31197]: pgmap v301: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 9 op/s 2024-09-12T02:57:07.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:57:07 smithi146 ceph-mon[35011]: pgmap v301: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 9 op/s 2024-09-12T02:57:07.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:07.914+0000 7f12a0469640 1 -- 172.21.15.5:0/841024471 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1298101de0 msgr2=0x7f12981021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:07.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:07.914+0000 7f12a0469640 1 --2- 172.21.15.5:0/841024471 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1298101de0 0x7f12981021c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f1288009ca0 tx=0x7f128802a8c0 comp rx=0 tx=0).stop 2024-09-12T02:57:07.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:07.914+0000 7f12a0469640 1 -- 172.21.15.5:0/841024471 shutdown_connections 2024-09-12T02:57:07.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:07.914+0000 7f12a0469640 1 --2- 172.21.15.5:0/841024471 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1298106f10 0x7f129810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:07.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:07.914+0000 7f12a0469640 1 --2- 172.21.15.5:0/841024471 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1298102790 0x7f12981067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:07.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:07.914+0000 7f12a0469640 1 --2- 172.21.15.5:0/841024471 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1298101de0 0x7f12981021c0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:07.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:07.914+0000 7f12a0469640 1 -- 172.21.15.5:0/841024471 >> 172.21.15.5:0/841024471 conn(0x7f12980fd8c0 msgr2=0x7f12980ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:07.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:07.914+0000 7f12a0469640 1 -- 172.21.15.5:0/841024471 shutdown_connections 2024-09-12T02:57:07.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:07.914+0000 7f12a0469640 1 -- 172.21.15.5:0/841024471 wait complete. 2024-09-12T02:57:07.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:07.915+0000 7f12a0469640 1 Processor -- start 2024-09-12T02:57:07.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:07.915+0000 7f12a0469640 1 -- start start 2024-09-12T02:57:07.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:07.915+0000 7f12a0469640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1298101de0 0x7f1298198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:07.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:07.915+0000 7f12a0469640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1298102790 0x7f1298198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:07.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:07.915+0000 7f12a0469640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1298106f10 0x7f129819cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:07.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:07.915+0000 7f12a0469640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1298111f30 con 0x7f1298106f10 2024-09-12T02:57:07.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:07.915+0000 7f12a0469640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f1298111db0 con 0x7f1298102790 2024-09-12T02:57:07.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:07.915+0000 7f12a0469640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f12981120b0 con 0x7f1298101de0 2024-09-12T02:57:07.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:07.915+0000 7f129ea06640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1298106f10 0x7f129819cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:07.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:07.915+0000 7f129ea06640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1298106f10 0x7f129819cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42910/0 (socket says 172.21.15.5:42910) 2024-09-12T02:57:07.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:07.915+0000 7f129ea06640 1 -- 172.21.15.5:0/2254390864 learned_addr learned my addr 172.21.15.5:0/2254390864 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:57:07.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:07.916+0000 7f129e205640 1 --2- 172.21.15.5:0/2254390864 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1298101de0 0x7f1298198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:07.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:07.916+0000 7f129da04640 1 --2- 172.21.15.5:0/2254390864 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1298102790 0x7f1298198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:07.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:07.916+0000 7f129ea06640 1 -- 172.21.15.5:0/2254390864 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1298101de0 msgr2=0x7f1298198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:07.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:07.916+0000 7f129ea06640 1 --2- 172.21.15.5:0/2254390864 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1298101de0 0x7f1298198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:07.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:07.916+0000 7f129ea06640 1 -- 172.21.15.5:0/2254390864 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1298102790 msgr2=0x7f1298198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:07.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:07.916+0000 7f129ea06640 1 --2- 172.21.15.5:0/2254390864 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1298102790 0x7f1298198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:07.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:07.916+0000 7f129ea06640 1 -- 172.21.15.5:0/2254390864 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f129819d3f0 con 0x7f1298106f10 2024-09-12T02:57:07.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:07.916+0000 7f129e205640 1 --2- 172.21.15.5:0/2254390864 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1298101de0 0x7f1298198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:57:07.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:07.916+0000 7f129da04640 1 --2- 172.21.15.5:0/2254390864 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1298102790 0x7f1298198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:57:07.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:07.916+0000 7f129ea06640 1 --2- 172.21.15.5:0/2254390864 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1298106f10 0x7f129819cd10 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f129000d750 tx=0x7f129000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:07.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:07.916+0000 7f12977fe640 1 -- 172.21.15.5:0/2254390864 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1290017070 con 0x7f1298106f10 2024-09-12T02:57:07.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:07.916+0000 7f12977fe640 1 -- 172.21.15.5:0/2254390864 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1290005500 con 0x7f1298106f10 2024-09-12T02:57:07.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:07.916+0000 7f12a0469640 1 -- 172.21.15.5:0/2254390864 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f129819d6e0 con 0x7f1298106f10 2024-09-12T02:57:07.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:07.916+0000 7f12977fe640 1 -- 172.21.15.5:0/2254390864 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1290004d90 con 0x7f1298106f10 2024-09-12T02:57:07.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:07.916+0000 7f12a0469640 1 -- 172.21.15.5:0/2254390864 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12981a4fc0 con 0x7f1298106f10 2024-09-12T02:57:07.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:07.917+0000 7f12a0469640 1 -- 172.21.15.5:0/2254390864 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12981021c0 con 0x7f1298106f10 2024-09-12T02:57:07.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:07.918+0000 7f12977fe640 1 -- 172.21.15.5:0/2254390864 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f1290004f30 con 0x7f1298106f10 2024-09-12T02:57:07.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:07.919+0000 7f12977fe640 1 --2- 172.21.15.5:0/2254390864 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f12800a2c90 0x7f12800a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:07.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:07.919+0000 7f129e205640 1 --2- 172.21.15.5:0/2254390864 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f12800a2c90 0x7f12800a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:07.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:07.919+0000 7f12977fe640 1 -- 172.21.15.5:0/2254390864 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f12900ceb80 con 0x7f1298106f10 2024-09-12T02:57:07.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:07.920+0000 7f129e205640 1 --2- 172.21.15.5:0/2254390864 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f12800a2c90 0x7f12800a5150 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1288006010 tx=0x7f12880093f0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:07.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:07.924+0000 7f12977fe640 1 -- 172.21.15.5:0/2254390864 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f129009b990 con 0x7f1298106f10 2024-09-12T02:57:08.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:08.090+0000 7f1c99eb4640 1 -- 172.21.15.5:0/3119013690 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c94102570 msgr2=0x7f1c941029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:08.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:08.090+0000 7f1c99eb4640 1 --2- 172.21.15.5:0/3119013690 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c94102570 0x7f1c941029b0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f1c88009900 tx=0x7f1c8802a480 comp rx=0 tx=0).stop 2024-09-12T02:57:08.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:08.090+0000 7f1c99eb4640 1 -- 172.21.15.5:0/3119013690 shutdown_connections 2024-09-12T02:57:08.106 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:08.090+0000 7f1c99eb4640 1 --2- 172.21.15.5:0/3119013690 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c94102570 0x7f1c941029b0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:08.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:08.090+0000 7f1c99eb4640 1 --2- 172.21.15.5:0/3119013690 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1c94102ef0 0x7f1c9410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:08.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:08.090+0000 7f1c99eb4640 1 --2- 172.21.15.5:0/3119013690 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c94100e20 0x7f1c94101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:08.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:08.090+0000 7f1c99eb4640 1 -- 172.21.15.5:0/3119013690 >> 172.21.15.5:0/3119013690 conn(0x7f1c940fc7d0 msgr2=0x7f1c940febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:08.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:08.090+0000 7f1c99eb4640 1 -- 172.21.15.5:0/3119013690 shutdown_connections 2024-09-12T02:57:08.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:08.090+0000 7f1c99eb4640 1 -- 172.21.15.5:0/3119013690 wait complete. 2024-09-12T02:57:08.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:08.090+0000 7f1c99eb4640 1 Processor -- start 2024-09-12T02:57:08.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:08.091+0000 7f1c99eb4640 1 -- start start 2024-09-12T02:57:08.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:08.091+0000 7f1c99eb4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c94100e20 0x7f1c9419a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:08.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:08.091+0000 7f1c99eb4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c94102570 0x7f1c9419ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:08.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:08.091+0000 7f1c99eb4640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1c94102ef0 0x7f1c9419efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:08.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:08.091+0000 7f1c99eb4640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1c94110f50 con 0x7f1c94100e20 2024-09-12T02:57:08.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:08.091+0000 7f1c99eb4640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f1c94110dd0 con 0x7f1c94102ef0 2024-09-12T02:57:08.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:08.091+0000 7f1c99eb4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1c941110d0 con 0x7f1c94102570 2024-09-12T02:57:08.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:08.091+0000 7f1c937fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c94100e20 0x7f1c9419a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:08.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:08.091+0000 7f1c937fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c94100e20 0x7f1c9419a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42922/0 (socket says 172.21.15.5:42922) 2024-09-12T02:57:08.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:08.091+0000 7f1c937fe640 1 -- 172.21.15.5:0/2305290047 learned_addr learned my addr 172.21.15.5:0/2305290047 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:57:08.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:08.091+0000 7f1c93fff640 1 --2- 172.21.15.5:0/2305290047 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1c94102ef0 0x7f1c9419efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:08.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:08.091+0000 7f1c92ffd640 1 --2- 172.21.15.5:0/2305290047 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c94102570 0x7f1c9419ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:08.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:08.091+0000 7f1c937fe640 1 -- 172.21.15.5:0/2305290047 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c94102570 msgr2=0x7f1c9419ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:08.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:08.091+0000 7f1c937fe640 1 --2- 172.21.15.5:0/2305290047 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c94102570 0x7f1c9419ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:08.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:08.091+0000 7f1c937fe640 1 -- 172.21.15.5:0/2305290047 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1c94102ef0 msgr2=0x7f1c9419efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:08.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:08.091+0000 7f1c937fe640 1 --2- 172.21.15.5:0/2305290047 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1c94102ef0 0x7f1c9419efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:08.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:08.091+0000 7f1c937fe640 1 -- 172.21.15.5:0/2305290047 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c9419f6b0 con 0x7f1c94100e20 2024-09-12T02:57:08.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:08.091+0000 7f1c937fe640 1 --2- 172.21.15.5:0/2305290047 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c94100e20 0x7f1c9419a700 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f1c8400d950 tx=0x7f1c8400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:08.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:08.092+0000 7f1c90ff9640 1 -- 172.21.15.5:0/2305290047 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c84017070 con 0x7f1c94100e20 2024-09-12T02:57:08.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:08.092+0000 7f1c90ff9640 1 -- 172.21.15.5:0/2305290047 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c84009c30 con 0x7f1c94100e20 2024-09-12T02:57:08.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:08.092+0000 7f1c99eb4640 1 -- 172.21.15.5:0/2305290047 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c9419f9a0 con 0x7f1c94100e20 2024-09-12T02:57:08.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:08.092+0000 7f1c90ff9640 1 -- 172.21.15.5:0/2305290047 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c8400f6b0 con 0x7f1c94100e20 2024-09-12T02:57:08.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:08.092+0000 7f1c99eb4640 1 -- 172.21.15.5:0/2305290047 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c941a7280 con 0x7f1c94100e20 2024-09-12T02:57:08.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:08.093+0000 7f1c99eb4640 1 -- 172.21.15.5:0/2305290047 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c94101200 con 0x7f1c94100e20 2024-09-12T02:57:08.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:08.095+0000 7f1c90ff9640 1 -- 172.21.15.5:0/2305290047 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f1c840109d0 con 0x7f1c94100e20 2024-09-12T02:57:08.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:08.096+0000 7f1c90ff9640 1 --2- 172.21.15.5:0/2305290047 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f1c7c0a2c90 0x7f1c7c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:08.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:08.096+0000 7f1c92ffd640 1 --2- 172.21.15.5:0/2305290047 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f1c7c0a2c90 0x7f1c7c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:08.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:08.096+0000 7f1c90ff9640 1 -- 172.21.15.5:0/2305290047 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f1c840ced30 con 0x7f1c94100e20 2024-09-12T02:57:08.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:08.096+0000 7f1c92ffd640 1 --2- 172.21.15.5:0/2305290047 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f1c7c0a2c90 0x7f1c7c0a5150 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1c88009230 tx=0x7f1c880058d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:08.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:08.098+0000 7f1c90ff9640 1 -- 172.21.15.5:0/2305290047 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1c84009de0 con 0x7f1c94100e20 2024-09-12T02:57:08.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:08.109+0000 7f12a0469640 1 -- 172.21.15.5:0/2254390864 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f129810a150 con 0x7f1298106f10 2024-09-12T02:57:08.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:08.109+0000 7f12977fe640 1 -- 172.21.15.5:0/2254390864 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f129009b290 con 0x7f1298106f10 2024-09-12T02:57:08.110 INFO:teuthology.orchestra.run.smithi005.stdout:34359738445 2024-09-12T02:57:08.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:08.111+0000 7f12a0469640 1 -- 172.21.15.5:0/2254390864 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f12800a2c90 msgr2=0x7f12800a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:08.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:08.111+0000 7f12a0469640 1 --2- 172.21.15.5:0/2254390864 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f12800a2c90 0x7f12800a5150 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1288006010 tx=0x7f12880093f0 comp rx=0 tx=0).stop 2024-09-12T02:57:08.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:08.111+0000 7f12a0469640 1 -- 172.21.15.5:0/2254390864 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1298106f10 msgr2=0x7f129819cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:08.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:08.111+0000 7f12a0469640 1 --2- 172.21.15.5:0/2254390864 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1298106f10 0x7f129819cd10 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f129000d750 tx=0x7f129000dc10 comp rx=0 tx=0).stop 2024-09-12T02:57:08.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:08.111+0000 7f12a0469640 1 -- 172.21.15.5:0/2254390864 shutdown_connections 2024-09-12T02:57:08.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:08.111+0000 7f12a0469640 1 --2- 172.21.15.5:0/2254390864 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f12800a2c90 0x7f12800a5150 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:08.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:08.111+0000 7f12a0469640 1 --2- 172.21.15.5:0/2254390864 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1298106f10 0x7f129819cd10 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:08.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:08.111+0000 7f12a0469640 1 --2- 172.21.15.5:0/2254390864 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1298102790 0x7f1298198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:08.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:08.111+0000 7f12a0469640 1 --2- 172.21.15.5:0/2254390864 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1298101de0 0x7f1298198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:08.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:08.111+0000 7f12a0469640 1 -- 172.21.15.5:0/2254390864 >> 172.21.15.5:0/2254390864 conn(0x7f12980fd8c0 msgr2=0x7f12980fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:08.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:08.111+0000 7f12a0469640 1 -- 172.21.15.5:0/2254390864 shutdown_connections 2024-09-12T02:57:08.112 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:08.112+0000 7f12a0469640 1 -- 172.21.15.5:0/2254390864 wait complete. 2024-09-12T02:57:08.289 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:57:08 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/2254390864' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-12T02:57:08.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:08.289+0000 7f1c99eb4640 1 -- 172.21.15.5:0/2305290047 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f1c941a7450 con 0x7f1c94100e20 2024-09-12T02:57:08.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:08.289+0000 7f1c90ff9640 1 -- 172.21.15.5:0/2305290047 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f1c8409b510 con 0x7f1c94100e20 2024-09-12T02:57:08.290 INFO:teuthology.orchestra.run.smithi005.stdout:107374182462 2024-09-12T02:57:08.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:08.293+0000 7f1c99eb4640 1 -- 172.21.15.5:0/2305290047 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f1c7c0a2c90 msgr2=0x7f1c7c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:08.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:08.293+0000 7f1c99eb4640 1 --2- 172.21.15.5:0/2305290047 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f1c7c0a2c90 0x7f1c7c0a5150 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1c88009230 tx=0x7f1c880058d0 comp rx=0 tx=0).stop 2024-09-12T02:57:08.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:08.293+0000 7f1c99eb4640 1 -- 172.21.15.5:0/2305290047 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c94100e20 msgr2=0x7f1c9419a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:08.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:08.293+0000 7f1c99eb4640 1 --2- 172.21.15.5:0/2305290047 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c94100e20 0x7f1c9419a700 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f1c8400d950 tx=0x7f1c8400de10 comp rx=0 tx=0).stop 2024-09-12T02:57:08.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:08.293+0000 7f1c99eb4640 1 -- 172.21.15.5:0/2305290047 shutdown_connections 2024-09-12T02:57:08.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:08.293+0000 7f1c99eb4640 1 --2- 172.21.15.5:0/2305290047 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1c94102ef0 0x7f1c9419efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:08.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:08.293+0000 7f1c99eb4640 1 --2- 172.21.15.5:0/2305290047 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c94102570 0x7f1c9419ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:08.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:08.293+0000 7f1c99eb4640 1 --2- 172.21.15.5:0/2305290047 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f1c7c0a2c90 0x7f1c7c0a5150 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:08.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:08.293+0000 7f1c99eb4640 1 --2- 172.21.15.5:0/2305290047 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c94100e20 0x7f1c9419a700 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:08.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:08.293+0000 7f1c99eb4640 1 -- 172.21.15.5:0/2305290047 >> 172.21.15.5:0/2305290047 conn(0x7f1c940fc7d0 msgr2=0x7f1c9406a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:08.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:08.293+0000 7f1c99eb4640 1 -- 172.21.15.5:0/2305290047 shutdown_connections 2024-09-12T02:57:08.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:08.293+0000 7f1c99eb4640 1 -- 172.21.15.5:0/2305290047 wait complete. 2024-09-12T02:57:08.415 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:57:08 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/2254390864' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-12T02:57:08.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:57:08 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/2254390864' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-12T02:57:09.017 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182460 got 107374182462 for osd.3 2024-09-12T02:57:09.017 DEBUG:teuthology.parallel:result is None 2024-09-12T02:57:09.260 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:57:09 smithi005 ceph-mon[31197]: pgmap v302: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-12T02:57:09.260 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:57:09 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/2305290047' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-12T02:57:09.415 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:57:09 smithi148 ceph-mon[35090]: pgmap v302: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-12T02:57:09.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:57:09 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/2305290047' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-12T02:57:09.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:57:09 smithi146 ceph-mon[35011]: pgmap v302: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-12T02:57:09.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:57:09 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/2305290047' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-12T02:57:09.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:09.761+0000 7f4f2a097640 1 -- 172.21.15.5:0/3170853715 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f241082a0 msgr2=0x7f4f2410ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:09.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:09.761+0000 7f4f2a097640 1 --2- 172.21.15.5:0/3170853715 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f241082a0 0x7f4f2410ec40 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c009cd0 tx=0x7f4f1c02a8e0 comp rx=0 tx=0).stop 2024-09-12T02:57:09.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:09.761+0000 7f4f2a097640 1 -- 172.21.15.5:0/3170853715 shutdown_connections 2024-09-12T02:57:09.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:09.761+0000 7f4f2a097640 1 --2- 172.21.15.5:0/3170853715 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f241082a0 0x7f4f2410ec40 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:09.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:09.761+0000 7f4f2a097640 1 --2- 172.21.15.5:0/3170853715 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4f241036e0 0x7f4f24107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:09.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:09.761+0000 7f4f2a097640 1 --2- 172.21.15.5:0/3170853715 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4f24102d30 0x7f4f24103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:09.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:09.761+0000 7f4f2a097640 1 -- 172.21.15.5:0/3170853715 >> 172.21.15.5:0/3170853715 conn(0x7f4f240fc7d0 msgr2=0x7f4f240febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:09.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:09.761+0000 7f4f2a097640 1 -- 172.21.15.5:0/3170853715 shutdown_connections 2024-09-12T02:57:09.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:09.761+0000 7f4f2a097640 1 -- 172.21.15.5:0/3170853715 wait complete. 2024-09-12T02:57:09.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:09.761+0000 7f4f2a097640 1 Processor -- start 2024-09-12T02:57:09.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:09.762+0000 7f4f2a097640 1 -- start start 2024-09-12T02:57:09.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:09.762+0000 7f4f2a097640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f24102d30 0x7f4f24100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:09.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:09.762+0000 7f4f2a097640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4f241036e0 0x7f4f24100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:09.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:09.762+0000 7f4f2a097640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4f241082a0 0x7f4f24101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:09.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:09.762+0000 7f4f2a097640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4f241120a0 con 0x7f4f24102d30 2024-09-12T02:57:09.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:09.762+0000 7f4f2a097640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f4f24111f20 con 0x7f4f241036e0 2024-09-12T02:57:09.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:09.762+0000 7f4f2a097640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4f24112220 con 0x7f4f241082a0 2024-09-12T02:57:09.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:09.762+0000 7f4f237fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f24102d30 0x7f4f24100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:09.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:09.762+0000 7f4f237fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f24102d30 0x7f4f24100990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42946/0 (socket says 172.21.15.5:42946) 2024-09-12T02:57:09.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:09.762+0000 7f4f237fe640 1 -- 172.21.15.5:0/2682836500 learned_addr learned my addr 172.21.15.5:0/2682836500 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:57:09.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:09.762+0000 7f4f22ffd640 1 --2- 172.21.15.5:0/2682836500 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4f241036e0 0x7f4f24100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:09.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:09.762+0000 7f4f23fff640 1 --2- 172.21.15.5:0/2682836500 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4f241082a0 0x7f4f24101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:09.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:09.762+0000 7f4f237fe640 1 -- 172.21.15.5:0/2682836500 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4f241082a0 msgr2=0x7f4f24101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:09.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:09.762+0000 7f4f237fe640 1 --2- 172.21.15.5:0/2682836500 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4f241082a0 0x7f4f24101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:09.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:09.762+0000 7f4f237fe640 1 -- 172.21.15.5:0/2682836500 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4f241036e0 msgr2=0x7f4f24100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:09.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:09.762+0000 7f4f237fe640 1 --2- 172.21.15.5:0/2682836500 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4f241036e0 0x7f4f24100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:09.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:09.763+0000 7f4f237fe640 1 -- 172.21.15.5:0/2682836500 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f241a6a30 con 0x7f4f24102d30 2024-09-12T02:57:09.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:09.763+0000 7f4f237fe640 1 --2- 172.21.15.5:0/2682836500 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f24102d30 0x7f4f24100990 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4f1400e9f0 tx=0x7f4f1400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:09.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:09.763+0000 7f4f20ff9640 1 -- 172.21.15.5:0/2682836500 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f1400cd80 con 0x7f4f24102d30 2024-09-12T02:57:09.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:09.763+0000 7f4f2a097640 1 -- 172.21.15.5:0/2682836500 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f241a6d20 con 0x7f4f24102d30 2024-09-12T02:57:09.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:09.763+0000 7f4f20ff9640 1 -- 172.21.15.5:0/2682836500 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f14005500 con 0x7f4f24102d30 2024-09-12T02:57:09.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:09.763+0000 7f4f20ff9640 1 -- 172.21.15.5:0/2682836500 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f14010690 con 0x7f4f24102d30 2024-09-12T02:57:09.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:09.763+0000 7f4f2a097640 1 -- 172.21.15.5:0/2682836500 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f241a7200 con 0x7f4f24102d30 2024-09-12T02:57:09.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:09.764+0000 7f4f2a097640 1 -- 172.21.15.5:0/2682836500 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f24103110 con 0x7f4f24102d30 2024-09-12T02:57:09.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:09.765+0000 7f4f20ff9640 1 -- 172.21.15.5:0/2682836500 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f4f140046c0 con 0x7f4f24102d30 2024-09-12T02:57:09.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:09.766+0000 7f4f20ff9640 1 --2- 172.21.15.5:0/2682836500 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f4f0c0a2c10 0x7f4f0c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:09.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:09.766+0000 7f4f20ff9640 1 -- 172.21.15.5:0/2682836500 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f4f140cf830 con 0x7f4f24102d30 2024-09-12T02:57:09.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:09.766+0000 7f4f22ffd640 1 --2- 172.21.15.5:0/2682836500 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f4f0c0a2c10 0x7f4f0c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:09.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:09.766+0000 7f4f22ffd640 1 --2- 172.21.15.5:0/2682836500 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f4f0c0a2c10 0x7f4f0c0a50d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4f24102420 tx=0x7f4f1800b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:09.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:09.769+0000 7f4f20ff9640 1 -- 172.21.15.5:0/2682836500 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4f140541b0 con 0x7f4f24102d30 2024-09-12T02:57:09.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:09.652+0000 7fe20c1d8640 1 -- 172.21.15.5:0/418327149 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe204101280 msgr2=0x7fe20410df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:09.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:09.652+0000 7fe20c1d8640 1 --2- 172.21.15.5:0/418327149 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe204101280 0x7fe20410df40 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe1f4009900 tx=0x7fe1f402a410 comp rx=0 tx=0).stop 2024-09-12T02:57:09.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:09.652+0000 7fe20c1d8640 1 -- 172.21.15.5:0/418327149 shutdown_connections 2024-09-12T02:57:09.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:09.652+0000 7fe20c1d8640 1 --2- 172.21.15.5:0/418327149 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe204101280 0x7fe20410df40 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:09.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:09.652+0000 7fe20c1d8640 1 --2- 172.21.15.5:0/418327149 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe20410e480 0x7fe204110870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:09.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:09.652+0000 7fe20c1d8640 1 --2- 172.21.15.5:0/418327149 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe2041008d0 0x7fe204100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:09.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:09.652+0000 7fe20c1d8640 1 -- 172.21.15.5:0/418327149 >> 172.21.15.5:0/418327149 conn(0x7fe2040fc7a0 msgr2=0x7fe2040febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:09.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:09.652+0000 7fe20c1d8640 1 -- 172.21.15.5:0/418327149 shutdown_connections 2024-09-12T02:57:09.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:09.652+0000 7fe20c1d8640 1 -- 172.21.15.5:0/418327149 wait complete. 2024-09-12T02:57:09.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:09.652+0000 7fe20c1d8640 1 Processor -- start 2024-09-12T02:57:09.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:09.652+0000 7fe20c1d8640 1 -- start start 2024-09-12T02:57:09.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:09.653+0000 7fe20c1d8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2041008d0 0x7fe20419a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:09.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:09.653+0000 7fe20c1d8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe204101280 0x7fe20419acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:09.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:09.653+0000 7fe20c1d8640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe20410e480 0x7fe20419df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:09.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:09.653+0000 7fe20c1d8640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe204114270 con 0x7fe2041008d0 2024-09-12T02:57:09.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:09.653+0000 7fe20c1d8640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fe2041140f0 con 0x7fe20410e480 2024-09-12T02:57:09.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:09.653+0000 7fe20c1d8640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe2041143f0 con 0x7fe204101280 2024-09-12T02:57:09.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:09.653+0000 7fe209f74640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2041008d0 0x7fe20419a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:09.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:09.653+0000 7fe209f74640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2041008d0 0x7fe20419a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42926/0 (socket says 172.21.15.5:42926) 2024-09-12T02:57:09.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:09.653+0000 7fe209f74640 1 -- 172.21.15.5:0/3488802286 learned_addr learned my addr 172.21.15.5:0/3488802286 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:57:09.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:09.653+0000 7fe20a775640 1 --2- 172.21.15.5:0/3488802286 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe20410e480 0x7fe20419df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:09.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:09.653+0000 7fe209773640 1 --2- 172.21.15.5:0/3488802286 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe204101280 0x7fe20419acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:09.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:09.653+0000 7fe209f74640 1 -- 172.21.15.5:0/3488802286 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe204101280 msgr2=0x7fe20419acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:09.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:09.653+0000 7fe209f74640 1 --2- 172.21.15.5:0/3488802286 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe204101280 0x7fe20419acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:09.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:09.653+0000 7fe209f74640 1 -- 172.21.15.5:0/3488802286 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe20410e480 msgr2=0x7fe20419df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:09.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:09.653+0000 7fe209f74640 1 --2- 172.21.15.5:0/3488802286 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe20410e480 0x7fe20419df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:09.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:09.653+0000 7fe209f74640 1 -- 172.21.15.5:0/3488802286 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe20419e6c0 con 0x7fe2041008d0 2024-09-12T02:57:09.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:09.653+0000 7fe209773640 1 --2- 172.21.15.5:0/3488802286 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe204101280 0x7fe20419acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:57:09.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:09.653+0000 7fe209f74640 1 --2- 172.21.15.5:0/3488802286 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2041008d0 0x7fe20419a770 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe20000b4a0 tx=0x7fe20000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:09.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:09.653+0000 7fe1faffd640 1 -- 172.21.15.5:0/3488802286 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe200014070 con 0x7fe2041008d0 2024-09-12T02:57:09.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:09.654+0000 7fe20c1d8640 1 -- 172.21.15.5:0/3488802286 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe20419e9b0 con 0x7fe2041008d0 2024-09-12T02:57:09.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:09.654+0000 7fe1faffd640 1 -- 172.21.15.5:0/3488802286 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe200010af0 con 0x7fe2041008d0 2024-09-12T02:57:09.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:09.654+0000 7fe1faffd640 1 -- 172.21.15.5:0/3488802286 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe200017d10 con 0x7fe2041008d0 2024-09-12T02:57:09.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:09.654+0000 7fe20c1d8640 1 -- 172.21.15.5:0/3488802286 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2041a7320 con 0x7fe2041008d0 2024-09-12T02:57:09.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:09.655+0000 7fe20c1d8640 1 -- 172.21.15.5:0/3488802286 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe204100cb0 con 0x7fe2041008d0 2024-09-12T02:57:09.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:09.656+0000 7fe1faffd640 1 -- 172.21.15.5:0/3488802286 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fe200017070 con 0x7fe2041008d0 2024-09-12T02:57:09.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:09.656+0000 7fe1faffd640 1 --2- 172.21.15.5:0/3488802286 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fe1f00a2c10 0x7fe1f00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:09.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:09.657+0000 7fe209773640 1 --2- 172.21.15.5:0/3488802286 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fe1f00a2c10 0x7fe1f00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:09.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:09.657+0000 7fe1faffd640 1 -- 172.21.15.5:0/3488802286 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fe2000ce8c0 con 0x7fe2041008d0 2024-09-12T02:57:09.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:09.657+0000 7fe209773640 1 --2- 172.21.15.5:0/3488802286 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fe1f00a2c10 0x7fe1f00a50d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe1f4005ab0 tx=0x7fe1f4000bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:09.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:09.660+0000 7fe1faffd640 1 -- 172.21.15.5:0/3488802286 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe20009b7e0 con 0x7fe2041008d0 2024-09-12T02:57:09.849 INFO:teuthology.orchestra.run.smithi005.stdout:322122547223 2024-09-12T02:57:09.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:09.840+0000 7fe20c1d8640 1 -- 172.21.15.5:0/3488802286 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7fe20419bad0 con 0x7fe2041008d0 2024-09-12T02:57:09.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:09.840+0000 7fe1faffd640 1 -- 172.21.15.5:0/3488802286 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fe20009b0e0 con 0x7fe2041008d0 2024-09-12T02:57:09.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:09.843+0000 7fe20c1d8640 1 -- 172.21.15.5:0/3488802286 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fe1f00a2c10 msgr2=0x7fe1f00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:09.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:09.843+0000 7fe20c1d8640 1 --2- 172.21.15.5:0/3488802286 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fe1f00a2c10 0x7fe1f00a50d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe1f4005ab0 tx=0x7fe1f4000bc0 comp rx=0 tx=0).stop 2024-09-12T02:57:09.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:09.843+0000 7fe20c1d8640 1 -- 172.21.15.5:0/3488802286 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2041008d0 msgr2=0x7fe20419a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:09.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:09.843+0000 7fe20c1d8640 1 --2- 172.21.15.5:0/3488802286 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2041008d0 0x7fe20419a770 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe20000b4a0 tx=0x7fe20000b960 comp rx=0 tx=0).stop 2024-09-12T02:57:09.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:09.844+0000 7fe20c1d8640 1 -- 172.21.15.5:0/3488802286 shutdown_connections 2024-09-12T02:57:09.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:09.844+0000 7fe20c1d8640 1 --2- 172.21.15.5:0/3488802286 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe20410e480 0x7fe20419df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:09.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:09.844+0000 7fe20c1d8640 1 --2- 172.21.15.5:0/3488802286 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe204101280 0x7fe20419acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:09.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:09.844+0000 7fe20c1d8640 1 --2- 172.21.15.5:0/3488802286 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fe1f00a2c10 0x7fe1f00a50d0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:09.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:09.844+0000 7fe20c1d8640 1 --2- 172.21.15.5:0/3488802286 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2041008d0 0x7fe20419a770 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:09.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:09.844+0000 7fe20c1d8640 1 -- 172.21.15.5:0/3488802286 >> 172.21.15.5:0/3488802286 conn(0x7fe2040fc7a0 msgr2=0x7fe2040feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:09.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:09.844+0000 7fe20c1d8640 1 -- 172.21.15.5:0/3488802286 shutdown_connections 2024-09-12T02:57:09.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:09.844+0000 7fe20c1d8640 1 -- 172.21.15.5:0/3488802286 wait complete. 2024-09-12T02:57:09.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:09.963+0000 7f4f2a097640 1 -- 172.21.15.5:0/2682836500 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f4f24101ec0 con 0x7f4f24102d30 2024-09-12T02:57:09.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:09.963+0000 7f4f20ff9640 1 -- 172.21.15.5:0/2682836500 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f4f24101ec0 con 0x7f4f24102d30 2024-09-12T02:57:09.963 INFO:teuthology.orchestra.run.smithi005.stdout:206158430250 2024-09-12T02:57:09.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:09.965+0000 7f4f2a097640 1 -- 172.21.15.5:0/2682836500 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f4f0c0a2c10 msgr2=0x7f4f0c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:09.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:09.965+0000 7f4f2a097640 1 --2- 172.21.15.5:0/2682836500 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f4f0c0a2c10 0x7f4f0c0a50d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4f24102420 tx=0x7f4f1800b410 comp rx=0 tx=0).stop 2024-09-12T02:57:09.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:09.965+0000 7f4f2a097640 1 -- 172.21.15.5:0/2682836500 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f24102d30 msgr2=0x7f4f24100990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:09.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:09.965+0000 7f4f2a097640 1 --2- 172.21.15.5:0/2682836500 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f24102d30 0x7f4f24100990 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4f1400e9f0 tx=0x7f4f1400edf0 comp rx=0 tx=0).stop 2024-09-12T02:57:09.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:09.965+0000 7f4f2a097640 1 -- 172.21.15.5:0/2682836500 shutdown_connections 2024-09-12T02:57:09.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:09.965+0000 7f4f2a097640 1 --2- 172.21.15.5:0/2682836500 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4f241082a0 0x7f4f24101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:09.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:09.965+0000 7f4f2a097640 1 --2- 172.21.15.5:0/2682836500 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4f241036e0 0x7f4f24100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:09.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:09.965+0000 7f4f2a097640 1 --2- 172.21.15.5:0/2682836500 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f4f0c0a2c10 0x7f4f0c0a50d0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:09.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:09.965+0000 7f4f2a097640 1 --2- 172.21.15.5:0/2682836500 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f24102d30 0x7f4f24100990 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:09.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:09.965+0000 7f4f2a097640 1 -- 172.21.15.5:0/2682836500 >> 172.21.15.5:0/2682836500 conn(0x7f4f240fc7d0 msgr2=0x7f4f24105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:09.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:09.965+0000 7f4f2a097640 1 -- 172.21.15.5:0/2682836500 shutdown_connections 2024-09-12T02:57:09.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:09.966+0000 7f4f2a097640 1 -- 172.21.15.5:0/2682836500 wait complete. 2024-09-12T02:57:10.382 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:57:10 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/3488802286' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-12T02:57:10.382 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:57:10 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/2682836500' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-12T02:57:10.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:57:10 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/3488802286' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-12T02:57:10.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:57:10 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/2682836500' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-12T02:57:10.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:57:10 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/3488802286' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-12T02:57:10.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:57:10 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/2682836500' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-12T02:57:10.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:10.728+0000 7f20e8faf640 1 -- 172.21.15.5:0/1633693537 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20e4101420 msgr2=0x7f20e410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:10.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:10.728+0000 7f20e8faf640 1 --2- 172.21.15.5:0/1633693537 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20e4101420 0x7f20e410d950 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f20dc009cd0 tx=0x7f20dc02a8e0 comp rx=0 tx=0).stop 2024-09-12T02:57:10.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:10.728+0000 7f20e8faf640 1 -- 172.21.15.5:0/1633693537 shutdown_connections 2024-09-12T02:57:10.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:10.728+0000 7f20e8faf640 1 --2- 172.21.15.5:0/1633693537 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20e4101420 0x7f20e410d950 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:10.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:10.728+0000 7f20e8faf640 1 --2- 172.21.15.5:0/1633693537 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f20e4100a80 0x7f20e4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:10.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:10.728+0000 7f20e8faf640 1 --2- 172.21.15.5:0/1633693537 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20e4106a80 0x7f20e4106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:10.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:10.728+0000 7f20e8faf640 1 -- 172.21.15.5:0/1633693537 >> 172.21.15.5:0/1633693537 conn(0x7f20e40fc7a0 msgr2=0x7f20e40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:10.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:10.728+0000 7f20e8faf640 1 -- 172.21.15.5:0/1633693537 shutdown_connections 2024-09-12T02:57:10.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:10.728+0000 7f20e8faf640 1 -- 172.21.15.5:0/1633693537 wait complete. 2024-09-12T02:57:10.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:10.729+0000 7f20e8faf640 1 Processor -- start 2024-09-12T02:57:10.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:10.729+0000 7f20e8faf640 1 -- start start 2024-09-12T02:57:10.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:10.729+0000 7f20e8faf640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f20e4100a80 0x7f20e4198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:10.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:10.729+0000 7f20e8faf640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20e4101420 0x7f20e4198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:10.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:10.729+0000 7f20e8faf640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20e4106a80 0x7f20e419cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:10.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:10.729+0000 7f20e8faf640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f20e4110ea0 con 0x7f20e4106a80 2024-09-12T02:57:10.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:10.729+0000 7f20e8faf640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f20e4110d20 con 0x7f20e4100a80 2024-09-12T02:57:10.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:10.729+0000 7f20e8faf640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f20e4111020 con 0x7f20e4101420 2024-09-12T02:57:10.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:10.729+0000 7f20e2d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20e4106a80 0x7f20e419cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:10.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:10.729+0000 7f20e2d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20e4106a80 0x7f20e419cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42966/0 (socket says 172.21.15.5:42966) 2024-09-12T02:57:10.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:10.729+0000 7f20e2d9d640 1 -- 172.21.15.5:0/2418093368 learned_addr learned my addr 172.21.15.5:0/2418093368 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:57:10.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:10.729+0000 7f20e259c640 1 --2- 172.21.15.5:0/2418093368 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f20e4100a80 0x7f20e4198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:10.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:10.730+0000 7f20d9d9b640 1 --2- 172.21.15.5:0/2418093368 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20e4101420 0x7f20e4198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:10.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:10.730+0000 7f20e2d9d640 1 -- 172.21.15.5:0/2418093368 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20e4101420 msgr2=0x7f20e4198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:10.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:10.730+0000 7f20e2d9d640 1 --2- 172.21.15.5:0/2418093368 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20e4101420 0x7f20e4198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:10.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:10.730+0000 7f20e2d9d640 1 -- 172.21.15.5:0/2418093368 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f20e4100a80 msgr2=0x7f20e4198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:10.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:10.730+0000 7f20e2d9d640 1 --2- 172.21.15.5:0/2418093368 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f20e4100a80 0x7f20e4198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:10.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:10.730+0000 7f20e2d9d640 1 -- 172.21.15.5:0/2418093368 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20e419d3f0 con 0x7f20e4106a80 2024-09-12T02:57:10.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:10.730+0000 7f20d9d9b640 1 --2- 172.21.15.5:0/2418093368 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20e4101420 0x7f20e4198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:57:10.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:10.730+0000 7f20e2d9d640 1 --2- 172.21.15.5:0/2418093368 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20e4106a80 0x7f20e419cd10 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f20dc006010 tx=0x7f20dc003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:10.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:10.730+0000 7f20dbfff640 1 -- 172.21.15.5:0/2418093368 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20dc004450 con 0x7f20e4106a80 2024-09-12T02:57:10.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:10.730+0000 7f20dbfff640 1 -- 172.21.15.5:0/2418093368 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20dc003e10 con 0x7f20e4106a80 2024-09-12T02:57:10.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:10.730+0000 7f20e8faf640 1 -- 172.21.15.5:0/2418093368 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20e419d680 con 0x7f20e4106a80 2024-09-12T02:57:10.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:10.730+0000 7f20dbfff640 1 -- 172.21.15.5:0/2418093368 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20dc046480 con 0x7f20e4106a80 2024-09-12T02:57:10.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:10.730+0000 7f20e8faf640 1 -- 172.21.15.5:0/2418093368 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20e41a4fc0 con 0x7f20e4106a80 2024-09-12T02:57:10.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:10.731+0000 7f20e8faf640 1 -- 172.21.15.5:0/2418093368 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20e4100ee0 con 0x7f20e4106a80 2024-09-12T02:57:10.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:10.732+0000 7f20dbfff640 1 -- 172.21.15.5:0/2418093368 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f20dc0045f0 con 0x7f20e4106a80 2024-09-12T02:57:10.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:10.733+0000 7f20dbfff640 1 --2- 172.21.15.5:0/2418093368 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f20d00a2c90 0x7f20d00a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:10.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:10.733+0000 7f20e259c640 1 --2- 172.21.15.5:0/2418093368 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f20d00a2c90 0x7f20d00a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:10.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:10.733+0000 7f20dbfff640 1 -- 172.21.15.5:0/2418093368 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f20dc02e030 con 0x7f20e4106a80 2024-09-12T02:57:10.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:10.734+0000 7f20e259c640 1 --2- 172.21.15.5:0/2418093368 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f20d00a2c90 0x7f20d00a5150 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f20d4009990 tx=0x7f20d4009110 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:10.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:10.736+0000 7f20dbfff640 1 -- 172.21.15.5:0/2418093368 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f20dc04b060 con 0x7f20e4106a80 2024-09-12T02:57:10.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:10.921+0000 7f20e8faf640 1 -- 172.21.15.5:0/2418093368 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f20e41997a0 con 0x7f20e4106a80 2024-09-12T02:57:10.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:10.921+0000 7f20dbfff640 1 -- 172.21.15.5:0/2418093368 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f20e41997a0 con 0x7f20e4106a80 2024-09-12T02:57:10.922 INFO:teuthology.orchestra.run.smithi005.stdout:77309411396 2024-09-12T02:57:10.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:10.925+0000 7f20e8faf640 1 -- 172.21.15.5:0/2418093368 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f20d00a2c90 msgr2=0x7f20d00a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:10.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:10.925+0000 7f20e8faf640 1 --2- 172.21.15.5:0/2418093368 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f20d00a2c90 0x7f20d00a5150 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f20d4009990 tx=0x7f20d4009110 comp rx=0 tx=0).stop 2024-09-12T02:57:10.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:10.925+0000 7f20e8faf640 1 -- 172.21.15.5:0/2418093368 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20e4106a80 msgr2=0x7f20e419cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:10.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:10.925+0000 7f20e8faf640 1 --2- 172.21.15.5:0/2418093368 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20e4106a80 0x7f20e419cd10 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f20dc006010 tx=0x7f20dc003c20 comp rx=0 tx=0).stop 2024-09-12T02:57:10.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:10.925+0000 7f20e8faf640 1 -- 172.21.15.5:0/2418093368 shutdown_connections 2024-09-12T02:57:10.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:10.925+0000 7f20e8faf640 1 --2- 172.21.15.5:0/2418093368 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f20d00a2c90 0x7f20d00a5150 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:10.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:10.925+0000 7f20e8faf640 1 --2- 172.21.15.5:0/2418093368 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20e4106a80 0x7f20e419cd10 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:10.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:10.925+0000 7f20e8faf640 1 --2- 172.21.15.5:0/2418093368 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f20e4101420 0x7f20e4198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:10.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:10.925+0000 7f20e8faf640 1 --2- 172.21.15.5:0/2418093368 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f20e4100a80 0x7f20e4198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:10.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:10.925+0000 7f20e8faf640 1 -- 172.21.15.5:0/2418093368 >> 172.21.15.5:0/2418093368 conn(0x7f20e40fc7a0 msgr2=0x7f20e406aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:10.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:10.925+0000 7f20e8faf640 1 -- 172.21.15.5:0/2418093368 shutdown_connections 2024-09-12T02:57:10.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:10.925+0000 7f20e8faf640 1 -- 172.21.15.5:0/2418093368 wait complete. 2024-09-12T02:57:11.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:57:11 smithi005 ceph-mon[31197]: pgmap v303: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-09-12T02:57:11.291 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:57:11 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/2418093368' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-12T02:57:11.320 INFO:tasks.cephadm.ceph_manager.ceph:need seq 206158430248 got 206158430250 for osd.7 2024-09-12T02:57:11.320 DEBUG:teuthology.parallel:result is None 2024-09-12T02:57:11.415 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:57:11 smithi148 ceph-mon[35090]: pgmap v303: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-09-12T02:57:11.415 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:57:11 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/2418093368' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-12T02:57:11.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:57:11 smithi146 ceph-mon[35011]: pgmap v303: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-09-12T02:57:11.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:57:11 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/2418093368' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-12T02:57:11.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:11.491+0000 7f8e7cf5c640 1 -- 172.21.15.5:0/856809351 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e78101420 msgr2=0x7f8e7810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:11.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:11.491+0000 7f8e7cf5c640 1 --2- 172.21.15.5:0/856809351 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e78101420 0x7f8e7810d950 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f8e70009cd0 tx=0x7f8e7002a8e0 comp rx=0 tx=0).stop 2024-09-12T02:57:11.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:11.491+0000 7f8e7cf5c640 1 -- 172.21.15.5:0/856809351 shutdown_connections 2024-09-12T02:57:11.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:11.491+0000 7f8e7cf5c640 1 --2- 172.21.15.5:0/856809351 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e78101420 0x7f8e7810d950 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:11.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:11.491+0000 7f8e7cf5c640 1 --2- 172.21.15.5:0/856809351 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e78100a80 0x7f8e78100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:11.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:11.491+0000 7f8e7cf5c640 1 --2- 172.21.15.5:0/856809351 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e78106a80 0x7f8e78106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:11.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:11.491+0000 7f8e7cf5c640 1 -- 172.21.15.5:0/856809351 >> 172.21.15.5:0/856809351 conn(0x7f8e780fc7a0 msgr2=0x7f8e780febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:11.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:11.491+0000 7f8e7cf5c640 1 -- 172.21.15.5:0/856809351 shutdown_connections 2024-09-12T02:57:11.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:11.491+0000 7f8e7cf5c640 1 -- 172.21.15.5:0/856809351 wait complete. 2024-09-12T02:57:11.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:11.492+0000 7f8e7cf5c640 1 Processor -- start 2024-09-12T02:57:11.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:11.492+0000 7f8e7cf5c640 1 -- start start 2024-09-12T02:57:11.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:11.492+0000 7f8e7cf5c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e78100a80 0x7f8e780fcd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:11.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:11.492+0000 7f8e7cf5c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e78101420 0x7f8e780fd2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:11.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:11.492+0000 7f8e7cf5c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e78106a80 0x7f8e780fd810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:11.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:11.492+0000 7f8e7cf5c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8e78110e80 con 0x7f8e78100a80 2024-09-12T02:57:11.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:11.492+0000 7f8e7cf5c640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f8e78110d00 con 0x7f8e78101420 2024-09-12T02:57:11.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:11.492+0000 7f8e7cf5c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8e78111000 con 0x7f8e78106a80 2024-09-12T02:57:11.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:11.492+0000 7f8e7659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e78100a80 0x7f8e780fcd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:11.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:11.492+0000 7f8e7659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e78100a80 0x7f8e780fcd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42994/0 (socket says 172.21.15.5:42994) 2024-09-12T02:57:11.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:11.492+0000 7f8e7659c640 1 -- 172.21.15.5:0/4222955716 learned_addr learned my addr 172.21.15.5:0/4222955716 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:57:11.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:11.492+0000 7f8e75d9b640 1 --2- 172.21.15.5:0/4222955716 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e78101420 0x7f8e780fd2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:11.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:11.493+0000 7f8e7659c640 1 -- 172.21.15.5:0/4222955716 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e78106a80 msgr2=0x7f8e780fd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:11.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:11.493+0000 7f8e7659c640 1 --2- 172.21.15.5:0/4222955716 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e78106a80 0x7f8e780fd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:11.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:11.493+0000 7f8e7659c640 1 -- 172.21.15.5:0/4222955716 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e78101420 msgr2=0x7f8e780fd2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:11.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:11.493+0000 7f8e7659c640 1 --2- 172.21.15.5:0/4222955716 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e78101420 0x7f8e780fd2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:11.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:11.493+0000 7f8e7659c640 1 -- 172.21.15.5:0/4222955716 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e781a6930 con 0x7f8e78100a80 2024-09-12T02:57:11.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:11.493+0000 7f8e7659c640 1 --2- 172.21.15.5:0/4222955716 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e78100a80 0x7f8e780fcd90 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f8e6800d770 tx=0x7f8e6800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:11.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:11.493+0000 7f8e677fe640 1 -- 172.21.15.5:0/4222955716 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e68017070 con 0x7f8e78100a80 2024-09-12T02:57:11.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:11.493+0000 7f8e7cf5c640 1 -- 172.21.15.5:0/4222955716 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e781a6c20 con 0x7f8e78100a80 2024-09-12T02:57:11.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:11.493+0000 7f8e677fe640 1 -- 172.21.15.5:0/4222955716 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e68005500 con 0x7f8e78100a80 2024-09-12T02:57:11.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:11.493+0000 7f8e677fe640 1 -- 172.21.15.5:0/4222955716 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e68004e00 con 0x7f8e78100a80 2024-09-12T02:57:11.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:11.493+0000 7f8e7cf5c640 1 -- 172.21.15.5:0/4222955716 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e781a7110 con 0x7f8e78100a80 2024-09-12T02:57:11.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:11.494+0000 7f8e7cf5c640 1 -- 172.21.15.5:0/4222955716 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e78100ee0 con 0x7f8e78100a80 2024-09-12T02:57:11.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:11.495+0000 7f8e677fe640 1 -- 172.21.15.5:0/4222955716 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f8e68004fa0 con 0x7f8e78100a80 2024-09-12T02:57:11.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:11.496+0000 7f8e677fe640 1 --2- 172.21.15.5:0/4222955716 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f8e600a2c10 0x7f8e600a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:11.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:11.496+0000 7f8e75d9b640 1 --2- 172.21.15.5:0/4222955716 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f8e600a2c10 0x7f8e600a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:11.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:11.496+0000 7f8e677fe640 1 -- 172.21.15.5:0/4222955716 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f8e680cecc0 con 0x7f8e78100a80 2024-09-12T02:57:11.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:11.496+0000 7f8e75d9b640 1 --2- 172.21.15.5:0/4222955716 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f8e600a2c10 0x7f8e600a50d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c005fd0 tx=0x7f8e6c005850 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:11.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:11.500+0000 7f8e677fe640 1 -- 172.21.15.5:0/4222955716 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8e680d5080 con 0x7f8e78100a80 2024-09-12T02:57:11.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:11.677+0000 7f8e7cf5c640 1 -- 172.21.15.5:0/4222955716 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f8e7810a030 con 0x7f8e78100a80 2024-09-12T02:57:11.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:11.678+0000 7f8e677fe640 1 -- 172.21.15.5:0/4222955716 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f8e6809ba70 con 0x7f8e78100a80 2024-09-12T02:57:11.679 INFO:teuthology.orchestra.run.smithi005.stdout:240518168615 2024-09-12T02:57:11.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:11.680+0000 7f8e7cf5c640 1 -- 172.21.15.5:0/4222955716 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f8e600a2c10 msgr2=0x7f8e600a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:11.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:11.680+0000 7f8e7cf5c640 1 --2- 172.21.15.5:0/4222955716 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f8e600a2c10 0x7f8e600a50d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c005fd0 tx=0x7f8e6c005850 comp rx=0 tx=0).stop 2024-09-12T02:57:11.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:11.680+0000 7f8e7cf5c640 1 -- 172.21.15.5:0/4222955716 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e78100a80 msgr2=0x7f8e780fcd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:11.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:11.680+0000 7f8e7cf5c640 1 --2- 172.21.15.5:0/4222955716 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e78100a80 0x7f8e780fcd90 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f8e6800d770 tx=0x7f8e6800dc30 comp rx=0 tx=0).stop 2024-09-12T02:57:11.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:11.680+0000 7f8e7cf5c640 1 -- 172.21.15.5:0/4222955716 shutdown_connections 2024-09-12T02:57:11.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:11.680+0000 7f8e7cf5c640 1 --2- 172.21.15.5:0/4222955716 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8e78106a80 0x7f8e780fd810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:11.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:11.680+0000 7f8e7cf5c640 1 --2- 172.21.15.5:0/4222955716 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8e78101420 0x7f8e780fd2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:11.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:11.680+0000 7f8e7cf5c640 1 --2- 172.21.15.5:0/4222955716 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f8e600a2c10 0x7f8e600a50d0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:11.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:11.680+0000 7f8e7cf5c640 1 --2- 172.21.15.5:0/4222955716 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e78100a80 0x7f8e780fcd90 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:11.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:11.680+0000 7f8e7cf5c640 1 -- 172.21.15.5:0/4222955716 >> 172.21.15.5:0/4222955716 conn(0x7f8e780fc7a0 msgr2=0x7f8e7806aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:11.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:11.680+0000 7f8e7cf5c640 1 -- 172.21.15.5:0/4222955716 shutdown_connections 2024-09-12T02:57:11.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:11.680+0000 7f8e7cf5c640 1 -- 172.21.15.5:0/4222955716 wait complete. 2024-09-12T02:57:12.001 INFO:tasks.cephadm.ceph_manager.ceph:need seq 322122547221 got 322122547223 for osd.11 2024-09-12T02:57:12.002 DEBUG:teuthology.parallel:result is None 2024-09-12T02:57:12.259 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:57:12 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/4222955716' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-12T02:57:12.464 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:57:12 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:57:12.466 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:57:12 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/4222955716' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-12T02:57:12.466 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:57:12 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:57:12.466 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:57:12 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/4222955716' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-12T02:57:12.466 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:57:12 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:57:12.547 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411393 got 77309411396 for osd.2 2024-09-12T02:57:12.548 DEBUG:teuthology.parallel:result is None 2024-09-12T02:57:13.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:13.107+0000 7f1fcbef4640 1 -- 172.21.15.5:0/3296683573 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1fc4102d30 msgr2=0x7f1fc4103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:13.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:13.107+0000 7f1fcbef4640 1 --2- 172.21.15.5:0/3296683573 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1fc4102d30 0x7f1fc4103110 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f1fc0009930 tx=0x7f1fc002a420 comp rx=0 tx=0).stop 2024-09-12T02:57:13.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:13.107+0000 7f1fcbef4640 1 -- 172.21.15.5:0/3296683573 shutdown_connections 2024-09-12T02:57:13.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:13.107+0000 7f1fcbef4640 1 --2- 172.21.15.5:0/3296683573 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1fc41082a0 0x7f1fc410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:13.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:13.107+0000 7f1fcbef4640 1 --2- 172.21.15.5:0/3296683573 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1fc41036e0 0x7f1fc4107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:13.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:13.107+0000 7f1fcbef4640 1 --2- 172.21.15.5:0/3296683573 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1fc4102d30 0x7f1fc4103110 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:13.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:13.107+0000 7f1fcbef4640 1 -- 172.21.15.5:0/3296683573 >> 172.21.15.5:0/3296683573 conn(0x7f1fc40fc7d0 msgr2=0x7f1fc40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:13.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:13.108+0000 7f1fcbef4640 1 -- 172.21.15.5:0/3296683573 shutdown_connections 2024-09-12T02:57:13.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:13.108+0000 7f1fcbef4640 1 -- 172.21.15.5:0/3296683573 wait complete. 2024-09-12T02:57:13.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:13.108+0000 7f1fcbef4640 1 Processor -- start 2024-09-12T02:57:13.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:13.108+0000 7f1fcbef4640 1 -- start start 2024-09-12T02:57:13.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:13.108+0000 7f1fcbef4640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1fc4102d30 0x7f1fc4100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:13.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:13.108+0000 7f1fcbef4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1fc41036e0 0x7f1fc4100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:13.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:13.108+0000 7f1fcbef4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1fc41082a0 0x7f1fc4101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:13.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:13.108+0000 7f1fcbef4640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1fc41120a0 con 0x7f1fc41036e0 2024-09-12T02:57:13.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:13.108+0000 7f1fcbef4640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f1fc4111f20 con 0x7f1fc4102d30 2024-09-12T02:57:13.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:13.108+0000 7f1fcbef4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1fc4112220 con 0x7f1fc41082a0 2024-09-12T02:57:13.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:13.109+0000 7f1fc948f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1fc41036e0 0x7f1fc4100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:13.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:13.109+0000 7f1fc948f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1fc41036e0 0x7f1fc4100ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43014/0 (socket says 172.21.15.5:43014) 2024-09-12T02:57:13.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:13.109+0000 7f1fc948f640 1 -- 172.21.15.5:0/2185405010 learned_addr learned my addr 172.21.15.5:0/2185405010 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:57:13.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:13.109+0000 7f1fc9c90640 1 --2- 172.21.15.5:0/2185405010 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1fc4102d30 0x7f1fc4100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:13.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:13.109+0000 7f1fca491640 1 --2- 172.21.15.5:0/2185405010 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1fc41082a0 0x7f1fc4101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:13.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:13.109+0000 7f1fc948f640 1 -- 172.21.15.5:0/2185405010 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1fc41082a0 msgr2=0x7f1fc4101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:13.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:13.109+0000 7f1fc948f640 1 --2- 172.21.15.5:0/2185405010 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1fc41082a0 0x7f1fc4101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:13.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:13.109+0000 7f1fc948f640 1 -- 172.21.15.5:0/2185405010 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1fc4102d30 msgr2=0x7f1fc4100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:13.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:13.109+0000 7f1fc948f640 1 --2- 172.21.15.5:0/2185405010 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1fc4102d30 0x7f1fc4100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:13.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:13.109+0000 7f1fc948f640 1 -- 172.21.15.5:0/2185405010 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fc41aaf90 con 0x7f1fc41036e0 2024-09-12T02:57:13.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:13.109+0000 7f1fc948f640 1 --2- 172.21.15.5:0/2185405010 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1fc41036e0 0x7f1fc4100ed0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f1fb400e9f0 tx=0x7f1fb400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:13.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:13.109+0000 7f1fbaffd640 1 -- 172.21.15.5:0/2185405010 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fb400ccc0 con 0x7f1fc41036e0 2024-09-12T02:57:13.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:13.109+0000 7f1fcbef4640 1 -- 172.21.15.5:0/2185405010 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fc41ab1c0 con 0x7f1fc41036e0 2024-09-12T02:57:13.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:13.109+0000 7f1fbaffd640 1 -- 172.21.15.5:0/2185405010 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1fb4010910 con 0x7f1fc41036e0 2024-09-12T02:57:13.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:13.109+0000 7f1fbaffd640 1 -- 172.21.15.5:0/2185405010 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fb4018920 con 0x7f1fc41036e0 2024-09-12T02:57:13.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:13.109+0000 7f1fcbef4640 1 -- 172.21.15.5:0/2185405010 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fc41ab700 con 0x7f1fc41036e0 2024-09-12T02:57:13.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:13.110+0000 7f1fcbef4640 1 -- 172.21.15.5:0/2185405010 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fc406a850 con 0x7f1fc41036e0 2024-09-12T02:57:13.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:13.111+0000 7f1fbaffd640 1 -- 172.21.15.5:0/2185405010 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f1fb4010ac0 con 0x7f1fc41036e0 2024-09-12T02:57:13.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:13.112+0000 7f1fbaffd640 1 --2- 172.21.15.5:0/2185405010 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f1fb00a2c10 0x7f1fb00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:13.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:13.112+0000 7f1fbaffd640 1 -- 172.21.15.5:0/2185405010 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f1fb40cf4b0 con 0x7f1fc41036e0 2024-09-12T02:57:13.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:13.112+0000 7f1fc9c90640 1 --2- 172.21.15.5:0/2185405010 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f1fb00a2c10 0x7f1fb00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:13.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:13.113+0000 7f1fc9c90640 1 --2- 172.21.15.5:0/2185405010 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f1fb00a2c10 0x7f1fb00a50d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1fc0009900 tx=0x7f1fc00058d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:13.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:13.117+0000 7f1fbaffd640 1 -- 172.21.15.5:0/2185405010 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1fb409bc10 con 0x7f1fc41036e0 2024-09-12T02:57:13.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:13.047+0000 7f887c9a2640 1 -- 172.21.15.5:0/1966211287 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8878100e20 msgr2=0x7f8878101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:13.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:13.047+0000 7f887c9a2640 1 --2- 172.21.15.5:0/1966211287 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8878100e20 0x7f8878101200 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f8868009900 tx=0x7f886802a480 comp rx=0 tx=0).stop 2024-09-12T02:57:13.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:13.048+0000 7f887c9a2640 1 -- 172.21.15.5:0/1966211287 shutdown_connections 2024-09-12T02:57:13.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:13.048+0000 7f887c9a2640 1 --2- 172.21.15.5:0/1966211287 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8878102ef0 0x7f887810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:13.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:13.048+0000 7f887c9a2640 1 --2- 172.21.15.5:0/1966211287 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8878102570 0x7f88781029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:13.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:13.048+0000 7f887c9a2640 1 --2- 172.21.15.5:0/1966211287 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8878100e20 0x7f8878101200 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:13.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:13.048+0000 7f887c9a2640 1 -- 172.21.15.5:0/1966211287 >> 172.21.15.5:0/1966211287 conn(0x7f88780fc7d0 msgr2=0x7f88780febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:13.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:13.048+0000 7f887c9a2640 1 -- 172.21.15.5:0/1966211287 shutdown_connections 2024-09-12T02:57:13.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:13.048+0000 7f887c9a2640 1 -- 172.21.15.5:0/1966211287 wait complete. 2024-09-12T02:57:13.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:13.049+0000 7f887c9a2640 1 Processor -- start 2024-09-12T02:57:13.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:13.049+0000 7f887c9a2640 1 -- start start 2024-09-12T02:57:13.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:13.049+0000 7f887c9a2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8878100e20 0x7f887810e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:13.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:13.049+0000 7f887c9a2640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8878102570 0x7f887810ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:13.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:13.049+0000 7f887c9a2640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8878102ef0 0x7f887810ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:13.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:13.049+0000 7f887c9a2640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f88781120a0 con 0x7f8878100e20 2024-09-12T02:57:13.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:13.049+0000 7f887c9a2640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f8878111f20 con 0x7f8878102ef0 2024-09-12T02:57:13.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:13.049+0000 7f887c9a2640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8878112220 con 0x7f8878102570 2024-09-12T02:57:13.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:13.049+0000 7f887659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8878100e20 0x7f887810e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:13.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:13.049+0000 7f886ffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8878102570 0x7f887810ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:13.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:13.049+0000 7f887659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8878100e20 0x7f887810e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43004/0 (socket says 172.21.15.5:43004) 2024-09-12T02:57:13.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:13.049+0000 7f887659c640 1 -- 172.21.15.5:0/2329750184 learned_addr learned my addr 172.21.15.5:0/2329750184 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:57:13.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:13.049+0000 7f8876d9d640 1 --2- 172.21.15.5:0/2329750184 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8878102ef0 0x7f887810ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:13.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:13.050+0000 7f887659c640 1 -- 172.21.15.5:0/2329750184 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8878102570 msgr2=0x7f887810ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:13.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:13.050+0000 7f887659c640 1 --2- 172.21.15.5:0/2329750184 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8878102570 0x7f887810ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:13.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:13.050+0000 7f887659c640 1 -- 172.21.15.5:0/2329750184 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8878102ef0 msgr2=0x7f887810ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:13.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:13.050+0000 7f887659c640 1 --2- 172.21.15.5:0/2329750184 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8878102ef0 0x7f887810ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:13.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:13.050+0000 7f887659c640 1 -- 172.21.15.5:0/2329750184 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88781a6a70 con 0x7f8878100e20 2024-09-12T02:57:13.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:13.050+0000 7f886ffff640 1 --2- 172.21.15.5:0/2329750184 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8878102570 0x7f887810ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:57:13.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:13.050+0000 7f8876d9d640 1 --2- 172.21.15.5:0/2329750184 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8878102ef0 0x7f887810ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:57:13.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:13.050+0000 7f887659c640 1 --2- 172.21.15.5:0/2329750184 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8878100e20 0x7f887810e4d0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f8868006010 tx=0x7f8868003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:13.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:13.050+0000 7f886f7fe640 1 -- 172.21.15.5:0/2329750184 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f886803f070 con 0x7f8878100e20 2024-09-12T02:57:13.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:13.050+0000 7f887c9a2640 1 -- 172.21.15.5:0/2329750184 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88781a6d00 con 0x7f8878100e20 2024-09-12T02:57:13.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:13.050+0000 7f886f7fe640 1 -- 172.21.15.5:0/2329750184 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8868031c30 con 0x7f8878100e20 2024-09-12T02:57:13.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:13.050+0000 7f886f7fe640 1 -- 172.21.15.5:0/2329750184 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8868035030 con 0x7f8878100e20 2024-09-12T02:57:13.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:13.050+0000 7f887c9a2640 1 -- 172.21.15.5:0/2329750184 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88781a7190 con 0x7f8878100e20 2024-09-12T02:57:13.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:13.051+0000 7f887c9a2640 1 -- 172.21.15.5:0/2329750184 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8878101200 con 0x7f8878100e20 2024-09-12T02:57:13.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:13.052+0000 7f886f7fe640 1 -- 172.21.15.5:0/2329750184 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f8868042070 con 0x7f8878100e20 2024-09-12T02:57:13.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:13.053+0000 7f886f7fe640 1 --2- 172.21.15.5:0/2329750184 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f88640a2c10 0x7f88640a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:13.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:13.053+0000 7f886f7fe640 1 -- 172.21.15.5:0/2329750184 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f88680ef980 con 0x7f8878100e20 2024-09-12T02:57:13.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:13.053+0000 7f886ffff640 1 --2- 172.21.15.5:0/2329750184 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f88640a2c10 0x7f88640a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:13.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:13.054+0000 7f886ffff640 1 --2- 172.21.15.5:0/2329750184 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f88640a2c10 0x7f88640a50d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f88780fff40 tx=0x7f886000b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:13.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:13.058+0000 7f886f7fe640 1 -- 172.21.15.5:0/2329750184 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8868031de0 con 0x7f8878100e20 2024-09-12T02:57:13.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:13.244+0000 7f887c9a2640 1 -- 172.21.15.5:0/2329750184 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f8878109090 con 0x7f8878100e20 2024-09-12T02:57:13.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:13.245+0000 7f886f7fe640 1 -- 172.21.15.5:0/2329750184 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f88680bc140 con 0x7f8878100e20 2024-09-12T02:57:13.246 INFO:teuthology.orchestra.run.smithi005.stdout:184683593777 2024-09-12T02:57:13.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:13.247+0000 7f887c9a2640 1 -- 172.21.15.5:0/2329750184 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f88640a2c10 msgr2=0x7f88640a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:13.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:13.247+0000 7f887c9a2640 1 --2- 172.21.15.5:0/2329750184 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f88640a2c10 0x7f88640a50d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f88780fff40 tx=0x7f886000b390 comp rx=0 tx=0).stop 2024-09-12T02:57:13.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:13.247+0000 7f887c9a2640 1 -- 172.21.15.5:0/2329750184 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8878100e20 msgr2=0x7f887810e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:13.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:13.247+0000 7f887c9a2640 1 --2- 172.21.15.5:0/2329750184 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8878100e20 0x7f887810e4d0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f8868006010 tx=0x7f8868003c10 comp rx=0 tx=0).stop 2024-09-12T02:57:13.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:13.247+0000 7f887c9a2640 1 -- 172.21.15.5:0/2329750184 shutdown_connections 2024-09-12T02:57:13.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:13.247+0000 7f887c9a2640 1 --2- 172.21.15.5:0/2329750184 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8878102ef0 0x7f887810ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:13.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:13.247+0000 7f887c9a2640 1 --2- 172.21.15.5:0/2329750184 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8878102570 0x7f887810ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:13.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:13.247+0000 7f887c9a2640 1 --2- 172.21.15.5:0/2329750184 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f88640a2c10 0x7f88640a50d0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:13.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:13.247+0000 7f887c9a2640 1 --2- 172.21.15.5:0/2329750184 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8878100e20 0x7f887810e4d0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:13.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:13.247+0000 7f887c9a2640 1 -- 172.21.15.5:0/2329750184 >> 172.21.15.5:0/2329750184 conn(0x7f88780fc7d0 msgr2=0x7f88780fe200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:13.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:13.247+0000 7f887c9a2640 1 -- 172.21.15.5:0/2329750184 shutdown_connections 2024-09-12T02:57:13.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:13.247+0000 7f887c9a2640 1 -- 172.21.15.5:0/2329750184 wait complete. 2024-09-12T02:57:13.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:13.306+0000 7f1fcbef4640 1 -- 172.21.15.5:0/2185405010 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f1fc410a2b0 con 0x7f1fc41036e0 2024-09-12T02:57:13.308 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:57:13 smithi005 ceph-mon[31197]: pgmap v304: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T02:57:13.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:13.306+0000 7f1fbaffd640 1 -- 172.21.15.5:0/2185405010 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f1fb409ff10 con 0x7f1fc41036e0 2024-09-12T02:57:13.309 INFO:teuthology.orchestra.run.smithi005.stdout:55834574920 2024-09-12T02:57:13.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:13.309+0000 7f1fcbef4640 1 -- 172.21.15.5:0/2185405010 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f1fb00a2c10 msgr2=0x7f1fb00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:13.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:13.309+0000 7f1fcbef4640 1 --2- 172.21.15.5:0/2185405010 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f1fb00a2c10 0x7f1fb00a50d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1fc0009900 tx=0x7f1fc00058d0 comp rx=0 tx=0).stop 2024-09-12T02:57:13.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:13.309+0000 7f1fcbef4640 1 -- 172.21.15.5:0/2185405010 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1fc41036e0 msgr2=0x7f1fc4100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:13.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:13.309+0000 7f1fcbef4640 1 --2- 172.21.15.5:0/2185405010 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1fc41036e0 0x7f1fc4100ed0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f1fb400e9f0 tx=0x7f1fb400eeb0 comp rx=0 tx=0).stop 2024-09-12T02:57:13.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:13.309+0000 7f1fcbef4640 1 -- 172.21.15.5:0/2185405010 shutdown_connections 2024-09-12T02:57:13.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:13.309+0000 7f1fcbef4640 1 --2- 172.21.15.5:0/2185405010 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f1fb00a2c10 0x7f1fb00a50d0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:13.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:13.309+0000 7f1fcbef4640 1 --2- 172.21.15.5:0/2185405010 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1fc41036e0 0x7f1fc4100ed0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:13.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:13.309+0000 7f1fcbef4640 1 --2- 172.21.15.5:0/2185405010 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1fc41082a0 0x7f1fc4101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:13.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:13.309+0000 7f1fcbef4640 1 --2- 172.21.15.5:0/2185405010 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1fc4102d30 0x7f1fc4100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:13.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:13.309+0000 7f1fcbef4640 1 -- 172.21.15.5:0/2185405010 >> 172.21.15.5:0/2185405010 conn(0x7f1fc40fc7d0 msgr2=0x7f1fc4105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:13.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:13.309+0000 7f1fcbef4640 1 -- 172.21.15.5:0/2185405010 shutdown_connections 2024-09-12T02:57:13.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:13.309+0000 7f1fcbef4640 1 -- 172.21.15.5:0/2185405010 wait complete. 2024-09-12T02:57:13.415 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:57:13 smithi148 ceph-mon[35090]: pgmap v304: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T02:57:13.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:57:13 smithi146 ceph-mon[35011]: pgmap v304: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T02:57:13.874 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738442 got 34359738445 for osd.0 2024-09-12T02:57:13.875 DEBUG:teuthology.parallel:result is None 2024-09-12T02:57:14.359 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:57:14 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:57:14.360 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:57:14 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:57:14.360 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:57:14 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:57:14.360 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:57:14 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:57:14.360 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:57:14 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/2329750184' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-12T02:57:14.360 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:57:14 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/2185405010' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-12T02:57:14.391 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574917 got 55834574920 for osd.1 2024-09-12T02:57:14.391 DEBUG:teuthology.parallel:result is None 2024-09-12T02:57:14.411 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:57:14.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:57:14 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:57:14.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:57:14 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:57:14.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:57:14 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:57:14.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:57:14 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:57:14.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:57:14 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/2329750184' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-12T02:57:14.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:57:14 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/2185405010' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-12T02:57:14.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:57:14 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:57:14.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:57:14 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:57:14.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:57:14 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:57:14.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:57:14 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:57:14.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:57:14 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/2329750184' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-12T02:57:14.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:57:14 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/2185405010' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-12T02:57:14.866 INFO:tasks.cephadm.ceph_manager.ceph:need seq 184683593773 got 184683593777 for osd.6 2024-09-12T02:57:14.866 DEBUG:teuthology.parallel:result is None 2024-09-12T02:57:15.236 INFO:tasks.cephadm.ceph_manager.ceph:need seq 240518168612 got 240518168615 for osd.8 2024-09-12T02:57:15.236 DEBUG:teuthology.parallel:result is None 2024-09-12T02:57:15.475 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:57:15 smithi005 ceph-mon[31197]: pgmap v305: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-12T02:57:15.665 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:57:15 smithi148 ceph-mon[35090]: pgmap v305: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-12T02:57:15.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:15.698+0000 7f6a904eb640 1 -- 172.21.15.5:0/1756212620 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a88100e20 msgr2=0x7f6a88101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:15.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:15.698+0000 7f6a904eb640 1 --2- 172.21.15.5:0/1756212620 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a88100e20 0x7f6a88101200 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f6a84009900 tx=0x7f6a8402a480 comp rx=0 tx=0).stop 2024-09-12T02:57:15.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:15.698+0000 7f6a904eb640 1 -- 172.21.15.5:0/1756212620 shutdown_connections 2024-09-12T02:57:15.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:15.698+0000 7f6a904eb640 1 --2- 172.21.15.5:0/1756212620 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6a88102ef0 0x7f6a8810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:15.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:15.698+0000 7f6a904eb640 1 --2- 172.21.15.5:0/1756212620 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a88102570 0x7f6a881029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:15.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:15.698+0000 7f6a904eb640 1 --2- 172.21.15.5:0/1756212620 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a88100e20 0x7f6a88101200 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:15.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:15.698+0000 7f6a904eb640 1 -- 172.21.15.5:0/1756212620 >> 172.21.15.5:0/1756212620 conn(0x7f6a880fc7d0 msgr2=0x7f6a880febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:15.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:15.698+0000 7f6a904eb640 1 -- 172.21.15.5:0/1756212620 shutdown_connections 2024-09-12T02:57:15.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:15.698+0000 7f6a904eb640 1 -- 172.21.15.5:0/1756212620 wait complete. 2024-09-12T02:57:15.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:15.698+0000 7f6a904eb640 1 Processor -- start 2024-09-12T02:57:15.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:15.698+0000 7f6a904eb640 1 -- start start 2024-09-12T02:57:15.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:15.698+0000 7f6a904eb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a88100e20 0x7f6a88198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:15.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:15.698+0000 7f6a904eb640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6a88102570 0x7f6a88198b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:15.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:15.698+0000 7f6a904eb640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a88102ef0 0x7f6a8819ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:15.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:15.698+0000 7f6a904eb640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6a881120a0 con 0x7f6a88100e20 2024-09-12T02:57:15.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:15.698+0000 7f6a904eb640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f6a88111f20 con 0x7f6a88102570 2024-09-12T02:57:15.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:15.698+0000 7f6a904eb640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f6a88112220 con 0x7f6a88102ef0 2024-09-12T02:57:15.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:15.698+0000 7f6a8e287640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a88100e20 0x7f6a88198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:15.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:15.698+0000 7f6a8e287640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a88100e20 0x7f6a88198600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59860/0 (socket says 172.21.15.5:59860) 2024-09-12T02:57:15.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:15.698+0000 7f6a8e287640 1 -- 172.21.15.5:0/2146643867 learned_addr learned my addr 172.21.15.5:0/2146643867 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:57:15.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:15.698+0000 7f6a8ea88640 1 --2- 172.21.15.5:0/2146643867 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a88102ef0 0x7f6a8819ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:15.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:15.698+0000 7f6a8da86640 1 --2- 172.21.15.5:0/2146643867 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6a88102570 0x7f6a88198b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:15.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:15.698+0000 7f6a8e287640 1 -- 172.21.15.5:0/2146643867 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a88102ef0 msgr2=0x7f6a8819ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:15.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:15.698+0000 7f6a8e287640 1 --2- 172.21.15.5:0/2146643867 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a88102ef0 0x7f6a8819ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:15.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:15.698+0000 7f6a8e287640 1 -- 172.21.15.5:0/2146643867 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6a88102570 msgr2=0x7f6a88198b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:15.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:15.698+0000 7f6a8e287640 1 --2- 172.21.15.5:0/2146643867 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6a88102570 0x7f6a88198b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:15.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:15.698+0000 7f6a8e287640 1 -- 172.21.15.5:0/2146643867 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a8819d5b0 con 0x7f6a88100e20 2024-09-12T02:57:15.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:15.698+0000 7f6a8da86640 1 --2- 172.21.15.5:0/2146643867 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6a88102570 0x7f6a88198b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:57:15.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:15.698+0000 7f6a8e287640 1 --2- 172.21.15.5:0/2146643867 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a88100e20 0x7f6a88198600 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f6a84006010 tx=0x7f6a84003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:15.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:15.699+0000 7f6a7f7fe640 1 -- 172.21.15.5:0/2146643867 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a8403f070 con 0x7f6a88100e20 2024-09-12T02:57:15.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:15.699+0000 7f6a7f7fe640 1 -- 172.21.15.5:0/2146643867 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a84031c30 con 0x7f6a88100e20 2024-09-12T02:57:15.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:15.699+0000 7f6a904eb640 1 -- 172.21.15.5:0/2146643867 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a8819d840 con 0x7f6a88100e20 2024-09-12T02:57:15.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:15.699+0000 7f6a7f7fe640 1 -- 172.21.15.5:0/2146643867 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a84035030 con 0x7f6a88100e20 2024-09-12T02:57:15.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:15.699+0000 7f6a904eb640 1 -- 172.21.15.5:0/2146643867 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a881a5180 con 0x7f6a88100e20 2024-09-12T02:57:15.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:15.700+0000 7f6a904eb640 1 -- 172.21.15.5:0/2146643867 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a881021b0 con 0x7f6a88100e20 2024-09-12T02:57:15.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:15.701+0000 7f6a7f7fe640 1 -- 172.21.15.5:0/2146643867 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f6a84042070 con 0x7f6a88100e20 2024-09-12T02:57:15.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:15.702+0000 7f6a7f7fe640 1 --2- 172.21.15.5:0/2146643867 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f6a740a2c10 0x7f6a740a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:15.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:15.702+0000 7f6a8da86640 1 --2- 172.21.15.5:0/2146643867 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f6a740a2c10 0x7f6a740a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:15.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:15.702+0000 7f6a7f7fe640 1 -- 172.21.15.5:0/2146643867 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f6a840efd90 con 0x7f6a88100e20 2024-09-12T02:57:15.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:15.702+0000 7f6a8da86640 1 --2- 172.21.15.5:0/2146643867 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f6a740a2c10 0x7f6a740a50d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6a780058c0 tx=0x7f6a78005850 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:15.713 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:57:15 smithi146 ceph-mon[35011]: pgmap v305: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-12T02:57:15.713 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:15.705+0000 7f6a7f7fe640 1 -- 172.21.15.5:0/2146643867 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6a840bcba0 con 0x7f6a88100e20 2024-09-12T02:57:15.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:15.892+0000 7f6a904eb640 1 -- 172.21.15.5:0/2146643867 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f6a88199960 con 0x7f6a88100e20 2024-09-12T02:57:15.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:15.892+0000 7f6a7f7fe640 1 -- 172.21.15.5:0/2146643867 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f6a840bc4a0 con 0x7f6a88100e20 2024-09-12T02:57:15.893 INFO:teuthology.orchestra.run.smithi005.stdout:270582939682 2024-09-12T02:57:15.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:15.895+0000 7f6a904eb640 1 -- 172.21.15.5:0/2146643867 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f6a740a2c10 msgr2=0x7f6a740a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:15.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:15.895+0000 7f6a904eb640 1 --2- 172.21.15.5:0/2146643867 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f6a740a2c10 0x7f6a740a50d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6a780058c0 tx=0x7f6a78005850 comp rx=0 tx=0).stop 2024-09-12T02:57:15.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:15.895+0000 7f6a904eb640 1 -- 172.21.15.5:0/2146643867 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a88100e20 msgr2=0x7f6a88198600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:15.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:15.895+0000 7f6a904eb640 1 --2- 172.21.15.5:0/2146643867 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a88100e20 0x7f6a88198600 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f6a84006010 tx=0x7f6a84003c10 comp rx=0 tx=0).stop 2024-09-12T02:57:15.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:15.895+0000 7f6a904eb640 1 -- 172.21.15.5:0/2146643867 shutdown_connections 2024-09-12T02:57:15.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:15.895+0000 7f6a904eb640 1 --2- 172.21.15.5:0/2146643867 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6a88102ef0 0x7f6a8819ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:15.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:15.895+0000 7f6a904eb640 1 --2- 172.21.15.5:0/2146643867 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6a88102570 0x7f6a88198b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:15.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:15.895+0000 7f6a904eb640 1 --2- 172.21.15.5:0/2146643867 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f6a740a2c10 0x7f6a740a50d0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:15.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:15.895+0000 7f6a904eb640 1 --2- 172.21.15.5:0/2146643867 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a88100e20 0x7f6a88198600 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:15.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:15.895+0000 7f6a904eb640 1 -- 172.21.15.5:0/2146643867 >> 172.21.15.5:0/2146643867 conn(0x7f6a880fc7d0 msgr2=0x7f6a880febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:15.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:15.895+0000 7f6a904eb640 1 -- 172.21.15.5:0/2146643867 shutdown_connections 2024-09-12T02:57:15.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:15.895+0000 7f6a904eb640 1 -- 172.21.15.5:0/2146643867 wait complete. 2024-09-12T02:57:16.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:57:16 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/2146643867' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-12T02:57:16.665 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:57:16 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/2146643867' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-12T02:57:16.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:57:16 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/2146643867' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-12T02:57:16.819 INFO:tasks.cephadm.ceph_manager.ceph:need seq 270582939681 got 270582939682 for osd.9 2024-09-12T02:57:16.819 DEBUG:teuthology.parallel:result is None 2024-09-12T02:57:16.819 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-12T02:57:16.819 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-12T02:57:17.215 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:57:17.486 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:57:17 smithi005 ceph-mon[31197]: pgmap v306: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-12T02:57:17.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:17.642+0000 7f3902981640 1 -- 172.21.15.5:0/2792031462 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f38fc100a80 msgr2=0x7f38fc100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:17.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:17.642+0000 7f3902981640 1 --2- 172.21.15.5:0/2792031462 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f38fc100a80 0x7f38fc100ee0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f38e4009930 tx=0x7f38e402a420 comp rx=0 tx=0).stop 2024-09-12T02:57:17.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:17.642+0000 7f3902981640 1 -- 172.21.15.5:0/2792031462 shutdown_connections 2024-09-12T02:57:17.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:17.642+0000 7f3902981640 1 --2- 172.21.15.5:0/2792031462 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f38fc100a80 0x7f38fc100ee0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:17.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:17.642+0000 7f3902981640 1 --2- 172.21.15.5:0/2792031462 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38fc101420 0x7f38fc10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:17.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:17.642+0000 7f3902981640 1 --2- 172.21.15.5:0/2792031462 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f38fc106a80 0x7f38fc106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:17.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:17.642+0000 7f3902981640 1 -- 172.21.15.5:0/2792031462 >> 172.21.15.5:0/2792031462 conn(0x7f38fc0fc7a0 msgr2=0x7f38fc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:17.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:17.642+0000 7f3902981640 1 -- 172.21.15.5:0/2792031462 shutdown_connections 2024-09-12T02:57:17.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:17.642+0000 7f3902981640 1 -- 172.21.15.5:0/2792031462 wait complete. 2024-09-12T02:57:17.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:17.643+0000 7f3902981640 1 Processor -- start 2024-09-12T02:57:17.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:17.643+0000 7f3902981640 1 -- start start 2024-09-12T02:57:17.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:17.643+0000 7f3902981640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f38fc100a80 0x7f38fc19a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:17.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:17.643+0000 7f3902981640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38fc101420 0x7f38fc19aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:17.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:17.643+0000 7f3902981640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f38fc106a80 0x7f38fc19ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:17.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:17.643+0000 7f3902981640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f38fc110e40 con 0x7f38fc100a80 2024-09-12T02:57:17.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:17.643+0000 7f3902981640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f38fc110cc0 con 0x7f38fc106a80 2024-09-12T02:57:17.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:17.643+0000 7f3902981640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f38fc110fc0 con 0x7f38fc101420 2024-09-12T02:57:17.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:17.643+0000 7f38fbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f38fc100a80 0x7f38fc19a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:17.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:17.643+0000 7f38fbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f38fc100a80 0x7f38fc19a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59872/0 (socket says 172.21.15.5:59872) 2024-09-12T02:57:17.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:17.643+0000 7f38fbfff640 1 -- 172.21.15.5:0/4009623527 learned_addr learned my addr 172.21.15.5:0/4009623527 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:57:17.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:17.643+0000 7f38f3fff640 1 --2- 172.21.15.5:0/4009623527 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38fc101420 0x7f38fc19aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:17.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:17.643+0000 7f3900f1e640 1 --2- 172.21.15.5:0/4009623527 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f38fc106a80 0x7f38fc19ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:17.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:17.643+0000 7f38fbfff640 1 -- 172.21.15.5:0/4009623527 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38fc101420 msgr2=0x7f38fc19aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:17.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:17.643+0000 7f38fbfff640 1 --2- 172.21.15.5:0/4009623527 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38fc101420 0x7f38fc19aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:17.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:17.643+0000 7f38fbfff640 1 -- 172.21.15.5:0/4009623527 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f38fc106a80 msgr2=0x7f38fc19ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:17.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:17.644+0000 7f38fbfff640 1 --2- 172.21.15.5:0/4009623527 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f38fc106a80 0x7f38fc19ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:17.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:17.644+0000 7f38fbfff640 1 -- 172.21.15.5:0/4009623527 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38fc19f610 con 0x7f38fc100a80 2024-09-12T02:57:17.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:17.644+0000 7f38fbfff640 1 --2- 172.21.15.5:0/4009623527 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f38fc100a80 0x7f38fc19a660 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f38ec00d950 tx=0x7f38ec00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:17.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:17.644+0000 7f38f9ffb640 1 -- 172.21.15.5:0/4009623527 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38ec017030 con 0x7f38fc100a80 2024-09-12T02:57:17.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:17.644+0000 7f38f9ffb640 1 -- 172.21.15.5:0/4009623527 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f38ec004020 con 0x7f38fc100a80 2024-09-12T02:57:17.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:17.644+0000 7f38f9ffb640 1 -- 172.21.15.5:0/4009623527 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38ec0047a0 con 0x7f38fc100a80 2024-09-12T02:57:17.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:17.644+0000 7f3902981640 1 -- 172.21.15.5:0/4009623527 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38fc19f900 con 0x7f38fc100a80 2024-09-12T02:57:17.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:17.644+0000 7f3902981640 1 -- 172.21.15.5:0/4009623527 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38fc1a71e0 con 0x7f38fc100a80 2024-09-12T02:57:17.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:17.645+0000 7f3902981640 1 -- 172.21.15.5:0/4009623527 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38fc100ee0 con 0x7f38fc100a80 2024-09-12T02:57:17.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:17.646+0000 7f38f9ffb640 1 -- 172.21.15.5:0/4009623527 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f38ec0041d0 con 0x7f38fc100a80 2024-09-12T02:57:17.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:17.647+0000 7f38f9ffb640 1 --2- 172.21.15.5:0/4009623527 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f38e80a2c10 0x7f38e80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:17.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:17.647+0000 7f38f9ffb640 1 -- 172.21.15.5:0/4009623527 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f38ec0cebf0 con 0x7f38fc100a80 2024-09-12T02:57:17.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:17.647+0000 7f38f3fff640 1 --2- 172.21.15.5:0/4009623527 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f38e80a2c10 0x7f38e80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:17.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:17.647+0000 7f38f3fff640 1 --2- 172.21.15.5:0/4009623527 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f38e80a2c10 0x7f38e80a50d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f38e4000c00 tx=0x7f38e4005a20 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:17.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:17.651+0000 7f38f9ffb640 1 -- 172.21.15.5:0/4009623527 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f38ec09ba00 con 0x7f38fc100a80 2024-09-12T02:57:17.665 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:57:17 smithi148 ceph-mon[35090]: pgmap v306: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-12T02:57:17.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:57:17 smithi146 ceph-mon[35011]: pgmap v306: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-12T02:57:17.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:17.837+0000 7f3902981640 1 -- 172.21.15.5:0/4009623527 --> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f38fc103a10 con 0x7f38e80a2c10 2024-09-12T02:57:17.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:17.845+0000 7f38f9ffb640 1 -- 172.21.15.5:0/4009623527 <== mgr.14148 v2:172.21.15.5:6800/3780631642 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258760 (secure 0 0 0) 0x7f38fc103a10 con 0x7f38e80a2c10 2024-09-12T02:57:17.848 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-12T02:57:17.849 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-09-12T02:57:17.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:17.848+0000 7f3902981640 1 -- 172.21.15.5:0/4009623527 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f38e80a2c10 msgr2=0x7f38e80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:17.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:17.848+0000 7f3902981640 1 --2- 172.21.15.5:0/4009623527 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f38e80a2c10 0x7f38e80a50d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f38e4000c00 tx=0x7f38e4005a20 comp rx=0 tx=0).stop 2024-09-12T02:57:17.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:17.848+0000 7f3902981640 1 -- 172.21.15.5:0/4009623527 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f38fc100a80 msgr2=0x7f38fc19a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:17.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:17.848+0000 7f3902981640 1 --2- 172.21.15.5:0/4009623527 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f38fc100a80 0x7f38fc19a660 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f38ec00d950 tx=0x7f38ec00de10 comp rx=0 tx=0).stop 2024-09-12T02:57:17.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:17.848+0000 7f3902981640 1 -- 172.21.15.5:0/4009623527 shutdown_connections 2024-09-12T02:57:17.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:17.848+0000 7f3902981640 1 --2- 172.21.15.5:0/4009623527 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f38fc106a80 0x7f38fc19ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:17.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:17.848+0000 7f3902981640 1 --2- 172.21.15.5:0/4009623527 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f38fc101420 0x7f38fc19aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:17.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:17.848+0000 7f3902981640 1 --2- 172.21.15.5:0/4009623527 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f38e80a2c10 0x7f38e80a50d0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:17.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:17.848+0000 7f3902981640 1 --2- 172.21.15.5:0/4009623527 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f38fc100a80 0x7f38fc19a660 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:17.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:17.848+0000 7f3902981640 1 -- 172.21.15.5:0/4009623527 >> 172.21.15.5:0/4009623527 conn(0x7f38fc0fc7a0 msgr2=0x7f38fc06a970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:17.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:17.848+0000 7f3902981640 1 -- 172.21.15.5:0/4009623527 shutdown_connections 2024-09-12T02:57:17.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:17.848+0000 7f3902981640 1 -- 172.21.15.5:0/4009623527 wait complete. 2024-09-12T02:57:18.486 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":307,"stamp":"2024-09-12T02:57:17.557208+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658977,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":636,"num_read_kb":433,"num_write":429,"num_write_kb":3075,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":1181470,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":350,"ondisk_log_size":350,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":374588,"kb_used_data":12828,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124567236,"statfs":{"total":1151940427776,"available":1151556849664,"internally_reserved":0,"allocated":13135872,"data_stored":7244847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":370193723},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":114,"num_read_kb":49,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003243"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.033421+0000","last_change":"2024-09-12T02:56:15.988690+0000","last_active":"2024-09-12T02:56:31.033421+0000","last_peered":"2024-09-12T02:56:31.033421+0000","last_clean":"2024-09-12T02:56:31.033421+0000","last_became_active":"2024-09-12T02:56:15.988320+0000","last_became_peered":"2024-09-12T02:56:15.988320+0000","last_unstale":"2024-09-12T02:56:31.033421+0000","last_undegraded":"2024-09-12T02:56:31.033421+0000","last_fullsized":"2024-09-12T02:56:31.033421+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:31:14.157622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.032544+0000","last_change":"2024-09-12T02:56:15.978368+0000","last_active":"2024-09-12T02:56:31.032544+0000","last_peered":"2024-09-12T02:56:31.032544+0000","last_clean":"2024-09-12T02:56:31.032544+0000","last_became_active":"2024-09-12T02:56:15.978092+0000","last_became_peered":"2024-09-12T02:56:15.978092+0000","last_unstale":"2024-09-12T02:56:31.032544+0000","last_undegraded":"2024-09-12T02:56:31.032544+0000","last_fullsized":"2024-09-12T02:56:31.032544+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:32:48.399005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.032403+0000","last_change":"2024-09-12T02:56:15.987249+0000","last_active":"2024-09-12T02:56:31.032403+0000","last_peered":"2024-09-12T02:56:31.032403+0000","last_clean":"2024-09-12T02:56:31.032403+0000","last_became_active":"2024-09-12T02:56:15.986921+0000","last_became_peered":"2024-09-12T02:56:15.986921+0000","last_unstale":"2024-09-12T02:56:31.032403+0000","last_undegraded":"2024-09-12T02:56:31.032403+0000","last_fullsized":"2024-09-12T02:56:31.032403+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:42:13.427871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.031295+0000","last_change":"2024-09-12T02:56:15.987673+0000","last_active":"2024-09-12T02:56:31.031295+0000","last_peered":"2024-09-12T02:56:31.031295+0000","last_clean":"2024-09-12T02:56:31.031295+0000","last_became_active":"2024-09-12T02:56:15.986964+0000","last_became_peered":"2024-09-12T02:56:15.986964+0000","last_unstale":"2024-09-12T02:56:31.031295+0000","last_undegraded":"2024-09-12T02:56:31.031295+0000","last_fullsized":"2024-09-12T02:56:31.031295+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:07:12.406974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.034804+0000","last_change":"2024-09-12T02:56:15.974473+0000","last_active":"2024-09-12T02:56:31.034804+0000","last_peered":"2024-09-12T02:56:31.034804+0000","last_clean":"2024-09-12T02:56:31.034804+0000","last_became_active":"2024-09-12T02:56:15.974124+0000","last_became_peered":"2024-09-12T02:56:15.974124+0000","last_unstale":"2024-09-12T02:56:31.034804+0000","last_undegraded":"2024-09-12T02:56:31.034804+0000","last_fullsized":"2024-09-12T02:56:31.034804+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:44:29.054841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.032976+0000","last_change":"2024-09-12T02:56:15.980242+0000","last_active":"2024-09-12T02:56:31.032976+0000","last_peered":"2024-09-12T02:56:31.032976+0000","last_clean":"2024-09-12T02:56:31.032976+0000","last_became_active":"2024-09-12T02:56:15.979937+0000","last_became_peered":"2024-09-12T02:56:15.979937+0000","last_unstale":"2024-09-12T02:56:31.032976+0000","last_undegraded":"2024-09-12T02:56:31.032976+0000","last_fullsized":"2024-09-12T02:56:31.032976+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:44:13.406341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.035162+0000","last_change":"2024-09-12T02:56:15.978746+0000","last_active":"2024-09-12T02:56:31.035162+0000","last_peered":"2024-09-12T02:56:31.035162+0000","last_clean":"2024-09-12T02:56:31.035162+0000","last_became_active":"2024-09-12T02:56:15.978304+0000","last_became_peered":"2024-09-12T02:56:15.978304+0000","last_unstale":"2024-09-12T02:56:31.035162+0000","last_undegraded":"2024-09-12T02:56:31.035162+0000","last_fullsized":"2024-09-12T02:56:31.035162+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:34:52.115630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.034360+0000","last_change":"2024-09-12T02:56:15.980083+0000","last_active":"2024-09-12T02:56:31.034360+0000","last_peered":"2024-09-12T02:56:31.034360+0000","last_clean":"2024-09-12T02:56:31.034360+0000","last_became_active":"2024-09-12T02:56:15.979771+0000","last_became_peered":"2024-09-12T02:56:15.979771+0000","last_unstale":"2024-09-12T02:56:31.034360+0000","last_undegraded":"2024-09-12T02:56:31.034360+0000","last_fullsized":"2024-09-12T02:56:31.034360+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:59:42.351970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.306732+0000","last_change":"2024-09-12T02:56:15.988091+0000","last_active":"2024-09-12T02:56:31.306732+0000","last_peered":"2024-09-12T02:56:31.306732+0000","last_clean":"2024-09-12T02:56:31.306732+0000","last_became_active":"2024-09-12T02:56:15.986295+0000","last_became_peered":"2024-09-12T02:56:15.986295+0000","last_unstale":"2024-09-12T02:56:31.306732+0000","last_undegraded":"2024-09-12T02:56:31.306732+0000","last_fullsized":"2024-09-12T02:56:31.306732+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:45:40.331805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.033168+0000","last_change":"2024-09-12T02:56:15.987349+0000","last_active":"2024-09-12T02:56:31.033168+0000","last_peered":"2024-09-12T02:56:31.033168+0000","last_clean":"2024-09-12T02:56:31.033168+0000","last_became_active":"2024-09-12T02:56:15.987018+0000","last_became_peered":"2024-09-12T02:56:15.987018+0000","last_unstale":"2024-09-12T02:56:31.033168+0000","last_undegraded":"2024-09-12T02:56:31.033168+0000","last_fullsized":"2024-09-12T02:56:31.033168+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:34:19.652348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.035420+0000","last_change":"2024-09-12T02:56:15.985164+0000","last_active":"2024-09-12T02:56:31.035420+0000","last_peered":"2024-09-12T02:56:31.035420+0000","last_clean":"2024-09-12T02:56:31.035420+0000","last_became_active":"2024-09-12T02:56:15.984787+0000","last_became_peered":"2024-09-12T02:56:15.984787+0000","last_unstale":"2024-09-12T02:56:31.035420+0000","last_undegraded":"2024-09-12T02:56:31.035420+0000","last_fullsized":"2024-09-12T02:56:31.035420+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:36:29.727442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.031157+0000","last_change":"2024-09-12T02:56:15.979963+0000","last_active":"2024-09-12T02:56:31.031157+0000","last_peered":"2024-09-12T02:56:31.031157+0000","last_clean":"2024-09-12T02:56:31.031157+0000","last_became_active":"2024-09-12T02:56:15.979639+0000","last_became_peered":"2024-09-12T02:56:15.979639+0000","last_unstale":"2024-09-12T02:56:31.031157+0000","last_undegraded":"2024-09-12T02:56:31.031157+0000","last_fullsized":"2024-09-12T02:56:31.031157+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:34:18.273245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.031973+0000","last_change":"2024-09-12T02:56:15.980415+0000","last_active":"2024-09-12T02:56:31.031973+0000","last_peered":"2024-09-12T02:56:31.031973+0000","last_clean":"2024-09-12T02:56:31.031973+0000","last_became_active":"2024-09-12T02:56:15.980059+0000","last_became_peered":"2024-09-12T02:56:15.980059+0000","last_unstale":"2024-09-12T02:56:31.031973+0000","last_undegraded":"2024-09-12T02:56:31.031973+0000","last_fullsized":"2024-09-12T02:56:31.031973+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:11:42.689081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.032089+0000","last_change":"2024-09-12T02:56:15.981423+0000","last_active":"2024-09-12T02:56:31.032089+0000","last_peered":"2024-09-12T02:56:31.032089+0000","last_clean":"2024-09-12T02:56:31.032089+0000","last_became_active":"2024-09-12T02:56:15.981186+0000","last_became_peered":"2024-09-12T02:56:15.981186+0000","last_unstale":"2024-09-12T02:56:31.032089+0000","last_undegraded":"2024-09-12T02:56:31.032089+0000","last_fullsized":"2024-09-12T02:56:31.032089+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:36:57.406907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.032139+0000","last_change":"2024-09-12T02:56:15.982587+0000","last_active":"2024-09-12T02:56:31.032139+0000","last_peered":"2024-09-12T02:56:31.032139+0000","last_clean":"2024-09-12T02:56:31.032139+0000","last_became_active":"2024-09-12T02:56:15.982315+0000","last_became_peered":"2024-09-12T02:56:15.982315+0000","last_unstale":"2024-09-12T02:56:31.032139+0000","last_undegraded":"2024-09-12T02:56:31.032139+0000","last_fullsized":"2024-09-12T02:56:31.032139+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:42:17.736999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.306084+0000","last_change":"2024-09-12T02:56:15.979482+0000","last_active":"2024-09-12T02:56:31.306084+0000","last_peered":"2024-09-12T02:56:31.306084+0000","last_clean":"2024-09-12T02:56:31.306084+0000","last_became_active":"2024-09-12T02:56:15.979122+0000","last_became_peered":"2024-09-12T02:56:15.979122+0000","last_unstale":"2024-09-12T02:56:31.306084+0000","last_undegraded":"2024-09-12T02:56:31.306084+0000","last_fullsized":"2024-09-12T02:56:31.306084+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:18:05.642487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.033518+0000","last_change":"2024-09-12T02:56:15.989908+0000","last_active":"2024-09-12T02:56:31.033518+0000","last_peered":"2024-09-12T02:56:31.033518+0000","last_clean":"2024-09-12T02:56:31.033518+0000","last_became_active":"2024-09-12T02:56:15.989515+0000","last_became_peered":"2024-09-12T02:56:15.989515+0000","last_unstale":"2024-09-12T02:56:31.033518+0000","last_undegraded":"2024-09-12T02:56:31.033518+0000","last_fullsized":"2024-09-12T02:56:31.033518+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:59:57.141607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.029078+0000","last_change":"2024-09-12T02:56:15.980323+0000","last_active":"2024-09-12T02:56:31.029078+0000","last_peered":"2024-09-12T02:56:31.029078+0000","last_clean":"2024-09-12T02:56:31.029078+0000","last_became_active":"2024-09-12T02:56:15.979989+0000","last_became_peered":"2024-09-12T02:56:15.979989+0000","last_unstale":"2024-09-12T02:56:31.029078+0000","last_undegraded":"2024-09-12T02:56:31.029078+0000","last_fullsized":"2024-09-12T02:56:31.029078+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:55:56.064125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.030503+0000","last_change":"2024-09-12T02:56:15.986003+0000","last_active":"2024-09-12T02:56:31.030503+0000","last_peered":"2024-09-12T02:56:31.030503+0000","last_clean":"2024-09-12T02:56:31.030503+0000","last_became_active":"2024-09-12T02:56:15.985335+0000","last_became_peered":"2024-09-12T02:56:15.985335+0000","last_unstale":"2024-09-12T02:56:31.030503+0000","last_undegraded":"2024-09-12T02:56:31.030503+0000","last_fullsized":"2024-09-12T02:56:31.030503+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:51:47.944530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.036870+0000","last_change":"2024-09-12T02:56:31.036870+0000","last_active":"2024-09-12T02:56:31.036870+0000","last_peered":"2024-09-12T02:56:31.036870+0000","last_clean":"2024-09-12T02:56:31.036870+0000","last_became_active":"2024-09-12T02:56:31.036443+0000","last_became_peered":"2024-09-12T02:56:31.036443+0000","last_unstale":"2024-09-12T02:56:31.036870+0000","last_undegraded":"2024-09-12T02:56:31.036870+0000","last_fullsized":"2024-09-12T02:56:31.036870+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:49:29.326184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.032066+0000","last_change":"2024-09-12T02:56:15.974547+0000","last_active":"2024-09-12T02:56:31.032066+0000","last_peered":"2024-09-12T02:56:31.032066+0000","last_clean":"2024-09-12T02:56:31.032066+0000","last_became_active":"2024-09-12T02:56:15.974174+0000","last_became_peered":"2024-09-12T02:56:15.974174+0000","last_unstale":"2024-09-12T02:56:31.032066+0000","last_undegraded":"2024-09-12T02:56:31.032066+0000","last_fullsized":"2024-09-12T02:56:31.032066+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:22:16.954309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.306220+0000","last_change":"2024-09-12T02:56:15.981771+0000","last_active":"2024-09-12T02:56:31.306220+0000","last_peered":"2024-09-12T02:56:31.306220+0000","last_clean":"2024-09-12T02:56:31.306220+0000","last_became_active":"2024-09-12T02:56:15.980948+0000","last_became_peered":"2024-09-12T02:56:15.980948+0000","last_unstale":"2024-09-12T02:56:31.306220+0000","last_undegraded":"2024-09-12T02:56:31.306220+0000","last_fullsized":"2024-09-12T02:56:31.306220+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:34:35.961348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.034487+0000","last_change":"2024-09-12T02:56:15.982318+0000","last_active":"2024-09-12T02:56:31.034487+0000","last_peered":"2024-09-12T02:56:31.034487+0000","last_clean":"2024-09-12T02:56:31.034487+0000","last_became_active":"2024-09-12T02:56:15.982013+0000","last_became_peered":"2024-09-12T02:56:15.982013+0000","last_unstale":"2024-09-12T02:56:31.034487+0000","last_undegraded":"2024-09-12T02:56:31.034487+0000","last_fullsized":"2024-09-12T02:56:31.034487+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:10:50.895579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.032248+0000","last_change":"2024-09-12T02:56:15.969992+0000","last_active":"2024-09-12T02:56:31.032248+0000","last_peered":"2024-09-12T02:56:31.032248+0000","last_clean":"2024-09-12T02:56:31.032248+0000","last_became_active":"2024-09-12T02:56:15.969621+0000","last_became_peered":"2024-09-12T02:56:15.969621+0000","last_unstale":"2024-09-12T02:56:31.032248+0000","last_undegraded":"2024-09-12T02:56:31.032248+0000","last_fullsized":"2024-09-12T02:56:31.032248+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:47:40.206042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.034306+0000","last_change":"2024-09-12T02:56:15.982315+0000","last_active":"2024-09-12T02:56:31.034306+0000","last_peered":"2024-09-12T02:56:31.034306+0000","last_clean":"2024-09-12T02:56:31.034306+0000","last_became_active":"2024-09-12T02:56:15.981992+0000","last_became_peered":"2024-09-12T02:56:15.981992+0000","last_unstale":"2024-09-12T02:56:31.034306+0000","last_undegraded":"2024-09-12T02:56:31.034306+0000","last_fullsized":"2024-09-12T02:56:31.034306+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:54:04.188418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.030767+0000","last_change":"2024-09-12T02:56:15.980712+0000","last_active":"2024-09-12T02:56:31.030767+0000","last_peered":"2024-09-12T02:56:31.030767+0000","last_clean":"2024-09-12T02:56:31.030767+0000","last_became_active":"2024-09-12T02:56:15.980410+0000","last_became_peered":"2024-09-12T02:56:15.980410+0000","last_unstale":"2024-09-12T02:56:31.030767+0000","last_undegraded":"2024-09-12T02:56:31.030767+0000","last_fullsized":"2024-09-12T02:56:31.030767+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:13:53.844490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.033126+0000","last_change":"2024-09-12T02:56:15.981675+0000","last_active":"2024-09-12T02:56:31.033126+0000","last_peered":"2024-09-12T02:56:31.033126+0000","last_clean":"2024-09-12T02:56:31.033126+0000","last_became_active":"2024-09-12T02:56:15.981372+0000","last_became_peered":"2024-09-12T02:56:15.981372+0000","last_unstale":"2024-09-12T02:56:31.033126+0000","last_undegraded":"2024-09-12T02:56:31.033126+0000","last_fullsized":"2024-09-12T02:56:31.033126+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:39:17.399199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.032248+0000","last_change":"2024-09-12T02:56:15.972750+0000","last_active":"2024-09-12T02:56:31.032248+0000","last_peered":"2024-09-12T02:56:31.032248+0000","last_clean":"2024-09-12T02:56:31.032248+0000","last_became_active":"2024-09-12T02:56:15.972434+0000","last_became_peered":"2024-09-12T02:56:15.972434+0000","last_unstale":"2024-09-12T02:56:31.032248+0000","last_undegraded":"2024-09-12T02:56:31.032248+0000","last_fullsized":"2024-09-12T02:56:31.032248+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:59:22.784908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.032502+0000","last_change":"2024-09-12T02:56:15.973567+0000","last_active":"2024-09-12T02:56:31.032502+0000","last_peered":"2024-09-12T02:56:31.032502+0000","last_clean":"2024-09-12T02:56:31.032502+0000","last_became_active":"2024-09-12T02:56:15.973251+0000","last_became_peered":"2024-09-12T02:56:15.973251+0000","last_unstale":"2024-09-12T02:56:31.032502+0000","last_undegraded":"2024-09-12T02:56:31.032502+0000","last_fullsized":"2024-09-12T02:56:31.032502+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:16:39.536817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.030130+0000","last_change":"2024-09-12T02:56:15.986108+0000","last_active":"2024-09-12T02:56:31.030130+0000","last_peered":"2024-09-12T02:56:31.030130+0000","last_clean":"2024-09-12T02:56:31.030130+0000","last_became_active":"2024-09-12T02:56:15.985701+0000","last_became_peered":"2024-09-12T02:56:15.985701+0000","last_unstale":"2024-09-12T02:56:31.030130+0000","last_undegraded":"2024-09-12T02:56:31.030130+0000","last_fullsized":"2024-09-12T02:56:31.030130+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:59.377473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.030220+0000","last_change":"2024-09-12T02:56:15.980078+0000","last_active":"2024-09-12T02:56:31.030220+0000","last_peered":"2024-09-12T02:56:31.030220+0000","last_clean":"2024-09-12T02:56:31.030220+0000","last_became_active":"2024-09-12T02:56:15.979785+0000","last_became_peered":"2024-09-12T02:56:15.979785+0000","last_unstale":"2024-09-12T02:56:31.030220+0000","last_undegraded":"2024-09-12T02:56:31.030220+0000","last_fullsized":"2024-09-12T02:56:31.030220+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:00:19.252398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.034505+0000","last_change":"2024-09-12T02:56:15.984170+0000","last_active":"2024-09-12T02:56:31.034505+0000","last_peered":"2024-09-12T02:56:31.034505+0000","last_clean":"2024-09-12T02:56:31.034505+0000","last_became_active":"2024-09-12T02:56:15.983416+0000","last_became_peered":"2024-09-12T02:56:15.983416+0000","last_unstale":"2024-09-12T02:56:31.034505+0000","last_undegraded":"2024-09-12T02:56:31.034505+0000","last_fullsized":"2024-09-12T02:56:31.034505+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:13:24.723923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.035795+0000","last_change":"2024-09-12T02:56:15.980987+0000","last_active":"2024-09-12T02:56:31.035795+0000","last_peered":"2024-09-12T02:56:31.035795+0000","last_clean":"2024-09-12T02:56:31.035795+0000","last_became_active":"2024-09-12T02:56:15.980661+0000","last_became_peered":"2024-09-12T02:56:15.980661+0000","last_unstale":"2024-09-12T02:56:31.035795+0000","last_undegraded":"2024-09-12T02:56:31.035795+0000","last_fullsized":"2024-09-12T02:56:31.035795+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:19:49.294698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"81'5","reported_seq":90,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:15.105763+0000","last_change":"2024-09-12T02:56:31.042914+0000","last_active":"2024-09-12T02:57:15.105763+0000","last_peered":"2024-09-12T02:57:15.105763+0000","last_clean":"2024-09-12T02:57:15.105763+0000","last_became_active":"2024-09-12T02:56:31.037356+0000","last_became_peered":"2024-09-12T02:56:31.037356+0000","last_unstale":"2024-09-12T02:57:15.105763+0000","last_undegraded":"2024-09-12T02:57:15.105763+0000","last_fullsized":"2024-09-12T02:57:15.105763+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:17:48.582725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":51,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":622,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.036619+0000","last_change":"2024-09-12T02:56:31.036170+0000","last_active":"2024-09-12T02:56:31.036619+0000","last_peered":"2024-09-12T02:56:31.036619+0000","last_clean":"2024-09-12T02:56:31.036619+0000","last_became_active":"2024-09-12T02:56:31.035754+0000","last_became_peered":"2024-09-12T02:56:31.035754+0000","last_unstale":"2024-09-12T02:56:31.036619+0000","last_undegraded":"2024-09-12T02:56:31.036619+0000","last_fullsized":"2024-09-12T02:56:31.036619+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:58:06.464818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1a","version":"81'7","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.032813+0000","last_change":"2024-09-12T02:56:13.931202+0000","last_active":"2024-09-12T02:56:31.032813+0000","last_peered":"2024-09-12T02:56:31.032813+0000","last_clean":"2024-09-12T02:56:31.032813+0000","last_became_active":"2024-09-12T02:56:13.930776+0000","last_became_peered":"2024-09-12T02:56:13.930776+0000","last_unstale":"2024-09-12T02:56:31.032813+0000","last_undegraded":"2024-09-12T02:56:31.032813+0000","last_fullsized":"2024-09-12T02:56:31.032813+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:22:16.919253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"81'3","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.031662+0000","last_change":"2024-09-12T02:56:13.924660+0000","last_active":"2024-09-12T02:56:31.031662+0000","last_peered":"2024-09-12T02:56:31.031662+0000","last_clean":"2024-09-12T02:56:31.031662+0000","last_became_active":"2024-09-12T02:56:13.924342+0000","last_became_peered":"2024-09-12T02:56:13.924342+0000","last_unstale":"2024-09-12T02:56:31.031662+0000","last_undegraded":"2024-09-12T02:56:31.031662+0000","last_fullsized":"2024-09-12T02:56:31.031662+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:48:53.907044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.034409+0000","last_change":"2024-09-12T02:56:15.978723+0000","last_active":"2024-09-12T02:56:31.034409+0000","last_peered":"2024-09-12T02:56:31.034409+0000","last_clean":"2024-09-12T02:56:31.034409+0000","last_became_active":"2024-09-12T02:56:15.978344+0000","last_became_peered":"2024-09-12T02:56:15.978344+0000","last_unstale":"2024-09-12T02:56:31.034409+0000","last_undegraded":"2024-09-12T02:56:31.034409+0000","last_fullsized":"2024-09-12T02:56:31.034409+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:25:04.559395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"81'3","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.032936+0000","last_change":"2024-09-12T02:56:13.933065+0000","last_active":"2024-09-12T02:56:31.032936+0000","last_peered":"2024-09-12T02:56:31.032936+0000","last_clean":"2024-09-12T02:56:31.032936+0000","last_became_active":"2024-09-12T02:56:13.932597+0000","last_became_peered":"2024-09-12T02:56:13.932597+0000","last_unstale":"2024-09-12T02:56:31.032936+0000","last_undegraded":"2024-09-12T02:56:31.032936+0000","last_fullsized":"2024-09-12T02:56:31.032936+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:57:26.563392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.028878+0000","last_change":"2024-09-12T02:56:15.975722+0000","last_active":"2024-09-12T02:56:31.028878+0000","last_peered":"2024-09-12T02:56:31.028878+0000","last_clean":"2024-09-12T02:56:31.028878+0000","last_became_active":"2024-09-12T02:56:15.975342+0000","last_became_peered":"2024-09-12T02:56:15.975342+0000","last_unstale":"2024-09-12T02:56:31.028878+0000","last_undegraded":"2024-09-12T02:56:31.028878+0000","last_fullsized":"2024-09-12T02:56:31.028878+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:33:17.613738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"81'4","reported_seq":116,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:11.909871+0000","last_change":"2024-09-12T02:56:13.929700+0000","last_active":"2024-09-12T02:57:11.909871+0000","last_peered":"2024-09-12T02:57:11.909871+0000","last_clean":"2024-09-12T02:57:11.909871+0000","last_became_active":"2024-09-12T02:56:13.929317+0000","last_became_peered":"2024-09-12T02:56:13.929317+0000","last_unstale":"2024-09-12T02:57:11.909871+0000","last_undegraded":"2024-09-12T02:57:11.909871+0000","last_fullsized":"2024-09-12T02:57:11.909871+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:46:32.172674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":90,"num_read_kb":90,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.306705+0000","last_change":"2024-09-12T02:56:15.985754+0000","last_active":"2024-09-12T02:56:31.306705+0000","last_peered":"2024-09-12T02:56:31.306705+0000","last_clean":"2024-09-12T02:56:31.306705+0000","last_became_active":"2024-09-12T02:56:15.985402+0000","last_became_peered":"2024-09-12T02:56:15.985402+0000","last_unstale":"2024-09-12T02:56:31.306705+0000","last_undegraded":"2024-09-12T02:56:31.306705+0000","last_fullsized":"2024-09-12T02:56:31.306705+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:05:29.979961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"81'4","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.035428+0000","last_change":"2024-09-12T02:56:13.933329+0000","last_active":"2024-09-12T02:56:31.035428+0000","last_peered":"2024-09-12T02:56:31.035428+0000","last_clean":"2024-09-12T02:56:31.035428+0000","last_became_active":"2024-09-12T02:56:13.932960+0000","last_became_peered":"2024-09-12T02:56:13.932960+0000","last_unstale":"2024-09-12T02:56:31.035428+0000","last_undegraded":"2024-09-12T02:56:31.035428+0000","last_fullsized":"2024-09-12T02:56:31.035428+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:02:57.564202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.037340+0000","last_change":"2024-09-12T02:56:31.037340+0000","last_active":"2024-09-12T02:56:31.037340+0000","last_peered":"2024-09-12T02:56:31.037340+0000","last_clean":"2024-09-12T02:56:31.037340+0000","last_became_active":"2024-09-12T02:56:31.036993+0000","last_became_peered":"2024-09-12T02:56:31.036993+0000","last_unstale":"2024-09-12T02:56:31.037340+0000","last_undegraded":"2024-09-12T02:56:31.037340+0000","last_fullsized":"2024-09-12T02:56:31.037340+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:44:27.788791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"81'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.033005+0000","last_change":"2024-09-12T02:56:14.077202+0000","last_active":"2024-09-12T02:56:31.033005+0000","last_peered":"2024-09-12T02:56:31.033005+0000","last_clean":"2024-09-12T02:56:31.033005+0000","last_became_active":"2024-09-12T02:56:14.076610+0000","last_became_peered":"2024-09-12T02:56:14.076610+0000","last_unstale":"2024-09-12T02:56:31.033005+0000","last_undegraded":"2024-09-12T02:56:31.033005+0000","last_fullsized":"2024-09-12T02:56:31.033005+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:37:50.333152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.306307+0000","last_change":"2024-09-12T02:56:15.977161+0000","last_active":"2024-09-12T02:56:31.306307+0000","last_peered":"2024-09-12T02:56:31.306307+0000","last_clean":"2024-09-12T02:56:31.306307+0000","last_became_active":"2024-09-12T02:56:15.976683+0000","last_became_peered":"2024-09-12T02:56:15.976683+0000","last_unstale":"2024-09-12T02:56:31.306307+0000","last_undegraded":"2024-09-12T02:56:31.306307+0000","last_fullsized":"2024-09-12T02:56:31.306307+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:34:07.526890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'31","reported_seq":152,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:11.909864+0000","last_change":"2024-09-12T02:56:13.931680+0000","last_active":"2024-09-12T02:57:11.909864+0000","last_peered":"2024-09-12T02:57:11.909864+0000","last_clean":"2024-09-12T02:57:11.909864+0000","last_became_active":"2024-09-12T02:56:13.931214+0000","last_became_peered":"2024-09-12T02:56:13.931214+0000","last_unstale":"2024-09-12T02:57:11.909864+0000","last_undegraded":"2024-09-12T02:57:11.909864+0000","last_fullsized":"2024-09-12T02:57:11.909864+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:38:34.151933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61167,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":99,"num_read_kb":108,"num_write":33,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.035329+0000","last_change":"2024-09-12T02:56:15.977720+0000","last_active":"2024-09-12T02:56:31.035329+0000","last_peered":"2024-09-12T02:56:31.035329+0000","last_clean":"2024-09-12T02:56:31.035329+0000","last_became_active":"2024-09-12T02:56:15.977380+0000","last_became_peered":"2024-09-12T02:56:15.977380+0000","last_unstale":"2024-09-12T02:56:31.035329+0000","last_undegraded":"2024-09-12T02:56:31.035329+0000","last_fullsized":"2024-09-12T02:56:31.035329+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:52:16.841909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"81'9","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.030362+0000","last_change":"2024-09-12T02:56:13.929496+0000","last_active":"2024-09-12T02:56:31.030362+0000","last_peered":"2024-09-12T02:56:31.030362+0000","last_clean":"2024-09-12T02:56:31.030362+0000","last_became_active":"2024-09-12T02:56:13.929117+0000","last_became_peered":"2024-09-12T02:56:13.929117+0000","last_unstale":"2024-09-12T02:56:31.030362+0000","last_undegraded":"2024-09-12T02:56:31.030362+0000","last_fullsized":"2024-09-12T02:56:31.030362+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:59:38.399135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.031759+0000","last_change":"2024-09-12T02:56:15.974323+0000","last_active":"2024-09-12T02:56:31.031759+0000","last_peered":"2024-09-12T02:56:31.031759+0000","last_clean":"2024-09-12T02:56:31.031759+0000","last_became_active":"2024-09-12T02:56:15.973819+0000","last_became_peered":"2024-09-12T02:56:15.973819+0000","last_unstale":"2024-09-12T02:56:31.031759+0000","last_undegraded":"2024-09-12T02:56:31.031759+0000","last_fullsized":"2024-09-12T02:56:31.031759+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:53:13.130695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"81'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.032473+0000","last_change":"2024-09-12T02:56:13.935263+0000","last_active":"2024-09-12T02:56:31.032473+0000","last_peered":"2024-09-12T02:56:31.032473+0000","last_clean":"2024-09-12T02:56:31.032473+0000","last_became_active":"2024-09-12T02:56:13.934805+0000","last_became_peered":"2024-09-12T02:56:13.934805+0000","last_unstale":"2024-09-12T02:56:31.032473+0000","last_undegraded":"2024-09-12T02:56:31.032473+0000","last_fullsized":"2024-09-12T02:56:31.032473+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:39:42.882031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.035564+0000","last_change":"2024-09-12T02:56:15.976630+0000","last_active":"2024-09-12T02:56:31.035564+0000","last_peered":"2024-09-12T02:56:31.035564+0000","last_clean":"2024-09-12T02:56:31.035564+0000","last_became_active":"2024-09-12T02:56:15.976304+0000","last_became_peered":"2024-09-12T02:56:15.976304+0000","last_unstale":"2024-09-12T02:56:31.035564+0000","last_undegraded":"2024-09-12T02:56:31.035564+0000","last_fullsized":"2024-09-12T02:56:31.035564+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:46:03.366740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"81'6","reported_seq":80,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:14.911496+0000","last_change":"2024-09-12T02:56:13.934224+0000","last_active":"2024-09-12T02:57:14.911496+0000","last_peered":"2024-09-12T02:57:14.911496+0000","last_clean":"2024-09-12T02:57:14.911496+0000","last_became_active":"2024-09-12T02:56:13.933838+0000","last_became_peered":"2024-09-12T02:56:13.933838+0000","last_unstale":"2024-09-12T02:57:14.911496+0000","last_undegraded":"2024-09-12T02:57:14.911496+0000","last_fullsized":"2024-09-12T02:57:14.911496+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:34:34.206051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":53,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.035229+0000","last_change":"2024-09-12T02:56:15.980364+0000","last_active":"2024-09-12T02:56:31.035229+0000","last_peered":"2024-09-12T02:56:31.035229+0000","last_clean":"2024-09-12T02:56:31.035229+0000","last_became_active":"2024-09-12T02:56:15.979437+0000","last_became_peered":"2024-09-12T02:56:15.979437+0000","last_unstale":"2024-09-12T02:56:31.035229+0000","last_undegraded":"2024-09-12T02:56:31.035229+0000","last_fullsized":"2024-09-12T02:56:31.035229+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:43:33.424867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"81'17","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.050509+0000","last_change":"2024-09-12T02:56:31.050046+0000","last_active":"2024-09-12T02:56:31.050509+0000","last_peered":"2024-09-12T02:56:31.050509+0000","last_clean":"2024-09-12T02:56:31.050509+0000","last_became_active":"2024-09-12T02:56:31.036990+0000","last_became_peered":"2024-09-12T02:56:31.036990+0000","last_unstale":"2024-09-12T02:56:31.050509+0000","last_undegraded":"2024-09-12T02:56:31.050509+0000","last_fullsized":"2024-09-12T02:56:31.050509+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:05:53.336691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.032226+0000","last_change":"2024-09-12T02:56:15.986434+0000","last_active":"2024-09-12T02:56:31.032226+0000","last_peered":"2024-09-12T02:56:31.032226+0000","last_clean":"2024-09-12T02:56:31.032226+0000","last_became_active":"2024-09-12T02:56:15.986101+0000","last_became_peered":"2024-09-12T02:56:15.986101+0000","last_unstale":"2024-09-12T02:56:31.032226+0000","last_undegraded":"2024-09-12T02:56:31.032226+0000","last_fullsized":"2024-09-12T02:56:31.032226+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:51:27.840951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"81'2","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.035974+0000","last_change":"2024-09-12T02:56:13.925068+0000","last_active":"2024-09-12T02:56:31.035974+0000","last_peered":"2024-09-12T02:56:31.035974+0000","last_clean":"2024-09-12T02:56:31.035974+0000","last_became_active":"2024-09-12T02:56:13.924535+0000","last_became_peered":"2024-09-12T02:56:13.924535+0000","last_unstale":"2024-09-12T02:56:31.035974+0000","last_undegraded":"2024-09-12T02:56:31.035974+0000","last_fullsized":"2024-09-12T02:56:31.035974+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:45:32.478632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.033033+0000","last_change":"2024-09-12T02:56:15.983850+0000","last_active":"2024-09-12T02:56:31.033033+0000","last_peered":"2024-09-12T02:56:31.033033+0000","last_clean":"2024-09-12T02:56:31.033033+0000","last_became_active":"2024-09-12T02:56:15.983480+0000","last_became_peered":"2024-09-12T02:56:15.983480+0000","last_unstale":"2024-09-12T02:56:31.033033+0000","last_undegraded":"2024-09-12T02:56:31.033033+0000","last_fullsized":"2024-09-12T02:56:31.033033+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:09:42.185237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"81'5","reported_seq":39,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.312450+0000","last_change":"2024-09-12T02:56:31.312450+0000","last_active":"2024-09-12T02:56:31.312450+0000","last_peered":"2024-09-12T02:56:31.312450+0000","last_clean":"2024-09-12T02:56:31.312450+0000","last_became_active":"2024-09-12T02:56:31.308030+0000","last_became_peered":"2024-09-12T02:56:31.308030+0000","last_unstale":"2024-09-12T02:56:31.312450+0000","last_undegraded":"2024-09-12T02:56:31.312450+0000","last_fullsized":"2024-09-12T02:56:31.312450+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:15:14.692292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.035489+0000","last_change":"2024-09-12T02:56:15.978996+0000","last_active":"2024-09-12T02:56:31.035489+0000","last_peered":"2024-09-12T02:56:31.035489+0000","last_clean":"2024-09-12T02:56:31.035489+0000","last_became_active":"2024-09-12T02:56:15.978636+0000","last_became_peered":"2024-09-12T02:56:15.978636+0000","last_unstale":"2024-09-12T02:56:31.035489+0000","last_undegraded":"2024-09-12T02:56:31.035489+0000","last_fullsized":"2024-09-12T02:56:31.035489+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:29:20.401664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"81'3","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.033119+0000","last_change":"2024-09-12T02:56:13.929388+0000","last_active":"2024-09-12T02:56:31.033119+0000","last_peered":"2024-09-12T02:56:31.033119+0000","last_clean":"2024-09-12T02:56:31.033119+0000","last_became_active":"2024-09-12T02:56:13.928998+0000","last_became_peered":"2024-09-12T02:56:13.928998+0000","last_unstale":"2024-09-12T02:56:31.033119+0000","last_undegraded":"2024-09-12T02:56:31.033119+0000","last_fullsized":"2024-09-12T02:56:31.033119+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:37:57.368050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'1","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.035355+0000","last_change":"2024-09-12T02:56:15.977491+0000","last_active":"2024-09-12T02:56:31.035355+0000","last_peered":"2024-09-12T02:56:31.035355+0000","last_clean":"2024-09-12T02:56:31.035355+0000","last_became_active":"2024-09-12T02:56:15.977160+0000","last_became_peered":"2024-09-12T02:56:15.977160+0000","last_unstale":"2024-09-12T02:56:31.035355+0000","last_undegraded":"2024-09-12T02:56:31.035355+0000","last_fullsized":"2024-09-12T02:56:31.035355+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:05:40.577342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"81'6","reported_seq":81,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:15.915412+0000","last_change":"2024-09-12T02:56:14.327147+0000","last_active":"2024-09-12T02:57:15.915412+0000","last_peered":"2024-09-12T02:57:15.915412+0000","last_clean":"2024-09-12T02:57:15.915412+0000","last_became_active":"2024-09-12T02:56:14.326737+0000","last_became_peered":"2024-09-12T02:56:14.326737+0000","last_unstale":"2024-09-12T02:57:15.915412+0000","last_undegraded":"2024-09-12T02:57:15.915412+0000","last_fullsized":"2024-09-12T02:57:15.915412+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:55:33.510599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1563,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":53,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.035591+0000","last_change":"2024-09-12T02:56:15.984363+0000","last_active":"2024-09-12T02:56:31.035591+0000","last_peered":"2024-09-12T02:56:31.035591+0000","last_clean":"2024-09-12T02:56:31.035591+0000","last_became_active":"2024-09-12T02:56:15.983822+0000","last_became_peered":"2024-09-12T02:56:15.983822+0000","last_unstale":"2024-09-12T02:56:31.035591+0000","last_undegraded":"2024-09-12T02:56:31.035591+0000","last_fullsized":"2024-09-12T02:56:31.035591+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:41:46.082716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":332,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.035507+0000","last_change":"2024-09-12T02:55:05.800210+0000","last_active":"2024-09-12T02:56:31.035507+0000","last_peered":"2024-09-12T02:56:31.035507+0000","last_clean":"2024-09-12T02:56:31.035507+0000","last_became_active":"2024-09-12T02:55:05.799791+0000","last_became_peered":"2024-09-12T02:55:05.799791+0000","last_unstale":"2024-09-12T02:56:31.035507+0000","last_undegraded":"2024-09-12T02:56:31.035507+0000","last_fullsized":"2024-09-12T02:56:31.035507+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:51:48.530016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:51:48.530016+0000","last_clean_scrub_stamp":"2024-09-12T02:51:48.530016+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:05:47.416697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"81'6","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.031169+0000","last_change":"2024-09-12T02:56:13.930269+0000","last_active":"2024-09-12T02:56:31.031169+0000","last_peered":"2024-09-12T02:56:31.031169+0000","last_clean":"2024-09-12T02:56:31.031169+0000","last_became_active":"2024-09-12T02:56:13.929831+0000","last_became_peered":"2024-09-12T02:56:13.929831+0000","last_unstale":"2024-09-12T02:56:31.031169+0000","last_undegraded":"2024-09-12T02:56:31.031169+0000","last_fullsized":"2024-09-12T02:56:31.031169+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:38:17.878398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.034774+0000","last_change":"2024-09-12T02:56:15.969744+0000","last_active":"2024-09-12T02:56:31.034774+0000","last_peered":"2024-09-12T02:56:31.034774+0000","last_clean":"2024-09-12T02:56:31.034774+0000","last_became_active":"2024-09-12T02:56:15.969354+0000","last_became_peered":"2024-09-12T02:56:15.969354+0000","last_unstale":"2024-09-12T02:56:31.034774+0000","last_undegraded":"2024-09-12T02:56:31.034774+0000","last_fullsized":"2024-09-12T02:56:31.034774+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:26:02.409510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"81'4","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.035385+0000","last_change":"2024-09-12T02:56:13.929231+0000","last_active":"2024-09-12T02:56:31.035385+0000","last_peered":"2024-09-12T02:56:31.035385+0000","last_clean":"2024-09-12T02:56:31.035385+0000","last_became_active":"2024-09-12T02:56:13.928843+0000","last_became_peered":"2024-09-12T02:56:13.928843+0000","last_unstale":"2024-09-12T02:56:31.035385+0000","last_undegraded":"2024-09-12T02:56:31.035385+0000","last_fullsized":"2024-09-12T02:56:31.035385+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:26:33.619755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.031654+0000","last_change":"2024-09-12T02:56:15.982503+0000","last_active":"2024-09-12T02:56:31.031654+0000","last_peered":"2024-09-12T02:56:31.031654+0000","last_clean":"2024-09-12T02:56:31.031654+0000","last_became_active":"2024-09-12T02:56:15.982192+0000","last_became_peered":"2024-09-12T02:56:15.982192+0000","last_unstale":"2024-09-12T02:56:31.031654+0000","last_undegraded":"2024-09-12T02:56:31.031654+0000","last_fullsized":"2024-09-12T02:56:31.031654+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:55:18.370967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"81'5","reported_seq":76,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:13.909347+0000","last_change":"2024-09-12T02:56:14.075486+0000","last_active":"2024-09-12T02:57:13.909347+0000","last_peered":"2024-09-12T02:57:13.909347+0000","last_clean":"2024-09-12T02:57:13.909347+0000","last_became_active":"2024-09-12T02:56:14.075152+0000","last_became_peered":"2024-09-12T02:56:14.075152+0000","last_unstale":"2024-09-12T02:57:13.909347+0000","last_undegraded":"2024-09-12T02:57:13.909347+0000","last_fullsized":"2024-09-12T02:57:13.909347+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:59:40.950191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":49,"num_read_kb":49,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.032420+0000","last_change":"2024-09-12T02:56:15.981584+0000","last_active":"2024-09-12T02:56:31.032420+0000","last_peered":"2024-09-12T02:56:31.032420+0000","last_clean":"2024-09-12T02:56:31.032420+0000","last_became_active":"2024-09-12T02:56:15.981240+0000","last_became_peered":"2024-09-12T02:56:15.981240+0000","last_unstale":"2024-09-12T02:56:31.032420+0000","last_undegraded":"2024-09-12T02:56:31.032420+0000","last_fullsized":"2024-09-12T02:56:31.032420+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:49:04.036830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"81'5","reported_seq":75,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:12.915095+0000","last_change":"2024-09-12T02:56:13.935143+0000","last_active":"2024-09-12T02:57:12.915095+0000","last_peered":"2024-09-12T02:57:12.915095+0000","last_clean":"2024-09-12T02:57:12.915095+0000","last_became_active":"2024-09-12T02:56:13.934792+0000","last_became_peered":"2024-09-12T02:56:13.934792+0000","last_unstale":"2024-09-12T02:57:12.915095+0000","last_undegraded":"2024-09-12T02:57:12.915095+0000","last_fullsized":"2024-09-12T02:57:12.915095+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:00:26.973612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":48,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.034825+0000","last_change":"2024-09-12T02:56:31.034825+0000","last_active":"2024-09-12T02:56:31.034825+0000","last_peered":"2024-09-12T02:56:31.034825+0000","last_clean":"2024-09-12T02:56:31.034825+0000","last_became_active":"2024-09-12T02:56:31.034516+0000","last_became_peered":"2024-09-12T02:56:31.034516+0000","last_unstale":"2024-09-12T02:56:31.034825+0000","last_undegraded":"2024-09-12T02:56:31.034825+0000","last_fullsized":"2024-09-12T02:56:31.034825+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:10:42.161023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"81'1","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.029027+0000","last_change":"2024-09-12T02:56:14.076762+0000","last_active":"2024-09-12T02:56:31.029027+0000","last_peered":"2024-09-12T02:56:31.029027+0000","last_clean":"2024-09-12T02:56:31.029027+0000","last_became_active":"2024-09-12T02:56:14.076421+0000","last_became_peered":"2024-09-12T02:56:14.076421+0000","last_unstale":"2024-09-12T02:56:31.029027+0000","last_undegraded":"2024-09-12T02:56:31.029027+0000","last_fullsized":"2024-09-12T02:56:31.029027+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:56:56.422910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'1","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.033103+0000","last_change":"2024-09-12T02:56:15.978810+0000","last_active":"2024-09-12T02:56:31.033103+0000","last_peered":"2024-09-12T02:56:31.033103+0000","last_clean":"2024-09-12T02:56:31.033103+0000","last_became_active":"2024-09-12T02:56:15.978038+0000","last_became_peered":"2024-09-12T02:56:15.978038+0000","last_unstale":"2024-09-12T02:56:31.033103+0000","last_undegraded":"2024-09-12T02:56:31.033103+0000","last_fullsized":"2024-09-12T02:56:31.033103+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:04:52.215784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"81'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.035440+0000","last_change":"2024-09-12T02:56:14.075987+0000","last_active":"2024-09-12T02:56:31.035440+0000","last_peered":"2024-09-12T02:56:31.035440+0000","last_clean":"2024-09-12T02:56:31.035440+0000","last_became_active":"2024-09-12T02:56:14.075596+0000","last_became_peered":"2024-09-12T02:56:14.075596+0000","last_unstale":"2024-09-12T02:56:31.035440+0000","last_undegraded":"2024-09-12T02:56:31.035440+0000","last_fullsized":"2024-09-12T02:56:31.035440+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:18:48.713387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.032345+0000","last_change":"2024-09-12T02:56:15.987842+0000","last_active":"2024-09-12T02:56:31.032345+0000","last_peered":"2024-09-12T02:56:31.032345+0000","last_clean":"2024-09-12T02:56:31.032345+0000","last_became_active":"2024-09-12T02:56:15.987507+0000","last_became_peered":"2024-09-12T02:56:15.987507+0000","last_unstale":"2024-09-12T02:56:31.032345+0000","last_undegraded":"2024-09-12T02:56:31.032345+0000","last_fullsized":"2024-09-12T02:56:31.032345+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:52:23.000549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'16","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:39.980001+0000","last_change":"2024-09-12T02:56:13.924372+0000","last_active":"2024-09-12T02:56:39.980001+0000","last_peered":"2024-09-12T02:56:39.980001+0000","last_clean":"2024-09-12T02:56:39.980001+0000","last_became_active":"2024-09-12T02:56:13.923955+0000","last_became_peered":"2024-09-12T02:56:13.923955+0000","last_unstale":"2024-09-12T02:56:39.980001+0000","last_undegraded":"2024-09-12T02:56:39.980001+0000","last_fullsized":"2024-09-12T02:56:39.980001+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:52:47.452410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.034382+0000","last_change":"2024-09-12T02:56:15.978262+0000","last_active":"2024-09-12T02:56:31.034382+0000","last_peered":"2024-09-12T02:56:31.034382+0000","last_clean":"2024-09-12T02:56:31.034382+0000","last_became_active":"2024-09-12T02:56:15.977913+0000","last_became_peered":"2024-09-12T02:56:15.977913+0000","last_unstale":"2024-09-12T02:56:31.034382+0000","last_undegraded":"2024-09-12T02:56:31.034382+0000","last_fullsized":"2024-09-12T02:56:31.034382+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:35:27.104635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"81'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.031488+0000","last_change":"2024-09-12T02:56:14.327417+0000","last_active":"2024-09-12T02:56:31.031488+0000","last_peered":"2024-09-12T02:56:31.031488+0000","last_clean":"2024-09-12T02:56:31.031488+0000","last_became_active":"2024-09-12T02:56:14.327001+0000","last_became_peered":"2024-09-12T02:56:14.327001+0000","last_unstale":"2024-09-12T02:56:31.031488+0000","last_undegraded":"2024-09-12T02:56:31.031488+0000","last_fullsized":"2024-09-12T02:56:31.031488+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:15:39.019252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.032658+0000","last_change":"2024-09-12T02:56:15.977286+0000","last_active":"2024-09-12T02:56:31.032658+0000","last_peered":"2024-09-12T02:56:31.032658+0000","last_clean":"2024-09-12T02:56:31.032658+0000","last_became_active":"2024-09-12T02:56:15.976944+0000","last_became_peered":"2024-09-12T02:56:15.976944+0000","last_unstale":"2024-09-12T02:56:31.032658+0000","last_undegraded":"2024-09-12T02:56:31.032658+0000","last_fullsized":"2024-09-12T02:56:31.032658+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:22:57.056781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.306212+0000","last_change":"2024-09-12T02:56:14.075701+0000","last_active":"2024-09-12T02:56:31.306212+0000","last_peered":"2024-09-12T02:56:31.306212+0000","last_clean":"2024-09-12T02:56:31.306212+0000","last_became_active":"2024-09-12T02:56:14.075340+0000","last_became_peered":"2024-09-12T02:56:14.075340+0000","last_unstale":"2024-09-12T02:56:31.306212+0000","last_undegraded":"2024-09-12T02:56:31.306212+0000","last_fullsized":"2024-09-12T02:56:31.306212+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:17:42.255058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.034874+0000","last_change":"2024-09-12T02:56:15.979783+0000","last_active":"2024-09-12T02:56:31.034874+0000","last_peered":"2024-09-12T02:56:31.034874+0000","last_clean":"2024-09-12T02:56:31.034874+0000","last_became_active":"2024-09-12T02:56:15.979424+0000","last_became_peered":"2024-09-12T02:56:15.979424+0000","last_unstale":"2024-09-12T02:56:31.034874+0000","last_undegraded":"2024-09-12T02:56:31.034874+0000","last_fullsized":"2024-09-12T02:56:31.034874+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:59:11.568138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"81'7","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.034660+0000","last_change":"2024-09-12T02:56:13.928996+0000","last_active":"2024-09-12T02:56:31.034660+0000","last_peered":"2024-09-12T02:56:31.034660+0000","last_clean":"2024-09-12T02:56:31.034660+0000","last_became_active":"2024-09-12T02:56:13.928664+0000","last_became_peered":"2024-09-12T02:56:13.928664+0000","last_unstale":"2024-09-12T02:56:31.034660+0000","last_undegraded":"2024-09-12T02:56:31.034660+0000","last_fullsized":"2024-09-12T02:56:31.034660+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:37:22.067930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.034581+0000","last_change":"2024-09-12T02:56:15.984096+0000","last_active":"2024-09-12T02:56:31.034581+0000","last_peered":"2024-09-12T02:56:31.034581+0000","last_clean":"2024-09-12T02:56:31.034581+0000","last_became_active":"2024-09-12T02:56:15.983737+0000","last_became_peered":"2024-09-12T02:56:15.983737+0000","last_unstale":"2024-09-12T02:56:31.034581+0000","last_undegraded":"2024-09-12T02:56:31.034581+0000","last_fullsized":"2024-09-12T02:56:31.034581+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:02:17.892318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"81'3","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.053785+0000","last_change":"2024-09-12T02:56:31.053343+0000","last_active":"2024-09-12T02:56:31.053785+0000","last_peered":"2024-09-12T02:56:31.053785+0000","last_clean":"2024-09-12T02:56:31.053785+0000","last_became_active":"2024-09-12T02:56:31.037918+0000","last_became_peered":"2024-09-12T02:56:31.037918+0000","last_unstale":"2024-09-12T02:56:31.053785+0000","last_undegraded":"2024-09-12T02:56:31.053785+0000","last_fullsized":"2024-09-12T02:56:31.053785+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:39:54.990724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.034343+0000","last_change":"2024-09-12T02:56:15.975640+0000","last_active":"2024-09-12T02:56:31.034343+0000","last_peered":"2024-09-12T02:56:31.034343+0000","last_clean":"2024-09-12T02:56:31.034343+0000","last_became_active":"2024-09-12T02:56:15.975342+0000","last_became_peered":"2024-09-12T02:56:15.975342+0000","last_unstale":"2024-09-12T02:56:31.034343+0000","last_undegraded":"2024-09-12T02:56:31.034343+0000","last_fullsized":"2024-09-12T02:56:31.034343+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:26:23.694876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.034922+0000","last_change":"2024-09-12T02:56:13.934149+0000","last_active":"2024-09-12T02:56:31.034922+0000","last_peered":"2024-09-12T02:56:31.034922+0000","last_clean":"2024-09-12T02:56:31.034922+0000","last_became_active":"2024-09-12T02:56:13.933743+0000","last_became_peered":"2024-09-12T02:56:13.933743+0000","last_unstale":"2024-09-12T02:56:31.034922+0000","last_undegraded":"2024-09-12T02:56:31.034922+0000","last_fullsized":"2024-09-12T02:56:31.034922+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:16:17.854457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.035435+0000","last_change":"2024-09-12T02:56:15.984782+0000","last_active":"2024-09-12T02:56:31.035435+0000","last_peered":"2024-09-12T02:56:31.035435+0000","last_clean":"2024-09-12T02:56:31.035435+0000","last_became_active":"2024-09-12T02:56:15.984336+0000","last_became_peered":"2024-09-12T02:56:15.984336+0000","last_unstale":"2024-09-12T02:56:31.035435+0000","last_undegraded":"2024-09-12T02:56:31.035435+0000","last_fullsized":"2024-09-12T02:56:31.035435+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:03:39.832897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.032443+0000","last_change":"2024-09-12T02:56:15.985486+0000","last_active":"2024-09-12T02:56:31.032443+0000","last_peered":"2024-09-12T02:56:31.032443+0000","last_clean":"2024-09-12T02:56:31.032443+0000","last_became_active":"2024-09-12T02:56:15.985167+0000","last_became_peered":"2024-09-12T02:56:15.985167+0000","last_unstale":"2024-09-12T02:56:31.032443+0000","last_undegraded":"2024-09-12T02:56:31.032443+0000","last_fullsized":"2024-09-12T02:56:31.032443+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:46:20.569453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"81'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.035106+0000","last_change":"2024-09-12T02:56:13.929324+0000","last_active":"2024-09-12T02:56:31.035106+0000","last_peered":"2024-09-12T02:56:31.035106+0000","last_clean":"2024-09-12T02:56:31.035106+0000","last_became_active":"2024-09-12T02:56:13.928994+0000","last_became_peered":"2024-09-12T02:56:13.928994+0000","last_unstale":"2024-09-12T02:56:31.035106+0000","last_undegraded":"2024-09-12T02:56:31.035106+0000","last_fullsized":"2024-09-12T02:56:31.035106+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:30:27.290328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.306230+0000","last_change":"2024-09-12T02:56:15.985889+0000","last_active":"2024-09-12T02:56:31.306230+0000","last_peered":"2024-09-12T02:56:31.306230+0000","last_clean":"2024-09-12T02:56:31.306230+0000","last_became_active":"2024-09-12T02:56:15.985509+0000","last_became_peered":"2024-09-12T02:56:15.985509+0000","last_unstale":"2024-09-12T02:56:31.306230+0000","last_undegraded":"2024-09-12T02:56:31.306230+0000","last_fullsized":"2024-09-12T02:56:31.306230+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:30:43.120718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"81'12","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.035386+0000","last_change":"2024-09-12T02:56:13.934192+0000","last_active":"2024-09-12T02:56:31.035386+0000","last_peered":"2024-09-12T02:56:31.035386+0000","last_clean":"2024-09-12T02:56:31.035386+0000","last_became_active":"2024-09-12T02:56:13.933789+0000","last_became_peered":"2024-09-12T02:56:13.933789+0000","last_unstale":"2024-09-12T02:56:31.035386+0000","last_undegraded":"2024-09-12T02:56:31.035386+0000","last_fullsized":"2024-09-12T02:56:31.035386+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:32:47.771085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.032496+0000","last_change":"2024-09-12T02:56:15.983531+0000","last_active":"2024-09-12T02:56:31.032496+0000","last_peered":"2024-09-12T02:56:31.032496+0000","last_clean":"2024-09-12T02:56:31.032496+0000","last_became_active":"2024-09-12T02:56:15.983192+0000","last_became_peered":"2024-09-12T02:56:15.983192+0000","last_unstale":"2024-09-12T02:56:31.032496+0000","last_undegraded":"2024-09-12T02:56:31.032496+0000","last_fullsized":"2024-09-12T02:56:31.032496+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:16:59.923379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.034752+0000","last_change":"2024-09-12T02:56:14.075396+0000","last_active":"2024-09-12T02:56:31.034752+0000","last_peered":"2024-09-12T02:56:31.034752+0000","last_clean":"2024-09-12T02:56:31.034752+0000","last_became_active":"2024-09-12T02:56:14.075023+0000","last_became_peered":"2024-09-12T02:56:14.075023+0000","last_unstale":"2024-09-12T02:56:31.034752+0000","last_undegraded":"2024-09-12T02:56:31.034752+0000","last_fullsized":"2024-09-12T02:56:31.034752+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:41:45.760689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.034442+0000","last_change":"2024-09-12T02:56:15.987276+0000","last_active":"2024-09-12T02:56:31.034442+0000","last_peered":"2024-09-12T02:56:31.034442+0000","last_clean":"2024-09-12T02:56:31.034442+0000","last_became_active":"2024-09-12T02:56:15.986949+0000","last_became_peered":"2024-09-12T02:56:15.986949+0000","last_unstale":"2024-09-12T02:56:31.034442+0000","last_undegraded":"2024-09-12T02:56:31.034442+0000","last_fullsized":"2024-09-12T02:56:31.034442+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:49:45.627093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"81'13","reported_seq":83,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:12.914741+0000","last_change":"2024-09-12T02:56:13.932365+0000","last_active":"2024-09-12T02:57:12.914741+0000","last_peered":"2024-09-12T02:57:12.914741+0000","last_clean":"2024-09-12T02:57:12.914741+0000","last_became_active":"2024-09-12T02:56:13.932062+0000","last_became_peered":"2024-09-12T02:56:13.932062+0000","last_unstale":"2024-09-12T02:57:12.914741+0000","last_undegraded":"2024-09-12T02:57:12.914741+0000","last_fullsized":"2024-09-12T02:57:12.914741+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:17:08.451839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":48,"num_read_kb":48,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68375,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":510,"num_read_kb":324,"num_write":246,"num_write_kb":135,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":734,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":205125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":211,"ondisk_log_size":211,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547225,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":917504,"data_stored":440690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776157,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32756,"kb_used_data":1572,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712396,"statfs":{"total":95995035648,"available":95961493504,"internally_reserved":0,"allocated":1609728,"data_stored":1092774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939683,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40416,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704736,"statfs":{"total":95995035648,"available":95953649664,"internally_reserved":0,"allocated":933888,"data_stored":440115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":40434112},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168616,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":921600,"data_stored":440381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430252,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28516,"kb_used_data":1492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716636,"statfs":{"total":95995035648,"available":95965835264,"internally_reserved":0,"allocated":1527808,"data_stored":1033013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593778,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":917504,"data_stored":439832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822710,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28528,"kb_used_data":1504,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716624,"statfs":{"total":95995035648,"available":95965822976,"internally_reserved":0,"allocated":1540096,"data_stored":1032298,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953532,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32096,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713056,"statfs":{"total":95995035648,"available":95962169344,"internally_reserved":0,"allocated":925696,"data_stored":440708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182464,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":933888,"data_stored":439657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411398,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36312,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708840,"statfs":{"total":95995035648,"available":95957852160,"internally_reserved":0,"allocated":991232,"data_stored":501903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574922,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":501912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738448,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":937984,"data_stored":441564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":62919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":62910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":63296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T02:57:18.492 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-12T02:57:18.784 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:57:19.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:19.259+0000 7f8136426640 1 -- 172.21.15.5:0/2652598182 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8130102d40 msgr2=0x7f8130103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:19.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:19.259+0000 7f8136426640 1 --2- 172.21.15.5:0/2652598182 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8130102d40 0x7f8130103120 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f8120009900 tx=0x7f812002a480 comp rx=0 tx=0).stop 2024-09-12T02:57:19.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:19.259+0000 7f8136426640 1 -- 172.21.15.5:0/2652598182 shutdown_connections 2024-09-12T02:57:19.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:19.259+0000 7f8136426640 1 --2- 172.21.15.5:0/2652598182 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f813010c170 0x7f813010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:19.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:19.259+0000 7f8136426640 1 --2- 172.21.15.5:0/2652598182 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8130103660 0x7f813010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:19.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:19.259+0000 7f8136426640 1 --2- 172.21.15.5:0/2652598182 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8130102d40 0x7f8130103120 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:19.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:19.259+0000 7f8136426640 1 -- 172.21.15.5:0/2652598182 >> 172.21.15.5:0/2652598182 conn(0x7f81300fc7a0 msgr2=0x7f81300febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:19.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:19.259+0000 7f8136426640 1 -- 172.21.15.5:0/2652598182 shutdown_connections 2024-09-12T02:57:19.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:19.259+0000 7f8136426640 1 -- 172.21.15.5:0/2652598182 wait complete. 2024-09-12T02:57:19.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:19.260+0000 7f8136426640 1 Processor -- start 2024-09-12T02:57:19.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:19.260+0000 7f8136426640 1 -- start start 2024-09-12T02:57:19.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:19.260+0000 7f8136426640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8130102d40 0x7f81301984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:19.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:19.260+0000 7f8136426640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8130103660 0x7f8130198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:19.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:19.260+0000 7f8136426640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f813010c170 0x7f813019cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:19.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:19.260+0000 7f8136426640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8130111f70 con 0x7f813010c170 2024-09-12T02:57:19.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:19.260+0000 7f8136426640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f8130111df0 con 0x7f8130102d40 2024-09-12T02:57:19.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:19.260+0000 7f8136426640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f81301120f0 con 0x7f8130103660 2024-09-12T02:57:19.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:19.260+0000 7f81349c3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f813010c170 0x7f813019cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:19.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:19.260+0000 7f81349c3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f813010c170 0x7f813019cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59898/0 (socket says 172.21.15.5:59898) 2024-09-12T02:57:19.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:19.260+0000 7f81349c3640 1 -- 172.21.15.5:0/1449190982 learned_addr learned my addr 172.21.15.5:0/1449190982 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:57:19.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:19.260+0000 7f812f7fe640 1 --2- 172.21.15.5:0/1449190982 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8130103660 0x7f8130198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:19.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:19.261+0000 7f812ffff640 1 --2- 172.21.15.5:0/1449190982 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8130102d40 0x7f81301984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:19.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:19.261+0000 7f81349c3640 1 -- 172.21.15.5:0/1449190982 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8130103660 msgr2=0x7f8130198a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:19.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:19.261+0000 7f81349c3640 1 --2- 172.21.15.5:0/1449190982 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8130103660 0x7f8130198a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:19.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:19.261+0000 7f81349c3640 1 -- 172.21.15.5:0/1449190982 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8130102d40 msgr2=0x7f81301984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:19.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:19.261+0000 7f81349c3640 1 --2- 172.21.15.5:0/1449190982 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8130102d40 0x7f81301984e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:19.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:19.261+0000 7f81349c3640 1 -- 172.21.15.5:0/1449190982 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f813019d490 con 0x7f813010c170 2024-09-12T02:57:19.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:19.261+0000 7f81349c3640 1 --2- 172.21.15.5:0/1449190982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f813010c170 0x7f813019cdb0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f812800eb10 tx=0x7f812800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:19.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:19.261+0000 7f812d7fa640 1 -- 172.21.15.5:0/1449190982 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8128004b20 con 0x7f813010c170 2024-09-12T02:57:19.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:19.261+0000 7f812d7fa640 1 -- 172.21.15.5:0/1449190982 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8128004cc0 con 0x7f813010c170 2024-09-12T02:57:19.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:19.261+0000 7f8136426640 1 -- 172.21.15.5:0/1449190982 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f813019d780 con 0x7f813010c170 2024-09-12T02:57:19.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:19.261+0000 7f812d7fa640 1 -- 172.21.15.5:0/1449190982 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8128010760 con 0x7f813010c170 2024-09-12T02:57:19.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:19.261+0000 7f8136426640 1 -- 172.21.15.5:0/1449190982 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81301a5060 con 0x7f813010c170 2024-09-12T02:57:19.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:19.262+0000 7f8136426640 1 -- 172.21.15.5:0/1449190982 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8130103120 con 0x7f813010c170 2024-09-12T02:57:19.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:19.263+0000 7f812d7fa640 1 -- 172.21.15.5:0/1449190982 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f8128005020 con 0x7f813010c170 2024-09-12T02:57:19.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:19.264+0000 7f812d7fa640 1 --2- 172.21.15.5:0/1449190982 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f81180a2c10 0x7f81180a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:19.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:19.265+0000 7f812ffff640 1 --2- 172.21.15.5:0/1449190982 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f81180a2c10 0x7f81180a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:19.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:19.265+0000 7f812d7fa640 1 -- 172.21.15.5:0/1449190982 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f81280cf710 con 0x7f813010c170 2024-09-12T02:57:19.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:19.265+0000 7f812ffff640 1 --2- 172.21.15.5:0/1449190982 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f81180a2c10 0x7f81180a50d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8120009230 tx=0x7f81200058d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:19.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:19.269+0000 7f812d7fa640 1 -- 172.21.15.5:0/1449190982 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f812809c4c0 con 0x7f813010c170 2024-09-12T02:57:19.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:19.441+0000 7f8136426640 1 -- 172.21.15.5:0/1449190982 --> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8130062630 con 0x7f81180a2c10 2024-09-12T02:57:19.442 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:57:19 smithi005 ceph-mon[31197]: pgmap v307: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-09-12T02:57:19.442 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:57:19 smithi005 ceph-mon[31197]: from='client.14919 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:57:19.442 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:57:19 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:57:19.442 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:57:19 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:57:19.442 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:57:19 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:57:19.442 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:57:19 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:57:19.442 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:57:19 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:57:19.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:19.447+0000 7f812d7fa640 1 -- 172.21.15.5:0/1449190982 <== mgr.14148 v2:172.21.15.5:6800/3780631642 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258760 (secure 0 0 0) 0x7f8130062630 con 0x7f81180a2c10 2024-09-12T02:57:19.451 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-12T02:57:19.451 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-09-12T02:57:19.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:19.450+0000 7f8136426640 1 -- 172.21.15.5:0/1449190982 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f81180a2c10 msgr2=0x7f81180a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:19.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:19.450+0000 7f8136426640 1 --2- 172.21.15.5:0/1449190982 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f81180a2c10 0x7f81180a50d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8120009230 tx=0x7f81200058d0 comp rx=0 tx=0).stop 2024-09-12T02:57:19.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:19.450+0000 7f8136426640 1 -- 172.21.15.5:0/1449190982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f813010c170 msgr2=0x7f813019cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:19.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:19.450+0000 7f8136426640 1 --2- 172.21.15.5:0/1449190982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f813010c170 0x7f813019cdb0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f812800eb10 tx=0x7f812800c2d0 comp rx=0 tx=0).stop 2024-09-12T02:57:19.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:19.450+0000 7f8136426640 1 -- 172.21.15.5:0/1449190982 shutdown_connections 2024-09-12T02:57:19.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:19.450+0000 7f8136426640 1 --2- 172.21.15.5:0/1449190982 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f81180a2c10 0x7f81180a50d0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:19.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:19.450+0000 7f8136426640 1 --2- 172.21.15.5:0/1449190982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f813010c170 0x7f813019cdb0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:19.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:19.450+0000 7f8136426640 1 --2- 172.21.15.5:0/1449190982 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8130103660 0x7f8130198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:19.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:19.450+0000 7f8136426640 1 --2- 172.21.15.5:0/1449190982 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8130102d40 0x7f81301984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:19.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:19.450+0000 7f8136426640 1 -- 172.21.15.5:0/1449190982 >> 172.21.15.5:0/1449190982 conn(0x7f81300fc7a0 msgr2=0x7f81300fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:19.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:19.450+0000 7f8136426640 1 -- 172.21.15.5:0/1449190982 shutdown_connections 2024-09-12T02:57:19.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:19.451+0000 7f8136426640 1 -- 172.21.15.5:0/1449190982 wait complete. 2024-09-12T02:57:19.665 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:57:19 smithi148 ceph-mon[35090]: pgmap v307: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-09-12T02:57:19.665 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:57:19 smithi148 ceph-mon[35090]: from='client.14919 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:57:19.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:57:19 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:57:19.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:57:19 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:57:19.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:57:19 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:57:19.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:57:19 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:57:19.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:57:19 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:57:19.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:57:19 smithi146 ceph-mon[35011]: pgmap v307: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s 2024-09-12T02:57:19.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:57:19 smithi146 ceph-mon[35011]: from='client.14919 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:57:19.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:57:19 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:57:19.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:57:19 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:57:19.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:57:19 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:57:19.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:57:19 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:57:19.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:57:19 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:57:20.131 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":307,"stamp":"2024-09-12T02:57:17.557208+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658977,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":636,"num_read_kb":433,"num_write":429,"num_write_kb":3075,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":1181470,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":350,"ondisk_log_size":350,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":374588,"kb_used_data":12828,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124567236,"statfs":{"total":1151940427776,"available":1151556849664,"internally_reserved":0,"allocated":13135872,"data_stored":7244847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":370193723},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":114,"num_read_kb":49,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003243"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.033421+0000","last_change":"2024-09-12T02:56:15.988690+0000","last_active":"2024-09-12T02:56:31.033421+0000","last_peered":"2024-09-12T02:56:31.033421+0000","last_clean":"2024-09-12T02:56:31.033421+0000","last_became_active":"2024-09-12T02:56:15.988320+0000","last_became_peered":"2024-09-12T02:56:15.988320+0000","last_unstale":"2024-09-12T02:56:31.033421+0000","last_undegraded":"2024-09-12T02:56:31.033421+0000","last_fullsized":"2024-09-12T02:56:31.033421+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:31:14.157622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.032544+0000","last_change":"2024-09-12T02:56:15.978368+0000","last_active":"2024-09-12T02:56:31.032544+0000","last_peered":"2024-09-12T02:56:31.032544+0000","last_clean":"2024-09-12T02:56:31.032544+0000","last_became_active":"2024-09-12T02:56:15.978092+0000","last_became_peered":"2024-09-12T02:56:15.978092+0000","last_unstale":"2024-09-12T02:56:31.032544+0000","last_undegraded":"2024-09-12T02:56:31.032544+0000","last_fullsized":"2024-09-12T02:56:31.032544+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:32:48.399005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.032403+0000","last_change":"2024-09-12T02:56:15.987249+0000","last_active":"2024-09-12T02:56:31.032403+0000","last_peered":"2024-09-12T02:56:31.032403+0000","last_clean":"2024-09-12T02:56:31.032403+0000","last_became_active":"2024-09-12T02:56:15.986921+0000","last_became_peered":"2024-09-12T02:56:15.986921+0000","last_unstale":"2024-09-12T02:56:31.032403+0000","last_undegraded":"2024-09-12T02:56:31.032403+0000","last_fullsized":"2024-09-12T02:56:31.032403+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:42:13.427871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.031295+0000","last_change":"2024-09-12T02:56:15.987673+0000","last_active":"2024-09-12T02:56:31.031295+0000","last_peered":"2024-09-12T02:56:31.031295+0000","last_clean":"2024-09-12T02:56:31.031295+0000","last_became_active":"2024-09-12T02:56:15.986964+0000","last_became_peered":"2024-09-12T02:56:15.986964+0000","last_unstale":"2024-09-12T02:56:31.031295+0000","last_undegraded":"2024-09-12T02:56:31.031295+0000","last_fullsized":"2024-09-12T02:56:31.031295+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:07:12.406974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.034804+0000","last_change":"2024-09-12T02:56:15.974473+0000","last_active":"2024-09-12T02:56:31.034804+0000","last_peered":"2024-09-12T02:56:31.034804+0000","last_clean":"2024-09-12T02:56:31.034804+0000","last_became_active":"2024-09-12T02:56:15.974124+0000","last_became_peered":"2024-09-12T02:56:15.974124+0000","last_unstale":"2024-09-12T02:56:31.034804+0000","last_undegraded":"2024-09-12T02:56:31.034804+0000","last_fullsized":"2024-09-12T02:56:31.034804+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:44:29.054841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.032976+0000","last_change":"2024-09-12T02:56:15.980242+0000","last_active":"2024-09-12T02:56:31.032976+0000","last_peered":"2024-09-12T02:56:31.032976+0000","last_clean":"2024-09-12T02:56:31.032976+0000","last_became_active":"2024-09-12T02:56:15.979937+0000","last_became_peered":"2024-09-12T02:56:15.979937+0000","last_unstale":"2024-09-12T02:56:31.032976+0000","last_undegraded":"2024-09-12T02:56:31.032976+0000","last_fullsized":"2024-09-12T02:56:31.032976+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:44:13.406341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.035162+0000","last_change":"2024-09-12T02:56:15.978746+0000","last_active":"2024-09-12T02:56:31.035162+0000","last_peered":"2024-09-12T02:56:31.035162+0000","last_clean":"2024-09-12T02:56:31.035162+0000","last_became_active":"2024-09-12T02:56:15.978304+0000","last_became_peered":"2024-09-12T02:56:15.978304+0000","last_unstale":"2024-09-12T02:56:31.035162+0000","last_undegraded":"2024-09-12T02:56:31.035162+0000","last_fullsized":"2024-09-12T02:56:31.035162+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:34:52.115630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.034360+0000","last_change":"2024-09-12T02:56:15.980083+0000","last_active":"2024-09-12T02:56:31.034360+0000","last_peered":"2024-09-12T02:56:31.034360+0000","last_clean":"2024-09-12T02:56:31.034360+0000","last_became_active":"2024-09-12T02:56:15.979771+0000","last_became_peered":"2024-09-12T02:56:15.979771+0000","last_unstale":"2024-09-12T02:56:31.034360+0000","last_undegraded":"2024-09-12T02:56:31.034360+0000","last_fullsized":"2024-09-12T02:56:31.034360+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:59:42.351970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.306732+0000","last_change":"2024-09-12T02:56:15.988091+0000","last_active":"2024-09-12T02:56:31.306732+0000","last_peered":"2024-09-12T02:56:31.306732+0000","last_clean":"2024-09-12T02:56:31.306732+0000","last_became_active":"2024-09-12T02:56:15.986295+0000","last_became_peered":"2024-09-12T02:56:15.986295+0000","last_unstale":"2024-09-12T02:56:31.306732+0000","last_undegraded":"2024-09-12T02:56:31.306732+0000","last_fullsized":"2024-09-12T02:56:31.306732+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:45:40.331805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.033168+0000","last_change":"2024-09-12T02:56:15.987349+0000","last_active":"2024-09-12T02:56:31.033168+0000","last_peered":"2024-09-12T02:56:31.033168+0000","last_clean":"2024-09-12T02:56:31.033168+0000","last_became_active":"2024-09-12T02:56:15.987018+0000","last_became_peered":"2024-09-12T02:56:15.987018+0000","last_unstale":"2024-09-12T02:56:31.033168+0000","last_undegraded":"2024-09-12T02:56:31.033168+0000","last_fullsized":"2024-09-12T02:56:31.033168+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:34:19.652348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.035420+0000","last_change":"2024-09-12T02:56:15.985164+0000","last_active":"2024-09-12T02:56:31.035420+0000","last_peered":"2024-09-12T02:56:31.035420+0000","last_clean":"2024-09-12T02:56:31.035420+0000","last_became_active":"2024-09-12T02:56:15.984787+0000","last_became_peered":"2024-09-12T02:56:15.984787+0000","last_unstale":"2024-09-12T02:56:31.035420+0000","last_undegraded":"2024-09-12T02:56:31.035420+0000","last_fullsized":"2024-09-12T02:56:31.035420+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:36:29.727442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.031157+0000","last_change":"2024-09-12T02:56:15.979963+0000","last_active":"2024-09-12T02:56:31.031157+0000","last_peered":"2024-09-12T02:56:31.031157+0000","last_clean":"2024-09-12T02:56:31.031157+0000","last_became_active":"2024-09-12T02:56:15.979639+0000","last_became_peered":"2024-09-12T02:56:15.979639+0000","last_unstale":"2024-09-12T02:56:31.031157+0000","last_undegraded":"2024-09-12T02:56:31.031157+0000","last_fullsized":"2024-09-12T02:56:31.031157+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:34:18.273245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.031973+0000","last_change":"2024-09-12T02:56:15.980415+0000","last_active":"2024-09-12T02:56:31.031973+0000","last_peered":"2024-09-12T02:56:31.031973+0000","last_clean":"2024-09-12T02:56:31.031973+0000","last_became_active":"2024-09-12T02:56:15.980059+0000","last_became_peered":"2024-09-12T02:56:15.980059+0000","last_unstale":"2024-09-12T02:56:31.031973+0000","last_undegraded":"2024-09-12T02:56:31.031973+0000","last_fullsized":"2024-09-12T02:56:31.031973+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:11:42.689081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.032089+0000","last_change":"2024-09-12T02:56:15.981423+0000","last_active":"2024-09-12T02:56:31.032089+0000","last_peered":"2024-09-12T02:56:31.032089+0000","last_clean":"2024-09-12T02:56:31.032089+0000","last_became_active":"2024-09-12T02:56:15.981186+0000","last_became_peered":"2024-09-12T02:56:15.981186+0000","last_unstale":"2024-09-12T02:56:31.032089+0000","last_undegraded":"2024-09-12T02:56:31.032089+0000","last_fullsized":"2024-09-12T02:56:31.032089+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:36:57.406907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.032139+0000","last_change":"2024-09-12T02:56:15.982587+0000","last_active":"2024-09-12T02:56:31.032139+0000","last_peered":"2024-09-12T02:56:31.032139+0000","last_clean":"2024-09-12T02:56:31.032139+0000","last_became_active":"2024-09-12T02:56:15.982315+0000","last_became_peered":"2024-09-12T02:56:15.982315+0000","last_unstale":"2024-09-12T02:56:31.032139+0000","last_undegraded":"2024-09-12T02:56:31.032139+0000","last_fullsized":"2024-09-12T02:56:31.032139+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:42:17.736999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.306084+0000","last_change":"2024-09-12T02:56:15.979482+0000","last_active":"2024-09-12T02:56:31.306084+0000","last_peered":"2024-09-12T02:56:31.306084+0000","last_clean":"2024-09-12T02:56:31.306084+0000","last_became_active":"2024-09-12T02:56:15.979122+0000","last_became_peered":"2024-09-12T02:56:15.979122+0000","last_unstale":"2024-09-12T02:56:31.306084+0000","last_undegraded":"2024-09-12T02:56:31.306084+0000","last_fullsized":"2024-09-12T02:56:31.306084+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:18:05.642487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.033518+0000","last_change":"2024-09-12T02:56:15.989908+0000","last_active":"2024-09-12T02:56:31.033518+0000","last_peered":"2024-09-12T02:56:31.033518+0000","last_clean":"2024-09-12T02:56:31.033518+0000","last_became_active":"2024-09-12T02:56:15.989515+0000","last_became_peered":"2024-09-12T02:56:15.989515+0000","last_unstale":"2024-09-12T02:56:31.033518+0000","last_undegraded":"2024-09-12T02:56:31.033518+0000","last_fullsized":"2024-09-12T02:56:31.033518+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:59:57.141607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.029078+0000","last_change":"2024-09-12T02:56:15.980323+0000","last_active":"2024-09-12T02:56:31.029078+0000","last_peered":"2024-09-12T02:56:31.029078+0000","last_clean":"2024-09-12T02:56:31.029078+0000","last_became_active":"2024-09-12T02:56:15.979989+0000","last_became_peered":"2024-09-12T02:56:15.979989+0000","last_unstale":"2024-09-12T02:56:31.029078+0000","last_undegraded":"2024-09-12T02:56:31.029078+0000","last_fullsized":"2024-09-12T02:56:31.029078+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:55:56.064125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.030503+0000","last_change":"2024-09-12T02:56:15.986003+0000","last_active":"2024-09-12T02:56:31.030503+0000","last_peered":"2024-09-12T02:56:31.030503+0000","last_clean":"2024-09-12T02:56:31.030503+0000","last_became_active":"2024-09-12T02:56:15.985335+0000","last_became_peered":"2024-09-12T02:56:15.985335+0000","last_unstale":"2024-09-12T02:56:31.030503+0000","last_undegraded":"2024-09-12T02:56:31.030503+0000","last_fullsized":"2024-09-12T02:56:31.030503+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:51:47.944530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.036870+0000","last_change":"2024-09-12T02:56:31.036870+0000","last_active":"2024-09-12T02:56:31.036870+0000","last_peered":"2024-09-12T02:56:31.036870+0000","last_clean":"2024-09-12T02:56:31.036870+0000","last_became_active":"2024-09-12T02:56:31.036443+0000","last_became_peered":"2024-09-12T02:56:31.036443+0000","last_unstale":"2024-09-12T02:56:31.036870+0000","last_undegraded":"2024-09-12T02:56:31.036870+0000","last_fullsized":"2024-09-12T02:56:31.036870+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:49:29.326184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.032066+0000","last_change":"2024-09-12T02:56:15.974547+0000","last_active":"2024-09-12T02:56:31.032066+0000","last_peered":"2024-09-12T02:56:31.032066+0000","last_clean":"2024-09-12T02:56:31.032066+0000","last_became_active":"2024-09-12T02:56:15.974174+0000","last_became_peered":"2024-09-12T02:56:15.974174+0000","last_unstale":"2024-09-12T02:56:31.032066+0000","last_undegraded":"2024-09-12T02:56:31.032066+0000","last_fullsized":"2024-09-12T02:56:31.032066+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:22:16.954309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.306220+0000","last_change":"2024-09-12T02:56:15.981771+0000","last_active":"2024-09-12T02:56:31.306220+0000","last_peered":"2024-09-12T02:56:31.306220+0000","last_clean":"2024-09-12T02:56:31.306220+0000","last_became_active":"2024-09-12T02:56:15.980948+0000","last_became_peered":"2024-09-12T02:56:15.980948+0000","last_unstale":"2024-09-12T02:56:31.306220+0000","last_undegraded":"2024-09-12T02:56:31.306220+0000","last_fullsized":"2024-09-12T02:56:31.306220+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:34:35.961348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.034487+0000","last_change":"2024-09-12T02:56:15.982318+0000","last_active":"2024-09-12T02:56:31.034487+0000","last_peered":"2024-09-12T02:56:31.034487+0000","last_clean":"2024-09-12T02:56:31.034487+0000","last_became_active":"2024-09-12T02:56:15.982013+0000","last_became_peered":"2024-09-12T02:56:15.982013+0000","last_unstale":"2024-09-12T02:56:31.034487+0000","last_undegraded":"2024-09-12T02:56:31.034487+0000","last_fullsized":"2024-09-12T02:56:31.034487+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:10:50.895579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.032248+0000","last_change":"2024-09-12T02:56:15.969992+0000","last_active":"2024-09-12T02:56:31.032248+0000","last_peered":"2024-09-12T02:56:31.032248+0000","last_clean":"2024-09-12T02:56:31.032248+0000","last_became_active":"2024-09-12T02:56:15.969621+0000","last_became_peered":"2024-09-12T02:56:15.969621+0000","last_unstale":"2024-09-12T02:56:31.032248+0000","last_undegraded":"2024-09-12T02:56:31.032248+0000","last_fullsized":"2024-09-12T02:56:31.032248+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:47:40.206042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.034306+0000","last_change":"2024-09-12T02:56:15.982315+0000","last_active":"2024-09-12T02:56:31.034306+0000","last_peered":"2024-09-12T02:56:31.034306+0000","last_clean":"2024-09-12T02:56:31.034306+0000","last_became_active":"2024-09-12T02:56:15.981992+0000","last_became_peered":"2024-09-12T02:56:15.981992+0000","last_unstale":"2024-09-12T02:56:31.034306+0000","last_undegraded":"2024-09-12T02:56:31.034306+0000","last_fullsized":"2024-09-12T02:56:31.034306+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:54:04.188418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.030767+0000","last_change":"2024-09-12T02:56:15.980712+0000","last_active":"2024-09-12T02:56:31.030767+0000","last_peered":"2024-09-12T02:56:31.030767+0000","last_clean":"2024-09-12T02:56:31.030767+0000","last_became_active":"2024-09-12T02:56:15.980410+0000","last_became_peered":"2024-09-12T02:56:15.980410+0000","last_unstale":"2024-09-12T02:56:31.030767+0000","last_undegraded":"2024-09-12T02:56:31.030767+0000","last_fullsized":"2024-09-12T02:56:31.030767+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:13:53.844490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.033126+0000","last_change":"2024-09-12T02:56:15.981675+0000","last_active":"2024-09-12T02:56:31.033126+0000","last_peered":"2024-09-12T02:56:31.033126+0000","last_clean":"2024-09-12T02:56:31.033126+0000","last_became_active":"2024-09-12T02:56:15.981372+0000","last_became_peered":"2024-09-12T02:56:15.981372+0000","last_unstale":"2024-09-12T02:56:31.033126+0000","last_undegraded":"2024-09-12T02:56:31.033126+0000","last_fullsized":"2024-09-12T02:56:31.033126+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:39:17.399199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.032248+0000","last_change":"2024-09-12T02:56:15.972750+0000","last_active":"2024-09-12T02:56:31.032248+0000","last_peered":"2024-09-12T02:56:31.032248+0000","last_clean":"2024-09-12T02:56:31.032248+0000","last_became_active":"2024-09-12T02:56:15.972434+0000","last_became_peered":"2024-09-12T02:56:15.972434+0000","last_unstale":"2024-09-12T02:56:31.032248+0000","last_undegraded":"2024-09-12T02:56:31.032248+0000","last_fullsized":"2024-09-12T02:56:31.032248+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:59:22.784908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.032502+0000","last_change":"2024-09-12T02:56:15.973567+0000","last_active":"2024-09-12T02:56:31.032502+0000","last_peered":"2024-09-12T02:56:31.032502+0000","last_clean":"2024-09-12T02:56:31.032502+0000","last_became_active":"2024-09-12T02:56:15.973251+0000","last_became_peered":"2024-09-12T02:56:15.973251+0000","last_unstale":"2024-09-12T02:56:31.032502+0000","last_undegraded":"2024-09-12T02:56:31.032502+0000","last_fullsized":"2024-09-12T02:56:31.032502+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:16:39.536817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.030130+0000","last_change":"2024-09-12T02:56:15.986108+0000","last_active":"2024-09-12T02:56:31.030130+0000","last_peered":"2024-09-12T02:56:31.030130+0000","last_clean":"2024-09-12T02:56:31.030130+0000","last_became_active":"2024-09-12T02:56:15.985701+0000","last_became_peered":"2024-09-12T02:56:15.985701+0000","last_unstale":"2024-09-12T02:56:31.030130+0000","last_undegraded":"2024-09-12T02:56:31.030130+0000","last_fullsized":"2024-09-12T02:56:31.030130+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:59.377473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.030220+0000","last_change":"2024-09-12T02:56:15.980078+0000","last_active":"2024-09-12T02:56:31.030220+0000","last_peered":"2024-09-12T02:56:31.030220+0000","last_clean":"2024-09-12T02:56:31.030220+0000","last_became_active":"2024-09-12T02:56:15.979785+0000","last_became_peered":"2024-09-12T02:56:15.979785+0000","last_unstale":"2024-09-12T02:56:31.030220+0000","last_undegraded":"2024-09-12T02:56:31.030220+0000","last_fullsized":"2024-09-12T02:56:31.030220+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:00:19.252398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.034505+0000","last_change":"2024-09-12T02:56:15.984170+0000","last_active":"2024-09-12T02:56:31.034505+0000","last_peered":"2024-09-12T02:56:31.034505+0000","last_clean":"2024-09-12T02:56:31.034505+0000","last_became_active":"2024-09-12T02:56:15.983416+0000","last_became_peered":"2024-09-12T02:56:15.983416+0000","last_unstale":"2024-09-12T02:56:31.034505+0000","last_undegraded":"2024-09-12T02:56:31.034505+0000","last_fullsized":"2024-09-12T02:56:31.034505+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:13:24.723923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.035795+0000","last_change":"2024-09-12T02:56:15.980987+0000","last_active":"2024-09-12T02:56:31.035795+0000","last_peered":"2024-09-12T02:56:31.035795+0000","last_clean":"2024-09-12T02:56:31.035795+0000","last_became_active":"2024-09-12T02:56:15.980661+0000","last_became_peered":"2024-09-12T02:56:15.980661+0000","last_unstale":"2024-09-12T02:56:31.035795+0000","last_undegraded":"2024-09-12T02:56:31.035795+0000","last_fullsized":"2024-09-12T02:56:31.035795+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:19:49.294698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"81'5","reported_seq":90,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:15.105763+0000","last_change":"2024-09-12T02:56:31.042914+0000","last_active":"2024-09-12T02:57:15.105763+0000","last_peered":"2024-09-12T02:57:15.105763+0000","last_clean":"2024-09-12T02:57:15.105763+0000","last_became_active":"2024-09-12T02:56:31.037356+0000","last_became_peered":"2024-09-12T02:56:31.037356+0000","last_unstale":"2024-09-12T02:57:15.105763+0000","last_undegraded":"2024-09-12T02:57:15.105763+0000","last_fullsized":"2024-09-12T02:57:15.105763+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:17:48.582725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":51,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":622,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.036619+0000","last_change":"2024-09-12T02:56:31.036170+0000","last_active":"2024-09-12T02:56:31.036619+0000","last_peered":"2024-09-12T02:56:31.036619+0000","last_clean":"2024-09-12T02:56:31.036619+0000","last_became_active":"2024-09-12T02:56:31.035754+0000","last_became_peered":"2024-09-12T02:56:31.035754+0000","last_unstale":"2024-09-12T02:56:31.036619+0000","last_undegraded":"2024-09-12T02:56:31.036619+0000","last_fullsized":"2024-09-12T02:56:31.036619+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:58:06.464818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1a","version":"81'7","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.032813+0000","last_change":"2024-09-12T02:56:13.931202+0000","last_active":"2024-09-12T02:56:31.032813+0000","last_peered":"2024-09-12T02:56:31.032813+0000","last_clean":"2024-09-12T02:56:31.032813+0000","last_became_active":"2024-09-12T02:56:13.930776+0000","last_became_peered":"2024-09-12T02:56:13.930776+0000","last_unstale":"2024-09-12T02:56:31.032813+0000","last_undegraded":"2024-09-12T02:56:31.032813+0000","last_fullsized":"2024-09-12T02:56:31.032813+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:22:16.919253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"81'3","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.031662+0000","last_change":"2024-09-12T02:56:13.924660+0000","last_active":"2024-09-12T02:56:31.031662+0000","last_peered":"2024-09-12T02:56:31.031662+0000","last_clean":"2024-09-12T02:56:31.031662+0000","last_became_active":"2024-09-12T02:56:13.924342+0000","last_became_peered":"2024-09-12T02:56:13.924342+0000","last_unstale":"2024-09-12T02:56:31.031662+0000","last_undegraded":"2024-09-12T02:56:31.031662+0000","last_fullsized":"2024-09-12T02:56:31.031662+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:48:53.907044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.034409+0000","last_change":"2024-09-12T02:56:15.978723+0000","last_active":"2024-09-12T02:56:31.034409+0000","last_peered":"2024-09-12T02:56:31.034409+0000","last_clean":"2024-09-12T02:56:31.034409+0000","last_became_active":"2024-09-12T02:56:15.978344+0000","last_became_peered":"2024-09-12T02:56:15.978344+0000","last_unstale":"2024-09-12T02:56:31.034409+0000","last_undegraded":"2024-09-12T02:56:31.034409+0000","last_fullsized":"2024-09-12T02:56:31.034409+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:25:04.559395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"81'3","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.032936+0000","last_change":"2024-09-12T02:56:13.933065+0000","last_active":"2024-09-12T02:56:31.032936+0000","last_peered":"2024-09-12T02:56:31.032936+0000","last_clean":"2024-09-12T02:56:31.032936+0000","last_became_active":"2024-09-12T02:56:13.932597+0000","last_became_peered":"2024-09-12T02:56:13.932597+0000","last_unstale":"2024-09-12T02:56:31.032936+0000","last_undegraded":"2024-09-12T02:56:31.032936+0000","last_fullsized":"2024-09-12T02:56:31.032936+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:57:26.563392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.028878+0000","last_change":"2024-09-12T02:56:15.975722+0000","last_active":"2024-09-12T02:56:31.028878+0000","last_peered":"2024-09-12T02:56:31.028878+0000","last_clean":"2024-09-12T02:56:31.028878+0000","last_became_active":"2024-09-12T02:56:15.975342+0000","last_became_peered":"2024-09-12T02:56:15.975342+0000","last_unstale":"2024-09-12T02:56:31.028878+0000","last_undegraded":"2024-09-12T02:56:31.028878+0000","last_fullsized":"2024-09-12T02:56:31.028878+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:33:17.613738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"81'4","reported_seq":116,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:11.909871+0000","last_change":"2024-09-12T02:56:13.929700+0000","last_active":"2024-09-12T02:57:11.909871+0000","last_peered":"2024-09-12T02:57:11.909871+0000","last_clean":"2024-09-12T02:57:11.909871+0000","last_became_active":"2024-09-12T02:56:13.929317+0000","last_became_peered":"2024-09-12T02:56:13.929317+0000","last_unstale":"2024-09-12T02:57:11.909871+0000","last_undegraded":"2024-09-12T02:57:11.909871+0000","last_fullsized":"2024-09-12T02:57:11.909871+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:46:32.172674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":90,"num_read_kb":90,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.306705+0000","last_change":"2024-09-12T02:56:15.985754+0000","last_active":"2024-09-12T02:56:31.306705+0000","last_peered":"2024-09-12T02:56:31.306705+0000","last_clean":"2024-09-12T02:56:31.306705+0000","last_became_active":"2024-09-12T02:56:15.985402+0000","last_became_peered":"2024-09-12T02:56:15.985402+0000","last_unstale":"2024-09-12T02:56:31.306705+0000","last_undegraded":"2024-09-12T02:56:31.306705+0000","last_fullsized":"2024-09-12T02:56:31.306705+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:05:29.979961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"81'4","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.035428+0000","last_change":"2024-09-12T02:56:13.933329+0000","last_active":"2024-09-12T02:56:31.035428+0000","last_peered":"2024-09-12T02:56:31.035428+0000","last_clean":"2024-09-12T02:56:31.035428+0000","last_became_active":"2024-09-12T02:56:13.932960+0000","last_became_peered":"2024-09-12T02:56:13.932960+0000","last_unstale":"2024-09-12T02:56:31.035428+0000","last_undegraded":"2024-09-12T02:56:31.035428+0000","last_fullsized":"2024-09-12T02:56:31.035428+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:02:57.564202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.037340+0000","last_change":"2024-09-12T02:56:31.037340+0000","last_active":"2024-09-12T02:56:31.037340+0000","last_peered":"2024-09-12T02:56:31.037340+0000","last_clean":"2024-09-12T02:56:31.037340+0000","last_became_active":"2024-09-12T02:56:31.036993+0000","last_became_peered":"2024-09-12T02:56:31.036993+0000","last_unstale":"2024-09-12T02:56:31.037340+0000","last_undegraded":"2024-09-12T02:56:31.037340+0000","last_fullsized":"2024-09-12T02:56:31.037340+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:44:27.788791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"81'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.033005+0000","last_change":"2024-09-12T02:56:14.077202+0000","last_active":"2024-09-12T02:56:31.033005+0000","last_peered":"2024-09-12T02:56:31.033005+0000","last_clean":"2024-09-12T02:56:31.033005+0000","last_became_active":"2024-09-12T02:56:14.076610+0000","last_became_peered":"2024-09-12T02:56:14.076610+0000","last_unstale":"2024-09-12T02:56:31.033005+0000","last_undegraded":"2024-09-12T02:56:31.033005+0000","last_fullsized":"2024-09-12T02:56:31.033005+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:37:50.333152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.306307+0000","last_change":"2024-09-12T02:56:15.977161+0000","last_active":"2024-09-12T02:56:31.306307+0000","last_peered":"2024-09-12T02:56:31.306307+0000","last_clean":"2024-09-12T02:56:31.306307+0000","last_became_active":"2024-09-12T02:56:15.976683+0000","last_became_peered":"2024-09-12T02:56:15.976683+0000","last_unstale":"2024-09-12T02:56:31.306307+0000","last_undegraded":"2024-09-12T02:56:31.306307+0000","last_fullsized":"2024-09-12T02:56:31.306307+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:34:07.526890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'31","reported_seq":152,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:11.909864+0000","last_change":"2024-09-12T02:56:13.931680+0000","last_active":"2024-09-12T02:57:11.909864+0000","last_peered":"2024-09-12T02:57:11.909864+0000","last_clean":"2024-09-12T02:57:11.909864+0000","last_became_active":"2024-09-12T02:56:13.931214+0000","last_became_peered":"2024-09-12T02:56:13.931214+0000","last_unstale":"2024-09-12T02:57:11.909864+0000","last_undegraded":"2024-09-12T02:57:11.909864+0000","last_fullsized":"2024-09-12T02:57:11.909864+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:38:34.151933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61167,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":99,"num_read_kb":108,"num_write":33,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.035329+0000","last_change":"2024-09-12T02:56:15.977720+0000","last_active":"2024-09-12T02:56:31.035329+0000","last_peered":"2024-09-12T02:56:31.035329+0000","last_clean":"2024-09-12T02:56:31.035329+0000","last_became_active":"2024-09-12T02:56:15.977380+0000","last_became_peered":"2024-09-12T02:56:15.977380+0000","last_unstale":"2024-09-12T02:56:31.035329+0000","last_undegraded":"2024-09-12T02:56:31.035329+0000","last_fullsized":"2024-09-12T02:56:31.035329+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:52:16.841909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"81'9","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.030362+0000","last_change":"2024-09-12T02:56:13.929496+0000","last_active":"2024-09-12T02:56:31.030362+0000","last_peered":"2024-09-12T02:56:31.030362+0000","last_clean":"2024-09-12T02:56:31.030362+0000","last_became_active":"2024-09-12T02:56:13.929117+0000","last_became_peered":"2024-09-12T02:56:13.929117+0000","last_unstale":"2024-09-12T02:56:31.030362+0000","last_undegraded":"2024-09-12T02:56:31.030362+0000","last_fullsized":"2024-09-12T02:56:31.030362+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:59:38.399135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.031759+0000","last_change":"2024-09-12T02:56:15.974323+0000","last_active":"2024-09-12T02:56:31.031759+0000","last_peered":"2024-09-12T02:56:31.031759+0000","last_clean":"2024-09-12T02:56:31.031759+0000","last_became_active":"2024-09-12T02:56:15.973819+0000","last_became_peered":"2024-09-12T02:56:15.973819+0000","last_unstale":"2024-09-12T02:56:31.031759+0000","last_undegraded":"2024-09-12T02:56:31.031759+0000","last_fullsized":"2024-09-12T02:56:31.031759+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:53:13.130695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"81'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.032473+0000","last_change":"2024-09-12T02:56:13.935263+0000","last_active":"2024-09-12T02:56:31.032473+0000","last_peered":"2024-09-12T02:56:31.032473+0000","last_clean":"2024-09-12T02:56:31.032473+0000","last_became_active":"2024-09-12T02:56:13.934805+0000","last_became_peered":"2024-09-12T02:56:13.934805+0000","last_unstale":"2024-09-12T02:56:31.032473+0000","last_undegraded":"2024-09-12T02:56:31.032473+0000","last_fullsized":"2024-09-12T02:56:31.032473+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:39:42.882031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.035564+0000","last_change":"2024-09-12T02:56:15.976630+0000","last_active":"2024-09-12T02:56:31.035564+0000","last_peered":"2024-09-12T02:56:31.035564+0000","last_clean":"2024-09-12T02:56:31.035564+0000","last_became_active":"2024-09-12T02:56:15.976304+0000","last_became_peered":"2024-09-12T02:56:15.976304+0000","last_unstale":"2024-09-12T02:56:31.035564+0000","last_undegraded":"2024-09-12T02:56:31.035564+0000","last_fullsized":"2024-09-12T02:56:31.035564+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:46:03.366740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"81'6","reported_seq":80,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:14.911496+0000","last_change":"2024-09-12T02:56:13.934224+0000","last_active":"2024-09-12T02:57:14.911496+0000","last_peered":"2024-09-12T02:57:14.911496+0000","last_clean":"2024-09-12T02:57:14.911496+0000","last_became_active":"2024-09-12T02:56:13.933838+0000","last_became_peered":"2024-09-12T02:56:13.933838+0000","last_unstale":"2024-09-12T02:57:14.911496+0000","last_undegraded":"2024-09-12T02:57:14.911496+0000","last_fullsized":"2024-09-12T02:57:14.911496+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:34:34.206051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":53,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.035229+0000","last_change":"2024-09-12T02:56:15.980364+0000","last_active":"2024-09-12T02:56:31.035229+0000","last_peered":"2024-09-12T02:56:31.035229+0000","last_clean":"2024-09-12T02:56:31.035229+0000","last_became_active":"2024-09-12T02:56:15.979437+0000","last_became_peered":"2024-09-12T02:56:15.979437+0000","last_unstale":"2024-09-12T02:56:31.035229+0000","last_undegraded":"2024-09-12T02:56:31.035229+0000","last_fullsized":"2024-09-12T02:56:31.035229+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:43:33.424867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"81'17","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.050509+0000","last_change":"2024-09-12T02:56:31.050046+0000","last_active":"2024-09-12T02:56:31.050509+0000","last_peered":"2024-09-12T02:56:31.050509+0000","last_clean":"2024-09-12T02:56:31.050509+0000","last_became_active":"2024-09-12T02:56:31.036990+0000","last_became_peered":"2024-09-12T02:56:31.036990+0000","last_unstale":"2024-09-12T02:56:31.050509+0000","last_undegraded":"2024-09-12T02:56:31.050509+0000","last_fullsized":"2024-09-12T02:56:31.050509+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:05:53.336691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.032226+0000","last_change":"2024-09-12T02:56:15.986434+0000","last_active":"2024-09-12T02:56:31.032226+0000","last_peered":"2024-09-12T02:56:31.032226+0000","last_clean":"2024-09-12T02:56:31.032226+0000","last_became_active":"2024-09-12T02:56:15.986101+0000","last_became_peered":"2024-09-12T02:56:15.986101+0000","last_unstale":"2024-09-12T02:56:31.032226+0000","last_undegraded":"2024-09-12T02:56:31.032226+0000","last_fullsized":"2024-09-12T02:56:31.032226+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:51:27.840951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"81'2","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.035974+0000","last_change":"2024-09-12T02:56:13.925068+0000","last_active":"2024-09-12T02:56:31.035974+0000","last_peered":"2024-09-12T02:56:31.035974+0000","last_clean":"2024-09-12T02:56:31.035974+0000","last_became_active":"2024-09-12T02:56:13.924535+0000","last_became_peered":"2024-09-12T02:56:13.924535+0000","last_unstale":"2024-09-12T02:56:31.035974+0000","last_undegraded":"2024-09-12T02:56:31.035974+0000","last_fullsized":"2024-09-12T02:56:31.035974+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:45:32.478632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.033033+0000","last_change":"2024-09-12T02:56:15.983850+0000","last_active":"2024-09-12T02:56:31.033033+0000","last_peered":"2024-09-12T02:56:31.033033+0000","last_clean":"2024-09-12T02:56:31.033033+0000","last_became_active":"2024-09-12T02:56:15.983480+0000","last_became_peered":"2024-09-12T02:56:15.983480+0000","last_unstale":"2024-09-12T02:56:31.033033+0000","last_undegraded":"2024-09-12T02:56:31.033033+0000","last_fullsized":"2024-09-12T02:56:31.033033+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:09:42.185237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"81'5","reported_seq":39,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.312450+0000","last_change":"2024-09-12T02:56:31.312450+0000","last_active":"2024-09-12T02:56:31.312450+0000","last_peered":"2024-09-12T02:56:31.312450+0000","last_clean":"2024-09-12T02:56:31.312450+0000","last_became_active":"2024-09-12T02:56:31.308030+0000","last_became_peered":"2024-09-12T02:56:31.308030+0000","last_unstale":"2024-09-12T02:56:31.312450+0000","last_undegraded":"2024-09-12T02:56:31.312450+0000","last_fullsized":"2024-09-12T02:56:31.312450+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:15:14.692292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.035489+0000","last_change":"2024-09-12T02:56:15.978996+0000","last_active":"2024-09-12T02:56:31.035489+0000","last_peered":"2024-09-12T02:56:31.035489+0000","last_clean":"2024-09-12T02:56:31.035489+0000","last_became_active":"2024-09-12T02:56:15.978636+0000","last_became_peered":"2024-09-12T02:56:15.978636+0000","last_unstale":"2024-09-12T02:56:31.035489+0000","last_undegraded":"2024-09-12T02:56:31.035489+0000","last_fullsized":"2024-09-12T02:56:31.035489+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:29:20.401664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"81'3","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.033119+0000","last_change":"2024-09-12T02:56:13.929388+0000","last_active":"2024-09-12T02:56:31.033119+0000","last_peered":"2024-09-12T02:56:31.033119+0000","last_clean":"2024-09-12T02:56:31.033119+0000","last_became_active":"2024-09-12T02:56:13.928998+0000","last_became_peered":"2024-09-12T02:56:13.928998+0000","last_unstale":"2024-09-12T02:56:31.033119+0000","last_undegraded":"2024-09-12T02:56:31.033119+0000","last_fullsized":"2024-09-12T02:56:31.033119+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:37:57.368050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'1","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.035355+0000","last_change":"2024-09-12T02:56:15.977491+0000","last_active":"2024-09-12T02:56:31.035355+0000","last_peered":"2024-09-12T02:56:31.035355+0000","last_clean":"2024-09-12T02:56:31.035355+0000","last_became_active":"2024-09-12T02:56:15.977160+0000","last_became_peered":"2024-09-12T02:56:15.977160+0000","last_unstale":"2024-09-12T02:56:31.035355+0000","last_undegraded":"2024-09-12T02:56:31.035355+0000","last_fullsized":"2024-09-12T02:56:31.035355+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:05:40.577342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"81'6","reported_seq":81,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:15.915412+0000","last_change":"2024-09-12T02:56:14.327147+0000","last_active":"2024-09-12T02:57:15.915412+0000","last_peered":"2024-09-12T02:57:15.915412+0000","last_clean":"2024-09-12T02:57:15.915412+0000","last_became_active":"2024-09-12T02:56:14.326737+0000","last_became_peered":"2024-09-12T02:56:14.326737+0000","last_unstale":"2024-09-12T02:57:15.915412+0000","last_undegraded":"2024-09-12T02:57:15.915412+0000","last_fullsized":"2024-09-12T02:57:15.915412+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:55:33.510599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1563,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":53,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.035591+0000","last_change":"2024-09-12T02:56:15.984363+0000","last_active":"2024-09-12T02:56:31.035591+0000","last_peered":"2024-09-12T02:56:31.035591+0000","last_clean":"2024-09-12T02:56:31.035591+0000","last_became_active":"2024-09-12T02:56:15.983822+0000","last_became_peered":"2024-09-12T02:56:15.983822+0000","last_unstale":"2024-09-12T02:56:31.035591+0000","last_undegraded":"2024-09-12T02:56:31.035591+0000","last_fullsized":"2024-09-12T02:56:31.035591+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:41:46.082716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":332,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.035507+0000","last_change":"2024-09-12T02:55:05.800210+0000","last_active":"2024-09-12T02:56:31.035507+0000","last_peered":"2024-09-12T02:56:31.035507+0000","last_clean":"2024-09-12T02:56:31.035507+0000","last_became_active":"2024-09-12T02:55:05.799791+0000","last_became_peered":"2024-09-12T02:55:05.799791+0000","last_unstale":"2024-09-12T02:56:31.035507+0000","last_undegraded":"2024-09-12T02:56:31.035507+0000","last_fullsized":"2024-09-12T02:56:31.035507+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:51:48.530016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:51:48.530016+0000","last_clean_scrub_stamp":"2024-09-12T02:51:48.530016+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:05:47.416697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"81'6","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.031169+0000","last_change":"2024-09-12T02:56:13.930269+0000","last_active":"2024-09-12T02:56:31.031169+0000","last_peered":"2024-09-12T02:56:31.031169+0000","last_clean":"2024-09-12T02:56:31.031169+0000","last_became_active":"2024-09-12T02:56:13.929831+0000","last_became_peered":"2024-09-12T02:56:13.929831+0000","last_unstale":"2024-09-12T02:56:31.031169+0000","last_undegraded":"2024-09-12T02:56:31.031169+0000","last_fullsized":"2024-09-12T02:56:31.031169+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:38:17.878398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.034774+0000","last_change":"2024-09-12T02:56:15.969744+0000","last_active":"2024-09-12T02:56:31.034774+0000","last_peered":"2024-09-12T02:56:31.034774+0000","last_clean":"2024-09-12T02:56:31.034774+0000","last_became_active":"2024-09-12T02:56:15.969354+0000","last_became_peered":"2024-09-12T02:56:15.969354+0000","last_unstale":"2024-09-12T02:56:31.034774+0000","last_undegraded":"2024-09-12T02:56:31.034774+0000","last_fullsized":"2024-09-12T02:56:31.034774+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:26:02.409510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"81'4","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.035385+0000","last_change":"2024-09-12T02:56:13.929231+0000","last_active":"2024-09-12T02:56:31.035385+0000","last_peered":"2024-09-12T02:56:31.035385+0000","last_clean":"2024-09-12T02:56:31.035385+0000","last_became_active":"2024-09-12T02:56:13.928843+0000","last_became_peered":"2024-09-12T02:56:13.928843+0000","last_unstale":"2024-09-12T02:56:31.035385+0000","last_undegraded":"2024-09-12T02:56:31.035385+0000","last_fullsized":"2024-09-12T02:56:31.035385+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:26:33.619755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.031654+0000","last_change":"2024-09-12T02:56:15.982503+0000","last_active":"2024-09-12T02:56:31.031654+0000","last_peered":"2024-09-12T02:56:31.031654+0000","last_clean":"2024-09-12T02:56:31.031654+0000","last_became_active":"2024-09-12T02:56:15.982192+0000","last_became_peered":"2024-09-12T02:56:15.982192+0000","last_unstale":"2024-09-12T02:56:31.031654+0000","last_undegraded":"2024-09-12T02:56:31.031654+0000","last_fullsized":"2024-09-12T02:56:31.031654+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:55:18.370967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"81'5","reported_seq":76,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:13.909347+0000","last_change":"2024-09-12T02:56:14.075486+0000","last_active":"2024-09-12T02:57:13.909347+0000","last_peered":"2024-09-12T02:57:13.909347+0000","last_clean":"2024-09-12T02:57:13.909347+0000","last_became_active":"2024-09-12T02:56:14.075152+0000","last_became_peered":"2024-09-12T02:56:14.075152+0000","last_unstale":"2024-09-12T02:57:13.909347+0000","last_undegraded":"2024-09-12T02:57:13.909347+0000","last_fullsized":"2024-09-12T02:57:13.909347+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:59:40.950191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":49,"num_read_kb":49,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.032420+0000","last_change":"2024-09-12T02:56:15.981584+0000","last_active":"2024-09-12T02:56:31.032420+0000","last_peered":"2024-09-12T02:56:31.032420+0000","last_clean":"2024-09-12T02:56:31.032420+0000","last_became_active":"2024-09-12T02:56:15.981240+0000","last_became_peered":"2024-09-12T02:56:15.981240+0000","last_unstale":"2024-09-12T02:56:31.032420+0000","last_undegraded":"2024-09-12T02:56:31.032420+0000","last_fullsized":"2024-09-12T02:56:31.032420+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:49:04.036830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"81'5","reported_seq":75,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:12.915095+0000","last_change":"2024-09-12T02:56:13.935143+0000","last_active":"2024-09-12T02:57:12.915095+0000","last_peered":"2024-09-12T02:57:12.915095+0000","last_clean":"2024-09-12T02:57:12.915095+0000","last_became_active":"2024-09-12T02:56:13.934792+0000","last_became_peered":"2024-09-12T02:56:13.934792+0000","last_unstale":"2024-09-12T02:57:12.915095+0000","last_undegraded":"2024-09-12T02:57:12.915095+0000","last_fullsized":"2024-09-12T02:57:12.915095+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:00:26.973612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":48,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.034825+0000","last_change":"2024-09-12T02:56:31.034825+0000","last_active":"2024-09-12T02:56:31.034825+0000","last_peered":"2024-09-12T02:56:31.034825+0000","last_clean":"2024-09-12T02:56:31.034825+0000","last_became_active":"2024-09-12T02:56:31.034516+0000","last_became_peered":"2024-09-12T02:56:31.034516+0000","last_unstale":"2024-09-12T02:56:31.034825+0000","last_undegraded":"2024-09-12T02:56:31.034825+0000","last_fullsized":"2024-09-12T02:56:31.034825+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:10:42.161023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"81'1","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.029027+0000","last_change":"2024-09-12T02:56:14.076762+0000","last_active":"2024-09-12T02:56:31.029027+0000","last_peered":"2024-09-12T02:56:31.029027+0000","last_clean":"2024-09-12T02:56:31.029027+0000","last_became_active":"2024-09-12T02:56:14.076421+0000","last_became_peered":"2024-09-12T02:56:14.076421+0000","last_unstale":"2024-09-12T02:56:31.029027+0000","last_undegraded":"2024-09-12T02:56:31.029027+0000","last_fullsized":"2024-09-12T02:56:31.029027+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:56:56.422910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"82'1","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.033103+0000","last_change":"2024-09-12T02:56:15.978810+0000","last_active":"2024-09-12T02:56:31.033103+0000","last_peered":"2024-09-12T02:56:31.033103+0000","last_clean":"2024-09-12T02:56:31.033103+0000","last_became_active":"2024-09-12T02:56:15.978038+0000","last_became_peered":"2024-09-12T02:56:15.978038+0000","last_unstale":"2024-09-12T02:56:31.033103+0000","last_undegraded":"2024-09-12T02:56:31.033103+0000","last_fullsized":"2024-09-12T02:56:31.033103+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:04:52.215784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"81'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.035440+0000","last_change":"2024-09-12T02:56:14.075987+0000","last_active":"2024-09-12T02:56:31.035440+0000","last_peered":"2024-09-12T02:56:31.035440+0000","last_clean":"2024-09-12T02:56:31.035440+0000","last_became_active":"2024-09-12T02:56:14.075596+0000","last_became_peered":"2024-09-12T02:56:14.075596+0000","last_unstale":"2024-09-12T02:56:31.035440+0000","last_undegraded":"2024-09-12T02:56:31.035440+0000","last_fullsized":"2024-09-12T02:56:31.035440+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:18:48.713387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.032345+0000","last_change":"2024-09-12T02:56:15.987842+0000","last_active":"2024-09-12T02:56:31.032345+0000","last_peered":"2024-09-12T02:56:31.032345+0000","last_clean":"2024-09-12T02:56:31.032345+0000","last_became_active":"2024-09-12T02:56:15.987507+0000","last_became_peered":"2024-09-12T02:56:15.987507+0000","last_unstale":"2024-09-12T02:56:31.032345+0000","last_undegraded":"2024-09-12T02:56:31.032345+0000","last_fullsized":"2024-09-12T02:56:31.032345+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:52:23.000549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'16","reported_seq":38,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:39.980001+0000","last_change":"2024-09-12T02:56:13.924372+0000","last_active":"2024-09-12T02:56:39.980001+0000","last_peered":"2024-09-12T02:56:39.980001+0000","last_clean":"2024-09-12T02:56:39.980001+0000","last_became_active":"2024-09-12T02:56:13.923955+0000","last_became_peered":"2024-09-12T02:56:13.923955+0000","last_unstale":"2024-09-12T02:56:39.980001+0000","last_undegraded":"2024-09-12T02:56:39.980001+0000","last_fullsized":"2024-09-12T02:56:39.980001+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:52:47.452410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.034382+0000","last_change":"2024-09-12T02:56:15.978262+0000","last_active":"2024-09-12T02:56:31.034382+0000","last_peered":"2024-09-12T02:56:31.034382+0000","last_clean":"2024-09-12T02:56:31.034382+0000","last_became_active":"2024-09-12T02:56:15.977913+0000","last_became_peered":"2024-09-12T02:56:15.977913+0000","last_unstale":"2024-09-12T02:56:31.034382+0000","last_undegraded":"2024-09-12T02:56:31.034382+0000","last_fullsized":"2024-09-12T02:56:31.034382+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:35:27.104635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"81'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.031488+0000","last_change":"2024-09-12T02:56:14.327417+0000","last_active":"2024-09-12T02:56:31.031488+0000","last_peered":"2024-09-12T02:56:31.031488+0000","last_clean":"2024-09-12T02:56:31.031488+0000","last_became_active":"2024-09-12T02:56:14.327001+0000","last_became_peered":"2024-09-12T02:56:14.327001+0000","last_unstale":"2024-09-12T02:56:31.031488+0000","last_undegraded":"2024-09-12T02:56:31.031488+0000","last_fullsized":"2024-09-12T02:56:31.031488+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:15:39.019252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.032658+0000","last_change":"2024-09-12T02:56:15.977286+0000","last_active":"2024-09-12T02:56:31.032658+0000","last_peered":"2024-09-12T02:56:31.032658+0000","last_clean":"2024-09-12T02:56:31.032658+0000","last_became_active":"2024-09-12T02:56:15.976944+0000","last_became_peered":"2024-09-12T02:56:15.976944+0000","last_unstale":"2024-09-12T02:56:31.032658+0000","last_undegraded":"2024-09-12T02:56:31.032658+0000","last_fullsized":"2024-09-12T02:56:31.032658+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:22:57.056781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.306212+0000","last_change":"2024-09-12T02:56:14.075701+0000","last_active":"2024-09-12T02:56:31.306212+0000","last_peered":"2024-09-12T02:56:31.306212+0000","last_clean":"2024-09-12T02:56:31.306212+0000","last_became_active":"2024-09-12T02:56:14.075340+0000","last_became_peered":"2024-09-12T02:56:14.075340+0000","last_unstale":"2024-09-12T02:56:31.306212+0000","last_undegraded":"2024-09-12T02:56:31.306212+0000","last_fullsized":"2024-09-12T02:56:31.306212+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:17:42.255058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.034874+0000","last_change":"2024-09-12T02:56:15.979783+0000","last_active":"2024-09-12T02:56:31.034874+0000","last_peered":"2024-09-12T02:56:31.034874+0000","last_clean":"2024-09-12T02:56:31.034874+0000","last_became_active":"2024-09-12T02:56:15.979424+0000","last_became_peered":"2024-09-12T02:56:15.979424+0000","last_unstale":"2024-09-12T02:56:31.034874+0000","last_undegraded":"2024-09-12T02:56:31.034874+0000","last_fullsized":"2024-09-12T02:56:31.034874+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:59:11.568138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"81'7","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.034660+0000","last_change":"2024-09-12T02:56:13.928996+0000","last_active":"2024-09-12T02:56:31.034660+0000","last_peered":"2024-09-12T02:56:31.034660+0000","last_clean":"2024-09-12T02:56:31.034660+0000","last_became_active":"2024-09-12T02:56:13.928664+0000","last_became_peered":"2024-09-12T02:56:13.928664+0000","last_unstale":"2024-09-12T02:56:31.034660+0000","last_undegraded":"2024-09-12T02:56:31.034660+0000","last_fullsized":"2024-09-12T02:56:31.034660+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:37:22.067930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.034581+0000","last_change":"2024-09-12T02:56:15.984096+0000","last_active":"2024-09-12T02:56:31.034581+0000","last_peered":"2024-09-12T02:56:31.034581+0000","last_clean":"2024-09-12T02:56:31.034581+0000","last_became_active":"2024-09-12T02:56:15.983737+0000","last_became_peered":"2024-09-12T02:56:15.983737+0000","last_unstale":"2024-09-12T02:56:31.034581+0000","last_undegraded":"2024-09-12T02:56:31.034581+0000","last_fullsized":"2024-09-12T02:56:31.034581+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:02:17.892318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"81'3","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.053785+0000","last_change":"2024-09-12T02:56:31.053343+0000","last_active":"2024-09-12T02:56:31.053785+0000","last_peered":"2024-09-12T02:56:31.053785+0000","last_clean":"2024-09-12T02:56:31.053785+0000","last_became_active":"2024-09-12T02:56:31.037918+0000","last_became_peered":"2024-09-12T02:56:31.037918+0000","last_unstale":"2024-09-12T02:56:31.053785+0000","last_undegraded":"2024-09-12T02:56:31.053785+0000","last_fullsized":"2024-09-12T02:56:31.053785+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:39:54.990724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.034343+0000","last_change":"2024-09-12T02:56:15.975640+0000","last_active":"2024-09-12T02:56:31.034343+0000","last_peered":"2024-09-12T02:56:31.034343+0000","last_clean":"2024-09-12T02:56:31.034343+0000","last_became_active":"2024-09-12T02:56:15.975342+0000","last_became_peered":"2024-09-12T02:56:15.975342+0000","last_unstale":"2024-09-12T02:56:31.034343+0000","last_undegraded":"2024-09-12T02:56:31.034343+0000","last_fullsized":"2024-09-12T02:56:31.034343+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:26:23.694876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.034922+0000","last_change":"2024-09-12T02:56:13.934149+0000","last_active":"2024-09-12T02:56:31.034922+0000","last_peered":"2024-09-12T02:56:31.034922+0000","last_clean":"2024-09-12T02:56:31.034922+0000","last_became_active":"2024-09-12T02:56:13.933743+0000","last_became_peered":"2024-09-12T02:56:13.933743+0000","last_unstale":"2024-09-12T02:56:31.034922+0000","last_undegraded":"2024-09-12T02:56:31.034922+0000","last_fullsized":"2024-09-12T02:56:31.034922+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:16:17.854457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.035435+0000","last_change":"2024-09-12T02:56:15.984782+0000","last_active":"2024-09-12T02:56:31.035435+0000","last_peered":"2024-09-12T02:56:31.035435+0000","last_clean":"2024-09-12T02:56:31.035435+0000","last_became_active":"2024-09-12T02:56:15.984336+0000","last_became_peered":"2024-09-12T02:56:15.984336+0000","last_unstale":"2024-09-12T02:56:31.035435+0000","last_undegraded":"2024-09-12T02:56:31.035435+0000","last_fullsized":"2024-09-12T02:56:31.035435+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:03:39.832897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.032443+0000","last_change":"2024-09-12T02:56:15.985486+0000","last_active":"2024-09-12T02:56:31.032443+0000","last_peered":"2024-09-12T02:56:31.032443+0000","last_clean":"2024-09-12T02:56:31.032443+0000","last_became_active":"2024-09-12T02:56:15.985167+0000","last_became_peered":"2024-09-12T02:56:15.985167+0000","last_unstale":"2024-09-12T02:56:31.032443+0000","last_undegraded":"2024-09-12T02:56:31.032443+0000","last_fullsized":"2024-09-12T02:56:31.032443+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:46:20.569453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"81'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.035106+0000","last_change":"2024-09-12T02:56:13.929324+0000","last_active":"2024-09-12T02:56:31.035106+0000","last_peered":"2024-09-12T02:56:31.035106+0000","last_clean":"2024-09-12T02:56:31.035106+0000","last_became_active":"2024-09-12T02:56:13.928994+0000","last_became_peered":"2024-09-12T02:56:13.928994+0000","last_unstale":"2024-09-12T02:56:31.035106+0000","last_undegraded":"2024-09-12T02:56:31.035106+0000","last_fullsized":"2024-09-12T02:56:31.035106+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:30:27.290328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.306230+0000","last_change":"2024-09-12T02:56:15.985889+0000","last_active":"2024-09-12T02:56:31.306230+0000","last_peered":"2024-09-12T02:56:31.306230+0000","last_clean":"2024-09-12T02:56:31.306230+0000","last_became_active":"2024-09-12T02:56:15.985509+0000","last_became_peered":"2024-09-12T02:56:15.985509+0000","last_unstale":"2024-09-12T02:56:31.306230+0000","last_undegraded":"2024-09-12T02:56:31.306230+0000","last_fullsized":"2024-09-12T02:56:31.306230+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:30:43.120718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"81'12","reported_seq":35,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.035386+0000","last_change":"2024-09-12T02:56:13.934192+0000","last_active":"2024-09-12T02:56:31.035386+0000","last_peered":"2024-09-12T02:56:31.035386+0000","last_clean":"2024-09-12T02:56:31.035386+0000","last_became_active":"2024-09-12T02:56:13.933789+0000","last_became_peered":"2024-09-12T02:56:13.933789+0000","last_unstale":"2024-09-12T02:56:31.035386+0000","last_undegraded":"2024-09-12T02:56:31.035386+0000","last_fullsized":"2024-09-12T02:56:31.035386+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:32:47.771085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.032496+0000","last_change":"2024-09-12T02:56:15.983531+0000","last_active":"2024-09-12T02:56:31.032496+0000","last_peered":"2024-09-12T02:56:31.032496+0000","last_clean":"2024-09-12T02:56:31.032496+0000","last_became_active":"2024-09-12T02:56:15.983192+0000","last_became_peered":"2024-09-12T02:56:15.983192+0000","last_unstale":"2024-09-12T02:56:31.032496+0000","last_undegraded":"2024-09-12T02:56:31.032496+0000","last_fullsized":"2024-09-12T02:56:31.032496+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:16:59.923379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.034752+0000","last_change":"2024-09-12T02:56:14.075396+0000","last_active":"2024-09-12T02:56:31.034752+0000","last_peered":"2024-09-12T02:56:31.034752+0000","last_clean":"2024-09-12T02:56:31.034752+0000","last_became_active":"2024-09-12T02:56:14.075023+0000","last_became_peered":"2024-09-12T02:56:14.075023+0000","last_unstale":"2024-09-12T02:56:31.034752+0000","last_undegraded":"2024-09-12T02:56:31.034752+0000","last_fullsized":"2024-09-12T02:56:31.034752+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:41:45.760689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.034442+0000","last_change":"2024-09-12T02:56:15.987276+0000","last_active":"2024-09-12T02:56:31.034442+0000","last_peered":"2024-09-12T02:56:31.034442+0000","last_clean":"2024-09-12T02:56:31.034442+0000","last_became_active":"2024-09-12T02:56:15.986949+0000","last_became_peered":"2024-09-12T02:56:15.986949+0000","last_unstale":"2024-09-12T02:56:31.034442+0000","last_undegraded":"2024-09-12T02:56:31.034442+0000","last_fullsized":"2024-09-12T02:56:31.034442+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:49:45.627093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"81'13","reported_seq":83,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:12.914741+0000","last_change":"2024-09-12T02:56:13.932365+0000","last_active":"2024-09-12T02:57:12.914741+0000","last_peered":"2024-09-12T02:57:12.914741+0000","last_clean":"2024-09-12T02:57:12.914741+0000","last_became_active":"2024-09-12T02:56:13.932062+0000","last_became_peered":"2024-09-12T02:56:13.932062+0000","last_unstale":"2024-09-12T02:57:12.914741+0000","last_undegraded":"2024-09-12T02:57:12.914741+0000","last_fullsized":"2024-09-12T02:57:12.914741+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:17:08.451839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":48,"num_read_kb":48,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68375,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":510,"num_read_kb":324,"num_write":246,"num_write_kb":135,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":734,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":205125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":211,"ondisk_log_size":211,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547225,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":917504,"data_stored":440690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776157,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32756,"kb_used_data":1572,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712396,"statfs":{"total":95995035648,"available":95961493504,"internally_reserved":0,"allocated":1609728,"data_stored":1092774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939683,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40416,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704736,"statfs":{"total":95995035648,"available":95953649664,"internally_reserved":0,"allocated":933888,"data_stored":440115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":40434112},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168616,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":921600,"data_stored":440381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430252,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28516,"kb_used_data":1492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716636,"statfs":{"total":95995035648,"available":95965835264,"internally_reserved":0,"allocated":1527808,"data_stored":1033013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593778,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":917504,"data_stored":439832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822710,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28528,"kb_used_data":1504,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716624,"statfs":{"total":95995035648,"available":95965822976,"internally_reserved":0,"allocated":1540096,"data_stored":1032298,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953532,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32096,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713056,"statfs":{"total":95995035648,"available":95962169344,"internally_reserved":0,"allocated":925696,"data_stored":440708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182464,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":933888,"data_stored":439657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411398,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36312,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708840,"statfs":{"total":95995035648,"available":95957852160,"internally_reserved":0,"allocated":991232,"data_stored":501903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574922,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":501912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738448,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":937984,"data_stored":441564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":62919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":62910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":63296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T02:57:20.135 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-12T02:57:20.135 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-12T02:57:20.135 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-12T02:57:20.136 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph health --format=json 2024-09-12T02:57:20.395 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:57:20.516 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:57:20 smithi005 ceph-mon[31197]: from='client.14925 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:57:20.665 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:57:20 smithi148 ceph-mon[35090]: from='client.14925 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:57:20.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:57:20 smithi146 ceph-mon[35011]: from='client.14925 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:57:20.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:20.822+0000 7ff726e94640 1 -- 172.21.15.5:0/1252975494 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff720100a80 msgr2=0x7ff720100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:20.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:20.822+0000 7ff726e94640 1 --2- 172.21.15.5:0/1252975494 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff720100a80 0x7ff720100ee0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7ff70c009930 tx=0x7ff70c02a420 comp rx=0 tx=0).stop 2024-09-12T02:57:20.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:20.822+0000 7ff726e94640 1 -- 172.21.15.5:0/1252975494 shutdown_connections 2024-09-12T02:57:20.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:20.822+0000 7ff726e94640 1 --2- 172.21.15.5:0/1252975494 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff720100a80 0x7ff720100ee0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:20.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:20.822+0000 7ff726e94640 1 --2- 172.21.15.5:0/1252975494 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff720101420 0x7ff72010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:20.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:20.822+0000 7ff726e94640 1 --2- 172.21.15.5:0/1252975494 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff720106a80 0x7ff720106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:20.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:20.822+0000 7ff726e94640 1 -- 172.21.15.5:0/1252975494 >> 172.21.15.5:0/1252975494 conn(0x7ff7200fc7a0 msgr2=0x7ff7200febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:20.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:20.822+0000 7ff726e94640 1 -- 172.21.15.5:0/1252975494 shutdown_connections 2024-09-12T02:57:20.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:20.823+0000 7ff726e94640 1 -- 172.21.15.5:0/1252975494 wait complete. 2024-09-12T02:57:20.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:20.823+0000 7ff726e94640 1 Processor -- start 2024-09-12T02:57:20.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:20.823+0000 7ff726e94640 1 -- start start 2024-09-12T02:57:20.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:20.823+0000 7ff726e94640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff720100a80 0x7ff7201984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:20.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:20.823+0000 7ff726e94640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff720101420 0x7ff7201989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:20.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:20.823+0000 7ff726e94640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff720106a80 0x7ff72019cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:20.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:20.823+0000 7ff726e94640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff720110ed0 con 0x7ff720100a80 2024-09-12T02:57:20.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:20.824+0000 7ff726e94640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ff720110d50 con 0x7ff720106a80 2024-09-12T02:57:20.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:20.824+0000 7ff726e94640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff720111050 con 0x7ff720101420 2024-09-12T02:57:20.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:20.824+0000 7ff724c30640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff720100a80 0x7ff7201984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:20.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:20.824+0000 7ff724c30640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff720100a80 0x7ff7201984a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59914/0 (socket says 172.21.15.5:59914) 2024-09-12T02:57:20.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:20.824+0000 7ff724c30640 1 -- 172.21.15.5:0/2867191917 learned_addr learned my addr 172.21.15.5:0/2867191917 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:57:20.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:20.824+0000 7ff725431640 1 --2- 172.21.15.5:0/2867191917 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff720106a80 0x7ff72019cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:20.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:20.824+0000 7ff71ffff640 1 --2- 172.21.15.5:0/2867191917 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff720101420 0x7ff7201989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:20.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:20.824+0000 7ff724c30640 1 -- 172.21.15.5:0/2867191917 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff720101420 msgr2=0x7ff7201989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:20.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:20.824+0000 7ff724c30640 1 --2- 172.21.15.5:0/2867191917 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff720101420 0x7ff7201989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:20.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:20.824+0000 7ff724c30640 1 -- 172.21.15.5:0/2867191917 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff720106a80 msgr2=0x7ff72019cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:20.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:20.824+0000 7ff724c30640 1 --2- 172.21.15.5:0/2867191917 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff720106a80 0x7ff72019cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:20.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:20.824+0000 7ff724c30640 1 -- 172.21.15.5:0/2867191917 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff72019d450 con 0x7ff720100a80 2024-09-12T02:57:20.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:20.824+0000 7ff71ffff640 1 --2- 172.21.15.5:0/2867191917 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff720101420 0x7ff7201989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:57:20.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:20.824+0000 7ff724c30640 1 --2- 172.21.15.5:0/2867191917 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff720100a80 0x7ff7201984a0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7ff71400d950 tx=0x7ff71400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:20.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:20.824+0000 7ff71dffb640 1 -- 172.21.15.5:0/2867191917 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff714017030 con 0x7ff720100a80 2024-09-12T02:57:20.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:20.824+0000 7ff726e94640 1 -- 172.21.15.5:0/2867191917 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff72019d740 con 0x7ff720100a80 2024-09-12T02:57:20.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:20.824+0000 7ff71dffb640 1 -- 172.21.15.5:0/2867191917 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff714004020 con 0x7ff720100a80 2024-09-12T02:57:20.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:20.824+0000 7ff71dffb640 1 -- 172.21.15.5:0/2867191917 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7140047a0 con 0x7ff720100a80 2024-09-12T02:57:20.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:20.825+0000 7ff726e94640 1 -- 172.21.15.5:0/2867191917 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7201a5020 con 0x7ff720100a80 2024-09-12T02:57:20.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:20.825+0000 7ff726e94640 1 -- 172.21.15.5:0/2867191917 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff720100ee0 con 0x7ff720100a80 2024-09-12T02:57:20.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:20.826+0000 7ff71dffb640 1 -- 172.21.15.5:0/2867191917 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7ff7140041d0 con 0x7ff720100a80 2024-09-12T02:57:20.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:20.827+0000 7ff71dffb640 1 --2- 172.21.15.5:0/2867191917 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7ff7100a2c10 0x7ff7100a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:20.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:20.827+0000 7ff71ffff640 1 --2- 172.21.15.5:0/2867191917 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7ff7100a2c10 0x7ff7100a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:20.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:20.827+0000 7ff71dffb640 1 -- 172.21.15.5:0/2867191917 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7ff7140cec30 con 0x7ff720100a80 2024-09-12T02:57:20.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:20.828+0000 7ff71ffff640 1 --2- 172.21.15.5:0/2867191917 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7ff7100a2c10 0x7ff7100a50d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff70c02a940 tx=0x7ff70c005fd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:20.831 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:20.831+0000 7ff71dffb640 1 -- 172.21.15.5:0/2867191917 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff71409ba40 con 0x7ff720100a80 2024-09-12T02:57:21.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:21.045+0000 7ff726e94640 1 -- 172.21.15.5:0/2867191917 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7ff7201a51f0 con 0x7ff720100a80 2024-09-12T02:57:21.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:21.045+0000 7ff71dffb640 1 -- 172.21.15.5:0/2867191917 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7ff71409b340 con 0x7ff720100a80 2024-09-12T02:57:21.046 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-12T02:57:21.046 INFO:teuthology.orchestra.run.smithi005.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-12T02:57:21.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:21.047+0000 7ff726e94640 1 -- 172.21.15.5:0/2867191917 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7ff7100a2c10 msgr2=0x7ff7100a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:21.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:21.047+0000 7ff726e94640 1 --2- 172.21.15.5:0/2867191917 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7ff7100a2c10 0x7ff7100a50d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff70c02a940 tx=0x7ff70c005fd0 comp rx=0 tx=0).stop 2024-09-12T02:57:21.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:21.047+0000 7ff726e94640 1 -- 172.21.15.5:0/2867191917 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff720100a80 msgr2=0x7ff7201984a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:21.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:21.047+0000 7ff726e94640 1 --2- 172.21.15.5:0/2867191917 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff720100a80 0x7ff7201984a0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7ff71400d950 tx=0x7ff71400de10 comp rx=0 tx=0).stop 2024-09-12T02:57:21.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:21.048+0000 7ff726e94640 1 -- 172.21.15.5:0/2867191917 shutdown_connections 2024-09-12T02:57:21.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:21.048+0000 7ff726e94640 1 --2- 172.21.15.5:0/2867191917 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff720106a80 0x7ff72019cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:21.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:21.048+0000 7ff726e94640 1 --2- 172.21.15.5:0/2867191917 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff720101420 0x7ff7201989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:21.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:21.048+0000 7ff726e94640 1 --2- 172.21.15.5:0/2867191917 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7ff7100a2c10 0x7ff7100a50d0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:21.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:21.048+0000 7ff726e94640 1 --2- 172.21.15.5:0/2867191917 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff720100a80 0x7ff7201984a0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:21.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:21.048+0000 7ff726e94640 1 -- 172.21.15.5:0/2867191917 >> 172.21.15.5:0/2867191917 conn(0x7ff7200fc7a0 msgr2=0x7ff7201048d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:21.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:21.048+0000 7ff726e94640 1 -- 172.21.15.5:0/2867191917 shutdown_connections 2024-09-12T02:57:21.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:21.048+0000 7ff726e94640 1 -- 172.21.15.5:0/2867191917 wait complete. 2024-09-12T02:57:21.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:57:21 smithi005 ceph-mon[31197]: pgmap v308: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-09-12T02:57:21.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:57:21 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/2867191917' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-12T02:57:21.594 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-12T02:57:21.594 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:57:21.665 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:57:21 smithi148 ceph-mon[35090]: pgmap v308: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-09-12T02:57:21.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:57:21 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/2867191917' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-12T02:57:21.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:57:21 smithi146 ceph-mon[35011]: pgmap v308: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-09-12T02:57:21.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:57:21 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/2867191917' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-12T02:57:21.852 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:57:22.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:22.283+0000 7ff69da45640 1 -- 172.21.15.5:0/3577053254 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff698102790 msgr2=0x7ff6981067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:22.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:22.283+0000 7ff69da45640 1 --2- 172.21.15.5:0/3577053254 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff698102790 0x7ff6981067e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7ff68c009900 tx=0x7ff68c02a410 comp rx=0 tx=0).stop 2024-09-12T02:57:22.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:22.283+0000 7ff69da45640 1 -- 172.21.15.5:0/3577053254 shutdown_connections 2024-09-12T02:57:22.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:22.283+0000 7ff69da45640 1 --2- 172.21.15.5:0/3577053254 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff698102790 0x7ff6981067e0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:22.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:22.283+0000 7ff69da45640 1 --2- 172.21.15.5:0/3577053254 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff698106f10 0x7ff69810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:22.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:22.283+0000 7ff69da45640 1 --2- 172.21.15.5:0/3577053254 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff698101de0 0x7ff6981021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:22.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:22.283+0000 7ff69da45640 1 -- 172.21.15.5:0/3577053254 >> 172.21.15.5:0/3577053254 conn(0x7ff6980fd8c0 msgr2=0x7ff6980ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:22.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:22.283+0000 7ff69da45640 1 -- 172.21.15.5:0/3577053254 shutdown_connections 2024-09-12T02:57:22.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:22.283+0000 7ff69da45640 1 -- 172.21.15.5:0/3577053254 wait complete. 2024-09-12T02:57:22.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:22.284+0000 7ff69da45640 1 Processor -- start 2024-09-12T02:57:22.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:22.284+0000 7ff69da45640 1 -- start start 2024-09-12T02:57:22.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:22.284+0000 7ff69da45640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff698101de0 0x7ff69819a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:22.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:22.284+0000 7ff69da45640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff698102790 0x7ff69819abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:22.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:22.284+0000 7ff69da45640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff698106f10 0x7ff69819ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:22.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:22.284+0000 7ff69da45640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff698111f70 con 0x7ff698102790 2024-09-12T02:57:22.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:22.284+0000 7ff69da45640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ff698111df0 con 0x7ff698106f10 2024-09-12T02:57:22.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:22.284+0000 7ff69da45640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff6981120f0 con 0x7ff698101de0 2024-09-12T02:57:22.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:22.284+0000 7ff6967fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff698102790 0x7ff69819abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:22.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:22.284+0000 7ff6967fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff698102790 0x7ff69819abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59918/0 (socket says 172.21.15.5:59918) 2024-09-12T02:57:22.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:22.284+0000 7ff6967fc640 1 -- 172.21.15.5:0/2035068365 learned_addr learned my addr 172.21.15.5:0/2035068365 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:57:22.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:22.284+0000 7ff696ffd640 1 --2- 172.21.15.5:0/2035068365 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff698101de0 0x7ff69819a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:22.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:22.284+0000 7ff6977fe640 1 --2- 172.21.15.5:0/2035068365 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff698106f10 0x7ff69819ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:22.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:22.284+0000 7ff6967fc640 1 -- 172.21.15.5:0/2035068365 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff698101de0 msgr2=0x7ff69819a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:22.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:22.284+0000 7ff6967fc640 1 --2- 172.21.15.5:0/2035068365 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff698101de0 0x7ff69819a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:22.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:22.284+0000 7ff6967fc640 1 -- 172.21.15.5:0/2035068365 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff698106f10 msgr2=0x7ff69819ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:22.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:22.285+0000 7ff6967fc640 1 --2- 172.21.15.5:0/2035068365 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff698106f10 0x7ff69819ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:22.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:22.285+0000 7ff6967fc640 1 -- 172.21.15.5:0/2035068365 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff69819f640 con 0x7ff698102790 2024-09-12T02:57:22.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:22.285+0000 7ff6977fe640 1 --2- 172.21.15.5:0/2035068365 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff698106f10 0x7ff69819ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:57:22.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:22.285+0000 7ff6967fc640 1 --2- 172.21.15.5:0/2035068365 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff698102790 0x7ff69819abd0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7ff68c006010 tx=0x7ff68c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:22.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:22.285+0000 7ff69ca43640 1 -- 172.21.15.5:0/2035068365 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff68c03f030 con 0x7ff698102790 2024-09-12T02:57:22.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:22.285+0000 7ff69ca43640 1 -- 172.21.15.5:0/2035068365 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff68c003ce0 con 0x7ff698102790 2024-09-12T02:57:22.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:22.285+0000 7ff69da45640 1 -- 172.21.15.5:0/2035068365 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff69819f8d0 con 0x7ff698102790 2024-09-12T02:57:22.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:22.285+0000 7ff69ca43640 1 -- 172.21.15.5:0/2035068365 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff68c035070 con 0x7ff698102790 2024-09-12T02:57:22.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:22.285+0000 7ff69da45640 1 -- 172.21.15.5:0/2035068365 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6981a7210 con 0x7ff698102790 2024-09-12T02:57:22.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:22.286+0000 7ff69da45640 1 -- 172.21.15.5:0/2035068365 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6981021c0 con 0x7ff698102790 2024-09-12T02:57:22.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:22.287+0000 7ff69ca43640 1 -- 172.21.15.5:0/2035068365 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7ff68c042030 con 0x7ff698102790 2024-09-12T02:57:22.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:22.288+0000 7ff69ca43640 1 --2- 172.21.15.5:0/2035068365 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7ff6800a2c10 0x7ff6800a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:22.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:22.288+0000 7ff696ffd640 1 --2- 172.21.15.5:0/2035068365 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7ff6800a2c10 0x7ff6800a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:22.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:22.288+0000 7ff69ca43640 1 -- 172.21.15.5:0/2035068365 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7ff68c0efbf0 con 0x7ff698102790 2024-09-12T02:57:22.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:22.289+0000 7ff696ffd640 1 --2- 172.21.15.5:0/2035068365 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7ff6800a2c10 0x7ff6800a50d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff688005fd0 tx=0x7ff68800b590 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:22.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:22.291+0000 7ff69ca43640 1 -- 172.21.15.5:0/2035068365 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff68c0bca60 con 0x7ff698102790 2024-09-12T02:57:22.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:22.503+0000 7ff69da45640 1 -- 172.21.15.5:0/2035068365 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff6981a73e0 con 0x7ff698102790 2024-09-12T02:57:22.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:22.503+0000 7ff69ca43640 1 -- 172.21.15.5:0/2035068365 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11966 (secure 0 0 0) 0x7ff68c0bc360 con 0x7ff698102790 2024-09-12T02:57:22.504 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-12T02:57:22.504 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":25,"btime":"2024-09-12T02:56:27:840212+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/4135232521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":4135232521},{"type":"v1","addr":"172.21.15.5:6835","nonce":4135232521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/4290221940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":4290221940},{"type":"v1","addr":"172.21.15.5:6837","nonce":4290221940}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T02:56:16.595562+0000","modified":"2024-09-12T02:56:27.840207+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24502,"mds_1":24487,"mds_2":24481,"mds_3":24479,"mds_4":24458},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/26447341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":26447341},{"type":"v1","addr":"172.21.15.5:6839","nonce":26447341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2393443295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2393443295},{"type":"v1","addr":"172.21.15.5:6841","nonce":2393443295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6833/3686970137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3686970137},{"type":"v1","addr":"172.21.15.146:6833","nonce":3686970137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6833/568512209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":568512209},{"type":"v1","addr":"172.21.15.148:6833","nonce":568512209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6835/2540740420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2540740420},{"type":"v1","addr":"172.21.15.146:6835","nonce":2540740420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.146:6837/1673370937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":1673370937},{"type":"v1","addr":"172.21.15.146:6837","nonce":1673370937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":3,"incarnation":22,"state":"up:active","state_seq":6,"addr":"172.21.15.146:6839/3036767228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":3036767228},{"type":"v1","addr":"172.21.15.146:6839","nonce":3036767228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":2,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.148:6835/4195086427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":4195086427},{"type":"v1","addr":"172.21.15.148:6835","nonce":4195086427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.148:6837/2606873922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2606873922},{"type":"v1","addr":"172.21.15.148:6837","nonce":2606873922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6839/2317600000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":2317600000},{"type":"v1","addr":"172.21.15.148:6839","nonce":2317600000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24502,"qdb_cluster":[24458,24479,24481,24487,24502]},"id":1}]} 2024-09-12T02:57:22.505 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 25 2024-09-12T02:57:22.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:22.505+0000 7ff69da45640 1 -- 172.21.15.5:0/2035068365 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7ff6800a2c10 msgr2=0x7ff6800a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:22.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:22.505+0000 7ff69da45640 1 --2- 172.21.15.5:0/2035068365 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7ff6800a2c10 0x7ff6800a50d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff688005fd0 tx=0x7ff68800b590 comp rx=0 tx=0).stop 2024-09-12T02:57:22.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:22.505+0000 7ff69da45640 1 -- 172.21.15.5:0/2035068365 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff698102790 msgr2=0x7ff69819abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:22.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:22.505+0000 7ff69da45640 1 --2- 172.21.15.5:0/2035068365 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff698102790 0x7ff69819abd0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7ff68c006010 tx=0x7ff68c0042f0 comp rx=0 tx=0).stop 2024-09-12T02:57:22.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:22.505+0000 7ff69da45640 1 -- 172.21.15.5:0/2035068365 shutdown_connections 2024-09-12T02:57:22.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:22.505+0000 7ff69da45640 1 --2- 172.21.15.5:0/2035068365 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7ff6800a2c10 0x7ff6800a50d0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:22.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:22.505+0000 7ff69da45640 1 --2- 172.21.15.5:0/2035068365 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff698102790 0x7ff69819abd0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:22.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:22.505+0000 7ff69da45640 1 --2- 172.21.15.5:0/2035068365 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff698106f10 0x7ff69819ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:22.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:22.505+0000 7ff69da45640 1 --2- 172.21.15.5:0/2035068365 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff698101de0 0x7ff69819a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:22.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:22.505+0000 7ff69da45640 1 -- 172.21.15.5:0/2035068365 >> 172.21.15.5:0/2035068365 conn(0x7ff6980fd8c0 msgr2=0x7ff6980febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:22.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:22.506+0000 7ff69da45640 1 -- 172.21.15.5:0/2035068365 shutdown_connections 2024-09-12T02:57:22.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:22.506+0000 7ff69da45640 1 -- 172.21.15.5:0/2035068365 wait complete. 2024-09-12T02:57:23.196 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:57:23.465 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:57:23.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:57:23 smithi005 ceph-mon[31197]: pgmap v309: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T02:57:23.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:57:23 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/2035068365' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:57:23.665 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:57:23 smithi148 ceph-mon[35090]: pgmap v309: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T02:57:23.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:57:23 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/2035068365' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:57:23.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:57:23 smithi146 ceph-mon[35011]: pgmap v309: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T02:57:23.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:57:23 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/2035068365' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:57:23.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:23.891+0000 7f7e924d6640 1 -- 172.21.15.5:0/898132082 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e8c102ef0 msgr2=0x7f7e8c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:23.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:23.891+0000 7f7e924d6640 1 --2- 172.21.15.5:0/898132082 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e8c102ef0 0x7f7e8c10da40 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f7e84009ca0 tx=0x7f7e8402a950 comp rx=0 tx=0).stop 2024-09-12T02:57:23.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:23.891+0000 7f7e924d6640 1 -- 172.21.15.5:0/898132082 shutdown_connections 2024-09-12T02:57:23.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:23.891+0000 7f7e924d6640 1 --2- 172.21.15.5:0/898132082 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e8c102ef0 0x7f7e8c10da40 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:23.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:23.891+0000 7f7e924d6640 1 --2- 172.21.15.5:0/898132082 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7e8c102570 0x7f7e8c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:23.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:23.891+0000 7f7e924d6640 1 --2- 172.21.15.5:0/898132082 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7e8c100e20 0x7f7e8c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:23.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:23.891+0000 7f7e924d6640 1 -- 172.21.15.5:0/898132082 >> 172.21.15.5:0/898132082 conn(0x7f7e8c0fc7d0 msgr2=0x7f7e8c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:23.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:23.891+0000 7f7e924d6640 1 -- 172.21.15.5:0/898132082 shutdown_connections 2024-09-12T02:57:23.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:23.891+0000 7f7e924d6640 1 -- 172.21.15.5:0/898132082 wait complete. 2024-09-12T02:57:23.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:23.892+0000 7f7e924d6640 1 Processor -- start 2024-09-12T02:57:23.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:23.892+0000 7f7e924d6640 1 -- start start 2024-09-12T02:57:23.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:23.892+0000 7f7e924d6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7e8c100e20 0x7f7e8c19a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:23.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:23.892+0000 7f7e924d6640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7e8c102570 0x7f7e8c19ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:23.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:23.892+0000 7f7e924d6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e8c102ef0 0x7f7e8c19ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:23.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:23.892+0000 7f7e924d6640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7e8c110f80 con 0x7f7e8c102ef0 2024-09-12T02:57:23.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:23.892+0000 7f7e924d6640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f7e8c110e00 con 0x7f7e8c102570 2024-09-12T02:57:23.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:23.892+0000 7f7e924d6640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f7e8c111100 con 0x7f7e8c100e20 2024-09-12T02:57:23.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:23.892+0000 7f7e90a73640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e8c102ef0 0x7f7e8c19ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:23.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:23.892+0000 7f7e90a73640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e8c102ef0 0x7f7e8c19ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45198/0 (socket says 172.21.15.5:45198) 2024-09-12T02:57:23.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:23.892+0000 7f7e90a73640 1 -- 172.21.15.5:0/1368489931 learned_addr learned my addr 172.21.15.5:0/1368489931 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:57:23.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:23.892+0000 7f7e8bfff640 1 --2- 172.21.15.5:0/1368489931 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7e8c100e20 0x7f7e8c19a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:23.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:23.893+0000 7f7e83fff640 1 --2- 172.21.15.5:0/1368489931 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7e8c102570 0x7f7e8c19ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:23.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:23.893+0000 7f7e90a73640 1 -- 172.21.15.5:0/1368489931 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7e8c100e20 msgr2=0x7f7e8c19a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:23.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:23.893+0000 7f7e90a73640 1 --2- 172.21.15.5:0/1368489931 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7e8c100e20 0x7f7e8c19a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:23.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:23.893+0000 7f7e90a73640 1 -- 172.21.15.5:0/1368489931 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7e8c102570 msgr2=0x7f7e8c19ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:23.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:23.893+0000 7f7e90a73640 1 --2- 172.21.15.5:0/1368489931 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7e8c102570 0x7f7e8c19ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:23.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:23.893+0000 7f7e90a73640 1 -- 172.21.15.5:0/1368489931 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e8c19f570 con 0x7f7e8c102ef0 2024-09-12T02:57:23.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:23.893+0000 7f7e90a73640 1 --2- 172.21.15.5:0/1368489931 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e8c102ef0 0x7f7e8c19ee90 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f7e8402ae70 tx=0x7f7e8402f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:23.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:23.893+0000 7f7e89ffb640 1 -- 172.21.15.5:0/1368489931 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e84034040 con 0x7f7e8c102ef0 2024-09-12T02:57:23.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:23.893+0000 7f7e924d6640 1 -- 172.21.15.5:0/1368489931 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e8c19f800 con 0x7f7e8c102ef0 2024-09-12T02:57:23.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:23.893+0000 7f7e89ffb640 1 -- 172.21.15.5:0/1368489931 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e84030920 con 0x7f7e8c102ef0 2024-09-12T02:57:23.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:23.893+0000 7f7e89ffb640 1 -- 172.21.15.5:0/1368489931 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e84045480 con 0x7f7e8c102ef0 2024-09-12T02:57:23.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:23.893+0000 7f7e924d6640 1 -- 172.21.15.5:0/1368489931 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e8c1a7140 con 0x7f7e8c102ef0 2024-09-12T02:57:23.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:23.894+0000 7f7e924d6640 1 -- 172.21.15.5:0/1368489931 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e8c1021b0 con 0x7f7e8c102ef0 2024-09-12T02:57:23.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:23.895+0000 7f7e89ffb640 1 -- 172.21.15.5:0/1368489931 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f7e84037020 con 0x7f7e8c102ef0 2024-09-12T02:57:23.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:23.896+0000 7f7e89ffb640 1 --2- 172.21.15.5:0/1368489931 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f7e780a2c10 0x7f7e780a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:23.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:23.896+0000 7f7e8bfff640 1 --2- 172.21.15.5:0/1368489931 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f7e780a2c10 0x7f7e780a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:23.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:23.896+0000 7f7e89ffb640 1 -- 172.21.15.5:0/1368489931 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f7e840effc0 con 0x7f7e8c102ef0 2024-09-12T02:57:23.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:23.896+0000 7f7e8bfff640 1 --2- 172.21.15.5:0/1368489931 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f7e780a2c10 0x7f7e780a50d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f7e7c005d00 tx=0x7f7e7c00b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:23.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:23.900+0000 7f7e89ffb640 1 -- 172.21.15.5:0/1368489931 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7e840bcee0 con 0x7f7e8c102ef0 2024-09-12T02:57:24.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:24.113+0000 7f7e924d6640 1 -- 172.21.15.5:0/1368489931 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7e8c19b950 con 0x7f7e8c102ef0 2024-09-12T02:57:24.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:24.113+0000 7f7e89ffb640 1 -- 172.21.15.5:0/1368489931 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11966 (secure 0 0 0) 0x7f7e8c19b950 con 0x7f7e8c102ef0 2024-09-12T02:57:24.114 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-12T02:57:24.114 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":25,"btime":"2024-09-12T02:56:27:840212+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/4135232521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":4135232521},{"type":"v1","addr":"172.21.15.5:6835","nonce":4135232521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/4290221940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":4290221940},{"type":"v1","addr":"172.21.15.5:6837","nonce":4290221940}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T02:56:16.595562+0000","modified":"2024-09-12T02:56:27.840207+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24502,"mds_1":24487,"mds_2":24481,"mds_3":24479,"mds_4":24458},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/26447341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":26447341},{"type":"v1","addr":"172.21.15.5:6839","nonce":26447341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2393443295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2393443295},{"type":"v1","addr":"172.21.15.5:6841","nonce":2393443295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6833/3686970137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3686970137},{"type":"v1","addr":"172.21.15.146:6833","nonce":3686970137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6833/568512209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":568512209},{"type":"v1","addr":"172.21.15.148:6833","nonce":568512209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6835/2540740420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2540740420},{"type":"v1","addr":"172.21.15.146:6835","nonce":2540740420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.146:6837/1673370937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":1673370937},{"type":"v1","addr":"172.21.15.146:6837","nonce":1673370937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":3,"incarnation":22,"state":"up:active","state_seq":6,"addr":"172.21.15.146:6839/3036767228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":3036767228},{"type":"v1","addr":"172.21.15.146:6839","nonce":3036767228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":2,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.148:6835/4195086427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":4195086427},{"type":"v1","addr":"172.21.15.148:6835","nonce":4195086427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.148:6837/2606873922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2606873922},{"type":"v1","addr":"172.21.15.148:6837","nonce":2606873922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6839/2317600000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":2317600000},{"type":"v1","addr":"172.21.15.148:6839","nonce":2317600000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24502,"qdb_cluster":[24458,24479,24481,24487,24502]},"id":1}]} 2024-09-12T02:57:24.114 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 25 2024-09-12T02:57:24.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:24.115+0000 7f7e924d6640 1 -- 172.21.15.5:0/1368489931 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f7e780a2c10 msgr2=0x7f7e780a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:24.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:24.115+0000 7f7e924d6640 1 --2- 172.21.15.5:0/1368489931 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f7e780a2c10 0x7f7e780a50d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f7e7c005d00 tx=0x7f7e7c00b410 comp rx=0 tx=0).stop 2024-09-12T02:57:24.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:24.115+0000 7f7e924d6640 1 -- 172.21.15.5:0/1368489931 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e8c102ef0 msgr2=0x7f7e8c19ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:24.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:24.115+0000 7f7e924d6640 1 --2- 172.21.15.5:0/1368489931 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e8c102ef0 0x7f7e8c19ee90 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f7e8402ae70 tx=0x7f7e8402f5d0 comp rx=0 tx=0).stop 2024-09-12T02:57:24.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:24.115+0000 7f7e924d6640 1 -- 172.21.15.5:0/1368489931 shutdown_connections 2024-09-12T02:57:24.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:24.115+0000 7f7e924d6640 1 --2- 172.21.15.5:0/1368489931 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f7e780a2c10 0x7f7e780a50d0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:24.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:24.115+0000 7f7e924d6640 1 --2- 172.21.15.5:0/1368489931 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e8c102ef0 0x7f7e8c19ee90 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:24.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:24.115+0000 7f7e924d6640 1 --2- 172.21.15.5:0/1368489931 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7e8c102570 0x7f7e8c19ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:24.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:24.115+0000 7f7e924d6640 1 --2- 172.21.15.5:0/1368489931 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7e8c100e20 0x7f7e8c19a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:24.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:24.115+0000 7f7e924d6640 1 -- 172.21.15.5:0/1368489931 >> 172.21.15.5:0/1368489931 conn(0x7f7e8c0fc7d0 msgr2=0x7f7e8c069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:24.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:24.115+0000 7f7e924d6640 1 -- 172.21.15.5:0/1368489931 shutdown_connections 2024-09-12T02:57:24.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:24.116+0000 7f7e924d6640 1 -- 172.21.15.5:0/1368489931 wait complete. 2024-09-12T02:57:24.410 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:57:24 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/1368489931' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:57:24.665 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:57:24 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/1368489931' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:57:24.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:57:24 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/1368489931' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:57:24.768 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T02:57:25.030 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:57:25.408 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:57:25 smithi005 ceph-mon[31197]: pgmap v310: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-12T02:57:25.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:25.494+0000 7fe269aec640 1 -- 172.21.15.5:0/2215728961 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe264106f10 msgr2=0x7fe26410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:25.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:25.494+0000 7fe269aec640 1 --2- 172.21.15.5:0/2215728961 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe264106f10 0x7fe26410ea90 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fe25c009ca0 tx=0x7fe25c02a8c0 comp rx=0 tx=0).stop 2024-09-12T02:57:25.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:25.494+0000 7fe269aec640 1 -- 172.21.15.5:0/2215728961 shutdown_connections 2024-09-12T02:57:25.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:25.494+0000 7fe269aec640 1 --2- 172.21.15.5:0/2215728961 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe264106f10 0x7fe26410ea90 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:25.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:25.494+0000 7fe269aec640 1 --2- 172.21.15.5:0/2215728961 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe264102790 0x7fe2641067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:25.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:25.494+0000 7fe269aec640 1 --2- 172.21.15.5:0/2215728961 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe264101de0 0x7fe2641021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:25.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:25.494+0000 7fe269aec640 1 -- 172.21.15.5:0/2215728961 >> 172.21.15.5:0/2215728961 conn(0x7fe2640fd8c0 msgr2=0x7fe2640ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:25.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:25.494+0000 7fe269aec640 1 -- 172.21.15.5:0/2215728961 shutdown_connections 2024-09-12T02:57:25.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:25.494+0000 7fe269aec640 1 -- 172.21.15.5:0/2215728961 wait complete. 2024-09-12T02:57:25.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:25.494+0000 7fe269aec640 1 Processor -- start 2024-09-12T02:57:25.526 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:25.494+0000 7fe269aec640 1 -- start start 2024-09-12T02:57:25.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:25.495+0000 7fe269aec640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe264101de0 0x7fe264198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:25.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:25.495+0000 7fe269aec640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe264102790 0x7fe2641989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:25.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:25.495+0000 7fe269aec640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe264106f10 0x7fe26419cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:25.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:25.495+0000 7fe269aec640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe264111f90 con 0x7fe264106f10 2024-09-12T02:57:25.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:25.495+0000 7fe269aec640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fe264111e10 con 0x7fe264102790 2024-09-12T02:57:25.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:25.495+0000 7fe269aec640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe264112110 con 0x7fe264101de0 2024-09-12T02:57:25.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:25.495+0000 7fe263fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe264106f10 0x7fe26419cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:25.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:25.495+0000 7fe263fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe264106f10 0x7fe26419cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45216/0 (socket says 172.21.15.5:45216) 2024-09-12T02:57:25.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:25.495+0000 7fe263fff640 1 -- 172.21.15.5:0/4069044945 learned_addr learned my addr 172.21.15.5:0/4069044945 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:57:25.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:25.495+0000 7fe25bfff640 1 --2- 172.21.15.5:0/4069044945 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe264102790 0x7fe2641989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:25.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:25.495+0000 7fe2637fe640 1 --2- 172.21.15.5:0/4069044945 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe264101de0 0x7fe264198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:25.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:25.495+0000 7fe263fff640 1 -- 172.21.15.5:0/4069044945 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe264101de0 msgr2=0x7fe264198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:25.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:25.495+0000 7fe263fff640 1 --2- 172.21.15.5:0/4069044945 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe264101de0 0x7fe264198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:25.527 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:25.495+0000 7fe263fff640 1 -- 172.21.15.5:0/4069044945 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe264102790 msgr2=0x7fe2641989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:25.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:25.495+0000 7fe263fff640 1 --2- 172.21.15.5:0/4069044945 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe264102790 0x7fe2641989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:25.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:25.495+0000 7fe263fff640 1 -- 172.21.15.5:0/4069044945 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe26419d420 con 0x7fe264106f10 2024-09-12T02:57:25.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:25.495+0000 7fe263fff640 1 --2- 172.21.15.5:0/4069044945 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe264106f10 0x7fe26419cd40 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fe25c031a20 tx=0x7fe25c031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:25.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:25.495+0000 7fe2617fa640 1 -- 172.21.15.5:0/4069044945 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe25c003c30 con 0x7fe264106f10 2024-09-12T02:57:25.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:25.495+0000 7fe269aec640 1 -- 172.21.15.5:0/4069044945 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe26419d6b0 con 0x7fe264106f10 2024-09-12T02:57:25.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:25.495+0000 7fe2617fa640 1 -- 172.21.15.5:0/4069044945 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe25c003dd0 con 0x7fe264106f10 2024-09-12T02:57:25.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:25.495+0000 7fe2617fa640 1 -- 172.21.15.5:0/4069044945 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe25c02fcf0 con 0x7fe264106f10 2024-09-12T02:57:25.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:25.496+0000 7fe269aec640 1 -- 172.21.15.5:0/4069044945 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2641a4ff0 con 0x7fe264106f10 2024-09-12T02:57:25.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:25.496+0000 7fe269aec640 1 -- 172.21.15.5:0/4069044945 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2641021c0 con 0x7fe264106f10 2024-09-12T02:57:25.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:25.498+0000 7fe2617fa640 1 -- 172.21.15.5:0/4069044945 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fe25c004050 con 0x7fe264106f10 2024-09-12T02:57:25.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:25.498+0000 7fe2617fa640 1 --2- 172.21.15.5:0/4069044945 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fe2500a2c10 0x7fe2500a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:25.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:25.498+0000 7fe2637fe640 1 --2- 172.21.15.5:0/4069044945 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fe2500a2c10 0x7fe2500a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:25.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:25.498+0000 7fe2617fa640 1 -- 172.21.15.5:0/4069044945 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fe25c0f0400 con 0x7fe264106f10 2024-09-12T02:57:25.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:25.499+0000 7fe2637fe640 1 --2- 172.21.15.5:0/4069044945 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fe2500a2c10 0x7fe2500a50d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe254009890 tx=0x7fe254008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:25.529 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:25.503+0000 7fe2617fa640 1 -- 172.21.15.5:0/4069044945 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe25c0bd3b0 con 0x7fe264106f10 2024-09-12T02:57:25.665 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:57:25 smithi148 ceph-mon[35090]: pgmap v310: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-12T02:57:25.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:25.680+0000 7fe269aec640 1 -- 172.21.15.5:0/4069044945 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe26410a150 con 0x7fe264106f10 2024-09-12T02:57:25.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:25.682+0000 7fe2617fa640 1 -- 172.21.15.5:0/4069044945 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+22656 (secure 0 0 0) 0x7fe25c02fe90 con 0x7fe264106f10 2024-09-12T02:57:25.682 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-12T02:57:25.683 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":83,"fsid":"618305f8-70b1-11ef-bcea-c7b262605968","created":"2024-09-12T02:48:42.352146+0000","modified":"2024-09-12T02:56:31.026079+0000","last_up_change":"2024-09-12T02:55:27.817161+0000","last_in_change":"2024-09-12T02:55:03.745281+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T02:51:47.970639+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T02:56:12.834271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T02:56:14.951437+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17da560a-6aa2-4f2c-a95f-67481dce4d3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":4248507573},{"type":"v1","addr":"172.21.15.5:6803","nonce":4248507573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":4248507573},{"type":"v1","addr":"172.21.15.5:6805","nonce":4248507573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":4248507573},{"type":"v1","addr":"172.21.15.5:6809","nonce":4248507573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":4248507573},{"type":"v1","addr":"172.21.15.5:6807","nonce":4248507573}]},"public_addr":"172.21.15.5:6803/4248507573","cluster_addr":"172.21.15.5:6805/4248507573","heartbeat_back_addr":"172.21.15.5:6809/4248507573","heartbeat_front_addr":"172.21.15.5:6807/4248507573","state":["exists","up"]},{"osd":1,"uuid":"a3237006-99af-4fc1-860c-d4917032647f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":1105990287},{"type":"v1","addr":"172.21.15.146:6801","nonce":1105990287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":1105990287},{"type":"v1","addr":"172.21.15.146:6803","nonce":1105990287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":1105990287},{"type":"v1","addr":"172.21.15.146:6807","nonce":1105990287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":1105990287},{"type":"v1","addr":"172.21.15.146:6805","nonce":1105990287}]},"public_addr":"172.21.15.146:6801/1105990287","cluster_addr":"172.21.15.146:6803/1105990287","heartbeat_back_addr":"172.21.15.146:6807/1105990287","heartbeat_front_addr":"172.21.15.146:6805/1105990287","state":["exists","up"]},{"osd":2,"uuid":"ea3ab84a-7739-4975-8244-31ddd5aad80d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3378174313},{"type":"v1","addr":"172.21.15.148:6801","nonce":3378174313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3378174313},{"type":"v1","addr":"172.21.15.148:6803","nonce":3378174313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3378174313},{"type":"v1","addr":"172.21.15.148:6807","nonce":3378174313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3378174313},{"type":"v1","addr":"172.21.15.148:6805","nonce":3378174313}]},"public_addr":"172.21.15.148:6801/3378174313","cluster_addr":"172.21.15.148:6803/3378174313","heartbeat_back_addr":"172.21.15.148:6807/3378174313","heartbeat_front_addr":"172.21.15.148:6805/3378174313","state":["exists","up"]},{"osd":3,"uuid":"4396a9ee-f3f3-410b-9fb1-124f1e33a1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":11741938},{"type":"v1","addr":"172.21.15.5:6811","nonce":11741938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":11741938},{"type":"v1","addr":"172.21.15.5:6813","nonce":11741938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":11741938},{"type":"v1","addr":"172.21.15.5:6817","nonce":11741938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":11741938},{"type":"v1","addr":"172.21.15.5:6815","nonce":11741938}]},"public_addr":"172.21.15.5:6811/11741938","cluster_addr":"172.21.15.5:6813/11741938","heartbeat_back_addr":"172.21.15.5:6817/11741938","heartbeat_front_addr":"172.21.15.5:6815/11741938","state":["exists","up"]},{"osd":4,"uuid":"989d76e7-c742-49d0-994a-c58f2771b1d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":3274923851},{"type":"v1","addr":"172.21.15.146:6809","nonce":3274923851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":3274923851},{"type":"v1","addr":"172.21.15.146:6811","nonce":3274923851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3274923851},{"type":"v1","addr":"172.21.15.146:6815","nonce":3274923851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":3274923851},{"type":"v1","addr":"172.21.15.146:6813","nonce":3274923851}]},"public_addr":"172.21.15.146:6809/3274923851","cluster_addr":"172.21.15.146:6811/3274923851","heartbeat_back_addr":"172.21.15.146:6815/3274923851","heartbeat_front_addr":"172.21.15.146:6813/3274923851","state":["exists","up"]},{"osd":5,"uuid":"64ada7d4-70a0-4b83-9af8-032be544608d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":2771670476},{"type":"v1","addr":"172.21.15.148:6809","nonce":2771670476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":2771670476},{"type":"v1","addr":"172.21.15.148:6811","nonce":2771670476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":2771670476},{"type":"v1","addr":"172.21.15.148:6815","nonce":2771670476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":2771670476},{"type":"v1","addr":"172.21.15.148:6813","nonce":2771670476}]},"public_addr":"172.21.15.148:6809/2771670476","cluster_addr":"172.21.15.148:6811/2771670476","heartbeat_back_addr":"172.21.15.148:6815/2771670476","heartbeat_front_addr":"172.21.15.148:6813/2771670476","state":["exists","up"]},{"osd":6,"uuid":"a70a88fd-2e4b-480c-bb98-62636cf70cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3176073169},{"type":"v1","addr":"172.21.15.5:6819","nonce":3176073169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":3176073169},{"type":"v1","addr":"172.21.15.5:6821","nonce":3176073169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":3176073169},{"type":"v1","addr":"172.21.15.5:6825","nonce":3176073169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3176073169},{"type":"v1","addr":"172.21.15.5:6823","nonce":3176073169}]},"public_addr":"172.21.15.5:6819/3176073169","cluster_addr":"172.21.15.5:6821/3176073169","heartbeat_back_addr":"172.21.15.5:6825/3176073169","heartbeat_front_addr":"172.21.15.5:6823/3176073169","state":["exists","up"]},{"osd":7,"uuid":"49451e8d-c741-4ad5-b8f5-89bad8d4d129","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2209508838},{"type":"v1","addr":"172.21.15.146:6817","nonce":2209508838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2209508838},{"type":"v1","addr":"172.21.15.146:6819","nonce":2209508838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2209508838},{"type":"v1","addr":"172.21.15.146:6823","nonce":2209508838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2209508838},{"type":"v1","addr":"172.21.15.146:6821","nonce":2209508838}]},"public_addr":"172.21.15.146:6817/2209508838","cluster_addr":"172.21.15.146:6819/2209508838","heartbeat_back_addr":"172.21.15.146:6823/2209508838","heartbeat_front_addr":"172.21.15.146:6821/2209508838","state":["exists","up"]},{"osd":8,"uuid":"57bfc121-a75e-4a7f-b1dc-64074b178b03","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":651045756},{"type":"v1","addr":"172.21.15.148:6817","nonce":651045756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":651045756},{"type":"v1","addr":"172.21.15.148:6819","nonce":651045756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":651045756},{"type":"v1","addr":"172.21.15.148:6823","nonce":651045756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":651045756},{"type":"v1","addr":"172.21.15.148:6821","nonce":651045756}]},"public_addr":"172.21.15.148:6817/651045756","cluster_addr":"172.21.15.148:6819/651045756","heartbeat_back_addr":"172.21.15.148:6823/651045756","heartbeat_front_addr":"172.21.15.148:6821/651045756","state":["exists","up"]},{"osd":9,"uuid":"d9726168-7494-4bf8-b433-350e85bfd461","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1961857360},{"type":"v1","addr":"172.21.15.5:6827","nonce":1961857360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1961857360},{"type":"v1","addr":"172.21.15.5:6829","nonce":1961857360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":1961857360},{"type":"v1","addr":"172.21.15.5:6833","nonce":1961857360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1961857360},{"type":"v1","addr":"172.21.15.5:6831","nonce":1961857360}]},"public_addr":"172.21.15.5:6827/1961857360","cluster_addr":"172.21.15.5:6829/1961857360","heartbeat_back_addr":"172.21.15.5:6833/1961857360","heartbeat_front_addr":"172.21.15.5:6831/1961857360","state":["exists","up"]},{"osd":10,"uuid":"519ec60f-3606-48ac-9043-6f59a798766d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":2041733103},{"type":"v1","addr":"172.21.15.146:6825","nonce":2041733103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":2041733103},{"type":"v1","addr":"172.21.15.146:6827","nonce":2041733103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":2041733103},{"type":"v1","addr":"172.21.15.146:6831","nonce":2041733103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":2041733103},{"type":"v1","addr":"172.21.15.146:6829","nonce":2041733103}]},"public_addr":"172.21.15.146:6825/2041733103","cluster_addr":"172.21.15.146:6827/2041733103","heartbeat_back_addr":"172.21.15.146:6831/2041733103","heartbeat_front_addr":"172.21.15.146:6829/2041733103","state":["exists","up"]},{"osd":11,"uuid":"0f92d0cb-ba54-4bc3-a6d0-3c54d910556a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":839290773},{"type":"v1","addr":"172.21.15.148:6825","nonce":839290773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":839290773},{"type":"v1","addr":"172.21.15.148:6827","nonce":839290773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":839290773},{"type":"v1","addr":"172.21.15.148:6831","nonce":839290773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":839290773},{"type":"v1","addr":"172.21.15.148:6829","nonce":839290773}]},"public_addr":"172.21.15.148:6825/839290773","cluster_addr":"172.21.15.148:6827/839290773","heartbeat_back_addr":"172.21.15.148:6831/839290773","heartbeat_front_addr":"172.21.15.148:6829/839290773","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:50:54.101755+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:51:19.037385+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:51:44.142837+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:52:09.150887+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:52:33.705079+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:52:58.845233+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:53:23.068339+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:53:47.520398+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:54:11.965370+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:54:36.776075+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:55:01.292236+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:55:25.346589+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:6800/3125821798":"2024-09-13T02:49:29.462216+0000","172.21.15.5:0/3479911451":"2024-09-13T02:49:29.462216+0000","172.21.15.5:0/1547753207":"2024-09-13T02:49:03.300872+0000","172.21.15.5:0/1905762627":"2024-09-13T02:49:03.300872+0000","172.21.15.5:0/4142340280":"2024-09-13T02:49:29.462216+0000","172.21.15.5:6801/3125821798":"2024-09-13T02:49:29.462216+0000","172.21.15.5:6801/2595781680":"2024-09-13T02:49:03.300872+0000","172.21.15.5:0/2273691311":"2024-09-13T02:49:29.462216+0000","172.21.15.5:6800/2595781680":"2024-09-13T02:49:03.300872+0000","172.21.15.5:0/1196201217":"2024-09-13T02:49:03.300872+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T02:57:25.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:25.684+0000 7fe269aec640 1 -- 172.21.15.5:0/4069044945 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fe2500a2c10 msgr2=0x7fe2500a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:25.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:25.684+0000 7fe269aec640 1 --2- 172.21.15.5:0/4069044945 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fe2500a2c10 0x7fe2500a50d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe254009890 tx=0x7fe254008040 comp rx=0 tx=0).stop 2024-09-12T02:57:25.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:25.684+0000 7fe269aec640 1 -- 172.21.15.5:0/4069044945 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe264106f10 msgr2=0x7fe26419cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:25.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:25.684+0000 7fe269aec640 1 --2- 172.21.15.5:0/4069044945 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe264106f10 0x7fe26419cd40 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fe25c031a20 tx=0x7fe25c031a50 comp rx=0 tx=0).stop 2024-09-12T02:57:25.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:25.684+0000 7fe269aec640 1 -- 172.21.15.5:0/4069044945 shutdown_connections 2024-09-12T02:57:25.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:25.684+0000 7fe269aec640 1 --2- 172.21.15.5:0/4069044945 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fe2500a2c10 0x7fe2500a50d0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:25.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:25.684+0000 7fe269aec640 1 --2- 172.21.15.5:0/4069044945 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe264106f10 0x7fe26419cd40 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:25.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:25.684+0000 7fe269aec640 1 --2- 172.21.15.5:0/4069044945 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe264102790 0x7fe2641989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:25.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:25.684+0000 7fe269aec640 1 --2- 172.21.15.5:0/4069044945 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe264101de0 0x7fe264198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:25.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:25.684+0000 7fe269aec640 1 -- 172.21.15.5:0/4069044945 >> 172.21.15.5:0/4069044945 conn(0x7fe2640fd8c0 msgr2=0x7fe2640fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:25.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:25.684+0000 7fe269aec640 1 -- 172.21.15.5:0/4069044945 shutdown_connections 2024-09-12T02:57:25.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:25.684+0000 7fe269aec640 1 -- 172.21.15.5:0/4069044945 wait complete. 2024-09-12T02:57:25.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:57:25 smithi146 ceph-mon[35011]: pgmap v310: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-12T02:57:26.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:57:26 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/4069044945' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:57:26.355 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:57:26.620 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:57:26.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:57:26 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/4069044945' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:57:26.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:57:26 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/4069044945' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:57:27.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:27.098+0000 7f7a3dddd640 1 -- 172.21.15.5:0/3981924102 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a38102df0 msgr2=0x7f7a381031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:27.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:27.098+0000 7f7a3dddd640 1 --2- 172.21.15.5:0/3981924102 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a38102df0 0x7f7a381031d0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f7a28009900 tx=0x7f7a2802a410 comp rx=0 tx=0).stop 2024-09-12T02:57:27.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:27.099+0000 7f7a3dddd640 1 -- 172.21.15.5:0/3981924102 shutdown_connections 2024-09-12T02:57:27.108 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:27.099+0000 7f7a3dddd640 1 --2- 172.21.15.5:0/3981924102 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7a3810c2f0 0x7f7a3810e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:27.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:27.099+0000 7f7a3dddd640 1 --2- 172.21.15.5:0/3981924102 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7a381037a0 0x7f7a3810bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:27.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:27.099+0000 7f7a3dddd640 1 --2- 172.21.15.5:0/3981924102 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a38102df0 0x7f7a381031d0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:27.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:27.099+0000 7f7a3dddd640 1 -- 172.21.15.5:0/3981924102 >> 172.21.15.5:0/3981924102 conn(0x7f7a380fc7d0 msgr2=0x7f7a380febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:27.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:27.099+0000 7f7a3dddd640 1 -- 172.21.15.5:0/3981924102 shutdown_connections 2024-09-12T02:57:27.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:27.099+0000 7f7a3dddd640 1 -- 172.21.15.5:0/3981924102 wait complete. 2024-09-12T02:57:27.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:27.099+0000 7f7a3dddd640 1 Processor -- start 2024-09-12T02:57:27.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:27.099+0000 7f7a3dddd640 1 -- start start 2024-09-12T02:57:27.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:27.100+0000 7f7a3dddd640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7a38102df0 0x7f7a381a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:27.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:27.100+0000 7f7a3dddd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a381037a0 0x7f7a381a0b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:27.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:27.100+0000 7f7a3dddd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7a3810c2f0 0x7f7a3819a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:27.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:27.100+0000 7f7a3dddd640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7a38112150 con 0x7f7a381037a0 2024-09-12T02:57:27.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:27.100+0000 7f7a3dddd640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f7a38111fd0 con 0x7f7a38102df0 2024-09-12T02:57:27.109 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:27.100+0000 7f7a3dddd640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f7a381122d0 con 0x7f7a3810c2f0 2024-09-12T02:57:27.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:27.100+0000 7f7a36ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a381037a0 0x7f7a381a0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:27.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:27.100+0000 7f7a36ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a381037a0 0x7f7a381a0b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45236/0 (socket says 172.21.15.5:45236) 2024-09-12T02:57:27.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:27.100+0000 7f7a36ffd640 1 -- 172.21.15.5:0/6519289 learned_addr learned my addr 172.21.15.5:0/6519289 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:57:27.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:27.100+0000 7f7a377fe640 1 --2- 172.21.15.5:0/6519289 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7a38102df0 0x7f7a381a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:27.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:27.100+0000 7f7a36ffd640 1 -- 172.21.15.5:0/6519289 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7a3810c2f0 msgr2=0x7f7a3819a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:27.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:27.100+0000 7f7a36ffd640 1 --2- 172.21.15.5:0/6519289 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7a3810c2f0 0x7f7a3819a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:27.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:27.100+0000 7f7a36ffd640 1 -- 172.21.15.5:0/6519289 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7a38102df0 msgr2=0x7f7a381a05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:27.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:27.100+0000 7f7a36ffd640 1 --2- 172.21.15.5:0/6519289 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7a38102df0 0x7f7a381a05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:27.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:27.100+0000 7f7a36ffd640 1 -- 172.21.15.5:0/6519289 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a3819af30 con 0x7f7a381037a0 2024-09-12T02:57:27.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:27.100+0000 7f7a377fe640 1 --2- 172.21.15.5:0/6519289 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7a38102df0 0x7f7a381a05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:57:27.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:27.100+0000 7f7a36ffd640 1 --2- 172.21.15.5:0/6519289 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a381037a0 0x7f7a381a0b30 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c006fd0 tx=0x7f7a2c00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:27.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:27.100+0000 7f7a34ff9640 1 -- 172.21.15.5:0/6519289 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a2c011830 con 0x7f7a381037a0 2024-09-12T02:57:27.110 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:27.100+0000 7f7a34ff9640 1 -- 172.21.15.5:0/6519289 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a2c016070 con 0x7f7a381037a0 2024-09-12T02:57:27.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:27.100+0000 7f7a3dddd640 1 -- 172.21.15.5:0/6519289 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a3819b220 con 0x7f7a381037a0 2024-09-12T02:57:27.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:27.100+0000 7f7a34ff9640 1 -- 172.21.15.5:0/6519289 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a2c00f2e0 con 0x7f7a381037a0 2024-09-12T02:57:27.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:27.100+0000 7f7a3dddd640 1 -- 172.21.15.5:0/6519289 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a381a7570 con 0x7f7a381037a0 2024-09-12T02:57:27.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:27.102+0000 7f7a3dddd640 1 -- 172.21.15.5:0/6519289 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a3806a790 con 0x7f7a381037a0 2024-09-12T02:57:27.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:27.103+0000 7f7a34ff9640 1 -- 172.21.15.5:0/6519289 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f7a2c01b460 con 0x7f7a381037a0 2024-09-12T02:57:27.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:27.103+0000 7f7a34ff9640 1 --2- 172.21.15.5:0/6519289 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f7a200a2c10 0x7f7a200a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:27.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:27.103+0000 7f7a377fe640 1 --2- 172.21.15.5:0/6519289 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f7a200a2c10 0x7f7a200a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:27.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:27.103+0000 7f7a34ff9640 1 -- 172.21.15.5:0/6519289 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f7a2c0ce7f0 con 0x7f7a381037a0 2024-09-12T02:57:27.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:27.104+0000 7f7a377fe640 1 --2- 172.21.15.5:0/6519289 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f7a200a2c10 0x7f7a200a50d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7a28005ab0 tx=0x7f7a28005a20 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:27.111 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:27.107+0000 7f7a34ff9640 1 -- 172.21.15.5:0/6519289 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7a2c09b710 con 0x7f7a381037a0 2024-09-12T02:57:27.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:57:27 smithi005 ceph-mon[31197]: pgmap v311: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-12T02:57:27.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:27.320+0000 7f7a3dddd640 1 -- 172.21.15.5:0/6519289 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7a3819bbf0 con 0x7f7a381037a0 2024-09-12T02:57:27.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:27.321+0000 7f7a34ff9640 1 -- 172.21.15.5:0/6519289 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11966 (secure 0 0 0) 0x7f7a2c09b010 con 0x7f7a381037a0 2024-09-12T02:57:27.321 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-12T02:57:27.321 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":25,"btime":"2024-09-12T02:56:27:840212+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/4135232521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":4135232521},{"type":"v1","addr":"172.21.15.5:6835","nonce":4135232521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/4290221940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":4290221940},{"type":"v1","addr":"172.21.15.5:6837","nonce":4290221940}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T02:56:16.595562+0000","modified":"2024-09-12T02:56:27.840207+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24502,"mds_1":24487,"mds_2":24481,"mds_3":24479,"mds_4":24458},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/26447341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":26447341},{"type":"v1","addr":"172.21.15.5:6839","nonce":26447341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2393443295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2393443295},{"type":"v1","addr":"172.21.15.5:6841","nonce":2393443295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6833/3686970137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3686970137},{"type":"v1","addr":"172.21.15.146:6833","nonce":3686970137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6833/568512209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":568512209},{"type":"v1","addr":"172.21.15.148:6833","nonce":568512209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6835/2540740420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2540740420},{"type":"v1","addr":"172.21.15.146:6835","nonce":2540740420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.146:6837/1673370937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":1673370937},{"type":"v1","addr":"172.21.15.146:6837","nonce":1673370937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":3,"incarnation":22,"state":"up:active","state_seq":6,"addr":"172.21.15.146:6839/3036767228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":3036767228},{"type":"v1","addr":"172.21.15.146:6839","nonce":3036767228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":2,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.148:6835/4195086427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":4195086427},{"type":"v1","addr":"172.21.15.148:6835","nonce":4195086427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.148:6837/2606873922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2606873922},{"type":"v1","addr":"172.21.15.148:6837","nonce":2606873922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6839/2317600000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":2317600000},{"type":"v1","addr":"172.21.15.148:6839","nonce":2317600000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24502,"qdb_cluster":[24458,24479,24481,24487,24502]},"id":1}]} 2024-09-12T02:57:27.322 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 25 2024-09-12T02:57:27.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:27.323+0000 7f7a3dddd640 1 -- 172.21.15.5:0/6519289 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f7a200a2c10 msgr2=0x7f7a200a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:27.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:27.323+0000 7f7a3dddd640 1 --2- 172.21.15.5:0/6519289 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f7a200a2c10 0x7f7a200a50d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7a28005ab0 tx=0x7f7a28005a20 comp rx=0 tx=0).stop 2024-09-12T02:57:27.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:27.323+0000 7f7a3dddd640 1 -- 172.21.15.5:0/6519289 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a381037a0 msgr2=0x7f7a381a0b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:27.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:27.323+0000 7f7a3dddd640 1 --2- 172.21.15.5:0/6519289 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a381037a0 0x7f7a381a0b30 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c006fd0 tx=0x7f7a2c00b8e0 comp rx=0 tx=0).stop 2024-09-12T02:57:27.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:27.323+0000 7f7a3dddd640 1 -- 172.21.15.5:0/6519289 shutdown_connections 2024-09-12T02:57:27.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:27.323+0000 7f7a3dddd640 1 --2- 172.21.15.5:0/6519289 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f7a200a2c10 0x7f7a200a50d0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:27.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:27.323+0000 7f7a3dddd640 1 --2- 172.21.15.5:0/6519289 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a381037a0 0x7f7a381a0b30 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:27.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:27.323+0000 7f7a3dddd640 1 --2- 172.21.15.5:0/6519289 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7a3810c2f0 0x7f7a3819a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:27.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:27.323+0000 7f7a3dddd640 1 --2- 172.21.15.5:0/6519289 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7a38102df0 0x7f7a381a05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:27.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:27.323+0000 7f7a3dddd640 1 -- 172.21.15.5:0/6519289 >> 172.21.15.5:0/6519289 conn(0x7f7a380fc7d0 msgr2=0x7f7a380fd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:27.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:27.323+0000 7f7a3dddd640 1 -- 172.21.15.5:0/6519289 shutdown_connections 2024-09-12T02:57:27.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:27.323+0000 7f7a3dddd640 1 -- 172.21.15.5:0/6519289 wait complete. 2024-09-12T02:57:27.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:57:27 smithi148 ceph-mon[35090]: pgmap v311: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-12T02:57:27.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:57:27 smithi146 ceph-mon[35011]: pgmap v311: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-12T02:57:28.003 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-12T02:56:16.595562+0000', 'modified': '2024-09-12T02:56:27.840207+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24502, 'mds_1': 24487, 'mds_2': 24481, 'mds_3': 24479, 'mds_4': 24458}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14601': {'gid': 14601, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.5:6839/26447341', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6838', 'nonce': 26447341}, {'type': 'v1', 'addr': '172.21.15.5:6839', 'nonce': 26447341}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14613': {'gid': 14613, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.5:6841/2393443295', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6840', 'nonce': 2393443295}, {'type': 'v1', 'addr': '172.21.15.5:6841', 'nonce': 2393443295}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24407': {'gid': 24407, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.146:6833/3686970137', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.146:6832', 'nonce': 3686970137}, {'type': 'v1', 'addr': '172.21.15.146:6833', 'nonce': 3686970137}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24439': {'gid': 24439, 'name': 'l', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.148:6833/568512209', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6832', 'nonce': 568512209}, {'type': 'v1', 'addr': '172.21.15.148:6833', 'nonce': 568512209}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24440': {'gid': 24440, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.146:6835/2540740420', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.146:6834', 'nonce': 2540740420}, {'type': 'v1', 'addr': '172.21.15.146:6835', 'nonce': 2540740420}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24458': {'gid': 24458, 'name': 'h', 'rank': 4, 'incarnation': 24, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.146:6837/1673370937', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.146:6836', 'nonce': 1673370937}, {'type': 'v1', 'addr': '172.21.15.146:6837', 'nonce': 1673370937}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24479': {'gid': 24479, 'name': 'b', 'rank': 3, 'incarnation': 22, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.146:6839/3036767228', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.146:6838', 'nonce': 3036767228}, {'type': 'v1', 'addr': '172.21.15.146:6839', 'nonce': 3036767228}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24481': {'gid': 24481, 'name': 'f', 'rank': 2, 'incarnation': 20, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.148:6835/4195086427', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6834', 'nonce': 4195086427}, {'type': 'v1', 'addr': '172.21.15.148:6835', 'nonce': 4195086427}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24487': {'gid': 24487, 'name': 'c', 'rank': 1, 'incarnation': 18, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.148:6837/2606873922', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6836', 'nonce': 2606873922}, {'type': 'v1', 'addr': '172.21.15.148:6837', 'nonce': 2606873922}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24502': {'gid': 24502, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.148:6839/2317600000', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6838', 'nonce': 2317600000}, {'type': 'v1', 'addr': '172.21.15.148:6839', 'nonce': 2317600000}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24502, 'qdb_cluster': [24458, 24479, 24481, 24487, 24502]} 2024-09-12T02:57:28.003 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-12T02:57:28.003 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.h status 2024-09-12T02:57:28.268 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:57:28.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:57:28 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/6519289' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:57:28.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:57:28 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/6519289' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:57:28.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:57:28 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/6519289' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:57:28.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.734+0000 7fe0792cf640 1 -- 172.21.15.5:0/417567656 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe074102570 msgr2=0x7fe0741029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:28.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.734+0000 7fe0792cf640 1 --2- 172.21.15.5:0/417567656 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe074102570 0x7fe0741029b0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fe068009900 tx=0x7fe06802a480 comp rx=0 tx=0).stop 2024-09-12T02:57:28.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.734+0000 7fe0792cf640 1 -- 172.21.15.5:0/417567656 shutdown_connections 2024-09-12T02:57:28.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.734+0000 7fe0792cf640 1 --2- 172.21.15.5:0/417567656 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe074102570 0x7fe0741029b0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:28.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.734+0000 7fe0792cf640 1 --2- 172.21.15.5:0/417567656 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe074102ef0 0x7fe07410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:28.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.734+0000 7fe0792cf640 1 --2- 172.21.15.5:0/417567656 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe074100e20 0x7fe074101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:28.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.735+0000 7fe0792cf640 1 -- 172.21.15.5:0/417567656 >> 172.21.15.5:0/417567656 conn(0x7fe0740fc7d0 msgr2=0x7fe0740febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:28.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.735+0000 7fe0792cf640 1 -- 172.21.15.5:0/417567656 shutdown_connections 2024-09-12T02:57:28.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.735+0000 7fe0792cf640 1 -- 172.21.15.5:0/417567656 wait complete. 2024-09-12T02:57:28.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.735+0000 7fe0792cf640 1 Processor -- start 2024-09-12T02:57:28.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.735+0000 7fe0792cf640 1 -- start start 2024-09-12T02:57:28.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.736+0000 7fe0792cf640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe074100e20 0x7fe0741994b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:28.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.736+0000 7fe0792cf640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe074102570 0x7fe0741999f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:28.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.736+0000 7fe0792cf640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe074102ef0 0x7fe0741a4530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:28.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.736+0000 7fe0792cf640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe074111020 con 0x7fe074102ef0 2024-09-12T02:57:28.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.736+0000 7fe0792cf640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fe074110ea0 con 0x7fe074102570 2024-09-12T02:57:28.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.736+0000 7fe0792cf640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe0741111a0 con 0x7fe074100e20 2024-09-12T02:57:28.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.736+0000 7fe0737fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe074102ef0 0x7fe0741a4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:28.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.736+0000 7fe0737fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe074102ef0 0x7fe0741a4530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45244/0 (socket says 172.21.15.5:45244) 2024-09-12T02:57:28.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.736+0000 7fe0737fe640 1 -- 172.21.15.5:0/1538613771 learned_addr learned my addr 172.21.15.5:0/1538613771 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:57:28.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.736+0000 7fe0727fc640 1 --2- 172.21.15.5:0/1538613771 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe074102570 0x7fe0741999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:28.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.736+0000 7fe072ffd640 1 --2- 172.21.15.5:0/1538613771 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe074100e20 0x7fe0741994b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:28.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.736+0000 7fe0737fe640 1 -- 172.21.15.5:0/1538613771 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe074100e20 msgr2=0x7fe0741994b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:28.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.736+0000 7fe0737fe640 1 --2- 172.21.15.5:0/1538613771 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe074100e20 0x7fe0741994b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:28.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.736+0000 7fe0737fe640 1 -- 172.21.15.5:0/1538613771 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe074102570 msgr2=0x7fe0741999f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:28.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.736+0000 7fe0737fe640 1 --2- 172.21.15.5:0/1538613771 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe074102570 0x7fe0741999f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:28.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.736+0000 7fe0737fe640 1 -- 172.21.15.5:0/1538613771 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0741a4a70 con 0x7fe074102ef0 2024-09-12T02:57:28.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.736+0000 7fe0727fc640 1 --2- 172.21.15.5:0/1538613771 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe074102570 0x7fe0741999f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:57:28.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.736+0000 7fe0737fe640 1 --2- 172.21.15.5:0/1538613771 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe074102ef0 0x7fe0741a4530 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fe06c009fd0 tx=0x7fe06c00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:28.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.736+0000 7fe05bfff640 1 -- 172.21.15.5:0/1538613771 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe06c011830 con 0x7fe074102ef0 2024-09-12T02:57:28.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.736+0000 7fe05bfff640 1 -- 172.21.15.5:0/1538613771 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe06c016070 con 0x7fe074102ef0 2024-09-12T02:57:28.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.736+0000 7fe0792cf640 1 -- 172.21.15.5:0/1538613771 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0741a4c40 con 0x7fe074102ef0 2024-09-12T02:57:28.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.736+0000 7fe05bfff640 1 -- 172.21.15.5:0/1538613771 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe06c00f2e0 con 0x7fe074102ef0 2024-09-12T02:57:28.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.736+0000 7fe0792cf640 1 -- 172.21.15.5:0/1538613771 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0741a5180 con 0x7fe074102ef0 2024-09-12T02:57:28.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.739+0000 7fe05bfff640 1 -- 172.21.15.5:0/1538613771 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fe06c0119d0 con 0x7fe074102ef0 2024-09-12T02:57:28.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.739+0000 7fe07b6a8740 1 Processor -- start 2024-09-12T02:57:28.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.739+0000 7fe07b6a8740 1 -- start start 2024-09-12T02:57:28.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.739+0000 7fe07b6a8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560bfc7799d0 0x560bfc779db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:28.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.739+0000 7fe07b6a8740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560bfc77a2f0 0x560bfc784670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:28.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.739+0000 7fe07b6a8740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560bfc784bb0 0x560bfc786fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:28.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.739+0000 7fe07b6a8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560bfc6e0090 con 0x560bfc7799d0 2024-09-12T02:57:28.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.739+0000 7fe07b6a8740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x560bfc687a70 con 0x560bfc77a2f0 2024-09-12T02:57:28.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.739+0000 7fe07b6a8740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x560bfc6dc050 con 0x560bfc784bb0 2024-09-12T02:57:28.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.739+0000 7fe0727fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560bfc7799d0 0x560bfc779db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:28.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.739+0000 7fe0727fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560bfc7799d0 0x560bfc779db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45258/0 (socket says 172.21.15.5:45258) 2024-09-12T02:57:28.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.739+0000 7fe0727fc640 1 -- 172.21.15.5:0/526145101 learned_addr learned my addr 172.21.15.5:0/526145101 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:57:28.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.740+0000 7fe0727fc640 1 --2- 172.21.15.5:0/526145101 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560bfc784bb0 0x560bfc786fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:28.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.740+0000 7fe05bfff640 1 --2- 172.21.15.5:0/1538613771 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fe05c0a2c90 0x7fe05c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:28.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.740+0000 7fe072ffd640 1 --2- 172.21.15.5:0/526145101 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560bfc77a2f0 0x560bfc784670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:28.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.740+0000 7fe0737fe640 1 --2- 172.21.15.5:0/1538613771 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fe05c0a2c90 0x7fe05c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:28.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.740+0000 7fe0727fc640 1 -- 172.21.15.5:0/526145101 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560bfc784bb0 msgr2=0x560bfc786fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:57:28.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.740+0000 7fe0727fc640 1 --2- 172.21.15.5:0/526145101 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560bfc784bb0 0x560bfc786fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:28.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.740+0000 7fe0727fc640 1 -- 172.21.15.5:0/526145101 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560bfc77a2f0 msgr2=0x560bfc784670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:57:28.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.740+0000 7fe0727fc640 1 --2- 172.21.15.5:0/526145101 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560bfc77a2f0 0x560bfc784670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:28.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.740+0000 7fe0727fc640 1 -- 172.21.15.5:0/526145101 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560bfc694130 con 0x560bfc7799d0 2024-09-12T02:57:28.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.740+0000 7fe05bfff640 1 -- 172.21.15.5:0/1538613771 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fe06c0ce750 con 0x7fe074102ef0 2024-09-12T02:57:28.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.740+0000 7fe0727fc640 1 --2- 172.21.15.5:0/526145101 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560bfc7799d0 0x560bfc779db0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fe068005c40 tx=0x7fe068005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8152753313d149f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:28.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.740+0000 7fe0597fa640 1 -- 172.21.15.5:0/526145101 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe068035030 con 0x560bfc7799d0 2024-09-12T02:57:28.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.740+0000 7fe0737fe640 1 --2- 172.21.15.5:0/1538613771 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fe05c0a2c90 0x7fe05c0a5150 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe06c097370 tx=0x7fe06c0bb040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:28.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.740+0000 7fe0597fa640 1 -- 172.21.15.5:0/526145101 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe06803d3f0 con 0x560bfc7799d0 2024-09-12T02:57:28.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.740+0000 7fe0597fa640 1 -- 172.21.15.5:0/526145101 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe06803a600 con 0x560bfc7799d0 2024-09-12T02:57:28.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.741+0000 7fe07b6a8740 1 -- 172.21.15.5:0/526145101 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560bfc7799d0 msgr2=0x560bfc779db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:28.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.741+0000 7fe07b6a8740 1 --2- 172.21.15.5:0/526145101 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560bfc7799d0 0x560bfc779db0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fe068005c40 tx=0x7fe068005b50 comp rx=0 tx=0).stop 2024-09-12T02:57:28.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.741+0000 7fe07b6a8740 1 -- 172.21.15.5:0/526145101 shutdown_connections 2024-09-12T02:57:28.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.741+0000 7fe07b6a8740 1 --2- 172.21.15.5:0/526145101 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560bfc784bb0 0x560bfc786fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:28.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.741+0000 7fe07b6a8740 1 --2- 172.21.15.5:0/526145101 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560bfc77a2f0 0x560bfc784670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:28.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.741+0000 7fe07b6a8740 1 --2- 172.21.15.5:0/526145101 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560bfc7799d0 0x560bfc779db0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:28.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.741+0000 7fe07b6a8740 1 -- 172.21.15.5:0/526145101 >> 172.21.15.5:0/526145101 conn(0x560bfc775f60 msgr2=0x560bfc776340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:28.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.741+0000 7fe07b6a8740 1 -- 172.21.15.5:0/526145101 shutdown_connections 2024-09-12T02:57:28.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.741+0000 7fe07b6a8740 1 -- 172.21.15.5:0/526145101 wait complete. 2024-09-12T02:57:28.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.741+0000 7fe07b6a8740 1 -- start start 2024-09-12T02:57:28.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.742+0000 7fe07b6a8740 1 Processor -- start 2024-09-12T02:57:28.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.742+0000 7fe07b6a8740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560bfc7799d0 0x560bfc784460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:28.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.742+0000 7fe07b6a8740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560bfc77a2f0 0x560bfc7817b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:28.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.742+0000 7fe07b6a8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560bfc784bb0 0x560bfc781cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:28.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.742+0000 7fe07b6a8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560bfc78bb90 con 0x560bfc784bb0 2024-09-12T02:57:28.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.742+0000 7fe07b6a8740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x560bfc69af20 con 0x560bfc7799d0 2024-09-12T02:57:28.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.742+0000 7fe07b6a8740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x560bfc78bd10 con 0x560bfc77a2f0 2024-09-12T02:57:28.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.742+0000 7fe07b6a8740 10 client.0 _ll_register_callbacks cb 0x560bfc664030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:57:28.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.742+0000 7fe0727fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560bfc784bb0 0x560bfc781cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:28.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.742+0000 7fe0727fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560bfc784bb0 0x560bfc781cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45270/0 (socket says 172.21.15.5:45270) 2024-09-12T02:57:28.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.742+0000 7fe0727fc640 1 -- 172.21.15.5:0/2594903482 learned_addr learned my addr 172.21.15.5:0/2594903482 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:57:28.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.742+0000 7fe0727fc640 1 --2- 172.21.15.5:0/2594903482 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560bfc7799d0 0x560bfc784460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:28.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.742+0000 7fe072ffd640 1 --2- 172.21.15.5:0/2594903482 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560bfc77a2f0 0x560bfc7817b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:28.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.742+0000 7fe0727fc640 1 -- 172.21.15.5:0/2594903482 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560bfc77a2f0 msgr2=0x560bfc7817b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:57:28.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.742+0000 7fe0727fc640 1 --2- 172.21.15.5:0/2594903482 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560bfc77a2f0 0x560bfc7817b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:28.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.742+0000 7fe0727fc640 1 -- 172.21.15.5:0/2594903482 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560bfc7799d0 msgr2=0x560bfc784460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:57:28.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.742+0000 7fe0727fc640 1 --2- 172.21.15.5:0/2594903482 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560bfc7799d0 0x560bfc784460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:28.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.742+0000 7fe0727fc640 1 -- 172.21.15.5:0/2594903482 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560bfc4512b0 con 0x560bfc784bb0 2024-09-12T02:57:28.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.742+0000 7fe072ffd640 1 --2- 172.21.15.5:0/2594903482 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560bfc77a2f0 0x560bfc7817b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:57:28.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.742+0000 7fe0727fc640 1 --2- 172.21.15.5:0/2594903482 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560bfc784bb0 0x560bfc781cf0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fe06802a9a0 tx=0x7fe06803dd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11d3f21f9c2a6f8b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:28.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.742+0000 7fe056ff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T02:57:28.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.743+0000 7fe056ff5640 1 -- 172.21.15.5:0/2594903482 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe068053060 con 0x560bfc784bb0 2024-09-12T02:57:28.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.743+0000 7fe07b6a8740 1 -- 172.21.15.5:0/2594903482 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe06803d3f0 con 0x560bfc784bb0 2024-09-12T02:57:28.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.743+0000 7fe056ff5640 1 -- 172.21.15.5:0/2594903482 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fe06803d9b0 con 0x560bfc784bb0 2024-09-12T02:57:28.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.743+0000 7fe056ff5640 1 -- 172.21.15.5:0/2594903482 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe068035030 con 0x560bfc784bb0 2024-09-12T02:57:28.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.743+0000 7fe07b6a8740 10 client.14967 fetch_fsmap learned FSMap version 25 2024-09-12T02:57:28.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.744+0000 7fe07b6a8740 1 -- 172.21.15.5:0/2594903482 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560bfc78bb90 con 0x560bfc784bb0 2024-09-12T02:57:28.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.744+0000 7fe056ff5640 1 -- 172.21.15.5:0/2594903482 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe06803d3f0 con 0x560bfc784bb0 2024-09-12T02:57:28.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.744+0000 7fe056ff5640 1 -- 172.21.15.5:0/2594903482 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8362+0+0 (secure 0 0 0) 0x7fe06803aa40 con 0x560bfc784bb0 2024-09-12T02:57:28.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.744+0000 7fe07b6a8740 10 client.14967 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:57:28.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.744+0000 7fe07b6a8740 10 client.14967 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-12T02:57:28.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.744+0000 7fe07b6a8740 20 client.14967 populate_metadata read hostname 'smithi005' 2024-09-12T02:57:28.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.744+0000 7fe07b6a8740 1 --2- 172.21.15.5:0/2594903482 >> [v2:172.21.15.146:6836/1673370937,v1:172.21.15.146:6837/1673370937] conn(0x560bfc799590 0x560bfc79aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:28.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.744+0000 7fe07b6a8740 4 client.14967 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:57:28.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.744+0000 7fe07b6a8740 1 -- 172.21.15.5:0/2594903482 --> [v2:172.21.15.146:6836/1673370937,v1:172.21.15.146:6837/1673370937] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560bfc6ed600 con 0x560bfc799590 2024-09-12T02:57:28.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.744+0000 7fe072ffd640 1 --2- 172.21.15.5:0/2594903482 >> [v2:172.21.15.146:6836/1673370937,v1:172.21.15.146:6837/1673370937] conn(0x560bfc799590 0x560bfc79aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:28.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.745+0000 7fe056ff5640 10 client.14967 ms_handle_connect on v2:172.21.15.146:6836/1673370937 2024-09-12T02:57:28.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.745+0000 7fe072ffd640 1 --2- 172.21.15.5:0/2594903482 >> [v2:172.21.15.146:6836/1673370937,v1:172.21.15.146:6837/1673370937] conn(0x560bfc799590 0x560bfc79aa00 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3591c531140bcd69 server_cookie=5432475c910e5333 in_seq=0 out_seq=0 2024-09-12T02:57:28.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.746+0000 7fe056ff5640 1 -- 172.21.15.5:0/2594903482 <== mds.4 v2:172.21.15.146:6836/1673370937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560bfc6ed600 con 0x560bfc799590 2024-09-12T02:57:28.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.746+0000 7fe056ff5640 10 client.14967 handle_command_reply: tid=0 2024-09-12T02:57:28.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.746+0000 7fe07b6a8740 1 client.14967 shutdown 2024-09-12T02:57:28.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.746+0000 7fe07b6a8740 1 -- 172.21.15.5:0/2594903482 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560bfc784bb0 msgr2=0x560bfc781cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:28.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.746+0000 7fe07b6a8740 1 --2- 172.21.15.5:0/2594903482 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560bfc784bb0 0x560bfc781cf0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fe06802a9a0 tx=0x7fe06803dd70 comp rx=0 tx=0).stop 2024-09-12T02:57:28.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.746+0000 7fe07b6a8740 1 -- 172.21.15.5:0/2594903482 shutdown_connections 2024-09-12T02:57:28.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.746+0000 7fe07b6a8740 1 --2- 172.21.15.5:0/2594903482 >> [v2:172.21.15.146:6836/1673370937,v1:172.21.15.146:6837/1673370937] conn(0x560bfc799590 0x560bfc79aa00 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:28.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.746+0000 7fe07b6a8740 1 --2- 172.21.15.5:0/2594903482 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560bfc784bb0 0x560bfc781cf0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:28.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.746+0000 7fe07b6a8740 1 --2- 172.21.15.5:0/2594903482 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560bfc77a2f0 0x560bfc7817b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:28.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.746+0000 7fe07b6a8740 1 --2- 172.21.15.5:0/2594903482 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560bfc7799d0 0x560bfc784460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:28.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.746+0000 7fe07b6a8740 1 -- 172.21.15.5:0/2594903482 >> 172.21.15.5:0/2594903482 conn(0x560bfc775f60 msgr2=0x560bfc77bc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:28.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.746+0000 7fe056ff5640 0 client.14967 ms_handle_reset on v2:172.21.15.146:6836/1673370937 2024-09-12T02:57:28.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.746+0000 7fe07b6a8740 1 -- 172.21.15.5:0/2594903482 shutdown_connections 2024-09-12T02:57:28.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.746+0000 7fe07b6a8740 1 -- 172.21.15.5:0/2594903482 wait complete. 2024-09-12T02:57:28.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.746+0000 7fe07b6a8740 20 client.14967 trim_cache size 0 max 16384 2024-09-12T02:57:28.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.764+0000 7fe07b6a8740 1 Processor -- start 2024-09-12T02:57:28.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.764+0000 7fe07b6a8740 1 -- start start 2024-09-12T02:57:28.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.764+0000 7fe07b6a8740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560bfc7799d0 0x560bfc79aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:28.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.764+0000 7fe07b6a8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560bfc77a2f0 0x560bfc7800a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:28.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.764+0000 7fe07b6a8740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560bfc784bb0 0x560bfc77ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:28.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.764+0000 7fe07b6a8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560bfc798d40 con 0x560bfc77a2f0 2024-09-12T02:57:28.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.764+0000 7fe07b6a8740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x560bfc799410 con 0x560bfc7799d0 2024-09-12T02:57:28.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.764+0000 7fe07b6a8740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x560bfc799ae0 con 0x560bfc784bb0 2024-09-12T02:57:28.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.764+0000 7fe072ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560bfc77a2f0 0x560bfc7800a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:28.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.764+0000 7fe072ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560bfc77a2f0 0x560bfc7800a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45282/0 (socket says 172.21.15.5:45282) 2024-09-12T02:57:28.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.764+0000 7fe072ffd640 1 -- 172.21.15.5:0/579745561 learned_addr learned my addr 172.21.15.5:0/579745561 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:57:28.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.765+0000 7fe0727fc640 1 --2- 172.21.15.5:0/579745561 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560bfc7799d0 0x560bfc79aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:28.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.765+0000 7fe072ffd640 1 -- 172.21.15.5:0/579745561 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560bfc784bb0 msgr2=0x560bfc77ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:57:28.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.765+0000 7fe072ffd640 1 --2- 172.21.15.5:0/579745561 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560bfc784bb0 0x560bfc77ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:28.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.765+0000 7fe072ffd640 1 -- 172.21.15.5:0/579745561 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560bfc7799d0 msgr2=0x560bfc79aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:57:28.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.765+0000 7fe072ffd640 1 --2- 172.21.15.5:0/579745561 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560bfc7799d0 0x560bfc79aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:28.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.765+0000 7fe072ffd640 1 -- 172.21.15.5:0/579745561 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560bfc77d3c0 con 0x560bfc77a2f0 2024-09-12T02:57:28.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.765+0000 7fe072ffd640 1 --2- 172.21.15.5:0/579745561 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560bfc77a2f0 0x560bfc7800a0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fe06400a510 tx=0x7fe06400e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e80741e948d7087d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:28.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.765+0000 7fe058ff9640 1 -- 172.21.15.5:0/579745561 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe064007790 con 0x560bfc77a2f0 2024-09-12T02:57:28.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.765+0000 7fe058ff9640 1 -- 172.21.15.5:0/579745561 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe064007e10 con 0x560bfc77a2f0 2024-09-12T02:57:28.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.765+0000 7fe058ff9640 1 -- 172.21.15.5:0/579745561 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0640203d0 con 0x560bfc77a2f0 2024-09-12T02:57:28.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.766+0000 7fe07b6a8740 1 -- 172.21.15.5:0/579745561 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560bfc77a2f0 msgr2=0x560bfc7800a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:28.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.766+0000 7fe07b6a8740 1 --2- 172.21.15.5:0/579745561 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560bfc77a2f0 0x560bfc7800a0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fe06400a510 tx=0x7fe06400e040 comp rx=0 tx=0).stop 2024-09-12T02:57:28.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.766+0000 7fe07b6a8740 1 -- 172.21.15.5:0/579745561 shutdown_connections 2024-09-12T02:57:28.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.766+0000 7fe07b6a8740 1 --2- 172.21.15.5:0/579745561 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560bfc77a2f0 0x560bfc7800a0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:28.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.766+0000 7fe07b6a8740 1 --2- 172.21.15.5:0/579745561 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560bfc784bb0 0x560bfc77ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:28.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.766+0000 7fe07b6a8740 1 --2- 172.21.15.5:0/579745561 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560bfc7799d0 0x560bfc79aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:28.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.766+0000 7fe07b6a8740 1 -- 172.21.15.5:0/579745561 >> 172.21.15.5:0/579745561 conn(0x560bfc775f60 msgr2=0x560bfc770ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:28.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.766+0000 7fe07b6a8740 1 -- 172.21.15.5:0/579745561 shutdown_connections 2024-09-12T02:57:28.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.766+0000 7fe07b6a8740 1 -- 172.21.15.5:0/579745561 wait complete. 2024-09-12T02:57:28.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.766+0000 7fe07b6a8740 1 -- start start 2024-09-12T02:57:28.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.766+0000 7fe07b6a8740 1 Processor -- start 2024-09-12T02:57:28.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.766+0000 7fe07b6a8740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560bfc7799d0 0x560bfc7a35b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:28.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.766+0000 7fe07b6a8740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560bfc77a2f0 0x560bfc79e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:28.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.767+0000 7fe07b6a8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560bfc784bb0 0x560bfc79f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:28.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.767+0000 7fe07b6a8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560bfc79a880 con 0x560bfc784bb0 2024-09-12T02:57:28.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.767+0000 7fe07b6a8740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x560bfc79a1b0 con 0x560bfc77a2f0 2024-09-12T02:57:28.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.767+0000 7fe07b6a8740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x560bfc79af50 con 0x560bfc7799d0 2024-09-12T02:57:28.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.767+0000 7fe07b6a8740 10 client.0 _ll_register_callbacks cb 0x7fe0640096d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:57:28.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.767+0000 7fe0727fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560bfc784bb0 0x560bfc79f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:28.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.767+0000 7fe0727fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560bfc784bb0 0x560bfc79f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45294/0 (socket says 172.21.15.5:45294) 2024-09-12T02:57:28.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.767+0000 7fe0727fc640 1 -- 172.21.15.5:0/1310174178 learned_addr learned my addr 172.21.15.5:0/1310174178 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:57:28.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.767+0000 7fe072ffd640 1 --2- 172.21.15.5:0/1310174178 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560bfc77a2f0 0x560bfc79e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:28.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.767+0000 7fe0727fc640 1 --2- 172.21.15.5:0/1310174178 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560bfc7799d0 0x560bfc7a35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:28.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.767+0000 7fe0727fc640 1 -- 172.21.15.5:0/1310174178 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560bfc7799d0 msgr2=0x560bfc7a35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:57:28.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.767+0000 7fe0727fc640 1 --2- 172.21.15.5:0/1310174178 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560bfc7799d0 0x560bfc7a35b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:28.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.767+0000 7fe0727fc640 1 -- 172.21.15.5:0/1310174178 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560bfc77a2f0 msgr2=0x560bfc79e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:57:28.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.767+0000 7fe0727fc640 1 --2- 172.21.15.5:0/1310174178 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560bfc77a2f0 0x560bfc79e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:28.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.767+0000 7fe0727fc640 1 -- 172.21.15.5:0/1310174178 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560bfc802ca0 con 0x560bfc784bb0 2024-09-12T02:57:28.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.767+0000 7fe0727fc640 1 --2- 172.21.15.5:0/1310174178 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560bfc784bb0 0x560bfc79f020 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fe06803ade0 tx=0x7fe06803dd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb5f248eb9394042 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:28.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.767+0000 7fe056ff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T02:57:28.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.767+0000 7fe056ff5640 1 -- 172.21.15.5:0/1310174178 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe06805e030 con 0x560bfc784bb0 2024-09-12T02:57:28.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.767+0000 7fe07b6a8740 1 -- 172.21.15.5:0/1310174178 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe064007e10 con 0x560bfc784bb0 2024-09-12T02:57:28.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.767+0000 7fe056ff5640 1 -- 172.21.15.5:0/1310174178 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fe06803d9b0 con 0x560bfc784bb0 2024-09-12T02:57:28.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.768+0000 7fe056ff5640 1 -- 172.21.15.5:0/1310174178 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe06806d050 con 0x560bfc784bb0 2024-09-12T02:57:28.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.768+0000 7fe07b6a8740 10 client.14973 fetch_fsmap learned FSMap version 25 2024-09-12T02:57:28.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.768+0000 7fe07b6a8740 1 -- 172.21.15.5:0/1310174178 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560bfc803e40 con 0x560bfc784bb0 2024-09-12T02:57:28.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.768+0000 7fe056ff5640 1 -- 172.21.15.5:0/1310174178 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe064007e10 con 0x560bfc784bb0 2024-09-12T02:57:28.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.768+0000 7fe056ff5640 1 -- 172.21.15.5:0/1310174178 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8362+0+0 (secure 0 0 0) 0x7fe068064070 con 0x560bfc784bb0 2024-09-12T02:57:28.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.768+0000 7fe07b6a8740 10 client.14973 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:57:28.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.768+0000 7fe07b6a8740 10 client.14973 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-12T02:57:28.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.768+0000 7fe07b6a8740 20 client.14973 populate_metadata read hostname 'smithi005' 2024-09-12T02:57:28.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.768+0000 7fe07b6a8740 1 --2- 172.21.15.5:0/1310174178 >> [v2:172.21.15.146:6836/1673370937,v1:172.21.15.146:6837/1673370937] conn(0x560bfc8088f0 0x560bfc807600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:28.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.768+0000 7fe07b6a8740 4 client.14973 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T02:57:28.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.768+0000 7fe07b6a8740 1 -- 172.21.15.5:0/1310174178 --> [v2:172.21.15.146:6836/1673370937,v1:172.21.15.146:6837/1673370937] -- command(tid 0: {"prefix": "status"}) -- 0x560bfc6e7b10 con 0x560bfc8088f0 2024-09-12T02:57:28.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.768+0000 7fe072ffd640 1 --2- 172.21.15.5:0/1310174178 >> [v2:172.21.15.146:6836/1673370937,v1:172.21.15.146:6837/1673370937] conn(0x560bfc8088f0 0x560bfc807600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:28.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.769+0000 7fe072ffd640 1 --2- 172.21.15.5:0/1310174178 >> [v2:172.21.15.146:6836/1673370937,v1:172.21.15.146:6837/1673370937] conn(0x560bfc8088f0 0x560bfc807600 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9a609f1d915c80ec server_cookie=1abe04357714f50c in_seq=0 out_seq=0 2024-09-12T02:57:28.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.769+0000 7fe056ff5640 10 client.14973 ms_handle_connect on v2:172.21.15.146:6836/1673370937 2024-09-12T02:57:28.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.769+0000 7fe056ff5640 1 -- 172.21.15.5:0/1310174178 <== mds.4 v2:172.21.15.146:6836/1673370937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x560bfc6e7b10 con 0x560bfc8088f0 2024-09-12T02:57:28.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.769+0000 7fe056ff5640 10 client.14973 handle_command_reply: tid=0 2024-09-12T02:57:28.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.769+0000 7fe07b6a8740 1 client.14973 shutdown 2024-09-12T02:57:28.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.769+0000 7fe07b6a8740 1 -- 172.21.15.5:0/1310174178 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560bfc784bb0 msgr2=0x560bfc79f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:28.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.769+0000 7fe07b6a8740 1 --2- 172.21.15.5:0/1310174178 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560bfc784bb0 0x560bfc79f020 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fe06803ade0 tx=0x7fe06803dd70 comp rx=0 tx=0).stop 2024-09-12T02:57:28.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.769+0000 7fe07b6a8740 1 -- 172.21.15.5:0/1310174178 shutdown_connections 2024-09-12T02:57:28.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.769+0000 7fe07b6a8740 1 --2- 172.21.15.5:0/1310174178 >> [v2:172.21.15.146:6836/1673370937,v1:172.21.15.146:6837/1673370937] conn(0x560bfc8088f0 0x560bfc807600 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:28.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.769+0000 7fe07b6a8740 1 --2- 172.21.15.5:0/1310174178 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560bfc784bb0 0x560bfc79f020 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:28.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.769+0000 7fe07b6a8740 1 --2- 172.21.15.5:0/1310174178 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560bfc77a2f0 0x560bfc79e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:28.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.769+0000 7fe07b6a8740 1 --2- 172.21.15.5:0/1310174178 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560bfc7799d0 0x560bfc7a35b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:28.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.769+0000 7fe07b6a8740 1 -- 172.21.15.5:0/1310174178 >> 172.21.15.5:0/1310174178 conn(0x560bfc775f60 msgr2=0x560bfc77fc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:28.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.769+0000 7fe056ff5640 0 client.14973 ms_handle_reset on v2:172.21.15.146:6836/1673370937 2024-09-12T02:57:28.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.769+0000 7fe07b6a8740 1 -- 172.21.15.5:0/1310174178 shutdown_connections 2024-09-12T02:57:28.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.769+0000 7fe07b6a8740 1 -- 172.21.15.5:0/1310174178 wait complete. 2024-09-12T02:57:28.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.769+0000 7fe07b6a8740 20 client.14973 trim_cache size 0 max 16384 2024-09-12T02:57:28.777 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-12T02:57:28.777 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "618305f8-70b1-11ef-bcea-c7b262605968", 2024-09-12T02:57:28.777 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 4, 2024-09-12T02:57:28.777 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24458, 2024-09-12T02:57:28.777 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-09-12T02:57:28.778 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-09-12T02:57:28.778 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-09-12T02:57:28.778 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 61.749249239000001, 2024-09-12T02:57:28.778 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 25, 2024-09-12T02:57:28.778 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 83, 2024-09-12T02:57:28.778 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 81, 2024-09-12T02:57:28.778 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 93.611163052999999 2024-09-12T02:57:28.778 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-12T02:57:28.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.770+0000 7fe0792cf640 1 -- 172.21.15.5:0/1538613771 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fe05c0a2c90 msgr2=0x7fe05c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:28.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.770+0000 7fe0792cf640 1 --2- 172.21.15.5:0/1538613771 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fe05c0a2c90 0x7fe05c0a5150 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe06c097370 tx=0x7fe06c0bb040 comp rx=0 tx=0).stop 2024-09-12T02:57:28.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.770+0000 7fe0792cf640 1 -- 172.21.15.5:0/1538613771 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe074102ef0 msgr2=0x7fe0741a4530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:28.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.770+0000 7fe0792cf640 1 --2- 172.21.15.5:0/1538613771 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe074102ef0 0x7fe0741a4530 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fe06c009fd0 tx=0x7fe06c00b8e0 comp rx=0 tx=0).stop 2024-09-12T02:57:28.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.770+0000 7fe0792cf640 1 -- 172.21.15.5:0/1538613771 shutdown_connections 2024-09-12T02:57:28.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.770+0000 7fe0792cf640 1 --2- 172.21.15.5:0/1538613771 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fe05c0a2c90 0x7fe05c0a5150 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:28.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.770+0000 7fe0792cf640 1 --2- 172.21.15.5:0/1538613771 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe074102ef0 0x7fe0741a4530 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:28.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.770+0000 7fe0792cf640 1 --2- 172.21.15.5:0/1538613771 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe074102570 0x7fe0741999f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:28.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.770+0000 7fe0792cf640 1 --2- 172.21.15.5:0/1538613771 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe074100e20 0x7fe0741994b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:28.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.770+0000 7fe0792cf640 1 -- 172.21.15.5:0/1538613771 >> 172.21.15.5:0/1538613771 conn(0x7fe0740fc7d0 msgr2=0x7fe0740febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:28.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.770+0000 7fe0792cf640 1 -- 172.21.15.5:0/1538613771 shutdown_connections 2024-09-12T02:57:28.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:28.770+0000 7fe0792cf640 1 -- 172.21.15.5:0/1538613771 wait complete. 2024-09-12T02:57:29.448 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:57:29 smithi005 ceph-mon[31197]: pgmap v312: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-12T02:57:29.480 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.b status 2024-09-12T02:57:29.665 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:57:29 smithi148 ceph-mon[35090]: pgmap v312: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-12T02:57:29.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:57:29 smithi146 ceph-mon[35011]: pgmap v312: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-12T02:57:29.742 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:57:30.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.225+0000 7fb3f7efb640 1 -- 172.21.15.5:0/3219198143 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3f0101de0 msgr2=0x7fb3f01021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:30.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.225+0000 7fb3f7efb640 1 --2- 172.21.15.5:0/3219198143 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3f0101de0 0x7fb3f01021c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fb3ec009900 tx=0x7fb3ec02a410 comp rx=0 tx=0).stop 2024-09-12T02:57:30.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.225+0000 7fb3f7efb640 1 -- 172.21.15.5:0/3219198143 shutdown_connections 2024-09-12T02:57:30.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.225+0000 7fb3f7efb640 1 --2- 172.21.15.5:0/3219198143 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb3f0106f10 0x7fb3f010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:30.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.225+0000 7fb3f7efb640 1 --2- 172.21.15.5:0/3219198143 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3f0102790 0x7fb3f01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:30.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.225+0000 7fb3f7efb640 1 --2- 172.21.15.5:0/3219198143 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3f0101de0 0x7fb3f01021c0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:30.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.225+0000 7fb3f7efb640 1 -- 172.21.15.5:0/3219198143 >> 172.21.15.5:0/3219198143 conn(0x7fb3f00fd8c0 msgr2=0x7fb3f00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:30.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.225+0000 7fb3f7efb640 1 -- 172.21.15.5:0/3219198143 shutdown_connections 2024-09-12T02:57:30.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.225+0000 7fb3f7efb640 1 -- 172.21.15.5:0/3219198143 wait complete. 2024-09-12T02:57:30.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.226+0000 7fb3f7efb640 1 Processor -- start 2024-09-12T02:57:30.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.226+0000 7fb3f7efb640 1 -- start start 2024-09-12T02:57:30.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.226+0000 7fb3f7efb640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb3f0101de0 0x7fb3f01983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:30.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.226+0000 7fb3f7efb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3f0102790 0x7fb3f0198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:30.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.226+0000 7fb3f7efb640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3f0106f10 0x7fb3f019cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:30.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.226+0000 7fb3f7efb640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb3f0111f70 con 0x7fb3f0102790 2024-09-12T02:57:30.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.226+0000 7fb3f7efb640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fb3f0111df0 con 0x7fb3f0101de0 2024-09-12T02:57:30.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.226+0000 7fb3f7efb640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb3f01120f0 con 0x7fb3f0106f10 2024-09-12T02:57:30.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.227+0000 7fb3f5496640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3f0102790 0x7fb3f0198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:30.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.227+0000 7fb3f5496640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3f0102790 0x7fb3f0198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45316/0 (socket says 172.21.15.5:45316) 2024-09-12T02:57:30.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.227+0000 7fb3f5496640 1 -- 172.21.15.5:0/1159548839 learned_addr learned my addr 172.21.15.5:0/1159548839 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:57:30.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.227+0000 7fb3f6498640 1 --2- 172.21.15.5:0/1159548839 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3f0106f10 0x7fb3f019cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:30.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.227+0000 7fb3f5c97640 1 --2- 172.21.15.5:0/1159548839 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb3f0101de0 0x7fb3f01983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:30.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.227+0000 7fb3f5496640 1 -- 172.21.15.5:0/1159548839 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3f0106f10 msgr2=0x7fb3f019cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:30.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.227+0000 7fb3f5496640 1 --2- 172.21.15.5:0/1159548839 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3f0106f10 0x7fb3f019cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:30.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.227+0000 7fb3f5496640 1 -- 172.21.15.5:0/1159548839 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb3f0101de0 msgr2=0x7fb3f01983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:30.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.227+0000 7fb3f5496640 1 --2- 172.21.15.5:0/1159548839 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb3f0101de0 0x7fb3f01983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:30.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.227+0000 7fb3f5496640 1 -- 172.21.15.5:0/1159548839 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3f019d380 con 0x7fb3f0102790 2024-09-12T02:57:30.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.227+0000 7fb3f5c97640 1 --2- 172.21.15.5:0/1159548839 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb3f0101de0 0x7fb3f01983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:57:30.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.227+0000 7fb3f5496640 1 --2- 172.21.15.5:0/1159548839 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3f0102790 0x7fb3f0198910 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fb3e000d770 tx=0x7fb3e000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:30.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.227+0000 7fb3e6ffd640 1 -- 172.21.15.5:0/1159548839 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3e0017070 con 0x7fb3f0102790 2024-09-12T02:57:30.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.227+0000 7fb3e6ffd640 1 -- 172.21.15.5:0/1159548839 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3e0005500 con 0x7fb3f0102790 2024-09-12T02:57:30.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.227+0000 7fb3f7efb640 1 -- 172.21.15.5:0/1159548839 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3f019d670 con 0x7fb3f0102790 2024-09-12T02:57:30.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.227+0000 7fb3e6ffd640 1 -- 172.21.15.5:0/1159548839 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3e0004e00 con 0x7fb3f0102790 2024-09-12T02:57:30.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.227+0000 7fb3f7efb640 1 -- 172.21.15.5:0/1159548839 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3f01a4f50 con 0x7fb3f0102790 2024-09-12T02:57:30.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.230+0000 7fb3e6ffd640 1 -- 172.21.15.5:0/1159548839 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fb3e0004fa0 con 0x7fb3f0102790 2024-09-12T02:57:30.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.230+0000 7fb3fa2d4740 1 Processor -- start 2024-09-12T02:57:30.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.230+0000 7fb3fa2d4740 1 -- start start 2024-09-12T02:57:30.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.230+0000 7fb3fa2d4740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559fa0a528d0 0x559fa0a52cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:30.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.230+0000 7fb3fa2d4740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fa0a531f0 0x559fa0a61a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:30.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.230+0000 7fb3fa2d4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fa0a61fc0 0x559fa0a643b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:30.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.230+0000 7fb3fa2d4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559fa09b9090 con 0x559fa0a61fc0 2024-09-12T02:57:30.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.230+0000 7fb3fa2d4740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x559fa0960a70 con 0x559fa0a528d0 2024-09-12T02:57:30.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.230+0000 7fb3fa2d4740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x559fa09b5050 con 0x559fa0a531f0 2024-09-12T02:57:30.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.230+0000 7fb3f5496640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fa0a61fc0 0x559fa0a643b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:30.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.231+0000 7fb3f6498640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559fa0a528d0 0x559fa0a52cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:30.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.231+0000 7fb3f5496640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fa0a61fc0 0x559fa0a643b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45318/0 (socket says 172.21.15.5:45318) 2024-09-12T02:57:30.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.231+0000 7fb3f5496640 1 -- 172.21.15.5:0/3925302101 learned_addr learned my addr 172.21.15.5:0/3925302101 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:57:30.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.231+0000 7fb3f6498640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559fa0a528d0 0x559fa0a52cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.5:54668/0 (socket says 172.21.15.5:54668) 2024-09-12T02:57:30.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.231+0000 7fb3e6ffd640 1 --2- 172.21.15.5:0/1159548839 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fb3dc0a2c10 0x7fb3dc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:30.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.231+0000 7fb3f5c97640 1 --2- 172.21.15.5:0/3925302101 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fa0a531f0 0x559fa0a61a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:30.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.231+0000 7fb3f6498640 1 --2- 172.21.15.5:0/1159548839 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fb3dc0a2c10 0x7fb3dc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:30.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.231+0000 7fb3e6ffd640 1 -- 172.21.15.5:0/1159548839 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fb3e00ced90 con 0x7fb3f0102790 2024-09-12T02:57:30.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.231+0000 7fb3f5496640 1 -- 172.21.15.5:0/3925302101 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fa0a531f0 msgr2=0x559fa0a61a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:57:30.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.231+0000 7fb3f5496640 1 --2- 172.21.15.5:0/3925302101 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fa0a531f0 0x559fa0a61a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:30.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.231+0000 7fb3f5496640 1 -- 172.21.15.5:0/3925302101 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559fa0a528d0 msgr2=0x559fa0a52cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:57:30.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.231+0000 7fb3f5496640 1 --2- 172.21.15.5:0/3925302101 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559fa0a528d0 0x559fa0a52cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:30.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.231+0000 7fb3f5496640 1 -- 172.21.15.5:0/3925302101 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559fa096d130 con 0x559fa0a61fc0 2024-09-12T02:57:30.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.231+0000 7fb3f5496640 1 --2- 172.21.15.5:0/3925302101 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fa0a61fc0 0x559fa0a643b0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fb3e00a4270 tx=0x7fb3e0009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=321d7fc414e8f15c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:30.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.231+0000 7fb3f6498640 1 --2- 172.21.15.5:0/1159548839 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fb3dc0a2c10 0x7fb3dc0a50d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb3e8007fd0 tx=0x7fb3e8004ab0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:30.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.231+0000 7fb3dbfff640 1 -- 172.21.15.5:0/3925302101 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3e0020070 con 0x559fa0a61fc0 2024-09-12T02:57:30.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.231+0000 7fb3dbfff640 1 -- 172.21.15.5:0/3925302101 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3e00d0030 con 0x559fa0a61fc0 2024-09-12T02:57:30.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.231+0000 7fb3dbfff640 1 -- 172.21.15.5:0/3925302101 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3e0009cc0 con 0x559fa0a61fc0 2024-09-12T02:57:30.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.232+0000 7fb3fa2d4740 1 -- 172.21.15.5:0/3925302101 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fa0a61fc0 msgr2=0x559fa0a643b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:30.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.232+0000 7fb3fa2d4740 1 --2- 172.21.15.5:0/3925302101 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fa0a61fc0 0x559fa0a643b0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fb3e00a4270 tx=0x7fb3e0009c30 comp rx=0 tx=0).stop 2024-09-12T02:57:30.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.232+0000 7fb3fa2d4740 1 -- 172.21.15.5:0/3925302101 shutdown_connections 2024-09-12T02:57:30.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.232+0000 7fb3fa2d4740 1 --2- 172.21.15.5:0/3925302101 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fa0a61fc0 0x559fa0a643b0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:30.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.232+0000 7fb3fa2d4740 1 --2- 172.21.15.5:0/3925302101 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fa0a531f0 0x559fa0a61a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:30.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.232+0000 7fb3fa2d4740 1 --2- 172.21.15.5:0/3925302101 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559fa0a528d0 0x559fa0a52cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:30.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.232+0000 7fb3fa2d4740 1 -- 172.21.15.5:0/3925302101 >> 172.21.15.5:0/3925302101 conn(0x559fa0a4ef60 msgr2=0x559fa0a4f340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:30.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.232+0000 7fb3fa2d4740 1 -- 172.21.15.5:0/3925302101 shutdown_connections 2024-09-12T02:57:30.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.232+0000 7fb3fa2d4740 1 -- 172.21.15.5:0/3925302101 wait complete. 2024-09-12T02:57:30.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.233+0000 7fb3fa2d4740 1 -- start start 2024-09-12T02:57:30.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.233+0000 7fb3fa2d4740 1 Processor -- start 2024-09-12T02:57:30.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.233+0000 7fb3fa2d4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fa0a528d0 0x559fa0a618a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:30.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.233+0000 7fb3fa2d4740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fa0a531f0 0x559fa0a5ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:30.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.233+0000 7fb3fa2d4740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559fa0a61fc0 0x559fa0a5f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:30.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.233+0000 7fb3fa2d4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559fa0a64b90 con 0x559fa0a528d0 2024-09-12T02:57:30.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.233+0000 7fb3fa2d4740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x559fa0973f20 con 0x559fa0a61fc0 2024-09-12T02:57:30.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.233+0000 7fb3fa2d4740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x559fa0a64d10 con 0x559fa0a531f0 2024-09-12T02:57:30.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.233+0000 7fb3fa2d4740 10 client.0 _ll_register_callbacks cb 0x559fa093d030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:57:30.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.233+0000 7fb3f5c97640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fa0a528d0 0x559fa0a618a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:30.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.233+0000 7fb3f5c97640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fa0a528d0 0x559fa0a618a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45320/0 (socket says 172.21.15.5:45320) 2024-09-12T02:57:30.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.233+0000 7fb3f5c97640 1 -- 172.21.15.5:0/3161374875 learned_addr learned my addr 172.21.15.5:0/3161374875 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:57:30.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.233+0000 7fb3f5496640 1 --2- 172.21.15.5:0/3161374875 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fa0a531f0 0x559fa0a5ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:30.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.233+0000 7fb3f6498640 1 --2- 172.21.15.5:0/3161374875 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559fa0a61fc0 0x559fa0a5f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:30.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.234+0000 7fb3f5c97640 1 -- 172.21.15.5:0/3161374875 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fa0a531f0 msgr2=0x559fa0a5ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:57:30.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.234+0000 7fb3f5c97640 1 --2- 172.21.15.5:0/3161374875 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fa0a531f0 0x559fa0a5ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:30.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.234+0000 7fb3f5c97640 1 -- 172.21.15.5:0/3161374875 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559fa0a61fc0 msgr2=0x559fa0a5f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:57:30.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.234+0000 7fb3f5c97640 1 --2- 172.21.15.5:0/3161374875 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559fa0a61fc0 0x559fa0a5f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:30.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.234+0000 7fb3f5c97640 1 -- 172.21.15.5:0/3161374875 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559fa072a2b0 con 0x559fa0a528d0 2024-09-12T02:57:30.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.234+0000 7fb3f6498640 1 --2- 172.21.15.5:0/3161374875 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559fa0a61fc0 0x559fa0a5f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:57:30.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.234+0000 7fb3f5c97640 1 --2- 172.21.15.5:0/3161374875 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fa0a528d0 0x559fa0a618a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fb3ec009900 tx=0x7fb3ec0317e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3725854c240f697b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:30.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.234+0000 7fb3d97fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T02:57:30.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.234+0000 7fb3d97fa640 1 -- 172.21.15.5:0/3161374875 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3ec040070 con 0x559fa0a528d0 2024-09-12T02:57:30.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.234+0000 7fb3d97fa640 1 -- 172.21.15.5:0/3161374875 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fb3ec004250 con 0x559fa0a528d0 2024-09-12T02:57:30.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.234+0000 7fb3fa2d4740 1 -- 172.21.15.5:0/3161374875 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3e00d0030 con 0x559fa0a528d0 2024-09-12T02:57:30.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.234+0000 7fb3d97fa640 1 -- 172.21.15.5:0/3161374875 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb3ec035040 con 0x559fa0a528d0 2024-09-12T02:57:30.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.234+0000 7fb3fa2d4740 10 client.14985 fetch_fsmap learned FSMap version 25 2024-09-12T02:57:30.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.234+0000 7fb3fa2d4740 1 -- 172.21.15.5:0/3161374875 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559fa0a70540 con 0x559fa0a528d0 2024-09-12T02:57:30.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.234+0000 7fb3d97fa640 1 -- 172.21.15.5:0/3161374875 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3e00d0030 con 0x559fa0a528d0 2024-09-12T02:57:30.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.234+0000 7fb3d97fa640 1 -- 172.21.15.5:0/3161374875 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8362+0+0 (secure 0 0 0) 0x7fb3ec004a50 con 0x559fa0a528d0 2024-09-12T02:57:30.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.235+0000 7fb3fa2d4740 10 client.14985 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:57:30.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.235+0000 7fb3fa2d4740 10 client.14985 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-12T02:57:30.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.235+0000 7fb3fa2d4740 20 client.14985 populate_metadata read hostname 'smithi005' 2024-09-12T02:57:30.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.235+0000 7fb3fa2d4740 1 --2- 172.21.15.5:0/3161374875 >> [v2:172.21.15.146:6838/3036767228,v1:172.21.15.146:6839/3036767228] conn(0x559fa0a72ba0 0x559fa0a74010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:30.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.235+0000 7fb3fa2d4740 4 client.14985 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:57:30.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.235+0000 7fb3fa2d4740 1 -- 172.21.15.5:0/3161374875 --> [v2:172.21.15.146:6838/3036767228,v1:172.21.15.146:6839/3036767228] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559fa09c6600 con 0x559fa0a72ba0 2024-09-12T02:57:30.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.235+0000 7fb3f5496640 1 --2- 172.21.15.5:0/3161374875 >> [v2:172.21.15.146:6838/3036767228,v1:172.21.15.146:6839/3036767228] conn(0x559fa0a72ba0 0x559fa0a74010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:30.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.235+0000 7fb3d97fa640 10 client.14985 ms_handle_connect on v2:172.21.15.146:6838/3036767228 2024-09-12T02:57:30.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.235+0000 7fb3f5496640 1 --2- 172.21.15.5:0/3161374875 >> [v2:172.21.15.146:6838/3036767228,v1:172.21.15.146:6839/3036767228] conn(0x559fa0a72ba0 0x559fa0a74010 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=288ab3a9320816bc server_cookie=e1db6c8091f6d874 in_seq=0 out_seq=0 2024-09-12T02:57:30.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.236+0000 7fb3d97fa640 1 -- 172.21.15.5:0/3161374875 <== mds.3 v2:172.21.15.146:6838/3036767228 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559fa09c6600 con 0x559fa0a72ba0 2024-09-12T02:57:30.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.236+0000 7fb3d97fa640 10 client.14985 handle_command_reply: tid=0 2024-09-12T02:57:30.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.236+0000 7fb3fa2d4740 1 client.14985 shutdown 2024-09-12T02:57:30.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.237+0000 7fb3fa2d4740 1 -- 172.21.15.5:0/3161374875 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fa0a528d0 msgr2=0x559fa0a618a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:30.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.237+0000 7fb3fa2d4740 1 --2- 172.21.15.5:0/3161374875 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fa0a528d0 0x559fa0a618a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fb3ec009900 tx=0x7fb3ec0317e0 comp rx=0 tx=0).stop 2024-09-12T02:57:30.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.237+0000 7fb3fa2d4740 1 -- 172.21.15.5:0/3161374875 shutdown_connections 2024-09-12T02:57:30.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.237+0000 7fb3fa2d4740 1 --2- 172.21.15.5:0/3161374875 >> [v2:172.21.15.146:6838/3036767228,v1:172.21.15.146:6839/3036767228] conn(0x559fa0a72ba0 0x559fa0a74010 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:30.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.237+0000 7fb3fa2d4740 1 --2- 172.21.15.5:0/3161374875 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559fa0a61fc0 0x559fa0a5f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:30.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.237+0000 7fb3fa2d4740 1 --2- 172.21.15.5:0/3161374875 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fa0a531f0 0x559fa0a5ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:30.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.237+0000 7fb3d97fa640 0 client.14985 ms_handle_reset on v2:172.21.15.146:6838/3036767228 2024-09-12T02:57:30.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.237+0000 7fb3fa2d4740 1 --2- 172.21.15.5:0/3161374875 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fa0a528d0 0x559fa0a618a0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:30.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.237+0000 7fb3fa2d4740 1 -- 172.21.15.5:0/3161374875 >> 172.21.15.5:0/3161374875 conn(0x559fa0a4ef60 msgr2=0x559fa0a54c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:30.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.237+0000 7fb3fa2d4740 1 -- 172.21.15.5:0/3161374875 shutdown_connections 2024-09-12T02:57:30.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.237+0000 7fb3fa2d4740 1 -- 172.21.15.5:0/3161374875 wait complete. 2024-09-12T02:57:30.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.237+0000 7fb3fa2d4740 20 client.14985 trim_cache size 0 max 16384 2024-09-12T02:57:30.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.255+0000 7fb3fa2d4740 1 Processor -- start 2024-09-12T02:57:30.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.255+0000 7fb3fa2d4740 1 -- start start 2024-09-12T02:57:30.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.255+0000 7fb3fa2d4740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fa0a528d0 0x559fa0a5be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:30.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.255+0000 7fb3fa2d4740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559fa0a531f0 0x559fa0a5c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:30.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.255+0000 7fb3fa2d4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fa0a61fc0 0x559fa0a54400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:30.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.255+0000 7fb3fa2d4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559fa0a72350 con 0x559fa0a61fc0 2024-09-12T02:57:30.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.255+0000 7fb3fa2d4740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x559fa0a72a20 con 0x559fa0a531f0 2024-09-12T02:57:30.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.255+0000 7fb3fa2d4740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x559fa0a730f0 con 0x559fa0a528d0 2024-09-12T02:57:30.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.255+0000 7fb3f6498640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fa0a61fc0 0x559fa0a54400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:30.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.255+0000 7fb3f6498640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fa0a61fc0 0x559fa0a54400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45328/0 (socket says 172.21.15.5:45328) 2024-09-12T02:57:30.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.255+0000 7fb3f6498640 1 -- 172.21.15.5:0/3156856716 learned_addr learned my addr 172.21.15.5:0/3156856716 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:57:30.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.255+0000 7fb3f5496640 1 --2- 172.21.15.5:0/3156856716 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559fa0a531f0 0x559fa0a5c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:30.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.255+0000 7fb3f5c97640 1 --2- 172.21.15.5:0/3156856716 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fa0a528d0 0x559fa0a5be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:30.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.255+0000 7fb3f6498640 1 -- 172.21.15.5:0/3156856716 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fa0a528d0 msgr2=0x559fa0a5be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:57:30.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.255+0000 7fb3f6498640 1 --2- 172.21.15.5:0/3156856716 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fa0a528d0 0x559fa0a5be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:30.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.255+0000 7fb3f6498640 1 -- 172.21.15.5:0/3156856716 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559fa0a531f0 msgr2=0x559fa0a5c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:57:30.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.255+0000 7fb3f6498640 1 --2- 172.21.15.5:0/3156856716 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559fa0a531f0 0x559fa0a5c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:30.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.255+0000 7fb3f6498640 1 -- 172.21.15.5:0/3156856716 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559fa0a64d00 con 0x559fa0a61fc0 2024-09-12T02:57:30.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.255+0000 7fb3f5496640 1 --2- 172.21.15.5:0/3156856716 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559fa0a531f0 0x559fa0a5c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:57:30.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.255+0000 7fb3f5c97640 1 --2- 172.21.15.5:0/3156856716 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fa0a528d0 0x559fa0a5be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:57:30.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.255+0000 7fb3f6498640 1 --2- 172.21.15.5:0/3156856716 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fa0a61fc0 0x559fa0a54400 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fb3e8007c40 tx=0x7fb3e80074e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9eb26db12de1833 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:30.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.256+0000 7fb3db7fe640 1 -- 172.21.15.5:0/3156856716 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3e8025070 con 0x559fa0a61fc0 2024-09-12T02:57:30.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.256+0000 7fb3db7fe640 1 -- 172.21.15.5:0/3156856716 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3e8010e70 con 0x559fa0a61fc0 2024-09-12T02:57:30.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.256+0000 7fb3db7fe640 1 -- 172.21.15.5:0/3156856716 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3e80216d0 con 0x559fa0a61fc0 2024-09-12T02:57:30.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.256+0000 7fb3fa2d4740 1 -- 172.21.15.5:0/3156856716 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fa0a61fc0 msgr2=0x559fa0a54400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:30.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.256+0000 7fb3fa2d4740 1 --2- 172.21.15.5:0/3156856716 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fa0a61fc0 0x559fa0a54400 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fb3e8007c40 tx=0x7fb3e80074e0 comp rx=0 tx=0).stop 2024-09-12T02:57:30.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.256+0000 7fb3fa2d4740 1 -- 172.21.15.5:0/3156856716 shutdown_connections 2024-09-12T02:57:30.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.256+0000 7fb3fa2d4740 1 --2- 172.21.15.5:0/3156856716 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fa0a61fc0 0x559fa0a54400 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:30.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.256+0000 7fb3fa2d4740 1 --2- 172.21.15.5:0/3156856716 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559fa0a531f0 0x559fa0a5c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:30.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.256+0000 7fb3fa2d4740 1 --2- 172.21.15.5:0/3156856716 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fa0a528d0 0x559fa0a5be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:30.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.256+0000 7fb3fa2d4740 1 -- 172.21.15.5:0/3156856716 >> 172.21.15.5:0/3156856716 conn(0x559fa0a4ef60 msgr2=0x559fa0a5d4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:30.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.256+0000 7fb3fa2d4740 1 -- 172.21.15.5:0/3156856716 shutdown_connections 2024-09-12T02:57:30.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.256+0000 7fb3fa2d4740 1 -- 172.21.15.5:0/3156856716 wait complete. 2024-09-12T02:57:30.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.257+0000 7fb3fa2d4740 1 -- start start 2024-09-12T02:57:30.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.257+0000 7fb3fa2d4740 1 Processor -- start 2024-09-12T02:57:30.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.257+0000 7fb3fa2d4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fa0a528d0 0x559fa0a7c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:30.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.257+0000 7fb3fa2d4740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fa0a531f0 0x559fa0a75cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:30.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.257+0000 7fb3fa2d4740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559fa0a61fc0 0x559fa0a761f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:30.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.257+0000 7fb3fa2d4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559fa0a73e90 con 0x559fa0a528d0 2024-09-12T02:57:30.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.257+0000 7fb3fa2d4740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x559fa0a737c0 con 0x559fa0a61fc0 2024-09-12T02:57:30.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.257+0000 7fb3fa2d4740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x559fa0a74560 con 0x559fa0a531f0 2024-09-12T02:57:30.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.257+0000 7fb3fa2d4740 10 client.0 _ll_register_callbacks cb 0x7fb3ec005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:57:30.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.257+0000 7fb3f5c97640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fa0a528d0 0x559fa0a7c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:30.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.257+0000 7fb3f5c97640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fa0a528d0 0x559fa0a7c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45338/0 (socket says 172.21.15.5:45338) 2024-09-12T02:57:30.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.257+0000 7fb3f5c97640 1 -- 172.21.15.5:0/4282755022 learned_addr learned my addr 172.21.15.5:0/4282755022 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:57:30.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.257+0000 7fb3f5496640 1 --2- 172.21.15.5:0/4282755022 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fa0a531f0 0x559fa0a75cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:30.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.258+0000 7fb3f6498640 1 --2- 172.21.15.5:0/4282755022 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559fa0a61fc0 0x559fa0a761f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:30.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.258+0000 7fb3f5c97640 1 -- 172.21.15.5:0/4282755022 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fa0a531f0 msgr2=0x559fa0a75cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:57:30.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.258+0000 7fb3f5c97640 1 --2- 172.21.15.5:0/4282755022 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fa0a531f0 0x559fa0a75cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:30.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.258+0000 7fb3f5c97640 1 -- 172.21.15.5:0/4282755022 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559fa0a61fc0 msgr2=0x559fa0a761f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:57:30.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.258+0000 7fb3f5c97640 1 --2- 172.21.15.5:0/4282755022 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559fa0a61fc0 0x559fa0a761f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:30.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.258+0000 7fb3f5c97640 1 -- 172.21.15.5:0/4282755022 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559fa0a76a20 con 0x559fa0a528d0 2024-09-12T02:57:30.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.258+0000 7fb3f5496640 1 --2- 172.21.15.5:0/4282755022 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fa0a531f0 0x559fa0a75cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:57:30.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.258+0000 7fb3f6498640 1 --2- 172.21.15.5:0/4282755022 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559fa0a61fc0 0x559fa0a761f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:57:30.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.258+0000 7fb3f5c97640 1 --2- 172.21.15.5:0/4282755022 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fa0a528d0 0x559fa0a7c980 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fb3ec031a60 tx=0x7fb3ec031a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0fda2a99391dcf server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:30.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.258+0000 7fb3d97fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T02:57:30.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.258+0000 7fb3d97fa640 1 -- 172.21.15.5:0/4282755022 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3ec040070 con 0x559fa0a528d0 2024-09-12T02:57:30.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.258+0000 7fb3d97fa640 1 -- 172.21.15.5:0/4282755022 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fb3ec003c10 con 0x559fa0a528d0 2024-09-12T02:57:30.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.258+0000 7fb3fa2d4740 1 -- 172.21.15.5:0/4282755022 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3e8010e70 con 0x559fa0a528d0 2024-09-12T02:57:30.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.258+0000 7fb3d97fa640 1 -- 172.21.15.5:0/4282755022 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb3ec054070 con 0x559fa0a528d0 2024-09-12T02:57:30.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.258+0000 7fb3fa2d4740 10 client.14991 fetch_fsmap learned FSMap version 25 2024-09-12T02:57:30.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.258+0000 7fb3fa2d4740 1 -- 172.21.15.5:0/4282755022 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559fa0add2b0 con 0x559fa0a528d0 2024-09-12T02:57:30.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.258+0000 7fb3d97fa640 1 -- 172.21.15.5:0/4282755022 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3e8010e70 con 0x559fa0a528d0 2024-09-12T02:57:30.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.259+0000 7fb3d97fa640 1 -- 172.21.15.5:0/4282755022 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8362+0+0 (secure 0 0 0) 0x7fb3ec0051c0 con 0x559fa0a528d0 2024-09-12T02:57:30.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.259+0000 7fb3fa2d4740 10 client.14991 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:57:30.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.259+0000 7fb3fa2d4740 10 client.14991 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-12T02:57:30.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.259+0000 7fb3fa2d4740 20 client.14991 populate_metadata read hostname 'smithi005' 2024-09-12T02:57:30.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.259+0000 7fb3fa2d4740 1 --2- 172.21.15.5:0/4282755022 >> [v2:172.21.15.146:6838/3036767228,v1:172.21.15.146:6839/3036767228] conn(0x559fa0ae1da0 0x559fa0ae0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:30.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.259+0000 7fb3fa2d4740 4 client.14991 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T02:57:30.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.259+0000 7fb3fa2d4740 1 -- 172.21.15.5:0/4282755022 --> [v2:172.21.15.146:6838/3036767228,v1:172.21.15.146:6839/3036767228] -- command(tid 0: {"prefix": "status"}) -- 0x559fa09c0b10 con 0x559fa0ae1da0 2024-09-12T02:57:30.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.259+0000 7fb3f5496640 1 --2- 172.21.15.5:0/4282755022 >> [v2:172.21.15.146:6838/3036767228,v1:172.21.15.146:6839/3036767228] conn(0x559fa0ae1da0 0x559fa0ae0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:30.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.259+0000 7fb3f5496640 1 --2- 172.21.15.5:0/4282755022 >> [v2:172.21.15.146:6838/3036767228,v1:172.21.15.146:6839/3036767228] conn(0x559fa0ae1da0 0x559fa0ae0ab0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=db70b6de9c2c6b0b server_cookie=1e5bafadb1673317 in_seq=0 out_seq=0 2024-09-12T02:57:30.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.259+0000 7fb3d97fa640 10 client.14991 ms_handle_connect on v2:172.21.15.146:6838/3036767228 2024-09-12T02:57:30.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.260+0000 7fb3d97fa640 1 -- 172.21.15.5:0/4282755022 <== mds.3 v2:172.21.15.146:6838/3036767228 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x559fa09c0b10 con 0x559fa0ae1da0 2024-09-12T02:57:30.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.260+0000 7fb3d97fa640 10 client.14991 handle_command_reply: tid=0 2024-09-12T02:57:30.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.260+0000 7fb3fa2d4740 1 client.14991 shutdown 2024-09-12T02:57:30.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.260+0000 7fb3fa2d4740 1 -- 172.21.15.5:0/4282755022 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fa0a528d0 msgr2=0x559fa0a7c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:30.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.260+0000 7fb3fa2d4740 1 --2- 172.21.15.5:0/4282755022 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fa0a528d0 0x559fa0a7c980 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fb3ec031a60 tx=0x7fb3ec031a90 comp rx=0 tx=0).stop 2024-09-12T02:57:30.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.260+0000 7fb3fa2d4740 1 -- 172.21.15.5:0/4282755022 shutdown_connections 2024-09-12T02:57:30.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.260+0000 7fb3fa2d4740 1 --2- 172.21.15.5:0/4282755022 >> [v2:172.21.15.146:6838/3036767228,v1:172.21.15.146:6839/3036767228] conn(0x559fa0ae1da0 0x559fa0ae0ab0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:30.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.260+0000 7fb3fa2d4740 1 --2- 172.21.15.5:0/4282755022 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559fa0a61fc0 0x559fa0a761f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:30.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.260+0000 7fb3fa2d4740 1 --2- 172.21.15.5:0/4282755022 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559fa0a531f0 0x559fa0a75cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:30.266 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.260+0000 7fb3d97fa640 0 client.14991 ms_handle_reset on v2:172.21.15.146:6838/3036767228 2024-09-12T02:57:30.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.260+0000 7fb3fa2d4740 1 --2- 172.21.15.5:0/4282755022 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fa0a528d0 0x559fa0a7c980 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:30.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.260+0000 7fb3fa2d4740 1 -- 172.21.15.5:0/4282755022 >> 172.21.15.5:0/4282755022 conn(0x559fa0a4ef60 msgr2=0x559fa0a5d8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:30.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.260+0000 7fb3fa2d4740 1 -- 172.21.15.5:0/4282755022 shutdown_connections 2024-09-12T02:57:30.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.260+0000 7fb3fa2d4740 1 -- 172.21.15.5:0/4282755022 wait complete. 2024-09-12T02:57:30.267 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.260+0000 7fb3fa2d4740 20 client.14991 trim_cache size 0 max 16384 2024-09-12T02:57:30.267 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-12T02:57:30.267 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "618305f8-70b1-11ef-bcea-c7b262605968", 2024-09-12T02:57:30.267 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 3, 2024-09-12T02:57:30.267 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24479, 2024-09-12T02:57:30.267 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-09-12T02:57:30.268 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-09-12T02:57:30.268 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-09-12T02:57:30.268 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 64.248381322, 2024-09-12T02:57:30.268 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 25, 2024-09-12T02:57:30.268 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 83, 2024-09-12T02:57:30.268 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 81, 2024-09-12T02:57:30.268 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 84.157415542999999 2024-09-12T02:57:30.268 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-12T02:57:30.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.261+0000 7fb3f7efb640 1 -- 172.21.15.5:0/1159548839 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fb3dc0a2c10 msgr2=0x7fb3dc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:30.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.261+0000 7fb3f7efb640 1 --2- 172.21.15.5:0/1159548839 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fb3dc0a2c10 0x7fb3dc0a50d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb3e8007fd0 tx=0x7fb3e8004ab0 comp rx=0 tx=0).stop 2024-09-12T02:57:30.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.261+0000 7fb3f7efb640 1 -- 172.21.15.5:0/1159548839 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3f0102790 msgr2=0x7fb3f0198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:30.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.261+0000 7fb3f7efb640 1 --2- 172.21.15.5:0/1159548839 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3f0102790 0x7fb3f0198910 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fb3e000d770 tx=0x7fb3e000dc30 comp rx=0 tx=0).stop 2024-09-12T02:57:30.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.261+0000 7fb3f7efb640 1 -- 172.21.15.5:0/1159548839 shutdown_connections 2024-09-12T02:57:30.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.261+0000 7fb3f7efb640 1 --2- 172.21.15.5:0/1159548839 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fb3dc0a2c10 0x7fb3dc0a50d0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:30.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.261+0000 7fb3f7efb640 1 --2- 172.21.15.5:0/1159548839 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3f0102790 0x7fb3f0198910 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:30.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.261+0000 7fb3f7efb640 1 --2- 172.21.15.5:0/1159548839 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3f0106f10 0x7fb3f019cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:30.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.261+0000 7fb3f7efb640 1 --2- 172.21.15.5:0/1159548839 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb3f0101de0 0x7fb3f01983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:30.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.261+0000 7fb3f7efb640 1 -- 172.21.15.5:0/1159548839 >> 172.21.15.5:0/1159548839 conn(0x7fb3f00fd8c0 msgr2=0x7fb3f00febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:30.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.261+0000 7fb3f7efb640 1 -- 172.21.15.5:0/1159548839 shutdown_connections 2024-09-12T02:57:30.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:30.261+0000 7fb3f7efb640 1 -- 172.21.15.5:0/1159548839 wait complete. 2024-09-12T02:57:30.980 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.f status 2024-09-12T02:57:31.238 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:57:31.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:57:31 smithi005 ceph-mon[31197]: pgmap v313: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-12T02:57:31.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.658+0000 7f8aba53a640 1 -- 172.21.15.5:0/784345775 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ab4103660 msgr2=0x7f8ab410bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:31.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.658+0000 7f8aba53a640 1 --2- 172.21.15.5:0/784345775 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ab4103660 0x7f8ab410bb50 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c009900 tx=0x7f8a9c02a480 comp rx=0 tx=0).stop 2024-09-12T02:57:31.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.659+0000 7f8aba53a640 1 -- 172.21.15.5:0/784345775 shutdown_connections 2024-09-12T02:57:31.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.659+0000 7f8aba53a640 1 --2- 172.21.15.5:0/784345775 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ab4103660 0x7f8ab410bb50 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:31.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.659+0000 7f8aba53a640 1 --2- 172.21.15.5:0/784345775 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8ab410c170 0x7f8ab410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:31.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.659+0000 7f8aba53a640 1 --2- 172.21.15.5:0/784345775 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ab4102d40 0x7f8ab4103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:31.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.659+0000 7f8aba53a640 1 -- 172.21.15.5:0/784345775 >> 172.21.15.5:0/784345775 conn(0x7f8ab40fc7a0 msgr2=0x7f8ab40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:31.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.659+0000 7f8aba53a640 1 -- 172.21.15.5:0/784345775 shutdown_connections 2024-09-12T02:57:31.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.659+0000 7f8aba53a640 1 -- 172.21.15.5:0/784345775 wait complete. 2024-09-12T02:57:31.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.659+0000 7f8aba53a640 1 Processor -- start 2024-09-12T02:57:31.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.659+0000 7f8aba53a640 1 -- start start 2024-09-12T02:57:31.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.660+0000 7f8aba53a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ab4102d40 0x7f8ab4198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:31.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.660+0000 7f8aba53a640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8ab4103660 0x7f8ab4198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:31.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.660+0000 7f8aba53a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ab410c170 0x7f8ab419cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:31.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.660+0000 7f8aba53a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8ab4111fc0 con 0x7f8ab410c170 2024-09-12T02:57:31.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.660+0000 7f8aba53a640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f8ab4111e40 con 0x7f8ab4103660 2024-09-12T02:57:31.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.660+0000 7f8aba53a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8ab4112140 con 0x7f8ab4102d40 2024-09-12T02:57:31.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.660+0000 7f8ab8ad7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ab410c170 0x7f8ab419cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:31.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.660+0000 7f8ab8ad7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ab410c170 0x7f8ab419cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45352/0 (socket says 172.21.15.5:45352) 2024-09-12T02:57:31.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.660+0000 7f8ab8ad7640 1 -- 172.21.15.5:0/3167827906 learned_addr learned my addr 172.21.15.5:0/3167827906 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:57:31.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.660+0000 7f8ab3fff640 1 --2- 172.21.15.5:0/3167827906 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ab4102d40 0x7f8ab4198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:31.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.660+0000 7f8aabfff640 1 --2- 172.21.15.5:0/3167827906 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8ab4103660 0x7f8ab4198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:31.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.660+0000 7f8ab8ad7640 1 -- 172.21.15.5:0/3167827906 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ab4102d40 msgr2=0x7f8ab4198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:31.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.660+0000 7f8ab8ad7640 1 --2- 172.21.15.5:0/3167827906 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ab4102d40 0x7f8ab4198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:31.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.660+0000 7f8ab8ad7640 1 -- 172.21.15.5:0/3167827906 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8ab4103660 msgr2=0x7f8ab4198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:31.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.660+0000 7f8ab8ad7640 1 --2- 172.21.15.5:0/3167827906 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8ab4103660 0x7f8ab4198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:31.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.660+0000 7f8ab8ad7640 1 -- 172.21.15.5:0/3167827906 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ab419d3e0 con 0x7f8ab410c170 2024-09-12T02:57:31.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.660+0000 7f8ab8ad7640 1 --2- 172.21.15.5:0/3167827906 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ab410c170 0x7f8ab419cd00 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f8aac00eaa0 tx=0x7f8aac00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:31.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.661+0000 7f8ab1ffb640 1 -- 172.21.15.5:0/3167827906 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aac00cdb0 con 0x7f8ab410c170 2024-09-12T02:57:31.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.661+0000 7f8aba53a640 1 -- 172.21.15.5:0/3167827906 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ab419d6d0 con 0x7f8ab410c170 2024-09-12T02:57:31.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.661+0000 7f8ab1ffb640 1 -- 172.21.15.5:0/3167827906 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8aac005020 con 0x7f8ab410c170 2024-09-12T02:57:31.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.661+0000 7f8aba53a640 1 -- 172.21.15.5:0/3167827906 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ab41a4fb0 con 0x7f8ab410c170 2024-09-12T02:57:31.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.661+0000 7f8ab1ffb640 1 -- 172.21.15.5:0/3167827906 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aac0116b0 con 0x7f8ab410c170 2024-09-12T02:57:31.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.663+0000 7f8ab1ffb640 1 -- 172.21.15.5:0/3167827906 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f8aac005560 con 0x7f8ab410c170 2024-09-12T02:57:31.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.663+0000 7f8abc913740 1 Processor -- start 2024-09-12T02:57:31.668 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.663+0000 7f8abc913740 1 -- start start 2024-09-12T02:57:31.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.664+0000 7f8abc913740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5640f10079d0 0x5640f1007db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:31.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.664+0000 7f8abc913740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5640f10082f0 0x5640f1012670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:31.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.664+0000 7f8abc913740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5640f1012bb0 0x5640f1014fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:31.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.664+0000 7f8abc913740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5640f0f6e090 con 0x5640f10079d0 2024-09-12T02:57:31.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.664+0000 7f8abc913740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5640f0f15a70 con 0x5640f10082f0 2024-09-12T02:57:31.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.664+0000 7f8abc913740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5640f0f6a050 con 0x5640f1012bb0 2024-09-12T02:57:31.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.664+0000 7f8aabfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5640f10079d0 0x5640f1007db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:31.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.664+0000 7f8ab3fff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5640f10082f0 0x5640f1012670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:31.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.664+0000 7f8ab1ffb640 1 --2- 172.21.15.5:0/3167827906 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f8aa00a2c90 0x7f8aa00a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:31.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.664+0000 7f8ab3fff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5640f10082f0 0x5640f1012670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.5:54722/0 (socket says 172.21.15.5:54722) 2024-09-12T02:57:31.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.664+0000 7f8ab3fff640 1 -- 172.21.15.5:0/3248908215 learned_addr learned my addr 172.21.15.5:0/3248908215 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:57:31.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.664+0000 7f8ab8ad7640 1 --2- 172.21.15.5:0/3167827906 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f8aa00a2c90 0x7f8aa00a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:31.669 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.664+0000 7f8aabfff640 1 --2- 172.21.15.5:0/3248908215 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5640f1012bb0 0x5640f1014fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:31.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.664+0000 7f8ab1ffb640 1 -- 172.21.15.5:0/3167827906 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f8aac0cf450 con 0x7f8ab410c170 2024-09-12T02:57:31.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.664+0000 7f8aabfff640 1 -- 172.21.15.5:0/3248908215 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5640f1012bb0 msgr2=0x5640f1014fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:57:31.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.664+0000 7f8aabfff640 1 --2- 172.21.15.5:0/3248908215 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5640f1012bb0 0x5640f1014fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:31.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.664+0000 7f8aabfff640 1 -- 172.21.15.5:0/3248908215 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5640f10082f0 msgr2=0x5640f1012670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:57:31.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.664+0000 7f8aabfff640 1 --2- 172.21.15.5:0/3248908215 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5640f10082f0 0x5640f1012670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:31.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.664+0000 7f8aabfff640 1 -- 172.21.15.5:0/3248908215 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640f0f22130 con 0x5640f10079d0 2024-09-12T02:57:31.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.664+0000 7f8ab3fff640 1 --2- 172.21.15.5:0/3248908215 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5640f10082f0 0x5640f1012670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:57:31.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.665+0000 7f8aabfff640 1 --2- 172.21.15.5:0/3248908215 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5640f10079d0 0x5640f1007db0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c02aa70 tx=0x7f8a9c009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7fee8a820c04422 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:31.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.665+0000 7f8ab8ad7640 1 --2- 172.21.15.5:0/3167827906 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f8aa00a2c90 0x7f8aa00a5150 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8aac0984d0 tx=0x7f8aac06db10 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:31.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.665+0000 7f8aaa7fc640 1 -- 172.21.15.5:0/3248908215 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a9c035030 con 0x5640f10079d0 2024-09-12T02:57:31.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.665+0000 7f8aaa7fc640 1 -- 172.21.15.5:0/3248908215 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a9c005520 con 0x5640f10079d0 2024-09-12T02:57:31.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.665+0000 7f8aaa7fc640 1 -- 172.21.15.5:0/3248908215 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a9c0035f0 con 0x5640f10079d0 2024-09-12T02:57:31.670 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.665+0000 7f8abc913740 1 -- 172.21.15.5:0/3248908215 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5640f10079d0 msgr2=0x5640f1007db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:31.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.665+0000 7f8abc913740 1 --2- 172.21.15.5:0/3248908215 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5640f10079d0 0x5640f1007db0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c02aa70 tx=0x7f8a9c009560 comp rx=0 tx=0).stop 2024-09-12T02:57:31.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.666+0000 7f8abc913740 1 -- 172.21.15.5:0/3248908215 shutdown_connections 2024-09-12T02:57:31.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.666+0000 7f8abc913740 1 --2- 172.21.15.5:0/3248908215 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5640f1012bb0 0x5640f1014fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:31.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.666+0000 7f8abc913740 1 --2- 172.21.15.5:0/3248908215 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5640f10082f0 0x5640f1012670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:31.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.666+0000 7f8abc913740 1 --2- 172.21.15.5:0/3248908215 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5640f10079d0 0x5640f1007db0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:31.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.666+0000 7f8abc913740 1 -- 172.21.15.5:0/3248908215 >> 172.21.15.5:0/3248908215 conn(0x5640f1003f60 msgr2=0x5640f1004340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:31.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.666+0000 7f8abc913740 1 -- 172.21.15.5:0/3248908215 shutdown_connections 2024-09-12T02:57:31.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.666+0000 7f8abc913740 1 -- 172.21.15.5:0/3248908215 wait complete. 2024-09-12T02:57:31.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.666+0000 7f8abc913740 1 -- start start 2024-09-12T02:57:31.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.666+0000 7f8abc913740 1 Processor -- start 2024-09-12T02:57:31.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.667+0000 7f8abc913740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5640f10079d0 0x5640f10124d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:31.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.667+0000 7f8abc913740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5640f10082f0 0x5640f100f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:31.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.667+0000 7f8abc913740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5640f1012bb0 0x5640f100fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:31.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.667+0000 7f8abc913740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5640f1015780 con 0x5640f10082f0 2024-09-12T02:57:31.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.667+0000 7f8abc913740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5640f0f28f20 con 0x5640f1012bb0 2024-09-12T02:57:31.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.667+0000 7f8abc913740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5640f1015900 con 0x5640f10079d0 2024-09-12T02:57:31.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.667+0000 7f8abc913740 10 client.0 _ll_register_callbacks cb 0x5640f0ef2030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:57:31.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.667+0000 7f8ab3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5640f10082f0 0x5640f100f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:31.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.667+0000 7f8ab3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5640f10082f0 0x5640f100f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45378/0 (socket says 172.21.15.5:45378) 2024-09-12T02:57:31.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.667+0000 7f8ab3fff640 1 -- 172.21.15.5:0/522022046 learned_addr learned my addr 172.21.15.5:0/522022046 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:57:31.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.667+0000 7f8aabfff640 1 --2- 172.21.15.5:0/522022046 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5640f1012bb0 0x5640f100fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:31.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.667+0000 7f8aabfff640 1 --2- 172.21.15.5:0/522022046 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5640f10079d0 0x5640f10124d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:31.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.667+0000 7f8ab3fff640 1 -- 172.21.15.5:0/522022046 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5640f10079d0 msgr2=0x5640f10124d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:57:31.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.667+0000 7f8ab3fff640 1 --2- 172.21.15.5:0/522022046 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5640f10079d0 0x5640f10124d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:31.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.667+0000 7f8ab3fff640 1 -- 172.21.15.5:0/522022046 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5640f1012bb0 msgr2=0x5640f100fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:57:31.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.667+0000 7f8ab3fff640 1 --2- 172.21.15.5:0/522022046 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5640f1012bb0 0x5640f100fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:31.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.667+0000 7f8ab3fff640 1 -- 172.21.15.5:0/522022046 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640f0cdf2b0 con 0x5640f10082f0 2024-09-12T02:57:31.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.667+0000 7f8aabfff640 1 --2- 172.21.15.5:0/522022046 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5640f10079d0 0x5640f10124d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:57:31.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.667+0000 7f8ab3fff640 1 --2- 172.21.15.5:0/522022046 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5640f10082f0 0x5640f100f820 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f8aa400bf70 tx=0x7f8aa40128e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9cc03521aaef91f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:31.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.667+0000 7f8a937fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T02:57:31.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.667+0000 7f8a937fe640 1 -- 172.21.15.5:0/522022046 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aa40035f0 con 0x5640f10082f0 2024-09-12T02:57:31.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.667+0000 7f8a937fe640 1 -- 172.21.15.5:0/522022046 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f8aa4003790 con 0x5640f10082f0 2024-09-12T02:57:31.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.667+0000 7f8abc913740 1 -- 172.21.15.5:0/522022046 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a9c005520 con 0x5640f10082f0 2024-09-12T02:57:31.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.668+0000 7f8a937fe640 1 -- 172.21.15.5:0/522022046 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8aa4003990 con 0x5640f10082f0 2024-09-12T02:57:31.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.668+0000 7f8abc913740 10 client.15003 fetch_fsmap learned FSMap version 25 2024-09-12T02:57:31.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.668+0000 7f8abc913740 1 -- 172.21.15.5:0/522022046 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5640f1021160 con 0x5640f10082f0 2024-09-12T02:57:31.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.668+0000 7f8a937fe640 1 -- 172.21.15.5:0/522022046 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a9c005520 con 0x5640f10082f0 2024-09-12T02:57:31.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.668+0000 7f8a937fe640 1 -- 172.21.15.5:0/522022046 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8362+0+0 (secure 0 0 0) 0x7f8aa4019070 con 0x5640f10082f0 2024-09-12T02:57:31.673 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.668+0000 7f8abc913740 10 client.15003 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:57:31.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.668+0000 7f8abc913740 10 client.15003 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-12T02:57:31.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.668+0000 7f8abc913740 20 client.15003 populate_metadata read hostname 'smithi005' 2024-09-12T02:57:31.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.668+0000 7f8abc913740 1 --2- 172.21.15.5:0/522022046 >> [v2:172.21.15.148:6834/4195086427,v1:172.21.15.148:6835/4195086427] conn(0x5640f10237d0 0x5640f1024c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:31.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.668+0000 7f8abc913740 4 client.15003 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:57:31.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.668+0000 7f8abc913740 1 -- 172.21.15.5:0/522022046 --> [v2:172.21.15.148:6834/4195086427,v1:172.21.15.148:6835/4195086427] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5640f0f7b600 con 0x5640f10237d0 2024-09-12T02:57:31.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.668+0000 7f8aabfff640 1 --2- 172.21.15.5:0/522022046 >> [v2:172.21.15.148:6834/4195086427,v1:172.21.15.148:6835/4195086427] conn(0x5640f10237d0 0x5640f1024c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:31.675 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:57:31 smithi148 ceph-mon[35090]: pgmap v313: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-12T02:57:31.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.669+0000 7f8aabfff640 1 --2- 172.21.15.5:0/522022046 >> [v2:172.21.15.148:6834/4195086427,v1:172.21.15.148:6835/4195086427] conn(0x5640f10237d0 0x5640f1024c40 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=49bdd59f7531b0d8 server_cookie=259cea95ab66f998 in_seq=0 out_seq=0 2024-09-12T02:57:31.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.669+0000 7f8a937fe640 10 client.15003 ms_handle_connect on v2:172.21.15.148:6834/4195086427 2024-09-12T02:57:31.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.670+0000 7f8a937fe640 1 -- 172.21.15.5:0/522022046 <== mds.2 v2:172.21.15.148:6834/4195086427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5640f0f7b600 con 0x5640f10237d0 2024-09-12T02:57:31.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.670+0000 7f8a937fe640 10 client.15003 handle_command_reply: tid=0 2024-09-12T02:57:31.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.670+0000 7f8abc913740 1 client.15003 shutdown 2024-09-12T02:57:31.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.670+0000 7f8abc913740 1 -- 172.21.15.5:0/522022046 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5640f10082f0 msgr2=0x5640f100f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:31.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.670+0000 7f8abc913740 1 --2- 172.21.15.5:0/522022046 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5640f10082f0 0x5640f100f820 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f8aa400bf70 tx=0x7f8aa40128e0 comp rx=0 tx=0).stop 2024-09-12T02:57:31.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.670+0000 7f8abc913740 1 -- 172.21.15.5:0/522022046 shutdown_connections 2024-09-12T02:57:31.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.670+0000 7f8abc913740 1 --2- 172.21.15.5:0/522022046 >> [v2:172.21.15.148:6834/4195086427,v1:172.21.15.148:6835/4195086427] conn(0x5640f10237d0 0x5640f1024c40 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:31.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.670+0000 7f8abc913740 1 --2- 172.21.15.5:0/522022046 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5640f10082f0 0x5640f100f820 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:31.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.670+0000 7f8abc913740 1 --2- 172.21.15.5:0/522022046 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5640f1012bb0 0x5640f100fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:31.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.670+0000 7f8abc913740 1 --2- 172.21.15.5:0/522022046 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5640f10079d0 0x5640f10124d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:31.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.670+0000 7f8abc913740 1 -- 172.21.15.5:0/522022046 >> 172.21.15.5:0/522022046 conn(0x5640f1003f60 msgr2=0x5640f1009c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:31.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.670+0000 7f8a937fe640 0 client.15003 ms_handle_reset on v2:172.21.15.148:6834/4195086427 2024-09-12T02:57:31.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.670+0000 7f8abc913740 1 -- 172.21.15.5:0/522022046 shutdown_connections 2024-09-12T02:57:31.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.670+0000 7f8abc913740 1 -- 172.21.15.5:0/522022046 wait complete. 2024-09-12T02:57:31.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.670+0000 7f8abc913740 20 client.15003 trim_cache size 0 max 16384 2024-09-12T02:57:31.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.688+0000 7f8abc913740 1 Processor -- start 2024-09-12T02:57:31.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.688+0000 7f8abc913740 1 -- start start 2024-09-12T02:57:31.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.688+0000 7f8abc913740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5640f10079d0 0x5640f10237d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:31.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.688+0000 7f8abc913740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5640f10082f0 0x5640f100e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:31.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.688+0000 7f8abc913740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5640f1012bb0 0x5640f102cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:31.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.688+0000 7f8abc913740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5640f1022f80 con 0x5640f10079d0 2024-09-12T02:57:31.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.688+0000 7f8abc913740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5640f1023650 con 0x5640f10082f0 2024-09-12T02:57:31.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.688+0000 7f8abc913740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5640f1023d20 con 0x5640f1012bb0 2024-09-12T02:57:31.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.688+0000 7f8aabfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5640f10079d0 0x5640f10237d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:31.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.688+0000 7f8aabfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5640f10079d0 0x5640f10237d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45382/0 (socket says 172.21.15.5:45382) 2024-09-12T02:57:31.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.688+0000 7f8aabfff640 1 -- 172.21.15.5:0/696127792 learned_addr learned my addr 172.21.15.5:0/696127792 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:57:31.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.689+0000 7f8ab3fff640 1 --2- 172.21.15.5:0/696127792 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5640f10082f0 0x5640f100e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:31.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.689+0000 7f8aabfff640 1 -- 172.21.15.5:0/696127792 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5640f1012bb0 msgr2=0x5640f102cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:57:31.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.689+0000 7f8aabfff640 1 --2- 172.21.15.5:0/696127792 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5640f1012bb0 0x5640f102cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:31.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.689+0000 7f8aabfff640 1 -- 172.21.15.5:0/696127792 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5640f10082f0 msgr2=0x5640f100e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:57:31.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.689+0000 7f8aabfff640 1 --2- 172.21.15.5:0/696127792 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5640f10082f0 0x5640f100e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:31.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.689+0000 7f8aabfff640 1 -- 172.21.15.5:0/696127792 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640f1015890 con 0x5640f10079d0 2024-09-12T02:57:31.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.689+0000 7f8ab3fff640 1 --2- 172.21.15.5:0/696127792 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5640f10082f0 0x5640f100e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:57:31.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.689+0000 7f8aabfff640 1 --2- 172.21.15.5:0/696127792 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5640f10079d0 0x5640f10237d0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c000c00 tx=0x7f8a9c009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4028e6d5f9f3429f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:31.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.689+0000 7f8aa9ffb640 1 -- 172.21.15.5:0/696127792 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a9c035040 con 0x5640f10079d0 2024-09-12T02:57:31.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.689+0000 7f8aa9ffb640 1 -- 172.21.15.5:0/696127792 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a9c0046c0 con 0x5640f10079d0 2024-09-12T02:57:31.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.689+0000 7f8aa9ffb640 1 -- 172.21.15.5:0/696127792 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a9c031810 con 0x5640f10079d0 2024-09-12T02:57:31.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.690+0000 7f8abc913740 1 -- 172.21.15.5:0/696127792 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5640f10079d0 msgr2=0x5640f10237d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:31.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.690+0000 7f8abc913740 1 --2- 172.21.15.5:0/696127792 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5640f10079d0 0x5640f10237d0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c000c00 tx=0x7f8a9c009560 comp rx=0 tx=0).stop 2024-09-12T02:57:31.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.690+0000 7f8abc913740 1 -- 172.21.15.5:0/696127792 shutdown_connections 2024-09-12T02:57:31.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.690+0000 7f8abc913740 1 --2- 172.21.15.5:0/696127792 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5640f1012bb0 0x5640f102cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:31.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.690+0000 7f8abc913740 1 --2- 172.21.15.5:0/696127792 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5640f10082f0 0x5640f100e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:31.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.690+0000 7f8abc913740 1 --2- 172.21.15.5:0/696127792 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5640f10079d0 0x5640f10237d0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:31.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.690+0000 7f8abc913740 1 -- 172.21.15.5:0/696127792 >> 172.21.15.5:0/696127792 conn(0x5640f1003f60 msgr2=0x5640f100e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:31.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.690+0000 7f8abc913740 1 -- 172.21.15.5:0/696127792 shutdown_connections 2024-09-12T02:57:31.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.690+0000 7f8abc913740 1 -- 172.21.15.5:0/696127792 wait complete. 2024-09-12T02:57:31.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.690+0000 7f8abc913740 1 -- start start 2024-09-12T02:57:31.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.690+0000 7f8abc913740 1 Processor -- start 2024-09-12T02:57:31.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.691+0000 7f8abc913740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5640f10079d0 0x5640f102d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:31.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.691+0000 7f8abc913740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5640f10082f0 0x5640f1027b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:31.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.691+0000 7f8abc913740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5640f1012bb0 0x5640f10282c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:31.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.691+0000 7f8abc913740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5640f1024ac0 con 0x5640f10082f0 2024-09-12T02:57:31.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.691+0000 7f8abc913740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5640f10243f0 con 0x5640f1012bb0 2024-09-12T02:57:31.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.691+0000 7f8abc913740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5640f1025190 con 0x5640f10079d0 2024-09-12T02:57:31.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.691+0000 7f8abc913740 10 client.0 _ll_register_callbacks cb 0x7f8aa400b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:57:31.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.691+0000 7f8ab3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5640f10082f0 0x5640f1027b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:31.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.691+0000 7f8ab3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5640f10082f0 0x5640f1027b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45390/0 (socket says 172.21.15.5:45390) 2024-09-12T02:57:31.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.691+0000 7f8ab3fff640 1 -- 172.21.15.5:0/588066545 learned_addr learned my addr 172.21.15.5:0/588066545 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:57:31.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.691+0000 7f8aabfff640 1 --2- 172.21.15.5:0/588066545 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5640f1012bb0 0x5640f10282c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:31.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.691+0000 7f8aabfff640 1 --2- 172.21.15.5:0/588066545 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5640f10079d0 0x5640f102d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:31.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.691+0000 7f8ab3fff640 1 -- 172.21.15.5:0/588066545 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5640f10079d0 msgr2=0x5640f102d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:57:31.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.691+0000 7f8ab3fff640 1 --2- 172.21.15.5:0/588066545 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5640f10079d0 0x5640f102d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:31.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.691+0000 7f8ab3fff640 1 -- 172.21.15.5:0/588066545 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5640f1012bb0 msgr2=0x5640f10282c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:57:31.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.691+0000 7f8ab3fff640 1 --2- 172.21.15.5:0/588066545 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5640f1012bb0 0x5640f10282c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:31.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.691+0000 7f8ab3fff640 1 -- 172.21.15.5:0/588066545 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640f108bed0 con 0x5640f10082f0 2024-09-12T02:57:31.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.691+0000 7f8aabfff640 1 --2- 172.21.15.5:0/588066545 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5640f1012bb0 0x5640f10282c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:57:31.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.691+0000 7f8ab3fff640 1 --2- 172.21.15.5:0/588066545 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5640f10082f0 0x5640f1027b30 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x5640f100f0b0 tx=0x7f8aa400bc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4efd86a49167932 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:31.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.691+0000 7f8a937fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T02:57:31.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.691+0000 7f8a937fe640 1 -- 172.21.15.5:0/588066545 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aa4003d30 con 0x5640f10082f0 2024-09-12T02:57:31.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.691+0000 7f8a937fe640 1 -- 172.21.15.5:0/588066545 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f8aa4005020 con 0x5640f10082f0 2024-09-12T02:57:31.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.691+0000 7f8abc913740 1 -- 172.21.15.5:0/588066545 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a9c0046c0 con 0x5640f10082f0 2024-09-12T02:57:31.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.692+0000 7f8a937fe640 1 -- 172.21.15.5:0/588066545 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8aa4005220 con 0x5640f10082f0 2024-09-12T02:57:31.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.692+0000 7f8abc913740 10 client.15009 fetch_fsmap learned FSMap version 25 2024-09-12T02:57:31.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.692+0000 7f8abc913740 1 -- 172.21.15.5:0/588066545 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5640f108d070 con 0x5640f10082f0 2024-09-12T02:57:31.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.692+0000 7f8a937fe640 1 -- 172.21.15.5:0/588066545 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a9c0046c0 con 0x5640f10082f0 2024-09-12T02:57:31.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.692+0000 7f8a937fe640 1 -- 172.21.15.5:0/588066545 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8362+0+0 (secure 0 0 0) 0x7f8aa4019040 con 0x5640f10082f0 2024-09-12T02:57:31.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.692+0000 7f8abc913740 10 client.15009 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:57:31.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.692+0000 7f8abc913740 10 client.15009 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-12T02:57:31.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.692+0000 7f8abc913740 20 client.15009 populate_metadata read hostname 'smithi005' 2024-09-12T02:57:31.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.692+0000 7f8abc913740 1 --2- 172.21.15.5:0/588066545 >> [v2:172.21.15.148:6834/4195086427,v1:172.21.15.148:6835/4195086427] conn(0x5640f1091c40 0x5640f1090950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:31.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.692+0000 7f8abc913740 4 client.15009 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T02:57:31.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.692+0000 7f8abc913740 1 -- 172.21.15.5:0/588066545 --> [v2:172.21.15.148:6834/4195086427,v1:172.21.15.148:6835/4195086427] -- command(tid 0: {"prefix": "status"}) -- 0x5640f0f75b10 con 0x5640f1091c40 2024-09-12T02:57:31.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.692+0000 7f8aabfff640 1 --2- 172.21.15.5:0/588066545 >> [v2:172.21.15.148:6834/4195086427,v1:172.21.15.148:6835/4195086427] conn(0x5640f1091c40 0x5640f1090950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:31.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.693+0000 7f8aabfff640 1 --2- 172.21.15.5:0/588066545 >> [v2:172.21.15.148:6834/4195086427,v1:172.21.15.148:6835/4195086427] conn(0x5640f1091c40 0x5640f1090950 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a24bb4e62776b026 server_cookie=7a392e879758d0ee in_seq=0 out_seq=0 2024-09-12T02:57:31.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.693+0000 7f8a937fe640 10 client.15009 ms_handle_connect on v2:172.21.15.148:6834/4195086427 2024-09-12T02:57:31.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.693+0000 7f8a937fe640 1 -- 172.21.15.5:0/588066545 <== mds.2 v2:172.21.15.148:6834/4195086427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5640f0f75b10 con 0x5640f1091c40 2024-09-12T02:57:31.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.693+0000 7f8a937fe640 10 client.15009 handle_command_reply: tid=0 2024-09-12T02:57:31.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.693+0000 7f8abc913740 1 client.15009 shutdown 2024-09-12T02:57:31.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.693+0000 7f8abc913740 1 -- 172.21.15.5:0/588066545 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5640f10082f0 msgr2=0x5640f1027b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:31.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.693+0000 7f8abc913740 1 --2- 172.21.15.5:0/588066545 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5640f10082f0 0x5640f1027b30 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x5640f100f0b0 tx=0x7f8aa400bc20 comp rx=0 tx=0).stop 2024-09-12T02:57:31.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.693+0000 7f8abc913740 1 -- 172.21.15.5:0/588066545 shutdown_connections 2024-09-12T02:57:31.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.693+0000 7f8abc913740 1 --2- 172.21.15.5:0/588066545 >> [v2:172.21.15.148:6834/4195086427,v1:172.21.15.148:6835/4195086427] conn(0x5640f1091c40 0x5640f1090950 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:31.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.693+0000 7f8abc913740 1 --2- 172.21.15.5:0/588066545 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5640f10082f0 0x5640f1027b30 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:31.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.693+0000 7f8abc913740 1 --2- 172.21.15.5:0/588066545 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5640f1012bb0 0x5640f10282c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:31.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.693+0000 7f8abc913740 1 --2- 172.21.15.5:0/588066545 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5640f10079d0 0x5640f102d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:31.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.693+0000 7f8abc913740 1 -- 172.21.15.5:0/588066545 >> 172.21.15.5:0/588066545 conn(0x5640f1003f60 msgr2=0x5640f100e910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:31.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.693+0000 7f8a937fe640 0 client.15009 ms_handle_reset on v2:172.21.15.148:6834/4195086427 2024-09-12T02:57:31.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.693+0000 7f8abc913740 1 -- 172.21.15.5:0/588066545 shutdown_connections 2024-09-12T02:57:31.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.693+0000 7f8abc913740 1 -- 172.21.15.5:0/588066545 wait complete. 2024-09-12T02:57:31.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.693+0000 7f8abc913740 20 client.15009 trim_cache size 0 max 16384 2024-09-12T02:57:31.701 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-12T02:57:31.701 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "618305f8-70b1-11ef-bcea-c7b262605968", 2024-09-12T02:57:31.701 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 2, 2024-09-12T02:57:31.701 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24481, 2024-09-12T02:57:31.701 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-09-12T02:57:31.701 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-09-12T02:57:31.701 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-09-12T02:57:31.701 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 66.693200716000007, 2024-09-12T02:57:31.702 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 25, 2024-09-12T02:57:31.702 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 83, 2024-09-12T02:57:31.702 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 81, 2024-09-12T02:57:31.702 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 101.467098554 2024-09-12T02:57:31.702 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-12T02:57:31.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.694+0000 7f8aba53a640 1 -- 172.21.15.5:0/3167827906 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f8aa00a2c90 msgr2=0x7f8aa00a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:31.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.694+0000 7f8aba53a640 1 --2- 172.21.15.5:0/3167827906 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f8aa00a2c90 0x7f8aa00a5150 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8aac0984d0 tx=0x7f8aac06db10 comp rx=0 tx=0).stop 2024-09-12T02:57:31.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.694+0000 7f8aba53a640 1 -- 172.21.15.5:0/3167827906 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ab410c170 msgr2=0x7f8ab419cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:31.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.694+0000 7f8aba53a640 1 --2- 172.21.15.5:0/3167827906 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ab410c170 0x7f8ab419cd00 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f8aac00eaa0 tx=0x7f8aac00ef60 comp rx=0 tx=0).stop 2024-09-12T02:57:31.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.694+0000 7f8aba53a640 1 -- 172.21.15.5:0/3167827906 shutdown_connections 2024-09-12T02:57:31.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.694+0000 7f8aba53a640 1 --2- 172.21.15.5:0/3167827906 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f8aa00a2c90 0x7f8aa00a5150 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:31.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.694+0000 7f8aba53a640 1 --2- 172.21.15.5:0/3167827906 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ab410c170 0x7f8ab419cd00 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:31.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.694+0000 7f8aba53a640 1 --2- 172.21.15.5:0/3167827906 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8ab4103660 0x7f8ab4198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:31.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.694+0000 7f8aba53a640 1 --2- 172.21.15.5:0/3167827906 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8ab4102d40 0x7f8ab4198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:31.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.694+0000 7f8aba53a640 1 -- 172.21.15.5:0/3167827906 >> 172.21.15.5:0/3167827906 conn(0x7f8ab40fc7a0 msgr2=0x7f8ab40fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:31.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.694+0000 7f8aba53a640 1 -- 172.21.15.5:0/3167827906 shutdown_connections 2024-09-12T02:57:31.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:31.694+0000 7f8aba53a640 1 -- 172.21.15.5:0/3167827906 wait complete. 2024-09-12T02:57:31.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:57:31 smithi146 ceph-mon[35011]: pgmap v313: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-12T02:57:32.311 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.c status 2024-09-12T02:57:32.567 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:57:33.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.119+0000 7f5243041640 1 -- 172.21.15.5:0/974758375 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f523c106f10 msgr2=0x7f523c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:33.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.119+0000 7f5243041640 1 --2- 172.21.15.5:0/974758375 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f523c106f10 0x7f523c10ea90 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f5234009ca0 tx=0x7f523402a8c0 comp rx=0 tx=0).stop 2024-09-12T02:57:33.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.119+0000 7f5243041640 1 -- 172.21.15.5:0/974758375 shutdown_connections 2024-09-12T02:57:33.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.119+0000 7f5243041640 1 --2- 172.21.15.5:0/974758375 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f523c106f10 0x7f523c10ea90 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:33.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.119+0000 7f5243041640 1 --2- 172.21.15.5:0/974758375 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f523c102790 0x7f523c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:33.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.119+0000 7f5243041640 1 --2- 172.21.15.5:0/974758375 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f523c101de0 0x7f523c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:33.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.119+0000 7f5243041640 1 -- 172.21.15.5:0/974758375 >> 172.21.15.5:0/974758375 conn(0x7f523c0fd8c0 msgr2=0x7f523c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:33.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.119+0000 7f5243041640 1 -- 172.21.15.5:0/974758375 shutdown_connections 2024-09-12T02:57:33.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.119+0000 7f5243041640 1 -- 172.21.15.5:0/974758375 wait complete. 2024-09-12T02:57:33.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.120+0000 7f5243041640 1 Processor -- start 2024-09-12T02:57:33.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.120+0000 7f5243041640 1 -- start start 2024-09-12T02:57:33.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.120+0000 7f5243041640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f523c101de0 0x7f523c198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:33.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.120+0000 7f5243041640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f523c102790 0x7f523c1988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:33.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.120+0000 7f5243041640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f523c106f10 0x7f523c19cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:33.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.120+0000 7f5243041640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f523c111f70 con 0x7f523c101de0 2024-09-12T02:57:33.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.120+0000 7f5243041640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f523c111df0 con 0x7f523c102790 2024-09-12T02:57:33.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.120+0000 7f5243041640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f523c1120f0 con 0x7f523c106f10 2024-09-12T02:57:33.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.120+0000 7f5240ddd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f523c101de0 0x7f523c198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:33.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.120+0000 7f5240ddd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f523c101de0 0x7f523c198390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45420/0 (socket says 172.21.15.5:45420) 2024-09-12T02:57:33.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.120+0000 7f5240ddd640 1 -- 172.21.15.5:0/1535332207 learned_addr learned my addr 172.21.15.5:0/1535332207 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:57:33.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.120+0000 7f52415de640 1 --2- 172.21.15.5:0/1535332207 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f523c106f10 0x7f523c19cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:33.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.120+0000 7f523bfff640 1 --2- 172.21.15.5:0/1535332207 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f523c102790 0x7f523c1988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:33.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.120+0000 7f5240ddd640 1 -- 172.21.15.5:0/1535332207 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f523c106f10 msgr2=0x7f523c19cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:33.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.120+0000 7f5240ddd640 1 --2- 172.21.15.5:0/1535332207 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f523c106f10 0x7f523c19cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:33.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.120+0000 7f5240ddd640 1 -- 172.21.15.5:0/1535332207 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f523c102790 msgr2=0x7f523c1988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:33.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.120+0000 7f5240ddd640 1 --2- 172.21.15.5:0/1535332207 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f523c102790 0x7f523c1988d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:33.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.120+0000 7f5240ddd640 1 -- 172.21.15.5:0/1535332207 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f523c19d340 con 0x7f523c101de0 2024-09-12T02:57:33.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.120+0000 7f52415de640 1 --2- 172.21.15.5:0/1535332207 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f523c106f10 0x7f523c19cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:57:33.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.121+0000 7f5240ddd640 1 --2- 172.21.15.5:0/1535332207 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f523c101de0 0x7f523c198390 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f522c00c920 tx=0x7f522c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:33.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.121+0000 7f5239ffb640 1 -- 172.21.15.5:0/1535332207 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f522c007c70 con 0x7f523c101de0 2024-09-12T02:57:33.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.121+0000 7f5243041640 1 -- 172.21.15.5:0/1535332207 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f523c19d630 con 0x7f523c101de0 2024-09-12T02:57:33.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.121+0000 7f5239ffb640 1 -- 172.21.15.5:0/1535332207 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f522c007e10 con 0x7f523c101de0 2024-09-12T02:57:33.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.121+0000 7f5239ffb640 1 -- 172.21.15.5:0/1535332207 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f522c018890 con 0x7f523c101de0 2024-09-12T02:57:33.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.121+0000 7f5243041640 1 -- 172.21.15.5:0/1535332207 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f523c1a4f10 con 0x7f523c101de0 2024-09-12T02:57:33.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.123+0000 7f5239ffb640 1 -- 172.21.15.5:0/1535332207 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f522c018a70 con 0x7f523c101de0 2024-09-12T02:57:33.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.123+0000 7f524541a740 1 Processor -- start 2024-09-12T02:57:33.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.123+0000 7f524541a740 1 -- start start 2024-09-12T02:57:33.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.124+0000 7f524541a740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5633985af9d0 0x5633985afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:33.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.124+0000 7f524541a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633985b02f0 0x5633985ba670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:33.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.124+0000 7f524541a740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5633985babb0 0x5633985bcfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:33.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.124+0000 7f524541a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x563398516090 con 0x5633985b02f0 2024-09-12T02:57:33.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.124+0000 7f524541a740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5633984bda70 con 0x5633985af9d0 2024-09-12T02:57:33.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.124+0000 7f524541a740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x563398512050 con 0x5633985babb0 2024-09-12T02:57:33.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.124+0000 7f5240ddd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633985b02f0 0x5633985ba670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:33.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.124+0000 7f5240ddd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633985b02f0 0x5633985ba670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45422/0 (socket says 172.21.15.5:45422) 2024-09-12T02:57:33.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.124+0000 7f5240ddd640 1 -- 172.21.15.5:0/3572066510 learned_addr learned my addr 172.21.15.5:0/3572066510 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:57:33.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.124+0000 7f52415de640 1 --2- 172.21.15.5:0/3572066510 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5633985af9d0 0x5633985afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:33.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.124+0000 7f5239ffb640 1 --2- 172.21.15.5:0/1535332207 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f52240a2c10 0x7f52240a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:33.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.124+0000 7f523bfff640 1 --2- 172.21.15.5:0/3572066510 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5633985babb0 0x5633985bcfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:33.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.124+0000 7f5240ddd640 1 -- 172.21.15.5:0/3572066510 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5633985babb0 msgr2=0x5633985bcfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:57:33.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.124+0000 7f5240ddd640 1 --2- 172.21.15.5:0/3572066510 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5633985babb0 0x5633985bcfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:33.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.124+0000 7f5240ddd640 1 -- 172.21.15.5:0/3572066510 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5633985af9d0 msgr2=0x5633985afdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:57:33.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.124+0000 7f5240ddd640 1 --2- 172.21.15.5:0/3572066510 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5633985af9d0 0x5633985afdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:33.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.124+0000 7f5240ddd640 1 -- 172.21.15.5:0/3572066510 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633984ca130 con 0x5633985b02f0 2024-09-12T02:57:33.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.124+0000 7f52415de640 1 --2- 172.21.15.5:0/3572066510 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5633985af9d0 0x5633985afdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:57:33.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.124+0000 7f52415de640 1 --2- 172.21.15.5:0/1535332207 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f52240a2c10 0x7f52240a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:33.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.124+0000 7f5240ddd640 1 --2- 172.21.15.5:0/3572066510 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633985b02f0 0x5633985ba670 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f522c0cccc0 tx=0x7f522c097640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81b6cd828a037fd2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:33.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.125+0000 7f5239ffb640 1 -- 172.21.15.5:0/1535332207 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f522c0cf420 con 0x7f523c101de0 2024-09-12T02:57:33.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.125+0000 7f5222ffd640 1 -- 172.21.15.5:0/3572066510 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f522c09b610 con 0x5633985b02f0 2024-09-12T02:57:33.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.125+0000 7f5222ffd640 1 -- 172.21.15.5:0/3572066510 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f522c09c1d0 con 0x5633985b02f0 2024-09-12T02:57:33.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.125+0000 7f5222ffd640 1 -- 172.21.15.5:0/3572066510 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f522c09b430 con 0x5633985b02f0 2024-09-12T02:57:33.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.125+0000 7f524541a740 1 -- 172.21.15.5:0/3572066510 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633985b02f0 msgr2=0x5633985ba670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:33.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.125+0000 7f52415de640 1 --2- 172.21.15.5:0/1535332207 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f52240a2c10 0x7f52240a50d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5234005e30 tx=0x7f52340095c0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:33.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.125+0000 7f524541a740 1 --2- 172.21.15.5:0/3572066510 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633985b02f0 0x5633985ba670 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f522c0cccc0 tx=0x7f522c097640 comp rx=0 tx=0).stop 2024-09-12T02:57:33.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.125+0000 7f524541a740 1 -- 172.21.15.5:0/3572066510 shutdown_connections 2024-09-12T02:57:33.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.125+0000 7f524541a740 1 --2- 172.21.15.5:0/3572066510 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633985b02f0 0x5633985ba670 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:33.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.125+0000 7f524541a740 1 --2- 172.21.15.5:0/3572066510 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5633985babb0 0x5633985bcfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:33.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.125+0000 7f524541a740 1 --2- 172.21.15.5:0/3572066510 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5633985af9d0 0x5633985afdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:33.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.125+0000 7f524541a740 1 -- 172.21.15.5:0/3572066510 >> 172.21.15.5:0/3572066510 conn(0x5633985abf60 msgr2=0x5633985ac340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:33.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.125+0000 7f524541a740 1 -- 172.21.15.5:0/3572066510 shutdown_connections 2024-09-12T02:57:33.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.126+0000 7f524541a740 1 -- 172.21.15.5:0/3572066510 wait complete. 2024-09-12T02:57:33.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.126+0000 7f524541a740 1 -- start start 2024-09-12T02:57:33.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.126+0000 7f524541a740 1 Processor -- start 2024-09-12T02:57:33.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.126+0000 7f524541a740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5633985af9d0 0x5633985ba430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:33.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.126+0000 7f524541a740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5633985b02f0 0x5633985b7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:33.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.126+0000 7f524541a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633985babb0 0x5633985b7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:33.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.126+0000 7f524541a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5633985bd780 con 0x5633985babb0 2024-09-12T02:57:33.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.126+0000 7f524541a740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5633984d0f20 con 0x5633985b02f0 2024-09-12T02:57:33.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.126+0000 7f524541a740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5633985bd900 con 0x5633985af9d0 2024-09-12T02:57:33.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.126+0000 7f524541a740 10 client.0 _ll_register_callbacks cb 0x56339849a030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:57:33.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.126+0000 7f52415de640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633985babb0 0x5633985b7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:33.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.126+0000 7f52415de640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633985babb0 0x5633985b7cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45436/0 (socket says 172.21.15.5:45436) 2024-09-12T02:57:33.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.127+0000 7f52415de640 1 -- 172.21.15.5:0/4083226761 learned_addr learned my addr 172.21.15.5:0/4083226761 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:57:33.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.127+0000 7f5240ddd640 1 --2- 172.21.15.5:0/4083226761 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5633985af9d0 0x5633985ba430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:33.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.127+0000 7f523bfff640 1 --2- 172.21.15.5:0/4083226761 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5633985b02f0 0x5633985b7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:33.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.127+0000 7f52415de640 1 -- 172.21.15.5:0/4083226761 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5633985af9d0 msgr2=0x5633985ba430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:57:33.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.127+0000 7f52415de640 1 --2- 172.21.15.5:0/4083226761 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5633985af9d0 0x5633985ba430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:33.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.127+0000 7f52415de640 1 -- 172.21.15.5:0/4083226761 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5633985b02f0 msgr2=0x5633985b7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:57:33.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.127+0000 7f52415de640 1 --2- 172.21.15.5:0/4083226761 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5633985b02f0 0x5633985b7780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:33.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.127+0000 7f52415de640 1 -- 172.21.15.5:0/4083226761 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5633982872b0 con 0x5633985babb0 2024-09-12T02:57:33.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.127+0000 7f52415de640 1 --2- 172.21.15.5:0/4083226761 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633985babb0 0x5633985b7cc0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f52340093b0 tx=0x7f523402a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa08cf22c88e9bba server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:33.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.127+0000 7f521bfff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T02:57:33.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.127+0000 7f521bfff640 1 -- 172.21.15.5:0/4083226761 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5234004240 con 0x5633985babb0 2024-09-12T02:57:33.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.127+0000 7f521bfff640 1 -- 172.21.15.5:0/4083226761 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f52340043e0 con 0x5633985babb0 2024-09-12T02:57:33.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.127+0000 7f524541a740 1 -- 172.21.15.5:0/4083226761 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f522c09c1d0 con 0x5633985babb0 2024-09-12T02:57:33.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.127+0000 7f521bfff640 1 -- 172.21.15.5:0/4083226761 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5234004620 con 0x5633985babb0 2024-09-12T02:57:33.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.127+0000 7f524541a740 10 client.15021 fetch_fsmap learned FSMap version 25 2024-09-12T02:57:33.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.127+0000 7f524541a740 1 -- 172.21.15.5:0/4083226761 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5633985c9090 con 0x5633985babb0 2024-09-12T02:57:33.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.128+0000 7f521bfff640 1 -- 172.21.15.5:0/4083226761 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f522c09c1d0 con 0x5633985babb0 2024-09-12T02:57:33.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.128+0000 7f521bfff640 1 -- 172.21.15.5:0/4083226761 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8362+0+0 (secure 0 0 0) 0x7f52340306e0 con 0x5633985babb0 2024-09-12T02:57:33.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.128+0000 7f524541a740 10 client.15021 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:57:33.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.128+0000 7f524541a740 10 client.15021 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-12T02:57:33.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.128+0000 7f524541a740 20 client.15021 populate_metadata read hostname 'smithi005' 2024-09-12T02:57:33.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.128+0000 7f524541a740 1 --2- 172.21.15.5:0/4083226761 >> [v2:172.21.15.148:6836/2606873922,v1:172.21.15.148:6837/2606873922] conn(0x5633985cb810 0x5633985ccc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:33.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.128+0000 7f524541a740 4 client.15021 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:57:33.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.128+0000 7f524541a740 1 -- 172.21.15.5:0/4083226761 --> [v2:172.21.15.148:6836/2606873922,v1:172.21.15.148:6837/2606873922] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563398523600 con 0x5633985cb810 2024-09-12T02:57:33.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.128+0000 7f5240ddd640 1 --2- 172.21.15.5:0/4083226761 >> [v2:172.21.15.148:6836/2606873922,v1:172.21.15.148:6837/2606873922] conn(0x5633985cb810 0x5633985ccc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:33.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.129+0000 7f5240ddd640 1 --2- 172.21.15.5:0/4083226761 >> [v2:172.21.15.148:6836/2606873922,v1:172.21.15.148:6837/2606873922] conn(0x5633985cb810 0x5633985ccc80 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=15868e3613ab1557 server_cookie=ad8ea56da9e5995e in_seq=0 out_seq=0 2024-09-12T02:57:33.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.129+0000 7f521bfff640 10 client.15021 ms_handle_connect on v2:172.21.15.148:6836/2606873922 2024-09-12T02:57:33.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.129+0000 7f521bfff640 1 -- 172.21.15.5:0/4083226761 <== mds.1 v2:172.21.15.148:6836/2606873922 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563398523600 con 0x5633985cb810 2024-09-12T02:57:33.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.129+0000 7f521bfff640 10 client.15021 handle_command_reply: tid=0 2024-09-12T02:57:33.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.129+0000 7f524541a740 1 client.15021 shutdown 2024-09-12T02:57:33.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.130+0000 7f524541a740 1 -- 172.21.15.5:0/4083226761 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633985babb0 msgr2=0x5633985b7cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:33.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.130+0000 7f524541a740 1 --2- 172.21.15.5:0/4083226761 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633985babb0 0x5633985b7cc0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f52340093b0 tx=0x7f523402a8c0 comp rx=0 tx=0).stop 2024-09-12T02:57:33.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.130+0000 7f524541a740 1 -- 172.21.15.5:0/4083226761 shutdown_connections 2024-09-12T02:57:33.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.130+0000 7f524541a740 1 --2- 172.21.15.5:0/4083226761 >> [v2:172.21.15.148:6836/2606873922,v1:172.21.15.148:6837/2606873922] conn(0x5633985cb810 0x5633985ccc80 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:33.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.130+0000 7f524541a740 1 --2- 172.21.15.5:0/4083226761 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633985babb0 0x5633985b7cc0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:33.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.130+0000 7f524541a740 1 --2- 172.21.15.5:0/4083226761 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5633985b02f0 0x5633985b7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:33.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.130+0000 7f521bfff640 0 client.15021 ms_handle_reset on v2:172.21.15.148:6836/2606873922 2024-09-12T02:57:33.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.130+0000 7f524541a740 1 --2- 172.21.15.5:0/4083226761 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5633985af9d0 0x5633985ba430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:33.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.130+0000 7f524541a740 1 -- 172.21.15.5:0/4083226761 >> 172.21.15.5:0/4083226761 conn(0x5633985abf60 msgr2=0x5633985b1c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:33.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.130+0000 7f524541a740 1 -- 172.21.15.5:0/4083226761 shutdown_connections 2024-09-12T02:57:33.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.130+0000 7f524541a740 1 -- 172.21.15.5:0/4083226761 wait complete. 2024-09-12T02:57:33.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.130+0000 7f524541a740 20 client.15021 trim_cache size 0 max 16384 2024-09-12T02:57:33.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.149+0000 7f524541a740 1 Processor -- start 2024-09-12T02:57:33.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.149+0000 7f524541a740 1 -- start start 2024-09-12T02:57:33.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.149+0000 7f524541a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633985af9d0 0x5633985ccc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:33.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.149+0000 7f524541a740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5633985b02f0 0x5633985ca3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:33.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.149+0000 7f524541a740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5633985babb0 0x5633985d5350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:33.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.149+0000 7f524541a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5633985cafc0 con 0x5633985af9d0 2024-09-12T02:57:33.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.149+0000 7f524541a740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5633985cb690 con 0x5633985babb0 2024-09-12T02:57:33.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.149+0000 7f524541a740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5633985cbd60 con 0x5633985b02f0 2024-09-12T02:57:33.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.150+0000 7f5240ddd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633985af9d0 0x5633985ccc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:33.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.150+0000 7f5240ddd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633985af9d0 0x5633985ccc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45448/0 (socket says 172.21.15.5:45448) 2024-09-12T02:57:33.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.150+0000 7f5240ddd640 1 -- 172.21.15.5:0/2161887049 learned_addr learned my addr 172.21.15.5:0/2161887049 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:57:33.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.150+0000 7f523bfff640 1 --2- 172.21.15.5:0/2161887049 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5633985b02f0 0x5633985ca3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:33.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.150+0000 7f52415de640 1 --2- 172.21.15.5:0/2161887049 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5633985babb0 0x5633985d5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:33.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.150+0000 7f5240ddd640 1 -- 172.21.15.5:0/2161887049 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5633985b02f0 msgr2=0x5633985ca3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:57:33.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.150+0000 7f5240ddd640 1 --2- 172.21.15.5:0/2161887049 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5633985b02f0 0x5633985ca3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:33.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.150+0000 7f5240ddd640 1 -- 172.21.15.5:0/2161887049 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5633985babb0 msgr2=0x5633985d5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:57:33.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.150+0000 7f5240ddd640 1 --2- 172.21.15.5:0/2161887049 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5633985babb0 0x5633985d5350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:33.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.150+0000 7f5240ddd640 1 -- 172.21.15.5:0/2161887049 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633985b0120 con 0x5633985af9d0 2024-09-12T02:57:33.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.150+0000 7f5240ddd640 1 --2- 172.21.15.5:0/2161887049 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633985af9d0 0x5633985ccc80 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f522c09b7d0 tx=0x7f522c09bad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b63719c3de1f86c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:33.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.150+0000 7f52227fc640 1 -- 172.21.15.5:0/2161887049 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f522c09b610 con 0x5633985af9d0 2024-09-12T02:57:33.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.150+0000 7f52227fc640 1 -- 172.21.15.5:0/2161887049 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f522c09c1d0 con 0x5633985af9d0 2024-09-12T02:57:33.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.150+0000 7f52227fc640 1 -- 172.21.15.5:0/2161887049 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f522c09fe80 con 0x5633985af9d0 2024-09-12T02:57:33.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.151+0000 7f524541a740 1 -- 172.21.15.5:0/2161887049 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633985af9d0 msgr2=0x5633985ccc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:33.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.151+0000 7f524541a740 1 --2- 172.21.15.5:0/2161887049 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633985af9d0 0x5633985ccc80 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f522c09b7d0 tx=0x7f522c09bad0 comp rx=0 tx=0).stop 2024-09-12T02:57:33.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.151+0000 7f524541a740 1 -- 172.21.15.5:0/2161887049 shutdown_connections 2024-09-12T02:57:33.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.151+0000 7f524541a740 1 --2- 172.21.15.5:0/2161887049 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5633985babb0 0x5633985d5350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:33.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.151+0000 7f524541a740 1 --2- 172.21.15.5:0/2161887049 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5633985b02f0 0x5633985ca3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:33.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.151+0000 7f524541a740 1 --2- 172.21.15.5:0/2161887049 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633985af9d0 0x5633985ccc80 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:33.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.151+0000 7f524541a740 1 -- 172.21.15.5:0/2161887049 >> 172.21.15.5:0/2161887049 conn(0x5633985abf60 msgr2=0x5633985a6ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:33.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.151+0000 7f524541a740 1 -- 172.21.15.5:0/2161887049 shutdown_connections 2024-09-12T02:57:33.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.151+0000 7f524541a740 1 -- 172.21.15.5:0/2161887049 wait complete. 2024-09-12T02:57:33.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.151+0000 7f524541a740 1 -- start start 2024-09-12T02:57:33.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.152+0000 7f524541a740 1 Processor -- start 2024-09-12T02:57:33.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.152+0000 7f524541a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633985af9d0 0x5633985d4520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:33.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.152+0000 7f524541a740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5633985b02f0 0x5633985d4a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:33.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.152+0000 7f524541a740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5633985babb0 0x563398637f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:33.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.152+0000 7f524541a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5633985ccb00 con 0x5633985af9d0 2024-09-12T02:57:33.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.152+0000 7f524541a740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5633985cc430 con 0x5633985b02f0 2024-09-12T02:57:33.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.152+0000 7f524541a740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5633985cd1d0 con 0x5633985babb0 2024-09-12T02:57:33.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.152+0000 7f524541a740 10 client.0 _ll_register_callbacks cb 0x7f5234009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:57:33.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.152+0000 7f5240ddd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633985af9d0 0x5633985d4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:33.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.152+0000 7f5240ddd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633985af9d0 0x5633985d4520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45464/0 (socket says 172.21.15.5:45464) 2024-09-12T02:57:33.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.152+0000 7f5240ddd640 1 -- 172.21.15.5:0/1787645499 learned_addr learned my addr 172.21.15.5:0/1787645499 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:57:33.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.152+0000 7f523bfff640 1 --2- 172.21.15.5:0/1787645499 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5633985b02f0 0x5633985d4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:33.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.152+0000 7f52415de640 1 --2- 172.21.15.5:0/1787645499 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5633985babb0 0x563398637f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:33.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.152+0000 7f5240ddd640 1 -- 172.21.15.5:0/1787645499 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5633985babb0 msgr2=0x563398637f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:57:33.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.152+0000 7f5240ddd640 1 --2- 172.21.15.5:0/1787645499 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5633985babb0 0x563398637f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:33.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.152+0000 7f5240ddd640 1 -- 172.21.15.5:0/1787645499 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5633985b02f0 msgr2=0x5633985d4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:57:33.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.152+0000 7f5240ddd640 1 --2- 172.21.15.5:0/1787645499 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5633985b02f0 0x5633985d4a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:33.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.153+0000 7f5240ddd640 1 -- 172.21.15.5:0/1787645499 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563398638480 con 0x5633985af9d0 2024-09-12T02:57:33.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.153+0000 7f5240ddd640 1 --2- 172.21.15.5:0/1787645499 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633985af9d0 0x5633985d4520 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f522c0d4010 tx=0x7f522c0d2be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a99643e66a314f46 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:33.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.153+0000 7f521bfff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T02:57:33.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.153+0000 7f521bfff640 1 -- 172.21.15.5:0/1787645499 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f522c00de20 con 0x5633985af9d0 2024-09-12T02:57:33.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.153+0000 7f521bfff640 1 -- 172.21.15.5:0/1787645499 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f522c097960 con 0x5633985af9d0 2024-09-12T02:57:33.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.153+0000 7f524541a740 1 -- 172.21.15.5:0/1787645499 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f522c09c1d0 con 0x5633985af9d0 2024-09-12T02:57:33.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.153+0000 7f521bfff640 1 -- 172.21.15.5:0/1787645499 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f522c0a4740 con 0x5633985af9d0 2024-09-12T02:57:33.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.153+0000 7f524541a740 10 client.15027 fetch_fsmap learned FSMap version 25 2024-09-12T02:57:33.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.153+0000 7f524541a740 1 -- 172.21.15.5:0/1787645499 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563398639680 con 0x5633985af9d0 2024-09-12T02:57:33.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.153+0000 7f521bfff640 1 -- 172.21.15.5:0/1787645499 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f522c09c1d0 con 0x5633985af9d0 2024-09-12T02:57:33.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.153+0000 7f521bfff640 1 -- 172.21.15.5:0/1787645499 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8362+0+0 (secure 0 0 0) 0x7f522c098140 con 0x5633985af9d0 2024-09-12T02:57:33.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.153+0000 7f524541a740 10 client.15027 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:57:33.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.153+0000 7f524541a740 10 client.15027 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-12T02:57:33.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.153+0000 7f524541a740 20 client.15027 populate_metadata read hostname 'smithi005' 2024-09-12T02:57:33.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.154+0000 7f524541a740 1 --2- 172.21.15.5:0/1787645499 >> [v2:172.21.15.148:6836/2606873922,v1:172.21.15.148:6837/2606873922] conn(0x56339863dc70 0x56339863c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:33.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.154+0000 7f524541a740 4 client.15027 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T02:57:33.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.154+0000 7f524541a740 1 -- 172.21.15.5:0/1787645499 --> [v2:172.21.15.148:6836/2606873922,v1:172.21.15.148:6837/2606873922] -- command(tid 0: {"prefix": "status"}) -- 0x56339851db10 con 0x56339863dc70 2024-09-12T02:57:33.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.154+0000 7f523bfff640 1 --2- 172.21.15.5:0/1787645499 >> [v2:172.21.15.148:6836/2606873922,v1:172.21.15.148:6837/2606873922] conn(0x56339863dc70 0x56339863c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:33.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.154+0000 7f521bfff640 10 client.15027 ms_handle_connect on v2:172.21.15.148:6836/2606873922 2024-09-12T02:57:33.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.154+0000 7f523bfff640 1 --2- 172.21.15.5:0/1787645499 >> [v2:172.21.15.148:6836/2606873922,v1:172.21.15.148:6837/2606873922] conn(0x56339863dc70 0x56339863c980 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fc8fe4b5a3a513d0 server_cookie=cc590793a0a00d3c in_seq=0 out_seq=0 2024-09-12T02:57:33.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.154+0000 7f521bfff640 1 -- 172.21.15.5:0/1787645499 <== mds.1 v2:172.21.15.148:6836/2606873922 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x56339851db10 con 0x56339863dc70 2024-09-12T02:57:33.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.154+0000 7f521bfff640 10 client.15027 handle_command_reply: tid=0 2024-09-12T02:57:33.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.154+0000 7f524541a740 1 client.15027 shutdown 2024-09-12T02:57:33.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.154+0000 7f524541a740 1 -- 172.21.15.5:0/1787645499 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633985af9d0 msgr2=0x5633985d4520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:33.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.154+0000 7f524541a740 1 --2- 172.21.15.5:0/1787645499 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633985af9d0 0x5633985d4520 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f522c0d4010 tx=0x7f522c0d2be0 comp rx=0 tx=0).stop 2024-09-12T02:57:33.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.154+0000 7f524541a740 1 -- 172.21.15.5:0/1787645499 shutdown_connections 2024-09-12T02:57:33.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.154+0000 7f524541a740 1 --2- 172.21.15.5:0/1787645499 >> [v2:172.21.15.148:6836/2606873922,v1:172.21.15.148:6837/2606873922] conn(0x56339863dc70 0x56339863c980 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:33.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.155+0000 7f521bfff640 0 client.15027 ms_handle_reset on v2:172.21.15.148:6836/2606873922 2024-09-12T02:57:33.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.155+0000 7f524541a740 1 --2- 172.21.15.5:0/1787645499 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5633985babb0 0x563398637f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:33.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.155+0000 7f524541a740 1 --2- 172.21.15.5:0/1787645499 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5633985b02f0 0x5633985d4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:33.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.155+0000 7f524541a740 1 --2- 172.21.15.5:0/1787645499 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633985af9d0 0x5633985d4520 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:33.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.155+0000 7f524541a740 1 -- 172.21.15.5:0/1787645499 >> 172.21.15.5:0/1787645499 conn(0x5633985abf60 msgr2=0x5633985ac340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:33.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.155+0000 7f524541a740 1 -- 172.21.15.5:0/1787645499 shutdown_connections 2024-09-12T02:57:33.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.155+0000 7f524541a740 1 -- 172.21.15.5:0/1787645499 wait complete. 2024-09-12T02:57:33.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.155+0000 7f524541a740 20 client.15027 trim_cache size 0 max 16384 2024-09-12T02:57:33.166 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-12T02:57:33.166 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "618305f8-70b1-11ef-bcea-c7b262605968", 2024-09-12T02:57:33.166 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 1, 2024-09-12T02:57:33.166 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24487, 2024-09-12T02:57:33.166 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-09-12T02:57:33.166 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-09-12T02:57:33.166 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-09-12T02:57:33.166 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 69.164136775000003, 2024-09-12T02:57:33.167 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 25, 2024-09-12T02:57:33.167 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 83, 2024-09-12T02:57:33.167 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 81, 2024-09-12T02:57:33.167 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 100.58254955300001 2024-09-12T02:57:33.167 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-12T02:57:33.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.155+0000 7f5243041640 1 -- 172.21.15.5:0/1535332207 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f52240a2c10 msgr2=0x7f52240a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:33.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.155+0000 7f5243041640 1 --2- 172.21.15.5:0/1535332207 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f52240a2c10 0x7f52240a50d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5234005e30 tx=0x7f52340095c0 comp rx=0 tx=0).stop 2024-09-12T02:57:33.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.155+0000 7f5243041640 1 -- 172.21.15.5:0/1535332207 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f523c101de0 msgr2=0x7f523c198390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:33.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.155+0000 7f5243041640 1 --2- 172.21.15.5:0/1535332207 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f523c101de0 0x7f523c198390 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f522c00c920 tx=0x7f522c00cde0 comp rx=0 tx=0).stop 2024-09-12T02:57:33.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.155+0000 7f5243041640 1 -- 172.21.15.5:0/1535332207 shutdown_connections 2024-09-12T02:57:33.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.155+0000 7f5243041640 1 --2- 172.21.15.5:0/1535332207 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f523c106f10 0x7f523c19cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:33.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.155+0000 7f5243041640 1 --2- 172.21.15.5:0/1535332207 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f523c102790 0x7f523c1988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:33.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.155+0000 7f5243041640 1 --2- 172.21.15.5:0/1535332207 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f52240a2c10 0x7f52240a50d0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:33.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.155+0000 7f5243041640 1 --2- 172.21.15.5:0/1535332207 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f523c101de0 0x7f523c198390 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:33.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.155+0000 7f5243041640 1 -- 172.21.15.5:0/1535332207 >> 172.21.15.5:0/1535332207 conn(0x7f523c0fd8c0 msgr2=0x7f523c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:33.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.156+0000 7f5243041640 1 -- 172.21.15.5:0/1535332207 shutdown_connections 2024-09-12T02:57:33.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:33.156+0000 7f5243041640 1 -- 172.21.15.5:0/1535332207 wait complete. 2024-09-12T02:57:33.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:57:33 smithi005 ceph-mon[31197]: pgmap v314: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T02:57:33.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:57:33 smithi148 ceph-mon[35090]: pgmap v314: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T02:57:33.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:57:33 smithi146 ceph-mon[35011]: pgmap v314: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T02:57:33.726 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.i status 2024-09-12T02:57:33.983 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:57:34.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.467+0000 7f585b59e640 1 -- 172.21.15.5:0/2299605291 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f585c10bfc0 msgr2=0x7f585c10e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:34.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.467+0000 7f585b59e640 1 --2- 172.21.15.5:0/2299605291 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f585c10bfc0 0x7f585c10e460 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f5854009ca0 tx=0x7f585402a950 comp rx=0 tx=0).stop 2024-09-12T02:57:34.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.468+0000 7f585b59e640 1 -- 172.21.15.5:0/2299605291 shutdown_connections 2024-09-12T02:57:34.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.468+0000 7f585b59e640 1 --2- 172.21.15.5:0/2299605291 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f585c10bfc0 0x7f585c10e460 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:34.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.468+0000 7f585b59e640 1 --2- 172.21.15.5:0/2299605291 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f585c1034b0 0x7f585c10b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:34.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.468+0000 7f585b59e640 1 --2- 172.21.15.5:0/2299605291 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f585c102b90 0x7f585c102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:34.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.468+0000 7f585b59e640 1 -- 172.21.15.5:0/2299605291 >> 172.21.15.5:0/2299605291 conn(0x7f585c0fc630 msgr2=0x7f585c0fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:34.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.468+0000 7f585b59e640 1 -- 172.21.15.5:0/2299605291 shutdown_connections 2024-09-12T02:57:34.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.468+0000 7f585b59e640 1 -- 172.21.15.5:0/2299605291 wait complete. 2024-09-12T02:57:34.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.469+0000 7f585b59e640 1 Processor -- start 2024-09-12T02:57:34.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.469+0000 7f585b59e640 1 -- start start 2024-09-12T02:57:34.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.469+0000 7f585b59e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f585c102b90 0x7f585c198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:34.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.469+0000 7f585b59e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f585c1034b0 0x7f585c198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:34.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.469+0000 7f585b59e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f585c10bfc0 0x7f585c19cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:34.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.469+0000 7f585b59e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f585c111e00 con 0x7f585c1034b0 2024-09-12T02:57:34.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.469+0000 7f585b59e640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f585c111c80 con 0x7f585c102b90 2024-09-12T02:57:34.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.469+0000 7f585b59e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f585c111f80 con 0x7f585c10bfc0 2024-09-12T02:57:34.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.469+0000 7f5859d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f585c1034b0 0x7f585c198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:34.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.469+0000 7f5859d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f585c1034b0 0x7f585c198780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51750/0 (socket says 172.21.15.5:51750) 2024-09-12T02:57:34.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.469+0000 7f5859d9b640 1 -- 172.21.15.5:0/1632504792 learned_addr learned my addr 172.21.15.5:0/1632504792 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:57:34.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.470+0000 7f585a59c640 1 --2- 172.21.15.5:0/1632504792 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f585c102b90 0x7f585c198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:34.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.470+0000 7f585ad9d640 1 --2- 172.21.15.5:0/1632504792 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f585c10bfc0 0x7f585c19cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:34.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.470+0000 7f5859d9b640 1 -- 172.21.15.5:0/1632504792 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f585c10bfc0 msgr2=0x7f585c19cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:34.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.470+0000 7f5859d9b640 1 --2- 172.21.15.5:0/1632504792 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f585c10bfc0 0x7f585c19cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:34.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.470+0000 7f5859d9b640 1 -- 172.21.15.5:0/1632504792 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f585c102b90 msgr2=0x7f585c198240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:34.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.470+0000 7f5859d9b640 1 --2- 172.21.15.5:0/1632504792 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f585c102b90 0x7f585c198240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:34.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.470+0000 7f5859d9b640 1 -- 172.21.15.5:0/1632504792 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f585c19d1f0 con 0x7f585c1034b0 2024-09-12T02:57:34.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.470+0000 7f5859d9b640 1 --2- 172.21.15.5:0/1632504792 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f585c1034b0 0x7f585c198780 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f585000b3f0 tx=0x7f585000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:34.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.470+0000 7f584b7fe640 1 -- 172.21.15.5:0/1632504792 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5850015020 con 0x7f585c1034b0 2024-09-12T02:57:34.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.470+0000 7f585b59e640 1 -- 172.21.15.5:0/1632504792 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f585c19d4e0 con 0x7f585c1034b0 2024-09-12T02:57:34.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.470+0000 7f584b7fe640 1 -- 172.21.15.5:0/1632504792 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5850011d10 con 0x7f585c1034b0 2024-09-12T02:57:34.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.470+0000 7f584b7fe640 1 -- 172.21.15.5:0/1632504792 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f585000f2e0 con 0x7f585c1034b0 2024-09-12T02:57:34.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.470+0000 7f585b59e640 1 -- 172.21.15.5:0/1632504792 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f585c1a4dc0 con 0x7f585c1034b0 2024-09-12T02:57:34.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.472+0000 7f584b7fe640 1 -- 172.21.15.5:0/1632504792 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f5850018020 con 0x7f585c1034b0 2024-09-12T02:57:34.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.473+0000 7f5862fd8740 1 Processor -- start 2024-09-12T02:57:34.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.473+0000 7f584b7fe640 1 --2- 172.21.15.5:0/1632504792 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f58440a2c10 0x7f58440a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:34.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.473+0000 7f585ad9d640 1 --2- 172.21.15.5:0/1632504792 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f58440a2c10 0x7f58440a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:34.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.473+0000 7f5862fd8740 1 -- start start 2024-09-12T02:57:34.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.473+0000 7f584b7fe640 1 -- 172.21.15.5:0/1632504792 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f58500cea00 con 0x7f585c1034b0 2024-09-12T02:57:34.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.473+0000 7f5862fd8740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558380b83bd0 0x558380b83fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:34.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.473+0000 7f5862fd8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558380b844f0 0x558380b8e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:34.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.473+0000 7f5862fd8740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558380b8edb0 0x558380b911a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:34.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.473+0000 7f5862fd8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558380aea090 con 0x558380b844f0 2024-09-12T02:57:34.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.473+0000 7f5862fd8740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x558380a91a70 con 0x558380b83bd0 2024-09-12T02:57:34.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.473+0000 7f5862fd8740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x558380ae6050 con 0x558380b8edb0 2024-09-12T02:57:34.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.474+0000 7f5859d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558380b844f0 0x558380b8e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:34.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.474+0000 7f5859d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558380b844f0 0x558380b8e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51764/0 (socket says 172.21.15.5:51764) 2024-09-12T02:57:34.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.474+0000 7f5859d9b640 1 -- 172.21.15.5:0/2668245865 learned_addr learned my addr 172.21.15.5:0/2668245865 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:57:34.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.474+0000 7f585a59c640 1 --2- 172.21.15.5:0/2668245865 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558380b83bd0 0x558380b83fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:34.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.474+0000 7f585ad9d640 1 --2- 172.21.15.5:0/1632504792 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f58440a2c10 0x7f58440a50d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5854009430 tx=0x7f5854005ef0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:34.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.474+0000 7f585ad9d640 1 --2- 172.21.15.5:0/2668245865 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558380b8edb0 0x558380b911a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:34.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.474+0000 7f5859d9b640 1 -- 172.21.15.5:0/2668245865 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558380b8edb0 msgr2=0x558380b911a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:57:34.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.474+0000 7f5859d9b640 1 --2- 172.21.15.5:0/2668245865 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558380b8edb0 0x558380b911a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:34.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.474+0000 7f5859d9b640 1 -- 172.21.15.5:0/2668245865 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558380b83bd0 msgr2=0x558380b83fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:57:34.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.474+0000 7f5859d9b640 1 --2- 172.21.15.5:0/2668245865 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558380b83bd0 0x558380b83fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:34.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.474+0000 7f5859d9b640 1 -- 172.21.15.5:0/2668245865 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558380a9e130 con 0x558380b844f0 2024-09-12T02:57:34.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.474+0000 7f5859d9b640 1 --2- 172.21.15.5:0/2668245865 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558380b844f0 0x558380b8e870 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f58500a3960 tx=0x7f585009ac80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4120980d013be429 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:34.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.474+0000 7f5848ff9640 1 -- 172.21.15.5:0/2668245865 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58500b9e60 con 0x558380b844f0 2024-09-12T02:57:34.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.474+0000 7f5848ff9640 1 -- 172.21.15.5:0/2668245865 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f585009b220 con 0x558380b844f0 2024-09-12T02:57:34.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.474+0000 7f5848ff9640 1 -- 172.21.15.5:0/2668245865 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5850097890 con 0x558380b844f0 2024-09-12T02:57:34.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.475+0000 7f5862fd8740 1 -- 172.21.15.5:0/2668245865 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558380b844f0 msgr2=0x558380b8e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:34.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.475+0000 7f5862fd8740 1 --2- 172.21.15.5:0/2668245865 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558380b844f0 0x558380b8e870 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f58500a3960 tx=0x7f585009ac80 comp rx=0 tx=0).stop 2024-09-12T02:57:34.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.475+0000 7f5862fd8740 1 -- 172.21.15.5:0/2668245865 shutdown_connections 2024-09-12T02:57:34.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.475+0000 7f5862fd8740 1 --2- 172.21.15.5:0/2668245865 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558380b844f0 0x558380b8e870 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:34.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.475+0000 7f5862fd8740 1 --2- 172.21.15.5:0/2668245865 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558380b8edb0 0x558380b911a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:34.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.475+0000 7f5862fd8740 1 --2- 172.21.15.5:0/2668245865 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558380b83bd0 0x558380b83fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:34.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.475+0000 7f5862fd8740 1 -- 172.21.15.5:0/2668245865 >> 172.21.15.5:0/2668245865 conn(0x558380b80160 msgr2=0x558380b80540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:34.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.475+0000 7f5862fd8740 1 -- 172.21.15.5:0/2668245865 shutdown_connections 2024-09-12T02:57:34.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.475+0000 7f5862fd8740 1 -- 172.21.15.5:0/2668245865 wait complete. 2024-09-12T02:57:34.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.475+0000 7f5862fd8740 1 -- start start 2024-09-12T02:57:34.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.476+0000 7f5862fd8740 1 Processor -- start 2024-09-12T02:57:34.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.476+0000 7f5862fd8740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558380b83bd0 0x558380b8e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:34.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.476+0000 7f5862fd8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558380b844f0 0x558380b8b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:34.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.476+0000 7f5862fd8740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558380b8edb0 0x558380b8be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:34.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.476+0000 7f5862fd8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558380b91980 con 0x558380b844f0 2024-09-12T02:57:34.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.476+0000 7f5862fd8740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x558380aa4f20 con 0x558380b8edb0 2024-09-12T02:57:34.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.476+0000 7f5862fd8740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x558380b91b00 con 0x558380b83bd0 2024-09-12T02:57:34.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.476+0000 7f5862fd8740 10 client.0 _ll_register_callbacks cb 0x558380a6e030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:57:34.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.476+0000 7f5859d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558380b844f0 0x558380b8b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:34.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.476+0000 7f5859d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558380b844f0 0x558380b8b940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51772/0 (socket says 172.21.15.5:51772) 2024-09-12T02:57:34.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.476+0000 7f5859d9b640 1 -- 172.21.15.5:0/2544360752 learned_addr learned my addr 172.21.15.5:0/2544360752 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:57:34.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.476+0000 7f585a59c640 1 --2- 172.21.15.5:0/2544360752 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558380b83bd0 0x558380b8e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:34.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.476+0000 7f585ad9d640 1 --2- 172.21.15.5:0/2544360752 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558380b8edb0 0x558380b8be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:34.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.476+0000 7f5859d9b640 1 -- 172.21.15.5:0/2544360752 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558380b83bd0 msgr2=0x558380b8e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:57:34.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.476+0000 7f5859d9b640 1 --2- 172.21.15.5:0/2544360752 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558380b83bd0 0x558380b8e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:34.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.476+0000 7f5859d9b640 1 -- 172.21.15.5:0/2544360752 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558380b8edb0 msgr2=0x558380b8be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:57:34.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.476+0000 7f5859d9b640 1 --2- 172.21.15.5:0/2544360752 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558380b8edb0 0x558380b8be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:34.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.476+0000 7f5859d9b640 1 -- 172.21.15.5:0/2544360752 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55838085b2b0 con 0x558380b844f0 2024-09-12T02:57:34.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.476+0000 7f585a59c640 1 --2- 172.21.15.5:0/2544360752 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558380b83bd0 0x558380b8e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:57:34.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.476+0000 7f585ad9d640 1 --2- 172.21.15.5:0/2544360752 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558380b8edb0 0x558380b8be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:57:34.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.477+0000 7f5859d9b640 1 --2- 172.21.15.5:0/2544360752 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558380b844f0 0x558380b8b940 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f58500cc550 tx=0x7f5850010470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65902fd0a4273431 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:34.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.477+0000 7f5841ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T02:57:34.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.477+0000 7f5841ffb640 1 -- 172.21.15.5:0/2544360752 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58500b9e60 con 0x558380b844f0 2024-09-12T02:57:34.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.477+0000 7f5841ffb640 1 -- 172.21.15.5:0/2544360752 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f5850097890 con 0x558380b844f0 2024-09-12T02:57:34.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.477+0000 7f5862fd8740 1 -- 172.21.15.5:0/2544360752 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f585009b220 con 0x558380b844f0 2024-09-12T02:57:34.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.477+0000 7f5841ffb640 1 -- 172.21.15.5:0/2544360752 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f58500bb0b0 con 0x558380b844f0 2024-09-12T02:57:34.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.477+0000 7f5862fd8740 10 client.15039 fetch_fsmap learned FSMap version 25 2024-09-12T02:57:34.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.477+0000 7f5862fd8740 1 -- 172.21.15.5:0/2544360752 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558380b91980 con 0x558380b844f0 2024-09-12T02:57:34.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.477+0000 7f5841ffb640 1 -- 172.21.15.5:0/2544360752 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f585009b220 con 0x558380b844f0 2024-09-12T02:57:34.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.477+0000 7f5841ffb640 1 -- 172.21.15.5:0/2544360752 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8362+0+0 (secure 0 0 0) 0x7f585009a670 con 0x558380b844f0 2024-09-12T02:57:34.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.477+0000 7f5862fd8740 10 client.15039 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:57:34.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.477+0000 7f5862fd8740 10 client.15039 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-12T02:57:34.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.477+0000 7f5862fd8740 20 client.15039 populate_metadata read hostname 'smithi005' 2024-09-12T02:57:34.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.477+0000 7f5862fd8740 1 --2- 172.21.15.5:0/2544360752 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x558380b9f350 0x558380ba07c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:34.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.477+0000 7f5862fd8740 4 client.15039 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:57:34.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.477+0000 7f5862fd8740 1 -- 172.21.15.5:0/2544360752 --> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558380af7600 con 0x558380b9f350 2024-09-12T02:57:34.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.478+0000 7f585a59c640 1 --2- 172.21.15.5:0/2544360752 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x558380b9f350 0x558380ba07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:34.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.478+0000 7f585a59c640 1 --2- 172.21.15.5:0/2544360752 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x558380b9f350 0x558380ba07c0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0c600860b840ef6 server_cookie=77ac1de3ab0e12e8 in_seq=0 out_seq=0 2024-09-12T02:57:34.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.478+0000 7f5841ffb640 10 client.15039 ms_handle_connect on v2:172.21.15.148:6838/2317600000 2024-09-12T02:57:34.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.479+0000 7f5841ffb640 1 -- 172.21.15.5:0/2544360752 <== mds.0 v2:172.21.15.148:6838/2317600000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558380af7600 con 0x558380b9f350 2024-09-12T02:57:34.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.479+0000 7f5841ffb640 10 client.15039 handle_command_reply: tid=0 2024-09-12T02:57:34.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.479+0000 7f5862fd8740 1 client.15039 shutdown 2024-09-12T02:57:34.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.479+0000 7f5862fd8740 1 -- 172.21.15.5:0/2544360752 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558380b844f0 msgr2=0x558380b8b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:34.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.479+0000 7f5862fd8740 1 --2- 172.21.15.5:0/2544360752 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558380b844f0 0x558380b8b940 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f58500cc550 tx=0x7f5850010470 comp rx=0 tx=0).stop 2024-09-12T02:57:34.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.479+0000 7f5862fd8740 1 -- 172.21.15.5:0/2544360752 shutdown_connections 2024-09-12T02:57:34.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.479+0000 7f5862fd8740 1 --2- 172.21.15.5:0/2544360752 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x558380b9f350 0x558380ba07c0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:34.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.479+0000 7f5862fd8740 1 --2- 172.21.15.5:0/2544360752 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558380b844f0 0x558380b8b940 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:34.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.479+0000 7f5862fd8740 1 --2- 172.21.15.5:0/2544360752 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558380b8edb0 0x558380b8be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:34.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.479+0000 7f5862fd8740 1 --2- 172.21.15.5:0/2544360752 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558380b83bd0 0x558380b8e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:34.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.479+0000 7f5841ffb640 0 client.15039 ms_handle_reset on v2:172.21.15.148:6838/2317600000 2024-09-12T02:57:34.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.479+0000 7f5862fd8740 1 -- 172.21.15.5:0/2544360752 >> 172.21.15.5:0/2544360752 conn(0x558380b80160 msgr2=0x558380b85e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:34.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.479+0000 7f5862fd8740 1 -- 172.21.15.5:0/2544360752 shutdown_connections 2024-09-12T02:57:34.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.479+0000 7f5862fd8740 1 -- 172.21.15.5:0/2544360752 wait complete. 2024-09-12T02:57:34.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.479+0000 7f5862fd8740 20 client.15039 trim_cache size 0 max 16384 2024-09-12T02:57:34.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.497+0000 7f5862fd8740 1 Processor -- start 2024-09-12T02:57:34.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.497+0000 7f5862fd8740 1 -- start start 2024-09-12T02:57:34.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.497+0000 7f5862fd8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558380b83bd0 0x558380ba07c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:34.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.497+0000 7f5862fd8740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558380b844f0 0x558380b87030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:34.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.497+0000 7f5862fd8740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558380b8edb0 0x558380b7ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:34.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.497+0000 7f5862fd8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558380b9eb00 con 0x558380b83bd0 2024-09-12T02:57:34.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.497+0000 7f5862fd8740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x558380b9f1d0 con 0x558380b844f0 2024-09-12T02:57:34.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.497+0000 7f5862fd8740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x558380b9f8a0 con 0x558380b8edb0 2024-09-12T02:57:34.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.497+0000 7f585a59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558380b83bd0 0x558380ba07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:34.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.497+0000 7f585a59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558380b83bd0 0x558380ba07c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51788/0 (socket says 172.21.15.5:51788) 2024-09-12T02:57:34.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.497+0000 7f585a59c640 1 -- 172.21.15.5:0/513901029 learned_addr learned my addr 172.21.15.5:0/513901029 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:57:34.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.497+0000 7f5859d9b640 1 --2- 172.21.15.5:0/513901029 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558380b844f0 0x558380b87030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:34.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.497+0000 7f585a59c640 1 -- 172.21.15.5:0/513901029 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558380b8edb0 msgr2=0x558380b7ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:57:34.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.497+0000 7f585a59c640 1 --2- 172.21.15.5:0/513901029 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558380b8edb0 0x558380b7ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:34.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.497+0000 7f585a59c640 1 -- 172.21.15.5:0/513901029 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558380b844f0 msgr2=0x558380b87030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:57:34.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.497+0000 7f585a59c640 1 --2- 172.21.15.5:0/513901029 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558380b844f0 0x558380b87030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:34.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.498+0000 7f585a59c640 1 -- 172.21.15.5:0/513901029 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558380b8e960 con 0x558380b83bd0 2024-09-12T02:57:34.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.498+0000 7f5859d9b640 1 --2- 172.21.15.5:0/513901029 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558380b844f0 0x558380b87030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:57:34.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.498+0000 7f585a59c640 1 --2- 172.21.15.5:0/513901029 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558380b83bd0 0x558380ba07c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f584c005e10 tx=0x7f584c00ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=989e6d782fd2b66d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:34.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.498+0000 7f5843fff640 1 -- 172.21.15.5:0/513901029 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f584c014020 con 0x558380b83bd0 2024-09-12T02:57:34.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.498+0000 7f5843fff640 1 -- 172.21.15.5:0/513901029 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f584c003c40 con 0x558380b83bd0 2024-09-12T02:57:34.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.498+0000 7f5843fff640 1 -- 172.21.15.5:0/513901029 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f584c010720 con 0x558380b83bd0 2024-09-12T02:57:34.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.498+0000 7f5862fd8740 1 -- 172.21.15.5:0/513901029 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558380b83bd0 msgr2=0x558380ba07c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:34.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.498+0000 7f5862fd8740 1 --2- 172.21.15.5:0/513901029 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558380b83bd0 0x558380ba07c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f584c005e10 tx=0x7f584c00ccf0 comp rx=0 tx=0).stop 2024-09-12T02:57:34.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.498+0000 7f5862fd8740 1 -- 172.21.15.5:0/513901029 shutdown_connections 2024-09-12T02:57:34.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.498+0000 7f5862fd8740 1 --2- 172.21.15.5:0/513901029 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558380b8edb0 0x558380b7ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:34.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.498+0000 7f5862fd8740 1 --2- 172.21.15.5:0/513901029 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558380b844f0 0x558380b87030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:34.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.498+0000 7f5862fd8740 1 --2- 172.21.15.5:0/513901029 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558380b83bd0 0x558380ba07c0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:34.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.498+0000 7f5862fd8740 1 -- 172.21.15.5:0/513901029 >> 172.21.15.5:0/513901029 conn(0x558380b80160 msgr2=0x558380b7e810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:34.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.498+0000 7f5862fd8740 1 -- 172.21.15.5:0/513901029 shutdown_connections 2024-09-12T02:57:34.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.499+0000 7f5862fd8740 1 -- 172.21.15.5:0/513901029 wait complete. 2024-09-12T02:57:34.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.499+0000 7f5862fd8740 1 -- start start 2024-09-12T02:57:34.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.499+0000 7f5862fd8740 1 Processor -- start 2024-09-12T02:57:34.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.499+0000 7f5862fd8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558380b83bd0 0x558380ba2b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:34.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.499+0000 7f5862fd8740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558380b844f0 0x558380ba3060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:34.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.499+0000 7f5862fd8740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558380b8edb0 0x558380ba35e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:34.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.499+0000 7f5862fd8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558380ba0640 con 0x558380b83bd0 2024-09-12T02:57:34.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.499+0000 7f5862fd8740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x558380b9ff70 con 0x558380b844f0 2024-09-12T02:57:34.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.499+0000 7f5862fd8740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x558380ba0d10 con 0x558380b8edb0 2024-09-12T02:57:34.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.499+0000 7f5862fd8740 10 client.0 _ll_register_callbacks cb 0x7f584c00c350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:57:34.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.499+0000 7f585a59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558380b83bd0 0x558380ba2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:34.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.499+0000 7f585a59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558380b83bd0 0x558380ba2b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51796/0 (socket says 172.21.15.5:51796) 2024-09-12T02:57:34.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.499+0000 7f585a59c640 1 -- 172.21.15.5:0/1703934248 learned_addr learned my addr 172.21.15.5:0/1703934248 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:57:34.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.500+0000 7f585ad9d640 1 --2- 172.21.15.5:0/1703934248 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558380b8edb0 0x558380ba35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:34.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.500+0000 7f585a59c640 1 -- 172.21.15.5:0/1703934248 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558380b8edb0 msgr2=0x558380ba35e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:57:34.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.500+0000 7f585a59c640 1 --2- 172.21.15.5:0/1703934248 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558380b8edb0 0x558380ba35e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:34.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.500+0000 7f5859d9b640 1 --2- 172.21.15.5:0/1703934248 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558380b844f0 0x558380ba3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:34.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.500+0000 7f585a59c640 1 -- 172.21.15.5:0/1703934248 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558380b844f0 msgr2=0x558380ba3060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:57:34.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.500+0000 7f585a59c640 1 --2- 172.21.15.5:0/1703934248 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558380b844f0 0x558380ba3060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:34.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.500+0000 7f585a59c640 1 -- 172.21.15.5:0/1703934248 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558380ba75b0 con 0x558380b83bd0 2024-09-12T02:57:34.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.500+0000 7f585ad9d640 1 --2- 172.21.15.5:0/1703934248 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558380b8edb0 0x558380ba35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:57:34.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.500+0000 7f585a59c640 1 --2- 172.21.15.5:0/1703934248 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558380b83bd0 0x558380ba2b20 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f584c00c790 tx=0x7f584c003b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3773b831beb590a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:34.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.500+0000 7f5841ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T02:57:34.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.500+0000 7f5841ffb640 1 -- 172.21.15.5:0/1703934248 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f584c01c960 con 0x558380b83bd0 2024-09-12T02:57:34.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.500+0000 7f5841ffb640 1 -- 172.21.15.5:0/1703934248 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f584c01cb00 con 0x558380b83bd0 2024-09-12T02:57:34.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.500+0000 7f5862fd8740 1 -- 172.21.15.5:0/1703934248 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f584c003c40 con 0x558380b83bd0 2024-09-12T02:57:34.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.500+0000 7f5841ffb640 1 -- 172.21.15.5:0/1703934248 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f584c0104a0 con 0x558380b83bd0 2024-09-12T02:57:34.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.500+0000 7f5862fd8740 10 client.15045 fetch_fsmap learned FSMap version 25 2024-09-12T02:57:34.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.500+0000 7f5862fd8740 1 -- 172.21.15.5:0/1703934248 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558380c09fc0 con 0x558380b83bd0 2024-09-12T02:57:34.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.500+0000 7f5841ffb640 1 -- 172.21.15.5:0/1703934248 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f584c003c40 con 0x558380b83bd0 2024-09-12T02:57:34.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.500+0000 7f5841ffb640 1 -- 172.21.15.5:0/1703934248 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8362+0+0 (secure 0 0 0) 0x7f584c027950 con 0x558380b83bd0 2024-09-12T02:57:34.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.501+0000 7f5862fd8740 10 client.15045 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:57:34.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.501+0000 7f5862fd8740 10 client.15045 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-12T02:57:34.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.501+0000 7f5862fd8740 20 client.15045 populate_metadata read hostname 'smithi005' 2024-09-12T02:57:34.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.501+0000 7f5862fd8740 1 --2- 172.21.15.5:0/1703934248 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x558380c0e740 0x558380c0d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:34.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.501+0000 7f5862fd8740 4 client.15045 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T02:57:34.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.501+0000 7f5862fd8740 1 -- 172.21.15.5:0/1703934248 --> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] -- command(tid 0: {"prefix": "status"}) -- 0x558380af1b10 con 0x558380c0e740 2024-09-12T02:57:34.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.501+0000 7f5859d9b640 1 --2- 172.21.15.5:0/1703934248 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x558380c0e740 0x558380c0d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:34.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.501+0000 7f5859d9b640 1 --2- 172.21.15.5:0/1703934248 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x558380c0e740 0x558380c0d450 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2158f384288b7d1 server_cookie=bf2e86771244d657 in_seq=0 out_seq=0 2024-09-12T02:57:34.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.501+0000 7f5841ffb640 10 client.15045 ms_handle_connect on v2:172.21.15.148:6838/2317600000 2024-09-12T02:57:34.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.502+0000 7f5841ffb640 1 -- 172.21.15.5:0/1703934248 <== mds.0 v2:172.21.15.148:6838/2317600000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x558380af1b10 con 0x558380c0e740 2024-09-12T02:57:34.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.502+0000 7f5841ffb640 10 client.15045 handle_command_reply: tid=0 2024-09-12T02:57:34.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.502+0000 7f5862fd8740 1 client.15045 shutdown 2024-09-12T02:57:34.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.502+0000 7f5862fd8740 1 -- 172.21.15.5:0/1703934248 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558380b83bd0 msgr2=0x558380ba2b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:34.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.502+0000 7f5862fd8740 1 --2- 172.21.15.5:0/1703934248 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558380b83bd0 0x558380ba2b20 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f584c00c790 tx=0x7f584c003b80 comp rx=0 tx=0).stop 2024-09-12T02:57:34.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.502+0000 7f5862fd8740 1 -- 172.21.15.5:0/1703934248 shutdown_connections 2024-09-12T02:57:34.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.502+0000 7f5862fd8740 1 --2- 172.21.15.5:0/1703934248 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558380b8edb0 0x558380ba35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:34.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.502+0000 7f5862fd8740 1 --2- 172.21.15.5:0/1703934248 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558380b844f0 0x558380ba3060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:34.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.502+0000 7f5862fd8740 1 --2- 172.21.15.5:0/1703934248 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x558380c0e740 0x558380c0d450 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:34.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.502+0000 7f5862fd8740 1 --2- 172.21.15.5:0/1703934248 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558380b83bd0 0x558380ba2b20 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:34.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.502+0000 7f5862fd8740 1 -- 172.21.15.5:0/1703934248 >> 172.21.15.5:0/1703934248 conn(0x558380b80160 msgr2=0x558380b7e810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:34.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.502+0000 7f5841ffb640 0 client.15045 ms_handle_reset on v2:172.21.15.148:6838/2317600000 2024-09-12T02:57:34.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.502+0000 7f5862fd8740 1 -- 172.21.15.5:0/1703934248 shutdown_connections 2024-09-12T02:57:34.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.502+0000 7f5862fd8740 1 -- 172.21.15.5:0/1703934248 wait complete. 2024-09-12T02:57:34.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.502+0000 7f5862fd8740 20 client.15045 trim_cache size 0 max 16384 2024-09-12T02:57:34.510 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-12T02:57:34.510 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "618305f8-70b1-11ef-bcea-c7b262605968", 2024-09-12T02:57:34.510 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 0, 2024-09-12T02:57:34.511 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24502, 2024-09-12T02:57:34.511 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-09-12T02:57:34.511 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-09-12T02:57:34.511 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-09-12T02:57:34.511 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 77.530677287000003, 2024-09-12T02:57:34.511 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 25, 2024-09-12T02:57:34.511 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 83, 2024-09-12T02:57:34.511 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 81, 2024-09-12T02:57:34.511 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 96.836373730000005 2024-09-12T02:57:34.511 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-12T02:57:34.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.503+0000 7f585b59e640 1 -- 172.21.15.5:0/1632504792 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f58440a2c10 msgr2=0x7f58440a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:34.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.503+0000 7f585b59e640 1 --2- 172.21.15.5:0/1632504792 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f58440a2c10 0x7f58440a50d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f5854009430 tx=0x7f5854005ef0 comp rx=0 tx=0).stop 2024-09-12T02:57:34.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.503+0000 7f585b59e640 1 -- 172.21.15.5:0/1632504792 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f585c1034b0 msgr2=0x7f585c198780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:34.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.503+0000 7f585b59e640 1 --2- 172.21.15.5:0/1632504792 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f585c1034b0 0x7f585c198780 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f585000b3f0 tx=0x7f585000b8b0 comp rx=0 tx=0).stop 2024-09-12T02:57:34.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.503+0000 7f585b59e640 1 -- 172.21.15.5:0/1632504792 shutdown_connections 2024-09-12T02:57:34.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.503+0000 7f585b59e640 1 --2- 172.21.15.5:0/1632504792 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f58440a2c10 0x7f58440a50d0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:34.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.503+0000 7f585b59e640 1 --2- 172.21.15.5:0/1632504792 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f585c1034b0 0x7f585c198780 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:34.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.503+0000 7f585b59e640 1 --2- 172.21.15.5:0/1632504792 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f585c10bfc0 0x7f585c19cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:34.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.503+0000 7f585b59e640 1 --2- 172.21.15.5:0/1632504792 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f585c102b90 0x7f585c198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:34.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.503+0000 7f585b59e640 1 -- 172.21.15.5:0/1632504792 >> 172.21.15.5:0/1632504792 conn(0x7f585c0fc630 msgr2=0x7f585c0fd230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:34.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.503+0000 7f585b59e640 1 -- 172.21.15.5:0/1632504792 shutdown_connections 2024-09-12T02:57:34.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:34.503+0000 7f585b59e640 1 -- 172.21.15.5:0/1632504792 wait complete. 2024-09-12T02:57:35.218 INFO:tasks.cephadm:Setup complete, yielding 2024-09-12T02:57:35.218 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-12T02:57:35.231 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi005.front.sepia.ceph.com 2024-09-12T02:57:35.231 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- bash -c 'ceph orch status' 2024-09-12T02:57:35.496 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:57:35.511 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:57:35 smithi005 ceph-mon[31197]: pgmap v315: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-12T02:57:35.665 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:57:35 smithi148 ceph-mon[35090]: pgmap v315: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-12T02:57:35.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:57:35 smithi146 ceph-mon[35011]: pgmap v315: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-12T02:57:35.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:35.935+0000 7f18a108e640 1 -- 172.21.15.5:0/63906344 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f189c102eb0 msgr2=0x7f189c103330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:35.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:35.935+0000 7f18a108e640 1 --2- 172.21.15.5:0/63906344 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f189c102eb0 0x7f189c103330 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f1890009930 tx=0x7f189002a420 comp rx=0 tx=0).stop 2024-09-12T02:57:35.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:35.935+0000 7f18a108e640 1 -- 172.21.15.5:0/63906344 shutdown_connections 2024-09-12T02:57:35.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:35.935+0000 7f18a108e640 1 --2- 172.21.15.5:0/63906344 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f189c102eb0 0x7f189c103330 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:35.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:35.935+0000 7f18a108e640 1 --2- 172.21.15.5:0/63906344 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f189c103870 0x7f189c10a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:35.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:35.935+0000 7f18a108e640 1 --2- 172.21.15.5:0/63906344 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f189c0ffed0 0x7f189c1002d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:35.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:35.935+0000 7f18a108e640 1 -- 172.21.15.5:0/63906344 >> 172.21.15.5:0/63906344 conn(0x7f189c0fba40 msgr2=0x7f189c0fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:35.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:35.935+0000 7f18a108e640 1 -- 172.21.15.5:0/63906344 shutdown_connections 2024-09-12T02:57:35.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:35.935+0000 7f18a108e640 1 -- 172.21.15.5:0/63906344 wait complete. 2024-09-12T02:57:35.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:35.936+0000 7f18a108e640 1 Processor -- start 2024-09-12T02:57:35.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:35.936+0000 7f18a108e640 1 -- start start 2024-09-12T02:57:35.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:35.936+0000 7f18a108e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f189c0ffed0 0x7f189c102810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:35.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:35.936+0000 7f18a108e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f189c102eb0 0x7f189c100e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:35.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:35.936+0000 7f18a108e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f189c103870 0x7f189c1013a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:35.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:35.936+0000 7f18a108e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f189c10d8b0 con 0x7f189c0ffed0 2024-09-12T02:57:35.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:35.936+0000 7f18a108e640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f189c10d730 con 0x7f189c103870 2024-09-12T02:57:35.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:35.936+0000 7f18a108e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f189c10da30 con 0x7f189c102eb0 2024-09-12T02:57:35.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:35.937+0000 7f189ad9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f189c0ffed0 0x7f189c102810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:35.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:35.937+0000 7f189ad9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f189c0ffed0 0x7f189c102810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51820/0 (socket says 172.21.15.5:51820) 2024-09-12T02:57:35.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:35.937+0000 7f189ad9d640 1 -- 172.21.15.5:0/1869169807 learned_addr learned my addr 172.21.15.5:0/1869169807 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:57:35.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:35.937+0000 7f189a59c640 1 --2- 172.21.15.5:0/1869169807 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f189c102eb0 0x7f189c100e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:35.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:35.937+0000 7f189b59e640 1 --2- 172.21.15.5:0/1869169807 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f189c103870 0x7f189c1013a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:35.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:35.937+0000 7f189ad9d640 1 -- 172.21.15.5:0/1869169807 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f189c102eb0 msgr2=0x7f189c100e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:35.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:35.937+0000 7f189ad9d640 1 --2- 172.21.15.5:0/1869169807 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f189c102eb0 0x7f189c100e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:35.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:35.937+0000 7f189ad9d640 1 -- 172.21.15.5:0/1869169807 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f189c103870 msgr2=0x7f189c1013a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:35.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:35.937+0000 7f189ad9d640 1 --2- 172.21.15.5:0/1869169807 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f189c103870 0x7f189c1013a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:35.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:35.937+0000 7f189ad9d640 1 -- 172.21.15.5:0/1869169807 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f189c101c60 con 0x7f189c0ffed0 2024-09-12T02:57:35.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:35.937+0000 7f189b59e640 1 --2- 172.21.15.5:0/1869169807 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f189c103870 0x7f189c1013a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:57:35.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:35.937+0000 7f189ad9d640 1 --2- 172.21.15.5:0/1869169807 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f189c0ffed0 0x7f189c102810 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f188c00c970 tx=0x7f188c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:35.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:35.937+0000 7f1883fff640 1 -- 172.21.15.5:0/1869169807 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f188c004220 con 0x7f189c0ffed0 2024-09-12T02:57:35.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:35.937+0000 7f1883fff640 1 -- 172.21.15.5:0/1869169807 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f188c0043c0 con 0x7f189c0ffed0 2024-09-12T02:57:35.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:35.937+0000 7f18a108e640 1 -- 172.21.15.5:0/1869169807 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f189c1a2180 con 0x7f189c0ffed0 2024-09-12T02:57:35.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:35.937+0000 7f1883fff640 1 -- 172.21.15.5:0/1869169807 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f188c004a00 con 0x7f189c0ffed0 2024-09-12T02:57:35.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:35.937+0000 7f18a108e640 1 -- 172.21.15.5:0/1869169807 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f189c1a2640 con 0x7f189c0ffed0 2024-09-12T02:57:35.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:35.938+0000 7f18a108e640 1 -- 172.21.15.5:0/1869169807 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f189c1091a0 con 0x7f189c0ffed0 2024-09-12T02:57:35.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:35.940+0000 7f1883fff640 1 -- 172.21.15.5:0/1869169807 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f188c0079b0 con 0x7f189c0ffed0 2024-09-12T02:57:35.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:35.940+0000 7f1883fff640 1 --2- 172.21.15.5:0/1869169807 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f18840a2c10 0x7f18840a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:35.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:35.940+0000 7f189a59c640 1 --2- 172.21.15.5:0/1869169807 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f18840a2c10 0x7f18840a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:35.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:35.941+0000 7f1883fff640 1 -- 172.21.15.5:0/1869169807 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f188c0cf350 con 0x7f189c0ffed0 2024-09-12T02:57:35.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:35.941+0000 7f189a59c640 1 --2- 172.21.15.5:0/1869169807 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f18840a2c10 0x7f18840a50d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1890006010 tx=0x7f18900315b0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:35.955 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:35.944+0000 7f1883fff640 1 -- 172.21.15.5:0/1869169807 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f188c014030 con 0x7f189c0ffed0 2024-09-12T02:57:36.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:36.141+0000 7f18a108e640 1 -- 172.21.15.5:0/1869169807 --> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f189c062630 con 0x7f18840a2c10 2024-09-12T02:57:36.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:36.142+0000 7f1883fff640 1 -- 172.21.15.5:0/1869169807 <== mgr.14148 v2:172.21.15.5:6800/3780631642 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f189c062630 con 0x7f18840a2c10 2024-09-12T02:57:36.142 INFO:teuthology.orchestra.run.smithi005.stdout:Backend: cephadm 2024-09-12T02:57:36.142 INFO:teuthology.orchestra.run.smithi005.stdout:Available: Yes 2024-09-12T02:57:36.142 INFO:teuthology.orchestra.run.smithi005.stdout:Paused: No 2024-09-12T02:57:36.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:36.144+0000 7f18a108e640 1 -- 172.21.15.5:0/1869169807 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f18840a2c10 msgr2=0x7f18840a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:36.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:36.144+0000 7f18a108e640 1 --2- 172.21.15.5:0/1869169807 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f18840a2c10 0x7f18840a50d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1890006010 tx=0x7f18900315b0 comp rx=0 tx=0).stop 2024-09-12T02:57:36.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:36.144+0000 7f18a108e640 1 -- 172.21.15.5:0/1869169807 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f189c0ffed0 msgr2=0x7f189c102810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:36.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:36.144+0000 7f18a108e640 1 --2- 172.21.15.5:0/1869169807 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f189c0ffed0 0x7f189c102810 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f188c00c970 tx=0x7f188c00ce30 comp rx=0 tx=0).stop 2024-09-12T02:57:36.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:36.144+0000 7f18a108e640 1 -- 172.21.15.5:0/1869169807 shutdown_connections 2024-09-12T02:57:36.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:36.144+0000 7f18a108e640 1 --2- 172.21.15.5:0/1869169807 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f189c103870 0x7f189c1013a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:36.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:36.144+0000 7f18a108e640 1 --2- 172.21.15.5:0/1869169807 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f189c102eb0 0x7f189c100e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:36.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:36.144+0000 7f18a108e640 1 --2- 172.21.15.5:0/1869169807 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f18840a2c10 0x7f18840a50d0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:36.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:36.144+0000 7f18a108e640 1 --2- 172.21.15.5:0/1869169807 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f189c0ffed0 0x7f189c102810 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:36.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:36.144+0000 7f18a108e640 1 -- 172.21.15.5:0/1869169807 >> 172.21.15.5:0/1869169807 conn(0x7f189c0fba40 msgr2=0x7f189c106370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:36.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:36.144+0000 7f18a108e640 1 -- 172.21.15.5:0/1869169807 shutdown_connections 2024-09-12T02:57:36.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:36.144+0000 7f18a108e640 1 -- 172.21.15.5:0/1869169807 wait complete. 2024-09-12T02:57:36.803 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- bash -c 'ceph orch ps' 2024-09-12T02:57:37.064 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:57:37.451 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:57:37 smithi005 ceph-mon[31197]: pgmap v316: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-12T02:57:37.451 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:57:37 smithi005 ceph-mon[31197]: from='client.15051 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:57:37.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:37.533+0000 7fdb486a5640 1 -- 172.21.15.5:0/3476772142 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb40101cd0 msgr2=0x7fdb40102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:37.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:37.533+0000 7fdb486a5640 1 --2- 172.21.15.5:0/3476772142 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb40101cd0 0x7fdb40102150 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fdb30009900 tx=0x7fdb3002a410 comp rx=0 tx=0).stop 2024-09-12T02:57:37.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:37.533+0000 7fdb486a5640 1 -- 172.21.15.5:0/3476772142 shutdown_connections 2024-09-12T02:57:37.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:37.533+0000 7fdb486a5640 1 --2- 172.21.15.5:0/3476772142 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb40101cd0 0x7fdb40102150 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:37.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:37.533+0000 7fdb486a5640 1 --2- 172.21.15.5:0/3476772142 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdb40102690 0x7fdb40108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:37.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:37.533+0000 7fdb486a5640 1 --2- 172.21.15.5:0/3476772142 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdb40100ad0 0x7fdb40100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:37.535 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:37.533+0000 7fdb486a5640 1 -- 172.21.15.5:0/3476772142 >> 172.21.15.5:0/3476772142 conn(0x7fdb400fc280 msgr2=0x7fdb400fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:37.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:37.533+0000 7fdb486a5640 1 -- 172.21.15.5:0/3476772142 shutdown_connections 2024-09-12T02:57:37.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:37.533+0000 7fdb486a5640 1 -- 172.21.15.5:0/3476772142 wait complete. 2024-09-12T02:57:37.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:37.533+0000 7fdb486a5640 1 Processor -- start 2024-09-12T02:57:37.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:37.533+0000 7fdb486a5640 1 -- start start 2024-09-12T02:57:37.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:37.534+0000 7fdb486a5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdb40100ad0 0x7fdb40197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:37.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:37.534+0000 7fdb486a5640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdb40101cd0 0x7fdb401984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:37.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:37.534+0000 7fdb486a5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb40102690 0x7fdb4019f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:37.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:37.534+0000 7fdb486a5640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdb4010c7b0 con 0x7fdb40102690 2024-09-12T02:57:37.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:37.534+0000 7fdb486a5640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fdb4010c630 con 0x7fdb40101cd0 2024-09-12T02:57:37.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:37.534+0000 7fdb486a5640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fdb4010c930 con 0x7fdb40100ad0 2024-09-12T02:57:37.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:37.534+0000 7fdb46c42640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb40102690 0x7fdb4019f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:37.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:37.534+0000 7fdb46c42640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb40102690 0x7fdb4019f540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51838/0 (socket says 172.21.15.5:51838) 2024-09-12T02:57:37.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:37.534+0000 7fdb46c42640 1 -- 172.21.15.5:0/3145936127 learned_addr learned my addr 172.21.15.5:0/3145936127 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:57:37.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:37.534+0000 7fdb45c40640 1 --2- 172.21.15.5:0/3145936127 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdb40101cd0 0x7fdb401984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:37.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:37.534+0000 7fdb46441640 1 --2- 172.21.15.5:0/3145936127 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdb40100ad0 0x7fdb40197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:37.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:37.534+0000 7fdb46c42640 1 -- 172.21.15.5:0/3145936127 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdb40100ad0 msgr2=0x7fdb40197f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:37.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:37.534+0000 7fdb46c42640 1 --2- 172.21.15.5:0/3145936127 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdb40100ad0 0x7fdb40197f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:37.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:37.534+0000 7fdb46c42640 1 -- 172.21.15.5:0/3145936127 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdb40101cd0 msgr2=0x7fdb401984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:37.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:37.534+0000 7fdb46c42640 1 --2- 172.21.15.5:0/3145936127 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdb40101cd0 0x7fdb401984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:37.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:37.534+0000 7fdb46c42640 1 -- 172.21.15.5:0/3145936127 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb4019fbb0 con 0x7fdb40102690 2024-09-12T02:57:37.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:37.534+0000 7fdb46c42640 1 --2- 172.21.15.5:0/3145936127 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb40102690 0x7fdb4019f540 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fdb3800b970 tx=0x7fdb3800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:37.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:37.535+0000 7fdb377fe640 1 -- 172.21.15.5:0/3145936127 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb38019070 con 0x7fdb40102690 2024-09-12T02:57:37.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:37.535+0000 7fdb377fe640 1 -- 172.21.15.5:0/3145936127 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb3800cb40 con 0x7fdb40102690 2024-09-12T02:57:37.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:37.535+0000 7fdb486a5640 1 -- 172.21.15.5:0/3145936127 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb4019fea0 con 0x7fdb40102690 2024-09-12T02:57:37.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:37.535+0000 7fdb377fe640 1 -- 172.21.15.5:0/3145936127 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb38007610 con 0x7fdb40102690 2024-09-12T02:57:37.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:37.535+0000 7fdb486a5640 1 -- 172.21.15.5:0/3145936127 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb401a03d0 con 0x7fdb40102690 2024-09-12T02:57:37.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:37.536+0000 7fdb486a5640 1 -- 172.21.15.5:0/3145936127 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb401a06b0 con 0x7fdb40102690 2024-09-12T02:57:37.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:37.537+0000 7fdb377fe640 1 -- 172.21.15.5:0/3145936127 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fdb3801c070 con 0x7fdb40102690 2024-09-12T02:57:37.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:37.537+0000 7fdb377fe640 1 --2- 172.21.15.5:0/3145936127 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fdb2c0a2c40 0x7fdb2c0a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:37.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:37.538+0000 7fdb46441640 1 --2- 172.21.15.5:0/3145936127 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fdb2c0a2c40 0x7fdb2c0a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:37.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:37.538+0000 7fdb377fe640 1 -- 172.21.15.5:0/3145936127 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fdb380ce610 con 0x7fdb40102690 2024-09-12T02:57:37.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:37.538+0000 7fdb46441640 1 --2- 172.21.15.5:0/3145936127 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fdb2c0a2c40 0x7fdb2c0a5100 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c005db0 tx=0x7fdb3c005d40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:37.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:37.542+0000 7fdb377fe640 1 -- 172.21.15.5:0/3145936127 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdb3809aee0 con 0x7fdb40102690 2024-09-12T02:57:37.665 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:57:37 smithi148 ceph-mon[35090]: pgmap v316: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-12T02:57:37.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:57:37 smithi148 ceph-mon[35090]: from='client.15051 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:57:37.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:57:37 smithi146 ceph-mon[35011]: pgmap v316: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-12T02:57:37.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:57:37 smithi146 ceph-mon[35011]: from='client.15051 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:57:37.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:37.725+0000 7fdb486a5640 1 -- 172.21.15.5:0/3145936127 --> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7fdb401a0880 con 0x7fdb2c0a2c40 2024-09-12T02:57:37.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:37.737+0000 7fdb377fe640 1 -- 172.21.15.5:0/3145936127 <== mgr.14148 v2:172.21.15.5:6800/3780631642 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4154 (secure 0 0 0) 0x7fdb401a0880 con 0x7fdb2c0a2c40 2024-09-12T02:57:37.737 INFO:teuthology.orchestra.run.smithi005.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-12T02:57:37.738 INFO:teuthology.orchestra.run.smithi005.stdout:mds.a smithi005 running (94s) 19s ago 94s 19.2M - 19.1.1-298-gb53b6d3e 54467abc55f8 be62dcbdf775 2024-09-12T02:57:37.738 INFO:teuthology.orchestra.run.smithi005.stdout:mds.b smithi146 running (91s) 24s ago 91s 17.7M - 19.1.1-298-gb53b6d3e 54467abc55f8 4dd5146b2890 2024-09-12T02:57:37.738 INFO:teuthology.orchestra.run.smithi005.stdout:mds.c smithi148 running (105s) 24s ago 105s 19.4M - 19.1.1-298-gb53b6d3e 54467abc55f8 3d621945b9f5 2024-09-12T02:57:37.738 INFO:teuthology.orchestra.run.smithi005.stdout:mds.d smithi005 running (112s) 19s ago 112s 16.6M - 19.1.1-298-gb53b6d3e 54467abc55f8 eaf2f38083f9 2024-09-12T02:57:37.738 INFO:teuthology.orchestra.run.smithi005.stdout:mds.e smithi146 running (110s) 24s ago 109s 19.5M - 19.1.1-298-gb53b6d3e 54467abc55f8 c6bdd085a6c1 2024-09-12T02:57:37.738 INFO:teuthology.orchestra.run.smithi005.stdout:mds.f smithi148 running (107s) 24s ago 107s 17.2M - 19.1.1-298-gb53b6d3e 54467abc55f8 3cb01b1b23a3 2024-09-12T02:57:37.738 INFO:teuthology.orchestra.run.smithi005.stdout:mds.g smithi005 running (97s) 19s ago 97s 19.2M - 19.1.1-298-gb53b6d3e 54467abc55f8 f37068bb74e3 2024-09-12T02:57:37.738 INFO:teuthology.orchestra.run.smithi005.stdout:mds.h smithi146 running (102s) 24s ago 102s 19.0M - 19.1.1-298-gb53b6d3e 54467abc55f8 f42ddd7f5459 2024-09-12T02:57:37.738 INFO:teuthology.orchestra.run.smithi005.stdout:mds.i smithi148 running (100s) 24s ago 99s 28.8M - 19.1.1-298-gb53b6d3e 54467abc55f8 6c536fc6aff2 2024-09-12T02:57:37.738 INFO:teuthology.orchestra.run.smithi005.stdout:mds.j smithi005 running (116s) 19s ago 115s 14.4M - 19.1.1-298-gb53b6d3e 54467abc55f8 6307c2f6d51a 2024-09-12T02:57:37.738 INFO:teuthology.orchestra.run.smithi005.stdout:mds.k smithi146 running (119s) 24s ago 119s 18.5M - 19.1.1-298-gb53b6d3e 54467abc55f8 889f13ae82d1 2024-09-12T02:57:37.738 INFO:teuthology.orchestra.run.smithi005.stdout:mds.l smithi148 running (2m) 24s ago 2m 19.2M - 19.1.1-298-gb53b6d3e 54467abc55f8 d3975d960a31 2024-09-12T02:57:37.739 INFO:teuthology.orchestra.run.smithi005.stdout:mgr.x smithi005 *:9283,8765 running (8m) 19s ago 8m 509M - 19.1.1-298-gb53b6d3e 54467abc55f8 929a8c33032e 2024-09-12T02:57:37.739 INFO:teuthology.orchestra.run.smithi005.stdout:mgr.y smithi146 *:8443,8765 running (7m) 24s ago 7m 442M - 19.1.1-298-gb53b6d3e 54467abc55f8 44071781cc32 2024-09-12T02:57:37.739 INFO:teuthology.orchestra.run.smithi005.stdout:mgr.z smithi148 *:8443,8765 running (7m) 24s ago 7m 441M - 19.1.1-298-gb53b6d3e 54467abc55f8 c05a997c2d4f 2024-09-12T02:57:37.739 INFO:teuthology.orchestra.run.smithi005.stdout:mon.a smithi005 running (8m) 19s ago 8m 50.6M 2048M 19.1.1-298-gb53b6d3e 54467abc55f8 62861099a2ac 2024-09-12T02:57:37.739 INFO:teuthology.orchestra.run.smithi005.stdout:mon.b smithi146 running (7m) 24s ago 7m 40.4M 2048M 19.1.1-298-gb53b6d3e 54467abc55f8 50219fda65c4 2024-09-12T02:57:37.739 INFO:teuthology.orchestra.run.smithi005.stdout:mon.c smithi148 running (7m) 24s ago 7m 40.5M 2048M 19.1.1-298-gb53b6d3e 54467abc55f8 faab55de658f 2024-09-12T02:57:37.739 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0 smithi005 running (6m) 19s ago 6m 52.5M 4273M 19.1.1-298-gb53b6d3e 54467abc55f8 67834bd03595 2024-09-12T02:57:37.739 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1 smithi146 running (6m) 24s ago 6m 48.3M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 d193e2b4cafe 2024-09-12T02:57:37.739 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2 smithi148 running (6m) 24s ago 6m 57.0M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 a58b36cf2e8c 2024-09-12T02:57:37.739 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3 smithi005 running (5m) 19s ago 5m 49.9M 4273M 19.1.1-298-gb53b6d3e 54467abc55f8 f15ebd611d25 2024-09-12T02:57:37.739 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4 smithi146 running (5m) 24s ago 5m 50.6M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 b7728f115085 2024-09-12T02:57:37.739 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5 smithi148 running (4m) 24s ago 4m 46.0M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 3b01fb95287a 2024-09-12T02:57:37.739 INFO:teuthology.orchestra.run.smithi005.stdout:osd.6 smithi005 running (4m) 19s ago 4m 47.1M 4273M 19.1.1-298-gb53b6d3e 54467abc55f8 f3c8dc1cccdc 2024-09-12T02:57:37.739 INFO:teuthology.orchestra.run.smithi005.stdout:osd.7 smithi146 running (3m) 24s ago 3m 48.9M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 f735ae846010 2024-09-12T02:57:37.740 INFO:teuthology.orchestra.run.smithi005.stdout:osd.8 smithi148 running (3m) 24s ago 3m 54.3M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 f89cd9d61bd2 2024-09-12T02:57:37.740 INFO:teuthology.orchestra.run.smithi005.stdout:osd.9 smithi005 running (3m) 19s ago 3m 50.8M 4273M 19.1.1-298-gb53b6d3e 54467abc55f8 140fd33a3c1b 2024-09-12T02:57:37.740 INFO:teuthology.orchestra.run.smithi005.stdout:osd.10 smithi146 running (2m) 24s ago 2m 53.3M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 d1f11e3bd1e2 2024-09-12T02:57:37.740 INFO:teuthology.orchestra.run.smithi005.stdout:osd.11 smithi148 running (2m) 24s ago 2m 51.2M 4272M 19.1.1-298-gb53b6d3e 54467abc55f8 26d68867ac17 2024-09-12T02:57:37.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:37.740+0000 7fdb486a5640 1 -- 172.21.15.5:0/3145936127 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fdb2c0a2c40 msgr2=0x7fdb2c0a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:37.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:37.740+0000 7fdb486a5640 1 --2- 172.21.15.5:0/3145936127 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fdb2c0a2c40 0x7fdb2c0a5100 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c005db0 tx=0x7fdb3c005d40 comp rx=0 tx=0).stop 2024-09-12T02:57:37.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:37.740+0000 7fdb486a5640 1 -- 172.21.15.5:0/3145936127 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb40102690 msgr2=0x7fdb4019f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:37.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:37.740+0000 7fdb486a5640 1 --2- 172.21.15.5:0/3145936127 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb40102690 0x7fdb4019f540 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fdb3800b970 tx=0x7fdb3800be30 comp rx=0 tx=0).stop 2024-09-12T02:57:37.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:37.740+0000 7fdb486a5640 1 -- 172.21.15.5:0/3145936127 shutdown_connections 2024-09-12T02:57:37.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:37.740+0000 7fdb486a5640 1 --2- 172.21.15.5:0/3145936127 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fdb2c0a2c40 0x7fdb2c0a5100 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:37.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:37.740+0000 7fdb486a5640 1 --2- 172.21.15.5:0/3145936127 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdb40102690 0x7fdb4019f540 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:37.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:37.740+0000 7fdb486a5640 1 --2- 172.21.15.5:0/3145936127 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdb40101cd0 0x7fdb401984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:37.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:37.740+0000 7fdb486a5640 1 --2- 172.21.15.5:0/3145936127 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdb40100ad0 0x7fdb40197f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:37.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:37.740+0000 7fdb486a5640 1 -- 172.21.15.5:0/3145936127 >> 172.21.15.5:0/3145936127 conn(0x7fdb400fc280 msgr2=0x7fdb40069fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:37.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:37.740+0000 7fdb486a5640 1 -- 172.21.15.5:0/3145936127 shutdown_connections 2024-09-12T02:57:37.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:37.740+0000 7fdb486a5640 1 -- 172.21.15.5:0/3145936127 wait complete. 2024-09-12T02:57:38.400 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- bash -c 'ceph orch ls' 2024-09-12T02:57:38.658 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:57:39.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:39.117+0000 7f7eaf37a640 1 -- 172.21.15.5:0/1939986485 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ea81037c0 msgr2=0x7f7ea810a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:39.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:39.117+0000 7f7eaf37a640 1 --2- 172.21.15.5:0/1939986485 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ea81037c0 0x7f7ea810a050 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f7ea0009ca0 tx=0x7f7ea002a8c0 comp rx=0 tx=0).stop 2024-09-12T02:57:39.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:39.117+0000 7f7eaf37a640 1 -- 172.21.15.5:0/1939986485 shutdown_connections 2024-09-12T02:57:39.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:39.117+0000 7f7eaf37a640 1 --2- 172.21.15.5:0/1939986485 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ea81037c0 0x7f7ea810a050 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:39.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:39.117+0000 7f7eaf37a640 1 --2- 172.21.15.5:0/1939986485 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7ea8102e00 0x7f7ea8103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:39.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:39.117+0000 7f7eaf37a640 1 --2- 172.21.15.5:0/1939986485 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ea8101c00 0x7f7ea8102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:39.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:39.117+0000 7f7eaf37a640 1 -- 172.21.15.5:0/1939986485 >> 172.21.15.5:0/1939986485 conn(0x7f7ea80fd390 msgr2=0x7f7ea80ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:39.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:39.117+0000 7f7eaf37a640 1 -- 172.21.15.5:0/1939986485 shutdown_connections 2024-09-12T02:57:39.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:39.117+0000 7f7eaf37a640 1 -- 172.21.15.5:0/1939986485 wait complete. 2024-09-12T02:57:39.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:39.117+0000 7f7eaf37a640 1 Processor -- start 2024-09-12T02:57:39.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:39.117+0000 7f7eaf37a640 1 -- start start 2024-09-12T02:57:39.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:39.118+0000 7f7eaf37a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ea8101c00 0x7f7ea8197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:39.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:39.118+0000 7f7eaf37a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ea8102e00 0x7f7ea8198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:39.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:39.118+0000 7f7eaf37a640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7ea81037c0 0x7f7ea819f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:39.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:39.118+0000 7f7eaf37a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea810d900 con 0x7f7ea8102e00 2024-09-12T02:57:39.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:39.118+0000 7f7eaf37a640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea810d780 con 0x7f7ea81037c0 2024-09-12T02:57:39.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:39.118+0000 7f7eaf37a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea810da80 con 0x7f7ea8101c00 2024-09-12T02:57:39.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:39.118+0000 7f7eac915640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ea8102e00 0x7f7ea8198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:39.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:39.118+0000 7f7eac915640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ea8102e00 0x7f7ea8198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51848/0 (socket says 172.21.15.5:51848) 2024-09-12T02:57:39.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:39.118+0000 7f7eac915640 1 -- 172.21.15.5:0/1542655970 learned_addr learned my addr 172.21.15.5:0/1542655970 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:57:39.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:39.118+0000 7f7ead917640 1 --2- 172.21.15.5:0/1542655970 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7ea81037c0 0x7f7ea819f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:39.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:39.118+0000 7f7ead116640 1 --2- 172.21.15.5:0/1542655970 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ea8101c00 0x7f7ea8197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:39.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:39.118+0000 7f7eac915640 1 -- 172.21.15.5:0/1542655970 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ea8101c00 msgr2=0x7f7ea8197ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:39.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:39.118+0000 7f7eac915640 1 --2- 172.21.15.5:0/1542655970 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ea8101c00 0x7f7ea8197ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:39.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:39.118+0000 7f7eac915640 1 -- 172.21.15.5:0/1542655970 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7ea81037c0 msgr2=0x7f7ea819f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:39.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:39.118+0000 7f7eac915640 1 --2- 172.21.15.5:0/1542655970 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7ea81037c0 0x7f7ea819f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:39.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:39.118+0000 7f7eac915640 1 -- 172.21.15.5:0/1542655970 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ea819fb20 con 0x7f7ea8102e00 2024-09-12T02:57:39.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:39.118+0000 7f7ead116640 1 --2- 172.21.15.5:0/1542655970 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ea8101c00 0x7f7ea8197ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:57:39.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:39.118+0000 7f7eac915640 1 --2- 172.21.15.5:0/1542655970 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ea8102e00 0x7f7ea8198430 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f7e9800d950 tx=0x7f7e9800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:39.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:39.118+0000 7f7e9e7fc640 1 -- 172.21.15.5:0/1542655970 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e98017070 con 0x7f7ea8102e00 2024-09-12T02:57:39.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:39.118+0000 7f7e9e7fc640 1 -- 172.21.15.5:0/1542655970 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e98009c30 con 0x7f7ea8102e00 2024-09-12T02:57:39.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:39.118+0000 7f7eaf37a640 1 -- 172.21.15.5:0/1542655970 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ea819fe10 con 0x7f7ea8102e00 2024-09-12T02:57:39.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:39.118+0000 7f7e9e7fc640 1 -- 172.21.15.5:0/1542655970 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e9800f6b0 con 0x7f7ea8102e00 2024-09-12T02:57:39.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:39.119+0000 7f7eaf37a640 1 -- 172.21.15.5:0/1542655970 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ea81a0340 con 0x7f7ea8102e00 2024-09-12T02:57:39.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:39.119+0000 7f7eaf37a640 1 -- 172.21.15.5:0/1542655970 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ea8108fb0 con 0x7f7ea8102e00 2024-09-12T02:57:39.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:39.121+0000 7f7e9e7fc640 1 -- 172.21.15.5:0/1542655970 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f7e980109d0 con 0x7f7ea8102e00 2024-09-12T02:57:39.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:39.122+0000 7f7e9e7fc640 1 --2- 172.21.15.5:0/1542655970 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f7e940a2c40 0x7f7e940a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:39.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:39.122+0000 7f7ead116640 1 --2- 172.21.15.5:0/1542655970 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f7e940a2c40 0x7f7e940a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:39.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:39.122+0000 7f7e9e7fc640 1 -- 172.21.15.5:0/1542655970 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f7e980ced90 con 0x7f7ea8102e00 2024-09-12T02:57:39.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:39.122+0000 7f7ead116640 1 --2- 172.21.15.5:0/1542655970 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f7e940a2c40 0x7f7e940a5100 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7ea8102c60 tx=0x7f7ea4008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:39.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:39.125+0000 7f7e9e7fc640 1 -- 172.21.15.5:0/1542655970 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7e98009de0 con 0x7f7ea8102e00 2024-09-12T02:57:39.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:57:39 smithi005 ceph-mon[31197]: pgmap v317: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-12T02:57:39.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:57:39 smithi005 ceph-mon[31197]: from='client.15057 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:57:39.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:39.312+0000 7f7eaf37a640 1 -- 172.21.15.5:0/1542655970 --> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f7ea8062630 con 0x7f7e940a2c40 2024-09-12T02:57:39.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:39.317+0000 7f7e9e7fc640 1 -- 172.21.15.5:0/1542655970 <== mgr.14148 v2:172.21.15.5:6800/3780631642 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f7ea8062630 con 0x7f7e940a2c40 2024-09-12T02:57:39.317 INFO:teuthology.orchestra.run.smithi005.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-09-12T02:57:39.318 INFO:teuthology.orchestra.run.smithi005.stdout:mds.all 12/12 26s ago 2m smithi005=a;smithi005=d;smithi005=g;smithi005=j;smithi146=b;smithi146=e;smithi146=h;smithi146=k;smithi148=c;smithi148=f;smithi148=i;smithi148=l;count:12 2024-09-12T02:57:39.318 INFO:teuthology.orchestra.run.smithi005.stdout:mgr 3/3 26s ago 7m smithi005=x;smithi146=y;smithi148=z;count:3 2024-09-12T02:57:39.318 INFO:teuthology.orchestra.run.smithi005.stdout:mon 3/3 26s ago 7m smithi005:172.21.15.5=a;smithi146:172.21.15.146=b;smithi148:172.21.15.148=c;count:3 2024-09-12T02:57:39.318 INFO:teuthology.orchestra.run.smithi005.stdout:osd 12 26s ago - 2024-09-12T02:57:39.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:39.319+0000 7f7eaf37a640 1 -- 172.21.15.5:0/1542655970 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f7e940a2c40 msgr2=0x7f7e940a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:39.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:39.319+0000 7f7eaf37a640 1 --2- 172.21.15.5:0/1542655970 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f7e940a2c40 0x7f7e940a5100 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7ea8102c60 tx=0x7f7ea4008040 comp rx=0 tx=0).stop 2024-09-12T02:57:39.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:39.319+0000 7f7eaf37a640 1 -- 172.21.15.5:0/1542655970 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ea8102e00 msgr2=0x7f7ea8198430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:39.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:39.319+0000 7f7eaf37a640 1 --2- 172.21.15.5:0/1542655970 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ea8102e00 0x7f7ea8198430 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f7e9800d950 tx=0x7f7e9800de10 comp rx=0 tx=0).stop 2024-09-12T02:57:39.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:39.319+0000 7f7eaf37a640 1 -- 172.21.15.5:0/1542655970 shutdown_connections 2024-09-12T02:57:39.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:39.319+0000 7f7eaf37a640 1 --2- 172.21.15.5:0/1542655970 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f7e940a2c40 0x7f7e940a5100 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:39.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:39.319+0000 7f7eaf37a640 1 --2- 172.21.15.5:0/1542655970 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ea8102e00 0x7f7ea8198430 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:39.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:39.319+0000 7f7eaf37a640 1 --2- 172.21.15.5:0/1542655970 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7ea81037c0 0x7f7ea819f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:39.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:39.319+0000 7f7eaf37a640 1 --2- 172.21.15.5:0/1542655970 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7ea8101c00 0x7f7ea8197ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:39.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:39.319+0000 7f7eaf37a640 1 -- 172.21.15.5:0/1542655970 >> 172.21.15.5:0/1542655970 conn(0x7f7ea80fd390 msgr2=0x7f7ea8106180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:39.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:39.320+0000 7f7eaf37a640 1 -- 172.21.15.5:0/1542655970 shutdown_connections 2024-09-12T02:57:39.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:39.320+0000 7f7eaf37a640 1 -- 172.21.15.5:0/1542655970 wait complete. 2024-09-12T02:57:39.526 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:57:39 smithi146 ceph-mon[35011]: pgmap v317: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-12T02:57:39.527 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:57:39 smithi146 ceph-mon[35011]: from='client.15057 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:57:39.665 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:57:39 smithi148 ceph-mon[35090]: pgmap v317: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-12T02:57:39.665 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:57:39 smithi148 ceph-mon[35090]: from='client.15057 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:57:39.957 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- bash -c 'ceph orch host ls' 2024-09-12T02:57:40.223 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:57:40.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:57:40 smithi005 ceph-mon[31197]: from='client.15063 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:57:40.665 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:57:40 smithi148 ceph-mon[35090]: from='client.15063 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:57:40.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:57:40 smithi146 ceph-mon[35011]: from='client.15063 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:57:40.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:40.702+0000 7f47a89a8640 1 -- 172.21.15.5:0/988608066 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47a4101bf0 msgr2=0x7f47a4101ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:40.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:40.702+0000 7f47a89a8640 1 --2- 172.21.15.5:0/988608066 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47a4101bf0 0x7f47a4101ff0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f4794009900 tx=0x7f479402a410 comp rx=0 tx=0).stop 2024-09-12T02:57:40.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:40.702+0000 7f47a89a8640 1 -- 172.21.15.5:0/988608066 shutdown_connections 2024-09-12T02:57:40.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:40.702+0000 7f47a89a8640 1 --2- 172.21.15.5:0/988608066 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f47a41037b0 0x7f47a410a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:40.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:40.702+0000 7f47a89a8640 1 --2- 172.21.15.5:0/988608066 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47a4102df0 0x7f47a4103270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:40.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:40.702+0000 7f47a89a8640 1 --2- 172.21.15.5:0/988608066 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47a4101bf0 0x7f47a4101ff0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:40.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:40.702+0000 7f47a89a8640 1 -- 172.21.15.5:0/988608066 >> 172.21.15.5:0/988608066 conn(0x7f47a40fd3c0 msgr2=0x7f47a40ff7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:40.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:40.702+0000 7f47a89a8640 1 -- 172.21.15.5:0/988608066 shutdown_connections 2024-09-12T02:57:40.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:40.702+0000 7f47a89a8640 1 -- 172.21.15.5:0/988608066 wait complete. 2024-09-12T02:57:40.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:40.703+0000 7f47a89a8640 1 Processor -- start 2024-09-12T02:57:40.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:40.703+0000 7f47a89a8640 1 -- start start 2024-09-12T02:57:40.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:40.703+0000 7f47a89a8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47a4101bf0 0x7f47a4075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:40.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:40.703+0000 7f47a89a8640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f47a4102df0 0x7f47a40745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:40.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:40.703+0000 7f47a89a8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47a41037b0 0x7f47a4074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:40.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:40.703+0000 7f47a89a8640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f47a410d8a0 con 0x7f47a41037b0 2024-09-12T02:57:40.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:40.703+0000 7f47a89a8640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f47a410d720 con 0x7f47a4102df0 2024-09-12T02:57:40.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:40.703+0000 7f47a89a8640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f47a410da20 con 0x7f47a4101bf0 2024-09-12T02:57:40.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:40.703+0000 7f47a2d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47a41037b0 0x7f47a4074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:40.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:40.703+0000 7f47a1d9b640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f47a4102df0 0x7f47a40745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:40.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:40.703+0000 7f47a2d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47a41037b0 0x7f47a4074b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51864/0 (socket says 172.21.15.5:51864) 2024-09-12T02:57:40.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:40.703+0000 7f47a2d9d640 1 -- 172.21.15.5:0/3127835850 learned_addr learned my addr 172.21.15.5:0/3127835850 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:57:40.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:40.703+0000 7f47a259c640 1 --2- 172.21.15.5:0/3127835850 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47a4101bf0 0x7f47a4075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:40.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:40.704+0000 7f47a1d9b640 1 -- 172.21.15.5:0/3127835850 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47a4101bf0 msgr2=0x7f47a4075f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:40.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:40.704+0000 7f47a1d9b640 1 --2- 172.21.15.5:0/3127835850 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47a4101bf0 0x7f47a4075f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:40.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:40.704+0000 7f47a1d9b640 1 -- 172.21.15.5:0/3127835850 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47a41037b0 msgr2=0x7f47a4074b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:40.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:40.704+0000 7f47a1d9b640 1 --2- 172.21.15.5:0/3127835850 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47a41037b0 0x7f47a4074b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:40.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:40.704+0000 7f47a1d9b640 1 -- 172.21.15.5:0/3127835850 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47a40753e0 con 0x7f47a4102df0 2024-09-12T02:57:40.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:40.704+0000 7f47a1d9b640 1 --2- 172.21.15.5:0/3127835850 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f47a4102df0 0x7f47a40745e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f479800e9f0 tx=0x7f479800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:40.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:40.704+0000 7f47937fe640 1 -- 172.21.15.5:0/3127835850 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f479800ccc0 con 0x7f47a4102df0 2024-09-12T02:57:40.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:40.704+0000 7f47937fe640 1 -- 172.21.15.5:0/3127835850 <== mon.2 v2:172.21.15.146:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4798010910 con 0x7f47a4102df0 2024-09-12T02:57:40.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:40.704+0000 7f47a89a8640 1 -- 172.21.15.5:0/3127835850 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47a41a2060 con 0x7f47a4102df0 2024-09-12T02:57:40.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:40.704+0000 7f47937fe640 1 -- 172.21.15.5:0/3127835850 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4798018960 con 0x7f47a4102df0 2024-09-12T02:57:40.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:40.704+0000 7f47a89a8640 1 -- 172.21.15.5:0/3127835850 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47a41a2570 con 0x7f47a4102df0 2024-09-12T02:57:40.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:40.705+0000 7f47a89a8640 1 -- 172.21.15.5:0/3127835850 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47a4108fa0 con 0x7f47a4102df0 2024-09-12T02:57:40.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:40.706+0000 7f47937fe640 1 -- 172.21.15.5:0/3127835850 <== mon.2 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f4798010ac0 con 0x7f47a4102df0 2024-09-12T02:57:40.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:40.707+0000 7f47937fe640 1 --2- 172.21.15.5:0/3127835850 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f478c0a2c10 0x7f478c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:40.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:40.707+0000 7f47a259c640 1 --2- 172.21.15.5:0/3127835850 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f478c0a2c10 0x7f478c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:40.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:40.707+0000 7f47937fe640 1 -- 172.21.15.5:0/3127835850 <== mon.2 v2:172.21.15.146:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f47980cf8e0 con 0x7f47a4102df0 2024-09-12T02:57:40.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:40.708+0000 7f47a259c640 1 --2- 172.21.15.5:0/3127835850 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f478c0a2c10 0x7f478c0a50d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f479402aa00 tx=0x7f47940315b0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:40.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:40.710+0000 7f47937fe640 1 -- 172.21.15.5:0/3127835850 <== mon.2 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f479809c040 con 0x7f47a4102df0 2024-09-12T02:57:40.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:40.892+0000 7f47a89a8640 1 -- 172.21.15.5:0/3127835850 --> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f47a4062630 con 0x7f478c0a2c10 2024-09-12T02:57:40.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:40.894+0000 7f47937fe640 1 -- 172.21.15.5:0/3127835850 <== mgr.14148 v2:172.21.15.5:6800/3780631642 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f47a4062630 con 0x7f478c0a2c10 2024-09-12T02:57:40.894 INFO:teuthology.orchestra.run.smithi005.stdout:HOST ADDR LABELS STATUS 2024-09-12T02:57:40.894 INFO:teuthology.orchestra.run.smithi005.stdout:smithi005 172.21.15.5 2024-09-12T02:57:40.894 INFO:teuthology.orchestra.run.smithi005.stdout:smithi146 172.21.15.146 2024-09-12T02:57:40.894 INFO:teuthology.orchestra.run.smithi005.stdout:smithi148 172.21.15.148 2024-09-12T02:57:40.894 INFO:teuthology.orchestra.run.smithi005.stdout:3 hosts in cluster 2024-09-12T02:57:40.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:40.895+0000 7f47a89a8640 1 -- 172.21.15.5:0/3127835850 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f478c0a2c10 msgr2=0x7f478c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:40.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:40.895+0000 7f47a89a8640 1 --2- 172.21.15.5:0/3127835850 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f478c0a2c10 0x7f478c0a50d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f479402aa00 tx=0x7f47940315b0 comp rx=0 tx=0).stop 2024-09-12T02:57:40.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:40.896+0000 7f47a89a8640 1 -- 172.21.15.5:0/3127835850 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f47a4102df0 msgr2=0x7f47a40745e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:40.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:40.896+0000 7f47a89a8640 1 --2- 172.21.15.5:0/3127835850 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f47a4102df0 0x7f47a40745e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f479800e9f0 tx=0x7f479800eeb0 comp rx=0 tx=0).stop 2024-09-12T02:57:40.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:40.896+0000 7f47a89a8640 1 -- 172.21.15.5:0/3127835850 shutdown_connections 2024-09-12T02:57:40.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:40.896+0000 7f47a89a8640 1 --2- 172.21.15.5:0/3127835850 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f478c0a2c10 0x7f478c0a50d0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:40.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:40.896+0000 7f47a89a8640 1 --2- 172.21.15.5:0/3127835850 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47a41037b0 0x7f47a4074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:40.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:40.896+0000 7f47a89a8640 1 --2- 172.21.15.5:0/3127835850 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f47a4102df0 0x7f47a40745e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:40.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:40.896+0000 7f47a89a8640 1 --2- 172.21.15.5:0/3127835850 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47a4101bf0 0x7f47a4075f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:40.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:40.896+0000 7f47a89a8640 1 -- 172.21.15.5:0/3127835850 >> 172.21.15.5:0/3127835850 conn(0x7f47a40fd3c0 msgr2=0x7f47a4106170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:40.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:40.896+0000 7f47a89a8640 1 -- 172.21.15.5:0/3127835850 shutdown_connections 2024-09-12T02:57:40.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:40.896+0000 7f47a89a8640 1 -- 172.21.15.5:0/3127835850 wait complete. 2024-09-12T02:57:41.487 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:57:41 smithi005 ceph-mon[31197]: pgmap v318: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-12T02:57:41.516 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- bash -c 'ceph orch device ls' 2024-09-12T02:57:41.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:57:41 smithi148 ceph-mon[35090]: pgmap v318: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-12T02:57:41.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:57:41 smithi146 ceph-mon[35011]: pgmap v318: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-12T02:57:41.774 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:57:42.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:42.242+0000 7f6377438640 1 -- 172.21.15.5:0/1206411451 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6370101d10 msgr2=0x7f6370102190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:42.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:42.242+0000 7f6377438640 1 --2- 172.21.15.5:0/1206411451 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6370101d10 0x7f6370102190 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f6360009900 tx=0x7f636002a410 comp rx=0 tx=0).stop 2024-09-12T02:57:42.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:42.242+0000 7f6377438640 1 -- 172.21.15.5:0/1206411451 shutdown_connections 2024-09-12T02:57:42.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:42.242+0000 7f6377438640 1 --2- 172.21.15.5:0/1206411451 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6370101d10 0x7f6370102190 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:42.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:42.242+0000 7f6377438640 1 --2- 172.21.15.5:0/1206411451 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f63701026d0 0x7f6370108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:42.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:42.242+0000 7f6377438640 1 --2- 172.21.15.5:0/1206411451 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6370100b10 0x7f6370100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:42.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:42.242+0000 7f6377438640 1 -- 172.21.15.5:0/1206411451 >> 172.21.15.5:0/1206411451 conn(0x7f63700fc2a0 msgr2=0x7f63700fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:42.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:42.242+0000 7f6377438640 1 -- 172.21.15.5:0/1206411451 shutdown_connections 2024-09-12T02:57:42.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:42.242+0000 7f6377438640 1 -- 172.21.15.5:0/1206411451 wait complete. 2024-09-12T02:57:42.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:42.243+0000 7f6377438640 1 Processor -- start 2024-09-12T02:57:42.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:42.243+0000 7f6377438640 1 -- start start 2024-09-12T02:57:42.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:42.243+0000 7f6377438640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6370100b10 0x7f6370196ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:42.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:42.243+0000 7f6377438640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6370101d10 0x7f6370197410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:42.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:42.243+0000 7f6377438640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f63701026d0 0x7f637019e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:42.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:42.243+0000 7f6377438640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f637010c810 con 0x7f6370101d10 2024-09-12T02:57:42.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:42.243+0000 7f6377438640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f637010c690 con 0x7f63701026d0 2024-09-12T02:57:42.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:42.243+0000 7f6377438640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f637010c990 con 0x7f6370100b10 2024-09-12T02:57:42.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:42.243+0000 7f63749d3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6370101d10 0x7f6370197410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:42.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:42.243+0000 7f63749d3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6370101d10 0x7f6370197410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51884/0 (socket says 172.21.15.5:51884) 2024-09-12T02:57:42.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:42.243+0000 7f63749d3640 1 -- 172.21.15.5:0/1286344018 learned_addr learned my addr 172.21.15.5:0/1286344018 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:57:42.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:42.243+0000 7f63749d3640 1 -- 172.21.15.5:0/1286344018 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6370100b10 msgr2=0x7f6370196ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:42.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:42.243+0000 7f63751d4640 1 --2- 172.21.15.5:0/1286344018 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6370100b10 0x7f6370196ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:42.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:42.243+0000 7f63749d3640 1 --2- 172.21.15.5:0/1286344018 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6370100b10 0x7f6370196ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:42.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:42.243+0000 7f63759d5640 1 --2- 172.21.15.5:0/1286344018 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f63701026d0 0x7f637019e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:42.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:42.243+0000 7f63749d3640 1 -- 172.21.15.5:0/1286344018 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f63701026d0 msgr2=0x7f637019e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:42.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:42.243+0000 7f63749d3640 1 --2- 172.21.15.5:0/1286344018 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f63701026d0 0x7f637019e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:42.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:42.243+0000 7f63749d3640 1 -- 172.21.15.5:0/1286344018 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f637019eb00 con 0x7f6370101d10 2024-09-12T02:57:42.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:42.243+0000 7f63751d4640 1 --2- 172.21.15.5:0/1286344018 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6370100b10 0x7f6370196ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:57:42.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:42.244+0000 7f63749d3640 1 --2- 172.21.15.5:0/1286344018 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6370101d10 0x7f6370197410 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f6360009230 tx=0x7f63600043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:42.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:42.244+0000 7f63667fc640 1 -- 172.21.15.5:0/1286344018 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6360046070 con 0x7f6370101d10 2024-09-12T02:57:42.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:42.244+0000 7f6377438640 1 -- 172.21.15.5:0/1286344018 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f637006a790 con 0x7f6370101d10 2024-09-12T02:57:42.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:42.244+0000 7f63667fc640 1 -- 172.21.15.5:0/1286344018 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6360003ce0 con 0x7f6370101d10 2024-09-12T02:57:42.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:42.244+0000 7f63667fc640 1 -- 172.21.15.5:0/1286344018 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6360035050 con 0x7f6370101d10 2024-09-12T02:57:42.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:42.244+0000 7f6377438640 1 -- 172.21.15.5:0/1286344018 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f637006ad40 con 0x7f6370101d10 2024-09-12T02:57:42.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:42.245+0000 7f6377438640 1 -- 172.21.15.5:0/1286344018 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6370100ef0 con 0x7f6370101d10 2024-09-12T02:57:42.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:42.246+0000 7f63667fc640 1 -- 172.21.15.5:0/1286344018 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f6360049070 con 0x7f6370101d10 2024-09-12T02:57:42.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:42.247+0000 7f63667fc640 1 --2- 172.21.15.5:0/1286344018 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f635c0a2cc0 0x7f635c0a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:42.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:42.247+0000 7f63751d4640 1 --2- 172.21.15.5:0/1286344018 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f635c0a2cc0 0x7f635c0a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:42.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:42.247+0000 7f63667fc640 1 -- 172.21.15.5:0/1286344018 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f63600f0f50 con 0x7f6370101d10 2024-09-12T02:57:42.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:42.247+0000 7f63751d4640 1 --2- 172.21.15.5:0/1286344018 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f635c0a2cc0 0x7f635c0a5180 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f636c006fd0 tx=0x7f636c008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:42.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:42.251+0000 7f63667fc640 1 -- 172.21.15.5:0/1286344018 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f636007e1b0 con 0x7f6370101d10 2024-09-12T02:57:42.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:42.436+0000 7f6377438640 1 -- 172.21.15.5:0/1286344018 --> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f637006afa0 con 0x7f635c0a2cc0 2024-09-12T02:57:42.437 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:57:42 smithi005 ceph-mon[31197]: from='client.24764 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:57:42.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:42.438+0000 7f63667fc640 1 -- 172.21.15.5:0/1286344018 <== mgr.14148 v2:172.21.15.5:6800/3780631642 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f637006afa0 con 0x7f635c0a2cc0 2024-09-12T02:57:42.438 INFO:teuthology.orchestra.run.smithi005.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-09-12T02:57:42.438 INFO:teuthology.orchestra.run.smithi005.stdout:smithi005 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB751300EW480DGN 447G No 3m ago Has a FileSystem, LVM detected 2024-09-12T02:57:42.438 INFO:teuthology.orchestra.run.smithi005.stdout:smithi146 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300GB400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-09-12T02:57:42.438 INFO:teuthology.orchestra.run.smithi005.stdout:smithi148 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6233005G400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-09-12T02:57:42.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:42.440+0000 7f6377438640 1 -- 172.21.15.5:0/1286344018 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f635c0a2cc0 msgr2=0x7f635c0a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:42.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:42.440+0000 7f6377438640 1 --2- 172.21.15.5:0/1286344018 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f635c0a2cc0 0x7f635c0a5180 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f636c006fd0 tx=0x7f636c008040 comp rx=0 tx=0).stop 2024-09-12T02:57:42.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:42.440+0000 7f6377438640 1 -- 172.21.15.5:0/1286344018 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6370101d10 msgr2=0x7f6370197410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:42.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:42.440+0000 7f6377438640 1 --2- 172.21.15.5:0/1286344018 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6370101d10 0x7f6370197410 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f6360009230 tx=0x7f63600043b0 comp rx=0 tx=0).stop 2024-09-12T02:57:42.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:42.440+0000 7f6377438640 1 -- 172.21.15.5:0/1286344018 shutdown_connections 2024-09-12T02:57:42.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:42.440+0000 7f6377438640 1 --2- 172.21.15.5:0/1286344018 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f635c0a2cc0 0x7f635c0a5180 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:42.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:42.440+0000 7f6377438640 1 --2- 172.21.15.5:0/1286344018 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6370101d10 0x7f6370197410 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:42.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:42.440+0000 7f6377438640 1 --2- 172.21.15.5:0/1286344018 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f63701026d0 0x7f637019e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:42.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:42.440+0000 7f6377438640 1 --2- 172.21.15.5:0/1286344018 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6370100b10 0x7f6370196ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:42.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:42.440+0000 7f6377438640 1 -- 172.21.15.5:0/1286344018 >> 172.21.15.5:0/1286344018 conn(0x7f63700fc2a0 msgr2=0x7f6370105150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:42.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:42.440+0000 7f6377438640 1 -- 172.21.15.5:0/1286344018 shutdown_connections 2024-09-12T02:57:42.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:42.440+0000 7f6377438640 1 -- 172.21.15.5:0/1286344018 wait complete. 2024-09-12T02:57:42.665 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:57:42 smithi148 ceph-mon[35090]: from='client.24764 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:57:42.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:57:42 smithi146 ceph-mon[35011]: from='client.24764 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:57:43.108 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-12T02:57:43.117 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi005.front.sepia.ceph.com 2024-09-12T02:57:43.117 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- bash -c 'ceph fs dump' 2024-09-12T02:57:43.376 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:57:43.482 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:57:43 smithi005 ceph-mon[31197]: pgmap v319: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T02:57:43.482 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:57:43 smithi005 ceph-mon[31197]: from='client.15075 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:57:43.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:57:43 smithi148 ceph-mon[35090]: pgmap v319: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T02:57:43.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:57:43 smithi148 ceph-mon[35090]: from='client.15075 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:57:43.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:57:43 smithi146 ceph-mon[35011]: pgmap v319: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T02:57:43.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:57:43 smithi146 ceph-mon[35011]: from='client.15075 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:57:43.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:43.800+0000 7f2876a56640 1 -- 172.21.15.5:0/534338323 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2870100ad0 msgr2=0x7f2870100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:43.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:43.800+0000 7f2876a56640 1 --2- 172.21.15.5:0/534338323 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2870100ad0 0x7f2870100ed0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f2860009900 tx=0x7f286002a410 comp rx=0 tx=0).stop 2024-09-12T02:57:43.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:43.800+0000 7f2876a56640 1 -- 172.21.15.5:0/534338323 shutdown_connections 2024-09-12T02:57:43.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:43.800+0000 7f2876a56640 1 --2- 172.21.15.5:0/534338323 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2870102690 0x7f2870108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:43.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:43.800+0000 7f2876a56640 1 --2- 172.21.15.5:0/534338323 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2870101cd0 0x7f2870102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:43.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:43.800+0000 7f2876a56640 1 --2- 172.21.15.5:0/534338323 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2870100ad0 0x7f2870100ed0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:43.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:43.800+0000 7f2876a56640 1 -- 172.21.15.5:0/534338323 >> 172.21.15.5:0/534338323 conn(0x7f28700fc280 msgr2=0x7f28700fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:43.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:43.800+0000 7f2876a56640 1 -- 172.21.15.5:0/534338323 shutdown_connections 2024-09-12T02:57:43.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:43.800+0000 7f2876a56640 1 -- 172.21.15.5:0/534338323 wait complete. 2024-09-12T02:57:43.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:43.801+0000 7f2876a56640 1 Processor -- start 2024-09-12T02:57:43.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:43.801+0000 7f2876a56640 1 -- start start 2024-09-12T02:57:43.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:43.801+0000 7f2876a56640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2870100ad0 0x7f2870104a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:43.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:43.801+0000 7f2876a56640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2870101cd0 0x7f28701030b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:43.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:43.801+0000 7f2876a56640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2870102690 0x7f28701035f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:43.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:43.801+0000 7f2876a56640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f287010c7b0 con 0x7f2870100ad0 2024-09-12T02:57:43.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:43.801+0000 7f2876a56640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f287010c630 con 0x7f2870102690 2024-09-12T02:57:43.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:43.801+0000 7f2876a56640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f287010c930 con 0x7f2870101cd0 2024-09-12T02:57:43.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:43.801+0000 7f286ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2870100ad0 0x7f2870104a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:43.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:43.801+0000 7f286ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2870100ad0 0x7f2870104a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37086/0 (socket says 172.21.15.5:37086) 2024-09-12T02:57:43.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:43.801+0000 7f286ffff640 1 -- 172.21.15.5:0/2958134895 learned_addr learned my addr 172.21.15.5:0/2958134895 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:57:43.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:43.802+0000 7f2867fff640 1 --2- 172.21.15.5:0/2958134895 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2870101cd0 0x7f28701030b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:43.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:43.802+0000 7f286ffff640 1 -- 172.21.15.5:0/2958134895 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2870101cd0 msgr2=0x7f28701030b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:43.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:43.802+0000 7f286ffff640 1 --2- 172.21.15.5:0/2958134895 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2870101cd0 0x7f28701030b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:43.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:43.802+0000 7f286ffff640 1 -- 172.21.15.5:0/2958134895 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2870102690 msgr2=0x7f28701035f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:43.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:43.802+0000 7f286ffff640 1 --2- 172.21.15.5:0/2958134895 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2870102690 0x7f28701035f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:43.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:43.802+0000 7f286ffff640 1 -- 172.21.15.5:0/2958134895 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2870103eb0 con 0x7f2870100ad0 2024-09-12T02:57:43.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:43.802+0000 7f286ffff640 1 --2- 172.21.15.5:0/2958134895 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2870100ad0 0x7f2870104a60 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f2860031610 tx=0x7f28600043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:43.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:43.802+0000 7f286dffb640 1 -- 172.21.15.5:0/2958134895 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f286003f030 con 0x7f2870100ad0 2024-09-12T02:57:43.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:43.802+0000 7f2876a56640 1 -- 172.21.15.5:0/2958134895 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28701a2060 con 0x7f2870100ad0 2024-09-12T02:57:43.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:43.802+0000 7f2876a56640 1 -- 172.21.15.5:0/2958134895 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28701a25e0 con 0x7f2870100ad0 2024-09-12T02:57:43.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:43.802+0000 7f286dffb640 1 -- 172.21.15.5:0/2958134895 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2860003ce0 con 0x7f2870100ad0 2024-09-12T02:57:43.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:43.802+0000 7f286dffb640 1 -- 172.21.15.5:0/2958134895 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2860035070 con 0x7f2870100ad0 2024-09-12T02:57:43.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:43.803+0000 7f2876a56640 1 -- 172.21.15.5:0/2958134895 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28701a2920 con 0x7f2870100ad0 2024-09-12T02:57:43.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:43.804+0000 7f286dffb640 1 -- 172.21.15.5:0/2958134895 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f2860042030 con 0x7f2870100ad0 2024-09-12T02:57:43.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:43.805+0000 7f286dffb640 1 --2- 172.21.15.5:0/2958134895 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f285c0a2c90 0x7f285c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:43.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:43.805+0000 7f2867fff640 1 --2- 172.21.15.5:0/2958134895 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f285c0a2c90 0x7f285c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:43.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:43.805+0000 7f286dffb640 1 -- 172.21.15.5:0/2958134895 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f28600ef880 con 0x7f2870100ad0 2024-09-12T02:57:43.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:43.805+0000 7f2867fff640 1 --2- 172.21.15.5:0/2958134895 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f285c0a2c90 0x7f285c0a5150 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f28701047e0 tx=0x7f2858005cc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:43.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:43.809+0000 7f286dffb640 1 -- 172.21.15.5:0/2958134895 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f28600bc690 con 0x7f2870100ad0 2024-09-12T02:57:44.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:44.024+0000 7f2876a56640 1 -- 172.21.15.5:0/2958134895 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f28701a2c10 con 0x7f2870100ad0 2024-09-12T02:57:44.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:44.024+0000 7f286dffb640 1 -- 172.21.15.5:0/2958134895 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 25 v25) ==== 76+0+3123 (secure 0 0 0) 0x7f28600bbf90 con 0x7f2870100ad0 2024-09-12T02:57:44.025 INFO:teuthology.orchestra.run.smithi005.stdout:e25 2024-09-12T02:57:44.025 INFO:teuthology.orchestra.run.smithi005.stdout:btime 2024-09-12T02:56:27:840212+0000 2024-09-12T02:57:44.025 INFO:teuthology.orchestra.run.smithi005.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-12T02:57:44.025 INFO:teuthology.orchestra.run.smithi005.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-12T02:57:44.025 INFO:teuthology.orchestra.run.smithi005.stdout:legacy client fscid: 1 2024-09-12T02:57:44.025 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-12T02:57:44.026 INFO:teuthology.orchestra.run.smithi005.stdout:Filesystem 'cephfs' (1) 2024-09-12T02:57:44.026 INFO:teuthology.orchestra.run.smithi005.stdout:fs_name cephfs 2024-09-12T02:57:44.026 INFO:teuthology.orchestra.run.smithi005.stdout:epoch 25 2024-09-12T02:57:44.026 INFO:teuthology.orchestra.run.smithi005.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-12T02:57:44.026 INFO:teuthology.orchestra.run.smithi005.stdout:created 2024-09-12T02:56:16.595562+0000 2024-09-12T02:57:44.026 INFO:teuthology.orchestra.run.smithi005.stdout:modified 2024-09-12T02:56:27.840207+0000 2024-09-12T02:57:44.026 INFO:teuthology.orchestra.run.smithi005.stdout:tableserver 0 2024-09-12T02:57:44.026 INFO:teuthology.orchestra.run.smithi005.stdout:root 0 2024-09-12T02:57:44.026 INFO:teuthology.orchestra.run.smithi005.stdout:session_timeout 300 2024-09-12T02:57:44.026 INFO:teuthology.orchestra.run.smithi005.stdout:session_autoclose 300 2024-09-12T02:57:44.026 INFO:teuthology.orchestra.run.smithi005.stdout:max_file_size 1099511627776 2024-09-12T02:57:44.026 INFO:teuthology.orchestra.run.smithi005.stdout:max_xattr_size 65536 2024-09-12T02:57:44.027 INFO:teuthology.orchestra.run.smithi005.stdout:required_client_features {} 2024-09-12T02:57:44.027 INFO:teuthology.orchestra.run.smithi005.stdout:last_failure 0 2024-09-12T02:57:44.027 INFO:teuthology.orchestra.run.smithi005.stdout:last_failure_osd_epoch 0 2024-09-12T02:57:44.027 INFO:teuthology.orchestra.run.smithi005.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-12T02:57:44.027 INFO:teuthology.orchestra.run.smithi005.stdout:max_mds 5 2024-09-12T02:57:44.027 INFO:teuthology.orchestra.run.smithi005.stdout:in 0,1,2,3,4 2024-09-12T02:57:44.027 INFO:teuthology.orchestra.run.smithi005.stdout:up {0=24502,1=24487,2=24481,3=24479,4=24458} 2024-09-12T02:57:44.027 INFO:teuthology.orchestra.run.smithi005.stdout:failed 2024-09-12T02:57:44.027 INFO:teuthology.orchestra.run.smithi005.stdout:damaged 2024-09-12T02:57:44.027 INFO:teuthology.orchestra.run.smithi005.stdout:stopped 2024-09-12T02:57:44.027 INFO:teuthology.orchestra.run.smithi005.stdout:data_pools [3] 2024-09-12T02:57:44.027 INFO:teuthology.orchestra.run.smithi005.stdout:metadata_pool 2 2024-09-12T02:57:44.027 INFO:teuthology.orchestra.run.smithi005.stdout:inline_data disabled 2024-09-12T02:57:44.028 INFO:teuthology.orchestra.run.smithi005.stdout:balancer 2024-09-12T02:57:44.028 INFO:teuthology.orchestra.run.smithi005.stdout:bal_rank_mask -1 2024-09-12T02:57:44.028 INFO:teuthology.orchestra.run.smithi005.stdout:standby_count_wanted 0 2024-09-12T02:57:44.028 INFO:teuthology.orchestra.run.smithi005.stdout:qdb_cluster leader: 24502 members: 24458,24479,24481,24487,24502 2024-09-12T02:57:44.028 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.i{0:24502} state up:active seq 6 addr [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T02:57:44.028 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.e{0:24440} state up:standby-replay seq 1 addr [v2:172.21.15.146:6834/2540740420,v1:172.21.15.146:6835/2540740420] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T02:57:44.028 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.c{1:24487} state up:active seq 9 addr [v2:172.21.15.148:6836/2606873922,v1:172.21.15.148:6837/2606873922] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T02:57:44.028 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.l{1:24439} state up:standby-replay seq 1 addr [v2:172.21.15.148:6832/568512209,v1:172.21.15.148:6833/568512209] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T02:57:44.028 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.f{2:24481} state up:active seq 10 addr [v2:172.21.15.148:6834/4195086427,v1:172.21.15.148:6835/4195086427] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T02:57:44.028 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.k{2:24407} state up:standby-replay seq 1 addr [v2:172.21.15.146:6832/3686970137,v1:172.21.15.146:6833/3686970137] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T02:57:44.028 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.b{3:24479} state up:active seq 6 addr [v2:172.21.15.146:6838/3036767228,v1:172.21.15.146:6839/3036767228] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T02:57:44.028 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.a{3:14613} state up:standby-replay seq 1 addr [v2:172.21.15.5:6840/2393443295,v1:172.21.15.5:6841/2393443295] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T02:57:44.029 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.h{4:24458} state up:active seq 9 addr [v2:172.21.15.146:6836/1673370937,v1:172.21.15.146:6837/1673370937] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T02:57:44.029 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.g{4:14601} state up:standby-replay seq 1 addr [v2:172.21.15.5:6838/26447341,v1:172.21.15.5:6839/26447341] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T02:57:44.029 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-12T02:57:44.029 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-12T02:57:44.029 INFO:teuthology.orchestra.run.smithi005.stdout:Standby daemons: 2024-09-12T02:57:44.029 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-12T02:57:44.029 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.j{-1:14535} state up:standby seq 1 addr [v2:172.21.15.5:6834/4135232521,v1:172.21.15.5:6835/4135232521] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T02:57:44.029 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.d{-1:14547} state up:standby seq 1 addr [v2:172.21.15.5:6836/4290221940,v1:172.21.15.5:6837/4290221940] compat {c=[1],r=[1],i=[1fff]}] 2024-09-12T02:57:44.030 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 25 2024-09-12T02:57:44.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:44.028+0000 7f2876a56640 1 -- 172.21.15.5:0/2958134895 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f285c0a2c90 msgr2=0x7f285c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:44.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:44.028+0000 7f2876a56640 1 --2- 172.21.15.5:0/2958134895 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f285c0a2c90 0x7f285c0a5150 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f28701047e0 tx=0x7f2858005cc0 comp rx=0 tx=0).stop 2024-09-12T02:57:44.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:44.028+0000 7f2876a56640 1 -- 172.21.15.5:0/2958134895 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2870100ad0 msgr2=0x7f2870104a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:44.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:44.028+0000 7f2876a56640 1 --2- 172.21.15.5:0/2958134895 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2870100ad0 0x7f2870104a60 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f2860031610 tx=0x7f28600043b0 comp rx=0 tx=0).stop 2024-09-12T02:57:44.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:44.028+0000 7f2876a56640 1 -- 172.21.15.5:0/2958134895 shutdown_connections 2024-09-12T02:57:44.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:44.028+0000 7f2876a56640 1 --2- 172.21.15.5:0/2958134895 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2870102690 0x7f28701035f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:44.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:44.028+0000 7f2876a56640 1 --2- 172.21.15.5:0/2958134895 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2870101cd0 0x7f28701030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:44.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:44.028+0000 7f2876a56640 1 --2- 172.21.15.5:0/2958134895 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f285c0a2c90 0x7f285c0a5150 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:44.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:44.028+0000 7f2876a56640 1 --2- 172.21.15.5:0/2958134895 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2870100ad0 0x7f2870104a60 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:44.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:44.028+0000 7f2876a56640 1 -- 172.21.15.5:0/2958134895 >> 172.21.15.5:0/2958134895 conn(0x7f28700fc280 msgr2=0x7f2870069fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:44.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:44.028+0000 7f2876a56640 1 -- 172.21.15.5:0/2958134895 shutdown_connections 2024-09-12T02:57:44.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:44.028+0000 7f2876a56640 1 -- 172.21.15.5:0/2958134895 wait complete. 2024-09-12T02:57:44.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:57:44 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/2958134895' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-12T02:57:44.665 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:57:44 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/2958134895' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-12T02:57:44.704 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- bash -c 'ceph osd dump' 2024-09-12T02:57:44.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:57:44 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/2958134895' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-12T02:57:44.964 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:57:45.392 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:57:45 smithi005 ceph-mon[31197]: pgmap v320: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-12T02:57:45.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:45.392+0000 7fc006f77640 1 -- 172.21.15.5:0/1213876871 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc000105030 msgr2=0x7fc000107420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:45.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:45.392+0000 7fc006f77640 1 --2- 172.21.15.5:0/1213876871 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc000105030 0x7fc000107420 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fbfec009900 tx=0x7fbfec02a410 comp rx=0 tx=0).stop 2024-09-12T02:57:45.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:45.392+0000 7fc006f77640 1 -- 172.21.15.5:0/1213876871 shutdown_connections 2024-09-12T02:57:45.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:45.392+0000 7fc006f77640 1 --2- 172.21.15.5:0/1213876871 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc000105030 0x7fc000107420 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:45.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:45.392+0000 7fc006f77640 1 --2- 172.21.15.5:0/1213876871 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc000107960 0x7fc000109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:45.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:45.392+0000 7fc006f77640 1 --2- 172.21.15.5:0/1213876871 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc000102700 0x7fc000104af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:45.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:45.392+0000 7fc006f77640 1 -- 172.21.15.5:0/1213876871 >> 172.21.15.5:0/1213876871 conn(0x7fc0000fc280 msgr2=0x7fc0000fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:45.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:45.392+0000 7fc006f77640 1 -- 172.21.15.5:0/1213876871 shutdown_connections 2024-09-12T02:57:45.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:45.392+0000 7fc006f77640 1 -- 172.21.15.5:0/1213876871 wait complete. 2024-09-12T02:57:45.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:45.392+0000 7fc006f77640 1 Processor -- start 2024-09-12T02:57:45.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:45.392+0000 7fc006f77640 1 -- start start 2024-09-12T02:57:45.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:45.393+0000 7fc006f77640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc000102700 0x7fc000102060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:45.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:45.393+0000 7fc006f77640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc000105030 0x7fc0001006b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:45.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:45.393+0000 7fc006f77640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc000107960 0x7fc000100bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:45.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:45.393+0000 7fc006f77640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc00010d850 con 0x7fc000107960 2024-09-12T02:57:45.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:45.393+0000 7fc006f77640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fc00010d6d0 con 0x7fc000102700 2024-09-12T02:57:45.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:45.393+0000 7fc006f77640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc00010d9d0 con 0x7fc000105030 2024-09-12T02:57:45.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:45.393+0000 7fc005514640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc000107960 0x7fc000100bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:45.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:45.393+0000 7fc005514640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc000107960 0x7fc000100bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37100/0 (socket says 172.21.15.5:37100) 2024-09-12T02:57:45.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:45.393+0000 7fc005514640 1 -- 172.21.15.5:0/4188103828 learned_addr learned my addr 172.21.15.5:0/4188103828 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:57:45.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:45.393+0000 7fc004d13640 1 --2- 172.21.15.5:0/4188103828 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc000102700 0x7fc000102060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:45.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:45.393+0000 7fbffffff640 1 --2- 172.21.15.5:0/4188103828 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc000105030 0x7fc0001006b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:45.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:45.393+0000 7fc005514640 1 -- 172.21.15.5:0/4188103828 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc000105030 msgr2=0x7fc0001006b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:45.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:45.393+0000 7fc005514640 1 --2- 172.21.15.5:0/4188103828 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc000105030 0x7fc0001006b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:45.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:45.393+0000 7fc005514640 1 -- 172.21.15.5:0/4188103828 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc000102700 msgr2=0x7fc000102060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:45.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:45.393+0000 7fc005514640 1 --2- 172.21.15.5:0/4188103828 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc000102700 0x7fc000102060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:45.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:45.393+0000 7fc005514640 1 -- 172.21.15.5:0/4188103828 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0001014b0 con 0x7fc000107960 2024-09-12T02:57:45.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:45.393+0000 7fc005514640 1 --2- 172.21.15.5:0/4188103828 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc000107960 0x7fc000100bf0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fbff8009fd0 tx=0x7fbff800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:45.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:45.393+0000 7fbffdffb640 1 -- 172.21.15.5:0/4188103828 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbff8017070 con 0x7fc000107960 2024-09-12T02:57:45.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:45.394+0000 7fc006f77640 1 -- 172.21.15.5:0/4188103828 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0001a20f0 con 0x7fc000107960 2024-09-12T02:57:45.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:45.394+0000 7fbffdffb640 1 -- 172.21.15.5:0/4188103828 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbff8003e50 con 0x7fc000107960 2024-09-12T02:57:45.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:45.394+0000 7fbffdffb640 1 -- 172.21.15.5:0/4188103828 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbff80106c0 con 0x7fc000107960 2024-09-12T02:57:45.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:45.394+0000 7fc006f77640 1 -- 172.21.15.5:0/4188103828 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0001a25b0 con 0x7fc000107960 2024-09-12T02:57:45.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:45.395+0000 7fc006f77640 1 -- 172.21.15.5:0/4188103828 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0000fdd70 con 0x7fc000107960 2024-09-12T02:57:45.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:45.396+0000 7fbffdffb640 1 -- 172.21.15.5:0/4188103828 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fbff8003970 con 0x7fc000107960 2024-09-12T02:57:45.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:45.396+0000 7fbffdffb640 1 --2- 172.21.15.5:0/4188103828 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fbff00a2c10 0x7fbff00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:45.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:45.397+0000 7fc004d13640 1 --2- 172.21.15.5:0/4188103828 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fbff00a2c10 0x7fbff00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:45.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:45.397+0000 7fbffdffb640 1 -- 172.21.15.5:0/4188103828 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fbff80cf010 con 0x7fc000107960 2024-09-12T02:57:45.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:45.397+0000 7fc004d13640 1 --2- 172.21.15.5:0/4188103828 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fbff00a2c10 0x7fbff00a50d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fbff4005fd0 tx=0x7fbff4005850 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:45.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:45.401+0000 7fbffdffb640 1 -- 172.21.15.5:0/4188103828 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbff809be20 con 0x7fc000107960 2024-09-12T02:57:45.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:45.579+0000 7fc006f77640 1 -- 172.21.15.5:0/4188103828 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7fc00002d7c0 con 0x7fc000107960 2024-09-12T02:57:45.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:45.580+0000 7fbffdffb640 1 -- 172.21.15.5:0/4188103828 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v83) ==== 56+0+5255 (secure 0 0 0) 0x7fbff809b720 con 0x7fc000107960 2024-09-12T02:57:45.581 INFO:teuthology.orchestra.run.smithi005.stdout:epoch 83 2024-09-12T02:57:45.581 INFO:teuthology.orchestra.run.smithi005.stdout:fsid 618305f8-70b1-11ef-bcea-c7b262605968 2024-09-12T02:57:45.581 INFO:teuthology.orchestra.run.smithi005.stdout:created 2024-09-12T02:48:42.352146+0000 2024-09-12T02:57:45.581 INFO:teuthology.orchestra.run.smithi005.stdout:modified 2024-09-12T02:56:31.026079+0000 2024-09-12T02:57:45.581 INFO:teuthology.orchestra.run.smithi005.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-12T02:57:45.581 INFO:teuthology.orchestra.run.smithi005.stdout:crush_version 26 2024-09-12T02:57:45.581 INFO:teuthology.orchestra.run.smithi005.stdout:full_ratio 0.95 2024-09-12T02:57:45.581 INFO:teuthology.orchestra.run.smithi005.stdout:backfillfull_ratio 0.9 2024-09-12T02:57:45.581 INFO:teuthology.orchestra.run.smithi005.stdout:nearfull_ratio 0.85 2024-09-12T02:57:45.582 INFO:teuthology.orchestra.run.smithi005.stdout:require_min_compat_client luminous 2024-09-12T02:57:45.582 INFO:teuthology.orchestra.run.smithi005.stdout:min_compat_client luminous 2024-09-12T02:57:45.582 INFO:teuthology.orchestra.run.smithi005.stdout:require_osd_release squid 2024-09-12T02:57:45.582 INFO:teuthology.orchestra.run.smithi005.stdout:stretch_mode_enabled false 2024-09-12T02:57:45.582 INFO:teuthology.orchestra.run.smithi005.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-09-12T02:57:45.582 INFO:teuthology.orchestra.run.smithi005.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 81 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-09-12T02:57:45.582 INFO:teuthology.orchestra.run.smithi005.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 81 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-09-12T02:57:45.582 INFO:teuthology.orchestra.run.smithi005.stdout:max_osd 12 2024-09-12T02:57:45.583 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0 up in weight 1 up_from 8 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.5:6802/4248507573,v1:172.21.15.5:6803/4248507573] [v2:172.21.15.5:6804/4248507573,v1:172.21.15.5:6805/4248507573] exists,up 17da560a-6aa2-4f2c-a95f-67481dce4d3a 2024-09-12T02:57:45.583 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1 up in weight 1 up_from 13 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.146:6800/1105990287,v1:172.21.15.146:6801/1105990287] [v2:172.21.15.146:6802/1105990287,v1:172.21.15.146:6803/1105990287] exists,up a3237006-99af-4fc1-860c-d4917032647f 2024-09-12T02:57:45.583 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2 up in weight 1 up_from 18 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.148:6800/3378174313,v1:172.21.15.148:6801/3378174313] [v2:172.21.15.148:6802/3378174313,v1:172.21.15.148:6803/3378174313] exists,up ea3ab84a-7739-4975-8244-31ddd5aad80d 2024-09-12T02:57:45.583 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3 up in weight 1 up_from 25 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.5:6810/11741938,v1:172.21.15.5:6811/11741938] [v2:172.21.15.5:6812/11741938,v1:172.21.15.5:6813/11741938] exists,up 4396a9ee-f3f3-410b-9fb1-124f1e33a1f1 2024-09-12T02:57:45.583 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4 up in weight 1 up_from 32 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.146:6808/3274923851,v1:172.21.15.146:6809/3274923851] [v2:172.21.15.146:6810/3274923851,v1:172.21.15.146:6811/3274923851] exists,up 989d76e7-c742-49d0-994a-c58f2771b1d7 2024-09-12T02:57:45.583 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5 up in weight 1 up_from 36 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.148:6808/2771670476,v1:172.21.15.148:6809/2771670476] [v2:172.21.15.148:6810/2771670476,v1:172.21.15.148:6811/2771670476] exists,up 64ada7d4-70a0-4b83-9af8-032be544608d 2024-09-12T02:57:45.583 INFO:teuthology.orchestra.run.smithi005.stdout:osd.6 up in weight 1 up_from 43 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.5:6818/3176073169,v1:172.21.15.5:6819/3176073169] [v2:172.21.15.5:6820/3176073169,v1:172.21.15.5:6821/3176073169] exists,up a70a88fd-2e4b-480c-bb98-62636cf70cc6 2024-09-12T02:57:45.583 INFO:teuthology.orchestra.run.smithi005.stdout:osd.7 up in weight 1 up_from 48 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.146:6816/2209508838,v1:172.21.15.146:6817/2209508838] [v2:172.21.15.146:6818/2209508838,v1:172.21.15.146:6819/2209508838] exists,up 49451e8d-c741-4ad5-b8f5-89bad8d4d129 2024-09-12T02:57:45.583 INFO:teuthology.orchestra.run.smithi005.stdout:osd.8 up in weight 1 up_from 56 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.148:6816/651045756,v1:172.21.15.148:6817/651045756] [v2:172.21.15.148:6818/651045756,v1:172.21.15.148:6819/651045756] exists,up 57bfc121-a75e-4a7f-b1dc-64074b178b03 2024-09-12T02:57:45.583 INFO:teuthology.orchestra.run.smithi005.stdout:osd.9 up in weight 1 up_from 63 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.5:6826/1961857360,v1:172.21.15.5:6827/1961857360] [v2:172.21.15.5:6828/1961857360,v1:172.21.15.5:6829/1961857360] exists,up d9726168-7494-4bf8-b433-350e85bfd461 2024-09-12T02:57:45.584 INFO:teuthology.orchestra.run.smithi005.stdout:osd.10 up in weight 1 up_from 68 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.146:6824/2041733103,v1:172.21.15.146:6825/2041733103] [v2:172.21.15.146:6826/2041733103,v1:172.21.15.146:6827/2041733103] exists,up 519ec60f-3606-48ac-9043-6f59a798766d 2024-09-12T02:57:45.584 INFO:teuthology.orchestra.run.smithi005.stdout:osd.11 up in weight 1 up_from 75 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.148:6824/839290773,v1:172.21.15.148:6825/839290773] [v2:172.21.15.148:6826/839290773,v1:172.21.15.148:6827/839290773] exists,up 0f92d0cb-ba54-4bc3-a6d0-3c54d910556a 2024-09-12T02:57:45.584 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 2.a [2,11] 2024-09-12T02:57:45.584 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 2.e [2,1] 2024-09-12T02:57:45.584 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 2.10 [8,11] 2024-09-12T02:57:45.584 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 2.1b [2,4] 2024-09-12T02:57:45.584 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 3.3 [4,9] 2024-09-12T02:57:45.584 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 3.17 [4,8] 2024-09-12T02:57:45.584 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 3.1b [5,9] 2024-09-12T02:57:45.585 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 3.2c [4,9] 2024-09-12T02:57:45.585 INFO:teuthology.orchestra.run.smithi005.stdout:blocklist 172.21.15.5:6800/3125821798 expires 2024-09-13T02:49:29.462216+0000 2024-09-12T02:57:45.585 INFO:teuthology.orchestra.run.smithi005.stdout:blocklist 172.21.15.5:0/3479911451 expires 2024-09-13T02:49:29.462216+0000 2024-09-12T02:57:45.585 INFO:teuthology.orchestra.run.smithi005.stdout:blocklist 172.21.15.5:0/1547753207 expires 2024-09-13T02:49:03.300872+0000 2024-09-12T02:57:45.585 INFO:teuthology.orchestra.run.smithi005.stdout:blocklist 172.21.15.5:0/1905762627 expires 2024-09-13T02:49:03.300872+0000 2024-09-12T02:57:45.585 INFO:teuthology.orchestra.run.smithi005.stdout:blocklist 172.21.15.5:0/4142340280 expires 2024-09-13T02:49:29.462216+0000 2024-09-12T02:57:45.585 INFO:teuthology.orchestra.run.smithi005.stdout:blocklist 172.21.15.5:6801/3125821798 expires 2024-09-13T02:49:29.462216+0000 2024-09-12T02:57:45.585 INFO:teuthology.orchestra.run.smithi005.stdout:blocklist 172.21.15.5:6801/2595781680 expires 2024-09-13T02:49:03.300872+0000 2024-09-12T02:57:45.585 INFO:teuthology.orchestra.run.smithi005.stdout:blocklist 172.21.15.5:0/2273691311 expires 2024-09-13T02:49:29.462216+0000 2024-09-12T02:57:45.585 INFO:teuthology.orchestra.run.smithi005.stdout:blocklist 172.21.15.5:6800/2595781680 expires 2024-09-13T02:49:03.300872+0000 2024-09-12T02:57:45.586 INFO:teuthology.orchestra.run.smithi005.stdout:blocklist 172.21.15.5:0/1196201217 expires 2024-09-13T02:49:03.300872+0000 2024-09-12T02:57:45.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:45.582+0000 7fc006f77640 1 -- 172.21.15.5:0/4188103828 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fbff00a2c10 msgr2=0x7fbff00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:45.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:45.582+0000 7fc006f77640 1 --2- 172.21.15.5:0/4188103828 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fbff00a2c10 0x7fbff00a50d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fbff4005fd0 tx=0x7fbff4005850 comp rx=0 tx=0).stop 2024-09-12T02:57:45.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:45.582+0000 7fc006f77640 1 -- 172.21.15.5:0/4188103828 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc000107960 msgr2=0x7fc000100bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:45.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:45.582+0000 7fc006f77640 1 --2- 172.21.15.5:0/4188103828 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc000107960 0x7fc000100bf0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fbff8009fd0 tx=0x7fbff800b260 comp rx=0 tx=0).stop 2024-09-12T02:57:45.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:45.583+0000 7fc006f77640 1 -- 172.21.15.5:0/4188103828 shutdown_connections 2024-09-12T02:57:45.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:45.583+0000 7fc006f77640 1 --2- 172.21.15.5:0/4188103828 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fbff00a2c10 0x7fbff00a50d0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:45.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:45.583+0000 7fc006f77640 1 --2- 172.21.15.5:0/4188103828 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc000107960 0x7fc000100bf0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:45.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:45.583+0000 7fc006f77640 1 --2- 172.21.15.5:0/4188103828 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc000105030 0x7fc0001006b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:45.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:45.583+0000 7fc006f77640 1 --2- 172.21.15.5:0/4188103828 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc000102700 0x7fc000102060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:45.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:45.583+0000 7fc006f77640 1 -- 172.21.15.5:0/4188103828 >> 172.21.15.5:0/4188103828 conn(0x7fc0000fc280 msgr2=0x7fc000103080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:45.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:45.583+0000 7fc006f77640 1 -- 172.21.15.5:0/4188103828 shutdown_connections 2024-09-12T02:57:45.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:45.583+0000 7fc006f77640 1 -- 172.21.15.5:0/4188103828 wait complete. 2024-09-12T02:57:45.665 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:57:45 smithi148 ceph-mon[35090]: pgmap v320: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-12T02:57:45.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:57:45 smithi146 ceph-mon[35011]: pgmap v320: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-12T02:57:46.201 INFO:teuthology.run_tasks:Running task fs.ready... 2024-09-12T02:57:46.213 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:57:46.472 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:57:46.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:57:46 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/4188103828' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-12T02:57:46.665 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:57:46 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/4188103828' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-12T02:57:46.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:57:46 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/4188103828' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-12T02:57:46.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:46.984+0000 7fafc2b80640 1 -- 172.21.15.5:0/3389281999 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fafbc10c170 msgr2=0x7fafbc10e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:46.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:46.984+0000 7fafc2b80640 1 --2- 172.21.15.5:0/3389281999 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fafbc10c170 0x7fafbc10e610 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fafb4009ca0 tx=0x7fafb402a950 comp rx=0 tx=0).stop 2024-09-12T02:57:46.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:46.984+0000 7fafc2b80640 1 -- 172.21.15.5:0/3389281999 shutdown_connections 2024-09-12T02:57:46.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:46.984+0000 7fafc2b80640 1 --2- 172.21.15.5:0/3389281999 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fafbc10c170 0x7fafbc10e610 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:46.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:46.984+0000 7fafc2b80640 1 --2- 172.21.15.5:0/3389281999 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fafbc103660 0x7fafbc10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:46.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:46.985+0000 7fafc2b80640 1 --2- 172.21.15.5:0/3389281999 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fafbc102d40 0x7fafbc103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:46.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:46.985+0000 7fafc2b80640 1 -- 172.21.15.5:0/3389281999 >> 172.21.15.5:0/3389281999 conn(0x7fafbc0fc7a0 msgr2=0x7fafbc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:46.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:46.985+0000 7fafc2b80640 1 -- 172.21.15.5:0/3389281999 shutdown_connections 2024-09-12T02:57:46.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:46.985+0000 7fafc2b80640 1 -- 172.21.15.5:0/3389281999 wait complete. 2024-09-12T02:57:46.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:46.985+0000 7fafc2b80640 1 Processor -- start 2024-09-12T02:57:46.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:46.985+0000 7fafc2b80640 1 -- start start 2024-09-12T02:57:46.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:46.985+0000 7fafc2b80640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fafbc102d40 0x7fafbc19a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:46.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:46.985+0000 7fafc2b80640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fafbc103660 0x7fafbc19ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:46.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:46.985+0000 7fafc2b80640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fafbc10c170 0x7fafbc19eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:46.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:46.985+0000 7fafc2b80640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fafbc111f70 con 0x7fafbc102d40 2024-09-12T02:57:46.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:46.985+0000 7fafc2b80640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fafbc111df0 con 0x7fafbc103660 2024-09-12T02:57:46.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:46.985+0000 7fafc2b80640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fafbc1120f0 con 0x7fafbc10c170 2024-09-12T02:57:46.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:46.986+0000 7fafc091c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fafbc102d40 0x7fafbc19a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:46.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:46.986+0000 7fafc091c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fafbc102d40 0x7fafbc19a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37114/0 (socket says 172.21.15.5:37114) 2024-09-12T02:57:46.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:46.986+0000 7fafc091c640 1 -- 172.21.15.5:0/758084585 learned_addr learned my addr 172.21.15.5:0/758084585 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:57:46.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:46.986+0000 7fafbbfff640 1 --2- 172.21.15.5:0/758084585 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fafbc103660 0x7fafbc19ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:46.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:46.986+0000 7fafc111d640 1 --2- 172.21.15.5:0/758084585 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fafbc10c170 0x7fafbc19eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:46.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:46.986+0000 7fafc091c640 1 -- 172.21.15.5:0/758084585 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fafbc10c170 msgr2=0x7fafbc19eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:46.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:46.986+0000 7fafc091c640 1 --2- 172.21.15.5:0/758084585 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fafbc10c170 0x7fafbc19eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:46.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:46.986+0000 7fafc091c640 1 -- 172.21.15.5:0/758084585 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fafbc103660 msgr2=0x7fafbc19ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:46.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:46.986+0000 7fafc091c640 1 --2- 172.21.15.5:0/758084585 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fafbc103660 0x7fafbc19ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:46.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:46.986+0000 7fafc091c640 1 -- 172.21.15.5:0/758084585 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafbc19f590 con 0x7fafbc102d40 2024-09-12T02:57:46.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:46.986+0000 7fafc091c640 1 --2- 172.21.15.5:0/758084585 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fafbc102d40 0x7fafbc19a5e0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fafb000d950 tx=0x7fafb000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:46.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:46.986+0000 7fafb9ffb640 1 -- 172.21.15.5:0/758084585 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb0017070 con 0x7fafbc102d40 2024-09-12T02:57:46.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:46.986+0000 7fafb9ffb640 1 -- 172.21.15.5:0/758084585 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fafb0009c30 con 0x7fafbc102d40 2024-09-12T02:57:46.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:46.986+0000 7fafc2b80640 1 -- 172.21.15.5:0/758084585 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafbc19f880 con 0x7fafbc102d40 2024-09-12T02:57:46.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:46.986+0000 7fafb9ffb640 1 -- 172.21.15.5:0/758084585 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb000f6b0 con 0x7fafbc102d40 2024-09-12T02:57:46.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:46.987+0000 7fafc2b80640 1 -- 172.21.15.5:0/758084585 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafbc1a7160 con 0x7fafbc102d40 2024-09-12T02:57:46.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:46.987+0000 7fafc2b80640 1 -- 172.21.15.5:0/758084585 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafbc103120 con 0x7fafbc102d40 2024-09-12T02:57:46.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:46.989+0000 7fafb9ffb640 1 -- 172.21.15.5:0/758084585 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fafb00109d0 con 0x7fafbc102d40 2024-09-12T02:57:46.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:46.989+0000 7fafb9ffb640 1 --2- 172.21.15.5:0/758084585 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fafac0a2c10 0x7fafac0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:46.990 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:46.989+0000 7fafbbfff640 1 --2- 172.21.15.5:0/758084585 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fafac0a2c10 0x7fafac0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:46.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:46.990+0000 7fafb9ffb640 1 -- 172.21.15.5:0/758084585 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fafb00ced50 con 0x7fafbc102d40 2024-09-12T02:57:46.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:46.990+0000 7fafbbfff640 1 --2- 172.21.15.5:0/758084585 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fafac0a2c10 0x7fafac0a50d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fafbc10dfd0 tx=0x7fafa8005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:46.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:46.992+0000 7fafb9ffb640 1 -- 172.21.15.5:0/758084585 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fafb0009de0 con 0x7fafbc102d40 2024-09-12T02:57:47.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:47.204+0000 7fafc2b80640 1 -- 172.21.15.5:0/758084585 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fafbc1a7400 con 0x7fafbc102d40 2024-09-12T02:57:47.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:47.205+0000 7fafb9ffb640 1 -- 172.21.15.5:0/758084585 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11966 (secure 0 0 0) 0x7fafb009b510 con 0x7fafbc102d40 2024-09-12T02:57:47.206 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-12T02:57:47.206 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":25,"btime":"2024-09-12T02:56:27:840212+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/4135232521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":4135232521},{"type":"v1","addr":"172.21.15.5:6835","nonce":4135232521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/4290221940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":4290221940},{"type":"v1","addr":"172.21.15.5:6837","nonce":4290221940}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T02:56:16.595562+0000","modified":"2024-09-12T02:56:27.840207+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24502,"mds_1":24487,"mds_2":24481,"mds_3":24479,"mds_4":24458},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/26447341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":26447341},{"type":"v1","addr":"172.21.15.5:6839","nonce":26447341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2393443295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2393443295},{"type":"v1","addr":"172.21.15.5:6841","nonce":2393443295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6833/3686970137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3686970137},{"type":"v1","addr":"172.21.15.146:6833","nonce":3686970137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6833/568512209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":568512209},{"type":"v1","addr":"172.21.15.148:6833","nonce":568512209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6835/2540740420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2540740420},{"type":"v1","addr":"172.21.15.146:6835","nonce":2540740420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.146:6837/1673370937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":1673370937},{"type":"v1","addr":"172.21.15.146:6837","nonce":1673370937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":3,"incarnation":22,"state":"up:active","state_seq":6,"addr":"172.21.15.146:6839/3036767228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":3036767228},{"type":"v1","addr":"172.21.15.146:6839","nonce":3036767228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":2,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.148:6835/4195086427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":4195086427},{"type":"v1","addr":"172.21.15.148:6835","nonce":4195086427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.148:6837/2606873922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2606873922},{"type":"v1","addr":"172.21.15.148:6837","nonce":2606873922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6839/2317600000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":2317600000},{"type":"v1","addr":"172.21.15.148:6839","nonce":2317600000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24502,"qdb_cluster":[24458,24479,24481,24487,24502]},"id":1}]} 2024-09-12T02:57:47.206 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 25 2024-09-12T02:57:47.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:47.207+0000 7fafc2b80640 1 -- 172.21.15.5:0/758084585 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fafac0a2c10 msgr2=0x7fafac0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:47.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:47.207+0000 7fafc2b80640 1 --2- 172.21.15.5:0/758084585 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fafac0a2c10 0x7fafac0a50d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fafbc10dfd0 tx=0x7fafa8005f90 comp rx=0 tx=0).stop 2024-09-12T02:57:47.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:47.207+0000 7fafc2b80640 1 -- 172.21.15.5:0/758084585 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fafbc102d40 msgr2=0x7fafbc19a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:47.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:47.207+0000 7fafc2b80640 1 --2- 172.21.15.5:0/758084585 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fafbc102d40 0x7fafbc19a5e0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fafb000d950 tx=0x7fafb000de10 comp rx=0 tx=0).stop 2024-09-12T02:57:47.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:47.207+0000 7fafc2b80640 1 -- 172.21.15.5:0/758084585 shutdown_connections 2024-09-12T02:57:47.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:47.207+0000 7fafc2b80640 1 --2- 172.21.15.5:0/758084585 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fafbc10c170 0x7fafbc19eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:47.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:47.207+0000 7fafc2b80640 1 --2- 172.21.15.5:0/758084585 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fafbc103660 0x7fafbc19ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:47.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:47.207+0000 7fafc2b80640 1 --2- 172.21.15.5:0/758084585 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fafac0a2c10 0x7fafac0a50d0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:47.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:47.207+0000 7fafc2b80640 1 --2- 172.21.15.5:0/758084585 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fafbc102d40 0x7fafbc19a5e0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:47.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:47.207+0000 7fafc2b80640 1 -- 172.21.15.5:0/758084585 >> 172.21.15.5:0/758084585 conn(0x7fafbc0fc7a0 msgr2=0x7fafbc10d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:47.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:47.207+0000 7fafc2b80640 1 -- 172.21.15.5:0/758084585 shutdown_connections 2024-09-12T02:57:47.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:47.207+0000 7fafc2b80640 1 -- 172.21.15.5:0/758084585 wait complete. 2024-09-12T02:57:47.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:57:47 smithi005 ceph-mon[31197]: pgmap v321: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-12T02:57:47.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:57:47 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/758084585' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:57:47.665 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:57:47 smithi148 ceph-mon[35090]: pgmap v321: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-12T02:57:47.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:57:47 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/758084585' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:57:47.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:57:47 smithi146 ceph-mon[35011]: pgmap v321: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-12T02:57:47.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:57:47 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/758084585' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:57:47.834 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:57:48.095 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:57:48.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:48.572+0000 7f28a3674640 1 -- 172.21.15.5:0/3530627484 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f289c101de0 msgr2=0x7f289c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:48.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:48.572+0000 7f28a3674640 1 --2- 172.21.15.5:0/3530627484 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f289c101de0 0x7f289c1021c0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f2898009ca0 tx=0x7f289802a8c0 comp rx=0 tx=0).stop 2024-09-12T02:57:48.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:48.572+0000 7f28a3674640 1 -- 172.21.15.5:0/3530627484 shutdown_connections 2024-09-12T02:57:48.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:48.572+0000 7f28a3674640 1 --2- 172.21.15.5:0/3530627484 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f289c106f10 0x7f289c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:48.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:48.572+0000 7f28a3674640 1 --2- 172.21.15.5:0/3530627484 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f289c102790 0x7f289c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:48.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:48.572+0000 7f28a3674640 1 --2- 172.21.15.5:0/3530627484 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f289c101de0 0x7f289c1021c0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:48.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:48.572+0000 7f28a3674640 1 -- 172.21.15.5:0/3530627484 >> 172.21.15.5:0/3530627484 conn(0x7f289c0fd8c0 msgr2=0x7f289c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:48.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:48.572+0000 7f28a3674640 1 -- 172.21.15.5:0/3530627484 shutdown_connections 2024-09-12T02:57:48.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:48.572+0000 7f28a3674640 1 -- 172.21.15.5:0/3530627484 wait complete. 2024-09-12T02:57:48.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:48.573+0000 7f28a3674640 1 Processor -- start 2024-09-12T02:57:48.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:48.573+0000 7f28a3674640 1 -- start start 2024-09-12T02:57:48.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:48.573+0000 7f28a3674640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f289c101de0 0x7f289c10deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:48.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:48.573+0000 7f28a3674640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f289c102790 0x7f289c10e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:48.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:48.573+0000 7f28a3674640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f289c106f10 0x7f289c1a34e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:48.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:48.573+0000 7f28a3674640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f289c112010 con 0x7f289c102790 2024-09-12T02:57:48.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:48.573+0000 7f28a3674640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f289c111e90 con 0x7f289c106f10 2024-09-12T02:57:48.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:48.573+0000 7f28a3674640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f289c112190 con 0x7f289c101de0 2024-09-12T02:57:48.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:48.573+0000 7f28a0c0f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f289c102790 0x7f289c10e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:48.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:48.573+0000 7f28a0c0f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f289c102790 0x7f289c10e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37138/0 (socket says 172.21.15.5:37138) 2024-09-12T02:57:48.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:48.573+0000 7f28a0c0f640 1 -- 172.21.15.5:0/2238550493 learned_addr learned my addr 172.21.15.5:0/2238550493 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:57:48.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:48.573+0000 7f28a1410640 1 --2- 172.21.15.5:0/2238550493 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f289c101de0 0x7f289c10deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:48.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:48.573+0000 7f28a1c11640 1 --2- 172.21.15.5:0/2238550493 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f289c106f10 0x7f289c1a34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:48.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:48.573+0000 7f28a0c0f640 1 -- 172.21.15.5:0/2238550493 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f289c101de0 msgr2=0x7f289c10deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:48.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:48.573+0000 7f28a0c0f640 1 --2- 172.21.15.5:0/2238550493 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f289c101de0 0x7f289c10deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:48.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:48.573+0000 7f28a0c0f640 1 -- 172.21.15.5:0/2238550493 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f289c106f10 msgr2=0x7f289c1a34e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:48.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:48.573+0000 7f28a0c0f640 1 --2- 172.21.15.5:0/2238550493 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f289c106f10 0x7f289c1a34e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:48.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:48.573+0000 7f28a0c0f640 1 -- 172.21.15.5:0/2238550493 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f289c1a3c10 con 0x7f289c102790 2024-09-12T02:57:48.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:48.573+0000 7f28a1410640 1 --2- 172.21.15.5:0/2238550493 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f289c101de0 0x7f289c10deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:57:48.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:48.573+0000 7f28a1c11640 1 --2- 172.21.15.5:0/2238550493 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f289c106f10 0x7f289c1a34e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:57:48.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:48.574+0000 7f28a0c0f640 1 --2- 172.21.15.5:0/2238550493 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f289c102790 0x7f289c10e3f0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f289000b4a0 tx=0x7f289000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:48.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:48.574+0000 7f288e7fc640 1 -- 172.21.15.5:0/2238550493 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2890014070 con 0x7f289c102790 2024-09-12T02:57:48.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:48.574+0000 7f288e7fc640 1 -- 172.21.15.5:0/2238550493 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2890010af0 con 0x7f289c102790 2024-09-12T02:57:48.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:48.574+0000 7f288e7fc640 1 -- 172.21.15.5:0/2238550493 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2890017d10 con 0x7f289c102790 2024-09-12T02:57:48.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:48.574+0000 7f28a3674640 1 -- 172.21.15.5:0/2238550493 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f289c1a3f00 con 0x7f289c102790 2024-09-12T02:57:48.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:48.574+0000 7f28a3674640 1 -- 172.21.15.5:0/2238550493 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f289c1ab790 con 0x7f289c102790 2024-09-12T02:57:48.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:48.575+0000 7f28a3674640 1 -- 172.21.15.5:0/2238550493 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f289c102240 con 0x7f289c102790 2024-09-12T02:57:48.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:48.576+0000 7f288e7fc640 1 -- 172.21.15.5:0/2238550493 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f2890017070 con 0x7f289c102790 2024-09-12T02:57:48.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:48.577+0000 7f288e7fc640 1 --2- 172.21.15.5:0/2238550493 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f28880a2c90 0x7f28880a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:48.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:48.577+0000 7f28a1410640 1 --2- 172.21.15.5:0/2238550493 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f28880a2c90 0x7f28880a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:48.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:48.577+0000 7f288e7fc640 1 -- 172.21.15.5:0/2238550493 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f28900ce4b0 con 0x7f289c102790 2024-09-12T02:57:48.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:48.577+0000 7f28a1410640 1 --2- 172.21.15.5:0/2238550493 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f28880a2c90 0x7f28880a5150 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f2898000f80 tx=0x7f2898005ef0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:48.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:48.581+0000 7f288e7fc640 1 -- 172.21.15.5:0/2238550493 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f289009b370 con 0x7f289c102790 2024-09-12T02:57:48.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:48.791+0000 7f28a3674640 1 -- 172.21.15.5:0/2238550493 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f289c105ed0 con 0x7f289c102790 2024-09-12T02:57:48.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:48.792+0000 7f288e7fc640 1 -- 172.21.15.5:0/2238550493 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11966 (secure 0 0 0) 0x7f289009ac70 con 0x7f289c102790 2024-09-12T02:57:48.793 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-12T02:57:48.795 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":25,"btime":"2024-09-12T02:56:27:840212+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/4135232521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":4135232521},{"type":"v1","addr":"172.21.15.5:6835","nonce":4135232521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/4290221940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":4290221940},{"type":"v1","addr":"172.21.15.5:6837","nonce":4290221940}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T02:56:16.595562+0000","modified":"2024-09-12T02:56:27.840207+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24502,"mds_1":24487,"mds_2":24481,"mds_3":24479,"mds_4":24458},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/26447341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":26447341},{"type":"v1","addr":"172.21.15.5:6839","nonce":26447341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2393443295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2393443295},{"type":"v1","addr":"172.21.15.5:6841","nonce":2393443295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6833/3686970137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3686970137},{"type":"v1","addr":"172.21.15.146:6833","nonce":3686970137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6833/568512209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":568512209},{"type":"v1","addr":"172.21.15.148:6833","nonce":568512209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6835/2540740420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2540740420},{"type":"v1","addr":"172.21.15.146:6835","nonce":2540740420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.146:6837/1673370937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":1673370937},{"type":"v1","addr":"172.21.15.146:6837","nonce":1673370937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":3,"incarnation":22,"state":"up:active","state_seq":6,"addr":"172.21.15.146:6839/3036767228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":3036767228},{"type":"v1","addr":"172.21.15.146:6839","nonce":3036767228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":2,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.148:6835/4195086427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":4195086427},{"type":"v1","addr":"172.21.15.148:6835","nonce":4195086427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.148:6837/2606873922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2606873922},{"type":"v1","addr":"172.21.15.148:6837","nonce":2606873922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6839/2317600000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":2317600000},{"type":"v1","addr":"172.21.15.148:6839","nonce":2317600000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24502,"qdb_cluster":[24458,24479,24481,24487,24502]},"id":1}]} 2024-09-12T02:57:48.795 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 25 2024-09-12T02:57:48.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:48.794+0000 7f28a3674640 1 -- 172.21.15.5:0/2238550493 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f28880a2c90 msgr2=0x7f28880a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:48.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:48.794+0000 7f28a3674640 1 --2- 172.21.15.5:0/2238550493 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f28880a2c90 0x7f28880a5150 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f2898000f80 tx=0x7f2898005ef0 comp rx=0 tx=0).stop 2024-09-12T02:57:48.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:48.794+0000 7f28a3674640 1 -- 172.21.15.5:0/2238550493 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f289c102790 msgr2=0x7f289c10e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:48.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:48.794+0000 7f28a3674640 1 --2- 172.21.15.5:0/2238550493 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f289c102790 0x7f289c10e3f0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f289000b4a0 tx=0x7f289000b960 comp rx=0 tx=0).stop 2024-09-12T02:57:48.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:48.794+0000 7f28a3674640 1 -- 172.21.15.5:0/2238550493 shutdown_connections 2024-09-12T02:57:48.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:48.794+0000 7f28a3674640 1 --2- 172.21.15.5:0/2238550493 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f28880a2c90 0x7f28880a5150 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:48.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:48.794+0000 7f28a3674640 1 --2- 172.21.15.5:0/2238550493 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f289c102790 0x7f289c10e3f0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:48.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:48.794+0000 7f28a3674640 1 --2- 172.21.15.5:0/2238550493 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f289c106f10 0x7f289c1a34e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:48.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:48.794+0000 7f28a3674640 1 --2- 172.21.15.5:0/2238550493 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f289c101de0 0x7f289c10deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:48.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:48.794+0000 7f28a3674640 1 -- 172.21.15.5:0/2238550493 >> 172.21.15.5:0/2238550493 conn(0x7f289c0fd8c0 msgr2=0x7f289c0fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:48.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:48.794+0000 7f28a3674640 1 -- 172.21.15.5:0/2238550493 shutdown_connections 2024-09-12T02:57:48.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:48.794+0000 7f28a3674640 1 -- 172.21.15.5:0/2238550493 wait complete. 2024-09-12T02:57:49.420 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:57:49 smithi005 ceph-mon[31197]: pgmap v322: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-12T02:57:49.420 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:57:49 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/2238550493' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:57:49.452 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T02:57:49.665 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:57:49 smithi148 ceph-mon[35090]: pgmap v322: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-12T02:57:49.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:57:49 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/2238550493' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:57:49.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:57:49 smithi146 ceph-mon[35011]: pgmap v322: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-12T02:57:49.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:57:49 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/2238550493' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:57:49.711 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:57:50.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:50.150+0000 7f857da3e640 1 -- 172.21.15.5:0/825162544 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f857810c170 msgr2=0x7f857810e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:50.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:50.150+0000 7f857da3e640 1 --2- 172.21.15.5:0/825162544 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f857810c170 0x7f857810e610 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f8570009ca0 tx=0x7f857002a950 comp rx=0 tx=0).stop 2024-09-12T02:57:50.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:50.150+0000 7f857da3e640 1 -- 172.21.15.5:0/825162544 shutdown_connections 2024-09-12T02:57:50.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:50.150+0000 7f857da3e640 1 --2- 172.21.15.5:0/825162544 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f857810c170 0x7f857810e610 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:50.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:50.150+0000 7f857da3e640 1 --2- 172.21.15.5:0/825162544 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8578103660 0x7f857810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:50.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:50.150+0000 7f857da3e640 1 --2- 172.21.15.5:0/825162544 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8578102d40 0x7f8578103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:50.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:50.150+0000 7f857da3e640 1 -- 172.21.15.5:0/825162544 >> 172.21.15.5:0/825162544 conn(0x7f85780fc7a0 msgr2=0x7f85780febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:50.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:50.150+0000 7f857da3e640 1 -- 172.21.15.5:0/825162544 shutdown_connections 2024-09-12T02:57:50.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:50.150+0000 7f857da3e640 1 -- 172.21.15.5:0/825162544 wait complete. 2024-09-12T02:57:50.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:50.151+0000 7f857da3e640 1 Processor -- start 2024-09-12T02:57:50.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:50.151+0000 7f857da3e640 1 -- start start 2024-09-12T02:57:50.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:50.152+0000 7f857da3e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8578102d40 0x7f8578198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:50.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:50.152+0000 7f857da3e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8578103660 0x7f8578198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:50.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:50.152+0000 7f857da3e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f857810c170 0x7f857819cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:50.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:50.152+0000 7f857da3e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8578111f70 con 0x7f8578103660 2024-09-12T02:57:50.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:50.152+0000 7f857da3e640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f8578111df0 con 0x7f8578102d40 2024-09-12T02:57:50.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:50.152+0000 7f857da3e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f85781120f0 con 0x7f857810c170 2024-09-12T02:57:50.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:50.152+0000 7f85767fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8578103660 0x7f8578198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:50.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:50.152+0000 7f85767fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8578103660 0x7f8578198870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37160/0 (socket says 172.21.15.5:37160) 2024-09-12T02:57:50.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:50.152+0000 7f85767fc640 1 -- 172.21.15.5:0/213726364 learned_addr learned my addr 172.21.15.5:0/213726364 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:57:50.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:50.152+0000 7f8576ffd640 1 --2- 172.21.15.5:0/213726364 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8578102d40 0x7f8578198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:50.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:50.152+0000 7f85777fe640 1 --2- 172.21.15.5:0/213726364 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f857810c170 0x7f857819cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:50.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:50.152+0000 7f85767fc640 1 -- 172.21.15.5:0/213726364 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f857810c170 msgr2=0x7f857819cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:50.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:50.152+0000 7f85767fc640 1 --2- 172.21.15.5:0/213726364 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f857810c170 0x7f857819cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:50.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:50.152+0000 7f85767fc640 1 -- 172.21.15.5:0/213726364 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8578102d40 msgr2=0x7f8578198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:50.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:50.152+0000 7f85767fc640 1 --2- 172.21.15.5:0/213726364 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8578102d40 0x7f8578198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:50.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:50.152+0000 7f85767fc640 1 -- 172.21.15.5:0/213726364 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f857819d2e0 con 0x7f8578103660 2024-09-12T02:57:50.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:50.152+0000 7f85767fc640 1 --2- 172.21.15.5:0/213726364 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8578103660 0x7f8578198870 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f856c00c970 tx=0x7f856c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:50.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:50.152+0000 7f857ca3c640 1 -- 172.21.15.5:0/213726364 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f856c007bb0 con 0x7f8578103660 2024-09-12T02:57:50.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:50.152+0000 7f857ca3c640 1 -- 172.21.15.5:0/213726364 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f856c007d50 con 0x7f8578103660 2024-09-12T02:57:50.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:50.152+0000 7f857da3e640 1 -- 172.21.15.5:0/213726364 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f857819d5d0 con 0x7f8578103660 2024-09-12T02:57:50.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:50.152+0000 7f857ca3c640 1 -- 172.21.15.5:0/213726364 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f856c018820 con 0x7f8578103660 2024-09-12T02:57:50.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:50.152+0000 7f857da3e640 1 -- 172.21.15.5:0/213726364 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85781a4eb0 con 0x7f8578103660 2024-09-12T02:57:50.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:50.153+0000 7f857da3e640 1 -- 172.21.15.5:0/213726364 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8578103120 con 0x7f8578103660 2024-09-12T02:57:50.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:50.154+0000 7f857ca3c640 1 -- 172.21.15.5:0/213726364 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f856c0189c0 con 0x7f8578103660 2024-09-12T02:57:50.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:50.155+0000 7f857ca3c640 1 --2- 172.21.15.5:0/213726364 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f85600a2c90 0x7f85600a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:50.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:50.155+0000 7f8576ffd640 1 --2- 172.21.15.5:0/213726364 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f85600a2c90 0x7f85600a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:50.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:50.155+0000 7f857ca3c640 1 -- 172.21.15.5:0/213726364 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f856c0cf220 con 0x7f8578103660 2024-09-12T02:57:50.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:50.156+0000 7f8576ffd640 1 --2- 172.21.15.5:0/213726364 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f85600a2c90 0x7f85600a5150 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f8568005fd0 tx=0x7f8568005850 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:50.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:50.159+0000 7f857ca3c640 1 -- 172.21.15.5:0/213726364 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f856c014030 con 0x7f8578103660 2024-09-12T02:57:50.339 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:50.339+0000 7f857da3e640 1 -- 172.21.15.5:0/213726364 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f85781a51b0 con 0x7f8578103660 2024-09-12T02:57:50.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:50.340+0000 7f857ca3c640 1 -- 172.21.15.5:0/213726364 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+22656 (secure 0 0 0) 0x7f856c01f300 con 0x7f8578103660 2024-09-12T02:57:50.341 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-12T02:57:50.341 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":83,"fsid":"618305f8-70b1-11ef-bcea-c7b262605968","created":"2024-09-12T02:48:42.352146+0000","modified":"2024-09-12T02:56:31.026079+0000","last_up_change":"2024-09-12T02:55:27.817161+0000","last_in_change":"2024-09-12T02:55:03.745281+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T02:51:47.970639+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T02:56:12.834271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T02:56:14.951437+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17da560a-6aa2-4f2c-a95f-67481dce4d3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":4248507573},{"type":"v1","addr":"172.21.15.5:6803","nonce":4248507573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":4248507573},{"type":"v1","addr":"172.21.15.5:6805","nonce":4248507573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":4248507573},{"type":"v1","addr":"172.21.15.5:6809","nonce":4248507573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":4248507573},{"type":"v1","addr":"172.21.15.5:6807","nonce":4248507573}]},"public_addr":"172.21.15.5:6803/4248507573","cluster_addr":"172.21.15.5:6805/4248507573","heartbeat_back_addr":"172.21.15.5:6809/4248507573","heartbeat_front_addr":"172.21.15.5:6807/4248507573","state":["exists","up"]},{"osd":1,"uuid":"a3237006-99af-4fc1-860c-d4917032647f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":1105990287},{"type":"v1","addr":"172.21.15.146:6801","nonce":1105990287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":1105990287},{"type":"v1","addr":"172.21.15.146:6803","nonce":1105990287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":1105990287},{"type":"v1","addr":"172.21.15.146:6807","nonce":1105990287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":1105990287},{"type":"v1","addr":"172.21.15.146:6805","nonce":1105990287}]},"public_addr":"172.21.15.146:6801/1105990287","cluster_addr":"172.21.15.146:6803/1105990287","heartbeat_back_addr":"172.21.15.146:6807/1105990287","heartbeat_front_addr":"172.21.15.146:6805/1105990287","state":["exists","up"]},{"osd":2,"uuid":"ea3ab84a-7739-4975-8244-31ddd5aad80d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3378174313},{"type":"v1","addr":"172.21.15.148:6801","nonce":3378174313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3378174313},{"type":"v1","addr":"172.21.15.148:6803","nonce":3378174313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3378174313},{"type":"v1","addr":"172.21.15.148:6807","nonce":3378174313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3378174313},{"type":"v1","addr":"172.21.15.148:6805","nonce":3378174313}]},"public_addr":"172.21.15.148:6801/3378174313","cluster_addr":"172.21.15.148:6803/3378174313","heartbeat_back_addr":"172.21.15.148:6807/3378174313","heartbeat_front_addr":"172.21.15.148:6805/3378174313","state":["exists","up"]},{"osd":3,"uuid":"4396a9ee-f3f3-410b-9fb1-124f1e33a1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":11741938},{"type":"v1","addr":"172.21.15.5:6811","nonce":11741938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":11741938},{"type":"v1","addr":"172.21.15.5:6813","nonce":11741938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":11741938},{"type":"v1","addr":"172.21.15.5:6817","nonce":11741938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":11741938},{"type":"v1","addr":"172.21.15.5:6815","nonce":11741938}]},"public_addr":"172.21.15.5:6811/11741938","cluster_addr":"172.21.15.5:6813/11741938","heartbeat_back_addr":"172.21.15.5:6817/11741938","heartbeat_front_addr":"172.21.15.5:6815/11741938","state":["exists","up"]},{"osd":4,"uuid":"989d76e7-c742-49d0-994a-c58f2771b1d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":3274923851},{"type":"v1","addr":"172.21.15.146:6809","nonce":3274923851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":3274923851},{"type":"v1","addr":"172.21.15.146:6811","nonce":3274923851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3274923851},{"type":"v1","addr":"172.21.15.146:6815","nonce":3274923851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":3274923851},{"type":"v1","addr":"172.21.15.146:6813","nonce":3274923851}]},"public_addr":"172.21.15.146:6809/3274923851","cluster_addr":"172.21.15.146:6811/3274923851","heartbeat_back_addr":"172.21.15.146:6815/3274923851","heartbeat_front_addr":"172.21.15.146:6813/3274923851","state":["exists","up"]},{"osd":5,"uuid":"64ada7d4-70a0-4b83-9af8-032be544608d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":2771670476},{"type":"v1","addr":"172.21.15.148:6809","nonce":2771670476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":2771670476},{"type":"v1","addr":"172.21.15.148:6811","nonce":2771670476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":2771670476},{"type":"v1","addr":"172.21.15.148:6815","nonce":2771670476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":2771670476},{"type":"v1","addr":"172.21.15.148:6813","nonce":2771670476}]},"public_addr":"172.21.15.148:6809/2771670476","cluster_addr":"172.21.15.148:6811/2771670476","heartbeat_back_addr":"172.21.15.148:6815/2771670476","heartbeat_front_addr":"172.21.15.148:6813/2771670476","state":["exists","up"]},{"osd":6,"uuid":"a70a88fd-2e4b-480c-bb98-62636cf70cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3176073169},{"type":"v1","addr":"172.21.15.5:6819","nonce":3176073169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":3176073169},{"type":"v1","addr":"172.21.15.5:6821","nonce":3176073169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":3176073169},{"type":"v1","addr":"172.21.15.5:6825","nonce":3176073169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3176073169},{"type":"v1","addr":"172.21.15.5:6823","nonce":3176073169}]},"public_addr":"172.21.15.5:6819/3176073169","cluster_addr":"172.21.15.5:6821/3176073169","heartbeat_back_addr":"172.21.15.5:6825/3176073169","heartbeat_front_addr":"172.21.15.5:6823/3176073169","state":["exists","up"]},{"osd":7,"uuid":"49451e8d-c741-4ad5-b8f5-89bad8d4d129","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2209508838},{"type":"v1","addr":"172.21.15.146:6817","nonce":2209508838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2209508838},{"type":"v1","addr":"172.21.15.146:6819","nonce":2209508838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2209508838},{"type":"v1","addr":"172.21.15.146:6823","nonce":2209508838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2209508838},{"type":"v1","addr":"172.21.15.146:6821","nonce":2209508838}]},"public_addr":"172.21.15.146:6817/2209508838","cluster_addr":"172.21.15.146:6819/2209508838","heartbeat_back_addr":"172.21.15.146:6823/2209508838","heartbeat_front_addr":"172.21.15.146:6821/2209508838","state":["exists","up"]},{"osd":8,"uuid":"57bfc121-a75e-4a7f-b1dc-64074b178b03","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":651045756},{"type":"v1","addr":"172.21.15.148:6817","nonce":651045756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":651045756},{"type":"v1","addr":"172.21.15.148:6819","nonce":651045756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":651045756},{"type":"v1","addr":"172.21.15.148:6823","nonce":651045756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":651045756},{"type":"v1","addr":"172.21.15.148:6821","nonce":651045756}]},"public_addr":"172.21.15.148:6817/651045756","cluster_addr":"172.21.15.148:6819/651045756","heartbeat_back_addr":"172.21.15.148:6823/651045756","heartbeat_front_addr":"172.21.15.148:6821/651045756","state":["exists","up"]},{"osd":9,"uuid":"d9726168-7494-4bf8-b433-350e85bfd461","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1961857360},{"type":"v1","addr":"172.21.15.5:6827","nonce":1961857360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1961857360},{"type":"v1","addr":"172.21.15.5:6829","nonce":1961857360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":1961857360},{"type":"v1","addr":"172.21.15.5:6833","nonce":1961857360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1961857360},{"type":"v1","addr":"172.21.15.5:6831","nonce":1961857360}]},"public_addr":"172.21.15.5:6827/1961857360","cluster_addr":"172.21.15.5:6829/1961857360","heartbeat_back_addr":"172.21.15.5:6833/1961857360","heartbeat_front_addr":"172.21.15.5:6831/1961857360","state":["exists","up"]},{"osd":10,"uuid":"519ec60f-3606-48ac-9043-6f59a798766d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":2041733103},{"type":"v1","addr":"172.21.15.146:6825","nonce":2041733103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":2041733103},{"type":"v1","addr":"172.21.15.146:6827","nonce":2041733103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":2041733103},{"type":"v1","addr":"172.21.15.146:6831","nonce":2041733103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":2041733103},{"type":"v1","addr":"172.21.15.146:6829","nonce":2041733103}]},"public_addr":"172.21.15.146:6825/2041733103","cluster_addr":"172.21.15.146:6827/2041733103","heartbeat_back_addr":"172.21.15.146:6831/2041733103","heartbeat_front_addr":"172.21.15.146:6829/2041733103","state":["exists","up"]},{"osd":11,"uuid":"0f92d0cb-ba54-4bc3-a6d0-3c54d910556a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":839290773},{"type":"v1","addr":"172.21.15.148:6825","nonce":839290773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":839290773},{"type":"v1","addr":"172.21.15.148:6827","nonce":839290773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":839290773},{"type":"v1","addr":"172.21.15.148:6831","nonce":839290773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":839290773},{"type":"v1","addr":"172.21.15.148:6829","nonce":839290773}]},"public_addr":"172.21.15.148:6825/839290773","cluster_addr":"172.21.15.148:6827/839290773","heartbeat_back_addr":"172.21.15.148:6831/839290773","heartbeat_front_addr":"172.21.15.148:6829/839290773","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:50:54.101755+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:51:19.037385+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:51:44.142837+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:52:09.150887+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:52:33.705079+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:52:58.845233+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:53:23.068339+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:53:47.520398+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:54:11.965370+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:54:36.776075+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:55:01.292236+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:55:25.346589+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:6800/3125821798":"2024-09-13T02:49:29.462216+0000","172.21.15.5:0/3479911451":"2024-09-13T02:49:29.462216+0000","172.21.15.5:0/1547753207":"2024-09-13T02:49:03.300872+0000","172.21.15.5:0/1905762627":"2024-09-13T02:49:03.300872+0000","172.21.15.5:0/4142340280":"2024-09-13T02:49:29.462216+0000","172.21.15.5:6801/3125821798":"2024-09-13T02:49:29.462216+0000","172.21.15.5:6801/2595781680":"2024-09-13T02:49:03.300872+0000","172.21.15.5:0/2273691311":"2024-09-13T02:49:29.462216+0000","172.21.15.5:6800/2595781680":"2024-09-13T02:49:03.300872+0000","172.21.15.5:0/1196201217":"2024-09-13T02:49:03.300872+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T02:57:50.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:50.342+0000 7f857da3e640 1 -- 172.21.15.5:0/213726364 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f85600a2c90 msgr2=0x7f85600a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:50.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:50.342+0000 7f857da3e640 1 --2- 172.21.15.5:0/213726364 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f85600a2c90 0x7f85600a5150 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f8568005fd0 tx=0x7f8568005850 comp rx=0 tx=0).stop 2024-09-12T02:57:50.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:50.342+0000 7f857da3e640 1 -- 172.21.15.5:0/213726364 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8578103660 msgr2=0x7f8578198870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:50.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:50.342+0000 7f857da3e640 1 --2- 172.21.15.5:0/213726364 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8578103660 0x7f8578198870 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f856c00c970 tx=0x7f856c00ce30 comp rx=0 tx=0).stop 2024-09-12T02:57:50.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:50.342+0000 7f857da3e640 1 -- 172.21.15.5:0/213726364 shutdown_connections 2024-09-12T02:57:50.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:50.342+0000 7f857da3e640 1 --2- 172.21.15.5:0/213726364 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f85600a2c90 0x7f85600a5150 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:50.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:50.342+0000 7f857da3e640 1 --2- 172.21.15.5:0/213726364 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8578103660 0x7f8578198870 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:50.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:50.342+0000 7f857da3e640 1 --2- 172.21.15.5:0/213726364 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f857810c170 0x7f857819cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:50.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:50.342+0000 7f857da3e640 1 --2- 172.21.15.5:0/213726364 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8578102d40 0x7f8578198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:50.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:50.343+0000 7f857da3e640 1 -- 172.21.15.5:0/213726364 >> 172.21.15.5:0/213726364 conn(0x7f85780fc7a0 msgr2=0x7f85780fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:50.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:50.343+0000 7f857da3e640 1 -- 172.21.15.5:0/213726364 shutdown_connections 2024-09-12T02:57:50.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:50.343+0000 7f857da3e640 1 -- 172.21.15.5:0/213726364 wait complete. 2024-09-12T02:57:51.005 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-12T02:56:16.595562+0000', 'modified': '2024-09-12T02:56:27.840207+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24502, 'mds_1': 24487, 'mds_2': 24481, 'mds_3': 24479, 'mds_4': 24458}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14601': {'gid': 14601, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.5:6839/26447341', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6838', 'nonce': 26447341}, {'type': 'v1', 'addr': '172.21.15.5:6839', 'nonce': 26447341}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14613': {'gid': 14613, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.5:6841/2393443295', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6840', 'nonce': 2393443295}, {'type': 'v1', 'addr': '172.21.15.5:6841', 'nonce': 2393443295}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24407': {'gid': 24407, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.146:6833/3686970137', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.146:6832', 'nonce': 3686970137}, {'type': 'v1', 'addr': '172.21.15.146:6833', 'nonce': 3686970137}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24439': {'gid': 24439, 'name': 'l', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.148:6833/568512209', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6832', 'nonce': 568512209}, {'type': 'v1', 'addr': '172.21.15.148:6833', 'nonce': 568512209}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24440': {'gid': 24440, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.146:6835/2540740420', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.146:6834', 'nonce': 2540740420}, {'type': 'v1', 'addr': '172.21.15.146:6835', 'nonce': 2540740420}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24458': {'gid': 24458, 'name': 'h', 'rank': 4, 'incarnation': 24, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.146:6837/1673370937', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.146:6836', 'nonce': 1673370937}, {'type': 'v1', 'addr': '172.21.15.146:6837', 'nonce': 1673370937}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24479': {'gid': 24479, 'name': 'b', 'rank': 3, 'incarnation': 22, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.146:6839/3036767228', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.146:6838', 'nonce': 3036767228}, {'type': 'v1', 'addr': '172.21.15.146:6839', 'nonce': 3036767228}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24481': {'gid': 24481, 'name': 'f', 'rank': 2, 'incarnation': 20, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.148:6835/4195086427', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6834', 'nonce': 4195086427}, {'type': 'v1', 'addr': '172.21.15.148:6835', 'nonce': 4195086427}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24487': {'gid': 24487, 'name': 'c', 'rank': 1, 'incarnation': 18, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.148:6837/2606873922', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6836', 'nonce': 2606873922}, {'type': 'v1', 'addr': '172.21.15.148:6837', 'nonce': 2606873922}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24502': {'gid': 24502, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.148:6839/2317600000', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6838', 'nonce': 2317600000}, {'type': 'v1', 'addr': '172.21.15.148:6839', 'nonce': 2317600000}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24502, 'qdb_cluster': [24458, 24479, 24481, 24487, 24502]} 2024-09-12T02:57:51.005 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-12T02:57:51.005 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.h status 2024-09-12T02:57:51.261 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:57:51.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:57:51 smithi005 ceph-mon[31197]: pgmap v323: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-12T02:57:51.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:57:51 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/213726364' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:57:51.665 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:57:51 smithi148 ceph-mon[35090]: pgmap v323: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-12T02:57:51.665 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:57:51 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/213726364' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:57:51.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:57:51 smithi146 ceph-mon[35011]: pgmap v323: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-12T02:57:51.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:57:51 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/213726364' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:57:51.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.748+0000 7f8da954b640 1 -- 172.21.15.5:0/1679826295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8da4101de0 msgr2=0x7f8da41021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:51.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.748+0000 7f8da954b640 1 --2- 172.21.15.5:0/1679826295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8da4101de0 0x7f8da41021c0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f8d94009900 tx=0x7f8d9402a410 comp rx=0 tx=0).stop 2024-09-12T02:57:51.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.748+0000 7f8da954b640 1 -- 172.21.15.5:0/1679826295 shutdown_connections 2024-09-12T02:57:51.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.748+0000 7f8da954b640 1 --2- 172.21.15.5:0/1679826295 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8da4106f10 0x7f8da410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:51.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.748+0000 7f8da954b640 1 --2- 172.21.15.5:0/1679826295 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8da4102790 0x7f8da41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:51.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.748+0000 7f8da954b640 1 --2- 172.21.15.5:0/1679826295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8da4101de0 0x7f8da41021c0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:51.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.748+0000 7f8da954b640 1 -- 172.21.15.5:0/1679826295 >> 172.21.15.5:0/1679826295 conn(0x7f8da40fd8c0 msgr2=0x7f8da40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:51.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.748+0000 7f8da954b640 1 -- 172.21.15.5:0/1679826295 shutdown_connections 2024-09-12T02:57:51.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.748+0000 7f8da954b640 1 -- 172.21.15.5:0/1679826295 wait complete. 2024-09-12T02:57:51.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.749+0000 7f8da954b640 1 Processor -- start 2024-09-12T02:57:51.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.749+0000 7f8da954b640 1 -- start start 2024-09-12T02:57:51.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.749+0000 7f8da954b640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8da4101de0 0x7f8da40744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:51.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.749+0000 7f8da954b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8da4102790 0x7f8da4074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:51.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.749+0000 7f8da954b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8da4106f10 0x7f8da4074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:51.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.749+0000 7f8da954b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8da4111f90 con 0x7f8da4102790 2024-09-12T02:57:51.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.749+0000 7f8da954b640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f8da4111e10 con 0x7f8da4101de0 2024-09-12T02:57:51.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.749+0000 7f8da954b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8da4112110 con 0x7f8da4106f10 2024-09-12T02:57:51.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.749+0000 7f8da27fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8da4102790 0x7f8da4074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:51.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.750+0000 7f8da27fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8da4102790 0x7f8da4074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37180/0 (socket says 172.21.15.5:37180) 2024-09-12T02:57:51.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.750+0000 7f8da27fc640 1 -- 172.21.15.5:0/663810199 learned_addr learned my addr 172.21.15.5:0/663810199 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:57:51.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.750+0000 7f8da37fe640 1 --2- 172.21.15.5:0/663810199 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8da4106f10 0x7f8da4074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:51.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.750+0000 7f8da2ffd640 1 --2- 172.21.15.5:0/663810199 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8da4101de0 0x7f8da40744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:51.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.750+0000 7f8da27fc640 1 -- 172.21.15.5:0/663810199 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8da4106f10 msgr2=0x7f8da4074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:51.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.750+0000 7f8da27fc640 1 --2- 172.21.15.5:0/663810199 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8da4106f10 0x7f8da4074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:51.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.750+0000 7f8da27fc640 1 -- 172.21.15.5:0/663810199 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8da4101de0 msgr2=0x7f8da40744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:51.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.750+0000 7f8da27fc640 1 --2- 172.21.15.5:0/663810199 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8da4101de0 0x7f8da40744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:51.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.750+0000 7f8da27fc640 1 -- 172.21.15.5:0/663810199 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8da41a6980 con 0x7f8da4102790 2024-09-12T02:57:51.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.750+0000 7f8da27fc640 1 --2- 172.21.15.5:0/663810199 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8da4102790 0x7f8da4074a30 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f8d9800c890 tx=0x7f8d9800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:51.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.750+0000 7f8d8bfff640 1 -- 172.21.15.5:0/663810199 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d98004220 con 0x7f8da4102790 2024-09-12T02:57:51.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.750+0000 7f8da954b640 1 -- 172.21.15.5:0/663810199 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8da41a6c70 con 0x7f8da4102790 2024-09-12T02:57:51.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.750+0000 7f8d8bfff640 1 -- 172.21.15.5:0/663810199 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d980043c0 con 0x7f8da4102790 2024-09-12T02:57:51.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.750+0000 7f8d8bfff640 1 -- 172.21.15.5:0/663810199 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d980049e0 con 0x7f8da4102790 2024-09-12T02:57:51.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.750+0000 7f8da954b640 1 -- 172.21.15.5:0/663810199 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8da41a71b0 con 0x7f8da4102790 2024-09-12T02:57:51.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.752+0000 7f8d8bfff640 1 -- 172.21.15.5:0/663810199 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f8d980079b0 con 0x7f8da4102790 2024-09-12T02:57:51.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.753+0000 7f8dab924740 1 Processor -- start 2024-09-12T02:57:51.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.753+0000 7f8dab924740 1 -- start start 2024-09-12T02:57:51.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.753+0000 7f8d8bfff640 1 --2- 172.21.15.5:0/663810199 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f8d8c0a2c10 0x7f8d8c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:51.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.753+0000 7f8dab924740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f9c575a9d0 0x55f9c575adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:51.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.753+0000 7f8dab924740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9c575b2f0 0x55f9c5765670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:51.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.753+0000 7f8dab924740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f9c5765bb0 0x55f9c5767fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:51.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.753+0000 7f8dab924740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f9c56c1090 con 0x55f9c575b2f0 2024-09-12T02:57:51.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.753+0000 7f8dab924740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55f9c5668a70 con 0x55f9c575a9d0 2024-09-12T02:57:51.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.753+0000 7f8dab924740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55f9c56bd050 con 0x55f9c5765bb0 2024-09-12T02:57:51.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.753+0000 7f8da27fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9c575b2f0 0x55f9c5765670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:51.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.753+0000 7f8da37fe640 1 --2- 172.21.15.5:0/663810199 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f8d8c0a2c10 0x7f8d8c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:51.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.753+0000 7f8da27fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9c575b2f0 0x55f9c5765670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37186/0 (socket says 172.21.15.5:37186) 2024-09-12T02:57:51.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.753+0000 7f8da27fc640 1 -- 172.21.15.5:0/694987595 learned_addr learned my addr 172.21.15.5:0/694987595 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:57:51.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.754+0000 7f8da2ffd640 1 --2- 172.21.15.5:0/694987595 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f9c575a9d0 0x55f9c575adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:51.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.754+0000 7f8d8bfff640 1 -- 172.21.15.5:0/663810199 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f8d980cf1d0 con 0x7f8da4102790 2024-09-12T02:57:51.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.754+0000 7f8da37fe640 1 --2- 172.21.15.5:0/694987595 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f9c5765bb0 0x55f9c5767fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:51.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.754+0000 7f8da27fc640 1 -- 172.21.15.5:0/694987595 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f9c5765bb0 msgr2=0x55f9c5767fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:57:51.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.754+0000 7f8da27fc640 1 --2- 172.21.15.5:0/694987595 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f9c5765bb0 0x55f9c5767fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:51.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.754+0000 7f8da27fc640 1 -- 172.21.15.5:0/694987595 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f9c575a9d0 msgr2=0x55f9c575adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:57:51.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.754+0000 7f8da27fc640 1 --2- 172.21.15.5:0/694987595 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f9c575a9d0 0x55f9c575adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:51.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.754+0000 7f8da27fc640 1 -- 172.21.15.5:0/694987595 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9c5675130 con 0x55f9c575b2f0 2024-09-12T02:57:51.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.754+0000 7f8da37fe640 1 --2- 172.21.15.5:0/663810199 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f8d8c0a2c10 0x7f8d8c0a50d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8d9c009900 tx=0x7f8d9c00aba0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:51.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.754+0000 7f8da27fc640 1 --2- 172.21.15.5:0/694987595 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9c575b2f0 0x55f9c5765670 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f8d980ccb60 tx=0x7f8d980054a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8b60693a9febf81 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:51.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.754+0000 7f8d897fa640 1 -- 172.21.15.5:0/694987595 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d9801d070 con 0x55f9c575b2f0 2024-09-12T02:57:51.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.754+0000 7f8d897fa640 1 -- 172.21.15.5:0/694987595 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d980ae190 con 0x55f9c575b2f0 2024-09-12T02:57:51.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.754+0000 7f8d897fa640 1 -- 172.21.15.5:0/694987595 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d98097fa0 con 0x55f9c575b2f0 2024-09-12T02:57:51.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.755+0000 7f8dab924740 1 -- 172.21.15.5:0/694987595 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9c575b2f0 msgr2=0x55f9c5765670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:51.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.755+0000 7f8dab924740 1 --2- 172.21.15.5:0/694987595 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9c575b2f0 0x55f9c5765670 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f8d980ccb60 tx=0x7f8d980054a0 comp rx=0 tx=0).stop 2024-09-12T02:57:51.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.755+0000 7f8dab924740 1 -- 172.21.15.5:0/694987595 shutdown_connections 2024-09-12T02:57:51.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.755+0000 7f8dab924740 1 --2- 172.21.15.5:0/694987595 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9c575b2f0 0x55f9c5765670 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:51.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.755+0000 7f8dab924740 1 --2- 172.21.15.5:0/694987595 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f9c5765bb0 0x55f9c5767fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:51.766 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.755+0000 7f8dab924740 1 --2- 172.21.15.5:0/694987595 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f9c575a9d0 0x55f9c575adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:51.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.755+0000 7f8dab924740 1 -- 172.21.15.5:0/694987595 >> 172.21.15.5:0/694987595 conn(0x55f9c5756f60 msgr2=0x55f9c5757340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:51.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.755+0000 7f8dab924740 1 -- 172.21.15.5:0/694987595 shutdown_connections 2024-09-12T02:57:51.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.755+0000 7f8dab924740 1 -- 172.21.15.5:0/694987595 wait complete. 2024-09-12T02:57:51.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.756+0000 7f8dab924740 1 -- start start 2024-09-12T02:57:51.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.756+0000 7f8dab924740 1 Processor -- start 2024-09-12T02:57:51.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.756+0000 7f8dab924740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9c575a9d0 0x55f9c57653b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:51.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.756+0000 7f8dab924740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f9c575b2f0 0x55f9c5762700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:51.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.756+0000 7f8dab924740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f9c5765bb0 0x55f9c5762c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:51.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.756+0000 7f8dab924740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f9c5768780 con 0x55f9c575a9d0 2024-09-12T02:57:51.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.756+0000 7f8dab924740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55f9c567bf20 con 0x55f9c575b2f0 2024-09-12T02:57:51.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.756+0000 7f8dab924740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55f9c5768900 con 0x55f9c5765bb0 2024-09-12T02:57:51.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.756+0000 7f8dab924740 10 client.0 _ll_register_callbacks cb 0x55f9c5645030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:57:51.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.756+0000 7f8da2ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9c575a9d0 0x55f9c57653b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:51.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.756+0000 7f8da2ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9c575a9d0 0x55f9c57653b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37190/0 (socket says 172.21.15.5:37190) 2024-09-12T02:57:51.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.756+0000 7f8da2ffd640 1 -- 172.21.15.5:0/3905536334 learned_addr learned my addr 172.21.15.5:0/3905536334 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:57:51.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.756+0000 7f8da27fc640 1 --2- 172.21.15.5:0/3905536334 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f9c575b2f0 0x55f9c5762700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:51.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.756+0000 7f8da37fe640 1 --2- 172.21.15.5:0/3905536334 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f9c5765bb0 0x55f9c5762c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:51.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.756+0000 7f8da2ffd640 1 -- 172.21.15.5:0/3905536334 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f9c5765bb0 msgr2=0x55f9c5762c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:57:51.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.756+0000 7f8da2ffd640 1 --2- 172.21.15.5:0/3905536334 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f9c5765bb0 0x55f9c5762c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:51.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.756+0000 7f8da2ffd640 1 -- 172.21.15.5:0/3905536334 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f9c575b2f0 msgr2=0x55f9c5762700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:57:51.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.756+0000 7f8da2ffd640 1 --2- 172.21.15.5:0/3905536334 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f9c575b2f0 0x55f9c5762700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:51.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.756+0000 7f8da2ffd640 1 -- 172.21.15.5:0/3905536334 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9c54322b0 con 0x55f9c575a9d0 2024-09-12T02:57:51.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.756+0000 7f8da27fc640 1 --2- 172.21.15.5:0/3905536334 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f9c575b2f0 0x55f9c5762700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:57:51.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.756+0000 7f8da37fe640 1 --2- 172.21.15.5:0/3905536334 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f9c5765bb0 0x55f9c5762c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:57:51.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.757+0000 7f8da2ffd640 1 --2- 172.21.15.5:0/3905536334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9c575a9d0 0x55f9c57653b0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f8d94009900 tx=0x7f8d94031a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4829e62bcce61bb1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:51.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.757+0000 7f8d86ff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T02:57:51.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.757+0000 7f8d86ff5640 1 -- 172.21.15.5:0/3905536334 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d94035040 con 0x55f9c575a9d0 2024-09-12T02:57:51.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.757+0000 7f8d86ff5640 1 -- 172.21.15.5:0/3905536334 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f8d94004380 con 0x55f9c575a9d0 2024-09-12T02:57:51.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.757+0000 7f8dab924740 1 -- 172.21.15.5:0/3905536334 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d980ae190 con 0x55f9c575a9d0 2024-09-12T02:57:51.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.757+0000 7f8d86ff5640 1 -- 172.21.15.5:0/3905536334 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8d94005020 con 0x55f9c575a9d0 2024-09-12T02:57:51.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.757+0000 7f8dab924740 10 client.15117 fetch_fsmap learned FSMap version 25 2024-09-12T02:57:51.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.757+0000 7f8dab924740 1 -- 172.21.15.5:0/3905536334 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f9c5774040 con 0x55f9c575a9d0 2024-09-12T02:57:51.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.757+0000 7f8d86ff5640 1 -- 172.21.15.5:0/3905536334 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d980ae190 con 0x55f9c575a9d0 2024-09-12T02:57:51.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.757+0000 7f8d86ff5640 1 -- 172.21.15.5:0/3905536334 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8362+0+0 (secure 0 0 0) 0x7f8d940035f0 con 0x55f9c575a9d0 2024-09-12T02:57:51.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.757+0000 7f8dab924740 10 client.15117 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:57:51.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.757+0000 7f8dab924740 10 client.15117 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-12T02:57:51.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.758+0000 7f8dab924740 20 client.15117 populate_metadata read hostname 'smithi005' 2024-09-12T02:57:51.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.758+0000 7f8dab924740 1 --2- 172.21.15.5:0/3905536334 >> [v2:172.21.15.146:6836/1673370937,v1:172.21.15.146:6837/1673370937] conn(0x55f9c5776660 0x55f9c5777ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:51.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.758+0000 7f8dab924740 4 client.15117 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:57:51.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.758+0000 7f8dab924740 1 -- 172.21.15.5:0/3905536334 --> [v2:172.21.15.146:6836/1673370937,v1:172.21.15.146:6837/1673370937] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f9c56ce600 con 0x55f9c5776660 2024-09-12T02:57:51.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.758+0000 7f8da27fc640 1 --2- 172.21.15.5:0/3905536334 >> [v2:172.21.15.146:6836/1673370937,v1:172.21.15.146:6837/1673370937] conn(0x55f9c5776660 0x55f9c5777ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:51.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.758+0000 7f8da27fc640 1 --2- 172.21.15.5:0/3905536334 >> [v2:172.21.15.146:6836/1673370937,v1:172.21.15.146:6837/1673370937] conn(0x55f9c5776660 0x55f9c5777ad0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6d66157adc81df04 server_cookie=acb4bcafe8cf5938 in_seq=0 out_seq=0 2024-09-12T02:57:51.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.758+0000 7f8d86ff5640 10 client.15117 ms_handle_connect on v2:172.21.15.146:6836/1673370937 2024-09-12T02:57:51.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.759+0000 7f8d86ff5640 1 -- 172.21.15.5:0/3905536334 <== mds.4 v2:172.21.15.146:6836/1673370937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f9c56ce600 con 0x55f9c5776660 2024-09-12T02:57:51.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.759+0000 7f8d86ff5640 10 client.15117 handle_command_reply: tid=0 2024-09-12T02:57:51.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.759+0000 7f8dab924740 1 client.15117 shutdown 2024-09-12T02:57:51.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.759+0000 7f8dab924740 1 -- 172.21.15.5:0/3905536334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9c575a9d0 msgr2=0x55f9c57653b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:51.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.759+0000 7f8dab924740 1 --2- 172.21.15.5:0/3905536334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9c575a9d0 0x55f9c57653b0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f8d94009900 tx=0x7f8d94031a60 comp rx=0 tx=0).stop 2024-09-12T02:57:51.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.759+0000 7f8dab924740 1 -- 172.21.15.5:0/3905536334 shutdown_connections 2024-09-12T02:57:51.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.759+0000 7f8dab924740 1 --2- 172.21.15.5:0/3905536334 >> [v2:172.21.15.146:6836/1673370937,v1:172.21.15.146:6837/1673370937] conn(0x55f9c5776660 0x55f9c5777ad0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:51.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.759+0000 7f8dab924740 1 --2- 172.21.15.5:0/3905536334 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f9c5765bb0 0x55f9c5762c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:51.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.759+0000 7f8dab924740 1 --2- 172.21.15.5:0/3905536334 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f9c575b2f0 0x55f9c5762700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:51.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.759+0000 7f8dab924740 1 --2- 172.21.15.5:0/3905536334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9c575a9d0 0x55f9c57653b0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:51.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.759+0000 7f8d86ff5640 0 client.15117 ms_handle_reset on v2:172.21.15.146:6836/1673370937 2024-09-12T02:57:51.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.759+0000 7f8dab924740 1 -- 172.21.15.5:0/3905536334 >> 172.21.15.5:0/3905536334 conn(0x55f9c5756f60 msgr2=0x55f9c575cc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:51.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.759+0000 7f8dab924740 1 -- 172.21.15.5:0/3905536334 shutdown_connections 2024-09-12T02:57:51.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.759+0000 7f8dab924740 1 -- 172.21.15.5:0/3905536334 wait complete. 2024-09-12T02:57:51.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.760+0000 7f8dab924740 20 client.15117 trim_cache size 0 max 16384 2024-09-12T02:57:51.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.777+0000 7f8dab924740 1 Processor -- start 2024-09-12T02:57:51.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.777+0000 7f8dab924740 1 -- start start 2024-09-12T02:57:51.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.777+0000 7f8dab924740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f9c575a9d0 0x55f9c575f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:51.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.777+0000 7f8dab924740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f9c575b2f0 0x55f9c575fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:51.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.777+0000 7f8dab924740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9c5765bb0 0x55f9c577fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:51.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.777+0000 7f8dab924740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f9c5775e10 con 0x55f9c5765bb0 2024-09-12T02:57:51.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.777+0000 7f8dab924740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55f9c57764e0 con 0x55f9c575a9d0 2024-09-12T02:57:51.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.777+0000 7f8dab924740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55f9c5776bb0 con 0x55f9c575b2f0 2024-09-12T02:57:51.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.777+0000 7f8da37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9c5765bb0 0x55f9c577fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:51.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.778+0000 7f8da37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9c5765bb0 0x55f9c577fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37196/0 (socket says 172.21.15.5:37196) 2024-09-12T02:57:51.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.778+0000 7f8da37fe640 1 -- 172.21.15.5:0/3141439948 learned_addr learned my addr 172.21.15.5:0/3141439948 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:57:51.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.778+0000 7f8da2ffd640 1 --2- 172.21.15.5:0/3141439948 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f9c575a9d0 0x55f9c575f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:51.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.778+0000 7f8da27fc640 1 --2- 172.21.15.5:0/3141439948 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f9c575b2f0 0x55f9c575fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:51.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.778+0000 7f8da37fe640 1 -- 172.21.15.5:0/3141439948 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f9c575b2f0 msgr2=0x55f9c575fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:57:51.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.778+0000 7f8da37fe640 1 --2- 172.21.15.5:0/3141439948 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f9c575b2f0 0x55f9c575fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:51.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.778+0000 7f8da37fe640 1 -- 172.21.15.5:0/3141439948 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f9c575a9d0 msgr2=0x55f9c575f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:57:51.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.778+0000 7f8da37fe640 1 --2- 172.21.15.5:0/3141439948 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f9c575a9d0 0x55f9c575f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:51.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.778+0000 7f8da37fe640 1 -- 172.21.15.5:0/3141439948 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9c5764fd0 con 0x55f9c5765bb0 2024-09-12T02:57:51.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.778+0000 7f8da2ffd640 1 --2- 172.21.15.5:0/3141439948 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f9c575a9d0 0x55f9c575f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:57:51.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.778+0000 7f8da37fe640 1 --2- 172.21.15.5:0/3141439948 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9c5765bb0 0x55f9c577fdb0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f8d9c009d00 tx=0x7f8d9c01c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d23de598f265115f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:51.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.778+0000 7f8d88ff9640 1 -- 172.21.15.5:0/3141439948 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d9c02b070 con 0x55f9c5765bb0 2024-09-12T02:57:51.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.778+0000 7f8d88ff9640 1 -- 172.21.15.5:0/3141439948 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d9c015d00 con 0x55f9c5765bb0 2024-09-12T02:57:51.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.778+0000 7f8d88ff9640 1 -- 172.21.15.5:0/3141439948 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d9c026de0 con 0x55f9c5765bb0 2024-09-12T02:57:51.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.779+0000 7f8dab924740 1 -- 172.21.15.5:0/3141439948 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9c5765bb0 msgr2=0x55f9c577fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:51.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.779+0000 7f8dab924740 1 --2- 172.21.15.5:0/3141439948 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9c5765bb0 0x55f9c577fdb0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f8d9c009d00 tx=0x7f8d9c01c000 comp rx=0 tx=0).stop 2024-09-12T02:57:51.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.779+0000 7f8dab924740 1 -- 172.21.15.5:0/3141439948 shutdown_connections 2024-09-12T02:57:51.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.779+0000 7f8dab924740 1 --2- 172.21.15.5:0/3141439948 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9c5765bb0 0x55f9c577fdb0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:51.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.779+0000 7f8dab924740 1 --2- 172.21.15.5:0/3141439948 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f9c575b2f0 0x55f9c575fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:51.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.779+0000 7f8dab924740 1 --2- 172.21.15.5:0/3141439948 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f9c575a9d0 0x55f9c575f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:51.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.779+0000 7f8dab924740 1 -- 172.21.15.5:0/3141439948 >> 172.21.15.5:0/3141439948 conn(0x55f9c5756f60 msgr2=0x55f9c5760fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:51.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.779+0000 7f8dab924740 1 -- 172.21.15.5:0/3141439948 shutdown_connections 2024-09-12T02:57:51.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.779+0000 7f8dab924740 1 -- 172.21.15.5:0/3141439948 wait complete. 2024-09-12T02:57:51.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.779+0000 7f8dab924740 1 -- start start 2024-09-12T02:57:51.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.779+0000 7f8dab924740 1 Processor -- start 2024-09-12T02:57:51.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.779+0000 7f8dab924740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9c575a9d0 0x55f9c5780180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:51.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.779+0000 7f8dab924740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f9c575b2f0 0x55f9c57806c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:51.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.779+0000 7f8dab924740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f9c5765bb0 0x55f9c577b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:51.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.779+0000 7f8dab924740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f9c5777950 con 0x55f9c575a9d0 2024-09-12T02:57:51.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.779+0000 7f8dab924740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55f9c5777280 con 0x55f9c575b2f0 2024-09-12T02:57:51.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.779+0000 7f8dab924740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55f9c5778020 con 0x55f9c5765bb0 2024-09-12T02:57:51.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.780+0000 7f8dab924740 10 client.0 _ll_register_callbacks cb 0x7f8d94009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:57:51.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.780+0000 7f8da2ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9c575a9d0 0x55f9c5780180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:51.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.780+0000 7f8da2ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9c575a9d0 0x55f9c5780180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37206/0 (socket says 172.21.15.5:37206) 2024-09-12T02:57:51.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.780+0000 7f8da2ffd640 1 -- 172.21.15.5:0/1189013993 learned_addr learned my addr 172.21.15.5:0/1189013993 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:57:51.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.780+0000 7f8da37fe640 1 --2- 172.21.15.5:0/1189013993 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f9c5765bb0 0x55f9c577b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:51.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.780+0000 7f8da27fc640 1 --2- 172.21.15.5:0/1189013993 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f9c575b2f0 0x55f9c57806c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:51.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.780+0000 7f8da2ffd640 1 -- 172.21.15.5:0/1189013993 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f9c5765bb0 msgr2=0x55f9c577b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:57:51.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.780+0000 7f8da2ffd640 1 --2- 172.21.15.5:0/1189013993 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f9c5765bb0 0x55f9c577b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:51.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.780+0000 7f8da2ffd640 1 -- 172.21.15.5:0/1189013993 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f9c575b2f0 msgr2=0x55f9c57806c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:57:51.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.780+0000 7f8da2ffd640 1 --2- 172.21.15.5:0/1189013993 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f9c575b2f0 0x55f9c57806c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:51.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.780+0000 7f8da2ffd640 1 -- 172.21.15.5:0/1189013993 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9c577bf30 con 0x55f9c575a9d0 2024-09-12T02:57:51.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.780+0000 7f8da2ffd640 1 --2- 172.21.15.5:0/1189013993 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9c575a9d0 0x55f9c5780180 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f8d94009850 tx=0x7f8d940038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=653bbddc556e25ad server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:51.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.780+0000 7f8d86ff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T02:57:51.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.780+0000 7f8d86ff5640 1 -- 172.21.15.5:0/1189013993 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d94035040 con 0x55f9c575a9d0 2024-09-12T02:57:51.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.780+0000 7f8d86ff5640 1 -- 172.21.15.5:0/1189013993 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f8d94004380 con 0x55f9c575a9d0 2024-09-12T02:57:51.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.780+0000 7f8dab924740 1 -- 172.21.15.5:0/1189013993 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d9c015d00 con 0x55f9c575a9d0 2024-09-12T02:57:51.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.780+0000 7f8d86ff5640 1 -- 172.21.15.5:0/1189013993 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8d94003e10 con 0x55f9c575a9d0 2024-09-12T02:57:51.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.780+0000 7f8dab924740 10 client.15123 fetch_fsmap learned FSMap version 25 2024-09-12T02:57:51.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.780+0000 7f8dab924740 1 -- 172.21.15.5:0/1189013993 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f9c57e0a20 con 0x55f9c575a9d0 2024-09-12T02:57:51.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.781+0000 7f8d86ff5640 1 -- 172.21.15.5:0/1189013993 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d9c015d00 con 0x55f9c575a9d0 2024-09-12T02:57:51.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.781+0000 7f8d86ff5640 1 -- 172.21.15.5:0/1189013993 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8362+0+0 (secure 0 0 0) 0x7f8d9405d4b0 con 0x55f9c575a9d0 2024-09-12T02:57:51.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.781+0000 7f8dab924740 10 client.15123 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:57:51.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.781+0000 7f8dab924740 10 client.15123 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-12T02:57:51.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.781+0000 7f8dab924740 20 client.15123 populate_metadata read hostname 'smithi005' 2024-09-12T02:57:51.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.781+0000 7f8dab924740 1 --2- 172.21.15.5:0/1189013993 >> [v2:172.21.15.146:6836/1673370937,v1:172.21.15.146:6837/1673370937] conn(0x55f9c57e5590 0x55f9c57e42a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:51.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.781+0000 7f8dab924740 4 client.15123 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T02:57:51.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.781+0000 7f8dab924740 1 -- 172.21.15.5:0/1189013993 --> [v2:172.21.15.146:6836/1673370937,v1:172.21.15.146:6837/1673370937] -- command(tid 0: {"prefix": "status"}) -- 0x55f9c56c8b10 con 0x55f9c57e5590 2024-09-12T02:57:51.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.781+0000 7f8da27fc640 1 --2- 172.21.15.5:0/1189013993 >> [v2:172.21.15.146:6836/1673370937,v1:172.21.15.146:6837/1673370937] conn(0x55f9c57e5590 0x55f9c57e42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:51.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.781+0000 7f8d86ff5640 10 client.15123 ms_handle_connect on v2:172.21.15.146:6836/1673370937 2024-09-12T02:57:51.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.781+0000 7f8da27fc640 1 --2- 172.21.15.5:0/1189013993 >> [v2:172.21.15.146:6836/1673370937,v1:172.21.15.146:6837/1673370937] conn(0x55f9c57e5590 0x55f9c57e42a0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7481d98e1fde4ccc server_cookie=bb2845b3b5520f33 in_seq=0 out_seq=0 2024-09-12T02:57:51.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.782+0000 7f8d86ff5640 1 -- 172.21.15.5:0/1189013993 <== mds.4 v2:172.21.15.146:6836/1673370937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55f9c56c8b10 con 0x55f9c57e5590 2024-09-12T02:57:51.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.782+0000 7f8d86ff5640 10 client.15123 handle_command_reply: tid=0 2024-09-12T02:57:51.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.782+0000 7f8dab924740 1 client.15123 shutdown 2024-09-12T02:57:51.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.782+0000 7f8dab924740 1 -- 172.21.15.5:0/1189013993 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9c575a9d0 msgr2=0x55f9c5780180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:51.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.782+0000 7f8dab924740 1 --2- 172.21.15.5:0/1189013993 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9c575a9d0 0x55f9c5780180 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f8d94009850 tx=0x7f8d940038a0 comp rx=0 tx=0).stop 2024-09-12T02:57:51.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.782+0000 7f8dab924740 1 -- 172.21.15.5:0/1189013993 shutdown_connections 2024-09-12T02:57:51.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.782+0000 7f8dab924740 1 --2- 172.21.15.5:0/1189013993 >> [v2:172.21.15.146:6836/1673370937,v1:172.21.15.146:6837/1673370937] conn(0x55f9c57e5590 0x55f9c57e42a0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:51.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.782+0000 7f8dab924740 1 --2- 172.21.15.5:0/1189013993 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f9c5765bb0 0x55f9c577b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:51.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.782+0000 7f8dab924740 1 --2- 172.21.15.5:0/1189013993 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f9c575b2f0 0x55f9c57806c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:51.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.782+0000 7f8d86ff5640 0 client.15123 ms_handle_reset on v2:172.21.15.146:6836/1673370937 2024-09-12T02:57:51.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.782+0000 7f8dab924740 1 --2- 172.21.15.5:0/1189013993 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9c575a9d0 0x55f9c5780180 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:51.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.782+0000 7f8dab924740 1 -- 172.21.15.5:0/1189013993 >> 172.21.15.5:0/1189013993 conn(0x55f9c5756f60 msgr2=0x55f9c57555c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:51.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.782+0000 7f8dab924740 1 -- 172.21.15.5:0/1189013993 shutdown_connections 2024-09-12T02:57:51.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.782+0000 7f8dab924740 1 -- 172.21.15.5:0/1189013993 wait complete. 2024-09-12T02:57:51.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.782+0000 7f8dab924740 20 client.15123 trim_cache size 0 max 16384 2024-09-12T02:57:51.796 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-12T02:57:51.796 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "618305f8-70b1-11ef-bcea-c7b262605968", 2024-09-12T02:57:51.796 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 4, 2024-09-12T02:57:51.796 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24458, 2024-09-12T02:57:51.796 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-09-12T02:57:51.796 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-09-12T02:57:51.796 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-09-12T02:57:51.797 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 84.762215378999997, 2024-09-12T02:57:51.797 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 25, 2024-09-12T02:57:51.797 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 83, 2024-09-12T02:57:51.797 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 81, 2024-09-12T02:57:51.797 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 116.62412902200001 2024-09-12T02:57:51.797 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-12T02:57:51.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.783+0000 7f8da954b640 1 -- 172.21.15.5:0/663810199 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f8d8c0a2c10 msgr2=0x7f8d8c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:51.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.783+0000 7f8da954b640 1 --2- 172.21.15.5:0/663810199 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f8d8c0a2c10 0x7f8d8c0a50d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8d9c009900 tx=0x7f8d9c00aba0 comp rx=0 tx=0).stop 2024-09-12T02:57:51.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.783+0000 7f8da954b640 1 -- 172.21.15.5:0/663810199 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8da4102790 msgr2=0x7f8da4074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:51.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.783+0000 7f8da954b640 1 --2- 172.21.15.5:0/663810199 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8da4102790 0x7f8da4074a30 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f8d9800c890 tx=0x7f8d9800cd50 comp rx=0 tx=0).stop 2024-09-12T02:57:51.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.783+0000 7f8da954b640 1 -- 172.21.15.5:0/663810199 shutdown_connections 2024-09-12T02:57:51.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.783+0000 7f8da954b640 1 --2- 172.21.15.5:0/663810199 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f8d8c0a2c10 0x7f8d8c0a50d0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:51.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.783+0000 7f8da954b640 1 --2- 172.21.15.5:0/663810199 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8da4102790 0x7f8da4074a30 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:51.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.783+0000 7f8da954b640 1 --2- 172.21.15.5:0/663810199 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8da4106f10 0x7f8da4074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:51.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.783+0000 7f8da954b640 1 --2- 172.21.15.5:0/663810199 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8da4101de0 0x7f8da40744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:51.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.783+0000 7f8da954b640 1 -- 172.21.15.5:0/663810199 >> 172.21.15.5:0/663810199 conn(0x7f8da40fd8c0 msgr2=0x7f8da40fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:51.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.783+0000 7f8da954b640 1 -- 172.21.15.5:0/663810199 shutdown_connections 2024-09-12T02:57:51.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:51.783+0000 7f8da954b640 1 -- 172.21.15.5:0/663810199 wait complete. 2024-09-12T02:57:52.441 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.b status 2024-09-12T02:57:52.698 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:57:53.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.174+0000 7f57eacec640 1 -- 172.21.15.5:0/390495519 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57e4101420 msgr2=0x7f57e410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:53.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.174+0000 7f57eacec640 1 --2- 172.21.15.5:0/390495519 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57e4101420 0x7f57e410d950 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f57dc009cd0 tx=0x7f57dc02a8e0 comp rx=0 tx=0).stop 2024-09-12T02:57:53.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.174+0000 7f57eacec640 1 -- 172.21.15.5:0/390495519 shutdown_connections 2024-09-12T02:57:53.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.174+0000 7f57eacec640 1 --2- 172.21.15.5:0/390495519 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57e4101420 0x7f57e410d950 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:53.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.174+0000 7f57eacec640 1 --2- 172.21.15.5:0/390495519 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57e4100a80 0x7f57e4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:53.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.174+0000 7f57eacec640 1 --2- 172.21.15.5:0/390495519 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f57e4106a80 0x7f57e4106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:53.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.174+0000 7f57eacec640 1 -- 172.21.15.5:0/390495519 >> 172.21.15.5:0/390495519 conn(0x7f57e40fc7a0 msgr2=0x7f57e40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:53.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.174+0000 7f57eacec640 1 -- 172.21.15.5:0/390495519 shutdown_connections 2024-09-12T02:57:53.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.174+0000 7f57eacec640 1 -- 172.21.15.5:0/390495519 wait complete. 2024-09-12T02:57:53.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.175+0000 7f57eacec640 1 Processor -- start 2024-09-12T02:57:53.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.175+0000 7f57eacec640 1 -- start start 2024-09-12T02:57:53.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.175+0000 7f57eacec640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57e4100a80 0x7f57e41992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:53.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.175+0000 7f57eacec640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57e4101420 0x7f57e41997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:53.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.175+0000 7f57eacec640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f57e4106a80 0x7f57e41a4330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:53.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.175+0000 7f57eacec640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f57e4110ea0 con 0x7f57e4100a80 2024-09-12T02:57:53.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.175+0000 7f57eacec640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f57e4110d20 con 0x7f57e4106a80 2024-09-12T02:57:53.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.175+0000 7f57eacec640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f57e4111020 con 0x7f57e4101420 2024-09-12T02:57:53.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.175+0000 7f57e8a88640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57e4100a80 0x7f57e41992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:53.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.175+0000 7f57e8a88640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57e4100a80 0x7f57e41992b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37218/0 (socket says 172.21.15.5:37218) 2024-09-12T02:57:53.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.175+0000 7f57e8a88640 1 -- 172.21.15.5:0/2492099940 learned_addr learned my addr 172.21.15.5:0/2492099940 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:57:53.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.175+0000 7f57e3fff640 1 --2- 172.21.15.5:0/2492099940 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57e4101420 0x7f57e41997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:53.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.176+0000 7f57e8a88640 1 -- 172.21.15.5:0/2492099940 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57e4101420 msgr2=0x7f57e41997f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:53.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.176+0000 7f57e8a88640 1 --2- 172.21.15.5:0/2492099940 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57e4101420 0x7f57e41997f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:53.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.176+0000 7f57e8a88640 1 -- 172.21.15.5:0/2492099940 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f57e4106a80 msgr2=0x7f57e41a4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:53.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.176+0000 7f57e8a88640 1 --2- 172.21.15.5:0/2492099940 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f57e4106a80 0x7f57e41a4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:53.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.176+0000 7f57e8a88640 1 -- 172.21.15.5:0/2492099940 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57e41a4870 con 0x7f57e4100a80 2024-09-12T02:57:53.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.176+0000 7f57e8a88640 1 --2- 172.21.15.5:0/2492099940 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57e4100a80 0x7f57e41992b0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f57d800b4a0 tx=0x7f57d800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:53.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.176+0000 7f57e1ffb640 1 -- 172.21.15.5:0/2492099940 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d8014070 con 0x7f57e4100a80 2024-09-12T02:57:53.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.176+0000 7f57e1ffb640 1 -- 172.21.15.5:0/2492099940 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57d8010af0 con 0x7f57e4100a80 2024-09-12T02:57:53.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.176+0000 7f57eacec640 1 -- 172.21.15.5:0/2492099940 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57e41a4a40 con 0x7f57e4100a80 2024-09-12T02:57:53.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.176+0000 7f57e1ffb640 1 -- 172.21.15.5:0/2492099940 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d8017d10 con 0x7f57e4100a80 2024-09-12T02:57:53.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.176+0000 7f57eacec640 1 -- 172.21.15.5:0/2492099940 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57e41a4f80 con 0x7f57e4100a80 2024-09-12T02:57:53.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.178+0000 7f57e1ffb640 1 -- 172.21.15.5:0/2492099940 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f57d8017070 con 0x7f57e4100a80 2024-09-12T02:57:53.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.179+0000 7f57ed0c5740 1 Processor -- start 2024-09-12T02:57:53.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.179+0000 7f57ed0c5740 1 -- start start 2024-09-12T02:57:53.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.179+0000 7f57e1ffb640 1 --2- 172.21.15.5:0/2492099940 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f57d40a2c90 0x7f57d40a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:53.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.179+0000 7f57ed0c5740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557faaf238b0 0x557faaf23c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:53.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.179+0000 7f57ed0c5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557faaf241d0 0x557faaf32a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:53.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.179+0000 7f57ed0c5740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557faaf32fc0 0x557faaf353b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:53.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.179+0000 7f57ed0c5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557faae8a090 con 0x557faaf241d0 2024-09-12T02:57:53.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.179+0000 7f57ed0c5740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x557faae31a70 con 0x557faaf238b0 2024-09-12T02:57:53.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.179+0000 7f57ed0c5740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x557faae86050 con 0x557faaf32fc0 2024-09-12T02:57:53.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.179+0000 7f57e9289640 1 --2- 172.21.15.5:0/2492099940 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f57d40a2c90 0x7f57d40a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:53.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.179+0000 7f57e3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557faaf241d0 0x557faaf32a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:53.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.179+0000 7f57e1ffb640 1 -- 172.21.15.5:0/2492099940 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f57d80ce4b0 con 0x7f57e4100a80 2024-09-12T02:57:53.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.179+0000 7f57e3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557faaf241d0 0x557faaf32a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37228/0 (socket says 172.21.15.5:37228) 2024-09-12T02:57:53.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.179+0000 7f57e3fff640 1 -- 172.21.15.5:0/2482092670 learned_addr learned my addr 172.21.15.5:0/2482092670 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:57:53.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.179+0000 7f57e9289640 1 --2- 172.21.15.5:0/2482092670 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557faaf32fc0 0x557faaf353b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:53.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.179+0000 7f57e8a88640 1 --2- 172.21.15.5:0/2482092670 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557faaf238b0 0x557faaf23c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:53.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.180+0000 7f57e3fff640 1 -- 172.21.15.5:0/2482092670 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557faaf32fc0 msgr2=0x557faaf353b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:57:53.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.180+0000 7f57e3fff640 1 --2- 172.21.15.5:0/2482092670 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557faaf32fc0 0x557faaf353b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:53.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.180+0000 7f57e3fff640 1 -- 172.21.15.5:0/2482092670 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557faaf238b0 msgr2=0x557faaf23c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:57:53.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.180+0000 7f57e3fff640 1 --2- 172.21.15.5:0/2482092670 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557faaf238b0 0x557faaf23c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:53.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.180+0000 7f57e3fff640 1 -- 172.21.15.5:0/2482092670 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557faae3e130 con 0x557faaf241d0 2024-09-12T02:57:53.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.180+0000 7f57e8a88640 1 --2- 172.21.15.5:0/2482092670 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557faaf238b0 0x557faaf23c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:57:53.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.180+0000 7f57e9289640 1 --2- 172.21.15.5:0/2492099940 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f57d40a2c90 0x7f57d40a5150 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f57dc006010 tx=0x7f57dc031510 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:53.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.180+0000 7f57e3fff640 1 --2- 172.21.15.5:0/2482092670 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557faaf241d0 0x557faaf32a80 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f57d000bc80 tx=0x7f57d000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5faed4601f382e24 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:53.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.180+0000 7f57ceffd640 1 -- 172.21.15.5:0/2482092670 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d0015070 con 0x557faaf241d0 2024-09-12T02:57:53.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.180+0000 7f57ceffd640 1 -- 172.21.15.5:0/2482092670 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57d0011920 con 0x557faaf241d0 2024-09-12T02:57:53.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.180+0000 7f57ceffd640 1 -- 172.21.15.5:0/2482092670 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d0010770 con 0x557faaf241d0 2024-09-12T02:57:53.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.181+0000 7f57ed0c5740 1 -- 172.21.15.5:0/2482092670 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557faaf241d0 msgr2=0x557faaf32a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:53.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.181+0000 7f57ed0c5740 1 --2- 172.21.15.5:0/2482092670 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557faaf241d0 0x557faaf32a80 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f57d000bc80 tx=0x7f57d000f730 comp rx=0 tx=0).stop 2024-09-12T02:57:53.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.181+0000 7f57ed0c5740 1 -- 172.21.15.5:0/2482092670 shutdown_connections 2024-09-12T02:57:53.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.181+0000 7f57ed0c5740 1 --2- 172.21.15.5:0/2482092670 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557faaf241d0 0x557faaf32a80 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:53.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.181+0000 7f57ed0c5740 1 --2- 172.21.15.5:0/2482092670 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557faaf32fc0 0x557faaf353b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:53.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.181+0000 7f57ed0c5740 1 --2- 172.21.15.5:0/2482092670 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557faaf238b0 0x557faaf23c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:53.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.181+0000 7f57ed0c5740 1 -- 172.21.15.5:0/2482092670 >> 172.21.15.5:0/2482092670 conn(0x557faaf1ff60 msgr2=0x557faaf20340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:53.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.181+0000 7f57ed0c5740 1 -- 172.21.15.5:0/2482092670 shutdown_connections 2024-09-12T02:57:53.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.181+0000 7f57ed0c5740 1 -- 172.21.15.5:0/2482092670 wait complete. 2024-09-12T02:57:53.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.181+0000 7f57ed0c5740 1 -- start start 2024-09-12T02:57:53.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.181+0000 7f57ed0c5740 1 Processor -- start 2024-09-12T02:57:53.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.182+0000 7f57ed0c5740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557faaf238b0 0x557faaf32850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:53.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.182+0000 7f57ed0c5740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557faaf241d0 0x557faaf2fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:53.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.182+0000 7f57ed0c5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557faaf32fc0 0x557faaf300e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:53.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.182+0000 7f57ed0c5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557faaf35b90 con 0x557faaf32fc0 2024-09-12T02:57:53.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.182+0000 7f57ed0c5740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x557faae44f20 con 0x557faaf238b0 2024-09-12T02:57:53.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.182+0000 7f57ed0c5740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x557faaf35d10 con 0x557faaf241d0 2024-09-12T02:57:53.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.182+0000 7f57ed0c5740 10 client.0 _ll_register_callbacks cb 0x557faae0e030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:57:53.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.182+0000 7f57e9289640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557faaf32fc0 0x557faaf300e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:53.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.182+0000 7f57e9289640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557faaf32fc0 0x557faaf300e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37232/0 (socket says 172.21.15.5:37232) 2024-09-12T02:57:53.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.182+0000 7f57e8a88640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557faaf238b0 0x557faaf32850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:53.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.182+0000 7f57e9289640 1 -- 172.21.15.5:0/2934459790 learned_addr learned my addr 172.21.15.5:0/2934459790 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:57:53.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.182+0000 7f57e3fff640 1 --2- 172.21.15.5:0/2934459790 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557faaf241d0 0x557faaf2fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:53.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.182+0000 7f57e9289640 1 -- 172.21.15.5:0/2934459790 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557faaf241d0 msgr2=0x557faaf2fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:57:53.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.182+0000 7f57e9289640 1 --2- 172.21.15.5:0/2934459790 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557faaf241d0 0x557faaf2fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:53.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.182+0000 7f57e9289640 1 -- 172.21.15.5:0/2934459790 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557faaf238b0 msgr2=0x557faaf32850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:57:53.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.182+0000 7f57e9289640 1 --2- 172.21.15.5:0/2934459790 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557faaf238b0 0x557faaf32850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:53.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.182+0000 7f57e9289640 1 -- 172.21.15.5:0/2934459790 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557faabfb2b0 con 0x557faaf32fc0 2024-09-12T02:57:53.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.182+0000 7f57e3fff640 1 --2- 172.21.15.5:0/2934459790 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557faaf241d0 0x557faaf2fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:57:53.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.182+0000 7f57e9289640 1 --2- 172.21.15.5:0/2934459790 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557faaf32fc0 0x557faaf300e0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f57dc035000 tx=0x7f57dc042040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=306c32efdd583e58 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:53.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.182+0000 7f57c3fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T02:57:53.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.182+0000 7f57c3fff640 1 -- 172.21.15.5:0/2934459790 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57dc04e070 con 0x557faaf32fc0 2024-09-12T02:57:53.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.182+0000 7f57c3fff640 1 -- 172.21.15.5:0/2934459790 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f57dc004750 con 0x557faaf32fc0 2024-09-12T02:57:53.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.182+0000 7f57ed0c5740 1 -- 172.21.15.5:0/2934459790 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57d0011920 con 0x557faaf32fc0 2024-09-12T02:57:53.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.183+0000 7f57c3fff640 1 -- 172.21.15.5:0/2934459790 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f57dc0305a0 con 0x557faaf32fc0 2024-09-12T02:57:53.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.183+0000 7f57ed0c5740 10 client.15135 fetch_fsmap learned FSMap version 25 2024-09-12T02:57:53.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.183+0000 7f57ed0c5740 1 -- 172.21.15.5:0/2934459790 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557faaf41510 con 0x557faaf32fc0 2024-09-12T02:57:53.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.183+0000 7f57c3fff640 1 -- 172.21.15.5:0/2934459790 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57d0011920 con 0x557faaf32fc0 2024-09-12T02:57:53.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.183+0000 7f57c3fff640 1 -- 172.21.15.5:0/2934459790 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8362+0+0 (secure 0 0 0) 0x7f57dc049070 con 0x557faaf32fc0 2024-09-12T02:57:53.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.183+0000 7f57ed0c5740 10 client.15135 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:57:53.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.183+0000 7f57ed0c5740 10 client.15135 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-12T02:57:53.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.183+0000 7f57ed0c5740 20 client.15135 populate_metadata read hostname 'smithi005' 2024-09-12T02:57:53.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.183+0000 7f57ed0c5740 1 --2- 172.21.15.5:0/2934459790 >> [v2:172.21.15.146:6838/3036767228,v1:172.21.15.146:6839/3036767228] conn(0x557faaf43c00 0x557faaf45070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:53.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.183+0000 7f57ed0c5740 4 client.15135 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:57:53.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.183+0000 7f57ed0c5740 1 -- 172.21.15.5:0/2934459790 --> [v2:172.21.15.146:6838/3036767228,v1:172.21.15.146:6839/3036767228] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557faae97600 con 0x557faaf43c00 2024-09-12T02:57:53.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.183+0000 7f57e8a88640 1 --2- 172.21.15.5:0/2934459790 >> [v2:172.21.15.146:6838/3036767228,v1:172.21.15.146:6839/3036767228] conn(0x557faaf43c00 0x557faaf45070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:53.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.184+0000 7f57e8a88640 1 --2- 172.21.15.5:0/2934459790 >> [v2:172.21.15.146:6838/3036767228,v1:172.21.15.146:6839/3036767228] conn(0x557faaf43c00 0x557faaf45070 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9651765e2320b6e7 server_cookie=b26efcbbdf6c0690 in_seq=0 out_seq=0 2024-09-12T02:57:53.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.184+0000 7f57c3fff640 10 client.15135 ms_handle_connect on v2:172.21.15.146:6838/3036767228 2024-09-12T02:57:53.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.185+0000 7f57c3fff640 1 -- 172.21.15.5:0/2934459790 <== mds.3 v2:172.21.15.146:6838/3036767228 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557faae97600 con 0x557faaf43c00 2024-09-12T02:57:53.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.185+0000 7f57c3fff640 10 client.15135 handle_command_reply: tid=0 2024-09-12T02:57:53.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.185+0000 7f57ed0c5740 1 client.15135 shutdown 2024-09-12T02:57:53.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.185+0000 7f57ed0c5740 1 -- 172.21.15.5:0/2934459790 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557faaf32fc0 msgr2=0x557faaf300e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:53.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.185+0000 7f57ed0c5740 1 --2- 172.21.15.5:0/2934459790 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557faaf32fc0 0x557faaf300e0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f57dc035000 tx=0x7f57dc042040 comp rx=0 tx=0).stop 2024-09-12T02:57:53.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.185+0000 7f57ed0c5740 1 -- 172.21.15.5:0/2934459790 shutdown_connections 2024-09-12T02:57:53.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.185+0000 7f57ed0c5740 1 --2- 172.21.15.5:0/2934459790 >> [v2:172.21.15.146:6838/3036767228,v1:172.21.15.146:6839/3036767228] conn(0x557faaf43c00 0x557faaf45070 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:53.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.185+0000 7f57ed0c5740 1 --2- 172.21.15.5:0/2934459790 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557faaf32fc0 0x557faaf300e0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:53.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.185+0000 7f57ed0c5740 1 --2- 172.21.15.5:0/2934459790 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557faaf241d0 0x557faaf2fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:53.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.185+0000 7f57ed0c5740 1 --2- 172.21.15.5:0/2934459790 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557faaf238b0 0x557faaf32850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:53.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.185+0000 7f57ed0c5740 1 -- 172.21.15.5:0/2934459790 >> 172.21.15.5:0/2934459790 conn(0x557faaf1ff60 msgr2=0x557faaf25cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:53.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.185+0000 7f57c3fff640 0 client.15135 ms_handle_reset on v2:172.21.15.146:6838/3036767228 2024-09-12T02:57:53.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.185+0000 7f57ed0c5740 1 -- 172.21.15.5:0/2934459790 shutdown_connections 2024-09-12T02:57:53.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.185+0000 7f57ed0c5740 1 -- 172.21.15.5:0/2934459790 wait complete. 2024-09-12T02:57:53.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.185+0000 7f57ed0c5740 20 client.15135 trim_cache size 0 max 16384 2024-09-12T02:57:53.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.203+0000 7f57ed0c5740 1 Processor -- start 2024-09-12T02:57:53.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.203+0000 7f57ed0c5740 1 -- start start 2024-09-12T02:57:53.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.203+0000 7f57ed0c5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557faaf238b0 0x557faaf43530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:53.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.203+0000 7f57ed0c5740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557faaf241d0 0x557faaf449a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:53.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.203+0000 7f57ed0c5740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557faaf32fc0 0x557faaf2fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:53.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.203+0000 7f57ed0c5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557faaf433b0 con 0x557faaf238b0 2024-09-12T02:57:53.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.203+0000 7f57ed0c5740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x557faaf43a80 con 0x557faaf241d0 2024-09-12T02:57:53.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.203+0000 7f57ed0c5740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x557faaf44150 con 0x557faaf32fc0 2024-09-12T02:57:53.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.203+0000 7f57e8a88640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557faaf238b0 0x557faaf43530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:53.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.203+0000 7f57e8a88640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557faaf238b0 0x557faaf43530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37244/0 (socket says 172.21.15.5:37244) 2024-09-12T02:57:53.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.203+0000 7f57e8a88640 1 -- 172.21.15.5:0/1339784292 learned_addr learned my addr 172.21.15.5:0/1339784292 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:57:53.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.203+0000 7f57e3fff640 1 --2- 172.21.15.5:0/1339784292 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557faaf241d0 0x557faaf449a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:53.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.203+0000 7f57e8a88640 1 -- 172.21.15.5:0/1339784292 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557faaf32fc0 msgr2=0x557faaf2fb60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T02:57:53.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.203+0000 7f57e8a88640 1 --2- 172.21.15.5:0/1339784292 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557faaf32fc0 0x557faaf2fb60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:53.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.203+0000 7f57e8a88640 1 -- 172.21.15.5:0/1339784292 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557faaf241d0 msgr2=0x557faaf449a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:57:53.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.203+0000 7f57e8a88640 1 --2- 172.21.15.5:0/1339784292 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557faaf241d0 0x557faaf449a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:53.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.203+0000 7f57e8a88640 1 -- 172.21.15.5:0/1339784292 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557faaf24000 con 0x557faaf238b0 2024-09-12T02:57:53.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.203+0000 7f57e3fff640 1 --2- 172.21.15.5:0/1339784292 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557faaf241d0 0x557faaf449a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:57:53.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.204+0000 7f57e8a88640 1 --2- 172.21.15.5:0/1339784292 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557faaf238b0 0x557faaf43530 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f57d800be60 tx=0x7f57d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfa661c9ca26184a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:53.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.204+0000 7f57ce7fc640 1 -- 172.21.15.5:0/1339784292 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d801e040 con 0x557faaf238b0 2024-09-12T02:57:53.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.204+0000 7f57ce7fc640 1 -- 172.21.15.5:0/1339784292 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57d809f020 con 0x557faaf238b0 2024-09-12T02:57:53.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.204+0000 7f57ce7fc640 1 -- 172.21.15.5:0/1339784292 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d80972e0 con 0x557faaf238b0 2024-09-12T02:57:53.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.204+0000 7f57ed0c5740 1 -- 172.21.15.5:0/1339784292 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557faaf238b0 msgr2=0x557faaf43530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:53.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.204+0000 7f57ed0c5740 1 --2- 172.21.15.5:0/1339784292 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557faaf238b0 0x557faaf43530 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f57d800be60 tx=0x7f57d80046c0 comp rx=0 tx=0).stop 2024-09-12T02:57:53.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.204+0000 7f57ed0c5740 1 -- 172.21.15.5:0/1339784292 shutdown_connections 2024-09-12T02:57:53.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.204+0000 7f57ed0c5740 1 --2- 172.21.15.5:0/1339784292 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557faaf32fc0 0x557faaf2fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:53.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.204+0000 7f57ed0c5740 1 --2- 172.21.15.5:0/1339784292 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557faaf241d0 0x557faaf449a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:53.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.204+0000 7f57ed0c5740 1 --2- 172.21.15.5:0/1339784292 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557faaf238b0 0x557faaf43530 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:53.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.204+0000 7f57ed0c5740 1 -- 172.21.15.5:0/1339784292 >> 172.21.15.5:0/1339784292 conn(0x557faaf1ff60 msgr2=0x557faaf1ace0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:53.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.204+0000 7f57ed0c5740 1 -- 172.21.15.5:0/1339784292 shutdown_connections 2024-09-12T02:57:53.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.204+0000 7f57ed0c5740 1 -- 172.21.15.5:0/1339784292 wait complete. 2024-09-12T02:57:53.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.205+0000 7f57ed0c5740 1 -- start start 2024-09-12T02:57:53.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.205+0000 7f57ed0c5740 1 Processor -- start 2024-09-12T02:57:53.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.205+0000 7f57ed0c5740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557faaf238b0 0x557faaf46120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:53.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.205+0000 7f57ed0c5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557faaf241d0 0x557faaf46660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:53.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.205+0000 7f57ed0c5740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557faaf32fc0 0x557faaf46ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:53.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.205+0000 7f57ed0c5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557faaf44ef0 con 0x557faaf241d0 2024-09-12T02:57:53.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.205+0000 7f57ed0c5740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x557faaf44820 con 0x557faaf238b0 2024-09-12T02:57:53.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.205+0000 7f57ed0c5740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x557faaf455c0 con 0x557faaf32fc0 2024-09-12T02:57:53.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.205+0000 7f57ed0c5740 10 client.0 _ll_register_callbacks cb 0x7f57dc009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:57:53.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.205+0000 7f57e3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557faaf241d0 0x557faaf46660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:53.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.205+0000 7f57e3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557faaf241d0 0x557faaf46660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37258/0 (socket says 172.21.15.5:37258) 2024-09-12T02:57:53.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.205+0000 7f57e3fff640 1 -- 172.21.15.5:0/2827905111 learned_addr learned my addr 172.21.15.5:0/2827905111 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:57:53.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.206+0000 7f57e3fff640 1 -- 172.21.15.5:0/2827905111 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557faaf32fc0 msgr2=0x557faaf46ba0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T02:57:53.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.206+0000 7f57e3fff640 1 --2- 172.21.15.5:0/2827905111 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557faaf32fc0 0x557faaf46ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:53.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.206+0000 7f57e3fff640 1 -- 172.21.15.5:0/2827905111 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557faaf238b0 msgr2=0x557faaf46120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:57:53.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.206+0000 7f57e3fff640 1 --2- 172.21.15.5:0/2827905111 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557faaf238b0 0x557faaf46120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:53.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.206+0000 7f57e3fff640 1 -- 172.21.15.5:0/2827905111 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557faafac300 con 0x557faaf241d0 2024-09-12T02:57:53.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.206+0000 7f57e3fff640 1 --2- 172.21.15.5:0/2827905111 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557faaf241d0 0x557faaf46660 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f57d0000c00 tx=0x7f57d000b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76b42c2382acaec4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:53.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.206+0000 7f57c3fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T02:57:53.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.206+0000 7f57c3fff640 1 -- 172.21.15.5:0/2827905111 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d0028060 con 0x557faaf241d0 2024-09-12T02:57:53.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.206+0000 7f57c3fff640 1 -- 172.21.15.5:0/2827905111 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f57d0005450 con 0x557faaf241d0 2024-09-12T02:57:53.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.206+0000 7f57ed0c5740 1 -- 172.21.15.5:0/2827905111 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57d809f020 con 0x557faaf241d0 2024-09-12T02:57:53.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.206+0000 7f57c3fff640 1 -- 172.21.15.5:0/2827905111 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f57d0015040 con 0x557faaf241d0 2024-09-12T02:57:53.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.206+0000 7f57ed0c5740 10 client.15141 fetch_fsmap learned FSMap version 25 2024-09-12T02:57:53.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.206+0000 7f57ed0c5740 1 -- 172.21.15.5:0/2827905111 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557faafad610 con 0x557faaf241d0 2024-09-12T02:57:53.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.206+0000 7f57c3fff640 1 -- 172.21.15.5:0/2827905111 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57d809f020 con 0x557faaf241d0 2024-09-12T02:57:53.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.207+0000 7f57c3fff640 1 -- 172.21.15.5:0/2827905111 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8362+0+0 (secure 0 0 0) 0x7f57d001a6c0 con 0x557faaf241d0 2024-09-12T02:57:53.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.207+0000 7f57ed0c5740 10 client.15141 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:57:53.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.207+0000 7f57ed0c5740 10 client.15141 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-12T02:57:53.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.207+0000 7f57ed0c5740 20 client.15141 populate_metadata read hostname 'smithi005' 2024-09-12T02:57:53.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.207+0000 7f57ed0c5740 1 --2- 172.21.15.5:0/2827905111 >> [v2:172.21.15.146:6838/3036767228,v1:172.21.15.146:6839/3036767228] conn(0x557faafb1ed0 0x557faafb0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:53.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.207+0000 7f57ed0c5740 4 client.15141 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T02:57:53.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.207+0000 7f57ed0c5740 1 -- 172.21.15.5:0/2827905111 --> [v2:172.21.15.146:6838/3036767228,v1:172.21.15.146:6839/3036767228] -- command(tid 0: {"prefix": "status"}) -- 0x557faae91b10 con 0x557faafb1ed0 2024-09-12T02:57:53.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.207+0000 7f57e8a88640 1 --2- 172.21.15.5:0/2827905111 >> [v2:172.21.15.146:6838/3036767228,v1:172.21.15.146:6839/3036767228] conn(0x557faafb1ed0 0x557faafb0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:53.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.207+0000 7f57e8a88640 1 --2- 172.21.15.5:0/2827905111 >> [v2:172.21.15.146:6838/3036767228,v1:172.21.15.146:6839/3036767228] conn(0x557faafb1ed0 0x557faafb0be0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=51b6b3b39cc626cd server_cookie=36e8f8a69695cf40 in_seq=0 out_seq=0 2024-09-12T02:57:53.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.207+0000 7f57c3fff640 10 client.15141 ms_handle_connect on v2:172.21.15.146:6838/3036767228 2024-09-12T02:57:53.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.208+0000 7f57c3fff640 1 -- 172.21.15.5:0/2827905111 <== mds.3 v2:172.21.15.146:6838/3036767228 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x557faae91b10 con 0x557faafb1ed0 2024-09-12T02:57:53.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.208+0000 7f57c3fff640 10 client.15141 handle_command_reply: tid=0 2024-09-12T02:57:53.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.208+0000 7f57ed0c5740 1 client.15141 shutdown 2024-09-12T02:57:53.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.208+0000 7f57ed0c5740 1 -- 172.21.15.5:0/2827905111 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557faaf241d0 msgr2=0x557faaf46660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:53.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.208+0000 7f57ed0c5740 1 --2- 172.21.15.5:0/2827905111 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557faaf241d0 0x557faaf46660 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f57d0000c00 tx=0x7f57d000b9f0 comp rx=0 tx=0).stop 2024-09-12T02:57:53.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.208+0000 7f57ed0c5740 1 -- 172.21.15.5:0/2827905111 shutdown_connections 2024-09-12T02:57:53.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.208+0000 7f57ed0c5740 1 --2- 172.21.15.5:0/2827905111 >> [v2:172.21.15.146:6838/3036767228,v1:172.21.15.146:6839/3036767228] conn(0x557faafb1ed0 0x557faafb0be0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:53.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.208+0000 7f57ed0c5740 1 --2- 172.21.15.5:0/2827905111 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557faaf241d0 0x557faaf46660 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:53.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.208+0000 7f57ed0c5740 1 --2- 172.21.15.5:0/2827905111 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557faaf32fc0 0x557faaf46ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:53.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.208+0000 7f57c3fff640 0 client.15141 ms_handle_reset on v2:172.21.15.146:6838/3036767228 2024-09-12T02:57:53.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.208+0000 7f57ed0c5740 1 --2- 172.21.15.5:0/2827905111 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557faaf238b0 0x557faaf46120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:53.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.208+0000 7f57ed0c5740 1 -- 172.21.15.5:0/2827905111 >> 172.21.15.5:0/2827905111 conn(0x557faaf1ff60 msgr2=0x557faaf20340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:53.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.208+0000 7f57ed0c5740 1 -- 172.21.15.5:0/2827905111 shutdown_connections 2024-09-12T02:57:53.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.208+0000 7f57ed0c5740 1 -- 172.21.15.5:0/2827905111 wait complete. 2024-09-12T02:57:53.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.208+0000 7f57ed0c5740 20 client.15141 trim_cache size 0 max 16384 2024-09-12T02:57:53.222 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-12T02:57:53.222 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "618305f8-70b1-11ef-bcea-c7b262605968", 2024-09-12T02:57:53.222 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 3, 2024-09-12T02:57:53.222 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24479, 2024-09-12T02:57:53.222 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-09-12T02:57:53.222 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-09-12T02:57:53.222 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-09-12T02:57:53.222 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 87.196285309999993, 2024-09-12T02:57:53.222 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 25, 2024-09-12T02:57:53.222 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 83, 2024-09-12T02:57:53.222 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 81, 2024-09-12T02:57:53.223 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 107.10531963699999 2024-09-12T02:57:53.223 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-12T02:57:53.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.209+0000 7f57eacec640 1 -- 172.21.15.5:0/2492099940 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f57d40a2c90 msgr2=0x7f57d40a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:53.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.209+0000 7f57eacec640 1 --2- 172.21.15.5:0/2492099940 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f57d40a2c90 0x7f57d40a5150 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f57dc006010 tx=0x7f57dc031510 comp rx=0 tx=0).stop 2024-09-12T02:57:53.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.209+0000 7f57eacec640 1 -- 172.21.15.5:0/2492099940 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57e4100a80 msgr2=0x7f57e41992b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:53.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.209+0000 7f57eacec640 1 --2- 172.21.15.5:0/2492099940 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57e4100a80 0x7f57e41992b0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f57d800b4a0 tx=0x7f57d800b960 comp rx=0 tx=0).stop 2024-09-12T02:57:53.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.209+0000 7f57eacec640 1 -- 172.21.15.5:0/2492099940 shutdown_connections 2024-09-12T02:57:53.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.209+0000 7f57eacec640 1 --2- 172.21.15.5:0/2492099940 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f57e4106a80 0x7f57e41a4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:53.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.209+0000 7f57eacec640 1 --2- 172.21.15.5:0/2492099940 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f57e4101420 0x7f57e41997f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:53.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.209+0000 7f57eacec640 1 --2- 172.21.15.5:0/2492099940 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f57d40a2c90 0x7f57d40a5150 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:53.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.209+0000 7f57eacec640 1 --2- 172.21.15.5:0/2492099940 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57e4100a80 0x7f57e41992b0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:53.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.209+0000 7f57eacec640 1 -- 172.21.15.5:0/2492099940 >> 172.21.15.5:0/2492099940 conn(0x7f57e40fc7a0 msgr2=0x7f57e40fded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:53.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.209+0000 7f57eacec640 1 -- 172.21.15.5:0/2492099940 shutdown_connections 2024-09-12T02:57:53.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:53.209+0000 7f57eacec640 1 -- 172.21.15.5:0/2492099940 wait complete. 2024-09-12T02:57:53.501 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:57:53 smithi005 ceph-mon[31197]: pgmap v324: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T02:57:53.665 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:57:53 smithi148 ceph-mon[35090]: pgmap v324: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T02:57:53.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:57:53 smithi146 ceph-mon[35011]: pgmap v324: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T02:57:53.928 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.f status 2024-09-12T02:57:54.191 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:57:54.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.651+0000 7f1499533640 1 -- 172.21.15.5:0/1765862504 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1494106a80 msgr2=0x7f1494106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:54.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.651+0000 7f1499533640 1 --2- 172.21.15.5:0/1765862504 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1494106a80 0x7f1494106e60 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f1484009930 tx=0x7f148402a420 comp rx=0 tx=0).stop 2024-09-12T02:57:54.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.651+0000 7f1499533640 1 -- 172.21.15.5:0/1765862504 shutdown_connections 2024-09-12T02:57:54.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.651+0000 7f1499533640 1 --2- 172.21.15.5:0/1765862504 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1494101420 0x7f149410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:54.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.651+0000 7f1499533640 1 --2- 172.21.15.5:0/1765862504 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1494100a80 0x7f1494100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:54.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.651+0000 7f1499533640 1 --2- 172.21.15.5:0/1765862504 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1494106a80 0x7f1494106e60 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:54.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.651+0000 7f1499533640 1 -- 172.21.15.5:0/1765862504 >> 172.21.15.5:0/1765862504 conn(0x7f14940fc7a0 msgr2=0x7f14940febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:54.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.651+0000 7f1499533640 1 -- 172.21.15.5:0/1765862504 shutdown_connections 2024-09-12T02:57:54.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.651+0000 7f1499533640 1 -- 172.21.15.5:0/1765862504 wait complete. 2024-09-12T02:57:54.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.652+0000 7f1499533640 1 Processor -- start 2024-09-12T02:57:54.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.652+0000 7f1499533640 1 -- start start 2024-09-12T02:57:54.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.652+0000 7f1499533640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1494100a80 0x7f1494198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:54.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.652+0000 7f1499533640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1494101420 0x7f14941988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:54.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.652+0000 7f1499533640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1494106a80 0x7f149419cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:54.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.652+0000 7f1499533640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1494110e80 con 0x7f1494106a80 2024-09-12T02:57:54.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.652+0000 7f1499533640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f1494110d00 con 0x7f1494101420 2024-09-12T02:57:54.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.652+0000 7f1499533640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1494111000 con 0x7f1494100a80 2024-09-12T02:57:54.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.652+0000 7f14937fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1494106a80 0x7f149419cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:54.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.653+0000 7f14937fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1494106a80 0x7f149419cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60208/0 (socket says 172.21.15.5:60208) 2024-09-12T02:57:54.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.653+0000 7f14937fe640 1 -- 172.21.15.5:0/4290266239 learned_addr learned my addr 172.21.15.5:0/4290266239 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:57:54.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.653+0000 7f1492ffd640 1 --2- 172.21.15.5:0/4290266239 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1494100a80 0x7f1494198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:54.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.653+0000 7f148bfff640 1 --2- 172.21.15.5:0/4290266239 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1494101420 0x7f14941988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:54.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.653+0000 7f14937fe640 1 -- 172.21.15.5:0/4290266239 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1494100a80 msgr2=0x7f1494198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:54.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.653+0000 7f14937fe640 1 --2- 172.21.15.5:0/4290266239 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1494100a80 0x7f1494198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:54.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.653+0000 7f14937fe640 1 -- 172.21.15.5:0/4290266239 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1494101420 msgr2=0x7f14941988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:54.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.653+0000 7f14937fe640 1 --2- 172.21.15.5:0/4290266239 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1494101420 0x7f14941988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:54.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.653+0000 7f14937fe640 1 -- 172.21.15.5:0/4290266239 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f149419d340 con 0x7f1494106a80 2024-09-12T02:57:54.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.653+0000 7f1492ffd640 1 --2- 172.21.15.5:0/4290266239 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1494100a80 0x7f1494198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:57:54.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.653+0000 7f148bfff640 1 --2- 172.21.15.5:0/4290266239 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1494101420 0x7f14941988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:57:54.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.653+0000 7f14937fe640 1 --2- 172.21.15.5:0/4290266239 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1494106a80 0x7f149419cc60 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f148c00eaa0 tx=0x7f148c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:54.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.653+0000 7f1490ff9640 1 -- 172.21.15.5:0/4290266239 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f148c00cdb0 con 0x7f1494106a80 2024-09-12T02:57:54.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.653+0000 7f1490ff9640 1 -- 172.21.15.5:0/4290266239 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f148c005020 con 0x7f1494106a80 2024-09-12T02:57:54.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.653+0000 7f1499533640 1 -- 172.21.15.5:0/4290266239 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f149419d630 con 0x7f1494106a80 2024-09-12T02:57:54.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.653+0000 7f1490ff9640 1 -- 172.21.15.5:0/4290266239 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f148c0116b0 con 0x7f1494106a80 2024-09-12T02:57:54.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.653+0000 7f1499533640 1 -- 172.21.15.5:0/4290266239 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14941a4f10 con 0x7f1494106a80 2024-09-12T02:57:54.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.655+0000 7f1490ff9640 1 -- 172.21.15.5:0/4290266239 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f148c005560 con 0x7f1494106a80 2024-09-12T02:57:54.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.656+0000 7f149b90c740 1 Processor -- start 2024-09-12T02:57:54.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.656+0000 7f149b90c740 1 -- start start 2024-09-12T02:57:54.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.656+0000 7f149b90c740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f52fe7e9d0 0x55f52fe7edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:54.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.656+0000 7f149b90c740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f52fe7f2f0 0x55f52fe89670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:54.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.656+0000 7f149b90c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f52fe89bb0 0x55f52fe8bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:54.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.656+0000 7f149b90c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f52fde5090 con 0x55f52fe89bb0 2024-09-12T02:57:54.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.656+0000 7f149b90c740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55f52fd8ca70 con 0x55f52fe7f2f0 2024-09-12T02:57:54.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.656+0000 7f149b90c740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55f52fde1050 con 0x55f52fe7e9d0 2024-09-12T02:57:54.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.656+0000 7f1490ff9640 1 --2- 172.21.15.5:0/4290266239 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f14800a2c90 0x7f14800a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:54.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.656+0000 7f148bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f52fe89bb0 0x55f52fe8bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:54.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.656+0000 7f148bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f52fe89bb0 0x55f52fe8bfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60218/0 (socket says 172.21.15.5:60218) 2024-09-12T02:57:54.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.656+0000 7f148bfff640 1 -- 172.21.15.5:0/81918926 learned_addr learned my addr 172.21.15.5:0/81918926 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:57:54.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.656+0000 7f14937fe640 1 --2- 172.21.15.5:0/4290266239 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f14800a2c90 0x7f14800a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:54.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.657+0000 7f1490ff9640 1 -- 172.21.15.5:0/4290266239 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f148c0cf3b0 con 0x7f1494106a80 2024-09-12T02:57:54.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.657+0000 7f148bfff640 1 -- 172.21.15.5:0/81918926 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f52fe7e9d0 msgr2=0x55f52fe7edb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T02:57:54.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.657+0000 7f148bfff640 1 --2- 172.21.15.5:0/81918926 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f52fe7e9d0 0x55f52fe7edb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:54.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.657+0000 7f148bfff640 1 -- 172.21.15.5:0/81918926 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f52fe7f2f0 msgr2=0x55f52fe89670 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T02:57:54.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.657+0000 7f148bfff640 1 --2- 172.21.15.5:0/81918926 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f52fe7f2f0 0x55f52fe89670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:54.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.657+0000 7f148bfff640 1 -- 172.21.15.5:0/81918926 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f52fd99130 con 0x55f52fe89bb0 2024-09-12T02:57:54.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.657+0000 7f14937fe640 1 --2- 172.21.15.5:0/4290266239 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f14800a2c90 0x7f14800a5150 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f148c097f20 tx=0x7f148c06dad0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:54.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.657+0000 7f148bfff640 1 --2- 172.21.15.5:0/81918926 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f52fe89bb0 0x55f52fe8bfa0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f147c00bbc0 tx=0x7f147c00d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d03d19e2f5ec0ab5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:54.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.657+0000 7f14897fa640 1 -- 172.21.15.5:0/81918926 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f147c017620 con 0x55f52fe89bb0 2024-09-12T02:57:54.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.657+0000 7f14897fa640 1 -- 172.21.15.5:0/81918926 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f147c017ca0 con 0x55f52fe89bb0 2024-09-12T02:57:54.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.657+0000 7f14897fa640 1 -- 172.21.15.5:0/81918926 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f147c00fda0 con 0x55f52fe89bb0 2024-09-12T02:57:54.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.658+0000 7f149b90c740 1 -- 172.21.15.5:0/81918926 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f52fe89bb0 msgr2=0x55f52fe8bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:54.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.658+0000 7f149b90c740 1 --2- 172.21.15.5:0/81918926 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f52fe89bb0 0x55f52fe8bfa0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f147c00bbc0 tx=0x7f147c00d710 comp rx=0 tx=0).stop 2024-09-12T02:57:54.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.658+0000 7f149b90c740 1 -- 172.21.15.5:0/81918926 shutdown_connections 2024-09-12T02:57:54.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.658+0000 7f149b90c740 1 --2- 172.21.15.5:0/81918926 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f52fe89bb0 0x55f52fe8bfa0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:54.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.658+0000 7f149b90c740 1 --2- 172.21.15.5:0/81918926 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f52fe7f2f0 0x55f52fe89670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:54.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.658+0000 7f149b90c740 1 --2- 172.21.15.5:0/81918926 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f52fe7e9d0 0x55f52fe7edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:54.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.658+0000 7f149b90c740 1 -- 172.21.15.5:0/81918926 >> 172.21.15.5:0/81918926 conn(0x55f52fe7af60 msgr2=0x55f52fe7b340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:54.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.658+0000 7f149b90c740 1 -- 172.21.15.5:0/81918926 shutdown_connections 2024-09-12T02:57:54.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.658+0000 7f149b90c740 1 -- 172.21.15.5:0/81918926 wait complete. 2024-09-12T02:57:54.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.658+0000 7f149b90c740 1 -- start start 2024-09-12T02:57:54.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.658+0000 7f149b90c740 1 Processor -- start 2024-09-12T02:57:54.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.659+0000 7f149b90c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f52fe7e9d0 0x55f52fe89450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:54.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.659+0000 7f149b90c740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f52fe7f2f0 0x55f52fe867a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:54.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.659+0000 7f149b90c740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f52fe89bb0 0x55f52fe86ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:54.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.659+0000 7f149b90c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f52fe8c780 con 0x55f52fe7e9d0 2024-09-12T02:57:54.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.659+0000 7f149b90c740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55f52fd9ff20 con 0x55f52fe89bb0 2024-09-12T02:57:54.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.659+0000 7f149b90c740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55f52fe8c900 con 0x55f52fe7f2f0 2024-09-12T02:57:54.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.659+0000 7f149b90c740 10 client.0 _ll_register_callbacks cb 0x55f52fd69030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:57:54.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.659+0000 7f148bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f52fe7e9d0 0x55f52fe89450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:54.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.659+0000 7f148bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f52fe7e9d0 0x55f52fe89450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60222/0 (socket says 172.21.15.5:60222) 2024-09-12T02:57:54.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.659+0000 7f148bfff640 1 -- 172.21.15.5:0/1446659204 learned_addr learned my addr 172.21.15.5:0/1446659204 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:57:54.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.659+0000 7f1492ffd640 1 --2- 172.21.15.5:0/1446659204 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f52fe7f2f0 0x55f52fe867a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:54.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.659+0000 7f148bfff640 1 --2- 172.21.15.5:0/1446659204 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f52fe89bb0 0x55f52fe86ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:54.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.659+0000 7f148bfff640 1 -- 172.21.15.5:0/1446659204 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f52fe7f2f0 msgr2=0x55f52fe867a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:57:54.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.659+0000 7f148bfff640 1 --2- 172.21.15.5:0/1446659204 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f52fe7f2f0 0x55f52fe867a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:54.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.659+0000 7f148bfff640 1 -- 172.21.15.5:0/1446659204 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f52fe89bb0 msgr2=0x55f52fe86ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:57:54.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.659+0000 7f148bfff640 1 --2- 172.21.15.5:0/1446659204 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f52fe89bb0 0x55f52fe86ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:54.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.659+0000 7f148bfff640 1 -- 172.21.15.5:0/1446659204 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f52fb562b0 con 0x55f52fe7e9d0 2024-09-12T02:57:54.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.659+0000 7f1492ffd640 1 --2- 172.21.15.5:0/1446659204 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f52fe7f2f0 0x55f52fe867a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:57:54.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.659+0000 7f148bfff640 1 --2- 172.21.15.5:0/1446659204 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f52fe7e9d0 0x55f52fe89450 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f147c006010 tx=0x7f147c00f9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0d451aea32fa0b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:54.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.659+0000 7f147a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T02:57:54.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.660+0000 7f147a7fc640 1 -- 172.21.15.5:0/1446659204 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f147c017420 con 0x55f52fe7e9d0 2024-09-12T02:57:54.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.660+0000 7f147a7fc640 1 -- 172.21.15.5:0/1446659204 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f147c0175c0 con 0x55f52fe7e9d0 2024-09-12T02:57:54.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.660+0000 7f149b90c740 1 -- 172.21.15.5:0/1446659204 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f147c017ca0 con 0x55f52fe7e9d0 2024-09-12T02:57:54.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.660+0000 7f147a7fc640 1 -- 172.21.15.5:0/1446659204 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f147c0177c0 con 0x55f52fe7e9d0 2024-09-12T02:57:54.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.660+0000 7f149b90c740 10 client.15153 fetch_fsmap learned FSMap version 25 2024-09-12T02:57:54.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.660+0000 7f149b90c740 1 -- 172.21.15.5:0/1446659204 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f52fe980a0 con 0x55f52fe7e9d0 2024-09-12T02:57:54.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.660+0000 7f147a7fc640 1 -- 172.21.15.5:0/1446659204 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f147c017ca0 con 0x55f52fe7e9d0 2024-09-12T02:57:54.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.660+0000 7f147a7fc640 1 -- 172.21.15.5:0/1446659204 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8362+0+0 (secure 0 0 0) 0x7f147c023040 con 0x55f52fe7e9d0 2024-09-12T02:57:54.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.660+0000 7f149b90c740 10 client.15153 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:57:54.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.660+0000 7f149b90c740 10 client.15153 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-12T02:57:54.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.660+0000 7f149b90c740 20 client.15153 populate_metadata read hostname 'smithi005' 2024-09-12T02:57:54.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.660+0000 7f149b90c740 1 --2- 172.21.15.5:0/1446659204 >> [v2:172.21.15.148:6834/4195086427,v1:172.21.15.148:6835/4195086427] conn(0x55f52fe99560 0x55f52fe9b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:54.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.660+0000 7f149b90c740 4 client.15153 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:57:54.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.660+0000 7f149b90c740 1 -- 172.21.15.5:0/1446659204 --> [v2:172.21.15.148:6834/4195086427,v1:172.21.15.148:6835/4195086427] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f52fdf2600 con 0x55f52fe99560 2024-09-12T02:57:54.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.661+0000 7f1492ffd640 1 --2- 172.21.15.5:0/1446659204 >> [v2:172.21.15.148:6834/4195086427,v1:172.21.15.148:6835/4195086427] conn(0x55f52fe99560 0x55f52fe9b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:54.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.661+0000 7f1492ffd640 1 --2- 172.21.15.5:0/1446659204 >> [v2:172.21.15.148:6834/4195086427,v1:172.21.15.148:6835/4195086427] conn(0x55f52fe99560 0x55f52fe9b760 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6ec4dc30c1963162 server_cookie=971947171cb1b82e in_seq=0 out_seq=0 2024-09-12T02:57:54.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.661+0000 7f147a7fc640 10 client.15153 ms_handle_connect on v2:172.21.15.148:6834/4195086427 2024-09-12T02:57:54.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.662+0000 7f147a7fc640 1 -- 172.21.15.5:0/1446659204 <== mds.2 v2:172.21.15.148:6834/4195086427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f52fdf2600 con 0x55f52fe99560 2024-09-12T02:57:54.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.662+0000 7f147a7fc640 10 client.15153 handle_command_reply: tid=0 2024-09-12T02:57:54.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.662+0000 7f149b90c740 1 client.15153 shutdown 2024-09-12T02:57:54.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.662+0000 7f149b90c740 1 -- 172.21.15.5:0/1446659204 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f52fe7e9d0 msgr2=0x55f52fe89450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:54.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.662+0000 7f149b90c740 1 --2- 172.21.15.5:0/1446659204 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f52fe7e9d0 0x55f52fe89450 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f147c006010 tx=0x7f147c00f9a0 comp rx=0 tx=0).stop 2024-09-12T02:57:54.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.662+0000 7f149b90c740 1 -- 172.21.15.5:0/1446659204 shutdown_connections 2024-09-12T02:57:54.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.662+0000 7f149b90c740 1 --2- 172.21.15.5:0/1446659204 >> [v2:172.21.15.148:6834/4195086427,v1:172.21.15.148:6835/4195086427] conn(0x55f52fe99560 0x55f52fe9b760 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:54.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.662+0000 7f149b90c740 1 --2- 172.21.15.5:0/1446659204 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f52fe89bb0 0x55f52fe86ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:54.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.662+0000 7f149b90c740 1 --2- 172.21.15.5:0/1446659204 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f52fe7f2f0 0x55f52fe867a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:54.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.662+0000 7f147a7fc640 0 client.15153 ms_handle_reset on v2:172.21.15.148:6834/4195086427 2024-09-12T02:57:54.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.662+0000 7f149b90c740 1 --2- 172.21.15.5:0/1446659204 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f52fe7e9d0 0x55f52fe89450 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:54.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.662+0000 7f149b90c740 1 -- 172.21.15.5:0/1446659204 >> 172.21.15.5:0/1446659204 conn(0x55f52fe7af60 msgr2=0x55f52fe80c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:54.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.662+0000 7f149b90c740 1 -- 172.21.15.5:0/1446659204 shutdown_connections 2024-09-12T02:57:54.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.662+0000 7f149b90c740 1 -- 172.21.15.5:0/1446659204 wait complete. 2024-09-12T02:57:54.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.662+0000 7f149b90c740 20 client.15153 trim_cache size 0 max 16384 2024-09-12T02:57:54.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.680+0000 7f149b90c740 1 Processor -- start 2024-09-12T02:57:54.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.680+0000 7f149b90c740 1 -- start start 2024-09-12T02:57:54.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.680+0000 7f149b90c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f52fe7e9d0 0x55f52fe99c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:54.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.680+0000 7f149b90c740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f52fe7f2f0 0x55f52fe9a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:54.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.680+0000 7f149b90c740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f52fe89bb0 0x55f52fe86080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:54.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.680+0000 7f149b90c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f52fe99aa0 con 0x55f52fe7e9d0 2024-09-12T02:57:54.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.680+0000 7f149b90c740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55f52fe9a170 con 0x55f52fe89bb0 2024-09-12T02:57:54.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.680+0000 7f149b90c740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55f52fe9a840 con 0x55f52fe7f2f0 2024-09-12T02:57:54.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.681+0000 7f148bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f52fe7e9d0 0x55f52fe99c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:54.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.681+0000 7f148bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f52fe7e9d0 0x55f52fe99c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60230/0 (socket says 172.21.15.5:60230) 2024-09-12T02:57:54.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.681+0000 7f148bfff640 1 -- 172.21.15.5:0/849038418 learned_addr learned my addr 172.21.15.5:0/849038418 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:57:54.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.681+0000 7f1492ffd640 1 --2- 172.21.15.5:0/849038418 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f52fe7f2f0 0x55f52fe9a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:54.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.681+0000 7f148bfff640 1 --2- 172.21.15.5:0/849038418 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f52fe89bb0 0x55f52fe86080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:54.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.681+0000 7f148bfff640 1 -- 172.21.15.5:0/849038418 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f52fe7f2f0 msgr2=0x55f52fe9a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:57:54.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.681+0000 7f148bfff640 1 --2- 172.21.15.5:0/849038418 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f52fe7f2f0 0x55f52fe9a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:54.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.681+0000 7f148bfff640 1 -- 172.21.15.5:0/849038418 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f52fe89bb0 msgr2=0x55f52fe86080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:57:54.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.681+0000 7f148bfff640 1 --2- 172.21.15.5:0/849038418 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f52fe89bb0 0x55f52fe86080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:54.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.681+0000 7f148bfff640 1 -- 172.21.15.5:0/849038418 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f52fe897c0 con 0x55f52fe7e9d0 2024-09-12T02:57:54.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.681+0000 7f148bfff640 1 --2- 172.21.15.5:0/849038418 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f52fe7e9d0 0x55f52fe99c20 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f147c0245c0 tx=0x7f147c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bee598b6ab869ae7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:54.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.681+0000 7f1488ff9640 1 -- 172.21.15.5:0/849038418 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f147c00b4b0 con 0x55f52fe7e9d0 2024-09-12T02:57:54.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.681+0000 7f1488ff9640 1 -- 172.21.15.5:0/849038418 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f147c017420 con 0x55f52fe7e9d0 2024-09-12T02:57:54.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.681+0000 7f1488ff9640 1 -- 172.21.15.5:0/849038418 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f147c024a40 con 0x55f52fe7e9d0 2024-09-12T02:57:54.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.682+0000 7f149b90c740 1 -- 172.21.15.5:0/849038418 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f52fe7e9d0 msgr2=0x55f52fe99c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:54.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.682+0000 7f149b90c740 1 --2- 172.21.15.5:0/849038418 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f52fe7e9d0 0x55f52fe99c20 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f147c0245c0 tx=0x7f147c00b960 comp rx=0 tx=0).stop 2024-09-12T02:57:54.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.682+0000 7f149b90c740 1 -- 172.21.15.5:0/849038418 shutdown_connections 2024-09-12T02:57:54.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.682+0000 7f149b90c740 1 --2- 172.21.15.5:0/849038418 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f52fe89bb0 0x55f52fe86080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:54.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.682+0000 7f149b90c740 1 --2- 172.21.15.5:0/849038418 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f52fe7f2f0 0x55f52fe9a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:54.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.682+0000 7f149b90c740 1 --2- 172.21.15.5:0/849038418 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f52fe7e9d0 0x55f52fe99c20 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:54.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.682+0000 7f149b90c740 1 -- 172.21.15.5:0/849038418 >> 172.21.15.5:0/849038418 conn(0x55f52fe7af60 msgr2=0x55f52fe815c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:54.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.682+0000 7f149b90c740 1 -- 172.21.15.5:0/849038418 shutdown_connections 2024-09-12T02:57:54.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.682+0000 7f149b90c740 1 -- 172.21.15.5:0/849038418 wait complete. 2024-09-12T02:57:54.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.683+0000 7f149b90c740 1 -- start start 2024-09-12T02:57:54.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.683+0000 7f149b90c740 1 Processor -- start 2024-09-12T02:57:54.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.683+0000 7f149b90c740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f52fe7e9d0 0x55f52fe83c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:54.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.683+0000 7f149b90c740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f52fe7f2f0 0x55f52fe84140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:54.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.683+0000 7f149b90c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f52fe89bb0 0x55f52fea2e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:54.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.683+0000 7f149b90c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f52fe9b5e0 con 0x55f52fe89bb0 2024-09-12T02:57:54.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.683+0000 7f149b90c740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55f52fe9af10 con 0x55f52fe7f2f0 2024-09-12T02:57:54.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.683+0000 7f149b90c740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55f52fe9bcb0 con 0x55f52fe7e9d0 2024-09-12T02:57:54.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.683+0000 7f149b90c740 10 client.0 _ll_register_callbacks cb 0x7f147c00ba90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:57:54.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.683+0000 7f148bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f52fe89bb0 0x55f52fea2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:54.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.683+0000 7f148bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f52fe89bb0 0x55f52fea2e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60234/0 (socket says 172.21.15.5:60234) 2024-09-12T02:57:54.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.683+0000 7f148bfff640 1 -- 172.21.15.5:0/3984053263 learned_addr learned my addr 172.21.15.5:0/3984053263 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:57:54.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.683+0000 7f1492ffd640 1 --2- 172.21.15.5:0/3984053263 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f52fe7f2f0 0x55f52fe84140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:54.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.683+0000 7f148bfff640 1 --2- 172.21.15.5:0/3984053263 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f52fe7e9d0 0x55f52fe83c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:54.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.683+0000 7f148bfff640 1 -- 172.21.15.5:0/3984053263 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f52fe7e9d0 msgr2=0x55f52fe83c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:57:54.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.683+0000 7f148bfff640 1 --2- 172.21.15.5:0/3984053263 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f52fe7e9d0 0x55f52fe83c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:54.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.683+0000 7f148bfff640 1 -- 172.21.15.5:0/3984053263 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f52fe7f2f0 msgr2=0x55f52fe84140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:57:54.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.683+0000 7f148bfff640 1 --2- 172.21.15.5:0/3984053263 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f52fe7f2f0 0x55f52fe84140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:54.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.683+0000 7f148bfff640 1 -- 172.21.15.5:0/3984053263 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f52fea34d0 con 0x55f52fe89bb0 2024-09-12T02:57:54.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.684+0000 7f1492ffd640 1 --2- 172.21.15.5:0/3984053263 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f52fe7f2f0 0x55f52fe84140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:57:54.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.684+0000 7f148bfff640 1 --2- 172.21.15.5:0/3984053263 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f52fe89bb0 0x55f52fea2e60 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f147c024590 tx=0x7f147c024500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=999d0288bfb13fb7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:54.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.684+0000 7f147a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T02:57:54.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.684+0000 7f147a7fc640 1 -- 172.21.15.5:0/3984053263 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f147c017c30 con 0x55f52fe89bb0 2024-09-12T02:57:54.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.684+0000 7f147a7fc640 1 -- 172.21.15.5:0/3984053263 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f147c03f040 con 0x55f52fe89bb0 2024-09-12T02:57:54.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.684+0000 7f149b90c740 1 -- 172.21.15.5:0/3984053263 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f147c017420 con 0x55f52fe89bb0 2024-09-12T02:57:54.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.684+0000 7f147a7fc640 1 -- 172.21.15.5:0/3984053263 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f147c00f420 con 0x55f52fe89bb0 2024-09-12T02:57:54.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.684+0000 7f149b90c740 10 client.15159 fetch_fsmap learned FSMap version 25 2024-09-12T02:57:54.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.684+0000 7f149b90c740 1 -- 172.21.15.5:0/3984053263 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f52ff08300 con 0x55f52fe89bb0 2024-09-12T02:57:54.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.684+0000 7f147a7fc640 1 -- 172.21.15.5:0/3984053263 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f147c017420 con 0x55f52fe89bb0 2024-09-12T02:57:54.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.684+0000 7f147a7fc640 1 -- 172.21.15.5:0/3984053263 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8362+0+0 (secure 0 0 0) 0x7f147c04f080 con 0x55f52fe89bb0 2024-09-12T02:57:54.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.685+0000 7f149b90c740 10 client.15159 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:57:54.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.685+0000 7f149b90c740 10 client.15159 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-12T02:57:54.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.685+0000 7f149b90c740 20 client.15159 populate_metadata read hostname 'smithi005' 2024-09-12T02:57:54.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.685+0000 7f149b90c740 1 --2- 172.21.15.5:0/3984053263 >> [v2:172.21.15.148:6834/4195086427,v1:172.21.15.148:6835/4195086427] conn(0x55f52ff0cbc0 0x55f52ff0b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:54.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.685+0000 7f149b90c740 4 client.15159 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T02:57:54.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.685+0000 7f149b90c740 1 -- 172.21.15.5:0/3984053263 --> [v2:172.21.15.148:6834/4195086427,v1:172.21.15.148:6835/4195086427] -- command(tid 0: {"prefix": "status"}) -- 0x55f52fdecb10 con 0x55f52ff0cbc0 2024-09-12T02:57:54.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.685+0000 7f1492ffd640 1 --2- 172.21.15.5:0/3984053263 >> [v2:172.21.15.148:6834/4195086427,v1:172.21.15.148:6835/4195086427] conn(0x55f52ff0cbc0 0x55f52ff0b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:54.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.685+0000 7f147a7fc640 10 client.15159 ms_handle_connect on v2:172.21.15.148:6834/4195086427 2024-09-12T02:57:54.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.685+0000 7f1492ffd640 1 --2- 172.21.15.5:0/3984053263 >> [v2:172.21.15.148:6834/4195086427,v1:172.21.15.148:6835/4195086427] conn(0x55f52ff0cbc0 0x55f52ff0b8d0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=82c2748dcddc6b12 server_cookie=86505af471c7bd7e in_seq=0 out_seq=0 2024-09-12T02:57:54.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.686+0000 7f147a7fc640 1 -- 172.21.15.5:0/3984053263 <== mds.2 v2:172.21.15.148:6834/4195086427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55f52fdecb10 con 0x55f52ff0cbc0 2024-09-12T02:57:54.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.686+0000 7f147a7fc640 10 client.15159 handle_command_reply: tid=0 2024-09-12T02:57:54.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.686+0000 7f149b90c740 1 client.15159 shutdown 2024-09-12T02:57:54.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.686+0000 7f149b90c740 1 -- 172.21.15.5:0/3984053263 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f52fe89bb0 msgr2=0x55f52fea2e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:54.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.686+0000 7f149b90c740 1 --2- 172.21.15.5:0/3984053263 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f52fe89bb0 0x55f52fea2e60 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f147c024590 tx=0x7f147c024500 comp rx=0 tx=0).stop 2024-09-12T02:57:54.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.686+0000 7f149b90c740 1 -- 172.21.15.5:0/3984053263 shutdown_connections 2024-09-12T02:57:54.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.686+0000 7f149b90c740 1 --2- 172.21.15.5:0/3984053263 >> [v2:172.21.15.148:6834/4195086427,v1:172.21.15.148:6835/4195086427] conn(0x55f52ff0cbc0 0x55f52ff0b8d0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:54.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.686+0000 7f149b90c740 1 --2- 172.21.15.5:0/3984053263 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f52fe89bb0 0x55f52fea2e60 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:54.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.686+0000 7f149b90c740 1 --2- 172.21.15.5:0/3984053263 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f52fe7f2f0 0x55f52fe84140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:54.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.686+0000 7f147a7fc640 0 client.15159 ms_handle_reset on v2:172.21.15.148:6834/4195086427 2024-09-12T02:57:54.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.686+0000 7f149b90c740 1 --2- 172.21.15.5:0/3984053263 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f52fe7e9d0 0x55f52fe83c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:54.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.686+0000 7f149b90c740 1 -- 172.21.15.5:0/3984053263 >> 172.21.15.5:0/3984053263 conn(0x55f52fe7af60 msgr2=0x55f52fe81da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:54.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.686+0000 7f149b90c740 1 -- 172.21.15.5:0/3984053263 shutdown_connections 2024-09-12T02:57:54.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.686+0000 7f149b90c740 1 -- 172.21.15.5:0/3984053263 wait complete. 2024-09-12T02:57:54.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.686+0000 7f149b90c740 20 client.15159 trim_cache size 0 max 16384 2024-09-12T02:57:54.701 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-12T02:57:54.702 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "618305f8-70b1-11ef-bcea-c7b262605968", 2024-09-12T02:57:54.702 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 2, 2024-09-12T02:57:54.702 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24481, 2024-09-12T02:57:54.702 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-09-12T02:57:54.702 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-09-12T02:57:54.702 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-09-12T02:57:54.702 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 89.685865730000003, 2024-09-12T02:57:54.702 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 25, 2024-09-12T02:57:54.702 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 83, 2024-09-12T02:57:54.702 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 81, 2024-09-12T02:57:54.702 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 124.459761813 2024-09-12T02:57:54.703 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-12T02:57:54.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.687+0000 7f1499533640 1 -- 172.21.15.5:0/4290266239 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f14800a2c90 msgr2=0x7f14800a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:54.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.687+0000 7f1499533640 1 --2- 172.21.15.5:0/4290266239 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f14800a2c90 0x7f14800a5150 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f148c097f20 tx=0x7f148c06dad0 comp rx=0 tx=0).stop 2024-09-12T02:57:54.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.687+0000 7f1499533640 1 -- 172.21.15.5:0/4290266239 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1494106a80 msgr2=0x7f149419cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:54.703 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.687+0000 7f1499533640 1 --2- 172.21.15.5:0/4290266239 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1494106a80 0x7f149419cc60 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f148c00eaa0 tx=0x7f148c00ef60 comp rx=0 tx=0).stop 2024-09-12T02:57:54.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.687+0000 7f1499533640 1 -- 172.21.15.5:0/4290266239 shutdown_connections 2024-09-12T02:57:54.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.687+0000 7f1499533640 1 --2- 172.21.15.5:0/4290266239 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f14800a2c90 0x7f14800a5150 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:54.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.687+0000 7f1499533640 1 --2- 172.21.15.5:0/4290266239 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1494106a80 0x7f149419cc60 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:54.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.687+0000 7f1499533640 1 --2- 172.21.15.5:0/4290266239 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1494101420 0x7f14941988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:54.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.687+0000 7f1499533640 1 --2- 172.21.15.5:0/4290266239 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1494100a80 0x7f1494198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:54.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.687+0000 7f1499533640 1 -- 172.21.15.5:0/4290266239 >> 172.21.15.5:0/4290266239 conn(0x7f14940fc7a0 msgr2=0x7f149406a970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:54.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.687+0000 7f1499533640 1 -- 172.21.15.5:0/4290266239 shutdown_connections 2024-09-12T02:57:54.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:54.687+0000 7f1499533640 1 -- 172.21.15.5:0/4290266239 wait complete. 2024-09-12T02:57:55.361 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.c status 2024-09-12T02:57:55.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:57:55 smithi005 ceph-mon[31197]: pgmap v325: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-12T02:57:55.619 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:57:55.665 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:57:55 smithi148 ceph-mon[35090]: pgmap v325: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-12T02:57:55.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:57:55 smithi146 ceph-mon[35011]: pgmap v325: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-12T02:57:56.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.044+0000 7f481a7bd640 1 -- 172.21.15.5:0/113599412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4814106a80 msgr2=0x7f4814106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:56.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.044+0000 7f481a7bd640 1 --2- 172.21.15.5:0/113599412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4814106a80 0x7f4814106e60 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f4804009930 tx=0x7f480402a440 comp rx=0 tx=0).stop 2024-09-12T02:57:56.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.044+0000 7f481a7bd640 1 -- 172.21.15.5:0/113599412 shutdown_connections 2024-09-12T02:57:56.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.044+0000 7f481a7bd640 1 --2- 172.21.15.5:0/113599412 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4814101420 0x7f481410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:56.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.044+0000 7f481a7bd640 1 --2- 172.21.15.5:0/113599412 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4814100a80 0x7f4814100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:56.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.044+0000 7f481a7bd640 1 --2- 172.21.15.5:0/113599412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4814106a80 0x7f4814106e60 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:56.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.044+0000 7f481a7bd640 1 -- 172.21.15.5:0/113599412 >> 172.21.15.5:0/113599412 conn(0x7f48140fc7a0 msgr2=0x7f48140febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:56.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.044+0000 7f481a7bd640 1 -- 172.21.15.5:0/113599412 shutdown_connections 2024-09-12T02:57:56.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.044+0000 7f481a7bd640 1 -- 172.21.15.5:0/113599412 wait complete. 2024-09-12T02:57:56.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.045+0000 7f481a7bd640 1 Processor -- start 2024-09-12T02:57:56.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.045+0000 7f481a7bd640 1 -- start start 2024-09-12T02:57:56.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.045+0000 7f481a7bd640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4814100a80 0x7f4814198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:56.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.045+0000 7f481a7bd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4814101420 0x7f4814198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:56.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.045+0000 7f481a7bd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4814106a80 0x7f481419ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:56.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.045+0000 7f481a7bd640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4814111fe0 con 0x7f4814101420 2024-09-12T02:57:56.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.045+0000 7f481a7bd640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f4814111e60 con 0x7f4814100a80 2024-09-12T02:57:56.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.045+0000 7f481a7bd640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4814112160 con 0x7f4814106a80 2024-09-12T02:57:56.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.045+0000 7f48137fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4814101420 0x7f4814198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:56.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.046+0000 7f48137fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4814101420 0x7f4814198a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60248/0 (socket says 172.21.15.5:60248) 2024-09-12T02:57:56.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.046+0000 7f48137fe640 1 -- 172.21.15.5:0/3756750141 learned_addr learned my addr 172.21.15.5:0/3756750141 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:57:56.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.046+0000 7f4818d5a640 1 --2- 172.21.15.5:0/3756750141 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4814106a80 0x7f481419ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:56.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.046+0000 7f4813fff640 1 --2- 172.21.15.5:0/3756750141 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4814100a80 0x7f4814198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:56.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.046+0000 7f48137fe640 1 -- 172.21.15.5:0/3756750141 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4814106a80 msgr2=0x7f481419ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:56.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.046+0000 7f48137fe640 1 --2- 172.21.15.5:0/3756750141 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4814106a80 0x7f481419ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:56.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.046+0000 7f48137fe640 1 -- 172.21.15.5:0/3756750141 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4814100a80 msgr2=0x7f4814198540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:56.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.046+0000 7f48137fe640 1 --2- 172.21.15.5:0/3756750141 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4814100a80 0x7f4814198540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:56.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.046+0000 7f48137fe640 1 -- 172.21.15.5:0/3756750141 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f481419d4f0 con 0x7f4814101420 2024-09-12T02:57:56.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.046+0000 7f4813fff640 1 --2- 172.21.15.5:0/3756750141 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4814100a80 0x7f4814198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:57:56.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.046+0000 7f48137fe640 1 --2- 172.21.15.5:0/3756750141 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4814101420 0x7f4814198a80 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f480800b450 tx=0x7f480800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:56.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.046+0000 7f48117fa640 1 -- 172.21.15.5:0/3756750141 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4808015020 con 0x7f4814101420 2024-09-12T02:57:56.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.046+0000 7f481a7bd640 1 -- 172.21.15.5:0/3756750141 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f481419d7e0 con 0x7f4814101420 2024-09-12T02:57:56.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.046+0000 7f48117fa640 1 -- 172.21.15.5:0/3756750141 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4808011d70 con 0x7f4814101420 2024-09-12T02:57:56.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.046+0000 7f48117fa640 1 -- 172.21.15.5:0/3756750141 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f480800f340 con 0x7f4814101420 2024-09-12T02:57:56.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.046+0000 7f481a7bd640 1 -- 172.21.15.5:0/3756750141 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48141a50c0 con 0x7f4814101420 2024-09-12T02:57:56.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.049+0000 7f48117fa640 1 -- 172.21.15.5:0/3756750141 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f4808018020 con 0x7f4814101420 2024-09-12T02:57:56.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.049+0000 7f481cb96740 1 Processor -- start 2024-09-12T02:57:56.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.049+0000 7f481cb96740 1 -- start start 2024-09-12T02:57:56.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.049+0000 7f481cb96740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633651b18b0 0x5633651b1c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:56.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.049+0000 7f481cb96740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5633651b21d0 0x5633651c0a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:56.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.049+0000 7f481cb96740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5633651c0fc0 0x5633651c33b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:56.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.049+0000 7f481cb96740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x563365118090 con 0x5633651b18b0 2024-09-12T02:57:56.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.049+0000 7f481cb96740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5633650bfa70 con 0x5633651c0fc0 2024-09-12T02:57:56.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.049+0000 7f481cb96740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x563365114050 con 0x5633651b21d0 2024-09-12T02:57:56.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.050+0000 7f4818d5a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633651b18b0 0x5633651b1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:56.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.050+0000 7f4818d5a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633651b18b0 0x5633651b1c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60262/0 (socket says 172.21.15.5:60262) 2024-09-12T02:57:56.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.050+0000 7f4818d5a640 1 -- 172.21.15.5:0/842063985 learned_addr learned my addr 172.21.15.5:0/842063985 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:57:56.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.050+0000 7f4813fff640 1 --2- 172.21.15.5:0/842063985 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5633651b21d0 0x5633651c0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:56.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.050+0000 7f48117fa640 1 --2- 172.21.15.5:0/3756750141 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f47fc0a2c10 0x7f47fc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:56.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.050+0000 7f48137fe640 1 --2- 172.21.15.5:0/842063985 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5633651c0fc0 0x5633651c33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:56.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.050+0000 7f4818d5a640 1 --2- 172.21.15.5:0/3756750141 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f47fc0a2c10 0x7f47fc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:56.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.050+0000 7f4818d5a640 1 -- 172.21.15.5:0/842063985 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5633651b21d0 msgr2=0x5633651c0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:57:56.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.050+0000 7f4818d5a640 1 --2- 172.21.15.5:0/842063985 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5633651b21d0 0x5633651c0a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:56.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.050+0000 7f4818d5a640 1 -- 172.21.15.5:0/842063985 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5633651c0fc0 msgr2=0x5633651c33b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:57:56.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.050+0000 7f4818d5a640 1 --2- 172.21.15.5:0/842063985 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5633651c0fc0 0x5633651c33b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:56.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.050+0000 7f4818d5a640 1 -- 172.21.15.5:0/842063985 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633650cc130 con 0x5633651b18b0 2024-09-12T02:57:56.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.050+0000 7f48137fe640 1 --2- 172.21.15.5:0/842063985 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5633651c0fc0 0x5633651c33b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:57:56.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.050+0000 7f48117fa640 1 -- 172.21.15.5:0/3756750141 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f48080cec70 con 0x7f4814101420 2024-09-12T02:57:56.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.050+0000 7f4818d5a640 1 --2- 172.21.15.5:0/842063985 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633651b18b0 0x5633651b1c90 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f480c012890 tx=0x7f480c012d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c818216630379ac server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:56.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.050+0000 7f47fa7fc640 1 -- 172.21.15.5:0/842063985 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f480c003d40 con 0x5633651b18b0 2024-09-12T02:57:56.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.050+0000 7f47fa7fc640 1 -- 172.21.15.5:0/842063985 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f480c008e10 con 0x5633651b18b0 2024-09-12T02:57:56.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.050+0000 7f47fa7fc640 1 -- 172.21.15.5:0/842063985 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f480c01c890 con 0x5633651b18b0 2024-09-12T02:57:56.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.051+0000 7f481cb96740 1 -- 172.21.15.5:0/842063985 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633651b18b0 msgr2=0x5633651b1c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:56.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.051+0000 7f4818d5a640 1 --2- 172.21.15.5:0/3756750141 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f47fc0a2c10 0x7f47fc0a50d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f480c01f000 tx=0x7f480c01e000 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:56.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.051+0000 7f481cb96740 1 --2- 172.21.15.5:0/842063985 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633651b18b0 0x5633651b1c90 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f480c012890 tx=0x7f480c012d50 comp rx=0 tx=0).stop 2024-09-12T02:57:56.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.051+0000 7f481cb96740 1 -- 172.21.15.5:0/842063985 shutdown_connections 2024-09-12T02:57:56.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.051+0000 7f481cb96740 1 --2- 172.21.15.5:0/842063985 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5633651c0fc0 0x5633651c33b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:56.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.051+0000 7f481cb96740 1 --2- 172.21.15.5:0/842063985 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5633651b21d0 0x5633651c0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:56.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.051+0000 7f481cb96740 1 --2- 172.21.15.5:0/842063985 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633651b18b0 0x5633651b1c90 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:56.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.051+0000 7f481cb96740 1 -- 172.21.15.5:0/842063985 >> 172.21.15.5:0/842063985 conn(0x5633651adf60 msgr2=0x5633651ae340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:56.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.051+0000 7f481cb96740 1 -- 172.21.15.5:0/842063985 shutdown_connections 2024-09-12T02:57:56.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.051+0000 7f481cb96740 1 -- 172.21.15.5:0/842063985 wait complete. 2024-09-12T02:57:56.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.052+0000 7f481cb96740 1 -- start start 2024-09-12T02:57:56.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.052+0000 7f481cb96740 1 Processor -- start 2024-09-12T02:57:56.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.052+0000 7f481cb96740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633651b18b0 0x5633651c0830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:56.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.052+0000 7f481cb96740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5633651b21d0 0x5633651bdb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:56.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.052+0000 7f481cb96740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5633651c0fc0 0x5633651be0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:56.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.052+0000 7f481cb96740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5633651c3b90 con 0x5633651b18b0 2024-09-12T02:57:56.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.052+0000 7f481cb96740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5633650d2f20 con 0x5633651c0fc0 2024-09-12T02:57:56.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.052+0000 7f481cb96740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5633651c3d10 con 0x5633651b21d0 2024-09-12T02:57:56.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.052+0000 7f481cb96740 10 client.0 _ll_register_callbacks cb 0x56336509c030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:57:56.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.052+0000 7f4813fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633651b18b0 0x5633651c0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:56.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.052+0000 7f4813fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633651b18b0 0x5633651c0830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60278/0 (socket says 172.21.15.5:60278) 2024-09-12T02:57:56.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.052+0000 7f4813fff640 1 -- 172.21.15.5:0/249718251 learned_addr learned my addr 172.21.15.5:0/249718251 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:57:56.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.052+0000 7f48137fe640 1 --2- 172.21.15.5:0/249718251 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5633651b21d0 0x5633651bdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:56.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.053+0000 7f4813fff640 1 -- 172.21.15.5:0/249718251 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5633651b21d0 msgr2=0x5633651bdb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:57:56.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.053+0000 7f4813fff640 1 --2- 172.21.15.5:0/249718251 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5633651b21d0 0x5633651bdb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:56.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.053+0000 7f4813fff640 1 -- 172.21.15.5:0/249718251 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5633651c0fc0 msgr2=0x5633651be0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:57:56.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.053+0000 7f4818d5a640 1 --2- 172.21.15.5:0/249718251 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5633651c0fc0 0x5633651be0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:56.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.053+0000 7f4813fff640 1 --2- 172.21.15.5:0/249718251 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5633651c0fc0 0x5633651be0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:56.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.053+0000 7f4813fff640 1 -- 172.21.15.5:0/249718251 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563364e892b0 con 0x5633651b18b0 2024-09-12T02:57:56.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.053+0000 7f4818d5a640 1 --2- 172.21.15.5:0/249718251 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5633651c0fc0 0x5633651be0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:57:56.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.053+0000 7f4813fff640 1 --2- 172.21.15.5:0/249718251 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633651b18b0 0x5633651c0830 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f4804009930 tx=0x7f4804031850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64b961397a5836af server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:56.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.053+0000 7f47f37fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T02:57:56.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.053+0000 7f47f37fe640 1 -- 172.21.15.5:0/249718251 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4804035040 con 0x5633651b18b0 2024-09-12T02:57:56.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.053+0000 7f47f37fe640 1 -- 172.21.15.5:0/249718251 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f4804004450 con 0x5633651b18b0 2024-09-12T02:57:56.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.053+0000 7f481cb96740 1 -- 172.21.15.5:0/249718251 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x563365125600 con 0x5633651b18b0 2024-09-12T02:57:56.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.053+0000 7f47f37fe640 1 -- 172.21.15.5:0/249718251 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4804031a00 con 0x5633651b18b0 2024-09-12T02:57:56.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.053+0000 7f481cb96740 10 client.15171 fetch_fsmap learned FSMap version 25 2024-09-12T02:57:56.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.053+0000 7f481cb96740 1 -- 172.21.15.5:0/249718251 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5633651cf4c0 con 0x5633651b18b0 2024-09-12T02:57:56.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.053+0000 7f47f37fe640 1 -- 172.21.15.5:0/249718251 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x563365125600 con 0x5633651b18b0 2024-09-12T02:57:56.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.054+0000 7f47f37fe640 1 -- 172.21.15.5:0/249718251 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8362+0+0 (secure 0 0 0) 0x7f48040035f0 con 0x5633651b18b0 2024-09-12T02:57:56.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.054+0000 7f481cb96740 10 client.15171 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:57:56.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.054+0000 7f481cb96740 10 client.15171 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-12T02:57:56.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.054+0000 7f481cb96740 20 client.15171 populate_metadata read hostname 'smithi005' 2024-09-12T02:57:56.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.054+0000 7f481cb96740 1 --2- 172.21.15.5:0/249718251 >> [v2:172.21.15.148:6836/2606873922,v1:172.21.15.148:6837/2606873922] conn(0x5633651d1b20 0x5633651d2f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:56.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.054+0000 7f481cb96740 4 client.15171 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:57:56.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.054+0000 7f481cb96740 1 -- 172.21.15.5:0/249718251 --> [v2:172.21.15.148:6836/2606873922,v1:172.21.15.148:6837/2606873922] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56336511fb10 con 0x5633651d1b20 2024-09-12T02:57:56.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.054+0000 7f48137fe640 1 --2- 172.21.15.5:0/249718251 >> [v2:172.21.15.148:6836/2606873922,v1:172.21.15.148:6837/2606873922] conn(0x5633651d1b20 0x5633651d2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:56.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.054+0000 7f47f37fe640 10 client.15171 ms_handle_connect on v2:172.21.15.148:6836/2606873922 2024-09-12T02:57:56.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.054+0000 7f48137fe640 1 --2- 172.21.15.5:0/249718251 >> [v2:172.21.15.148:6836/2606873922,v1:172.21.15.148:6837/2606873922] conn(0x5633651d1b20 0x5633651d2f90 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f387b29fd5a54e30 server_cookie=fd6732f50eeec3 in_seq=0 out_seq=0 2024-09-12T02:57:56.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.056+0000 7f47f37fe640 1 -- 172.21.15.5:0/249718251 <== mds.1 v2:172.21.15.148:6836/2606873922 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56336511fb10 con 0x5633651d1b20 2024-09-12T02:57:56.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.056+0000 7f47f37fe640 10 client.15171 handle_command_reply: tid=0 2024-09-12T02:57:56.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.056+0000 7f481cb96740 1 client.15171 shutdown 2024-09-12T02:57:56.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.056+0000 7f481cb96740 1 -- 172.21.15.5:0/249718251 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633651b18b0 msgr2=0x5633651c0830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:56.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.056+0000 7f481cb96740 1 --2- 172.21.15.5:0/249718251 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633651b18b0 0x5633651c0830 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f4804009930 tx=0x7f4804031850 comp rx=0 tx=0).stop 2024-09-12T02:57:56.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.056+0000 7f481cb96740 1 -- 172.21.15.5:0/249718251 shutdown_connections 2024-09-12T02:57:56.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.056+0000 7f481cb96740 1 --2- 172.21.15.5:0/249718251 >> [v2:172.21.15.148:6836/2606873922,v1:172.21.15.148:6837/2606873922] conn(0x5633651d1b20 0x5633651d2f90 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:56.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.056+0000 7f481cb96740 1 --2- 172.21.15.5:0/249718251 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5633651c0fc0 0x5633651be0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:56.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.056+0000 7f481cb96740 1 --2- 172.21.15.5:0/249718251 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5633651b21d0 0x5633651bdb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:56.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.056+0000 7f481cb96740 1 --2- 172.21.15.5:0/249718251 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633651b18b0 0x5633651c0830 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:56.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.056+0000 7f47f37fe640 0 client.15171 ms_handle_reset on v2:172.21.15.148:6836/2606873922 2024-09-12T02:57:56.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.056+0000 7f481cb96740 1 -- 172.21.15.5:0/249718251 >> 172.21.15.5:0/249718251 conn(0x5633651adf60 msgr2=0x5633651b3cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:56.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.056+0000 7f481cb96740 1 -- 172.21.15.5:0/249718251 shutdown_connections 2024-09-12T02:57:56.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.056+0000 7f481cb96740 1 -- 172.21.15.5:0/249718251 wait complete. 2024-09-12T02:57:56.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.056+0000 7f481cb96740 20 client.15171 trim_cache size 0 max 16384 2024-09-12T02:57:56.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.074+0000 7f481cb96740 1 Processor -- start 2024-09-12T02:57:56.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.074+0000 7f481cb96740 1 -- start start 2024-09-12T02:57:56.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.074+0000 7f481cb96740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5633651b18b0 0x5633651d21f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:56.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.074+0000 7f481cb96740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5633651b21d0 0x5633651bc450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:56.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.074+0000 7f481cb96740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633651c0fc0 0x5633651bade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:56.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.074+0000 7f481cb96740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5633651d12d0 con 0x5633651c0fc0 2024-09-12T02:57:56.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.074+0000 7f481cb96740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5633651d19a0 con 0x5633651b18b0 2024-09-12T02:57:56.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.074+0000 7f481cb96740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5633651d2070 con 0x5633651b21d0 2024-09-12T02:57:56.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.074+0000 7f4818d5a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633651c0fc0 0x5633651bade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:56.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.074+0000 7f4818d5a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633651c0fc0 0x5633651bade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60284/0 (socket says 172.21.15.5:60284) 2024-09-12T02:57:56.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.074+0000 7f4818d5a640 1 -- 172.21.15.5:0/553894750 learned_addr learned my addr 172.21.15.5:0/553894750 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:57:56.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.074+0000 7f48137fe640 1 --2- 172.21.15.5:0/553894750 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5633651b21d0 0x5633651bc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:56.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.074+0000 7f4813fff640 1 --2- 172.21.15.5:0/553894750 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5633651b18b0 0x5633651d21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:56.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.074+0000 7f4818d5a640 1 -- 172.21.15.5:0/553894750 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5633651b21d0 msgr2=0x5633651bc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:57:56.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.074+0000 7f4818d5a640 1 --2- 172.21.15.5:0/553894750 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5633651b21d0 0x5633651bc450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:56.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.074+0000 7f4818d5a640 1 -- 172.21.15.5:0/553894750 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5633651b18b0 msgr2=0x5633651d21f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:57:56.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.074+0000 7f4818d5a640 1 --2- 172.21.15.5:0/553894750 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5633651b18b0 0x5633651d21f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:56.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.074+0000 7f4818d5a640 1 -- 172.21.15.5:0/553894750 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633651bcb30 con 0x5633651c0fc0 2024-09-12T02:57:56.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.074+0000 7f4818d5a640 1 --2- 172.21.15.5:0/553894750 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633651c0fc0 0x5633651bade0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f480c000f80 tx=0x7f480c0107f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1157a2ff5bbe2fcc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:56.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.075+0000 7f47f9ffb640 1 -- 172.21.15.5:0/553894750 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f480c003d40 con 0x5633651c0fc0 2024-09-12T02:57:56.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.075+0000 7f47f9ffb640 1 -- 172.21.15.5:0/553894750 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f480c01cbe0 con 0x5633651c0fc0 2024-09-12T02:57:56.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.075+0000 7f47f9ffb640 1 -- 172.21.15.5:0/553894750 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f480c01c880 con 0x5633651c0fc0 2024-09-12T02:57:56.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.075+0000 7f481cb96740 1 -- 172.21.15.5:0/553894750 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633651c0fc0 msgr2=0x5633651bade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:56.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.075+0000 7f481cb96740 1 --2- 172.21.15.5:0/553894750 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633651c0fc0 0x5633651bade0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f480c000f80 tx=0x7f480c0107f0 comp rx=0 tx=0).stop 2024-09-12T02:57:56.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.075+0000 7f481cb96740 1 -- 172.21.15.5:0/553894750 shutdown_connections 2024-09-12T02:57:56.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.075+0000 7f481cb96740 1 --2- 172.21.15.5:0/553894750 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633651c0fc0 0x5633651bade0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:56.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.075+0000 7f481cb96740 1 --2- 172.21.15.5:0/553894750 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5633651b21d0 0x5633651bc450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:56.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.075+0000 7f481cb96740 1 --2- 172.21.15.5:0/553894750 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5633651b18b0 0x5633651d21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:56.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.075+0000 7f481cb96740 1 -- 172.21.15.5:0/553894750 >> 172.21.15.5:0/553894750 conn(0x5633651adf60 msgr2=0x5633651b9240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:56.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.075+0000 7f481cb96740 1 -- 172.21.15.5:0/553894750 shutdown_connections 2024-09-12T02:57:56.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.075+0000 7f481cb96740 1 -- 172.21.15.5:0/553894750 wait complete. 2024-09-12T02:57:56.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.076+0000 7f481cb96740 1 -- start start 2024-09-12T02:57:56.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.076+0000 7f481cb96740 1 Processor -- start 2024-09-12T02:57:56.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.076+0000 7f481cb96740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633651b18b0 0x5633651db640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:56.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.076+0000 7f481cb96740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5633651b21d0 0x5633651dbb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:56.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.076+0000 7f481cb96740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5633651c0fc0 0x5633651d4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:56.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.076+0000 7f481cb96740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5633651d2e10 con 0x5633651b18b0 2024-09-12T02:57:56.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.076+0000 7f481cb96740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5633651d2740 con 0x5633651c0fc0 2024-09-12T02:57:56.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.076+0000 7f481cb96740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5633651d34e0 con 0x5633651b21d0 2024-09-12T02:57:56.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.076+0000 7f481cb96740 10 client.0 _ll_register_callbacks cb 0x7f4804032ed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:57:56.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.076+0000 7f4813fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633651b18b0 0x5633651db640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:56.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.076+0000 7f4813fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633651b18b0 0x5633651db640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60294/0 (socket says 172.21.15.5:60294) 2024-09-12T02:57:56.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.076+0000 7f4813fff640 1 -- 172.21.15.5:0/237872332 learned_addr learned my addr 172.21.15.5:0/237872332 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:57:56.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.076+0000 7f48137fe640 1 --2- 172.21.15.5:0/237872332 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5633651b21d0 0x5633651dbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:56.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.076+0000 7f4818d5a640 1 --2- 172.21.15.5:0/237872332 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5633651c0fc0 0x5633651d4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:56.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.077+0000 7f4813fff640 1 -- 172.21.15.5:0/237872332 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5633651b21d0 msgr2=0x5633651dbb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:57:56.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.077+0000 7f4813fff640 1 --2- 172.21.15.5:0/237872332 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5633651b21d0 0x5633651dbb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:56.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.077+0000 7f4813fff640 1 -- 172.21.15.5:0/237872332 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5633651c0fc0 msgr2=0x5633651d4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:57:56.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.077+0000 7f4813fff640 1 --2- 172.21.15.5:0/237872332 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5633651c0fc0 0x5633651d4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:56.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.077+0000 7f4813fff640 1 -- 172.21.15.5:0/237872332 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5633651d53f0 con 0x5633651b18b0 2024-09-12T02:57:56.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.077+0000 7f4818d5a640 1 --2- 172.21.15.5:0/237872332 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5633651c0fc0 0x5633651d4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:57:56.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.077+0000 7f4813fff640 1 --2- 172.21.15.5:0/237872332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633651b18b0 0x5633651db640 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f4804005ec0 tx=0x7f480402a440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb1700f75a28c1bc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:56.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.077+0000 7f47f37fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T02:57:56.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.077+0000 7f47f37fe640 1 -- 172.21.15.5:0/237872332 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4804035070 con 0x5633651b18b0 2024-09-12T02:57:56.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.077+0000 7f47f37fe640 1 -- 172.21.15.5:0/237872332 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f4804004450 con 0x5633651b18b0 2024-09-12T02:57:56.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.077+0000 7f481cb96740 1 -- 172.21.15.5:0/237872332 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f480c01cbe0 con 0x5633651b18b0 2024-09-12T02:57:56.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.077+0000 7f47f37fe640 1 -- 172.21.15.5:0/237872332 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f480402aa80 con 0x5633651b18b0 2024-09-12T02:57:56.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.077+0000 7f481cb96740 10 client.15177 fetch_fsmap learned FSMap version 25 2024-09-12T02:57:56.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.077+0000 7f481cb96740 1 -- 172.21.15.5:0/237872332 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5633651d9e90 con 0x5633651b18b0 2024-09-12T02:57:56.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.077+0000 7f47f37fe640 1 -- 172.21.15.5:0/237872332 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f480c01cbe0 con 0x5633651b18b0 2024-09-12T02:57:56.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.078+0000 7f47f37fe640 1 -- 172.21.15.5:0/237872332 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8362+0+0 (secure 0 0 0) 0x7f480403c440 con 0x5633651b18b0 2024-09-12T02:57:56.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.078+0000 7f481cb96740 10 client.15177 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:57:56.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.078+0000 7f481cb96740 10 client.15177 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-12T02:57:56.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.078+0000 7f481cb96740 20 client.15177 populate_metadata read hostname 'smithi005' 2024-09-12T02:57:56.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.078+0000 7f481cb96740 1 --2- 172.21.15.5:0/237872332 >> [v2:172.21.15.148:6836/2606873922,v1:172.21.15.148:6837/2606873922] conn(0x563365240b20 0x56336523f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:56.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.078+0000 7f481cb96740 4 client.15177 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T02:57:56.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.078+0000 7f481cb96740 1 -- 172.21.15.5:0/237872332 --> [v2:172.21.15.148:6836/2606873922,v1:172.21.15.148:6837/2606873922] -- command(tid 0: {"prefix": "status"}) -- 0x56336509c0e0 con 0x563365240b20 2024-09-12T02:57:56.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.078+0000 7f48137fe640 1 --2- 172.21.15.5:0/237872332 >> [v2:172.21.15.148:6836/2606873922,v1:172.21.15.148:6837/2606873922] conn(0x563365240b20 0x56336523f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:56.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.078+0000 7f48137fe640 1 --2- 172.21.15.5:0/237872332 >> [v2:172.21.15.148:6836/2606873922,v1:172.21.15.148:6837/2606873922] conn(0x563365240b20 0x56336523f830 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4f5839fa152d0840 server_cookie=3e57373f267c3a1a in_seq=0 out_seq=0 2024-09-12T02:57:56.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.078+0000 7f47f37fe640 10 client.15177 ms_handle_connect on v2:172.21.15.148:6836/2606873922 2024-09-12T02:57:56.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.079+0000 7f47f37fe640 1 -- 172.21.15.5:0/237872332 <== mds.1 v2:172.21.15.148:6836/2606873922 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x56336509c0e0 con 0x563365240b20 2024-09-12T02:57:56.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.079+0000 7f47f37fe640 10 client.15177 handle_command_reply: tid=0 2024-09-12T02:57:56.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.079+0000 7f481cb96740 1 client.15177 shutdown 2024-09-12T02:57:56.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.079+0000 7f481cb96740 1 -- 172.21.15.5:0/237872332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633651b18b0 msgr2=0x5633651db640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:56.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.079+0000 7f481cb96740 1 --2- 172.21.15.5:0/237872332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633651b18b0 0x5633651db640 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f4804005ec0 tx=0x7f480402a440 comp rx=0 tx=0).stop 2024-09-12T02:57:56.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.079+0000 7f481cb96740 1 -- 172.21.15.5:0/237872332 shutdown_connections 2024-09-12T02:57:56.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.079+0000 7f481cb96740 1 --2- 172.21.15.5:0/237872332 >> [v2:172.21.15.148:6836/2606873922,v1:172.21.15.148:6837/2606873922] conn(0x563365240b20 0x56336523f830 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:56.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.079+0000 7f47f37fe640 0 client.15177 ms_handle_reset on v2:172.21.15.148:6836/2606873922 2024-09-12T02:57:56.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.079+0000 7f481cb96740 1 --2- 172.21.15.5:0/237872332 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5633651c0fc0 0x5633651d4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:56.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.079+0000 7f481cb96740 1 --2- 172.21.15.5:0/237872332 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5633651b21d0 0x5633651dbb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:56.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.079+0000 7f481cb96740 1 --2- 172.21.15.5:0/237872332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5633651b18b0 0x5633651db640 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:56.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.079+0000 7f481cb96740 1 -- 172.21.15.5:0/237872332 >> 172.21.15.5:0/237872332 conn(0x5633651adf60 msgr2=0x5633651b9240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:56.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.079+0000 7f481cb96740 1 -- 172.21.15.5:0/237872332 shutdown_connections 2024-09-12T02:57:56.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.079+0000 7f481cb96740 1 -- 172.21.15.5:0/237872332 wait complete. 2024-09-12T02:57:56.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.079+0000 7f481cb96740 20 client.15177 trim_cache size 0 max 16384 2024-09-12T02:57:56.084 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-12T02:57:56.085 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "618305f8-70b1-11ef-bcea-c7b262605968", 2024-09-12T02:57:56.085 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 1, 2024-09-12T02:57:56.085 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24487, 2024-09-12T02:57:56.085 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-09-12T02:57:56.085 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-09-12T02:57:56.085 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-09-12T02:57:56.085 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 92.088428742999994, 2024-09-12T02:57:56.085 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 25, 2024-09-12T02:57:56.085 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 83, 2024-09-12T02:57:56.085 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 81, 2024-09-12T02:57:56.085 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 123.506841599 2024-09-12T02:57:56.085 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-12T02:57:56.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.080+0000 7f481a7bd640 1 -- 172.21.15.5:0/3756750141 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f47fc0a2c10 msgr2=0x7f47fc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:56.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.080+0000 7f481a7bd640 1 --2- 172.21.15.5:0/3756750141 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f47fc0a2c10 0x7f47fc0a50d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f480c01f000 tx=0x7f480c01e000 comp rx=0 tx=0).stop 2024-09-12T02:57:56.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.080+0000 7f481a7bd640 1 -- 172.21.15.5:0/3756750141 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4814101420 msgr2=0x7f4814198a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:56.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.080+0000 7f481a7bd640 1 --2- 172.21.15.5:0/3756750141 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4814101420 0x7f4814198a80 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f480800b450 tx=0x7f480800b910 comp rx=0 tx=0).stop 2024-09-12T02:57:56.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.080+0000 7f481a7bd640 1 -- 172.21.15.5:0/3756750141 shutdown_connections 2024-09-12T02:57:56.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.080+0000 7f481a7bd640 1 --2- 172.21.15.5:0/3756750141 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f47fc0a2c10 0x7f47fc0a50d0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:56.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.080+0000 7f481a7bd640 1 --2- 172.21.15.5:0/3756750141 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4814101420 0x7f4814198a80 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:56.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.080+0000 7f481a7bd640 1 --2- 172.21.15.5:0/3756750141 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4814106a80 0x7f481419ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:56.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.080+0000 7f481a7bd640 1 --2- 172.21.15.5:0/3756750141 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4814100a80 0x7f4814198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:56.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.080+0000 7f481a7bd640 1 -- 172.21.15.5:0/3756750141 >> 172.21.15.5:0/3756750141 conn(0x7f48140fc7a0 msgr2=0x7f48140fd070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:56.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.080+0000 7f481a7bd640 1 -- 172.21.15.5:0/3756750141 shutdown_connections 2024-09-12T02:57:56.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:56.080+0000 7f481a7bd640 1 -- 172.21.15.5:0/3756750141 wait complete. 2024-09-12T02:57:56.863 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.i status 2024-09-12T02:57:57.121 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:57:57.538 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:57:57 smithi005 ceph-mon[31197]: pgmap v326: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-12T02:57:57.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.546+0000 7f43c19dc640 1 -- 172.21.15.5:0/3865541682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f43bc100e20 msgr2=0x7f43bc101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:57.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.546+0000 7f43c19dc640 1 --2- 172.21.15.5:0/3865541682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f43bc100e20 0x7f43bc101200 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f43ac009900 tx=0x7f43ac02a480 comp rx=0 tx=0).stop 2024-09-12T02:57:57.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.546+0000 7f43c19dc640 1 -- 172.21.15.5:0/3865541682 shutdown_connections 2024-09-12T02:57:57.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.546+0000 7f43c19dc640 1 --2- 172.21.15.5:0/3865541682 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43bc102ef0 0x7f43bc10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:57.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.546+0000 7f43c19dc640 1 --2- 172.21.15.5:0/3865541682 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f43bc102570 0x7f43bc1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:57.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.546+0000 7f43c19dc640 1 --2- 172.21.15.5:0/3865541682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f43bc100e20 0x7f43bc101200 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:57.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.546+0000 7f43c19dc640 1 -- 172.21.15.5:0/3865541682 >> 172.21.15.5:0/3865541682 conn(0x7f43bc0fc7d0 msgr2=0x7f43bc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:57.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.546+0000 7f43c19dc640 1 -- 172.21.15.5:0/3865541682 shutdown_connections 2024-09-12T02:57:57.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.546+0000 7f43c19dc640 1 -- 172.21.15.5:0/3865541682 wait complete. 2024-09-12T02:57:57.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.547+0000 7f43c19dc640 1 Processor -- start 2024-09-12T02:57:57.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.547+0000 7f43c19dc640 1 -- start start 2024-09-12T02:57:57.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.547+0000 7f43c19dc640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f43bc100e20 0x7f43bc198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:57.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.547+0000 7f43c19dc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43bc102570 0x7f43bc198bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:57.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.547+0000 7f43c19dc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f43bc102ef0 0x7f43bc19cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:57.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.547+0000 7f43c19dc640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f43bc111020 con 0x7f43bc102ef0 2024-09-12T02:57:57.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.547+0000 7f43c19dc640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f43bc110ea0 con 0x7f43bc100e20 2024-09-12T02:57:57.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.547+0000 7f43c19dc640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f43bc1111a0 con 0x7f43bc102570 2024-09-12T02:57:57.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.547+0000 7f43bb7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f43bc102ef0 0x7f43bc19cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:57.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.547+0000 7f43bb7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f43bc102ef0 0x7f43bc19cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60322/0 (socket says 172.21.15.5:60322) 2024-09-12T02:57:57.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.547+0000 7f43bb7fe640 1 -- 172.21.15.5:0/1662758212 learned_addr learned my addr 172.21.15.5:0/1662758212 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:57:57.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.548+0000 7f43bb7fe640 1 -- 172.21.15.5:0/1662758212 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43bc102570 msgr2=0x7f43bc198bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:57.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.548+0000 7f43bb7fe640 1 --2- 172.21.15.5:0/1662758212 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43bc102570 0x7f43bc198bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:57.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.548+0000 7f43bb7fe640 1 -- 172.21.15.5:0/1662758212 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f43bc100e20 msgr2=0x7f43bc198670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:57.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.548+0000 7f43bb7fe640 1 --2- 172.21.15.5:0/1662758212 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f43bc100e20 0x7f43bc198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:57.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.548+0000 7f43bb7fe640 1 -- 172.21.15.5:0/1662758212 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43bc19d620 con 0x7f43bc102ef0 2024-09-12T02:57:57.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.548+0000 7f43bb7fe640 1 --2- 172.21.15.5:0/1662758212 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f43bc102ef0 0x7f43bc19cf40 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f43b400b970 tx=0x7f43b400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:57.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.548+0000 7f43c09da640 1 -- 172.21.15.5:0/1662758212 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b4019070 con 0x7f43bc102ef0 2024-09-12T02:57:57.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.548+0000 7f43c09da640 1 -- 172.21.15.5:0/1662758212 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43b400cb40 con 0x7f43bc102ef0 2024-09-12T02:57:57.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.548+0000 7f43c19dc640 1 -- 172.21.15.5:0/1662758212 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43bc19d910 con 0x7f43bc102ef0 2024-09-12T02:57:57.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.548+0000 7f43c09da640 1 -- 172.21.15.5:0/1662758212 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b4007650 con 0x7f43bc102ef0 2024-09-12T02:57:57.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.548+0000 7f43c19dc640 1 -- 172.21.15.5:0/1662758212 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43bc1a51f0 con 0x7f43bc102ef0 2024-09-12T02:57:57.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.550+0000 7f43c09da640 1 -- 172.21.15.5:0/1662758212 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7f43b401c070 con 0x7f43bc102ef0 2024-09-12T02:57:57.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.550+0000 7f43c3db5740 1 Processor -- start 2024-09-12T02:57:57.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.551+0000 7f43c3db5740 1 -- start start 2024-09-12T02:57:57.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.551+0000 7f43c3db5740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561e29c279d0 0x561e29c27db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:57.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.551+0000 7f43c3db5740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561e29c282f0 0x561e29c32670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:57.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.551+0000 7f43c3db5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e29c32bb0 0x561e29c34fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:57.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.551+0000 7f43c3db5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561e29b8e090 con 0x561e29c32bb0 2024-09-12T02:57:57.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.551+0000 7f43c3db5740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x561e29b35a70 con 0x561e29c282f0 2024-09-12T02:57:57.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.551+0000 7f43c3db5740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x561e29b8a050 con 0x561e29c279d0 2024-09-12T02:57:57.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.551+0000 7f43b3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e29c32bb0 0x561e29c34fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:57.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.551+0000 7f43b3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e29c32bb0 0x561e29c34fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60326/0 (socket says 172.21.15.5:60326) 2024-09-12T02:57:57.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.551+0000 7f43b3fff640 1 -- 172.21.15.5:0/739823569 learned_addr learned my addr 172.21.15.5:0/739823569 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:57:57.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.551+0000 7f43baffd640 1 --2- 172.21.15.5:0/739823569 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561e29c282f0 0x561e29c32670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:57.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.551+0000 7f43b3fff640 1 --2- 172.21.15.5:0/739823569 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561e29c279d0 0x561e29c27db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:57.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.551+0000 7f43b3fff640 1 -- 172.21.15.5:0/739823569 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561e29c279d0 msgr2=0x561e29c27db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:57:57.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.551+0000 7f43b3fff640 1 --2- 172.21.15.5:0/739823569 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561e29c279d0 0x561e29c27db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:57.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.551+0000 7f43b3fff640 1 -- 172.21.15.5:0/739823569 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561e29c282f0 msgr2=0x561e29c32670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:57:57.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.551+0000 7f43b3fff640 1 --2- 172.21.15.5:0/739823569 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561e29c282f0 0x561e29c32670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:57.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.551+0000 7f43c09da640 1 --2- 172.21.15.5:0/1662758212 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f43a80a2c90 0x7f43a80a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:57.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.551+0000 7f43b3fff640 1 -- 172.21.15.5:0/739823569 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e29b42130 con 0x561e29c32bb0 2024-09-12T02:57:57.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.552+0000 7f43c09da640 1 -- 172.21.15.5:0/1662758212 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f43b40ce6e0 con 0x7f43bc102ef0 2024-09-12T02:57:57.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.552+0000 7f43b3fff640 1 --2- 172.21.15.5:0/739823569 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e29c32bb0 0x561e29c34fa0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f43a4001a80 tx=0x7f43a4009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22ab919f511d1bd2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:57.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.552+0000 7f43b3fff640 1 --2- 172.21.15.5:0/1662758212 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f43a80a2c90 0x7f43a80a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:57.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.552+0000 7f43b1ffb640 1 -- 172.21.15.5:0/739823569 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a400eaf0 con 0x561e29c32bb0 2024-09-12T02:57:57.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.552+0000 7f43b1ffb640 1 -- 172.21.15.5:0/739823569 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43a40070f0 con 0x561e29c32bb0 2024-09-12T02:57:57.555 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.552+0000 7f43b1ffb640 1 -- 172.21.15.5:0/739823569 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a4009040 con 0x561e29c32bb0 2024-09-12T02:57:57.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.553+0000 7f43c3db5740 1 -- 172.21.15.5:0/739823569 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e29c32bb0 msgr2=0x561e29c34fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:57.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.553+0000 7f43b3fff640 1 --2- 172.21.15.5:0/1662758212 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f43a80a2c90 0x7f43a80a5150 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f43a4018000 tx=0x7f43a4009770 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:57.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.553+0000 7f43c3db5740 1 --2- 172.21.15.5:0/739823569 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e29c32bb0 0x561e29c34fa0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f43a4001a80 tx=0x7f43a4009f90 comp rx=0 tx=0).stop 2024-09-12T02:57:57.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.553+0000 7f43c3db5740 1 -- 172.21.15.5:0/739823569 shutdown_connections 2024-09-12T02:57:57.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.553+0000 7f43c3db5740 1 --2- 172.21.15.5:0/739823569 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e29c32bb0 0x561e29c34fa0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:57.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.553+0000 7f43c3db5740 1 --2- 172.21.15.5:0/739823569 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561e29c282f0 0x561e29c32670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:57.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.553+0000 7f43c3db5740 1 --2- 172.21.15.5:0/739823569 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561e29c279d0 0x561e29c27db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:57.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.553+0000 7f43c3db5740 1 -- 172.21.15.5:0/739823569 >> 172.21.15.5:0/739823569 conn(0x561e29c23f60 msgr2=0x561e29c24340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:57.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.553+0000 7f43c3db5740 1 -- 172.21.15.5:0/739823569 shutdown_connections 2024-09-12T02:57:57.556 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.553+0000 7f43c3db5740 1 -- 172.21.15.5:0/739823569 wait complete. 2024-09-12T02:57:57.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.553+0000 7f43c3db5740 1 -- start start 2024-09-12T02:57:57.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.553+0000 7f43c3db5740 1 Processor -- start 2024-09-12T02:57:57.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.554+0000 7f43c3db5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e29c279d0 0x561e29c323b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:57.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.554+0000 7f43c3db5740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561e29c282f0 0x561e29c2f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:57.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.554+0000 7f43c3db5740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561e29c32bb0 0x561e29c2fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:57.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.554+0000 7f43c3db5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561e29c35780 con 0x561e29c279d0 2024-09-12T02:57:57.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.554+0000 7f43c3db5740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x561e29b48f20 con 0x561e29c32bb0 2024-09-12T02:57:57.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.554+0000 7f43c3db5740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x561e29c35900 con 0x561e29c282f0 2024-09-12T02:57:57.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.554+0000 7f43c3db5740 10 client.0 _ll_register_callbacks cb 0x561e29b12030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:57:57.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.554+0000 7f43baffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e29c279d0 0x561e29c323b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:57.561 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.554+0000 7f43baffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e29c279d0 0x561e29c323b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60328/0 (socket says 172.21.15.5:60328) 2024-09-12T02:57:57.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.554+0000 7f43baffd640 1 -- 172.21.15.5:0/3673300314 learned_addr learned my addr 172.21.15.5:0/3673300314 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:57:57.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.554+0000 7f43bb7fe640 1 --2- 172.21.15.5:0/3673300314 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561e29c32bb0 0x561e29c2fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:57.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.554+0000 7f43baffd640 1 -- 172.21.15.5:0/3673300314 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561e29c282f0 msgr2=0x561e29c2f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:57:57.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.554+0000 7f43b3fff640 1 --2- 172.21.15.5:0/3673300314 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561e29c282f0 0x561e29c2f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:57.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.554+0000 7f43baffd640 1 --2- 172.21.15.5:0/3673300314 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561e29c282f0 0x561e29c2f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:57.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.554+0000 7f43baffd640 1 -- 172.21.15.5:0/3673300314 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561e29c32bb0 msgr2=0x561e29c2fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:57:57.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.554+0000 7f43baffd640 1 --2- 172.21.15.5:0/3673300314 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561e29c32bb0 0x561e29c2fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:57.562 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.554+0000 7f43baffd640 1 -- 172.21.15.5:0/3673300314 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e298ff2b0 con 0x561e29c279d0 2024-09-12T02:57:57.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.554+0000 7f43bb7fe640 1 --2- 172.21.15.5:0/3673300314 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561e29c32bb0 0x561e29c2fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:57:57.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.554+0000 7f43b3fff640 1 --2- 172.21.15.5:0/3673300314 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561e29c282f0 0x561e29c2f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:57:57.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.554+0000 7f43baffd640 1 --2- 172.21.15.5:0/3673300314 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e29c279d0 0x561e29c323b0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f43ac033000 tx=0x7f43ac0043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=223365302fa37641 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:57.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.554+0000 7f439affd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T02:57:57.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.555+0000 7f439affd640 1 -- 172.21.15.5:0/3673300314 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43ac040070 con 0x561e29c279d0 2024-09-12T02:57:57.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.555+0000 7f439affd640 1 -- 172.21.15.5:0/3673300314 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f43ac003a00 con 0x561e29c279d0 2024-09-12T02:57:57.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.555+0000 7f43c3db5740 1 -- 172.21.15.5:0/3673300314 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43a40070f0 con 0x561e29c279d0 2024-09-12T02:57:57.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.555+0000 7f439affd640 1 -- 172.21.15.5:0/3673300314 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f43ac035040 con 0x561e29c279d0 2024-09-12T02:57:57.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.555+0000 7f43c3db5740 10 client.15189 fetch_fsmap learned FSMap version 25 2024-09-12T02:57:57.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.555+0000 7f43c3db5740 1 -- 172.21.15.5:0/3673300314 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561e29c41040 con 0x561e29c279d0 2024-09-12T02:57:57.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.555+0000 7f439affd640 1 -- 172.21.15.5:0/3673300314 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43a40070f0 con 0x561e29c279d0 2024-09-12T02:57:57.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.555+0000 7f439affd640 1 -- 172.21.15.5:0/3673300314 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8362+0+0 (secure 0 0 0) 0x7f43ac045460 con 0x561e29c279d0 2024-09-12T02:57:57.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.555+0000 7f43c3db5740 10 client.15189 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:57:57.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.555+0000 7f43c3db5740 10 client.15189 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-12T02:57:57.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.555+0000 7f43c3db5740 20 client.15189 populate_metadata read hostname 'smithi005' 2024-09-12T02:57:57.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.555+0000 7f43c3db5740 1 --2- 172.21.15.5:0/3673300314 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x561e29c43750 0x561e29c44bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:57.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.555+0000 7f43c3db5740 4 client.15189 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:57:57.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.555+0000 7f43c3db5740 1 -- 172.21.15.5:0/3673300314 --> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561e29b9b600 con 0x561e29c43750 2024-09-12T02:57:57.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.556+0000 7f43b3fff640 1 --2- 172.21.15.5:0/3673300314 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x561e29c43750 0x561e29c44bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:57.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.556+0000 7f43b3fff640 1 --2- 172.21.15.5:0/3673300314 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x561e29c43750 0x561e29c44bc0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1cd85df28e9ee07c server_cookie=1388b43b99bee163 in_seq=0 out_seq=0 2024-09-12T02:57:57.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.556+0000 7f439affd640 10 client.15189 ms_handle_connect on v2:172.21.15.148:6838/2317600000 2024-09-12T02:57:57.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.557+0000 7f439affd640 1 -- 172.21.15.5:0/3673300314 <== mds.0 v2:172.21.15.148:6838/2317600000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561e29b9b600 con 0x561e29c43750 2024-09-12T02:57:57.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.557+0000 7f439affd640 10 client.15189 handle_command_reply: tid=0 2024-09-12T02:57:57.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.557+0000 7f43c3db5740 1 client.15189 shutdown 2024-09-12T02:57:57.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.557+0000 7f43c3db5740 1 -- 172.21.15.5:0/3673300314 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e29c279d0 msgr2=0x561e29c323b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:57.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.557+0000 7f43c3db5740 1 --2- 172.21.15.5:0/3673300314 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e29c279d0 0x561e29c323b0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f43ac033000 tx=0x7f43ac0043b0 comp rx=0 tx=0).stop 2024-09-12T02:57:57.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.557+0000 7f43c3db5740 1 -- 172.21.15.5:0/3673300314 shutdown_connections 2024-09-12T02:57:57.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.557+0000 7f43c3db5740 1 --2- 172.21.15.5:0/3673300314 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561e29c32bb0 0x561e29c2fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:57.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.557+0000 7f43c3db5740 1 --2- 172.21.15.5:0/3673300314 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561e29c282f0 0x561e29c2f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:57.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.557+0000 7f43c3db5740 1 --2- 172.21.15.5:0/3673300314 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x561e29c43750 0x561e29c44bc0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:57.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.557+0000 7f43c3db5740 1 --2- 172.21.15.5:0/3673300314 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e29c279d0 0x561e29c323b0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:57.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.557+0000 7f43c3db5740 1 -- 172.21.15.5:0/3673300314 >> 172.21.15.5:0/3673300314 conn(0x561e29c23f60 msgr2=0x561e29c29c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:57.565 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.557+0000 7f439affd640 0 client.15189 ms_handle_reset on v2:172.21.15.148:6838/2317600000 2024-09-12T02:57:57.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.557+0000 7f43c3db5740 1 -- 172.21.15.5:0/3673300314 shutdown_connections 2024-09-12T02:57:57.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.557+0000 7f43c3db5740 1 -- 172.21.15.5:0/3673300314 wait complete. 2024-09-12T02:57:57.566 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.557+0000 7f43c3db5740 20 client.15189 trim_cache size 0 max 16384 2024-09-12T02:57:57.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.575+0000 7f43c3db5740 1 Processor -- start 2024-09-12T02:57:57.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.575+0000 7f43c3db5740 1 -- start start 2024-09-12T02:57:57.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.575+0000 7f43c3db5740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561e29c279d0 0x561e29c43080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:57.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.575+0000 7f43c3db5740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561e29c282f0 0x561e29c44bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:57.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.575+0000 7f43c3db5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e29c32bb0 0x561e29c4cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:57.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.575+0000 7f43c3db5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561e29c42f00 con 0x561e29c32bb0 2024-09-12T02:57:57.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.575+0000 7f43c3db5740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x561e29c435d0 con 0x561e29c279d0 2024-09-12T02:57:57.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.575+0000 7f43c3db5740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x561e29c43ca0 con 0x561e29c282f0 2024-09-12T02:57:57.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.575+0000 7f43bb7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e29c32bb0 0x561e29c4cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:57.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.575+0000 7f43bb7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e29c32bb0 0x561e29c4cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60330/0 (socket says 172.21.15.5:60330) 2024-09-12T02:57:57.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.575+0000 7f43bb7fe640 1 -- 172.21.15.5:0/2844348356 learned_addr learned my addr 172.21.15.5:0/2844348356 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:57:57.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.575+0000 7f43baffd640 1 --2- 172.21.15.5:0/2844348356 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561e29c279d0 0x561e29c43080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:57.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.576+0000 7f43bb7fe640 1 -- 172.21.15.5:0/2844348356 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561e29c282f0 msgr2=0x561e29c44bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:57:57.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.576+0000 7f43b3fff640 1 --2- 172.21.15.5:0/2844348356 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561e29c282f0 0x561e29c44bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:57.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.576+0000 7f43bb7fe640 1 --2- 172.21.15.5:0/2844348356 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561e29c282f0 0x561e29c44bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:57.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.576+0000 7f43bb7fe640 1 -- 172.21.15.5:0/2844348356 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561e29c279d0 msgr2=0x561e29c43080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:57:57.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.576+0000 7f43bb7fe640 1 --2- 172.21.15.5:0/2844348356 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561e29c279d0 0x561e29c43080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:57.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.576+0000 7f43bb7fe640 1 -- 172.21.15.5:0/2844348356 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e29c32720 con 0x561e29c32bb0 2024-09-12T02:57:57.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.576+0000 7f43bb7fe640 1 --2- 172.21.15.5:0/2844348356 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e29c32bb0 0x561e29c4cea0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f43b40a3640 tx=0x7f43b409ab70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8198880403b3ada2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:57.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.576+0000 7f43b17fa640 1 -- 172.21.15.5:0/2844348356 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b409f2b0 con 0x561e29c32bb0 2024-09-12T02:57:57.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.576+0000 7f43b17fa640 1 -- 172.21.15.5:0/2844348356 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43b40a3b70 con 0x561e29c32bb0 2024-09-12T02:57:57.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.576+0000 7f43b17fa640 1 -- 172.21.15.5:0/2844348356 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b40a3960 con 0x561e29c32bb0 2024-09-12T02:57:57.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.577+0000 7f43c3db5740 1 -- 172.21.15.5:0/2844348356 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e29c32bb0 msgr2=0x561e29c4cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:57.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.577+0000 7f43c3db5740 1 --2- 172.21.15.5:0/2844348356 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e29c32bb0 0x561e29c4cea0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f43b40a3640 tx=0x7f43b409ab70 comp rx=0 tx=0).stop 2024-09-12T02:57:57.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.577+0000 7f43c3db5740 1 -- 172.21.15.5:0/2844348356 shutdown_connections 2024-09-12T02:57:57.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.577+0000 7f43c3db5740 1 --2- 172.21.15.5:0/2844348356 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e29c32bb0 0x561e29c4cea0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:57.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.577+0000 7f43c3db5740 1 --2- 172.21.15.5:0/2844348356 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561e29c282f0 0x561e29c44bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:57.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.577+0000 7f43c3db5740 1 --2- 172.21.15.5:0/2844348356 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561e29c279d0 0x561e29c43080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:57.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.577+0000 7f43c3db5740 1 -- 172.21.15.5:0/2844348356 >> 172.21.15.5:0/2844348356 conn(0x561e29c23f60 msgr2=0x561e29c1ece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:57.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.577+0000 7f43c3db5740 1 -- 172.21.15.5:0/2844348356 shutdown_connections 2024-09-12T02:57:57.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.577+0000 7f43c3db5740 1 -- 172.21.15.5:0/2844348356 wait complete. 2024-09-12T02:57:57.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.577+0000 7f43c3db5740 1 -- start start 2024-09-12T02:57:57.580 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.577+0000 7f43c3db5740 1 Processor -- start 2024-09-12T02:57:57.581 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.577+0000 7f43c3db5740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561e29c279d0 0x561e29c47e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:57.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.577+0000 7f43c3db5740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e29c282f0 0x561e29c48370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:57.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.577+0000 7f43c3db5740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561e29c32bb0 0x561e29c488b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:57.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.578+0000 7f43c3db5740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561e29c44a40 con 0x561e29c282f0 2024-09-12T02:57:57.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.578+0000 7f43c3db5740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x561e29c44370 con 0x561e29c279d0 2024-09-12T02:57:57.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.578+0000 7f43c3db5740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x561e29c45110 con 0x561e29c32bb0 2024-09-12T02:57:57.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.578+0000 7f43b3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e29c282f0 0x561e29c48370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:57.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.578+0000 7f43b3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e29c282f0 0x561e29c48370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60334/0 (socket says 172.21.15.5:60334) 2024-09-12T02:57:57.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.578+0000 7f43c3db5740 10 client.0 _ll_register_callbacks cb 0x7f43ac004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:57:57.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.578+0000 7f43b3fff640 1 -- 172.21.15.5:0/889573308 learned_addr learned my addr 172.21.15.5:0/889573308 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:57:57.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.578+0000 7f43b3fff640 1 -- 172.21.15.5:0/889573308 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561e29c32bb0 msgr2=0x561e29c488b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T02:57:57.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.578+0000 7f43b3fff640 1 --2- 172.21.15.5:0/889573308 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561e29c32bb0 0x561e29c488b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:57.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.578+0000 7f43b3fff640 1 -- 172.21.15.5:0/889573308 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561e29c279d0 msgr2=0x561e29c47e30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T02:57:57.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.578+0000 7f43b3fff640 1 --2- 172.21.15.5:0/889573308 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561e29c279d0 0x561e29c47e30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:57.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.578+0000 7f43b3fff640 1 -- 172.21.15.5:0/889573308 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e29cac0a0 con 0x561e29c282f0 2024-09-12T02:57:57.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.578+0000 7f43b3fff640 1 --2- 172.21.15.5:0/889573308 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e29c282f0 0x561e29c48370 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f43a4009b80 tx=0x7f43a400b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2510a13a394ff215 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:57.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.578+0000 7f439affd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T02:57:57.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.578+0000 7f439affd640 1 -- 172.21.15.5:0/889573308 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a4009db0 con 0x561e29c282f0 2024-09-12T02:57:57.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.578+0000 7f439affd640 1 -- 172.21.15.5:0/889573308 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f43a4012040 con 0x561e29c282f0 2024-09-12T02:57:57.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.578+0000 7f43c3db5740 1 -- 172.21.15.5:0/889573308 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43b40a3b70 con 0x561e29c282f0 2024-09-12T02:57:57.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.578+0000 7f439affd640 1 -- 172.21.15.5:0/889573308 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f43a4009040 con 0x561e29c282f0 2024-09-12T02:57:57.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.578+0000 7f43c3db5740 10 client.15195 fetch_fsmap learned FSMap version 25 2024-09-12T02:57:57.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.578+0000 7f43c3db5740 1 -- 172.21.15.5:0/889573308 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561e29cad3a0 con 0x561e29c282f0 2024-09-12T02:57:57.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.579+0000 7f439affd640 1 -- 172.21.15.5:0/889573308 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43b40a3b70 con 0x561e29c282f0 2024-09-12T02:57:57.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.579+0000 7f439affd640 1 -- 172.21.15.5:0/889573308 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) ==== 8362+0+0 (secure 0 0 0) 0x7f43a4019cb0 con 0x561e29c282f0 2024-09-12T02:57:57.583 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.579+0000 7f43c3db5740 10 client.15195 fetch_fsmap finished waiting for FSMap version 25 2024-09-12T02:57:57.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.579+0000 7f43c3db5740 10 client.15195 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-12T02:57:57.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.579+0000 7f43c3db5740 20 client.15195 populate_metadata read hostname 'smithi005' 2024-09-12T02:57:57.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.579+0000 7f43c3db5740 1 --2- 172.21.15.5:0/889573308 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x561e29cb1ec0 0x561e29cb0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:57.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.579+0000 7f43c3db5740 4 client.15195 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T02:57:57.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.579+0000 7f43c3db5740 1 -- 172.21.15.5:0/889573308 --> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] -- command(tid 0: {"prefix": "status"}) -- 0x561e29b95b10 con 0x561e29cb1ec0 2024-09-12T02:57:57.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.581+0000 7f43baffd640 1 --2- 172.21.15.5:0/889573308 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x561e29cb1ec0 0x561e29cb0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:57.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.585+0000 7f43baffd640 1 --2- 172.21.15.5:0/889573308 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x561e29cb1ec0 0x561e29cb0bd0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=156c52171f51d746 server_cookie=d9fa5db220551ee in_seq=0 out_seq=0 2024-09-12T02:57:57.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.585+0000 7f439affd640 10 client.15195 ms_handle_connect on v2:172.21.15.148:6838/2317600000 2024-09-12T02:57:57.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.586+0000 7f439affd640 1 -- 172.21.15.5:0/889573308 <== mds.0 v2:172.21.15.148:6838/2317600000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+321 (crc 0 0 0) 0x561e29b95b10 con 0x561e29cb1ec0 2024-09-12T02:57:57.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.586+0000 7f439affd640 10 client.15195 handle_command_reply: tid=0 2024-09-12T02:57:57.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.586+0000 7f43c3db5740 1 client.15195 shutdown 2024-09-12T02:57:57.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.586+0000 7f43c3db5740 1 -- 172.21.15.5:0/889573308 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e29c282f0 msgr2=0x561e29c48370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:57.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.586+0000 7f43c3db5740 1 --2- 172.21.15.5:0/889573308 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e29c282f0 0x561e29c48370 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f43a4009b80 tx=0x7f43a400b000 comp rx=0 tx=0).stop 2024-09-12T02:57:57.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.586+0000 7f43c3db5740 1 -- 172.21.15.5:0/889573308 shutdown_connections 2024-09-12T02:57:57.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.586+0000 7f43c3db5740 1 --2- 172.21.15.5:0/889573308 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x561e29cb1ec0 0x561e29cb0bd0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:57.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.586+0000 7f43c3db5740 1 --2- 172.21.15.5:0/889573308 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e29c282f0 0x561e29c48370 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:57.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.586+0000 7f43c3db5740 1 --2- 172.21.15.5:0/889573308 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561e29c32bb0 0x561e29c488b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:57.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.586+0000 7f43c3db5740 1 --2- 172.21.15.5:0/889573308 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561e29c279d0 0x561e29c47e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:57.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.586+0000 7f43c3db5740 1 -- 172.21.15.5:0/889573308 >> 172.21.15.5:0/889573308 conn(0x561e29c23f60 msgr2=0x561e29c2efc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:57.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.586+0000 7f439affd640 0 client.15195 ms_handle_reset on v2:172.21.15.148:6838/2317600000 2024-09-12T02:57:57.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.586+0000 7f43c3db5740 1 -- 172.21.15.5:0/889573308 shutdown_connections 2024-09-12T02:57:57.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.586+0000 7f43c3db5740 1 -- 172.21.15.5:0/889573308 wait complete. 2024-09-12T02:57:57.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.586+0000 7f43c3db5740 20 client.15195 trim_cache size 0 max 16384 2024-09-12T02:57:57.588 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-12T02:57:57.588 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "618305f8-70b1-11ef-bcea-c7b262605968", 2024-09-12T02:57:57.588 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 0, 2024-09-12T02:57:57.588 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24502, 2024-09-12T02:57:57.588 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-09-12T02:57:57.588 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-09-12T02:57:57.588 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-09-12T02:57:57.588 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 100.61431256, 2024-09-12T02:57:57.588 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 25, 2024-09-12T02:57:57.589 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 83, 2024-09-12T02:57:57.589 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 81, 2024-09-12T02:57:57.589 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 119.920007799 2024-09-12T02:57:57.589 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-12T02:57:57.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.587+0000 7f43c19dc640 1 -- 172.21.15.5:0/1662758212 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f43a80a2c90 msgr2=0x7f43a80a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:57.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.587+0000 7f43c19dc640 1 --2- 172.21.15.5:0/1662758212 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f43a80a2c90 0x7f43a80a5150 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f43a4018000 tx=0x7f43a4009770 comp rx=0 tx=0).stop 2024-09-12T02:57:57.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.587+0000 7f43c19dc640 1 -- 172.21.15.5:0/1662758212 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f43bc102ef0 msgr2=0x7f43bc19cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:57.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.587+0000 7f43c19dc640 1 --2- 172.21.15.5:0/1662758212 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f43bc102ef0 0x7f43bc19cf40 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f43b400b970 tx=0x7f43b400be30 comp rx=0 tx=0).stop 2024-09-12T02:57:57.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.587+0000 7f43c19dc640 1 -- 172.21.15.5:0/1662758212 shutdown_connections 2024-09-12T02:57:57.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.587+0000 7f43c19dc640 1 --2- 172.21.15.5:0/1662758212 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f43a80a2c90 0x7f43a80a5150 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:57.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.587+0000 7f43c19dc640 1 --2- 172.21.15.5:0/1662758212 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f43bc102ef0 0x7f43bc19cf40 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:57.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.587+0000 7f43c19dc640 1 --2- 172.21.15.5:0/1662758212 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f43bc102570 0x7f43bc198bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:57.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.587+0000 7f43c19dc640 1 --2- 172.21.15.5:0/1662758212 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f43bc100e20 0x7f43bc198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:57.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.587+0000 7f43c19dc640 1 -- 172.21.15.5:0/1662758212 >> 172.21.15.5:0/1662758212 conn(0x7f43bc0fc7d0 msgr2=0x7f43bc06a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:57.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.587+0000 7f43c19dc640 1 -- 172.21.15.5:0/1662758212 shutdown_connections 2024-09-12T02:57:57.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:57.587+0000 7f43c19dc640 1 -- 172.21.15.5:0/1662758212 wait complete. 2024-09-12T02:57:57.665 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:57:57 smithi148 ceph-mon[35090]: pgmap v326: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-12T02:57:57.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:57:57 smithi146 ceph-mon[35011]: pgmap v326: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-12T02:57:58.366 INFO:teuthology.run_tasks:Running task sequential... 2024-09-12T02:57:58.381 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-09-12T02:57:58.381 INFO:teuthology.task.sequential:In sequential, running task print... 2024-09-12T02:57:58.382 INFO:teuthology.task.print:Enabling mgr modules 2024-09-12T02:57:58.382 INFO:teuthology.run_tasks:Running task kclient... 2024-09-12T02:57:58.454 INFO:tasks.kclient:Mounting kernel clients... 2024-09-12T02:57:58.456 INFO:tasks.kclient:config is {'client.0': {}} 2024-09-12T02:57:58.456 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T02:57:58.456 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list 2024-09-12T02:57:58.473 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T02:57:58.473 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link delete ceph-brx 2024-09-12T02:57:58.542 INFO:teuthology.orchestra.run.smithi005.stderr:Cannot find device "ceph-brx" 2024-09-12T02:57:58.543 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T02:57:58.544 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=secure', 'wsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2024-09-12T02:57:58.544 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-12T02:57:58.544 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-12T02:57:58.545 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph fs ls 2024-09-12T02:57:58.840 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:57:59.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:59.321+0000 7fa524a8a640 1 -- 172.21.15.5:0/2087956451 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa520100a80 msgr2=0x7fa520100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:59.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:59.321+0000 7fa524a8a640 1 --2- 172.21.15.5:0/2087956451 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa520100a80 0x7fa520100ee0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fa508009930 tx=0x7fa50802a420 comp rx=0 tx=0).stop 2024-09-12T02:57:59.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:59.321+0000 7fa524a8a640 1 -- 172.21.15.5:0/2087956451 shutdown_connections 2024-09-12T02:57:59.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:59.321+0000 7fa524a8a640 1 --2- 172.21.15.5:0/2087956451 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa520100a80 0x7fa520100ee0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:59.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:59.321+0000 7fa524a8a640 1 --2- 172.21.15.5:0/2087956451 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa520101420 0x7fa52010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:59.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:59.321+0000 7fa524a8a640 1 --2- 172.21.15.5:0/2087956451 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa520106a80 0x7fa520106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:59.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:59.321+0000 7fa524a8a640 1 -- 172.21.15.5:0/2087956451 >> 172.21.15.5:0/2087956451 conn(0x7fa5200fc7a0 msgr2=0x7fa5200febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:59.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:59.321+0000 7fa524a8a640 1 -- 172.21.15.5:0/2087956451 shutdown_connections 2024-09-12T02:57:59.325 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:59.321+0000 7fa524a8a640 1 -- 172.21.15.5:0/2087956451 wait complete. 2024-09-12T02:57:59.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:59.322+0000 7fa524a8a640 1 Processor -- start 2024-09-12T02:57:59.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:59.322+0000 7fa524a8a640 1 -- start start 2024-09-12T02:57:59.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:59.322+0000 7fa524a8a640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa520100a80 0x7fa5201992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:59.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:59.322+0000 7fa524a8a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa520101420 0x7fa5201997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:59.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:59.322+0000 7fa524a8a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa520106a80 0x7fa5201a4320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:59.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:59.322+0000 7fa524a8a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa520110e80 con 0x7fa520101420 2024-09-12T02:57:59.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:59.322+0000 7fa524a8a640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fa520110d00 con 0x7fa520100a80 2024-09-12T02:57:59.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:59.322+0000 7fa524a8a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa520111000 con 0x7fa520106a80 2024-09-12T02:57:59.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:59.323+0000 7fa517fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa520101420 0x7fa5201997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:59.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:59.323+0000 7fa517fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa520101420 0x7fa5201997e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60348/0 (socket says 172.21.15.5:60348) 2024-09-12T02:57:59.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:59.323+0000 7fa517fff640 1 -- 172.21.15.5:0/3334984726 learned_addr learned my addr 172.21.15.5:0/3334984726 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:57:59.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:59.323+0000 7fa51e59c640 1 --2- 172.21.15.5:0/3334984726 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa520100a80 0x7fa5201992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:59.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:59.323+0000 7fa51ed9d640 1 --2- 172.21.15.5:0/3334984726 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa520106a80 0x7fa5201a4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:59.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:59.323+0000 7fa517fff640 1 -- 172.21.15.5:0/3334984726 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa520106a80 msgr2=0x7fa5201a4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:59.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:59.323+0000 7fa517fff640 1 --2- 172.21.15.5:0/3334984726 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa520106a80 0x7fa5201a4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:59.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:59.323+0000 7fa517fff640 1 -- 172.21.15.5:0/3334984726 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa520100a80 msgr2=0x7fa5201992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:59.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:59.323+0000 7fa517fff640 1 --2- 172.21.15.5:0/3334984726 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa520100a80 0x7fa5201992a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:59.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:59.323+0000 7fa517fff640 1 -- 172.21.15.5:0/3334984726 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5201a4860 con 0x7fa520101420 2024-09-12T02:57:59.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:59.323+0000 7fa517fff640 1 --2- 172.21.15.5:0/3334984726 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa520101420 0x7fa5201997e0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fa508009230 tx=0x7fa508004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:59.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:59.323+0000 7fa5177fe640 1 -- 172.21.15.5:0/3334984726 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa50803f030 con 0x7fa520101420 2024-09-12T02:57:59.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:59.323+0000 7fa5177fe640 1 -- 172.21.15.5:0/3334984726 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa508004440 con 0x7fa520101420 2024-09-12T02:57:59.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:59.323+0000 7fa524a8a640 1 -- 172.21.15.5:0/3334984726 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5201a4a30 con 0x7fa520101420 2024-09-12T02:57:59.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:59.323+0000 7fa5177fe640 1 -- 172.21.15.5:0/3334984726 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa508035070 con 0x7fa520101420 2024-09-12T02:57:59.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:59.323+0000 7fa524a8a640 1 -- 172.21.15.5:0/3334984726 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5201a4e80 con 0x7fa520101420 2024-09-12T02:57:59.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:59.324+0000 7fa524a8a640 1 -- 172.21.15.5:0/3334984726 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa520100ee0 con 0x7fa520101420 2024-09-12T02:57:59.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:59.325+0000 7fa5177fe640 1 -- 172.21.15.5:0/3334984726 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) ==== 137883+0+0 (secure 0 0 0) 0x7fa508042030 con 0x7fa520101420 2024-09-12T02:57:59.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:59.326+0000 7fa5177fe640 1 --2- 172.21.15.5:0/3334984726 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fa50c0a2c10 0x7fa50c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:57:59.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:59.326+0000 7fa51e59c640 1 --2- 172.21.15.5:0/3334984726 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fa50c0a2c10 0x7fa50c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:57:59.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:59.326+0000 7fa5177fe640 1 -- 172.21.15.5:0/3334984726 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fa5080ef500 con 0x7fa520101420 2024-09-12T02:57:59.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:59.327+0000 7fa51e59c640 1 --2- 172.21.15.5:0/3334984726 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fa50c0a2c10 0x7fa50c0a50d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa510009b50 tx=0x7fa510005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:57:59.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:59.330+0000 7fa5177fe640 1 -- 172.21.15.5:0/3334984726 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa5080bbc60 con 0x7fa520101420 2024-09-12T02:57:59.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:57:59 smithi005 ceph-mon[31197]: pgmap v327: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-12T02:57:59.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:59.570+0000 7fa524a8a640 1 -- 172.21.15.5:0/3334984726 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fa520108a80 con 0x7fa520101420 2024-09-12T02:57:59.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:59.570+0000 7fa5177fe640 1 -- 172.21.15.5:0/3334984726 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v25) ==== 53+0+73 (secure 0 0 0) 0x7fa5080bff60 con 0x7fa520101420 2024-09-12T02:57:59.571 INFO:teuthology.orchestra.run.smithi005.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-12T02:57:59.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:59.572+0000 7fa524a8a640 1 -- 172.21.15.5:0/3334984726 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fa50c0a2c10 msgr2=0x7fa50c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:59.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:59.572+0000 7fa524a8a640 1 --2- 172.21.15.5:0/3334984726 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fa50c0a2c10 0x7fa50c0a50d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa510009b50 tx=0x7fa510005f90 comp rx=0 tx=0).stop 2024-09-12T02:57:59.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:59.572+0000 7fa524a8a640 1 -- 172.21.15.5:0/3334984726 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa520101420 msgr2=0x7fa5201997e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:57:59.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:59.572+0000 7fa524a8a640 1 --2- 172.21.15.5:0/3334984726 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa520101420 0x7fa5201997e0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fa508009230 tx=0x7fa508004040 comp rx=0 tx=0).stop 2024-09-12T02:57:59.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:59.572+0000 7fa524a8a640 1 -- 172.21.15.5:0/3334984726 shutdown_connections 2024-09-12T02:57:59.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:59.572+0000 7fa524a8a640 1 --2- 172.21.15.5:0/3334984726 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fa50c0a2c10 0x7fa50c0a50d0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:59.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:59.572+0000 7fa524a8a640 1 --2- 172.21.15.5:0/3334984726 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa520101420 0x7fa5201997e0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:59.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:59.572+0000 7fa524a8a640 1 --2- 172.21.15.5:0/3334984726 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa520106a80 0x7fa5201a4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:59.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:59.572+0000 7fa524a8a640 1 --2- 172.21.15.5:0/3334984726 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa520100a80 0x7fa5201992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:57:59.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:59.572+0000 7fa524a8a640 1 -- 172.21.15.5:0/3334984726 >> 172.21.15.5:0/3334984726 conn(0x7fa5200fc7a0 msgr2=0x7fa5200fcfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:57:59.578 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:59.572+0000 7fa524a8a640 1 -- 172.21.15.5:0/3334984726 shutdown_connections 2024-09-12T02:57:59.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:57:59.572+0000 7fa524a8a640 1 -- 172.21.15.5:0/3334984726 wait complete. 2024-09-12T02:57:59.665 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:57:59 smithi148 ceph-mon[35090]: pgmap v327: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-12T02:57:59.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:57:59 smithi146 ceph-mon[35011]: pgmap v327: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-12T02:58:00.352 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-12T02:58:00.352 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-12T02:58:00.352 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi005.front.sepia.ceph.com 2024-09-12T02:58:00.352 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-12T02:58:00.352 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-12T02:58:00.352 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-12T02:58:00.352 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-12T02:58:00.352 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-12T02:58:00.352 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-12T02:58:00.352 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T02:58:00.353 DEBUG:teuthology.orchestra.run.smithi005:> ip addr 2024-09-12T02:58:00.370 INFO:teuthology.orchestra.run.smithi005.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-12T02:58:00.370 INFO:teuthology.orchestra.run.smithi005.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-12T02:58:00.370 INFO:teuthology.orchestra.run.smithi005.stdout: inet 127.0.0.1/8 scope host lo 2024-09-12T02:58:00.370 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-09-12T02:58:00.370 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 ::1/128 scope host 2024-09-12T02:58:00.370 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-09-12T02:58:00.371 INFO:teuthology.orchestra.run.smithi005.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T02:58:00.371 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:de brd ff:ff:ff:ff:ff:ff 2024-09-12T02:58:00.371 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f0 2024-09-12T02:58:00.371 INFO:teuthology.orchestra.run.smithi005.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T02:58:00.371 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:df brd ff:ff:ff:ff:ff:ff 2024-09-12T02:58:00.371 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f1 2024-09-12T02:58:00.371 INFO:teuthology.orchestra.run.smithi005.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-12T02:58:00.371 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:42 brd ff:ff:ff:ff:ff:ff 2024-09-12T02:58:00.371 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f0 2024-09-12T02:58:00.371 INFO:teuthology.orchestra.run.smithi005.stdout: inet 172.21.15.5/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-12T02:58:00.371 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft 42222sec preferred_lft 42222sec 2024-09-12T02:58:00.371 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::ec4:7aff:febd:1442/64 scope link 2024-09-12T02:58:00.371 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2024-09-12T02:58:00.372 INFO:teuthology.orchestra.run.smithi005.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-12T02:58:00.372 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:43 brd ff:ff:ff:ff:ff:ff 2024-09-12T02:58:00.372 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f1 2024-09-12T02:58:00.372 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-12T02:58:00.372 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T02:58:00.372 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-09-12T02:58:00.373 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link add name ceph-brx type bridge 2024-09-12T02:58:00.373 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip addr flush dev ceph-brx 2024-09-12T02:58:00.373 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link set ceph-brx up 2024-09-12T02:58:00.373 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-12T02:58:00.373 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-09-12T02:58:00.461 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:00 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T02:58:00.570 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:58:00 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/3334984726' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-12T02:58:00.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T02:58:00.581 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T02:58:00.581 DEBUG:teuthology.orchestra.run.smithi005:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-12T02:58:00.607 INFO:teuthology.orchestra.run.smithi005.stdout:1 2024-09-12T02:58:00.608 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T02:58:00.609 DEBUG:teuthology.orchestra.run.smithi005:> route 2024-09-12T02:58:00.665 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:58:00 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/3334984726' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-12T02:58:00.688 INFO:teuthology.orchestra.run.smithi005.stdout:Kernel IP routing table 2024-09-12T02:58:00.688 INFO:teuthology.orchestra.run.smithi005.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-12T02:58:00.688 INFO:teuthology.orchestra.run.smithi005.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-12T02:58:00.688 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-12T02:58:00.688 INFO:teuthology.orchestra.run.smithi005.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-12T02:58:00.689 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T02:58:00.689 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-09-12T02:58:00.689 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-12T02:58:00.689 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-12T02:58:00.689 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-12T02:58:00.689 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-09-12T02:58:00.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:58:00 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/3334984726' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-12T02:58:00.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:00 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T02:58:00.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T02:58:00.986 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T02:58:00.986 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list 2024-09-12T02:58:01.003 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T02:58:01.003 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list-id 2024-09-12T02:58:01.059 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T02:58:01.059 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-09-12T02:58:01.059 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T02:58:01.059 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-12T02:58:01.059 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-09-12T02:58:01.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:01 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T02:58:01.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T02:58:01.181 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-12T02:58:01.181 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T02:58:01.181 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-09-12T02:58:01.181 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-12T02:58:01.181 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-12T02:58:01.181 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-12T02:58:01.181 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-12T02:58:01.181 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-12T02:58:01.181 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-09-12T02:58:01.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:01 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T02:58:01.306 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T02:58:01.312 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-12T02:58:01.313 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2024-09-12T02:58:01.313 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link set brx.0 up 2024-09-12T02:58:01.313 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link set dev brx.0 master ceph-brx 2024-09-12T02:58:01.313 DEBUG:teuthology.orchestra.run.smithi005:> ') 2024-09-12T02:58:01.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:01 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T02:58:01.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T02:58:01.393 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T02:58:01.393 DEBUG:teuthology.orchestra.run.smithi005:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-12T02:58:01.408 INFO:teuthology.orchestra.run.smithi005.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-12T02:58:01.409 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-12T02:58:01.409 DEBUG:teuthology.orchestra.run.smithi005:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-12T02:58:01.465 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-09-12T02:58:01.465 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-09-12T02:58:01.465 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/e6595afd-b733-4dfd-b518-50fa51993b54 2024-09-12T02:58:01.465 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T02:58:01.465 DEBUG:teuthology.orchestra.run.smithi005:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/e6595afd-b733-4dfd-b518-50fa51993b54 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,wsync,nofallback 2024-09-12T02:58:01.527 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:58:01 smithi005 ceph-mon[31197]: pgmap v328: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-12T02:58:01.527 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:58:01 smithi005 ceph-mon[31197]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-09-12T02:58:01.581 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,wsync,nofallback 2024-09-12T02:58:01.581 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,wsync". 2024-09-12T02:58:01.665 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:58:01 smithi148 ceph-mon[35090]: pgmap v328: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-12T02:58:01.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:58:01 smithi148 ceph-mon[35090]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-09-12T02:58:01.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:58:01 smithi146 ceph-mon[35011]: pgmap v328: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-12T02:58:01.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:58:01 smithi146 ceph-mon[35011]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-09-12T02:58:01.707 INFO:teuthology.orchestra.run.smithi005.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,wsync,nofallback 2024-09-12T02:58:01.707 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,wsync". 2024-09-12T02:58:01.707 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: resolved to: "172.21.15.5:3300,172.21.15.146:3300,172.21.15.148:3300" 2024-09-12T02:58:01.707 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: trying mount with new device syntax: 0@618305f8-70b1-11ef-bcea-c7b262605968.cephfs=/volumes/qa/sv_1/e6595afd-b733-4dfd-b518-50fa51993b54 2024-09-12T02:58:01.707 INFO:teuthology.orchestra.run.smithi005.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,wsync,name=0,key=0,mon_addr=172.21.15.5:3300/172.21.15.146:3300/172.21.15.148:3300" will pass to kernel 2024-09-12T02:58:01.708 INFO:teuthology.orchestra.run.smithi005.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-12T02:58:01.708 INFO:teuthology.orchestra.run.smithi005.stdout: You just mounted a file system that supports labels which does not 2024-09-12T02:58:01.708 INFO:teuthology.orchestra.run.smithi005.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-12T02:58:01.708 INFO:teuthology.orchestra.run.smithi005.stdout: applications will generate AVC messages and not be allowed access to 2024-09-12T02:58:01.708 INFO:teuthology.orchestra.run.smithi005.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-12T02:58:01.710 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-12T02:58:01.710 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T02:58:01.710 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-12T02:58:01.781 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-12T02:58:01.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:01 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-12T02:58:01.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T02:58:01.881 INFO:teuthology.orchestra.run.smithi005.stdout:client15204 2024-09-12T02:58:01.881 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-09-12T02:58:01.881 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T02:58:01.898 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-12T02:58:01.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:01 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-12T02:58:01.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T02:58:01.997 INFO:teuthology.orchestra.run.smithi005.stdout:client15204 2024-09-12T02:58:01.998 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-12T02:58:02.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:02 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-12T02:58:02.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T02:58:02.055 INFO:teuthology.orchestra.run.smithi005.stdout:client15204 2024-09-12T02:58:02.055 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/618305f8-70b1-11ef-bcea-c7b262605968.client15204/status') 2024-09-12T02:58:02.104 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:02 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-12T02:58:02.115 INFO:teuthology.orchestra.run.smithi005.stdout:instance: client.15204 (3)192.168.0.1:0/666136916 2024-09-12T02:58:02.115 INFO:teuthology.orchestra.run.smithi005.stdout:blocklisted: false 2024-09-12T02:58:02.115 INFO:teuthology.orchestra.run.smithi005.stderr:0+1 records in 2024-09-12T02:58:02.115 INFO:teuthology.orchestra.run.smithi005.stderr:0+1 records out 2024-09-12T02:58:02.115 INFO:teuthology.orchestra.run.smithi005.stderr:69 bytes copied, 2.4687e-05 s, 2.8 MB/s 2024-09-12T02:58:02.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-12T02:58:02.123 INFO:teuthology.run_tasks:Running task exec... 2024-09-12T02:58:02.133 INFO:teuthology.task.exec:Executing custom commands... 2024-09-12T02:58:02.133 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi005.front.sepia.ceph.com 2024-09-12T02:58:02.133 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-09-12T02:58:02.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:02.237+0000 7f07b89d8640 1 -- 172.21.15.5:0/3615241540 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f07b4102610 msgr2=0x7f07b4108ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:02.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:02.237+0000 7f07b89d8640 1 --2- 172.21.15.5:0/3615241540 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f07b4102610 0x7f07b4108ea0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f07ac009ca0 tx=0x7f07ac02a8c0 comp rx=0 tx=0).stop 2024-09-12T02:58:02.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:02.237+0000 7f07b89d8640 1 -- 172.21.15.5:0/3615241540 shutdown_connections 2024-09-12T02:58:02.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:02.237+0000 7f07b89d8640 1 --2- 172.21.15.5:0/3615241540 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f07b4102610 0x7f07b4108ea0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:02.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:02.237+0000 7f07b89d8640 1 --2- 172.21.15.5:0/3615241540 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f07b4101c50 0x7f07b41020d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:02.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:02.237+0000 7f07b89d8640 1 --2- 172.21.15.5:0/3615241540 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f07b4100a60 0x7f07b4100e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:02.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:02.237+0000 7f07b89d8640 1 -- 172.21.15.5:0/3615241540 >> 172.21.15.5:0/3615241540 conn(0x7f07b40fc1e0 msgr2=0x7f07b40fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:02.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:02.237+0000 7f07b89d8640 1 -- 172.21.15.5:0/3615241540 shutdown_connections 2024-09-12T02:58:02.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:02.237+0000 7f07b89d8640 1 -- 172.21.15.5:0/3615241540 wait complete. 2024-09-12T02:58:02.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:02.238+0000 7f07b89d8640 1 Processor -- start 2024-09-12T02:58:02.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:02.238+0000 7f07b89d8640 1 -- start start 2024-09-12T02:58:02.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:02.238+0000 7f07b89d8640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f07b4100a60 0x7f07b4197cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:02.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:02.238+0000 7f07b89d8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f07b4101c50 0x7f07b4198200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:02.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:02.238+0000 7f07b89d8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f07b4102610 0x7f07b419f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:02.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:02.238+0000 7f07b89d8640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f07b410d850 con 0x7f07b4102610 2024-09-12T02:58:02.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:02.238+0000 7f07b89d8640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f07b410d6d0 con 0x7f07b4100a60 2024-09-12T02:58:02.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:02.238+0000 7f07b89d8640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f07b410d9d0 con 0x7f07b4101c50 2024-09-12T02:58:02.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:02.238+0000 7f07b2d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f07b4102610 0x7f07b419f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:02.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:02.238+0000 7f07b2d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f07b4102610 0x7f07b419f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60360/0 (socket says 172.21.15.5:60360) 2024-09-12T02:58:02.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:02.238+0000 7f07b2d9d640 1 -- 172.21.15.5:0/2958700201 learned_addr learned my addr 172.21.15.5:0/2958700201 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:58:02.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:02.239+0000 7f07b259c640 1 --2- 172.21.15.5:0/2958700201 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f07b4100a60 0x7f07b4197cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:02.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:02.239+0000 7f07b2d9d640 1 -- 172.21.15.5:0/2958700201 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f07b4101c50 msgr2=0x7f07b4198200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:02.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:02.239+0000 7f07b2d9d640 1 --2- 172.21.15.5:0/2958700201 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f07b4101c50 0x7f07b4198200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:02.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:02.239+0000 7f07b2d9d640 1 -- 172.21.15.5:0/2958700201 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f07b4100a60 msgr2=0x7f07b4197cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:02.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:02.239+0000 7f07b2d9d640 1 --2- 172.21.15.5:0/2958700201 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f07b4100a60 0x7f07b4197cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:02.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:02.239+0000 7f07b2d9d640 1 -- 172.21.15.5:0/2958700201 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07b419f900 con 0x7f07b4102610 2024-09-12T02:58:02.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:02.239+0000 7f07b259c640 1 --2- 172.21.15.5:0/2958700201 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f07b4100a60 0x7f07b4197cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:58:02.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:02.239+0000 7f07b2d9d640 1 --2- 172.21.15.5:0/2958700201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f07b4102610 0x7f07b419f260 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f07ac0319f0 tx=0x7f07ac02f450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:02.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:02.239+0000 7f07a37fe640 1 -- 172.21.15.5:0/2958700201 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07ac003c70 con 0x7f07b4102610 2024-09-12T02:58:02.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:02.239+0000 7f07a37fe640 1 -- 172.21.15.5:0/2958700201 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07ac003e10 con 0x7f07b4102610 2024-09-12T02:58:02.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:02.239+0000 7f07b89d8640 1 -- 172.21.15.5:0/2958700201 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07b419fb90 con 0x7f07b4102610 2024-09-12T02:58:02.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:02.239+0000 7f07a37fe640 1 -- 172.21.15.5:0/2958700201 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07ac02fd30 con 0x7f07b4102610 2024-09-12T02:58:02.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:02.239+0000 7f07b89d8640 1 -- 172.21.15.5:0/2958700201 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07b41a00e0 con 0x7f07b4102610 2024-09-12T02:58:02.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:02.240+0000 7f07b89d8640 1 -- 172.21.15.5:0/2958700201 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07b4107d20 con 0x7f07b4102610 2024-09-12T02:58:02.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:02.241+0000 7f07a37fe640 1 -- 172.21.15.5:0/2958700201 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f07ac004040 con 0x7f07b4102610 2024-09-12T02:58:02.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:02.242+0000 7f07a37fe640 1 --2- 172.21.15.5:0/2958700201 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f079c0a2c30 0x7f079c0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:02.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:02.242+0000 7f07a37fe640 1 -- 172.21.15.5:0/2958700201 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f07ac0f08a0 con 0x7f07b4102610 2024-09-12T02:58:02.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:02.242+0000 7f07b259c640 1 --2- 172.21.15.5:0/2958700201 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f079c0a2c30 0x7f079c0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:02.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:02.243+0000 7f07b259c640 1 --2- 172.21.15.5:0/2958700201 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f079c0a2c30 0x7f079c0a50f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f07b4198b90 tx=0x7f07a4005dd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:02.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:02.246+0000 7f07a37fe640 1 -- 172.21.15.5:0/2958700201 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f07ac0bd7b0 con 0x7f07b4102610 2024-09-12T02:58:02.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:02.462+0000 7f07b89d8640 1 -- 172.21.15.5:0/2958700201 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7f07b4062e90 con 0x7f07b4102610 2024-09-12T02:58:03.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:03.328+0000 7f07a37fe640 1 -- 172.21.15.5:0/2958700201 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v26) ==== 118+0+0 (secure 0 0 0) 0x7f07ac0bd0b0 con 0x7f07b4102610 2024-09-12T02:58:03.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:03.332+0000 7f07b89d8640 1 -- 172.21.15.5:0/2958700201 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f079c0a2c30 msgr2=0x7f079c0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:03.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:03.332+0000 7f07b89d8640 1 --2- 172.21.15.5:0/2958700201 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f079c0a2c30 0x7f079c0a50f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f07b4198b90 tx=0x7f07a4005dd0 comp rx=0 tx=0).stop 2024-09-12T02:58:03.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:03.332+0000 7f07b89d8640 1 -- 172.21.15.5:0/2958700201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f07b4102610 msgr2=0x7f07b419f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:03.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:03.332+0000 7f07b89d8640 1 --2- 172.21.15.5:0/2958700201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f07b4102610 0x7f07b419f260 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f07ac0319f0 tx=0x7f07ac02f450 comp rx=0 tx=0).stop 2024-09-12T02:58:03.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:03.332+0000 7f07b89d8640 1 -- 172.21.15.5:0/2958700201 shutdown_connections 2024-09-12T02:58:03.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:03.332+0000 7f07b89d8640 1 --2- 172.21.15.5:0/2958700201 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f079c0a2c30 0x7f079c0a50f0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:03.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:03.332+0000 7f07b89d8640 1 --2- 172.21.15.5:0/2958700201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f07b4102610 0x7f07b419f260 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:03.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:03.332+0000 7f07b89d8640 1 --2- 172.21.15.5:0/2958700201 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f07b4101c50 0x7f07b4198200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:03.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:03.332+0000 7f07b89d8640 1 --2- 172.21.15.5:0/2958700201 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f07b4100a60 0x7f07b4197cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:03.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:03.332+0000 7f07b89d8640 1 -- 172.21.15.5:0/2958700201 >> 172.21.15.5:0/2958700201 conn(0x7f07b40fc1e0 msgr2=0x7f07b4104ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:03.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:03.333+0000 7f07b89d8640 1 -- 172.21.15.5:0/2958700201 shutdown_connections 2024-09-12T02:58:03.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:03.333+0000 7f07b89d8640 1 -- 172.21.15.5:0/2958700201 wait complete. 2024-09-12T02:58:03.346 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-09-12T02:58:03.462 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:03.460+0000 7f9296fd8640 1 -- 172.21.15.5:0/4189805917 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9290102de0 msgr2=0x7f9290103260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:03.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:03.460+0000 7f9296fd8640 1 --2- 172.21.15.5:0/4189805917 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9290102de0 0x7f9290103260 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f9280009ca0 tx=0x7f928002a8c0 comp rx=0 tx=0).stop 2024-09-12T02:58:03.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:03.460+0000 7f9296fd8640 1 -- 172.21.15.5:0/4189805917 shutdown_connections 2024-09-12T02:58:03.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:03.460+0000 7f9296fd8640 1 --2- 172.21.15.5:0/4189805917 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9290102de0 0x7f9290103260 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:03.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:03.460+0000 7f9296fd8640 1 --2- 172.21.15.5:0/4189805917 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92901037a0 0x7f929010a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:03.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:03.460+0000 7f9296fd8640 1 --2- 172.21.15.5:0/4189805917 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f92900ffe10 0x7f9290100210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:03.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:03.460+0000 7f9296fd8640 1 -- 172.21.15.5:0/4189805917 >> 172.21.15.5:0/4189805917 conn(0x7f92900fb930 msgr2=0x7f92900fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:03.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:03.460+0000 7f9296fd8640 1 -- 172.21.15.5:0/4189805917 shutdown_connections 2024-09-12T02:58:03.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:03.460+0000 7f9296fd8640 1 -- 172.21.15.5:0/4189805917 wait complete. 2024-09-12T02:58:03.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:03.461+0000 7f9296fd8640 1 Processor -- start 2024-09-12T02:58:03.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:03.461+0000 7f9296fd8640 1 -- start start 2024-09-12T02:58:03.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:03.461+0000 7f9296fd8640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f92900ffe10 0x7f9290197e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:03.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:03.461+0000 7f9296fd8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9290102de0 0x7f9290198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:03.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:03.461+0000 7f9296fd8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92901037a0 0x7f929019f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:03.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:03.461+0000 7f9296fd8640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f929010d840 con 0x7f9290102de0 2024-09-12T02:58:03.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:03.461+0000 7f9296fd8640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f929010d6c0 con 0x7f92900ffe10 2024-09-12T02:58:03.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:03.461+0000 7f9296fd8640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f929010d9c0 con 0x7f92901037a0 2024-09-12T02:58:03.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:03.461+0000 7f928ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9290102de0 0x7f9290198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:03.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:03.461+0000 7f928ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9290102de0 0x7f9290198380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60384/0 (socket says 172.21.15.5:60384) 2024-09-12T02:58:03.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:03.461+0000 7f928ffff640 1 -- 172.21.15.5:0/1727863350 learned_addr learned my addr 172.21.15.5:0/1727863350 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:58:03.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:03.461+0000 7f9294d74640 1 --2- 172.21.15.5:0/1727863350 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f92900ffe10 0x7f9290197e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:03.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:03.461+0000 7f928ffff640 1 -- 172.21.15.5:0/1727863350 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92901037a0 msgr2=0x7f929019f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:03.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:03.461+0000 7f928ffff640 1 --2- 172.21.15.5:0/1727863350 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92901037a0 0x7f929019f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:03.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:03.461+0000 7f928ffff640 1 -- 172.21.15.5:0/1727863350 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f92900ffe10 msgr2=0x7f9290197e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:03.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:03.461+0000 7f928ffff640 1 --2- 172.21.15.5:0/1727863350 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f92900ffe10 0x7f9290197e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:03.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:03.461+0000 7f928ffff640 1 -- 172.21.15.5:0/1727863350 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f929019fa60 con 0x7f9290102de0 2024-09-12T02:58:03.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:03.461+0000 7f928ffff640 1 --2- 172.21.15.5:0/1727863350 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9290102de0 0x7f9290198380 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f9280031a40 tx=0x7f928002f800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:03.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:03.462+0000 7f928dffb640 1 -- 172.21.15.5:0/1727863350 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f928002fd80 con 0x7f9290102de0 2024-09-12T02:58:03.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:03.462+0000 7f9296fd8640 1 -- 172.21.15.5:0/1727863350 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f929019fcf0 con 0x7f9290102de0 2024-09-12T02:58:03.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:03.462+0000 7f928dffb640 1 -- 172.21.15.5:0/1727863350 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9280030aa0 con 0x7f9290102de0 2024-09-12T02:58:03.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:03.462+0000 7f928dffb640 1 -- 172.21.15.5:0/1727863350 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9280046a70 con 0x7f9290102de0 2024-09-12T02:58:03.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:03.462+0000 7f9296fd8640 1 -- 172.21.15.5:0/1727863350 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92901a0260 con 0x7f9290102de0 2024-09-12T02:58:03.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:03.463+0000 7f9296fd8640 1 -- 172.21.15.5:0/1727863350 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9290108f80 con 0x7f9290102de0 2024-09-12T02:58:03.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:03.464+0000 7f928dffb640 1 -- 172.21.15.5:0/1727863350 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f9280030c50 con 0x7f9290102de0 2024-09-12T02:58:03.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:03.465+0000 7f928dffb640 1 --2- 172.21.15.5:0/1727863350 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f927c0a2c30 0x7f927c0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:03.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:03.465+0000 7f9294d74640 1 --2- 172.21.15.5:0/1727863350 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f927c0a2c30 0x7f927c0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:03.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:03.465+0000 7f928dffb640 1 -- 172.21.15.5:0/1727863350 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f92800f0d60 con 0x7f9290102de0 2024-09-12T02:58:03.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:03.465+0000 7f9294d74640 1 --2- 172.21.15.5:0/1727863350 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f927c0a2c30 0x7f927c0a50f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f929006a140 tx=0x7f9278005c60 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:03.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:03.469+0000 7f928dffb640 1 -- 172.21.15.5:0/1727863350 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f92800bdc70 con 0x7f9290102de0 2024-09-12T02:58:03.665 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:58:03 smithi148 ceph-mon[35090]: pgmap v329: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-12T02:58:03.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:58:03 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/2958700201' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-12T02:58:03.671 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:03.671+0000 7f9296fd8640 1 -- 172.21.15.5:0/1727863350 --> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7f92900626d0 con 0x7f927c0a2c30 2024-09-12T02:58:03.677 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:58:03 smithi005 ceph-mon[31197]: pgmap v329: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-12T02:58:03.677 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:58:03 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/2958700201' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-12T02:58:03.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:03.696+0000 7f928dffb640 1 -- 172.21.15.5:0/1727863350 <== mgr.14148 v2:172.21.15.5:6800/3780631642 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7f92900626d0 con 0x7f927c0a2c30 2024-09-12T02:58:03.696 INFO:teuthology.orchestra.run.smithi005.stdout:{} 2024-09-12T02:58:03.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:03.698+0000 7f9296fd8640 1 -- 172.21.15.5:0/1727863350 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f927c0a2c30 msgr2=0x7f927c0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:03.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:03.698+0000 7f9296fd8640 1 --2- 172.21.15.5:0/1727863350 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f927c0a2c30 0x7f927c0a50f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f929006a140 tx=0x7f9278005c60 comp rx=0 tx=0).stop 2024-09-12T02:58:03.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:03.698+0000 7f9296fd8640 1 -- 172.21.15.5:0/1727863350 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9290102de0 msgr2=0x7f9290198380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:03.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:03.698+0000 7f9296fd8640 1 --2- 172.21.15.5:0/1727863350 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9290102de0 0x7f9290198380 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f9280031a40 tx=0x7f928002f800 comp rx=0 tx=0).stop 2024-09-12T02:58:03.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:03.698+0000 7f9296fd8640 1 -- 172.21.15.5:0/1727863350 shutdown_connections 2024-09-12T02:58:03.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:03.698+0000 7f9296fd8640 1 --2- 172.21.15.5:0/1727863350 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f927c0a2c30 0x7f927c0a50f0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:03.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:03.698+0000 7f9296fd8640 1 --2- 172.21.15.5:0/1727863350 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9290102de0 0x7f9290198380 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:03.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:03.698+0000 7f9296fd8640 1 --2- 172.21.15.5:0/1727863350 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92901037a0 0x7f929019f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:03.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:03.698+0000 7f9296fd8640 1 --2- 172.21.15.5:0/1727863350 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f92900ffe10 0x7f9290197e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:03.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:03.698+0000 7f9296fd8640 1 -- 172.21.15.5:0/1727863350 >> 172.21.15.5:0/1727863350 conn(0x7f92900fb930 msgr2=0x7f9290106150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:03.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:03.698+0000 7f9296fd8640 1 -- 172.21.15.5:0/1727863350 shutdown_connections 2024-09-12T02:58:03.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:03.698+0000 7f9296fd8640 1 -- 172.21.15.5:0/1727863350 wait complete. 2024-09-12T02:58:03.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:58:03 smithi146 ceph-mon[35011]: pgmap v329: 97 pgs: 97 active+clean; 644 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-09-12T02:58:03.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:58:03 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/2958700201' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-12T02:58:03.709 INFO:teuthology.run_tasks:Running task check-counter... 2024-09-12T02:58:03.720 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-09-12T02:58:03.770 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T02:58:03.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:03.874+0000 7f714e8a5640 1 -- 172.21.15.5:0/1165385727 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71481025e0 msgr2=0x7f7148108e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:03.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:03.874+0000 7f714e8a5640 1 --2- 172.21.15.5:0/1165385727 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71481025e0 0x7f7148108e70 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f7140009ca0 tx=0x7f714002a8c0 comp rx=0 tx=0).stop 2024-09-12T02:58:03.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:03.874+0000 7f714e8a5640 1 -- 172.21.15.5:0/1165385727 shutdown_connections 2024-09-12T02:58:03.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:03.874+0000 7f714e8a5640 1 --2- 172.21.15.5:0/1165385727 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71481025e0 0x7f7148108e70 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:03.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:03.874+0000 7f714e8a5640 1 --2- 172.21.15.5:0/1165385727 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7148101c20 0x7f71481020a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:03.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:03.874+0000 7f714e8a5640 1 --2- 172.21.15.5:0/1165385727 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7148100a30 0x7f7148100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:03.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:03.874+0000 7f714e8a5640 1 -- 172.21.15.5:0/1165385727 >> 172.21.15.5:0/1165385727 conn(0x7f71480fc1b0 msgr2=0x7f71480fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:03.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:03.874+0000 7f714e8a5640 1 -- 172.21.15.5:0/1165385727 shutdown_connections 2024-09-12T02:58:03.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:03.874+0000 7f714e8a5640 1 -- 172.21.15.5:0/1165385727 wait complete. 2024-09-12T02:58:03.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:03.875+0000 7f714e8a5640 1 Processor -- start 2024-09-12T02:58:03.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:03.875+0000 7f714e8a5640 1 -- start start 2024-09-12T02:58:03.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:03.875+0000 7f714e8a5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7148100a30 0x7f7148197e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:03.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:03.875+0000 7f714e8a5640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7148101c20 0x7f7148198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:03.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:03.875+0000 7f714e8a5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71481025e0 0x7f714819f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:03.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:03.875+0000 7f714e8a5640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f714810d800 con 0x7f71481025e0 2024-09-12T02:58:03.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:03.875+0000 7f714e8a5640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f714810d680 con 0x7f7148101c20 2024-09-12T02:58:03.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:03.875+0000 7f714e8a5640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f714810d980 con 0x7f7148100a30 2024-09-12T02:58:03.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:03.875+0000 7f714ce42640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71481025e0 0x7f714819f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:03.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:03.875+0000 7f714ce42640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71481025e0 0x7f714819f3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55334/0 (socket says 172.21.15.5:55334) 2024-09-12T02:58:03.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:03.875+0000 7f714ce42640 1 -- 172.21.15.5:0/475689725 learned_addr learned my addr 172.21.15.5:0/475689725 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:58:03.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:03.876+0000 7f71477fe640 1 --2- 172.21.15.5:0/475689725 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7148101c20 0x7f7148198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:03.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:03.876+0000 7f7147fff640 1 --2- 172.21.15.5:0/475689725 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7148100a30 0x7f7148197e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:03.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:03.876+0000 7f714ce42640 1 -- 172.21.15.5:0/475689725 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7148100a30 msgr2=0x7f7148197e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:03.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:03.876+0000 7f714ce42640 1 --2- 172.21.15.5:0/475689725 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7148100a30 0x7f7148197e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:03.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:03.876+0000 7f714ce42640 1 -- 172.21.15.5:0/475689725 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7148101c20 msgr2=0x7f7148198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:03.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:03.876+0000 7f714ce42640 1 --2- 172.21.15.5:0/475689725 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7148101c20 0x7f7148198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:03.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:03.876+0000 7f714ce42640 1 -- 172.21.15.5:0/475689725 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f714819fa40 con 0x7f71481025e0 2024-09-12T02:58:03.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:03.876+0000 7f714ce42640 1 --2- 172.21.15.5:0/475689725 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71481025e0 0x7f714819f3d0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f7140005e30 tx=0x7f7140031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:03.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:03.876+0000 7f71457fa640 1 -- 172.21.15.5:0/475689725 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7140003c30 con 0x7f71481025e0 2024-09-12T02:58:03.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:03.876+0000 7f71457fa640 1 -- 172.21.15.5:0/475689725 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7140003dd0 con 0x7f71481025e0 2024-09-12T02:58:03.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:03.876+0000 7f714e8a5640 1 -- 172.21.15.5:0/475689725 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f714819fcd0 con 0x7f71481025e0 2024-09-12T02:58:03.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:03.876+0000 7f71457fa640 1 -- 172.21.15.5:0/475689725 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f714002fde0 con 0x7f71481025e0 2024-09-12T02:58:03.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:03.876+0000 7f714e8a5640 1 -- 172.21.15.5:0/475689725 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71481a01d0 con 0x7f71481025e0 2024-09-12T02:58:03.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:03.877+0000 7f714e8a5640 1 -- 172.21.15.5:0/475689725 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7148107df0 con 0x7f71481025e0 2024-09-12T02:58:03.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:03.878+0000 7f71457fa640 1 -- 172.21.15.5:0/475689725 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f7140004050 con 0x7f71481025e0 2024-09-12T02:58:03.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:03.879+0000 7f71457fa640 1 --2- 172.21.15.5:0/475689725 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f71300a2b90 0x7f71300a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:03.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:03.879+0000 7f71457fa640 1 -- 172.21.15.5:0/475689725 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f71400f04b0 con 0x7f71481025e0 2024-09-12T02:58:03.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:03.879+0000 7f7147fff640 1 --2- 172.21.15.5:0/475689725 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f71300a2b90 0x7f71300a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:03.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:03.879+0000 7f7147fff640 1 --2- 172.21.15.5:0/475689725 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f71300a2b90 0x7f71300a5050 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f7138005fd0 tx=0x7f7138005850 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:03.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:03.883+0000 7f71457fa640 1 -- 172.21.15.5:0/475689725 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f71400bd420 con 0x7f71481025e0 2024-09-12T02:58:04.001 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:58:03 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: 2024-09-12T02:58:03.675+0000 7ff89129a640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T02:58:04.002 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:58:03 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: 2024-09-12T02:58:03.675+0000 7ff89129a640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T02:58:04.002 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:58:03 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: 2024-09-12T02:58:03.675+0000 7ff89129a640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T02:58:04.002 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:58:03 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: 2024-09-12T02:58:03.675+0000 7ff89129a640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T02:58:04.002 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 02:58:03 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: 2024-09-12T02:58:03.675+0000 7ff89129a640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-12T02:58:04.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.063+0000 7f714e8a5640 1 -- 172.21.15.5:0/475689725 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7148062e90 con 0x7f71481025e0 2024-09-12T02:58:04.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.064+0000 7f71457fa640 1 -- 172.21.15.5:0/475689725 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+22656 (secure 0 0 0) 0x7f71400bcd20 con 0x7f71481025e0 2024-09-12T02:58:04.065 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-12T02:58:04.065 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":83,"fsid":"618305f8-70b1-11ef-bcea-c7b262605968","created":"2024-09-12T02:48:42.352146+0000","modified":"2024-09-12T02:56:31.026079+0000","last_up_change":"2024-09-12T02:55:27.817161+0000","last_in_change":"2024-09-12T02:55:03.745281+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T02:51:47.970639+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T02:56:12.834271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T02:56:14.951437+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17da560a-6aa2-4f2c-a95f-67481dce4d3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":4248507573},{"type":"v1","addr":"172.21.15.5:6803","nonce":4248507573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":4248507573},{"type":"v1","addr":"172.21.15.5:6805","nonce":4248507573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":4248507573},{"type":"v1","addr":"172.21.15.5:6809","nonce":4248507573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":4248507573},{"type":"v1","addr":"172.21.15.5:6807","nonce":4248507573}]},"public_addr":"172.21.15.5:6803/4248507573","cluster_addr":"172.21.15.5:6805/4248507573","heartbeat_back_addr":"172.21.15.5:6809/4248507573","heartbeat_front_addr":"172.21.15.5:6807/4248507573","state":["exists","up"]},{"osd":1,"uuid":"a3237006-99af-4fc1-860c-d4917032647f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":1105990287},{"type":"v1","addr":"172.21.15.146:6801","nonce":1105990287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":1105990287},{"type":"v1","addr":"172.21.15.146:6803","nonce":1105990287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":1105990287},{"type":"v1","addr":"172.21.15.146:6807","nonce":1105990287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":1105990287},{"type":"v1","addr":"172.21.15.146:6805","nonce":1105990287}]},"public_addr":"172.21.15.146:6801/1105990287","cluster_addr":"172.21.15.146:6803/1105990287","heartbeat_back_addr":"172.21.15.146:6807/1105990287","heartbeat_front_addr":"172.21.15.146:6805/1105990287","state":["exists","up"]},{"osd":2,"uuid":"ea3ab84a-7739-4975-8244-31ddd5aad80d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3378174313},{"type":"v1","addr":"172.21.15.148:6801","nonce":3378174313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3378174313},{"type":"v1","addr":"172.21.15.148:6803","nonce":3378174313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3378174313},{"type":"v1","addr":"172.21.15.148:6807","nonce":3378174313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3378174313},{"type":"v1","addr":"172.21.15.148:6805","nonce":3378174313}]},"public_addr":"172.21.15.148:6801/3378174313","cluster_addr":"172.21.15.148:6803/3378174313","heartbeat_back_addr":"172.21.15.148:6807/3378174313","heartbeat_front_addr":"172.21.15.148:6805/3378174313","state":["exists","up"]},{"osd":3,"uuid":"4396a9ee-f3f3-410b-9fb1-124f1e33a1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":11741938},{"type":"v1","addr":"172.21.15.5:6811","nonce":11741938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":11741938},{"type":"v1","addr":"172.21.15.5:6813","nonce":11741938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":11741938},{"type":"v1","addr":"172.21.15.5:6817","nonce":11741938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":11741938},{"type":"v1","addr":"172.21.15.5:6815","nonce":11741938}]},"public_addr":"172.21.15.5:6811/11741938","cluster_addr":"172.21.15.5:6813/11741938","heartbeat_back_addr":"172.21.15.5:6817/11741938","heartbeat_front_addr":"172.21.15.5:6815/11741938","state":["exists","up"]},{"osd":4,"uuid":"989d76e7-c742-49d0-994a-c58f2771b1d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":3274923851},{"type":"v1","addr":"172.21.15.146:6809","nonce":3274923851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":3274923851},{"type":"v1","addr":"172.21.15.146:6811","nonce":3274923851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3274923851},{"type":"v1","addr":"172.21.15.146:6815","nonce":3274923851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":3274923851},{"type":"v1","addr":"172.21.15.146:6813","nonce":3274923851}]},"public_addr":"172.21.15.146:6809/3274923851","cluster_addr":"172.21.15.146:6811/3274923851","heartbeat_back_addr":"172.21.15.146:6815/3274923851","heartbeat_front_addr":"172.21.15.146:6813/3274923851","state":["exists","up"]},{"osd":5,"uuid":"64ada7d4-70a0-4b83-9af8-032be544608d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":2771670476},{"type":"v1","addr":"172.21.15.148:6809","nonce":2771670476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":2771670476},{"type":"v1","addr":"172.21.15.148:6811","nonce":2771670476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":2771670476},{"type":"v1","addr":"172.21.15.148:6815","nonce":2771670476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":2771670476},{"type":"v1","addr":"172.21.15.148:6813","nonce":2771670476}]},"public_addr":"172.21.15.148:6809/2771670476","cluster_addr":"172.21.15.148:6811/2771670476","heartbeat_back_addr":"172.21.15.148:6815/2771670476","heartbeat_front_addr":"172.21.15.148:6813/2771670476","state":["exists","up"]},{"osd":6,"uuid":"a70a88fd-2e4b-480c-bb98-62636cf70cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3176073169},{"type":"v1","addr":"172.21.15.5:6819","nonce":3176073169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":3176073169},{"type":"v1","addr":"172.21.15.5:6821","nonce":3176073169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":3176073169},{"type":"v1","addr":"172.21.15.5:6825","nonce":3176073169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3176073169},{"type":"v1","addr":"172.21.15.5:6823","nonce":3176073169}]},"public_addr":"172.21.15.5:6819/3176073169","cluster_addr":"172.21.15.5:6821/3176073169","heartbeat_back_addr":"172.21.15.5:6825/3176073169","heartbeat_front_addr":"172.21.15.5:6823/3176073169","state":["exists","up"]},{"osd":7,"uuid":"49451e8d-c741-4ad5-b8f5-89bad8d4d129","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2209508838},{"type":"v1","addr":"172.21.15.146:6817","nonce":2209508838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2209508838},{"type":"v1","addr":"172.21.15.146:6819","nonce":2209508838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2209508838},{"type":"v1","addr":"172.21.15.146:6823","nonce":2209508838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2209508838},{"type":"v1","addr":"172.21.15.146:6821","nonce":2209508838}]},"public_addr":"172.21.15.146:6817/2209508838","cluster_addr":"172.21.15.146:6819/2209508838","heartbeat_back_addr":"172.21.15.146:6823/2209508838","heartbeat_front_addr":"172.21.15.146:6821/2209508838","state":["exists","up"]},{"osd":8,"uuid":"57bfc121-a75e-4a7f-b1dc-64074b178b03","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":651045756},{"type":"v1","addr":"172.21.15.148:6817","nonce":651045756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":651045756},{"type":"v1","addr":"172.21.15.148:6819","nonce":651045756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":651045756},{"type":"v1","addr":"172.21.15.148:6823","nonce":651045756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":651045756},{"type":"v1","addr":"172.21.15.148:6821","nonce":651045756}]},"public_addr":"172.21.15.148:6817/651045756","cluster_addr":"172.21.15.148:6819/651045756","heartbeat_back_addr":"172.21.15.148:6823/651045756","heartbeat_front_addr":"172.21.15.148:6821/651045756","state":["exists","up"]},{"osd":9,"uuid":"d9726168-7494-4bf8-b433-350e85bfd461","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1961857360},{"type":"v1","addr":"172.21.15.5:6827","nonce":1961857360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1961857360},{"type":"v1","addr":"172.21.15.5:6829","nonce":1961857360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":1961857360},{"type":"v1","addr":"172.21.15.5:6833","nonce":1961857360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1961857360},{"type":"v1","addr":"172.21.15.5:6831","nonce":1961857360}]},"public_addr":"172.21.15.5:6827/1961857360","cluster_addr":"172.21.15.5:6829/1961857360","heartbeat_back_addr":"172.21.15.5:6833/1961857360","heartbeat_front_addr":"172.21.15.5:6831/1961857360","state":["exists","up"]},{"osd":10,"uuid":"519ec60f-3606-48ac-9043-6f59a798766d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":2041733103},{"type":"v1","addr":"172.21.15.146:6825","nonce":2041733103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":2041733103},{"type":"v1","addr":"172.21.15.146:6827","nonce":2041733103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":2041733103},{"type":"v1","addr":"172.21.15.146:6831","nonce":2041733103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":2041733103},{"type":"v1","addr":"172.21.15.146:6829","nonce":2041733103}]},"public_addr":"172.21.15.146:6825/2041733103","cluster_addr":"172.21.15.146:6827/2041733103","heartbeat_back_addr":"172.21.15.146:6831/2041733103","heartbeat_front_addr":"172.21.15.146:6829/2041733103","state":["exists","up"]},{"osd":11,"uuid":"0f92d0cb-ba54-4bc3-a6d0-3c54d910556a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":839290773},{"type":"v1","addr":"172.21.15.148:6825","nonce":839290773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":839290773},{"type":"v1","addr":"172.21.15.148:6827","nonce":839290773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":839290773},{"type":"v1","addr":"172.21.15.148:6831","nonce":839290773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":839290773},{"type":"v1","addr":"172.21.15.148:6829","nonce":839290773}]},"public_addr":"172.21.15.148:6825/839290773","cluster_addr":"172.21.15.148:6827/839290773","heartbeat_back_addr":"172.21.15.148:6831/839290773","heartbeat_front_addr":"172.21.15.148:6829/839290773","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:50:54.101755+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:51:19.037385+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:51:44.142837+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:52:09.150887+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:52:33.705079+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:52:58.845233+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:53:23.068339+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:53:47.520398+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:54:11.965370+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:54:36.776075+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:55:01.292236+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:55:25.346589+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:6800/3125821798":"2024-09-13T02:49:29.462216+0000","172.21.15.5:0/3479911451":"2024-09-13T02:49:29.462216+0000","172.21.15.5:0/1547753207":"2024-09-13T02:49:03.300872+0000","172.21.15.5:0/1905762627":"2024-09-13T02:49:03.300872+0000","172.21.15.5:0/4142340280":"2024-09-13T02:49:29.462216+0000","172.21.15.5:6801/3125821798":"2024-09-13T02:49:29.462216+0000","172.21.15.5:6801/2595781680":"2024-09-13T02:49:03.300872+0000","172.21.15.5:0/2273691311":"2024-09-13T02:49:29.462216+0000","172.21.15.5:6800/2595781680":"2024-09-13T02:49:03.300872+0000","172.21.15.5:0/1196201217":"2024-09-13T02:49:03.300872+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T02:58:04.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.067+0000 7f714e8a5640 1 -- 172.21.15.5:0/475689725 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f71300a2b90 msgr2=0x7f71300a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:04.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.067+0000 7f714e8a5640 1 --2- 172.21.15.5:0/475689725 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f71300a2b90 0x7f71300a5050 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f7138005fd0 tx=0x7f7138005850 comp rx=0 tx=0).stop 2024-09-12T02:58:04.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.067+0000 7f714e8a5640 1 -- 172.21.15.5:0/475689725 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71481025e0 msgr2=0x7f714819f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:04.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.067+0000 7f714e8a5640 1 --2- 172.21.15.5:0/475689725 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71481025e0 0x7f714819f3d0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f7140005e30 tx=0x7f7140031ee0 comp rx=0 tx=0).stop 2024-09-12T02:58:04.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.067+0000 7f714e8a5640 1 -- 172.21.15.5:0/475689725 shutdown_connections 2024-09-12T02:58:04.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.067+0000 7f714e8a5640 1 --2- 172.21.15.5:0/475689725 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f71300a2b90 0x7f71300a5050 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:04.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.067+0000 7f714e8a5640 1 --2- 172.21.15.5:0/475689725 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71481025e0 0x7f714819f3d0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:04.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.067+0000 7f714e8a5640 1 --2- 172.21.15.5:0/475689725 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7148101c20 0x7f7148198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:04.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.067+0000 7f714e8a5640 1 --2- 172.21.15.5:0/475689725 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7148100a30 0x7f7148197e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:04.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.067+0000 7f714e8a5640 1 -- 172.21.15.5:0/475689725 >> 172.21.15.5:0/475689725 conn(0x7f71480fc1b0 msgr2=0x7f7148104fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:04.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.067+0000 7f714e8a5640 1 -- 172.21.15.5:0/475689725 shutdown_connections 2024-09-12T02:58:04.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.067+0000 7f714e8a5640 1 -- 172.21.15.5:0/475689725 wait complete. 2024-09-12T02:58:04.078 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-12T02:51:47.970639+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-12T02:56:12.834271+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-12T02:56:14.951437+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-12T02:58:04.079 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-12T02:58:04.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.185+0000 7f9947f9a640 1 -- 172.21.15.5:0/3114808340 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9940101c20 msgr2=0x7f99401020a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:04.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.185+0000 7f9947f9a640 1 --2- 172.21.15.5:0/3114808340 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9940101c20 0x7f99401020a0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f9930009900 tx=0x7f993002a410 comp rx=0 tx=0).stop 2024-09-12T02:58:04.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.185+0000 7f9947f9a640 1 -- 172.21.15.5:0/3114808340 shutdown_connections 2024-09-12T02:58:04.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.185+0000 7f9947f9a640 1 --2- 172.21.15.5:0/3114808340 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9940101c20 0x7f99401020a0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:04.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.185+0000 7f9947f9a640 1 --2- 172.21.15.5:0/3114808340 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f99401025e0 0x7f9940108e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:04.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.185+0000 7f9947f9a640 1 --2- 172.21.15.5:0/3114808340 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9940100a30 0x7f9940100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:04.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.185+0000 7f9947f9a640 1 -- 172.21.15.5:0/3114808340 >> 172.21.15.5:0/3114808340 conn(0x7f99400fc1b0 msgr2=0x7f99400fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:04.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.185+0000 7f9947f9a640 1 -- 172.21.15.5:0/3114808340 shutdown_connections 2024-09-12T02:58:04.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.185+0000 7f9947f9a640 1 -- 172.21.15.5:0/3114808340 wait complete. 2024-09-12T02:58:04.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.186+0000 7f9947f9a640 1 Processor -- start 2024-09-12T02:58:04.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.186+0000 7f9947f9a640 1 -- start start 2024-09-12T02:58:04.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.186+0000 7f9947f9a640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9940100a30 0x7f9940196db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:04.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.186+0000 7f9947f9a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9940101c20 0x7f99401972f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:04.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.186+0000 7f9947f9a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99401025e0 0x7f994019e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:04.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.186+0000 7f9947f9a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f994010c700 con 0x7f99401025e0 2024-09-12T02:58:04.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.186+0000 7f9947f9a640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f994010c580 con 0x7f9940100a30 2024-09-12T02:58:04.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.186+0000 7f9947f9a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f994010c880 con 0x7f9940101c20 2024-09-12T02:58:04.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.186+0000 7f9946537640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99401025e0 0x7f994019e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:04.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.186+0000 7f9946537640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99401025e0 0x7f994019e360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55352/0 (socket says 172.21.15.5:55352) 2024-09-12T02:58:04.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.186+0000 7f9946537640 1 -- 172.21.15.5:0/2645251996 learned_addr learned my addr 172.21.15.5:0/2645251996 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:58:04.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.186+0000 7f9945d36640 1 --2- 172.21.15.5:0/2645251996 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9940100a30 0x7f9940196db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:04.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.186+0000 7f9946537640 1 -- 172.21.15.5:0/2645251996 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9940101c20 msgr2=0x7f99401972f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:04.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.186+0000 7f9946537640 1 --2- 172.21.15.5:0/2645251996 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9940101c20 0x7f99401972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:04.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.187+0000 7f9946537640 1 -- 172.21.15.5:0/2645251996 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9940100a30 msgr2=0x7f9940196db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:04.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.187+0000 7f9946537640 1 --2- 172.21.15.5:0/2645251996 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9940100a30 0x7f9940196db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:04.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.187+0000 7f9946537640 1 -- 172.21.15.5:0/2645251996 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f994019e9d0 con 0x7f99401025e0 2024-09-12T02:58:04.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.187+0000 7f9945d36640 1 --2- 172.21.15.5:0/2645251996 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9940100a30 0x7f9940196db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:58:04.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.187+0000 7f9946537640 1 --2- 172.21.15.5:0/2645251996 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99401025e0 0x7f994019e360 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f993800eaa0 tx=0x7f993800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:04.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.187+0000 7f9936ffd640 1 -- 172.21.15.5:0/2645251996 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f993800cdb0 con 0x7f99401025e0 2024-09-12T02:58:04.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.187+0000 7f9936ffd640 1 -- 172.21.15.5:0/2645251996 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9938005020 con 0x7f99401025e0 2024-09-12T02:58:04.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.187+0000 7f9947f9a640 1 -- 172.21.15.5:0/2645251996 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f994019ecc0 con 0x7f99401025e0 2024-09-12T02:58:04.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.187+0000 7f9936ffd640 1 -- 172.21.15.5:0/2645251996 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99380116b0 con 0x7f99401025e0 2024-09-12T02:58:04.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.187+0000 7f9947f9a640 1 -- 172.21.15.5:0/2645251996 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f994019f210 con 0x7f99401025e0 2024-09-12T02:58:04.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.188+0000 7f9947f9a640 1 -- 172.21.15.5:0/2645251996 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9940107ec0 con 0x7f99401025e0 2024-09-12T02:58:04.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.189+0000 7f9936ffd640 1 -- 172.21.15.5:0/2645251996 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f9938005560 con 0x7f99401025e0 2024-09-12T02:58:04.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.190+0000 7f9936ffd640 1 --2- 172.21.15.5:0/2645251996 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f992c0a2b90 0x7f992c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:04.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.190+0000 7f9945d36640 1 --2- 172.21.15.5:0/2645251996 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f992c0a2b90 0x7f992c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:04.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.190+0000 7f9936ffd640 1 -- 172.21.15.5:0/2645251996 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f99380cf330 con 0x7f99401025e0 2024-09-12T02:58:04.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.191+0000 7f9945d36640 1 --2- 172.21.15.5:0/2645251996 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f992c0a2b90 0x7f992c0a5050 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9940101a80 tx=0x7f993c005d30 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:04.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.194+0000 7f9936ffd640 1 -- 172.21.15.5:0/2645251996 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9938016030 con 0x7f99401025e0 2024-09-12T02:58:04.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.370+0000 7f9947f9a640 1 -- 172.21.15.5:0/2645251996 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9940062e90 con 0x7f99401025e0 2024-09-12T02:58:04.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.371+0000 7f9936ffd640 1 -- 172.21.15.5:0/2645251996 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) ==== 93+0+10 (secure 0 0 0) 0x7f993809c2a0 con 0x7f99401025e0 2024-09-12T02:58:04.371 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-09-12T02:58:04.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.373+0000 7f9947f9a640 1 -- 172.21.15.5:0/2645251996 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f992c0a2b90 msgr2=0x7f992c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:04.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.373+0000 7f9947f9a640 1 --2- 172.21.15.5:0/2645251996 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f992c0a2b90 0x7f992c0a5050 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9940101a80 tx=0x7f993c005d30 comp rx=0 tx=0).stop 2024-09-12T02:58:04.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.373+0000 7f9947f9a640 1 -- 172.21.15.5:0/2645251996 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99401025e0 msgr2=0x7f994019e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:04.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.373+0000 7f9947f9a640 1 --2- 172.21.15.5:0/2645251996 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99401025e0 0x7f994019e360 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f993800eaa0 tx=0x7f993800ef60 comp rx=0 tx=0).stop 2024-09-12T02:58:04.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.373+0000 7f9947f9a640 1 -- 172.21.15.5:0/2645251996 shutdown_connections 2024-09-12T02:58:04.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.373+0000 7f9947f9a640 1 --2- 172.21.15.5:0/2645251996 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f992c0a2b90 0x7f992c0a5050 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:04.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.373+0000 7f9947f9a640 1 --2- 172.21.15.5:0/2645251996 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99401025e0 0x7f994019e360 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:04.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.373+0000 7f9947f9a640 1 --2- 172.21.15.5:0/2645251996 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9940101c20 0x7f99401972f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:04.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.373+0000 7f9947f9a640 1 --2- 172.21.15.5:0/2645251996 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9940100a30 0x7f9940196db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:04.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.373+0000 7f9947f9a640 1 -- 172.21.15.5:0/2645251996 >> 172.21.15.5:0/2645251996 conn(0x7f99400fc1b0 msgr2=0x7f9940105090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:04.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.373+0000 7f9947f9a640 1 -- 172.21.15.5:0/2645251996 shutdown_connections 2024-09-12T02:58:04.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.373+0000 7f9947f9a640 1 -- 172.21.15.5:0/2645251996 wait complete. 2024-09-12T02:58:04.385 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-12T02:58:04.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.492+0000 7fd43f00e640 1 -- 172.21.15.5:0/1878925001 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4381077f0 msgr2=0x7fd438109c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:04.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.492+0000 7fd43f00e640 1 --2- 172.21.15.5:0/1878925001 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4381077f0 0x7fd438109c60 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fd430009ca0 tx=0x7fd43002a8c0 comp rx=0 tx=0).stop 2024-09-12T02:58:04.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.492+0000 7fd43f00e640 1 -- 172.21.15.5:0/1878925001 shutdown_connections 2024-09-12T02:58:04.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.492+0000 7fd43f00e640 1 --2- 172.21.15.5:0/1878925001 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4381077f0 0x7fd438109c60 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:04.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.492+0000 7fd43f00e640 1 --2- 172.21.15.5:0/1878925001 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd438104ec0 0x7fd4381072b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:04.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.492+0000 7fd43f00e640 1 --2- 172.21.15.5:0/1878925001 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd438102590 0x7fd438104980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:04.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.492+0000 7fd43f00e640 1 -- 172.21.15.5:0/1878925001 >> 172.21.15.5:0/1878925001 conn(0x7fd4380fc190 msgr2=0x7fd4380fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:04.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.492+0000 7fd43f00e640 1 -- 172.21.15.5:0/1878925001 shutdown_connections 2024-09-12T02:58:04.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.492+0000 7fd43f00e640 1 -- 172.21.15.5:0/1878925001 wait complete. 2024-09-12T02:58:04.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.493+0000 7fd43f00e640 1 Processor -- start 2024-09-12T02:58:04.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.493+0000 7fd43f00e640 1 -- start start 2024-09-12T02:58:04.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.493+0000 7fd43f00e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd438102590 0x7fd438101f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:04.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.493+0000 7fd43f00e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd438104ec0 0x7fd438100540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:04.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.493+0000 7fd43f00e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4381077f0 0x7fd438100a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:04.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.493+0000 7fd43f00e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd43810d840 con 0x7fd4381077f0 2024-09-12T02:58:04.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.493+0000 7fd43f00e640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fd43810d6c0 con 0x7fd438102590 2024-09-12T02:58:04.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.493+0000 7fd43f00e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd43810d9c0 con 0x7fd438104ec0 2024-09-12T02:58:04.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.493+0000 7fd43d5ab640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4381077f0 0x7fd438100a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:04.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.493+0000 7fd43d5ab640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4381077f0 0x7fd438100a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55382/0 (socket says 172.21.15.5:55382) 2024-09-12T02:58:04.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.493+0000 7fd43d5ab640 1 -- 172.21.15.5:0/391773220 learned_addr learned my addr 172.21.15.5:0/391773220 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:58:04.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.493+0000 7fd437fff640 1 --2- 172.21.15.5:0/391773220 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd438104ec0 0x7fd438100540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:04.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.493+0000 7fd43cdaa640 1 --2- 172.21.15.5:0/391773220 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd438102590 0x7fd438101f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:04.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.493+0000 7fd43d5ab640 1 -- 172.21.15.5:0/391773220 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd438104ec0 msgr2=0x7fd438100540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:04.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.493+0000 7fd43d5ab640 1 --2- 172.21.15.5:0/391773220 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd438104ec0 0x7fd438100540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:04.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.493+0000 7fd43d5ab640 1 -- 172.21.15.5:0/391773220 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd438102590 msgr2=0x7fd438101f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:04.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.493+0000 7fd43d5ab640 1 --2- 172.21.15.5:0/391773220 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd438102590 0x7fd438101f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:04.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.493+0000 7fd43d5ab640 1 -- 172.21.15.5:0/391773220 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd438101310 con 0x7fd4381077f0 2024-09-12T02:58:04.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.494+0000 7fd43d5ab640 1 --2- 172.21.15.5:0/391773220 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4381077f0 0x7fd438100a80 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fd430031a40 tx=0x7fd430031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:04.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.494+0000 7fd435ffb640 1 -- 172.21.15.5:0/391773220 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd430003b70 con 0x7fd4381077f0 2024-09-12T02:58:04.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.494+0000 7fd435ffb640 1 -- 172.21.15.5:0/391773220 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd430003d10 con 0x7fd4381077f0 2024-09-12T02:58:04.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.494+0000 7fd43f00e640 1 -- 172.21.15.5:0/391773220 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4381a1fe0 con 0x7fd4381077f0 2024-09-12T02:58:04.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.494+0000 7fd435ffb640 1 -- 172.21.15.5:0/391773220 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd43002fca0 con 0x7fd4381077f0 2024-09-12T02:58:04.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.494+0000 7fd43f00e640 1 -- 172.21.15.5:0/391773220 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4381a2520 con 0x7fd4381077f0 2024-09-12T02:58:04.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.495+0000 7fd43f00e640 1 -- 172.21.15.5:0/391773220 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4380fdc80 con 0x7fd4381077f0 2024-09-12T02:58:04.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.496+0000 7fd435ffb640 1 -- 172.21.15.5:0/391773220 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7fd430004050 con 0x7fd4381077f0 2024-09-12T02:58:04.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.497+0000 7fd435ffb640 1 --2- 172.21.15.5:0/391773220 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fd4280a2b90 0x7fd4280a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:04.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.497+0000 7fd435ffb640 1 -- 172.21.15.5:0/391773220 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fd4300f02e0 con 0x7fd4381077f0 2024-09-12T02:58:04.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.497+0000 7fd43cdaa640 1 --2- 172.21.15.5:0/391773220 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fd4280a2b90 0x7fd4280a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:04.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.498+0000 7fd43cdaa640 1 --2- 172.21.15.5:0/391773220 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fd4280a2b90 0x7fd4280a5050 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd42c005fd0 tx=0x7fd42c005850 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:04.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.501+0000 7fd435ffb640 1 -- 172.21.15.5:0/391773220 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd4300bd310 con 0x7fd4381077f0 2024-09-12T02:58:04.597 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:58:04 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/2958700201' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-12T02:58:04.597 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:58:04 smithi005 ceph-mon[31197]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T02:58:04.597 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:58:04 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/475689725' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:58:04.665 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:58:04 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/2958700201' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-12T02:58:04.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:58:04 smithi148 ceph-mon[35090]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T02:58:04.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:58:04 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/475689725' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:58:04.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.678+0000 7fd43f00e640 1 -- 172.21.15.5:0/391773220 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd438062df0 con 0x7fd4381077f0 2024-09-12T02:58:04.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.679+0000 7fd435ffb640 1 -- 172.21.15.5:0/391773220 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v83) ==== 104+0+11 (secure 0 0 0) 0x7fd4300bcc10 con 0x7fd4381077f0 2024-09-12T02:58:04.679 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-09-12T02:58:04.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.681+0000 7fd43f00e640 1 -- 172.21.15.5:0/391773220 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fd4280a2b90 msgr2=0x7fd4280a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:04.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.681+0000 7fd43f00e640 1 --2- 172.21.15.5:0/391773220 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fd4280a2b90 0x7fd4280a5050 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd42c005fd0 tx=0x7fd42c005850 comp rx=0 tx=0).stop 2024-09-12T02:58:04.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.681+0000 7fd43f00e640 1 -- 172.21.15.5:0/391773220 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4381077f0 msgr2=0x7fd438100a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:04.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.681+0000 7fd43f00e640 1 --2- 172.21.15.5:0/391773220 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4381077f0 0x7fd438100a80 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fd430031a40 tx=0x7fd430031ee0 comp rx=0 tx=0).stop 2024-09-12T02:58:04.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.681+0000 7fd43f00e640 1 -- 172.21.15.5:0/391773220 shutdown_connections 2024-09-12T02:58:04.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.681+0000 7fd43f00e640 1 --2- 172.21.15.5:0/391773220 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fd4280a2b90 0x7fd4280a5050 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:04.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.681+0000 7fd43f00e640 1 --2- 172.21.15.5:0/391773220 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4381077f0 0x7fd438100a80 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:04.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.681+0000 7fd43f00e640 1 --2- 172.21.15.5:0/391773220 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd438104ec0 0x7fd438100540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:04.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.681+0000 7fd43f00e640 1 --2- 172.21.15.5:0/391773220 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd438102590 0x7fd438101f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:04.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.681+0000 7fd43f00e640 1 -- 172.21.15.5:0/391773220 >> 172.21.15.5:0/391773220 conn(0x7fd4380fc190 msgr2=0x7fd438102f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:04.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.681+0000 7fd43f00e640 1 -- 172.21.15.5:0/391773220 shutdown_connections 2024-09-12T02:58:04.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.681+0000 7fd43f00e640 1 -- 172.21.15.5:0/391773220 wait complete. 2024-09-12T02:58:04.692 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-12T02:58:04.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:58:04 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/2958700201' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-12T02:58:04.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:58:04 smithi146 ceph-mon[35011]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T02:58:04.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:58:04 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/475689725' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:58:04.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.798+0000 7f758d6e4640 1 -- 172.21.15.5:0/1434775668 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7588100cf0 msgr2=0x7f7588101170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:04.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.798+0000 7f758d6e4640 1 --2- 172.21.15.5:0/1434775668 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7588100cf0 0x7f7588101170 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f757c009900 tx=0x7f757c02a410 comp rx=0 tx=0).stop 2024-09-12T02:58:04.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.798+0000 7f758d6e4640 1 -- 172.21.15.5:0/1434775668 shutdown_connections 2024-09-12T02:58:04.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.798+0000 7f758d6e4640 1 --2- 172.21.15.5:0/1434775668 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7588100cf0 0x7f7588101170 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:04.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.798+0000 7f758d6e4640 1 --2- 172.21.15.5:0/1434775668 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f75881016b0 0x7f7588108f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:04.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.798+0000 7f758d6e4640 1 --2- 172.21.15.5:0/1434775668 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7588101fc0 0x7f75881023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:04.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.798+0000 7f758d6e4640 1 -- 172.21.15.5:0/1434775668 >> 172.21.15.5:0/1434775668 conn(0x7f75880fb950 msgr2=0x7f75880fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:04.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.798+0000 7f758d6e4640 1 -- 172.21.15.5:0/1434775668 shutdown_connections 2024-09-12T02:58:04.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.798+0000 7f758d6e4640 1 -- 172.21.15.5:0/1434775668 wait complete. 2024-09-12T02:58:04.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.799+0000 7f758d6e4640 1 Processor -- start 2024-09-12T02:58:04.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.799+0000 7f758d6e4640 1 -- start start 2024-09-12T02:58:04.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.799+0000 7f758d6e4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7588100cf0 0x7f7588100750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:04.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.799+0000 7f758d6e4640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f75881016b0 0x7f758819cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:04.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.799+0000 7f758d6e4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7588101fc0 0x7f758819f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:04.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.799+0000 7f758d6e4640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f758810c6a0 con 0x7f7588100cf0 2024-09-12T02:58:04.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.799+0000 7f758d6e4640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f758810c520 con 0x7f75881016b0 2024-09-12T02:58:04.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.799+0000 7f758d6e4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f758810c820 con 0x7f7588101fc0 2024-09-12T02:58:04.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.799+0000 7f7586ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7588100cf0 0x7f7588100750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:04.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.799+0000 7f7586ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7588100cf0 0x7f7588100750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55400/0 (socket says 172.21.15.5:55400) 2024-09-12T02:58:04.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.799+0000 7f7586ffd640 1 -- 172.21.15.5:0/2879090111 learned_addr learned my addr 172.21.15.5:0/2879090111 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:58:04.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.800+0000 7f7586ffd640 1 -- 172.21.15.5:0/2879090111 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7588101fc0 msgr2=0x7f758819f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:04.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.800+0000 7f75877fe640 1 --2- 172.21.15.5:0/2879090111 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7588101fc0 0x7f758819f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:04.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.800+0000 7f75867fc640 1 --2- 172.21.15.5:0/2879090111 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f75881016b0 0x7f758819cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:04.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.800+0000 7f7586ffd640 1 --2- 172.21.15.5:0/2879090111 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7588101fc0 0x7f758819f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:04.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.800+0000 7f7586ffd640 1 -- 172.21.15.5:0/2879090111 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f75881016b0 msgr2=0x7f758819cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:04.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.800+0000 7f7586ffd640 1 --2- 172.21.15.5:0/2879090111 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f75881016b0 0x7f758819cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:04.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.800+0000 7f7586ffd640 1 -- 172.21.15.5:0/2879090111 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f758819fad0 con 0x7f7588100cf0 2024-09-12T02:58:04.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.800+0000 7f75867fc640 1 --2- 172.21.15.5:0/2879090111 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f75881016b0 0x7f758819cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:58:04.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.800+0000 7f7586ffd640 1 --2- 172.21.15.5:0/2879090111 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7588100cf0 0x7f7588100750 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f757800c970 tx=0x7f757800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:04.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.800+0000 7f756ffff640 1 -- 172.21.15.5:0/2879090111 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7578007c10 con 0x7f7588100cf0 2024-09-12T02:58:04.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.800+0000 7f758d6e4640 1 -- 172.21.15.5:0/2879090111 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f758819fd60 con 0x7f7588100cf0 2024-09-12T02:58:04.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.800+0000 7f756ffff640 1 -- 172.21.15.5:0/2879090111 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7578007db0 con 0x7f7588100cf0 2024-09-12T02:58:04.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.800+0000 7f756ffff640 1 -- 172.21.15.5:0/2879090111 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f757800f8a0 con 0x7f7588100cf0 2024-09-12T02:58:04.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.800+0000 7f758d6e4640 1 -- 172.21.15.5:0/2879090111 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75881a0220 con 0x7f7588100cf0 2024-09-12T02:58:04.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.801+0000 7f758d6e4640 1 -- 172.21.15.5:0/2879090111 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7588107f20 con 0x7f7588100cf0 2024-09-12T02:58:04.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.802+0000 7f756ffff640 1 -- 172.21.15.5:0/2879090111 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f757800fb00 con 0x7f7588100cf0 2024-09-12T02:58:04.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.803+0000 7f756ffff640 1 --2- 172.21.15.5:0/2879090111 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f75700a2b90 0x7f75700a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:04.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.803+0000 7f75867fc640 1 --2- 172.21.15.5:0/2879090111 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f75700a2b90 0x7f75700a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:04.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.803+0000 7f756ffff640 1 -- 172.21.15.5:0/2879090111 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f75780cf400 con 0x7f7588100cf0 2024-09-12T02:58:04.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.803+0000 7f75867fc640 1 --2- 172.21.15.5:0/2879090111 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f75700a2b90 0x7f75700a5050 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f757c000c00 tx=0x7f757c009560 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:04.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.807+0000 7f756ffff640 1 -- 172.21.15.5:0/2879090111 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f757809c400 con 0x7f7588100cf0 2024-09-12T02:58:04.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.982+0000 7f758d6e4640 1 -- 172.21.15.5:0/2879090111 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f7588062f30 con 0x7f7588100cf0 2024-09-12T02:58:04.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.982+0000 7f756ffff640 1 -- 172.21.15.5:0/2879090111 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v83) ==== 100+0+11 (secure 0 0 0) 0x7f757809bd00 con 0x7f7588100cf0 2024-09-12T02:58:04.983 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-09-12T02:58:04.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.984+0000 7f758d6e4640 1 -- 172.21.15.5:0/2879090111 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f75700a2b90 msgr2=0x7f75700a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:04.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.984+0000 7f758d6e4640 1 --2- 172.21.15.5:0/2879090111 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f75700a2b90 0x7f75700a5050 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f757c000c00 tx=0x7f757c009560 comp rx=0 tx=0).stop 2024-09-12T02:58:04.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.984+0000 7f758d6e4640 1 -- 172.21.15.5:0/2879090111 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7588100cf0 msgr2=0x7f7588100750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:04.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.984+0000 7f758d6e4640 1 --2- 172.21.15.5:0/2879090111 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7588100cf0 0x7f7588100750 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f757800c970 tx=0x7f757800ce30 comp rx=0 tx=0).stop 2024-09-12T02:58:04.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.984+0000 7f758d6e4640 1 -- 172.21.15.5:0/2879090111 shutdown_connections 2024-09-12T02:58:04.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.984+0000 7f758d6e4640 1 --2- 172.21.15.5:0/2879090111 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7588101fc0 0x7f758819f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:04.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.984+0000 7f758d6e4640 1 --2- 172.21.15.5:0/2879090111 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f75881016b0 0x7f758819cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:04.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.984+0000 7f758d6e4640 1 --2- 172.21.15.5:0/2879090111 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f75700a2b90 0x7f75700a5050 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:04.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.984+0000 7f758d6e4640 1 --2- 172.21.15.5:0/2879090111 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7588100cf0 0x7f7588100750 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:04.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.984+0000 7f758d6e4640 1 -- 172.21.15.5:0/2879090111 >> 172.21.15.5:0/2879090111 conn(0x7f75880fb950 msgr2=0x7f75881050f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:04.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.984+0000 7f758d6e4640 1 -- 172.21.15.5:0/2879090111 shutdown_connections 2024-09-12T02:58:04.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:04.984+0000 7f758d6e4640 1 -- 172.21.15.5:0/2879090111 wait complete. 2024-09-12T02:58:04.996 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-09-12T02:58:04.996 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:58:05.255 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:58:05.627 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:58:05 smithi005 ceph-mon[31197]: pgmap v330: 97 pgs: 97 active+clean; 649 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-09-12T02:58:05.627 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:58:05 smithi005 ceph-mon[31197]: from='client.15216 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:58:05.627 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:58:05 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/2645251996' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-12T02:58:05.627 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:58:05 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/391773220' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-12T02:58:05.627 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:58:05 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/2879090111' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-12T02:58:05.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:58:05 smithi148 ceph-mon[35090]: pgmap v330: 97 pgs: 97 active+clean; 649 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-09-12T02:58:05.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:58:05 smithi148 ceph-mon[35090]: from='client.15216 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:58:05.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:58:05 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/2645251996' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-12T02:58:05.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:58:05 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/391773220' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-12T02:58:05.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:58:05 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/2879090111' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-12T02:58:05.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:58:05 smithi146 ceph-mon[35011]: pgmap v330: 97 pgs: 97 active+clean; 649 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.1 KiB/s wr, 11 op/s 2024-09-12T02:58:05.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:58:05 smithi146 ceph-mon[35011]: from='client.15216 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-09-12T02:58:05.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:58:05 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/2645251996' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-12T02:58:05.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:58:05 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/391773220' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-12T02:58:05.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:58:05 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/2879090111' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-12T02:58:05.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:05.711+0000 7f28da517640 1 -- 172.21.15.5:0/2317112504 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28d4102de0 msgr2=0x7f28d41031c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:05.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:05.711+0000 7f28da517640 1 --2- 172.21.15.5:0/2317112504 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28d4102de0 0x7f28d41031c0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f28c4009900 tx=0x7f28c402a4a0 comp rx=0 tx=0).stop 2024-09-12T02:58:05.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:05.711+0000 7f28da517640 1 -- 172.21.15.5:0/2317112504 shutdown_connections 2024-09-12T02:58:05.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:05.711+0000 7f28da517640 1 --2- 172.21.15.5:0/2317112504 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28d410c180 0x7f28d410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:05.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:05.711+0000 7f28da517640 1 --2- 172.21.15.5:0/2317112504 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f28d4103700 0x7f28d410bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:05.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:05.711+0000 7f28da517640 1 --2- 172.21.15.5:0/2317112504 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28d4102de0 0x7f28d41031c0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:05.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:05.711+0000 7f28da517640 1 -- 172.21.15.5:0/2317112504 >> 172.21.15.5:0/2317112504 conn(0x7f28d40fc7a0 msgr2=0x7f28d40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:05.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:05.711+0000 7f28da517640 1 -- 172.21.15.5:0/2317112504 shutdown_connections 2024-09-12T02:58:05.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:05.711+0000 7f28da517640 1 -- 172.21.15.5:0/2317112504 wait complete. 2024-09-12T02:58:05.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:05.712+0000 7f28da517640 1 Processor -- start 2024-09-12T02:58:05.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:05.712+0000 7f28da517640 1 -- start start 2024-09-12T02:58:05.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:05.712+0000 7f28da517640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28d4102de0 0x7f28d41985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:05.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:05.712+0000 7f28da517640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f28d4103700 0x7f28d4198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:05.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:05.712+0000 7f28da517640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28d410c180 0x7f28d419cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:05.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:05.712+0000 7f28da517640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f28d41120a0 con 0x7f28d4102de0 2024-09-12T02:58:05.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:05.712+0000 7f28da517640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f28d4111f20 con 0x7f28d4103700 2024-09-12T02:58:05.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:05.712+0000 7f28da517640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f28d4112220 con 0x7f28d410c180 2024-09-12T02:58:05.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:05.712+0000 7f28d3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28d4102de0 0x7f28d41985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:05.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:05.712+0000 7f28d3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28d4102de0 0x7f28d41985d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55428/0 (socket says 172.21.15.5:55428) 2024-09-12T02:58:05.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:05.712+0000 7f28d3fff640 1 -- 172.21.15.5:0/1221028510 learned_addr learned my addr 172.21.15.5:0/1221028510 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:58:05.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:05.713+0000 7f28d8ab4640 1 --2- 172.21.15.5:0/1221028510 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28d410c180 0x7f28d419cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:05.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:05.713+0000 7f28d37fe640 1 --2- 172.21.15.5:0/1221028510 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f28d4103700 0x7f28d4198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:05.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:05.713+0000 7f28d3fff640 1 -- 172.21.15.5:0/1221028510 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28d410c180 msgr2=0x7f28d419cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:05.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:05.713+0000 7f28d3fff640 1 --2- 172.21.15.5:0/1221028510 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28d410c180 0x7f28d419cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:05.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:05.713+0000 7f28d3fff640 1 -- 172.21.15.5:0/1221028510 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f28d4103700 msgr2=0x7f28d4198b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:05.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:05.713+0000 7f28d3fff640 1 --2- 172.21.15.5:0/1221028510 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f28d4103700 0x7f28d4198b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:05.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:05.713+0000 7f28d3fff640 1 -- 172.21.15.5:0/1221028510 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28d419d580 con 0x7f28d4102de0 2024-09-12T02:58:05.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:05.713+0000 7f28d8ab4640 1 --2- 172.21.15.5:0/1221028510 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28d410c180 0x7f28d419cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:58:05.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:05.713+0000 7f28d3fff640 1 --2- 172.21.15.5:0/1221028510 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28d4102de0 0x7f28d41985d0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f28c402aa90 tx=0x7f28c4003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:05.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:05.713+0000 7f28d17fa640 1 -- 172.21.15.5:0/1221028510 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c403f070 con 0x7f28d4102de0 2024-09-12T02:58:05.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:05.713+0000 7f28d17fa640 1 -- 172.21.15.5:0/1221028510 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28c403b460 con 0x7f28d4102de0 2024-09-12T02:58:05.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:05.713+0000 7f28da517640 1 -- 172.21.15.5:0/1221028510 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28d419d810 con 0x7f28d4102de0 2024-09-12T02:58:05.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:05.713+0000 7f28d17fa640 1 -- 172.21.15.5:0/1221028510 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c4035030 con 0x7f28d4102de0 2024-09-12T02:58:05.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:05.713+0000 7f28da517640 1 -- 172.21.15.5:0/1221028510 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28d41a5150 con 0x7f28d4102de0 2024-09-12T02:58:05.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:05.714+0000 7f28da517640 1 -- 172.21.15.5:0/1221028510 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28d4103240 con 0x7f28d4102de0 2024-09-12T02:58:05.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:05.715+0000 7f28d17fa640 1 -- 172.21.15.5:0/1221028510 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) ==== 137832+0+0 (secure 0 0 0) 0x7f28c4042070 con 0x7f28d4102de0 2024-09-12T02:58:05.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:05.716+0000 7f28d17fa640 1 --2- 172.21.15.5:0/1221028510 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f28bc0a2b80 0x7f28bc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:05.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:05.716+0000 7f28d17fa640 1 -- 172.21.15.5:0/1221028510 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f28c40efb30 con 0x7f28d4102de0 2024-09-12T02:58:05.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:05.716+0000 7f28d37fe640 1 --2- 172.21.15.5:0/1221028510 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f28bc0a2b80 0x7f28bc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:05.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:05.716+0000 7f28d37fe640 1 --2- 172.21.15.5:0/1221028510 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f28bc0a2b80 0x7f28bc0a5040 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f28d40fee10 tx=0x7f28c8008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:05.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:05.720+0000 7f28d17fa640 1 -- 172.21.15.5:0/1221028510 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f28c40bcaa0 con 0x7f28d4102de0 2024-09-12T02:58:05.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:05.934+0000 7f28da517640 1 -- 172.21.15.5:0/1221028510 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f28d4107240 con 0x7f28d4102de0 2024-09-12T02:58:05.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:05.935+0000 7f28d17fa640 1 -- 172.21.15.5:0/1221028510 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11966 (secure 0 0 0) 0x7f28c40bc3a0 con 0x7f28d4102de0 2024-09-12T02:58:05.935 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-12T02:58:05.935 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":26,"btime":"2024-09-12T02:58:03:323671+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/4135232521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":4135232521},{"type":"v1","addr":"172.21.15.5:6835","nonce":4135232521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/4290221940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":4290221940},{"type":"v1","addr":"172.21.15.5:6837","nonce":4290221940}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T02:56:16.595562+0000","modified":"2024-09-12T02:58:02.463400+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24502,"mds_1":24487,"mds_2":24481,"mds_3":24479,"mds_4":24458},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/26447341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":26447341},{"type":"v1","addr":"172.21.15.5:6839","nonce":26447341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2393443295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2393443295},{"type":"v1","addr":"172.21.15.5:6841","nonce":2393443295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6833/3686970137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3686970137},{"type":"v1","addr":"172.21.15.146:6833","nonce":3686970137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6833/568512209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":568512209},{"type":"v1","addr":"172.21.15.148:6833","nonce":568512209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6835/2540740420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2540740420},{"type":"v1","addr":"172.21.15.146:6835","nonce":2540740420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.146:6837/1673370937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":1673370937},{"type":"v1","addr":"172.21.15.146:6837","nonce":1673370937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":3,"incarnation":22,"state":"up:active","state_seq":6,"addr":"172.21.15.146:6839/3036767228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":3036767228},{"type":"v1","addr":"172.21.15.146:6839","nonce":3036767228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":2,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.148:6835/4195086427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":4195086427},{"type":"v1","addr":"172.21.15.148:6835","nonce":4195086427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.148:6837/2606873922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2606873922},{"type":"v1","addr":"172.21.15.148:6837","nonce":2606873922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6839/2317600000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":2317600000},{"type":"v1","addr":"172.21.15.148:6839","nonce":2317600000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24502,"qdb_cluster":[24502,24487,24481,24479,24458]},"id":1}]} 2024-09-12T02:58:05.936 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 26 2024-09-12T02:58:05.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:05.937+0000 7f28da517640 1 -- 172.21.15.5:0/1221028510 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f28bc0a2b80 msgr2=0x7f28bc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:05.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:05.937+0000 7f28da517640 1 --2- 172.21.15.5:0/1221028510 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f28bc0a2b80 0x7f28bc0a5040 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f28d40fee10 tx=0x7f28c8008040 comp rx=0 tx=0).stop 2024-09-12T02:58:05.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:05.937+0000 7f28da517640 1 -- 172.21.15.5:0/1221028510 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28d4102de0 msgr2=0x7f28d41985d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:05.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:05.937+0000 7f28da517640 1 --2- 172.21.15.5:0/1221028510 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28d4102de0 0x7f28d41985d0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f28c402aa90 tx=0x7f28c4003d50 comp rx=0 tx=0).stop 2024-09-12T02:58:05.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:05.937+0000 7f28da517640 1 -- 172.21.15.5:0/1221028510 shutdown_connections 2024-09-12T02:58:05.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:05.937+0000 7f28da517640 1 --2- 172.21.15.5:0/1221028510 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28d410c180 0x7f28d419cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:05.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:05.937+0000 7f28da517640 1 --2- 172.21.15.5:0/1221028510 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f28d4103700 0x7f28d4198b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:05.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:05.937+0000 7f28da517640 1 --2- 172.21.15.5:0/1221028510 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f28bc0a2b80 0x7f28bc0a5040 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:05.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:05.937+0000 7f28da517640 1 --2- 172.21.15.5:0/1221028510 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28d4102de0 0x7f28d41985d0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:05.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:05.937+0000 7f28da517640 1 -- 172.21.15.5:0/1221028510 >> 172.21.15.5:0/1221028510 conn(0x7f28d40fc7a0 msgr2=0x7f28d40fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:05.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:05.937+0000 7f28da517640 1 -- 172.21.15.5:0/1221028510 shutdown_connections 2024-09-12T02:58:05.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:05.937+0000 7f28da517640 1 -- 172.21.15.5:0/1221028510 wait complete. 2024-09-12T02:58:06.545 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-09-12T02:58:06.545 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-09-12T02:58:06.546 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T02:58:06.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:06.653+0000 7fc8eec37640 1 -- 172.21.15.5:0/3571395554 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc8e81025c0 msgr2=0x7fc8e8108e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:06.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:06.653+0000 7fc8eec37640 1 --2- 172.21.15.5:0/3571395554 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc8e81025c0 0x7fc8e8108e50 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fc8d0009900 tx=0x7fc8d002a410 comp rx=0 tx=0).stop 2024-09-12T02:58:06.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:06.653+0000 7fc8eec37640 1 -- 172.21.15.5:0/3571395554 shutdown_connections 2024-09-12T02:58:06.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:06.653+0000 7fc8eec37640 1 --2- 172.21.15.5:0/3571395554 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc8e81025c0 0x7fc8e8108e50 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:06.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:06.653+0000 7fc8eec37640 1 --2- 172.21.15.5:0/3571395554 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc8e8101c00 0x7fc8e8102080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:06.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:06.653+0000 7fc8eec37640 1 --2- 172.21.15.5:0/3571395554 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc8e8100a10 0x7fc8e8100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:06.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:06.653+0000 7fc8eec37640 1 -- 172.21.15.5:0/3571395554 >> 172.21.15.5:0/3571395554 conn(0x7fc8e80fc170 msgr2=0x7fc8e80fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:06.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:06.653+0000 7fc8eec37640 1 -- 172.21.15.5:0/3571395554 shutdown_connections 2024-09-12T02:58:06.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:06.653+0000 7fc8eec37640 1 -- 172.21.15.5:0/3571395554 wait complete. 2024-09-12T02:58:06.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:06.653+0000 7fc8eec37640 1 Processor -- start 2024-09-12T02:58:06.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:06.654+0000 7fc8eec37640 1 -- start start 2024-09-12T02:58:06.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:06.654+0000 7fc8eec37640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc8e8100a10 0x7fc8e81049a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:06.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:06.654+0000 7fc8eec37640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc8e8101c00 0x7fc8e8102fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:06.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:06.654+0000 7fc8eec37640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc8e81025c0 0x7fc8e8103520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:06.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:06.654+0000 7fc8eec37640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e810c6c0 con 0x7fc8e8101c00 2024-09-12T02:58:06.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:06.654+0000 7fc8eec37640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e810c540 con 0x7fc8e8100a10 2024-09-12T02:58:06.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:06.654+0000 7fc8eec37640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e810c840 con 0x7fc8e81025c0 2024-09-12T02:58:06.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:06.654+0000 7fc8e7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc8e8101c00 0x7fc8e8102fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:06.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:06.654+0000 7fc8e7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc8e8101c00 0x7fc8e8102fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55440/0 (socket says 172.21.15.5:55440) 2024-09-12T02:58:06.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:06.654+0000 7fc8e7fff640 1 -- 172.21.15.5:0/2622585665 learned_addr learned my addr 172.21.15.5:0/2622585665 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:58:06.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:06.654+0000 7fc8ed1d4640 1 --2- 172.21.15.5:0/2622585665 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc8e81025c0 0x7fc8e8103520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:06.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:06.654+0000 7fc8ec9d3640 1 --2- 172.21.15.5:0/2622585665 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc8e8100a10 0x7fc8e81049a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:06.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:06.654+0000 7fc8e7fff640 1 -- 172.21.15.5:0/2622585665 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc8e81025c0 msgr2=0x7fc8e8103520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:06.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:06.654+0000 7fc8e7fff640 1 --2- 172.21.15.5:0/2622585665 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc8e81025c0 0x7fc8e8103520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:06.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:06.654+0000 7fc8e7fff640 1 -- 172.21.15.5:0/2622585665 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc8e8100a10 msgr2=0x7fc8e81049a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:06.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:06.654+0000 7fc8e7fff640 1 --2- 172.21.15.5:0/2622585665 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc8e8100a10 0x7fc8e81049a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:06.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:06.654+0000 7fc8e7fff640 1 -- 172.21.15.5:0/2622585665 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8e8103db0 con 0x7fc8e8101c00 2024-09-12T02:58:06.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:06.654+0000 7fc8ec9d3640 1 --2- 172.21.15.5:0/2622585665 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc8e8100a10 0x7fc8e81049a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:58:06.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:06.654+0000 7fc8e7fff640 1 --2- 172.21.15.5:0/2622585665 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc8e8101c00 0x7fc8e8102fe0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fc8d800d950 tx=0x7fc8d800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:06.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:06.655+0000 7fc8e5ffb640 1 -- 172.21.15.5:0/2622585665 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d8017070 con 0x7fc8e8101c00 2024-09-12T02:58:06.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:06.655+0000 7fc8e5ffb640 1 -- 172.21.15.5:0/2622585665 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8d8009c30 con 0x7fc8e8101c00 2024-09-12T02:58:06.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:06.655+0000 7fc8eec37640 1 -- 172.21.15.5:0/2622585665 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8e81a1f80 con 0x7fc8e8101c00 2024-09-12T02:58:06.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:06.655+0000 7fc8e5ffb640 1 -- 172.21.15.5:0/2622585665 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d800f6b0 con 0x7fc8e8101c00 2024-09-12T02:58:06.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:06.655+0000 7fc8eec37640 1 -- 172.21.15.5:0/2622585665 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8e81a23e0 con 0x7fc8e8101c00 2024-09-12T02:58:06.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:06.656+0000 7fc8eec37640 1 -- 172.21.15.5:0/2622585665 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8e81a26d0 con 0x7fc8e8101c00 2024-09-12T02:58:06.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:06.657+0000 7fc8e5ffb640 1 -- 172.21.15.5:0/2622585665 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7fc8d80109d0 con 0x7fc8e8101c00 2024-09-12T02:58:06.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:06.657+0000 7fc8e5ffb640 1 --2- 172.21.15.5:0/2622585665 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fc8d40a2c20 0x7fc8d40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:06.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:06.658+0000 7fc8ec9d3640 1 --2- 172.21.15.5:0/2622585665 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fc8d40a2c20 0x7fc8d40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:06.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:06.658+0000 7fc8e5ffb640 1 -- 172.21.15.5:0/2622585665 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fc8d80cecd0 con 0x7fc8e8101c00 2024-09-12T02:58:06.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:06.658+0000 7fc8ec9d3640 1 --2- 172.21.15.5:0/2622585665 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fc8d40a2c20 0x7fc8d40a50e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc8e8101a60 tx=0x7fc8e00095c0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:06.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:06.662+0000 7fc8e5ffb640 1 -- 172.21.15.5:0/2622585665 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc8d8009de0 con 0x7fc8e8101c00 2024-09-12T02:58:06.665 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:58:06 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/1221028510' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:58:06.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:58:06 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/1221028510' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:58:06.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:58:06 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/1221028510' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:58:06.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:06.835+0000 7fc8eec37640 1 -- 172.21.15.5:0/2622585665 --> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc8e80fe380 con 0x7fc8d40a2c20 2024-09-12T02:58:06.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:06.842+0000 7fc8e5ffb640 1 -- 172.21.15.5:0/2622585665 <== mgr.14148 v2:172.21.15.5:6800/3780631642 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259021 (secure 0 0 0) 0x7fc8e80fe380 con 0x7fc8d40a2c20 2024-09-12T02:58:06.844 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-12T02:58:06.846 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-09-12T02:58:06.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:06.844+0000 7fc8eec37640 1 -- 172.21.15.5:0/2622585665 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fc8d40a2c20 msgr2=0x7fc8d40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:06.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:06.844+0000 7fc8eec37640 1 --2- 172.21.15.5:0/2622585665 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fc8d40a2c20 0x7fc8d40a50e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc8e8101a60 tx=0x7fc8e00095c0 comp rx=0 tx=0).stop 2024-09-12T02:58:06.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:06.844+0000 7fc8eec37640 1 -- 172.21.15.5:0/2622585665 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc8e8101c00 msgr2=0x7fc8e8102fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:06.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:06.844+0000 7fc8eec37640 1 --2- 172.21.15.5:0/2622585665 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc8e8101c00 0x7fc8e8102fe0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fc8d800d950 tx=0x7fc8d800de10 comp rx=0 tx=0).stop 2024-09-12T02:58:06.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:06.845+0000 7fc8eec37640 1 -- 172.21.15.5:0/2622585665 shutdown_connections 2024-09-12T02:58:06.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:06.845+0000 7fc8eec37640 1 --2- 172.21.15.5:0/2622585665 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fc8d40a2c20 0x7fc8d40a50e0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:06.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:06.845+0000 7fc8eec37640 1 --2- 172.21.15.5:0/2622585665 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc8e8101c00 0x7fc8e8102fe0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:06.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:06.845+0000 7fc8eec37640 1 --2- 172.21.15.5:0/2622585665 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc8e81025c0 0x7fc8e8103520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:06.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:06.845+0000 7fc8eec37640 1 --2- 172.21.15.5:0/2622585665 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc8e8100a10 0x7fc8e81049a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:06.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:06.845+0000 7fc8eec37640 1 -- 172.21.15.5:0/2622585665 >> 172.21.15.5:0/2622585665 conn(0x7fc8e80fc170 msgr2=0x7fc8e80fdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:06.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:06.845+0000 7fc8eec37640 1 -- 172.21.15.5:0/2622585665 shutdown_connections 2024-09-12T02:58:06.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:06.845+0000 7fc8eec37640 1 -- 172.21.15.5:0/2622585665 wait complete. 2024-09-12T02:58:06.856 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":331,"stamp":"2024-09-12T02:58:05.568647+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":664831,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1126,"num_read_kb":689,"num_write":449,"num_write_kb":3094,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":1181470,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":866,"ondisk_log_size":866,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":374556,"kb_used_data":12828,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124567268,"statfs":{"total":1151940427776,"available":1151556882432,"internally_reserved":0,"allocated":13135872,"data_stored":7249552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":370193723},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5854,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":120,"num_read_kb":66,"num_write":20,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002569"},"pg_stats":[{"pgid":"3.3f","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.565050+0000","last_change":"2024-09-12T02:56:15.988690+0000","last_active":"2024-09-12T02:57:59.565050+0000","last_peered":"2024-09-12T02:57:59.565050+0000","last_clean":"2024-09-12T02:57:59.565050+0000","last_became_active":"2024-09-12T02:56:15.988320+0000","last_became_peered":"2024-09-12T02:56:15.988320+0000","last_unstale":"2024-09-12T02:57:59.565050+0000","last_undegraded":"2024-09-12T02:57:59.565050+0000","last_fullsized":"2024-09-12T02:57:59.565050+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:31:14.157622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"83'4","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.553842+0000","last_change":"2024-09-12T02:56:15.978368+0000","last_active":"2024-09-12T02:57:59.553842+0000","last_peered":"2024-09-12T02:57:59.553842+0000","last_clean":"2024-09-12T02:57:59.553842+0000","last_became_active":"2024-09-12T02:56:15.978092+0000","last_became_peered":"2024-09-12T02:56:15.978092+0000","last_unstale":"2024-09-12T02:57:59.553842+0000","last_undegraded":"2024-09-12T02:57:59.553842+0000","last_fullsized":"2024-09-12T02:57:59.553842+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:32:48.399005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.564003+0000","last_change":"2024-09-12T02:56:15.987249+0000","last_active":"2024-09-12T02:57:59.564003+0000","last_peered":"2024-09-12T02:57:59.564003+0000","last_clean":"2024-09-12T02:57:59.564003+0000","last_became_active":"2024-09-12T02:56:15.986921+0000","last_became_peered":"2024-09-12T02:56:15.986921+0000","last_unstale":"2024-09-12T02:57:59.564003+0000","last_undegraded":"2024-09-12T02:57:59.564003+0000","last_fullsized":"2024-09-12T02:57:59.564003+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:42:13.427871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.555703+0000","last_change":"2024-09-12T02:56:15.987673+0000","last_active":"2024-09-12T02:57:59.555703+0000","last_peered":"2024-09-12T02:57:59.555703+0000","last_clean":"2024-09-12T02:57:59.555703+0000","last_became_active":"2024-09-12T02:56:15.986964+0000","last_became_peered":"2024-09-12T02:56:15.986964+0000","last_unstale":"2024-09-12T02:57:59.555703+0000","last_undegraded":"2024-09-12T02:57:59.555703+0000","last_fullsized":"2024-09-12T02:57:59.555703+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:07:12.406974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"83'4","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.548461+0000","last_change":"2024-09-12T02:56:15.974473+0000","last_active":"2024-09-12T02:57:59.548461+0000","last_peered":"2024-09-12T02:57:59.548461+0000","last_clean":"2024-09-12T02:57:59.548461+0000","last_became_active":"2024-09-12T02:56:15.974124+0000","last_became_peered":"2024-09-12T02:56:15.974124+0000","last_unstale":"2024-09-12T02:57:59.548461+0000","last_undegraded":"2024-09-12T02:57:59.548461+0000","last_fullsized":"2024-09-12T02:57:59.548461+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:44:29.054841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.560740+0000","last_change":"2024-09-12T02:56:15.980242+0000","last_active":"2024-09-12T02:57:59.560740+0000","last_peered":"2024-09-12T02:57:59.560740+0000","last_clean":"2024-09-12T02:57:59.560740+0000","last_became_active":"2024-09-12T02:56:15.979937+0000","last_became_peered":"2024-09-12T02:56:15.979937+0000","last_unstale":"2024-09-12T02:57:59.560740+0000","last_undegraded":"2024-09-12T02:57:59.560740+0000","last_fullsized":"2024-09-12T02:57:59.560740+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:44:13.406341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"83'5","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.556845+0000","last_change":"2024-09-12T02:56:15.978746+0000","last_active":"2024-09-12T02:57:59.556845+0000","last_peered":"2024-09-12T02:57:59.556845+0000","last_clean":"2024-09-12T02:57:59.556845+0000","last_became_active":"2024-09-12T02:56:15.978304+0000","last_became_peered":"2024-09-12T02:56:15.978304+0000","last_unstale":"2024-09-12T02:57:59.556845+0000","last_undegraded":"2024-09-12T02:57:59.556845+0000","last_fullsized":"2024-09-12T02:57:59.556845+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:34:52.115630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.562507+0000","last_change":"2024-09-12T02:56:15.980083+0000","last_active":"2024-09-12T02:57:59.562507+0000","last_peered":"2024-09-12T02:57:59.562507+0000","last_clean":"2024-09-12T02:57:59.562507+0000","last_became_active":"2024-09-12T02:56:15.979771+0000","last_became_peered":"2024-09-12T02:56:15.979771+0000","last_unstale":"2024-09-12T02:57:59.562507+0000","last_undegraded":"2024-09-12T02:57:59.562507+0000","last_fullsized":"2024-09-12T02:57:59.562507+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:59:42.351970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"83'11","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.565703+0000","last_change":"2024-09-12T02:56:15.988091+0000","last_active":"2024-09-12T02:57:59.565703+0000","last_peered":"2024-09-12T02:57:59.565703+0000","last_clean":"2024-09-12T02:57:59.565703+0000","last_became_active":"2024-09-12T02:56:15.986295+0000","last_became_peered":"2024-09-12T02:56:15.986295+0000","last_unstale":"2024-09-12T02:57:59.565703+0000","last_undegraded":"2024-09-12T02:57:59.565703+0000","last_fullsized":"2024-09-12T02:57:59.565703+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:45:40.331805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"83'16","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.562128+0000","last_change":"2024-09-12T02:56:15.987349+0000","last_active":"2024-09-12T02:57:59.562128+0000","last_peered":"2024-09-12T02:57:59.562128+0000","last_clean":"2024-09-12T02:57:59.562128+0000","last_became_active":"2024-09-12T02:56:15.987018+0000","last_became_peered":"2024-09-12T02:56:15.987018+0000","last_unstale":"2024-09-12T02:57:59.562128+0000","last_undegraded":"2024-09-12T02:57:59.562128+0000","last_fullsized":"2024-09-12T02:57:59.562128+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:34:19.652348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.563544+0000","last_change":"2024-09-12T02:56:15.985164+0000","last_active":"2024-09-12T02:57:59.563544+0000","last_peered":"2024-09-12T02:57:59.563544+0000","last_clean":"2024-09-12T02:57:59.563544+0000","last_became_active":"2024-09-12T02:56:15.984787+0000","last_became_peered":"2024-09-12T02:56:15.984787+0000","last_unstale":"2024-09-12T02:57:59.563544+0000","last_undegraded":"2024-09-12T02:57:59.563544+0000","last_fullsized":"2024-09-12T02:57:59.563544+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:36:29.727442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.552957+0000","last_change":"2024-09-12T02:56:15.979963+0000","last_active":"2024-09-12T02:57:59.552957+0000","last_peered":"2024-09-12T02:57:59.552957+0000","last_clean":"2024-09-12T02:57:59.552957+0000","last_became_active":"2024-09-12T02:56:15.979639+0000","last_became_peered":"2024-09-12T02:56:15.979639+0000","last_unstale":"2024-09-12T02:57:59.552957+0000","last_undegraded":"2024-09-12T02:57:59.552957+0000","last_fullsized":"2024-09-12T02:57:59.552957+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:34:18.273245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"83'10","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.561647+0000","last_change":"2024-09-12T02:56:15.980415+0000","last_active":"2024-09-12T02:57:59.561647+0000","last_peered":"2024-09-12T02:57:59.561647+0000","last_clean":"2024-09-12T02:57:59.561647+0000","last_became_active":"2024-09-12T02:56:15.980059+0000","last_became_peered":"2024-09-12T02:56:15.980059+0000","last_unstale":"2024-09-12T02:57:59.561647+0000","last_undegraded":"2024-09-12T02:57:59.561647+0000","last_fullsized":"2024-09-12T02:57:59.561647+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:11:42.689081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.553574+0000","last_change":"2024-09-12T02:56:15.981423+0000","last_active":"2024-09-12T02:57:59.553574+0000","last_peered":"2024-09-12T02:57:59.553574+0000","last_clean":"2024-09-12T02:57:59.553574+0000","last_became_active":"2024-09-12T02:56:15.981186+0000","last_became_peered":"2024-09-12T02:56:15.981186+0000","last_unstale":"2024-09-12T02:57:59.553574+0000","last_undegraded":"2024-09-12T02:57:59.553574+0000","last_fullsized":"2024-09-12T02:57:59.553574+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:36:57.406907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.547696+0000","last_change":"2024-09-12T02:56:15.982587+0000","last_active":"2024-09-12T02:57:59.547696+0000","last_peered":"2024-09-12T02:57:59.547696+0000","last_clean":"2024-09-12T02:57:59.547696+0000","last_became_active":"2024-09-12T02:56:15.982315+0000","last_became_peered":"2024-09-12T02:56:15.982315+0000","last_unstale":"2024-09-12T02:57:59.547696+0000","last_undegraded":"2024-09-12T02:57:59.547696+0000","last_fullsized":"2024-09-12T02:57:59.547696+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:42:17.736999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.564760+0000","last_change":"2024-09-12T02:56:15.979482+0000","last_active":"2024-09-12T02:57:59.564760+0000","last_peered":"2024-09-12T02:57:59.564760+0000","last_clean":"2024-09-12T02:57:59.564760+0000","last_became_active":"2024-09-12T02:56:15.979122+0000","last_became_peered":"2024-09-12T02:56:15.979122+0000","last_unstale":"2024-09-12T02:57:59.564760+0000","last_undegraded":"2024-09-12T02:57:59.564760+0000","last_fullsized":"2024-09-12T02:57:59.564760+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:18:05.642487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.565218+0000","last_change":"2024-09-12T02:56:15.989908+0000","last_active":"2024-09-12T02:57:59.565218+0000","last_peered":"2024-09-12T02:57:59.565218+0000","last_clean":"2024-09-12T02:57:59.565218+0000","last_became_active":"2024-09-12T02:56:15.989515+0000","last_became_peered":"2024-09-12T02:56:15.989515+0000","last_unstale":"2024-09-12T02:57:59.565218+0000","last_undegraded":"2024-09-12T02:57:59.565218+0000","last_fullsized":"2024-09-12T02:57:59.565218+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:59:57.141607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"83'10","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.557609+0000","last_change":"2024-09-12T02:56:15.980323+0000","last_active":"2024-09-12T02:57:59.557609+0000","last_peered":"2024-09-12T02:57:59.557609+0000","last_clean":"2024-09-12T02:57:59.557609+0000","last_became_active":"2024-09-12T02:56:15.979989+0000","last_became_peered":"2024-09-12T02:56:15.979989+0000","last_unstale":"2024-09-12T02:57:59.557609+0000","last_undegraded":"2024-09-12T02:57:59.557609+0000","last_fullsized":"2024-09-12T02:57:59.557609+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:55:56.064125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.556275+0000","last_change":"2024-09-12T02:56:15.986003+0000","last_active":"2024-09-12T02:57:59.556275+0000","last_peered":"2024-09-12T02:57:59.556275+0000","last_clean":"2024-09-12T02:57:59.556275+0000","last_became_active":"2024-09-12T02:56:15.985335+0000","last_became_peered":"2024-09-12T02:56:15.985335+0000","last_unstale":"2024-09-12T02:57:59.556275+0000","last_undegraded":"2024-09-12T02:57:59.556275+0000","last_fullsized":"2024-09-12T02:57:59.556275+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:51:47.944530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"83'8","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.554254+0000","last_change":"2024-09-12T02:56:31.036870+0000","last_active":"2024-09-12T02:57:59.554254+0000","last_peered":"2024-09-12T02:57:59.554254+0000","last_clean":"2024-09-12T02:57:59.554254+0000","last_became_active":"2024-09-12T02:56:31.036443+0000","last_became_peered":"2024-09-12T02:56:31.036443+0000","last_unstale":"2024-09-12T02:57:59.554254+0000","last_undegraded":"2024-09-12T02:57:59.554254+0000","last_fullsized":"2024-09-12T02:57:59.554254+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:49:29.326184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"83'11","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.555414+0000","last_change":"2024-09-12T02:56:15.974547+0000","last_active":"2024-09-12T02:57:59.555414+0000","last_peered":"2024-09-12T02:57:59.555414+0000","last_clean":"2024-09-12T02:57:59.555414+0000","last_became_active":"2024-09-12T02:56:15.974174+0000","last_became_peered":"2024-09-12T02:56:15.974174+0000","last_unstale":"2024-09-12T02:57:59.555414+0000","last_undegraded":"2024-09-12T02:57:59.555414+0000","last_fullsized":"2024-09-12T02:57:59.555414+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:22:16.954309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"83'10","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.565778+0000","last_change":"2024-09-12T02:56:15.981771+0000","last_active":"2024-09-12T02:57:59.565778+0000","last_peered":"2024-09-12T02:57:59.565778+0000","last_clean":"2024-09-12T02:57:59.565778+0000","last_became_active":"2024-09-12T02:56:15.980948+0000","last_became_peered":"2024-09-12T02:56:15.980948+0000","last_unstale":"2024-09-12T02:57:59.565778+0000","last_undegraded":"2024-09-12T02:57:59.565778+0000","last_fullsized":"2024-09-12T02:57:59.565778+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:34:35.961348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.562061+0000","last_change":"2024-09-12T02:56:15.982318+0000","last_active":"2024-09-12T02:57:59.562061+0000","last_peered":"2024-09-12T02:57:59.562061+0000","last_clean":"2024-09-12T02:57:59.562061+0000","last_became_active":"2024-09-12T02:56:15.982013+0000","last_became_peered":"2024-09-12T02:56:15.982013+0000","last_unstale":"2024-09-12T02:57:59.562061+0000","last_undegraded":"2024-09-12T02:57:59.562061+0000","last_fullsized":"2024-09-12T02:57:59.562061+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:10:50.895579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.559580+0000","last_change":"2024-09-12T02:56:15.969992+0000","last_active":"2024-09-12T02:57:59.559580+0000","last_peered":"2024-09-12T02:57:59.559580+0000","last_clean":"2024-09-12T02:57:59.559580+0000","last_became_active":"2024-09-12T02:56:15.969621+0000","last_became_peered":"2024-09-12T02:56:15.969621+0000","last_unstale":"2024-09-12T02:57:59.559580+0000","last_undegraded":"2024-09-12T02:57:59.559580+0000","last_fullsized":"2024-09-12T02:57:59.559580+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:47:40.206042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"83'2","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.560304+0000","last_change":"2024-09-12T02:56:15.982315+0000","last_active":"2024-09-12T02:57:59.560304+0000","last_peered":"2024-09-12T02:57:59.560304+0000","last_clean":"2024-09-12T02:57:59.560304+0000","last_became_active":"2024-09-12T02:56:15.981992+0000","last_became_peered":"2024-09-12T02:56:15.981992+0000","last_unstale":"2024-09-12T02:57:59.560304+0000","last_undegraded":"2024-09-12T02:57:59.560304+0000","last_fullsized":"2024-09-12T02:57:59.560304+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:54:04.188418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"83'15","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.560612+0000","last_change":"2024-09-12T02:56:15.980712+0000","last_active":"2024-09-12T02:57:59.560612+0000","last_peered":"2024-09-12T02:57:59.560612+0000","last_clean":"2024-09-12T02:57:59.560612+0000","last_became_active":"2024-09-12T02:56:15.980410+0000","last_became_peered":"2024-09-12T02:56:15.980410+0000","last_unstale":"2024-09-12T02:57:59.560612+0000","last_undegraded":"2024-09-12T02:57:59.560612+0000","last_fullsized":"2024-09-12T02:57:59.560612+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:13:53.844490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.557058+0000","last_change":"2024-09-12T02:56:15.981675+0000","last_active":"2024-09-12T02:57:59.557058+0000","last_peered":"2024-09-12T02:57:59.557058+0000","last_clean":"2024-09-12T02:57:59.557058+0000","last_became_active":"2024-09-12T02:56:15.981372+0000","last_became_peered":"2024-09-12T02:56:15.981372+0000","last_unstale":"2024-09-12T02:57:59.557058+0000","last_undegraded":"2024-09-12T02:57:59.557058+0000","last_fullsized":"2024-09-12T02:57:59.557058+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:39:17.399199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"83'4","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.557413+0000","last_change":"2024-09-12T02:56:15.972750+0000","last_active":"2024-09-12T02:57:59.557413+0000","last_peered":"2024-09-12T02:57:59.557413+0000","last_clean":"2024-09-12T02:57:59.557413+0000","last_became_active":"2024-09-12T02:56:15.972434+0000","last_became_peered":"2024-09-12T02:56:15.972434+0000","last_unstale":"2024-09-12T02:57:59.557413+0000","last_undegraded":"2024-09-12T02:57:59.557413+0000","last_fullsized":"2024-09-12T02:57:59.557413+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:59:22.784908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.545235+0000","last_change":"2024-09-12T02:56:15.973567+0000","last_active":"2024-09-12T02:57:59.545235+0000","last_peered":"2024-09-12T02:57:59.545235+0000","last_clean":"2024-09-12T02:57:59.545235+0000","last_became_active":"2024-09-12T02:56:15.973251+0000","last_became_peered":"2024-09-12T02:56:15.973251+0000","last_unstale":"2024-09-12T02:57:59.545235+0000","last_undegraded":"2024-09-12T02:57:59.545235+0000","last_fullsized":"2024-09-12T02:57:59.545235+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:16:39.536817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.540308+0000","last_change":"2024-09-12T02:56:15.986108+0000","last_active":"2024-09-12T02:57:59.540308+0000","last_peered":"2024-09-12T02:57:59.540308+0000","last_clean":"2024-09-12T02:57:59.540308+0000","last_became_active":"2024-09-12T02:56:15.985701+0000","last_became_peered":"2024-09-12T02:56:15.985701+0000","last_unstale":"2024-09-12T02:57:59.540308+0000","last_undegraded":"2024-09-12T02:57:59.540308+0000","last_fullsized":"2024-09-12T02:57:59.540308+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:59.377473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"83'12","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.555638+0000","last_change":"2024-09-12T02:56:15.980078+0000","last_active":"2024-09-12T02:57:59.555638+0000","last_peered":"2024-09-12T02:57:59.555638+0000","last_clean":"2024-09-12T02:57:59.555638+0000","last_became_active":"2024-09-12T02:56:15.979785+0000","last_became_peered":"2024-09-12T02:56:15.979785+0000","last_unstale":"2024-09-12T02:57:59.555638+0000","last_undegraded":"2024-09-12T02:57:59.555638+0000","last_fullsized":"2024-09-12T02:57:59.555638+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:00:19.252398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.559119+0000","last_change":"2024-09-12T02:56:15.984170+0000","last_active":"2024-09-12T02:57:59.559119+0000","last_peered":"2024-09-12T02:57:59.559119+0000","last_clean":"2024-09-12T02:57:59.559119+0000","last_became_active":"2024-09-12T02:56:15.983416+0000","last_became_peered":"2024-09-12T02:56:15.983416+0000","last_unstale":"2024-09-12T02:57:59.559119+0000","last_undegraded":"2024-09-12T02:57:59.559119+0000","last_fullsized":"2024-09-12T02:57:59.559119+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:13:24.723923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.556295+0000","last_change":"2024-09-12T02:56:15.980987+0000","last_active":"2024-09-12T02:57:59.556295+0000","last_peered":"2024-09-12T02:57:59.556295+0000","last_clean":"2024-09-12T02:57:59.556295+0000","last_became_active":"2024-09-12T02:56:15.980661+0000","last_became_peered":"2024-09-12T02:56:15.980661+0000","last_unstale":"2024-09-12T02:57:59.556295+0000","last_undegraded":"2024-09-12T02:57:59.556295+0000","last_fullsized":"2024-09-12T02:57:59.556295+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:19:49.294698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"81'5","reported_seq":135,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:58:00.172973+0000","last_change":"2024-09-12T02:56:31.042914+0000","last_active":"2024-09-12T02:58:00.172973+0000","last_peered":"2024-09-12T02:58:00.172973+0000","last_clean":"2024-09-12T02:58:00.172973+0000","last_became_active":"2024-09-12T02:56:31.037356+0000","last_became_peered":"2024-09-12T02:56:31.037356+0000","last_unstale":"2024-09-12T02:58:00.172973+0000","last_undegraded":"2024-09-12T02:58:00.172973+0000","last_fullsized":"2024-09-12T02:58:00.172973+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:17:48.582725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":622,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"83'8","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.565986+0000","last_change":"2024-09-12T02:56:31.036170+0000","last_active":"2024-09-12T02:57:59.565986+0000","last_peered":"2024-09-12T02:57:59.565986+0000","last_clean":"2024-09-12T02:57:59.565986+0000","last_became_active":"2024-09-12T02:56:31.035754+0000","last_became_peered":"2024-09-12T02:56:31.035754+0000","last_unstale":"2024-09-12T02:57:59.565986+0000","last_undegraded":"2024-09-12T02:57:59.565986+0000","last_fullsized":"2024-09-12T02:57:59.565986+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:58:06.464818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1a","version":"81'7","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.032813+0000","last_change":"2024-09-12T02:56:13.931202+0000","last_active":"2024-09-12T02:56:31.032813+0000","last_peered":"2024-09-12T02:56:31.032813+0000","last_clean":"2024-09-12T02:56:31.032813+0000","last_became_active":"2024-09-12T02:56:13.930776+0000","last_became_peered":"2024-09-12T02:56:13.930776+0000","last_unstale":"2024-09-12T02:56:31.032813+0000","last_undegraded":"2024-09-12T02:56:31.032813+0000","last_fullsized":"2024-09-12T02:56:31.032813+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:22:16.919253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"81'3","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.031662+0000","last_change":"2024-09-12T02:56:13.924660+0000","last_active":"2024-09-12T02:56:31.031662+0000","last_peered":"2024-09-12T02:56:31.031662+0000","last_clean":"2024-09-12T02:56:31.031662+0000","last_became_active":"2024-09-12T02:56:13.924342+0000","last_became_peered":"2024-09-12T02:56:13.924342+0000","last_unstale":"2024-09-12T02:56:31.031662+0000","last_undegraded":"2024-09-12T02:56:31.031662+0000","last_fullsized":"2024-09-12T02:56:31.031662+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:48:53.907044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.563814+0000","last_change":"2024-09-12T02:56:15.978723+0000","last_active":"2024-09-12T02:57:59.563814+0000","last_peered":"2024-09-12T02:57:59.563814+0000","last_clean":"2024-09-12T02:57:59.563814+0000","last_became_active":"2024-09-12T02:56:15.978344+0000","last_became_peered":"2024-09-12T02:56:15.978344+0000","last_unstale":"2024-09-12T02:57:59.563814+0000","last_undegraded":"2024-09-12T02:57:59.563814+0000","last_fullsized":"2024-09-12T02:57:59.563814+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:25:04.559395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"81'3","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.032936+0000","last_change":"2024-09-12T02:56:13.933065+0000","last_active":"2024-09-12T02:56:31.032936+0000","last_peered":"2024-09-12T02:56:31.032936+0000","last_clean":"2024-09-12T02:56:31.032936+0000","last_became_active":"2024-09-12T02:56:13.932597+0000","last_became_peered":"2024-09-12T02:56:13.932597+0000","last_unstale":"2024-09-12T02:56:31.032936+0000","last_undegraded":"2024-09-12T02:56:31.032936+0000","last_fullsized":"2024-09-12T02:56:31.032936+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:57:26.563392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"83'11","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.566801+0000","last_change":"2024-09-12T02:56:15.975722+0000","last_active":"2024-09-12T02:57:59.566801+0000","last_peered":"2024-09-12T02:57:59.566801+0000","last_clean":"2024-09-12T02:57:59.566801+0000","last_became_active":"2024-09-12T02:56:15.975342+0000","last_became_peered":"2024-09-12T02:56:15.975342+0000","last_unstale":"2024-09-12T02:57:59.566801+0000","last_undegraded":"2024-09-12T02:57:59.566801+0000","last_fullsized":"2024-09-12T02:57:59.566801+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:33:17.613738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'5","reported_seq":217,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:58:01.977448+0000","last_change":"2024-09-12T02:56:13.929700+0000","last_active":"2024-09-12T02:58:01.977448+0000","last_peered":"2024-09-12T02:58:01.977448+0000","last_clean":"2024-09-12T02:58:01.977448+0000","last_became_active":"2024-09-12T02:56:13.929317+0000","last_became_peered":"2024-09-12T02:56:13.929317+0000","last_unstale":"2024-09-12T02:58:01.977448+0000","last_undegraded":"2024-09-12T02:58:01.977448+0000","last_fullsized":"2024-09-12T02:58:01.977448+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:46:32.172674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":190,"num_read_kb":190,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"83'10","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.564710+0000","last_change":"2024-09-12T02:56:15.985754+0000","last_active":"2024-09-12T02:57:59.564710+0000","last_peered":"2024-09-12T02:57:59.564710+0000","last_clean":"2024-09-12T02:57:59.564710+0000","last_became_active":"2024-09-12T02:56:15.985402+0000","last_became_peered":"2024-09-12T02:56:15.985402+0000","last_unstale":"2024-09-12T02:57:59.564710+0000","last_undegraded":"2024-09-12T02:57:59.564710+0000","last_fullsized":"2024-09-12T02:57:59.564710+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:05:29.979961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"81'4","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.035428+0000","last_change":"2024-09-12T02:56:13.933329+0000","last_active":"2024-09-12T02:56:31.035428+0000","last_peered":"2024-09-12T02:56:31.035428+0000","last_clean":"2024-09-12T02:56:31.035428+0000","last_became_active":"2024-09-12T02:56:13.932960+0000","last_became_peered":"2024-09-12T02:56:13.932960+0000","last_unstale":"2024-09-12T02:56:31.035428+0000","last_undegraded":"2024-09-12T02:56:31.035428+0000","last_fullsized":"2024-09-12T02:56:31.035428+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:02:57.564202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"83'9","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.556804+0000","last_change":"2024-09-12T02:56:31.037340+0000","last_active":"2024-09-12T02:57:59.556804+0000","last_peered":"2024-09-12T02:57:59.556804+0000","last_clean":"2024-09-12T02:57:59.556804+0000","last_became_active":"2024-09-12T02:56:31.036993+0000","last_became_peered":"2024-09-12T02:56:31.036993+0000","last_unstale":"2024-09-12T02:57:59.556804+0000","last_undegraded":"2024-09-12T02:57:59.556804+0000","last_fullsized":"2024-09-12T02:57:59.556804+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:44:27.788791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"81'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.033005+0000","last_change":"2024-09-12T02:56:14.077202+0000","last_active":"2024-09-12T02:56:31.033005+0000","last_peered":"2024-09-12T02:56:31.033005+0000","last_clean":"2024-09-12T02:56:31.033005+0000","last_became_active":"2024-09-12T02:56:14.076610+0000","last_became_peered":"2024-09-12T02:56:14.076610+0000","last_unstale":"2024-09-12T02:56:31.033005+0000","last_undegraded":"2024-09-12T02:56:31.033005+0000","last_fullsized":"2024-09-12T02:56:31.033005+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:37:50.333152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"83'11","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.558754+0000","last_change":"2024-09-12T02:56:15.977161+0000","last_active":"2024-09-12T02:57:59.558754+0000","last_peered":"2024-09-12T02:57:59.558754+0000","last_clean":"2024-09-12T02:57:59.558754+0000","last_became_active":"2024-09-12T02:56:15.976683+0000","last_became_peered":"2024-09-12T02:56:15.976683+0000","last_unstale":"2024-09-12T02:57:59.558754+0000","last_undegraded":"2024-09-12T02:57:59.558754+0000","last_fullsized":"2024-09-12T02:57:59.558754+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:34:07.526890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'36","reported_seq":259,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:58:01.977471+0000","last_change":"2024-09-12T02:56:13.931680+0000","last_active":"2024-09-12T02:58:01.977471+0000","last_peered":"2024-09-12T02:58:01.977471+0000","last_clean":"2024-09-12T02:58:01.977471+0000","last_became_active":"2024-09-12T02:56:13.931214+0000","last_became_peered":"2024-09-12T02:56:13.931214+0000","last_unstale":"2024-09-12T02:58:01.977471+0000","last_undegraded":"2024-09-12T02:58:01.977471+0000","last_fullsized":"2024-09-12T02:58:01.977471+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:38:34.151933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66038,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":201,"num_read_kb":160,"num_write":38,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.555616+0000","last_change":"2024-09-12T02:56:15.977720+0000","last_active":"2024-09-12T02:57:59.555616+0000","last_peered":"2024-09-12T02:57:59.555616+0000","last_clean":"2024-09-12T02:57:59.555616+0000","last_became_active":"2024-09-12T02:56:15.977380+0000","last_became_peered":"2024-09-12T02:56:15.977380+0000","last_unstale":"2024-09-12T02:57:59.555616+0000","last_undegraded":"2024-09-12T02:57:59.555616+0000","last_fullsized":"2024-09-12T02:57:59.555616+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:52:16.841909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"81'9","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.030362+0000","last_change":"2024-09-12T02:56:13.929496+0000","last_active":"2024-09-12T02:56:31.030362+0000","last_peered":"2024-09-12T02:56:31.030362+0000","last_clean":"2024-09-12T02:56:31.030362+0000","last_became_active":"2024-09-12T02:56:13.929117+0000","last_became_peered":"2024-09-12T02:56:13.929117+0000","last_unstale":"2024-09-12T02:56:31.030362+0000","last_undegraded":"2024-09-12T02:56:31.030362+0000","last_fullsized":"2024-09-12T02:56:31.030362+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:59:38.399135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.564811+0000","last_change":"2024-09-12T02:56:15.974323+0000","last_active":"2024-09-12T02:57:59.564811+0000","last_peered":"2024-09-12T02:57:59.564811+0000","last_clean":"2024-09-12T02:57:59.564811+0000","last_became_active":"2024-09-12T02:56:15.973819+0000","last_became_peered":"2024-09-12T02:56:15.973819+0000","last_unstale":"2024-09-12T02:57:59.564811+0000","last_undegraded":"2024-09-12T02:57:59.564811+0000","last_fullsized":"2024-09-12T02:57:59.564811+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:53:13.130695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"81'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.032473+0000","last_change":"2024-09-12T02:56:13.935263+0000","last_active":"2024-09-12T02:56:31.032473+0000","last_peered":"2024-09-12T02:56:31.032473+0000","last_clean":"2024-09-12T02:56:31.032473+0000","last_became_active":"2024-09-12T02:56:13.934805+0000","last_became_peered":"2024-09-12T02:56:13.934805+0000","last_unstale":"2024-09-12T02:56:31.032473+0000","last_undegraded":"2024-09-12T02:56:31.032473+0000","last_fullsized":"2024-09-12T02:56:31.032473+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:39:42.882031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"83'15","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.560318+0000","last_change":"2024-09-12T02:56:15.976630+0000","last_active":"2024-09-12T02:57:59.560318+0000","last_peered":"2024-09-12T02:57:59.560318+0000","last_clean":"2024-09-12T02:57:59.560318+0000","last_became_active":"2024-09-12T02:56:15.976304+0000","last_became_peered":"2024-09-12T02:56:15.976304+0000","last_unstale":"2024-09-12T02:57:59.560318+0000","last_undegraded":"2024-09-12T02:57:59.560318+0000","last_fullsized":"2024-09-12T02:57:59.560318+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:46:03.366740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'7","reported_seq":126,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.973453+0000","last_change":"2024-09-12T02:56:13.934224+0000","last_active":"2024-09-12T02:57:59.973453+0000","last_peered":"2024-09-12T02:57:59.973453+0000","last_clean":"2024-09-12T02:57:59.973453+0000","last_became_active":"2024-09-12T02:56:13.933838+0000","last_became_peered":"2024-09-12T02:56:13.933838+0000","last_unstale":"2024-09-12T02:57:59.973453+0000","last_undegraded":"2024-09-12T02:57:59.973453+0000","last_fullsized":"2024-09-12T02:57:59.973453+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:34:34.206051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":98,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.557213+0000","last_change":"2024-09-12T02:56:15.980364+0000","last_active":"2024-09-12T02:57:59.557213+0000","last_peered":"2024-09-12T02:57:59.557213+0000","last_clean":"2024-09-12T02:57:59.557213+0000","last_became_active":"2024-09-12T02:56:15.979437+0000","last_became_peered":"2024-09-12T02:56:15.979437+0000","last_unstale":"2024-09-12T02:57:59.557213+0000","last_undegraded":"2024-09-12T02:57:59.557213+0000","last_fullsized":"2024-09-12T02:57:59.557213+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:43:33.424867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"81'17","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.050509+0000","last_change":"2024-09-12T02:56:31.050046+0000","last_active":"2024-09-12T02:56:31.050509+0000","last_peered":"2024-09-12T02:56:31.050509+0000","last_clean":"2024-09-12T02:56:31.050509+0000","last_became_active":"2024-09-12T02:56:31.036990+0000","last_became_peered":"2024-09-12T02:56:31.036990+0000","last_unstale":"2024-09-12T02:56:31.050509+0000","last_undegraded":"2024-09-12T02:56:31.050509+0000","last_fullsized":"2024-09-12T02:56:31.050509+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:05:53.336691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.563297+0000","last_change":"2024-09-12T02:56:15.986434+0000","last_active":"2024-09-12T02:57:59.563297+0000","last_peered":"2024-09-12T02:57:59.563297+0000","last_clean":"2024-09-12T02:57:59.563297+0000","last_became_active":"2024-09-12T02:56:15.986101+0000","last_became_peered":"2024-09-12T02:56:15.986101+0000","last_unstale":"2024-09-12T02:57:59.563297+0000","last_undegraded":"2024-09-12T02:57:59.563297+0000","last_fullsized":"2024-09-12T02:57:59.563297+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:51:27.840951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"81'2","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.035974+0000","last_change":"2024-09-12T02:56:13.925068+0000","last_active":"2024-09-12T02:56:31.035974+0000","last_peered":"2024-09-12T02:56:31.035974+0000","last_clean":"2024-09-12T02:56:31.035974+0000","last_became_active":"2024-09-12T02:56:13.924535+0000","last_became_peered":"2024-09-12T02:56:13.924535+0000","last_unstale":"2024-09-12T02:56:31.035974+0000","last_undegraded":"2024-09-12T02:56:31.035974+0000","last_fullsized":"2024-09-12T02:56:31.035974+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:45:32.478632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"83'5","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.559020+0000","last_change":"2024-09-12T02:56:15.983850+0000","last_active":"2024-09-12T02:57:59.559020+0000","last_peered":"2024-09-12T02:57:59.559020+0000","last_clean":"2024-09-12T02:57:59.559020+0000","last_became_active":"2024-09-12T02:56:15.983480+0000","last_became_peered":"2024-09-12T02:56:15.983480+0000","last_unstale":"2024-09-12T02:57:59.559020+0000","last_undegraded":"2024-09-12T02:57:59.559020+0000","last_fullsized":"2024-09-12T02:57:59.559020+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:09:42.185237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"81'5","reported_seq":39,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.312450+0000","last_change":"2024-09-12T02:56:31.312450+0000","last_active":"2024-09-12T02:56:31.312450+0000","last_peered":"2024-09-12T02:56:31.312450+0000","last_clean":"2024-09-12T02:56:31.312450+0000","last_became_active":"2024-09-12T02:56:31.308030+0000","last_became_peered":"2024-09-12T02:56:31.308030+0000","last_unstale":"2024-09-12T02:56:31.312450+0000","last_undegraded":"2024-09-12T02:56:31.312450+0000","last_fullsized":"2024-09-12T02:56:31.312450+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:15:14.692292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.562640+0000","last_change":"2024-09-12T02:56:15.978996+0000","last_active":"2024-09-12T02:57:59.562640+0000","last_peered":"2024-09-12T02:57:59.562640+0000","last_clean":"2024-09-12T02:57:59.562640+0000","last_became_active":"2024-09-12T02:56:15.978636+0000","last_became_peered":"2024-09-12T02:56:15.978636+0000","last_unstale":"2024-09-12T02:57:59.562640+0000","last_undegraded":"2024-09-12T02:57:59.562640+0000","last_fullsized":"2024-09-12T02:57:59.562640+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:29:20.401664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"81'3","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.033119+0000","last_change":"2024-09-12T02:56:13.929388+0000","last_active":"2024-09-12T02:56:31.033119+0000","last_peered":"2024-09-12T02:56:31.033119+0000","last_clean":"2024-09-12T02:56:31.033119+0000","last_became_active":"2024-09-12T02:56:13.928998+0000","last_became_peered":"2024-09-12T02:56:13.928998+0000","last_unstale":"2024-09-12T02:56:31.033119+0000","last_undegraded":"2024-09-12T02:56:31.033119+0000","last_fullsized":"2024-09-12T02:56:31.033119+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:37:57.368050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.560190+0000","last_change":"2024-09-12T02:56:15.977491+0000","last_active":"2024-09-12T02:57:59.560190+0000","last_peered":"2024-09-12T02:57:59.560190+0000","last_clean":"2024-09-12T02:57:59.560190+0000","last_became_active":"2024-09-12T02:56:15.977160+0000","last_became_peered":"2024-09-12T02:56:15.977160+0000","last_unstale":"2024-09-12T02:57:59.560190+0000","last_undegraded":"2024-09-12T02:57:59.560190+0000","last_fullsized":"2024-09-12T02:57:59.560190+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:05:40.577342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'7","reported_seq":128,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:58:00.976755+0000","last_change":"2024-09-12T02:56:14.327147+0000","last_active":"2024-09-12T02:58:00.976755+0000","last_peered":"2024-09-12T02:58:00.976755+0000","last_clean":"2024-09-12T02:58:00.976755+0000","last_became_active":"2024-09-12T02:56:14.326737+0000","last_became_peered":"2024-09-12T02:56:14.326737+0000","last_unstale":"2024-09-12T02:58:00.976755+0000","last_undegraded":"2024-09-12T02:58:00.976755+0000","last_fullsized":"2024-09-12T02:58:00.976755+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:55:33.510599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1704,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":99,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.563684+0000","last_change":"2024-09-12T02:56:15.984363+0000","last_active":"2024-09-12T02:57:59.563684+0000","last_peered":"2024-09-12T02:57:59.563684+0000","last_clean":"2024-09-12T02:57:59.563684+0000","last_became_active":"2024-09-12T02:56:15.983822+0000","last_became_peered":"2024-09-12T02:56:15.983822+0000","last_unstale":"2024-09-12T02:57:59.563684+0000","last_undegraded":"2024-09-12T02:57:59.563684+0000","last_fullsized":"2024-09-12T02:57:59.563684+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:41:46.082716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":333,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.035564+0000","last_change":"2024-09-12T02:55:05.800210+0000","last_active":"2024-09-12T02:56:31.035564+0000","last_peered":"2024-09-12T02:56:31.035564+0000","last_clean":"2024-09-12T02:56:31.035564+0000","last_became_active":"2024-09-12T02:55:05.799791+0000","last_became_peered":"2024-09-12T02:55:05.799791+0000","last_unstale":"2024-09-12T02:56:31.035564+0000","last_undegraded":"2024-09-12T02:56:31.035564+0000","last_fullsized":"2024-09-12T02:56:31.035564+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:51:48.530016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:51:48.530016+0000","last_clean_scrub_stamp":"2024-09-12T02:51:48.530016+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:05:47.416697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"81'6","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.031169+0000","last_change":"2024-09-12T02:56:13.930269+0000","last_active":"2024-09-12T02:56:31.031169+0000","last_peered":"2024-09-12T02:56:31.031169+0000","last_clean":"2024-09-12T02:56:31.031169+0000","last_became_active":"2024-09-12T02:56:13.929831+0000","last_became_peered":"2024-09-12T02:56:13.929831+0000","last_unstale":"2024-09-12T02:56:31.031169+0000","last_undegraded":"2024-09-12T02:56:31.031169+0000","last_fullsized":"2024-09-12T02:56:31.031169+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:38:17.878398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.561332+0000","last_change":"2024-09-12T02:56:15.969744+0000","last_active":"2024-09-12T02:57:59.561332+0000","last_peered":"2024-09-12T02:57:59.561332+0000","last_clean":"2024-09-12T02:57:59.561332+0000","last_became_active":"2024-09-12T02:56:15.969354+0000","last_became_peered":"2024-09-12T02:56:15.969354+0000","last_unstale":"2024-09-12T02:57:59.561332+0000","last_undegraded":"2024-09-12T02:57:59.561332+0000","last_fullsized":"2024-09-12T02:57:59.561332+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:26:02.409510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"81'4","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.035385+0000","last_change":"2024-09-12T02:56:13.929231+0000","last_active":"2024-09-12T02:56:31.035385+0000","last_peered":"2024-09-12T02:56:31.035385+0000","last_clean":"2024-09-12T02:56:31.035385+0000","last_became_active":"2024-09-12T02:56:13.928843+0000","last_became_peered":"2024-09-12T02:56:13.928843+0000","last_unstale":"2024-09-12T02:56:31.035385+0000","last_undegraded":"2024-09-12T02:56:31.035385+0000","last_fullsized":"2024-09-12T02:56:31.035385+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:26:33.619755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"83'5","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.555927+0000","last_change":"2024-09-12T02:56:15.982503+0000","last_active":"2024-09-12T02:57:59.555927+0000","last_peered":"2024-09-12T02:57:59.555927+0000","last_clean":"2024-09-12T02:57:59.555927+0000","last_became_active":"2024-09-12T02:56:15.982192+0000","last_became_peered":"2024-09-12T02:56:15.982192+0000","last_unstale":"2024-09-12T02:57:59.555927+0000","last_undegraded":"2024-09-12T02:57:59.555927+0000","last_fullsized":"2024-09-12T02:57:59.555927+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:55:18.370967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"83'6","reported_seq":127,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:58:03.979145+0000","last_change":"2024-09-12T02:56:14.075486+0000","last_active":"2024-09-12T02:58:03.979145+0000","last_peered":"2024-09-12T02:58:03.979145+0000","last_clean":"2024-09-12T02:58:03.979145+0000","last_became_active":"2024-09-12T02:56:14.075152+0000","last_became_peered":"2024-09-12T02:56:14.075152+0000","last_unstale":"2024-09-12T02:58:03.979145+0000","last_undegraded":"2024-09-12T02:58:03.979145+0000","last_fullsized":"2024-09-12T02:58:03.979145+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:59:40.950191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":99,"num_read_kb":99,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.556340+0000","last_change":"2024-09-12T02:56:15.981584+0000","last_active":"2024-09-12T02:57:59.556340+0000","last_peered":"2024-09-12T02:57:59.556340+0000","last_clean":"2024-09-12T02:57:59.556340+0000","last_became_active":"2024-09-12T02:56:15.981240+0000","last_became_peered":"2024-09-12T02:56:15.981240+0000","last_unstale":"2024-09-12T02:57:59.556340+0000","last_undegraded":"2024-09-12T02:57:59.556340+0000","last_fullsized":"2024-09-12T02:57:59.556340+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:49:04.036830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"83'7","reported_seq":129,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:58:03.979513+0000","last_change":"2024-09-12T02:56:13.935143+0000","last_active":"2024-09-12T02:58:03.979513+0000","last_peered":"2024-09-12T02:58:03.979513+0000","last_clean":"2024-09-12T02:58:03.979513+0000","last_became_active":"2024-09-12T02:56:13.934792+0000","last_became_peered":"2024-09-12T02:56:13.934792+0000","last_unstale":"2024-09-12T02:58:03.979513+0000","last_undegraded":"2024-09-12T02:58:03.979513+0000","last_fullsized":"2024-09-12T02:58:03.979513+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:00:26.973612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1289,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":100,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"83'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.558893+0000","last_change":"2024-09-12T02:56:31.034825+0000","last_active":"2024-09-12T02:57:59.558893+0000","last_peered":"2024-09-12T02:57:59.558893+0000","last_clean":"2024-09-12T02:57:59.558893+0000","last_became_active":"2024-09-12T02:56:31.034516+0000","last_became_peered":"2024-09-12T02:56:31.034516+0000","last_unstale":"2024-09-12T02:57:59.558893+0000","last_undegraded":"2024-09-12T02:57:59.558893+0000","last_fullsized":"2024-09-12T02:57:59.558893+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:10:42.161023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"81'1","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.029027+0000","last_change":"2024-09-12T02:56:14.076762+0000","last_active":"2024-09-12T02:56:31.029027+0000","last_peered":"2024-09-12T02:56:31.029027+0000","last_clean":"2024-09-12T02:56:31.029027+0000","last_became_active":"2024-09-12T02:56:14.076421+0000","last_became_peered":"2024-09-12T02:56:14.076421+0000","last_unstale":"2024-09-12T02:56:31.029027+0000","last_undegraded":"2024-09-12T02:56:31.029027+0000","last_fullsized":"2024-09-12T02:56:31.029027+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:56:56.422910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'5","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.555835+0000","last_change":"2024-09-12T02:56:15.978810+0000","last_active":"2024-09-12T02:57:59.555835+0000","last_peered":"2024-09-12T02:57:59.555835+0000","last_clean":"2024-09-12T02:57:59.555835+0000","last_became_active":"2024-09-12T02:56:15.978038+0000","last_became_peered":"2024-09-12T02:56:15.978038+0000","last_unstale":"2024-09-12T02:57:59.555835+0000","last_undegraded":"2024-09-12T02:57:59.555835+0000","last_fullsized":"2024-09-12T02:57:59.555835+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:04:52.215784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"81'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.035440+0000","last_change":"2024-09-12T02:56:14.075987+0000","last_active":"2024-09-12T02:56:31.035440+0000","last_peered":"2024-09-12T02:56:31.035440+0000","last_clean":"2024-09-12T02:56:31.035440+0000","last_became_active":"2024-09-12T02:56:14.075596+0000","last_became_peered":"2024-09-12T02:56:14.075596+0000","last_unstale":"2024-09-12T02:56:31.035440+0000","last_undegraded":"2024-09-12T02:56:31.035440+0000","last_fullsized":"2024-09-12T02:56:31.035440+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:18:48.713387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.552400+0000","last_change":"2024-09-12T02:56:15.987842+0000","last_active":"2024-09-12T02:57:59.552400+0000","last_peered":"2024-09-12T02:57:59.552400+0000","last_clean":"2024-09-12T02:57:59.552400+0000","last_became_active":"2024-09-12T02:56:15.987507+0000","last_became_peered":"2024-09-12T02:56:15.987507+0000","last_unstale":"2024-09-12T02:57:59.552400+0000","last_undegraded":"2024-09-12T02:57:59.552400+0000","last_fullsized":"2024-09-12T02:57:59.552400+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:52:23.000549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'19","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:58:01.991961+0000","last_change":"2024-09-12T02:56:13.924372+0000","last_active":"2024-09-12T02:58:01.991961+0000","last_peered":"2024-09-12T02:58:01.991961+0000","last_clean":"2024-09-12T02:58:01.991961+0000","last_became_active":"2024-09-12T02:56:13.923955+0000","last_became_peered":"2024-09-12T02:56:13.923955+0000","last_unstale":"2024-09-12T02:58:01.991961+0000","last_undegraded":"2024-09-12T02:58:01.991961+0000","last_fullsized":"2024-09-12T02:58:01.991961+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:52:47.452410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":34,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"83'5","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.562919+0000","last_change":"2024-09-12T02:56:15.978262+0000","last_active":"2024-09-12T02:57:59.562919+0000","last_peered":"2024-09-12T02:57:59.562919+0000","last_clean":"2024-09-12T02:57:59.562919+0000","last_became_active":"2024-09-12T02:56:15.977913+0000","last_became_peered":"2024-09-12T02:56:15.977913+0000","last_unstale":"2024-09-12T02:57:59.562919+0000","last_undegraded":"2024-09-12T02:57:59.562919+0000","last_fullsized":"2024-09-12T02:57:59.562919+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:35:27.104635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"81'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.031488+0000","last_change":"2024-09-12T02:56:14.327417+0000","last_active":"2024-09-12T02:56:31.031488+0000","last_peered":"2024-09-12T02:56:31.031488+0000","last_clean":"2024-09-12T02:56:31.031488+0000","last_became_active":"2024-09-12T02:56:14.327001+0000","last_became_peered":"2024-09-12T02:56:14.327001+0000","last_unstale":"2024-09-12T02:56:31.031488+0000","last_undegraded":"2024-09-12T02:56:31.031488+0000","last_fullsized":"2024-09-12T02:56:31.031488+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:15:39.019252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.565859+0000","last_change":"2024-09-12T02:56:15.977286+0000","last_active":"2024-09-12T02:57:59.565859+0000","last_peered":"2024-09-12T02:57:59.565859+0000","last_clean":"2024-09-12T02:57:59.565859+0000","last_became_active":"2024-09-12T02:56:15.976944+0000","last_became_peered":"2024-09-12T02:56:15.976944+0000","last_unstale":"2024-09-12T02:57:59.565859+0000","last_undegraded":"2024-09-12T02:57:59.565859+0000","last_fullsized":"2024-09-12T02:57:59.565859+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:22:57.056781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.306212+0000","last_change":"2024-09-12T02:56:14.075701+0000","last_active":"2024-09-12T02:56:31.306212+0000","last_peered":"2024-09-12T02:56:31.306212+0000","last_clean":"2024-09-12T02:56:31.306212+0000","last_became_active":"2024-09-12T02:56:14.075340+0000","last_became_peered":"2024-09-12T02:56:14.075340+0000","last_unstale":"2024-09-12T02:56:31.306212+0000","last_undegraded":"2024-09-12T02:56:31.306212+0000","last_fullsized":"2024-09-12T02:56:31.306212+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:17:42.255058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"83'2","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.528187+0000","last_change":"2024-09-12T02:56:15.979783+0000","last_active":"2024-09-12T02:57:59.528187+0000","last_peered":"2024-09-12T02:57:59.528187+0000","last_clean":"2024-09-12T02:57:59.528187+0000","last_became_active":"2024-09-12T02:56:15.979424+0000","last_became_peered":"2024-09-12T02:56:15.979424+0000","last_unstale":"2024-09-12T02:57:59.528187+0000","last_undegraded":"2024-09-12T02:57:59.528187+0000","last_fullsized":"2024-09-12T02:57:59.528187+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:59:11.568138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"81'7","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.034660+0000","last_change":"2024-09-12T02:56:13.928996+0000","last_active":"2024-09-12T02:56:31.034660+0000","last_peered":"2024-09-12T02:56:31.034660+0000","last_clean":"2024-09-12T02:56:31.034660+0000","last_became_active":"2024-09-12T02:56:13.928664+0000","last_became_peered":"2024-09-12T02:56:13.928664+0000","last_unstale":"2024-09-12T02:56:31.034660+0000","last_undegraded":"2024-09-12T02:56:31.034660+0000","last_fullsized":"2024-09-12T02:56:31.034660+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:37:22.067930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.560217+0000","last_change":"2024-09-12T02:56:15.984096+0000","last_active":"2024-09-12T02:57:59.560217+0000","last_peered":"2024-09-12T02:57:59.560217+0000","last_clean":"2024-09-12T02:57:59.560217+0000","last_became_active":"2024-09-12T02:56:15.983737+0000","last_became_peered":"2024-09-12T02:56:15.983737+0000","last_unstale":"2024-09-12T02:57:59.560217+0000","last_undegraded":"2024-09-12T02:57:59.560217+0000","last_fullsized":"2024-09-12T02:57:59.560217+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:02:17.892318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"81'3","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.053785+0000","last_change":"2024-09-12T02:56:31.053343+0000","last_active":"2024-09-12T02:56:31.053785+0000","last_peered":"2024-09-12T02:56:31.053785+0000","last_clean":"2024-09-12T02:56:31.053785+0000","last_became_active":"2024-09-12T02:56:31.037918+0000","last_became_peered":"2024-09-12T02:56:31.037918+0000","last_unstale":"2024-09-12T02:56:31.053785+0000","last_undegraded":"2024-09-12T02:56:31.053785+0000","last_fullsized":"2024-09-12T02:56:31.053785+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:39:54.990724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.563189+0000","last_change":"2024-09-12T02:56:15.975640+0000","last_active":"2024-09-12T02:57:59.563189+0000","last_peered":"2024-09-12T02:57:59.563189+0000","last_clean":"2024-09-12T02:57:59.563189+0000","last_became_active":"2024-09-12T02:56:15.975342+0000","last_became_peered":"2024-09-12T02:56:15.975342+0000","last_unstale":"2024-09-12T02:57:59.563189+0000","last_undegraded":"2024-09-12T02:57:59.563189+0000","last_fullsized":"2024-09-12T02:57:59.563189+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:26:23.694876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.034922+0000","last_change":"2024-09-12T02:56:13.934149+0000","last_active":"2024-09-12T02:56:31.034922+0000","last_peered":"2024-09-12T02:56:31.034922+0000","last_clean":"2024-09-12T02:56:31.034922+0000","last_became_active":"2024-09-12T02:56:13.933743+0000","last_became_peered":"2024-09-12T02:56:13.933743+0000","last_unstale":"2024-09-12T02:56:31.034922+0000","last_undegraded":"2024-09-12T02:56:31.034922+0000","last_fullsized":"2024-09-12T02:56:31.034922+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:16:17.854457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"83'5","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.563348+0000","last_change":"2024-09-12T02:56:15.984782+0000","last_active":"2024-09-12T02:57:59.563348+0000","last_peered":"2024-09-12T02:57:59.563348+0000","last_clean":"2024-09-12T02:57:59.563348+0000","last_became_active":"2024-09-12T02:56:15.984336+0000","last_became_peered":"2024-09-12T02:56:15.984336+0000","last_unstale":"2024-09-12T02:57:59.563348+0000","last_undegraded":"2024-09-12T02:57:59.563348+0000","last_fullsized":"2024-09-12T02:57:59.563348+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:03:39.832897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"83'15","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.564252+0000","last_change":"2024-09-12T02:56:15.985486+0000","last_active":"2024-09-12T02:57:59.564252+0000","last_peered":"2024-09-12T02:57:59.564252+0000","last_clean":"2024-09-12T02:57:59.564252+0000","last_became_active":"2024-09-12T02:56:15.985167+0000","last_became_peered":"2024-09-12T02:56:15.985167+0000","last_unstale":"2024-09-12T02:57:59.564252+0000","last_undegraded":"2024-09-12T02:57:59.564252+0000","last_fullsized":"2024-09-12T02:57:59.564252+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:46:20.569453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"81'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.035106+0000","last_change":"2024-09-12T02:56:13.929324+0000","last_active":"2024-09-12T02:56:31.035106+0000","last_peered":"2024-09-12T02:56:31.035106+0000","last_clean":"2024-09-12T02:56:31.035106+0000","last_became_active":"2024-09-12T02:56:13.928994+0000","last_became_peered":"2024-09-12T02:56:13.928994+0000","last_unstale":"2024-09-12T02:56:31.035106+0000","last_undegraded":"2024-09-12T02:56:31.035106+0000","last_fullsized":"2024-09-12T02:56:31.035106+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:30:27.290328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.562582+0000","last_change":"2024-09-12T02:56:15.985889+0000","last_active":"2024-09-12T02:57:59.562582+0000","last_peered":"2024-09-12T02:57:59.562582+0000","last_clean":"2024-09-12T02:57:59.562582+0000","last_became_active":"2024-09-12T02:56:15.985509+0000","last_became_peered":"2024-09-12T02:56:15.985509+0000","last_unstale":"2024-09-12T02:57:59.562582+0000","last_undegraded":"2024-09-12T02:57:59.562582+0000","last_fullsized":"2024-09-12T02:57:59.562582+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:30:43.120718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"83'13","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.035437+0000","last_change":"2024-09-12T02:56:13.934192+0000","last_active":"2024-09-12T02:56:31.035437+0000","last_peered":"2024-09-12T02:56:31.035437+0000","last_clean":"2024-09-12T02:56:31.035437+0000","last_became_active":"2024-09-12T02:56:13.933789+0000","last_became_peered":"2024-09-12T02:56:13.933789+0000","last_unstale":"2024-09-12T02:56:31.035437+0000","last_undegraded":"2024-09-12T02:56:31.035437+0000","last_fullsized":"2024-09-12T02:56:31.035437+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:32:47.771085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.563285+0000","last_change":"2024-09-12T02:56:15.983531+0000","last_active":"2024-09-12T02:57:59.563285+0000","last_peered":"2024-09-12T02:57:59.563285+0000","last_clean":"2024-09-12T02:57:59.563285+0000","last_became_active":"2024-09-12T02:56:15.983192+0000","last_became_peered":"2024-09-12T02:56:15.983192+0000","last_unstale":"2024-09-12T02:57:59.563285+0000","last_undegraded":"2024-09-12T02:57:59.563285+0000","last_fullsized":"2024-09-12T02:57:59.563285+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:16:59.923379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.034752+0000","last_change":"2024-09-12T02:56:14.075396+0000","last_active":"2024-09-12T02:56:31.034752+0000","last_peered":"2024-09-12T02:56:31.034752+0000","last_clean":"2024-09-12T02:56:31.034752+0000","last_became_active":"2024-09-12T02:56:14.075023+0000","last_became_peered":"2024-09-12T02:56:14.075023+0000","last_unstale":"2024-09-12T02:56:31.034752+0000","last_undegraded":"2024-09-12T02:56:31.034752+0000","last_fullsized":"2024-09-12T02:56:31.034752+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:41:45.760689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"83'5","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.564093+0000","last_change":"2024-09-12T02:56:15.987276+0000","last_active":"2024-09-12T02:57:59.564093+0000","last_peered":"2024-09-12T02:57:59.564093+0000","last_clean":"2024-09-12T02:57:59.564093+0000","last_became_active":"2024-09-12T02:56:15.986949+0000","last_became_peered":"2024-09-12T02:56:15.986949+0000","last_unstale":"2024-09-12T02:57:59.564093+0000","last_undegraded":"2024-09-12T02:57:59.564093+0000","last_fullsized":"2024-09-12T02:57:59.564093+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:49:45.627093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"83'14","reported_seq":134,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:58:02.977467+0000","last_change":"2024-09-12T02:56:13.932365+0000","last_active":"2024-09-12T02:58:02.977467+0000","last_peered":"2024-09-12T02:58:02.977467+0000","last_clean":"2024-09-12T02:58:02.977467+0000","last_became_active":"2024-09-12T02:56:13.932062+0000","last_became_peered":"2024-09-12T02:56:13.932062+0000","last_unstale":"2024-09-12T02:58:02.977467+0000","last_undegraded":"2024-09-12T02:58:02.977467+0000","last_fullsized":"2024-09-12T02:58:02.977467+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:17:08.451839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":98,"num_read_kb":98,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":74229,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1000,"num_read_kb":580,"num_write":266,"num_write_kb":154,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":734,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":569344,"data_stored":214452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":227,"ondisk_log_size":227,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547235,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":917504,"data_stored":440831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776167,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32756,"kb_used_data":1572,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712396,"statfs":{"total":95995035648,"available":95961493504,"internally_reserved":0,"allocated":1609728,"data_stored":1093472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939693,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40416,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704736,"statfs":{"total":95995035648,"available":95953649664,"internally_reserved":0,"allocated":933888,"data_stored":440115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":40434112},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168626,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":921600,"data_stored":440381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430262,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28516,"kb_used_data":1492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716636,"statfs":{"total":95995035648,"available":95965835264,"internally_reserved":0,"allocated":1527808,"data_stored":1034641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593788,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":917504,"data_stored":440533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822719,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28528,"kb_used_data":1504,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716624,"statfs":{"total":95995035648,"available":95965822976,"internally_reserved":0,"allocated":1540096,"data_stored":1032298,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953541,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":925696,"data_stored":440849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182473,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":933888,"data_stored":439657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411407,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36312,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708840,"statfs":{"total":95995035648,"available":95957852160,"internally_reserved":0,"allocated":991232,"data_stored":502601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574931,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":502469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738457,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":937984,"data_stored":441705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":63784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":63608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":68308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T02:58:06.860 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T02:58:07.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:07.000+0000 7f3ddb111640 1 -- 172.21.15.5:0/3990456533 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3dd4102d20 msgr2=0x7f3dd41031a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:07.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:07.000+0000 7f3ddb111640 1 --2- 172.21.15.5:0/3990456533 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3dd4102d20 0x7f3dd41031a0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f3dc4009ca0 tx=0x7f3dc402a8c0 comp rx=0 tx=0).stop 2024-09-12T02:58:07.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:07.000+0000 7f3ddb111640 1 -- 172.21.15.5:0/3990456533 shutdown_connections 2024-09-12T02:58:07.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:07.000+0000 7f3ddb111640 1 --2- 172.21.15.5:0/3990456533 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3dd4102d20 0x7f3dd41031a0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:07.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:07.000+0000 7f3ddb111640 1 --2- 172.21.15.5:0/3990456533 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3dd41036e0 0x7f3dd4109f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:07.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:07.000+0000 7f3ddb111640 1 --2- 172.21.15.5:0/3990456533 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3dd4101b30 0x7f3dd4101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:07.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:07.000+0000 7f3ddb111640 1 -- 172.21.15.5:0/3990456533 >> 172.21.15.5:0/3990456533 conn(0x7f3dd40fd2b0 msgr2=0x7f3dd40ff6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:07.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:07.000+0000 7f3ddb111640 1 -- 172.21.15.5:0/3990456533 shutdown_connections 2024-09-12T02:58:07.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:07.000+0000 7f3ddb111640 1 -- 172.21.15.5:0/3990456533 wait complete. 2024-09-12T02:58:07.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:07.001+0000 7f3ddb111640 1 Processor -- start 2024-09-12T02:58:07.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:07.001+0000 7f3ddb111640 1 -- start start 2024-09-12T02:58:07.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:07.001+0000 7f3ddb111640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3dd4101b30 0x7f3dd4075fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:07.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:07.001+0000 7f3ddb111640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3dd4102d20 0x7f3dd40745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:07.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:07.001+0000 7f3ddb111640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3dd41036e0 0x7f3dd4074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:07.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:07.001+0000 7f3ddb111640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3dd410d800 con 0x7f3dd41036e0 2024-09-12T02:58:07.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:07.001+0000 7f3ddb111640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f3dd410d680 con 0x7f3dd4102d20 2024-09-12T02:58:07.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:07.001+0000 7f3ddb111640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3dd410d980 con 0x7f3dd4101b30 2024-09-12T02:58:07.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:07.001+0000 7f3dd96ae640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3dd41036e0 0x7f3dd4074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:07.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:07.001+0000 7f3dd96ae640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3dd41036e0 0x7f3dd4074b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55458/0 (socket says 172.21.15.5:55458) 2024-09-12T02:58:07.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:07.001+0000 7f3dd96ae640 1 -- 172.21.15.5:0/3383310854 learned_addr learned my addr 172.21.15.5:0/3383310854 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:58:07.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:07.001+0000 7f3dd8ead640 1 --2- 172.21.15.5:0/3383310854 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3dd4101b30 0x7f3dd4075fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:07.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:07.001+0000 7f3dd3fff640 1 --2- 172.21.15.5:0/3383310854 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3dd4102d20 0x7f3dd40745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:07.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:07.001+0000 7f3dd96ae640 1 -- 172.21.15.5:0/3383310854 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3dd4101b30 msgr2=0x7f3dd4075fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:07.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:07.001+0000 7f3dd96ae640 1 --2- 172.21.15.5:0/3383310854 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3dd4101b30 0x7f3dd4075fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:07.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:07.001+0000 7f3dd96ae640 1 -- 172.21.15.5:0/3383310854 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3dd4102d20 msgr2=0x7f3dd40745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:07.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:07.001+0000 7f3dd96ae640 1 --2- 172.21.15.5:0/3383310854 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3dd4102d20 0x7f3dd40745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:07.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:07.001+0000 7f3dd96ae640 1 -- 172.21.15.5:0/3383310854 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dd40753b0 con 0x7f3dd41036e0 2024-09-12T02:58:07.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:07.001+0000 7f3dd3fff640 1 --2- 172.21.15.5:0/3383310854 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3dd4102d20 0x7f3dd40745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:58:07.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:07.002+0000 7f3dd96ae640 1 --2- 172.21.15.5:0/3383310854 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3dd41036e0 0x7f3dd4074b20 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc00d750 tx=0x7f3dcc00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:07.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:07.002+0000 7f3dd1ffb640 1 -- 172.21.15.5:0/3383310854 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dcc017070 con 0x7f3dd41036e0 2024-09-12T02:58:07.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:07.002+0000 7f3dd1ffb640 1 -- 172.21.15.5:0/3383310854 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3dcc005500 con 0x7f3dd41036e0 2024-09-12T02:58:07.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:07.002+0000 7f3dd1ffb640 1 -- 172.21.15.5:0/3383310854 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dcc004d90 con 0x7f3dd41036e0 2024-09-12T02:58:07.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:07.002+0000 7f3ddb111640 1 -- 172.21.15.5:0/3383310854 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3dd41a1f80 con 0x7f3dd41036e0 2024-09-12T02:58:07.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:07.002+0000 7f3ddb111640 1 -- 172.21.15.5:0/3383310854 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3dd41a2390 con 0x7f3dd41036e0 2024-09-12T02:58:07.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:07.003+0000 7f3ddb111640 1 -- 172.21.15.5:0/3383310854 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3dd4108f20 con 0x7f3dd41036e0 2024-09-12T02:58:07.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:07.004+0000 7f3dd1ffb640 1 -- 172.21.15.5:0/3383310854 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7f3dcc004f30 con 0x7f3dd41036e0 2024-09-12T02:58:07.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:07.005+0000 7f3dd1ffb640 1 --2- 172.21.15.5:0/3383310854 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f3dc00a2c20 0x7f3dc00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:07.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:07.005+0000 7f3dd8ead640 1 --2- 172.21.15.5:0/3383310854 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f3dc00a2c20 0x7f3dc00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:07.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:07.005+0000 7f3dd1ffb640 1 -- 172.21.15.5:0/3383310854 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f3dcc0cebc0 con 0x7f3dd41036e0 2024-09-12T02:58:07.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:07.005+0000 7f3dd8ead640 1 --2- 172.21.15.5:0/3383310854 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f3dc00a2c20 0x7f3dc00a50e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc005fd0 tx=0x7f3dbc005850 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:07.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:07.009+0000 7f3dd1ffb640 1 -- 172.21.15.5:0/3383310854 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3dcc09b9d0 con 0x7f3dd41036e0 2024-09-12T02:58:07.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:07.182+0000 7f3ddb111640 1 -- 172.21.15.5:0/3383310854 --> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3dd4062630 con 0x7f3dc00a2c20 2024-09-12T02:58:07.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:07.188+0000 7f3dd1ffb640 1 -- 172.21.15.5:0/3383310854 <== mgr.14148 v2:172.21.15.5:6800/3780631642 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259021 (secure 0 0 0) 0x7f3dd4062630 con 0x7f3dc00a2c20 2024-09-12T02:58:07.191 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-12T02:58:07.192 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-09-12T02:58:07.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:07.191+0000 7f3ddb111640 1 -- 172.21.15.5:0/3383310854 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f3dc00a2c20 msgr2=0x7f3dc00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:07.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:07.191+0000 7f3ddb111640 1 --2- 172.21.15.5:0/3383310854 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f3dc00a2c20 0x7f3dc00a50e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f3dbc005fd0 tx=0x7f3dbc005850 comp rx=0 tx=0).stop 2024-09-12T02:58:07.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:07.191+0000 7f3ddb111640 1 -- 172.21.15.5:0/3383310854 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3dd41036e0 msgr2=0x7f3dd4074b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:07.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:07.191+0000 7f3ddb111640 1 --2- 172.21.15.5:0/3383310854 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3dd41036e0 0x7f3dd4074b20 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc00d750 tx=0x7f3dcc00dc10 comp rx=0 tx=0).stop 2024-09-12T02:58:07.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:07.191+0000 7f3ddb111640 1 -- 172.21.15.5:0/3383310854 shutdown_connections 2024-09-12T02:58:07.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:07.191+0000 7f3ddb111640 1 --2- 172.21.15.5:0/3383310854 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f3dc00a2c20 0x7f3dc00a50e0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:07.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:07.191+0000 7f3ddb111640 1 --2- 172.21.15.5:0/3383310854 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3dd41036e0 0x7f3dd4074b20 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:07.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:07.191+0000 7f3ddb111640 1 --2- 172.21.15.5:0/3383310854 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3dd4102d20 0x7f3dd40745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:07.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:07.191+0000 7f3ddb111640 1 --2- 172.21.15.5:0/3383310854 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3dd4101b30 0x7f3dd4075fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:07.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:07.191+0000 7f3ddb111640 1 -- 172.21.15.5:0/3383310854 >> 172.21.15.5:0/3383310854 conn(0x7f3dd40fd2b0 msgr2=0x7f3dd41060f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:07.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:07.191+0000 7f3ddb111640 1 -- 172.21.15.5:0/3383310854 shutdown_connections 2024-09-12T02:58:07.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:07.191+0000 7f3ddb111640 1 -- 172.21.15.5:0/3383310854 wait complete. 2024-09-12T02:58:07.202 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":331,"stamp":"2024-09-12T02:58:05.568647+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":664831,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1126,"num_read_kb":689,"num_write":449,"num_write_kb":3094,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":1181470,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":866,"ondisk_log_size":866,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":374556,"kb_used_data":12828,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124567268,"statfs":{"total":1151940427776,"available":1151556882432,"internally_reserved":0,"allocated":13135872,"data_stored":7249552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":370193723},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5854,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":120,"num_read_kb":66,"num_write":20,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002569"},"pg_stats":[{"pgid":"3.3f","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.565050+0000","last_change":"2024-09-12T02:56:15.988690+0000","last_active":"2024-09-12T02:57:59.565050+0000","last_peered":"2024-09-12T02:57:59.565050+0000","last_clean":"2024-09-12T02:57:59.565050+0000","last_became_active":"2024-09-12T02:56:15.988320+0000","last_became_peered":"2024-09-12T02:56:15.988320+0000","last_unstale":"2024-09-12T02:57:59.565050+0000","last_undegraded":"2024-09-12T02:57:59.565050+0000","last_fullsized":"2024-09-12T02:57:59.565050+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:31:14.157622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"83'4","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.553842+0000","last_change":"2024-09-12T02:56:15.978368+0000","last_active":"2024-09-12T02:57:59.553842+0000","last_peered":"2024-09-12T02:57:59.553842+0000","last_clean":"2024-09-12T02:57:59.553842+0000","last_became_active":"2024-09-12T02:56:15.978092+0000","last_became_peered":"2024-09-12T02:56:15.978092+0000","last_unstale":"2024-09-12T02:57:59.553842+0000","last_undegraded":"2024-09-12T02:57:59.553842+0000","last_fullsized":"2024-09-12T02:57:59.553842+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:32:48.399005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.564003+0000","last_change":"2024-09-12T02:56:15.987249+0000","last_active":"2024-09-12T02:57:59.564003+0000","last_peered":"2024-09-12T02:57:59.564003+0000","last_clean":"2024-09-12T02:57:59.564003+0000","last_became_active":"2024-09-12T02:56:15.986921+0000","last_became_peered":"2024-09-12T02:56:15.986921+0000","last_unstale":"2024-09-12T02:57:59.564003+0000","last_undegraded":"2024-09-12T02:57:59.564003+0000","last_fullsized":"2024-09-12T02:57:59.564003+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:42:13.427871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.555703+0000","last_change":"2024-09-12T02:56:15.987673+0000","last_active":"2024-09-12T02:57:59.555703+0000","last_peered":"2024-09-12T02:57:59.555703+0000","last_clean":"2024-09-12T02:57:59.555703+0000","last_became_active":"2024-09-12T02:56:15.986964+0000","last_became_peered":"2024-09-12T02:56:15.986964+0000","last_unstale":"2024-09-12T02:57:59.555703+0000","last_undegraded":"2024-09-12T02:57:59.555703+0000","last_fullsized":"2024-09-12T02:57:59.555703+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:07:12.406974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"83'4","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.548461+0000","last_change":"2024-09-12T02:56:15.974473+0000","last_active":"2024-09-12T02:57:59.548461+0000","last_peered":"2024-09-12T02:57:59.548461+0000","last_clean":"2024-09-12T02:57:59.548461+0000","last_became_active":"2024-09-12T02:56:15.974124+0000","last_became_peered":"2024-09-12T02:56:15.974124+0000","last_unstale":"2024-09-12T02:57:59.548461+0000","last_undegraded":"2024-09-12T02:57:59.548461+0000","last_fullsized":"2024-09-12T02:57:59.548461+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:44:29.054841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.560740+0000","last_change":"2024-09-12T02:56:15.980242+0000","last_active":"2024-09-12T02:57:59.560740+0000","last_peered":"2024-09-12T02:57:59.560740+0000","last_clean":"2024-09-12T02:57:59.560740+0000","last_became_active":"2024-09-12T02:56:15.979937+0000","last_became_peered":"2024-09-12T02:56:15.979937+0000","last_unstale":"2024-09-12T02:57:59.560740+0000","last_undegraded":"2024-09-12T02:57:59.560740+0000","last_fullsized":"2024-09-12T02:57:59.560740+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:44:13.406341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"83'5","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.556845+0000","last_change":"2024-09-12T02:56:15.978746+0000","last_active":"2024-09-12T02:57:59.556845+0000","last_peered":"2024-09-12T02:57:59.556845+0000","last_clean":"2024-09-12T02:57:59.556845+0000","last_became_active":"2024-09-12T02:56:15.978304+0000","last_became_peered":"2024-09-12T02:56:15.978304+0000","last_unstale":"2024-09-12T02:57:59.556845+0000","last_undegraded":"2024-09-12T02:57:59.556845+0000","last_fullsized":"2024-09-12T02:57:59.556845+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:34:52.115630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.562507+0000","last_change":"2024-09-12T02:56:15.980083+0000","last_active":"2024-09-12T02:57:59.562507+0000","last_peered":"2024-09-12T02:57:59.562507+0000","last_clean":"2024-09-12T02:57:59.562507+0000","last_became_active":"2024-09-12T02:56:15.979771+0000","last_became_peered":"2024-09-12T02:56:15.979771+0000","last_unstale":"2024-09-12T02:57:59.562507+0000","last_undegraded":"2024-09-12T02:57:59.562507+0000","last_fullsized":"2024-09-12T02:57:59.562507+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:59:42.351970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"83'11","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.565703+0000","last_change":"2024-09-12T02:56:15.988091+0000","last_active":"2024-09-12T02:57:59.565703+0000","last_peered":"2024-09-12T02:57:59.565703+0000","last_clean":"2024-09-12T02:57:59.565703+0000","last_became_active":"2024-09-12T02:56:15.986295+0000","last_became_peered":"2024-09-12T02:56:15.986295+0000","last_unstale":"2024-09-12T02:57:59.565703+0000","last_undegraded":"2024-09-12T02:57:59.565703+0000","last_fullsized":"2024-09-12T02:57:59.565703+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:45:40.331805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"83'16","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.562128+0000","last_change":"2024-09-12T02:56:15.987349+0000","last_active":"2024-09-12T02:57:59.562128+0000","last_peered":"2024-09-12T02:57:59.562128+0000","last_clean":"2024-09-12T02:57:59.562128+0000","last_became_active":"2024-09-12T02:56:15.987018+0000","last_became_peered":"2024-09-12T02:56:15.987018+0000","last_unstale":"2024-09-12T02:57:59.562128+0000","last_undegraded":"2024-09-12T02:57:59.562128+0000","last_fullsized":"2024-09-12T02:57:59.562128+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:34:19.652348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.563544+0000","last_change":"2024-09-12T02:56:15.985164+0000","last_active":"2024-09-12T02:57:59.563544+0000","last_peered":"2024-09-12T02:57:59.563544+0000","last_clean":"2024-09-12T02:57:59.563544+0000","last_became_active":"2024-09-12T02:56:15.984787+0000","last_became_peered":"2024-09-12T02:56:15.984787+0000","last_unstale":"2024-09-12T02:57:59.563544+0000","last_undegraded":"2024-09-12T02:57:59.563544+0000","last_fullsized":"2024-09-12T02:57:59.563544+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:36:29.727442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.552957+0000","last_change":"2024-09-12T02:56:15.979963+0000","last_active":"2024-09-12T02:57:59.552957+0000","last_peered":"2024-09-12T02:57:59.552957+0000","last_clean":"2024-09-12T02:57:59.552957+0000","last_became_active":"2024-09-12T02:56:15.979639+0000","last_became_peered":"2024-09-12T02:56:15.979639+0000","last_unstale":"2024-09-12T02:57:59.552957+0000","last_undegraded":"2024-09-12T02:57:59.552957+0000","last_fullsized":"2024-09-12T02:57:59.552957+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:34:18.273245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"83'10","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.561647+0000","last_change":"2024-09-12T02:56:15.980415+0000","last_active":"2024-09-12T02:57:59.561647+0000","last_peered":"2024-09-12T02:57:59.561647+0000","last_clean":"2024-09-12T02:57:59.561647+0000","last_became_active":"2024-09-12T02:56:15.980059+0000","last_became_peered":"2024-09-12T02:56:15.980059+0000","last_unstale":"2024-09-12T02:57:59.561647+0000","last_undegraded":"2024-09-12T02:57:59.561647+0000","last_fullsized":"2024-09-12T02:57:59.561647+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:11:42.689081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.553574+0000","last_change":"2024-09-12T02:56:15.981423+0000","last_active":"2024-09-12T02:57:59.553574+0000","last_peered":"2024-09-12T02:57:59.553574+0000","last_clean":"2024-09-12T02:57:59.553574+0000","last_became_active":"2024-09-12T02:56:15.981186+0000","last_became_peered":"2024-09-12T02:56:15.981186+0000","last_unstale":"2024-09-12T02:57:59.553574+0000","last_undegraded":"2024-09-12T02:57:59.553574+0000","last_fullsized":"2024-09-12T02:57:59.553574+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:36:57.406907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.547696+0000","last_change":"2024-09-12T02:56:15.982587+0000","last_active":"2024-09-12T02:57:59.547696+0000","last_peered":"2024-09-12T02:57:59.547696+0000","last_clean":"2024-09-12T02:57:59.547696+0000","last_became_active":"2024-09-12T02:56:15.982315+0000","last_became_peered":"2024-09-12T02:56:15.982315+0000","last_unstale":"2024-09-12T02:57:59.547696+0000","last_undegraded":"2024-09-12T02:57:59.547696+0000","last_fullsized":"2024-09-12T02:57:59.547696+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:42:17.736999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.564760+0000","last_change":"2024-09-12T02:56:15.979482+0000","last_active":"2024-09-12T02:57:59.564760+0000","last_peered":"2024-09-12T02:57:59.564760+0000","last_clean":"2024-09-12T02:57:59.564760+0000","last_became_active":"2024-09-12T02:56:15.979122+0000","last_became_peered":"2024-09-12T02:56:15.979122+0000","last_unstale":"2024-09-12T02:57:59.564760+0000","last_undegraded":"2024-09-12T02:57:59.564760+0000","last_fullsized":"2024-09-12T02:57:59.564760+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:18:05.642487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.565218+0000","last_change":"2024-09-12T02:56:15.989908+0000","last_active":"2024-09-12T02:57:59.565218+0000","last_peered":"2024-09-12T02:57:59.565218+0000","last_clean":"2024-09-12T02:57:59.565218+0000","last_became_active":"2024-09-12T02:56:15.989515+0000","last_became_peered":"2024-09-12T02:56:15.989515+0000","last_unstale":"2024-09-12T02:57:59.565218+0000","last_undegraded":"2024-09-12T02:57:59.565218+0000","last_fullsized":"2024-09-12T02:57:59.565218+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:59:57.141607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"83'10","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.557609+0000","last_change":"2024-09-12T02:56:15.980323+0000","last_active":"2024-09-12T02:57:59.557609+0000","last_peered":"2024-09-12T02:57:59.557609+0000","last_clean":"2024-09-12T02:57:59.557609+0000","last_became_active":"2024-09-12T02:56:15.979989+0000","last_became_peered":"2024-09-12T02:56:15.979989+0000","last_unstale":"2024-09-12T02:57:59.557609+0000","last_undegraded":"2024-09-12T02:57:59.557609+0000","last_fullsized":"2024-09-12T02:57:59.557609+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:55:56.064125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.556275+0000","last_change":"2024-09-12T02:56:15.986003+0000","last_active":"2024-09-12T02:57:59.556275+0000","last_peered":"2024-09-12T02:57:59.556275+0000","last_clean":"2024-09-12T02:57:59.556275+0000","last_became_active":"2024-09-12T02:56:15.985335+0000","last_became_peered":"2024-09-12T02:56:15.985335+0000","last_unstale":"2024-09-12T02:57:59.556275+0000","last_undegraded":"2024-09-12T02:57:59.556275+0000","last_fullsized":"2024-09-12T02:57:59.556275+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:51:47.944530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"83'8","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.554254+0000","last_change":"2024-09-12T02:56:31.036870+0000","last_active":"2024-09-12T02:57:59.554254+0000","last_peered":"2024-09-12T02:57:59.554254+0000","last_clean":"2024-09-12T02:57:59.554254+0000","last_became_active":"2024-09-12T02:56:31.036443+0000","last_became_peered":"2024-09-12T02:56:31.036443+0000","last_unstale":"2024-09-12T02:57:59.554254+0000","last_undegraded":"2024-09-12T02:57:59.554254+0000","last_fullsized":"2024-09-12T02:57:59.554254+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:49:29.326184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"83'11","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.555414+0000","last_change":"2024-09-12T02:56:15.974547+0000","last_active":"2024-09-12T02:57:59.555414+0000","last_peered":"2024-09-12T02:57:59.555414+0000","last_clean":"2024-09-12T02:57:59.555414+0000","last_became_active":"2024-09-12T02:56:15.974174+0000","last_became_peered":"2024-09-12T02:56:15.974174+0000","last_unstale":"2024-09-12T02:57:59.555414+0000","last_undegraded":"2024-09-12T02:57:59.555414+0000","last_fullsized":"2024-09-12T02:57:59.555414+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:22:16.954309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"83'10","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.565778+0000","last_change":"2024-09-12T02:56:15.981771+0000","last_active":"2024-09-12T02:57:59.565778+0000","last_peered":"2024-09-12T02:57:59.565778+0000","last_clean":"2024-09-12T02:57:59.565778+0000","last_became_active":"2024-09-12T02:56:15.980948+0000","last_became_peered":"2024-09-12T02:56:15.980948+0000","last_unstale":"2024-09-12T02:57:59.565778+0000","last_undegraded":"2024-09-12T02:57:59.565778+0000","last_fullsized":"2024-09-12T02:57:59.565778+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:34:35.961348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.562061+0000","last_change":"2024-09-12T02:56:15.982318+0000","last_active":"2024-09-12T02:57:59.562061+0000","last_peered":"2024-09-12T02:57:59.562061+0000","last_clean":"2024-09-12T02:57:59.562061+0000","last_became_active":"2024-09-12T02:56:15.982013+0000","last_became_peered":"2024-09-12T02:56:15.982013+0000","last_unstale":"2024-09-12T02:57:59.562061+0000","last_undegraded":"2024-09-12T02:57:59.562061+0000","last_fullsized":"2024-09-12T02:57:59.562061+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:10:50.895579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.559580+0000","last_change":"2024-09-12T02:56:15.969992+0000","last_active":"2024-09-12T02:57:59.559580+0000","last_peered":"2024-09-12T02:57:59.559580+0000","last_clean":"2024-09-12T02:57:59.559580+0000","last_became_active":"2024-09-12T02:56:15.969621+0000","last_became_peered":"2024-09-12T02:56:15.969621+0000","last_unstale":"2024-09-12T02:57:59.559580+0000","last_undegraded":"2024-09-12T02:57:59.559580+0000","last_fullsized":"2024-09-12T02:57:59.559580+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:47:40.206042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"83'2","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.560304+0000","last_change":"2024-09-12T02:56:15.982315+0000","last_active":"2024-09-12T02:57:59.560304+0000","last_peered":"2024-09-12T02:57:59.560304+0000","last_clean":"2024-09-12T02:57:59.560304+0000","last_became_active":"2024-09-12T02:56:15.981992+0000","last_became_peered":"2024-09-12T02:56:15.981992+0000","last_unstale":"2024-09-12T02:57:59.560304+0000","last_undegraded":"2024-09-12T02:57:59.560304+0000","last_fullsized":"2024-09-12T02:57:59.560304+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:54:04.188418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"83'15","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.560612+0000","last_change":"2024-09-12T02:56:15.980712+0000","last_active":"2024-09-12T02:57:59.560612+0000","last_peered":"2024-09-12T02:57:59.560612+0000","last_clean":"2024-09-12T02:57:59.560612+0000","last_became_active":"2024-09-12T02:56:15.980410+0000","last_became_peered":"2024-09-12T02:56:15.980410+0000","last_unstale":"2024-09-12T02:57:59.560612+0000","last_undegraded":"2024-09-12T02:57:59.560612+0000","last_fullsized":"2024-09-12T02:57:59.560612+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:13:53.844490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.557058+0000","last_change":"2024-09-12T02:56:15.981675+0000","last_active":"2024-09-12T02:57:59.557058+0000","last_peered":"2024-09-12T02:57:59.557058+0000","last_clean":"2024-09-12T02:57:59.557058+0000","last_became_active":"2024-09-12T02:56:15.981372+0000","last_became_peered":"2024-09-12T02:56:15.981372+0000","last_unstale":"2024-09-12T02:57:59.557058+0000","last_undegraded":"2024-09-12T02:57:59.557058+0000","last_fullsized":"2024-09-12T02:57:59.557058+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:39:17.399199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"83'4","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.557413+0000","last_change":"2024-09-12T02:56:15.972750+0000","last_active":"2024-09-12T02:57:59.557413+0000","last_peered":"2024-09-12T02:57:59.557413+0000","last_clean":"2024-09-12T02:57:59.557413+0000","last_became_active":"2024-09-12T02:56:15.972434+0000","last_became_peered":"2024-09-12T02:56:15.972434+0000","last_unstale":"2024-09-12T02:57:59.557413+0000","last_undegraded":"2024-09-12T02:57:59.557413+0000","last_fullsized":"2024-09-12T02:57:59.557413+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:59:22.784908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.545235+0000","last_change":"2024-09-12T02:56:15.973567+0000","last_active":"2024-09-12T02:57:59.545235+0000","last_peered":"2024-09-12T02:57:59.545235+0000","last_clean":"2024-09-12T02:57:59.545235+0000","last_became_active":"2024-09-12T02:56:15.973251+0000","last_became_peered":"2024-09-12T02:56:15.973251+0000","last_unstale":"2024-09-12T02:57:59.545235+0000","last_undegraded":"2024-09-12T02:57:59.545235+0000","last_fullsized":"2024-09-12T02:57:59.545235+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:16:39.536817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.540308+0000","last_change":"2024-09-12T02:56:15.986108+0000","last_active":"2024-09-12T02:57:59.540308+0000","last_peered":"2024-09-12T02:57:59.540308+0000","last_clean":"2024-09-12T02:57:59.540308+0000","last_became_active":"2024-09-12T02:56:15.985701+0000","last_became_peered":"2024-09-12T02:56:15.985701+0000","last_unstale":"2024-09-12T02:57:59.540308+0000","last_undegraded":"2024-09-12T02:57:59.540308+0000","last_fullsized":"2024-09-12T02:57:59.540308+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:59.377473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"83'12","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.555638+0000","last_change":"2024-09-12T02:56:15.980078+0000","last_active":"2024-09-12T02:57:59.555638+0000","last_peered":"2024-09-12T02:57:59.555638+0000","last_clean":"2024-09-12T02:57:59.555638+0000","last_became_active":"2024-09-12T02:56:15.979785+0000","last_became_peered":"2024-09-12T02:56:15.979785+0000","last_unstale":"2024-09-12T02:57:59.555638+0000","last_undegraded":"2024-09-12T02:57:59.555638+0000","last_fullsized":"2024-09-12T02:57:59.555638+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:00:19.252398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.559119+0000","last_change":"2024-09-12T02:56:15.984170+0000","last_active":"2024-09-12T02:57:59.559119+0000","last_peered":"2024-09-12T02:57:59.559119+0000","last_clean":"2024-09-12T02:57:59.559119+0000","last_became_active":"2024-09-12T02:56:15.983416+0000","last_became_peered":"2024-09-12T02:56:15.983416+0000","last_unstale":"2024-09-12T02:57:59.559119+0000","last_undegraded":"2024-09-12T02:57:59.559119+0000","last_fullsized":"2024-09-12T02:57:59.559119+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:13:24.723923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.556295+0000","last_change":"2024-09-12T02:56:15.980987+0000","last_active":"2024-09-12T02:57:59.556295+0000","last_peered":"2024-09-12T02:57:59.556295+0000","last_clean":"2024-09-12T02:57:59.556295+0000","last_became_active":"2024-09-12T02:56:15.980661+0000","last_became_peered":"2024-09-12T02:56:15.980661+0000","last_unstale":"2024-09-12T02:57:59.556295+0000","last_undegraded":"2024-09-12T02:57:59.556295+0000","last_fullsized":"2024-09-12T02:57:59.556295+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:19:49.294698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"81'5","reported_seq":135,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:58:00.172973+0000","last_change":"2024-09-12T02:56:31.042914+0000","last_active":"2024-09-12T02:58:00.172973+0000","last_peered":"2024-09-12T02:58:00.172973+0000","last_clean":"2024-09-12T02:58:00.172973+0000","last_became_active":"2024-09-12T02:56:31.037356+0000","last_became_peered":"2024-09-12T02:56:31.037356+0000","last_unstale":"2024-09-12T02:58:00.172973+0000","last_undegraded":"2024-09-12T02:58:00.172973+0000","last_fullsized":"2024-09-12T02:58:00.172973+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:17:48.582725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":622,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"83'8","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.565986+0000","last_change":"2024-09-12T02:56:31.036170+0000","last_active":"2024-09-12T02:57:59.565986+0000","last_peered":"2024-09-12T02:57:59.565986+0000","last_clean":"2024-09-12T02:57:59.565986+0000","last_became_active":"2024-09-12T02:56:31.035754+0000","last_became_peered":"2024-09-12T02:56:31.035754+0000","last_unstale":"2024-09-12T02:57:59.565986+0000","last_undegraded":"2024-09-12T02:57:59.565986+0000","last_fullsized":"2024-09-12T02:57:59.565986+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:58:06.464818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1a","version":"81'7","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.032813+0000","last_change":"2024-09-12T02:56:13.931202+0000","last_active":"2024-09-12T02:56:31.032813+0000","last_peered":"2024-09-12T02:56:31.032813+0000","last_clean":"2024-09-12T02:56:31.032813+0000","last_became_active":"2024-09-12T02:56:13.930776+0000","last_became_peered":"2024-09-12T02:56:13.930776+0000","last_unstale":"2024-09-12T02:56:31.032813+0000","last_undegraded":"2024-09-12T02:56:31.032813+0000","last_fullsized":"2024-09-12T02:56:31.032813+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:22:16.919253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"81'3","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.031662+0000","last_change":"2024-09-12T02:56:13.924660+0000","last_active":"2024-09-12T02:56:31.031662+0000","last_peered":"2024-09-12T02:56:31.031662+0000","last_clean":"2024-09-12T02:56:31.031662+0000","last_became_active":"2024-09-12T02:56:13.924342+0000","last_became_peered":"2024-09-12T02:56:13.924342+0000","last_unstale":"2024-09-12T02:56:31.031662+0000","last_undegraded":"2024-09-12T02:56:31.031662+0000","last_fullsized":"2024-09-12T02:56:31.031662+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:48:53.907044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.563814+0000","last_change":"2024-09-12T02:56:15.978723+0000","last_active":"2024-09-12T02:57:59.563814+0000","last_peered":"2024-09-12T02:57:59.563814+0000","last_clean":"2024-09-12T02:57:59.563814+0000","last_became_active":"2024-09-12T02:56:15.978344+0000","last_became_peered":"2024-09-12T02:56:15.978344+0000","last_unstale":"2024-09-12T02:57:59.563814+0000","last_undegraded":"2024-09-12T02:57:59.563814+0000","last_fullsized":"2024-09-12T02:57:59.563814+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:25:04.559395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"81'3","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.032936+0000","last_change":"2024-09-12T02:56:13.933065+0000","last_active":"2024-09-12T02:56:31.032936+0000","last_peered":"2024-09-12T02:56:31.032936+0000","last_clean":"2024-09-12T02:56:31.032936+0000","last_became_active":"2024-09-12T02:56:13.932597+0000","last_became_peered":"2024-09-12T02:56:13.932597+0000","last_unstale":"2024-09-12T02:56:31.032936+0000","last_undegraded":"2024-09-12T02:56:31.032936+0000","last_fullsized":"2024-09-12T02:56:31.032936+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:57:26.563392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"83'11","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.566801+0000","last_change":"2024-09-12T02:56:15.975722+0000","last_active":"2024-09-12T02:57:59.566801+0000","last_peered":"2024-09-12T02:57:59.566801+0000","last_clean":"2024-09-12T02:57:59.566801+0000","last_became_active":"2024-09-12T02:56:15.975342+0000","last_became_peered":"2024-09-12T02:56:15.975342+0000","last_unstale":"2024-09-12T02:57:59.566801+0000","last_undegraded":"2024-09-12T02:57:59.566801+0000","last_fullsized":"2024-09-12T02:57:59.566801+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:33:17.613738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'5","reported_seq":217,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:58:01.977448+0000","last_change":"2024-09-12T02:56:13.929700+0000","last_active":"2024-09-12T02:58:01.977448+0000","last_peered":"2024-09-12T02:58:01.977448+0000","last_clean":"2024-09-12T02:58:01.977448+0000","last_became_active":"2024-09-12T02:56:13.929317+0000","last_became_peered":"2024-09-12T02:56:13.929317+0000","last_unstale":"2024-09-12T02:58:01.977448+0000","last_undegraded":"2024-09-12T02:58:01.977448+0000","last_fullsized":"2024-09-12T02:58:01.977448+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:46:32.172674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":190,"num_read_kb":190,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"83'10","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.564710+0000","last_change":"2024-09-12T02:56:15.985754+0000","last_active":"2024-09-12T02:57:59.564710+0000","last_peered":"2024-09-12T02:57:59.564710+0000","last_clean":"2024-09-12T02:57:59.564710+0000","last_became_active":"2024-09-12T02:56:15.985402+0000","last_became_peered":"2024-09-12T02:56:15.985402+0000","last_unstale":"2024-09-12T02:57:59.564710+0000","last_undegraded":"2024-09-12T02:57:59.564710+0000","last_fullsized":"2024-09-12T02:57:59.564710+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:05:29.979961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"81'4","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.035428+0000","last_change":"2024-09-12T02:56:13.933329+0000","last_active":"2024-09-12T02:56:31.035428+0000","last_peered":"2024-09-12T02:56:31.035428+0000","last_clean":"2024-09-12T02:56:31.035428+0000","last_became_active":"2024-09-12T02:56:13.932960+0000","last_became_peered":"2024-09-12T02:56:13.932960+0000","last_unstale":"2024-09-12T02:56:31.035428+0000","last_undegraded":"2024-09-12T02:56:31.035428+0000","last_fullsized":"2024-09-12T02:56:31.035428+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:02:57.564202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"83'9","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.556804+0000","last_change":"2024-09-12T02:56:31.037340+0000","last_active":"2024-09-12T02:57:59.556804+0000","last_peered":"2024-09-12T02:57:59.556804+0000","last_clean":"2024-09-12T02:57:59.556804+0000","last_became_active":"2024-09-12T02:56:31.036993+0000","last_became_peered":"2024-09-12T02:56:31.036993+0000","last_unstale":"2024-09-12T02:57:59.556804+0000","last_undegraded":"2024-09-12T02:57:59.556804+0000","last_fullsized":"2024-09-12T02:57:59.556804+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:44:27.788791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"81'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.033005+0000","last_change":"2024-09-12T02:56:14.077202+0000","last_active":"2024-09-12T02:56:31.033005+0000","last_peered":"2024-09-12T02:56:31.033005+0000","last_clean":"2024-09-12T02:56:31.033005+0000","last_became_active":"2024-09-12T02:56:14.076610+0000","last_became_peered":"2024-09-12T02:56:14.076610+0000","last_unstale":"2024-09-12T02:56:31.033005+0000","last_undegraded":"2024-09-12T02:56:31.033005+0000","last_fullsized":"2024-09-12T02:56:31.033005+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:37:50.333152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"83'11","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.558754+0000","last_change":"2024-09-12T02:56:15.977161+0000","last_active":"2024-09-12T02:57:59.558754+0000","last_peered":"2024-09-12T02:57:59.558754+0000","last_clean":"2024-09-12T02:57:59.558754+0000","last_became_active":"2024-09-12T02:56:15.976683+0000","last_became_peered":"2024-09-12T02:56:15.976683+0000","last_unstale":"2024-09-12T02:57:59.558754+0000","last_undegraded":"2024-09-12T02:57:59.558754+0000","last_fullsized":"2024-09-12T02:57:59.558754+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:34:07.526890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'36","reported_seq":259,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:58:01.977471+0000","last_change":"2024-09-12T02:56:13.931680+0000","last_active":"2024-09-12T02:58:01.977471+0000","last_peered":"2024-09-12T02:58:01.977471+0000","last_clean":"2024-09-12T02:58:01.977471+0000","last_became_active":"2024-09-12T02:56:13.931214+0000","last_became_peered":"2024-09-12T02:56:13.931214+0000","last_unstale":"2024-09-12T02:58:01.977471+0000","last_undegraded":"2024-09-12T02:58:01.977471+0000","last_fullsized":"2024-09-12T02:58:01.977471+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:38:34.151933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66038,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":201,"num_read_kb":160,"num_write":38,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.555616+0000","last_change":"2024-09-12T02:56:15.977720+0000","last_active":"2024-09-12T02:57:59.555616+0000","last_peered":"2024-09-12T02:57:59.555616+0000","last_clean":"2024-09-12T02:57:59.555616+0000","last_became_active":"2024-09-12T02:56:15.977380+0000","last_became_peered":"2024-09-12T02:56:15.977380+0000","last_unstale":"2024-09-12T02:57:59.555616+0000","last_undegraded":"2024-09-12T02:57:59.555616+0000","last_fullsized":"2024-09-12T02:57:59.555616+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:52:16.841909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"81'9","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.030362+0000","last_change":"2024-09-12T02:56:13.929496+0000","last_active":"2024-09-12T02:56:31.030362+0000","last_peered":"2024-09-12T02:56:31.030362+0000","last_clean":"2024-09-12T02:56:31.030362+0000","last_became_active":"2024-09-12T02:56:13.929117+0000","last_became_peered":"2024-09-12T02:56:13.929117+0000","last_unstale":"2024-09-12T02:56:31.030362+0000","last_undegraded":"2024-09-12T02:56:31.030362+0000","last_fullsized":"2024-09-12T02:56:31.030362+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:59:38.399135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.564811+0000","last_change":"2024-09-12T02:56:15.974323+0000","last_active":"2024-09-12T02:57:59.564811+0000","last_peered":"2024-09-12T02:57:59.564811+0000","last_clean":"2024-09-12T02:57:59.564811+0000","last_became_active":"2024-09-12T02:56:15.973819+0000","last_became_peered":"2024-09-12T02:56:15.973819+0000","last_unstale":"2024-09-12T02:57:59.564811+0000","last_undegraded":"2024-09-12T02:57:59.564811+0000","last_fullsized":"2024-09-12T02:57:59.564811+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:53:13.130695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"81'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.032473+0000","last_change":"2024-09-12T02:56:13.935263+0000","last_active":"2024-09-12T02:56:31.032473+0000","last_peered":"2024-09-12T02:56:31.032473+0000","last_clean":"2024-09-12T02:56:31.032473+0000","last_became_active":"2024-09-12T02:56:13.934805+0000","last_became_peered":"2024-09-12T02:56:13.934805+0000","last_unstale":"2024-09-12T02:56:31.032473+0000","last_undegraded":"2024-09-12T02:56:31.032473+0000","last_fullsized":"2024-09-12T02:56:31.032473+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:39:42.882031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"83'15","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.560318+0000","last_change":"2024-09-12T02:56:15.976630+0000","last_active":"2024-09-12T02:57:59.560318+0000","last_peered":"2024-09-12T02:57:59.560318+0000","last_clean":"2024-09-12T02:57:59.560318+0000","last_became_active":"2024-09-12T02:56:15.976304+0000","last_became_peered":"2024-09-12T02:56:15.976304+0000","last_unstale":"2024-09-12T02:57:59.560318+0000","last_undegraded":"2024-09-12T02:57:59.560318+0000","last_fullsized":"2024-09-12T02:57:59.560318+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:46:03.366740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'7","reported_seq":126,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.973453+0000","last_change":"2024-09-12T02:56:13.934224+0000","last_active":"2024-09-12T02:57:59.973453+0000","last_peered":"2024-09-12T02:57:59.973453+0000","last_clean":"2024-09-12T02:57:59.973453+0000","last_became_active":"2024-09-12T02:56:13.933838+0000","last_became_peered":"2024-09-12T02:56:13.933838+0000","last_unstale":"2024-09-12T02:57:59.973453+0000","last_undegraded":"2024-09-12T02:57:59.973453+0000","last_fullsized":"2024-09-12T02:57:59.973453+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:34:34.206051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":98,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.557213+0000","last_change":"2024-09-12T02:56:15.980364+0000","last_active":"2024-09-12T02:57:59.557213+0000","last_peered":"2024-09-12T02:57:59.557213+0000","last_clean":"2024-09-12T02:57:59.557213+0000","last_became_active":"2024-09-12T02:56:15.979437+0000","last_became_peered":"2024-09-12T02:56:15.979437+0000","last_unstale":"2024-09-12T02:57:59.557213+0000","last_undegraded":"2024-09-12T02:57:59.557213+0000","last_fullsized":"2024-09-12T02:57:59.557213+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:43:33.424867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"81'17","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.050509+0000","last_change":"2024-09-12T02:56:31.050046+0000","last_active":"2024-09-12T02:56:31.050509+0000","last_peered":"2024-09-12T02:56:31.050509+0000","last_clean":"2024-09-12T02:56:31.050509+0000","last_became_active":"2024-09-12T02:56:31.036990+0000","last_became_peered":"2024-09-12T02:56:31.036990+0000","last_unstale":"2024-09-12T02:56:31.050509+0000","last_undegraded":"2024-09-12T02:56:31.050509+0000","last_fullsized":"2024-09-12T02:56:31.050509+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:05:53.336691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.563297+0000","last_change":"2024-09-12T02:56:15.986434+0000","last_active":"2024-09-12T02:57:59.563297+0000","last_peered":"2024-09-12T02:57:59.563297+0000","last_clean":"2024-09-12T02:57:59.563297+0000","last_became_active":"2024-09-12T02:56:15.986101+0000","last_became_peered":"2024-09-12T02:56:15.986101+0000","last_unstale":"2024-09-12T02:57:59.563297+0000","last_undegraded":"2024-09-12T02:57:59.563297+0000","last_fullsized":"2024-09-12T02:57:59.563297+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:51:27.840951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"81'2","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.035974+0000","last_change":"2024-09-12T02:56:13.925068+0000","last_active":"2024-09-12T02:56:31.035974+0000","last_peered":"2024-09-12T02:56:31.035974+0000","last_clean":"2024-09-12T02:56:31.035974+0000","last_became_active":"2024-09-12T02:56:13.924535+0000","last_became_peered":"2024-09-12T02:56:13.924535+0000","last_unstale":"2024-09-12T02:56:31.035974+0000","last_undegraded":"2024-09-12T02:56:31.035974+0000","last_fullsized":"2024-09-12T02:56:31.035974+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:45:32.478632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"83'5","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.559020+0000","last_change":"2024-09-12T02:56:15.983850+0000","last_active":"2024-09-12T02:57:59.559020+0000","last_peered":"2024-09-12T02:57:59.559020+0000","last_clean":"2024-09-12T02:57:59.559020+0000","last_became_active":"2024-09-12T02:56:15.983480+0000","last_became_peered":"2024-09-12T02:56:15.983480+0000","last_unstale":"2024-09-12T02:57:59.559020+0000","last_undegraded":"2024-09-12T02:57:59.559020+0000","last_fullsized":"2024-09-12T02:57:59.559020+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:09:42.185237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"81'5","reported_seq":39,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.312450+0000","last_change":"2024-09-12T02:56:31.312450+0000","last_active":"2024-09-12T02:56:31.312450+0000","last_peered":"2024-09-12T02:56:31.312450+0000","last_clean":"2024-09-12T02:56:31.312450+0000","last_became_active":"2024-09-12T02:56:31.308030+0000","last_became_peered":"2024-09-12T02:56:31.308030+0000","last_unstale":"2024-09-12T02:56:31.312450+0000","last_undegraded":"2024-09-12T02:56:31.312450+0000","last_fullsized":"2024-09-12T02:56:31.312450+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:15:14.692292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.562640+0000","last_change":"2024-09-12T02:56:15.978996+0000","last_active":"2024-09-12T02:57:59.562640+0000","last_peered":"2024-09-12T02:57:59.562640+0000","last_clean":"2024-09-12T02:57:59.562640+0000","last_became_active":"2024-09-12T02:56:15.978636+0000","last_became_peered":"2024-09-12T02:56:15.978636+0000","last_unstale":"2024-09-12T02:57:59.562640+0000","last_undegraded":"2024-09-12T02:57:59.562640+0000","last_fullsized":"2024-09-12T02:57:59.562640+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:29:20.401664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"81'3","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.033119+0000","last_change":"2024-09-12T02:56:13.929388+0000","last_active":"2024-09-12T02:56:31.033119+0000","last_peered":"2024-09-12T02:56:31.033119+0000","last_clean":"2024-09-12T02:56:31.033119+0000","last_became_active":"2024-09-12T02:56:13.928998+0000","last_became_peered":"2024-09-12T02:56:13.928998+0000","last_unstale":"2024-09-12T02:56:31.033119+0000","last_undegraded":"2024-09-12T02:56:31.033119+0000","last_fullsized":"2024-09-12T02:56:31.033119+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:37:57.368050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.560190+0000","last_change":"2024-09-12T02:56:15.977491+0000","last_active":"2024-09-12T02:57:59.560190+0000","last_peered":"2024-09-12T02:57:59.560190+0000","last_clean":"2024-09-12T02:57:59.560190+0000","last_became_active":"2024-09-12T02:56:15.977160+0000","last_became_peered":"2024-09-12T02:56:15.977160+0000","last_unstale":"2024-09-12T02:57:59.560190+0000","last_undegraded":"2024-09-12T02:57:59.560190+0000","last_fullsized":"2024-09-12T02:57:59.560190+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:05:40.577342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'7","reported_seq":128,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:58:00.976755+0000","last_change":"2024-09-12T02:56:14.327147+0000","last_active":"2024-09-12T02:58:00.976755+0000","last_peered":"2024-09-12T02:58:00.976755+0000","last_clean":"2024-09-12T02:58:00.976755+0000","last_became_active":"2024-09-12T02:56:14.326737+0000","last_became_peered":"2024-09-12T02:56:14.326737+0000","last_unstale":"2024-09-12T02:58:00.976755+0000","last_undegraded":"2024-09-12T02:58:00.976755+0000","last_fullsized":"2024-09-12T02:58:00.976755+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:55:33.510599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1704,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":99,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.563684+0000","last_change":"2024-09-12T02:56:15.984363+0000","last_active":"2024-09-12T02:57:59.563684+0000","last_peered":"2024-09-12T02:57:59.563684+0000","last_clean":"2024-09-12T02:57:59.563684+0000","last_became_active":"2024-09-12T02:56:15.983822+0000","last_became_peered":"2024-09-12T02:56:15.983822+0000","last_unstale":"2024-09-12T02:57:59.563684+0000","last_undegraded":"2024-09-12T02:57:59.563684+0000","last_fullsized":"2024-09-12T02:57:59.563684+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:41:46.082716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":333,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.035564+0000","last_change":"2024-09-12T02:55:05.800210+0000","last_active":"2024-09-12T02:56:31.035564+0000","last_peered":"2024-09-12T02:56:31.035564+0000","last_clean":"2024-09-12T02:56:31.035564+0000","last_became_active":"2024-09-12T02:55:05.799791+0000","last_became_peered":"2024-09-12T02:55:05.799791+0000","last_unstale":"2024-09-12T02:56:31.035564+0000","last_undegraded":"2024-09-12T02:56:31.035564+0000","last_fullsized":"2024-09-12T02:56:31.035564+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:51:48.530016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:51:48.530016+0000","last_clean_scrub_stamp":"2024-09-12T02:51:48.530016+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:05:47.416697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"81'6","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.031169+0000","last_change":"2024-09-12T02:56:13.930269+0000","last_active":"2024-09-12T02:56:31.031169+0000","last_peered":"2024-09-12T02:56:31.031169+0000","last_clean":"2024-09-12T02:56:31.031169+0000","last_became_active":"2024-09-12T02:56:13.929831+0000","last_became_peered":"2024-09-12T02:56:13.929831+0000","last_unstale":"2024-09-12T02:56:31.031169+0000","last_undegraded":"2024-09-12T02:56:31.031169+0000","last_fullsized":"2024-09-12T02:56:31.031169+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:38:17.878398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.561332+0000","last_change":"2024-09-12T02:56:15.969744+0000","last_active":"2024-09-12T02:57:59.561332+0000","last_peered":"2024-09-12T02:57:59.561332+0000","last_clean":"2024-09-12T02:57:59.561332+0000","last_became_active":"2024-09-12T02:56:15.969354+0000","last_became_peered":"2024-09-12T02:56:15.969354+0000","last_unstale":"2024-09-12T02:57:59.561332+0000","last_undegraded":"2024-09-12T02:57:59.561332+0000","last_fullsized":"2024-09-12T02:57:59.561332+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:26:02.409510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"81'4","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.035385+0000","last_change":"2024-09-12T02:56:13.929231+0000","last_active":"2024-09-12T02:56:31.035385+0000","last_peered":"2024-09-12T02:56:31.035385+0000","last_clean":"2024-09-12T02:56:31.035385+0000","last_became_active":"2024-09-12T02:56:13.928843+0000","last_became_peered":"2024-09-12T02:56:13.928843+0000","last_unstale":"2024-09-12T02:56:31.035385+0000","last_undegraded":"2024-09-12T02:56:31.035385+0000","last_fullsized":"2024-09-12T02:56:31.035385+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:26:33.619755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"83'5","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.555927+0000","last_change":"2024-09-12T02:56:15.982503+0000","last_active":"2024-09-12T02:57:59.555927+0000","last_peered":"2024-09-12T02:57:59.555927+0000","last_clean":"2024-09-12T02:57:59.555927+0000","last_became_active":"2024-09-12T02:56:15.982192+0000","last_became_peered":"2024-09-12T02:56:15.982192+0000","last_unstale":"2024-09-12T02:57:59.555927+0000","last_undegraded":"2024-09-12T02:57:59.555927+0000","last_fullsized":"2024-09-12T02:57:59.555927+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:55:18.370967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"83'6","reported_seq":127,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:58:03.979145+0000","last_change":"2024-09-12T02:56:14.075486+0000","last_active":"2024-09-12T02:58:03.979145+0000","last_peered":"2024-09-12T02:58:03.979145+0000","last_clean":"2024-09-12T02:58:03.979145+0000","last_became_active":"2024-09-12T02:56:14.075152+0000","last_became_peered":"2024-09-12T02:56:14.075152+0000","last_unstale":"2024-09-12T02:58:03.979145+0000","last_undegraded":"2024-09-12T02:58:03.979145+0000","last_fullsized":"2024-09-12T02:58:03.979145+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:59:40.950191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":99,"num_read_kb":99,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.556340+0000","last_change":"2024-09-12T02:56:15.981584+0000","last_active":"2024-09-12T02:57:59.556340+0000","last_peered":"2024-09-12T02:57:59.556340+0000","last_clean":"2024-09-12T02:57:59.556340+0000","last_became_active":"2024-09-12T02:56:15.981240+0000","last_became_peered":"2024-09-12T02:56:15.981240+0000","last_unstale":"2024-09-12T02:57:59.556340+0000","last_undegraded":"2024-09-12T02:57:59.556340+0000","last_fullsized":"2024-09-12T02:57:59.556340+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:49:04.036830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"83'7","reported_seq":129,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:58:03.979513+0000","last_change":"2024-09-12T02:56:13.935143+0000","last_active":"2024-09-12T02:58:03.979513+0000","last_peered":"2024-09-12T02:58:03.979513+0000","last_clean":"2024-09-12T02:58:03.979513+0000","last_became_active":"2024-09-12T02:56:13.934792+0000","last_became_peered":"2024-09-12T02:56:13.934792+0000","last_unstale":"2024-09-12T02:58:03.979513+0000","last_undegraded":"2024-09-12T02:58:03.979513+0000","last_fullsized":"2024-09-12T02:58:03.979513+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:00:26.973612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1289,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":100,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"83'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.558893+0000","last_change":"2024-09-12T02:56:31.034825+0000","last_active":"2024-09-12T02:57:59.558893+0000","last_peered":"2024-09-12T02:57:59.558893+0000","last_clean":"2024-09-12T02:57:59.558893+0000","last_became_active":"2024-09-12T02:56:31.034516+0000","last_became_peered":"2024-09-12T02:56:31.034516+0000","last_unstale":"2024-09-12T02:57:59.558893+0000","last_undegraded":"2024-09-12T02:57:59.558893+0000","last_fullsized":"2024-09-12T02:57:59.558893+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:10:42.161023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"81'1","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.029027+0000","last_change":"2024-09-12T02:56:14.076762+0000","last_active":"2024-09-12T02:56:31.029027+0000","last_peered":"2024-09-12T02:56:31.029027+0000","last_clean":"2024-09-12T02:56:31.029027+0000","last_became_active":"2024-09-12T02:56:14.076421+0000","last_became_peered":"2024-09-12T02:56:14.076421+0000","last_unstale":"2024-09-12T02:56:31.029027+0000","last_undegraded":"2024-09-12T02:56:31.029027+0000","last_fullsized":"2024-09-12T02:56:31.029027+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:56:56.422910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'5","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.555835+0000","last_change":"2024-09-12T02:56:15.978810+0000","last_active":"2024-09-12T02:57:59.555835+0000","last_peered":"2024-09-12T02:57:59.555835+0000","last_clean":"2024-09-12T02:57:59.555835+0000","last_became_active":"2024-09-12T02:56:15.978038+0000","last_became_peered":"2024-09-12T02:56:15.978038+0000","last_unstale":"2024-09-12T02:57:59.555835+0000","last_undegraded":"2024-09-12T02:57:59.555835+0000","last_fullsized":"2024-09-12T02:57:59.555835+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:04:52.215784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"81'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.035440+0000","last_change":"2024-09-12T02:56:14.075987+0000","last_active":"2024-09-12T02:56:31.035440+0000","last_peered":"2024-09-12T02:56:31.035440+0000","last_clean":"2024-09-12T02:56:31.035440+0000","last_became_active":"2024-09-12T02:56:14.075596+0000","last_became_peered":"2024-09-12T02:56:14.075596+0000","last_unstale":"2024-09-12T02:56:31.035440+0000","last_undegraded":"2024-09-12T02:56:31.035440+0000","last_fullsized":"2024-09-12T02:56:31.035440+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:18:48.713387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.552400+0000","last_change":"2024-09-12T02:56:15.987842+0000","last_active":"2024-09-12T02:57:59.552400+0000","last_peered":"2024-09-12T02:57:59.552400+0000","last_clean":"2024-09-12T02:57:59.552400+0000","last_became_active":"2024-09-12T02:56:15.987507+0000","last_became_peered":"2024-09-12T02:56:15.987507+0000","last_unstale":"2024-09-12T02:57:59.552400+0000","last_undegraded":"2024-09-12T02:57:59.552400+0000","last_fullsized":"2024-09-12T02:57:59.552400+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:52:23.000549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'19","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:58:01.991961+0000","last_change":"2024-09-12T02:56:13.924372+0000","last_active":"2024-09-12T02:58:01.991961+0000","last_peered":"2024-09-12T02:58:01.991961+0000","last_clean":"2024-09-12T02:58:01.991961+0000","last_became_active":"2024-09-12T02:56:13.923955+0000","last_became_peered":"2024-09-12T02:56:13.923955+0000","last_unstale":"2024-09-12T02:58:01.991961+0000","last_undegraded":"2024-09-12T02:58:01.991961+0000","last_fullsized":"2024-09-12T02:58:01.991961+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:52:47.452410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":34,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"83'5","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.562919+0000","last_change":"2024-09-12T02:56:15.978262+0000","last_active":"2024-09-12T02:57:59.562919+0000","last_peered":"2024-09-12T02:57:59.562919+0000","last_clean":"2024-09-12T02:57:59.562919+0000","last_became_active":"2024-09-12T02:56:15.977913+0000","last_became_peered":"2024-09-12T02:56:15.977913+0000","last_unstale":"2024-09-12T02:57:59.562919+0000","last_undegraded":"2024-09-12T02:57:59.562919+0000","last_fullsized":"2024-09-12T02:57:59.562919+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:35:27.104635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"81'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.031488+0000","last_change":"2024-09-12T02:56:14.327417+0000","last_active":"2024-09-12T02:56:31.031488+0000","last_peered":"2024-09-12T02:56:31.031488+0000","last_clean":"2024-09-12T02:56:31.031488+0000","last_became_active":"2024-09-12T02:56:14.327001+0000","last_became_peered":"2024-09-12T02:56:14.327001+0000","last_unstale":"2024-09-12T02:56:31.031488+0000","last_undegraded":"2024-09-12T02:56:31.031488+0000","last_fullsized":"2024-09-12T02:56:31.031488+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:15:39.019252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.565859+0000","last_change":"2024-09-12T02:56:15.977286+0000","last_active":"2024-09-12T02:57:59.565859+0000","last_peered":"2024-09-12T02:57:59.565859+0000","last_clean":"2024-09-12T02:57:59.565859+0000","last_became_active":"2024-09-12T02:56:15.976944+0000","last_became_peered":"2024-09-12T02:56:15.976944+0000","last_unstale":"2024-09-12T02:57:59.565859+0000","last_undegraded":"2024-09-12T02:57:59.565859+0000","last_fullsized":"2024-09-12T02:57:59.565859+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:22:57.056781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.306212+0000","last_change":"2024-09-12T02:56:14.075701+0000","last_active":"2024-09-12T02:56:31.306212+0000","last_peered":"2024-09-12T02:56:31.306212+0000","last_clean":"2024-09-12T02:56:31.306212+0000","last_became_active":"2024-09-12T02:56:14.075340+0000","last_became_peered":"2024-09-12T02:56:14.075340+0000","last_unstale":"2024-09-12T02:56:31.306212+0000","last_undegraded":"2024-09-12T02:56:31.306212+0000","last_fullsized":"2024-09-12T02:56:31.306212+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:17:42.255058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"83'2","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.528187+0000","last_change":"2024-09-12T02:56:15.979783+0000","last_active":"2024-09-12T02:57:59.528187+0000","last_peered":"2024-09-12T02:57:59.528187+0000","last_clean":"2024-09-12T02:57:59.528187+0000","last_became_active":"2024-09-12T02:56:15.979424+0000","last_became_peered":"2024-09-12T02:56:15.979424+0000","last_unstale":"2024-09-12T02:57:59.528187+0000","last_undegraded":"2024-09-12T02:57:59.528187+0000","last_fullsized":"2024-09-12T02:57:59.528187+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:59:11.568138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"81'7","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.034660+0000","last_change":"2024-09-12T02:56:13.928996+0000","last_active":"2024-09-12T02:56:31.034660+0000","last_peered":"2024-09-12T02:56:31.034660+0000","last_clean":"2024-09-12T02:56:31.034660+0000","last_became_active":"2024-09-12T02:56:13.928664+0000","last_became_peered":"2024-09-12T02:56:13.928664+0000","last_unstale":"2024-09-12T02:56:31.034660+0000","last_undegraded":"2024-09-12T02:56:31.034660+0000","last_fullsized":"2024-09-12T02:56:31.034660+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:37:22.067930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.560217+0000","last_change":"2024-09-12T02:56:15.984096+0000","last_active":"2024-09-12T02:57:59.560217+0000","last_peered":"2024-09-12T02:57:59.560217+0000","last_clean":"2024-09-12T02:57:59.560217+0000","last_became_active":"2024-09-12T02:56:15.983737+0000","last_became_peered":"2024-09-12T02:56:15.983737+0000","last_unstale":"2024-09-12T02:57:59.560217+0000","last_undegraded":"2024-09-12T02:57:59.560217+0000","last_fullsized":"2024-09-12T02:57:59.560217+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:02:17.892318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"81'3","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.053785+0000","last_change":"2024-09-12T02:56:31.053343+0000","last_active":"2024-09-12T02:56:31.053785+0000","last_peered":"2024-09-12T02:56:31.053785+0000","last_clean":"2024-09-12T02:56:31.053785+0000","last_became_active":"2024-09-12T02:56:31.037918+0000","last_became_peered":"2024-09-12T02:56:31.037918+0000","last_unstale":"2024-09-12T02:56:31.053785+0000","last_undegraded":"2024-09-12T02:56:31.053785+0000","last_fullsized":"2024-09-12T02:56:31.053785+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:39:54.990724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.563189+0000","last_change":"2024-09-12T02:56:15.975640+0000","last_active":"2024-09-12T02:57:59.563189+0000","last_peered":"2024-09-12T02:57:59.563189+0000","last_clean":"2024-09-12T02:57:59.563189+0000","last_became_active":"2024-09-12T02:56:15.975342+0000","last_became_peered":"2024-09-12T02:56:15.975342+0000","last_unstale":"2024-09-12T02:57:59.563189+0000","last_undegraded":"2024-09-12T02:57:59.563189+0000","last_fullsized":"2024-09-12T02:57:59.563189+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:26:23.694876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.034922+0000","last_change":"2024-09-12T02:56:13.934149+0000","last_active":"2024-09-12T02:56:31.034922+0000","last_peered":"2024-09-12T02:56:31.034922+0000","last_clean":"2024-09-12T02:56:31.034922+0000","last_became_active":"2024-09-12T02:56:13.933743+0000","last_became_peered":"2024-09-12T02:56:13.933743+0000","last_unstale":"2024-09-12T02:56:31.034922+0000","last_undegraded":"2024-09-12T02:56:31.034922+0000","last_fullsized":"2024-09-12T02:56:31.034922+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:16:17.854457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"83'5","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.563348+0000","last_change":"2024-09-12T02:56:15.984782+0000","last_active":"2024-09-12T02:57:59.563348+0000","last_peered":"2024-09-12T02:57:59.563348+0000","last_clean":"2024-09-12T02:57:59.563348+0000","last_became_active":"2024-09-12T02:56:15.984336+0000","last_became_peered":"2024-09-12T02:56:15.984336+0000","last_unstale":"2024-09-12T02:57:59.563348+0000","last_undegraded":"2024-09-12T02:57:59.563348+0000","last_fullsized":"2024-09-12T02:57:59.563348+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:03:39.832897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"83'15","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.564252+0000","last_change":"2024-09-12T02:56:15.985486+0000","last_active":"2024-09-12T02:57:59.564252+0000","last_peered":"2024-09-12T02:57:59.564252+0000","last_clean":"2024-09-12T02:57:59.564252+0000","last_became_active":"2024-09-12T02:56:15.985167+0000","last_became_peered":"2024-09-12T02:56:15.985167+0000","last_unstale":"2024-09-12T02:57:59.564252+0000","last_undegraded":"2024-09-12T02:57:59.564252+0000","last_fullsized":"2024-09-12T02:57:59.564252+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:46:20.569453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"81'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.035106+0000","last_change":"2024-09-12T02:56:13.929324+0000","last_active":"2024-09-12T02:56:31.035106+0000","last_peered":"2024-09-12T02:56:31.035106+0000","last_clean":"2024-09-12T02:56:31.035106+0000","last_became_active":"2024-09-12T02:56:13.928994+0000","last_became_peered":"2024-09-12T02:56:13.928994+0000","last_unstale":"2024-09-12T02:56:31.035106+0000","last_undegraded":"2024-09-12T02:56:31.035106+0000","last_fullsized":"2024-09-12T02:56:31.035106+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:30:27.290328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.562582+0000","last_change":"2024-09-12T02:56:15.985889+0000","last_active":"2024-09-12T02:57:59.562582+0000","last_peered":"2024-09-12T02:57:59.562582+0000","last_clean":"2024-09-12T02:57:59.562582+0000","last_became_active":"2024-09-12T02:56:15.985509+0000","last_became_peered":"2024-09-12T02:56:15.985509+0000","last_unstale":"2024-09-12T02:57:59.562582+0000","last_undegraded":"2024-09-12T02:57:59.562582+0000","last_fullsized":"2024-09-12T02:57:59.562582+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:30:43.120718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"83'13","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.035437+0000","last_change":"2024-09-12T02:56:13.934192+0000","last_active":"2024-09-12T02:56:31.035437+0000","last_peered":"2024-09-12T02:56:31.035437+0000","last_clean":"2024-09-12T02:56:31.035437+0000","last_became_active":"2024-09-12T02:56:13.933789+0000","last_became_peered":"2024-09-12T02:56:13.933789+0000","last_unstale":"2024-09-12T02:56:31.035437+0000","last_undegraded":"2024-09-12T02:56:31.035437+0000","last_fullsized":"2024-09-12T02:56:31.035437+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:32:47.771085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.563285+0000","last_change":"2024-09-12T02:56:15.983531+0000","last_active":"2024-09-12T02:57:59.563285+0000","last_peered":"2024-09-12T02:57:59.563285+0000","last_clean":"2024-09-12T02:57:59.563285+0000","last_became_active":"2024-09-12T02:56:15.983192+0000","last_became_peered":"2024-09-12T02:56:15.983192+0000","last_unstale":"2024-09-12T02:57:59.563285+0000","last_undegraded":"2024-09-12T02:57:59.563285+0000","last_fullsized":"2024-09-12T02:57:59.563285+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:16:59.923379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.034752+0000","last_change":"2024-09-12T02:56:14.075396+0000","last_active":"2024-09-12T02:56:31.034752+0000","last_peered":"2024-09-12T02:56:31.034752+0000","last_clean":"2024-09-12T02:56:31.034752+0000","last_became_active":"2024-09-12T02:56:14.075023+0000","last_became_peered":"2024-09-12T02:56:14.075023+0000","last_unstale":"2024-09-12T02:56:31.034752+0000","last_undegraded":"2024-09-12T02:56:31.034752+0000","last_fullsized":"2024-09-12T02:56:31.034752+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:41:45.760689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"83'5","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.564093+0000","last_change":"2024-09-12T02:56:15.987276+0000","last_active":"2024-09-12T02:57:59.564093+0000","last_peered":"2024-09-12T02:57:59.564093+0000","last_clean":"2024-09-12T02:57:59.564093+0000","last_became_active":"2024-09-12T02:56:15.986949+0000","last_became_peered":"2024-09-12T02:56:15.986949+0000","last_unstale":"2024-09-12T02:57:59.564093+0000","last_undegraded":"2024-09-12T02:57:59.564093+0000","last_fullsized":"2024-09-12T02:57:59.564093+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:49:45.627093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"83'14","reported_seq":134,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:58:02.977467+0000","last_change":"2024-09-12T02:56:13.932365+0000","last_active":"2024-09-12T02:58:02.977467+0000","last_peered":"2024-09-12T02:58:02.977467+0000","last_clean":"2024-09-12T02:58:02.977467+0000","last_became_active":"2024-09-12T02:56:13.932062+0000","last_became_peered":"2024-09-12T02:56:13.932062+0000","last_unstale":"2024-09-12T02:58:02.977467+0000","last_undegraded":"2024-09-12T02:58:02.977467+0000","last_fullsized":"2024-09-12T02:58:02.977467+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:17:08.451839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":98,"num_read_kb":98,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":74229,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1000,"num_read_kb":580,"num_write":266,"num_write_kb":154,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":734,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":569344,"data_stored":214452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":227,"ondisk_log_size":227,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547235,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":917504,"data_stored":440831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776167,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32756,"kb_used_data":1572,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712396,"statfs":{"total":95995035648,"available":95961493504,"internally_reserved":0,"allocated":1609728,"data_stored":1093472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939693,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40416,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704736,"statfs":{"total":95995035648,"available":95953649664,"internally_reserved":0,"allocated":933888,"data_stored":440115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":40434112},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168626,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":921600,"data_stored":440381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430262,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28516,"kb_used_data":1492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716636,"statfs":{"total":95995035648,"available":95965835264,"internally_reserved":0,"allocated":1527808,"data_stored":1034641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593788,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":917504,"data_stored":440533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822719,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28528,"kb_used_data":1504,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716624,"statfs":{"total":95995035648,"available":95965822976,"internally_reserved":0,"allocated":1540096,"data_stored":1032298,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953541,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":925696,"data_stored":440849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182473,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":933888,"data_stored":439657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411407,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36312,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708840,"statfs":{"total":95995035648,"available":95957852160,"internally_reserved":0,"allocated":991232,"data_stored":502601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574931,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":502469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738457,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":937984,"data_stored":441705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":63784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":63608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":68308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T02:58:07.206 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-09-12T02:58:07.206 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-12T02:58:07.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:07.347+0000 7f5887b8d640 1 -- 172.21.15.5:0/1968976986 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5880100a10 msgr2=0x7f5880100e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:07.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:07.347+0000 7f5887b8d640 1 --2- 172.21.15.5:0/1968976986 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5880100a10 0x7f5880100e10 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f587c009900 tx=0x7f587c02a410 comp rx=0 tx=0).stop 2024-09-12T02:58:07.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:07.347+0000 7f5887b8d640 1 -- 172.21.15.5:0/1968976986 shutdown_connections 2024-09-12T02:58:07.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:07.347+0000 7f5887b8d640 1 --2- 172.21.15.5:0/1968976986 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f58801025c0 0x7f5880108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:07.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:07.347+0000 7f5887b8d640 1 --2- 172.21.15.5:0/1968976986 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5880101c00 0x7f5880102080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:07.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:07.347+0000 7f5887b8d640 1 --2- 172.21.15.5:0/1968976986 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5880100a10 0x7f5880100e10 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:07.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:07.347+0000 7f5887b8d640 1 -- 172.21.15.5:0/1968976986 >> 172.21.15.5:0/1968976986 conn(0x7f58800fc170 msgr2=0x7f58800fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:07.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:07.347+0000 7f5887b8d640 1 -- 172.21.15.5:0/1968976986 shutdown_connections 2024-09-12T02:58:07.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:07.347+0000 7f5887b8d640 1 -- 172.21.15.5:0/1968976986 wait complete. 2024-09-12T02:58:07.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:07.348+0000 7f5887b8d640 1 Processor -- start 2024-09-12T02:58:07.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:07.348+0000 7f5887b8d640 1 -- start start 2024-09-12T02:58:07.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:07.348+0000 7f5887b8d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5880100a10 0x7f5880197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:07.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:07.348+0000 7f5887b8d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5880101c00 0x7f58801983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:07.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:07.348+0000 7f5887b8d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f58801025c0 0x7f588019f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:07.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:07.348+0000 7f5887b8d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f588010c6e0 con 0x7f5880100a10 2024-09-12T02:58:07.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:07.348+0000 7f5887b8d640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f588010c560 con 0x7f5880101c00 2024-09-12T02:58:07.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:07.348+0000 7f5887b8d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f588010c860 con 0x7f58801025c0 2024-09-12T02:58:07.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:07.348+0000 7f5885929640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5880100a10 0x7f5880197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:07.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:07.348+0000 7f5885929640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5880100a10 0x7f5880197e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55470/0 (socket says 172.21.15.5:55470) 2024-09-12T02:58:07.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:07.348+0000 7f5885929640 1 -- 172.21.15.5:0/349515788 learned_addr learned my addr 172.21.15.5:0/349515788 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:58:07.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:07.348+0000 7f5885128640 1 --2- 172.21.15.5:0/349515788 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5880101c00 0x7f58801983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:07.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:07.349+0000 7f5885929640 1 -- 172.21.15.5:0/349515788 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f58801025c0 msgr2=0x7f588019f430 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-12T02:58:07.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:07.349+0000 7f5885929640 1 --2- 172.21.15.5:0/349515788 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f58801025c0 0x7f588019f430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:07.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:07.349+0000 7f5885929640 1 -- 172.21.15.5:0/349515788 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5880101c00 msgr2=0x7f58801983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:07.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:07.349+0000 7f5885929640 1 --2- 172.21.15.5:0/349515788 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5880101c00 0x7f58801983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:07.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:07.349+0000 7f5885929640 1 -- 172.21.15.5:0/349515788 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f588019faa0 con 0x7f5880100a10 2024-09-12T02:58:07.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:07.349+0000 7f5885929640 1 --2- 172.21.15.5:0/349515788 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5880100a10 0x7f5880197e80 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f587c031440 tx=0x7f587c032e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:07.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:07.349+0000 7f5876ffd640 1 -- 172.21.15.5:0/349515788 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f587c03f030 con 0x7f5880100a10 2024-09-12T02:58:07.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:07.349+0000 7f5876ffd640 1 -- 172.21.15.5:0/349515788 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f587c003ce0 con 0x7f5880100a10 2024-09-12T02:58:07.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:07.349+0000 7f5887b8d640 1 -- 172.21.15.5:0/349515788 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f588019fd30 con 0x7f5880100a10 2024-09-12T02:58:07.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:07.349+0000 7f5876ffd640 1 -- 172.21.15.5:0/349515788 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f587c035070 con 0x7f5880100a10 2024-09-12T02:58:07.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:07.349+0000 7f5887b8d640 1 -- 172.21.15.5:0/349515788 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58801a01c0 con 0x7f5880100a10 2024-09-12T02:58:07.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:07.350+0000 7f5887b8d640 1 -- 172.21.15.5:0/349515788 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58801a03f0 con 0x7f5880100a10 2024-09-12T02:58:07.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:07.351+0000 7f5876ffd640 1 -- 172.21.15.5:0/349515788 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7f587c042030 con 0x7f5880100a10 2024-09-12T02:58:07.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:07.352+0000 7f5876ffd640 1 --2- 172.21.15.5:0/349515788 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f586c0a2ca0 0x7f586c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:07.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:07.352+0000 7f5885128640 1 --2- 172.21.15.5:0/349515788 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f586c0a2ca0 0x7f586c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:07.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:07.352+0000 7f5876ffd640 1 -- 172.21.15.5:0/349515788 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f587c0ef850 con 0x7f5880100a10 2024-09-12T02:58:07.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:07.352+0000 7f5885128640 1 --2- 172.21.15.5:0/349515788 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f586c0a2ca0 0x7f586c0a5160 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5880199390 tx=0x7f587000b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:07.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:07.356+0000 7f5876ffd640 1 -- 172.21.15.5:0/349515788 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f587c0bc600 con 0x7f5880100a10 2024-09-12T02:58:07.528 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:07.528+0000 7f5887b8d640 1 -- 172.21.15.5:0/349515788 --> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5880069ae0 con 0x7f586c0a2ca0 2024-09-12T02:58:07.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:07.535+0000 7f5876ffd640 1 -- 172.21.15.5:0/349515788 <== mgr.14148 v2:172.21.15.5:6800/3780631642 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259021 (secure 0 0 0) 0x7f5880069ae0 con 0x7f586c0a2ca0 2024-09-12T02:58:07.537 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-12T02:58:07.538 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-09-12T02:58:07.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:07.538+0000 7f5887b8d640 1 -- 172.21.15.5:0/349515788 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f586c0a2ca0 msgr2=0x7f586c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:07.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:07.538+0000 7f5887b8d640 1 --2- 172.21.15.5:0/349515788 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f586c0a2ca0 0x7f586c0a5160 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5880199390 tx=0x7f587000b410 comp rx=0 tx=0).stop 2024-09-12T02:58:07.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:07.538+0000 7f5887b8d640 1 -- 172.21.15.5:0/349515788 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5880100a10 msgr2=0x7f5880197e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:07.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:07.538+0000 7f5887b8d640 1 --2- 172.21.15.5:0/349515788 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5880100a10 0x7f5880197e80 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f587c031440 tx=0x7f587c032e20 comp rx=0 tx=0).stop 2024-09-12T02:58:07.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:07.538+0000 7f5887b8d640 1 -- 172.21.15.5:0/349515788 shutdown_connections 2024-09-12T02:58:07.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:07.538+0000 7f5887b8d640 1 --2- 172.21.15.5:0/349515788 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f58801025c0 0x7f588019f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:07.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:07.538+0000 7f5887b8d640 1 --2- 172.21.15.5:0/349515788 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5880101c00 0x7f58801983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:07.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:07.538+0000 7f5887b8d640 1 --2- 172.21.15.5:0/349515788 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f586c0a2ca0 0x7f586c0a5160 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:07.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:07.538+0000 7f5887b8d640 1 --2- 172.21.15.5:0/349515788 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5880100a10 0x7f5880197e80 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:07.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:07.538+0000 7f5887b8d640 1 -- 172.21.15.5:0/349515788 >> 172.21.15.5:0/349515788 conn(0x7f58800fc170 msgr2=0x7f58800693d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:07.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:07.538+0000 7f5887b8d640 1 -- 172.21.15.5:0/349515788 shutdown_connections 2024-09-12T02:58:07.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:07.538+0000 7f5887b8d640 1 -- 172.21.15.5:0/349515788 wait complete. 2024-09-12T02:58:07.548 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:58:07 smithi005 ceph-mon[31197]: pgmap v331: 97 pgs: 97 active+clean; 649 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-09-12T02:58:07.548 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:58:07 smithi005 ceph-mon[31197]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-09-12T02:58:07.549 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":331,"stamp":"2024-09-12T02:58:05.568647+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":664831,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1126,"num_read_kb":689,"num_write":449,"num_write_kb":3094,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":1181470,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":866,"ondisk_log_size":866,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":374556,"kb_used_data":12828,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124567268,"statfs":{"total":1151940427776,"available":1151556882432,"internally_reserved":0,"allocated":13135872,"data_stored":7249552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":370193723},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5854,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":120,"num_read_kb":66,"num_write":20,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002569"},"pg_stats":[{"pgid":"3.3f","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.565050+0000","last_change":"2024-09-12T02:56:15.988690+0000","last_active":"2024-09-12T02:57:59.565050+0000","last_peered":"2024-09-12T02:57:59.565050+0000","last_clean":"2024-09-12T02:57:59.565050+0000","last_became_active":"2024-09-12T02:56:15.988320+0000","last_became_peered":"2024-09-12T02:56:15.988320+0000","last_unstale":"2024-09-12T02:57:59.565050+0000","last_undegraded":"2024-09-12T02:57:59.565050+0000","last_fullsized":"2024-09-12T02:57:59.565050+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:31:14.157622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"83'4","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.553842+0000","last_change":"2024-09-12T02:56:15.978368+0000","last_active":"2024-09-12T02:57:59.553842+0000","last_peered":"2024-09-12T02:57:59.553842+0000","last_clean":"2024-09-12T02:57:59.553842+0000","last_became_active":"2024-09-12T02:56:15.978092+0000","last_became_peered":"2024-09-12T02:56:15.978092+0000","last_unstale":"2024-09-12T02:57:59.553842+0000","last_undegraded":"2024-09-12T02:57:59.553842+0000","last_fullsized":"2024-09-12T02:57:59.553842+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:32:48.399005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.564003+0000","last_change":"2024-09-12T02:56:15.987249+0000","last_active":"2024-09-12T02:57:59.564003+0000","last_peered":"2024-09-12T02:57:59.564003+0000","last_clean":"2024-09-12T02:57:59.564003+0000","last_became_active":"2024-09-12T02:56:15.986921+0000","last_became_peered":"2024-09-12T02:56:15.986921+0000","last_unstale":"2024-09-12T02:57:59.564003+0000","last_undegraded":"2024-09-12T02:57:59.564003+0000","last_fullsized":"2024-09-12T02:57:59.564003+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:42:13.427871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.555703+0000","last_change":"2024-09-12T02:56:15.987673+0000","last_active":"2024-09-12T02:57:59.555703+0000","last_peered":"2024-09-12T02:57:59.555703+0000","last_clean":"2024-09-12T02:57:59.555703+0000","last_became_active":"2024-09-12T02:56:15.986964+0000","last_became_peered":"2024-09-12T02:56:15.986964+0000","last_unstale":"2024-09-12T02:57:59.555703+0000","last_undegraded":"2024-09-12T02:57:59.555703+0000","last_fullsized":"2024-09-12T02:57:59.555703+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:07:12.406974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"83'4","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.548461+0000","last_change":"2024-09-12T02:56:15.974473+0000","last_active":"2024-09-12T02:57:59.548461+0000","last_peered":"2024-09-12T02:57:59.548461+0000","last_clean":"2024-09-12T02:57:59.548461+0000","last_became_active":"2024-09-12T02:56:15.974124+0000","last_became_peered":"2024-09-12T02:56:15.974124+0000","last_unstale":"2024-09-12T02:57:59.548461+0000","last_undegraded":"2024-09-12T02:57:59.548461+0000","last_fullsized":"2024-09-12T02:57:59.548461+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:44:29.054841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.560740+0000","last_change":"2024-09-12T02:56:15.980242+0000","last_active":"2024-09-12T02:57:59.560740+0000","last_peered":"2024-09-12T02:57:59.560740+0000","last_clean":"2024-09-12T02:57:59.560740+0000","last_became_active":"2024-09-12T02:56:15.979937+0000","last_became_peered":"2024-09-12T02:56:15.979937+0000","last_unstale":"2024-09-12T02:57:59.560740+0000","last_undegraded":"2024-09-12T02:57:59.560740+0000","last_fullsized":"2024-09-12T02:57:59.560740+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:44:13.406341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"83'5","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.556845+0000","last_change":"2024-09-12T02:56:15.978746+0000","last_active":"2024-09-12T02:57:59.556845+0000","last_peered":"2024-09-12T02:57:59.556845+0000","last_clean":"2024-09-12T02:57:59.556845+0000","last_became_active":"2024-09-12T02:56:15.978304+0000","last_became_peered":"2024-09-12T02:56:15.978304+0000","last_unstale":"2024-09-12T02:57:59.556845+0000","last_undegraded":"2024-09-12T02:57:59.556845+0000","last_fullsized":"2024-09-12T02:57:59.556845+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:34:52.115630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.562507+0000","last_change":"2024-09-12T02:56:15.980083+0000","last_active":"2024-09-12T02:57:59.562507+0000","last_peered":"2024-09-12T02:57:59.562507+0000","last_clean":"2024-09-12T02:57:59.562507+0000","last_became_active":"2024-09-12T02:56:15.979771+0000","last_became_peered":"2024-09-12T02:56:15.979771+0000","last_unstale":"2024-09-12T02:57:59.562507+0000","last_undegraded":"2024-09-12T02:57:59.562507+0000","last_fullsized":"2024-09-12T02:57:59.562507+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:59:42.351970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"83'11","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.565703+0000","last_change":"2024-09-12T02:56:15.988091+0000","last_active":"2024-09-12T02:57:59.565703+0000","last_peered":"2024-09-12T02:57:59.565703+0000","last_clean":"2024-09-12T02:57:59.565703+0000","last_became_active":"2024-09-12T02:56:15.986295+0000","last_became_peered":"2024-09-12T02:56:15.986295+0000","last_unstale":"2024-09-12T02:57:59.565703+0000","last_undegraded":"2024-09-12T02:57:59.565703+0000","last_fullsized":"2024-09-12T02:57:59.565703+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:45:40.331805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"83'16","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.562128+0000","last_change":"2024-09-12T02:56:15.987349+0000","last_active":"2024-09-12T02:57:59.562128+0000","last_peered":"2024-09-12T02:57:59.562128+0000","last_clean":"2024-09-12T02:57:59.562128+0000","last_became_active":"2024-09-12T02:56:15.987018+0000","last_became_peered":"2024-09-12T02:56:15.987018+0000","last_unstale":"2024-09-12T02:57:59.562128+0000","last_undegraded":"2024-09-12T02:57:59.562128+0000","last_fullsized":"2024-09-12T02:57:59.562128+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:34:19.652348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.563544+0000","last_change":"2024-09-12T02:56:15.985164+0000","last_active":"2024-09-12T02:57:59.563544+0000","last_peered":"2024-09-12T02:57:59.563544+0000","last_clean":"2024-09-12T02:57:59.563544+0000","last_became_active":"2024-09-12T02:56:15.984787+0000","last_became_peered":"2024-09-12T02:56:15.984787+0000","last_unstale":"2024-09-12T02:57:59.563544+0000","last_undegraded":"2024-09-12T02:57:59.563544+0000","last_fullsized":"2024-09-12T02:57:59.563544+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:36:29.727442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.552957+0000","last_change":"2024-09-12T02:56:15.979963+0000","last_active":"2024-09-12T02:57:59.552957+0000","last_peered":"2024-09-12T02:57:59.552957+0000","last_clean":"2024-09-12T02:57:59.552957+0000","last_became_active":"2024-09-12T02:56:15.979639+0000","last_became_peered":"2024-09-12T02:56:15.979639+0000","last_unstale":"2024-09-12T02:57:59.552957+0000","last_undegraded":"2024-09-12T02:57:59.552957+0000","last_fullsized":"2024-09-12T02:57:59.552957+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:34:18.273245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"83'10","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.561647+0000","last_change":"2024-09-12T02:56:15.980415+0000","last_active":"2024-09-12T02:57:59.561647+0000","last_peered":"2024-09-12T02:57:59.561647+0000","last_clean":"2024-09-12T02:57:59.561647+0000","last_became_active":"2024-09-12T02:56:15.980059+0000","last_became_peered":"2024-09-12T02:56:15.980059+0000","last_unstale":"2024-09-12T02:57:59.561647+0000","last_undegraded":"2024-09-12T02:57:59.561647+0000","last_fullsized":"2024-09-12T02:57:59.561647+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:11:42.689081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.553574+0000","last_change":"2024-09-12T02:56:15.981423+0000","last_active":"2024-09-12T02:57:59.553574+0000","last_peered":"2024-09-12T02:57:59.553574+0000","last_clean":"2024-09-12T02:57:59.553574+0000","last_became_active":"2024-09-12T02:56:15.981186+0000","last_became_peered":"2024-09-12T02:56:15.981186+0000","last_unstale":"2024-09-12T02:57:59.553574+0000","last_undegraded":"2024-09-12T02:57:59.553574+0000","last_fullsized":"2024-09-12T02:57:59.553574+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:36:57.406907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.547696+0000","last_change":"2024-09-12T02:56:15.982587+0000","last_active":"2024-09-12T02:57:59.547696+0000","last_peered":"2024-09-12T02:57:59.547696+0000","last_clean":"2024-09-12T02:57:59.547696+0000","last_became_active":"2024-09-12T02:56:15.982315+0000","last_became_peered":"2024-09-12T02:56:15.982315+0000","last_unstale":"2024-09-12T02:57:59.547696+0000","last_undegraded":"2024-09-12T02:57:59.547696+0000","last_fullsized":"2024-09-12T02:57:59.547696+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:42:17.736999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.564760+0000","last_change":"2024-09-12T02:56:15.979482+0000","last_active":"2024-09-12T02:57:59.564760+0000","last_peered":"2024-09-12T02:57:59.564760+0000","last_clean":"2024-09-12T02:57:59.564760+0000","last_became_active":"2024-09-12T02:56:15.979122+0000","last_became_peered":"2024-09-12T02:56:15.979122+0000","last_unstale":"2024-09-12T02:57:59.564760+0000","last_undegraded":"2024-09-12T02:57:59.564760+0000","last_fullsized":"2024-09-12T02:57:59.564760+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:18:05.642487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.565218+0000","last_change":"2024-09-12T02:56:15.989908+0000","last_active":"2024-09-12T02:57:59.565218+0000","last_peered":"2024-09-12T02:57:59.565218+0000","last_clean":"2024-09-12T02:57:59.565218+0000","last_became_active":"2024-09-12T02:56:15.989515+0000","last_became_peered":"2024-09-12T02:56:15.989515+0000","last_unstale":"2024-09-12T02:57:59.565218+0000","last_undegraded":"2024-09-12T02:57:59.565218+0000","last_fullsized":"2024-09-12T02:57:59.565218+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:59:57.141607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"83'10","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.557609+0000","last_change":"2024-09-12T02:56:15.980323+0000","last_active":"2024-09-12T02:57:59.557609+0000","last_peered":"2024-09-12T02:57:59.557609+0000","last_clean":"2024-09-12T02:57:59.557609+0000","last_became_active":"2024-09-12T02:56:15.979989+0000","last_became_peered":"2024-09-12T02:56:15.979989+0000","last_unstale":"2024-09-12T02:57:59.557609+0000","last_undegraded":"2024-09-12T02:57:59.557609+0000","last_fullsized":"2024-09-12T02:57:59.557609+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:55:56.064125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.556275+0000","last_change":"2024-09-12T02:56:15.986003+0000","last_active":"2024-09-12T02:57:59.556275+0000","last_peered":"2024-09-12T02:57:59.556275+0000","last_clean":"2024-09-12T02:57:59.556275+0000","last_became_active":"2024-09-12T02:56:15.985335+0000","last_became_peered":"2024-09-12T02:56:15.985335+0000","last_unstale":"2024-09-12T02:57:59.556275+0000","last_undegraded":"2024-09-12T02:57:59.556275+0000","last_fullsized":"2024-09-12T02:57:59.556275+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:51:47.944530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"83'8","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.554254+0000","last_change":"2024-09-12T02:56:31.036870+0000","last_active":"2024-09-12T02:57:59.554254+0000","last_peered":"2024-09-12T02:57:59.554254+0000","last_clean":"2024-09-12T02:57:59.554254+0000","last_became_active":"2024-09-12T02:56:31.036443+0000","last_became_peered":"2024-09-12T02:56:31.036443+0000","last_unstale":"2024-09-12T02:57:59.554254+0000","last_undegraded":"2024-09-12T02:57:59.554254+0000","last_fullsized":"2024-09-12T02:57:59.554254+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:49:29.326184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"83'11","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.555414+0000","last_change":"2024-09-12T02:56:15.974547+0000","last_active":"2024-09-12T02:57:59.555414+0000","last_peered":"2024-09-12T02:57:59.555414+0000","last_clean":"2024-09-12T02:57:59.555414+0000","last_became_active":"2024-09-12T02:56:15.974174+0000","last_became_peered":"2024-09-12T02:56:15.974174+0000","last_unstale":"2024-09-12T02:57:59.555414+0000","last_undegraded":"2024-09-12T02:57:59.555414+0000","last_fullsized":"2024-09-12T02:57:59.555414+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:22:16.954309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"83'10","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.565778+0000","last_change":"2024-09-12T02:56:15.981771+0000","last_active":"2024-09-12T02:57:59.565778+0000","last_peered":"2024-09-12T02:57:59.565778+0000","last_clean":"2024-09-12T02:57:59.565778+0000","last_became_active":"2024-09-12T02:56:15.980948+0000","last_became_peered":"2024-09-12T02:56:15.980948+0000","last_unstale":"2024-09-12T02:57:59.565778+0000","last_undegraded":"2024-09-12T02:57:59.565778+0000","last_fullsized":"2024-09-12T02:57:59.565778+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:34:35.961348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.562061+0000","last_change":"2024-09-12T02:56:15.982318+0000","last_active":"2024-09-12T02:57:59.562061+0000","last_peered":"2024-09-12T02:57:59.562061+0000","last_clean":"2024-09-12T02:57:59.562061+0000","last_became_active":"2024-09-12T02:56:15.982013+0000","last_became_peered":"2024-09-12T02:56:15.982013+0000","last_unstale":"2024-09-12T02:57:59.562061+0000","last_undegraded":"2024-09-12T02:57:59.562061+0000","last_fullsized":"2024-09-12T02:57:59.562061+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:10:50.895579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.559580+0000","last_change":"2024-09-12T02:56:15.969992+0000","last_active":"2024-09-12T02:57:59.559580+0000","last_peered":"2024-09-12T02:57:59.559580+0000","last_clean":"2024-09-12T02:57:59.559580+0000","last_became_active":"2024-09-12T02:56:15.969621+0000","last_became_peered":"2024-09-12T02:56:15.969621+0000","last_unstale":"2024-09-12T02:57:59.559580+0000","last_undegraded":"2024-09-12T02:57:59.559580+0000","last_fullsized":"2024-09-12T02:57:59.559580+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:47:40.206042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"83'2","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.560304+0000","last_change":"2024-09-12T02:56:15.982315+0000","last_active":"2024-09-12T02:57:59.560304+0000","last_peered":"2024-09-12T02:57:59.560304+0000","last_clean":"2024-09-12T02:57:59.560304+0000","last_became_active":"2024-09-12T02:56:15.981992+0000","last_became_peered":"2024-09-12T02:56:15.981992+0000","last_unstale":"2024-09-12T02:57:59.560304+0000","last_undegraded":"2024-09-12T02:57:59.560304+0000","last_fullsized":"2024-09-12T02:57:59.560304+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:54:04.188418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"83'15","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.560612+0000","last_change":"2024-09-12T02:56:15.980712+0000","last_active":"2024-09-12T02:57:59.560612+0000","last_peered":"2024-09-12T02:57:59.560612+0000","last_clean":"2024-09-12T02:57:59.560612+0000","last_became_active":"2024-09-12T02:56:15.980410+0000","last_became_peered":"2024-09-12T02:56:15.980410+0000","last_unstale":"2024-09-12T02:57:59.560612+0000","last_undegraded":"2024-09-12T02:57:59.560612+0000","last_fullsized":"2024-09-12T02:57:59.560612+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:13:53.844490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.557058+0000","last_change":"2024-09-12T02:56:15.981675+0000","last_active":"2024-09-12T02:57:59.557058+0000","last_peered":"2024-09-12T02:57:59.557058+0000","last_clean":"2024-09-12T02:57:59.557058+0000","last_became_active":"2024-09-12T02:56:15.981372+0000","last_became_peered":"2024-09-12T02:56:15.981372+0000","last_unstale":"2024-09-12T02:57:59.557058+0000","last_undegraded":"2024-09-12T02:57:59.557058+0000","last_fullsized":"2024-09-12T02:57:59.557058+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:39:17.399199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"83'4","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.557413+0000","last_change":"2024-09-12T02:56:15.972750+0000","last_active":"2024-09-12T02:57:59.557413+0000","last_peered":"2024-09-12T02:57:59.557413+0000","last_clean":"2024-09-12T02:57:59.557413+0000","last_became_active":"2024-09-12T02:56:15.972434+0000","last_became_peered":"2024-09-12T02:56:15.972434+0000","last_unstale":"2024-09-12T02:57:59.557413+0000","last_undegraded":"2024-09-12T02:57:59.557413+0000","last_fullsized":"2024-09-12T02:57:59.557413+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:59:22.784908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.545235+0000","last_change":"2024-09-12T02:56:15.973567+0000","last_active":"2024-09-12T02:57:59.545235+0000","last_peered":"2024-09-12T02:57:59.545235+0000","last_clean":"2024-09-12T02:57:59.545235+0000","last_became_active":"2024-09-12T02:56:15.973251+0000","last_became_peered":"2024-09-12T02:56:15.973251+0000","last_unstale":"2024-09-12T02:57:59.545235+0000","last_undegraded":"2024-09-12T02:57:59.545235+0000","last_fullsized":"2024-09-12T02:57:59.545235+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:16:39.536817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.540308+0000","last_change":"2024-09-12T02:56:15.986108+0000","last_active":"2024-09-12T02:57:59.540308+0000","last_peered":"2024-09-12T02:57:59.540308+0000","last_clean":"2024-09-12T02:57:59.540308+0000","last_became_active":"2024-09-12T02:56:15.985701+0000","last_became_peered":"2024-09-12T02:56:15.985701+0000","last_unstale":"2024-09-12T02:57:59.540308+0000","last_undegraded":"2024-09-12T02:57:59.540308+0000","last_fullsized":"2024-09-12T02:57:59.540308+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:59.377473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"83'12","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.555638+0000","last_change":"2024-09-12T02:56:15.980078+0000","last_active":"2024-09-12T02:57:59.555638+0000","last_peered":"2024-09-12T02:57:59.555638+0000","last_clean":"2024-09-12T02:57:59.555638+0000","last_became_active":"2024-09-12T02:56:15.979785+0000","last_became_peered":"2024-09-12T02:56:15.979785+0000","last_unstale":"2024-09-12T02:57:59.555638+0000","last_undegraded":"2024-09-12T02:57:59.555638+0000","last_fullsized":"2024-09-12T02:57:59.555638+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:00:19.252398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.559119+0000","last_change":"2024-09-12T02:56:15.984170+0000","last_active":"2024-09-12T02:57:59.559119+0000","last_peered":"2024-09-12T02:57:59.559119+0000","last_clean":"2024-09-12T02:57:59.559119+0000","last_became_active":"2024-09-12T02:56:15.983416+0000","last_became_peered":"2024-09-12T02:56:15.983416+0000","last_unstale":"2024-09-12T02:57:59.559119+0000","last_undegraded":"2024-09-12T02:57:59.559119+0000","last_fullsized":"2024-09-12T02:57:59.559119+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:13:24.723923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.556295+0000","last_change":"2024-09-12T02:56:15.980987+0000","last_active":"2024-09-12T02:57:59.556295+0000","last_peered":"2024-09-12T02:57:59.556295+0000","last_clean":"2024-09-12T02:57:59.556295+0000","last_became_active":"2024-09-12T02:56:15.980661+0000","last_became_peered":"2024-09-12T02:56:15.980661+0000","last_unstale":"2024-09-12T02:57:59.556295+0000","last_undegraded":"2024-09-12T02:57:59.556295+0000","last_fullsized":"2024-09-12T02:57:59.556295+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:19:49.294698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"81'5","reported_seq":135,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:58:00.172973+0000","last_change":"2024-09-12T02:56:31.042914+0000","last_active":"2024-09-12T02:58:00.172973+0000","last_peered":"2024-09-12T02:58:00.172973+0000","last_clean":"2024-09-12T02:58:00.172973+0000","last_became_active":"2024-09-12T02:56:31.037356+0000","last_became_peered":"2024-09-12T02:56:31.037356+0000","last_unstale":"2024-09-12T02:58:00.172973+0000","last_undegraded":"2024-09-12T02:58:00.172973+0000","last_fullsized":"2024-09-12T02:58:00.172973+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:17:48.582725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":622,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"83'8","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.565986+0000","last_change":"2024-09-12T02:56:31.036170+0000","last_active":"2024-09-12T02:57:59.565986+0000","last_peered":"2024-09-12T02:57:59.565986+0000","last_clean":"2024-09-12T02:57:59.565986+0000","last_became_active":"2024-09-12T02:56:31.035754+0000","last_became_peered":"2024-09-12T02:56:31.035754+0000","last_unstale":"2024-09-12T02:57:59.565986+0000","last_undegraded":"2024-09-12T02:57:59.565986+0000","last_fullsized":"2024-09-12T02:57:59.565986+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:58:06.464818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1a","version":"81'7","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.032813+0000","last_change":"2024-09-12T02:56:13.931202+0000","last_active":"2024-09-12T02:56:31.032813+0000","last_peered":"2024-09-12T02:56:31.032813+0000","last_clean":"2024-09-12T02:56:31.032813+0000","last_became_active":"2024-09-12T02:56:13.930776+0000","last_became_peered":"2024-09-12T02:56:13.930776+0000","last_unstale":"2024-09-12T02:56:31.032813+0000","last_undegraded":"2024-09-12T02:56:31.032813+0000","last_fullsized":"2024-09-12T02:56:31.032813+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:22:16.919253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"81'3","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.031662+0000","last_change":"2024-09-12T02:56:13.924660+0000","last_active":"2024-09-12T02:56:31.031662+0000","last_peered":"2024-09-12T02:56:31.031662+0000","last_clean":"2024-09-12T02:56:31.031662+0000","last_became_active":"2024-09-12T02:56:13.924342+0000","last_became_peered":"2024-09-12T02:56:13.924342+0000","last_unstale":"2024-09-12T02:56:31.031662+0000","last_undegraded":"2024-09-12T02:56:31.031662+0000","last_fullsized":"2024-09-12T02:56:31.031662+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:48:53.907044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.563814+0000","last_change":"2024-09-12T02:56:15.978723+0000","last_active":"2024-09-12T02:57:59.563814+0000","last_peered":"2024-09-12T02:57:59.563814+0000","last_clean":"2024-09-12T02:57:59.563814+0000","last_became_active":"2024-09-12T02:56:15.978344+0000","last_became_peered":"2024-09-12T02:56:15.978344+0000","last_unstale":"2024-09-12T02:57:59.563814+0000","last_undegraded":"2024-09-12T02:57:59.563814+0000","last_fullsized":"2024-09-12T02:57:59.563814+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:25:04.559395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"81'3","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.032936+0000","last_change":"2024-09-12T02:56:13.933065+0000","last_active":"2024-09-12T02:56:31.032936+0000","last_peered":"2024-09-12T02:56:31.032936+0000","last_clean":"2024-09-12T02:56:31.032936+0000","last_became_active":"2024-09-12T02:56:13.932597+0000","last_became_peered":"2024-09-12T02:56:13.932597+0000","last_unstale":"2024-09-12T02:56:31.032936+0000","last_undegraded":"2024-09-12T02:56:31.032936+0000","last_fullsized":"2024-09-12T02:56:31.032936+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:57:26.563392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"83'11","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.566801+0000","last_change":"2024-09-12T02:56:15.975722+0000","last_active":"2024-09-12T02:57:59.566801+0000","last_peered":"2024-09-12T02:57:59.566801+0000","last_clean":"2024-09-12T02:57:59.566801+0000","last_became_active":"2024-09-12T02:56:15.975342+0000","last_became_peered":"2024-09-12T02:56:15.975342+0000","last_unstale":"2024-09-12T02:57:59.566801+0000","last_undegraded":"2024-09-12T02:57:59.566801+0000","last_fullsized":"2024-09-12T02:57:59.566801+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:33:17.613738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'5","reported_seq":217,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:58:01.977448+0000","last_change":"2024-09-12T02:56:13.929700+0000","last_active":"2024-09-12T02:58:01.977448+0000","last_peered":"2024-09-12T02:58:01.977448+0000","last_clean":"2024-09-12T02:58:01.977448+0000","last_became_active":"2024-09-12T02:56:13.929317+0000","last_became_peered":"2024-09-12T02:56:13.929317+0000","last_unstale":"2024-09-12T02:58:01.977448+0000","last_undegraded":"2024-09-12T02:58:01.977448+0000","last_fullsized":"2024-09-12T02:58:01.977448+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:46:32.172674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":190,"num_read_kb":190,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"83'10","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.564710+0000","last_change":"2024-09-12T02:56:15.985754+0000","last_active":"2024-09-12T02:57:59.564710+0000","last_peered":"2024-09-12T02:57:59.564710+0000","last_clean":"2024-09-12T02:57:59.564710+0000","last_became_active":"2024-09-12T02:56:15.985402+0000","last_became_peered":"2024-09-12T02:56:15.985402+0000","last_unstale":"2024-09-12T02:57:59.564710+0000","last_undegraded":"2024-09-12T02:57:59.564710+0000","last_fullsized":"2024-09-12T02:57:59.564710+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:05:29.979961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"81'4","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.035428+0000","last_change":"2024-09-12T02:56:13.933329+0000","last_active":"2024-09-12T02:56:31.035428+0000","last_peered":"2024-09-12T02:56:31.035428+0000","last_clean":"2024-09-12T02:56:31.035428+0000","last_became_active":"2024-09-12T02:56:13.932960+0000","last_became_peered":"2024-09-12T02:56:13.932960+0000","last_unstale":"2024-09-12T02:56:31.035428+0000","last_undegraded":"2024-09-12T02:56:31.035428+0000","last_fullsized":"2024-09-12T02:56:31.035428+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:02:57.564202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"83'9","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.556804+0000","last_change":"2024-09-12T02:56:31.037340+0000","last_active":"2024-09-12T02:57:59.556804+0000","last_peered":"2024-09-12T02:57:59.556804+0000","last_clean":"2024-09-12T02:57:59.556804+0000","last_became_active":"2024-09-12T02:56:31.036993+0000","last_became_peered":"2024-09-12T02:56:31.036993+0000","last_unstale":"2024-09-12T02:57:59.556804+0000","last_undegraded":"2024-09-12T02:57:59.556804+0000","last_fullsized":"2024-09-12T02:57:59.556804+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:44:27.788791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"81'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.033005+0000","last_change":"2024-09-12T02:56:14.077202+0000","last_active":"2024-09-12T02:56:31.033005+0000","last_peered":"2024-09-12T02:56:31.033005+0000","last_clean":"2024-09-12T02:56:31.033005+0000","last_became_active":"2024-09-12T02:56:14.076610+0000","last_became_peered":"2024-09-12T02:56:14.076610+0000","last_unstale":"2024-09-12T02:56:31.033005+0000","last_undegraded":"2024-09-12T02:56:31.033005+0000","last_fullsized":"2024-09-12T02:56:31.033005+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:37:50.333152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"83'11","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.558754+0000","last_change":"2024-09-12T02:56:15.977161+0000","last_active":"2024-09-12T02:57:59.558754+0000","last_peered":"2024-09-12T02:57:59.558754+0000","last_clean":"2024-09-12T02:57:59.558754+0000","last_became_active":"2024-09-12T02:56:15.976683+0000","last_became_peered":"2024-09-12T02:56:15.976683+0000","last_unstale":"2024-09-12T02:57:59.558754+0000","last_undegraded":"2024-09-12T02:57:59.558754+0000","last_fullsized":"2024-09-12T02:57:59.558754+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:34:07.526890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'36","reported_seq":259,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:58:01.977471+0000","last_change":"2024-09-12T02:56:13.931680+0000","last_active":"2024-09-12T02:58:01.977471+0000","last_peered":"2024-09-12T02:58:01.977471+0000","last_clean":"2024-09-12T02:58:01.977471+0000","last_became_active":"2024-09-12T02:56:13.931214+0000","last_became_peered":"2024-09-12T02:56:13.931214+0000","last_unstale":"2024-09-12T02:58:01.977471+0000","last_undegraded":"2024-09-12T02:58:01.977471+0000","last_fullsized":"2024-09-12T02:58:01.977471+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:38:34.151933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66038,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":201,"num_read_kb":160,"num_write":38,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.555616+0000","last_change":"2024-09-12T02:56:15.977720+0000","last_active":"2024-09-12T02:57:59.555616+0000","last_peered":"2024-09-12T02:57:59.555616+0000","last_clean":"2024-09-12T02:57:59.555616+0000","last_became_active":"2024-09-12T02:56:15.977380+0000","last_became_peered":"2024-09-12T02:56:15.977380+0000","last_unstale":"2024-09-12T02:57:59.555616+0000","last_undegraded":"2024-09-12T02:57:59.555616+0000","last_fullsized":"2024-09-12T02:57:59.555616+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:52:16.841909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"81'9","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.030362+0000","last_change":"2024-09-12T02:56:13.929496+0000","last_active":"2024-09-12T02:56:31.030362+0000","last_peered":"2024-09-12T02:56:31.030362+0000","last_clean":"2024-09-12T02:56:31.030362+0000","last_became_active":"2024-09-12T02:56:13.929117+0000","last_became_peered":"2024-09-12T02:56:13.929117+0000","last_unstale":"2024-09-12T02:56:31.030362+0000","last_undegraded":"2024-09-12T02:56:31.030362+0000","last_fullsized":"2024-09-12T02:56:31.030362+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:59:38.399135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.564811+0000","last_change":"2024-09-12T02:56:15.974323+0000","last_active":"2024-09-12T02:57:59.564811+0000","last_peered":"2024-09-12T02:57:59.564811+0000","last_clean":"2024-09-12T02:57:59.564811+0000","last_became_active":"2024-09-12T02:56:15.973819+0000","last_became_peered":"2024-09-12T02:56:15.973819+0000","last_unstale":"2024-09-12T02:57:59.564811+0000","last_undegraded":"2024-09-12T02:57:59.564811+0000","last_fullsized":"2024-09-12T02:57:59.564811+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:53:13.130695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"81'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.032473+0000","last_change":"2024-09-12T02:56:13.935263+0000","last_active":"2024-09-12T02:56:31.032473+0000","last_peered":"2024-09-12T02:56:31.032473+0000","last_clean":"2024-09-12T02:56:31.032473+0000","last_became_active":"2024-09-12T02:56:13.934805+0000","last_became_peered":"2024-09-12T02:56:13.934805+0000","last_unstale":"2024-09-12T02:56:31.032473+0000","last_undegraded":"2024-09-12T02:56:31.032473+0000","last_fullsized":"2024-09-12T02:56:31.032473+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:39:42.882031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"83'15","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.560318+0000","last_change":"2024-09-12T02:56:15.976630+0000","last_active":"2024-09-12T02:57:59.560318+0000","last_peered":"2024-09-12T02:57:59.560318+0000","last_clean":"2024-09-12T02:57:59.560318+0000","last_became_active":"2024-09-12T02:56:15.976304+0000","last_became_peered":"2024-09-12T02:56:15.976304+0000","last_unstale":"2024-09-12T02:57:59.560318+0000","last_undegraded":"2024-09-12T02:57:59.560318+0000","last_fullsized":"2024-09-12T02:57:59.560318+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:46:03.366740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'7","reported_seq":126,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.973453+0000","last_change":"2024-09-12T02:56:13.934224+0000","last_active":"2024-09-12T02:57:59.973453+0000","last_peered":"2024-09-12T02:57:59.973453+0000","last_clean":"2024-09-12T02:57:59.973453+0000","last_became_active":"2024-09-12T02:56:13.933838+0000","last_became_peered":"2024-09-12T02:56:13.933838+0000","last_unstale":"2024-09-12T02:57:59.973453+0000","last_undegraded":"2024-09-12T02:57:59.973453+0000","last_fullsized":"2024-09-12T02:57:59.973453+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:34:34.206051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":98,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.557213+0000","last_change":"2024-09-12T02:56:15.980364+0000","last_active":"2024-09-12T02:57:59.557213+0000","last_peered":"2024-09-12T02:57:59.557213+0000","last_clean":"2024-09-12T02:57:59.557213+0000","last_became_active":"2024-09-12T02:56:15.979437+0000","last_became_peered":"2024-09-12T02:56:15.979437+0000","last_unstale":"2024-09-12T02:57:59.557213+0000","last_undegraded":"2024-09-12T02:57:59.557213+0000","last_fullsized":"2024-09-12T02:57:59.557213+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:43:33.424867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"81'17","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.050509+0000","last_change":"2024-09-12T02:56:31.050046+0000","last_active":"2024-09-12T02:56:31.050509+0000","last_peered":"2024-09-12T02:56:31.050509+0000","last_clean":"2024-09-12T02:56:31.050509+0000","last_became_active":"2024-09-12T02:56:31.036990+0000","last_became_peered":"2024-09-12T02:56:31.036990+0000","last_unstale":"2024-09-12T02:56:31.050509+0000","last_undegraded":"2024-09-12T02:56:31.050509+0000","last_fullsized":"2024-09-12T02:56:31.050509+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:05:53.336691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.563297+0000","last_change":"2024-09-12T02:56:15.986434+0000","last_active":"2024-09-12T02:57:59.563297+0000","last_peered":"2024-09-12T02:57:59.563297+0000","last_clean":"2024-09-12T02:57:59.563297+0000","last_became_active":"2024-09-12T02:56:15.986101+0000","last_became_peered":"2024-09-12T02:56:15.986101+0000","last_unstale":"2024-09-12T02:57:59.563297+0000","last_undegraded":"2024-09-12T02:57:59.563297+0000","last_fullsized":"2024-09-12T02:57:59.563297+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:51:27.840951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"81'2","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.035974+0000","last_change":"2024-09-12T02:56:13.925068+0000","last_active":"2024-09-12T02:56:31.035974+0000","last_peered":"2024-09-12T02:56:31.035974+0000","last_clean":"2024-09-12T02:56:31.035974+0000","last_became_active":"2024-09-12T02:56:13.924535+0000","last_became_peered":"2024-09-12T02:56:13.924535+0000","last_unstale":"2024-09-12T02:56:31.035974+0000","last_undegraded":"2024-09-12T02:56:31.035974+0000","last_fullsized":"2024-09-12T02:56:31.035974+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:45:32.478632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"83'5","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.559020+0000","last_change":"2024-09-12T02:56:15.983850+0000","last_active":"2024-09-12T02:57:59.559020+0000","last_peered":"2024-09-12T02:57:59.559020+0000","last_clean":"2024-09-12T02:57:59.559020+0000","last_became_active":"2024-09-12T02:56:15.983480+0000","last_became_peered":"2024-09-12T02:56:15.983480+0000","last_unstale":"2024-09-12T02:57:59.559020+0000","last_undegraded":"2024-09-12T02:57:59.559020+0000","last_fullsized":"2024-09-12T02:57:59.559020+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:09:42.185237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"81'5","reported_seq":39,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.312450+0000","last_change":"2024-09-12T02:56:31.312450+0000","last_active":"2024-09-12T02:56:31.312450+0000","last_peered":"2024-09-12T02:56:31.312450+0000","last_clean":"2024-09-12T02:56:31.312450+0000","last_became_active":"2024-09-12T02:56:31.308030+0000","last_became_peered":"2024-09-12T02:56:31.308030+0000","last_unstale":"2024-09-12T02:56:31.312450+0000","last_undegraded":"2024-09-12T02:56:31.312450+0000","last_fullsized":"2024-09-12T02:56:31.312450+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:15:14.692292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.562640+0000","last_change":"2024-09-12T02:56:15.978996+0000","last_active":"2024-09-12T02:57:59.562640+0000","last_peered":"2024-09-12T02:57:59.562640+0000","last_clean":"2024-09-12T02:57:59.562640+0000","last_became_active":"2024-09-12T02:56:15.978636+0000","last_became_peered":"2024-09-12T02:56:15.978636+0000","last_unstale":"2024-09-12T02:57:59.562640+0000","last_undegraded":"2024-09-12T02:57:59.562640+0000","last_fullsized":"2024-09-12T02:57:59.562640+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:29:20.401664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"81'3","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.033119+0000","last_change":"2024-09-12T02:56:13.929388+0000","last_active":"2024-09-12T02:56:31.033119+0000","last_peered":"2024-09-12T02:56:31.033119+0000","last_clean":"2024-09-12T02:56:31.033119+0000","last_became_active":"2024-09-12T02:56:13.928998+0000","last_became_peered":"2024-09-12T02:56:13.928998+0000","last_unstale":"2024-09-12T02:56:31.033119+0000","last_undegraded":"2024-09-12T02:56:31.033119+0000","last_fullsized":"2024-09-12T02:56:31.033119+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:37:57.368050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.560190+0000","last_change":"2024-09-12T02:56:15.977491+0000","last_active":"2024-09-12T02:57:59.560190+0000","last_peered":"2024-09-12T02:57:59.560190+0000","last_clean":"2024-09-12T02:57:59.560190+0000","last_became_active":"2024-09-12T02:56:15.977160+0000","last_became_peered":"2024-09-12T02:56:15.977160+0000","last_unstale":"2024-09-12T02:57:59.560190+0000","last_undegraded":"2024-09-12T02:57:59.560190+0000","last_fullsized":"2024-09-12T02:57:59.560190+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:05:40.577342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'7","reported_seq":128,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:58:00.976755+0000","last_change":"2024-09-12T02:56:14.327147+0000","last_active":"2024-09-12T02:58:00.976755+0000","last_peered":"2024-09-12T02:58:00.976755+0000","last_clean":"2024-09-12T02:58:00.976755+0000","last_became_active":"2024-09-12T02:56:14.326737+0000","last_became_peered":"2024-09-12T02:56:14.326737+0000","last_unstale":"2024-09-12T02:58:00.976755+0000","last_undegraded":"2024-09-12T02:58:00.976755+0000","last_fullsized":"2024-09-12T02:58:00.976755+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:55:33.510599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1704,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":99,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.563684+0000","last_change":"2024-09-12T02:56:15.984363+0000","last_active":"2024-09-12T02:57:59.563684+0000","last_peered":"2024-09-12T02:57:59.563684+0000","last_clean":"2024-09-12T02:57:59.563684+0000","last_became_active":"2024-09-12T02:56:15.983822+0000","last_became_peered":"2024-09-12T02:56:15.983822+0000","last_unstale":"2024-09-12T02:57:59.563684+0000","last_undegraded":"2024-09-12T02:57:59.563684+0000","last_fullsized":"2024-09-12T02:57:59.563684+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:41:46.082716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":333,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.035564+0000","last_change":"2024-09-12T02:55:05.800210+0000","last_active":"2024-09-12T02:56:31.035564+0000","last_peered":"2024-09-12T02:56:31.035564+0000","last_clean":"2024-09-12T02:56:31.035564+0000","last_became_active":"2024-09-12T02:55:05.799791+0000","last_became_peered":"2024-09-12T02:55:05.799791+0000","last_unstale":"2024-09-12T02:56:31.035564+0000","last_undegraded":"2024-09-12T02:56:31.035564+0000","last_fullsized":"2024-09-12T02:56:31.035564+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:51:48.530016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:51:48.530016+0000","last_clean_scrub_stamp":"2024-09-12T02:51:48.530016+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:05:47.416697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"81'6","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.031169+0000","last_change":"2024-09-12T02:56:13.930269+0000","last_active":"2024-09-12T02:56:31.031169+0000","last_peered":"2024-09-12T02:56:31.031169+0000","last_clean":"2024-09-12T02:56:31.031169+0000","last_became_active":"2024-09-12T02:56:13.929831+0000","last_became_peered":"2024-09-12T02:56:13.929831+0000","last_unstale":"2024-09-12T02:56:31.031169+0000","last_undegraded":"2024-09-12T02:56:31.031169+0000","last_fullsized":"2024-09-12T02:56:31.031169+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:38:17.878398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.561332+0000","last_change":"2024-09-12T02:56:15.969744+0000","last_active":"2024-09-12T02:57:59.561332+0000","last_peered":"2024-09-12T02:57:59.561332+0000","last_clean":"2024-09-12T02:57:59.561332+0000","last_became_active":"2024-09-12T02:56:15.969354+0000","last_became_peered":"2024-09-12T02:56:15.969354+0000","last_unstale":"2024-09-12T02:57:59.561332+0000","last_undegraded":"2024-09-12T02:57:59.561332+0000","last_fullsized":"2024-09-12T02:57:59.561332+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:26:02.409510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"81'4","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.035385+0000","last_change":"2024-09-12T02:56:13.929231+0000","last_active":"2024-09-12T02:56:31.035385+0000","last_peered":"2024-09-12T02:56:31.035385+0000","last_clean":"2024-09-12T02:56:31.035385+0000","last_became_active":"2024-09-12T02:56:13.928843+0000","last_became_peered":"2024-09-12T02:56:13.928843+0000","last_unstale":"2024-09-12T02:56:31.035385+0000","last_undegraded":"2024-09-12T02:56:31.035385+0000","last_fullsized":"2024-09-12T02:56:31.035385+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:26:33.619755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"83'5","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.555927+0000","last_change":"2024-09-12T02:56:15.982503+0000","last_active":"2024-09-12T02:57:59.555927+0000","last_peered":"2024-09-12T02:57:59.555927+0000","last_clean":"2024-09-12T02:57:59.555927+0000","last_became_active":"2024-09-12T02:56:15.982192+0000","last_became_peered":"2024-09-12T02:56:15.982192+0000","last_unstale":"2024-09-12T02:57:59.555927+0000","last_undegraded":"2024-09-12T02:57:59.555927+0000","last_fullsized":"2024-09-12T02:57:59.555927+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:55:18.370967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"83'6","reported_seq":127,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:58:03.979145+0000","last_change":"2024-09-12T02:56:14.075486+0000","last_active":"2024-09-12T02:58:03.979145+0000","last_peered":"2024-09-12T02:58:03.979145+0000","last_clean":"2024-09-12T02:58:03.979145+0000","last_became_active":"2024-09-12T02:56:14.075152+0000","last_became_peered":"2024-09-12T02:56:14.075152+0000","last_unstale":"2024-09-12T02:58:03.979145+0000","last_undegraded":"2024-09-12T02:58:03.979145+0000","last_fullsized":"2024-09-12T02:58:03.979145+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:59:40.950191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":99,"num_read_kb":99,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.556340+0000","last_change":"2024-09-12T02:56:15.981584+0000","last_active":"2024-09-12T02:57:59.556340+0000","last_peered":"2024-09-12T02:57:59.556340+0000","last_clean":"2024-09-12T02:57:59.556340+0000","last_became_active":"2024-09-12T02:56:15.981240+0000","last_became_peered":"2024-09-12T02:56:15.981240+0000","last_unstale":"2024-09-12T02:57:59.556340+0000","last_undegraded":"2024-09-12T02:57:59.556340+0000","last_fullsized":"2024-09-12T02:57:59.556340+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:49:04.036830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"83'7","reported_seq":129,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:58:03.979513+0000","last_change":"2024-09-12T02:56:13.935143+0000","last_active":"2024-09-12T02:58:03.979513+0000","last_peered":"2024-09-12T02:58:03.979513+0000","last_clean":"2024-09-12T02:58:03.979513+0000","last_became_active":"2024-09-12T02:56:13.934792+0000","last_became_peered":"2024-09-12T02:56:13.934792+0000","last_unstale":"2024-09-12T02:58:03.979513+0000","last_undegraded":"2024-09-12T02:58:03.979513+0000","last_fullsized":"2024-09-12T02:58:03.979513+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:00:26.973612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1289,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":100,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"83'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.558893+0000","last_change":"2024-09-12T02:56:31.034825+0000","last_active":"2024-09-12T02:57:59.558893+0000","last_peered":"2024-09-12T02:57:59.558893+0000","last_clean":"2024-09-12T02:57:59.558893+0000","last_became_active":"2024-09-12T02:56:31.034516+0000","last_became_peered":"2024-09-12T02:56:31.034516+0000","last_unstale":"2024-09-12T02:57:59.558893+0000","last_undegraded":"2024-09-12T02:57:59.558893+0000","last_fullsized":"2024-09-12T02:57:59.558893+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:10:42.161023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"81'1","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.029027+0000","last_change":"2024-09-12T02:56:14.076762+0000","last_active":"2024-09-12T02:56:31.029027+0000","last_peered":"2024-09-12T02:56:31.029027+0000","last_clean":"2024-09-12T02:56:31.029027+0000","last_became_active":"2024-09-12T02:56:14.076421+0000","last_became_peered":"2024-09-12T02:56:14.076421+0000","last_unstale":"2024-09-12T02:56:31.029027+0000","last_undegraded":"2024-09-12T02:56:31.029027+0000","last_fullsized":"2024-09-12T02:56:31.029027+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:56:56.422910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'5","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.555835+0000","last_change":"2024-09-12T02:56:15.978810+0000","last_active":"2024-09-12T02:57:59.555835+0000","last_peered":"2024-09-12T02:57:59.555835+0000","last_clean":"2024-09-12T02:57:59.555835+0000","last_became_active":"2024-09-12T02:56:15.978038+0000","last_became_peered":"2024-09-12T02:56:15.978038+0000","last_unstale":"2024-09-12T02:57:59.555835+0000","last_undegraded":"2024-09-12T02:57:59.555835+0000","last_fullsized":"2024-09-12T02:57:59.555835+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:04:52.215784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"81'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.035440+0000","last_change":"2024-09-12T02:56:14.075987+0000","last_active":"2024-09-12T02:56:31.035440+0000","last_peered":"2024-09-12T02:56:31.035440+0000","last_clean":"2024-09-12T02:56:31.035440+0000","last_became_active":"2024-09-12T02:56:14.075596+0000","last_became_peered":"2024-09-12T02:56:14.075596+0000","last_unstale":"2024-09-12T02:56:31.035440+0000","last_undegraded":"2024-09-12T02:56:31.035440+0000","last_fullsized":"2024-09-12T02:56:31.035440+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:18:48.713387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.552400+0000","last_change":"2024-09-12T02:56:15.987842+0000","last_active":"2024-09-12T02:57:59.552400+0000","last_peered":"2024-09-12T02:57:59.552400+0000","last_clean":"2024-09-12T02:57:59.552400+0000","last_became_active":"2024-09-12T02:56:15.987507+0000","last_became_peered":"2024-09-12T02:56:15.987507+0000","last_unstale":"2024-09-12T02:57:59.552400+0000","last_undegraded":"2024-09-12T02:57:59.552400+0000","last_fullsized":"2024-09-12T02:57:59.552400+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:52:23.000549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'19","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:58:01.991961+0000","last_change":"2024-09-12T02:56:13.924372+0000","last_active":"2024-09-12T02:58:01.991961+0000","last_peered":"2024-09-12T02:58:01.991961+0000","last_clean":"2024-09-12T02:58:01.991961+0000","last_became_active":"2024-09-12T02:56:13.923955+0000","last_became_peered":"2024-09-12T02:56:13.923955+0000","last_unstale":"2024-09-12T02:58:01.991961+0000","last_undegraded":"2024-09-12T02:58:01.991961+0000","last_fullsized":"2024-09-12T02:58:01.991961+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:52:47.452410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":34,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"83'5","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.562919+0000","last_change":"2024-09-12T02:56:15.978262+0000","last_active":"2024-09-12T02:57:59.562919+0000","last_peered":"2024-09-12T02:57:59.562919+0000","last_clean":"2024-09-12T02:57:59.562919+0000","last_became_active":"2024-09-12T02:56:15.977913+0000","last_became_peered":"2024-09-12T02:56:15.977913+0000","last_unstale":"2024-09-12T02:57:59.562919+0000","last_undegraded":"2024-09-12T02:57:59.562919+0000","last_fullsized":"2024-09-12T02:57:59.562919+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:35:27.104635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"81'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.031488+0000","last_change":"2024-09-12T02:56:14.327417+0000","last_active":"2024-09-12T02:56:31.031488+0000","last_peered":"2024-09-12T02:56:31.031488+0000","last_clean":"2024-09-12T02:56:31.031488+0000","last_became_active":"2024-09-12T02:56:14.327001+0000","last_became_peered":"2024-09-12T02:56:14.327001+0000","last_unstale":"2024-09-12T02:56:31.031488+0000","last_undegraded":"2024-09-12T02:56:31.031488+0000","last_fullsized":"2024-09-12T02:56:31.031488+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:15:39.019252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.565859+0000","last_change":"2024-09-12T02:56:15.977286+0000","last_active":"2024-09-12T02:57:59.565859+0000","last_peered":"2024-09-12T02:57:59.565859+0000","last_clean":"2024-09-12T02:57:59.565859+0000","last_became_active":"2024-09-12T02:56:15.976944+0000","last_became_peered":"2024-09-12T02:56:15.976944+0000","last_unstale":"2024-09-12T02:57:59.565859+0000","last_undegraded":"2024-09-12T02:57:59.565859+0000","last_fullsized":"2024-09-12T02:57:59.565859+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:22:57.056781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.306212+0000","last_change":"2024-09-12T02:56:14.075701+0000","last_active":"2024-09-12T02:56:31.306212+0000","last_peered":"2024-09-12T02:56:31.306212+0000","last_clean":"2024-09-12T02:56:31.306212+0000","last_became_active":"2024-09-12T02:56:14.075340+0000","last_became_peered":"2024-09-12T02:56:14.075340+0000","last_unstale":"2024-09-12T02:56:31.306212+0000","last_undegraded":"2024-09-12T02:56:31.306212+0000","last_fullsized":"2024-09-12T02:56:31.306212+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:17:42.255058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"83'2","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.528187+0000","last_change":"2024-09-12T02:56:15.979783+0000","last_active":"2024-09-12T02:57:59.528187+0000","last_peered":"2024-09-12T02:57:59.528187+0000","last_clean":"2024-09-12T02:57:59.528187+0000","last_became_active":"2024-09-12T02:56:15.979424+0000","last_became_peered":"2024-09-12T02:56:15.979424+0000","last_unstale":"2024-09-12T02:57:59.528187+0000","last_undegraded":"2024-09-12T02:57:59.528187+0000","last_fullsized":"2024-09-12T02:57:59.528187+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:59:11.568138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"81'7","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.034660+0000","last_change":"2024-09-12T02:56:13.928996+0000","last_active":"2024-09-12T02:56:31.034660+0000","last_peered":"2024-09-12T02:56:31.034660+0000","last_clean":"2024-09-12T02:56:31.034660+0000","last_became_active":"2024-09-12T02:56:13.928664+0000","last_became_peered":"2024-09-12T02:56:13.928664+0000","last_unstale":"2024-09-12T02:56:31.034660+0000","last_undegraded":"2024-09-12T02:56:31.034660+0000","last_fullsized":"2024-09-12T02:56:31.034660+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:37:22.067930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.560217+0000","last_change":"2024-09-12T02:56:15.984096+0000","last_active":"2024-09-12T02:57:59.560217+0000","last_peered":"2024-09-12T02:57:59.560217+0000","last_clean":"2024-09-12T02:57:59.560217+0000","last_became_active":"2024-09-12T02:56:15.983737+0000","last_became_peered":"2024-09-12T02:56:15.983737+0000","last_unstale":"2024-09-12T02:57:59.560217+0000","last_undegraded":"2024-09-12T02:57:59.560217+0000","last_fullsized":"2024-09-12T02:57:59.560217+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:02:17.892318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"81'3","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.053785+0000","last_change":"2024-09-12T02:56:31.053343+0000","last_active":"2024-09-12T02:56:31.053785+0000","last_peered":"2024-09-12T02:56:31.053785+0000","last_clean":"2024-09-12T02:56:31.053785+0000","last_became_active":"2024-09-12T02:56:31.037918+0000","last_became_peered":"2024-09-12T02:56:31.037918+0000","last_unstale":"2024-09-12T02:56:31.053785+0000","last_undegraded":"2024-09-12T02:56:31.053785+0000","last_fullsized":"2024-09-12T02:56:31.053785+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:39:54.990724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.563189+0000","last_change":"2024-09-12T02:56:15.975640+0000","last_active":"2024-09-12T02:57:59.563189+0000","last_peered":"2024-09-12T02:57:59.563189+0000","last_clean":"2024-09-12T02:57:59.563189+0000","last_became_active":"2024-09-12T02:56:15.975342+0000","last_became_peered":"2024-09-12T02:56:15.975342+0000","last_unstale":"2024-09-12T02:57:59.563189+0000","last_undegraded":"2024-09-12T02:57:59.563189+0000","last_fullsized":"2024-09-12T02:57:59.563189+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:26:23.694876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.034922+0000","last_change":"2024-09-12T02:56:13.934149+0000","last_active":"2024-09-12T02:56:31.034922+0000","last_peered":"2024-09-12T02:56:31.034922+0000","last_clean":"2024-09-12T02:56:31.034922+0000","last_became_active":"2024-09-12T02:56:13.933743+0000","last_became_peered":"2024-09-12T02:56:13.933743+0000","last_unstale":"2024-09-12T02:56:31.034922+0000","last_undegraded":"2024-09-12T02:56:31.034922+0000","last_fullsized":"2024-09-12T02:56:31.034922+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:16:17.854457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"83'5","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.563348+0000","last_change":"2024-09-12T02:56:15.984782+0000","last_active":"2024-09-12T02:57:59.563348+0000","last_peered":"2024-09-12T02:57:59.563348+0000","last_clean":"2024-09-12T02:57:59.563348+0000","last_became_active":"2024-09-12T02:56:15.984336+0000","last_became_peered":"2024-09-12T02:56:15.984336+0000","last_unstale":"2024-09-12T02:57:59.563348+0000","last_undegraded":"2024-09-12T02:57:59.563348+0000","last_fullsized":"2024-09-12T02:57:59.563348+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:03:39.832897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"83'15","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.564252+0000","last_change":"2024-09-12T02:56:15.985486+0000","last_active":"2024-09-12T02:57:59.564252+0000","last_peered":"2024-09-12T02:57:59.564252+0000","last_clean":"2024-09-12T02:57:59.564252+0000","last_became_active":"2024-09-12T02:56:15.985167+0000","last_became_peered":"2024-09-12T02:56:15.985167+0000","last_unstale":"2024-09-12T02:57:59.564252+0000","last_undegraded":"2024-09-12T02:57:59.564252+0000","last_fullsized":"2024-09-12T02:57:59.564252+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:46:20.569453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"81'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.035106+0000","last_change":"2024-09-12T02:56:13.929324+0000","last_active":"2024-09-12T02:56:31.035106+0000","last_peered":"2024-09-12T02:56:31.035106+0000","last_clean":"2024-09-12T02:56:31.035106+0000","last_became_active":"2024-09-12T02:56:13.928994+0000","last_became_peered":"2024-09-12T02:56:13.928994+0000","last_unstale":"2024-09-12T02:56:31.035106+0000","last_undegraded":"2024-09-12T02:56:31.035106+0000","last_fullsized":"2024-09-12T02:56:31.035106+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:30:27.290328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.562582+0000","last_change":"2024-09-12T02:56:15.985889+0000","last_active":"2024-09-12T02:57:59.562582+0000","last_peered":"2024-09-12T02:57:59.562582+0000","last_clean":"2024-09-12T02:57:59.562582+0000","last_became_active":"2024-09-12T02:56:15.985509+0000","last_became_peered":"2024-09-12T02:56:15.985509+0000","last_unstale":"2024-09-12T02:57:59.562582+0000","last_undegraded":"2024-09-12T02:57:59.562582+0000","last_fullsized":"2024-09-12T02:57:59.562582+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:30:43.120718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"83'13","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.035437+0000","last_change":"2024-09-12T02:56:13.934192+0000","last_active":"2024-09-12T02:56:31.035437+0000","last_peered":"2024-09-12T02:56:31.035437+0000","last_clean":"2024-09-12T02:56:31.035437+0000","last_became_active":"2024-09-12T02:56:13.933789+0000","last_became_peered":"2024-09-12T02:56:13.933789+0000","last_unstale":"2024-09-12T02:56:31.035437+0000","last_undegraded":"2024-09-12T02:56:31.035437+0000","last_fullsized":"2024-09-12T02:56:31.035437+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:32:47.771085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.563285+0000","last_change":"2024-09-12T02:56:15.983531+0000","last_active":"2024-09-12T02:57:59.563285+0000","last_peered":"2024-09-12T02:57:59.563285+0000","last_clean":"2024-09-12T02:57:59.563285+0000","last_became_active":"2024-09-12T02:56:15.983192+0000","last_became_peered":"2024-09-12T02:56:15.983192+0000","last_unstale":"2024-09-12T02:57:59.563285+0000","last_undegraded":"2024-09-12T02:57:59.563285+0000","last_fullsized":"2024-09-12T02:57:59.563285+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:16:59.923379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.034752+0000","last_change":"2024-09-12T02:56:14.075396+0000","last_active":"2024-09-12T02:56:31.034752+0000","last_peered":"2024-09-12T02:56:31.034752+0000","last_clean":"2024-09-12T02:56:31.034752+0000","last_became_active":"2024-09-12T02:56:14.075023+0000","last_became_peered":"2024-09-12T02:56:14.075023+0000","last_unstale":"2024-09-12T02:56:31.034752+0000","last_undegraded":"2024-09-12T02:56:31.034752+0000","last_fullsized":"2024-09-12T02:56:31.034752+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:41:45.760689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"83'5","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.564093+0000","last_change":"2024-09-12T02:56:15.987276+0000","last_active":"2024-09-12T02:57:59.564093+0000","last_peered":"2024-09-12T02:57:59.564093+0000","last_clean":"2024-09-12T02:57:59.564093+0000","last_became_active":"2024-09-12T02:56:15.986949+0000","last_became_peered":"2024-09-12T02:56:15.986949+0000","last_unstale":"2024-09-12T02:57:59.564093+0000","last_undegraded":"2024-09-12T02:57:59.564093+0000","last_fullsized":"2024-09-12T02:57:59.564093+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:49:45.627093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"83'14","reported_seq":134,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:58:02.977467+0000","last_change":"2024-09-12T02:56:13.932365+0000","last_active":"2024-09-12T02:58:02.977467+0000","last_peered":"2024-09-12T02:58:02.977467+0000","last_clean":"2024-09-12T02:58:02.977467+0000","last_became_active":"2024-09-12T02:56:13.932062+0000","last_became_peered":"2024-09-12T02:56:13.932062+0000","last_unstale":"2024-09-12T02:58:02.977467+0000","last_undegraded":"2024-09-12T02:58:02.977467+0000","last_fullsized":"2024-09-12T02:58:02.977467+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:17:08.451839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":98,"num_read_kb":98,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":74229,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1000,"num_read_kb":580,"num_write":266,"num_write_kb":154,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":734,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":569344,"data_stored":214452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":227,"ondisk_log_size":227,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547235,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":917504,"data_stored":440831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776167,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32756,"kb_used_data":1572,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712396,"statfs":{"total":95995035648,"available":95961493504,"internally_reserved":0,"allocated":1609728,"data_stored":1093472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939693,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40416,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704736,"statfs":{"total":95995035648,"available":95953649664,"internally_reserved":0,"allocated":933888,"data_stored":440115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":40434112},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168626,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":921600,"data_stored":440381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430262,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28516,"kb_used_data":1492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716636,"statfs":{"total":95995035648,"available":95965835264,"internally_reserved":0,"allocated":1527808,"data_stored":1034641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593788,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":917504,"data_stored":440533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822719,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28528,"kb_used_data":1504,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716624,"statfs":{"total":95995035648,"available":95965822976,"internally_reserved":0,"allocated":1540096,"data_stored":1032298,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953541,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":925696,"data_stored":440849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182473,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":933888,"data_stored":439657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411407,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36312,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708840,"statfs":{"total":95995035648,"available":95957852160,"internally_reserved":0,"allocated":991232,"data_stored":502601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574931,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":502469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738457,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":937984,"data_stored":441705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":63784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":63608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":68308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T02:58:07.553 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:58:07.665 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:58:07 smithi148 ceph-mon[35090]: pgmap v331: 97 pgs: 97 active+clean; 649 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-09-12T02:58:07.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:58:07 smithi148 ceph-mon[35090]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-09-12T02:58:07.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:58:07 smithi146 ceph-mon[35011]: pgmap v331: 97 pgs: 97 active+clean; 649 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-09-12T02:58:07.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:58:07 smithi146 ceph-mon[35011]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-09-12T02:58:07.847 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:58:08.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:08.344+0000 7f5d40fea640 1 -- 172.21.15.5:0/2232733028 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d3c102ef0 msgr2=0x7f5d3c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:08.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:08.344+0000 7f5d40fea640 1 --2- 172.21.15.5:0/2232733028 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d3c102ef0 0x7f5d3c10da40 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f5d34009ca0 tx=0x7f5d3402a950 comp rx=0 tx=0).stop 2024-09-12T02:58:08.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:08.344+0000 7f5d40fea640 1 -- 172.21.15.5:0/2232733028 shutdown_connections 2024-09-12T02:58:08.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:08.344+0000 7f5d40fea640 1 --2- 172.21.15.5:0/2232733028 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d3c102ef0 0x7f5d3c10da40 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:08.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:08.344+0000 7f5d40fea640 1 --2- 172.21.15.5:0/2232733028 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5d3c102570 0x7f5d3c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:08.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:08.344+0000 7f5d40fea640 1 --2- 172.21.15.5:0/2232733028 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5d3c100e20 0x7f5d3c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:08.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:08.344+0000 7f5d40fea640 1 -- 172.21.15.5:0/2232733028 >> 172.21.15.5:0/2232733028 conn(0x7f5d3c0fc7d0 msgr2=0x7f5d3c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:08.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:08.344+0000 7f5d40fea640 1 -- 172.21.15.5:0/2232733028 shutdown_connections 2024-09-12T02:58:08.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:08.344+0000 7f5d40fea640 1 -- 172.21.15.5:0/2232733028 wait complete. 2024-09-12T02:58:08.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:08.345+0000 7f5d40fea640 1 Processor -- start 2024-09-12T02:58:08.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:08.345+0000 7f5d40fea640 1 -- start start 2024-09-12T02:58:08.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:08.346+0000 7f5d40fea640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5d3c100e20 0x7f5d3c0fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:08.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:08.346+0000 7f5d40fea640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d3c102570 0x7f5d3c0fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:08.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:08.346+0000 7f5d40fea640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5d3c102ef0 0x7f5d3c0fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:08.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:08.346+0000 7f5d40fea640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5d3c110fd0 con 0x7f5d3c102570 2024-09-12T02:58:08.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:08.346+0000 7f5d40fea640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f5d3c110e50 con 0x7f5d3c100e20 2024-09-12T02:58:08.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:08.346+0000 7f5d40fea640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5d3c111150 con 0x7f5d3c102ef0 2024-09-12T02:58:08.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:08.346+0000 7f5d33fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d3c102570 0x7f5d3c0fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:08.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:08.346+0000 7f5d3a59c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5d3c100e20 0x7f5d3c0fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:08.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:08.346+0000 7f5d33fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d3c102570 0x7f5d3c0fd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55484/0 (socket says 172.21.15.5:55484) 2024-09-12T02:58:08.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:08.346+0000 7f5d33fff640 1 -- 172.21.15.5:0/3143487025 learned_addr learned my addr 172.21.15.5:0/3143487025 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:58:08.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:08.346+0000 7f5d3ad9d640 1 --2- 172.21.15.5:0/3143487025 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5d3c102ef0 0x7f5d3c0fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:08.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:08.346+0000 7f5d33fff640 1 -- 172.21.15.5:0/3143487025 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5d3c102ef0 msgr2=0x7f5d3c0fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:08.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:08.346+0000 7f5d33fff640 1 --2- 172.21.15.5:0/3143487025 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5d3c102ef0 0x7f5d3c0fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:08.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:08.346+0000 7f5d33fff640 1 -- 172.21.15.5:0/3143487025 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5d3c100e20 msgr2=0x7f5d3c0fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:08.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:08.346+0000 7f5d33fff640 1 --2- 172.21.15.5:0/3143487025 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5d3c100e20 0x7f5d3c0fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:08.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:08.346+0000 7f5d33fff640 1 -- 172.21.15.5:0/3143487025 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d3c1a6920 con 0x7f5d3c102570 2024-09-12T02:58:08.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:08.347+0000 7f5d33fff640 1 --2- 172.21.15.5:0/3143487025 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d3c102570 0x7f5d3c0fd1c0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f5d2400c890 tx=0x7f5d2400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:08.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:08.347+0000 7f5d337fe640 1 -- 172.21.15.5:0/3143487025 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d24004220 con 0x7f5d3c102570 2024-09-12T02:58:08.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:08.347+0000 7f5d337fe640 1 -- 172.21.15.5:0/3143487025 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d240043c0 con 0x7f5d3c102570 2024-09-12T02:58:08.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:08.347+0000 7f5d40fea640 1 -- 172.21.15.5:0/3143487025 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d3c1a6c10 con 0x7f5d3c102570 2024-09-12T02:58:08.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:08.347+0000 7f5d337fe640 1 -- 172.21.15.5:0/3143487025 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d240049e0 con 0x7f5d3c102570 2024-09-12T02:58:08.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:08.348+0000 7f5d40fea640 1 -- 172.21.15.5:0/3143487025 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d3c1a7150 con 0x7f5d3c102570 2024-09-12T02:58:08.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:08.349+0000 7f5d40fea640 1 -- 172.21.15.5:0/3143487025 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d3c101200 con 0x7f5d3c102570 2024-09-12T02:58:08.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:08.350+0000 7f5d337fe640 1 -- 172.21.15.5:0/3143487025 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7f5d240079b0 con 0x7f5d3c102570 2024-09-12T02:58:08.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:08.351+0000 7f5d337fe640 1 --2- 172.21.15.5:0/3143487025 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f5d280a2c10 0x7f5d280a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:08.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:08.351+0000 7f5d3a59c640 1 --2- 172.21.15.5:0/3143487025 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f5d280a2c10 0x7f5d280a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:08.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:08.351+0000 7f5d337fe640 1 -- 172.21.15.5:0/3143487025 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f5d240cf2c0 con 0x7f5d3c102570 2024-09-12T02:58:08.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:08.351+0000 7f5d3a59c640 1 --2- 172.21.15.5:0/3143487025 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f5d280a2c10 0x7f5d280a50d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c00bbe0 tx=0x7f5d2c00b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:08.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:08.357+0000 7f5d337fe640 1 -- 172.21.15.5:0/3143487025 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5d24014030 con 0x7f5d3c102570 2024-09-12T02:58:08.540 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:58:08 smithi005 ceph-mon[31197]: from='client.15258 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:58:08.540 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:58:08 smithi005 ceph-mon[31197]: from='client.15264 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:58:08.569 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:08.569+0000 7f5d40fea640 1 -- 172.21.15.5:0/3143487025 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5d3c0fe1b0 con 0x7f5d3c102570 2024-09-12T02:58:08.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:08.570+0000 7f5d337fe640 1 -- 172.21.15.5:0/3143487025 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11966 (secure 0 0 0) 0x7f5d3c0fe1b0 con 0x7f5d3c102570 2024-09-12T02:58:08.570 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-12T02:58:08.571 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":26,"btime":"2024-09-12T02:58:03:323671+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/4135232521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":4135232521},{"type":"v1","addr":"172.21.15.5:6835","nonce":4135232521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/4290221940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":4290221940},{"type":"v1","addr":"172.21.15.5:6837","nonce":4290221940}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T02:56:16.595562+0000","modified":"2024-09-12T02:58:02.463400+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24502,"mds_1":24487,"mds_2":24481,"mds_3":24479,"mds_4":24458},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/26447341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":26447341},{"type":"v1","addr":"172.21.15.5:6839","nonce":26447341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2393443295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2393443295},{"type":"v1","addr":"172.21.15.5:6841","nonce":2393443295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6833/3686970137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3686970137},{"type":"v1","addr":"172.21.15.146:6833","nonce":3686970137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6833/568512209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":568512209},{"type":"v1","addr":"172.21.15.148:6833","nonce":568512209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6835/2540740420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2540740420},{"type":"v1","addr":"172.21.15.146:6835","nonce":2540740420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.146:6837/1673370937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":1673370937},{"type":"v1","addr":"172.21.15.146:6837","nonce":1673370937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":3,"incarnation":22,"state":"up:active","state_seq":6,"addr":"172.21.15.146:6839/3036767228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":3036767228},{"type":"v1","addr":"172.21.15.146:6839","nonce":3036767228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":2,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.148:6835/4195086427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":4195086427},{"type":"v1","addr":"172.21.15.148:6835","nonce":4195086427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.148:6837/2606873922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2606873922},{"type":"v1","addr":"172.21.15.148:6837","nonce":2606873922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6839/2317600000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":2317600000},{"type":"v1","addr":"172.21.15.148:6839","nonce":2317600000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24502,"qdb_cluster":[24502,24487,24481,24479,24458]},"id":1}]} 2024-09-12T02:58:08.571 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 26 2024-09-12T02:58:08.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:08.572+0000 7f5d40fea640 1 -- 172.21.15.5:0/3143487025 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f5d280a2c10 msgr2=0x7f5d280a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:08.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:08.572+0000 7f5d40fea640 1 --2- 172.21.15.5:0/3143487025 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f5d280a2c10 0x7f5d280a50d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c00bbe0 tx=0x7f5d2c00b390 comp rx=0 tx=0).stop 2024-09-12T02:58:08.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:08.572+0000 7f5d40fea640 1 -- 172.21.15.5:0/3143487025 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d3c102570 msgr2=0x7f5d3c0fd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:08.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:08.572+0000 7f5d40fea640 1 --2- 172.21.15.5:0/3143487025 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d3c102570 0x7f5d3c0fd1c0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f5d2400c890 tx=0x7f5d2400cd50 comp rx=0 tx=0).stop 2024-09-12T02:58:08.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:08.572+0000 7f5d40fea640 1 -- 172.21.15.5:0/3143487025 shutdown_connections 2024-09-12T02:58:08.572 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:08.572+0000 7f5d40fea640 1 --2- 172.21.15.5:0/3143487025 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f5d280a2c10 0x7f5d280a50d0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:08.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:08.572+0000 7f5d40fea640 1 --2- 172.21.15.5:0/3143487025 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d3c102570 0x7f5d3c0fd1c0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:08.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:08.572+0000 7f5d40fea640 1 --2- 172.21.15.5:0/3143487025 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5d3c102ef0 0x7f5d3c0fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:08.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:08.572+0000 7f5d40fea640 1 --2- 172.21.15.5:0/3143487025 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5d3c100e20 0x7f5d3c0fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:08.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:08.572+0000 7f5d40fea640 1 -- 172.21.15.5:0/3143487025 >> 172.21.15.5:0/3143487025 conn(0x7f5d3c0fc7d0 msgr2=0x7f5d3c06a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:08.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:08.572+0000 7f5d40fea640 1 -- 172.21.15.5:0/3143487025 shutdown_connections 2024-09-12T02:58:08.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:08.572+0000 7f5d40fea640 1 -- 172.21.15.5:0/3143487025 wait complete. 2024-09-12T02:58:08.665 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:58:08 smithi148 ceph-mon[35090]: from='client.15258 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:58:08.665 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:58:08 smithi148 ceph-mon[35090]: from='client.15264 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:58:08.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:58:08 smithi146 ceph-mon[35011]: from='client.15258 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:58:08.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:58:08 smithi146 ceph-mon[35011]: from='client.15264 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:58:09.278 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T02:58:09.541 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:58:09.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:58:09 smithi005 ceph-mon[31197]: from='client.15270 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:58:09.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:58:09 smithi005 ceph-mon[31197]: pgmap v332: 97 pgs: 97 active+clean; 650 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-09-12T02:58:09.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:58:09 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/3143487025' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:58:09.607 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:58:09 smithi146 ceph-mon[35011]: from='client.15270 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:58:09.607 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:58:09 smithi146 ceph-mon[35011]: pgmap v332: 97 pgs: 97 active+clean; 650 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-09-12T02:58:09.608 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:58:09 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/3143487025' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:58:09.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:58:09 smithi148 ceph-mon[35090]: from='client.15270 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:58:09.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:58:09 smithi148 ceph-mon[35090]: pgmap v332: 97 pgs: 97 active+clean; 650 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-09-12T02:58:09.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:58:09 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/3143487025' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:58:09.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:09.952+0000 7fc9e5dd3640 1 -- 172.21.15.5:0/2281278671 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9e0074f40 msgr2=0x7fc9e010de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:09.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:09.952+0000 7fc9e5dd3640 1 --2- 172.21.15.5:0/2281278671 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9e0074f40 0x7fc9e010de80 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fc9d4009900 tx=0x7fc9d402a480 comp rx=0 tx=0).stop 2024-09-12T02:58:09.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:09.952+0000 7fc9e5dd3640 1 -- 172.21.15.5:0/2281278671 shutdown_connections 2024-09-12T02:58:09.978 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:09.952+0000 7fc9e5dd3640 1 --2- 172.21.15.5:0/2281278671 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9e0074f40 0x7fc9e010de80 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:09.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:09.952+0000 7fc9e5dd3640 1 --2- 172.21.15.5:0/2281278671 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc9e010e3c0 0x7fc9e0110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:09.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:09.952+0000 7fc9e5dd3640 1 --2- 172.21.15.5:0/2281278671 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc9e0074620 0x7fc9e0074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:09.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:09.952+0000 7fc9e5dd3640 1 -- 172.21.15.5:0/2281278671 >> 172.21.15.5:0/2281278671 conn(0x7fc9e00fd8c0 msgr2=0x7fc9e00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:09.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:09.952+0000 7fc9e5dd3640 1 -- 172.21.15.5:0/2281278671 shutdown_connections 2024-09-12T02:58:09.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:09.952+0000 7fc9e5dd3640 1 -- 172.21.15.5:0/2281278671 wait complete. 2024-09-12T02:58:09.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:09.953+0000 7fc9e5dd3640 1 Processor -- start 2024-09-12T02:58:09.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:09.953+0000 7fc9e5dd3640 1 -- start start 2024-09-12T02:58:09.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:09.953+0000 7fc9e5dd3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9e0074620 0x7fc9e019a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:09.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:09.953+0000 7fc9e5dd3640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc9e0074f40 0x7fc9e019ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:09.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:09.953+0000 7fc9e5dd3640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc9e010e3c0 0x7fc9e019efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:09.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:09.953+0000 7fc9e5dd3640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc9e01141c0 con 0x7fc9e0074620 2024-09-12T02:58:09.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:09.953+0000 7fc9e5dd3640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fc9e0114040 con 0x7fc9e0074f40 2024-09-12T02:58:09.979 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:09.953+0000 7fc9e5dd3640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc9e0114340 con 0x7fc9e010e3c0 2024-09-12T02:58:09.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:09.954+0000 7fc9df7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9e0074620 0x7fc9e019a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:09.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:09.954+0000 7fc9df7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9e0074620 0x7fc9e019a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55494/0 (socket says 172.21.15.5:55494) 2024-09-12T02:58:09.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:09.954+0000 7fc9df7fe640 1 -- 172.21.15.5:0/3568170752 learned_addr learned my addr 172.21.15.5:0/3568170752 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:58:09.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:09.954+0000 7fc9deffd640 1 --2- 172.21.15.5:0/3568170752 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc9e0074f40 0x7fc9e019ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:09.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:09.954+0000 7fc9dffff640 1 --2- 172.21.15.5:0/3568170752 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc9e010e3c0 0x7fc9e019efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:09.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:09.954+0000 7fc9df7fe640 1 -- 172.21.15.5:0/3568170752 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc9e010e3c0 msgr2=0x7fc9e019efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:09.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:09.954+0000 7fc9df7fe640 1 --2- 172.21.15.5:0/3568170752 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc9e010e3c0 0x7fc9e019efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:09.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:09.954+0000 7fc9df7fe640 1 -- 172.21.15.5:0/3568170752 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc9e0074f40 msgr2=0x7fc9e019ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:09.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:09.954+0000 7fc9df7fe640 1 --2- 172.21.15.5:0/3568170752 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc9e0074f40 0x7fc9e019ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:09.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:09.954+0000 7fc9df7fe640 1 -- 172.21.15.5:0/3568170752 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9e019f6a0 con 0x7fc9e0074620 2024-09-12T02:58:09.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:09.954+0000 7fc9df7fe640 1 --2- 172.21.15.5:0/3568170752 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9e0074620 0x7fc9e019a6f0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fc9d000b4a0 tx=0x7fc9d000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:09.980 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:09.954+0000 7fc9dcff9640 1 -- 172.21.15.5:0/3568170752 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9d0014070 con 0x7fc9e0074620 2024-09-12T02:58:09.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:09.954+0000 7fc9dcff9640 1 -- 172.21.15.5:0/3568170752 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9d0010af0 con 0x7fc9e0074620 2024-09-12T02:58:09.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:09.954+0000 7fc9dcff9640 1 -- 172.21.15.5:0/3568170752 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9d0017d10 con 0x7fc9e0074620 2024-09-12T02:58:09.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:09.954+0000 7fc9e5dd3640 1 -- 172.21.15.5:0/3568170752 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9e019f990 con 0x7fc9e0074620 2024-09-12T02:58:09.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:09.955+0000 7fc9e5dd3640 1 -- 172.21.15.5:0/3568170752 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9e01a7270 con 0x7fc9e0074620 2024-09-12T02:58:09.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:09.955+0000 7fc9e5dd3640 1 -- 172.21.15.5:0/3568170752 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9e0074a00 con 0x7fc9e0074620 2024-09-12T02:58:09.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:09.957+0000 7fc9dcff9640 1 -- 172.21.15.5:0/3568170752 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7fc9d0017070 con 0x7fc9e0074620 2024-09-12T02:58:09.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:09.957+0000 7fc9dcff9640 1 --2- 172.21.15.5:0/3568170752 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fc9c80a2c10 0x7fc9c80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:09.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:09.958+0000 7fc9deffd640 1 --2- 172.21.15.5:0/3568170752 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fc9c80a2c10 0x7fc9c80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:09.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:09.958+0000 7fc9dcff9640 1 -- 172.21.15.5:0/3568170752 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fc9d00ce480 con 0x7fc9e0074620 2024-09-12T02:58:09.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:09.958+0000 7fc9deffd640 1 --2- 172.21.15.5:0/3568170752 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fc9c80a2c10 0x7fc9c80a50d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc9d4000c00 tx=0x7fc9d40058d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:09.981 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:09.962+0000 7fc9dcff9640 1 -- 172.21.15.5:0/3568170752 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc9d009b3a0 con 0x7fc9e0074620 2024-09-12T02:58:10.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:10.142+0000 7fc9e5dd3640 1 -- 172.21.15.5:0/3568170752 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc9e0109540 con 0x7fc9e0074620 2024-09-12T02:58:10.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:10.143+0000 7fc9dcff9640 1 -- 172.21.15.5:0/3568170752 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+22656 (secure 0 0 0) 0x7fc9d009aca0 con 0x7fc9e0074620 2024-09-12T02:58:10.144 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-12T02:58:10.144 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":83,"fsid":"618305f8-70b1-11ef-bcea-c7b262605968","created":"2024-09-12T02:48:42.352146+0000","modified":"2024-09-12T02:56:31.026079+0000","last_up_change":"2024-09-12T02:55:27.817161+0000","last_in_change":"2024-09-12T02:55:03.745281+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T02:51:47.970639+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T02:56:12.834271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T02:56:14.951437+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17da560a-6aa2-4f2c-a95f-67481dce4d3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":4248507573},{"type":"v1","addr":"172.21.15.5:6803","nonce":4248507573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":4248507573},{"type":"v1","addr":"172.21.15.5:6805","nonce":4248507573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":4248507573},{"type":"v1","addr":"172.21.15.5:6809","nonce":4248507573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":4248507573},{"type":"v1","addr":"172.21.15.5:6807","nonce":4248507573}]},"public_addr":"172.21.15.5:6803/4248507573","cluster_addr":"172.21.15.5:6805/4248507573","heartbeat_back_addr":"172.21.15.5:6809/4248507573","heartbeat_front_addr":"172.21.15.5:6807/4248507573","state":["exists","up"]},{"osd":1,"uuid":"a3237006-99af-4fc1-860c-d4917032647f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":1105990287},{"type":"v1","addr":"172.21.15.146:6801","nonce":1105990287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":1105990287},{"type":"v1","addr":"172.21.15.146:6803","nonce":1105990287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":1105990287},{"type":"v1","addr":"172.21.15.146:6807","nonce":1105990287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":1105990287},{"type":"v1","addr":"172.21.15.146:6805","nonce":1105990287}]},"public_addr":"172.21.15.146:6801/1105990287","cluster_addr":"172.21.15.146:6803/1105990287","heartbeat_back_addr":"172.21.15.146:6807/1105990287","heartbeat_front_addr":"172.21.15.146:6805/1105990287","state":["exists","up"]},{"osd":2,"uuid":"ea3ab84a-7739-4975-8244-31ddd5aad80d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3378174313},{"type":"v1","addr":"172.21.15.148:6801","nonce":3378174313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3378174313},{"type":"v1","addr":"172.21.15.148:6803","nonce":3378174313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3378174313},{"type":"v1","addr":"172.21.15.148:6807","nonce":3378174313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3378174313},{"type":"v1","addr":"172.21.15.148:6805","nonce":3378174313}]},"public_addr":"172.21.15.148:6801/3378174313","cluster_addr":"172.21.15.148:6803/3378174313","heartbeat_back_addr":"172.21.15.148:6807/3378174313","heartbeat_front_addr":"172.21.15.148:6805/3378174313","state":["exists","up"]},{"osd":3,"uuid":"4396a9ee-f3f3-410b-9fb1-124f1e33a1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":11741938},{"type":"v1","addr":"172.21.15.5:6811","nonce":11741938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":11741938},{"type":"v1","addr":"172.21.15.5:6813","nonce":11741938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":11741938},{"type":"v1","addr":"172.21.15.5:6817","nonce":11741938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":11741938},{"type":"v1","addr":"172.21.15.5:6815","nonce":11741938}]},"public_addr":"172.21.15.5:6811/11741938","cluster_addr":"172.21.15.5:6813/11741938","heartbeat_back_addr":"172.21.15.5:6817/11741938","heartbeat_front_addr":"172.21.15.5:6815/11741938","state":["exists","up"]},{"osd":4,"uuid":"989d76e7-c742-49d0-994a-c58f2771b1d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":3274923851},{"type":"v1","addr":"172.21.15.146:6809","nonce":3274923851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":3274923851},{"type":"v1","addr":"172.21.15.146:6811","nonce":3274923851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3274923851},{"type":"v1","addr":"172.21.15.146:6815","nonce":3274923851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":3274923851},{"type":"v1","addr":"172.21.15.146:6813","nonce":3274923851}]},"public_addr":"172.21.15.146:6809/3274923851","cluster_addr":"172.21.15.146:6811/3274923851","heartbeat_back_addr":"172.21.15.146:6815/3274923851","heartbeat_front_addr":"172.21.15.146:6813/3274923851","state":["exists","up"]},{"osd":5,"uuid":"64ada7d4-70a0-4b83-9af8-032be544608d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":2771670476},{"type":"v1","addr":"172.21.15.148:6809","nonce":2771670476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":2771670476},{"type":"v1","addr":"172.21.15.148:6811","nonce":2771670476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":2771670476},{"type":"v1","addr":"172.21.15.148:6815","nonce":2771670476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":2771670476},{"type":"v1","addr":"172.21.15.148:6813","nonce":2771670476}]},"public_addr":"172.21.15.148:6809/2771670476","cluster_addr":"172.21.15.148:6811/2771670476","heartbeat_back_addr":"172.21.15.148:6815/2771670476","heartbeat_front_addr":"172.21.15.148:6813/2771670476","state":["exists","up"]},{"osd":6,"uuid":"a70a88fd-2e4b-480c-bb98-62636cf70cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3176073169},{"type":"v1","addr":"172.21.15.5:6819","nonce":3176073169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":3176073169},{"type":"v1","addr":"172.21.15.5:6821","nonce":3176073169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":3176073169},{"type":"v1","addr":"172.21.15.5:6825","nonce":3176073169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3176073169},{"type":"v1","addr":"172.21.15.5:6823","nonce":3176073169}]},"public_addr":"172.21.15.5:6819/3176073169","cluster_addr":"172.21.15.5:6821/3176073169","heartbeat_back_addr":"172.21.15.5:6825/3176073169","heartbeat_front_addr":"172.21.15.5:6823/3176073169","state":["exists","up"]},{"osd":7,"uuid":"49451e8d-c741-4ad5-b8f5-89bad8d4d129","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2209508838},{"type":"v1","addr":"172.21.15.146:6817","nonce":2209508838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2209508838},{"type":"v1","addr":"172.21.15.146:6819","nonce":2209508838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2209508838},{"type":"v1","addr":"172.21.15.146:6823","nonce":2209508838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2209508838},{"type":"v1","addr":"172.21.15.146:6821","nonce":2209508838}]},"public_addr":"172.21.15.146:6817/2209508838","cluster_addr":"172.21.15.146:6819/2209508838","heartbeat_back_addr":"172.21.15.146:6823/2209508838","heartbeat_front_addr":"172.21.15.146:6821/2209508838","state":["exists","up"]},{"osd":8,"uuid":"57bfc121-a75e-4a7f-b1dc-64074b178b03","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":651045756},{"type":"v1","addr":"172.21.15.148:6817","nonce":651045756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":651045756},{"type":"v1","addr":"172.21.15.148:6819","nonce":651045756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":651045756},{"type":"v1","addr":"172.21.15.148:6823","nonce":651045756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":651045756},{"type":"v1","addr":"172.21.15.148:6821","nonce":651045756}]},"public_addr":"172.21.15.148:6817/651045756","cluster_addr":"172.21.15.148:6819/651045756","heartbeat_back_addr":"172.21.15.148:6823/651045756","heartbeat_front_addr":"172.21.15.148:6821/651045756","state":["exists","up"]},{"osd":9,"uuid":"d9726168-7494-4bf8-b433-350e85bfd461","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1961857360},{"type":"v1","addr":"172.21.15.5:6827","nonce":1961857360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1961857360},{"type":"v1","addr":"172.21.15.5:6829","nonce":1961857360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":1961857360},{"type":"v1","addr":"172.21.15.5:6833","nonce":1961857360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1961857360},{"type":"v1","addr":"172.21.15.5:6831","nonce":1961857360}]},"public_addr":"172.21.15.5:6827/1961857360","cluster_addr":"172.21.15.5:6829/1961857360","heartbeat_back_addr":"172.21.15.5:6833/1961857360","heartbeat_front_addr":"172.21.15.5:6831/1961857360","state":["exists","up"]},{"osd":10,"uuid":"519ec60f-3606-48ac-9043-6f59a798766d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":2041733103},{"type":"v1","addr":"172.21.15.146:6825","nonce":2041733103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":2041733103},{"type":"v1","addr":"172.21.15.146:6827","nonce":2041733103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":2041733103},{"type":"v1","addr":"172.21.15.146:6831","nonce":2041733103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":2041733103},{"type":"v1","addr":"172.21.15.146:6829","nonce":2041733103}]},"public_addr":"172.21.15.146:6825/2041733103","cluster_addr":"172.21.15.146:6827/2041733103","heartbeat_back_addr":"172.21.15.146:6831/2041733103","heartbeat_front_addr":"172.21.15.146:6829/2041733103","state":["exists","up"]},{"osd":11,"uuid":"0f92d0cb-ba54-4bc3-a6d0-3c54d910556a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":839290773},{"type":"v1","addr":"172.21.15.148:6825","nonce":839290773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":839290773},{"type":"v1","addr":"172.21.15.148:6827","nonce":839290773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":839290773},{"type":"v1","addr":"172.21.15.148:6831","nonce":839290773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":839290773},{"type":"v1","addr":"172.21.15.148:6829","nonce":839290773}]},"public_addr":"172.21.15.148:6825/839290773","cluster_addr":"172.21.15.148:6827/839290773","heartbeat_back_addr":"172.21.15.148:6831/839290773","heartbeat_front_addr":"172.21.15.148:6829/839290773","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:50:54.101755+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:51:19.037385+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:51:44.142837+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:52:09.150887+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:52:33.705079+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:52:58.845233+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:53:23.068339+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:53:47.520398+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:54:11.965370+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:54:36.776075+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:55:01.292236+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:55:25.346589+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:6800/3125821798":"2024-09-13T02:49:29.462216+0000","172.21.15.5:0/3479911451":"2024-09-13T02:49:29.462216+0000","172.21.15.5:0/1547753207":"2024-09-13T02:49:03.300872+0000","172.21.15.5:0/1905762627":"2024-09-13T02:49:03.300872+0000","172.21.15.5:0/4142340280":"2024-09-13T02:49:29.462216+0000","172.21.15.5:6801/3125821798":"2024-09-13T02:49:29.462216+0000","172.21.15.5:6801/2595781680":"2024-09-13T02:49:03.300872+0000","172.21.15.5:0/2273691311":"2024-09-13T02:49:29.462216+0000","172.21.15.5:6800/2595781680":"2024-09-13T02:49:03.300872+0000","172.21.15.5:0/1196201217":"2024-09-13T02:49:03.300872+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T02:58:10.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:10.146+0000 7fc9e5dd3640 1 -- 172.21.15.5:0/3568170752 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fc9c80a2c10 msgr2=0x7fc9c80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:10.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:10.146+0000 7fc9e5dd3640 1 --2- 172.21.15.5:0/3568170752 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fc9c80a2c10 0x7fc9c80a50d0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc9d4000c00 tx=0x7fc9d40058d0 comp rx=0 tx=0).stop 2024-09-12T02:58:10.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:10.146+0000 7fc9e5dd3640 1 -- 172.21.15.5:0/3568170752 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9e0074620 msgr2=0x7fc9e019a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:10.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:10.146+0000 7fc9e5dd3640 1 --2- 172.21.15.5:0/3568170752 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9e0074620 0x7fc9e019a6f0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fc9d000b4a0 tx=0x7fc9d000b960 comp rx=0 tx=0).stop 2024-09-12T02:58:10.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:10.146+0000 7fc9e5dd3640 1 -- 172.21.15.5:0/3568170752 shutdown_connections 2024-09-12T02:58:10.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:10.146+0000 7fc9e5dd3640 1 --2- 172.21.15.5:0/3568170752 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc9e010e3c0 0x7fc9e019efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:10.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:10.146+0000 7fc9e5dd3640 1 --2- 172.21.15.5:0/3568170752 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc9e0074f40 0x7fc9e019ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:10.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:10.146+0000 7fc9e5dd3640 1 --2- 172.21.15.5:0/3568170752 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fc9c80a2c10 0x7fc9c80a50d0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:10.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:10.146+0000 7fc9e5dd3640 1 --2- 172.21.15.5:0/3568170752 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9e0074620 0x7fc9e019a6f0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:10.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:10.146+0000 7fc9e5dd3640 1 -- 172.21.15.5:0/3568170752 >> 172.21.15.5:0/3568170752 conn(0x7fc9e00fd8c0 msgr2=0x7fc9e00fefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:10.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:10.146+0000 7fc9e5dd3640 1 -- 172.21.15.5:0/3568170752 shutdown_connections 2024-09-12T02:58:10.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:10.146+0000 7fc9e5dd3640 1 -- 172.21.15.5:0/3568170752 wait complete. 2024-09-12T02:58:10.456 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:58:10 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/3568170752' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:58:10.665 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:58:10 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/3568170752' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:58:10.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:58:10 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/3568170752' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:58:10.837 DEBUG:tasks.fwd_scrub:Yielding 2024-09-12T02:58:10.837 INFO:teuthology.run_tasks:Running task quiescer... 2024-09-12T02:58:10.852 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-12T02:58:10.852 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-12T02:58:10.852 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-09-12T02:58:10.852 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T02:58:10.853 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T02:58:11.124 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:58:11.129 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:58:11.553 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:58:11 smithi005 ceph-mon[31197]: pgmap v333: 97 pgs: 97 active+clean; 653 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 2.2 KiB/s wr, 13 op/s 2024-09-12T02:58:11.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:58:11 smithi148 ceph-mon[35090]: pgmap v333: 97 pgs: 97 active+clean; 653 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 2.2 KiB/s wr, 13 op/s 2024-09-12T02:58:11.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:58:11 smithi146 ceph-mon[35011]: pgmap v333: 97 pgs: 97 active+clean; 653 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 2.2 KiB/s wr, 13 op/s 2024-09-12T02:58:11.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.746+0000 7f15919d6640 1 -- 172.21.15.5:0/2264416117 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f158c101380 msgr2=0x7f158c10dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:11.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.746+0000 7f15919d6640 1 --2- 172.21.15.5:0/2264416117 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f158c101380 0x7f158c10dfb0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f1580009900 tx=0x7f158002a410 comp rx=0 tx=0).stop 2024-09-12T02:58:11.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.747+0000 7f15919d6640 1 -- 172.21.15.5:0/2264416117 shutdown_connections 2024-09-12T02:58:11.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.747+0000 7f15919d6640 1 --2- 172.21.15.5:0/2264416117 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f158c101380 0x7f158c10dfb0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:11.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.747+0000 7f15919d6640 1 --2- 172.21.15.5:0/2264416117 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f158c10e4f0 0x7f158c110900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:11.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.747+0000 7f15919d6640 1 --2- 172.21.15.5:0/2264416117 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f158c1009d0 0x7f158c100db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:11.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.747+0000 7f15919d6640 1 -- 172.21.15.5:0/2264416117 >> 172.21.15.5:0/2264416117 conn(0x7f158c0fc7d0 msgr2=0x7f158c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:11.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.747+0000 7f15919d6640 1 -- 172.21.15.5:0/2264416117 shutdown_connections 2024-09-12T02:58:11.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.747+0000 7f15919d6640 1 -- 172.21.15.5:0/2264416117 wait complete. 2024-09-12T02:58:11.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.747+0000 7f15919d6640 1 Processor -- start 2024-09-12T02:58:11.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.747+0000 7f15919d6640 1 -- start start 2024-09-12T02:58:11.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.748+0000 7f15919d6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f158c1009d0 0x7f158c19a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:11.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.748+0000 7f15919d6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f158c101380 0x7f158c19ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:11.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.748+0000 7f15919d6640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f158c10e4f0 0x7f158c19f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:11.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.748+0000 7f15919d6640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f158c114260 con 0x7f158c101380 2024-09-12T02:58:11.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.748+0000 7f158affd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f158c1009d0 0x7f158c19a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:11.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.748+0000 7f158a7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f158c101380 0x7f158c19ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:11.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.748+0000 7f15919d6640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f158c1140e0 con 0x7f158c10e4f0 2024-09-12T02:58:11.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.748+0000 7f15919d6640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f158c1143e0 con 0x7f158c1009d0 2024-09-12T02:58:11.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.748+0000 7f158affd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f158c1009d0 0x7f158c19a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.5:46278/0 (socket says 172.21.15.5:46278) 2024-09-12T02:58:11.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.748+0000 7f158affd640 1 -- 172.21.15.5:0/2915751330 learned_addr learned my addr 172.21.15.5:0/2915751330 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:58:11.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.748+0000 7f158b7fe640 1 --2- 172.21.15.5:0/2915751330 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f158c10e4f0 0x7f158c19f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:11.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.748+0000 7f158affd640 1 -- 172.21.15.5:0/2915751330 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f158c10e4f0 msgr2=0x7f158c19f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:11.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.748+0000 7f158affd640 1 --2- 172.21.15.5:0/2915751330 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f158c10e4f0 0x7f158c19f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:11.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.748+0000 7f158affd640 1 -- 172.21.15.5:0/2915751330 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f158c101380 msgr2=0x7f158c19ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:11.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.748+0000 7f158affd640 1 --2- 172.21.15.5:0/2915751330 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f158c101380 0x7f158c19ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:11.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.748+0000 7f158affd640 1 -- 172.21.15.5:0/2915751330 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f158c19f700 con 0x7f158c1009d0 2024-09-12T02:58:11.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.748+0000 7f158a7fc640 1 --2- 172.21.15.5:0/2915751330 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f158c101380 0x7f158c19ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:58:11.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.749+0000 7f158affd640 1 --2- 172.21.15.5:0/2915751330 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f158c1009d0 0x7f158c19a750 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f157c00e9f0 tx=0x7f157c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:11.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.749+0000 7f15909d4640 1 -- 172.21.15.5:0/2915751330 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f157c00cd80 con 0x7f158c1009d0 2024-09-12T02:58:11.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.749+0000 7f15919d6640 1 -- 172.21.15.5:0/2915751330 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f158c19f9f0 con 0x7f158c1009d0 2024-09-12T02:58:11.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.749+0000 7f15909d4640 1 -- 172.21.15.5:0/2915751330 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f157c010910 con 0x7f158c1009d0 2024-09-12T02:58:11.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.749+0000 7f15909d4640 1 -- 172.21.15.5:0/2915751330 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f157c0188f0 con 0x7f158c1009d0 2024-09-12T02:58:11.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.749+0000 7f15919d6640 1 -- 172.21.15.5:0/2915751330 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f158c1a72d0 con 0x7f158c1009d0 2024-09-12T02:58:11.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.750+0000 7f15919d6640 1 -- 172.21.15.5:0/2915751330 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f158c100db0 con 0x7f158c1009d0 2024-09-12T02:58:11.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.751+0000 7f15909d4640 1 -- 172.21.15.5:0/2915751330 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7f157c010ac0 con 0x7f158c1009d0 2024-09-12T02:58:11.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.752+0000 7f15909d4640 1 --2- 172.21.15.5:0/2915751330 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f15740a2c10 0x7f15740a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:11.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.752+0000 7f158a7fc640 1 --2- 172.21.15.5:0/2915751330 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f15740a2c10 0x7f15740a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:11.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.752+0000 7f15909d4640 1 -- 172.21.15.5:0/2915751330 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f157c0cf950 con 0x7f158c1009d0 2024-09-12T02:58:11.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.753+0000 7f158a7fc640 1 --2- 172.21.15.5:0/2915751330 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f15740a2c10 0x7f15740a50d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f1580009230 tx=0x7f15800058d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:11.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.755+0000 7f15909d4640 1 -- 172.21.15.5:0/2915751330 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f157c09c110 con 0x7f158c1009d0 2024-09-12T02:58:11.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.932+0000 7f15919d6640 1 -- 172.21.15.5:0/2915751330 --> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f158c1096f0 con 0x7f15740a2c10 2024-09-12T02:58:11.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.938+0000 7f15909d4640 1 -- 172.21.15.5:0/2915751330 <== mgr.14148 v2:172.21.15.5:6800/3780631642 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258894 (secure 0 0 0) 0x7f15800316b0 con 0x7f15740a2c10 2024-09-12T02:58:11.941 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-12T02:58:11.942 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-09-12T02:58:11.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.941+0000 7f15919d6640 1 -- 172.21.15.5:0/2915751330 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f15740a2c10 msgr2=0x7f15740a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:11.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.941+0000 7f15919d6640 1 --2- 172.21.15.5:0/2915751330 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f15740a2c10 0x7f15740a50d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f1580009230 tx=0x7f15800058d0 comp rx=0 tx=0).stop 2024-09-12T02:58:11.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.941+0000 7f15919d6640 1 -- 172.21.15.5:0/2915751330 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f158c1009d0 msgr2=0x7f158c19a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:11.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.941+0000 7f15919d6640 1 --2- 172.21.15.5:0/2915751330 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f158c1009d0 0x7f158c19a750 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f157c00e9f0 tx=0x7f157c00eeb0 comp rx=0 tx=0).stop 2024-09-12T02:58:11.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.941+0000 7f15919d6640 1 -- 172.21.15.5:0/2915751330 shutdown_connections 2024-09-12T02:58:11.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.941+0000 7f15919d6640 1 --2- 172.21.15.5:0/2915751330 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f15740a2c10 0x7f15740a50d0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:11.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.941+0000 7f15919d6640 1 --2- 172.21.15.5:0/2915751330 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f158c101380 0x7f158c19ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:11.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.941+0000 7f15919d6640 1 --2- 172.21.15.5:0/2915751330 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f158c10e4f0 0x7f158c19f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:11.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.941+0000 7f15919d6640 1 --2- 172.21.15.5:0/2915751330 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f158c1009d0 0x7f158c19a750 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:11.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.941+0000 7f15919d6640 1 -- 172.21.15.5:0/2915751330 >> 172.21.15.5:0/2915751330 conn(0x7f158c0fc7d0 msgr2=0x7f158c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:11.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.942+0000 7f15919d6640 1 -- 172.21.15.5:0/2915751330 shutdown_connections 2024-09-12T02:58:11.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.942+0000 7f15919d6640 1 -- 172.21.15.5:0/2915751330 wait complete. 2024-09-12T02:58:11.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.947+0000 7f2b9ec26640 1 -- 172.21.15.5:0/3975415706 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b98102790 msgr2=0x7f2b981067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:11.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.947+0000 7f2b9ec26640 1 --2- 172.21.15.5:0/3975415706 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b98102790 0x7f2b981067e0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f2b84009900 tx=0x7f2b8402a410 comp rx=0 tx=0).stop 2024-09-12T02:58:11.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.947+0000 7f2b9ec26640 1 -- 172.21.15.5:0/3975415706 shutdown_connections 2024-09-12T02:58:11.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.947+0000 7f2b9ec26640 1 --2- 172.21.15.5:0/3975415706 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b98102790 0x7f2b981067e0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:11.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.947+0000 7f2b9ec26640 1 --2- 172.21.15.5:0/3975415706 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2b98106f10 0x7f2b9810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:11.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.947+0000 7f2b9ec26640 1 --2- 172.21.15.5:0/3975415706 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2b98101de0 0x7f2b981021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:11.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.947+0000 7f2b9ec26640 1 -- 172.21.15.5:0/3975415706 >> 172.21.15.5:0/3975415706 conn(0x7f2b980fd880 msgr2=0x7f2b980ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:11.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.947+0000 7f2b9ec26640 1 -- 172.21.15.5:0/3975415706 shutdown_connections 2024-09-12T02:58:11.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.947+0000 7f2b9ec26640 1 -- 172.21.15.5:0/3975415706 wait complete. 2024-09-12T02:58:11.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.947+0000 7f2b9ec26640 1 Processor -- start 2024-09-12T02:58:11.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.947+0000 7f2b9ec26640 1 -- start start 2024-09-12T02:58:11.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.948+0000 7f2b9ec26640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2b98101de0 0x7f2b981983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:11.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.948+0000 7f2b9ec26640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2b98102790 0x7f2b98198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:11.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.948+0000 7f2b9ec26640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b98106f10 0x7f2b9819cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:11.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.948+0000 7f2b9ec26640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2b98111f70 con 0x7f2b98106f10 2024-09-12T02:58:11.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.948+0000 7f2b9ec26640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f2b98111df0 con 0x7f2b98102790 2024-09-12T02:58:11.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.948+0000 7f2b9ec26640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f2b981120f0 con 0x7f2b98101de0 2024-09-12T02:58:11.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.948+0000 7f2b9d1c3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b98106f10 0x7f2b9819cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:11.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.948+0000 7f2b9d1c3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b98106f10 0x7f2b9819cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55520/0 (socket says 172.21.15.5:55520) 2024-09-12T02:58:11.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.948+0000 7f2b9d1c3640 1 -- 172.21.15.5:0/695917037 learned_addr learned my addr 172.21.15.5:0/695917037 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:58:11.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.948+0000 7f2b9c9c2640 1 --2- 172.21.15.5:0/695917037 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2b98101de0 0x7f2b981983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:11.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.948+0000 7f2b97fff640 1 --2- 172.21.15.5:0/695917037 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2b98102790 0x7f2b98198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:11.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.948+0000 7f2b9d1c3640 1 -- 172.21.15.5:0/695917037 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2b98101de0 msgr2=0x7f2b981983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:11.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.948+0000 7f2b9d1c3640 1 --2- 172.21.15.5:0/695917037 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2b98101de0 0x7f2b981983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:11.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.948+0000 7f2b9d1c3640 1 -- 172.21.15.5:0/695917037 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2b98102790 msgr2=0x7f2b98198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:11.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.948+0000 7f2b9d1c3640 1 --2- 172.21.15.5:0/695917037 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2b98102790 0x7f2b98198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:11.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.948+0000 7f2b9d1c3640 1 -- 172.21.15.5:0/695917037 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b9819d380 con 0x7f2b98106f10 2024-09-12T02:58:11.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.948+0000 7f2b9d1c3640 1 --2- 172.21.15.5:0/695917037 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b98106f10 0x7f2b9819cca0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f2b9000eaa0 tx=0x7f2b9000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:11.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.949+0000 7f2b95ffb640 1 -- 172.21.15.5:0/695917037 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b9000cdb0 con 0x7f2b98106f10 2024-09-12T02:58:11.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.949+0000 7f2b9ec26640 1 -- 172.21.15.5:0/695917037 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b9819d670 con 0x7f2b98106f10 2024-09-12T02:58:11.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.949+0000 7f2b95ffb640 1 -- 172.21.15.5:0/695917037 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b90005020 con 0x7f2b98106f10 2024-09-12T02:58:11.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.949+0000 7f2b95ffb640 1 -- 172.21.15.5:0/695917037 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b900116b0 con 0x7f2b98106f10 2024-09-12T02:58:11.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.949+0000 7f2b9ec26640 1 -- 172.21.15.5:0/695917037 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b981a4f50 con 0x7f2b98106f10 2024-09-12T02:58:11.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.951+0000 7f2b95ffb640 1 -- 172.21.15.5:0/695917037 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7f2b90005560 con 0x7f2b98106f10 2024-09-12T02:58:11.968 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.952+0000 7f2ba0fff740 1 Processor -- start 2024-09-12T02:58:11.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.952+0000 7f2ba0fff740 1 -- start start 2024-09-12T02:58:11.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.952+0000 7f2ba0fff740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562c8e7aa20 0x5562c8e7ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:11.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.952+0000 7f2ba0fff740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5562c8e7b340 0x5562c8e89c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:11.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.952+0000 7f2ba0fff740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5562c8e8a150 0x5562c8e8c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:11.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.952+0000 7f2ba0fff740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5562c8de0280 con 0x5562c8e7aa20 2024-09-12T02:58:11.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.952+0000 7f2ba0fff740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5562c8d87b10 con 0x5562c8e7b340 2024-09-12T02:58:11.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.952+0000 7f2ba0fff740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5562c8ddc240 con 0x5562c8e8a150 2024-09-12T02:58:11.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.952+0000 7f2b95ffb640 1 --2- 172.21.15.5:0/695917037 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f2b880a2c10 0x7f2b880a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:11.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.952+0000 7f2b97fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562c8e7aa20 0x5562c8e7ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:11.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.952+0000 7f2b97fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562c8e7aa20 0x5562c8e7ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55528/0 (socket says 172.21.15.5:55528) 2024-09-12T02:58:11.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.952+0000 7f2b97fff640 1 -- 172.21.15.5:0/3309988398 learned_addr learned my addr 172.21.15.5:0/3309988398 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:58:11.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.952+0000 7f2b9c9c2640 1 --2- 172.21.15.5:0/3309988398 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5562c8e7b340 0x5562c8e89c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:11.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.953+0000 7f2b9d1c3640 1 --2- 172.21.15.5:0/695917037 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f2b880a2c10 0x7f2b880a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:11.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.953+0000 7f2b97fff640 1 --2- 172.21.15.5:0/3309988398 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5562c8e8a150 0x5562c8e8c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:11.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.953+0000 7f2b97fff640 1 -- 172.21.15.5:0/3309988398 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5562c8e8a150 msgr2=0x5562c8e8c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:11.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.953+0000 7f2b97fff640 1 --2- 172.21.15.5:0/3309988398 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5562c8e8a150 0x5562c8e8c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:11.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.953+0000 7f2b97fff640 1 -- 172.21.15.5:0/3309988398 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5562c8e7b340 msgr2=0x5562c8e89c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:11.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.953+0000 7f2b97fff640 1 --2- 172.21.15.5:0/3309988398 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5562c8e7b340 0x5562c8e89c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:11.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.953+0000 7f2b97fff640 1 -- 172.21.15.5:0/3309988398 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562c8d941f0 con 0x5562c8e7aa20 2024-09-12T02:58:11.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.953+0000 7f2b97fff640 1 --2- 172.21.15.5:0/3309988398 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562c8e7aa20 0x5562c8e7ae00 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f2b84009900 tx=0x7f2b84000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11822002a5fedbd5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:11.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.953+0000 7f2b82ffd640 1 -- 172.21.15.5:0/3309988398 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b84044030 con 0x5562c8e7aa20 2024-09-12T02:58:11.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.953+0000 7f2b95ffb640 1 -- 172.21.15.5:0/695917037 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f2b900cf300 con 0x7f2b98106f10 2024-09-12T02:58:11.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.953+0000 7f2b9d1c3640 1 --2- 172.21.15.5:0/695917037 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f2b880a2c10 0x7f2b880a50d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2b90098490 tx=0x7f2b90097710 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:11.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.953+0000 7f2b82ffd640 1 -- 172.21.15.5:0/3309988398 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b84005020 con 0x5562c8e7aa20 2024-09-12T02:58:11.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.953+0000 7f2b82ffd640 1 -- 172.21.15.5:0/3309988398 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b84035040 con 0x5562c8e7aa20 2024-09-12T02:58:11.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.954+0000 7f2ba0fff740 1 -- 172.21.15.5:0/3309988398 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562c8e7aa20 msgr2=0x5562c8e7ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:11.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.954+0000 7f2ba0fff740 1 --2- 172.21.15.5:0/3309988398 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562c8e7aa20 0x5562c8e7ae00 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f2b84009900 tx=0x7f2b84000bc0 comp rx=0 tx=0).stop 2024-09-12T02:58:11.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.954+0000 7f2ba0fff740 1 -- 172.21.15.5:0/3309988398 shutdown_connections 2024-09-12T02:58:11.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.954+0000 7f2ba0fff740 1 --2- 172.21.15.5:0/3309988398 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5562c8e8a150 0x5562c8e8c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:11.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.954+0000 7f2ba0fff740 1 --2- 172.21.15.5:0/3309988398 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5562c8e7b340 0x5562c8e89c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:11.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.954+0000 7f2ba0fff740 1 --2- 172.21.15.5:0/3309988398 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562c8e7aa20 0x5562c8e7ae00 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:11.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.954+0000 7f2ba0fff740 1 -- 172.21.15.5:0/3309988398 >> 172.21.15.5:0/3309988398 conn(0x5562c8e770f0 msgr2=0x5562c8e774d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:11.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.954+0000 7f2ba0fff740 1 -- 172.21.15.5:0/3309988398 shutdown_connections 2024-09-12T02:58:11.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.954+0000 7f2ba0fff740 1 -- 172.21.15.5:0/3309988398 wait complete. 2024-09-12T02:58:11.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.954+0000 7f2ba0fff740 1 -- start start 2024-09-12T02:58:11.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.955+0000 7f2ba0fff740 1 Processor -- start 2024-09-12T02:58:11.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.955+0000 7f2ba0fff740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562c8e7aa20 0x5562c8e87090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:11.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.955+0000 7f2ba0fff740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5562c8e7b340 0x5562c8e875d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:11.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.955+0000 7f2ba0fff740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5562c8e8a150 0x5562c8e87b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:11.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.955+0000 7f2ba0fff740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5562c8e8ccc0 con 0x5562c8e7aa20 2024-09-12T02:58:11.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.955+0000 7f2ba0fff740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5562c8d9af10 con 0x5562c8e7b340 2024-09-12T02:58:11.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.955+0000 7f2ba0fff740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5562c8e8ce40 con 0x5562c8e8a150 2024-09-12T02:58:11.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.955+0000 7f2ba0fff740 10 client.0 _ll_register_callbacks cb 0x5562c8d63a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:58:11.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.955+0000 7f2b97fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562c8e7aa20 0x5562c8e87090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:11.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.955+0000 7f2b97fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562c8e7aa20 0x5562c8e87090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55540/0 (socket says 172.21.15.5:55540) 2024-09-12T02:58:11.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.955+0000 7f2b97fff640 1 -- 172.21.15.5:0/2880031755 learned_addr learned my addr 172.21.15.5:0/2880031755 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:58:11.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.955+0000 7f2b9c9c2640 1 --2- 172.21.15.5:0/2880031755 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5562c8e7b340 0x5562c8e875d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:11.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.955+0000 7f2b97fff640 1 -- 172.21.15.5:0/2880031755 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5562c8e8a150 msgr2=0x5562c8e87b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:11.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.955+0000 7f2b97fff640 1 --2- 172.21.15.5:0/2880031755 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5562c8e8a150 0x5562c8e87b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:11.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.955+0000 7f2b97fff640 1 -- 172.21.15.5:0/2880031755 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5562c8e7b340 msgr2=0x5562c8e875d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:11.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.955+0000 7f2b97fff640 1 --2- 172.21.15.5:0/2880031755 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5562c8e7b340 0x5562c8e875d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:11.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.955+0000 7f2b97fff640 1 -- 172.21.15.5:0/2880031755 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562c8b516a0 con 0x5562c8e7aa20 2024-09-12T02:58:11.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.956+0000 7f2b97fff640 1 --2- 172.21.15.5:0/2880031755 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562c8e7aa20 0x5562c8e87090 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f2b84009340 tx=0x7f2b8402ab80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94d2f75a799133dc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:11.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.956+0000 7f2b77fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T02:58:11.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.956+0000 7f2b77fff640 1 -- 172.21.15.5:0/2880031755 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b8404d070 con 0x5562c8e7aa20 2024-09-12T02:58:11.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.956+0000 7f2b77fff640 1 -- 172.21.15.5:0/2880031755 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f2b840046c0 con 0x5562c8e7aa20 2024-09-12T02:58:11.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.956+0000 7f2ba0fff740 1 -- 172.21.15.5:0/2880031755 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b84005020 con 0x5562c8e7aa20 2024-09-12T02:58:11.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.956+0000 7f2b77fff640 1 -- 172.21.15.5:0/2880031755 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2b84059050 con 0x5562c8e7aa20 2024-09-12T02:58:11.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.956+0000 7f2ba0fff740 10 client.15300 fetch_fsmap learned FSMap version 26 2024-09-12T02:58:11.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.956+0000 7f2ba0fff740 1 -- 172.21.15.5:0/2880031755 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5562c8e98960 con 0x5562c8e7aa20 2024-09-12T02:58:11.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.956+0000 7f2b77fff640 1 -- 172.21.15.5:0/2880031755 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b84005020 con 0x5562c8e7aa20 2024-09-12T02:58:11.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.957+0000 7f2b77fff640 1 -- 172.21.15.5:0/2880031755 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f2b84049b40 con 0x5562c8e7aa20 2024-09-12T02:58:11.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.957+0000 7f2ba0fff740 10 client.15300 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T02:58:11.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.957+0000 7f2ba0fff740 10 client.15300 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T02:58:11.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.957+0000 7f2ba0fff740 20 client.15300 populate_metadata read hostname 'smithi005' 2024-09-12T02:58:11.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.957+0000 7f2ba0fff740 1 --2- 172.21.15.5:0/2880031755 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x5562c8e99cb0 0x5562c8e9c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:11.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.957+0000 7f2ba0fff740 4 client.15300 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:58:11.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.957+0000 7f2ba0fff740 1 -- 172.21.15.5:0/2880031755 --> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5562c8ded7f0 con 0x5562c8e99cb0 2024-09-12T02:58:11.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.957+0000 7f2b9c9c2640 1 --2- 172.21.15.5:0/2880031755 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x5562c8e99cb0 0x5562c8e9c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:11.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.958+0000 7f2b9c9c2640 1 --2- 172.21.15.5:0/2880031755 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x5562c8e99cb0 0x5562c8e9c0a0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80531b3a9ef71d95 server_cookie=f354b619afaf7b9a in_seq=0 out_seq=0 2024-09-12T02:58:11.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.958+0000 7f2b77fff640 10 client.15300 ms_handle_connect on v2:172.21.15.148:6838/2317600000 2024-09-12T02:58:11.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.958+0000 7f2b77fff640 1 -- 172.21.15.5:0/2880031755 <== mds.0 v2:172.21.15.148:6838/2317600000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5562c8ded7f0 con 0x5562c8e99cb0 2024-09-12T02:58:11.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.958+0000 7f2b77fff640 10 client.15300 handle_command_reply: tid=0 2024-09-12T02:58:11.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.958+0000 7f2ba0fff740 1 client.15300 shutdown 2024-09-12T02:58:11.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.958+0000 7f2ba0fff740 1 -- 172.21.15.5:0/2880031755 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562c8e7aa20 msgr2=0x5562c8e87090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:11.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.958+0000 7f2ba0fff740 1 --2- 172.21.15.5:0/2880031755 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562c8e7aa20 0x5562c8e87090 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f2b84009340 tx=0x7f2b8402ab80 comp rx=0 tx=0).stop 2024-09-12T02:58:11.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.958+0000 7f2ba0fff740 1 -- 172.21.15.5:0/2880031755 shutdown_connections 2024-09-12T02:58:11.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.958+0000 7f2ba0fff740 1 --2- 172.21.15.5:0/2880031755 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5562c8e8a150 0x5562c8e87b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:11.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.958+0000 7f2ba0fff740 1 --2- 172.21.15.5:0/2880031755 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5562c8e7b340 0x5562c8e875d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:11.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.958+0000 7f2ba0fff740 1 --2- 172.21.15.5:0/2880031755 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x5562c8e99cb0 0x5562c8e9c0a0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:11.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.958+0000 7f2ba0fff740 1 --2- 172.21.15.5:0/2880031755 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562c8e7aa20 0x5562c8e87090 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:11.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.958+0000 7f2ba0fff740 1 -- 172.21.15.5:0/2880031755 >> 172.21.15.5:0/2880031755 conn(0x5562c8e770f0 msgr2=0x5562c8e7ce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:11.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.959+0000 7f2b77fff640 0 client.15300 ms_handle_reset on v2:172.21.15.148:6838/2317600000 2024-09-12T02:58:11.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.959+0000 7f2ba0fff740 1 -- 172.21.15.5:0/2880031755 shutdown_connections 2024-09-12T02:58:11.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.959+0000 7f2ba0fff740 1 -- 172.21.15.5:0/2880031755 wait complete. 2024-09-12T02:58:11.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.959+0000 7f2ba0fff740 20 client.15300 trim_cache size 0 max 16384 2024-09-12T02:58:11.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.986+0000 7f2ba0fff740 1 Processor -- start 2024-09-12T02:58:11.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.986+0000 7f2ba0fff740 1 -- start start 2024-09-12T02:58:11.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.986+0000 7f2ba0fff740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5562c8e7aa20 0x5562c8e9f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:11.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.986+0000 7f2ba0fff740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562c8e7b340 0x5562c8e9f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:11.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.986+0000 7f2ba0fff740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5562c8e8a150 0x5562c8e842f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:11.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.986+0000 7f2ba0fff740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2b981ab1e0 con 0x5562c8e7b340 2024-09-12T02:58:11.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.986+0000 7f2ba0fff740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f2b981acd20 con 0x5562c8e8a150 2024-09-12T02:58:11.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.986+0000 7f2ba0fff740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f2b981ab8b0 con 0x5562c8e7aa20 2024-09-12T02:58:11.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.986+0000 7f2b9c9c2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562c8e7b340 0x5562c8e9f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:11.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.986+0000 7f2b9c9c2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562c8e7b340 0x5562c8e9f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55548/0 (socket says 172.21.15.5:55548) 2024-09-12T02:58:11.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.986+0000 7f2b9c9c2640 1 -- 172.21.15.5:0/1840486804 learned_addr learned my addr 172.21.15.5:0/1840486804 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:58:11.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.986+0000 7f2b97fff640 1 --2- 172.21.15.5:0/1840486804 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5562c8e7aa20 0x5562c8e9f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:11.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.986+0000 7f2b97fff640 1 --2- 172.21.15.5:0/1840486804 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5562c8e8a150 0x5562c8e842f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:11.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.986+0000 7f2b9c9c2640 1 -- 172.21.15.5:0/1840486804 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5562c8e7aa20 msgr2=0x5562c8e9f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:11.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.986+0000 7f2b9c9c2640 1 --2- 172.21.15.5:0/1840486804 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5562c8e7aa20 0x5562c8e9f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:11.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.986+0000 7f2b9c9c2640 1 -- 172.21.15.5:0/1840486804 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5562c8e8a150 msgr2=0x5562c8e842f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:11.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.986+0000 7f2b9c9c2640 1 --2- 172.21.15.5:0/1840486804 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5562c8e8a150 0x5562c8e842f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:11.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.986+0000 7f2b9c9c2640 1 -- 172.21.15.5:0/1840486804 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562c8e855e0 con 0x5562c8e7b340 2024-09-12T02:58:11.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.986+0000 7f2b97fff640 1 --2- 172.21.15.5:0/1840486804 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5562c8e8a150 0x5562c8e842f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:58:11.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.987+0000 7f2b9c9c2640 1 --2- 172.21.15.5:0/1840486804 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562c8e7b340 0x5562c8e9f600 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x5562c8e8cf00 tx=0x7f2b8c00b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dad21a08c412993 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:11.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.987+0000 7f2b827fc640 1 -- 172.21.15.5:0/1840486804 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b8c003de0 con 0x5562c8e7b340 2024-09-12T02:58:11.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.987+0000 7f2b827fc640 1 -- 172.21.15.5:0/1840486804 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b8c026920 con 0x5562c8e7b340 2024-09-12T02:58:11.989 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.987+0000 7f2b827fc640 1 -- 172.21.15.5:0/1840486804 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b8c011a50 con 0x5562c8e7b340 2024-09-12T02:58:11.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.987+0000 7f2ba0fff740 1 -- 172.21.15.5:0/1840486804 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562c8e7b340 msgr2=0x5562c8e9f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:11.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.987+0000 7f2ba0fff740 1 --2- 172.21.15.5:0/1840486804 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562c8e7b340 0x5562c8e9f600 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x5562c8e8cf00 tx=0x7f2b8c00b390 comp rx=0 tx=0).stop 2024-09-12T02:58:11.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.988+0000 7f2ba0fff740 1 -- 172.21.15.5:0/1840486804 shutdown_connections 2024-09-12T02:58:11.991 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.988+0000 7f2ba0fff740 1 --2- 172.21.15.5:0/1840486804 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562c8e7b340 0x5562c8e9f600 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:11.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.988+0000 7f2ba0fff740 1 --2- 172.21.15.5:0/1840486804 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5562c8e8a150 0x5562c8e842f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:11.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.988+0000 7f2ba0fff740 1 --2- 172.21.15.5:0/1840486804 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5562c8e7aa20 0x5562c8e9f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:11.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.988+0000 7f2ba0fff740 1 -- 172.21.15.5:0/1840486804 >> 172.21.15.5:0/1840486804 conn(0x5562c8e770f0 msgr2=0x5562c8e76270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:11.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.988+0000 7f2ba0fff740 1 -- 172.21.15.5:0/1840486804 shutdown_connections 2024-09-12T02:58:11.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.988+0000 7f2ba0fff740 1 -- 172.21.15.5:0/1840486804 wait complete. 2024-09-12T02:58:11.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.988+0000 7f2ba0fff740 1 -- start start 2024-09-12T02:58:11.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.988+0000 7f2ba0fff740 1 Processor -- start 2024-09-12T02:58:11.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.988+0000 7f2ba0fff740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5562c8e7aa20 0x5562c8e9f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:11.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.988+0000 7f2ba0fff740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562c8e7b340 0x5562c8e9f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:11.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.988+0000 7f2ba0fff740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5562c8e8a150 0x5562c8e9d920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:11.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.988+0000 7f2ba0fff740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2b981ac650 con 0x5562c8e7b340 2024-09-12T02:58:11.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.988+0000 7f2ba0fff740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f2b981abf80 con 0x5562c8e8a150 2024-09-12T02:58:11.992 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.989+0000 7f2ba0fff740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f2b981a91c0 con 0x5562c8e7aa20 2024-09-12T02:58:11.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.989+0000 7f2ba0fff740 10 client.0 _ll_register_callbacks cb 0x7f2b8c00b770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:58:11.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.989+0000 7f2b9c9c2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562c8e7b340 0x5562c8e9f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:11.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.989+0000 7f2b9c9c2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562c8e7b340 0x5562c8e9f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55562/0 (socket says 172.21.15.5:55562) 2024-09-12T02:58:11.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.989+0000 7f2b9c9c2640 1 -- 172.21.15.5:0/569751052 learned_addr learned my addr 172.21.15.5:0/569751052 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:58:11.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.989+0000 7f2b97fff640 1 --2- 172.21.15.5:0/569751052 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5562c8e8a150 0x5562c8e9d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:11.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.989+0000 7f2b97fff640 1 --2- 172.21.15.5:0/569751052 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5562c8e7aa20 0x5562c8e9f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:11.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.989+0000 7f2b9c9c2640 1 -- 172.21.15.5:0/569751052 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5562c8e7aa20 msgr2=0x5562c8e9f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:11.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.989+0000 7f2b9c9c2640 1 --2- 172.21.15.5:0/569751052 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5562c8e7aa20 0x5562c8e9f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:11.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.989+0000 7f2b9c9c2640 1 -- 172.21.15.5:0/569751052 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5562c8e8a150 msgr2=0x5562c8e9d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:11.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.989+0000 7f2b9c9c2640 1 --2- 172.21.15.5:0/569751052 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5562c8e8a150 0x5562c8e9d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:11.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.989+0000 7f2b9c9c2640 1 -- 172.21.15.5:0/569751052 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562c8e9e150 con 0x5562c8e7b340 2024-09-12T02:58:11.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.989+0000 7f2b97fff640 1 --2- 172.21.15.5:0/569751052 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5562c8e7aa20 0x5562c8e9f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:58:11.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.989+0000 7f2b9c9c2640 1 --2- 172.21.15.5:0/569751052 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562c8e7b340 0x5562c8e9f920 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f2b8c004190 tx=0x7f2b8c0114c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6a9fa9a6862dc6e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:11.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.989+0000 7f2b77fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T02:58:11.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.990+0000 7f2b77fff640 1 -- 172.21.15.5:0/569751052 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b8c026440 con 0x5562c8e7b340 2024-09-12T02:58:11.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.990+0000 7f2b77fff640 1 -- 172.21.15.5:0/569751052 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f2b8c0265e0 con 0x5562c8e7b340 2024-09-12T02:58:11.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.990+0000 7f2ba0fff740 1 -- 172.21.15.5:0/569751052 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b8c026920 con 0x5562c8e7b340 2024-09-12T02:58:11.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.990+0000 7f2b77fff640 1 -- 172.21.15.5:0/569751052 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2b8c026ad0 con 0x5562c8e7b340 2024-09-12T02:58:11.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.990+0000 7f2ba0fff740 10 client.15306 fetch_fsmap learned FSMap version 26 2024-09-12T02:58:11.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.990+0000 7f2ba0fff740 1 -- 172.21.15.5:0/569751052 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5562c8f052b0 con 0x5562c8e7b340 2024-09-12T02:58:11.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.990+0000 7f2b77fff640 1 -- 172.21.15.5:0/569751052 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2b8c026920 con 0x5562c8e7b340 2024-09-12T02:58:11.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.990+0000 7f2b77fff640 1 -- 172.21.15.5:0/569751052 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f2b8c02d020 con 0x5562c8e7b340 2024-09-12T02:58:11.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.990+0000 7f2ba0fff740 10 client.15306 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T02:58:11.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.990+0000 7f2ba0fff740 10 client.15306 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T02:58:11.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.990+0000 7f2ba0fff740 20 client.15306 populate_metadata read hostname 'smithi005' 2024-09-12T02:58:11.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.990+0000 7f2ba0fff740 1 --2- 172.21.15.5:0/569751052 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x5562c8f059b0 0x5562c8f07dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:11.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.990+0000 7f2ba0fff740 4 client.15306 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T02:58:11.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.990+0000 7f2ba0fff740 1 -- 172.21.15.5:0/569751052 --> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5562c8de7d00 con 0x5562c8f059b0 2024-09-12T02:58:11.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.991+0000 7f2b97fff640 1 --2- 172.21.15.5:0/569751052 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x5562c8f059b0 0x5562c8f07dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:11.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.991+0000 7f2b97fff640 1 --2- 172.21.15.5:0/569751052 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x5562c8f059b0 0x5562c8f07dd0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75f37d3f317b7417 server_cookie=7530e88ee72626ca in_seq=0 out_seq=0 2024-09-12T02:58:11.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.991+0000 7f2b77fff640 10 client.15306 ms_handle_connect on v2:172.21.15.148:6838/2317600000 2024-09-12T02:58:11.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.992+0000 7f2b77fff640 1 -- 172.21.15.5:0/569751052 <== mds.0 v2:172.21.15.148:6838/2317600000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5562c8de7d00 con 0x5562c8f059b0 2024-09-12T02:58:11.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.992+0000 7f2b77fff640 10 client.15306 handle_command_reply: tid=0 2024-09-12T02:58:11.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.992+0000 7f2ba0fff740 1 client.15306 shutdown 2024-09-12T02:58:11.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.992+0000 7f2ba0fff740 1 -- 172.21.15.5:0/569751052 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562c8e7b340 msgr2=0x5562c8e9f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:11.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.992+0000 7f2ba0fff740 1 --2- 172.21.15.5:0/569751052 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562c8e7b340 0x5562c8e9f920 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f2b8c004190 tx=0x7f2b8c0114c0 comp rx=0 tx=0).stop 2024-09-12T02:58:11.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.992+0000 7f2ba0fff740 1 -- 172.21.15.5:0/569751052 shutdown_connections 2024-09-12T02:58:11.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.992+0000 7f2ba0fff740 1 --2- 172.21.15.5:0/569751052 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x5562c8f059b0 0x5562c8f07dd0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:11.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.992+0000 7f2b77fff640 0 client.15306 ms_handle_reset on v2:172.21.15.148:6838/2317600000 2024-09-12T02:58:11.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.992+0000 7f2ba0fff740 1 --2- 172.21.15.5:0/569751052 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5562c8e7b340 0x5562c8e9f920 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:11.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.992+0000 7f2ba0fff740 1 --2- 172.21.15.5:0/569751052 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5562c8e8a150 0x5562c8e9d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:11.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.992+0000 7f2ba0fff740 1 --2- 172.21.15.5:0/569751052 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5562c8e7aa20 0x5562c8e9f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:11.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.992+0000 7f2ba0fff740 1 -- 172.21.15.5:0/569751052 >> 172.21.15.5:0/569751052 conn(0x5562c8e770f0 msgr2=0x5562c8e950d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:11.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.993+0000 7f2ba0fff740 1 -- 172.21.15.5:0/569751052 shutdown_connections 2024-09-12T02:58:11.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.993+0000 7f2ba0fff740 1 -- 172.21.15.5:0/569751052 wait complete. 2024-09-12T02:58:11.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.993+0000 7f2ba0fff740 20 client.15306 trim_cache size 0 max 16384 2024-09-12T02:58:11.997 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-12T02:58:11.997 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-09-12T02:58:11.997 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "9734a5ee-e292-4bee-b391-561c540e414e", 2024-09-12T02:58:11.997 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-09-12T02:58:11.997 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-12T02:58:11.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.993+0000 7f2b9ec26640 1 -- 172.21.15.5:0/695917037 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f2b880a2c10 msgr2=0x7f2b880a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:11.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.993+0000 7f2b9ec26640 1 --2- 172.21.15.5:0/695917037 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f2b880a2c10 0x7f2b880a50d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2b90098490 tx=0x7f2b90097710 comp rx=0 tx=0).stop 2024-09-12T02:58:11.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.993+0000 7f2b9ec26640 1 -- 172.21.15.5:0/695917037 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b98106f10 msgr2=0x7f2b9819cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:11.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.993+0000 7f2b9ec26640 1 --2- 172.21.15.5:0/695917037 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b98106f10 0x7f2b9819cca0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f2b9000eaa0 tx=0x7f2b9000ef60 comp rx=0 tx=0).stop 2024-09-12T02:58:11.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.993+0000 7f2b9ec26640 1 -- 172.21.15.5:0/695917037 shutdown_connections 2024-09-12T02:58:11.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.993+0000 7f2b9ec26640 1 --2- 172.21.15.5:0/695917037 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f2b880a2c10 0x7f2b880a50d0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:11.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.993+0000 7f2b9ec26640 1 --2- 172.21.15.5:0/695917037 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b98106f10 0x7f2b9819cca0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:11.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.993+0000 7f2b9ec26640 1 --2- 172.21.15.5:0/695917037 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2b98102790 0x7f2b98198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:11.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.993+0000 7f2b9ec26640 1 --2- 172.21.15.5:0/695917037 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2b98101de0 0x7f2b981983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:11.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.993+0000 7f2b9ec26640 1 -- 172.21.15.5:0/695917037 >> 172.21.15.5:0/695917037 conn(0x7f2b980fd880 msgr2=0x7f2b980febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:11.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.994+0000 7f2b9ec26640 1 -- 172.21.15.5:0/695917037 shutdown_connections 2024-09-12T02:58:11.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:11.994+0000 7f2b9ec26640 1 -- 172.21.15.5:0/695917037 wait complete. 2024-09-12T02:58:12.982 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":334,"stamp":"2024-09-12T02:58:11.570028+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":668530,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1185,"num_read_kb":722,"num_write":456,"num_write_kb":3102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":1181470,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":872,"ondisk_log_size":872,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":374580,"kb_used_data":12852,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124567244,"statfs":{"total":1151940427776,"available":1151556857856,"internally_reserved":0,"allocated":13160448,"data_stored":7265946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":370193723},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":9553,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":120,"num_read_kb":69,"num_write":27,"num_write_kb":27,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002641"},"pg_stats":[{"pgid":"3.3f","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.565050+0000","last_change":"2024-09-12T02:56:15.988690+0000","last_active":"2024-09-12T02:57:59.565050+0000","last_peered":"2024-09-12T02:57:59.565050+0000","last_clean":"2024-09-12T02:57:59.565050+0000","last_became_active":"2024-09-12T02:56:15.988320+0000","last_became_peered":"2024-09-12T02:56:15.988320+0000","last_unstale":"2024-09-12T02:57:59.565050+0000","last_undegraded":"2024-09-12T02:57:59.565050+0000","last_fullsized":"2024-09-12T02:57:59.565050+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:31:14.157622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"83'4","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.553842+0000","last_change":"2024-09-12T02:56:15.978368+0000","last_active":"2024-09-12T02:57:59.553842+0000","last_peered":"2024-09-12T02:57:59.553842+0000","last_clean":"2024-09-12T02:57:59.553842+0000","last_became_active":"2024-09-12T02:56:15.978092+0000","last_became_peered":"2024-09-12T02:56:15.978092+0000","last_unstale":"2024-09-12T02:57:59.553842+0000","last_undegraded":"2024-09-12T02:57:59.553842+0000","last_fullsized":"2024-09-12T02:57:59.553842+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:32:48.399005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.564003+0000","last_change":"2024-09-12T02:56:15.987249+0000","last_active":"2024-09-12T02:57:59.564003+0000","last_peered":"2024-09-12T02:57:59.564003+0000","last_clean":"2024-09-12T02:57:59.564003+0000","last_became_active":"2024-09-12T02:56:15.986921+0000","last_became_peered":"2024-09-12T02:56:15.986921+0000","last_unstale":"2024-09-12T02:57:59.564003+0000","last_undegraded":"2024-09-12T02:57:59.564003+0000","last_fullsized":"2024-09-12T02:57:59.564003+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:42:13.427871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.555703+0000","last_change":"2024-09-12T02:56:15.987673+0000","last_active":"2024-09-12T02:57:59.555703+0000","last_peered":"2024-09-12T02:57:59.555703+0000","last_clean":"2024-09-12T02:57:59.555703+0000","last_became_active":"2024-09-12T02:56:15.986964+0000","last_became_peered":"2024-09-12T02:56:15.986964+0000","last_unstale":"2024-09-12T02:57:59.555703+0000","last_undegraded":"2024-09-12T02:57:59.555703+0000","last_fullsized":"2024-09-12T02:57:59.555703+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:07:12.406974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"83'4","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.548461+0000","last_change":"2024-09-12T02:56:15.974473+0000","last_active":"2024-09-12T02:57:59.548461+0000","last_peered":"2024-09-12T02:57:59.548461+0000","last_clean":"2024-09-12T02:57:59.548461+0000","last_became_active":"2024-09-12T02:56:15.974124+0000","last_became_peered":"2024-09-12T02:56:15.974124+0000","last_unstale":"2024-09-12T02:57:59.548461+0000","last_undegraded":"2024-09-12T02:57:59.548461+0000","last_fullsized":"2024-09-12T02:57:59.548461+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:44:29.054841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.560740+0000","last_change":"2024-09-12T02:56:15.980242+0000","last_active":"2024-09-12T02:57:59.560740+0000","last_peered":"2024-09-12T02:57:59.560740+0000","last_clean":"2024-09-12T02:57:59.560740+0000","last_became_active":"2024-09-12T02:56:15.979937+0000","last_became_peered":"2024-09-12T02:56:15.979937+0000","last_unstale":"2024-09-12T02:57:59.560740+0000","last_undegraded":"2024-09-12T02:57:59.560740+0000","last_fullsized":"2024-09-12T02:57:59.560740+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:44:13.406341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"83'5","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.556845+0000","last_change":"2024-09-12T02:56:15.978746+0000","last_active":"2024-09-12T02:57:59.556845+0000","last_peered":"2024-09-12T02:57:59.556845+0000","last_clean":"2024-09-12T02:57:59.556845+0000","last_became_active":"2024-09-12T02:56:15.978304+0000","last_became_peered":"2024-09-12T02:56:15.978304+0000","last_unstale":"2024-09-12T02:57:59.556845+0000","last_undegraded":"2024-09-12T02:57:59.556845+0000","last_fullsized":"2024-09-12T02:57:59.556845+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:34:52.115630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.562507+0000","last_change":"2024-09-12T02:56:15.980083+0000","last_active":"2024-09-12T02:57:59.562507+0000","last_peered":"2024-09-12T02:57:59.562507+0000","last_clean":"2024-09-12T02:57:59.562507+0000","last_became_active":"2024-09-12T02:56:15.979771+0000","last_became_peered":"2024-09-12T02:56:15.979771+0000","last_unstale":"2024-09-12T02:57:59.562507+0000","last_undegraded":"2024-09-12T02:57:59.562507+0000","last_fullsized":"2024-09-12T02:57:59.562507+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:59:42.351970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"83'11","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.565703+0000","last_change":"2024-09-12T02:56:15.988091+0000","last_active":"2024-09-12T02:57:59.565703+0000","last_peered":"2024-09-12T02:57:59.565703+0000","last_clean":"2024-09-12T02:57:59.565703+0000","last_became_active":"2024-09-12T02:56:15.986295+0000","last_became_peered":"2024-09-12T02:56:15.986295+0000","last_unstale":"2024-09-12T02:57:59.565703+0000","last_undegraded":"2024-09-12T02:57:59.565703+0000","last_fullsized":"2024-09-12T02:57:59.565703+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:45:40.331805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"83'16","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.562128+0000","last_change":"2024-09-12T02:56:15.987349+0000","last_active":"2024-09-12T02:57:59.562128+0000","last_peered":"2024-09-12T02:57:59.562128+0000","last_clean":"2024-09-12T02:57:59.562128+0000","last_became_active":"2024-09-12T02:56:15.987018+0000","last_became_peered":"2024-09-12T02:56:15.987018+0000","last_unstale":"2024-09-12T02:57:59.562128+0000","last_undegraded":"2024-09-12T02:57:59.562128+0000","last_fullsized":"2024-09-12T02:57:59.562128+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:34:19.652348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.563544+0000","last_change":"2024-09-12T02:56:15.985164+0000","last_active":"2024-09-12T02:57:59.563544+0000","last_peered":"2024-09-12T02:57:59.563544+0000","last_clean":"2024-09-12T02:57:59.563544+0000","last_became_active":"2024-09-12T02:56:15.984787+0000","last_became_peered":"2024-09-12T02:56:15.984787+0000","last_unstale":"2024-09-12T02:57:59.563544+0000","last_undegraded":"2024-09-12T02:57:59.563544+0000","last_fullsized":"2024-09-12T02:57:59.563544+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:36:29.727442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.552957+0000","last_change":"2024-09-12T02:56:15.979963+0000","last_active":"2024-09-12T02:57:59.552957+0000","last_peered":"2024-09-12T02:57:59.552957+0000","last_clean":"2024-09-12T02:57:59.552957+0000","last_became_active":"2024-09-12T02:56:15.979639+0000","last_became_peered":"2024-09-12T02:56:15.979639+0000","last_unstale":"2024-09-12T02:57:59.552957+0000","last_undegraded":"2024-09-12T02:57:59.552957+0000","last_fullsized":"2024-09-12T02:57:59.552957+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:34:18.273245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"83'10","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.561647+0000","last_change":"2024-09-12T02:56:15.980415+0000","last_active":"2024-09-12T02:57:59.561647+0000","last_peered":"2024-09-12T02:57:59.561647+0000","last_clean":"2024-09-12T02:57:59.561647+0000","last_became_active":"2024-09-12T02:56:15.980059+0000","last_became_peered":"2024-09-12T02:56:15.980059+0000","last_unstale":"2024-09-12T02:57:59.561647+0000","last_undegraded":"2024-09-12T02:57:59.561647+0000","last_fullsized":"2024-09-12T02:57:59.561647+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:11:42.689081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.553574+0000","last_change":"2024-09-12T02:56:15.981423+0000","last_active":"2024-09-12T02:57:59.553574+0000","last_peered":"2024-09-12T02:57:59.553574+0000","last_clean":"2024-09-12T02:57:59.553574+0000","last_became_active":"2024-09-12T02:56:15.981186+0000","last_became_peered":"2024-09-12T02:56:15.981186+0000","last_unstale":"2024-09-12T02:57:59.553574+0000","last_undegraded":"2024-09-12T02:57:59.553574+0000","last_fullsized":"2024-09-12T02:57:59.553574+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:36:57.406907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.547696+0000","last_change":"2024-09-12T02:56:15.982587+0000","last_active":"2024-09-12T02:57:59.547696+0000","last_peered":"2024-09-12T02:57:59.547696+0000","last_clean":"2024-09-12T02:57:59.547696+0000","last_became_active":"2024-09-12T02:56:15.982315+0000","last_became_peered":"2024-09-12T02:56:15.982315+0000","last_unstale":"2024-09-12T02:57:59.547696+0000","last_undegraded":"2024-09-12T02:57:59.547696+0000","last_fullsized":"2024-09-12T02:57:59.547696+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:42:17.736999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.564760+0000","last_change":"2024-09-12T02:56:15.979482+0000","last_active":"2024-09-12T02:57:59.564760+0000","last_peered":"2024-09-12T02:57:59.564760+0000","last_clean":"2024-09-12T02:57:59.564760+0000","last_became_active":"2024-09-12T02:56:15.979122+0000","last_became_peered":"2024-09-12T02:56:15.979122+0000","last_unstale":"2024-09-12T02:57:59.564760+0000","last_undegraded":"2024-09-12T02:57:59.564760+0000","last_fullsized":"2024-09-12T02:57:59.564760+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:18:05.642487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.565218+0000","last_change":"2024-09-12T02:56:15.989908+0000","last_active":"2024-09-12T02:57:59.565218+0000","last_peered":"2024-09-12T02:57:59.565218+0000","last_clean":"2024-09-12T02:57:59.565218+0000","last_became_active":"2024-09-12T02:56:15.989515+0000","last_became_peered":"2024-09-12T02:56:15.989515+0000","last_unstale":"2024-09-12T02:57:59.565218+0000","last_undegraded":"2024-09-12T02:57:59.565218+0000","last_fullsized":"2024-09-12T02:57:59.565218+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:59:57.141607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"83'10","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.557609+0000","last_change":"2024-09-12T02:56:15.980323+0000","last_active":"2024-09-12T02:57:59.557609+0000","last_peered":"2024-09-12T02:57:59.557609+0000","last_clean":"2024-09-12T02:57:59.557609+0000","last_became_active":"2024-09-12T02:56:15.979989+0000","last_became_peered":"2024-09-12T02:56:15.979989+0000","last_unstale":"2024-09-12T02:57:59.557609+0000","last_undegraded":"2024-09-12T02:57:59.557609+0000","last_fullsized":"2024-09-12T02:57:59.557609+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:55:56.064125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.556275+0000","last_change":"2024-09-12T02:56:15.986003+0000","last_active":"2024-09-12T02:57:59.556275+0000","last_peered":"2024-09-12T02:57:59.556275+0000","last_clean":"2024-09-12T02:57:59.556275+0000","last_became_active":"2024-09-12T02:56:15.985335+0000","last_became_peered":"2024-09-12T02:56:15.985335+0000","last_unstale":"2024-09-12T02:57:59.556275+0000","last_undegraded":"2024-09-12T02:57:59.556275+0000","last_fullsized":"2024-09-12T02:57:59.556275+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:51:47.944530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"83'8","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.554254+0000","last_change":"2024-09-12T02:56:31.036870+0000","last_active":"2024-09-12T02:57:59.554254+0000","last_peered":"2024-09-12T02:57:59.554254+0000","last_clean":"2024-09-12T02:57:59.554254+0000","last_became_active":"2024-09-12T02:56:31.036443+0000","last_became_peered":"2024-09-12T02:56:31.036443+0000","last_unstale":"2024-09-12T02:57:59.554254+0000","last_undegraded":"2024-09-12T02:57:59.554254+0000","last_fullsized":"2024-09-12T02:57:59.554254+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:49:29.326184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"83'11","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.555414+0000","last_change":"2024-09-12T02:56:15.974547+0000","last_active":"2024-09-12T02:57:59.555414+0000","last_peered":"2024-09-12T02:57:59.555414+0000","last_clean":"2024-09-12T02:57:59.555414+0000","last_became_active":"2024-09-12T02:56:15.974174+0000","last_became_peered":"2024-09-12T02:56:15.974174+0000","last_unstale":"2024-09-12T02:57:59.555414+0000","last_undegraded":"2024-09-12T02:57:59.555414+0000","last_fullsized":"2024-09-12T02:57:59.555414+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:22:16.954309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"83'10","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.565778+0000","last_change":"2024-09-12T02:56:15.981771+0000","last_active":"2024-09-12T02:57:59.565778+0000","last_peered":"2024-09-12T02:57:59.565778+0000","last_clean":"2024-09-12T02:57:59.565778+0000","last_became_active":"2024-09-12T02:56:15.980948+0000","last_became_peered":"2024-09-12T02:56:15.980948+0000","last_unstale":"2024-09-12T02:57:59.565778+0000","last_undegraded":"2024-09-12T02:57:59.565778+0000","last_fullsized":"2024-09-12T02:57:59.565778+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:34:35.961348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.562061+0000","last_change":"2024-09-12T02:56:15.982318+0000","last_active":"2024-09-12T02:57:59.562061+0000","last_peered":"2024-09-12T02:57:59.562061+0000","last_clean":"2024-09-12T02:57:59.562061+0000","last_became_active":"2024-09-12T02:56:15.982013+0000","last_became_peered":"2024-09-12T02:56:15.982013+0000","last_unstale":"2024-09-12T02:57:59.562061+0000","last_undegraded":"2024-09-12T02:57:59.562061+0000","last_fullsized":"2024-09-12T02:57:59.562061+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:10:50.895579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.559580+0000","last_change":"2024-09-12T02:56:15.969992+0000","last_active":"2024-09-12T02:57:59.559580+0000","last_peered":"2024-09-12T02:57:59.559580+0000","last_clean":"2024-09-12T02:57:59.559580+0000","last_became_active":"2024-09-12T02:56:15.969621+0000","last_became_peered":"2024-09-12T02:56:15.969621+0000","last_unstale":"2024-09-12T02:57:59.559580+0000","last_undegraded":"2024-09-12T02:57:59.559580+0000","last_fullsized":"2024-09-12T02:57:59.559580+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:47:40.206042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"83'2","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.560304+0000","last_change":"2024-09-12T02:56:15.982315+0000","last_active":"2024-09-12T02:57:59.560304+0000","last_peered":"2024-09-12T02:57:59.560304+0000","last_clean":"2024-09-12T02:57:59.560304+0000","last_became_active":"2024-09-12T02:56:15.981992+0000","last_became_peered":"2024-09-12T02:56:15.981992+0000","last_unstale":"2024-09-12T02:57:59.560304+0000","last_undegraded":"2024-09-12T02:57:59.560304+0000","last_fullsized":"2024-09-12T02:57:59.560304+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:54:04.188418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"83'15","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.560612+0000","last_change":"2024-09-12T02:56:15.980712+0000","last_active":"2024-09-12T02:57:59.560612+0000","last_peered":"2024-09-12T02:57:59.560612+0000","last_clean":"2024-09-12T02:57:59.560612+0000","last_became_active":"2024-09-12T02:56:15.980410+0000","last_became_peered":"2024-09-12T02:56:15.980410+0000","last_unstale":"2024-09-12T02:57:59.560612+0000","last_undegraded":"2024-09-12T02:57:59.560612+0000","last_fullsized":"2024-09-12T02:57:59.560612+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:13:53.844490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.557058+0000","last_change":"2024-09-12T02:56:15.981675+0000","last_active":"2024-09-12T02:57:59.557058+0000","last_peered":"2024-09-12T02:57:59.557058+0000","last_clean":"2024-09-12T02:57:59.557058+0000","last_became_active":"2024-09-12T02:56:15.981372+0000","last_became_peered":"2024-09-12T02:56:15.981372+0000","last_unstale":"2024-09-12T02:57:59.557058+0000","last_undegraded":"2024-09-12T02:57:59.557058+0000","last_fullsized":"2024-09-12T02:57:59.557058+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:39:17.399199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"83'4","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.557413+0000","last_change":"2024-09-12T02:56:15.972750+0000","last_active":"2024-09-12T02:57:59.557413+0000","last_peered":"2024-09-12T02:57:59.557413+0000","last_clean":"2024-09-12T02:57:59.557413+0000","last_became_active":"2024-09-12T02:56:15.972434+0000","last_became_peered":"2024-09-12T02:56:15.972434+0000","last_unstale":"2024-09-12T02:57:59.557413+0000","last_undegraded":"2024-09-12T02:57:59.557413+0000","last_fullsized":"2024-09-12T02:57:59.557413+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:59:22.784908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.545235+0000","last_change":"2024-09-12T02:56:15.973567+0000","last_active":"2024-09-12T02:57:59.545235+0000","last_peered":"2024-09-12T02:57:59.545235+0000","last_clean":"2024-09-12T02:57:59.545235+0000","last_became_active":"2024-09-12T02:56:15.973251+0000","last_became_peered":"2024-09-12T02:56:15.973251+0000","last_unstale":"2024-09-12T02:57:59.545235+0000","last_undegraded":"2024-09-12T02:57:59.545235+0000","last_fullsized":"2024-09-12T02:57:59.545235+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:16:39.536817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.540308+0000","last_change":"2024-09-12T02:56:15.986108+0000","last_active":"2024-09-12T02:57:59.540308+0000","last_peered":"2024-09-12T02:57:59.540308+0000","last_clean":"2024-09-12T02:57:59.540308+0000","last_became_active":"2024-09-12T02:56:15.985701+0000","last_became_peered":"2024-09-12T02:56:15.985701+0000","last_unstale":"2024-09-12T02:57:59.540308+0000","last_undegraded":"2024-09-12T02:57:59.540308+0000","last_fullsized":"2024-09-12T02:57:59.540308+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:59.377473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"83'12","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.555638+0000","last_change":"2024-09-12T02:56:15.980078+0000","last_active":"2024-09-12T02:57:59.555638+0000","last_peered":"2024-09-12T02:57:59.555638+0000","last_clean":"2024-09-12T02:57:59.555638+0000","last_became_active":"2024-09-12T02:56:15.979785+0000","last_became_peered":"2024-09-12T02:56:15.979785+0000","last_unstale":"2024-09-12T02:57:59.555638+0000","last_undegraded":"2024-09-12T02:57:59.555638+0000","last_fullsized":"2024-09-12T02:57:59.555638+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:00:19.252398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.559119+0000","last_change":"2024-09-12T02:56:15.984170+0000","last_active":"2024-09-12T02:57:59.559119+0000","last_peered":"2024-09-12T02:57:59.559119+0000","last_clean":"2024-09-12T02:57:59.559119+0000","last_became_active":"2024-09-12T02:56:15.983416+0000","last_became_peered":"2024-09-12T02:56:15.983416+0000","last_unstale":"2024-09-12T02:57:59.559119+0000","last_undegraded":"2024-09-12T02:57:59.559119+0000","last_fullsized":"2024-09-12T02:57:59.559119+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:13:24.723923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.556295+0000","last_change":"2024-09-12T02:56:15.980987+0000","last_active":"2024-09-12T02:57:59.556295+0000","last_peered":"2024-09-12T02:57:59.556295+0000","last_clean":"2024-09-12T02:57:59.556295+0000","last_became_active":"2024-09-12T02:56:15.980661+0000","last_became_peered":"2024-09-12T02:56:15.980661+0000","last_unstale":"2024-09-12T02:57:59.556295+0000","last_undegraded":"2024-09-12T02:57:59.556295+0000","last_fullsized":"2024-09-12T02:57:59.556295+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:19:49.294698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"83'6","reported_seq":142,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:58:05.180949+0000","last_change":"2024-09-12T02:56:31.042914+0000","last_active":"2024-09-12T02:58:05.180949+0000","last_peered":"2024-09-12T02:58:05.180949+0000","last_clean":"2024-09-12T02:58:05.180949+0000","last_became_active":"2024-09-12T02:56:31.037356+0000","last_became_peered":"2024-09-12T02:56:31.037356+0000","last_unstale":"2024-09-12T02:58:05.180949+0000","last_undegraded":"2024-09-12T02:58:05.180949+0000","last_fullsized":"2024-09-12T02:58:05.180949+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:17:48.582725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":102,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":622,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"83'8","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.565986+0000","last_change":"2024-09-12T02:56:31.036170+0000","last_active":"2024-09-12T02:57:59.565986+0000","last_peered":"2024-09-12T02:57:59.565986+0000","last_clean":"2024-09-12T02:57:59.565986+0000","last_became_active":"2024-09-12T02:56:31.035754+0000","last_became_peered":"2024-09-12T02:56:31.035754+0000","last_unstale":"2024-09-12T02:57:59.565986+0000","last_undegraded":"2024-09-12T02:57:59.565986+0000","last_fullsized":"2024-09-12T02:57:59.565986+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:58:06.464818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1a","version":"81'7","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.032813+0000","last_change":"2024-09-12T02:56:13.931202+0000","last_active":"2024-09-12T02:56:31.032813+0000","last_peered":"2024-09-12T02:56:31.032813+0000","last_clean":"2024-09-12T02:56:31.032813+0000","last_became_active":"2024-09-12T02:56:13.930776+0000","last_became_peered":"2024-09-12T02:56:13.930776+0000","last_unstale":"2024-09-12T02:56:31.032813+0000","last_undegraded":"2024-09-12T02:56:31.032813+0000","last_fullsized":"2024-09-12T02:56:31.032813+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:22:16.919253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"81'3","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.031662+0000","last_change":"2024-09-12T02:56:13.924660+0000","last_active":"2024-09-12T02:56:31.031662+0000","last_peered":"2024-09-12T02:56:31.031662+0000","last_clean":"2024-09-12T02:56:31.031662+0000","last_became_active":"2024-09-12T02:56:13.924342+0000","last_became_peered":"2024-09-12T02:56:13.924342+0000","last_unstale":"2024-09-12T02:56:31.031662+0000","last_undegraded":"2024-09-12T02:56:31.031662+0000","last_fullsized":"2024-09-12T02:56:31.031662+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:48:53.907044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.563814+0000","last_change":"2024-09-12T02:56:15.978723+0000","last_active":"2024-09-12T02:57:59.563814+0000","last_peered":"2024-09-12T02:57:59.563814+0000","last_clean":"2024-09-12T02:57:59.563814+0000","last_became_active":"2024-09-12T02:56:15.978344+0000","last_became_peered":"2024-09-12T02:56:15.978344+0000","last_unstale":"2024-09-12T02:57:59.563814+0000","last_undegraded":"2024-09-12T02:57:59.563814+0000","last_fullsized":"2024-09-12T02:57:59.563814+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:25:04.559395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"81'3","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.032936+0000","last_change":"2024-09-12T02:56:13.933065+0000","last_active":"2024-09-12T02:56:31.032936+0000","last_peered":"2024-09-12T02:56:31.032936+0000","last_clean":"2024-09-12T02:56:31.032936+0000","last_became_active":"2024-09-12T02:56:13.932597+0000","last_became_peered":"2024-09-12T02:56:13.932597+0000","last_unstale":"2024-09-12T02:56:31.032936+0000","last_undegraded":"2024-09-12T02:56:31.032936+0000","last_fullsized":"2024-09-12T02:56:31.032936+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:57:26.563392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"83'11","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.566801+0000","last_change":"2024-09-12T02:56:15.975722+0000","last_active":"2024-09-12T02:57:59.566801+0000","last_peered":"2024-09-12T02:57:59.566801+0000","last_clean":"2024-09-12T02:57:59.566801+0000","last_became_active":"2024-09-12T02:56:15.975342+0000","last_became_peered":"2024-09-12T02:56:15.975342+0000","last_unstale":"2024-09-12T02:57:59.566801+0000","last_undegraded":"2024-09-12T02:57:59.566801+0000","last_fullsized":"2024-09-12T02:57:59.566801+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:33:17.613738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'6","reported_seq":228,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:58:06.984214+0000","last_change":"2024-09-12T02:56:13.929700+0000","last_active":"2024-09-12T02:58:06.984214+0000","last_peered":"2024-09-12T02:58:06.984214+0000","last_clean":"2024-09-12T02:58:06.984214+0000","last_became_active":"2024-09-12T02:56:13.929317+0000","last_became_peered":"2024-09-12T02:56:13.929317+0000","last_unstale":"2024-09-12T02:58:06.984214+0000","last_undegraded":"2024-09-12T02:58:06.984214+0000","last_fullsized":"2024-09-12T02:58:06.984214+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:46:32.172674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":200,"num_read_kb":200,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"83'10","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.564710+0000","last_change":"2024-09-12T02:56:15.985754+0000","last_active":"2024-09-12T02:57:59.564710+0000","last_peered":"2024-09-12T02:57:59.564710+0000","last_clean":"2024-09-12T02:57:59.564710+0000","last_became_active":"2024-09-12T02:56:15.985402+0000","last_became_peered":"2024-09-12T02:56:15.985402+0000","last_unstale":"2024-09-12T02:57:59.564710+0000","last_undegraded":"2024-09-12T02:57:59.564710+0000","last_fullsized":"2024-09-12T02:57:59.564710+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:05:29.979961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"81'4","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.035428+0000","last_change":"2024-09-12T02:56:13.933329+0000","last_active":"2024-09-12T02:56:31.035428+0000","last_peered":"2024-09-12T02:56:31.035428+0000","last_clean":"2024-09-12T02:56:31.035428+0000","last_became_active":"2024-09-12T02:56:13.932960+0000","last_became_peered":"2024-09-12T02:56:13.932960+0000","last_unstale":"2024-09-12T02:56:31.035428+0000","last_undegraded":"2024-09-12T02:56:31.035428+0000","last_fullsized":"2024-09-12T02:56:31.035428+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:02:57.564202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"83'9","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.556804+0000","last_change":"2024-09-12T02:56:31.037340+0000","last_active":"2024-09-12T02:57:59.556804+0000","last_peered":"2024-09-12T02:57:59.556804+0000","last_clean":"2024-09-12T02:57:59.556804+0000","last_became_active":"2024-09-12T02:56:31.036993+0000","last_became_peered":"2024-09-12T02:56:31.036993+0000","last_unstale":"2024-09-12T02:57:59.556804+0000","last_undegraded":"2024-09-12T02:57:59.556804+0000","last_fullsized":"2024-09-12T02:57:59.556804+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:44:27.788791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"81'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.033005+0000","last_change":"2024-09-12T02:56:14.077202+0000","last_active":"2024-09-12T02:56:31.033005+0000","last_peered":"2024-09-12T02:56:31.033005+0000","last_clean":"2024-09-12T02:56:31.033005+0000","last_became_active":"2024-09-12T02:56:14.076610+0000","last_became_peered":"2024-09-12T02:56:14.076610+0000","last_unstale":"2024-09-12T02:56:31.033005+0000","last_undegraded":"2024-09-12T02:56:31.033005+0000","last_fullsized":"2024-09-12T02:56:31.033005+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:37:50.333152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"83'11","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.558754+0000","last_change":"2024-09-12T02:56:15.977161+0000","last_active":"2024-09-12T02:57:59.558754+0000","last_peered":"2024-09-12T02:57:59.558754+0000","last_clean":"2024-09-12T02:57:59.558754+0000","last_became_active":"2024-09-12T02:56:15.976683+0000","last_became_peered":"2024-09-12T02:56:15.976683+0000","last_unstale":"2024-09-12T02:57:59.558754+0000","last_undegraded":"2024-09-12T02:57:59.558754+0000","last_fullsized":"2024-09-12T02:57:59.558754+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:34:07.526890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'38","reported_seq":273,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:58:06.984664+0000","last_change":"2024-09-12T02:56:13.931680+0000","last_active":"2024-09-12T02:58:06.984664+0000","last_peered":"2024-09-12T02:58:06.984664+0000","last_clean":"2024-09-12T02:58:06.984664+0000","last_became_active":"2024-09-12T02:56:13.931214+0000","last_became_peered":"2024-09-12T02:56:13.931214+0000","last_unstale":"2024-09-12T02:58:06.984664+0000","last_undegraded":"2024-09-12T02:58:06.984664+0000","last_fullsized":"2024-09-12T02:58:06.984664+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:38:34.151933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68951,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":213,"num_read_kb":171,"num_write":40,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.555616+0000","last_change":"2024-09-12T02:56:15.977720+0000","last_active":"2024-09-12T02:57:59.555616+0000","last_peered":"2024-09-12T02:57:59.555616+0000","last_clean":"2024-09-12T02:57:59.555616+0000","last_became_active":"2024-09-12T02:56:15.977380+0000","last_became_peered":"2024-09-12T02:56:15.977380+0000","last_unstale":"2024-09-12T02:57:59.555616+0000","last_undegraded":"2024-09-12T02:57:59.555616+0000","last_fullsized":"2024-09-12T02:57:59.555616+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:52:16.841909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"81'9","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.030362+0000","last_change":"2024-09-12T02:56:13.929496+0000","last_active":"2024-09-12T02:56:31.030362+0000","last_peered":"2024-09-12T02:56:31.030362+0000","last_clean":"2024-09-12T02:56:31.030362+0000","last_became_active":"2024-09-12T02:56:13.929117+0000","last_became_peered":"2024-09-12T02:56:13.929117+0000","last_unstale":"2024-09-12T02:56:31.030362+0000","last_undegraded":"2024-09-12T02:56:31.030362+0000","last_fullsized":"2024-09-12T02:56:31.030362+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:59:38.399135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.564811+0000","last_change":"2024-09-12T02:56:15.974323+0000","last_active":"2024-09-12T02:57:59.564811+0000","last_peered":"2024-09-12T02:57:59.564811+0000","last_clean":"2024-09-12T02:57:59.564811+0000","last_became_active":"2024-09-12T02:56:15.973819+0000","last_became_peered":"2024-09-12T02:56:15.973819+0000","last_unstale":"2024-09-12T02:57:59.564811+0000","last_undegraded":"2024-09-12T02:57:59.564811+0000","last_fullsized":"2024-09-12T02:57:59.564811+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:53:13.130695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"81'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.032473+0000","last_change":"2024-09-12T02:56:13.935263+0000","last_active":"2024-09-12T02:56:31.032473+0000","last_peered":"2024-09-12T02:56:31.032473+0000","last_clean":"2024-09-12T02:56:31.032473+0000","last_became_active":"2024-09-12T02:56:13.934805+0000","last_became_peered":"2024-09-12T02:56:13.934805+0000","last_unstale":"2024-09-12T02:56:31.032473+0000","last_undegraded":"2024-09-12T02:56:31.032473+0000","last_fullsized":"2024-09-12T02:56:31.032473+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:39:42.882031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"83'15","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.560318+0000","last_change":"2024-09-12T02:56:15.976630+0000","last_active":"2024-09-12T02:57:59.560318+0000","last_peered":"2024-09-12T02:57:59.560318+0000","last_clean":"2024-09-12T02:57:59.560318+0000","last_became_active":"2024-09-12T02:56:15.976304+0000","last_became_peered":"2024-09-12T02:56:15.976304+0000","last_unstale":"2024-09-12T02:57:59.560318+0000","last_undegraded":"2024-09-12T02:57:59.560318+0000","last_fullsized":"2024-09-12T02:57:59.560318+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:46:03.366740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'8","reported_seq":138,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:58:08.987094+0000","last_change":"2024-09-12T02:56:13.934224+0000","last_active":"2024-09-12T02:58:08.987094+0000","last_peered":"2024-09-12T02:58:08.987094+0000","last_clean":"2024-09-12T02:58:08.987094+0000","last_became_active":"2024-09-12T02:56:13.933838+0000","last_became_peered":"2024-09-12T02:56:13.933838+0000","last_unstale":"2024-09-12T02:58:08.987094+0000","last_undegraded":"2024-09-12T02:58:08.987094+0000","last_fullsized":"2024-09-12T02:58:08.987094+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:34:34.206051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1515,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":109,"num_read_kb":6,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.557213+0000","last_change":"2024-09-12T02:56:15.980364+0000","last_active":"2024-09-12T02:57:59.557213+0000","last_peered":"2024-09-12T02:57:59.557213+0000","last_clean":"2024-09-12T02:57:59.557213+0000","last_became_active":"2024-09-12T02:56:15.979437+0000","last_became_peered":"2024-09-12T02:56:15.979437+0000","last_unstale":"2024-09-12T02:57:59.557213+0000","last_undegraded":"2024-09-12T02:57:59.557213+0000","last_fullsized":"2024-09-12T02:57:59.557213+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:43:33.424867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"81'17","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.050509+0000","last_change":"2024-09-12T02:56:31.050046+0000","last_active":"2024-09-12T02:56:31.050509+0000","last_peered":"2024-09-12T02:56:31.050509+0000","last_clean":"2024-09-12T02:56:31.050509+0000","last_became_active":"2024-09-12T02:56:31.036990+0000","last_became_peered":"2024-09-12T02:56:31.036990+0000","last_unstale":"2024-09-12T02:56:31.050509+0000","last_undegraded":"2024-09-12T02:56:31.050509+0000","last_fullsized":"2024-09-12T02:56:31.050509+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:05:53.336691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.563297+0000","last_change":"2024-09-12T02:56:15.986434+0000","last_active":"2024-09-12T02:57:59.563297+0000","last_peered":"2024-09-12T02:57:59.563297+0000","last_clean":"2024-09-12T02:57:59.563297+0000","last_became_active":"2024-09-12T02:56:15.986101+0000","last_became_peered":"2024-09-12T02:56:15.986101+0000","last_unstale":"2024-09-12T02:57:59.563297+0000","last_undegraded":"2024-09-12T02:57:59.563297+0000","last_fullsized":"2024-09-12T02:57:59.563297+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:51:27.840951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"81'2","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.035974+0000","last_change":"2024-09-12T02:56:13.925068+0000","last_active":"2024-09-12T02:56:31.035974+0000","last_peered":"2024-09-12T02:56:31.035974+0000","last_clean":"2024-09-12T02:56:31.035974+0000","last_became_active":"2024-09-12T02:56:13.924535+0000","last_became_peered":"2024-09-12T02:56:13.924535+0000","last_unstale":"2024-09-12T02:56:31.035974+0000","last_undegraded":"2024-09-12T02:56:31.035974+0000","last_fullsized":"2024-09-12T02:56:31.035974+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:45:32.478632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"83'5","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.559020+0000","last_change":"2024-09-12T02:56:15.983850+0000","last_active":"2024-09-12T02:57:59.559020+0000","last_peered":"2024-09-12T02:57:59.559020+0000","last_clean":"2024-09-12T02:57:59.559020+0000","last_became_active":"2024-09-12T02:56:15.983480+0000","last_became_peered":"2024-09-12T02:56:15.983480+0000","last_unstale":"2024-09-12T02:57:59.559020+0000","last_undegraded":"2024-09-12T02:57:59.559020+0000","last_fullsized":"2024-09-12T02:57:59.559020+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:09:42.185237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"81'5","reported_seq":39,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.312450+0000","last_change":"2024-09-12T02:56:31.312450+0000","last_active":"2024-09-12T02:56:31.312450+0000","last_peered":"2024-09-12T02:56:31.312450+0000","last_clean":"2024-09-12T02:56:31.312450+0000","last_became_active":"2024-09-12T02:56:31.308030+0000","last_became_peered":"2024-09-12T02:56:31.308030+0000","last_unstale":"2024-09-12T02:56:31.312450+0000","last_undegraded":"2024-09-12T02:56:31.312450+0000","last_fullsized":"2024-09-12T02:56:31.312450+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:15:14.692292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.562640+0000","last_change":"2024-09-12T02:56:15.978996+0000","last_active":"2024-09-12T02:57:59.562640+0000","last_peered":"2024-09-12T02:57:59.562640+0000","last_clean":"2024-09-12T02:57:59.562640+0000","last_became_active":"2024-09-12T02:56:15.978636+0000","last_became_peered":"2024-09-12T02:56:15.978636+0000","last_unstale":"2024-09-12T02:57:59.562640+0000","last_undegraded":"2024-09-12T02:57:59.562640+0000","last_fullsized":"2024-09-12T02:57:59.562640+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:29:20.401664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"81'3","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.033119+0000","last_change":"2024-09-12T02:56:13.929388+0000","last_active":"2024-09-12T02:56:31.033119+0000","last_peered":"2024-09-12T02:56:31.033119+0000","last_clean":"2024-09-12T02:56:31.033119+0000","last_became_active":"2024-09-12T02:56:13.928998+0000","last_became_peered":"2024-09-12T02:56:13.928998+0000","last_unstale":"2024-09-12T02:56:31.033119+0000","last_undegraded":"2024-09-12T02:56:31.033119+0000","last_fullsized":"2024-09-12T02:56:31.033119+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:37:57.368050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.560190+0000","last_change":"2024-09-12T02:56:15.977491+0000","last_active":"2024-09-12T02:57:59.560190+0000","last_peered":"2024-09-12T02:57:59.560190+0000","last_clean":"2024-09-12T02:57:59.560190+0000","last_became_active":"2024-09-12T02:56:15.977160+0000","last_became_peered":"2024-09-12T02:56:15.977160+0000","last_unstale":"2024-09-12T02:57:59.560190+0000","last_undegraded":"2024-09-12T02:57:59.560190+0000","last_fullsized":"2024-09-12T02:57:59.560190+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:05:40.577342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'7","reported_seq":133,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:58:05.983433+0000","last_change":"2024-09-12T02:56:14.327147+0000","last_active":"2024-09-12T02:58:05.983433+0000","last_peered":"2024-09-12T02:58:05.983433+0000","last_clean":"2024-09-12T02:58:05.983433+0000","last_became_active":"2024-09-12T02:56:14.326737+0000","last_became_peered":"2024-09-12T02:56:14.326737+0000","last_unstale":"2024-09-12T02:58:05.983433+0000","last_undegraded":"2024-09-12T02:58:05.983433+0000","last_fullsized":"2024-09-12T02:58:05.983433+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:55:33.510599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1704,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":104,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.563684+0000","last_change":"2024-09-12T02:56:15.984363+0000","last_active":"2024-09-12T02:57:59.563684+0000","last_peered":"2024-09-12T02:57:59.563684+0000","last_clean":"2024-09-12T02:57:59.563684+0000","last_became_active":"2024-09-12T02:56:15.983822+0000","last_became_peered":"2024-09-12T02:56:15.983822+0000","last_unstale":"2024-09-12T02:57:59.563684+0000","last_undegraded":"2024-09-12T02:57:59.563684+0000","last_fullsized":"2024-09-12T02:57:59.563684+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:41:46.082716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":333,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.035564+0000","last_change":"2024-09-12T02:55:05.800210+0000","last_active":"2024-09-12T02:56:31.035564+0000","last_peered":"2024-09-12T02:56:31.035564+0000","last_clean":"2024-09-12T02:56:31.035564+0000","last_became_active":"2024-09-12T02:55:05.799791+0000","last_became_peered":"2024-09-12T02:55:05.799791+0000","last_unstale":"2024-09-12T02:56:31.035564+0000","last_undegraded":"2024-09-12T02:56:31.035564+0000","last_fullsized":"2024-09-12T02:56:31.035564+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:51:48.530016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:51:48.530016+0000","last_clean_scrub_stamp":"2024-09-12T02:51:48.530016+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:05:47.416697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"81'6","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.031169+0000","last_change":"2024-09-12T02:56:13.930269+0000","last_active":"2024-09-12T02:56:31.031169+0000","last_peered":"2024-09-12T02:56:31.031169+0000","last_clean":"2024-09-12T02:56:31.031169+0000","last_became_active":"2024-09-12T02:56:13.929831+0000","last_became_peered":"2024-09-12T02:56:13.929831+0000","last_unstale":"2024-09-12T02:56:31.031169+0000","last_undegraded":"2024-09-12T02:56:31.031169+0000","last_fullsized":"2024-09-12T02:56:31.031169+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:38:17.878398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.561332+0000","last_change":"2024-09-12T02:56:15.969744+0000","last_active":"2024-09-12T02:57:59.561332+0000","last_peered":"2024-09-12T02:57:59.561332+0000","last_clean":"2024-09-12T02:57:59.561332+0000","last_became_active":"2024-09-12T02:56:15.969354+0000","last_became_peered":"2024-09-12T02:56:15.969354+0000","last_unstale":"2024-09-12T02:57:59.561332+0000","last_undegraded":"2024-09-12T02:57:59.561332+0000","last_fullsized":"2024-09-12T02:57:59.561332+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:26:02.409510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"81'4","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.035385+0000","last_change":"2024-09-12T02:56:13.929231+0000","last_active":"2024-09-12T02:56:31.035385+0000","last_peered":"2024-09-12T02:56:31.035385+0000","last_clean":"2024-09-12T02:56:31.035385+0000","last_became_active":"2024-09-12T02:56:13.928843+0000","last_became_peered":"2024-09-12T02:56:13.928843+0000","last_unstale":"2024-09-12T02:56:31.035385+0000","last_undegraded":"2024-09-12T02:56:31.035385+0000","last_fullsized":"2024-09-12T02:56:31.035385+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:26:33.619755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"83'5","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.555927+0000","last_change":"2024-09-12T02:56:15.982503+0000","last_active":"2024-09-12T02:57:59.555927+0000","last_peered":"2024-09-12T02:57:59.555927+0000","last_clean":"2024-09-12T02:57:59.555927+0000","last_became_active":"2024-09-12T02:56:15.982192+0000","last_became_peered":"2024-09-12T02:56:15.982192+0000","last_unstale":"2024-09-12T02:57:59.555927+0000","last_undegraded":"2024-09-12T02:57:59.555927+0000","last_fullsized":"2024-09-12T02:57:59.555927+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:55:18.370967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"83'6","reported_seq":132,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:58:08.986599+0000","last_change":"2024-09-12T02:56:14.075486+0000","last_active":"2024-09-12T02:58:08.986599+0000","last_peered":"2024-09-12T02:58:08.986599+0000","last_clean":"2024-09-12T02:58:08.986599+0000","last_became_active":"2024-09-12T02:56:14.075152+0000","last_became_peered":"2024-09-12T02:56:14.075152+0000","last_unstale":"2024-09-12T02:58:08.986599+0000","last_undegraded":"2024-09-12T02:58:08.986599+0000","last_fullsized":"2024-09-12T02:58:08.986599+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:59:40.950191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":104,"num_read_kb":104,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.556340+0000","last_change":"2024-09-12T02:56:15.981584+0000","last_active":"2024-09-12T02:57:59.556340+0000","last_peered":"2024-09-12T02:57:59.556340+0000","last_clean":"2024-09-12T02:57:59.556340+0000","last_became_active":"2024-09-12T02:56:15.981240+0000","last_became_peered":"2024-09-12T02:56:15.981240+0000","last_unstale":"2024-09-12T02:57:59.556340+0000","last_undegraded":"2024-09-12T02:57:59.556340+0000","last_fullsized":"2024-09-12T02:57:59.556340+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:49:04.036830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"83'7","reported_seq":134,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:58:07.985338+0000","last_change":"2024-09-12T02:56:13.935143+0000","last_active":"2024-09-12T02:58:07.985338+0000","last_peered":"2024-09-12T02:58:07.985338+0000","last_clean":"2024-09-12T02:58:07.985338+0000","last_became_active":"2024-09-12T02:56:13.934792+0000","last_became_peered":"2024-09-12T02:56:13.934792+0000","last_unstale":"2024-09-12T02:58:07.985338+0000","last_undegraded":"2024-09-12T02:58:07.985338+0000","last_fullsized":"2024-09-12T02:58:07.985338+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:00:26.973612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1289,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":105,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"83'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.558893+0000","last_change":"2024-09-12T02:56:31.034825+0000","last_active":"2024-09-12T02:57:59.558893+0000","last_peered":"2024-09-12T02:57:59.558893+0000","last_clean":"2024-09-12T02:57:59.558893+0000","last_became_active":"2024-09-12T02:56:31.034516+0000","last_became_peered":"2024-09-12T02:56:31.034516+0000","last_unstale":"2024-09-12T02:57:59.558893+0000","last_undegraded":"2024-09-12T02:57:59.558893+0000","last_fullsized":"2024-09-12T02:57:59.558893+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:10:42.161023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"81'1","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.029027+0000","last_change":"2024-09-12T02:56:14.076762+0000","last_active":"2024-09-12T02:56:31.029027+0000","last_peered":"2024-09-12T02:56:31.029027+0000","last_clean":"2024-09-12T02:56:31.029027+0000","last_became_active":"2024-09-12T02:56:14.076421+0000","last_became_peered":"2024-09-12T02:56:14.076421+0000","last_unstale":"2024-09-12T02:56:31.029027+0000","last_undegraded":"2024-09-12T02:56:31.029027+0000","last_fullsized":"2024-09-12T02:56:31.029027+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:56:56.422910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'5","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.555835+0000","last_change":"2024-09-12T02:56:15.978810+0000","last_active":"2024-09-12T02:57:59.555835+0000","last_peered":"2024-09-12T02:57:59.555835+0000","last_clean":"2024-09-12T02:57:59.555835+0000","last_became_active":"2024-09-12T02:56:15.978038+0000","last_became_peered":"2024-09-12T02:56:15.978038+0000","last_unstale":"2024-09-12T02:57:59.555835+0000","last_undegraded":"2024-09-12T02:57:59.555835+0000","last_fullsized":"2024-09-12T02:57:59.555835+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:04:52.215784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"81'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.035440+0000","last_change":"2024-09-12T02:56:14.075987+0000","last_active":"2024-09-12T02:56:31.035440+0000","last_peered":"2024-09-12T02:56:31.035440+0000","last_clean":"2024-09-12T02:56:31.035440+0000","last_became_active":"2024-09-12T02:56:14.075596+0000","last_became_peered":"2024-09-12T02:56:14.075596+0000","last_unstale":"2024-09-12T02:56:31.035440+0000","last_undegraded":"2024-09-12T02:56:31.035440+0000","last_fullsized":"2024-09-12T02:56:31.035440+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:18:48.713387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.552400+0000","last_change":"2024-09-12T02:56:15.987842+0000","last_active":"2024-09-12T02:57:59.552400+0000","last_peered":"2024-09-12T02:57:59.552400+0000","last_clean":"2024-09-12T02:57:59.552400+0000","last_became_active":"2024-09-12T02:56:15.987507+0000","last_became_peered":"2024-09-12T02:56:15.987507+0000","last_unstale":"2024-09-12T02:57:59.552400+0000","last_undegraded":"2024-09-12T02:57:59.552400+0000","last_fullsized":"2024-09-12T02:57:59.552400+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:52:23.000549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'19","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:58:01.991961+0000","last_change":"2024-09-12T02:56:13.924372+0000","last_active":"2024-09-12T02:58:01.991961+0000","last_peered":"2024-09-12T02:58:01.991961+0000","last_clean":"2024-09-12T02:58:01.991961+0000","last_became_active":"2024-09-12T02:56:13.923955+0000","last_became_peered":"2024-09-12T02:56:13.923955+0000","last_unstale":"2024-09-12T02:58:01.991961+0000","last_undegraded":"2024-09-12T02:58:01.991961+0000","last_fullsized":"2024-09-12T02:58:01.991961+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:52:47.452410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":34,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"83'5","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.562919+0000","last_change":"2024-09-12T02:56:15.978262+0000","last_active":"2024-09-12T02:57:59.562919+0000","last_peered":"2024-09-12T02:57:59.562919+0000","last_clean":"2024-09-12T02:57:59.562919+0000","last_became_active":"2024-09-12T02:56:15.977913+0000","last_became_peered":"2024-09-12T02:56:15.977913+0000","last_unstale":"2024-09-12T02:57:59.562919+0000","last_undegraded":"2024-09-12T02:57:59.562919+0000","last_fullsized":"2024-09-12T02:57:59.562919+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:35:27.104635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"81'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.031488+0000","last_change":"2024-09-12T02:56:14.327417+0000","last_active":"2024-09-12T02:56:31.031488+0000","last_peered":"2024-09-12T02:56:31.031488+0000","last_clean":"2024-09-12T02:56:31.031488+0000","last_became_active":"2024-09-12T02:56:14.327001+0000","last_became_peered":"2024-09-12T02:56:14.327001+0000","last_unstale":"2024-09-12T02:56:31.031488+0000","last_undegraded":"2024-09-12T02:56:31.031488+0000","last_fullsized":"2024-09-12T02:56:31.031488+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:15:39.019252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.565859+0000","last_change":"2024-09-12T02:56:15.977286+0000","last_active":"2024-09-12T02:57:59.565859+0000","last_peered":"2024-09-12T02:57:59.565859+0000","last_clean":"2024-09-12T02:57:59.565859+0000","last_became_active":"2024-09-12T02:56:15.976944+0000","last_became_peered":"2024-09-12T02:56:15.976944+0000","last_unstale":"2024-09-12T02:57:59.565859+0000","last_undegraded":"2024-09-12T02:57:59.565859+0000","last_fullsized":"2024-09-12T02:57:59.565859+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:22:57.056781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.306212+0000","last_change":"2024-09-12T02:56:14.075701+0000","last_active":"2024-09-12T02:56:31.306212+0000","last_peered":"2024-09-12T02:56:31.306212+0000","last_clean":"2024-09-12T02:56:31.306212+0000","last_became_active":"2024-09-12T02:56:14.075340+0000","last_became_peered":"2024-09-12T02:56:14.075340+0000","last_unstale":"2024-09-12T02:56:31.306212+0000","last_undegraded":"2024-09-12T02:56:31.306212+0000","last_fullsized":"2024-09-12T02:56:31.306212+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:17:42.255058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"83'2","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.528187+0000","last_change":"2024-09-12T02:56:15.979783+0000","last_active":"2024-09-12T02:57:59.528187+0000","last_peered":"2024-09-12T02:57:59.528187+0000","last_clean":"2024-09-12T02:57:59.528187+0000","last_became_active":"2024-09-12T02:56:15.979424+0000","last_became_peered":"2024-09-12T02:56:15.979424+0000","last_unstale":"2024-09-12T02:57:59.528187+0000","last_undegraded":"2024-09-12T02:57:59.528187+0000","last_fullsized":"2024-09-12T02:57:59.528187+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:59:11.568138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"81'7","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.034660+0000","last_change":"2024-09-12T02:56:13.928996+0000","last_active":"2024-09-12T02:56:31.034660+0000","last_peered":"2024-09-12T02:56:31.034660+0000","last_clean":"2024-09-12T02:56:31.034660+0000","last_became_active":"2024-09-12T02:56:13.928664+0000","last_became_peered":"2024-09-12T02:56:13.928664+0000","last_unstale":"2024-09-12T02:56:31.034660+0000","last_undegraded":"2024-09-12T02:56:31.034660+0000","last_fullsized":"2024-09-12T02:56:31.034660+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:37:22.067930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.560217+0000","last_change":"2024-09-12T02:56:15.984096+0000","last_active":"2024-09-12T02:57:59.560217+0000","last_peered":"2024-09-12T02:57:59.560217+0000","last_clean":"2024-09-12T02:57:59.560217+0000","last_became_active":"2024-09-12T02:56:15.983737+0000","last_became_peered":"2024-09-12T02:56:15.983737+0000","last_unstale":"2024-09-12T02:57:59.560217+0000","last_undegraded":"2024-09-12T02:57:59.560217+0000","last_fullsized":"2024-09-12T02:57:59.560217+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:02:17.892318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'4","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:33.042035+0000","last_change":"2024-09-12T02:56:31.053343+0000","last_active":"2024-09-12T02:56:33.042035+0000","last_peered":"2024-09-12T02:56:33.042035+0000","last_clean":"2024-09-12T02:56:33.042035+0000","last_became_active":"2024-09-12T02:56:31.037918+0000","last_became_peered":"2024-09-12T02:56:31.037918+0000","last_unstale":"2024-09-12T02:56:33.042035+0000","last_undegraded":"2024-09-12T02:56:33.042035+0000","last_fullsized":"2024-09-12T02:56:33.042035+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:39:54.990724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.563189+0000","last_change":"2024-09-12T02:56:15.975640+0000","last_active":"2024-09-12T02:57:59.563189+0000","last_peered":"2024-09-12T02:57:59.563189+0000","last_clean":"2024-09-12T02:57:59.563189+0000","last_became_active":"2024-09-12T02:56:15.975342+0000","last_became_peered":"2024-09-12T02:56:15.975342+0000","last_unstale":"2024-09-12T02:57:59.563189+0000","last_undegraded":"2024-09-12T02:57:59.563189+0000","last_fullsized":"2024-09-12T02:57:59.563189+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:26:23.694876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.034922+0000","last_change":"2024-09-12T02:56:13.934149+0000","last_active":"2024-09-12T02:56:31.034922+0000","last_peered":"2024-09-12T02:56:31.034922+0000","last_clean":"2024-09-12T02:56:31.034922+0000","last_became_active":"2024-09-12T02:56:13.933743+0000","last_became_peered":"2024-09-12T02:56:13.933743+0000","last_unstale":"2024-09-12T02:56:31.034922+0000","last_undegraded":"2024-09-12T02:56:31.034922+0000","last_fullsized":"2024-09-12T02:56:31.034922+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:16:17.854457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"83'5","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.563348+0000","last_change":"2024-09-12T02:56:15.984782+0000","last_active":"2024-09-12T02:57:59.563348+0000","last_peered":"2024-09-12T02:57:59.563348+0000","last_clean":"2024-09-12T02:57:59.563348+0000","last_became_active":"2024-09-12T02:56:15.984336+0000","last_became_peered":"2024-09-12T02:56:15.984336+0000","last_unstale":"2024-09-12T02:57:59.563348+0000","last_undegraded":"2024-09-12T02:57:59.563348+0000","last_fullsized":"2024-09-12T02:57:59.563348+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:03:39.832897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"83'15","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.564252+0000","last_change":"2024-09-12T02:56:15.985486+0000","last_active":"2024-09-12T02:57:59.564252+0000","last_peered":"2024-09-12T02:57:59.564252+0000","last_clean":"2024-09-12T02:57:59.564252+0000","last_became_active":"2024-09-12T02:56:15.985167+0000","last_became_peered":"2024-09-12T02:56:15.985167+0000","last_unstale":"2024-09-12T02:57:59.564252+0000","last_undegraded":"2024-09-12T02:57:59.564252+0000","last_fullsized":"2024-09-12T02:57:59.564252+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:46:20.569453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"81'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.035106+0000","last_change":"2024-09-12T02:56:13.929324+0000","last_active":"2024-09-12T02:56:31.035106+0000","last_peered":"2024-09-12T02:56:31.035106+0000","last_clean":"2024-09-12T02:56:31.035106+0000","last_became_active":"2024-09-12T02:56:13.928994+0000","last_became_peered":"2024-09-12T02:56:13.928994+0000","last_unstale":"2024-09-12T02:56:31.035106+0000","last_undegraded":"2024-09-12T02:56:31.035106+0000","last_fullsized":"2024-09-12T02:56:31.035106+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:30:27.290328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.562582+0000","last_change":"2024-09-12T02:56:15.985889+0000","last_active":"2024-09-12T02:57:59.562582+0000","last_peered":"2024-09-12T02:57:59.562582+0000","last_clean":"2024-09-12T02:57:59.562582+0000","last_became_active":"2024-09-12T02:56:15.985509+0000","last_became_peered":"2024-09-12T02:56:15.985509+0000","last_unstale":"2024-09-12T02:57:59.562582+0000","last_undegraded":"2024-09-12T02:57:59.562582+0000","last_fullsized":"2024-09-12T02:57:59.562582+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:30:43.120718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"83'13","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.035437+0000","last_change":"2024-09-12T02:56:13.934192+0000","last_active":"2024-09-12T02:56:31.035437+0000","last_peered":"2024-09-12T02:56:31.035437+0000","last_clean":"2024-09-12T02:56:31.035437+0000","last_became_active":"2024-09-12T02:56:13.933789+0000","last_became_peered":"2024-09-12T02:56:13.933789+0000","last_unstale":"2024-09-12T02:56:31.035437+0000","last_undegraded":"2024-09-12T02:56:31.035437+0000","last_fullsized":"2024-09-12T02:56:31.035437+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:32:47.771085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.563285+0000","last_change":"2024-09-12T02:56:15.983531+0000","last_active":"2024-09-12T02:57:59.563285+0000","last_peered":"2024-09-12T02:57:59.563285+0000","last_clean":"2024-09-12T02:57:59.563285+0000","last_became_active":"2024-09-12T02:56:15.983192+0000","last_became_peered":"2024-09-12T02:56:15.983192+0000","last_unstale":"2024-09-12T02:57:59.563285+0000","last_undegraded":"2024-09-12T02:57:59.563285+0000","last_fullsized":"2024-09-12T02:57:59.563285+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:16:59.923379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.034752+0000","last_change":"2024-09-12T02:56:14.075396+0000","last_active":"2024-09-12T02:56:31.034752+0000","last_peered":"2024-09-12T02:56:31.034752+0000","last_clean":"2024-09-12T02:56:31.034752+0000","last_became_active":"2024-09-12T02:56:14.075023+0000","last_became_peered":"2024-09-12T02:56:14.075023+0000","last_unstale":"2024-09-12T02:56:31.034752+0000","last_undegraded":"2024-09-12T02:56:31.034752+0000","last_fullsized":"2024-09-12T02:56:31.034752+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:41:45.760689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"83'5","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.564093+0000","last_change":"2024-09-12T02:56:15.987276+0000","last_active":"2024-09-12T02:57:59.564093+0000","last_peered":"2024-09-12T02:57:59.564093+0000","last_clean":"2024-09-12T02:57:59.564093+0000","last_became_active":"2024-09-12T02:56:15.986949+0000","last_became_peered":"2024-09-12T02:56:15.986949+0000","last_unstale":"2024-09-12T02:57:59.564093+0000","last_undegraded":"2024-09-12T02:57:59.564093+0000","last_fullsized":"2024-09-12T02:57:59.564093+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:49:45.627093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"83'14","reported_seq":139,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:58:07.984885+0000","last_change":"2024-09-12T02:56:13.932365+0000","last_active":"2024-09-12T02:58:07.984885+0000","last_peered":"2024-09-12T02:58:07.984885+0000","last_clean":"2024-09-12T02:58:07.984885+0000","last_became_active":"2024-09-12T02:56:13.932062+0000","last_became_peered":"2024-09-12T02:56:13.932062+0000","last_unstale":"2024-09-12T02:58:07.984885+0000","last_undegraded":"2024-09-12T02:58:07.984885+0000","last_fullsized":"2024-09-12T02:58:07.984885+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:17:08.451839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":103,"num_read_kb":103,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":77928,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1059,"num_read_kb":613,"num_write":273,"num_write_kb":162,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":734,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":589824,"data_stored":228744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":233,"ondisk_log_size":233,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547236,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":917504,"data_stored":440831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776168,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32764,"kb_used_data":1580,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712388,"statfs":{"total":95995035648,"available":95961485312,"internally_reserved":0,"allocated":1617920,"data_stored":1098572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939694,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40416,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704736,"statfs":{"total":95995035648,"available":95953649664,"internally_reserved":0,"allocated":933888,"data_stored":440115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":40434112},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168627,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":921600,"data_stored":440381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430263,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28516,"kb_used_data":1492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716636,"statfs":{"total":95995035648,"available":95965835264,"internally_reserved":0,"allocated":1527808,"data_stored":1034641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593789,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":917504,"data_stored":440533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822720,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28528,"kb_used_data":1504,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716624,"statfs":{"total":95995035648,"available":95965822976,"internally_reserved":0,"allocated":1540096,"data_stored":1032691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953543,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":925696,"data_stored":441635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182474,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":933888,"data_stored":439657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411408,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36320,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708832,"statfs":{"total":95995035648,"available":95957843968,"internally_reserved":0,"allocated":999424,"data_stored":507308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574932,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":987136,"data_stored":507877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738459,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":937984,"data_stored":441705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":68884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":68315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":71614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T02:58:12.986 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-12T02:58:13.352 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T02:58:13.417 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:58:13.576 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:58:13 smithi005 ceph-mon[31197]: pgmap v334: 97 pgs: 97 active+clean; 653 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.2 KiB/s wr, 12 op/s 2024-09-12T02:58:13.576 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:58:13 smithi005 ceph-mon[31197]: from='client.24931 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:58:13.576 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:58:13 smithi005 ceph-mon[31197]: scrub queued for path: / 2024-09-12T02:58:13.576 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:58:13 smithi005 ceph-mon[31197]: scrub summary: idle+waiting paths [/] 2024-09-12T02:58:13.576 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:58:13 smithi005 ceph-mon[31197]: scrub summary: active paths [/] 2024-09-12T02:58:13.576 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:58:13 smithi005 ceph-mon[31197]: scrub summary: idle+waiting paths [/] 2024-09-12T02:58:13.665 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:58:13 smithi148 ceph-mon[35090]: pgmap v334: 97 pgs: 97 active+clean; 653 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.2 KiB/s wr, 12 op/s 2024-09-12T02:58:13.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:58:13 smithi148 ceph-mon[35090]: from='client.24931 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:58:13.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:58:13 smithi148 ceph-mon[35090]: scrub queued for path: / 2024-09-12T02:58:13.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:58:13 smithi148 ceph-mon[35090]: scrub summary: idle+waiting paths [/] 2024-09-12T02:58:13.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:58:13 smithi148 ceph-mon[35090]: scrub summary: active paths [/] 2024-09-12T02:58:13.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:58:13 smithi148 ceph-mon[35090]: scrub summary: idle+waiting paths [/] 2024-09-12T02:58:13.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:58:13 smithi146 ceph-mon[35011]: pgmap v334: 97 pgs: 97 active+clean; 653 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.2 KiB/s wr, 12 op/s 2024-09-12T02:58:13.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:58:13 smithi146 ceph-mon[35011]: from='client.24931 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:58:13.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:58:13 smithi146 ceph-mon[35011]: scrub queued for path: / 2024-09-12T02:58:13.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:58:13 smithi146 ceph-mon[35011]: scrub summary: idle+waiting paths [/] 2024-09-12T02:58:13.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:58:13 smithi146 ceph-mon[35011]: scrub summary: active paths [/] 2024-09-12T02:58:13.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:58:13 smithi146 ceph-mon[35011]: scrub summary: idle+waiting paths [/] 2024-09-12T02:58:13.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:13.901+0000 7f8c0b101640 1 -- 172.21.15.5:0/1363658048 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c04102ef0 msgr2=0x7f8c0410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:13.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:13.901+0000 7f8c0b101640 1 --2- 172.21.15.5:0/1363658048 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c04102ef0 0x7f8c0410da40 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc009ca0 tx=0x7f8bfc02a950 comp rx=0 tx=0).stop 2024-09-12T02:58:13.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:13.901+0000 7f8c0b101640 1 -- 172.21.15.5:0/1363658048 shutdown_connections 2024-09-12T02:58:13.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:13.901+0000 7f8c0b101640 1 --2- 172.21.15.5:0/1363658048 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c04102ef0 0x7f8c0410da40 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:13.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:13.901+0000 7f8c0b101640 1 --2- 172.21.15.5:0/1363658048 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c04102570 0x7f8c041029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:13.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:13.901+0000 7f8c0b101640 1 --2- 172.21.15.5:0/1363658048 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c04100e20 0x7f8c04101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:13.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:13.901+0000 7f8c0b101640 1 -- 172.21.15.5:0/1363658048 >> 172.21.15.5:0/1363658048 conn(0x7f8c040fc7d0 msgr2=0x7f8c040febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:13.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:13.901+0000 7f8c0b101640 1 -- 172.21.15.5:0/1363658048 shutdown_connections 2024-09-12T02:58:13.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:13.901+0000 7f8c0b101640 1 -- 172.21.15.5:0/1363658048 wait complete. 2024-09-12T02:58:13.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:13.901+0000 7f8c0b101640 1 Processor -- start 2024-09-12T02:58:13.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:13.901+0000 7f8c0b101640 1 -- start start 2024-09-12T02:58:13.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:13.902+0000 7f8c0b101640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c04100e20 0x7f8c0410e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:13.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:13.902+0000 7f8c0b101640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c04102570 0x7f8c0410ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:13.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:13.902+0000 7f8c0b101640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c04102ef0 0x7f8c0410ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:13.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:13.902+0000 7f8c0b101640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8c041120a0 con 0x7f8c04100e20 2024-09-12T02:58:13.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:13.902+0000 7f8c0b101640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f8c04111f20 con 0x7f8c04102ef0 2024-09-12T02:58:13.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:13.902+0000 7f8c0b101640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8c04112220 con 0x7f8c04102570 2024-09-12T02:58:13.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:13.902+0000 7f8c08e9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c04100e20 0x7f8c0410e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:13.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:13.902+0000 7f8c08e9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c04100e20 0x7f8c0410e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50512/0 (socket says 172.21.15.5:50512) 2024-09-12T02:58:13.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:13.902+0000 7f8c08e9d640 1 -- 172.21.15.5:0/931262372 learned_addr learned my addr 172.21.15.5:0/931262372 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:58:13.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:13.902+0000 7f8c03fff640 1 --2- 172.21.15.5:0/931262372 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c04102570 0x7f8c0410ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:13.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:13.902+0000 7f8c0969e640 1 --2- 172.21.15.5:0/931262372 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c04102ef0 0x7f8c0410ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:13.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:13.902+0000 7f8c08e9d640 1 -- 172.21.15.5:0/931262372 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c04102570 msgr2=0x7f8c0410ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:13.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:13.902+0000 7f8c08e9d640 1 --2- 172.21.15.5:0/931262372 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c04102570 0x7f8c0410ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:13.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:13.902+0000 7f8c08e9d640 1 -- 172.21.15.5:0/931262372 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c04102ef0 msgr2=0x7f8c0410ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:13.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:13.902+0000 7f8c08e9d640 1 --2- 172.21.15.5:0/931262372 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c04102ef0 0x7f8c0410ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:13.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:13.902+0000 7f8c08e9d640 1 -- 172.21.15.5:0/931262372 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c041a6940 con 0x7f8c04100e20 2024-09-12T02:58:13.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:13.902+0000 7f8c0969e640 1 --2- 172.21.15.5:0/931262372 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c04102ef0 0x7f8c0410ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:58:13.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:13.902+0000 7f8c08e9d640 1 --2- 172.21.15.5:0/931262372 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c04100e20 0x7f8c0410e4d0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f8bf400d770 tx=0x7f8bf400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:13.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:13.903+0000 7f8c01ffb640 1 -- 172.21.15.5:0/931262372 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bf4017070 con 0x7f8c04100e20 2024-09-12T02:58:13.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:13.903+0000 7f8c01ffb640 1 -- 172.21.15.5:0/931262372 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8bf4005500 con 0x7f8c04100e20 2024-09-12T02:58:13.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:13.903+0000 7f8c0b101640 1 -- 172.21.15.5:0/931262372 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c041a6c30 con 0x7f8c04100e20 2024-09-12T02:58:13.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:13.903+0000 7f8c01ffb640 1 -- 172.21.15.5:0/931262372 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bf4004e00 con 0x7f8c04100e20 2024-09-12T02:58:13.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:13.903+0000 7f8c0b101640 1 -- 172.21.15.5:0/931262372 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c041a7120 con 0x7f8c04100e20 2024-09-12T02:58:13.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:13.904+0000 7f8c0b101640 1 -- 172.21.15.5:0/931262372 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c04101200 con 0x7f8c04100e20 2024-09-12T02:58:13.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:13.905+0000 7f8c01ffb640 1 -- 172.21.15.5:0/931262372 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7f8bf4004fa0 con 0x7f8c04100e20 2024-09-12T02:58:13.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:13.906+0000 7f8c01ffb640 1 --2- 172.21.15.5:0/931262372 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f8bec0a2c10 0x7f8bec0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:13.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:13.906+0000 7f8c03fff640 1 --2- 172.21.15.5:0/931262372 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f8bec0a2c10 0x7f8bec0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:13.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:13.906+0000 7f8c01ffb640 1 -- 172.21.15.5:0/931262372 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f8bf40cec40 con 0x7f8c04100e20 2024-09-12T02:58:13.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:13.906+0000 7f8c03fff640 1 --2- 172.21.15.5:0/931262372 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f8bec0a2c10 0x7f8bec0a50d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8c0410ff60 tx=0x7f8bf800b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:13.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:13.910+0000 7f8c01ffb640 1 -- 172.21.15.5:0/931262372 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8bf40d5080 con 0x7f8c04100e20 2024-09-12T02:58:13.964 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:58:14.107 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.106+0000 7f8c0b101640 1 -- 172.21.15.5:0/931262372 --> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8c04106f30 con 0x7f8bec0a2c10 2024-09-12T02:58:14.113 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.113+0000 7f8c01ffb640 1 -- 172.21.15.5:0/931262372 <== mgr.14148 v2:172.21.15.5:6800/3780631642 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258894 (secure 0 0 0) 0x7f8c04106f30 con 0x7f8bec0a2c10 2024-09-12T02:58:14.115 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-12T02:58:14.116 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-09-12T02:58:14.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.116+0000 7f8c0b101640 1 -- 172.21.15.5:0/931262372 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f8bec0a2c10 msgr2=0x7f8bec0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:14.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.116+0000 7f8c0b101640 1 --2- 172.21.15.5:0/931262372 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f8bec0a2c10 0x7f8bec0a50d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8c0410ff60 tx=0x7f8bf800b390 comp rx=0 tx=0).stop 2024-09-12T02:58:14.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.116+0000 7f8c0b101640 1 -- 172.21.15.5:0/931262372 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c04100e20 msgr2=0x7f8c0410e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:14.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.116+0000 7f8c0b101640 1 --2- 172.21.15.5:0/931262372 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c04100e20 0x7f8c0410e4d0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f8bf400d770 tx=0x7f8bf400dc30 comp rx=0 tx=0).stop 2024-09-12T02:58:14.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.116+0000 7f8c0b101640 1 -- 172.21.15.5:0/931262372 shutdown_connections 2024-09-12T02:58:14.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.116+0000 7f8c0b101640 1 --2- 172.21.15.5:0/931262372 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8c04102ef0 0x7f8c0410ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:14.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.116+0000 7f8c0b101640 1 --2- 172.21.15.5:0/931262372 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8c04102570 0x7f8c0410ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:14.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.116+0000 7f8c0b101640 1 --2- 172.21.15.5:0/931262372 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f8bec0a2c10 0x7f8bec0a50d0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:14.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.116+0000 7f8c0b101640 1 --2- 172.21.15.5:0/931262372 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c04100e20 0x7f8c0410e4d0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:14.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.116+0000 7f8c0b101640 1 -- 172.21.15.5:0/931262372 >> 172.21.15.5:0/931262372 conn(0x7f8c040fc7d0 msgr2=0x7f8c04106820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:14.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.116+0000 7f8c0b101640 1 -- 172.21.15.5:0/931262372 shutdown_connections 2024-09-12T02:58:14.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.116+0000 7f8c0b101640 1 -- 172.21.15.5:0/931262372 wait complete. 2024-09-12T02:58:14.577 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.577+0000 7f44256ab640 1 -- 172.21.15.5:0/856629505 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44201082a0 msgr2=0x7f442010ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:15.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.577+0000 7f44256ab640 1 --2- 172.21.15.5:0/856629505 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44201082a0 0x7f442010ec40 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f4418009cd0 tx=0x7f441802a8e0 comp rx=0 tx=0).stop 2024-09-12T02:58:15.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.577+0000 7f44256ab640 1 -- 172.21.15.5:0/856629505 shutdown_connections 2024-09-12T02:58:15.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.577+0000 7f44256ab640 1 --2- 172.21.15.5:0/856629505 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44201082a0 0x7f442010ec40 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:15.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.577+0000 7f44256ab640 1 --2- 172.21.15.5:0/856629505 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44201036e0 0x7f4420107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:15.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.577+0000 7f44256ab640 1 --2- 172.21.15.5:0/856629505 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4420102d30 0x7f4420103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:15.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.577+0000 7f44256ab640 1 -- 172.21.15.5:0/856629505 >> 172.21.15.5:0/856629505 conn(0x7f44200fc7d0 msgr2=0x7f44200febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:15.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.577+0000 7f44256ab640 1 -- 172.21.15.5:0/856629505 shutdown_connections 2024-09-12T02:58:15.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.577+0000 7f44256ab640 1 -- 172.21.15.5:0/856629505 wait complete. 2024-09-12T02:58:15.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.578+0000 7f44256ab640 1 Processor -- start 2024-09-12T02:58:15.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.578+0000 7f44256ab640 1 -- start start 2024-09-12T02:58:15.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.578+0000 7f44256ab640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4420102d30 0x7f4420198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:15.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.578+0000 7f44256ab640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44201036e0 0x7f4420198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:15.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.578+0000 7f44256ab640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44201082a0 0x7f442019ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:15.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.578+0000 7f44256ab640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f44201120a0 con 0x7f44201082a0 2024-09-12T02:58:15.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.578+0000 7f44256ab640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f4420111f20 con 0x7f4420102d30 2024-09-12T02:58:15.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.578+0000 7f44256ab640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4420112220 con 0x7f44201036e0 2024-09-12T02:58:15.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.578+0000 7f441f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44201082a0 0x7f442019ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:15.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.578+0000 7f441f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44201082a0 0x7f442019ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50540/0 (socket says 172.21.15.5:50540) 2024-09-12T02:58:15.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.578+0000 7f441f7fe640 1 -- 172.21.15.5:0/1178586952 learned_addr learned my addr 172.21.15.5:0/1178586952 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:58:15.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.578+0000 7f441effd640 1 --2- 172.21.15.5:0/1178586952 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4420102d30 0x7f4420198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:15.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.578+0000 7f441e7fc640 1 --2- 172.21.15.5:0/1178586952 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44201036e0 0x7f4420198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:15.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.579+0000 7f441f7fe640 1 -- 172.21.15.5:0/1178586952 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44201036e0 msgr2=0x7f4420198a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:15.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.579+0000 7f441f7fe640 1 --2- 172.21.15.5:0/1178586952 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44201036e0 0x7f4420198a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:15.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.579+0000 7f441f7fe640 1 -- 172.21.15.5:0/1178586952 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4420102d30 msgr2=0x7f4420198540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:15.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.579+0000 7f441f7fe640 1 --2- 172.21.15.5:0/1178586952 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4420102d30 0x7f4420198540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:15.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.579+0000 7f441f7fe640 1 -- 172.21.15.5:0/1178586952 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f442019d4f0 con 0x7f44201082a0 2024-09-12T02:58:15.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.579+0000 7f441e7fc640 1 --2- 172.21.15.5:0/1178586952 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44201036e0 0x7f4420198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:58:15.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.579+0000 7f441f7fe640 1 --2- 172.21.15.5:0/1178586952 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44201082a0 0x7f442019ce10 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f4418005cc0 tx=0x7f4418032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:15.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.579+0000 7f4407fff640 1 -- 172.21.15.5:0/1178586952 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4418004040 con 0x7f44201082a0 2024-09-12T02:58:15.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.579+0000 7f4407fff640 1 -- 172.21.15.5:0/1178586952 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44180046c0 con 0x7f44201082a0 2024-09-12T02:58:15.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.579+0000 7f44256ab640 1 -- 172.21.15.5:0/1178586952 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f442019d780 con 0x7f44201082a0 2024-09-12T02:58:15.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.579+0000 7f4407fff640 1 -- 172.21.15.5:0/1178586952 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4418030e00 con 0x7f44201082a0 2024-09-12T02:58:15.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.579+0000 7f44256ab640 1 -- 172.21.15.5:0/1178586952 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44201a50c0 con 0x7f44201082a0 2024-09-12T02:58:15.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.582+0000 7f4407fff640 1 -- 172.21.15.5:0/1178586952 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7f44180305a0 con 0x7f44201082a0 2024-09-12T02:58:15.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.582+0000 7f4427a84740 1 Processor -- start 2024-09-12T02:58:15.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.582+0000 7f4427a84740 1 -- start start 2024-09-12T02:58:15.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.582+0000 7f4427a84740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bacd4d64a0 0x55bacd4d6880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:15.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.582+0000 7f4427a84740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bacd4d6dc0 0x55bacd4e1140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:15.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.583+0000 7f4427a84740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bacd4e1680 0x55bacd4e3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:15.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.583+0000 7f4427a84740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55bacd43c2e0 con 0x55bacd4d64a0 2024-09-12T02:58:15.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.583+0000 7f4427a84740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55bacd3e3ce0 con 0x55bacd4e1680 2024-09-12T02:58:15.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.583+0000 7f4427a84740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55bacd4382a0 con 0x55bacd4d6dc0 2024-09-12T02:58:15.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.583+0000 7f441e7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bacd4d64a0 0x55bacd4d6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:15.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.583+0000 7f441e7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bacd4d64a0 0x55bacd4d6880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50554/0 (socket says 172.21.15.5:50554) 2024-09-12T02:58:15.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.583+0000 7f441e7fc640 1 -- 172.21.15.5:0/3122552844 learned_addr learned my addr 172.21.15.5:0/3122552844 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:58:15.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.583+0000 7f441e7fc640 1 --2- 172.21.15.5:0/3122552844 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bacd4e1680 0x55bacd4e3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:15.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.583+0000 7f4407fff640 1 --2- 172.21.15.5:0/1178586952 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f44080a2c10 0x7f44080a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:15.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.583+0000 7f441effd640 1 --2- 172.21.15.5:0/3122552844 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bacd4d6dc0 0x55bacd4e1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:15.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.583+0000 7f441e7fc640 1 -- 172.21.15.5:0/3122552844 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bacd4d6dc0 msgr2=0x55bacd4e1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:15.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.583+0000 7f441e7fc640 1 --2- 172.21.15.5:0/3122552844 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bacd4d6dc0 0x55bacd4e1140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:15.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.583+0000 7f441e7fc640 1 -- 172.21.15.5:0/3122552844 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bacd4e1680 msgr2=0x55bacd4e3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:15.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.583+0000 7f441e7fc640 1 --2- 172.21.15.5:0/3122552844 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bacd4e1680 0x55bacd4e3a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:15.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.583+0000 7f441e7fc640 1 -- 172.21.15.5:0/3122552844 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bacd3c8440 con 0x55bacd4d64a0 2024-09-12T02:58:15.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.583+0000 7f441f7fe640 1 --2- 172.21.15.5:0/1178586952 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f44080a2c10 0x7f44080a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:15.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.583+0000 7f4407fff640 1 -- 172.21.15.5:0/1178586952 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f441802e030 con 0x7f44201082a0 2024-09-12T02:58:15.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.583+0000 7f441e7fc640 1 --2- 172.21.15.5:0/3122552844 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bacd4d64a0 0x55bacd4d6880 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f4414005bc0 tx=0x7f441400b670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6787684c37ac7bf server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:15.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.583+0000 7f44057fa640 1 -- 172.21.15.5:0/3122552844 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4414019030 con 0x55bacd4d64a0 2024-09-12T02:58:15.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.583+0000 7f44057fa640 1 -- 172.21.15.5:0/3122552844 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44140035f0 con 0x55bacd4d64a0 2024-09-12T02:58:15.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.583+0000 7f44057fa640 1 -- 172.21.15.5:0/3122552844 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4414011720 con 0x55bacd4d64a0 2024-09-12T02:58:15.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.584+0000 7f4427a84740 1 -- 172.21.15.5:0/3122552844 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bacd4d64a0 msgr2=0x55bacd4d6880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:15.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.584+0000 7f4427a84740 1 --2- 172.21.15.5:0/3122552844 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bacd4d64a0 0x55bacd4d6880 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f4414005bc0 tx=0x7f441400b670 comp rx=0 tx=0).stop 2024-09-12T02:58:15.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.584+0000 7f441f7fe640 1 --2- 172.21.15.5:0/1178586952 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f44080a2c10 0x7f44080a50d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f44180bd130 tx=0x7f4418090260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:15.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.584+0000 7f4427a84740 1 -- 172.21.15.5:0/3122552844 shutdown_connections 2024-09-12T02:58:15.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.584+0000 7f4427a84740 1 --2- 172.21.15.5:0/3122552844 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bacd4e1680 0x55bacd4e3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:15.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.584+0000 7f4427a84740 1 --2- 172.21.15.5:0/3122552844 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bacd4d6dc0 0x55bacd4e1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:15.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.584+0000 7f4427a84740 1 --2- 172.21.15.5:0/3122552844 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bacd4d64a0 0x55bacd4d6880 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:15.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.584+0000 7f4427a84740 1 -- 172.21.15.5:0/3122552844 >> 172.21.15.5:0/3122552844 conn(0x55bacd4d2a30 msgr2=0x55bacd4d2e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:15.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.584+0000 7f4427a84740 1 -- 172.21.15.5:0/3122552844 shutdown_connections 2024-09-12T02:58:15.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.584+0000 7f4427a84740 1 -- 172.21.15.5:0/3122552844 wait complete. 2024-09-12T02:58:15.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.585+0000 7f4427a84740 1 -- start start 2024-09-12T02:58:15.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.585+0000 7f4427a84740 1 Processor -- start 2024-09-12T02:58:15.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.585+0000 7f4427a84740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bacd4d64a0 0x55bacd4e0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:15.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.585+0000 7f4427a84740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bacd4d6dc0 0x55bacd4de260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:15.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.585+0000 7f4427a84740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bacd4e1680 0x55bacd4de7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:15.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.585+0000 7f4427a84740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55bacd4e4250 con 0x55bacd4d64a0 2024-09-12T02:58:15.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.585+0000 7f4427a84740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55bacd3f7050 con 0x55bacd4d6dc0 2024-09-12T02:58:15.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.585+0000 7f4427a84740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55bacd4e43d0 con 0x55bacd4e1680 2024-09-12T02:58:15.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.585+0000 7f4427a84740 10 client.0 _ll_register_callbacks cb 0x55bacd437160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:58:15.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.585+0000 7f441e7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bacd4d64a0 0x55bacd4e0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:15.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.585+0000 7f441e7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bacd4d64a0 0x55bacd4e0fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50570/0 (socket says 172.21.15.5:50570) 2024-09-12T02:58:15.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.585+0000 7f441e7fc640 1 -- 172.21.15.5:0/3244745982 learned_addr learned my addr 172.21.15.5:0/3244745982 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:58:15.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.585+0000 7f441effd640 1 --2- 172.21.15.5:0/3244745982 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bacd4d6dc0 0x55bacd4de260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:15.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.585+0000 7f441e7fc640 1 -- 172.21.15.5:0/3244745982 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bacd4e1680 msgr2=0x55bacd4de7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:15.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.585+0000 7f441e7fc640 1 --2- 172.21.15.5:0/3244745982 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bacd4e1680 0x55bacd4de7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:15.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.585+0000 7f441e7fc640 1 -- 172.21.15.5:0/3244745982 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bacd4d6dc0 msgr2=0x55bacd4de260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:15.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.585+0000 7f441e7fc640 1 --2- 172.21.15.5:0/3244745982 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bacd4d6dc0 0x55bacd4de260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:15.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.585+0000 7f441e7fc640 1 -- 172.21.15.5:0/3244745982 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bacd1ad3c0 con 0x55bacd4d64a0 2024-09-12T02:58:15.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.586+0000 7f441e7fc640 1 --2- 172.21.15.5:0/3244745982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bacd4d64a0 0x55bacd4e0fb0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f4414005d50 tx=0x7f4414004250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad16c14c2d8082b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:15.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.586+0000 7f4402ff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T02:58:15.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.586+0000 7f4402ff5640 1 -- 172.21.15.5:0/3244745982 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4414019030 con 0x55bacd4d64a0 2024-09-12T02:58:15.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.586+0000 7f4427a84740 1 -- 172.21.15.5:0/3244745982 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f44140035f0 con 0x55bacd4d64a0 2024-09-12T02:58:15.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.586+0000 7f4402ff5640 1 -- 172.21.15.5:0/3244745982 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f44140039a0 con 0x55bacd4d64a0 2024-09-12T02:58:15.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.586+0000 7f4402ff5640 1 -- 172.21.15.5:0/3244745982 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4414003ba0 con 0x55bacd4d64a0 2024-09-12T02:58:15.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.586+0000 7f4427a84740 10 client.15324 fetch_fsmap learned FSMap version 26 2024-09-12T02:58:15.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.586+0000 7f4427a84740 1 -- 172.21.15.5:0/3244745982 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55bacd4efc00 con 0x55bacd4d64a0 2024-09-12T02:58:15.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.586+0000 7f4402ff5640 1 -- 172.21.15.5:0/3244745982 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44140035f0 con 0x55bacd4d64a0 2024-09-12T02:58:15.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.586+0000 7f4402ff5640 1 -- 172.21.15.5:0/3244745982 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f4414026020 con 0x55bacd4d64a0 2024-09-12T02:58:15.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.587+0000 7f4427a84740 10 client.15324 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T02:58:15.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.587+0000 7f4427a84740 10 client.15324 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T02:58:15.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.587+0000 7f4427a84740 20 client.15324 populate_metadata read hostname 'smithi005' 2024-09-12T02:58:15.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.587+0000 7f4427a84740 1 --2- 172.21.15.5:0/3244745982 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55bacd4f1010 0x55bacd4f3400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:15.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.587+0000 7f4427a84740 4 client.15324 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:58:15.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.587+0000 7f4427a84740 1 -- 172.21.15.5:0/3244745982 --> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bacd449850 con 0x55bacd4f1010 2024-09-12T02:58:15.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.587+0000 7f441effd640 1 --2- 172.21.15.5:0/3244745982 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55bacd4f1010 0x55bacd4f3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:15.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.587+0000 7f441effd640 1 --2- 172.21.15.5:0/3244745982 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55bacd4f1010 0x55bacd4f3400 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e66a321c725ad039 server_cookie=a15f1ca7d12ab1a2 in_seq=0 out_seq=0 2024-09-12T02:58:15.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.587+0000 7f4402ff5640 10 client.15324 ms_handle_connect on v2:172.21.15.148:6838/2317600000 2024-09-12T02:58:15.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.588+0000 7f4402ff5640 1 -- 172.21.15.5:0/3244745982 <== mds.0 v2:172.21.15.148:6838/2317600000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bacd449850 con 0x55bacd4f1010 2024-09-12T02:58:15.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.588+0000 7f4402ff5640 10 client.15324 handle_command_reply: tid=0 2024-09-12T02:58:15.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.588+0000 7f4427a84740 1 client.15324 shutdown 2024-09-12T02:58:15.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.588+0000 7f4427a84740 1 -- 172.21.15.5:0/3244745982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bacd4d64a0 msgr2=0x55bacd4e0fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:15.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.588+0000 7f4427a84740 1 --2- 172.21.15.5:0/3244745982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bacd4d64a0 0x55bacd4e0fb0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f4414005d50 tx=0x7f4414004250 comp rx=0 tx=0).stop 2024-09-12T02:58:15.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.588+0000 7f4427a84740 1 -- 172.21.15.5:0/3244745982 shutdown_connections 2024-09-12T02:58:15.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.588+0000 7f4427a84740 1 --2- 172.21.15.5:0/3244745982 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bacd4e1680 0x55bacd4de7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:15.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.588+0000 7f4427a84740 1 --2- 172.21.15.5:0/3244745982 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bacd4d6dc0 0x55bacd4de260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:15.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.588+0000 7f4427a84740 1 --2- 172.21.15.5:0/3244745982 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55bacd4f1010 0x55bacd4f3400 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:15.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.588+0000 7f4427a84740 1 --2- 172.21.15.5:0/3244745982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bacd4d64a0 0x55bacd4e0fb0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:15.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.588+0000 7f4402ff5640 0 client.15324 ms_handle_reset on v2:172.21.15.148:6838/2317600000 2024-09-12T02:58:15.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.588+0000 7f4427a84740 1 -- 172.21.15.5:0/3244745982 >> 172.21.15.5:0/3244745982 conn(0x55bacd4d2a30 msgr2=0x55bacd4d8760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:15.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.589+0000 7f4427a84740 1 -- 172.21.15.5:0/3244745982 shutdown_connections 2024-09-12T02:58:15.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.589+0000 7f4427a84740 1 -- 172.21.15.5:0/3244745982 wait complete. 2024-09-12T02:58:15.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.589+0000 7f4427a84740 20 client.15324 trim_cache size 0 max 16384 2024-09-12T02:58:15.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.606+0000 7f4427a84740 1 Processor -- start 2024-09-12T02:58:15.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.606+0000 7f4427a84740 1 -- start start 2024-09-12T02:58:15.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.606+0000 7f4427a84740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bacd4d64a0 0x55bacd4dcb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:15.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.606+0000 7f4427a84740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bacd4d6dc0 0x55bacd4e9a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:15.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.606+0000 7f4427a84740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bacd4e1680 0x55bacd4e9f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:15.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.606+0000 7f4427a84740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f44201ad4e0 con 0x55bacd4e1680 2024-09-12T02:58:15.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.606+0000 7f4427a84740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f44201af020 con 0x55bacd4d64a0 2024-09-12T02:58:15.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.606+0000 7f4427a84740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f44201adbb0 con 0x55bacd4d6dc0 2024-09-12T02:58:15.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.606+0000 7f441e7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bacd4e1680 0x55bacd4e9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:15.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.606+0000 7f441e7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bacd4e1680 0x55bacd4e9f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50580/0 (socket says 172.21.15.5:50580) 2024-09-12T02:58:15.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.606+0000 7f441e7fc640 1 -- 172.21.15.5:0/4039228269 learned_addr learned my addr 172.21.15.5:0/4039228269 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:58:15.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.606+0000 7f441effd640 1 --2- 172.21.15.5:0/4039228269 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bacd4d6dc0 0x55bacd4e9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:15.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.606+0000 7f441e7fc640 1 --2- 172.21.15.5:0/4039228269 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bacd4d64a0 0x55bacd4dcb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:15.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.607+0000 7f441e7fc640 1 -- 172.21.15.5:0/4039228269 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bacd4d6dc0 msgr2=0x55bacd4e9a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:15.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.607+0000 7f441e7fc640 1 --2- 172.21.15.5:0/4039228269 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bacd4d6dc0 0x55bacd4e9a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:15.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.607+0000 7f441e7fc640 1 -- 172.21.15.5:0/4039228269 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bacd4d64a0 msgr2=0x55bacd4dcb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:15.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.607+0000 7f441e7fc640 1 --2- 172.21.15.5:0/4039228269 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bacd4d64a0 0x55bacd4dcb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:15.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.607+0000 7f441e7fc640 1 -- 172.21.15.5:0/4039228269 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bacd4d62d0 con 0x55bacd4e1680 2024-09-12T02:58:15.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.607+0000 7f441e7fc640 1 --2- 172.21.15.5:0/4039228269 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bacd4e1680 0x55bacd4e9f40 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f4414001eb0 tx=0x7f4414005fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82b65b391968a6b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:15.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.607+0000 7f4404ff9640 1 -- 172.21.15.5:0/4039228269 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4414019040 con 0x55bacd4e1680 2024-09-12T02:58:15.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.607+0000 7f4404ff9640 1 -- 172.21.15.5:0/4039228269 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f441401a040 con 0x55bacd4e1680 2024-09-12T02:58:15.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.607+0000 7f4404ff9640 1 -- 172.21.15.5:0/4039228269 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4414005000 con 0x55bacd4e1680 2024-09-12T02:58:15.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.607+0000 7f4427a84740 1 -- 172.21.15.5:0/4039228269 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bacd4e1680 msgr2=0x55bacd4e9f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:15.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.607+0000 7f4427a84740 1 --2- 172.21.15.5:0/4039228269 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bacd4e1680 0x55bacd4e9f40 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f4414001eb0 tx=0x7f4414005fb0 comp rx=0 tx=0).stop 2024-09-12T02:58:15.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.608+0000 7f4427a84740 1 -- 172.21.15.5:0/4039228269 shutdown_connections 2024-09-12T02:58:15.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.608+0000 7f4427a84740 1 --2- 172.21.15.5:0/4039228269 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bacd4e1680 0x55bacd4e9f40 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:15.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.608+0000 7f4427a84740 1 --2- 172.21.15.5:0/4039228269 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bacd4d6dc0 0x55bacd4e9a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:15.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.608+0000 7f4427a84740 1 --2- 172.21.15.5:0/4039228269 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bacd4d64a0 0x55bacd4dcb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:15.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.608+0000 7f4427a84740 1 -- 172.21.15.5:0/4039228269 >> 172.21.15.5:0/4039228269 conn(0x55bacd4d2a30 msgr2=0x55bacd4d1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:15.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.608+0000 7f4427a84740 1 -- 172.21.15.5:0/4039228269 shutdown_connections 2024-09-12T02:58:15.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.608+0000 7f4427a84740 1 -- 172.21.15.5:0/4039228269 wait complete. 2024-09-12T02:58:15.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.608+0000 7f4427a84740 1 -- start start 2024-09-12T02:58:15.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.608+0000 7f4427a84740 1 Processor -- start 2024-09-12T02:58:15.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.608+0000 7f4427a84740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bacd4d64a0 0x55bacd4db380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:15.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.608+0000 7f4427a84740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bacd4d6dc0 0x55bacd4db8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:15.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.608+0000 7f4427a84740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bacd4e1680 0x55bacd557080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:15.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.608+0000 7f4427a84740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f44201ae950 con 0x55bacd4e1680 2024-09-12T02:58:15.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.608+0000 7f4427a84740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f44201ae280 con 0x55bacd4d6dc0 2024-09-12T02:58:15.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.608+0000 7f4427a84740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f44201ab4c0 con 0x55bacd4d64a0 2024-09-12T02:58:15.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.608+0000 7f4427a84740 10 client.0 _ll_register_callbacks cb 0x7f4414013400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:58:15.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.608+0000 7f441e7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bacd4e1680 0x55bacd557080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:15.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.608+0000 7f441e7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bacd4e1680 0x55bacd557080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50584/0 (socket says 172.21.15.5:50584) 2024-09-12T02:58:15.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.608+0000 7f441e7fc640 1 -- 172.21.15.5:0/3496087328 learned_addr learned my addr 172.21.15.5:0/3496087328 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:58:15.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.609+0000 7f441effd640 1 --2- 172.21.15.5:0/3496087328 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bacd4d6dc0 0x55bacd4db8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:15.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.609+0000 7f441e7fc640 1 -- 172.21.15.5:0/3496087328 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bacd4d64a0 msgr2=0x55bacd4db380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:15.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.609+0000 7f441e7fc640 1 --2- 172.21.15.5:0/3496087328 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bacd4d64a0 0x55bacd4db380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:15.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.609+0000 7f441e7fc640 1 -- 172.21.15.5:0/3496087328 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bacd4d6dc0 msgr2=0x55bacd4db8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:15.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.609+0000 7f441e7fc640 1 --2- 172.21.15.5:0/3496087328 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bacd4d6dc0 0x55bacd4db8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:15.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.609+0000 7f441e7fc640 1 -- 172.21.15.5:0/3496087328 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bacd557720 con 0x55bacd4e1680 2024-09-12T02:58:15.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.609+0000 7f441e7fc640 1 --2- 172.21.15.5:0/3496087328 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bacd4e1680 0x55bacd557080 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f441400bf60 tx=0x7f4414000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d932032d6018a2ee server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:15.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.609+0000 7f4402ff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T02:58:15.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.609+0000 7f4402ff5640 1 -- 172.21.15.5:0/3496087328 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4414019040 con 0x55bacd4e1680 2024-09-12T02:58:15.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.609+0000 7f4402ff5640 1 -- 172.21.15.5:0/3496087328 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f4414004680 con 0x55bacd4e1680 2024-09-12T02:58:15.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.609+0000 7f4427a84740 1 -- 172.21.15.5:0/3496087328 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f441401a040 con 0x55bacd4e1680 2024-09-12T02:58:15.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.609+0000 7f4402ff5640 1 -- 172.21.15.5:0/3496087328 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4414004880 con 0x55bacd4e1680 2024-09-12T02:58:15.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.609+0000 7f4427a84740 10 client.15330 fetch_fsmap learned FSMap version 26 2024-09-12T02:58:15.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.609+0000 7f4427a84740 1 -- 172.21.15.5:0/3496087328 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55bacd55fc60 con 0x55bacd4e1680 2024-09-12T02:58:15.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.610+0000 7f4402ff5640 1 -- 172.21.15.5:0/3496087328 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f441401a040 con 0x55bacd4e1680 2024-09-12T02:58:15.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.610+0000 7f4402ff5640 1 -- 172.21.15.5:0/3496087328 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f4414026040 con 0x55bacd4e1680 2024-09-12T02:58:15.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.610+0000 7f4427a84740 10 client.15330 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T02:58:15.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.610+0000 7f4427a84740 10 client.15330 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T02:58:15.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.610+0000 7f4427a84740 20 client.15330 populate_metadata read hostname 'smithi005' 2024-09-12T02:58:15.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.610+0000 7f4427a84740 1 --2- 172.21.15.5:0/3496087328 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55bacd5602d0 0x55bacd5626c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:15.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.610+0000 7f4427a84740 4 client.15330 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T02:58:15.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.610+0000 7f4427a84740 1 -- 172.21.15.5:0/3496087328 --> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55bacd4deb30 con 0x55bacd5602d0 2024-09-12T02:58:15.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.610+0000 7f441effd640 1 --2- 172.21.15.5:0/3496087328 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55bacd5602d0 0x55bacd5626c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:15.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.610+0000 7f441effd640 1 --2- 172.21.15.5:0/3496087328 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55bacd5602d0 0x55bacd5626c0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e7b056a235874a server_cookie=5097dc9301c654c2 in_seq=0 out_seq=0 2024-09-12T02:58:15.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.610+0000 7f4402ff5640 10 client.15330 ms_handle_connect on v2:172.21.15.148:6838/2317600000 2024-09-12T02:58:15.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.611+0000 7f4402ff5640 1 -- 172.21.15.5:0/3496087328 <== mds.0 v2:172.21.15.148:6838/2317600000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55bacd4deb30 con 0x55bacd5602d0 2024-09-12T02:58:15.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.611+0000 7f4402ff5640 10 client.15330 handle_command_reply: tid=0 2024-09-12T02:58:15.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.611+0000 7f4427a84740 1 client.15330 shutdown 2024-09-12T02:58:15.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.611+0000 7f4427a84740 1 -- 172.21.15.5:0/3496087328 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bacd4e1680 msgr2=0x55bacd557080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:15.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.611+0000 7f4427a84740 1 --2- 172.21.15.5:0/3496087328 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bacd4e1680 0x55bacd557080 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f441400bf60 tx=0x7f4414000bc0 comp rx=0 tx=0).stop 2024-09-12T02:58:15.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.611+0000 7f4427a84740 1 -- 172.21.15.5:0/3496087328 shutdown_connections 2024-09-12T02:58:15.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.611+0000 7f4427a84740 1 --2- 172.21.15.5:0/3496087328 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55bacd5602d0 0x55bacd5626c0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:15.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.611+0000 7f4427a84740 1 --2- 172.21.15.5:0/3496087328 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bacd4e1680 0x55bacd557080 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:15.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.611+0000 7f4427a84740 1 --2- 172.21.15.5:0/3496087328 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bacd4d6dc0 0x55bacd4db8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:15.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.611+0000 7f4402ff5640 0 client.15330 ms_handle_reset on v2:172.21.15.148:6838/2317600000 2024-09-12T02:58:15.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.611+0000 7f4427a84740 1 --2- 172.21.15.5:0/3496087328 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bacd4d64a0 0x55bacd4db380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:15.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.611+0000 7f4427a84740 1 -- 172.21.15.5:0/3496087328 >> 172.21.15.5:0/3496087328 conn(0x55bacd4d2a30 msgr2=0x55bacd4dfac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:15.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.611+0000 7f4427a84740 1 -- 172.21.15.5:0/3496087328 shutdown_connections 2024-09-12T02:58:15.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.611+0000 7f4427a84740 1 -- 172.21.15.5:0/3496087328 wait complete. 2024-09-12T02:58:15.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.611+0000 7f4427a84740 20 client.15330 trim_cache size 0 max 16384 2024-09-12T02:58:15.057 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-12T02:58:15.057 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-12T02:58:15.058 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-09-12T02:58:15.058 INFO:teuthology.orchestra.run.smithi005.stdout: "9734a5ee-e292-4bee-b391-561c540e414e": { 2024-09-12T02:58:15.058 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-09-12T02:58:15.058 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "9734a5ee-e292-4bee-b391-561c540e414e", 2024-09-12T02:58:15.058 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-09-12T02:58:15.058 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T02:58:15.058 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T02:58:15.058 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-12T02:58:15.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.611+0000 7f44256ab640 1 -- 172.21.15.5:0/1178586952 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f44080a2c10 msgr2=0x7f44080a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:15.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.611+0000 7f44256ab640 1 --2- 172.21.15.5:0/1178586952 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f44080a2c10 0x7f44080a50d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f44180bd130 tx=0x7f4418090260 comp rx=0 tx=0).stop 2024-09-12T02:58:15.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.611+0000 7f44256ab640 1 -- 172.21.15.5:0/1178586952 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44201082a0 msgr2=0x7f442019ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:15.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.611+0000 7f44256ab640 1 --2- 172.21.15.5:0/1178586952 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44201082a0 0x7f442019ce10 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f4418005cc0 tx=0x7f4418032f40 comp rx=0 tx=0).stop 2024-09-12T02:58:15.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.612+0000 7f44256ab640 1 -- 172.21.15.5:0/1178586952 shutdown_connections 2024-09-12T02:58:15.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.612+0000 7f44256ab640 1 --2- 172.21.15.5:0/1178586952 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f44080a2c10 0x7f44080a50d0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:15.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.612+0000 7f44256ab640 1 --2- 172.21.15.5:0/1178586952 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f44201082a0 0x7f442019ce10 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:15.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.612+0000 7f44256ab640 1 --2- 172.21.15.5:0/1178586952 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f44201036e0 0x7f4420198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:15.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.612+0000 7f44256ab640 1 --2- 172.21.15.5:0/1178586952 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4420102d30 0x7f4420198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:15.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.612+0000 7f44256ab640 1 -- 172.21.15.5:0/1178586952 >> 172.21.15.5:0/1178586952 conn(0x7f44200fc7d0 msgr2=0x7f4420105d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:15.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.612+0000 7f44256ab640 1 -- 172.21.15.5:0/1178586952 shutdown_connections 2024-09-12T02:58:15.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:14.612+0000 7f44256ab640 1 -- 172.21.15.5:0/1178586952 wait complete. 2024-09-12T02:58:15.115 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":335,"stamp":"2024-09-12T02:58:13.570501+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":668530,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1217,"num_read_kb":741,"num_write":456,"num_write_kb":3102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":1181470,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":872,"ondisk_log_size":872,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":374580,"kb_used_data":12852,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124567244,"statfs":{"total":1151940427776,"available":1151556857856,"internally_reserved":0,"allocated":13160448,"data_stored":7273506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":370193723},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":9412,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":141,"num_read_kb":82,"num_write":25,"num_write_kb":25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002690"},"pg_stats":[{"pgid":"3.3f","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.565050+0000","last_change":"2024-09-12T02:56:15.988690+0000","last_active":"2024-09-12T02:57:59.565050+0000","last_peered":"2024-09-12T02:57:59.565050+0000","last_clean":"2024-09-12T02:57:59.565050+0000","last_became_active":"2024-09-12T02:56:15.988320+0000","last_became_peered":"2024-09-12T02:56:15.988320+0000","last_unstale":"2024-09-12T02:57:59.565050+0000","last_undegraded":"2024-09-12T02:57:59.565050+0000","last_fullsized":"2024-09-12T02:57:59.565050+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:31:14.157622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"83'4","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.553842+0000","last_change":"2024-09-12T02:56:15.978368+0000","last_active":"2024-09-12T02:57:59.553842+0000","last_peered":"2024-09-12T02:57:59.553842+0000","last_clean":"2024-09-12T02:57:59.553842+0000","last_became_active":"2024-09-12T02:56:15.978092+0000","last_became_peered":"2024-09-12T02:56:15.978092+0000","last_unstale":"2024-09-12T02:57:59.553842+0000","last_undegraded":"2024-09-12T02:57:59.553842+0000","last_fullsized":"2024-09-12T02:57:59.553842+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:32:48.399005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.564003+0000","last_change":"2024-09-12T02:56:15.987249+0000","last_active":"2024-09-12T02:57:59.564003+0000","last_peered":"2024-09-12T02:57:59.564003+0000","last_clean":"2024-09-12T02:57:59.564003+0000","last_became_active":"2024-09-12T02:56:15.986921+0000","last_became_peered":"2024-09-12T02:56:15.986921+0000","last_unstale":"2024-09-12T02:57:59.564003+0000","last_undegraded":"2024-09-12T02:57:59.564003+0000","last_fullsized":"2024-09-12T02:57:59.564003+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:42:13.427871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.555703+0000","last_change":"2024-09-12T02:56:15.987673+0000","last_active":"2024-09-12T02:57:59.555703+0000","last_peered":"2024-09-12T02:57:59.555703+0000","last_clean":"2024-09-12T02:57:59.555703+0000","last_became_active":"2024-09-12T02:56:15.986964+0000","last_became_peered":"2024-09-12T02:56:15.986964+0000","last_unstale":"2024-09-12T02:57:59.555703+0000","last_undegraded":"2024-09-12T02:57:59.555703+0000","last_fullsized":"2024-09-12T02:57:59.555703+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:07:12.406974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"83'4","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.548461+0000","last_change":"2024-09-12T02:56:15.974473+0000","last_active":"2024-09-12T02:57:59.548461+0000","last_peered":"2024-09-12T02:57:59.548461+0000","last_clean":"2024-09-12T02:57:59.548461+0000","last_became_active":"2024-09-12T02:56:15.974124+0000","last_became_peered":"2024-09-12T02:56:15.974124+0000","last_unstale":"2024-09-12T02:57:59.548461+0000","last_undegraded":"2024-09-12T02:57:59.548461+0000","last_fullsized":"2024-09-12T02:57:59.548461+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:44:29.054841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.560740+0000","last_change":"2024-09-12T02:56:15.980242+0000","last_active":"2024-09-12T02:57:59.560740+0000","last_peered":"2024-09-12T02:57:59.560740+0000","last_clean":"2024-09-12T02:57:59.560740+0000","last_became_active":"2024-09-12T02:56:15.979937+0000","last_became_peered":"2024-09-12T02:56:15.979937+0000","last_unstale":"2024-09-12T02:57:59.560740+0000","last_undegraded":"2024-09-12T02:57:59.560740+0000","last_fullsized":"2024-09-12T02:57:59.560740+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:44:13.406341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"83'5","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.556845+0000","last_change":"2024-09-12T02:56:15.978746+0000","last_active":"2024-09-12T02:57:59.556845+0000","last_peered":"2024-09-12T02:57:59.556845+0000","last_clean":"2024-09-12T02:57:59.556845+0000","last_became_active":"2024-09-12T02:56:15.978304+0000","last_became_peered":"2024-09-12T02:56:15.978304+0000","last_unstale":"2024-09-12T02:57:59.556845+0000","last_undegraded":"2024-09-12T02:57:59.556845+0000","last_fullsized":"2024-09-12T02:57:59.556845+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:34:52.115630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.562507+0000","last_change":"2024-09-12T02:56:15.980083+0000","last_active":"2024-09-12T02:57:59.562507+0000","last_peered":"2024-09-12T02:57:59.562507+0000","last_clean":"2024-09-12T02:57:59.562507+0000","last_became_active":"2024-09-12T02:56:15.979771+0000","last_became_peered":"2024-09-12T02:56:15.979771+0000","last_unstale":"2024-09-12T02:57:59.562507+0000","last_undegraded":"2024-09-12T02:57:59.562507+0000","last_fullsized":"2024-09-12T02:57:59.562507+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:59:42.351970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"83'11","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.565703+0000","last_change":"2024-09-12T02:56:15.988091+0000","last_active":"2024-09-12T02:57:59.565703+0000","last_peered":"2024-09-12T02:57:59.565703+0000","last_clean":"2024-09-12T02:57:59.565703+0000","last_became_active":"2024-09-12T02:56:15.986295+0000","last_became_peered":"2024-09-12T02:56:15.986295+0000","last_unstale":"2024-09-12T02:57:59.565703+0000","last_undegraded":"2024-09-12T02:57:59.565703+0000","last_fullsized":"2024-09-12T02:57:59.565703+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:45:40.331805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"83'16","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.562128+0000","last_change":"2024-09-12T02:56:15.987349+0000","last_active":"2024-09-12T02:57:59.562128+0000","last_peered":"2024-09-12T02:57:59.562128+0000","last_clean":"2024-09-12T02:57:59.562128+0000","last_became_active":"2024-09-12T02:56:15.987018+0000","last_became_peered":"2024-09-12T02:56:15.987018+0000","last_unstale":"2024-09-12T02:57:59.562128+0000","last_undegraded":"2024-09-12T02:57:59.562128+0000","last_fullsized":"2024-09-12T02:57:59.562128+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:34:19.652348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.563544+0000","last_change":"2024-09-12T02:56:15.985164+0000","last_active":"2024-09-12T02:57:59.563544+0000","last_peered":"2024-09-12T02:57:59.563544+0000","last_clean":"2024-09-12T02:57:59.563544+0000","last_became_active":"2024-09-12T02:56:15.984787+0000","last_became_peered":"2024-09-12T02:56:15.984787+0000","last_unstale":"2024-09-12T02:57:59.563544+0000","last_undegraded":"2024-09-12T02:57:59.563544+0000","last_fullsized":"2024-09-12T02:57:59.563544+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:36:29.727442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.552957+0000","last_change":"2024-09-12T02:56:15.979963+0000","last_active":"2024-09-12T02:57:59.552957+0000","last_peered":"2024-09-12T02:57:59.552957+0000","last_clean":"2024-09-12T02:57:59.552957+0000","last_became_active":"2024-09-12T02:56:15.979639+0000","last_became_peered":"2024-09-12T02:56:15.979639+0000","last_unstale":"2024-09-12T02:57:59.552957+0000","last_undegraded":"2024-09-12T02:57:59.552957+0000","last_fullsized":"2024-09-12T02:57:59.552957+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:34:18.273245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"83'10","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.561647+0000","last_change":"2024-09-12T02:56:15.980415+0000","last_active":"2024-09-12T02:57:59.561647+0000","last_peered":"2024-09-12T02:57:59.561647+0000","last_clean":"2024-09-12T02:57:59.561647+0000","last_became_active":"2024-09-12T02:56:15.980059+0000","last_became_peered":"2024-09-12T02:56:15.980059+0000","last_unstale":"2024-09-12T02:57:59.561647+0000","last_undegraded":"2024-09-12T02:57:59.561647+0000","last_fullsized":"2024-09-12T02:57:59.561647+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:11:42.689081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.553574+0000","last_change":"2024-09-12T02:56:15.981423+0000","last_active":"2024-09-12T02:57:59.553574+0000","last_peered":"2024-09-12T02:57:59.553574+0000","last_clean":"2024-09-12T02:57:59.553574+0000","last_became_active":"2024-09-12T02:56:15.981186+0000","last_became_peered":"2024-09-12T02:56:15.981186+0000","last_unstale":"2024-09-12T02:57:59.553574+0000","last_undegraded":"2024-09-12T02:57:59.553574+0000","last_fullsized":"2024-09-12T02:57:59.553574+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:36:57.406907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.547696+0000","last_change":"2024-09-12T02:56:15.982587+0000","last_active":"2024-09-12T02:57:59.547696+0000","last_peered":"2024-09-12T02:57:59.547696+0000","last_clean":"2024-09-12T02:57:59.547696+0000","last_became_active":"2024-09-12T02:56:15.982315+0000","last_became_peered":"2024-09-12T02:56:15.982315+0000","last_unstale":"2024-09-12T02:57:59.547696+0000","last_undegraded":"2024-09-12T02:57:59.547696+0000","last_fullsized":"2024-09-12T02:57:59.547696+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:42:17.736999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.564760+0000","last_change":"2024-09-12T02:56:15.979482+0000","last_active":"2024-09-12T02:57:59.564760+0000","last_peered":"2024-09-12T02:57:59.564760+0000","last_clean":"2024-09-12T02:57:59.564760+0000","last_became_active":"2024-09-12T02:56:15.979122+0000","last_became_peered":"2024-09-12T02:56:15.979122+0000","last_unstale":"2024-09-12T02:57:59.564760+0000","last_undegraded":"2024-09-12T02:57:59.564760+0000","last_fullsized":"2024-09-12T02:57:59.564760+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:18:05.642487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.565218+0000","last_change":"2024-09-12T02:56:15.989908+0000","last_active":"2024-09-12T02:57:59.565218+0000","last_peered":"2024-09-12T02:57:59.565218+0000","last_clean":"2024-09-12T02:57:59.565218+0000","last_became_active":"2024-09-12T02:56:15.989515+0000","last_became_peered":"2024-09-12T02:56:15.989515+0000","last_unstale":"2024-09-12T02:57:59.565218+0000","last_undegraded":"2024-09-12T02:57:59.565218+0000","last_fullsized":"2024-09-12T02:57:59.565218+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:59:57.141607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"83'10","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.557609+0000","last_change":"2024-09-12T02:56:15.980323+0000","last_active":"2024-09-12T02:57:59.557609+0000","last_peered":"2024-09-12T02:57:59.557609+0000","last_clean":"2024-09-12T02:57:59.557609+0000","last_became_active":"2024-09-12T02:56:15.979989+0000","last_became_peered":"2024-09-12T02:56:15.979989+0000","last_unstale":"2024-09-12T02:57:59.557609+0000","last_undegraded":"2024-09-12T02:57:59.557609+0000","last_fullsized":"2024-09-12T02:57:59.557609+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:55:56.064125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.556275+0000","last_change":"2024-09-12T02:56:15.986003+0000","last_active":"2024-09-12T02:57:59.556275+0000","last_peered":"2024-09-12T02:57:59.556275+0000","last_clean":"2024-09-12T02:57:59.556275+0000","last_became_active":"2024-09-12T02:56:15.985335+0000","last_became_peered":"2024-09-12T02:56:15.985335+0000","last_unstale":"2024-09-12T02:57:59.556275+0000","last_undegraded":"2024-09-12T02:57:59.556275+0000","last_fullsized":"2024-09-12T02:57:59.556275+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:51:47.944530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"83'8","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.554254+0000","last_change":"2024-09-12T02:56:31.036870+0000","last_active":"2024-09-12T02:57:59.554254+0000","last_peered":"2024-09-12T02:57:59.554254+0000","last_clean":"2024-09-12T02:57:59.554254+0000","last_became_active":"2024-09-12T02:56:31.036443+0000","last_became_peered":"2024-09-12T02:56:31.036443+0000","last_unstale":"2024-09-12T02:57:59.554254+0000","last_undegraded":"2024-09-12T02:57:59.554254+0000","last_fullsized":"2024-09-12T02:57:59.554254+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:49:29.326184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"83'11","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.555414+0000","last_change":"2024-09-12T02:56:15.974547+0000","last_active":"2024-09-12T02:57:59.555414+0000","last_peered":"2024-09-12T02:57:59.555414+0000","last_clean":"2024-09-12T02:57:59.555414+0000","last_became_active":"2024-09-12T02:56:15.974174+0000","last_became_peered":"2024-09-12T02:56:15.974174+0000","last_unstale":"2024-09-12T02:57:59.555414+0000","last_undegraded":"2024-09-12T02:57:59.555414+0000","last_fullsized":"2024-09-12T02:57:59.555414+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:22:16.954309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"83'10","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.565778+0000","last_change":"2024-09-12T02:56:15.981771+0000","last_active":"2024-09-12T02:57:59.565778+0000","last_peered":"2024-09-12T02:57:59.565778+0000","last_clean":"2024-09-12T02:57:59.565778+0000","last_became_active":"2024-09-12T02:56:15.980948+0000","last_became_peered":"2024-09-12T02:56:15.980948+0000","last_unstale":"2024-09-12T02:57:59.565778+0000","last_undegraded":"2024-09-12T02:57:59.565778+0000","last_fullsized":"2024-09-12T02:57:59.565778+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:34:35.961348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.562061+0000","last_change":"2024-09-12T02:56:15.982318+0000","last_active":"2024-09-12T02:57:59.562061+0000","last_peered":"2024-09-12T02:57:59.562061+0000","last_clean":"2024-09-12T02:57:59.562061+0000","last_became_active":"2024-09-12T02:56:15.982013+0000","last_became_peered":"2024-09-12T02:56:15.982013+0000","last_unstale":"2024-09-12T02:57:59.562061+0000","last_undegraded":"2024-09-12T02:57:59.562061+0000","last_fullsized":"2024-09-12T02:57:59.562061+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:10:50.895579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.559580+0000","last_change":"2024-09-12T02:56:15.969992+0000","last_active":"2024-09-12T02:57:59.559580+0000","last_peered":"2024-09-12T02:57:59.559580+0000","last_clean":"2024-09-12T02:57:59.559580+0000","last_became_active":"2024-09-12T02:56:15.969621+0000","last_became_peered":"2024-09-12T02:56:15.969621+0000","last_unstale":"2024-09-12T02:57:59.559580+0000","last_undegraded":"2024-09-12T02:57:59.559580+0000","last_fullsized":"2024-09-12T02:57:59.559580+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:47:40.206042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"83'2","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.560304+0000","last_change":"2024-09-12T02:56:15.982315+0000","last_active":"2024-09-12T02:57:59.560304+0000","last_peered":"2024-09-12T02:57:59.560304+0000","last_clean":"2024-09-12T02:57:59.560304+0000","last_became_active":"2024-09-12T02:56:15.981992+0000","last_became_peered":"2024-09-12T02:56:15.981992+0000","last_unstale":"2024-09-12T02:57:59.560304+0000","last_undegraded":"2024-09-12T02:57:59.560304+0000","last_fullsized":"2024-09-12T02:57:59.560304+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:54:04.188418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"83'15","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.560612+0000","last_change":"2024-09-12T02:56:15.980712+0000","last_active":"2024-09-12T02:57:59.560612+0000","last_peered":"2024-09-12T02:57:59.560612+0000","last_clean":"2024-09-12T02:57:59.560612+0000","last_became_active":"2024-09-12T02:56:15.980410+0000","last_became_peered":"2024-09-12T02:56:15.980410+0000","last_unstale":"2024-09-12T02:57:59.560612+0000","last_undegraded":"2024-09-12T02:57:59.560612+0000","last_fullsized":"2024-09-12T02:57:59.560612+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:13:53.844490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.557058+0000","last_change":"2024-09-12T02:56:15.981675+0000","last_active":"2024-09-12T02:57:59.557058+0000","last_peered":"2024-09-12T02:57:59.557058+0000","last_clean":"2024-09-12T02:57:59.557058+0000","last_became_active":"2024-09-12T02:56:15.981372+0000","last_became_peered":"2024-09-12T02:56:15.981372+0000","last_unstale":"2024-09-12T02:57:59.557058+0000","last_undegraded":"2024-09-12T02:57:59.557058+0000","last_fullsized":"2024-09-12T02:57:59.557058+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:39:17.399199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"83'4","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.557413+0000","last_change":"2024-09-12T02:56:15.972750+0000","last_active":"2024-09-12T02:57:59.557413+0000","last_peered":"2024-09-12T02:57:59.557413+0000","last_clean":"2024-09-12T02:57:59.557413+0000","last_became_active":"2024-09-12T02:56:15.972434+0000","last_became_peered":"2024-09-12T02:56:15.972434+0000","last_unstale":"2024-09-12T02:57:59.557413+0000","last_undegraded":"2024-09-12T02:57:59.557413+0000","last_fullsized":"2024-09-12T02:57:59.557413+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:59:22.784908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.545235+0000","last_change":"2024-09-12T02:56:15.973567+0000","last_active":"2024-09-12T02:57:59.545235+0000","last_peered":"2024-09-12T02:57:59.545235+0000","last_clean":"2024-09-12T02:57:59.545235+0000","last_became_active":"2024-09-12T02:56:15.973251+0000","last_became_peered":"2024-09-12T02:56:15.973251+0000","last_unstale":"2024-09-12T02:57:59.545235+0000","last_undegraded":"2024-09-12T02:57:59.545235+0000","last_fullsized":"2024-09-12T02:57:59.545235+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:16:39.536817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.540308+0000","last_change":"2024-09-12T02:56:15.986108+0000","last_active":"2024-09-12T02:57:59.540308+0000","last_peered":"2024-09-12T02:57:59.540308+0000","last_clean":"2024-09-12T02:57:59.540308+0000","last_became_active":"2024-09-12T02:56:15.985701+0000","last_became_peered":"2024-09-12T02:56:15.985701+0000","last_unstale":"2024-09-12T02:57:59.540308+0000","last_undegraded":"2024-09-12T02:57:59.540308+0000","last_fullsized":"2024-09-12T02:57:59.540308+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:59.377473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"83'12","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.555638+0000","last_change":"2024-09-12T02:56:15.980078+0000","last_active":"2024-09-12T02:57:59.555638+0000","last_peered":"2024-09-12T02:57:59.555638+0000","last_clean":"2024-09-12T02:57:59.555638+0000","last_became_active":"2024-09-12T02:56:15.979785+0000","last_became_peered":"2024-09-12T02:56:15.979785+0000","last_unstale":"2024-09-12T02:57:59.555638+0000","last_undegraded":"2024-09-12T02:57:59.555638+0000","last_fullsized":"2024-09-12T02:57:59.555638+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:00:19.252398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.559119+0000","last_change":"2024-09-12T02:56:15.984170+0000","last_active":"2024-09-12T02:57:59.559119+0000","last_peered":"2024-09-12T02:57:59.559119+0000","last_clean":"2024-09-12T02:57:59.559119+0000","last_became_active":"2024-09-12T02:56:15.983416+0000","last_became_peered":"2024-09-12T02:56:15.983416+0000","last_unstale":"2024-09-12T02:57:59.559119+0000","last_undegraded":"2024-09-12T02:57:59.559119+0000","last_fullsized":"2024-09-12T02:57:59.559119+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:13:24.723923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.556295+0000","last_change":"2024-09-12T02:56:15.980987+0000","last_active":"2024-09-12T02:57:59.556295+0000","last_peered":"2024-09-12T02:57:59.556295+0000","last_clean":"2024-09-12T02:57:59.556295+0000","last_became_active":"2024-09-12T02:56:15.980661+0000","last_became_peered":"2024-09-12T02:56:15.980661+0000","last_unstale":"2024-09-12T02:57:59.556295+0000","last_undegraded":"2024-09-12T02:57:59.556295+0000","last_fullsized":"2024-09-12T02:57:59.556295+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:19:49.294698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"83'6","reported_seq":147,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:58:10.188613+0000","last_change":"2024-09-12T02:56:31.042914+0000","last_active":"2024-09-12T02:58:10.188613+0000","last_peered":"2024-09-12T02:58:10.188613+0000","last_clean":"2024-09-12T02:58:10.188613+0000","last_became_active":"2024-09-12T02:56:31.037356+0000","last_became_peered":"2024-09-12T02:56:31.037356+0000","last_unstale":"2024-09-12T02:58:10.188613+0000","last_undegraded":"2024-09-12T02:58:10.188613+0000","last_fullsized":"2024-09-12T02:58:10.188613+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:17:48.582725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":107,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":622,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"83'8","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.565986+0000","last_change":"2024-09-12T02:56:31.036170+0000","last_active":"2024-09-12T02:57:59.565986+0000","last_peered":"2024-09-12T02:57:59.565986+0000","last_clean":"2024-09-12T02:57:59.565986+0000","last_became_active":"2024-09-12T02:56:31.035754+0000","last_became_peered":"2024-09-12T02:56:31.035754+0000","last_unstale":"2024-09-12T02:57:59.565986+0000","last_undegraded":"2024-09-12T02:57:59.565986+0000","last_fullsized":"2024-09-12T02:57:59.565986+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:58:06.464818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1a","version":"81'7","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.032813+0000","last_change":"2024-09-12T02:56:13.931202+0000","last_active":"2024-09-12T02:56:31.032813+0000","last_peered":"2024-09-12T02:56:31.032813+0000","last_clean":"2024-09-12T02:56:31.032813+0000","last_became_active":"2024-09-12T02:56:13.930776+0000","last_became_peered":"2024-09-12T02:56:13.930776+0000","last_unstale":"2024-09-12T02:56:31.032813+0000","last_undegraded":"2024-09-12T02:56:31.032813+0000","last_fullsized":"2024-09-12T02:56:31.032813+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:22:16.919253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"81'3","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.031662+0000","last_change":"2024-09-12T02:56:13.924660+0000","last_active":"2024-09-12T02:56:31.031662+0000","last_peered":"2024-09-12T02:56:31.031662+0000","last_clean":"2024-09-12T02:56:31.031662+0000","last_became_active":"2024-09-12T02:56:13.924342+0000","last_became_peered":"2024-09-12T02:56:13.924342+0000","last_unstale":"2024-09-12T02:56:31.031662+0000","last_undegraded":"2024-09-12T02:56:31.031662+0000","last_fullsized":"2024-09-12T02:56:31.031662+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:48:53.907044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.563814+0000","last_change":"2024-09-12T02:56:15.978723+0000","last_active":"2024-09-12T02:57:59.563814+0000","last_peered":"2024-09-12T02:57:59.563814+0000","last_clean":"2024-09-12T02:57:59.563814+0000","last_became_active":"2024-09-12T02:56:15.978344+0000","last_became_peered":"2024-09-12T02:56:15.978344+0000","last_unstale":"2024-09-12T02:57:59.563814+0000","last_undegraded":"2024-09-12T02:57:59.563814+0000","last_fullsized":"2024-09-12T02:57:59.563814+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:25:04.559395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"81'3","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.032936+0000","last_change":"2024-09-12T02:56:13.933065+0000","last_active":"2024-09-12T02:56:31.032936+0000","last_peered":"2024-09-12T02:56:31.032936+0000","last_clean":"2024-09-12T02:56:31.032936+0000","last_became_active":"2024-09-12T02:56:13.932597+0000","last_became_peered":"2024-09-12T02:56:13.932597+0000","last_unstale":"2024-09-12T02:56:31.032936+0000","last_undegraded":"2024-09-12T02:56:31.032936+0000","last_fullsized":"2024-09-12T02:56:31.032936+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:57:26.563392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"83'11","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.566801+0000","last_change":"2024-09-12T02:56:15.975722+0000","last_active":"2024-09-12T02:57:59.566801+0000","last_peered":"2024-09-12T02:57:59.566801+0000","last_clean":"2024-09-12T02:57:59.566801+0000","last_became_active":"2024-09-12T02:56:15.975342+0000","last_became_peered":"2024-09-12T02:56:15.975342+0000","last_unstale":"2024-09-12T02:57:59.566801+0000","last_undegraded":"2024-09-12T02:57:59.566801+0000","last_fullsized":"2024-09-12T02:57:59.566801+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:33:17.613738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'6","reported_seq":238,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:58:11.991007+0000","last_change":"2024-09-12T02:56:13.929700+0000","last_active":"2024-09-12T02:58:11.991007+0000","last_peered":"2024-09-12T02:58:11.991007+0000","last_clean":"2024-09-12T02:58:11.991007+0000","last_became_active":"2024-09-12T02:56:13.929317+0000","last_became_peered":"2024-09-12T02:56:13.929317+0000","last_unstale":"2024-09-12T02:58:11.991007+0000","last_undegraded":"2024-09-12T02:58:11.991007+0000","last_fullsized":"2024-09-12T02:58:11.991007+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:46:32.172674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":210,"num_read_kb":210,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"83'10","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.564710+0000","last_change":"2024-09-12T02:56:15.985754+0000","last_active":"2024-09-12T02:57:59.564710+0000","last_peered":"2024-09-12T02:57:59.564710+0000","last_clean":"2024-09-12T02:57:59.564710+0000","last_became_active":"2024-09-12T02:56:15.985402+0000","last_became_peered":"2024-09-12T02:56:15.985402+0000","last_unstale":"2024-09-12T02:57:59.564710+0000","last_undegraded":"2024-09-12T02:57:59.564710+0000","last_fullsized":"2024-09-12T02:57:59.564710+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:05:29.979961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"81'4","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.035428+0000","last_change":"2024-09-12T02:56:13.933329+0000","last_active":"2024-09-12T02:56:31.035428+0000","last_peered":"2024-09-12T02:56:31.035428+0000","last_clean":"2024-09-12T02:56:31.035428+0000","last_became_active":"2024-09-12T02:56:13.932960+0000","last_became_peered":"2024-09-12T02:56:13.932960+0000","last_unstale":"2024-09-12T02:56:31.035428+0000","last_undegraded":"2024-09-12T02:56:31.035428+0000","last_fullsized":"2024-09-12T02:56:31.035428+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:02:57.564202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"83'9","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.556804+0000","last_change":"2024-09-12T02:56:31.037340+0000","last_active":"2024-09-12T02:57:59.556804+0000","last_peered":"2024-09-12T02:57:59.556804+0000","last_clean":"2024-09-12T02:57:59.556804+0000","last_became_active":"2024-09-12T02:56:31.036993+0000","last_became_peered":"2024-09-12T02:56:31.036993+0000","last_unstale":"2024-09-12T02:57:59.556804+0000","last_undegraded":"2024-09-12T02:57:59.556804+0000","last_fullsized":"2024-09-12T02:57:59.556804+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:44:27.788791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"81'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.033005+0000","last_change":"2024-09-12T02:56:14.077202+0000","last_active":"2024-09-12T02:56:31.033005+0000","last_peered":"2024-09-12T02:56:31.033005+0000","last_clean":"2024-09-12T02:56:31.033005+0000","last_became_active":"2024-09-12T02:56:14.076610+0000","last_became_peered":"2024-09-12T02:56:14.076610+0000","last_unstale":"2024-09-12T02:56:31.033005+0000","last_undegraded":"2024-09-12T02:56:31.033005+0000","last_fullsized":"2024-09-12T02:56:31.033005+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:37:50.333152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"83'11","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.558754+0000","last_change":"2024-09-12T02:56:15.977161+0000","last_active":"2024-09-12T02:57:59.558754+0000","last_peered":"2024-09-12T02:57:59.558754+0000","last_clean":"2024-09-12T02:57:59.558754+0000","last_became_active":"2024-09-12T02:56:15.976683+0000","last_became_peered":"2024-09-12T02:56:15.976683+0000","last_unstale":"2024-09-12T02:57:59.558754+0000","last_undegraded":"2024-09-12T02:57:59.558754+0000","last_fullsized":"2024-09-12T02:57:59.558754+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:34:07.526890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'38","reported_seq":285,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:58:11.991482+0000","last_change":"2024-09-12T02:56:13.931680+0000","last_active":"2024-09-12T02:58:11.991482+0000","last_peered":"2024-09-12T02:58:11.991482+0000","last_clean":"2024-09-12T02:58:11.991482+0000","last_became_active":"2024-09-12T02:56:13.931214+0000","last_became_peered":"2024-09-12T02:56:13.931214+0000","last_unstale":"2024-09-12T02:58:11.991482+0000","last_undegraded":"2024-09-12T02:58:11.991482+0000","last_fullsized":"2024-09-12T02:58:11.991482+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:38:34.151933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68951,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":225,"num_read_kb":180,"num_write":40,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.555616+0000","last_change":"2024-09-12T02:56:15.977720+0000","last_active":"2024-09-12T02:57:59.555616+0000","last_peered":"2024-09-12T02:57:59.555616+0000","last_clean":"2024-09-12T02:57:59.555616+0000","last_became_active":"2024-09-12T02:56:15.977380+0000","last_became_peered":"2024-09-12T02:56:15.977380+0000","last_unstale":"2024-09-12T02:57:59.555616+0000","last_undegraded":"2024-09-12T02:57:59.555616+0000","last_fullsized":"2024-09-12T02:57:59.555616+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:52:16.841909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"81'9","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.030362+0000","last_change":"2024-09-12T02:56:13.929496+0000","last_active":"2024-09-12T02:56:31.030362+0000","last_peered":"2024-09-12T02:56:31.030362+0000","last_clean":"2024-09-12T02:56:31.030362+0000","last_became_active":"2024-09-12T02:56:13.929117+0000","last_became_peered":"2024-09-12T02:56:13.929117+0000","last_unstale":"2024-09-12T02:56:31.030362+0000","last_undegraded":"2024-09-12T02:56:31.030362+0000","last_fullsized":"2024-09-12T02:56:31.030362+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:59:38.399135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.564811+0000","last_change":"2024-09-12T02:56:15.974323+0000","last_active":"2024-09-12T02:57:59.564811+0000","last_peered":"2024-09-12T02:57:59.564811+0000","last_clean":"2024-09-12T02:57:59.564811+0000","last_became_active":"2024-09-12T02:56:15.973819+0000","last_became_peered":"2024-09-12T02:56:15.973819+0000","last_unstale":"2024-09-12T02:57:59.564811+0000","last_undegraded":"2024-09-12T02:57:59.564811+0000","last_fullsized":"2024-09-12T02:57:59.564811+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:53:13.130695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"81'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.032473+0000","last_change":"2024-09-12T02:56:13.935263+0000","last_active":"2024-09-12T02:56:31.032473+0000","last_peered":"2024-09-12T02:56:31.032473+0000","last_clean":"2024-09-12T02:56:31.032473+0000","last_became_active":"2024-09-12T02:56:13.934805+0000","last_became_peered":"2024-09-12T02:56:13.934805+0000","last_unstale":"2024-09-12T02:56:31.032473+0000","last_undegraded":"2024-09-12T02:56:31.032473+0000","last_fullsized":"2024-09-12T02:56:31.032473+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:39:42.882031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"83'15","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.560318+0000","last_change":"2024-09-12T02:56:15.976630+0000","last_active":"2024-09-12T02:57:59.560318+0000","last_peered":"2024-09-12T02:57:59.560318+0000","last_clean":"2024-09-12T02:57:59.560318+0000","last_became_active":"2024-09-12T02:56:15.976304+0000","last_became_peered":"2024-09-12T02:56:15.976304+0000","last_unstale":"2024-09-12T02:57:59.560318+0000","last_undegraded":"2024-09-12T02:57:59.560318+0000","last_fullsized":"2024-09-12T02:57:59.560318+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:46:03.366740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'8","reported_seq":138,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:58:08.987094+0000","last_change":"2024-09-12T02:56:13.934224+0000","last_active":"2024-09-12T02:58:08.987094+0000","last_peered":"2024-09-12T02:58:08.987094+0000","last_clean":"2024-09-12T02:58:08.987094+0000","last_became_active":"2024-09-12T02:56:13.933838+0000","last_became_peered":"2024-09-12T02:56:13.933838+0000","last_unstale":"2024-09-12T02:58:08.987094+0000","last_undegraded":"2024-09-12T02:58:08.987094+0000","last_fullsized":"2024-09-12T02:58:08.987094+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:34:34.206051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1515,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":109,"num_read_kb":6,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.557213+0000","last_change":"2024-09-12T02:56:15.980364+0000","last_active":"2024-09-12T02:57:59.557213+0000","last_peered":"2024-09-12T02:57:59.557213+0000","last_clean":"2024-09-12T02:57:59.557213+0000","last_became_active":"2024-09-12T02:56:15.979437+0000","last_became_peered":"2024-09-12T02:56:15.979437+0000","last_unstale":"2024-09-12T02:57:59.557213+0000","last_undegraded":"2024-09-12T02:57:59.557213+0000","last_fullsized":"2024-09-12T02:57:59.557213+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:43:33.424867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"81'17","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.050509+0000","last_change":"2024-09-12T02:56:31.050046+0000","last_active":"2024-09-12T02:56:31.050509+0000","last_peered":"2024-09-12T02:56:31.050509+0000","last_clean":"2024-09-12T02:56:31.050509+0000","last_became_active":"2024-09-12T02:56:31.036990+0000","last_became_peered":"2024-09-12T02:56:31.036990+0000","last_unstale":"2024-09-12T02:56:31.050509+0000","last_undegraded":"2024-09-12T02:56:31.050509+0000","last_fullsized":"2024-09-12T02:56:31.050509+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:05:53.336691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.563297+0000","last_change":"2024-09-12T02:56:15.986434+0000","last_active":"2024-09-12T02:57:59.563297+0000","last_peered":"2024-09-12T02:57:59.563297+0000","last_clean":"2024-09-12T02:57:59.563297+0000","last_became_active":"2024-09-12T02:56:15.986101+0000","last_became_peered":"2024-09-12T02:56:15.986101+0000","last_unstale":"2024-09-12T02:57:59.563297+0000","last_undegraded":"2024-09-12T02:57:59.563297+0000","last_fullsized":"2024-09-12T02:57:59.563297+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:51:27.840951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"81'2","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.035974+0000","last_change":"2024-09-12T02:56:13.925068+0000","last_active":"2024-09-12T02:56:31.035974+0000","last_peered":"2024-09-12T02:56:31.035974+0000","last_clean":"2024-09-12T02:56:31.035974+0000","last_became_active":"2024-09-12T02:56:13.924535+0000","last_became_peered":"2024-09-12T02:56:13.924535+0000","last_unstale":"2024-09-12T02:56:31.035974+0000","last_undegraded":"2024-09-12T02:56:31.035974+0000","last_fullsized":"2024-09-12T02:56:31.035974+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:45:32.478632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"83'5","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.559020+0000","last_change":"2024-09-12T02:56:15.983850+0000","last_active":"2024-09-12T02:57:59.559020+0000","last_peered":"2024-09-12T02:57:59.559020+0000","last_clean":"2024-09-12T02:57:59.559020+0000","last_became_active":"2024-09-12T02:56:15.983480+0000","last_became_peered":"2024-09-12T02:56:15.983480+0000","last_unstale":"2024-09-12T02:57:59.559020+0000","last_undegraded":"2024-09-12T02:57:59.559020+0000","last_fullsized":"2024-09-12T02:57:59.559020+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:09:42.185237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"81'5","reported_seq":39,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.312450+0000","last_change":"2024-09-12T02:56:31.312450+0000","last_active":"2024-09-12T02:56:31.312450+0000","last_peered":"2024-09-12T02:56:31.312450+0000","last_clean":"2024-09-12T02:56:31.312450+0000","last_became_active":"2024-09-12T02:56:31.308030+0000","last_became_peered":"2024-09-12T02:56:31.308030+0000","last_unstale":"2024-09-12T02:56:31.312450+0000","last_undegraded":"2024-09-12T02:56:31.312450+0000","last_fullsized":"2024-09-12T02:56:31.312450+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:15:14.692292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.562640+0000","last_change":"2024-09-12T02:56:15.978996+0000","last_active":"2024-09-12T02:57:59.562640+0000","last_peered":"2024-09-12T02:57:59.562640+0000","last_clean":"2024-09-12T02:57:59.562640+0000","last_became_active":"2024-09-12T02:56:15.978636+0000","last_became_peered":"2024-09-12T02:56:15.978636+0000","last_unstale":"2024-09-12T02:57:59.562640+0000","last_undegraded":"2024-09-12T02:57:59.562640+0000","last_fullsized":"2024-09-12T02:57:59.562640+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:29:20.401664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"81'3","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.033119+0000","last_change":"2024-09-12T02:56:13.929388+0000","last_active":"2024-09-12T02:56:31.033119+0000","last_peered":"2024-09-12T02:56:31.033119+0000","last_clean":"2024-09-12T02:56:31.033119+0000","last_became_active":"2024-09-12T02:56:13.928998+0000","last_became_peered":"2024-09-12T02:56:13.928998+0000","last_unstale":"2024-09-12T02:56:31.033119+0000","last_undegraded":"2024-09-12T02:56:31.033119+0000","last_fullsized":"2024-09-12T02:56:31.033119+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:37:57.368050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.560190+0000","last_change":"2024-09-12T02:56:15.977491+0000","last_active":"2024-09-12T02:57:59.560190+0000","last_peered":"2024-09-12T02:57:59.560190+0000","last_clean":"2024-09-12T02:57:59.560190+0000","last_became_active":"2024-09-12T02:56:15.977160+0000","last_became_peered":"2024-09-12T02:56:15.977160+0000","last_unstale":"2024-09-12T02:57:59.560190+0000","last_undegraded":"2024-09-12T02:57:59.560190+0000","last_fullsized":"2024-09-12T02:57:59.560190+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:05:40.577342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'7","reported_seq":138,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:58:10.990412+0000","last_change":"2024-09-12T02:56:14.327147+0000","last_active":"2024-09-12T02:58:10.990412+0000","last_peered":"2024-09-12T02:58:10.990412+0000","last_clean":"2024-09-12T02:58:10.990412+0000","last_became_active":"2024-09-12T02:56:14.326737+0000","last_became_peered":"2024-09-12T02:56:14.326737+0000","last_unstale":"2024-09-12T02:58:10.990412+0000","last_undegraded":"2024-09-12T02:58:10.990412+0000","last_fullsized":"2024-09-12T02:58:10.990412+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:55:33.510599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1704,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":109,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.563684+0000","last_change":"2024-09-12T02:56:15.984363+0000","last_active":"2024-09-12T02:57:59.563684+0000","last_peered":"2024-09-12T02:57:59.563684+0000","last_clean":"2024-09-12T02:57:59.563684+0000","last_became_active":"2024-09-12T02:56:15.983822+0000","last_became_peered":"2024-09-12T02:56:15.983822+0000","last_unstale":"2024-09-12T02:57:59.563684+0000","last_undegraded":"2024-09-12T02:57:59.563684+0000","last_fullsized":"2024-09-12T02:57:59.563684+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:41:46.082716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":333,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.035564+0000","last_change":"2024-09-12T02:55:05.800210+0000","last_active":"2024-09-12T02:56:31.035564+0000","last_peered":"2024-09-12T02:56:31.035564+0000","last_clean":"2024-09-12T02:56:31.035564+0000","last_became_active":"2024-09-12T02:55:05.799791+0000","last_became_peered":"2024-09-12T02:55:05.799791+0000","last_unstale":"2024-09-12T02:56:31.035564+0000","last_undegraded":"2024-09-12T02:56:31.035564+0000","last_fullsized":"2024-09-12T02:56:31.035564+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:51:48.530016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:51:48.530016+0000","last_clean_scrub_stamp":"2024-09-12T02:51:48.530016+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:05:47.416697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"81'6","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.031169+0000","last_change":"2024-09-12T02:56:13.930269+0000","last_active":"2024-09-12T02:56:31.031169+0000","last_peered":"2024-09-12T02:56:31.031169+0000","last_clean":"2024-09-12T02:56:31.031169+0000","last_became_active":"2024-09-12T02:56:13.929831+0000","last_became_peered":"2024-09-12T02:56:13.929831+0000","last_unstale":"2024-09-12T02:56:31.031169+0000","last_undegraded":"2024-09-12T02:56:31.031169+0000","last_fullsized":"2024-09-12T02:56:31.031169+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:38:17.878398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.561332+0000","last_change":"2024-09-12T02:56:15.969744+0000","last_active":"2024-09-12T02:57:59.561332+0000","last_peered":"2024-09-12T02:57:59.561332+0000","last_clean":"2024-09-12T02:57:59.561332+0000","last_became_active":"2024-09-12T02:56:15.969354+0000","last_became_peered":"2024-09-12T02:56:15.969354+0000","last_unstale":"2024-09-12T02:57:59.561332+0000","last_undegraded":"2024-09-12T02:57:59.561332+0000","last_fullsized":"2024-09-12T02:57:59.561332+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:26:02.409510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"81'4","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.035385+0000","last_change":"2024-09-12T02:56:13.929231+0000","last_active":"2024-09-12T02:56:31.035385+0000","last_peered":"2024-09-12T02:56:31.035385+0000","last_clean":"2024-09-12T02:56:31.035385+0000","last_became_active":"2024-09-12T02:56:13.928843+0000","last_became_peered":"2024-09-12T02:56:13.928843+0000","last_unstale":"2024-09-12T02:56:31.035385+0000","last_undegraded":"2024-09-12T02:56:31.035385+0000","last_fullsized":"2024-09-12T02:56:31.035385+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:26:33.619755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"83'5","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.555927+0000","last_change":"2024-09-12T02:56:15.982503+0000","last_active":"2024-09-12T02:57:59.555927+0000","last_peered":"2024-09-12T02:57:59.555927+0000","last_clean":"2024-09-12T02:57:59.555927+0000","last_became_active":"2024-09-12T02:56:15.982192+0000","last_became_peered":"2024-09-12T02:56:15.982192+0000","last_unstale":"2024-09-12T02:57:59.555927+0000","last_undegraded":"2024-09-12T02:57:59.555927+0000","last_fullsized":"2024-09-12T02:57:59.555927+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:55:18.370967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"83'6","reported_seq":132,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:58:08.986599+0000","last_change":"2024-09-12T02:56:14.075486+0000","last_active":"2024-09-12T02:58:08.986599+0000","last_peered":"2024-09-12T02:58:08.986599+0000","last_clean":"2024-09-12T02:58:08.986599+0000","last_became_active":"2024-09-12T02:56:14.075152+0000","last_became_peered":"2024-09-12T02:56:14.075152+0000","last_unstale":"2024-09-12T02:58:08.986599+0000","last_undegraded":"2024-09-12T02:58:08.986599+0000","last_fullsized":"2024-09-12T02:58:08.986599+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:59:40.950191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":104,"num_read_kb":104,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.556340+0000","last_change":"2024-09-12T02:56:15.981584+0000","last_active":"2024-09-12T02:57:59.556340+0000","last_peered":"2024-09-12T02:57:59.556340+0000","last_clean":"2024-09-12T02:57:59.556340+0000","last_became_active":"2024-09-12T02:56:15.981240+0000","last_became_peered":"2024-09-12T02:56:15.981240+0000","last_unstale":"2024-09-12T02:57:59.556340+0000","last_undegraded":"2024-09-12T02:57:59.556340+0000","last_fullsized":"2024-09-12T02:57:59.556340+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:49:04.036830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"83'7","reported_seq":134,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:58:07.985338+0000","last_change":"2024-09-12T02:56:13.935143+0000","last_active":"2024-09-12T02:58:07.985338+0000","last_peered":"2024-09-12T02:58:07.985338+0000","last_clean":"2024-09-12T02:58:07.985338+0000","last_became_active":"2024-09-12T02:56:13.934792+0000","last_became_peered":"2024-09-12T02:56:13.934792+0000","last_unstale":"2024-09-12T02:58:07.985338+0000","last_undegraded":"2024-09-12T02:58:07.985338+0000","last_fullsized":"2024-09-12T02:58:07.985338+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:00:26.973612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1289,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":105,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"83'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.558893+0000","last_change":"2024-09-12T02:56:31.034825+0000","last_active":"2024-09-12T02:57:59.558893+0000","last_peered":"2024-09-12T02:57:59.558893+0000","last_clean":"2024-09-12T02:57:59.558893+0000","last_became_active":"2024-09-12T02:56:31.034516+0000","last_became_peered":"2024-09-12T02:56:31.034516+0000","last_unstale":"2024-09-12T02:57:59.558893+0000","last_undegraded":"2024-09-12T02:57:59.558893+0000","last_fullsized":"2024-09-12T02:57:59.558893+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:10:42.161023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"81'1","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.029027+0000","last_change":"2024-09-12T02:56:14.076762+0000","last_active":"2024-09-12T02:56:31.029027+0000","last_peered":"2024-09-12T02:56:31.029027+0000","last_clean":"2024-09-12T02:56:31.029027+0000","last_became_active":"2024-09-12T02:56:14.076421+0000","last_became_peered":"2024-09-12T02:56:14.076421+0000","last_unstale":"2024-09-12T02:56:31.029027+0000","last_undegraded":"2024-09-12T02:56:31.029027+0000","last_fullsized":"2024-09-12T02:56:31.029027+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:56:56.422910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'5","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.555835+0000","last_change":"2024-09-12T02:56:15.978810+0000","last_active":"2024-09-12T02:57:59.555835+0000","last_peered":"2024-09-12T02:57:59.555835+0000","last_clean":"2024-09-12T02:57:59.555835+0000","last_became_active":"2024-09-12T02:56:15.978038+0000","last_became_peered":"2024-09-12T02:56:15.978038+0000","last_unstale":"2024-09-12T02:57:59.555835+0000","last_undegraded":"2024-09-12T02:57:59.555835+0000","last_fullsized":"2024-09-12T02:57:59.555835+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:04:52.215784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"81'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.035440+0000","last_change":"2024-09-12T02:56:14.075987+0000","last_active":"2024-09-12T02:56:31.035440+0000","last_peered":"2024-09-12T02:56:31.035440+0000","last_clean":"2024-09-12T02:56:31.035440+0000","last_became_active":"2024-09-12T02:56:14.075596+0000","last_became_peered":"2024-09-12T02:56:14.075596+0000","last_unstale":"2024-09-12T02:56:31.035440+0000","last_undegraded":"2024-09-12T02:56:31.035440+0000","last_fullsized":"2024-09-12T02:56:31.035440+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:18:48.713387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.552400+0000","last_change":"2024-09-12T02:56:15.987842+0000","last_active":"2024-09-12T02:57:59.552400+0000","last_peered":"2024-09-12T02:57:59.552400+0000","last_clean":"2024-09-12T02:57:59.552400+0000","last_became_active":"2024-09-12T02:56:15.987507+0000","last_became_peered":"2024-09-12T02:56:15.987507+0000","last_unstale":"2024-09-12T02:57:59.552400+0000","last_undegraded":"2024-09-12T02:57:59.552400+0000","last_fullsized":"2024-09-12T02:57:59.552400+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:52:23.000549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'19","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:58:01.991961+0000","last_change":"2024-09-12T02:56:13.924372+0000","last_active":"2024-09-12T02:58:01.991961+0000","last_peered":"2024-09-12T02:58:01.991961+0000","last_clean":"2024-09-12T02:58:01.991961+0000","last_became_active":"2024-09-12T02:56:13.923955+0000","last_became_peered":"2024-09-12T02:56:13.923955+0000","last_unstale":"2024-09-12T02:58:01.991961+0000","last_undegraded":"2024-09-12T02:58:01.991961+0000","last_fullsized":"2024-09-12T02:58:01.991961+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:52:47.452410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":34,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"83'5","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.562919+0000","last_change":"2024-09-12T02:56:15.978262+0000","last_active":"2024-09-12T02:57:59.562919+0000","last_peered":"2024-09-12T02:57:59.562919+0000","last_clean":"2024-09-12T02:57:59.562919+0000","last_became_active":"2024-09-12T02:56:15.977913+0000","last_became_peered":"2024-09-12T02:56:15.977913+0000","last_unstale":"2024-09-12T02:57:59.562919+0000","last_undegraded":"2024-09-12T02:57:59.562919+0000","last_fullsized":"2024-09-12T02:57:59.562919+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:35:27.104635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"81'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.031488+0000","last_change":"2024-09-12T02:56:14.327417+0000","last_active":"2024-09-12T02:56:31.031488+0000","last_peered":"2024-09-12T02:56:31.031488+0000","last_clean":"2024-09-12T02:56:31.031488+0000","last_became_active":"2024-09-12T02:56:14.327001+0000","last_became_peered":"2024-09-12T02:56:14.327001+0000","last_unstale":"2024-09-12T02:56:31.031488+0000","last_undegraded":"2024-09-12T02:56:31.031488+0000","last_fullsized":"2024-09-12T02:56:31.031488+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:15:39.019252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.565859+0000","last_change":"2024-09-12T02:56:15.977286+0000","last_active":"2024-09-12T02:57:59.565859+0000","last_peered":"2024-09-12T02:57:59.565859+0000","last_clean":"2024-09-12T02:57:59.565859+0000","last_became_active":"2024-09-12T02:56:15.976944+0000","last_became_peered":"2024-09-12T02:56:15.976944+0000","last_unstale":"2024-09-12T02:57:59.565859+0000","last_undegraded":"2024-09-12T02:57:59.565859+0000","last_fullsized":"2024-09-12T02:57:59.565859+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:22:57.056781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.306212+0000","last_change":"2024-09-12T02:56:14.075701+0000","last_active":"2024-09-12T02:56:31.306212+0000","last_peered":"2024-09-12T02:56:31.306212+0000","last_clean":"2024-09-12T02:56:31.306212+0000","last_became_active":"2024-09-12T02:56:14.075340+0000","last_became_peered":"2024-09-12T02:56:14.075340+0000","last_unstale":"2024-09-12T02:56:31.306212+0000","last_undegraded":"2024-09-12T02:56:31.306212+0000","last_fullsized":"2024-09-12T02:56:31.306212+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:17:42.255058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"83'2","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.528187+0000","last_change":"2024-09-12T02:56:15.979783+0000","last_active":"2024-09-12T02:57:59.528187+0000","last_peered":"2024-09-12T02:57:59.528187+0000","last_clean":"2024-09-12T02:57:59.528187+0000","last_became_active":"2024-09-12T02:56:15.979424+0000","last_became_peered":"2024-09-12T02:56:15.979424+0000","last_unstale":"2024-09-12T02:57:59.528187+0000","last_undegraded":"2024-09-12T02:57:59.528187+0000","last_fullsized":"2024-09-12T02:57:59.528187+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:59:11.568138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"81'7","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.034660+0000","last_change":"2024-09-12T02:56:13.928996+0000","last_active":"2024-09-12T02:56:31.034660+0000","last_peered":"2024-09-12T02:56:31.034660+0000","last_clean":"2024-09-12T02:56:31.034660+0000","last_became_active":"2024-09-12T02:56:13.928664+0000","last_became_peered":"2024-09-12T02:56:13.928664+0000","last_unstale":"2024-09-12T02:56:31.034660+0000","last_undegraded":"2024-09-12T02:56:31.034660+0000","last_fullsized":"2024-09-12T02:56:31.034660+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:37:22.067930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.560217+0000","last_change":"2024-09-12T02:56:15.984096+0000","last_active":"2024-09-12T02:57:59.560217+0000","last_peered":"2024-09-12T02:57:59.560217+0000","last_clean":"2024-09-12T02:57:59.560217+0000","last_became_active":"2024-09-12T02:56:15.983737+0000","last_became_peered":"2024-09-12T02:56:15.983737+0000","last_unstale":"2024-09-12T02:57:59.560217+0000","last_undegraded":"2024-09-12T02:57:59.560217+0000","last_fullsized":"2024-09-12T02:57:59.560217+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:02:17.892318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'4","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:33.042035+0000","last_change":"2024-09-12T02:56:31.053343+0000","last_active":"2024-09-12T02:56:33.042035+0000","last_peered":"2024-09-12T02:56:33.042035+0000","last_clean":"2024-09-12T02:56:33.042035+0000","last_became_active":"2024-09-12T02:56:31.037918+0000","last_became_peered":"2024-09-12T02:56:31.037918+0000","last_unstale":"2024-09-12T02:56:33.042035+0000","last_undegraded":"2024-09-12T02:56:33.042035+0000","last_fullsized":"2024-09-12T02:56:33.042035+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:39:54.990724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.563189+0000","last_change":"2024-09-12T02:56:15.975640+0000","last_active":"2024-09-12T02:57:59.563189+0000","last_peered":"2024-09-12T02:57:59.563189+0000","last_clean":"2024-09-12T02:57:59.563189+0000","last_became_active":"2024-09-12T02:56:15.975342+0000","last_became_peered":"2024-09-12T02:56:15.975342+0000","last_unstale":"2024-09-12T02:57:59.563189+0000","last_undegraded":"2024-09-12T02:57:59.563189+0000","last_fullsized":"2024-09-12T02:57:59.563189+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:26:23.694876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.034922+0000","last_change":"2024-09-12T02:56:13.934149+0000","last_active":"2024-09-12T02:56:31.034922+0000","last_peered":"2024-09-12T02:56:31.034922+0000","last_clean":"2024-09-12T02:56:31.034922+0000","last_became_active":"2024-09-12T02:56:13.933743+0000","last_became_peered":"2024-09-12T02:56:13.933743+0000","last_unstale":"2024-09-12T02:56:31.034922+0000","last_undegraded":"2024-09-12T02:56:31.034922+0000","last_fullsized":"2024-09-12T02:56:31.034922+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:16:17.854457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"83'5","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.563348+0000","last_change":"2024-09-12T02:56:15.984782+0000","last_active":"2024-09-12T02:57:59.563348+0000","last_peered":"2024-09-12T02:57:59.563348+0000","last_clean":"2024-09-12T02:57:59.563348+0000","last_became_active":"2024-09-12T02:56:15.984336+0000","last_became_peered":"2024-09-12T02:56:15.984336+0000","last_unstale":"2024-09-12T02:57:59.563348+0000","last_undegraded":"2024-09-12T02:57:59.563348+0000","last_fullsized":"2024-09-12T02:57:59.563348+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:03:39.832897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"83'15","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.564252+0000","last_change":"2024-09-12T02:56:15.985486+0000","last_active":"2024-09-12T02:57:59.564252+0000","last_peered":"2024-09-12T02:57:59.564252+0000","last_clean":"2024-09-12T02:57:59.564252+0000","last_became_active":"2024-09-12T02:56:15.985167+0000","last_became_peered":"2024-09-12T02:56:15.985167+0000","last_unstale":"2024-09-12T02:57:59.564252+0000","last_undegraded":"2024-09-12T02:57:59.564252+0000","last_fullsized":"2024-09-12T02:57:59.564252+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:46:20.569453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"81'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.035106+0000","last_change":"2024-09-12T02:56:13.929324+0000","last_active":"2024-09-12T02:56:31.035106+0000","last_peered":"2024-09-12T02:56:31.035106+0000","last_clean":"2024-09-12T02:56:31.035106+0000","last_became_active":"2024-09-12T02:56:13.928994+0000","last_became_peered":"2024-09-12T02:56:13.928994+0000","last_unstale":"2024-09-12T02:56:31.035106+0000","last_undegraded":"2024-09-12T02:56:31.035106+0000","last_fullsized":"2024-09-12T02:56:31.035106+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:30:27.290328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.562582+0000","last_change":"2024-09-12T02:56:15.985889+0000","last_active":"2024-09-12T02:57:59.562582+0000","last_peered":"2024-09-12T02:57:59.562582+0000","last_clean":"2024-09-12T02:57:59.562582+0000","last_became_active":"2024-09-12T02:56:15.985509+0000","last_became_peered":"2024-09-12T02:56:15.985509+0000","last_unstale":"2024-09-12T02:57:59.562582+0000","last_undegraded":"2024-09-12T02:57:59.562582+0000","last_fullsized":"2024-09-12T02:57:59.562582+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:30:43.120718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"83'13","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.035437+0000","last_change":"2024-09-12T02:56:13.934192+0000","last_active":"2024-09-12T02:56:31.035437+0000","last_peered":"2024-09-12T02:56:31.035437+0000","last_clean":"2024-09-12T02:56:31.035437+0000","last_became_active":"2024-09-12T02:56:13.933789+0000","last_became_peered":"2024-09-12T02:56:13.933789+0000","last_unstale":"2024-09-12T02:56:31.035437+0000","last_undegraded":"2024-09-12T02:56:31.035437+0000","last_fullsized":"2024-09-12T02:56:31.035437+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:32:47.771085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.563285+0000","last_change":"2024-09-12T02:56:15.983531+0000","last_active":"2024-09-12T02:57:59.563285+0000","last_peered":"2024-09-12T02:57:59.563285+0000","last_clean":"2024-09-12T02:57:59.563285+0000","last_became_active":"2024-09-12T02:56:15.983192+0000","last_became_peered":"2024-09-12T02:56:15.983192+0000","last_unstale":"2024-09-12T02:57:59.563285+0000","last_undegraded":"2024-09-12T02:57:59.563285+0000","last_fullsized":"2024-09-12T02:57:59.563285+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:16:59.923379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.034752+0000","last_change":"2024-09-12T02:56:14.075396+0000","last_active":"2024-09-12T02:56:31.034752+0000","last_peered":"2024-09-12T02:56:31.034752+0000","last_clean":"2024-09-12T02:56:31.034752+0000","last_became_active":"2024-09-12T02:56:14.075023+0000","last_became_peered":"2024-09-12T02:56:14.075023+0000","last_unstale":"2024-09-12T02:56:31.034752+0000","last_undegraded":"2024-09-12T02:56:31.034752+0000","last_fullsized":"2024-09-12T02:56:31.034752+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:41:45.760689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"83'5","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.564093+0000","last_change":"2024-09-12T02:56:15.987276+0000","last_active":"2024-09-12T02:57:59.564093+0000","last_peered":"2024-09-12T02:57:59.564093+0000","last_clean":"2024-09-12T02:57:59.564093+0000","last_became_active":"2024-09-12T02:56:15.986949+0000","last_became_peered":"2024-09-12T02:56:15.986949+0000","last_unstale":"2024-09-12T02:57:59.564093+0000","last_undegraded":"2024-09-12T02:57:59.564093+0000","last_fullsized":"2024-09-12T02:57:59.564093+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:49:45.627093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"83'14","reported_seq":139,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:58:07.984885+0000","last_change":"2024-09-12T02:56:13.932365+0000","last_active":"2024-09-12T02:58:07.984885+0000","last_peered":"2024-09-12T02:58:07.984885+0000","last_clean":"2024-09-12T02:58:07.984885+0000","last_became_active":"2024-09-12T02:56:13.932062+0000","last_became_peered":"2024-09-12T02:56:13.932062+0000","last_unstale":"2024-09-12T02:58:07.984885+0000","last_undegraded":"2024-09-12T02:58:07.984885+0000","last_fullsized":"2024-09-12T02:58:07.984885+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:17:08.451839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":103,"num_read_kb":103,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":77928,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1091,"num_read_kb":632,"num_write":273,"num_write_kb":162,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":734,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":589824,"data_stored":233784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":233,"ondisk_log_size":233,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547237,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":917504,"data_stored":440831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776169,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32764,"kb_used_data":1580,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712388,"statfs":{"total":95995035648,"available":95961485312,"internally_reserved":0,"allocated":1617920,"data_stored":1101092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939694,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40416,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704736,"statfs":{"total":95995035648,"available":95953649664,"internally_reserved":0,"allocated":933888,"data_stored":440115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":40434112},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168627,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":921600,"data_stored":440381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430263,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28516,"kb_used_data":1492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716636,"statfs":{"total":95995035648,"available":95965835264,"internally_reserved":0,"allocated":1527808,"data_stored":1034641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593789,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":917504,"data_stored":440533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822721,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28528,"kb_used_data":1504,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716624,"statfs":{"total":95995035648,"available":95965822976,"internally_reserved":0,"allocated":1540096,"data_stored":1032691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953543,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":925696,"data_stored":441635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182475,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":933888,"data_stored":439657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411409,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36320,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708832,"statfs":{"total":95995035648,"available":95957843968,"internally_reserved":0,"allocated":999424,"data_stored":509828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574933,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":987136,"data_stored":510397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738459,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":937984,"data_stored":441705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":71404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":70835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":71614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T02:58:15.119 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-12T02:58:15.120 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-12T02:58:15.378 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:58:15.665 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:58:15 smithi148 ceph-mon[35090]: pgmap v335: 97 pgs: 97 active+clean; 653 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 2.1 KiB/s wr, 13 op/s 2024-09-12T02:58:15.665 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:58:15 smithi148 ceph-mon[35090]: from='client.15312 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:58:15.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:58:15 smithi146 ceph-mon[35011]: pgmap v335: 97 pgs: 97 active+clean; 653 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 2.1 KiB/s wr, 13 op/s 2024-09-12T02:58:15.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:58:15 smithi146 ceph-mon[35011]: from='client.15312 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:58:15.766 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:58:15 smithi005 ceph-mon[31197]: pgmap v335: 97 pgs: 97 active+clean; 653 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 2.1 KiB/s wr, 13 op/s 2024-09-12T02:58:15.766 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:58:15 smithi005 ceph-mon[31197]: from='client.15312 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:58:15.795 INFO:tasks.cephfs.filesystem:scrub status for tag:9734a5ee-e292-4bee-b391-561c540e414e - {'path': '/', 'tag': '9734a5ee-e292-4bee-b391-561c540e414e', 'options': 'recursive,force'} 2024-09-12T02:58:16.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:16.244+0000 7f6d0a4b3640 1 -- 172.21.15.5:0/845730604 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d04100e20 msgr2=0x7f6d04101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:16.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:16.244+0000 7f6d0a4b3640 1 --2- 172.21.15.5:0/845730604 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d04100e20 0x7f6d04101200 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f6cf4009900 tx=0x7f6cf402a480 comp rx=0 tx=0).stop 2024-09-12T02:58:16.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:16.244+0000 7f6d0a4b3640 1 -- 172.21.15.5:0/845730604 shutdown_connections 2024-09-12T02:58:16.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:16.244+0000 7f6d0a4b3640 1 --2- 172.21.15.5:0/845730604 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d04102ef0 0x7f6d0410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:16.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:16.244+0000 7f6d0a4b3640 1 --2- 172.21.15.5:0/845730604 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6d04102570 0x7f6d041029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:16.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:16.244+0000 7f6d0a4b3640 1 --2- 172.21.15.5:0/845730604 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d04100e20 0x7f6d04101200 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:16.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:16.244+0000 7f6d0a4b3640 1 -- 172.21.15.5:0/845730604 >> 172.21.15.5:0/845730604 conn(0x7f6d040fc7d0 msgr2=0x7f6d040febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:16.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:16.244+0000 7f6d0a4b3640 1 -- 172.21.15.5:0/845730604 shutdown_connections 2024-09-12T02:58:16.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:16.244+0000 7f6d0a4b3640 1 -- 172.21.15.5:0/845730604 wait complete. 2024-09-12T02:58:16.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:16.244+0000 7f6d0a4b3640 1 Processor -- start 2024-09-12T02:58:16.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:16.244+0000 7f6d0a4b3640 1 -- start start 2024-09-12T02:58:16.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:16.245+0000 7f6d0a4b3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d04100e20 0x7f6d041984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:16.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:16.245+0000 7f6d0a4b3640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d04102570 0x7f6d04198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:16.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:16.245+0000 7f6d0a4b3640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6d04102ef0 0x7f6d0419cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:16.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:16.245+0000 7f6d0a4b3640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6d04110f80 con 0x7f6d04100e20 2024-09-12T02:58:16.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:16.245+0000 7f6d0a4b3640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f6d04110e00 con 0x7f6d04102ef0 2024-09-12T02:58:16.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:16.245+0000 7f6d0a4b3640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f6d04111100 con 0x7f6d04102570 2024-09-12T02:58:16.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:16.245+0000 7f6d03fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d04100e20 0x7f6d041984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:16.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:16.245+0000 7f6d03fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d04100e20 0x7f6d041984d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50612/0 (socket says 172.21.15.5:50612) 2024-09-12T02:58:16.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:16.245+0000 7f6d03fff640 1 -- 172.21.15.5:0/2703261201 learned_addr learned my addr 172.21.15.5:0/2703261201 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:58:16.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:16.245+0000 7f6d08a50640 1 --2- 172.21.15.5:0/2703261201 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6d04102ef0 0x7f6d0419cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:16.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:16.245+0000 7f6d037fe640 1 --2- 172.21.15.5:0/2703261201 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d04102570 0x7f6d04198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:16.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:16.245+0000 7f6d03fff640 1 -- 172.21.15.5:0/2703261201 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d04102570 msgr2=0x7f6d04198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:16.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:16.245+0000 7f6d03fff640 1 --2- 172.21.15.5:0/2703261201 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d04102570 0x7f6d04198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:16.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:16.245+0000 7f6d03fff640 1 -- 172.21.15.5:0/2703261201 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6d04102ef0 msgr2=0x7f6d0419cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:16.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:16.245+0000 7f6d03fff640 1 --2- 172.21.15.5:0/2703261201 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6d04102ef0 0x7f6d0419cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:16.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:16.245+0000 7f6d03fff640 1 -- 172.21.15.5:0/2703261201 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d0419d480 con 0x7f6d04100e20 2024-09-12T02:58:16.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:16.245+0000 7f6d08a50640 1 --2- 172.21.15.5:0/2703261201 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6d04102ef0 0x7f6d0419cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:58:16.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:16.245+0000 7f6d03fff640 1 --2- 172.21.15.5:0/2703261201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d04100e20 0x7f6d041984d0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f6cf4006010 tx=0x7f6cf4003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:16.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:16.246+0000 7f6d017fa640 1 -- 172.21.15.5:0/2703261201 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cf403f070 con 0x7f6d04100e20 2024-09-12T02:58:16.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:16.246+0000 7f6d017fa640 1 -- 172.21.15.5:0/2703261201 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6cf4031c30 con 0x7f6d04100e20 2024-09-12T02:58:16.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:16.246+0000 7f6d017fa640 1 -- 172.21.15.5:0/2703261201 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cf4035030 con 0x7f6d04100e20 2024-09-12T02:58:16.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:16.246+0000 7f6d0a4b3640 1 -- 172.21.15.5:0/2703261201 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d0419d710 con 0x7f6d04100e20 2024-09-12T02:58:16.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:16.246+0000 7f6d0a4b3640 1 -- 172.21.15.5:0/2703261201 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d0419dba0 con 0x7f6d04100e20 2024-09-12T02:58:16.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:16.247+0000 7f6d0a4b3640 1 -- 172.21.15.5:0/2703261201 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d04101200 con 0x7f6d04100e20 2024-09-12T02:58:16.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:16.248+0000 7f6d017fa640 1 -- 172.21.15.5:0/2703261201 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7f6cf4042070 con 0x7f6d04100e20 2024-09-12T02:58:16.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:16.249+0000 7f6d017fa640 1 --2- 172.21.15.5:0/2703261201 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f6cec0a2c10 0x7f6cec0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:16.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:16.249+0000 7f6d037fe640 1 --2- 172.21.15.5:0/2703261201 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f6cec0a2c10 0x7f6cec0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:16.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:16.249+0000 7f6d017fa640 1 -- 172.21.15.5:0/2703261201 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f6cf40efb30 con 0x7f6d04100e20 2024-09-12T02:58:16.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:16.249+0000 7f6d037fe640 1 --2- 172.21.15.5:0/2703261201 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f6cec0a2c10 0x7f6cec0a50d0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f6cf8005fd0 tx=0x7f6cf8005850 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:16.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:16.253+0000 7f6d017fa640 1 -- 172.21.15.5:0/2703261201 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6cf4031de0 con 0x7f6d04100e20 2024-09-12T02:58:16.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:16.424+0000 7f6d0a4b3640 1 -- 172.21.15.5:0/2703261201 --> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6d041068a0 con 0x7f6cec0a2c10 2024-09-12T02:58:16.431 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:16.431+0000 7f6d017fa640 1 -- 172.21.15.5:0/2703261201 <== mgr.14148 v2:172.21.15.5:6800/3780631642 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258894 (secure 0 0 0) 0x7f6d041068a0 con 0x7f6cec0a2c10 2024-09-12T02:58:16.433 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-12T02:58:16.434 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2024-09-12T02:58:16.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:16.433+0000 7f6d0a4b3640 1 -- 172.21.15.5:0/2703261201 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f6cec0a2c10 msgr2=0x7f6cec0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:16.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:16.433+0000 7f6d0a4b3640 1 --2- 172.21.15.5:0/2703261201 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f6cec0a2c10 0x7f6cec0a50d0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f6cf8005fd0 tx=0x7f6cf8005850 comp rx=0 tx=0).stop 2024-09-12T02:58:16.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:16.433+0000 7f6d0a4b3640 1 -- 172.21.15.5:0/2703261201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d04100e20 msgr2=0x7f6d041984d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:16.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:16.433+0000 7f6d0a4b3640 1 --2- 172.21.15.5:0/2703261201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d04100e20 0x7f6d041984d0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f6cf4006010 tx=0x7f6cf4003c10 comp rx=0 tx=0).stop 2024-09-12T02:58:16.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:16.434+0000 7f6d0a4b3640 1 -- 172.21.15.5:0/2703261201 shutdown_connections 2024-09-12T02:58:16.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:16.434+0000 7f6d0a4b3640 1 --2- 172.21.15.5:0/2703261201 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6d04102ef0 0x7f6d0419cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:16.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:16.434+0000 7f6d0a4b3640 1 --2- 172.21.15.5:0/2703261201 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6d04102570 0x7f6d04198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:16.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:16.434+0000 7f6d0a4b3640 1 --2- 172.21.15.5:0/2703261201 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f6cec0a2c10 0x7f6cec0a50d0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:16.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:16.434+0000 7f6d0a4b3640 1 --2- 172.21.15.5:0/2703261201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d04100e20 0x7f6d041984d0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:16.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:16.434+0000 7f6d0a4b3640 1 -- 172.21.15.5:0/2703261201 >> 172.21.15.5:0/2703261201 conn(0x7f6d040fc7d0 msgr2=0x7f6d0406a180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:16.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:16.434+0000 7f6d0a4b3640 1 -- 172.21.15.5:0/2703261201 shutdown_connections 2024-09-12T02:58:16.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:16.434+0000 7f6d0a4b3640 1 -- 172.21.15.5:0/2703261201 wait complete. 2024-09-12T02:58:17.152 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":336,"stamp":"2024-09-12T02:58:15.571092+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":668530,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1233,"num_read_kb":752,"num_write":456,"num_write_kb":3102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":1181470,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":872,"ondisk_log_size":872,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":374580,"kb_used_data":12852,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124567244,"statfs":{"total":1151940427776,"available":1151556857856,"internally_reserved":0,"allocated":13160448,"data_stored":7273506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19141,"internal_metadata":370193723},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4400,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":124,"num_read_kb":75,"num_write":18,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002880"},"pg_stats":[{"pgid":"3.3f","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.565050+0000","last_change":"2024-09-12T02:56:15.988690+0000","last_active":"2024-09-12T02:57:59.565050+0000","last_peered":"2024-09-12T02:57:59.565050+0000","last_clean":"2024-09-12T02:57:59.565050+0000","last_became_active":"2024-09-12T02:56:15.988320+0000","last_became_peered":"2024-09-12T02:56:15.988320+0000","last_unstale":"2024-09-12T02:57:59.565050+0000","last_undegraded":"2024-09-12T02:57:59.565050+0000","last_fullsized":"2024-09-12T02:57:59.565050+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:31:14.157622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"83'4","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.553842+0000","last_change":"2024-09-12T02:56:15.978368+0000","last_active":"2024-09-12T02:57:59.553842+0000","last_peered":"2024-09-12T02:57:59.553842+0000","last_clean":"2024-09-12T02:57:59.553842+0000","last_became_active":"2024-09-12T02:56:15.978092+0000","last_became_peered":"2024-09-12T02:56:15.978092+0000","last_unstale":"2024-09-12T02:57:59.553842+0000","last_undegraded":"2024-09-12T02:57:59.553842+0000","last_fullsized":"2024-09-12T02:57:59.553842+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:32:48.399005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.564003+0000","last_change":"2024-09-12T02:56:15.987249+0000","last_active":"2024-09-12T02:57:59.564003+0000","last_peered":"2024-09-12T02:57:59.564003+0000","last_clean":"2024-09-12T02:57:59.564003+0000","last_became_active":"2024-09-12T02:56:15.986921+0000","last_became_peered":"2024-09-12T02:56:15.986921+0000","last_unstale":"2024-09-12T02:57:59.564003+0000","last_undegraded":"2024-09-12T02:57:59.564003+0000","last_fullsized":"2024-09-12T02:57:59.564003+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:42:13.427871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.555703+0000","last_change":"2024-09-12T02:56:15.987673+0000","last_active":"2024-09-12T02:57:59.555703+0000","last_peered":"2024-09-12T02:57:59.555703+0000","last_clean":"2024-09-12T02:57:59.555703+0000","last_became_active":"2024-09-12T02:56:15.986964+0000","last_became_peered":"2024-09-12T02:56:15.986964+0000","last_unstale":"2024-09-12T02:57:59.555703+0000","last_undegraded":"2024-09-12T02:57:59.555703+0000","last_fullsized":"2024-09-12T02:57:59.555703+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:07:12.406974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"83'4","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.548461+0000","last_change":"2024-09-12T02:56:15.974473+0000","last_active":"2024-09-12T02:57:59.548461+0000","last_peered":"2024-09-12T02:57:59.548461+0000","last_clean":"2024-09-12T02:57:59.548461+0000","last_became_active":"2024-09-12T02:56:15.974124+0000","last_became_peered":"2024-09-12T02:56:15.974124+0000","last_unstale":"2024-09-12T02:57:59.548461+0000","last_undegraded":"2024-09-12T02:57:59.548461+0000","last_fullsized":"2024-09-12T02:57:59.548461+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:44:29.054841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.560740+0000","last_change":"2024-09-12T02:56:15.980242+0000","last_active":"2024-09-12T02:57:59.560740+0000","last_peered":"2024-09-12T02:57:59.560740+0000","last_clean":"2024-09-12T02:57:59.560740+0000","last_became_active":"2024-09-12T02:56:15.979937+0000","last_became_peered":"2024-09-12T02:56:15.979937+0000","last_unstale":"2024-09-12T02:57:59.560740+0000","last_undegraded":"2024-09-12T02:57:59.560740+0000","last_fullsized":"2024-09-12T02:57:59.560740+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:44:13.406341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"83'5","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.556845+0000","last_change":"2024-09-12T02:56:15.978746+0000","last_active":"2024-09-12T02:57:59.556845+0000","last_peered":"2024-09-12T02:57:59.556845+0000","last_clean":"2024-09-12T02:57:59.556845+0000","last_became_active":"2024-09-12T02:56:15.978304+0000","last_became_peered":"2024-09-12T02:56:15.978304+0000","last_unstale":"2024-09-12T02:57:59.556845+0000","last_undegraded":"2024-09-12T02:57:59.556845+0000","last_fullsized":"2024-09-12T02:57:59.556845+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:34:52.115630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.562507+0000","last_change":"2024-09-12T02:56:15.980083+0000","last_active":"2024-09-12T02:57:59.562507+0000","last_peered":"2024-09-12T02:57:59.562507+0000","last_clean":"2024-09-12T02:57:59.562507+0000","last_became_active":"2024-09-12T02:56:15.979771+0000","last_became_peered":"2024-09-12T02:56:15.979771+0000","last_unstale":"2024-09-12T02:57:59.562507+0000","last_undegraded":"2024-09-12T02:57:59.562507+0000","last_fullsized":"2024-09-12T02:57:59.562507+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:59:42.351970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"83'11","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.565703+0000","last_change":"2024-09-12T02:56:15.988091+0000","last_active":"2024-09-12T02:57:59.565703+0000","last_peered":"2024-09-12T02:57:59.565703+0000","last_clean":"2024-09-12T02:57:59.565703+0000","last_became_active":"2024-09-12T02:56:15.986295+0000","last_became_peered":"2024-09-12T02:56:15.986295+0000","last_unstale":"2024-09-12T02:57:59.565703+0000","last_undegraded":"2024-09-12T02:57:59.565703+0000","last_fullsized":"2024-09-12T02:57:59.565703+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:45:40.331805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"83'16","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.562128+0000","last_change":"2024-09-12T02:56:15.987349+0000","last_active":"2024-09-12T02:57:59.562128+0000","last_peered":"2024-09-12T02:57:59.562128+0000","last_clean":"2024-09-12T02:57:59.562128+0000","last_became_active":"2024-09-12T02:56:15.987018+0000","last_became_peered":"2024-09-12T02:56:15.987018+0000","last_unstale":"2024-09-12T02:57:59.562128+0000","last_undegraded":"2024-09-12T02:57:59.562128+0000","last_fullsized":"2024-09-12T02:57:59.562128+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:34:19.652348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.563544+0000","last_change":"2024-09-12T02:56:15.985164+0000","last_active":"2024-09-12T02:57:59.563544+0000","last_peered":"2024-09-12T02:57:59.563544+0000","last_clean":"2024-09-12T02:57:59.563544+0000","last_became_active":"2024-09-12T02:56:15.984787+0000","last_became_peered":"2024-09-12T02:56:15.984787+0000","last_unstale":"2024-09-12T02:57:59.563544+0000","last_undegraded":"2024-09-12T02:57:59.563544+0000","last_fullsized":"2024-09-12T02:57:59.563544+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:36:29.727442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.552957+0000","last_change":"2024-09-12T02:56:15.979963+0000","last_active":"2024-09-12T02:57:59.552957+0000","last_peered":"2024-09-12T02:57:59.552957+0000","last_clean":"2024-09-12T02:57:59.552957+0000","last_became_active":"2024-09-12T02:56:15.979639+0000","last_became_peered":"2024-09-12T02:56:15.979639+0000","last_unstale":"2024-09-12T02:57:59.552957+0000","last_undegraded":"2024-09-12T02:57:59.552957+0000","last_fullsized":"2024-09-12T02:57:59.552957+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:34:18.273245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"83'10","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.561647+0000","last_change":"2024-09-12T02:56:15.980415+0000","last_active":"2024-09-12T02:57:59.561647+0000","last_peered":"2024-09-12T02:57:59.561647+0000","last_clean":"2024-09-12T02:57:59.561647+0000","last_became_active":"2024-09-12T02:56:15.980059+0000","last_became_peered":"2024-09-12T02:56:15.980059+0000","last_unstale":"2024-09-12T02:57:59.561647+0000","last_undegraded":"2024-09-12T02:57:59.561647+0000","last_fullsized":"2024-09-12T02:57:59.561647+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:11:42.689081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.553574+0000","last_change":"2024-09-12T02:56:15.981423+0000","last_active":"2024-09-12T02:57:59.553574+0000","last_peered":"2024-09-12T02:57:59.553574+0000","last_clean":"2024-09-12T02:57:59.553574+0000","last_became_active":"2024-09-12T02:56:15.981186+0000","last_became_peered":"2024-09-12T02:56:15.981186+0000","last_unstale":"2024-09-12T02:57:59.553574+0000","last_undegraded":"2024-09-12T02:57:59.553574+0000","last_fullsized":"2024-09-12T02:57:59.553574+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:36:57.406907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.547696+0000","last_change":"2024-09-12T02:56:15.982587+0000","last_active":"2024-09-12T02:57:59.547696+0000","last_peered":"2024-09-12T02:57:59.547696+0000","last_clean":"2024-09-12T02:57:59.547696+0000","last_became_active":"2024-09-12T02:56:15.982315+0000","last_became_peered":"2024-09-12T02:56:15.982315+0000","last_unstale":"2024-09-12T02:57:59.547696+0000","last_undegraded":"2024-09-12T02:57:59.547696+0000","last_fullsized":"2024-09-12T02:57:59.547696+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:42:17.736999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.564760+0000","last_change":"2024-09-12T02:56:15.979482+0000","last_active":"2024-09-12T02:57:59.564760+0000","last_peered":"2024-09-12T02:57:59.564760+0000","last_clean":"2024-09-12T02:57:59.564760+0000","last_became_active":"2024-09-12T02:56:15.979122+0000","last_became_peered":"2024-09-12T02:56:15.979122+0000","last_unstale":"2024-09-12T02:57:59.564760+0000","last_undegraded":"2024-09-12T02:57:59.564760+0000","last_fullsized":"2024-09-12T02:57:59.564760+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:18:05.642487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.565218+0000","last_change":"2024-09-12T02:56:15.989908+0000","last_active":"2024-09-12T02:57:59.565218+0000","last_peered":"2024-09-12T02:57:59.565218+0000","last_clean":"2024-09-12T02:57:59.565218+0000","last_became_active":"2024-09-12T02:56:15.989515+0000","last_became_peered":"2024-09-12T02:56:15.989515+0000","last_unstale":"2024-09-12T02:57:59.565218+0000","last_undegraded":"2024-09-12T02:57:59.565218+0000","last_fullsized":"2024-09-12T02:57:59.565218+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:59:57.141607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"83'10","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.557609+0000","last_change":"2024-09-12T02:56:15.980323+0000","last_active":"2024-09-12T02:57:59.557609+0000","last_peered":"2024-09-12T02:57:59.557609+0000","last_clean":"2024-09-12T02:57:59.557609+0000","last_became_active":"2024-09-12T02:56:15.979989+0000","last_became_peered":"2024-09-12T02:56:15.979989+0000","last_unstale":"2024-09-12T02:57:59.557609+0000","last_undegraded":"2024-09-12T02:57:59.557609+0000","last_fullsized":"2024-09-12T02:57:59.557609+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:55:56.064125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.556275+0000","last_change":"2024-09-12T02:56:15.986003+0000","last_active":"2024-09-12T02:57:59.556275+0000","last_peered":"2024-09-12T02:57:59.556275+0000","last_clean":"2024-09-12T02:57:59.556275+0000","last_became_active":"2024-09-12T02:56:15.985335+0000","last_became_peered":"2024-09-12T02:56:15.985335+0000","last_unstale":"2024-09-12T02:57:59.556275+0000","last_undegraded":"2024-09-12T02:57:59.556275+0000","last_fullsized":"2024-09-12T02:57:59.556275+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:51:47.944530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"83'8","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.554254+0000","last_change":"2024-09-12T02:56:31.036870+0000","last_active":"2024-09-12T02:57:59.554254+0000","last_peered":"2024-09-12T02:57:59.554254+0000","last_clean":"2024-09-12T02:57:59.554254+0000","last_became_active":"2024-09-12T02:56:31.036443+0000","last_became_peered":"2024-09-12T02:56:31.036443+0000","last_unstale":"2024-09-12T02:57:59.554254+0000","last_undegraded":"2024-09-12T02:57:59.554254+0000","last_fullsized":"2024-09-12T02:57:59.554254+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:49:29.326184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"83'11","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.555414+0000","last_change":"2024-09-12T02:56:15.974547+0000","last_active":"2024-09-12T02:57:59.555414+0000","last_peered":"2024-09-12T02:57:59.555414+0000","last_clean":"2024-09-12T02:57:59.555414+0000","last_became_active":"2024-09-12T02:56:15.974174+0000","last_became_peered":"2024-09-12T02:56:15.974174+0000","last_unstale":"2024-09-12T02:57:59.555414+0000","last_undegraded":"2024-09-12T02:57:59.555414+0000","last_fullsized":"2024-09-12T02:57:59.555414+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:22:16.954309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"83'10","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.565778+0000","last_change":"2024-09-12T02:56:15.981771+0000","last_active":"2024-09-12T02:57:59.565778+0000","last_peered":"2024-09-12T02:57:59.565778+0000","last_clean":"2024-09-12T02:57:59.565778+0000","last_became_active":"2024-09-12T02:56:15.980948+0000","last_became_peered":"2024-09-12T02:56:15.980948+0000","last_unstale":"2024-09-12T02:57:59.565778+0000","last_undegraded":"2024-09-12T02:57:59.565778+0000","last_fullsized":"2024-09-12T02:57:59.565778+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:34:35.961348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.562061+0000","last_change":"2024-09-12T02:56:15.982318+0000","last_active":"2024-09-12T02:57:59.562061+0000","last_peered":"2024-09-12T02:57:59.562061+0000","last_clean":"2024-09-12T02:57:59.562061+0000","last_became_active":"2024-09-12T02:56:15.982013+0000","last_became_peered":"2024-09-12T02:56:15.982013+0000","last_unstale":"2024-09-12T02:57:59.562061+0000","last_undegraded":"2024-09-12T02:57:59.562061+0000","last_fullsized":"2024-09-12T02:57:59.562061+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:10:50.895579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.559580+0000","last_change":"2024-09-12T02:56:15.969992+0000","last_active":"2024-09-12T02:57:59.559580+0000","last_peered":"2024-09-12T02:57:59.559580+0000","last_clean":"2024-09-12T02:57:59.559580+0000","last_became_active":"2024-09-12T02:56:15.969621+0000","last_became_peered":"2024-09-12T02:56:15.969621+0000","last_unstale":"2024-09-12T02:57:59.559580+0000","last_undegraded":"2024-09-12T02:57:59.559580+0000","last_fullsized":"2024-09-12T02:57:59.559580+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:47:40.206042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"83'2","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.560304+0000","last_change":"2024-09-12T02:56:15.982315+0000","last_active":"2024-09-12T02:57:59.560304+0000","last_peered":"2024-09-12T02:57:59.560304+0000","last_clean":"2024-09-12T02:57:59.560304+0000","last_became_active":"2024-09-12T02:56:15.981992+0000","last_became_peered":"2024-09-12T02:56:15.981992+0000","last_unstale":"2024-09-12T02:57:59.560304+0000","last_undegraded":"2024-09-12T02:57:59.560304+0000","last_fullsized":"2024-09-12T02:57:59.560304+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:54:04.188418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"83'15","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.560612+0000","last_change":"2024-09-12T02:56:15.980712+0000","last_active":"2024-09-12T02:57:59.560612+0000","last_peered":"2024-09-12T02:57:59.560612+0000","last_clean":"2024-09-12T02:57:59.560612+0000","last_became_active":"2024-09-12T02:56:15.980410+0000","last_became_peered":"2024-09-12T02:56:15.980410+0000","last_unstale":"2024-09-12T02:57:59.560612+0000","last_undegraded":"2024-09-12T02:57:59.560612+0000","last_fullsized":"2024-09-12T02:57:59.560612+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:13:53.844490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.557058+0000","last_change":"2024-09-12T02:56:15.981675+0000","last_active":"2024-09-12T02:57:59.557058+0000","last_peered":"2024-09-12T02:57:59.557058+0000","last_clean":"2024-09-12T02:57:59.557058+0000","last_became_active":"2024-09-12T02:56:15.981372+0000","last_became_peered":"2024-09-12T02:56:15.981372+0000","last_unstale":"2024-09-12T02:57:59.557058+0000","last_undegraded":"2024-09-12T02:57:59.557058+0000","last_fullsized":"2024-09-12T02:57:59.557058+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:39:17.399199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"83'4","reported_seq":21,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.557413+0000","last_change":"2024-09-12T02:56:15.972750+0000","last_active":"2024-09-12T02:57:59.557413+0000","last_peered":"2024-09-12T02:57:59.557413+0000","last_clean":"2024-09-12T02:57:59.557413+0000","last_became_active":"2024-09-12T02:56:15.972434+0000","last_became_peered":"2024-09-12T02:56:15.972434+0000","last_unstale":"2024-09-12T02:57:59.557413+0000","last_undegraded":"2024-09-12T02:57:59.557413+0000","last_fullsized":"2024-09-12T02:57:59.557413+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:59:22.784908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.545235+0000","last_change":"2024-09-12T02:56:15.973567+0000","last_active":"2024-09-12T02:57:59.545235+0000","last_peered":"2024-09-12T02:57:59.545235+0000","last_clean":"2024-09-12T02:57:59.545235+0000","last_became_active":"2024-09-12T02:56:15.973251+0000","last_became_peered":"2024-09-12T02:56:15.973251+0000","last_unstale":"2024-09-12T02:57:59.545235+0000","last_undegraded":"2024-09-12T02:57:59.545235+0000","last_fullsized":"2024-09-12T02:57:59.545235+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:16:39.536817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.540308+0000","last_change":"2024-09-12T02:56:15.986108+0000","last_active":"2024-09-12T02:57:59.540308+0000","last_peered":"2024-09-12T02:57:59.540308+0000","last_clean":"2024-09-12T02:57:59.540308+0000","last_became_active":"2024-09-12T02:56:15.985701+0000","last_became_peered":"2024-09-12T02:56:15.985701+0000","last_unstale":"2024-09-12T02:57:59.540308+0000","last_undegraded":"2024-09-12T02:57:59.540308+0000","last_fullsized":"2024-09-12T02:57:59.540308+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:13:59.377473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"83'12","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.555638+0000","last_change":"2024-09-12T02:56:15.980078+0000","last_active":"2024-09-12T02:57:59.555638+0000","last_peered":"2024-09-12T02:57:59.555638+0000","last_clean":"2024-09-12T02:57:59.555638+0000","last_became_active":"2024-09-12T02:56:15.979785+0000","last_became_peered":"2024-09-12T02:56:15.979785+0000","last_unstale":"2024-09-12T02:57:59.555638+0000","last_undegraded":"2024-09-12T02:57:59.555638+0000","last_fullsized":"2024-09-12T02:57:59.555638+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:00:19.252398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.559119+0000","last_change":"2024-09-12T02:56:15.984170+0000","last_active":"2024-09-12T02:57:59.559119+0000","last_peered":"2024-09-12T02:57:59.559119+0000","last_clean":"2024-09-12T02:57:59.559119+0000","last_became_active":"2024-09-12T02:56:15.983416+0000","last_became_peered":"2024-09-12T02:56:15.983416+0000","last_unstale":"2024-09-12T02:57:59.559119+0000","last_undegraded":"2024-09-12T02:57:59.559119+0000","last_fullsized":"2024-09-12T02:57:59.559119+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:13:24.723923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.556295+0000","last_change":"2024-09-12T02:56:15.980987+0000","last_active":"2024-09-12T02:57:59.556295+0000","last_peered":"2024-09-12T02:57:59.556295+0000","last_clean":"2024-09-12T02:57:59.556295+0000","last_became_active":"2024-09-12T02:56:15.980661+0000","last_became_peered":"2024-09-12T02:56:15.980661+0000","last_unstale":"2024-09-12T02:57:59.556295+0000","last_undegraded":"2024-09-12T02:57:59.556295+0000","last_fullsized":"2024-09-12T02:57:59.556295+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:19:49.294698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"83'6","reported_seq":147,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:58:10.188613+0000","last_change":"2024-09-12T02:56:31.042914+0000","last_active":"2024-09-12T02:58:10.188613+0000","last_peered":"2024-09-12T02:58:10.188613+0000","last_clean":"2024-09-12T02:58:10.188613+0000","last_became_active":"2024-09-12T02:56:31.037356+0000","last_became_peered":"2024-09-12T02:56:31.037356+0000","last_unstale":"2024-09-12T02:58:10.188613+0000","last_undegraded":"2024-09-12T02:58:10.188613+0000","last_fullsized":"2024-09-12T02:58:10.188613+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:17:48.582725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":107,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":622,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"83'8","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.565986+0000","last_change":"2024-09-12T02:56:31.036170+0000","last_active":"2024-09-12T02:57:59.565986+0000","last_peered":"2024-09-12T02:57:59.565986+0000","last_clean":"2024-09-12T02:57:59.565986+0000","last_became_active":"2024-09-12T02:56:31.035754+0000","last_became_peered":"2024-09-12T02:56:31.035754+0000","last_unstale":"2024-09-12T02:57:59.565986+0000","last_undegraded":"2024-09-12T02:57:59.565986+0000","last_fullsized":"2024-09-12T02:57:59.565986+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:58:06.464818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1a","version":"81'7","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.032813+0000","last_change":"2024-09-12T02:56:13.931202+0000","last_active":"2024-09-12T02:56:31.032813+0000","last_peered":"2024-09-12T02:56:31.032813+0000","last_clean":"2024-09-12T02:56:31.032813+0000","last_became_active":"2024-09-12T02:56:13.930776+0000","last_became_peered":"2024-09-12T02:56:13.930776+0000","last_unstale":"2024-09-12T02:56:31.032813+0000","last_undegraded":"2024-09-12T02:56:31.032813+0000","last_fullsized":"2024-09-12T02:56:31.032813+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:22:16.919253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"81'3","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.031662+0000","last_change":"2024-09-12T02:56:13.924660+0000","last_active":"2024-09-12T02:56:31.031662+0000","last_peered":"2024-09-12T02:56:31.031662+0000","last_clean":"2024-09-12T02:56:31.031662+0000","last_became_active":"2024-09-12T02:56:13.924342+0000","last_became_peered":"2024-09-12T02:56:13.924342+0000","last_unstale":"2024-09-12T02:56:31.031662+0000","last_undegraded":"2024-09-12T02:56:31.031662+0000","last_fullsized":"2024-09-12T02:56:31.031662+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:48:53.907044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.563814+0000","last_change":"2024-09-12T02:56:15.978723+0000","last_active":"2024-09-12T02:57:59.563814+0000","last_peered":"2024-09-12T02:57:59.563814+0000","last_clean":"2024-09-12T02:57:59.563814+0000","last_became_active":"2024-09-12T02:56:15.978344+0000","last_became_peered":"2024-09-12T02:56:15.978344+0000","last_unstale":"2024-09-12T02:57:59.563814+0000","last_undegraded":"2024-09-12T02:57:59.563814+0000","last_fullsized":"2024-09-12T02:57:59.563814+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:25:04.559395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"81'3","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.032936+0000","last_change":"2024-09-12T02:56:13.933065+0000","last_active":"2024-09-12T02:56:31.032936+0000","last_peered":"2024-09-12T02:56:31.032936+0000","last_clean":"2024-09-12T02:56:31.032936+0000","last_became_active":"2024-09-12T02:56:13.932597+0000","last_became_peered":"2024-09-12T02:56:13.932597+0000","last_unstale":"2024-09-12T02:56:31.032936+0000","last_undegraded":"2024-09-12T02:56:31.032936+0000","last_fullsized":"2024-09-12T02:56:31.032936+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:57:26.563392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"83'11","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.566801+0000","last_change":"2024-09-12T02:56:15.975722+0000","last_active":"2024-09-12T02:57:59.566801+0000","last_peered":"2024-09-12T02:57:59.566801+0000","last_clean":"2024-09-12T02:57:59.566801+0000","last_became_active":"2024-09-12T02:56:15.975342+0000","last_became_peered":"2024-09-12T02:56:15.975342+0000","last_unstale":"2024-09-12T02:57:59.566801+0000","last_undegraded":"2024-09-12T02:57:59.566801+0000","last_fullsized":"2024-09-12T02:57:59.566801+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:33:17.613738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'6","reported_seq":238,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:58:11.991007+0000","last_change":"2024-09-12T02:56:13.929700+0000","last_active":"2024-09-12T02:58:11.991007+0000","last_peered":"2024-09-12T02:58:11.991007+0000","last_clean":"2024-09-12T02:58:11.991007+0000","last_became_active":"2024-09-12T02:56:13.929317+0000","last_became_peered":"2024-09-12T02:56:13.929317+0000","last_unstale":"2024-09-12T02:58:11.991007+0000","last_undegraded":"2024-09-12T02:58:11.991007+0000","last_fullsized":"2024-09-12T02:58:11.991007+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:46:32.172674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":210,"num_read_kb":210,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"83'10","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.564710+0000","last_change":"2024-09-12T02:56:15.985754+0000","last_active":"2024-09-12T02:57:59.564710+0000","last_peered":"2024-09-12T02:57:59.564710+0000","last_clean":"2024-09-12T02:57:59.564710+0000","last_became_active":"2024-09-12T02:56:15.985402+0000","last_became_peered":"2024-09-12T02:56:15.985402+0000","last_unstale":"2024-09-12T02:57:59.564710+0000","last_undegraded":"2024-09-12T02:57:59.564710+0000","last_fullsized":"2024-09-12T02:57:59.564710+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:05:29.979961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"81'4","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.035428+0000","last_change":"2024-09-12T02:56:13.933329+0000","last_active":"2024-09-12T02:56:31.035428+0000","last_peered":"2024-09-12T02:56:31.035428+0000","last_clean":"2024-09-12T02:56:31.035428+0000","last_became_active":"2024-09-12T02:56:13.932960+0000","last_became_peered":"2024-09-12T02:56:13.932960+0000","last_unstale":"2024-09-12T02:56:31.035428+0000","last_undegraded":"2024-09-12T02:56:31.035428+0000","last_fullsized":"2024-09-12T02:56:31.035428+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:02:57.564202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"83'9","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.556804+0000","last_change":"2024-09-12T02:56:31.037340+0000","last_active":"2024-09-12T02:57:59.556804+0000","last_peered":"2024-09-12T02:57:59.556804+0000","last_clean":"2024-09-12T02:57:59.556804+0000","last_became_active":"2024-09-12T02:56:31.036993+0000","last_became_peered":"2024-09-12T02:56:31.036993+0000","last_unstale":"2024-09-12T02:57:59.556804+0000","last_undegraded":"2024-09-12T02:57:59.556804+0000","last_fullsized":"2024-09-12T02:57:59.556804+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:44:27.788791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"81'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.033005+0000","last_change":"2024-09-12T02:56:14.077202+0000","last_active":"2024-09-12T02:56:31.033005+0000","last_peered":"2024-09-12T02:56:31.033005+0000","last_clean":"2024-09-12T02:56:31.033005+0000","last_became_active":"2024-09-12T02:56:14.076610+0000","last_became_peered":"2024-09-12T02:56:14.076610+0000","last_unstale":"2024-09-12T02:56:31.033005+0000","last_undegraded":"2024-09-12T02:56:31.033005+0000","last_fullsized":"2024-09-12T02:56:31.033005+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:37:50.333152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"83'11","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.558754+0000","last_change":"2024-09-12T02:56:15.977161+0000","last_active":"2024-09-12T02:57:59.558754+0000","last_peered":"2024-09-12T02:57:59.558754+0000","last_clean":"2024-09-12T02:57:59.558754+0000","last_became_active":"2024-09-12T02:56:15.976683+0000","last_became_peered":"2024-09-12T02:56:15.976683+0000","last_unstale":"2024-09-12T02:57:59.558754+0000","last_undegraded":"2024-09-12T02:57:59.558754+0000","last_fullsized":"2024-09-12T02:57:59.558754+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:34:07.526890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"83'38","reported_seq":285,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:58:11.991482+0000","last_change":"2024-09-12T02:56:13.931680+0000","last_active":"2024-09-12T02:58:11.991482+0000","last_peered":"2024-09-12T02:58:11.991482+0000","last_clean":"2024-09-12T02:58:11.991482+0000","last_became_active":"2024-09-12T02:56:13.931214+0000","last_became_peered":"2024-09-12T02:56:13.931214+0000","last_unstale":"2024-09-12T02:58:11.991482+0000","last_undegraded":"2024-09-12T02:58:11.991482+0000","last_fullsized":"2024-09-12T02:58:11.991482+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:38:34.151933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68951,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":225,"num_read_kb":180,"num_write":40,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.555616+0000","last_change":"2024-09-12T02:56:15.977720+0000","last_active":"2024-09-12T02:57:59.555616+0000","last_peered":"2024-09-12T02:57:59.555616+0000","last_clean":"2024-09-12T02:57:59.555616+0000","last_became_active":"2024-09-12T02:56:15.977380+0000","last_became_peered":"2024-09-12T02:56:15.977380+0000","last_unstale":"2024-09-12T02:57:59.555616+0000","last_undegraded":"2024-09-12T02:57:59.555616+0000","last_fullsized":"2024-09-12T02:57:59.555616+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:52:16.841909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"81'9","reported_seq":33,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.030362+0000","last_change":"2024-09-12T02:56:13.929496+0000","last_active":"2024-09-12T02:56:31.030362+0000","last_peered":"2024-09-12T02:56:31.030362+0000","last_clean":"2024-09-12T02:56:31.030362+0000","last_became_active":"2024-09-12T02:56:13.929117+0000","last_became_peered":"2024-09-12T02:56:13.929117+0000","last_unstale":"2024-09-12T02:56:31.030362+0000","last_undegraded":"2024-09-12T02:56:31.030362+0000","last_fullsized":"2024-09-12T02:56:31.030362+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:59:38.399135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.564811+0000","last_change":"2024-09-12T02:56:15.974323+0000","last_active":"2024-09-12T02:57:59.564811+0000","last_peered":"2024-09-12T02:57:59.564811+0000","last_clean":"2024-09-12T02:57:59.564811+0000","last_became_active":"2024-09-12T02:56:15.973819+0000","last_became_peered":"2024-09-12T02:56:15.973819+0000","last_unstale":"2024-09-12T02:57:59.564811+0000","last_undegraded":"2024-09-12T02:57:59.564811+0000","last_fullsized":"2024-09-12T02:57:59.564811+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:53:13.130695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"81'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.032473+0000","last_change":"2024-09-12T02:56:13.935263+0000","last_active":"2024-09-12T02:56:31.032473+0000","last_peered":"2024-09-12T02:56:31.032473+0000","last_clean":"2024-09-12T02:56:31.032473+0000","last_became_active":"2024-09-12T02:56:13.934805+0000","last_became_peered":"2024-09-12T02:56:13.934805+0000","last_unstale":"2024-09-12T02:56:31.032473+0000","last_undegraded":"2024-09-12T02:56:31.032473+0000","last_fullsized":"2024-09-12T02:56:31.032473+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:39:42.882031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"83'15","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.560318+0000","last_change":"2024-09-12T02:56:15.976630+0000","last_active":"2024-09-12T02:57:59.560318+0000","last_peered":"2024-09-12T02:57:59.560318+0000","last_clean":"2024-09-12T02:57:59.560318+0000","last_became_active":"2024-09-12T02:56:15.976304+0000","last_became_peered":"2024-09-12T02:56:15.976304+0000","last_unstale":"2024-09-12T02:57:59.560318+0000","last_undegraded":"2024-09-12T02:57:59.560318+0000","last_fullsized":"2024-09-12T02:57:59.560318+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:46:03.366740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"83'8","reported_seq":138,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:58:08.987094+0000","last_change":"2024-09-12T02:56:13.934224+0000","last_active":"2024-09-12T02:58:08.987094+0000","last_peered":"2024-09-12T02:58:08.987094+0000","last_clean":"2024-09-12T02:58:08.987094+0000","last_became_active":"2024-09-12T02:56:13.933838+0000","last_became_peered":"2024-09-12T02:56:13.933838+0000","last_unstale":"2024-09-12T02:58:08.987094+0000","last_undegraded":"2024-09-12T02:58:08.987094+0000","last_fullsized":"2024-09-12T02:58:08.987094+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:34:34.206051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1515,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":109,"num_read_kb":6,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.557213+0000","last_change":"2024-09-12T02:56:15.980364+0000","last_active":"2024-09-12T02:57:59.557213+0000","last_peered":"2024-09-12T02:57:59.557213+0000","last_clean":"2024-09-12T02:57:59.557213+0000","last_became_active":"2024-09-12T02:56:15.979437+0000","last_became_peered":"2024-09-12T02:56:15.979437+0000","last_unstale":"2024-09-12T02:57:59.557213+0000","last_undegraded":"2024-09-12T02:57:59.557213+0000","last_fullsized":"2024-09-12T02:57:59.557213+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:43:33.424867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"81'17","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.050509+0000","last_change":"2024-09-12T02:56:31.050046+0000","last_active":"2024-09-12T02:56:31.050509+0000","last_peered":"2024-09-12T02:56:31.050509+0000","last_clean":"2024-09-12T02:56:31.050509+0000","last_became_active":"2024-09-12T02:56:31.036990+0000","last_became_peered":"2024-09-12T02:56:31.036990+0000","last_unstale":"2024-09-12T02:56:31.050509+0000","last_undegraded":"2024-09-12T02:56:31.050509+0000","last_fullsized":"2024-09-12T02:56:31.050509+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:05:53.336691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.563297+0000","last_change":"2024-09-12T02:56:15.986434+0000","last_active":"2024-09-12T02:57:59.563297+0000","last_peered":"2024-09-12T02:57:59.563297+0000","last_clean":"2024-09-12T02:57:59.563297+0000","last_became_active":"2024-09-12T02:56:15.986101+0000","last_became_peered":"2024-09-12T02:56:15.986101+0000","last_unstale":"2024-09-12T02:57:59.563297+0000","last_undegraded":"2024-09-12T02:57:59.563297+0000","last_fullsized":"2024-09-12T02:57:59.563297+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:51:27.840951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"81'2","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.035974+0000","last_change":"2024-09-12T02:56:13.925068+0000","last_active":"2024-09-12T02:56:31.035974+0000","last_peered":"2024-09-12T02:56:31.035974+0000","last_clean":"2024-09-12T02:56:31.035974+0000","last_became_active":"2024-09-12T02:56:13.924535+0000","last_became_peered":"2024-09-12T02:56:13.924535+0000","last_unstale":"2024-09-12T02:56:31.035974+0000","last_undegraded":"2024-09-12T02:56:31.035974+0000","last_fullsized":"2024-09-12T02:56:31.035974+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:45:32.478632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"83'5","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.559020+0000","last_change":"2024-09-12T02:56:15.983850+0000","last_active":"2024-09-12T02:57:59.559020+0000","last_peered":"2024-09-12T02:57:59.559020+0000","last_clean":"2024-09-12T02:57:59.559020+0000","last_became_active":"2024-09-12T02:56:15.983480+0000","last_became_peered":"2024-09-12T02:56:15.983480+0000","last_unstale":"2024-09-12T02:57:59.559020+0000","last_undegraded":"2024-09-12T02:57:59.559020+0000","last_fullsized":"2024-09-12T02:57:59.559020+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:09:42.185237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"81'5","reported_seq":39,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.312450+0000","last_change":"2024-09-12T02:56:31.312450+0000","last_active":"2024-09-12T02:56:31.312450+0000","last_peered":"2024-09-12T02:56:31.312450+0000","last_clean":"2024-09-12T02:56:31.312450+0000","last_became_active":"2024-09-12T02:56:31.308030+0000","last_became_peered":"2024-09-12T02:56:31.308030+0000","last_unstale":"2024-09-12T02:56:31.312450+0000","last_undegraded":"2024-09-12T02:56:31.312450+0000","last_fullsized":"2024-09-12T02:56:31.312450+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:15:14.692292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.562640+0000","last_change":"2024-09-12T02:56:15.978996+0000","last_active":"2024-09-12T02:57:59.562640+0000","last_peered":"2024-09-12T02:57:59.562640+0000","last_clean":"2024-09-12T02:57:59.562640+0000","last_became_active":"2024-09-12T02:56:15.978636+0000","last_became_peered":"2024-09-12T02:56:15.978636+0000","last_unstale":"2024-09-12T02:57:59.562640+0000","last_undegraded":"2024-09-12T02:57:59.562640+0000","last_fullsized":"2024-09-12T02:57:59.562640+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:29:20.401664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"81'3","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.033119+0000","last_change":"2024-09-12T02:56:13.929388+0000","last_active":"2024-09-12T02:56:31.033119+0000","last_peered":"2024-09-12T02:56:31.033119+0000","last_clean":"2024-09-12T02:56:31.033119+0000","last_became_active":"2024-09-12T02:56:13.928998+0000","last_became_peered":"2024-09-12T02:56:13.928998+0000","last_unstale":"2024-09-12T02:56:31.033119+0000","last_undegraded":"2024-09-12T02:56:31.033119+0000","last_fullsized":"2024-09-12T02:56:31.033119+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:37:57.368050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.560190+0000","last_change":"2024-09-12T02:56:15.977491+0000","last_active":"2024-09-12T02:57:59.560190+0000","last_peered":"2024-09-12T02:57:59.560190+0000","last_clean":"2024-09-12T02:57:59.560190+0000","last_became_active":"2024-09-12T02:56:15.977160+0000","last_became_peered":"2024-09-12T02:56:15.977160+0000","last_unstale":"2024-09-12T02:57:59.560190+0000","last_undegraded":"2024-09-12T02:57:59.560190+0000","last_fullsized":"2024-09-12T02:57:59.560190+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:05:40.577342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"83'7","reported_seq":138,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:58:10.990412+0000","last_change":"2024-09-12T02:56:14.327147+0000","last_active":"2024-09-12T02:58:10.990412+0000","last_peered":"2024-09-12T02:58:10.990412+0000","last_clean":"2024-09-12T02:58:10.990412+0000","last_became_active":"2024-09-12T02:56:14.326737+0000","last_became_peered":"2024-09-12T02:56:14.326737+0000","last_unstale":"2024-09-12T02:58:10.990412+0000","last_undegraded":"2024-09-12T02:58:10.990412+0000","last_fullsized":"2024-09-12T02:58:10.990412+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:55:33.510599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1704,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":109,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.563684+0000","last_change":"2024-09-12T02:56:15.984363+0000","last_active":"2024-09-12T02:57:59.563684+0000","last_peered":"2024-09-12T02:57:59.563684+0000","last_clean":"2024-09-12T02:57:59.563684+0000","last_became_active":"2024-09-12T02:56:15.983822+0000","last_became_peered":"2024-09-12T02:56:15.983822+0000","last_unstale":"2024-09-12T02:57:59.563684+0000","last_undegraded":"2024-09-12T02:57:59.563684+0000","last_fullsized":"2024-09-12T02:57:59.563684+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T10:41:46.082716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":333,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.035564+0000","last_change":"2024-09-12T02:55:05.800210+0000","last_active":"2024-09-12T02:56:31.035564+0000","last_peered":"2024-09-12T02:56:31.035564+0000","last_clean":"2024-09-12T02:56:31.035564+0000","last_became_active":"2024-09-12T02:55:05.799791+0000","last_became_peered":"2024-09-12T02:55:05.799791+0000","last_unstale":"2024-09-12T02:56:31.035564+0000","last_undegraded":"2024-09-12T02:56:31.035564+0000","last_fullsized":"2024-09-12T02:56:31.035564+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:51:48.530016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:51:48.530016+0000","last_clean_scrub_stamp":"2024-09-12T02:51:48.530016+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:05:47.416697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"81'6","reported_seq":31,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.031169+0000","last_change":"2024-09-12T02:56:13.930269+0000","last_active":"2024-09-12T02:56:31.031169+0000","last_peered":"2024-09-12T02:56:31.031169+0000","last_clean":"2024-09-12T02:56:31.031169+0000","last_became_active":"2024-09-12T02:56:13.929831+0000","last_became_peered":"2024-09-12T02:56:13.929831+0000","last_unstale":"2024-09-12T02:56:31.031169+0000","last_undegraded":"2024-09-12T02:56:31.031169+0000","last_fullsized":"2024-09-12T02:56:31.031169+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:38:17.878398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.561332+0000","last_change":"2024-09-12T02:56:15.969744+0000","last_active":"2024-09-12T02:57:59.561332+0000","last_peered":"2024-09-12T02:57:59.561332+0000","last_clean":"2024-09-12T02:57:59.561332+0000","last_became_active":"2024-09-12T02:56:15.969354+0000","last_became_peered":"2024-09-12T02:56:15.969354+0000","last_unstale":"2024-09-12T02:57:59.561332+0000","last_undegraded":"2024-09-12T02:57:59.561332+0000","last_fullsized":"2024-09-12T02:57:59.561332+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:26:02.409510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"81'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.035447+0000","last_change":"2024-09-12T02:56:13.929231+0000","last_active":"2024-09-12T02:56:31.035447+0000","last_peered":"2024-09-12T02:56:31.035447+0000","last_clean":"2024-09-12T02:56:31.035447+0000","last_became_active":"2024-09-12T02:56:13.928843+0000","last_became_peered":"2024-09-12T02:56:13.928843+0000","last_unstale":"2024-09-12T02:56:31.035447+0000","last_undegraded":"2024-09-12T02:56:31.035447+0000","last_fullsized":"2024-09-12T02:56:31.035447+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:26:33.619755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"83'5","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.555927+0000","last_change":"2024-09-12T02:56:15.982503+0000","last_active":"2024-09-12T02:57:59.555927+0000","last_peered":"2024-09-12T02:57:59.555927+0000","last_clean":"2024-09-12T02:57:59.555927+0000","last_became_active":"2024-09-12T02:56:15.982192+0000","last_became_peered":"2024-09-12T02:56:15.982192+0000","last_unstale":"2024-09-12T02:57:59.555927+0000","last_undegraded":"2024-09-12T02:57:59.555927+0000","last_fullsized":"2024-09-12T02:57:59.555927+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:55:18.370967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"83'6","reported_seq":137,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:58:13.993514+0000","last_change":"2024-09-12T02:56:14.075486+0000","last_active":"2024-09-12T02:58:13.993514+0000","last_peered":"2024-09-12T02:58:13.993514+0000","last_clean":"2024-09-12T02:58:13.993514+0000","last_became_active":"2024-09-12T02:56:14.075152+0000","last_became_peered":"2024-09-12T02:56:14.075152+0000","last_unstale":"2024-09-12T02:58:13.993514+0000","last_undegraded":"2024-09-12T02:58:13.993514+0000","last_fullsized":"2024-09-12T02:58:13.993514+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:59:40.950191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":109,"num_read_kb":109,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.556340+0000","last_change":"2024-09-12T02:56:15.981584+0000","last_active":"2024-09-12T02:57:59.556340+0000","last_peered":"2024-09-12T02:57:59.556340+0000","last_clean":"2024-09-12T02:57:59.556340+0000","last_became_active":"2024-09-12T02:56:15.981240+0000","last_became_peered":"2024-09-12T02:56:15.981240+0000","last_unstale":"2024-09-12T02:57:59.556340+0000","last_undegraded":"2024-09-12T02:57:59.556340+0000","last_fullsized":"2024-09-12T02:57:59.556340+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:49:04.036830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"83'7","reported_seq":139,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:58:12.992524+0000","last_change":"2024-09-12T02:56:13.935143+0000","last_active":"2024-09-12T02:58:12.992524+0000","last_peered":"2024-09-12T02:58:12.992524+0000","last_clean":"2024-09-12T02:58:12.992524+0000","last_became_active":"2024-09-12T02:56:13.934792+0000","last_became_peered":"2024-09-12T02:56:13.934792+0000","last_unstale":"2024-09-12T02:58:12.992524+0000","last_undegraded":"2024-09-12T02:58:12.992524+0000","last_fullsized":"2024-09-12T02:58:12.992524+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T04:00:26.973612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1289,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":110,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"83'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.558893+0000","last_change":"2024-09-12T02:56:31.034825+0000","last_active":"2024-09-12T02:57:59.558893+0000","last_peered":"2024-09-12T02:57:59.558893+0000","last_clean":"2024-09-12T02:57:59.558893+0000","last_became_active":"2024-09-12T02:56:31.034516+0000","last_became_peered":"2024-09-12T02:56:31.034516+0000","last_unstale":"2024-09-12T02:57:59.558893+0000","last_undegraded":"2024-09-12T02:57:59.558893+0000","last_fullsized":"2024-09-12T02:57:59.558893+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:10:42.161023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"81'1","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.029027+0000","last_change":"2024-09-12T02:56:14.076762+0000","last_active":"2024-09-12T02:56:31.029027+0000","last_peered":"2024-09-12T02:56:31.029027+0000","last_clean":"2024-09-12T02:56:31.029027+0000","last_became_active":"2024-09-12T02:56:14.076421+0000","last_became_peered":"2024-09-12T02:56:14.076421+0000","last_unstale":"2024-09-12T02:56:31.029027+0000","last_undegraded":"2024-09-12T02:56:31.029027+0000","last_fullsized":"2024-09-12T02:56:31.029027+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:56:56.422910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'5","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.555835+0000","last_change":"2024-09-12T02:56:15.978810+0000","last_active":"2024-09-12T02:57:59.555835+0000","last_peered":"2024-09-12T02:57:59.555835+0000","last_clean":"2024-09-12T02:57:59.555835+0000","last_became_active":"2024-09-12T02:56:15.978038+0000","last_became_peered":"2024-09-12T02:56:15.978038+0000","last_unstale":"2024-09-12T02:57:59.555835+0000","last_undegraded":"2024-09-12T02:57:59.555835+0000","last_fullsized":"2024-09-12T02:57:59.555835+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T05:04:52.215784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"81'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.035440+0000","last_change":"2024-09-12T02:56:14.075987+0000","last_active":"2024-09-12T02:56:31.035440+0000","last_peered":"2024-09-12T02:56:31.035440+0000","last_clean":"2024-09-12T02:56:31.035440+0000","last_became_active":"2024-09-12T02:56:14.075596+0000","last_became_peered":"2024-09-12T02:56:14.075596+0000","last_unstale":"2024-09-12T02:56:31.035440+0000","last_undegraded":"2024-09-12T02:56:31.035440+0000","last_fullsized":"2024-09-12T02:56:31.035440+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:18:48.713387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.552400+0000","last_change":"2024-09-12T02:56:15.987842+0000","last_active":"2024-09-12T02:57:59.552400+0000","last_peered":"2024-09-12T02:57:59.552400+0000","last_clean":"2024-09-12T02:57:59.552400+0000","last_became_active":"2024-09-12T02:56:15.987507+0000","last_became_peered":"2024-09-12T02:56:15.987507+0000","last_unstale":"2024-09-12T02:57:59.552400+0000","last_undegraded":"2024-09-12T02:57:59.552400+0000","last_fullsized":"2024-09-12T02:57:59.552400+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:52:23.000549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"83'19","reported_seq":41,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:58:01.991961+0000","last_change":"2024-09-12T02:56:13.924372+0000","last_active":"2024-09-12T02:58:01.991961+0000","last_peered":"2024-09-12T02:58:01.991961+0000","last_clean":"2024-09-12T02:58:01.991961+0000","last_became_active":"2024-09-12T02:56:13.923955+0000","last_became_peered":"2024-09-12T02:56:13.923955+0000","last_unstale":"2024-09-12T02:58:01.991961+0000","last_undegraded":"2024-09-12T02:58:01.991961+0000","last_fullsized":"2024-09-12T02:58:01.991961+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:52:47.452410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":34,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"83'5","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.562919+0000","last_change":"2024-09-12T02:56:15.978262+0000","last_active":"2024-09-12T02:57:59.562919+0000","last_peered":"2024-09-12T02:57:59.562919+0000","last_clean":"2024-09-12T02:57:59.562919+0000","last_became_active":"2024-09-12T02:56:15.977913+0000","last_became_peered":"2024-09-12T02:56:15.977913+0000","last_unstale":"2024-09-12T02:57:59.562919+0000","last_undegraded":"2024-09-12T02:57:59.562919+0000","last_fullsized":"2024-09-12T02:57:59.562919+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:35:27.104635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"81'7","reported_seq":30,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.031488+0000","last_change":"2024-09-12T02:56:14.327417+0000","last_active":"2024-09-12T02:56:31.031488+0000","last_peered":"2024-09-12T02:56:31.031488+0000","last_clean":"2024-09-12T02:56:31.031488+0000","last_became_active":"2024-09-12T02:56:14.327001+0000","last_became_peered":"2024-09-12T02:56:14.327001+0000","last_unstale":"2024-09-12T02:56:31.031488+0000","last_undegraded":"2024-09-12T02:56:31.031488+0000","last_fullsized":"2024-09-12T02:56:31.031488+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:15:39.019252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"83'7","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.565859+0000","last_change":"2024-09-12T02:56:15.977286+0000","last_active":"2024-09-12T02:57:59.565859+0000","last_peered":"2024-09-12T02:57:59.565859+0000","last_clean":"2024-09-12T02:57:59.565859+0000","last_became_active":"2024-09-12T02:56:15.976944+0000","last_became_peered":"2024-09-12T02:56:15.976944+0000","last_unstale":"2024-09-12T02:57:59.565859+0000","last_undegraded":"2024-09-12T02:57:59.565859+0000","last_fullsized":"2024-09-12T02:57:59.565859+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:22:57.056781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.306212+0000","last_change":"2024-09-12T02:56:14.075701+0000","last_active":"2024-09-12T02:56:31.306212+0000","last_peered":"2024-09-12T02:56:31.306212+0000","last_clean":"2024-09-12T02:56:31.306212+0000","last_became_active":"2024-09-12T02:56:14.075340+0000","last_became_peered":"2024-09-12T02:56:14.075340+0000","last_unstale":"2024-09-12T02:56:31.306212+0000","last_undegraded":"2024-09-12T02:56:31.306212+0000","last_fullsized":"2024-09-12T02:56:31.306212+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:17:42.255058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"83'2","reported_seq":19,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.528187+0000","last_change":"2024-09-12T02:56:15.979783+0000","last_active":"2024-09-12T02:57:59.528187+0000","last_peered":"2024-09-12T02:57:59.528187+0000","last_clean":"2024-09-12T02:57:59.528187+0000","last_became_active":"2024-09-12T02:56:15.979424+0000","last_became_peered":"2024-09-12T02:56:15.979424+0000","last_unstale":"2024-09-12T02:57:59.528187+0000","last_undegraded":"2024-09-12T02:57:59.528187+0000","last_fullsized":"2024-09-12T02:57:59.528187+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:59:11.568138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"81'7","reported_seq":29,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.034660+0000","last_change":"2024-09-12T02:56:13.928996+0000","last_active":"2024-09-12T02:56:31.034660+0000","last_peered":"2024-09-12T02:56:31.034660+0000","last_clean":"2024-09-12T02:56:31.034660+0000","last_became_active":"2024-09-12T02:56:13.928664+0000","last_became_peered":"2024-09-12T02:56:13.928664+0000","last_unstale":"2024-09-12T02:56:31.034660+0000","last_undegraded":"2024-09-12T02:56:31.034660+0000","last_fullsized":"2024-09-12T02:56:31.034660+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:37:22.067930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.560217+0000","last_change":"2024-09-12T02:56:15.984096+0000","last_active":"2024-09-12T02:57:59.560217+0000","last_peered":"2024-09-12T02:57:59.560217+0000","last_clean":"2024-09-12T02:57:59.560217+0000","last_became_active":"2024-09-12T02:56:15.983737+0000","last_became_peered":"2024-09-12T02:56:15.983737+0000","last_unstale":"2024-09-12T02:57:59.560217+0000","last_undegraded":"2024-09-12T02:57:59.560217+0000","last_fullsized":"2024-09-12T02:57:59.560217+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:02:17.892318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"83'4","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:33.042035+0000","last_change":"2024-09-12T02:56:31.053343+0000","last_active":"2024-09-12T02:56:33.042035+0000","last_peered":"2024-09-12T02:56:33.042035+0000","last_clean":"2024-09-12T02:56:33.042035+0000","last_became_active":"2024-09-12T02:56:31.037918+0000","last_became_peered":"2024-09-12T02:56:31.037918+0000","last_unstale":"2024-09-12T02:56:33.042035+0000","last_undegraded":"2024-09-12T02:56:33.042035+0000","last_fullsized":"2024-09-12T02:56:33.042035+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T09:39:54.990724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"83'8","reported_seq":25,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.563189+0000","last_change":"2024-09-12T02:56:15.975640+0000","last_active":"2024-09-12T02:57:59.563189+0000","last_peered":"2024-09-12T02:57:59.563189+0000","last_clean":"2024-09-12T02:57:59.563189+0000","last_became_active":"2024-09-12T02:56:15.975342+0000","last_became_peered":"2024-09-12T02:56:15.975342+0000","last_unstale":"2024-09-12T02:57:59.563189+0000","last_undegraded":"2024-09-12T02:57:59.563189+0000","last_fullsized":"2024-09-12T02:57:59.563189+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T07:26:23.694876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":27,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.034922+0000","last_change":"2024-09-12T02:56:13.934149+0000","last_active":"2024-09-12T02:56:31.034922+0000","last_peered":"2024-09-12T02:56:31.034922+0000","last_clean":"2024-09-12T02:56:31.034922+0000","last_became_active":"2024-09-12T02:56:13.933743+0000","last_became_peered":"2024-09-12T02:56:13.933743+0000","last_unstale":"2024-09-12T02:56:31.034922+0000","last_undegraded":"2024-09-12T02:56:31.034922+0000","last_fullsized":"2024-09-12T02:56:31.034922+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:16:17.854457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"83'5","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.563348+0000","last_change":"2024-09-12T02:56:15.984782+0000","last_active":"2024-09-12T02:57:59.563348+0000","last_peered":"2024-09-12T02:57:59.563348+0000","last_clean":"2024-09-12T02:57:59.563348+0000","last_became_active":"2024-09-12T02:56:15.984336+0000","last_became_peered":"2024-09-12T02:56:15.984336+0000","last_unstale":"2024-09-12T02:57:59.563348+0000","last_undegraded":"2024-09-12T02:57:59.563348+0000","last_fullsized":"2024-09-12T02:57:59.563348+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T03:03:39.832897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"83'15","reported_seq":32,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.564252+0000","last_change":"2024-09-12T02:56:15.985486+0000","last_active":"2024-09-12T02:57:59.564252+0000","last_peered":"2024-09-12T02:57:59.564252+0000","last_clean":"2024-09-12T02:57:59.564252+0000","last_became_active":"2024-09-12T02:56:15.985167+0000","last_became_peered":"2024-09-12T02:56:15.985167+0000","last_unstale":"2024-09-12T02:57:59.564252+0000","last_undegraded":"2024-09-12T02:57:59.564252+0000","last_fullsized":"2024-09-12T02:57:59.564252+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:46:20.569453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"81'5","reported_seq":28,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.035106+0000","last_change":"2024-09-12T02:56:13.929324+0000","last_active":"2024-09-12T02:56:31.035106+0000","last_peered":"2024-09-12T02:56:31.035106+0000","last_clean":"2024-09-12T02:56:31.035106+0000","last_became_active":"2024-09-12T02:56:13.928994+0000","last_became_peered":"2024-09-12T02:56:13.928994+0000","last_unstale":"2024-09-12T02:56:31.035106+0000","last_undegraded":"2024-09-12T02:56:31.035106+0000","last_fullsized":"2024-09-12T02:56:31.035106+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T12:30:27.290328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"83'9","reported_seq":26,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.562582+0000","last_change":"2024-09-12T02:56:15.985889+0000","last_active":"2024-09-12T02:57:59.562582+0000","last_peered":"2024-09-12T02:57:59.562582+0000","last_clean":"2024-09-12T02:57:59.562582+0000","last_became_active":"2024-09-12T02:56:15.985509+0000","last_became_peered":"2024-09-12T02:56:15.985509+0000","last_unstale":"2024-09-12T02:57:59.562582+0000","last_undegraded":"2024-09-12T02:57:59.562582+0000","last_fullsized":"2024-09-12T02:57:59.562582+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T06:30:43.120718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"83'13","reported_seq":36,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.035437+0000","last_change":"2024-09-12T02:56:13.934192+0000","last_active":"2024-09-12T02:56:31.035437+0000","last_peered":"2024-09-12T02:56:31.035437+0000","last_clean":"2024-09-12T02:56:31.035437+0000","last_became_active":"2024-09-12T02:56:13.933789+0000","last_became_peered":"2024-09-12T02:56:13.933789+0000","last_unstale":"2024-09-12T02:56:31.035437+0000","last_undegraded":"2024-09-12T02:56:31.035437+0000","last_fullsized":"2024-09-12T02:56:31.035437+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:32:47.771085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"83'6","reported_seq":23,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.563285+0000","last_change":"2024-09-12T02:56:15.983531+0000","last_active":"2024-09-12T02:57:59.563285+0000","last_peered":"2024-09-12T02:57:59.563285+0000","last_clean":"2024-09-12T02:57:59.563285+0000","last_became_active":"2024-09-12T02:56:15.983192+0000","last_became_peered":"2024-09-12T02:56:15.983192+0000","last_unstale":"2024-09-12T02:57:59.563285+0000","last_undegraded":"2024-09-12T02:57:59.563285+0000","last_fullsized":"2024-09-12T02:57:59.563285+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T14:16:59.923379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":24,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:56:31.034752+0000","last_change":"2024-09-12T02:56:14.075396+0000","last_active":"2024-09-12T02:56:31.034752+0000","last_peered":"2024-09-12T02:56:31.034752+0000","last_clean":"2024-09-12T02:56:31.034752+0000","last_became_active":"2024-09-12T02:56:14.075023+0000","last_became_peered":"2024-09-12T02:56:14.075023+0000","last_unstale":"2024-09-12T02:56:31.034752+0000","last_undegraded":"2024-09-12T02:56:31.034752+0000","last_fullsized":"2024-09-12T02:56:31.034752+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T11:41:45.760689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"83'5","reported_seq":22,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:57:59.564093+0000","last_change":"2024-09-12T02:56:15.987276+0000","last_active":"2024-09-12T02:57:59.564093+0000","last_peered":"2024-09-12T02:57:59.564093+0000","last_clean":"2024-09-12T02:57:59.564093+0000","last_became_active":"2024-09-12T02:56:15.986949+0000","last_became_peered":"2024-09-12T02:56:15.986949+0000","last_unstale":"2024-09-12T02:57:59.564093+0000","last_undegraded":"2024-09-12T02:57:59.564093+0000","last_fullsized":"2024-09-12T02:57:59.564093+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:14.951866+0000","last_clean_scrub_stamp":"2024-09-12T02:56:14.951866+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T13:49:45.627093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"83'14","reported_seq":145,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-12T02:58:12.992085+0000","last_change":"2024-09-12T02:56:13.932365+0000","last_active":"2024-09-12T02:58:12.992085+0000","last_peered":"2024-09-12T02:58:12.992085+0000","last_clean":"2024-09-12T02:58:12.992085+0000","last_became_active":"2024-09-12T02:56:13.932062+0000","last_became_peered":"2024-09-12T02:56:13.932062+0000","last_unstale":"2024-09-12T02:58:12.992085+0000","last_undegraded":"2024-09-12T02:58:12.992085+0000","last_fullsized":"2024-09-12T02:58:12.992085+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-12T02:56:12.906665+0000","last_clean_scrub_stamp":"2024-09-12T02:56:12.906665+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T08:17:08.451839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":109,"num_read_kb":109,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":77928,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1107,"num_read_kb":643,"num_write":273,"num_write_kb":162,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":734,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":589824,"data_stored":233784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":233,"ondisk_log_size":233,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547237,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":917504,"data_stored":440831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776169,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32764,"kb_used_data":1580,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712388,"statfs":{"total":95995035648,"available":95961485312,"internally_reserved":0,"allocated":1617920,"data_stored":1101092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939695,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40416,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704736,"statfs":{"total":95995035648,"available":95953649664,"internally_reserved":0,"allocated":933888,"data_stored":440115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":40434112},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168628,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32084,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713068,"statfs":{"total":95995035648,"available":95962181632,"internally_reserved":0,"allocated":921600,"data_stored":440381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430264,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28516,"kb_used_data":1492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716636,"statfs":{"total":95995035648,"available":95965835264,"internally_reserved":0,"allocated":1527808,"data_stored":1034641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593790,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":917504,"data_stored":440533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822721,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28528,"kb_used_data":1504,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716624,"statfs":{"total":95995035648,"available":95965822976,"internally_reserved":0,"allocated":1540096,"data_stored":1032691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953543,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":925696,"data_stored":441635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182475,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":933888,"data_stored":439657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411409,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36320,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708832,"statfs":{"total":95995035648,"available":95957843968,"internally_reserved":0,"allocated":999424,"data_stored":509828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574933,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":987136,"data_stored":510397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738459,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":937984,"data_stored":441705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":71404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":70835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":5163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":71614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-12T02:58:17.157 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:58:17.419 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:58:17.540 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:58:17 smithi005 ceph-mon[31197]: pgmap v336: 97 pgs: 97 active+clean; 653 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-09-12T02:58:17.540 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:58:17 smithi005 ceph-mon[31197]: from='client.15336 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:58:17.665 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:58:17 smithi148 ceph-mon[35090]: pgmap v336: 97 pgs: 97 active+clean; 653 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-09-12T02:58:17.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:58:17 smithi148 ceph-mon[35090]: from='client.15336 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:58:17.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:58:17 smithi146 ceph-mon[35011]: pgmap v336: 97 pgs: 97 active+clean; 653 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-09-12T02:58:17.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:58:17 smithi146 ceph-mon[35011]: from='client.15336 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-12T02:58:17.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:17.870+0000 7ffbe8142640 1 -- 172.21.15.5:0/3355204348 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbe0102b50 msgr2=0x7ffbe0100090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:17.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:17.870+0000 7ffbe8142640 1 --2- 172.21.15.5:0/3355204348 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbe0102b50 0x7ffbe0100090 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc009900 tx=0x7ffbdc02a3e0 comp rx=0 tx=0).stop 2024-09-12T02:58:17.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:17.870+0000 7ffbe8142640 1 -- 172.21.15.5:0/3355204348 shutdown_connections 2024-09-12T02:58:17.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:17.870+0000 7ffbe8142640 1 --2- 172.21.15.5:0/3355204348 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffbe010e810 0x7ffbe0110c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:17.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:17.870+0000 7ffbe8142640 1 --2- 172.21.15.5:0/3355204348 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffbe0102fc0 0x7ffbe01005d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:17.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:17.870+0000 7ffbe8142640 1 --2- 172.21.15.5:0/3355204348 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbe0102b50 0x7ffbe0100090 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:17.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:17.870+0000 7ffbe8142640 1 -- 172.21.15.5:0/3355204348 >> 172.21.15.5:0/3355204348 conn(0x7ffbe0077210 msgr2=0x7ffbe00fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:17.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:17.871+0000 7ffbe8142640 1 -- 172.21.15.5:0/3355204348 shutdown_connections 2024-09-12T02:58:17.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:17.871+0000 7ffbe8142640 1 -- 172.21.15.5:0/3355204348 wait complete. 2024-09-12T02:58:17.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:17.871+0000 7ffbe8142640 1 Processor -- start 2024-09-12T02:58:17.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:17.871+0000 7ffbe8142640 1 -- start start 2024-09-12T02:58:17.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:17.871+0000 7ffbe8142640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbe0102b50 0x7ffbe019a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:17.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:17.871+0000 7ffbe8142640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffbe010e810 0x7ffbe019aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:17.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:17.871+0000 7ffbe8142640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffbe019ee30 0x7ffbe019f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:17.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:17.871+0000 7ffbe8142640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ffbe01141c0 con 0x7ffbe0102b50 2024-09-12T02:58:17.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:17.871+0000 7ffbe8142640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ffbe0114040 con 0x7ffbe019ee30 2024-09-12T02:58:17.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:17.871+0000 7ffbe8142640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ffbe0114340 con 0x7ffbe010e810 2024-09-12T02:58:17.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:17.872+0000 7ffbe5ede640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbe0102b50 0x7ffbe019a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:17.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:17.872+0000 7ffbe5ede640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbe0102b50 0x7ffbe019a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50638/0 (socket says 172.21.15.5:50638) 2024-09-12T02:58:17.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:17.872+0000 7ffbe5ede640 1 -- 172.21.15.5:0/3913234639 learned_addr learned my addr 172.21.15.5:0/3913234639 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:58:17.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:17.872+0000 7ffbe56dd640 1 --2- 172.21.15.5:0/3913234639 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffbe010e810 0x7ffbe019aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:17.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:17.872+0000 7ffbe66df640 1 --2- 172.21.15.5:0/3913234639 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffbe019ee30 0x7ffbe019f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:17.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:17.872+0000 7ffbe5ede640 1 -- 172.21.15.5:0/3913234639 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffbe010e810 msgr2=0x7ffbe019aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:17.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:17.872+0000 7ffbe5ede640 1 --2- 172.21.15.5:0/3913234639 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffbe010e810 0x7ffbe019aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:17.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:17.872+0000 7ffbe5ede640 1 -- 172.21.15.5:0/3913234639 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffbe019ee30 msgr2=0x7ffbe019f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:17.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:17.872+0000 7ffbe5ede640 1 --2- 172.21.15.5:0/3913234639 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffbe019ee30 0x7ffbe019f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:17.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:17.872+0000 7ffbe5ede640 1 -- 172.21.15.5:0/3913234639 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbe019f8f0 con 0x7ffbe0102b50 2024-09-12T02:58:17.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:17.872+0000 7ffbe66df640 1 --2- 172.21.15.5:0/3913234639 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffbe019ee30 0x7ffbe019f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:58:17.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:17.872+0000 7ffbe5ede640 1 --2- 172.21.15.5:0/3913234639 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbe0102b50 0x7ffbe019a560 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc005ab0 tx=0x7ffbdc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:17.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:17.872+0000 7ffbd2ffd640 1 -- 172.21.15.5:0/3913234639 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbdc03f030 con 0x7ffbe0102b50 2024-09-12T02:58:17.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:17.872+0000 7ffbe8142640 1 -- 172.21.15.5:0/3913234639 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbe019fb80 con 0x7ffbe0102b50 2024-09-12T02:58:17.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:17.872+0000 7ffbd2ffd640 1 -- 172.21.15.5:0/3913234639 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffbdc003ce0 con 0x7ffbe0102b50 2024-09-12T02:58:17.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:17.872+0000 7ffbd2ffd640 1 -- 172.21.15.5:0/3913234639 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbdc035070 con 0x7ffbe0102b50 2024-09-12T02:58:17.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:17.872+0000 7ffbe8142640 1 -- 172.21.15.5:0/3913234639 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbe01a7360 con 0x7ffbe0102b50 2024-09-12T02:58:17.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:17.873+0000 7ffbe8142640 1 -- 172.21.15.5:0/3913234639 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbe0100090 con 0x7ffbe0102b50 2024-09-12T02:58:17.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:17.875+0000 7ffbd2ffd640 1 -- 172.21.15.5:0/3913234639 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7ffbdc042030 con 0x7ffbe0102b50 2024-09-12T02:58:17.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:17.875+0000 7ffbd2ffd640 1 --2- 172.21.15.5:0/3913234639 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7ffbcc0a2c10 0x7ffbcc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:17.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:17.875+0000 7ffbd2ffd640 1 -- 172.21.15.5:0/3913234639 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7ffbdc0efd80 con 0x7ffbe0102b50 2024-09-12T02:58:17.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:17.875+0000 7ffbe56dd640 1 --2- 172.21.15.5:0/3913234639 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7ffbcc0a2c10 0x7ffbcc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:17.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:17.876+0000 7ffbe56dd640 1 --2- 172.21.15.5:0/3913234639 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7ffbcc0a2c10 0x7ffbcc0a50d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ffbe00fedf0 tx=0x7ffbd400a2d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:17.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:17.879+0000 7ffbd2ffd640 1 -- 172.21.15.5:0/3913234639 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ffbdc0bcb90 con 0x7ffbe0102b50 2024-09-12T02:58:18.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:18.093+0000 7ffbe8142640 1 -- 172.21.15.5:0/3913234639 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffbe0103060 con 0x7ffbe0102b50 2024-09-12T02:58:18.094 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:18.093+0000 7ffbd2ffd640 1 -- 172.21.15.5:0/3913234639 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11966 (secure 0 0 0) 0x7ffbdc0bc490 con 0x7ffbe0102b50 2024-09-12T02:58:18.094 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-12T02:58:18.094 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":26,"btime":"2024-09-12T02:58:03:323671+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/4135232521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":4135232521},{"type":"v1","addr":"172.21.15.5:6835","nonce":4135232521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/4290221940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":4290221940},{"type":"v1","addr":"172.21.15.5:6837","nonce":4290221940}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T02:56:16.595562+0000","modified":"2024-09-12T02:58:02.463400+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24502,"mds_1":24487,"mds_2":24481,"mds_3":24479,"mds_4":24458},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/26447341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":26447341},{"type":"v1","addr":"172.21.15.5:6839","nonce":26447341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2393443295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2393443295},{"type":"v1","addr":"172.21.15.5:6841","nonce":2393443295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6833/3686970137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3686970137},{"type":"v1","addr":"172.21.15.146:6833","nonce":3686970137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6833/568512209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":568512209},{"type":"v1","addr":"172.21.15.148:6833","nonce":568512209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6835/2540740420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2540740420},{"type":"v1","addr":"172.21.15.146:6835","nonce":2540740420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.146:6837/1673370937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":1673370937},{"type":"v1","addr":"172.21.15.146:6837","nonce":1673370937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":3,"incarnation":22,"state":"up:active","state_seq":6,"addr":"172.21.15.146:6839/3036767228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":3036767228},{"type":"v1","addr":"172.21.15.146:6839","nonce":3036767228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":2,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.148:6835/4195086427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":4195086427},{"type":"v1","addr":"172.21.15.148:6835","nonce":4195086427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.148:6837/2606873922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2606873922},{"type":"v1","addr":"172.21.15.148:6837","nonce":2606873922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6839/2317600000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":2317600000},{"type":"v1","addr":"172.21.15.148:6839","nonce":2317600000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24502,"qdb_cluster":[24502,24487,24481,24479,24458]},"id":1}]} 2024-09-12T02:58:18.094 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 26 2024-09-12T02:58:18.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:18.095+0000 7ffbe8142640 1 -- 172.21.15.5:0/3913234639 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7ffbcc0a2c10 msgr2=0x7ffbcc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:18.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:18.095+0000 7ffbe8142640 1 --2- 172.21.15.5:0/3913234639 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7ffbcc0a2c10 0x7ffbcc0a50d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ffbe00fedf0 tx=0x7ffbd400a2d0 comp rx=0 tx=0).stop 2024-09-12T02:58:18.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:18.095+0000 7ffbe8142640 1 -- 172.21.15.5:0/3913234639 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbe0102b50 msgr2=0x7ffbe019a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:18.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:18.095+0000 7ffbe8142640 1 --2- 172.21.15.5:0/3913234639 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbe0102b50 0x7ffbe019a560 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7ffbdc005ab0 tx=0x7ffbdc0042f0 comp rx=0 tx=0).stop 2024-09-12T02:58:18.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:18.096+0000 7ffbe8142640 1 -- 172.21.15.5:0/3913234639 shutdown_connections 2024-09-12T02:58:18.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:18.096+0000 7ffbe8142640 1 --2- 172.21.15.5:0/3913234639 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ffbe019ee30 0x7ffbe019f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:18.096 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:18.096+0000 7ffbe8142640 1 --2- 172.21.15.5:0/3913234639 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ffbe010e810 0x7ffbe019aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:18.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:18.096+0000 7ffbe8142640 1 --2- 172.21.15.5:0/3913234639 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7ffbcc0a2c10 0x7ffbcc0a50d0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:18.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:18.096+0000 7ffbe8142640 1 --2- 172.21.15.5:0/3913234639 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbe0102b50 0x7ffbe019a560 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:18.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:18.096+0000 7ffbe8142640 1 -- 172.21.15.5:0/3913234639 >> 172.21.15.5:0/3913234639 conn(0x7ffbe0077210 msgr2=0x7ffbe010fc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:18.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:18.096+0000 7ffbe8142640 1 -- 172.21.15.5:0/3913234639 shutdown_connections 2024-09-12T02:58:18.097 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:18.096+0000 7ffbe8142640 1 -- 172.21.15.5:0/3913234639 wait complete. 2024-09-12T02:58:18.665 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:58:18 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/3913234639' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:58:18.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:58:18 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/3913234639' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:58:18.743 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:58:18 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/3913234639' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:58:18.773 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:58:19.038 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:58:19.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:19.528+0000 7fb73d625640 1 -- 172.21.15.5:0/515251448 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb738106a80 msgr2=0x7fb738106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:19.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:19.528+0000 7fb73d625640 1 --2- 172.21.15.5:0/515251448 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb738106a80 0x7fb738106e60 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fb728009930 tx=0x7fb72802a420 comp rx=0 tx=0).stop 2024-09-12T02:58:19.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:19.528+0000 7fb73d625640 1 -- 172.21.15.5:0/515251448 shutdown_connections 2024-09-12T02:58:19.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:19.528+0000 7fb73d625640 1 --2- 172.21.15.5:0/515251448 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb738101420 0x7fb73810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:19.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:19.528+0000 7fb73d625640 1 --2- 172.21.15.5:0/515251448 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb738100a80 0x7fb738100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:19.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:19.528+0000 7fb73d625640 1 --2- 172.21.15.5:0/515251448 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb738106a80 0x7fb738106e60 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:19.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:19.528+0000 7fb73d625640 1 -- 172.21.15.5:0/515251448 >> 172.21.15.5:0/515251448 conn(0x7fb7380fc7a0 msgr2=0x7fb7380febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:19.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:19.528+0000 7fb73d625640 1 -- 172.21.15.5:0/515251448 shutdown_connections 2024-09-12T02:58:19.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:19.528+0000 7fb73d625640 1 -- 172.21.15.5:0/515251448 wait complete. 2024-09-12T02:58:19.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:19.529+0000 7fb73d625640 1 Processor -- start 2024-09-12T02:58:19.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:19.529+0000 7fb73d625640 1 -- start start 2024-09-12T02:58:19.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:19.529+0000 7fb73d625640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb738100a80 0x7fb738197360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:19.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:19.529+0000 7fb73d625640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb738101420 0x7fb7381978a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:19.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:19.529+0000 7fb73d625640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb738106a80 0x7fb73819bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:19.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:19.529+0000 7fb73d625640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb738110e40 con 0x7fb738106a80 2024-09-12T02:58:19.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:19.529+0000 7fb73d625640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fb738110cc0 con 0x7fb738101420 2024-09-12T02:58:19.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:19.529+0000 7fb73d625640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb738110fc0 con 0x7fb738100a80 2024-09-12T02:58:19.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:19.529+0000 7fb7377fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb738106a80 0x7fb73819bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:19.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:19.530+0000 7fb736ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb738100a80 0x7fb738197360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:19.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:19.530+0000 7fb736ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb738100a80 0x7fb738197360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.5:41088/0 (socket says 172.21.15.5:41088) 2024-09-12T02:58:19.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:19.530+0000 7fb72ffff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb738101420 0x7fb7381978a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:19.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:19.530+0000 7fb736ffd640 1 -- 172.21.15.5:0/1715888613 learned_addr learned my addr 172.21.15.5:0/1715888613 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:58:19.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:19.530+0000 7fb7377fe640 1 -- 172.21.15.5:0/1715888613 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb738100a80 msgr2=0x7fb738197360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:19.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:19.530+0000 7fb7377fe640 1 --2- 172.21.15.5:0/1715888613 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb738100a80 0x7fb738197360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:19.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:19.530+0000 7fb7377fe640 1 -- 172.21.15.5:0/1715888613 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb738101420 msgr2=0x7fb7381978a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:19.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:19.530+0000 7fb7377fe640 1 --2- 172.21.15.5:0/1715888613 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb738101420 0x7fb7381978a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:19.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:19.530+0000 7fb7377fe640 1 -- 172.21.15.5:0/1715888613 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb73819c310 con 0x7fb738106a80 2024-09-12T02:58:19.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:19.530+0000 7fb736ffd640 1 --2- 172.21.15.5:0/1715888613 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb738100a80 0x7fb738197360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:58:19.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:19.530+0000 7fb7377fe640 1 --2- 172.21.15.5:0/1715888613 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb738106a80 0x7fb73819bc30 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fb730009fd0 tx=0x7fb73000ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:19.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:19.530+0000 7fb734ff9640 1 -- 172.21.15.5:0/1715888613 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb73000ce20 con 0x7fb738106a80 2024-09-12T02:58:19.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:19.530+0000 7fb734ff9640 1 -- 172.21.15.5:0/1715888613 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb730005020 con 0x7fb738106a80 2024-09-12T02:58:19.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:19.530+0000 7fb73d625640 1 -- 172.21.15.5:0/1715888613 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb73819c600 con 0x7fb738106a80 2024-09-12T02:58:19.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:19.530+0000 7fb734ff9640 1 -- 172.21.15.5:0/1715888613 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb730010720 con 0x7fb738106a80 2024-09-12T02:58:19.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:19.530+0000 7fb73d625640 1 -- 172.21.15.5:0/1715888613 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb73806a790 con 0x7fb738106a80 2024-09-12T02:58:19.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:19.531+0000 7fb73d625640 1 -- 172.21.15.5:0/1715888613 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb738100ee0 con 0x7fb738106a80 2024-09-12T02:58:19.549 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:19.533+0000 7fb734ff9640 1 -- 172.21.15.5:0/1715888613 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7fb730005540 con 0x7fb738106a80 2024-09-12T02:58:19.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:19.533+0000 7fb734ff9640 1 --2- 172.21.15.5:0/1715888613 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fb7240a2c90 0x7fb7240a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:19.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:19.533+0000 7fb736ffd640 1 --2- 172.21.15.5:0/1715888613 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fb7240a2c90 0x7fb7240a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:19.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:19.534+0000 7fb734ff9640 1 -- 172.21.15.5:0/1715888613 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fb7300cf740 con 0x7fb738106a80 2024-09-12T02:58:19.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:19.534+0000 7fb736ffd640 1 --2- 172.21.15.5:0/1715888613 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fb7240a2c90 0x7fb7240a5150 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fb728000c00 tx=0x7fb728009560 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:19.550 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:19.537+0000 7fb734ff9640 1 -- 172.21.15.5:0/1715888613 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb73009c550 con 0x7fb738106a80 2024-09-12T02:58:19.643 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:58:19 smithi005 ceph-mon[31197]: pgmap v337: 97 pgs: 97 active+clean; 653 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 10 op/s 2024-09-12T02:58:19.643 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:58:19 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:58:19.665 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:58:19 smithi148 ceph-mon[35090]: pgmap v337: 97 pgs: 97 active+clean; 653 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 10 op/s 2024-09-12T02:58:19.665 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:58:19 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:58:19.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:58:19 smithi146 ceph-mon[35011]: pgmap v337: 97 pgs: 97 active+clean; 653 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 10 op/s 2024-09-12T02:58:19.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:58:19 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:58:19.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:19.768+0000 7fb73d625640 1 -- 172.21.15.5:0/1715888613 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb7381986c0 con 0x7fb738106a80 2024-09-12T02:58:19.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:19.768+0000 7fb734ff9640 1 -- 172.21.15.5:0/1715888613 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11966 (secure 0 0 0) 0x7fb7381986c0 con 0x7fb738106a80 2024-09-12T02:58:19.769 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-12T02:58:19.769 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":26,"btime":"2024-09-12T02:58:03:323671+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/4135232521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":4135232521},{"type":"v1","addr":"172.21.15.5:6835","nonce":4135232521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/4290221940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":4290221940},{"type":"v1","addr":"172.21.15.5:6837","nonce":4290221940}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T02:56:16.595562+0000","modified":"2024-09-12T02:58:02.463400+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24502,"mds_1":24487,"mds_2":24481,"mds_3":24479,"mds_4":24458},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/26447341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":26447341},{"type":"v1","addr":"172.21.15.5:6839","nonce":26447341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2393443295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2393443295},{"type":"v1","addr":"172.21.15.5:6841","nonce":2393443295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6833/3686970137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3686970137},{"type":"v1","addr":"172.21.15.146:6833","nonce":3686970137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6833/568512209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":568512209},{"type":"v1","addr":"172.21.15.148:6833","nonce":568512209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6835/2540740420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2540740420},{"type":"v1","addr":"172.21.15.146:6835","nonce":2540740420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.146:6837/1673370937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":1673370937},{"type":"v1","addr":"172.21.15.146:6837","nonce":1673370937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":3,"incarnation":22,"state":"up:active","state_seq":6,"addr":"172.21.15.146:6839/3036767228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":3036767228},{"type":"v1","addr":"172.21.15.146:6839","nonce":3036767228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":2,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.148:6835/4195086427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":4195086427},{"type":"v1","addr":"172.21.15.148:6835","nonce":4195086427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.148:6837/2606873922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2606873922},{"type":"v1","addr":"172.21.15.148:6837","nonce":2606873922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6839/2317600000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":2317600000},{"type":"v1","addr":"172.21.15.148:6839","nonce":2317600000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24502,"qdb_cluster":[24502,24487,24481,24479,24458]},"id":1}]} 2024-09-12T02:58:19.769 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 26 2024-09-12T02:58:19.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:19.770+0000 7fb73d625640 1 -- 172.21.15.5:0/1715888613 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fb7240a2c90 msgr2=0x7fb7240a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:19.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:19.770+0000 7fb73d625640 1 --2- 172.21.15.5:0/1715888613 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fb7240a2c90 0x7fb7240a5150 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fb728000c00 tx=0x7fb728009560 comp rx=0 tx=0).stop 2024-09-12T02:58:19.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:19.770+0000 7fb73d625640 1 -- 172.21.15.5:0/1715888613 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb738106a80 msgr2=0x7fb73819bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:19.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:19.770+0000 7fb73d625640 1 --2- 172.21.15.5:0/1715888613 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb738106a80 0x7fb73819bc30 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fb730009fd0 tx=0x7fb73000ef10 comp rx=0 tx=0).stop 2024-09-12T02:58:19.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:19.771+0000 7fb73d625640 1 -- 172.21.15.5:0/1715888613 shutdown_connections 2024-09-12T02:58:19.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:19.771+0000 7fb73d625640 1 --2- 172.21.15.5:0/1715888613 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fb7240a2c90 0x7fb7240a5150 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:19.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:19.771+0000 7fb73d625640 1 --2- 172.21.15.5:0/1715888613 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb738106a80 0x7fb73819bc30 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:19.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:19.771+0000 7fb73d625640 1 --2- 172.21.15.5:0/1715888613 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb738101420 0x7fb7381978a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:19.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:19.771+0000 7fb73d625640 1 --2- 172.21.15.5:0/1715888613 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb738100a80 0x7fb738197360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:19.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:19.771+0000 7fb73d625640 1 -- 172.21.15.5:0/1715888613 >> 172.21.15.5:0/1715888613 conn(0x7fb7380fc7a0 msgr2=0x7fb7380fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:19.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:19.771+0000 7fb73d625640 1 -- 172.21.15.5:0/1715888613 shutdown_connections 2024-09-12T02:58:19.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:19.771+0000 7fb73d625640 1 -- 172.21.15.5:0/1715888613 wait complete. 2024-09-12T02:58:20.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:58:20 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/1715888613' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:58:20.665 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:58:20 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/1715888613' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:58:20.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:58:20 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/1715888613' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:58:21.069 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T02:58:21.343 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:58:21.665 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:58:21 smithi148 ceph-mon[35090]: pgmap v338: 97 pgs: 97 active+clean; 653 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 511 B/s wr, 11 op/s 2024-09-12T02:58:21.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:58:21 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:58:21.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:58:21 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:58:21.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:58:21 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:58:21.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:58:21 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:58:21.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:58:21 smithi146 ceph-mon[35011]: pgmap v338: 97 pgs: 97 active+clean; 653 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 511 B/s wr, 11 op/s 2024-09-12T02:58:21.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:58:21 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:58:21.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:58:21 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:58:21.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:58:21 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:58:21.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:58:21 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:58:21.725 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:58:21 smithi005 ceph-mon[31197]: pgmap v338: 97 pgs: 97 active+clean; 653 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 511 B/s wr, 11 op/s 2024-09-12T02:58:21.725 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:58:21 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:58:21.725 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:58:21 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:58:21.725 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:58:21 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:58:21.725 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:58:21 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:58:21.825 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:21.810+0000 7f30825f7640 1 -- 172.21.15.5:0/1707038145 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f307c1049a0 msgr2=0x7f307c1089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:21.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:21.810+0000 7f30825f7640 1 --2- 172.21.15.5:0/1707038145 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f307c1049a0 0x7f307c1089f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3070009900 tx=0x7f307002a410 comp rx=0 tx=0).stop 2024-09-12T02:58:21.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:21.810+0000 7f30825f7640 1 -- 172.21.15.5:0/1707038145 shutdown_connections 2024-09-12T02:58:21.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:21.810+0000 7f30825f7640 1 --2- 172.21.15.5:0/1707038145 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f307c109120 0x7f307c110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:21.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:21.810+0000 7f30825f7640 1 --2- 172.21.15.5:0/1707038145 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f307c1049a0 0x7f307c1089f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:21.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:21.810+0000 7f30825f7640 1 --2- 172.21.15.5:0/1707038145 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f307c103ff0 0x7f307c1043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:21.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:21.810+0000 7f30825f7640 1 -- 172.21.15.5:0/1707038145 >> 172.21.15.5:0/1707038145 conn(0x7f307c0ffa90 msgr2=0x7f307c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:21.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:21.810+0000 7f30825f7640 1 -- 172.21.15.5:0/1707038145 shutdown_connections 2024-09-12T02:58:21.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:21.810+0000 7f30825f7640 1 -- 172.21.15.5:0/1707038145 wait complete. 2024-09-12T02:58:21.826 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:21.811+0000 7f30825f7640 1 Processor -- start 2024-09-12T02:58:21.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:21.811+0000 7f30825f7640 1 -- start start 2024-09-12T02:58:21.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:21.811+0000 7f30825f7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f307c103ff0 0x7f307c19a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:21.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:21.811+0000 7f30825f7640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f307c1049a0 0x7f307c19ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:21.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:21.811+0000 7f30825f7640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f307c109120 0x7f307c19ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:21.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:21.811+0000 7f30825f7640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f307c114170 con 0x7f307c103ff0 2024-09-12T02:58:21.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:21.811+0000 7f30825f7640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f307c113ff0 con 0x7f307c1049a0 2024-09-12T02:58:21.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:21.811+0000 7f30825f7640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f307c1142f0 con 0x7f307c109120 2024-09-12T02:58:21.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:21.812+0000 7f307bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f307c103ff0 0x7f307c19a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:21.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:21.812+0000 7f307bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f307c103ff0 0x7f307c19a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50668/0 (socket says 172.21.15.5:50668) 2024-09-12T02:58:21.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:21.812+0000 7f307bfff640 1 -- 172.21.15.5:0/2663477387 learned_addr learned my addr 172.21.15.5:0/2663477387 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:58:21.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:21.812+0000 7f3080b94640 1 --2- 172.21.15.5:0/2663477387 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f307c109120 0x7f307c19ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:21.827 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:21.812+0000 7f307b7fe640 1 --2- 172.21.15.5:0/2663477387 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f307c1049a0 0x7f307c19ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:21.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:21.812+0000 7f307bfff640 1 -- 172.21.15.5:0/2663477387 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f307c109120 msgr2=0x7f307c19ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:21.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:21.812+0000 7f307bfff640 1 --2- 172.21.15.5:0/2663477387 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f307c109120 0x7f307c19ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:21.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:21.812+0000 7f307bfff640 1 -- 172.21.15.5:0/2663477387 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f307c1049a0 msgr2=0x7f307c19ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:21.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:21.812+0000 7f307bfff640 1 --2- 172.21.15.5:0/2663477387 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f307c1049a0 0x7f307c19ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:21.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:21.812+0000 7f307bfff640 1 -- 172.21.15.5:0/2663477387 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f307c19f600 con 0x7f307c103ff0 2024-09-12T02:58:21.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:21.812+0000 7f3080b94640 1 --2- 172.21.15.5:0/2663477387 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f307c109120 0x7f307c19ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:58:21.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:21.812+0000 7f307bfff640 1 --2- 172.21.15.5:0/2663477387 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f307c103ff0 0x7f307c19a650 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f306c00ea60 tx=0x7f306c00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:21.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:21.812+0000 7f30797fa640 1 -- 172.21.15.5:0/2663477387 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f306c00cde0 con 0x7f307c103ff0 2024-09-12T02:58:21.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:21.812+0000 7f30825f7640 1 -- 172.21.15.5:0/2663477387 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f307c19f8f0 con 0x7f307c103ff0 2024-09-12T02:58:21.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:21.812+0000 7f30797fa640 1 -- 172.21.15.5:0/2663477387 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f306c0035f0 con 0x7f307c103ff0 2024-09-12T02:58:21.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:21.812+0000 7f30797fa640 1 -- 172.21.15.5:0/2663477387 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f306c010670 con 0x7f307c103ff0 2024-09-12T02:58:21.828 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:21.812+0000 7f30825f7640 1 -- 172.21.15.5:0/2663477387 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f307c1a71d0 con 0x7f307c103ff0 2024-09-12T02:58:21.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:21.813+0000 7f30825f7640 1 -- 172.21.15.5:0/2663477387 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f307c1043d0 con 0x7f307c103ff0 2024-09-12T02:58:21.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:21.814+0000 7f30797fa640 1 -- 172.21.15.5:0/2663477387 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7f306c0037a0 con 0x7f307c103ff0 2024-09-12T02:58:21.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:21.815+0000 7f30797fa640 1 --2- 172.21.15.5:0/2663477387 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f30640a2c10 0x7f30640a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:21.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:21.815+0000 7f307b7fe640 1 --2- 172.21.15.5:0/2663477387 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f30640a2c10 0x7f30640a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:21.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:21.815+0000 7f30797fa640 1 -- 172.21.15.5:0/2663477387 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f306c0cf440 con 0x7f307c103ff0 2024-09-12T02:58:21.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:21.816+0000 7f307b7fe640 1 --2- 172.21.15.5:0/2663477387 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f30640a2c10 0x7f30640a50d0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f3070000c00 tx=0x7f3070005b50 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:21.829 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:21.819+0000 7f30797fa640 1 -- 172.21.15.5:0/2663477387 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f306c09c250 con 0x7f307c103ff0 2024-09-12T02:58:22.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:22.001+0000 7f30825f7640 1 -- 172.21.15.5:0/2663477387 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f307c10c360 con 0x7f307c103ff0 2024-09-12T02:58:22.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:22.003+0000 7f30797fa640 1 -- 172.21.15.5:0/2663477387 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+22656 (secure 0 0 0) 0x7f306c09bb50 con 0x7f307c103ff0 2024-09-12T02:58:22.003 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-12T02:58:22.004 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":83,"fsid":"618305f8-70b1-11ef-bcea-c7b262605968","created":"2024-09-12T02:48:42.352146+0000","modified":"2024-09-12T02:56:31.026079+0000","last_up_change":"2024-09-12T02:55:27.817161+0000","last_in_change":"2024-09-12T02:55:03.745281+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T02:51:47.970639+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T02:56:12.834271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T02:56:14.951437+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17da560a-6aa2-4f2c-a95f-67481dce4d3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":4248507573},{"type":"v1","addr":"172.21.15.5:6803","nonce":4248507573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":4248507573},{"type":"v1","addr":"172.21.15.5:6805","nonce":4248507573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":4248507573},{"type":"v1","addr":"172.21.15.5:6809","nonce":4248507573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":4248507573},{"type":"v1","addr":"172.21.15.5:6807","nonce":4248507573}]},"public_addr":"172.21.15.5:6803/4248507573","cluster_addr":"172.21.15.5:6805/4248507573","heartbeat_back_addr":"172.21.15.5:6809/4248507573","heartbeat_front_addr":"172.21.15.5:6807/4248507573","state":["exists","up"]},{"osd":1,"uuid":"a3237006-99af-4fc1-860c-d4917032647f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":1105990287},{"type":"v1","addr":"172.21.15.146:6801","nonce":1105990287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":1105990287},{"type":"v1","addr":"172.21.15.146:6803","nonce":1105990287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":1105990287},{"type":"v1","addr":"172.21.15.146:6807","nonce":1105990287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":1105990287},{"type":"v1","addr":"172.21.15.146:6805","nonce":1105990287}]},"public_addr":"172.21.15.146:6801/1105990287","cluster_addr":"172.21.15.146:6803/1105990287","heartbeat_back_addr":"172.21.15.146:6807/1105990287","heartbeat_front_addr":"172.21.15.146:6805/1105990287","state":["exists","up"]},{"osd":2,"uuid":"ea3ab84a-7739-4975-8244-31ddd5aad80d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3378174313},{"type":"v1","addr":"172.21.15.148:6801","nonce":3378174313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3378174313},{"type":"v1","addr":"172.21.15.148:6803","nonce":3378174313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3378174313},{"type":"v1","addr":"172.21.15.148:6807","nonce":3378174313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3378174313},{"type":"v1","addr":"172.21.15.148:6805","nonce":3378174313}]},"public_addr":"172.21.15.148:6801/3378174313","cluster_addr":"172.21.15.148:6803/3378174313","heartbeat_back_addr":"172.21.15.148:6807/3378174313","heartbeat_front_addr":"172.21.15.148:6805/3378174313","state":["exists","up"]},{"osd":3,"uuid":"4396a9ee-f3f3-410b-9fb1-124f1e33a1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":11741938},{"type":"v1","addr":"172.21.15.5:6811","nonce":11741938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":11741938},{"type":"v1","addr":"172.21.15.5:6813","nonce":11741938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":11741938},{"type":"v1","addr":"172.21.15.5:6817","nonce":11741938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":11741938},{"type":"v1","addr":"172.21.15.5:6815","nonce":11741938}]},"public_addr":"172.21.15.5:6811/11741938","cluster_addr":"172.21.15.5:6813/11741938","heartbeat_back_addr":"172.21.15.5:6817/11741938","heartbeat_front_addr":"172.21.15.5:6815/11741938","state":["exists","up"]},{"osd":4,"uuid":"989d76e7-c742-49d0-994a-c58f2771b1d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":3274923851},{"type":"v1","addr":"172.21.15.146:6809","nonce":3274923851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":3274923851},{"type":"v1","addr":"172.21.15.146:6811","nonce":3274923851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3274923851},{"type":"v1","addr":"172.21.15.146:6815","nonce":3274923851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":3274923851},{"type":"v1","addr":"172.21.15.146:6813","nonce":3274923851}]},"public_addr":"172.21.15.146:6809/3274923851","cluster_addr":"172.21.15.146:6811/3274923851","heartbeat_back_addr":"172.21.15.146:6815/3274923851","heartbeat_front_addr":"172.21.15.146:6813/3274923851","state":["exists","up"]},{"osd":5,"uuid":"64ada7d4-70a0-4b83-9af8-032be544608d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":2771670476},{"type":"v1","addr":"172.21.15.148:6809","nonce":2771670476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":2771670476},{"type":"v1","addr":"172.21.15.148:6811","nonce":2771670476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":2771670476},{"type":"v1","addr":"172.21.15.148:6815","nonce":2771670476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":2771670476},{"type":"v1","addr":"172.21.15.148:6813","nonce":2771670476}]},"public_addr":"172.21.15.148:6809/2771670476","cluster_addr":"172.21.15.148:6811/2771670476","heartbeat_back_addr":"172.21.15.148:6815/2771670476","heartbeat_front_addr":"172.21.15.148:6813/2771670476","state":["exists","up"]},{"osd":6,"uuid":"a70a88fd-2e4b-480c-bb98-62636cf70cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3176073169},{"type":"v1","addr":"172.21.15.5:6819","nonce":3176073169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":3176073169},{"type":"v1","addr":"172.21.15.5:6821","nonce":3176073169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":3176073169},{"type":"v1","addr":"172.21.15.5:6825","nonce":3176073169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3176073169},{"type":"v1","addr":"172.21.15.5:6823","nonce":3176073169}]},"public_addr":"172.21.15.5:6819/3176073169","cluster_addr":"172.21.15.5:6821/3176073169","heartbeat_back_addr":"172.21.15.5:6825/3176073169","heartbeat_front_addr":"172.21.15.5:6823/3176073169","state":["exists","up"]},{"osd":7,"uuid":"49451e8d-c741-4ad5-b8f5-89bad8d4d129","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2209508838},{"type":"v1","addr":"172.21.15.146:6817","nonce":2209508838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2209508838},{"type":"v1","addr":"172.21.15.146:6819","nonce":2209508838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2209508838},{"type":"v1","addr":"172.21.15.146:6823","nonce":2209508838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2209508838},{"type":"v1","addr":"172.21.15.146:6821","nonce":2209508838}]},"public_addr":"172.21.15.146:6817/2209508838","cluster_addr":"172.21.15.146:6819/2209508838","heartbeat_back_addr":"172.21.15.146:6823/2209508838","heartbeat_front_addr":"172.21.15.146:6821/2209508838","state":["exists","up"]},{"osd":8,"uuid":"57bfc121-a75e-4a7f-b1dc-64074b178b03","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":651045756},{"type":"v1","addr":"172.21.15.148:6817","nonce":651045756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":651045756},{"type":"v1","addr":"172.21.15.148:6819","nonce":651045756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":651045756},{"type":"v1","addr":"172.21.15.148:6823","nonce":651045756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":651045756},{"type":"v1","addr":"172.21.15.148:6821","nonce":651045756}]},"public_addr":"172.21.15.148:6817/651045756","cluster_addr":"172.21.15.148:6819/651045756","heartbeat_back_addr":"172.21.15.148:6823/651045756","heartbeat_front_addr":"172.21.15.148:6821/651045756","state":["exists","up"]},{"osd":9,"uuid":"d9726168-7494-4bf8-b433-350e85bfd461","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1961857360},{"type":"v1","addr":"172.21.15.5:6827","nonce":1961857360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1961857360},{"type":"v1","addr":"172.21.15.5:6829","nonce":1961857360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":1961857360},{"type":"v1","addr":"172.21.15.5:6833","nonce":1961857360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1961857360},{"type":"v1","addr":"172.21.15.5:6831","nonce":1961857360}]},"public_addr":"172.21.15.5:6827/1961857360","cluster_addr":"172.21.15.5:6829/1961857360","heartbeat_back_addr":"172.21.15.5:6833/1961857360","heartbeat_front_addr":"172.21.15.5:6831/1961857360","state":["exists","up"]},{"osd":10,"uuid":"519ec60f-3606-48ac-9043-6f59a798766d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":2041733103},{"type":"v1","addr":"172.21.15.146:6825","nonce":2041733103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":2041733103},{"type":"v1","addr":"172.21.15.146:6827","nonce":2041733103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":2041733103},{"type":"v1","addr":"172.21.15.146:6831","nonce":2041733103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":2041733103},{"type":"v1","addr":"172.21.15.146:6829","nonce":2041733103}]},"public_addr":"172.21.15.146:6825/2041733103","cluster_addr":"172.21.15.146:6827/2041733103","heartbeat_back_addr":"172.21.15.146:6831/2041733103","heartbeat_front_addr":"172.21.15.146:6829/2041733103","state":["exists","up"]},{"osd":11,"uuid":"0f92d0cb-ba54-4bc3-a6d0-3c54d910556a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":839290773},{"type":"v1","addr":"172.21.15.148:6825","nonce":839290773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":839290773},{"type":"v1","addr":"172.21.15.148:6827","nonce":839290773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":839290773},{"type":"v1","addr":"172.21.15.148:6831","nonce":839290773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":839290773},{"type":"v1","addr":"172.21.15.148:6829","nonce":839290773}]},"public_addr":"172.21.15.148:6825/839290773","cluster_addr":"172.21.15.148:6827/839290773","heartbeat_back_addr":"172.21.15.148:6831/839290773","heartbeat_front_addr":"172.21.15.148:6829/839290773","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:50:54.101755+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:51:19.037385+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:51:44.142837+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:52:09.150887+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:52:33.705079+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:52:58.845233+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:53:23.068339+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:53:47.520398+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:54:11.965370+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:54:36.776075+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:55:01.292236+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:55:25.346589+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:6800/3125821798":"2024-09-13T02:49:29.462216+0000","172.21.15.5:0/3479911451":"2024-09-13T02:49:29.462216+0000","172.21.15.5:0/1547753207":"2024-09-13T02:49:03.300872+0000","172.21.15.5:0/1905762627":"2024-09-13T02:49:03.300872+0000","172.21.15.5:0/4142340280":"2024-09-13T02:49:29.462216+0000","172.21.15.5:6801/3125821798":"2024-09-13T02:49:29.462216+0000","172.21.15.5:6801/2595781680":"2024-09-13T02:49:03.300872+0000","172.21.15.5:0/2273691311":"2024-09-13T02:49:29.462216+0000","172.21.15.5:6800/2595781680":"2024-09-13T02:49:03.300872+0000","172.21.15.5:0/1196201217":"2024-09-13T02:49:03.300872+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T02:58:22.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:22.006+0000 7f30825f7640 1 -- 172.21.15.5:0/2663477387 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f30640a2c10 msgr2=0x7f30640a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:22.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:22.006+0000 7f30825f7640 1 --2- 172.21.15.5:0/2663477387 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f30640a2c10 0x7f30640a50d0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f3070000c00 tx=0x7f3070005b50 comp rx=0 tx=0).stop 2024-09-12T02:58:22.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:22.006+0000 7f30825f7640 1 -- 172.21.15.5:0/2663477387 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f307c103ff0 msgr2=0x7f307c19a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:22.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:22.006+0000 7f30825f7640 1 --2- 172.21.15.5:0/2663477387 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f307c103ff0 0x7f307c19a650 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f306c00ea60 tx=0x7f306c00ef20 comp rx=0 tx=0).stop 2024-09-12T02:58:22.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:22.006+0000 7f30825f7640 1 -- 172.21.15.5:0/2663477387 shutdown_connections 2024-09-12T02:58:22.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:22.006+0000 7f30825f7640 1 --2- 172.21.15.5:0/2663477387 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f307c109120 0x7f307c19ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:22.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:22.006+0000 7f30825f7640 1 --2- 172.21.15.5:0/2663477387 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f307c1049a0 0x7f307c19ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:22.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:22.006+0000 7f30825f7640 1 --2- 172.21.15.5:0/2663477387 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f30640a2c10 0x7f30640a50d0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:22.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:22.006+0000 7f30825f7640 1 --2- 172.21.15.5:0/2663477387 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f307c103ff0 0x7f307c19a650 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:22.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:22.006+0000 7f30825f7640 1 -- 172.21.15.5:0/2663477387 >> 172.21.15.5:0/2663477387 conn(0x7f307c0ffa90 msgr2=0x7f307c100da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:22.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:22.006+0000 7f30825f7640 1 -- 172.21.15.5:0/2663477387 shutdown_connections 2024-09-12T02:58:22.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:22.006+0000 7f30825f7640 1 -- 172.21.15.5:0/2663477387 wait complete. 2024-09-12T02:58:22.713 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:58:22 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:58:22.714 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:58:22 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:58:22.714 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:58:22 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:58:22.714 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:58:22 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:58:22.714 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:58:22 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:58:22.714 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:58:22 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/2663477387' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:58:22.748 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph daemon mon.a --format=json config get run_dir 2024-09-12T02:58:22.915 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:58:22 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:58:22.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:58:22 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:58:22.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:58:22 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:58:22.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:58:22 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:58:22.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:58:22 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:58:22.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:58:22 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/2663477387' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:58:22.956 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:58:22 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:58:22.956 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:58:22 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:58:22.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:58:22 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:58:22.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:58:22 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:58:22.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:58:22 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:58:22.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:58:22 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/2663477387' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T02:58:23.010 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:58:23.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:58:23 smithi005 ceph-mon[31197]: pgmap v339: 97 pgs: 97 active+clean; 653 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-12T02:58:23.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:58:23 smithi005 ceph-mon[31197]: scrub summary: idle 2024-09-12T02:58:23.915 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:58:23 smithi148 ceph-mon[35090]: pgmap v339: 97 pgs: 97 active+clean; 653 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-12T02:58:23.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:58:23 smithi148 ceph-mon[35090]: scrub summary: idle 2024-09-12T02:58:23.956 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:58:23 smithi146 ceph-mon[35011]: pgmap v339: 97 pgs: 97 active+clean; 653 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-12T02:58:23.956 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:58:23 smithi146 ceph-mon[35011]: scrub summary: idle 2024-09-12T02:58:24.133 INFO:teuthology.orchestra.run.smithi005.stdout:{"run_dir":"/var/run/ceph"} 2024-09-12T02:58:24.133 DEBUG:tasks.cephfs.filesystem:_json_asok output { "run_dir": "/var/run/ceph" } 2024-09-12T02:58:24.133 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with seed 481 2024-09-12T02:58:24.134 DEBUG:tasks.quiescer:Yielding 2024-09-12T02:58:24.134 INFO:teuthology.run_tasks:Running task workunit... 2024-09-12T02:58:24.148 INFO:tasks.workunit:Pulling workunits from ref 086c737fa75c22d521a3932c59b31c5e1d1c9ef4 2024-09-12T02:58:24.148 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-12T02:58:24.148 INFO:tasks.workunit:timeout=3h 2024-09-12T02:58:24.148 INFO:tasks.workunit:cleanup=True 2024-09-12T02:58:24.149 DEBUG:teuthology.orchestra.run.smithi005:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-12T02:58:24.149 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:58:24.167 INFO:teuthology.orchestra.run.smithi005.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-09-12T02:58:24.167 INFO:teuthology.orchestra.run.smithi005.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-09-12T02:58:24.167 INFO:teuthology.orchestra.run.smithi005.stdout:Device: 78h/120d Inode: 1099511628283 Links: 2 2024-09-12T02:58:24.167 INFO:teuthology.orchestra.run.smithi005.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-12T02:58:24.167 INFO:teuthology.orchestra.run.smithi005.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-09-12T02:58:24.167 INFO:teuthology.orchestra.run.smithi005.stdout:Access: 2024-09-12 02:56:34.160240368 +0000 2024-09-12T02:58:24.168 INFO:teuthology.orchestra.run.smithi005.stdout:Modify: 2024-09-12 02:56:34.160240368 +0000 2024-09-12T02:58:24.168 INFO:teuthology.orchestra.run.smithi005.stdout:Change: 2024-09-12 02:58:01.773917456 +0000 2024-09-12T02:58:24.168 INFO:teuthology.orchestra.run.smithi005.stdout: Birth: 2024-09-12 02:56:34.160240368 +0000 2024-09-12T02:58:24.168 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-09-12T02:58:24.168 DEBUG:teuthology.orchestra.run.smithi005:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-09-12T02:58:24.241 DEBUG:teuthology.orchestra.run.smithi005:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 086c737fa75c22d521a3932c59b31c5e1d1c9ef4 2024-09-12T02:58:24.298 INFO:tasks.workunit.client.0.smithi005.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-12T02:58:24.421 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:58:24.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:24.959+0000 7fc3ffcf5640 1 -- 172.21.15.5:0/2825593609 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc3f8101430 msgr2=0x7fc3f810d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:24.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:24.959+0000 7fc3ffcf5640 1 --2- 172.21.15.5:0/2825593609 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc3f8101430 0x7fc3f810d960 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fc3f0009cd0 tx=0x7fc3f002a8e0 comp rx=0 tx=0).stop 2024-09-12T02:58:24.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:24.959+0000 7fc3ffcf5640 1 -- 172.21.15.5:0/2825593609 shutdown_connections 2024-09-12T02:58:24.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:24.959+0000 7fc3ffcf5640 1 --2- 172.21.15.5:0/2825593609 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc3f8101430 0x7fc3f810d960 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:24.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:24.959+0000 7fc3ffcf5640 1 --2- 172.21.15.5:0/2825593609 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc3f8100a90 0x7fc3f8100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:24.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:24.959+0000 7fc3ffcf5640 1 --2- 172.21.15.5:0/2825593609 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc3f8106a90 0x7fc3f8106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:24.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:24.959+0000 7fc3ffcf5640 1 -- 172.21.15.5:0/2825593609 >> 172.21.15.5:0/2825593609 conn(0x7fc3f80fc7d0 msgr2=0x7fc3f80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:24.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:24.959+0000 7fc3ffcf5640 1 -- 172.21.15.5:0/2825593609 shutdown_connections 2024-09-12T02:58:24.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:24.959+0000 7fc3ffcf5640 1 -- 172.21.15.5:0/2825593609 wait complete. 2024-09-12T02:58:24.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:24.959+0000 7fc3ffcf5640 1 Processor -- start 2024-09-12T02:58:24.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:24.960+0000 7fc3ffcf5640 1 -- start start 2024-09-12T02:58:24.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:24.960+0000 7fc3ffcf5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc3f8100a90 0x7fc3f819a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:24.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:24.960+0000 7fc3ffcf5640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc3f8101430 0x7fc3f819ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:24.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:24.960+0000 7fc3ffcf5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc3f8106a90 0x7fc3f819eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:24.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:24.960+0000 7fc3ffcf5640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc3f8110e90 con 0x7fc3f8106a90 2024-09-12T02:58:24.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:24.960+0000 7fc3ffcf5640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fc3f8110d10 con 0x7fc3f8101430 2024-09-12T02:58:24.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:24.960+0000 7fc3ffcf5640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc3f8111010 con 0x7fc3f8100a90 2024-09-12T02:58:24.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:24.960+0000 7fc3fe292640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc3f8106a90 0x7fc3f819eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:24.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:24.960+0000 7fc3fe292640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc3f8106a90 0x7fc3f819eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39146/0 (socket says 172.21.15.5:39146) 2024-09-12T02:58:24.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:24.960+0000 7fc3fe292640 1 -- 172.21.15.5:0/4266173645 learned_addr learned my addr 172.21.15.5:0/4266173645 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:58:24.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:24.960+0000 7fc3fda91640 1 --2- 172.21.15.5:0/4266173645 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc3f8100a90 0x7fc3f819a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:24.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:24.960+0000 7fc3fd290640 1 --2- 172.21.15.5:0/4266173645 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc3f8101430 0x7fc3f819ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:24.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:24.961+0000 7fc3fe292640 1 -- 172.21.15.5:0/4266173645 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc3f8100a90 msgr2=0x7fc3f819a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:24.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:24.961+0000 7fc3fe292640 1 --2- 172.21.15.5:0/4266173645 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc3f8100a90 0x7fc3f819a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:24.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:24.961+0000 7fc3fe292640 1 -- 172.21.15.5:0/4266173645 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc3f8101430 msgr2=0x7fc3f819ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:24.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:24.961+0000 7fc3fe292640 1 --2- 172.21.15.5:0/4266173645 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc3f8101430 0x7fc3f819ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:24.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:24.961+0000 7fc3fe292640 1 -- 172.21.15.5:0/4266173645 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3f819f5d0 con 0x7fc3f8106a90 2024-09-12T02:58:24.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:24.961+0000 7fc3fda91640 1 --2- 172.21.15.5:0/4266173645 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc3f8100a90 0x7fc3f819a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:58:24.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:24.961+0000 7fc3fe292640 1 --2- 172.21.15.5:0/4266173645 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc3f8106a90 0x7fc3f819eef0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fc3f0005cc0 tx=0x7fc3f0032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:24.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:24.961+0000 7fc3eeffd640 1 -- 172.21.15.5:0/4266173645 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3f0004040 con 0x7fc3f8106a90 2024-09-12T02:58:24.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:24.961+0000 7fc3ffcf5640 1 -- 172.21.15.5:0/4266173645 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3f819f860 con 0x7fc3f8106a90 2024-09-12T02:58:24.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:24.961+0000 7fc3eeffd640 1 -- 172.21.15.5:0/4266173645 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc3f00046c0 con 0x7fc3f8106a90 2024-09-12T02:58:24.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:24.961+0000 7fc3eeffd640 1 -- 172.21.15.5:0/4266173645 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3f0030e00 con 0x7fc3f8106a90 2024-09-12T02:58:24.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:24.961+0000 7fc3ffcf5640 1 -- 172.21.15.5:0/4266173645 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3f81a71a0 con 0x7fc3f8106a90 2024-09-12T02:58:24.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:24.963+0000 7fc3ffcf5640 1 -- 172.21.15.5:0/4266173645 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3f8100ef0 con 0x7fc3f8106a90 2024-09-12T02:58:24.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:24.964+0000 7fc3eeffd640 1 -- 172.21.15.5:0/4266173645 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7fc3f00305a0 con 0x7fc3f8106a90 2024-09-12T02:58:24.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:24.964+0000 7fc3eeffd640 1 --2- 172.21.15.5:0/4266173645 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fc3e40a2c10 0x7fc3e40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:24.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:24.965+0000 7fc3eeffd640 1 -- 172.21.15.5:0/4266173645 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fc3f002e030 con 0x7fc3f8106a90 2024-09-12T02:58:24.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:24.965+0000 7fc3fda91640 1 --2- 172.21.15.5:0/4266173645 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fc3e40a2c10 0x7fc3e40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:24.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:24.966+0000 7fc3fda91640 1 --2- 172.21.15.5:0/4266173645 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fc3e40a2c10 0x7fc3e40a50d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fc3f4005fd0 tx=0x7fc3f400b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:24.969 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:24.969+0000 7fc3eeffd640 1 -- 172.21.15.5:0/4266173645 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc3f004b060 con 0x7fc3f8106a90 2024-09-12T02:58:25.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:25.182+0000 7fc3ffcf5640 1 -- 172.21.15.5:0/4266173645 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc3f819b980 con 0x7fc3f8106a90 2024-09-12T02:58:25.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:25.182+0000 7fc3eeffd640 1 -- 172.21.15.5:0/4266173645 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11966 (secure 0 0 0) 0x7fc3f819b980 con 0x7fc3f8106a90 2024-09-12T02:58:25.184 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-12T02:58:25.184 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":26,"btime":"2024-09-12T02:58:03:323671+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/4135232521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":4135232521},{"type":"v1","addr":"172.21.15.5:6835","nonce":4135232521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/4290221940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":4290221940},{"type":"v1","addr":"172.21.15.5:6837","nonce":4290221940}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T02:56:16.595562+0000","modified":"2024-09-12T02:58:02.463400+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24502,"mds_1":24487,"mds_2":24481,"mds_3":24479,"mds_4":24458},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/26447341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":26447341},{"type":"v1","addr":"172.21.15.5:6839","nonce":26447341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2393443295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2393443295},{"type":"v1","addr":"172.21.15.5:6841","nonce":2393443295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6833/3686970137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3686970137},{"type":"v1","addr":"172.21.15.146:6833","nonce":3686970137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6833/568512209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":568512209},{"type":"v1","addr":"172.21.15.148:6833","nonce":568512209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6835/2540740420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2540740420},{"type":"v1","addr":"172.21.15.146:6835","nonce":2540740420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.146:6837/1673370937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":1673370937},{"type":"v1","addr":"172.21.15.146:6837","nonce":1673370937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":3,"incarnation":22,"state":"up:active","state_seq":6,"addr":"172.21.15.146:6839/3036767228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":3036767228},{"type":"v1","addr":"172.21.15.146:6839","nonce":3036767228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":2,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.148:6835/4195086427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":4195086427},{"type":"v1","addr":"172.21.15.148:6835","nonce":4195086427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.148:6837/2606873922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2606873922},{"type":"v1","addr":"172.21.15.148:6837","nonce":2606873922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6839/2317600000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":2317600000},{"type":"v1","addr":"172.21.15.148:6839","nonce":2317600000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24502,"qdb_cluster":[24502,24487,24481,24479,24458]},"id":1}]} 2024-09-12T02:58:25.186 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 26 2024-09-12T02:58:25.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:25.184+0000 7fc3ffcf5640 1 -- 172.21.15.5:0/4266173645 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fc3e40a2c10 msgr2=0x7fc3e40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:25.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:25.184+0000 7fc3ffcf5640 1 --2- 172.21.15.5:0/4266173645 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fc3e40a2c10 0x7fc3e40a50d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fc3f4005fd0 tx=0x7fc3f400b390 comp rx=0 tx=0).stop 2024-09-12T02:58:25.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:25.184+0000 7fc3ffcf5640 1 -- 172.21.15.5:0/4266173645 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc3f8106a90 msgr2=0x7fc3f819eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:25.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:25.184+0000 7fc3ffcf5640 1 --2- 172.21.15.5:0/4266173645 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc3f8106a90 0x7fc3f819eef0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fc3f0005cc0 tx=0x7fc3f0032f40 comp rx=0 tx=0).stop 2024-09-12T02:58:25.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:25.185+0000 7fc3ffcf5640 1 -- 172.21.15.5:0/4266173645 shutdown_connections 2024-09-12T02:58:25.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:25.185+0000 7fc3ffcf5640 1 --2- 172.21.15.5:0/4266173645 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fc3e40a2c10 0x7fc3e40a50d0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:25.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:25.185+0000 7fc3ffcf5640 1 --2- 172.21.15.5:0/4266173645 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc3f8106a90 0x7fc3f819eef0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:25.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:25.185+0000 7fc3ffcf5640 1 --2- 172.21.15.5:0/4266173645 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc3f8101430 0x7fc3f819ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:25.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:25.185+0000 7fc3ffcf5640 1 --2- 172.21.15.5:0/4266173645 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc3f8100a90 0x7fc3f819a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:25.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:25.185+0000 7fc3ffcf5640 1 -- 172.21.15.5:0/4266173645 >> 172.21.15.5:0/4266173645 conn(0x7fc3f80fc7d0 msgr2=0x7fc3f806a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:25.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:25.185+0000 7fc3ffcf5640 1 -- 172.21.15.5:0/4266173645 shutdown_connections 2024-09-12T02:58:25.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:25.185+0000 7fc3ffcf5640 1 -- 172.21.15.5:0/4266173645 wait complete. 2024-09-12T02:58:25.771 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:58:25 smithi005 ceph-mon[31197]: pgmap v340: 97 pgs: 97 active+clean; 653 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-12T02:58:25.771 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:58:25 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/4266173645' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:58:25.799 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-12T02:56:16.595562+0000', 'modified': '2024-09-12T02:58:02.463400+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24502, 'mds_1': 24487, 'mds_2': 24481, 'mds_3': 24479, 'mds_4': 24458}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14601': {'gid': 14601, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.5:6839/26447341', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6838', 'nonce': 26447341}, {'type': 'v1', 'addr': '172.21.15.5:6839', 'nonce': 26447341}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14613': {'gid': 14613, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.5:6841/2393443295', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.5:6840', 'nonce': 2393443295}, {'type': 'v1', 'addr': '172.21.15.5:6841', 'nonce': 2393443295}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24407': {'gid': 24407, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.146:6833/3686970137', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.146:6832', 'nonce': 3686970137}, {'type': 'v1', 'addr': '172.21.15.146:6833', 'nonce': 3686970137}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24439': {'gid': 24439, 'name': 'l', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.148:6833/568512209', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6832', 'nonce': 568512209}, {'type': 'v1', 'addr': '172.21.15.148:6833', 'nonce': 568512209}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24440': {'gid': 24440, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.146:6835/2540740420', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.146:6834', 'nonce': 2540740420}, {'type': 'v1', 'addr': '172.21.15.146:6835', 'nonce': 2540740420}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24458': {'gid': 24458, 'name': 'h', 'rank': 4, 'incarnation': 24, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.146:6837/1673370937', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.146:6836', 'nonce': 1673370937}, {'type': 'v1', 'addr': '172.21.15.146:6837', 'nonce': 1673370937}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24479': {'gid': 24479, 'name': 'b', 'rank': 3, 'incarnation': 22, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.146:6839/3036767228', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.146:6838', 'nonce': 3036767228}, {'type': 'v1', 'addr': '172.21.15.146:6839', 'nonce': 3036767228}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24481': {'gid': 24481, 'name': 'f', 'rank': 2, 'incarnation': 20, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.148:6835/4195086427', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6834', 'nonce': 4195086427}, {'type': 'v1', 'addr': '172.21.15.148:6835', 'nonce': 4195086427}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24487': {'gid': 24487, 'name': 'c', 'rank': 1, 'incarnation': 18, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.148:6837/2606873922', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6836', 'nonce': 2606873922}, {'type': 'v1', 'addr': '172.21.15.148:6837', 'nonce': 2606873922}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24502': {'gid': 24502, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.148:6839/2317600000', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.148:6838', 'nonce': 2317600000}, {'type': 'v1', 'addr': '172.21.15.148:6839', 'nonce': 2317600000}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24502, 'qdb_cluster': [24502, 24487, 24481, 24479, 24458]} 2024-09-12T02:58:25.799 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-09-12T02:58:25.800 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.h status 2024-09-12T02:58:25.915 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:58:25 smithi148 ceph-mon[35090]: pgmap v340: 97 pgs: 97 active+clean; 653 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-12T02:58:25.915 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:58:25 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/4266173645' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:58:25.956 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:58:25 smithi146 ceph-mon[35011]: pgmap v340: 97 pgs: 97 active+clean; 653 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-09-12T02:58:25.956 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:58:25 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/4266173645' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:58:26.099 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:58:26.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.593+0000 7f16aeaff640 1 -- 172.21.15.5:0/2065614097 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16a8106a70 msgr2=0x7f16a8106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:26.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.593+0000 7f16aeaff640 1 --2- 172.21.15.5:0/2065614097 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16a8106a70 0x7f16a8106e50 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f1698009930 tx=0x7f169802a420 comp rx=0 tx=0).stop 2024-09-12T02:58:26.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.593+0000 7f16aeaff640 1 -- 172.21.15.5:0/2065614097 shutdown_connections 2024-09-12T02:58:26.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.593+0000 7f16aeaff640 1 --2- 172.21.15.5:0/2065614097 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f16a8101410 0x7f16a810d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:26.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.593+0000 7f16aeaff640 1 --2- 172.21.15.5:0/2065614097 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16a8100a70 0x7f16a8100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:26.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.593+0000 7f16aeaff640 1 --2- 172.21.15.5:0/2065614097 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16a8106a70 0x7f16a8106e50 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:26.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.593+0000 7f16aeaff640 1 -- 172.21.15.5:0/2065614097 >> 172.21.15.5:0/2065614097 conn(0x7f16a80fc790 msgr2=0x7f16a80febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:26.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.593+0000 7f16aeaff640 1 -- 172.21.15.5:0/2065614097 shutdown_connections 2024-09-12T02:58:26.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.593+0000 7f16aeaff640 1 -- 172.21.15.5:0/2065614097 wait complete. 2024-09-12T02:58:26.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.593+0000 7f16aeaff640 1 Processor -- start 2024-09-12T02:58:26.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.593+0000 7f16aeaff640 1 -- start start 2024-09-12T02:58:26.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.594+0000 7f16aeaff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16a8100a70 0x7f16a810e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:26.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.594+0000 7f16aeaff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f16a8101410 0x7f16a810e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:26.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.594+0000 7f16aeaff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16a8106a70 0x7f16a810ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:26.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.594+0000 7f16aeaff640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f16a8111f90 con 0x7f16a8106a70 2024-09-12T02:58:26.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.594+0000 7f16aeaff640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f16a8111e10 con 0x7f16a8101410 2024-09-12T02:58:26.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.594+0000 7f16aeaff640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f16a8112110 con 0x7f16a8100a70 2024-09-12T02:58:26.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.594+0000 7f16ad09c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16a8106a70 0x7f16a810ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:26.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.594+0000 7f16ad09c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16a8106a70 0x7f16a810ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39170/0 (socket says 172.21.15.5:39170) 2024-09-12T02:58:26.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.594+0000 7f16ad09c640 1 -- 172.21.15.5:0/2768003097 learned_addr learned my addr 172.21.15.5:0/2768003097 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:58:26.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.594+0000 7f16ac89b640 1 --2- 172.21.15.5:0/2768003097 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16a8100a70 0x7f16a810e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:26.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.594+0000 7f16a7fff640 1 --2- 172.21.15.5:0/2768003097 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f16a8101410 0x7f16a810e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:26.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.594+0000 7f16ad09c640 1 -- 172.21.15.5:0/2768003097 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16a8100a70 msgr2=0x7f16a810e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:26.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.594+0000 7f16ad09c640 1 --2- 172.21.15.5:0/2768003097 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16a8100a70 0x7f16a810e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:26.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.594+0000 7f16ad09c640 1 -- 172.21.15.5:0/2768003097 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f16a8101410 msgr2=0x7f16a810e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:26.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.594+0000 7f16ad09c640 1 --2- 172.21.15.5:0/2768003097 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f16a8101410 0x7f16a810e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:26.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.594+0000 7f16ad09c640 1 -- 172.21.15.5:0/2768003097 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16a81a6990 con 0x7f16a8106a70 2024-09-12T02:58:26.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.594+0000 7f16a7fff640 1 --2- 172.21.15.5:0/2768003097 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f16a8101410 0x7f16a810e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:58:26.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.594+0000 7f16ac89b640 1 --2- 172.21.15.5:0/2768003097 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16a8100a70 0x7f16a810e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:58:26.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.594+0000 7f16ad09c640 1 --2- 172.21.15.5:0/2768003097 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16a8106a70 0x7f16a810ee50 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f16a000cce0 tx=0x7f16a000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:26.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.594+0000 7f16a5ffb640 1 -- 172.21.15.5:0/2768003097 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a000f450 con 0x7f16a8106a70 2024-09-12T02:58:26.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.595+0000 7f16aeaff640 1 -- 172.21.15.5:0/2768003097 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16a81a6c80 con 0x7f16a8106a70 2024-09-12T02:58:26.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.595+0000 7f16a5ffb640 1 -- 172.21.15.5:0/2768003097 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16a000fad0 con 0x7f16a8106a70 2024-09-12T02:58:26.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.595+0000 7f16a5ffb640 1 -- 172.21.15.5:0/2768003097 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a0018c10 con 0x7f16a8106a70 2024-09-12T02:58:26.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.595+0000 7f16aeaff640 1 -- 172.21.15.5:0/2768003097 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16a81a7170 con 0x7f16a8106a70 2024-09-12T02:58:26.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.598+0000 7f16a5ffb640 1 -- 172.21.15.5:0/2768003097 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7f16a000f5f0 con 0x7f16a8106a70 2024-09-12T02:58:26.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.598+0000 7f16b0ed8740 1 Processor -- start 2024-09-12T02:58:26.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.598+0000 7f16b0ed8740 1 -- start start 2024-09-12T02:58:26.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.598+0000 7f16b0ed8740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558e500cc9d0 0x558e500ccdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:26.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.598+0000 7f16b0ed8740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558e500cd2f0 0x558e500d7670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:26.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.598+0000 7f16b0ed8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e500d7bb0 0x558e500d9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:26.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.598+0000 7f16b0ed8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558e50033090 con 0x558e500d7bb0 2024-09-12T02:58:26.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.598+0000 7f16b0ed8740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x558e4ffdaa70 con 0x558e500cd2f0 2024-09-12T02:58:26.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.598+0000 7f16b0ed8740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x558e5002f050 con 0x558e500cc9d0 2024-09-12T02:58:26.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.598+0000 7f16a5ffb640 1 --2- 172.21.15.5:0/2768003097 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f16940a2c10 0x7f16940a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:26.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.598+0000 7f16a7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e500d7bb0 0x558e500d9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:26.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.598+0000 7f16a7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e500d7bb0 0x558e500d9fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39186/0 (socket says 172.21.15.5:39186) 2024-09-12T02:58:26.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.598+0000 7f16a7fff640 1 -- 172.21.15.5:0/661139722 learned_addr learned my addr 172.21.15.5:0/661139722 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:58:26.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.599+0000 7f16a7fff640 1 --2- 172.21.15.5:0/661139722 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558e500cc9d0 0x558e500ccdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:26.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.599+0000 7f16a5ffb640 1 -- 172.21.15.5:0/2768003097 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f16a00cf930 con 0x7f16a8106a70 2024-09-12T02:58:26.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.599+0000 7f16ad09c640 1 --2- 172.21.15.5:0/2768003097 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f16940a2c10 0x7f16940a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:26.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.599+0000 7f16a7fff640 1 -- 172.21.15.5:0/661139722 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558e500cc9d0 msgr2=0x558e500ccdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:26.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.599+0000 7f16a7fff640 1 --2- 172.21.15.5:0/661139722 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558e500cc9d0 0x558e500ccdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:26.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.599+0000 7f16a7fff640 1 -- 172.21.15.5:0/661139722 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558e500cd2f0 msgr2=0x558e500d7670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:26.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.599+0000 7f16a7fff640 1 --2- 172.21.15.5:0/661139722 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558e500cd2f0 0x558e500d7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:26.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.599+0000 7f16a7fff640 1 -- 172.21.15.5:0/661139722 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e4ffe7130 con 0x558e500d7bb0 2024-09-12T02:58:26.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.599+0000 7f16a7fff640 1 --2- 172.21.15.5:0/661139722 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e500d7bb0 0x558e500d9fa0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f1690016560 tx=0x7f169000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39c7f74507568783 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:26.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.599+0000 7f16ad09c640 1 --2- 172.21.15.5:0/2768003097 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f16940a2c10 0x7f16940a50d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f16a0098a10 tx=0x7f16a00185b0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:26.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.599+0000 7f169effd640 1 -- 172.21.15.5:0/661139722 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1690018430 con 0x558e500d7bb0 2024-09-12T02:58:26.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.599+0000 7f169effd640 1 -- 172.21.15.5:0/661139722 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1690018ab0 con 0x558e500d7bb0 2024-09-12T02:58:26.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.599+0000 7f169effd640 1 -- 172.21.15.5:0/661139722 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1690017870 con 0x558e500d7bb0 2024-09-12T02:58:26.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.600+0000 7f16b0ed8740 1 -- 172.21.15.5:0/661139722 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e500d7bb0 msgr2=0x558e500d9fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:26.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.600+0000 7f16b0ed8740 1 --2- 172.21.15.5:0/661139722 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e500d7bb0 0x558e500d9fa0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f1690016560 tx=0x7f169000a340 comp rx=0 tx=0).stop 2024-09-12T02:58:26.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.600+0000 7f16b0ed8740 1 -- 172.21.15.5:0/661139722 shutdown_connections 2024-09-12T02:58:26.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.600+0000 7f16b0ed8740 1 --2- 172.21.15.5:0/661139722 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e500d7bb0 0x558e500d9fa0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:26.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.600+0000 7f16b0ed8740 1 --2- 172.21.15.5:0/661139722 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558e500cd2f0 0x558e500d7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:26.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.600+0000 7f16b0ed8740 1 --2- 172.21.15.5:0/661139722 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558e500cc9d0 0x558e500ccdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:26.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.600+0000 7f16b0ed8740 1 -- 172.21.15.5:0/661139722 >> 172.21.15.5:0/661139722 conn(0x558e500c8f60 msgr2=0x558e500c9340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:26.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.600+0000 7f16b0ed8740 1 -- 172.21.15.5:0/661139722 shutdown_connections 2024-09-12T02:58:26.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.600+0000 7f16b0ed8740 1 -- 172.21.15.5:0/661139722 wait complete. 2024-09-12T02:58:26.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.601+0000 7f16b0ed8740 1 -- start start 2024-09-12T02:58:26.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.601+0000 7f16b0ed8740 1 Processor -- start 2024-09-12T02:58:26.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.601+0000 7f16b0ed8740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558e500cc9d0 0x558e500d7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:26.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.601+0000 7f16b0ed8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e500cd2f0 0x558e500d4750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:26.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.601+0000 7f16b0ed8740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558e500d7bb0 0x558e500d4c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:26.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.601+0000 7f16b0ed8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558e500da780 con 0x558e500cd2f0 2024-09-12T02:58:26.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.601+0000 7f16b0ed8740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x558e4ffedf20 con 0x558e500d7bb0 2024-09-12T02:58:26.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.601+0000 7f16b0ed8740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x558e500da900 con 0x558e500cc9d0 2024-09-12T02:58:26.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.601+0000 7f16b0ed8740 10 client.0 _ll_register_callbacks cb 0x558e4ffb7030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:58:26.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.601+0000 7f16ac89b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e500cd2f0 0x558e500d4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:26.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.601+0000 7f16ac89b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e500cd2f0 0x558e500d4750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39202/0 (socket says 172.21.15.5:39202) 2024-09-12T02:58:26.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.601+0000 7f16ac89b640 1 -- 172.21.15.5:0/1841030869 learned_addr learned my addr 172.21.15.5:0/1841030869 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:58:26.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.601+0000 7f16a7fff640 1 --2- 172.21.15.5:0/1841030869 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558e500cc9d0 0x558e500d7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:26.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.602+0000 7f16a7fff640 1 --2- 172.21.15.5:0/1841030869 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558e500d7bb0 0x558e500d4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:26.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.602+0000 7f16ac89b640 1 -- 172.21.15.5:0/1841030869 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558e500cc9d0 msgr2=0x558e500d7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:26.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.602+0000 7f16ac89b640 1 --2- 172.21.15.5:0/1841030869 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558e500cc9d0 0x558e500d7400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:26.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.602+0000 7f16ac89b640 1 -- 172.21.15.5:0/1841030869 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558e500d7bb0 msgr2=0x558e500d4c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:26.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.602+0000 7f16ac89b640 1 --2- 172.21.15.5:0/1841030869 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558e500d7bb0 0x558e500d4c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:26.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.602+0000 7f16ac89b640 1 -- 172.21.15.5:0/1841030869 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e4fda42b0 con 0x558e500cd2f0 2024-09-12T02:58:26.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.602+0000 7f16ac89b640 1 --2- 172.21.15.5:0/1841030869 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e500cd2f0 0x558e500d4750 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f1698009340 tx=0x7f16980042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7ad71feb97beed8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:26.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.602+0000 7f1687fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T02:58:26.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.602+0000 7f1687fff640 1 -- 172.21.15.5:0/1841030869 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f169803f070 con 0x558e500cd2f0 2024-09-12T02:58:26.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.602+0000 7f1687fff640 1 -- 172.21.15.5:0/1841030869 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f1698003a00 con 0x558e500cd2f0 2024-09-12T02:58:26.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.602+0000 7f16b0ed8740 1 -- 172.21.15.5:0/1841030869 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1690018ab0 con 0x558e500cd2f0 2024-09-12T02:58:26.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.602+0000 7f1687fff640 1 -- 172.21.15.5:0/1841030869 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1698035070 con 0x558e500cd2f0 2024-09-12T02:58:26.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.602+0000 7f16b0ed8740 10 client.15372 fetch_fsmap learned FSMap version 26 2024-09-12T02:58:26.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.602+0000 7f16b0ed8740 1 -- 172.21.15.5:0/1841030869 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558e500e6050 con 0x558e500cd2f0 2024-09-12T02:58:26.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.602+0000 7f1687fff640 1 -- 172.21.15.5:0/1841030869 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1690018ab0 con 0x558e500cd2f0 2024-09-12T02:58:26.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.603+0000 7f1687fff640 1 -- 172.21.15.5:0/1841030869 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f1698044ab0 con 0x558e500cd2f0 2024-09-12T02:58:26.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.603+0000 7f16b0ed8740 10 client.15372 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T02:58:26.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.603+0000 7f16b0ed8740 10 client.15372 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-12T02:58:26.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.603+0000 7f16b0ed8740 20 client.15372 populate_metadata read hostname 'smithi005' 2024-09-12T02:58:26.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.603+0000 7f16b0ed8740 1 --2- 172.21.15.5:0/1841030869 >> [v2:172.21.15.146:6836/1673370937,v1:172.21.15.146:6837/1673370937] conn(0x558e500e8670 0x558e500e9ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:26.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.603+0000 7f16b0ed8740 4 client.15372 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:58:26.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.603+0000 7f16b0ed8740 1 -- 172.21.15.5:0/1841030869 --> [v2:172.21.15.146:6836/1673370937,v1:172.21.15.146:6837/1673370937] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558e50040600 con 0x558e500e8670 2024-09-12T02:58:26.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.603+0000 7f16a7fff640 1 --2- 172.21.15.5:0/1841030869 >> [v2:172.21.15.146:6836/1673370937,v1:172.21.15.146:6837/1673370937] conn(0x558e500e8670 0x558e500e9ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:26.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.604+0000 7f16a7fff640 1 --2- 172.21.15.5:0/1841030869 >> [v2:172.21.15.146:6836/1673370937,v1:172.21.15.146:6837/1673370937] conn(0x558e500e8670 0x558e500e9ae0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=435685b2bd6e48c4 server_cookie=ee1ffcc1b399dad3 in_seq=0 out_seq=0 2024-09-12T02:58:26.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.604+0000 7f1687fff640 10 client.15372 ms_handle_connect on v2:172.21.15.146:6836/1673370937 2024-09-12T02:58:26.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.604+0000 7f1687fff640 1 -- 172.21.15.5:0/1841030869 <== mds.4 v2:172.21.15.146:6836/1673370937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558e50040600 con 0x558e500e8670 2024-09-12T02:58:26.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.604+0000 7f1687fff640 10 client.15372 handle_command_reply: tid=0 2024-09-12T02:58:26.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.604+0000 7f16b0ed8740 1 client.15372 shutdown 2024-09-12T02:58:26.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.604+0000 7f16b0ed8740 1 -- 172.21.15.5:0/1841030869 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e500cd2f0 msgr2=0x558e500d4750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:26.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.604+0000 7f16b0ed8740 1 --2- 172.21.15.5:0/1841030869 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e500cd2f0 0x558e500d4750 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f1698009340 tx=0x7f16980042f0 comp rx=0 tx=0).stop 2024-09-12T02:58:26.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.604+0000 7f16b0ed8740 1 -- 172.21.15.5:0/1841030869 shutdown_connections 2024-09-12T02:58:26.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.604+0000 7f16b0ed8740 1 --2- 172.21.15.5:0/1841030869 >> [v2:172.21.15.146:6836/1673370937,v1:172.21.15.146:6837/1673370937] conn(0x558e500e8670 0x558e500e9ae0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:26.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.604+0000 7f16b0ed8740 1 --2- 172.21.15.5:0/1841030869 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e500cd2f0 0x558e500d4750 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:26.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.604+0000 7f16b0ed8740 1 --2- 172.21.15.5:0/1841030869 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558e500d7bb0 0x558e500d4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:26.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.604+0000 7f16b0ed8740 1 --2- 172.21.15.5:0/1841030869 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558e500cc9d0 0x558e500d7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:26.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.604+0000 7f1687fff640 0 client.15372 ms_handle_reset on v2:172.21.15.146:6836/1673370937 2024-09-12T02:58:26.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.604+0000 7f16b0ed8740 1 -- 172.21.15.5:0/1841030869 >> 172.21.15.5:0/1841030869 conn(0x558e500c8f60 msgr2=0x558e500cec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:26.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.605+0000 7f16b0ed8740 1 -- 172.21.15.5:0/1841030869 shutdown_connections 2024-09-12T02:58:26.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.605+0000 7f16b0ed8740 1 -- 172.21.15.5:0/1841030869 wait complete. 2024-09-12T02:58:26.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.605+0000 7f16b0ed8740 20 client.15372 trim_cache size 0 max 16384 2024-09-12T02:58:26.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.622+0000 7f16b0ed8740 1 Processor -- start 2024-09-12T02:58:26.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.622+0000 7f16b0ed8740 1 -- start start 2024-09-12T02:58:26.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.623+0000 7f16b0ed8740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558e500cc9d0 0x558e500e9410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:26.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.623+0000 7f16b0ed8740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558e500cd2f0 0x558e500d19b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:26.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.623+0000 7f16b0ed8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e500d7bb0 0x558e500d1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:26.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.623+0000 7f16b0ed8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558e500e7e20 con 0x558e500d7bb0 2024-09-12T02:58:26.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.623+0000 7f16b0ed8740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x558e500e84f0 con 0x558e500cd2f0 2024-09-12T02:58:26.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.623+0000 7f16b0ed8740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x558e500e8bc0 con 0x558e500cc9d0 2024-09-12T02:58:26.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.623+0000 7f16a7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e500d7bb0 0x558e500d1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:26.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.623+0000 7f16a7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e500d7bb0 0x558e500d1ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39204/0 (socket says 172.21.15.5:39204) 2024-09-12T02:58:26.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.623+0000 7f16a7fff640 1 -- 172.21.15.5:0/3452438983 learned_addr learned my addr 172.21.15.5:0/3452438983 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:58:26.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.623+0000 7f16a7fff640 1 --2- 172.21.15.5:0/3452438983 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558e500cc9d0 0x558e500e9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:26.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.623+0000 7f16ac89b640 1 --2- 172.21.15.5:0/3452438983 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558e500cd2f0 0x558e500d19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:26.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.623+0000 7f16a7fff640 1 -- 172.21.15.5:0/3452438983 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558e500cc9d0 msgr2=0x558e500e9410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:26.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.623+0000 7f16a7fff640 1 --2- 172.21.15.5:0/3452438983 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558e500cc9d0 0x558e500e9410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:26.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.623+0000 7f16a7fff640 1 -- 172.21.15.5:0/3452438983 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558e500cd2f0 msgr2=0x558e500d19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:26.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.623+0000 7f16a7fff640 1 --2- 172.21.15.5:0/3452438983 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558e500cd2f0 0x558e500d19b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:26.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.623+0000 7f16a7fff640 1 -- 172.21.15.5:0/3452438983 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e500cd120 con 0x558e500d7bb0 2024-09-12T02:58:26.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.623+0000 7f16a7fff640 1 --2- 172.21.15.5:0/3452438983 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e500d7bb0 0x558e500d1ef0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f1690006aa0 tx=0x7f1690017590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe9891792a8f10e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:26.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.623+0000 7f169e7fc640 1 -- 172.21.15.5:0/3452438983 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16900188c0 con 0x558e500d7bb0 2024-09-12T02:58:26.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.623+0000 7f169e7fc640 1 -- 172.21.15.5:0/3452438983 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1690018a60 con 0x558e500d7bb0 2024-09-12T02:58:26.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.623+0000 7f169e7fc640 1 -- 172.21.15.5:0/3452438983 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1690029520 con 0x558e500d7bb0 2024-09-12T02:58:26.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.624+0000 7f16b0ed8740 1 -- 172.21.15.5:0/3452438983 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e500d7bb0 msgr2=0x558e500d1ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:26.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.624+0000 7f16b0ed8740 1 --2- 172.21.15.5:0/3452438983 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e500d7bb0 0x558e500d1ef0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f1690006aa0 tx=0x7f1690017590 comp rx=0 tx=0).stop 2024-09-12T02:58:26.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.624+0000 7f16b0ed8740 1 -- 172.21.15.5:0/3452438983 shutdown_connections 2024-09-12T02:58:26.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.624+0000 7f16b0ed8740 1 --2- 172.21.15.5:0/3452438983 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e500d7bb0 0x558e500d1ef0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:26.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.624+0000 7f16b0ed8740 1 --2- 172.21.15.5:0/3452438983 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558e500cd2f0 0x558e500d19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:26.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.624+0000 7f16b0ed8740 1 --2- 172.21.15.5:0/3452438983 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558e500cc9d0 0x558e500e9410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:26.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.624+0000 7f16b0ed8740 1 -- 172.21.15.5:0/3452438983 >> 172.21.15.5:0/3452438983 conn(0x558e500c8f60 msgr2=0x558e500c9340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:26.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.624+0000 7f16b0ed8740 1 -- 172.21.15.5:0/3452438983 shutdown_connections 2024-09-12T02:58:26.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.624+0000 7f16b0ed8740 1 -- 172.21.15.5:0/3452438983 wait complete. 2024-09-12T02:58:26.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.624+0000 7f16b0ed8740 1 -- start start 2024-09-12T02:58:26.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.625+0000 7f16b0ed8740 1 Processor -- start 2024-09-12T02:58:26.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.625+0000 7f16b0ed8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e500cc9d0 0x558e500f22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:26.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.625+0000 7f16b0ed8740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558e500cd2f0 0x558e500eb610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:26.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.625+0000 7f16b0ed8740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558e500d7bb0 0x558e500ebb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:26.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.625+0000 7f16b0ed8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558e500e9960 con 0x558e500cc9d0 2024-09-12T02:58:26.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.625+0000 7f16b0ed8740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x558e500e9290 con 0x558e500cd2f0 2024-09-12T02:58:26.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.625+0000 7f16b0ed8740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x558e500ea030 con 0x558e500d7bb0 2024-09-12T02:58:26.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.625+0000 7f16b0ed8740 10 client.0 _ll_register_callbacks cb 0x7f1698009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:58:26.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.625+0000 7f16a7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e500cc9d0 0x558e500f22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:26.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.625+0000 7f16a7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e500cc9d0 0x558e500f22e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39216/0 (socket says 172.21.15.5:39216) 2024-09-12T02:58:26.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.625+0000 7f16a7fff640 1 -- 172.21.15.5:0/1711006104 learned_addr learned my addr 172.21.15.5:0/1711006104 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:58:26.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.625+0000 7f16ac89b640 1 --2- 172.21.15.5:0/1711006104 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558e500cd2f0 0x558e500eb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:26.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.625+0000 7f16a7fff640 1 --2- 172.21.15.5:0/1711006104 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558e500d7bb0 0x558e500ebb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:26.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.626+0000 7f16a7fff640 1 -- 172.21.15.5:0/1711006104 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558e500d7bb0 msgr2=0x558e500ebb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:26.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.626+0000 7f16a7fff640 1 --2- 172.21.15.5:0/1711006104 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558e500d7bb0 0x558e500ebb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:26.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.626+0000 7f16a7fff640 1 -- 172.21.15.5:0/1711006104 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558e500cd2f0 msgr2=0x558e500eb610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:26.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.626+0000 7f16a7fff640 1 --2- 172.21.15.5:0/1711006104 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558e500cd2f0 0x558e500eb610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:26.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.626+0000 7f16a7fff640 1 -- 172.21.15.5:0/1711006104 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e500ec190 con 0x558e500cc9d0 2024-09-12T02:58:26.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.626+0000 7f16a7fff640 1 --2- 172.21.15.5:0/1711006104 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e500cc9d0 0x558e500f22e0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f1690006a70 tx=0x7f1690017f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75eee595568b9eee server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:26.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.626+0000 7f1687fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T02:58:26.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.626+0000 7f1687fff640 1 -- 172.21.15.5:0/1711006104 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1690017800 con 0x558e500cc9d0 2024-09-12T02:58:26.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.626+0000 7f1687fff640 1 -- 172.21.15.5:0/1711006104 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f16900179a0 con 0x558e500cc9d0 2024-09-12T02:58:26.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.626+0000 7f16b0ed8740 1 -- 172.21.15.5:0/1711006104 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1690018a60 con 0x558e500cc9d0 2024-09-12T02:58:26.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.626+0000 7f1687fff640 1 -- 172.21.15.5:0/1711006104 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1690017ba0 con 0x558e500cc9d0 2024-09-12T02:58:26.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.626+0000 7f16b0ed8740 10 client.15378 fetch_fsmap learned FSMap version 26 2024-09-12T02:58:26.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.626+0000 7f16b0ed8740 1 -- 172.21.15.5:0/1711006104 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558e50152d80 con 0x558e500cc9d0 2024-09-12T02:58:26.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.626+0000 7f1687fff640 1 -- 172.21.15.5:0/1711006104 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1690018a60 con 0x558e500cc9d0 2024-09-12T02:58:26.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.626+0000 7f1687fff640 1 -- 172.21.15.5:0/1711006104 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f1690012050 con 0x558e500cc9d0 2024-09-12T02:58:26.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.627+0000 7f16b0ed8740 10 client.15378 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T02:58:26.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.627+0000 7f16b0ed8740 10 client.15378 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-12T02:58:26.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.627+0000 7f16b0ed8740 20 client.15378 populate_metadata read hostname 'smithi005' 2024-09-12T02:58:26.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.627+0000 7f16b0ed8740 1 --2- 172.21.15.5:0/1711006104 >> [v2:172.21.15.146:6836/1673370937,v1:172.21.15.146:6837/1673370937] conn(0x558e50156f10 0x558e50155c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:26.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.627+0000 7f16b0ed8740 4 client.15378 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T02:58:26.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.627+0000 7f16b0ed8740 1 -- 172.21.15.5:0/1711006104 --> [v2:172.21.15.146:6836/1673370937,v1:172.21.15.146:6837/1673370937] -- command(tid 0: {"prefix": "status"}) -- 0x558e5003ab10 con 0x558e50156f10 2024-09-12T02:58:26.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.627+0000 7f16ac89b640 1 --2- 172.21.15.5:0/1711006104 >> [v2:172.21.15.146:6836/1673370937,v1:172.21.15.146:6837/1673370937] conn(0x558e50156f10 0x558e50155c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:26.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.627+0000 7f1687fff640 10 client.15378 ms_handle_connect on v2:172.21.15.146:6836/1673370937 2024-09-12T02:58:26.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.627+0000 7f16ac89b640 1 --2- 172.21.15.5:0/1711006104 >> [v2:172.21.15.146:6836/1673370937,v1:172.21.15.146:6837/1673370937] conn(0x558e50156f10 0x558e50155c20 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e9804472221de09a server_cookie=1d21a98ea57f8fdf in_seq=0 out_seq=0 2024-09-12T02:58:26.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.628+0000 7f1687fff640 1 -- 172.21.15.5:0/1711006104 <== mds.4 v2:172.21.15.146:6836/1673370937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x558e5003ab10 con 0x558e50156f10 2024-09-12T02:58:26.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.628+0000 7f1687fff640 10 client.15378 handle_command_reply: tid=0 2024-09-12T02:58:26.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.628+0000 7f16b0ed8740 1 client.15378 shutdown 2024-09-12T02:58:26.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.628+0000 7f16b0ed8740 1 -- 172.21.15.5:0/1711006104 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e500cc9d0 msgr2=0x558e500f22e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:26.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.628+0000 7f16b0ed8740 1 --2- 172.21.15.5:0/1711006104 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e500cc9d0 0x558e500f22e0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f1690006a70 tx=0x7f1690017f90 comp rx=0 tx=0).stop 2024-09-12T02:58:26.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.628+0000 7f16b0ed8740 1 -- 172.21.15.5:0/1711006104 shutdown_connections 2024-09-12T02:58:26.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.628+0000 7f16b0ed8740 1 --2- 172.21.15.5:0/1711006104 >> [v2:172.21.15.146:6836/1673370937,v1:172.21.15.146:6837/1673370937] conn(0x558e50156f10 0x558e50155c20 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:26.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.628+0000 7f16b0ed8740 1 --2- 172.21.15.5:0/1711006104 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558e500d7bb0 0x558e500ebb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:26.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.628+0000 7f16b0ed8740 1 --2- 172.21.15.5:0/1711006104 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558e500cd2f0 0x558e500eb610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:26.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.628+0000 7f1687fff640 0 client.15378 ms_handle_reset on v2:172.21.15.146:6836/1673370937 2024-09-12T02:58:26.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.628+0000 7f16b0ed8740 1 --2- 172.21.15.5:0/1711006104 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558e500cc9d0 0x558e500f22e0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:26.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.628+0000 7f16b0ed8740 1 -- 172.21.15.5:0/1711006104 >> 172.21.15.5:0/1711006104 conn(0x558e500c8f60 msgr2=0x558e500c77e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:26.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.628+0000 7f16b0ed8740 1 -- 172.21.15.5:0/1711006104 shutdown_connections 2024-09-12T02:58:26.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.628+0000 7f16b0ed8740 1 -- 172.21.15.5:0/1711006104 wait complete. 2024-09-12T02:58:26.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.628+0000 7f16b0ed8740 20 client.15378 trim_cache size 0 max 16384 2024-09-12T02:58:26.633 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-12T02:58:26.634 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "618305f8-70b1-11ef-bcea-c7b262605968", 2024-09-12T02:58:26.634 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 4, 2024-09-12T02:58:26.634 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24458, 2024-09-12T02:58:26.634 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-09-12T02:58:26.634 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-09-12T02:58:26.634 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-09-12T02:58:26.634 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 119.608137132, 2024-09-12T02:58:26.634 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 26, 2024-09-12T02:58:26.634 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 83, 2024-09-12T02:58:26.634 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 83, 2024-09-12T02:58:26.634 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 151.47005701500001 2024-09-12T02:58:26.634 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-12T02:58:26.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.629+0000 7f16aeaff640 1 -- 172.21.15.5:0/2768003097 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f16940a2c10 msgr2=0x7f16940a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:26.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.629+0000 7f16aeaff640 1 --2- 172.21.15.5:0/2768003097 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f16940a2c10 0x7f16940a50d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f16a0098a10 tx=0x7f16a00185b0 comp rx=0 tx=0).stop 2024-09-12T02:58:26.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.629+0000 7f16aeaff640 1 -- 172.21.15.5:0/2768003097 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16a8106a70 msgr2=0x7f16a810ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:26.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.629+0000 7f16aeaff640 1 --2- 172.21.15.5:0/2768003097 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16a8106a70 0x7f16a810ee50 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f16a000cce0 tx=0x7f16a000a620 comp rx=0 tx=0).stop 2024-09-12T02:58:26.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.629+0000 7f16aeaff640 1 -- 172.21.15.5:0/2768003097 shutdown_connections 2024-09-12T02:58:26.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.629+0000 7f16aeaff640 1 --2- 172.21.15.5:0/2768003097 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f16940a2c10 0x7f16940a50d0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:26.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.629+0000 7f16aeaff640 1 --2- 172.21.15.5:0/2768003097 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16a8106a70 0x7f16a810ee50 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:26.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.629+0000 7f16aeaff640 1 --2- 172.21.15.5:0/2768003097 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f16a8101410 0x7f16a810e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:26.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.629+0000 7f16aeaff640 1 --2- 172.21.15.5:0/2768003097 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f16a8100a70 0x7f16a810e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:26.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.629+0000 7f16aeaff640 1 -- 172.21.15.5:0/2768003097 >> 172.21.15.5:0/2768003097 conn(0x7f16a80fc790 msgr2=0x7f16a80fd010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:26.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.629+0000 7f16aeaff640 1 -- 172.21.15.5:0/2768003097 shutdown_connections 2024-09-12T02:58:26.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:26.629+0000 7f16aeaff640 1 -- 172.21.15.5:0/2768003097 wait complete. 2024-09-12T02:58:26.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:58:26 smithi005 ceph-mon[31197]: pgmap v341: 97 pgs: 97 active+clean; 653 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 9 op/s 2024-09-12T02:58:26.915 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:58:26 smithi148 ceph-mon[35090]: pgmap v341: 97 pgs: 97 active+clean; 653 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 9 op/s 2024-09-12T02:58:26.956 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:58:26 smithi146 ceph-mon[35011]: pgmap v341: 97 pgs: 97 active+clean; 653 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 9 op/s 2024-09-12T02:58:27.251 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.b status 2024-09-12T02:58:27.511 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:58:28.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.032+0000 7f1ab7965640 1 -- 172.21.15.5:0/3894246412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ab0074eb0 msgr2=0x7f1ab0112650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:28.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.032+0000 7f1ab7965640 1 --2- 172.21.15.5:0/3894246412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ab0074eb0 0x7f1ab0112650 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f1aa0009900 tx=0x7f1aa002a480 comp rx=0 tx=0).stop 2024-09-12T02:58:28.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.032+0000 7f1ab7965640 1 -- 172.21.15.5:0/3894246412 shutdown_connections 2024-09-12T02:58:28.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.032+0000 7f1ab7965640 1 --2- 172.21.15.5:0/3894246412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ab0074eb0 0x7f1ab0112650 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:28.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.032+0000 7f1ab7965640 1 --2- 172.21.15.5:0/3894246412 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1ab0112b90 0x7f1ab0114f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:28.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.032+0000 7f1ab7965640 1 --2- 172.21.15.5:0/3894246412 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ab0074590 0x7f1ab0074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:28.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.032+0000 7f1ab7965640 1 -- 172.21.15.5:0/3894246412 >> 172.21.15.5:0/3894246412 conn(0x7f1ab00fd8c0 msgr2=0x7f1ab00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:28.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.032+0000 7f1ab7965640 1 -- 172.21.15.5:0/3894246412 shutdown_connections 2024-09-12T02:58:28.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.032+0000 7f1ab7965640 1 -- 172.21.15.5:0/3894246412 wait complete. 2024-09-12T02:58:28.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.033+0000 7f1ab7965640 1 Processor -- start 2024-09-12T02:58:28.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.033+0000 7f1ab7965640 1 -- start start 2024-09-12T02:58:28.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.033+0000 7f1ab7965640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ab0074590 0x7f1ab01a4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:28.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.033+0000 7f1ab7965640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1ab0074eb0 0x7f1ab01a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:28.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.033+0000 7f1ab7965640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ab0112b90 0x7f1ab019ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:28.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.033+0000 7f1ab7965640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab0118650 con 0x7f1ab0112b90 2024-09-12T02:58:28.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.033+0000 7f1ab7965640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab01184d0 con 0x7f1ab0074eb0 2024-09-12T02:58:28.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.033+0000 7f1ab7965640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab01187d0 con 0x7f1ab0074590 2024-09-12T02:58:28.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.033+0000 7f1ab5f02640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ab0112b90 0x7f1ab019ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:28.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.033+0000 7f1ab5f02640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ab0112b90 0x7f1ab019ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39220/0 (socket says 172.21.15.5:39220) 2024-09-12T02:58:28.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.033+0000 7f1ab5f02640 1 -- 172.21.15.5:0/3295213945 learned_addr learned my addr 172.21.15.5:0/3295213945 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:58:28.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.033+0000 7f1ab5701640 1 --2- 172.21.15.5:0/3295213945 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ab0074590 0x7f1ab01a4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:28.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.034+0000 7f1ab4f00640 1 --2- 172.21.15.5:0/3295213945 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1ab0074eb0 0x7f1ab01a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:28.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.034+0000 7f1ab5f02640 1 -- 172.21.15.5:0/3295213945 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ab0074590 msgr2=0x7f1ab01a4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:28.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.034+0000 7f1ab5f02640 1 --2- 172.21.15.5:0/3295213945 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ab0074590 0x7f1ab01a4890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:28.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.034+0000 7f1ab5f02640 1 -- 172.21.15.5:0/3295213945 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1ab0074eb0 msgr2=0x7f1ab01a4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:28.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.034+0000 7f1ab5f02640 1 --2- 172.21.15.5:0/3295213945 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1ab0074eb0 0x7f1ab01a4dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:28.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.034+0000 7f1ab5f02640 1 -- 172.21.15.5:0/3295213945 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ab019f180 con 0x7f1ab0112b90 2024-09-12T02:58:28.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.034+0000 7f1ab4f00640 1 --2- 172.21.15.5:0/3295213945 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1ab0074eb0 0x7f1ab01a4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:58:28.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.034+0000 7f1ab5f02640 1 --2- 172.21.15.5:0/3295213945 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ab0112b90 0x7f1ab019ea50 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f1aa800b9e0 tx=0x7f1aa800bde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:28.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.034+0000 7f1aa67fc640 1 -- 172.21.15.5:0/3295213945 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aa8019070 con 0x7f1ab0112b90 2024-09-12T02:58:28.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.034+0000 7f1ab7965640 1 -- 172.21.15.5:0/3295213945 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ab019f470 con 0x7f1ab0112b90 2024-09-12T02:58:28.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.034+0000 7f1aa67fc640 1 -- 172.21.15.5:0/3295213945 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1aa8014440 con 0x7f1ab0112b90 2024-09-12T02:58:28.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.034+0000 7f1aa67fc640 1 -- 172.21.15.5:0/3295213945 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aa80126b0 con 0x7f1ab0112b90 2024-09-12T02:58:28.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.034+0000 7f1ab7965640 1 -- 172.21.15.5:0/3295213945 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ab01ab760 con 0x7f1ab0112b90 2024-09-12T02:58:28.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.036+0000 7f1aa67fc640 1 -- 172.21.15.5:0/3295213945 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7f1aa801c070 con 0x7f1ab0112b90 2024-09-12T02:58:28.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.037+0000 7f1ab9d3e740 1 Processor -- start 2024-09-12T02:58:28.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.037+0000 7f1ab9d3e740 1 -- start start 2024-09-12T02:58:28.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.037+0000 7f1ab9d3e740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b1829bb9d0 0x55b1829bbdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:28.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.037+0000 7f1ab9d3e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b1829bc2f0 0x55b1829c6670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:28.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.037+0000 7f1ab9d3e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1829c6bb0 0x55b1829c8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:28.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.037+0000 7f1ab9d3e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b182922090 con 0x55b1829c6bb0 2024-09-12T02:58:28.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.037+0000 7f1ab9d3e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55b1828c9a70 con 0x55b1829bc2f0 2024-09-12T02:58:28.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.037+0000 7f1ab9d3e740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55b18291e050 con 0x55b1829bb9d0 2024-09-12T02:58:28.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.037+0000 7f1ab4f00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1829c6bb0 0x55b1829c8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:28.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.037+0000 7f1ab4f00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1829c6bb0 0x55b1829c8fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39234/0 (socket says 172.21.15.5:39234) 2024-09-12T02:58:28.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.037+0000 7f1ab4f00640 1 -- 172.21.15.5:0/251484406 learned_addr learned my addr 172.21.15.5:0/251484406 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:58:28.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.037+0000 7f1ab5701640 1 --2- 172.21.15.5:0/251484406 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b1829bc2f0 0x55b1829c6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:28.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.037+0000 7f1aa67fc640 1 --2- 172.21.15.5:0/3295213945 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f1a9c0a2c10 0x7f1a9c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:28.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.037+0000 7f1ab4f00640 1 --2- 172.21.15.5:0/251484406 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b1829bb9d0 0x55b1829bbdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:28.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.038+0000 7f1ab5f02640 1 --2- 172.21.15.5:0/3295213945 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f1a9c0a2c10 0x7f1a9c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:28.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.038+0000 7f1ab4f00640 1 -- 172.21.15.5:0/251484406 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b1829bb9d0 msgr2=0x55b1829bbdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:28.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.038+0000 7f1ab4f00640 1 --2- 172.21.15.5:0/251484406 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b1829bb9d0 0x55b1829bbdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:28.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.038+0000 7f1ab4f00640 1 -- 172.21.15.5:0/251484406 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b1829bc2f0 msgr2=0x55b1829c6670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:28.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.038+0000 7f1ab4f00640 1 --2- 172.21.15.5:0/251484406 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b1829bc2f0 0x55b1829c6670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:28.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.038+0000 7f1ab4f00640 1 -- 172.21.15.5:0/251484406 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1828d6130 con 0x55b1829c6bb0 2024-09-12T02:58:28.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.038+0000 7f1aa67fc640 1 -- 172.21.15.5:0/3295213945 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f1aa80ce780 con 0x7f1ab0112b90 2024-09-12T02:58:28.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.038+0000 7f1ab4f00640 1 --2- 172.21.15.5:0/251484406 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1829c6bb0 0x55b1829c8fa0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f1aa00096c0 tx=0x7f1aa0031480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e2df1537bc63fb7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:28.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.038+0000 7f1ab5f02640 1 --2- 172.21.15.5:0/3295213945 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f1a9c0a2c10 0x7f1a9c0a50d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f1aa8095a40 tx=0x7f1aa8096ba0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:28.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.038+0000 7f1a9b7fe640 1 -- 172.21.15.5:0/251484406 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aa0035040 con 0x55b1829c6bb0 2024-09-12T02:58:28.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.038+0000 7f1a9b7fe640 1 -- 172.21.15.5:0/251484406 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1aa0031800 con 0x55b1829c6bb0 2024-09-12T02:58:28.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.038+0000 7f1a9b7fe640 1 -- 172.21.15.5:0/251484406 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aa00047a0 con 0x55b1829c6bb0 2024-09-12T02:58:28.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.039+0000 7f1ab9d3e740 1 -- 172.21.15.5:0/251484406 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1829c6bb0 msgr2=0x55b1829c8fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:28.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.039+0000 7f1ab9d3e740 1 --2- 172.21.15.5:0/251484406 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1829c6bb0 0x55b1829c8fa0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f1aa00096c0 tx=0x7f1aa0031480 comp rx=0 tx=0).stop 2024-09-12T02:58:28.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.039+0000 7f1ab9d3e740 1 -- 172.21.15.5:0/251484406 shutdown_connections 2024-09-12T02:58:28.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.039+0000 7f1ab9d3e740 1 --2- 172.21.15.5:0/251484406 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1829c6bb0 0x55b1829c8fa0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:28.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.039+0000 7f1ab9d3e740 1 --2- 172.21.15.5:0/251484406 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b1829bc2f0 0x55b1829c6670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:28.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.039+0000 7f1ab9d3e740 1 --2- 172.21.15.5:0/251484406 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b1829bb9d0 0x55b1829bbdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:28.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.039+0000 7f1ab9d3e740 1 -- 172.21.15.5:0/251484406 >> 172.21.15.5:0/251484406 conn(0x55b1829b7f60 msgr2=0x55b1829b8340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:28.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.039+0000 7f1ab9d3e740 1 -- 172.21.15.5:0/251484406 shutdown_connections 2024-09-12T02:58:28.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.039+0000 7f1ab9d3e740 1 -- 172.21.15.5:0/251484406 wait complete. 2024-09-12T02:58:28.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.039+0000 7f1ab9d3e740 1 -- start start 2024-09-12T02:58:28.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.039+0000 7f1ab9d3e740 1 Processor -- start 2024-09-12T02:58:28.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.040+0000 7f1ab9d3e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1829bb9d0 0x55b1829c6420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:28.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.040+0000 7f1ab9d3e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b1829bc2f0 0x55b1829c3770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:28.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.040+0000 7f1ab9d3e740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b1829c6bb0 0x55b1829c3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:28.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.040+0000 7f1ab9d3e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b1829c9780 con 0x55b1829bb9d0 2024-09-12T02:58:28.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.040+0000 7f1ab9d3e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55b1828dcf20 con 0x55b1829bc2f0 2024-09-12T02:58:28.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.040+0000 7f1ab9d3e740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55b1829c9900 con 0x55b1829c6bb0 2024-09-12T02:58:28.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.040+0000 7f1ab9d3e740 10 client.0 _ll_register_callbacks cb 0x55b1828a6030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:58:28.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.040+0000 7f1ab4f00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1829bb9d0 0x55b1829c6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:28.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.040+0000 7f1ab4f00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1829bb9d0 0x55b1829c6420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39246/0 (socket says 172.21.15.5:39246) 2024-09-12T02:58:28.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.040+0000 7f1ab4f00640 1 -- 172.21.15.5:0/2386811903 learned_addr learned my addr 172.21.15.5:0/2386811903 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:58:28.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.040+0000 7f1ab5701640 1 --2- 172.21.15.5:0/2386811903 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b1829bc2f0 0x55b1829c3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:28.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.040+0000 7f1ab4f00640 1 -- 172.21.15.5:0/2386811903 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b1829c6bb0 msgr2=0x55b1829c3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:28.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.040+0000 7f1ab4f00640 1 --2- 172.21.15.5:0/2386811903 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b1829c6bb0 0x55b1829c3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:28.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.040+0000 7f1ab4f00640 1 -- 172.21.15.5:0/2386811903 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b1829bc2f0 msgr2=0x55b1829c3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:28.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.040+0000 7f1ab4f00640 1 --2- 172.21.15.5:0/2386811903 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b1829bc2f0 0x55b1829c3770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:28.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.040+0000 7f1ab4f00640 1 -- 172.21.15.5:0/2386811903 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b1826932b0 con 0x55b1829bb9d0 2024-09-12T02:58:28.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.040+0000 7f1ab4f00640 1 --2- 172.21.15.5:0/2386811903 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1829bb9d0 0x55b1829c6420 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f1aa002a9a0 tx=0x7f1aa0032e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47574bb94afcde40 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:28.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.040+0000 7f1a98ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T02:58:28.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.041+0000 7f1a98ff9640 1 -- 172.21.15.5:0/2386811903 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aa0000e60 con 0x55b1829bb9d0 2024-09-12T02:58:28.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.041+0000 7f1a98ff9640 1 -- 172.21.15.5:0/2386811903 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f1aa0004300 con 0x55b1829bb9d0 2024-09-12T02:58:28.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.041+0000 7f1ab9d3e740 1 -- 172.21.15.5:0/2386811903 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1aa0031800 con 0x55b1829bb9d0 2024-09-12T02:58:28.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.041+0000 7f1a98ff9640 1 -- 172.21.15.5:0/2386811903 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1aa0035040 con 0x55b1829bb9d0 2024-09-12T02:58:28.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.041+0000 7f1ab9d3e740 10 client.15390 fetch_fsmap learned FSMap version 26 2024-09-12T02:58:28.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.041+0000 7f1ab9d3e740 1 -- 172.21.15.5:0/2386811903 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b1829d5070 con 0x55b1829bb9d0 2024-09-12T02:58:28.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.041+0000 7f1a98ff9640 1 -- 172.21.15.5:0/2386811903 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1aa0031800 con 0x55b1829bb9d0 2024-09-12T02:58:28.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.041+0000 7f1a98ff9640 1 -- 172.21.15.5:0/2386811903 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f1aa003a040 con 0x55b1829bb9d0 2024-09-12T02:58:28.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.041+0000 7f1ab9d3e740 10 client.15390 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T02:58:28.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.041+0000 7f1ab9d3e740 10 client.15390 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-12T02:58:28.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.041+0000 7f1ab9d3e740 20 client.15390 populate_metadata read hostname 'smithi005' 2024-09-12T02:58:28.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.041+0000 7f1ab9d3e740 1 --2- 172.21.15.5:0/2386811903 >> [v2:172.21.15.146:6838/3036767228,v1:172.21.15.146:6839/3036767228] conn(0x55b1829d7210 0x55b1829d8680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:28.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.041+0000 7f1ab9d3e740 4 client.15390 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:58:28.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.041+0000 7f1ab9d3e740 1 -- 172.21.15.5:0/2386811903 --> [v2:172.21.15.146:6838/3036767228,v1:172.21.15.146:6839/3036767228] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b18292f600 con 0x55b1829d7210 2024-09-12T02:58:28.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.042+0000 7f1ab5701640 1 --2- 172.21.15.5:0/2386811903 >> [v2:172.21.15.146:6838/3036767228,v1:172.21.15.146:6839/3036767228] conn(0x55b1829d7210 0x55b1829d8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:28.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.042+0000 7f1ab5701640 1 --2- 172.21.15.5:0/2386811903 >> [v2:172.21.15.146:6838/3036767228,v1:172.21.15.146:6839/3036767228] conn(0x55b1829d7210 0x55b1829d8680 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=596b328433d7ca1f server_cookie=d89d2e410c1e1078 in_seq=0 out_seq=0 2024-09-12T02:58:28.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.042+0000 7f1a98ff9640 10 client.15390 ms_handle_connect on v2:172.21.15.146:6838/3036767228 2024-09-12T02:58:28.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.043+0000 7f1a98ff9640 1 -- 172.21.15.5:0/2386811903 <== mds.3 v2:172.21.15.146:6838/3036767228 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b18292f600 con 0x55b1829d7210 2024-09-12T02:58:28.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.043+0000 7f1a98ff9640 10 client.15390 handle_command_reply: tid=0 2024-09-12T02:58:28.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.043+0000 7f1ab9d3e740 1 client.15390 shutdown 2024-09-12T02:58:28.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.043+0000 7f1ab9d3e740 1 -- 172.21.15.5:0/2386811903 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1829bb9d0 msgr2=0x55b1829c6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:28.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.043+0000 7f1ab9d3e740 1 --2- 172.21.15.5:0/2386811903 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1829bb9d0 0x55b1829c6420 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f1aa002a9a0 tx=0x7f1aa0032e50 comp rx=0 tx=0).stop 2024-09-12T02:58:28.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.043+0000 7f1ab9d3e740 1 -- 172.21.15.5:0/2386811903 shutdown_connections 2024-09-12T02:58:28.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.043+0000 7f1ab9d3e740 1 --2- 172.21.15.5:0/2386811903 >> [v2:172.21.15.146:6838/3036767228,v1:172.21.15.146:6839/3036767228] conn(0x55b1829d7210 0x55b1829d8680 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:28.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.043+0000 7f1ab9d3e740 1 --2- 172.21.15.5:0/2386811903 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b1829c6bb0 0x55b1829c3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:28.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.043+0000 7f1ab9d3e740 1 --2- 172.21.15.5:0/2386811903 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b1829bc2f0 0x55b1829c3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:28.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.043+0000 7f1ab9d3e740 1 --2- 172.21.15.5:0/2386811903 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1829bb9d0 0x55b1829c6420 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:28.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.043+0000 7f1ab9d3e740 1 -- 172.21.15.5:0/2386811903 >> 172.21.15.5:0/2386811903 conn(0x55b1829b7f60 msgr2=0x55b1829bdc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:28.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.043+0000 7f1a98ff9640 0 client.15390 ms_handle_reset on v2:172.21.15.146:6838/3036767228 2024-09-12T02:58:28.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.043+0000 7f1ab9d3e740 1 -- 172.21.15.5:0/2386811903 shutdown_connections 2024-09-12T02:58:28.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.043+0000 7f1ab9d3e740 1 -- 172.21.15.5:0/2386811903 wait complete. 2024-09-12T02:58:28.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.043+0000 7f1ab9d3e740 20 client.15390 trim_cache size 0 max 16384 2024-09-12T02:58:28.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.061+0000 7f1ab9d3e740 1 Processor -- start 2024-09-12T02:58:28.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.061+0000 7f1ab9d3e740 1 -- start start 2024-09-12T02:58:28.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.061+0000 7f1ab9d3e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b1829bb9d0 0x55b1829d7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:28.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.061+0000 7f1ab9d3e740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b1829bc2f0 0x55b1829d7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:28.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.061+0000 7f1ab9d3e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1829c6bb0 0x55b1829d6b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:28.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.061+0000 7f1ab9d3e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b1829d69c0 con 0x55b1829c6bb0 2024-09-12T02:58:28.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.061+0000 7f1ab9d3e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55b1829d7090 con 0x55b1829bb9d0 2024-09-12T02:58:28.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.061+0000 7f1ab9d3e740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55b1829d7760 con 0x55b1829bc2f0 2024-09-12T02:58:28.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.061+0000 7f1ab4f00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1829c6bb0 0x55b1829d6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:28.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.062+0000 7f1ab4f00640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1829c6bb0 0x55b1829d6b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39262/0 (socket says 172.21.15.5:39262) 2024-09-12T02:58:28.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.062+0000 7f1ab4f00640 1 -- 172.21.15.5:0/1542447183 learned_addr learned my addr 172.21.15.5:0/1542447183 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:58:28.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.062+0000 7f1ab5701640 1 --2- 172.21.15.5:0/1542447183 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b1829bc2f0 0x55b1829d7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:28.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.062+0000 7f1ab4f00640 1 --2- 172.21.15.5:0/1542447183 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b1829bb9d0 0x55b1829d7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:28.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.062+0000 7f1ab4f00640 1 -- 172.21.15.5:0/1542447183 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b1829bc2f0 msgr2=0x55b1829d7fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:28.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.062+0000 7f1ab4f00640 1 --2- 172.21.15.5:0/1542447183 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b1829bc2f0 0x55b1829d7fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:28.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.062+0000 7f1ab4f00640 1 -- 172.21.15.5:0/1542447183 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b1829bb9d0 msgr2=0x55b1829d7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:28.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.062+0000 7f1ab4f00640 1 --2- 172.21.15.5:0/1542447183 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b1829bb9d0 0x55b1829d7210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:28.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.062+0000 7f1ab4f00640 1 -- 172.21.15.5:0/1542447183 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b1829c6790 con 0x55b1829c6bb0 2024-09-12T02:58:28.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.062+0000 7f1ab4f00640 1 --2- 172.21.15.5:0/1542447183 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1829c6bb0 0x55b1829d6b40 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f1aa0003be0 tx=0x7f1aa002a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45eb45e39ac6ec94 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:28.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.062+0000 7f1a9affd640 1 -- 172.21.15.5:0/1542447183 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aa0031780 con 0x55b1829c6bb0 2024-09-12T02:58:28.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.062+0000 7f1a9affd640 1 -- 172.21.15.5:0/1542447183 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1aa0061020 con 0x55b1829c6bb0 2024-09-12T02:58:28.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.062+0000 7f1a9affd640 1 -- 172.21.15.5:0/1542447183 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aa0003850 con 0x55b1829c6bb0 2024-09-12T02:58:28.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.063+0000 7f1ab9d3e740 1 -- 172.21.15.5:0/1542447183 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1829c6bb0 msgr2=0x55b1829d6b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:28.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.063+0000 7f1ab9d3e740 1 --2- 172.21.15.5:0/1542447183 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1829c6bb0 0x55b1829d6b40 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f1aa0003be0 tx=0x7f1aa002a480 comp rx=0 tx=0).stop 2024-09-12T02:58:28.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.063+0000 7f1ab9d3e740 1 -- 172.21.15.5:0/1542447183 shutdown_connections 2024-09-12T02:58:28.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.063+0000 7f1ab9d3e740 1 --2- 172.21.15.5:0/1542447183 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1829c6bb0 0x55b1829d6b40 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:28.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.063+0000 7f1ab9d3e740 1 --2- 172.21.15.5:0/1542447183 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b1829bc2f0 0x55b1829d7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:28.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.063+0000 7f1ab9d3e740 1 --2- 172.21.15.5:0/1542447183 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b1829bb9d0 0x55b1829d7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:28.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.063+0000 7f1ab9d3e740 1 -- 172.21.15.5:0/1542447183 >> 172.21.15.5:0/1542447183 conn(0x55b1829b7f60 msgr2=0x55b1829b8340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:28.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.063+0000 7f1ab9d3e740 1 -- 172.21.15.5:0/1542447183 shutdown_connections 2024-09-12T02:58:28.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.063+0000 7f1ab9d3e740 1 -- 172.21.15.5:0/1542447183 wait complete. 2024-09-12T02:58:28.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.063+0000 7f1ab9d3e740 1 -- start start 2024-09-12T02:58:28.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.063+0000 7f1ab9d3e740 1 Processor -- start 2024-09-12T02:58:28.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.064+0000 7f1ab9d3e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b1829bb9d0 0x55b1829db6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:28.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.064+0000 7f1ab9d3e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1829bc2f0 0x55b1829dbbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:28.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.064+0000 7f1ab9d3e740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b1829c6bb0 0x55b1829dc120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:28.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.064+0000 7f1ab9d3e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b1829d8500 con 0x55b1829bc2f0 2024-09-12T02:58:28.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.064+0000 7f1ab9d3e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55b1829d7e30 con 0x55b1829bb9d0 2024-09-12T02:58:28.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.064+0000 7f1ab9d3e740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55b1829d8bd0 con 0x55b1829c6bb0 2024-09-12T02:58:28.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.064+0000 7f1ab9d3e740 10 client.0 _ll_register_callbacks cb 0x7f1aac0093c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:58:28.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.064+0000 7f1ab5701640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1829bc2f0 0x55b1829dbbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:28.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.064+0000 7f1ab5701640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1829bc2f0 0x55b1829dbbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39272/0 (socket says 172.21.15.5:39272) 2024-09-12T02:58:28.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.064+0000 7f1ab5701640 1 -- 172.21.15.5:0/838160993 learned_addr learned my addr 172.21.15.5:0/838160993 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:58:28.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.064+0000 7f1ab4f00640 1 --2- 172.21.15.5:0/838160993 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b1829bb9d0 0x55b1829db6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:28.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.064+0000 7f1ab4f00640 1 --2- 172.21.15.5:0/838160993 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b1829c6bb0 0x55b1829dc120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:28.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.064+0000 7f1ab5701640 1 -- 172.21.15.5:0/838160993 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b1829c6bb0 msgr2=0x55b1829dc120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:28.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.064+0000 7f1ab5701640 1 --2- 172.21.15.5:0/838160993 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b1829c6bb0 0x55b1829dc120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:28.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.064+0000 7f1ab5701640 1 -- 172.21.15.5:0/838160993 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b1829bb9d0 msgr2=0x55b1829db6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:28.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.064+0000 7f1ab5701640 1 --2- 172.21.15.5:0/838160993 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b1829bb9d0 0x55b1829db6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:28.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.064+0000 7f1ab5701640 1 -- 172.21.15.5:0/838160993 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b182a3f940 con 0x55b1829bc2f0 2024-09-12T02:58:28.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.064+0000 7f1ab4f00640 1 --2- 172.21.15.5:0/838160993 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b1829c6bb0 0x55b1829dc120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:58:28.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.064+0000 7f1ab5701640 1 --2- 172.21.15.5:0/838160993 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1829bc2f0 0x55b1829dbbe0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f1ab019fb90 tx=0x7f1aac0092f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6be5326b793c6ea server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:28.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.064+0000 7f1a98ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T02:58:28.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.065+0000 7f1a98ff9640 1 -- 172.21.15.5:0/838160993 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aac015690 con 0x55b1829bc2f0 2024-09-12T02:58:28.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.065+0000 7f1a98ff9640 1 -- 172.21.15.5:0/838160993 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f1aac015830 con 0x55b1829bc2f0 2024-09-12T02:58:28.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.065+0000 7f1ab9d3e740 1 -- 172.21.15.5:0/838160993 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1aa0061020 con 0x55b1829bc2f0 2024-09-12T02:58:28.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.065+0000 7f1a98ff9640 1 -- 172.21.15.5:0/838160993 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1aac015a30 con 0x55b1829bc2f0 2024-09-12T02:58:28.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.065+0000 7f1ab9d3e740 10 client.15396 fetch_fsmap learned FSMap version 26 2024-09-12T02:58:28.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.065+0000 7f1ab9d3e740 1 -- 172.21.15.5:0/838160993 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b182a40cb0 con 0x55b1829bc2f0 2024-09-12T02:58:28.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.065+0000 7f1a98ff9640 1 -- 172.21.15.5:0/838160993 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1aa0061020 con 0x55b1829bc2f0 2024-09-12T02:58:28.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.065+0000 7f1a98ff9640 1 -- 172.21.15.5:0/838160993 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f1aac0253d0 con 0x55b1829bc2f0 2024-09-12T02:58:28.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.065+0000 7f1ab9d3e740 10 client.15396 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T02:58:28.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.065+0000 7f1ab9d3e740 10 client.15396 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-12T02:58:28.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.065+0000 7f1ab9d3e740 20 client.15396 populate_metadata read hostname 'smithi005' 2024-09-12T02:58:28.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.065+0000 7f1ab9d3e740 1 --2- 172.21.15.5:0/838160993 >> [v2:172.21.15.146:6838/3036767228,v1:172.21.15.146:6839/3036767228] conn(0x55b182a458b0 0x55b182a445c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:28.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.065+0000 7f1ab9d3e740 4 client.15396 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T02:58:28.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.065+0000 7f1ab9d3e740 1 -- 172.21.15.5:0/838160993 --> [v2:172.21.15.146:6838/3036767228,v1:172.21.15.146:6839/3036767228] -- command(tid 0: {"prefix": "status"}) -- 0x55b182929b10 con 0x55b182a458b0 2024-09-12T02:58:28.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.066+0000 7f1ab4f00640 1 --2- 172.21.15.5:0/838160993 >> [v2:172.21.15.146:6838/3036767228,v1:172.21.15.146:6839/3036767228] conn(0x55b182a458b0 0x55b182a445c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:28.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.066+0000 7f1ab4f00640 1 --2- 172.21.15.5:0/838160993 >> [v2:172.21.15.146:6838/3036767228,v1:172.21.15.146:6839/3036767228] conn(0x55b182a458b0 0x55b182a445c0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d347bab7c74b8a66 server_cookie=82b32cd5f7bec978 in_seq=0 out_seq=0 2024-09-12T02:58:28.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.066+0000 7f1a98ff9640 10 client.15396 ms_handle_connect on v2:172.21.15.146:6838/3036767228 2024-09-12T02:58:28.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.066+0000 7f1a98ff9640 1 -- 172.21.15.5:0/838160993 <== mds.3 v2:172.21.15.146:6838/3036767228 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55b182929b10 con 0x55b182a458b0 2024-09-12T02:58:28.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.066+0000 7f1a98ff9640 10 client.15396 handle_command_reply: tid=0 2024-09-12T02:58:28.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.066+0000 7f1ab9d3e740 1 client.15396 shutdown 2024-09-12T02:58:28.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.066+0000 7f1ab9d3e740 1 -- 172.21.15.5:0/838160993 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1829bc2f0 msgr2=0x55b1829dbbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:28.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.066+0000 7f1ab9d3e740 1 --2- 172.21.15.5:0/838160993 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1829bc2f0 0x55b1829dbbe0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f1ab019fb90 tx=0x7f1aac0092f0 comp rx=0 tx=0).stop 2024-09-12T02:58:28.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.066+0000 7f1ab9d3e740 1 -- 172.21.15.5:0/838160993 shutdown_connections 2024-09-12T02:58:28.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.066+0000 7f1ab9d3e740 1 --2- 172.21.15.5:0/838160993 >> [v2:172.21.15.146:6838/3036767228,v1:172.21.15.146:6839/3036767228] conn(0x55b182a458b0 0x55b182a445c0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:28.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.066+0000 7f1ab9d3e740 1 --2- 172.21.15.5:0/838160993 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b1829bc2f0 0x55b1829dbbe0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:28.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.066+0000 7f1ab9d3e740 1 --2- 172.21.15.5:0/838160993 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b1829c6bb0 0x55b1829dc120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:28.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.066+0000 7f1ab9d3e740 1 --2- 172.21.15.5:0/838160993 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b1829bb9d0 0x55b1829db6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:28.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.066+0000 7f1ab9d3e740 1 -- 172.21.15.5:0/838160993 >> 172.21.15.5:0/838160993 conn(0x55b1829b7f60 msgr2=0x55b1829b67e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:28.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.066+0000 7f1a98ff9640 0 client.15396 ms_handle_reset on v2:172.21.15.146:6838/3036767228 2024-09-12T02:58:28.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.067+0000 7f1ab9d3e740 1 -- 172.21.15.5:0/838160993 shutdown_connections 2024-09-12T02:58:28.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.067+0000 7f1ab9d3e740 1 -- 172.21.15.5:0/838160993 wait complete. 2024-09-12T02:58:28.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.067+0000 7f1ab9d3e740 20 client.15396 trim_cache size 0 max 16384 2024-09-12T02:58:28.073 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-12T02:58:28.073 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "618305f8-70b1-11ef-bcea-c7b262605968", 2024-09-12T02:58:28.073 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 3, 2024-09-12T02:58:28.073 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24479, 2024-09-12T02:58:28.073 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-09-12T02:58:28.073 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-09-12T02:58:28.073 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-09-12T02:58:28.073 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 122.054893561, 2024-09-12T02:58:28.073 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 26, 2024-09-12T02:58:28.073 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 83, 2024-09-12T02:58:28.073 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 83, 2024-09-12T02:58:28.073 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 141.96392745599999 2024-09-12T02:58:28.073 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-12T02:58:28.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.067+0000 7f1ab7965640 1 -- 172.21.15.5:0/3295213945 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f1a9c0a2c10 msgr2=0x7f1a9c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:28.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.067+0000 7f1ab7965640 1 --2- 172.21.15.5:0/3295213945 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f1a9c0a2c10 0x7f1a9c0a50d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f1aa8095a40 tx=0x7f1aa8096ba0 comp rx=0 tx=0).stop 2024-09-12T02:58:28.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.067+0000 7f1ab7965640 1 -- 172.21.15.5:0/3295213945 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ab0112b90 msgr2=0x7f1ab019ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:28.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.067+0000 7f1ab7965640 1 --2- 172.21.15.5:0/3295213945 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ab0112b90 0x7f1ab019ea50 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f1aa800b9e0 tx=0x7f1aa800bde0 comp rx=0 tx=0).stop 2024-09-12T02:58:28.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.067+0000 7f1ab7965640 1 -- 172.21.15.5:0/3295213945 shutdown_connections 2024-09-12T02:58:28.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.067+0000 7f1ab7965640 1 --2- 172.21.15.5:0/3295213945 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f1a9c0a2c10 0x7f1a9c0a50d0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:28.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.067+0000 7f1ab7965640 1 --2- 172.21.15.5:0/3295213945 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ab0112b90 0x7f1ab019ea50 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:28.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.067+0000 7f1ab7965640 1 --2- 172.21.15.5:0/3295213945 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1ab0074eb0 0x7f1ab01a4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:28.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.067+0000 7f1ab7965640 1 --2- 172.21.15.5:0/3295213945 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ab0074590 0x7f1ab01a4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:28.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.067+0000 7f1ab7965640 1 -- 172.21.15.5:0/3295213945 >> 172.21.15.5:0/3295213945 conn(0x7f1ab00fd8c0 msgr2=0x7f1ab00ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:28.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.067+0000 7f1ab7965640 1 -- 172.21.15.5:0/3295213945 shutdown_connections 2024-09-12T02:58:28.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:28.067+0000 7f1ab7965640 1 -- 172.21.15.5:0/3295213945 wait complete. 2024-09-12T02:58:28.724 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.f status 2024-09-12T02:58:28.983 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:58:29.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:58:28 smithi005 ceph-mon[31197]: pgmap v342: 97 pgs: 97 active+clean; 653 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-12T02:58:29.165 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:58:28 smithi148 ceph-mon[35090]: pgmap v342: 97 pgs: 97 active+clean; 653 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-12T02:58:29.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:58:28 smithi146 ceph-mon[35011]: pgmap v342: 97 pgs: 97 active+clean; 653 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-12T02:58:29.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.397+0000 7fc82330f640 1 -- 172.21.15.5:0/1443442886 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc81c076720 msgr2=0x7fc81c076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:29.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.397+0000 7fc82330f640 1 --2- 172.21.15.5:0/1443442886 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc81c076720 0x7fc81c076b00 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fc818009900 tx=0x7fc81802a410 comp rx=0 tx=0).stop 2024-09-12T02:58:29.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.397+0000 7fc82330f640 1 -- 172.21.15.5:0/1443442886 shutdown_connections 2024-09-12T02:58:29.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.397+0000 7fc82330f640 1 --2- 172.21.15.5:0/1443442886 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc81c112a30 0x7fc81c114e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:29.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.397+0000 7fc82330f640 1 --2- 172.21.15.5:0/1443442886 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc81c077040 0x7fc81c1124f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:29.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.397+0000 7fc82330f640 1 --2- 172.21.15.5:0/1443442886 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc81c076720 0x7fc81c076b00 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:29.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.398+0000 7fc82330f640 1 -- 172.21.15.5:0/1443442886 >> 172.21.15.5:0/1443442886 conn(0x7fc81c0ffa90 msgr2=0x7fc81c101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:29.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.398+0000 7fc82330f640 1 -- 172.21.15.5:0/1443442886 shutdown_connections 2024-09-12T02:58:29.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.398+0000 7fc82330f640 1 -- 172.21.15.5:0/1443442886 wait complete. 2024-09-12T02:58:29.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.398+0000 7fc82330f640 1 Processor -- start 2024-09-12T02:58:29.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.398+0000 7fc82330f640 1 -- start start 2024-09-12T02:58:29.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.399+0000 7fc82330f640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc81c076720 0x7fc81c1a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:29.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.399+0000 7fc82330f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc81c077040 0x7fc81c1a4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:29.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.399+0000 7fc82330f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc81c112a30 0x7fc81c19e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:29.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.399+0000 7fc82330f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc81c1185a0 con 0x7fc81c077040 2024-09-12T02:58:29.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.399+0000 7fc82330f640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fc81c118420 con 0x7fc81c076720 2024-09-12T02:58:29.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.399+0000 7fc82330f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc81c118720 con 0x7fc81c112a30 2024-09-12T02:58:29.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.399+0000 7fc8208aa640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc81c077040 0x7fc81c1a4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:29.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.399+0000 7fc8208aa640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc81c077040 0x7fc81c1a4c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39280/0 (socket says 172.21.15.5:39280) 2024-09-12T02:58:29.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.399+0000 7fc8208aa640 1 -- 172.21.15.5:0/3606360881 learned_addr learned my addr 172.21.15.5:0/3606360881 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:58:29.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.399+0000 7fc8210ab640 1 --2- 172.21.15.5:0/3606360881 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc81c076720 0x7fc81c1a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:29.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.399+0000 7fc8218ac640 1 --2- 172.21.15.5:0/3606360881 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc81c112a30 0x7fc81c19e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:29.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.399+0000 7fc8208aa640 1 -- 172.21.15.5:0/3606360881 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc81c112a30 msgr2=0x7fc81c19e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:29.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.399+0000 7fc8208aa640 1 --2- 172.21.15.5:0/3606360881 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc81c112a30 0x7fc81c19e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:29.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.399+0000 7fc8208aa640 1 -- 172.21.15.5:0/3606360881 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc81c076720 msgr2=0x7fc81c1a46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:29.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.399+0000 7fc8208aa640 1 --2- 172.21.15.5:0/3606360881 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc81c076720 0x7fc81c1a46f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:29.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.399+0000 7fc8208aa640 1 -- 172.21.15.5:0/3606360881 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc81c19efe0 con 0x7fc81c077040 2024-09-12T02:58:29.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.399+0000 7fc8208aa640 1 --2- 172.21.15.5:0/3606360881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc81c077040 0x7fc81c1a4c30 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fc80c00d950 tx=0x7fc80c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:29.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.400+0000 7fc8127fc640 1 -- 172.21.15.5:0/3606360881 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc80c017070 con 0x7fc81c077040 2024-09-12T02:58:29.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.400+0000 7fc82330f640 1 -- 172.21.15.5:0/3606360881 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc81c19f2d0 con 0x7fc81c077040 2024-09-12T02:58:29.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.400+0000 7fc8127fc640 1 -- 172.21.15.5:0/3606360881 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc80c009c30 con 0x7fc81c077040 2024-09-12T02:58:29.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.400+0000 7fc8127fc640 1 -- 172.21.15.5:0/3606360881 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc80c00f6b0 con 0x7fc81c077040 2024-09-12T02:58:29.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.400+0000 7fc82330f640 1 -- 172.21.15.5:0/3606360881 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc81c1ab5c0 con 0x7fc81c077040 2024-09-12T02:58:29.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.402+0000 7fc8127fc640 1 -- 172.21.15.5:0/3606360881 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7fc80c0109d0 con 0x7fc81c077040 2024-09-12T02:58:29.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.403+0000 7fc8256e8740 1 Processor -- start 2024-09-12T02:58:29.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.403+0000 7fc8256e8740 1 -- start start 2024-09-12T02:58:29.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.403+0000 7fc8127fc640 1 --2- 172.21.15.5:0/3606360881 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fc8080a2c10 0x7fc8080a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:29.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.403+0000 7fc8256e8740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5630caafa9d0 0x5630caafadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:29.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.403+0000 7fc8218ac640 1 --2- 172.21.15.5:0/3606360881 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fc8080a2c10 0x7fc8080a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:29.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.403+0000 7fc8256e8740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5630caafb2f0 0x5630cab05670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:29.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.403+0000 7fc8256e8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630cab05bb0 0x5630cab07fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:29.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.403+0000 7fc8256e8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5630caa61090 con 0x5630cab05bb0 2024-09-12T02:58:29.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.403+0000 7fc8256e8740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5630caa08a70 con 0x5630caafa9d0 2024-09-12T02:58:29.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.403+0000 7fc8256e8740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5630caa5d050 con 0x5630caafb2f0 2024-09-12T02:58:29.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.403+0000 7fc8127fc640 1 -- 172.21.15.5:0/3606360881 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fc80c0cf1f0 con 0x7fc81c077040 2024-09-12T02:58:29.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.403+0000 7fc8218ac640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630cab05bb0 0x5630cab07fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:29.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.403+0000 7fc8218ac640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630cab05bb0 0x5630cab07fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39284/0 (socket says 172.21.15.5:39284) 2024-09-12T02:58:29.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.403+0000 7fc8218ac640 1 -- 172.21.15.5:0/1692202334 learned_addr learned my addr 172.21.15.5:0/1692202334 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:58:29.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.404+0000 7fc8208aa640 1 --2- 172.21.15.5:0/1692202334 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5630caafb2f0 0x5630cab05670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:29.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.404+0000 7fc8210ab640 1 --2- 172.21.15.5:0/1692202334 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5630caafa9d0 0x5630caafadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:29.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.404+0000 7fc8218ac640 1 --2- 172.21.15.5:0/3606360881 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fc8080a2c10 0x7fc8080a50d0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc814014680 tx=0x7fc814014a80 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:29.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.404+0000 7fc8218ac640 1 -- 172.21.15.5:0/1692202334 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5630caafb2f0 msgr2=0x5630cab05670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:29.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.404+0000 7fc8218ac640 1 --2- 172.21.15.5:0/1692202334 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5630caafb2f0 0x5630cab05670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:29.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.404+0000 7fc8218ac640 1 -- 172.21.15.5:0/1692202334 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5630caafa9d0 msgr2=0x5630caafadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:29.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.404+0000 7fc8218ac640 1 --2- 172.21.15.5:0/1692202334 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5630caafa9d0 0x5630caafadb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:29.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.404+0000 7fc8218ac640 1 -- 172.21.15.5:0/1692202334 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630caa15130 con 0x5630cab05bb0 2024-09-12T02:58:29.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.404+0000 7fc8210ab640 1 --2- 172.21.15.5:0/1692202334 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5630caafa9d0 0x5630caafadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:58:29.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.404+0000 7fc8218ac640 1 --2- 172.21.15.5:0/1692202334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630cab05bb0 0x5630cab07fa0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fc814012920 tx=0x7fc81401e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b53d424f3ca6a094 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:29.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.404+0000 7fc8077fe640 1 -- 172.21.15.5:0/1692202334 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc814012ea0 con 0x5630cab05bb0 2024-09-12T02:58:29.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.404+0000 7fc8077fe640 1 -- 172.21.15.5:0/1692202334 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc814004d20 con 0x5630cab05bb0 2024-09-12T02:58:29.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.404+0000 7fc8077fe640 1 -- 172.21.15.5:0/1692202334 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc814005810 con 0x5630cab05bb0 2024-09-12T02:58:29.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.405+0000 7fc8256e8740 1 -- 172.21.15.5:0/1692202334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630cab05bb0 msgr2=0x5630cab07fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:29.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.405+0000 7fc8256e8740 1 --2- 172.21.15.5:0/1692202334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630cab05bb0 0x5630cab07fa0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fc814012920 tx=0x7fc81401e000 comp rx=0 tx=0).stop 2024-09-12T02:58:29.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.405+0000 7fc8256e8740 1 -- 172.21.15.5:0/1692202334 shutdown_connections 2024-09-12T02:58:29.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.405+0000 7fc8256e8740 1 --2- 172.21.15.5:0/1692202334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630cab05bb0 0x5630cab07fa0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:29.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.405+0000 7fc8256e8740 1 --2- 172.21.15.5:0/1692202334 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5630caafb2f0 0x5630cab05670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:29.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.405+0000 7fc8256e8740 1 --2- 172.21.15.5:0/1692202334 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5630caafa9d0 0x5630caafadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:29.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.405+0000 7fc8256e8740 1 -- 172.21.15.5:0/1692202334 >> 172.21.15.5:0/1692202334 conn(0x5630caaf6f60 msgr2=0x5630caaf7340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:29.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.405+0000 7fc8256e8740 1 -- 172.21.15.5:0/1692202334 shutdown_connections 2024-09-12T02:58:29.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.405+0000 7fc8256e8740 1 -- 172.21.15.5:0/1692202334 wait complete. 2024-09-12T02:58:29.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.405+0000 7fc8256e8740 1 -- start start 2024-09-12T02:58:29.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.405+0000 7fc8256e8740 1 Processor -- start 2024-09-12T02:58:29.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.406+0000 7fc8256e8740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5630caafa9d0 0x5630cab05410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:29.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.406+0000 7fc8256e8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630caafb2f0 0x5630cab02760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:29.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.406+0000 7fc8256e8740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5630cab05bb0 0x5630cab02ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:29.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.406+0000 7fc8256e8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5630cab08780 con 0x5630caafb2f0 2024-09-12T02:58:29.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.406+0000 7fc8256e8740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5630caa1bf20 con 0x5630cab05bb0 2024-09-12T02:58:29.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.406+0000 7fc8256e8740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5630cab08900 con 0x5630caafa9d0 2024-09-12T02:58:29.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.406+0000 7fc8256e8740 10 client.0 _ll_register_callbacks cb 0x5630ca9e5030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:58:29.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.406+0000 7fc8208aa640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630caafb2f0 0x5630cab02760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:29.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.406+0000 7fc8208aa640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630caafb2f0 0x5630cab02760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39300/0 (socket says 172.21.15.5:39300) 2024-09-12T02:58:29.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.406+0000 7fc8208aa640 1 -- 172.21.15.5:0/1738034 learned_addr learned my addr 172.21.15.5:0/1738034 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:58:29.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.406+0000 7fc8210ab640 1 --2- 172.21.15.5:0/1738034 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5630caafa9d0 0x5630cab05410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:29.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.406+0000 7fc8208aa640 1 -- 172.21.15.5:0/1738034 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5630caafa9d0 msgr2=0x5630cab05410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:29.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.406+0000 7fc8208aa640 1 --2- 172.21.15.5:0/1738034 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5630caafa9d0 0x5630cab05410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:29.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.406+0000 7fc8208aa640 1 -- 172.21.15.5:0/1738034 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5630cab05bb0 msgr2=0x5630cab02ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:29.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.406+0000 7fc8218ac640 1 --2- 172.21.15.5:0/1738034 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5630cab05bb0 0x5630cab02ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:29.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.406+0000 7fc8208aa640 1 --2- 172.21.15.5:0/1738034 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5630cab05bb0 0x5630cab02ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:29.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.406+0000 7fc8208aa640 1 -- 172.21.15.5:0/1738034 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630ca7d22b0 con 0x5630caafb2f0 2024-09-12T02:58:29.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.407+0000 7fc8208aa640 1 --2- 172.21.15.5:0/1738034 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630caafb2f0 0x5630cab02760 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fc80c000c00 tx=0x7fc80c09bfa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46feb5e9dcb8616a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:29.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.407+0000 7fc804ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T02:58:29.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.407+0000 7fc804ff9640 1 -- 172.21.15.5:0/1738034 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc80c020040 con 0x5630caafb2f0 2024-09-12T02:58:29.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.407+0000 7fc804ff9640 1 -- 172.21.15.5:0/1738034 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fc80c0860c0 con 0x5630caafb2f0 2024-09-12T02:58:29.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.407+0000 7fc8256e8740 1 -- 172.21.15.5:0/1738034 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc814004d20 con 0x5630caafb2f0 2024-09-12T02:58:29.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.407+0000 7fc804ff9640 1 -- 172.21.15.5:0/1738034 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc80c0862c0 con 0x5630caafb2f0 2024-09-12T02:58:29.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.407+0000 7fc8256e8740 10 client.15408 fetch_fsmap learned FSMap version 26 2024-09-12T02:58:29.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.407+0000 7fc8256e8740 1 -- 172.21.15.5:0/1738034 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5630cab14070 con 0x5630caafb2f0 2024-09-12T02:58:29.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.407+0000 7fc804ff9640 1 -- 172.21.15.5:0/1738034 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc814004d20 con 0x5630caafb2f0 2024-09-12T02:58:29.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.407+0000 7fc804ff9640 1 -- 172.21.15.5:0/1738034 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7fc80c0a4080 con 0x5630caafb2f0 2024-09-12T02:58:29.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.407+0000 7fc8256e8740 10 client.15408 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T02:58:29.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.407+0000 7fc8256e8740 10 client.15408 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-12T02:58:29.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.407+0000 7fc8256e8740 20 client.15408 populate_metadata read hostname 'smithi005' 2024-09-12T02:58:29.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.407+0000 7fc8256e8740 1 --2- 172.21.15.5:0/1738034 >> [v2:172.21.15.148:6834/4195086427,v1:172.21.15.148:6835/4195086427] conn(0x5630cab16620 0x5630cab17a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:29.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.407+0000 7fc8256e8740 4 client.15408 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:58:29.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.407+0000 7fc8256e8740 1 -- 172.21.15.5:0/1738034 --> [v2:172.21.15.148:6834/4195086427,v1:172.21.15.148:6835/4195086427] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5630caa6e600 con 0x5630cab16620 2024-09-12T02:58:29.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.408+0000 7fc8210ab640 1 --2- 172.21.15.5:0/1738034 >> [v2:172.21.15.148:6834/4195086427,v1:172.21.15.148:6835/4195086427] conn(0x5630cab16620 0x5630cab17a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:29.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.408+0000 7fc804ff9640 10 client.15408 ms_handle_connect on v2:172.21.15.148:6834/4195086427 2024-09-12T02:58:29.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.408+0000 7fc8210ab640 1 --2- 172.21.15.5:0/1738034 >> [v2:172.21.15.148:6834/4195086427,v1:172.21.15.148:6835/4195086427] conn(0x5630cab16620 0x5630cab17a90 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=18d70c01394e9811 server_cookie=400bcb408c6edca in_seq=0 out_seq=0 2024-09-12T02:58:29.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.409+0000 7fc804ff9640 1 -- 172.21.15.5:0/1738034 <== mds.2 v2:172.21.15.148:6834/4195086427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5630caa6e600 con 0x5630cab16620 2024-09-12T02:58:29.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.409+0000 7fc804ff9640 10 client.15408 handle_command_reply: tid=0 2024-09-12T02:58:29.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.409+0000 7fc8256e8740 1 client.15408 shutdown 2024-09-12T02:58:29.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.409+0000 7fc8256e8740 1 -- 172.21.15.5:0/1738034 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630caafb2f0 msgr2=0x5630cab02760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:29.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.409+0000 7fc8256e8740 1 --2- 172.21.15.5:0/1738034 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630caafb2f0 0x5630cab02760 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fc80c000c00 tx=0x7fc80c09bfa0 comp rx=0 tx=0).stop 2024-09-12T02:58:29.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.409+0000 7fc8256e8740 1 -- 172.21.15.5:0/1738034 shutdown_connections 2024-09-12T02:58:29.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.409+0000 7fc8256e8740 1 --2- 172.21.15.5:0/1738034 >> [v2:172.21.15.148:6834/4195086427,v1:172.21.15.148:6835/4195086427] conn(0x5630cab16620 0x5630cab17a90 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:29.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.409+0000 7fc8256e8740 1 --2- 172.21.15.5:0/1738034 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630caafb2f0 0x5630cab02760 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:29.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.409+0000 7fc8256e8740 1 --2- 172.21.15.5:0/1738034 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5630cab05bb0 0x5630cab02ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:29.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.409+0000 7fc8256e8740 1 --2- 172.21.15.5:0/1738034 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5630caafa9d0 0x5630cab05410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:29.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.409+0000 7fc8256e8740 1 -- 172.21.15.5:0/1738034 >> 172.21.15.5:0/1738034 conn(0x5630caaf6f60 msgr2=0x5630caafcc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:29.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.409+0000 7fc804ff9640 0 client.15408 ms_handle_reset on v2:172.21.15.148:6834/4195086427 2024-09-12T02:58:29.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.409+0000 7fc8256e8740 1 -- 172.21.15.5:0/1738034 shutdown_connections 2024-09-12T02:58:29.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.409+0000 7fc8256e8740 1 -- 172.21.15.5:0/1738034 wait complete. 2024-09-12T02:58:29.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.409+0000 7fc8256e8740 20 client.15408 trim_cache size 0 max 16384 2024-09-12T02:58:29.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.427+0000 7fc8256e8740 1 Processor -- start 2024-09-12T02:58:29.428 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.427+0000 7fc8256e8740 1 -- start start 2024-09-12T02:58:29.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.427+0000 7fc8256e8740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5630caafa9d0 0x5630cab15f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:29.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.427+0000 7fc8256e8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630caafb2f0 0x5630cab173c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:29.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.427+0000 7fc8256e8740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5630cab05bb0 0x5630cab02720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:29.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.427+0000 7fc8256e8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5630cab15dd0 con 0x5630caafb2f0 2024-09-12T02:58:29.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.427+0000 7fc8256e8740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5630cab164a0 con 0x5630caafa9d0 2024-09-12T02:58:29.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.427+0000 7fc8256e8740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5630cab16b70 con 0x5630cab05bb0 2024-09-12T02:58:29.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.428+0000 7fc8208aa640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630caafb2f0 0x5630cab173c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:29.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.428+0000 7fc8208aa640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630caafb2f0 0x5630cab173c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39312/0 (socket says 172.21.15.5:39312) 2024-09-12T02:58:29.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.428+0000 7fc8208aa640 1 -- 172.21.15.5:0/2543700181 learned_addr learned my addr 172.21.15.5:0/2543700181 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:58:29.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.428+0000 7fc8210ab640 1 --2- 172.21.15.5:0/2543700181 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5630caafa9d0 0x5630cab15f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:29.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.428+0000 7fc8218ac640 1 --2- 172.21.15.5:0/2543700181 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5630cab05bb0 0x5630cab02720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:29.429 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.428+0000 7fc8208aa640 1 -- 172.21.15.5:0/2543700181 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5630cab05bb0 msgr2=0x5630cab02720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:29.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.428+0000 7fc8208aa640 1 --2- 172.21.15.5:0/2543700181 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5630cab05bb0 0x5630cab02720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:29.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.428+0000 7fc8208aa640 1 -- 172.21.15.5:0/2543700181 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5630caafa9d0 msgr2=0x5630cab15f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:29.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.428+0000 7fc8208aa640 1 --2- 172.21.15.5:0/2543700181 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5630caafa9d0 0x5630cab15f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:29.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.428+0000 7fc8208aa640 1 -- 172.21.15.5:0/2543700181 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630caaf80f0 con 0x5630caafb2f0 2024-09-12T02:58:29.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.428+0000 7fc8208aa640 1 --2- 172.21.15.5:0/2543700181 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630caafb2f0 0x5630cab173c0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fc80c086460 tx=0x7fc80c097c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2f930093fa08a9d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:29.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.428+0000 7fc806ffd640 1 -- 172.21.15.5:0/2543700181 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc80c020070 con 0x5630caafb2f0 2024-09-12T02:58:29.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.428+0000 7fc806ffd640 1 -- 172.21.15.5:0/2543700181 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc80c0ba2a0 con 0x5630caafb2f0 2024-09-12T02:58:29.430 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.428+0000 7fc806ffd640 1 -- 172.21.15.5:0/2543700181 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc80c09ad40 con 0x5630caafb2f0 2024-09-12T02:58:29.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.429+0000 7fc8256e8740 1 -- 172.21.15.5:0/2543700181 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630caafb2f0 msgr2=0x5630cab173c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:29.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.429+0000 7fc8256e8740 1 --2- 172.21.15.5:0/2543700181 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630caafb2f0 0x5630cab173c0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fc80c086460 tx=0x7fc80c097c70 comp rx=0 tx=0).stop 2024-09-12T02:58:29.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.429+0000 7fc8256e8740 1 -- 172.21.15.5:0/2543700181 shutdown_connections 2024-09-12T02:58:29.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.429+0000 7fc8256e8740 1 --2- 172.21.15.5:0/2543700181 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630caafb2f0 0x5630cab173c0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:29.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.429+0000 7fc8256e8740 1 --2- 172.21.15.5:0/2543700181 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5630cab05bb0 0x5630cab02720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:29.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.429+0000 7fc8256e8740 1 --2- 172.21.15.5:0/2543700181 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5630caafa9d0 0x5630cab15f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:29.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.429+0000 7fc8256e8740 1 -- 172.21.15.5:0/2543700181 >> 172.21.15.5:0/2543700181 conn(0x5630caaf6f60 msgr2=0x5630caaf1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:29.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.429+0000 7fc8256e8740 1 -- 172.21.15.5:0/2543700181 shutdown_connections 2024-09-12T02:58:29.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.429+0000 7fc8256e8740 1 -- 172.21.15.5:0/2543700181 wait complete. 2024-09-12T02:58:29.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.429+0000 7fc8256e8740 1 -- start start 2024-09-12T02:58:29.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.429+0000 7fc8256e8740 1 Processor -- start 2024-09-12T02:58:29.434 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.430+0000 7fc8256e8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630caafa9d0 0x5630caafe9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:29.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.430+0000 7fc8256e8740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5630caafb2f0 0x5630caafef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:29.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.430+0000 7fc8256e8740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5630cab05bb0 0x5630caaff450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:29.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.430+0000 7fc8256e8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5630cab17910 con 0x5630caafa9d0 2024-09-12T02:58:29.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.430+0000 7fc8256e8740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5630cab17240 con 0x5630caafb2f0 2024-09-12T02:58:29.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.430+0000 7fc8256e8740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5630cab17fe0 con 0x5630cab05bb0 2024-09-12T02:58:29.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.430+0000 7fc8256e8740 10 client.0 _ll_register_callbacks cb 0x7fc80c097b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:58:29.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.430+0000 7fc8210ab640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630caafa9d0 0x5630caafe9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:29.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.430+0000 7fc8210ab640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630caafa9d0 0x5630caafe9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39318/0 (socket says 172.21.15.5:39318) 2024-09-12T02:58:29.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.430+0000 7fc8210ab640 1 -- 172.21.15.5:0/2836522445 learned_addr learned my addr 172.21.15.5:0/2836522445 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:58:29.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.430+0000 7fc8208aa640 1 --2- 172.21.15.5:0/2836522445 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5630caafb2f0 0x5630caafef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:29.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.430+0000 7fc8218ac640 1 --2- 172.21.15.5:0/2836522445 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5630cab05bb0 0x5630caaff450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:29.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.430+0000 7fc8210ab640 1 -- 172.21.15.5:0/2836522445 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5630cab05bb0 msgr2=0x5630caaff450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:29.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.430+0000 7fc8210ab640 1 --2- 172.21.15.5:0/2836522445 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5630cab05bb0 0x5630caaff450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:29.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.430+0000 7fc8210ab640 1 -- 172.21.15.5:0/2836522445 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5630caafb2f0 msgr2=0x5630caafef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:29.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.430+0000 7fc8210ab640 1 --2- 172.21.15.5:0/2836522445 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5630caafb2f0 0x5630caafef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:29.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.430+0000 7fc8210ab640 1 -- 172.21.15.5:0/2836522445 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630cab1ed10 con 0x5630caafa9d0 2024-09-12T02:58:29.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.430+0000 7fc8208aa640 1 --2- 172.21.15.5:0/2836522445 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5630caafb2f0 0x5630caafef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:58:29.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.431+0000 7fc8210ab640 1 --2- 172.21.15.5:0/2836522445 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630caafa9d0 0x5630caafe9d0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fc818009230 tx=0x7fc818031860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f70eab7fec0b526 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:29.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.431+0000 7fc804ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T02:58:29.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.431+0000 7fc804ff9640 1 -- 172.21.15.5:0/2836522445 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc818049070 con 0x5630caafa9d0 2024-09-12T02:58:29.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.431+0000 7fc804ff9640 1 -- 172.21.15.5:0/2836522445 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fc818004450 con 0x5630caafa9d0 2024-09-12T02:58:29.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.431+0000 7fc8256e8740 1 -- 172.21.15.5:0/2836522445 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5630caa68b10 con 0x5630caafa9d0 2024-09-12T02:58:29.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.431+0000 7fc804ff9640 1 -- 172.21.15.5:0/2836522445 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc818035040 con 0x5630caafa9d0 2024-09-12T02:58:29.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.431+0000 7fc8256e8740 10 client.15414 fetch_fsmap learned FSMap version 26 2024-09-12T02:58:29.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.431+0000 7fc8256e8740 1 -- 172.21.15.5:0/2836522445 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5630cab20070 con 0x5630caafa9d0 2024-09-12T02:58:29.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.431+0000 7fc804ff9640 1 -- 172.21.15.5:0/2836522445 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5630caa68b10 con 0x5630caafa9d0 2024-09-12T02:58:29.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.431+0000 7fc804ff9640 1 -- 172.21.15.5:0/2836522445 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7fc81803c740 con 0x5630caafa9d0 2024-09-12T02:58:29.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.431+0000 7fc8256e8740 10 client.15414 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T02:58:29.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.431+0000 7fc8256e8740 10 client.15414 resolve_mds: resolved name 'f' to daemon mds.f 2024-09-12T02:58:29.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.431+0000 7fc8256e8740 20 client.15414 populate_metadata read hostname 'smithi005' 2024-09-12T02:58:29.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.431+0000 7fc8256e8740 1 --2- 172.21.15.5:0/2836522445 >> [v2:172.21.15.148:6834/4195086427,v1:172.21.15.148:6835/4195086427] conn(0x5630cab84dd0 0x5630cab83ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:29.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.431+0000 7fc8256e8740 4 client.15414 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T02:58:29.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.431+0000 7fc8256e8740 1 -- 172.21.15.5:0/2836522445 --> [v2:172.21.15.148:6834/4195086427,v1:172.21.15.148:6835/4195086427] -- command(tid 0: {"prefix": "status"}) -- 0x5630ca9e50e0 con 0x5630cab84dd0 2024-09-12T02:58:29.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.432+0000 7fc8208aa640 1 --2- 172.21.15.5:0/2836522445 >> [v2:172.21.15.148:6834/4195086427,v1:172.21.15.148:6835/4195086427] conn(0x5630cab84dd0 0x5630cab83ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:29.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.432+0000 7fc804ff9640 10 client.15414 ms_handle_connect on v2:172.21.15.148:6834/4195086427 2024-09-12T02:58:29.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.432+0000 7fc8208aa640 1 --2- 172.21.15.5:0/2836522445 >> [v2:172.21.15.148:6834/4195086427,v1:172.21.15.148:6835/4195086427] conn(0x5630cab84dd0 0x5630cab83ae0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d3d45d1a670ae2ef server_cookie=21f34d6d50f3cea1 in_seq=0 out_seq=0 2024-09-12T02:58:29.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.432+0000 7fc804ff9640 1 -- 172.21.15.5:0/2836522445 <== mds.2 v2:172.21.15.148:6834/4195086427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5630ca9e50e0 con 0x5630cab84dd0 2024-09-12T02:58:29.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.432+0000 7fc804ff9640 10 client.15414 handle_command_reply: tid=0 2024-09-12T02:58:29.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.432+0000 7fc8256e8740 1 client.15414 shutdown 2024-09-12T02:58:29.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.432+0000 7fc8256e8740 1 -- 172.21.15.5:0/2836522445 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630caafa9d0 msgr2=0x5630caafe9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:29.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.432+0000 7fc8256e8740 1 --2- 172.21.15.5:0/2836522445 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630caafa9d0 0x5630caafe9d0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fc818009230 tx=0x7fc818031860 comp rx=0 tx=0).stop 2024-09-12T02:58:29.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.432+0000 7fc8256e8740 1 -- 172.21.15.5:0/2836522445 shutdown_connections 2024-09-12T02:58:29.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.433+0000 7fc8256e8740 1 --2- 172.21.15.5:0/2836522445 >> [v2:172.21.15.148:6834/4195086427,v1:172.21.15.148:6835/4195086427] conn(0x5630cab84dd0 0x5630cab83ae0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:29.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.433+0000 7fc8256e8740 1 --2- 172.21.15.5:0/2836522445 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5630cab05bb0 0x5630caaff450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:29.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.433+0000 7fc804ff9640 0 client.15414 ms_handle_reset on v2:172.21.15.148:6834/4195086427 2024-09-12T02:58:29.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.433+0000 7fc8256e8740 1 --2- 172.21.15.5:0/2836522445 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5630caafb2f0 0x5630caafef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:29.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.433+0000 7fc8256e8740 1 --2- 172.21.15.5:0/2836522445 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630caafa9d0 0x5630caafe9d0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:29.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.433+0000 7fc8256e8740 1 -- 172.21.15.5:0/2836522445 >> 172.21.15.5:0/2836522445 conn(0x5630caaf6f60 msgr2=0x5630cab02720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:29.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.433+0000 7fc8256e8740 1 -- 172.21.15.5:0/2836522445 shutdown_connections 2024-09-12T02:58:29.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.433+0000 7fc8256e8740 1 -- 172.21.15.5:0/2836522445 wait complete. 2024-09-12T02:58:29.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.433+0000 7fc8256e8740 20 client.15414 trim_cache size 0 max 16384 2024-09-12T02:58:29.439 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-12T02:58:29.439 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "618305f8-70b1-11ef-bcea-c7b262605968", 2024-09-12T02:58:29.439 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 2, 2024-09-12T02:58:29.439 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24481, 2024-09-12T02:58:29.439 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-09-12T02:58:29.439 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-09-12T02:58:29.439 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-09-12T02:58:29.439 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 124.43255638700001, 2024-09-12T02:58:29.439 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 26, 2024-09-12T02:58:29.439 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 83, 2024-09-12T02:58:29.440 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 83, 2024-09-12T02:58:29.440 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 159.20645260800001 2024-09-12T02:58:29.440 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-12T02:58:29.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.433+0000 7fc82330f640 1 -- 172.21.15.5:0/3606360881 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fc8080a2c10 msgr2=0x7fc8080a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:29.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.433+0000 7fc82330f640 1 --2- 172.21.15.5:0/3606360881 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fc8080a2c10 0x7fc8080a50d0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc814014680 tx=0x7fc814014a80 comp rx=0 tx=0).stop 2024-09-12T02:58:29.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.433+0000 7fc82330f640 1 -- 172.21.15.5:0/3606360881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc81c077040 msgr2=0x7fc81c1a4c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:29.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.433+0000 7fc82330f640 1 --2- 172.21.15.5:0/3606360881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc81c077040 0x7fc81c1a4c30 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fc80c00d950 tx=0x7fc80c00de10 comp rx=0 tx=0).stop 2024-09-12T02:58:29.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.433+0000 7fc82330f640 1 -- 172.21.15.5:0/3606360881 shutdown_connections 2024-09-12T02:58:29.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.433+0000 7fc82330f640 1 --2- 172.21.15.5:0/3606360881 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fc8080a2c10 0x7fc8080a50d0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:29.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.433+0000 7fc82330f640 1 --2- 172.21.15.5:0/3606360881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc81c077040 0x7fc81c1a4c30 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:29.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.433+0000 7fc82330f640 1 --2- 172.21.15.5:0/3606360881 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc81c112a30 0x7fc81c19e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:29.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.434+0000 7fc82330f640 1 --2- 172.21.15.5:0/3606360881 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc81c076720 0x7fc81c1a46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:29.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.434+0000 7fc82330f640 1 -- 172.21.15.5:0/3606360881 >> 172.21.15.5:0/3606360881 conn(0x7fc81c0ffa90 msgr2=0x7fc81c101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:29.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.434+0000 7fc82330f640 1 -- 172.21.15.5:0/3606360881 shutdown_connections 2024-09-12T02:58:29.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:29.434+0000 7fc82330f640 1 -- 172.21.15.5:0/3606360881 wait complete. 2024-09-12T02:58:30.147 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.c status 2024-09-12T02:58:30.404 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:58:30.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.855+0000 7f34d48a0640 1 -- 172.21.15.5:0/2953603235 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f34d0101de0 msgr2=0x7f34d01021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:30.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.855+0000 7f34d48a0640 1 --2- 172.21.15.5:0/2953603235 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f34d0101de0 0x7f34d01021c0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f34c0009900 tx=0x7f34c002a3f0 comp rx=0 tx=0).stop 2024-09-12T02:58:30.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.855+0000 7f34d48a0640 1 -- 172.21.15.5:0/2953603235 shutdown_connections 2024-09-12T02:58:30.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.855+0000 7f34d48a0640 1 --2- 172.21.15.5:0/2953603235 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f34d0106f10 0x7f34d010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:30.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.855+0000 7f34d48a0640 1 --2- 172.21.15.5:0/2953603235 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34d0102790 0x7f34d01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:30.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.855+0000 7f34d48a0640 1 --2- 172.21.15.5:0/2953603235 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f34d0101de0 0x7f34d01021c0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:30.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.855+0000 7f34d48a0640 1 -- 172.21.15.5:0/2953603235 >> 172.21.15.5:0/2953603235 conn(0x7f34d00fd8c0 msgr2=0x7f34d00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:30.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.856+0000 7f34d48a0640 1 -- 172.21.15.5:0/2953603235 shutdown_connections 2024-09-12T02:58:30.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.856+0000 7f34d48a0640 1 -- 172.21.15.5:0/2953603235 wait complete. 2024-09-12T02:58:30.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.857+0000 7f34d48a0640 1 Processor -- start 2024-09-12T02:58:30.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.857+0000 7f34d48a0640 1 -- start start 2024-09-12T02:58:30.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.857+0000 7f34d48a0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f34d0101de0 0x7f34d00739a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:30.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.857+0000 7f34d48a0640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34d0102790 0x7f34d0073ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:30.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.857+0000 7f34d48a0640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f34d0106f10 0x7f34d006cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:30.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.857+0000 7f34d48a0640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f34d0111f90 con 0x7f34d0101de0 2024-09-12T02:58:30.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.857+0000 7f34d48a0640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f34d0111e10 con 0x7f34d0106f10 2024-09-12T02:58:30.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.857+0000 7f34d48a0640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f34d0112110 con 0x7f34d0102790 2024-09-12T02:58:30.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.857+0000 7f34ce59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f34d0101de0 0x7f34d00739a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:30.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.857+0000 7f34ce59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f34d0101de0 0x7f34d00739a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39342/0 (socket says 172.21.15.5:39342) 2024-09-12T02:58:30.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.857+0000 7f34ce59c640 1 -- 172.21.15.5:0/3665344204 learned_addr learned my addr 172.21.15.5:0/3665344204 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:58:30.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.857+0000 7f34cdd9b640 1 --2- 172.21.15.5:0/3665344204 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34d0102790 0x7f34d0073ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:30.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.857+0000 7f34ced9d640 1 --2- 172.21.15.5:0/3665344204 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f34d0106f10 0x7f34d006cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:30.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.858+0000 7f34ce59c640 1 -- 172.21.15.5:0/3665344204 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34d0102790 msgr2=0x7f34d0073ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:30.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.858+0000 7f34ce59c640 1 --2- 172.21.15.5:0/3665344204 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34d0102790 0x7f34d0073ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:30.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.858+0000 7f34ce59c640 1 -- 172.21.15.5:0/3665344204 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f34d0106f10 msgr2=0x7f34d006cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:30.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.858+0000 7f34ce59c640 1 --2- 172.21.15.5:0/3665344204 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f34d0106f10 0x7f34d006cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:30.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.858+0000 7f34ce59c640 1 -- 172.21.15.5:0/3665344204 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34d006d4d0 con 0x7f34d0101de0 2024-09-12T02:58:30.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.858+0000 7f34ced9d640 1 --2- 172.21.15.5:0/3665344204 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f34d0106f10 0x7f34d006cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:58:30.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.858+0000 7f34ce59c640 1 --2- 172.21.15.5:0/3665344204 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f34d0101de0 0x7f34d00739a0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f34c0005ab0 tx=0x7f34c00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:30.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.858+0000 7f34bf7fe640 1 -- 172.21.15.5:0/3665344204 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34c003f030 con 0x7f34d0101de0 2024-09-12T02:58:30.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.858+0000 7f34d48a0640 1 -- 172.21.15.5:0/3665344204 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34d006d760 con 0x7f34d0101de0 2024-09-12T02:58:30.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.858+0000 7f34bf7fe640 1 -- 172.21.15.5:0/3665344204 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34c0003ce0 con 0x7f34d0101de0 2024-09-12T02:58:30.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.858+0000 7f34bf7fe640 1 -- 172.21.15.5:0/3665344204 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34c0035070 con 0x7f34d0101de0 2024-09-12T02:58:30.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.858+0000 7f34d48a0640 1 -- 172.21.15.5:0/3665344204 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34d0071700 con 0x7f34d0101de0 2024-09-12T02:58:30.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.860+0000 7f34bf7fe640 1 -- 172.21.15.5:0/3665344204 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7f34c0042030 con 0x7f34d0101de0 2024-09-12T02:58:30.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.861+0000 7f34d6c79740 1 Processor -- start 2024-09-12T02:58:30.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.861+0000 7f34d6c79740 1 -- start start 2024-09-12T02:58:30.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.861+0000 7f34d6c79740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558a7f9a39d0 0x558a7f9a3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:30.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.861+0000 7f34d6c79740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558a7f9a42f0 0x558a7f9ae670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:30.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.861+0000 7f34d6c79740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a7f9aebb0 0x558a7f9b0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:30.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.861+0000 7f34d6c79740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558a7f90a090 con 0x558a7f9aebb0 2024-09-12T02:58:30.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.861+0000 7f34d6c79740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x558a7f8b1a70 con 0x558a7f9a39d0 2024-09-12T02:58:30.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.861+0000 7f34d6c79740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x558a7f906050 con 0x558a7f9a42f0 2024-09-12T02:58:30.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.861+0000 7f34cdd9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a7f9aebb0 0x558a7f9b0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:30.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.861+0000 7f34bf7fe640 1 --2- 172.21.15.5:0/3665344204 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f34b80a2c10 0x7f34b80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:30.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.861+0000 7f34cdd9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a7f9aebb0 0x558a7f9b0fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39350/0 (socket says 172.21.15.5:39350) 2024-09-12T02:58:30.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.861+0000 7f34cdd9b640 1 -- 172.21.15.5:0/1745972044 learned_addr learned my addr 172.21.15.5:0/1745972044 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:58:30.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.861+0000 7f34ce59c640 1 --2- 172.21.15.5:0/1745972044 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558a7f9a42f0 0x558a7f9ae670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:30.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.861+0000 7f34ced9d640 1 --2- 172.21.15.5:0/1745972044 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558a7f9a39d0 0x558a7f9a3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:30.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.862+0000 7f34ced9d640 1 --2- 172.21.15.5:0/3665344204 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f34b80a2c10 0x7f34b80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:30.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.862+0000 7f34bf7fe640 1 -- 172.21.15.5:0/3665344204 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f34c00ef850 con 0x7f34d0101de0 2024-09-12T02:58:30.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.862+0000 7f34cdd9b640 1 -- 172.21.15.5:0/1745972044 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558a7f9a42f0 msgr2=0x558a7f9ae670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:30.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.862+0000 7f34cdd9b640 1 --2- 172.21.15.5:0/1745972044 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558a7f9a42f0 0x558a7f9ae670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:30.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.862+0000 7f34cdd9b640 1 -- 172.21.15.5:0/1745972044 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558a7f9a39d0 msgr2=0x558a7f9a3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:30.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.862+0000 7f34cdd9b640 1 --2- 172.21.15.5:0/1745972044 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558a7f9a39d0 0x558a7f9a3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:30.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.862+0000 7f34cdd9b640 1 -- 172.21.15.5:0/1745972044 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a7f8be130 con 0x558a7f9aebb0 2024-09-12T02:58:30.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.862+0000 7f34cdd9b640 1 --2- 172.21.15.5:0/1745972044 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a7f9aebb0 0x558a7f9b0fa0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f34c400b7c0 tx=0x7f34c400bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d347328401718f5d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:30.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.862+0000 7f34ced9d640 1 --2- 172.21.15.5:0/3665344204 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f34b80a2c10 0x7f34b80a50d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f34c8005fd0 tx=0x7f34c8005f40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:30.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.862+0000 7f34bcff9640 1 -- 172.21.15.5:0/1745972044 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34c400d950 con 0x558a7f9aebb0 2024-09-12T02:58:30.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.862+0000 7f34bcff9640 1 -- 172.21.15.5:0/1745972044 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34c4018070 con 0x558a7f9aebb0 2024-09-12T02:58:30.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.862+0000 7f34bcff9640 1 -- 172.21.15.5:0/1745972044 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34c4012640 con 0x558a7f9aebb0 2024-09-12T02:58:30.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.863+0000 7f34d6c79740 1 -- 172.21.15.5:0/1745972044 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a7f9aebb0 msgr2=0x558a7f9b0fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:30.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.863+0000 7f34d6c79740 1 --2- 172.21.15.5:0/1745972044 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a7f9aebb0 0x558a7f9b0fa0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f34c400b7c0 tx=0x7f34c400bc80 comp rx=0 tx=0).stop 2024-09-12T02:58:30.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.863+0000 7f34d6c79740 1 -- 172.21.15.5:0/1745972044 shutdown_connections 2024-09-12T02:58:30.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.863+0000 7f34d6c79740 1 --2- 172.21.15.5:0/1745972044 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a7f9aebb0 0x558a7f9b0fa0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:30.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.863+0000 7f34d6c79740 1 --2- 172.21.15.5:0/1745972044 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558a7f9a42f0 0x558a7f9ae670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:30.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.863+0000 7f34d6c79740 1 --2- 172.21.15.5:0/1745972044 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558a7f9a39d0 0x558a7f9a3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:30.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.863+0000 7f34d6c79740 1 -- 172.21.15.5:0/1745972044 >> 172.21.15.5:0/1745972044 conn(0x558a7f99ff60 msgr2=0x558a7f9a0340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:30.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.863+0000 7f34d6c79740 1 -- 172.21.15.5:0/1745972044 shutdown_connections 2024-09-12T02:58:30.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.863+0000 7f34d6c79740 1 -- 172.21.15.5:0/1745972044 wait complete. 2024-09-12T02:58:30.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.864+0000 7f34d6c79740 1 -- start start 2024-09-12T02:58:30.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.864+0000 7f34d6c79740 1 Processor -- start 2024-09-12T02:58:30.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.864+0000 7f34d6c79740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558a7f9a39d0 0x558a7f9ae450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:30.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.864+0000 7f34d6c79740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558a7f9a42f0 0x558a7f9ab7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:30.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.864+0000 7f34d6c79740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a7f9aebb0 0x558a7f9abce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:30.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.864+0000 7f34d6c79740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558a7f9b1780 con 0x558a7f9aebb0 2024-09-12T02:58:30.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.864+0000 7f34d6c79740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x558a7f8c4f20 con 0x558a7f9a39d0 2024-09-12T02:58:30.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.864+0000 7f34d6c79740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x558a7f9b1900 con 0x558a7f9a42f0 2024-09-12T02:58:30.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.864+0000 7f34d6c79740 10 client.0 _ll_register_callbacks cb 0x558a7f88e030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:58:30.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.864+0000 7f34ced9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a7f9aebb0 0x558a7f9abce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:30.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.864+0000 7f34ced9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a7f9aebb0 0x558a7f9abce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39358/0 (socket says 172.21.15.5:39358) 2024-09-12T02:58:30.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.864+0000 7f34ced9d640 1 -- 172.21.15.5:0/1895061237 learned_addr learned my addr 172.21.15.5:0/1895061237 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:58:30.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.864+0000 7f34cdd9b640 1 --2- 172.21.15.5:0/1895061237 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558a7f9a42f0 0x558a7f9ab7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:30.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.864+0000 7f34ce59c640 1 --2- 172.21.15.5:0/1895061237 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558a7f9a39d0 0x558a7f9ae450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:30.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.864+0000 7f34ced9d640 1 -- 172.21.15.5:0/1895061237 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558a7f9a42f0 msgr2=0x558a7f9ab7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:30.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.864+0000 7f34ced9d640 1 --2- 172.21.15.5:0/1895061237 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558a7f9a42f0 0x558a7f9ab7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:30.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.864+0000 7f34ced9d640 1 -- 172.21.15.5:0/1895061237 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558a7f9a39d0 msgr2=0x558a7f9ae450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:30.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.864+0000 7f34ced9d640 1 --2- 172.21.15.5:0/1895061237 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558a7f9a39d0 0x558a7f9ae450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:30.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.864+0000 7f34ced9d640 1 -- 172.21.15.5:0/1895061237 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a7f67b2b0 con 0x558a7f9aebb0 2024-09-12T02:58:30.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.864+0000 7f34ce59c640 1 --2- 172.21.15.5:0/1895061237 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558a7f9a39d0 0x558a7f9ae450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:58:30.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.865+0000 7f34ced9d640 1 --2- 172.21.15.5:0/1895061237 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a7f9aebb0 0x558a7f9abce0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f34c800fb70 tx=0x7f34c8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38cb4b0372273711 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:30.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.865+0000 7f34b5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T02:58:30.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.865+0000 7f34b5ffb640 1 -- 172.21.15.5:0/1895061237 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34c801e450 con 0x558a7f9aebb0 2024-09-12T02:58:30.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.865+0000 7f34b5ffb640 1 -- 172.21.15.5:0/1895061237 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f34c801e5f0 con 0x558a7f9aebb0 2024-09-12T02:58:30.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.865+0000 7f34d6c79740 1 -- 172.21.15.5:0/1895061237 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34c4018070 con 0x558a7f9aebb0 2024-09-12T02:58:30.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.865+0000 7f34b5ffb640 1 -- 172.21.15.5:0/1895061237 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f34c801e7f0 con 0x558a7f9aebb0 2024-09-12T02:58:30.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.865+0000 7f34d6c79740 10 client.15426 fetch_fsmap learned FSMap version 26 2024-09-12T02:58:30.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.865+0000 7f34d6c79740 1 -- 172.21.15.5:0/1895061237 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558a7f9bd100 con 0x558a7f9aebb0 2024-09-12T02:58:30.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.865+0000 7f34b5ffb640 1 -- 172.21.15.5:0/1895061237 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34c4018070 con 0x558a7f9aebb0 2024-09-12T02:58:30.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.865+0000 7f34b5ffb640 1 -- 172.21.15.5:0/1895061237 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f34c801b040 con 0x558a7f9aebb0 2024-09-12T02:58:30.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.865+0000 7f34d6c79740 10 client.15426 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T02:58:30.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.866+0000 7f34d6c79740 10 client.15426 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-12T02:58:30.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.866+0000 7f34d6c79740 20 client.15426 populate_metadata read hostname 'smithi005' 2024-09-12T02:58:30.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.866+0000 7f34d6c79740 1 --2- 172.21.15.5:0/1895061237 >> [v2:172.21.15.148:6836/2606873922,v1:172.21.15.148:6837/2606873922] conn(0x558a7f9bf760 0x558a7f9c0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:30.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.866+0000 7f34d6c79740 4 client.15426 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:58:30.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.866+0000 7f34d6c79740 1 -- 172.21.15.5:0/1895061237 --> [v2:172.21.15.148:6836/2606873922,v1:172.21.15.148:6837/2606873922] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558a7f917600 con 0x558a7f9bf760 2024-09-12T02:58:30.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.866+0000 7f34ce59c640 1 --2- 172.21.15.5:0/1895061237 >> [v2:172.21.15.148:6836/2606873922,v1:172.21.15.148:6837/2606873922] conn(0x558a7f9bf760 0x558a7f9c0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:30.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.866+0000 7f34b5ffb640 10 client.15426 ms_handle_connect on v2:172.21.15.148:6836/2606873922 2024-09-12T02:58:30.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.866+0000 7f34ce59c640 1 --2- 172.21.15.5:0/1895061237 >> [v2:172.21.15.148:6836/2606873922,v1:172.21.15.148:6837/2606873922] conn(0x558a7f9bf760 0x558a7f9c0bd0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=411fb7560dfd4a42 server_cookie=6059821ba7755986 in_seq=0 out_seq=0 2024-09-12T02:58:30.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.867+0000 7f34b5ffb640 1 -- 172.21.15.5:0/1895061237 <== mds.1 v2:172.21.15.148:6836/2606873922 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558a7f917600 con 0x558a7f9bf760 2024-09-12T02:58:30.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.867+0000 7f34b5ffb640 10 client.15426 handle_command_reply: tid=0 2024-09-12T02:58:30.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.867+0000 7f34d6c79740 1 client.15426 shutdown 2024-09-12T02:58:30.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.867+0000 7f34d6c79740 1 -- 172.21.15.5:0/1895061237 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a7f9aebb0 msgr2=0x558a7f9abce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:30.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.867+0000 7f34d6c79740 1 --2- 172.21.15.5:0/1895061237 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a7f9aebb0 0x558a7f9abce0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f34c800fb70 tx=0x7f34c8004a40 comp rx=0 tx=0).stop 2024-09-12T02:58:30.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.867+0000 7f34d6c79740 1 -- 172.21.15.5:0/1895061237 shutdown_connections 2024-09-12T02:58:30.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.867+0000 7f34d6c79740 1 --2- 172.21.15.5:0/1895061237 >> [v2:172.21.15.148:6836/2606873922,v1:172.21.15.148:6837/2606873922] conn(0x558a7f9bf760 0x558a7f9c0bd0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:30.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.867+0000 7f34d6c79740 1 --2- 172.21.15.5:0/1895061237 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a7f9aebb0 0x558a7f9abce0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:30.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.867+0000 7f34b5ffb640 0 client.15426 ms_handle_reset on v2:172.21.15.148:6836/2606873922 2024-09-12T02:58:30.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.867+0000 7f34d6c79740 1 --2- 172.21.15.5:0/1895061237 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558a7f9a42f0 0x558a7f9ab7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:30.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.867+0000 7f34d6c79740 1 --2- 172.21.15.5:0/1895061237 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558a7f9a39d0 0x558a7f9ae450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:30.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.867+0000 7f34d6c79740 1 -- 172.21.15.5:0/1895061237 >> 172.21.15.5:0/1895061237 conn(0x558a7f99ff60 msgr2=0x558a7f9a5c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:30.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.867+0000 7f34d6c79740 1 -- 172.21.15.5:0/1895061237 shutdown_connections 2024-09-12T02:58:30.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.867+0000 7f34d6c79740 1 -- 172.21.15.5:0/1895061237 wait complete. 2024-09-12T02:58:30.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.868+0000 7f34d6c79740 20 client.15426 trim_cache size 0 max 16384 2024-09-12T02:58:30.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.886+0000 7f34d6c79740 1 Processor -- start 2024-09-12T02:58:30.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.886+0000 7f34d6c79740 1 -- start start 2024-09-12T02:58:30.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.886+0000 7f34d6c79740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558a7f9a39d0 0x558a7f9aa070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:30.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.886+0000 7f34d6c79740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a7f9a42f0 0x558a7f9aa5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:30.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.886+0000 7f34d6c79740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558a7f9aebb0 0x558a7f9a8a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:30.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.886+0000 7f34d6c79740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558a7f9bef10 con 0x558a7f9a42f0 2024-09-12T02:58:30.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.886+0000 7f34d6c79740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x558a7f9bf5e0 con 0x558a7f9aebb0 2024-09-12T02:58:30.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.886+0000 7f34d6c79740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x558a7f9bfcb0 con 0x558a7f9a39d0 2024-09-12T02:58:30.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.886+0000 7f34cdd9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a7f9a42f0 0x558a7f9aa5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:30.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.886+0000 7f34cdd9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a7f9a42f0 0x558a7f9aa5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39360/0 (socket says 172.21.15.5:39360) 2024-09-12T02:58:30.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.886+0000 7f34cdd9b640 1 -- 172.21.15.5:0/2734747351 learned_addr learned my addr 172.21.15.5:0/2734747351 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:58:30.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.886+0000 7f34ce59c640 1 --2- 172.21.15.5:0/2734747351 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558a7f9a39d0 0x558a7f9aa070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:30.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.886+0000 7f34ced9d640 1 --2- 172.21.15.5:0/2734747351 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558a7f9aebb0 0x558a7f9a8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:30.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.886+0000 7f34cdd9b640 1 -- 172.21.15.5:0/2734747351 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558a7f9a39d0 msgr2=0x558a7f9aa070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:30.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.886+0000 7f34cdd9b640 1 --2- 172.21.15.5:0/2734747351 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558a7f9a39d0 0x558a7f9aa070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:30.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.886+0000 7f34cdd9b640 1 -- 172.21.15.5:0/2734747351 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558a7f9aebb0 msgr2=0x558a7f9a8a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:30.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.886+0000 7f34cdd9b640 1 --2- 172.21.15.5:0/2734747351 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558a7f9aebb0 0x558a7f9a8a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:30.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.886+0000 7f34cdd9b640 1 -- 172.21.15.5:0/2734747351 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a7f9ae070 con 0x558a7f9a42f0 2024-09-12T02:58:30.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.887+0000 7f34cdd9b640 1 --2- 172.21.15.5:0/2734747351 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a7f9a42f0 0x558a7f9aa5b0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f34c400b7c0 tx=0x7f34c400dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb799df5cd000d23 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:30.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.887+0000 7f34b7fff640 1 -- 172.21.15.5:0/2734747351 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34c4029070 con 0x558a7f9a42f0 2024-09-12T02:58:30.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.887+0000 7f34b7fff640 1 -- 172.21.15.5:0/2734747351 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34c4018040 con 0x558a7f9a42f0 2024-09-12T02:58:30.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.887+0000 7f34b7fff640 1 -- 172.21.15.5:0/2734747351 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34c4024980 con 0x558a7f9a42f0 2024-09-12T02:58:30.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.887+0000 7f34d6c79740 1 -- 172.21.15.5:0/2734747351 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a7f9a42f0 msgr2=0x558a7f9aa5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:30.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.887+0000 7f34d6c79740 1 --2- 172.21.15.5:0/2734747351 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a7f9a42f0 0x558a7f9aa5b0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f34c400b7c0 tx=0x7f34c400dae0 comp rx=0 tx=0).stop 2024-09-12T02:58:30.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.888+0000 7f34d6c79740 1 -- 172.21.15.5:0/2734747351 shutdown_connections 2024-09-12T02:58:30.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.888+0000 7f34d6c79740 1 --2- 172.21.15.5:0/2734747351 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a7f9a42f0 0x558a7f9aa5b0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:30.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.888+0000 7f34d6c79740 1 --2- 172.21.15.5:0/2734747351 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558a7f9aebb0 0x558a7f9a8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:30.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.888+0000 7f34d6c79740 1 --2- 172.21.15.5:0/2734747351 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558a7f9a39d0 0x558a7f9aa070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:30.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.888+0000 7f34d6c79740 1 -- 172.21.15.5:0/2734747351 >> 172.21.15.5:0/2734747351 conn(0x558a7f99ff60 msgr2=0x558a7f9ab760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:30.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.888+0000 7f34d6c79740 1 -- 172.21.15.5:0/2734747351 shutdown_connections 2024-09-12T02:58:30.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.888+0000 7f34d6c79740 1 -- 172.21.15.5:0/2734747351 wait complete. 2024-09-12T02:58:30.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.888+0000 7f34d6c79740 1 -- start start 2024-09-12T02:58:30.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.888+0000 7f34d6c79740 1 Processor -- start 2024-09-12T02:58:30.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.888+0000 7f34d6c79740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558a7f9a39d0 0x558a7f9c43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:30.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.888+0000 7f34d6c79740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558a7f9a42f0 0x558a7f9c48f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:30.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.888+0000 7f34d6c79740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a7f9aebb0 0x558a7fa27e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:30.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.888+0000 7f34d6c79740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558a7f9c0a50 con 0x558a7f9aebb0 2024-09-12T02:58:30.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.888+0000 7f34d6c79740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x558a7f9c0380 con 0x558a7f9a39d0 2024-09-12T02:58:30.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.888+0000 7f34d6c79740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x558a7f9c1120 con 0x558a7f9a42f0 2024-09-12T02:58:30.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.888+0000 7f34d6c79740 10 client.0 _ll_register_callbacks cb 0x7f34c800ac10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:58:30.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.888+0000 7f34ced9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a7f9aebb0 0x558a7fa27e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:30.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.889+0000 7f34ced9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a7f9aebb0 0x558a7fa27e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39368/0 (socket says 172.21.15.5:39368) 2024-09-12T02:58:30.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.889+0000 7f34ced9d640 1 -- 172.21.15.5:0/2067404154 learned_addr learned my addr 172.21.15.5:0/2067404154 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:58:30.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.889+0000 7f34cdd9b640 1 --2- 172.21.15.5:0/2067404154 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558a7f9a42f0 0x558a7f9c48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:30.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.889+0000 7f34ce59c640 1 --2- 172.21.15.5:0/2067404154 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558a7f9a39d0 0x558a7f9c43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:30.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.889+0000 7f34ced9d640 1 -- 172.21.15.5:0/2067404154 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558a7f9a42f0 msgr2=0x558a7f9c48f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:30.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.889+0000 7f34ced9d640 1 --2- 172.21.15.5:0/2067404154 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558a7f9a42f0 0x558a7f9c48f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:30.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.889+0000 7f34ced9d640 1 -- 172.21.15.5:0/2067404154 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558a7f9a39d0 msgr2=0x558a7f9c43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:30.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.889+0000 7f34ced9d640 1 --2- 172.21.15.5:0/2067404154 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558a7f9a39d0 0x558a7f9c43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:30.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.889+0000 7f34ced9d640 1 -- 172.21.15.5:0/2067404154 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a7fa28450 con 0x558a7f9aebb0 2024-09-12T02:58:30.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.889+0000 7f34cdd9b640 1 --2- 172.21.15.5:0/2067404154 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558a7f9a42f0 0x558a7f9c48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:58:30.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.889+0000 7f34ce59c640 1 --2- 172.21.15.5:0/2067404154 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558a7f9a39d0 0x558a7f9c43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:58:30.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.889+0000 7f34ced9d640 1 --2- 172.21.15.5:0/2067404154 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a7f9aebb0 0x558a7fa27e60 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f34c800abe0 tx=0x7f34c800a3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed472fd004104ff server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:30.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.889+0000 7f34b5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T02:58:30.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.889+0000 7f34b5ffb640 1 -- 172.21.15.5:0/2067404154 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34c8004cb0 con 0x558a7f9aebb0 2024-09-12T02:58:30.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.889+0000 7f34b5ffb640 1 -- 172.21.15.5:0/2067404154 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f34c80283e0 con 0x558a7f9aebb0 2024-09-12T02:58:30.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.889+0000 7f34d6c79740 1 -- 172.21.15.5:0/2067404154 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34c4018040 con 0x558a7f9aebb0 2024-09-12T02:58:30.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.889+0000 7f34b5ffb640 1 -- 172.21.15.5:0/2067404154 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f34c8004e50 con 0x558a7f9aebb0 2024-09-12T02:58:30.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.890+0000 7f34d6c79740 10 client.15432 fetch_fsmap learned FSMap version 26 2024-09-12T02:58:30.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.890+0000 7f34d6c79740 1 -- 172.21.15.5:0/2067404154 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x558a7fa29730 con 0x558a7f9aebb0 2024-09-12T02:58:30.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.890+0000 7f34b5ffb640 1 -- 172.21.15.5:0/2067404154 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34c4018040 con 0x558a7f9aebb0 2024-09-12T02:58:30.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.890+0000 7f34b5ffb640 1 -- 172.21.15.5:0/2067404154 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f34c803a020 con 0x558a7f9aebb0 2024-09-12T02:58:30.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.890+0000 7f34d6c79740 10 client.15432 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T02:58:30.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.890+0000 7f34d6c79740 10 client.15432 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-12T02:58:30.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.890+0000 7f34d6c79740 20 client.15432 populate_metadata read hostname 'smithi005' 2024-09-12T02:58:30.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.890+0000 7f34d6c79740 1 --2- 172.21.15.5:0/2067404154 >> [v2:172.21.15.148:6836/2606873922,v1:172.21.15.148:6837/2606873922] conn(0x558a7f9bf760 0x558a7fa2d000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:30.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.890+0000 7f34d6c79740 4 client.15432 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T02:58:30.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.890+0000 7f34d6c79740 1 -- 172.21.15.5:0/2067404154 --> [v2:172.21.15.148:6836/2606873922,v1:172.21.15.148:6837/2606873922] -- command(tid 0: {"prefix": "status"}) -- 0x558a7f911b10 con 0x558a7f9bf760 2024-09-12T02:58:30.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.890+0000 7f34ce59c640 1 --2- 172.21.15.5:0/2067404154 >> [v2:172.21.15.148:6836/2606873922,v1:172.21.15.148:6837/2606873922] conn(0x558a7f9bf760 0x558a7fa2d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:30.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.891+0000 7f34b5ffb640 10 client.15432 ms_handle_connect on v2:172.21.15.148:6836/2606873922 2024-09-12T02:58:30.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.891+0000 7f34ce59c640 1 --2- 172.21.15.5:0/2067404154 >> [v2:172.21.15.148:6836/2606873922,v1:172.21.15.148:6837/2606873922] conn(0x558a7f9bf760 0x558a7fa2d000 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5d3a99a4ef913eec server_cookie=db8ad7120233d2e3 in_seq=0 out_seq=0 2024-09-12T02:58:30.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.891+0000 7f34b5ffb640 1 -- 172.21.15.5:0/2067404154 <== mds.1 v2:172.21.15.148:6836/2606873922 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x558a7f911b10 con 0x558a7f9bf760 2024-09-12T02:58:30.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.891+0000 7f34b5ffb640 10 client.15432 handle_command_reply: tid=0 2024-09-12T02:58:30.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.891+0000 7f34d6c79740 1 client.15432 shutdown 2024-09-12T02:58:30.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.891+0000 7f34d6c79740 1 -- 172.21.15.5:0/2067404154 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a7f9aebb0 msgr2=0x558a7fa27e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:30.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.891+0000 7f34d6c79740 1 --2- 172.21.15.5:0/2067404154 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a7f9aebb0 0x558a7fa27e60 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f34c800abe0 tx=0x7f34c800a3c0 comp rx=0 tx=0).stop 2024-09-12T02:58:30.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.891+0000 7f34d6c79740 1 -- 172.21.15.5:0/2067404154 shutdown_connections 2024-09-12T02:58:30.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.891+0000 7f34d6c79740 1 --2- 172.21.15.5:0/2067404154 >> [v2:172.21.15.148:6836/2606873922,v1:172.21.15.148:6837/2606873922] conn(0x558a7f9bf760 0x558a7fa2d000 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:30.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.891+0000 7f34d6c79740 1 --2- 172.21.15.5:0/2067404154 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a7f9aebb0 0x558a7fa27e60 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:30.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.891+0000 7f34d6c79740 1 --2- 172.21.15.5:0/2067404154 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558a7f9a42f0 0x558a7f9c48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:30.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.891+0000 7f34d6c79740 1 --2- 172.21.15.5:0/2067404154 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558a7f9a39d0 0x558a7f9c43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:30.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.891+0000 7f34d6c79740 1 -- 172.21.15.5:0/2067404154 >> 172.21.15.5:0/2067404154 conn(0x558a7f99ff60 msgr2=0x558a7f9a9c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:30.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.891+0000 7f34b5ffb640 0 client.15432 ms_handle_reset on v2:172.21.15.148:6836/2606873922 2024-09-12T02:58:30.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.891+0000 7f34d6c79740 1 -- 172.21.15.5:0/2067404154 shutdown_connections 2024-09-12T02:58:30.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.891+0000 7f34d6c79740 1 -- 172.21.15.5:0/2067404154 wait complete. 2024-09-12T02:58:30.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.891+0000 7f34d6c79740 20 client.15432 trim_cache size 0 max 16384 2024-09-12T02:58:30.898 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-12T02:58:30.898 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "618305f8-70b1-11ef-bcea-c7b262605968", 2024-09-12T02:58:30.898 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 1, 2024-09-12T02:58:30.898 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24487, 2024-09-12T02:58:30.898 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-09-12T02:58:30.898 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-09-12T02:58:30.898 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-09-12T02:58:30.898 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 126.900757754, 2024-09-12T02:58:30.899 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 26, 2024-09-12T02:58:30.899 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 83, 2024-09-12T02:58:30.899 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 83, 2024-09-12T02:58:30.899 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 158.319171877 2024-09-12T02:58:30.899 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-12T02:58:30.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.892+0000 7f34d48a0640 1 -- 172.21.15.5:0/3665344204 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f34b80a2c10 msgr2=0x7f34b80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:30.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.892+0000 7f34d48a0640 1 --2- 172.21.15.5:0/3665344204 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f34b80a2c10 0x7f34b80a50d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f34c8005fd0 tx=0x7f34c8005f40 comp rx=0 tx=0).stop 2024-09-12T02:58:30.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.892+0000 7f34d48a0640 1 -- 172.21.15.5:0/3665344204 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f34d0101de0 msgr2=0x7f34d00739a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:30.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.892+0000 7f34d48a0640 1 --2- 172.21.15.5:0/3665344204 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f34d0101de0 0x7f34d00739a0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f34c0005ab0 tx=0x7f34c00042f0 comp rx=0 tx=0).stop 2024-09-12T02:58:30.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.892+0000 7f34d48a0640 1 -- 172.21.15.5:0/3665344204 shutdown_connections 2024-09-12T02:58:30.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.892+0000 7f34d48a0640 1 --2- 172.21.15.5:0/3665344204 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f34d0106f10 0x7f34d006cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:30.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.892+0000 7f34d48a0640 1 --2- 172.21.15.5:0/3665344204 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f34d0102790 0x7f34d0073ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:30.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.892+0000 7f34d48a0640 1 --2- 172.21.15.5:0/3665344204 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f34b80a2c10 0x7f34b80a50d0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:30.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.892+0000 7f34d48a0640 1 --2- 172.21.15.5:0/3665344204 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f34d0101de0 0x7f34d00739a0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:30.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.892+0000 7f34d48a0640 1 -- 172.21.15.5:0/3665344204 >> 172.21.15.5:0/3665344204 conn(0x7f34d00fd8c0 msgr2=0x7f34d00fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:30.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.892+0000 7f34d48a0640 1 -- 172.21.15.5:0/3665344204 shutdown_connections 2024-09-12T02:58:30.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:30.892+0000 7f34d48a0640 1 -- 172.21.15.5:0/3665344204 wait complete. 2024-09-12T02:58:31.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:58:30 smithi005 ceph-mon[31197]: pgmap v343: 97 pgs: 97 active+clean; 658 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-12T02:58:31.165 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:58:30 smithi148 ceph-mon[35090]: pgmap v343: 97 pgs: 97 active+clean; 658 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-12T02:58:31.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:58:30 smithi146 ceph-mon[35011]: pgmap v343: 97 pgs: 97 active+clean; 658 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-12T02:58:31.556 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.i status 2024-09-12T02:58:31.812 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:58:32.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.314+0000 7f2a3a452640 1 -- 172.21.15.5:0/3778533129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a3410c1a0 msgr2=0x7f2a3410e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:32.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.314+0000 7f2a3a452640 1 --2- 172.21.15.5:0/3778533129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a3410c1a0 0x7f2a3410e610 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c009ca0 tx=0x7f2a2c02a950 comp rx=0 tx=0).stop 2024-09-12T02:58:32.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.314+0000 7f2a3a452640 1 -- 172.21.15.5:0/3778533129 shutdown_connections 2024-09-12T02:58:32.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.314+0000 7f2a3a452640 1 --2- 172.21.15.5:0/3778533129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a3410c1a0 0x7f2a3410e610 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:32.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.314+0000 7f2a3a452640 1 --2- 172.21.15.5:0/3778533129 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2a34103720 0x7f2a3410bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:32.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.314+0000 7f2a3a452640 1 --2- 172.21.15.5:0/3778533129 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2a34102e00 0x7f2a341031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:32.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.314+0000 7f2a3a452640 1 -- 172.21.15.5:0/3778533129 >> 172.21.15.5:0/3778533129 conn(0x7f2a340fc7a0 msgr2=0x7f2a340febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:32.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.314+0000 7f2a3a452640 1 -- 172.21.15.5:0/3778533129 shutdown_connections 2024-09-12T02:58:32.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.314+0000 7f2a3a452640 1 -- 172.21.15.5:0/3778533129 wait complete. 2024-09-12T02:58:32.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.315+0000 7f2a3a452640 1 Processor -- start 2024-09-12T02:58:32.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.315+0000 7f2a3a452640 1 -- start start 2024-09-12T02:58:32.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.315+0000 7f2a3a452640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2a34102e00 0x7f2a341a03f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:32.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.315+0000 7f2a3a452640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2a34103720 0x7f2a341a0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:32.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.315+0000 7f2a3a452640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a3410c1a0 0x7f2a3419a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:32.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.315+0000 7f2a3a452640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2a34112080 con 0x7f2a3410c1a0 2024-09-12T02:58:32.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.315+0000 7f2a3a452640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f2a34111f00 con 0x7f2a34103720 2024-09-12T02:58:32.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.315+0000 7f2a3a452640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f2a34112200 con 0x7f2a34102e00 2024-09-12T02:58:32.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.315+0000 7f2a389ef640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a3410c1a0 0x7f2a3419a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:32.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.315+0000 7f2a389ef640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a3410c1a0 0x7f2a3419a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39376/0 (socket says 172.21.15.5:39376) 2024-09-12T02:58:32.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.315+0000 7f2a389ef640 1 -- 172.21.15.5:0/601332483 learned_addr learned my addr 172.21.15.5:0/601332483 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:58:32.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.315+0000 7f2a33fff640 1 --2- 172.21.15.5:0/601332483 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2a34102e00 0x7f2a341a03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:32.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.315+0000 7f2a337fe640 1 --2- 172.21.15.5:0/601332483 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2a34103720 0x7f2a341a0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:32.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.316+0000 7f2a389ef640 1 -- 172.21.15.5:0/601332483 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2a34102e00 msgr2=0x7f2a341a03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:32.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.316+0000 7f2a389ef640 1 --2- 172.21.15.5:0/601332483 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2a34102e00 0x7f2a341a03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:32.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.316+0000 7f2a389ef640 1 -- 172.21.15.5:0/601332483 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2a34103720 msgr2=0x7f2a341a0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:32.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.316+0000 7f2a389ef640 1 --2- 172.21.15.5:0/601332483 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2a34103720 0x7f2a341a0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:32.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.316+0000 7f2a389ef640 1 -- 172.21.15.5:0/601332483 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a3419ace0 con 0x7f2a3410c1a0 2024-09-12T02:58:32.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.316+0000 7f2a33fff640 1 --2- 172.21.15.5:0/601332483 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2a34102e00 0x7f2a341a03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:58:32.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.316+0000 7f2a389ef640 1 --2- 172.21.15.5:0/601332483 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a3410c1a0 0x7f2a3419a5b0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c0098d0 tx=0x7f2a2c003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:32.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.316+0000 7f2a317fa640 1 -- 172.21.15.5:0/601332483 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a2c034040 con 0x7f2a3410c1a0 2024-09-12T02:58:32.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.316+0000 7f2a317fa640 1 -- 172.21.15.5:0/601332483 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a2c02fe20 con 0x7f2a3410c1a0 2024-09-12T02:58:32.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.316+0000 7f2a317fa640 1 -- 172.21.15.5:0/601332483 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a2c039450 con 0x7f2a3410c1a0 2024-09-12T02:58:32.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.316+0000 7f2a3a452640 1 -- 172.21.15.5:0/601332483 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a3419af70 con 0x7f2a3410c1a0 2024-09-12T02:58:32.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.316+0000 7f2a3a452640 1 -- 172.21.15.5:0/601332483 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a341a72c0 con 0x7f2a3410c1a0 2024-09-12T02:58:32.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.318+0000 7f2a317fa640 1 -- 172.21.15.5:0/601332483 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7f2a2c037020 con 0x7f2a3410c1a0 2024-09-12T02:58:32.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.319+0000 7f2a3c82b740 1 Processor -- start 2024-09-12T02:58:32.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.319+0000 7f2a3c82b740 1 -- start start 2024-09-12T02:58:32.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.319+0000 7f2a3c82b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55966fc139d0 0x55966fc13db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:32.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.319+0000 7f2a3c82b740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55966fc142f0 0x55966fc1e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:32.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.319+0000 7f2a3c82b740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55966fc1ebb0 0x55966fc20fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:32.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.319+0000 7f2a3c82b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55966fb7a090 con 0x55966fc139d0 2024-09-12T02:58:32.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.319+0000 7f2a3c82b740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55966fb21a70 con 0x55966fc142f0 2024-09-12T02:58:32.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.319+0000 7f2a3c82b740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55966fb76050 con 0x55966fc1ebb0 2024-09-12T02:58:32.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.319+0000 7f2a337fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55966fc139d0 0x55966fc13db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:32.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.319+0000 7f2a317fa640 1 --2- 172.21.15.5:0/601332483 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f2a1c0a2c10 0x7f2a1c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:32.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.319+0000 7f2a337fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55966fc139d0 0x55966fc13db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39384/0 (socket says 172.21.15.5:39384) 2024-09-12T02:58:32.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.319+0000 7f2a337fe640 1 -- 172.21.15.5:0/2126362092 learned_addr learned my addr 172.21.15.5:0/2126362092 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:58:32.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.319+0000 7f2a33fff640 1 --2- 172.21.15.5:0/2126362092 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55966fc142f0 0x55966fc1e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:32.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.319+0000 7f2a389ef640 1 --2- 172.21.15.5:0/601332483 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f2a1c0a2c10 0x7f2a1c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:32.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.319+0000 7f2a337fe640 1 --2- 172.21.15.5:0/2126362092 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55966fc1ebb0 0x55966fc20fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:32.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.319+0000 7f2a317fa640 1 -- 172.21.15.5:0/601332483 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f2a2c0f01e0 con 0x7f2a3410c1a0 2024-09-12T02:58:32.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.320+0000 7f2a337fe640 1 -- 172.21.15.5:0/2126362092 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55966fc1ebb0 msgr2=0x55966fc20fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:32.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.320+0000 7f2a337fe640 1 --2- 172.21.15.5:0/2126362092 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55966fc1ebb0 0x55966fc20fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:32.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.320+0000 7f2a337fe640 1 -- 172.21.15.5:0/2126362092 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55966fc142f0 msgr2=0x55966fc1e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:32.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.320+0000 7f2a337fe640 1 --2- 172.21.15.5:0/2126362092 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55966fc142f0 0x55966fc1e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:32.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.320+0000 7f2a337fe640 1 -- 172.21.15.5:0/2126362092 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55966fb2e130 con 0x55966fc139d0 2024-09-12T02:58:32.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.320+0000 7f2a337fe640 1 --2- 172.21.15.5:0/2126362092 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55966fc139d0 0x55966fc13db0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f2a28006e10 tx=0x7f2a2800b400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5db5818ef7c5070e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:32.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.320+0000 7f2a389ef640 1 --2- 172.21.15.5:0/601332483 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f2a1c0a2c10 0x7f2a1c0a50d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c0bc990 tx=0x7f2a2c0a7040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:32.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.320+0000 7f2a1a7fc640 1 -- 172.21.15.5:0/2126362092 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a28019680 con 0x55966fc139d0 2024-09-12T02:58:32.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.320+0000 7f2a1a7fc640 1 -- 172.21.15.5:0/2126362092 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a28010040 con 0x55966fc139d0 2024-09-12T02:58:32.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.320+0000 7f2a1a7fc640 1 -- 172.21.15.5:0/2126362092 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a280176c0 con 0x55966fc139d0 2024-09-12T02:58:32.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.321+0000 7f2a3c82b740 1 -- 172.21.15.5:0/2126362092 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55966fc139d0 msgr2=0x55966fc13db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:32.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.321+0000 7f2a3c82b740 1 --2- 172.21.15.5:0/2126362092 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55966fc139d0 0x55966fc13db0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f2a28006e10 tx=0x7f2a2800b400 comp rx=0 tx=0).stop 2024-09-12T02:58:32.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.321+0000 7f2a3c82b740 1 -- 172.21.15.5:0/2126362092 shutdown_connections 2024-09-12T02:58:32.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.321+0000 7f2a3c82b740 1 --2- 172.21.15.5:0/2126362092 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55966fc1ebb0 0x55966fc20fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:32.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.321+0000 7f2a3c82b740 1 --2- 172.21.15.5:0/2126362092 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55966fc142f0 0x55966fc1e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:32.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.321+0000 7f2a3c82b740 1 --2- 172.21.15.5:0/2126362092 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55966fc139d0 0x55966fc13db0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:32.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.321+0000 7f2a3c82b740 1 -- 172.21.15.5:0/2126362092 >> 172.21.15.5:0/2126362092 conn(0x55966fc0ff60 msgr2=0x55966fc10340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:32.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.321+0000 7f2a3c82b740 1 -- 172.21.15.5:0/2126362092 shutdown_connections 2024-09-12T02:58:32.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.321+0000 7f2a3c82b740 1 -- 172.21.15.5:0/2126362092 wait complete. 2024-09-12T02:58:32.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.321+0000 7f2a3c82b740 1 -- start start 2024-09-12T02:58:32.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.321+0000 7f2a3c82b740 1 Processor -- start 2024-09-12T02:58:32.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.322+0000 7f2a3c82b740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55966fc139d0 0x55966fc1e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:32.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.322+0000 7f2a3c82b740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55966fc142f0 0x55966fc1b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:32.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.322+0000 7f2a3c82b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55966fc1ebb0 0x55966fc1bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:32.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.322+0000 7f2a3c82b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55966fc21780 con 0x55966fc1ebb0 2024-09-12T02:58:32.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.322+0000 7f2a3c82b740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55966fb34f20 con 0x55966fc139d0 2024-09-12T02:58:32.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.322+0000 7f2a3c82b740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55966fc21900 con 0x55966fc142f0 2024-09-12T02:58:32.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.322+0000 7f2a3c82b740 10 client.0 _ll_register_callbacks cb 0x55966fafe030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:58:32.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.322+0000 7f2a337fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55966fc1ebb0 0x55966fc1bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:32.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.322+0000 7f2a337fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55966fc1ebb0 0x55966fc1bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39400/0 (socket says 172.21.15.5:39400) 2024-09-12T02:58:32.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.322+0000 7f2a337fe640 1 -- 172.21.15.5:0/2780744481 learned_addr learned my addr 172.21.15.5:0/2780744481 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:58:32.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.322+0000 7f2a337fe640 1 --2- 172.21.15.5:0/2780744481 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55966fc139d0 0x55966fc1e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:32.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.322+0000 7f2a33fff640 1 --2- 172.21.15.5:0/2780744481 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55966fc142f0 0x55966fc1b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:32.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.322+0000 7f2a337fe640 1 -- 172.21.15.5:0/2780744481 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55966fc142f0 msgr2=0x55966fc1b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:32.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.322+0000 7f2a337fe640 1 --2- 172.21.15.5:0/2780744481 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55966fc142f0 0x55966fc1b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:32.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.322+0000 7f2a337fe640 1 -- 172.21.15.5:0/2780744481 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55966fc139d0 msgr2=0x55966fc1e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:32.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.322+0000 7f2a337fe640 1 --2- 172.21.15.5:0/2780744481 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55966fc139d0 0x55966fc1e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:32.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.322+0000 7f2a337fe640 1 -- 172.21.15.5:0/2780744481 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55966f8eb2b0 con 0x55966fc1ebb0 2024-09-12T02:58:32.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.322+0000 7f2a337fe640 1 --2- 172.21.15.5:0/2780744481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55966fc1ebb0 0x55966fc1bd30 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f2a28011e50 tx=0x7f2a28017380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30190c99b8b0e581 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:32.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.322+0000 7f2a137fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T02:58:32.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.323+0000 7f2a137fe640 1 -- 172.21.15.5:0/2780744481 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a28019480 con 0x55966fc1ebb0 2024-09-12T02:58:32.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.323+0000 7f2a137fe640 1 -- 172.21.15.5:0/2780744481 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f2a28019620 con 0x55966fc1ebb0 2024-09-12T02:58:32.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.323+0000 7f2a3c82b740 1 -- 172.21.15.5:0/2780744481 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a28010040 con 0x55966fc1ebb0 2024-09-12T02:58:32.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.323+0000 7f2a137fe640 1 -- 172.21.15.5:0/2780744481 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2a28019820 con 0x55966fc1ebb0 2024-09-12T02:58:32.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.323+0000 7f2a3c82b740 10 client.15444 fetch_fsmap learned FSMap version 26 2024-09-12T02:58:32.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.323+0000 7f2a3c82b740 1 -- 172.21.15.5:0/2780744481 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55966fc2d0f0 con 0x55966fc1ebb0 2024-09-12T02:58:32.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.323+0000 7f2a137fe640 1 -- 172.21.15.5:0/2780744481 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a28010040 con 0x55966fc1ebb0 2024-09-12T02:58:32.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.323+0000 7f2a137fe640 1 -- 172.21.15.5:0/2780744481 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f2a28017aa0 con 0x55966fc1ebb0 2024-09-12T02:58:32.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.323+0000 7f2a3c82b740 10 client.15444 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T02:58:32.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.323+0000 7f2a3c82b740 10 client.15444 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-12T02:58:32.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.323+0000 7f2a3c82b740 20 client.15444 populate_metadata read hostname 'smithi005' 2024-09-12T02:58:32.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.323+0000 7f2a3c82b740 1 --2- 172.21.15.5:0/2780744481 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55966fc2f3d0 0x55966fc30840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:32.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.323+0000 7f2a3c82b740 4 client.15444 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:58:32.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.323+0000 7f2a3c82b740 1 -- 172.21.15.5:0/2780744481 --> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55966fb87600 con 0x55966fc2f3d0 2024-09-12T02:58:32.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.324+0000 7f2a33fff640 1 --2- 172.21.15.5:0/2780744481 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55966fc2f3d0 0x55966fc30840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:32.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.324+0000 7f2a33fff640 1 --2- 172.21.15.5:0/2780744481 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55966fc2f3d0 0x55966fc30840 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51f68ee55c661983 server_cookie=2456a1ae55c76ede in_seq=0 out_seq=0 2024-09-12T02:58:32.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.324+0000 7f2a137fe640 10 client.15444 ms_handle_connect on v2:172.21.15.148:6838/2317600000 2024-09-12T02:58:32.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.325+0000 7f2a137fe640 1 -- 172.21.15.5:0/2780744481 <== mds.0 v2:172.21.15.148:6838/2317600000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55966fb87600 con 0x55966fc2f3d0 2024-09-12T02:58:32.330 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.325+0000 7f2a137fe640 10 client.15444 handle_command_reply: tid=0 2024-09-12T02:58:32.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.325+0000 7f2a3c82b740 1 client.15444 shutdown 2024-09-12T02:58:32.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.325+0000 7f2a3c82b740 1 -- 172.21.15.5:0/2780744481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55966fc1ebb0 msgr2=0x55966fc1bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:32.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.325+0000 7f2a3c82b740 1 --2- 172.21.15.5:0/2780744481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55966fc1ebb0 0x55966fc1bd30 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f2a28011e50 tx=0x7f2a28017380 comp rx=0 tx=0).stop 2024-09-12T02:58:32.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.325+0000 7f2a3c82b740 1 -- 172.21.15.5:0/2780744481 shutdown_connections 2024-09-12T02:58:32.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.325+0000 7f2a3c82b740 1 --2- 172.21.15.5:0/2780744481 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55966fc2f3d0 0x55966fc30840 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:32.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.325+0000 7f2a3c82b740 1 --2- 172.21.15.5:0/2780744481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55966fc1ebb0 0x55966fc1bd30 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:32.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.325+0000 7f2a3c82b740 1 --2- 172.21.15.5:0/2780744481 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55966fc142f0 0x55966fc1b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:32.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.325+0000 7f2a3c82b740 1 --2- 172.21.15.5:0/2780744481 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55966fc139d0 0x55966fc1e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:32.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.325+0000 7f2a3c82b740 1 -- 172.21.15.5:0/2780744481 >> 172.21.15.5:0/2780744481 conn(0x55966fc0ff60 msgr2=0x55966fc15c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:32.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.325+0000 7f2a137fe640 0 client.15444 ms_handle_reset on v2:172.21.15.148:6838/2317600000 2024-09-12T02:58:32.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.325+0000 7f2a3c82b740 1 -- 172.21.15.5:0/2780744481 shutdown_connections 2024-09-12T02:58:32.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.325+0000 7f2a3c82b740 1 -- 172.21.15.5:0/2780744481 wait complete. 2024-09-12T02:58:32.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.325+0000 7f2a3c82b740 20 client.15444 trim_cache size 0 max 16384 2024-09-12T02:58:32.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.343+0000 7f2a3c82b740 1 Processor -- start 2024-09-12T02:58:32.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.343+0000 7f2a3c82b740 1 -- start start 2024-09-12T02:58:32.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.343+0000 7f2a3c82b740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55966fc139d0 0x55966fc2ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:32.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.343+0000 7f2a3c82b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55966fc142f0 0x55966fc0ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:32.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.343+0000 7f2a3c82b740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55966fc1ebb0 0x55966fc1a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:32.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.343+0000 7f2a3c82b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55966fc2eb80 con 0x55966fc142f0 2024-09-12T02:58:32.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.343+0000 7f2a3c82b740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55966fc2f250 con 0x55966fc139d0 2024-09-12T02:58:32.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.343+0000 7f2a3c82b740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55966fc2f920 con 0x55966fc1ebb0 2024-09-12T02:58:32.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.344+0000 7f2a33fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55966fc142f0 0x55966fc0ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:32.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.344+0000 7f2a33fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55966fc142f0 0x55966fc0ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39412/0 (socket says 172.21.15.5:39412) 2024-09-12T02:58:32.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.344+0000 7f2a33fff640 1 -- 172.21.15.5:0/1585316782 learned_addr learned my addr 172.21.15.5:0/1585316782 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:58:32.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.344+0000 7f2a337fe640 1 --2- 172.21.15.5:0/1585316782 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55966fc139d0 0x55966fc2ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:32.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.344+0000 7f2a337fe640 1 --2- 172.21.15.5:0/1585316782 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55966fc1ebb0 0x55966fc1a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:32.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.344+0000 7f2a33fff640 1 -- 172.21.15.5:0/1585316782 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55966fc1ebb0 msgr2=0x55966fc1a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:32.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.344+0000 7f2a33fff640 1 --2- 172.21.15.5:0/1585316782 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55966fc1ebb0 0x55966fc1a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:32.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.344+0000 7f2a33fff640 1 -- 172.21.15.5:0/1585316782 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55966fc139d0 msgr2=0x55966fc2ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:32.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.344+0000 7f2a33fff640 1 --2- 172.21.15.5:0/1585316782 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55966fc139d0 0x55966fc2ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:32.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.344+0000 7f2a33fff640 1 -- 172.21.15.5:0/1585316782 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55966fc1e810 con 0x55966fc142f0 2024-09-12T02:58:32.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.344+0000 7f2a337fe640 1 --2- 172.21.15.5:0/1585316782 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55966fc1ebb0 0x55966fc1a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:58:32.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.344+0000 7f2a33fff640 1 --2- 172.21.15.5:0/1585316782 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55966fc142f0 0x55966fc0ace0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x55966fc10dc0 tx=0x7f2a24008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf61beaff6d89416 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:32.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.344+0000 7f2a19ffb640 1 -- 172.21.15.5:0/1585316782 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a24020070 con 0x55966fc142f0 2024-09-12T02:58:32.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.344+0000 7f2a19ffb640 1 -- 172.21.15.5:0/1585316782 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a2401c250 con 0x55966fc142f0 2024-09-12T02:58:32.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.344+0000 7f2a19ffb640 1 -- 172.21.15.5:0/1585316782 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a24014610 con 0x55966fc142f0 2024-09-12T02:58:32.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.345+0000 7f2a3c82b740 1 -- 172.21.15.5:0/1585316782 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55966fc142f0 msgr2=0x55966fc0ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:32.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.345+0000 7f2a3c82b740 1 --2- 172.21.15.5:0/1585316782 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55966fc142f0 0x55966fc0ace0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x55966fc10dc0 tx=0x7f2a24008040 comp rx=0 tx=0).stop 2024-09-12T02:58:32.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.345+0000 7f2a3c82b740 1 -- 172.21.15.5:0/1585316782 shutdown_connections 2024-09-12T02:58:32.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.345+0000 7f2a3c82b740 1 --2- 172.21.15.5:0/1585316782 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55966fc142f0 0x55966fc0ace0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:32.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.345+0000 7f2a3c82b740 1 --2- 172.21.15.5:0/1585316782 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55966fc1ebb0 0x55966fc1a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:32.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.345+0000 7f2a3c82b740 1 --2- 172.21.15.5:0/1585316782 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55966fc139d0 0x55966fc2ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:32.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.345+0000 7f2a3c82b740 1 -- 172.21.15.5:0/1585316782 >> 172.21.15.5:0/1585316782 conn(0x55966fc0ff60 msgr2=0x55966fc2e440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:32.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.345+0000 7f2a3c82b740 1 -- 172.21.15.5:0/1585316782 shutdown_connections 2024-09-12T02:58:32.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.345+0000 7f2a3c82b740 1 -- 172.21.15.5:0/1585316782 wait complete. 2024-09-12T02:58:32.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.345+0000 7f2a3c82b740 1 -- start start 2024-09-12T02:58:32.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.346+0000 7f2a3c82b740 1 Processor -- start 2024-09-12T02:58:32.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.346+0000 7f2a3c82b740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55966fc139d0 0x55966fc33b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:32.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.346+0000 7f2a3c82b740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55966fc142f0 0x55966fc340d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:32.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.346+0000 7f2a3c82b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55966fc1ebb0 0x55966fc97ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:32.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.346+0000 7f2a3c82b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55966fc306c0 con 0x55966fc1ebb0 2024-09-12T02:58:32.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.346+0000 7f2a3c82b740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55966fc2fff0 con 0x55966fc142f0 2024-09-12T02:58:32.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.346+0000 7f2a3c82b740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55966fc30d90 con 0x55966fc139d0 2024-09-12T02:58:32.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.346+0000 7f2a3c82b740 10 client.0 _ll_register_callbacks cb 0x7f2a240071a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:58:32.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.346+0000 7f2a337fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55966fc1ebb0 0x55966fc97ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:32.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.346+0000 7f2a33fff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55966fc142f0 0x55966fc340d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:32.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.346+0000 7f2a337fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55966fc1ebb0 0x55966fc97ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39420/0 (socket says 172.21.15.5:39420) 2024-09-12T02:58:32.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.346+0000 7f2a337fe640 1 -- 172.21.15.5:0/652310282 learned_addr learned my addr 172.21.15.5:0/652310282 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:58:32.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.346+0000 7f2a337fe640 1 --2- 172.21.15.5:0/652310282 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55966fc139d0 0x55966fc33b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:32.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.346+0000 7f2a337fe640 1 -- 172.21.15.5:0/652310282 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55966fc139d0 msgr2=0x55966fc33b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:32.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.346+0000 7f2a337fe640 1 --2- 172.21.15.5:0/652310282 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55966fc139d0 0x55966fc33b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:32.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.346+0000 7f2a337fe640 1 -- 172.21.15.5:0/652310282 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55966fc142f0 msgr2=0x55966fc340d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:32.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.346+0000 7f2a337fe640 1 --2- 172.21.15.5:0/652310282 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55966fc142f0 0x55966fc340d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:32.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.346+0000 7f2a337fe640 1 -- 172.21.15.5:0/652310282 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55966fc349a0 con 0x55966fc1ebb0 2024-09-12T02:58:32.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.347+0000 7f2a337fe640 1 --2- 172.21.15.5:0/652310282 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55966fc1ebb0 0x55966fc97ad0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f2a28017ef0 tx=0x7f2a28011f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee325fb0a18ac0fc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:32.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.347+0000 7f2a137fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T02:58:32.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.347+0000 7f2a137fe640 1 -- 172.21.15.5:0/652310282 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a28011ae0 con 0x55966fc1ebb0 2024-09-12T02:58:32.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.347+0000 7f2a137fe640 1 -- 172.21.15.5:0/652310282 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f2a28019480 con 0x55966fc1ebb0 2024-09-12T02:58:32.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.347+0000 7f2a3c82b740 1 -- 172.21.15.5:0/652310282 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a2401c250 con 0x55966fc1ebb0 2024-09-12T02:58:32.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.347+0000 7f2a137fe640 1 -- 172.21.15.5:0/652310282 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2a28019680 con 0x55966fc1ebb0 2024-09-12T02:58:32.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.347+0000 7f2a3c82b740 10 client.15450 fetch_fsmap learned FSMap version 26 2024-09-12T02:58:32.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.347+0000 7f2a3c82b740 1 -- 172.21.15.5:0/652310282 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55966fc990e0 con 0x55966fc1ebb0 2024-09-12T02:58:32.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.347+0000 7f2a137fe640 1 -- 172.21.15.5:0/652310282 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2a2401c250 con 0x55966fc1ebb0 2024-09-12T02:58:32.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.347+0000 7f2a137fe640 1 -- 172.21.15.5:0/652310282 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f2a280173f0 con 0x55966fc1ebb0 2024-09-12T02:58:32.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.347+0000 7f2a3c82b740 10 client.15450 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T02:58:32.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.347+0000 7f2a3c82b740 10 client.15450 resolve_mds: resolved name 'i' to daemon mds.i 2024-09-12T02:58:32.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.347+0000 7f2a3c82b740 20 client.15450 populate_metadata read hostname 'smithi005' 2024-09-12T02:58:32.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.347+0000 7f2a3c82b740 1 --2- 172.21.15.5:0/652310282 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55966fc9daf0 0x55966fc9c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:32.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.347+0000 7f2a3c82b740 4 client.15450 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-12T02:58:32.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.347+0000 7f2a3c82b740 1 -- 172.21.15.5:0/652310282 --> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] -- command(tid 0: {"prefix": "status"}) -- 0x55966fb81b10 con 0x55966fc9daf0 2024-09-12T02:58:32.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.348+0000 7f2a33fff640 1 --2- 172.21.15.5:0/652310282 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55966fc9daf0 0x55966fc9c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:32.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.348+0000 7f2a33fff640 1 --2- 172.21.15.5:0/652310282 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55966fc9daf0 0x55966fc9c800 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=782bcbef2606453e server_cookie=636d5702b1231c74 in_seq=0 out_seq=0 2024-09-12T02:58:32.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.348+0000 7f2a137fe640 10 client.15450 ms_handle_connect on v2:172.21.15.148:6838/2317600000 2024-09-12T02:58:32.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.348+0000 7f2a137fe640 1 -- 172.21.15.5:0/652310282 <== mds.0 v2:172.21.15.148:6838/2317600000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55966fb81b10 con 0x55966fc9daf0 2024-09-12T02:58:32.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.348+0000 7f2a137fe640 10 client.15450 handle_command_reply: tid=0 2024-09-12T02:58:32.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.348+0000 7f2a3c82b740 1 client.15450 shutdown 2024-09-12T02:58:32.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.349+0000 7f2a3c82b740 1 -- 172.21.15.5:0/652310282 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55966fc1ebb0 msgr2=0x55966fc97ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:32.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.349+0000 7f2a3c82b740 1 --2- 172.21.15.5:0/652310282 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55966fc1ebb0 0x55966fc97ad0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f2a28017ef0 tx=0x7f2a28011f50 comp rx=0 tx=0).stop 2024-09-12T02:58:32.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.349+0000 7f2a3c82b740 1 -- 172.21.15.5:0/652310282 shutdown_connections 2024-09-12T02:58:32.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.349+0000 7f2a3c82b740 1 --2- 172.21.15.5:0/652310282 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55966fc9daf0 0x55966fc9c800 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:32.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.349+0000 7f2a3c82b740 1 --2- 172.21.15.5:0/652310282 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55966fc1ebb0 0x55966fc97ad0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:32.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.349+0000 7f2a3c82b740 1 --2- 172.21.15.5:0/652310282 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55966fc142f0 0x55966fc340d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:32.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.349+0000 7f2a3c82b740 1 --2- 172.21.15.5:0/652310282 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55966fc139d0 0x55966fc33b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:32.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.349+0000 7f2a3c82b740 1 -- 172.21.15.5:0/652310282 >> 172.21.15.5:0/652310282 conn(0x55966fc0ff60 msgr2=0x55966fc0ace0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:32.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.349+0000 7f2a137fe640 0 client.15450 ms_handle_reset on v2:172.21.15.148:6838/2317600000 2024-09-12T02:58:32.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.349+0000 7f2a3c82b740 1 -- 172.21.15.5:0/652310282 shutdown_connections 2024-09-12T02:58:32.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.349+0000 7f2a3c82b740 1 -- 172.21.15.5:0/652310282 wait complete. 2024-09-12T02:58:32.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.349+0000 7f2a3c82b740 20 client.15450 trim_cache size 0 max 16384 2024-09-12T02:58:32.355 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-12T02:58:32.355 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "618305f8-70b1-11ef-bcea-c7b262605968", 2024-09-12T02:58:32.355 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 0, 2024-09-12T02:58:32.355 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24502, 2024-09-12T02:58:32.355 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2024-09-12T02:58:32.355 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2024-09-12T02:58:32.355 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2024-09-12T02:58:32.356 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 135.37738647899999, 2024-09-12T02:58:32.356 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 26, 2024-09-12T02:58:32.356 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 83, 2024-09-12T02:58:32.356 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 83, 2024-09-12T02:58:32.356 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 154.68308164300001 2024-09-12T02:58:32.356 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-12T02:58:32.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.349+0000 7f2a3a452640 1 -- 172.21.15.5:0/601332483 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f2a1c0a2c10 msgr2=0x7f2a1c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:32.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.349+0000 7f2a3a452640 1 --2- 172.21.15.5:0/601332483 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f2a1c0a2c10 0x7f2a1c0a50d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c0bc990 tx=0x7f2a2c0a7040 comp rx=0 tx=0).stop 2024-09-12T02:58:32.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.349+0000 7f2a3a452640 1 -- 172.21.15.5:0/601332483 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a3410c1a0 msgr2=0x7f2a3419a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:32.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.349+0000 7f2a3a452640 1 --2- 172.21.15.5:0/601332483 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a3410c1a0 0x7f2a3419a5b0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c0098d0 tx=0x7f2a2c003f10 comp rx=0 tx=0).stop 2024-09-12T02:58:32.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.350+0000 7f2a3a452640 1 -- 172.21.15.5:0/601332483 shutdown_connections 2024-09-12T02:58:32.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.350+0000 7f2a3a452640 1 --2- 172.21.15.5:0/601332483 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f2a1c0a2c10 0x7f2a1c0a50d0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:32.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.350+0000 7f2a3a452640 1 --2- 172.21.15.5:0/601332483 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a3410c1a0 0x7f2a3419a5b0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:32.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.350+0000 7f2a3a452640 1 --2- 172.21.15.5:0/601332483 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2a34103720 0x7f2a341a0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:32.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.350+0000 7f2a3a452640 1 --2- 172.21.15.5:0/601332483 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2a34102e00 0x7f2a341a03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:32.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.350+0000 7f2a3a452640 1 -- 172.21.15.5:0/601332483 >> 172.21.15.5:0/601332483 conn(0x7f2a340fc7a0 msgr2=0x7f2a34100990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:32.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.350+0000 7f2a3a452640 1 -- 172.21.15.5:0/601332483 shutdown_connections 2024-09-12T02:58:32.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:32.350+0000 7f2a3a452640 1 -- 172.21.15.5:0/601332483 wait complete. 2024-09-12T02:58:32.960 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:58:32 smithi005 ceph-mon[31197]: pgmap v344: 97 pgs: 97 active+clean; 658 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 767 B/s wr, 9 op/s 2024-09-12T02:58:32.991 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 15 sec 2024-09-12T02:58:33.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:58:32 smithi148 ceph-mon[35090]: pgmap v344: 97 pgs: 97 active+clean; 658 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 767 B/s wr, 9 op/s 2024-09-12T02:58:33.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:58:32 smithi146 ceph-mon[35011]: pgmap v344: 97 pgs: 97 active+clean; 658 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 767 B/s wr, 9 op/s 2024-09-12T02:58:35.165 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:58:34 smithi148 ceph-mon[35090]: pgmap v345: 97 pgs: 97 active+clean; 658 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-12T02:58:35.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:58:34 smithi146 ceph-mon[35011]: pgmap v345: 97 pgs: 97 active+clean; 658 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-12T02:58:35.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:58:34 smithi005 ceph-mon[31197]: pgmap v345: 97 pgs: 97 active+clean; 658 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-12T02:58:37.165 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:58:36 smithi148 ceph-mon[35090]: pgmap v346: 97 pgs: 97 active+clean; 658 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-12T02:58:37.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:58:36 smithi146 ceph-mon[35011]: pgmap v346: 97 pgs: 97 active+clean; 658 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-12T02:58:37.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:58:36 smithi005 ceph-mon[31197]: pgmap v346: 97 pgs: 97 active+clean; 658 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-12T02:58:39.165 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:58:38 smithi148 ceph-mon[35090]: pgmap v347: 97 pgs: 97 active+clean; 658 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-12T02:58:39.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:58:38 smithi146 ceph-mon[35011]: pgmap v347: 97 pgs: 97 active+clean; 658 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-12T02:58:39.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:58:38 smithi005 ceph-mon[31197]: pgmap v347: 97 pgs: 97 active+clean; 658 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-12T02:58:41.165 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:58:40 smithi148 ceph-mon[35090]: pgmap v348: 97 pgs: 97 active+clean; 658 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-12T02:58:41.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:58:40 smithi146 ceph-mon[35011]: pgmap v348: 97 pgs: 97 active+clean; 658 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-12T02:58:41.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:58:40 smithi005 ceph-mon[31197]: pgmap v348: 97 pgs: 97 active+clean; 658 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-12T02:58:43.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:58:42 smithi148 ceph-mon[35090]: pgmap v349: 97 pgs: 97 active+clean; 658 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T02:58:43.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:58:42 smithi146 ceph-mon[35011]: pgmap v349: 97 pgs: 97 active+clean; 658 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T02:58:43.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:58:42 smithi005 ceph-mon[31197]: pgmap v349: 97 pgs: 97 active+clean; 658 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T02:58:45.165 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:58:44 smithi148 ceph-mon[35090]: pgmap v350: 97 pgs: 97 active+clean; 658 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-12T02:58:45.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:58:44 smithi146 ceph-mon[35011]: pgmap v350: 97 pgs: 97 active+clean; 658 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-12T02:58:45.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:58:44 smithi005 ceph-mon[31197]: pgmap v350: 97 pgs: 97 active+clean; 658 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-12T02:58:45.796 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T02:58:46.068 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:58:46.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.717+0000 7f30a86eb640 1 -- 172.21.15.5:0/3320249180 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f30a0102ef0 msgr2=0x7f30a010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:46.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.717+0000 7f30a86eb640 1 --2- 172.21.15.5:0/3320249180 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f30a0102ef0 0x7f30a010da40 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f3090009ca0 tx=0x7f309002a950 comp rx=0 tx=0).stop 2024-09-12T02:58:46.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.718+0000 7f30a86eb640 1 -- 172.21.15.5:0/3320249180 shutdown_connections 2024-09-12T02:58:46.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.718+0000 7f30a86eb640 1 --2- 172.21.15.5:0/3320249180 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f30a0102ef0 0x7f30a010da40 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:46.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.718+0000 7f30a86eb640 1 --2- 172.21.15.5:0/3320249180 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f30a0102570 0x7f30a01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:46.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.718+0000 7f30a86eb640 1 --2- 172.21.15.5:0/3320249180 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f30a0100e20 0x7f30a0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:46.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.718+0000 7f30a86eb640 1 -- 172.21.15.5:0/3320249180 >> 172.21.15.5:0/3320249180 conn(0x7f30a00fc7d0 msgr2=0x7f30a00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:46.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.718+0000 7f30a86eb640 1 -- 172.21.15.5:0/3320249180 shutdown_connections 2024-09-12T02:58:46.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.718+0000 7f30a86eb640 1 -- 172.21.15.5:0/3320249180 wait complete. 2024-09-12T02:58:46.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.718+0000 7f30a86eb640 1 Processor -- start 2024-09-12T02:58:46.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.719+0000 7f30a86eb640 1 -- start start 2024-09-12T02:58:46.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.719+0000 7f30a86eb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f30a0100e20 0x7f30a01993b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:46.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.719+0000 7f30a86eb640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f30a0102570 0x7f30a01998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:46.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.719+0000 7f30a86eb640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f30a0102ef0 0x7f30a0199e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:46.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.719+0000 7f30a86eb640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f30a0110f80 con 0x7f30a0100e20 2024-09-12T02:58:46.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.719+0000 7f30a86eb640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f30a0110e00 con 0x7f30a0102ef0 2024-09-12T02:58:46.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.719+0000 7f30a86eb640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f30a0111100 con 0x7f30a0102570 2024-09-12T02:58:46.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.719+0000 7f30a6487640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f30a0100e20 0x7f30a01993b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:46.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.719+0000 7f30a6487640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f30a0100e20 0x7f30a01993b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39754/0 (socket says 172.21.15.5:39754) 2024-09-12T02:58:46.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.719+0000 7f30a6487640 1 -- 172.21.15.5:0/2480083567 learned_addr learned my addr 172.21.15.5:0/2480083567 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:58:46.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.719+0000 7f30a6c88640 1 --2- 172.21.15.5:0/2480083567 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f30a0102ef0 0x7f30a0199e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:46.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.719+0000 7f30a5c86640 1 --2- 172.21.15.5:0/2480083567 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f30a0102570 0x7f30a01998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:46.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.719+0000 7f30a6487640 1 -- 172.21.15.5:0/2480083567 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f30a0102570 msgr2=0x7f30a01998f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:46.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.719+0000 7f30a6487640 1 --2- 172.21.15.5:0/2480083567 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f30a0102570 0x7f30a01998f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:46.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.719+0000 7f30a6487640 1 -- 172.21.15.5:0/2480083567 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f30a0102ef0 msgr2=0x7f30a0199e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:46.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.719+0000 7f30a6487640 1 --2- 172.21.15.5:0/2480083567 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f30a0102ef0 0x7f30a0199e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:46.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.719+0000 7f30a6487640 1 -- 172.21.15.5:0/2480083567 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30a01a47b0 con 0x7f30a0100e20 2024-09-12T02:58:46.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.720+0000 7f30a6487640 1 --2- 172.21.15.5:0/2480083567 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f30a0100e20 0x7f30a01993b0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f309800d950 tx=0x7f309800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:46.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.720+0000 7f309f7fe640 1 -- 172.21.15.5:0/2480083567 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3098017070 con 0x7f30a0100e20 2024-09-12T02:58:46.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.720+0000 7f309f7fe640 1 -- 172.21.15.5:0/2480083567 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3098009c30 con 0x7f30a0100e20 2024-09-12T02:58:46.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.720+0000 7f30a86eb640 1 -- 172.21.15.5:0/2480083567 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30a01a4aa0 con 0x7f30a0100e20 2024-09-12T02:58:46.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.720+0000 7f309f7fe640 1 -- 172.21.15.5:0/2480083567 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f309800f6b0 con 0x7f30a0100e20 2024-09-12T02:58:46.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.720+0000 7f30a86eb640 1 -- 172.21.15.5:0/2480083567 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30a01a4fe0 con 0x7f30a0100e20 2024-09-12T02:58:46.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.722+0000 7f309f7fe640 1 -- 172.21.15.5:0/2480083567 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7f30980109d0 con 0x7f30a0100e20 2024-09-12T02:58:46.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.722+0000 7f30aaac4740 1 Processor -- start 2024-09-12T02:58:46.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.723+0000 7f30aaac4740 1 -- start start 2024-09-12T02:58:46.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.723+0000 7f30aaac4740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d4826b94a0 0x55d4826b9880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:46.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.723+0000 7f30aaac4740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d4826b9dc0 0x55d4826c4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:46.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.723+0000 7f30aaac4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4826c4680 0x55d4826c6a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:46.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.723+0000 7f30aaac4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d48261f2e0 con 0x55d4826c4680 2024-09-12T02:58:46.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.723+0000 7f30aaac4740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55d4825c6ce0 con 0x55d4826b94a0 2024-09-12T02:58:46.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.723+0000 7f30aaac4740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55d48261b2a0 con 0x55d4826b9dc0 2024-09-12T02:58:46.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.723+0000 7f30a5c86640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4826c4680 0x55d4826c6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:46.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.723+0000 7f30a5c86640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4826c4680 0x55d4826c6a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39766/0 (socket says 172.21.15.5:39766) 2024-09-12T02:58:46.735 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.723+0000 7f30a5c86640 1 -- 172.21.15.5:0/577911989 learned_addr learned my addr 172.21.15.5:0/577911989 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:58:46.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.723+0000 7f30a6c88640 1 --2- 172.21.15.5:0/577911989 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d4826b94a0 0x55d4826b9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:46.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.723+0000 7f30a6487640 1 --2- 172.21.15.5:0/577911989 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d4826b9dc0 0x55d4826c4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:46.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.723+0000 7f30a5c86640 1 -- 172.21.15.5:0/577911989 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d4826b9dc0 msgr2=0x55d4826c4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:46.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.723+0000 7f30a5c86640 1 --2- 172.21.15.5:0/577911989 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d4826b9dc0 0x55d4826c4140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:46.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.723+0000 7f30a5c86640 1 -- 172.21.15.5:0/577911989 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d4826b94a0 msgr2=0x55d4826b9880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:46.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.723+0000 7f30a5c86640 1 --2- 172.21.15.5:0/577911989 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d4826b94a0 0x55d4826b9880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:46.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.723+0000 7f30a5c86640 1 -- 172.21.15.5:0/577911989 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4825ab440 con 0x55d4826c4680 2024-09-12T02:58:46.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.723+0000 7f30a6487640 1 --2- 172.21.15.5:0/577911989 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d4826b9dc0 0x55d4826c4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:58:46.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.724+0000 7f309f7fe640 1 --2- 172.21.15.5:0/2480083567 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f30880a2c10 0x7f30880a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:46.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.724+0000 7f30a6c88640 1 --2- 172.21.15.5:0/2480083567 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f30880a2c10 0x7f30880a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:46.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.724+0000 7f30a5c86640 1 --2- 172.21.15.5:0/577911989 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4826c4680 0x55d4826c6a70 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f3094009ae0 tx=0x7f309400e2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58e0c9e0b849dfce server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:46.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.724+0000 7f309f7fe640 1 -- 172.21.15.5:0/2480083567 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f30980ceee0 con 0x7f30a0100e20 2024-09-12T02:58:46.736 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.724+0000 7f309cff9640 1 -- 172.21.15.5:0/577911989 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3094016070 con 0x55d4826c4680 2024-09-12T02:58:46.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.724+0000 7f309cff9640 1 -- 172.21.15.5:0/577911989 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3094005500 con 0x55d4826c4680 2024-09-12T02:58:46.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.724+0000 7f309cff9640 1 -- 172.21.15.5:0/577911989 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3094011470 con 0x55d4826c4680 2024-09-12T02:58:46.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.725+0000 7f30aaac4740 1 -- 172.21.15.5:0/577911989 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4826c4680 msgr2=0x55d4826c6a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:46.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.725+0000 7f30aaac4740 1 --2- 172.21.15.5:0/577911989 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4826c4680 0x55d4826c6a70 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f3094009ae0 tx=0x7f309400e2f0 comp rx=0 tx=0).stop 2024-09-12T02:58:46.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.725+0000 7f30a6c88640 1 --2- 172.21.15.5:0/2480083567 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f30880a2c10 0x7f30880a50d0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f3090004af0 tx=0x7f3090005d80 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:46.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.725+0000 7f30aaac4740 1 -- 172.21.15.5:0/577911989 shutdown_connections 2024-09-12T02:58:46.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.725+0000 7f30aaac4740 1 --2- 172.21.15.5:0/577911989 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4826c4680 0x55d4826c6a70 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:46.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.725+0000 7f30aaac4740 1 --2- 172.21.15.5:0/577911989 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d4826b9dc0 0x55d4826c4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:46.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.725+0000 7f30aaac4740 1 --2- 172.21.15.5:0/577911989 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d4826b94a0 0x55d4826b9880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:46.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.725+0000 7f30aaac4740 1 -- 172.21.15.5:0/577911989 >> 172.21.15.5:0/577911989 conn(0x55d4826b5a30 msgr2=0x55d4826b5e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:46.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.725+0000 7f30aaac4740 1 -- 172.21.15.5:0/577911989 shutdown_connections 2024-09-12T02:58:46.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.725+0000 7f30aaac4740 1 -- 172.21.15.5:0/577911989 wait complete. 2024-09-12T02:58:46.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.725+0000 7f30aaac4740 1 -- start start 2024-09-12T02:58:46.737 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.726+0000 7f30aaac4740 1 Processor -- start 2024-09-12T02:58:46.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.726+0000 7f30aaac4740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d4826b94a0 0x55d4826c3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:46.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.726+0000 7f30aaac4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4826b9dc0 0x55d4826c11f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:46.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.726+0000 7f30aaac4740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d4826c4680 0x55d4826c1730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:46.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.726+0000 7f30aaac4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d4826c7250 con 0x55d4826b9dc0 2024-09-12T02:58:46.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.726+0000 7f30aaac4740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55d4825da050 con 0x55d4826c4680 2024-09-12T02:58:46.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.726+0000 7f30aaac4740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55d4826c73d0 con 0x55d4826b94a0 2024-09-12T02:58:46.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.726+0000 7f30aaac4740 10 client.0 _ll_register_callbacks cb 0x55d48261a160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:58:46.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.726+0000 7f30a5c86640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4826b9dc0 0x55d4826c11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:46.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.726+0000 7f30a5c86640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4826b9dc0 0x55d4826c11f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39774/0 (socket says 172.21.15.5:39774) 2024-09-12T02:58:46.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.726+0000 7f30a5c86640 1 -- 172.21.15.5:0/2823162892 learned_addr learned my addr 172.21.15.5:0/2823162892 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:58:46.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.726+0000 7f30a6c88640 1 --2- 172.21.15.5:0/2823162892 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d4826c4680 0x55d4826c1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:46.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.726+0000 7f30a6487640 1 --2- 172.21.15.5:0/2823162892 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d4826b94a0 0x55d4826c3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:46.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.726+0000 7f30a5c86640 1 -- 172.21.15.5:0/2823162892 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d4826b94a0 msgr2=0x55d4826c3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:46.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.726+0000 7f30a5c86640 1 --2- 172.21.15.5:0/2823162892 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d4826b94a0 0x55d4826c3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:46.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.726+0000 7f30a5c86640 1 -- 172.21.15.5:0/2823162892 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d4826c4680 msgr2=0x55d4826c1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:46.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.726+0000 7f30a5c86640 1 --2- 172.21.15.5:0/2823162892 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d4826c4680 0x55d4826c1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:46.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.726+0000 7f30a5c86640 1 -- 172.21.15.5:0/2823162892 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4823903c0 con 0x55d4826b9dc0 2024-09-12T02:58:46.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.727+0000 7f30a5c86640 1 --2- 172.21.15.5:0/2823162892 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4826b9dc0 0x55d4826c11f0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f3094004a90 tx=0x7f3094005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe50f5ed71437627 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:46.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.727+0000 7f3085ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T02:58:46.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.727+0000 7f3085ffb640 1 -- 172.21.15.5:0/2823162892 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3094028060 con 0x55d4826b9dc0 2024-09-12T02:58:46.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.727+0000 7f3085ffb640 1 -- 172.21.15.5:0/2823162892 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f3094004c90 con 0x55d4826b9dc0 2024-09-12T02:58:46.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.727+0000 7f30aaac4740 1 -- 172.21.15.5:0/2823162892 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3094005500 con 0x55d4826b9dc0 2024-09-12T02:58:46.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.727+0000 7f3085ffb640 1 -- 172.21.15.5:0/2823162892 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3094016040 con 0x55d4826b9dc0 2024-09-12T02:58:46.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.727+0000 7f30aaac4740 10 client.15462 fetch_fsmap learned FSMap version 26 2024-09-12T02:58:46.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.727+0000 7f30aaac4740 1 -- 172.21.15.5:0/2823162892 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d4826d6f50 con 0x55d4826b9dc0 2024-09-12T02:58:46.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.727+0000 7f3085ffb640 1 -- 172.21.15.5:0/2823162892 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3094005500 con 0x55d4826b9dc0 2024-09-12T02:58:46.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.727+0000 7f3085ffb640 1 -- 172.21.15.5:0/2823162892 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f309400f040 con 0x55d4826b9dc0 2024-09-12T02:58:46.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.727+0000 7f30aaac4740 10 client.15462 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T02:58:46.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.727+0000 7f30aaac4740 10 client.15462 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T02:58:46.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.727+0000 7f30aaac4740 20 client.15462 populate_metadata read hostname 'smithi005' 2024-09-12T02:58:46.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.728+0000 7f30aaac4740 1 --2- 172.21.15.5:0/2823162892 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55d4826d8410 0x55d4826da800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:46.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.728+0000 7f30aaac4740 4 client.15462 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:58:46.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.728+0000 7f30aaac4740 1 -- 172.21.15.5:0/2823162892 --> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d48262c850 con 0x55d4826d8410 2024-09-12T02:58:46.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.728+0000 7f30a6487640 1 --2- 172.21.15.5:0/2823162892 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55d4826d8410 0x55d4826da800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:46.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.728+0000 7f30a6487640 1 --2- 172.21.15.5:0/2823162892 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55d4826d8410 0x55d4826da800 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4050309aa00e64ba server_cookie=17a8782b0c311759 in_seq=0 out_seq=0 2024-09-12T02:58:46.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.728+0000 7f3085ffb640 10 client.15462 ms_handle_connect on v2:172.21.15.148:6838/2317600000 2024-09-12T02:58:46.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.730+0000 7f3085ffb640 1 -- 172.21.15.5:0/2823162892 <== mds.0 v2:172.21.15.148:6838/2317600000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d48262c850 con 0x55d4826d8410 2024-09-12T02:58:46.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.730+0000 7f3085ffb640 10 client.15462 handle_command_reply: tid=0 2024-09-12T02:58:46.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.730+0000 7f30aaac4740 1 client.15462 shutdown 2024-09-12T02:58:46.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.730+0000 7f30aaac4740 1 -- 172.21.15.5:0/2823162892 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4826b9dc0 msgr2=0x55d4826c11f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:46.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.730+0000 7f30aaac4740 1 --2- 172.21.15.5:0/2823162892 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4826b9dc0 0x55d4826c11f0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f3094004a90 tx=0x7f3094005710 comp rx=0 tx=0).stop 2024-09-12T02:58:46.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.730+0000 7f30aaac4740 1 -- 172.21.15.5:0/2823162892 shutdown_connections 2024-09-12T02:58:46.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.730+0000 7f30aaac4740 1 --2- 172.21.15.5:0/2823162892 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55d4826d8410 0x55d4826da800 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:46.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.730+0000 7f30aaac4740 1 --2- 172.21.15.5:0/2823162892 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4826b9dc0 0x55d4826c11f0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:46.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.730+0000 7f30aaac4740 1 --2- 172.21.15.5:0/2823162892 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d4826c4680 0x55d4826c1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:46.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.730+0000 7f30aaac4740 1 --2- 172.21.15.5:0/2823162892 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d4826b94a0 0x55d4826c3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:46.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.730+0000 7f30aaac4740 1 -- 172.21.15.5:0/2823162892 >> 172.21.15.5:0/2823162892 conn(0x55d4826b5a30 msgr2=0x55d4826bb710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:46.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.730+0000 7f3085ffb640 0 client.15462 ms_handle_reset on v2:172.21.15.148:6838/2317600000 2024-09-12T02:58:46.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.730+0000 7f30aaac4740 1 -- 172.21.15.5:0/2823162892 shutdown_connections 2024-09-12T02:58:46.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.730+0000 7f30aaac4740 1 -- 172.21.15.5:0/2823162892 wait complete. 2024-09-12T02:58:46.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.730+0000 7f30aaac4740 20 client.15462 trim_cache size 0 max 16384 2024-09-12T02:58:46.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.748+0000 7f30aaac4740 1 Processor -- start 2024-09-12T02:58:46.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.748+0000 7f30aaac4740 1 -- start start 2024-09-12T02:58:46.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.748+0000 7f30aaac4740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d4826b94a0 0x55d4826bfac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:46.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.748+0000 7f30aaac4740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d4826b9dc0 0x55d4826bc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:46.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.748+0000 7f30aaac4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4826c4680 0x55d4826dd440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:46.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.748+0000 7f30aaac4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f30a01ab420 con 0x55d4826c4680 2024-09-12T02:58:46.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.748+0000 7f30aaac4740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f30a01acf60 con 0x55d4826b9dc0 2024-09-12T02:58:46.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.748+0000 7f30aaac4740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f30a01abaf0 con 0x55d4826b94a0 2024-09-12T02:58:46.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.748+0000 7f30a6c88640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4826c4680 0x55d4826dd440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:46.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.748+0000 7f30a6c88640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4826c4680 0x55d4826dd440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39780/0 (socket says 172.21.15.5:39780) 2024-09-12T02:58:46.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.748+0000 7f30a6c88640 1 -- 172.21.15.5:0/3428029242 learned_addr learned my addr 172.21.15.5:0/3428029242 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:58:46.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.748+0000 7f30a5c86640 1 --2- 172.21.15.5:0/3428029242 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d4826b9dc0 0x55d4826bc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:46.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.748+0000 7f30a6487640 1 --2- 172.21.15.5:0/3428029242 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d4826b94a0 0x55d4826bfac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:46.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.748+0000 7f30a6c88640 1 -- 172.21.15.5:0/3428029242 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d4826b94a0 msgr2=0x55d4826bfac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:46.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.749+0000 7f30a6c88640 1 --2- 172.21.15.5:0/3428029242 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d4826b94a0 0x55d4826bfac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:46.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.749+0000 7f30a6c88640 1 -- 172.21.15.5:0/3428029242 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d4826b9dc0 msgr2=0x55d4826bc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:46.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.749+0000 7f30a6c88640 1 --2- 172.21.15.5:0/3428029242 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d4826b9dc0 0x55d4826bc760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:46.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.749+0000 7f30a6c88640 1 -- 172.21.15.5:0/3428029242 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4826c01a0 con 0x55d4826c4680 2024-09-12T02:58:46.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.749+0000 7f30a6c88640 1 --2- 172.21.15.5:0/3428029242 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4826c4680 0x55d4826dd440 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f3090009ca0 tx=0x7f3090040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=797293e73ccffc3d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:46.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.749+0000 7f3087fff640 1 -- 172.21.15.5:0/3428029242 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3090034040 con 0x55d4826c4680 2024-09-12T02:58:46.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.749+0000 7f3087fff640 1 -- 172.21.15.5:0/3428029242 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3090003b70 con 0x55d4826c4680 2024-09-12T02:58:46.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.749+0000 7f3087fff640 1 -- 172.21.15.5:0/3428029242 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30900307c0 con 0x55d4826c4680 2024-09-12T02:58:46.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.750+0000 7f30aaac4740 1 -- 172.21.15.5:0/3428029242 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4826c4680 msgr2=0x55d4826dd440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:46.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.750+0000 7f30aaac4740 1 --2- 172.21.15.5:0/3428029242 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4826c4680 0x55d4826dd440 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f3090009ca0 tx=0x7f3090040000 comp rx=0 tx=0).stop 2024-09-12T02:58:46.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.750+0000 7f30aaac4740 1 -- 172.21.15.5:0/3428029242 shutdown_connections 2024-09-12T02:58:46.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.750+0000 7f30aaac4740 1 --2- 172.21.15.5:0/3428029242 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4826c4680 0x55d4826dd440 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:46.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.750+0000 7f30aaac4740 1 --2- 172.21.15.5:0/3428029242 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d4826b9dc0 0x55d4826bc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:46.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.750+0000 7f30aaac4740 1 --2- 172.21.15.5:0/3428029242 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d4826b94a0 0x55d4826bfac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:46.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.750+0000 7f30aaac4740 1 -- 172.21.15.5:0/3428029242 >> 172.21.15.5:0/3428029242 conn(0x55d4826b5a30 msgr2=0x55d4826bc050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:46.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.750+0000 7f30aaac4740 1 -- 172.21.15.5:0/3428029242 shutdown_connections 2024-09-12T02:58:46.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.750+0000 7f30aaac4740 1 -- 172.21.15.5:0/3428029242 wait complete. 2024-09-12T02:58:46.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.750+0000 7f30aaac4740 1 -- start start 2024-09-12T02:58:46.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.750+0000 7f30aaac4740 1 Processor -- start 2024-09-12T02:58:46.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.751+0000 7f30aaac4740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d4826b94a0 0x55d4826dd990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:46.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.751+0000 7f30aaac4740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d4826b9dc0 0x55d4826dad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:46.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.751+0000 7f30aaac4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4826c4680 0x55d4826db270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:46.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.751+0000 7f30aaac4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f30a01ac890 con 0x55d4826c4680 2024-09-12T02:58:46.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.751+0000 7f30aaac4740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f30a01ac1c0 con 0x55d4826b9dc0 2024-09-12T02:58:46.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.751+0000 7f30aaac4740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f30a01a9400 con 0x55d4826b94a0 2024-09-12T02:58:46.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.751+0000 7f30aaac4740 10 client.0 _ll_register_callbacks cb 0x7f3090009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:58:46.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.751+0000 7f30a6c88640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4826c4680 0x55d4826db270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:46.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.751+0000 7f30a6c88640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4826c4680 0x55d4826db270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39790/0 (socket says 172.21.15.5:39790) 2024-09-12T02:58:46.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.751+0000 7f30a6c88640 1 -- 172.21.15.5:0/1364222591 learned_addr learned my addr 172.21.15.5:0/1364222591 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:58:46.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.751+0000 7f30a6487640 1 --2- 172.21.15.5:0/1364222591 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d4826b94a0 0x55d4826dd990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:46.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.751+0000 7f30a5c86640 1 --2- 172.21.15.5:0/1364222591 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d4826b9dc0 0x55d4826dad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:46.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.751+0000 7f30a6c88640 1 -- 172.21.15.5:0/1364222591 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d4826b94a0 msgr2=0x55d4826dd990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:46.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.751+0000 7f30a6c88640 1 --2- 172.21.15.5:0/1364222591 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d4826b94a0 0x55d4826dd990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:46.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.751+0000 7f30a6c88640 1 -- 172.21.15.5:0/1364222591 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d4826b9dc0 msgr2=0x55d4826dad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:46.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.751+0000 7f30a6c88640 1 --2- 172.21.15.5:0/1364222591 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d4826b9dc0 0x55d4826dad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:46.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.751+0000 7f30a6c88640 1 -- 172.21.15.5:0/1364222591 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4826c42b0 con 0x55d4826c4680 2024-09-12T02:58:46.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.752+0000 7f30a6c88640 1 --2- 172.21.15.5:0/1364222591 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4826c4680 0x55d4826db270 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f3090005c90 tx=0x7f309002fa40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d8c641e61d0ec47 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:46.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.752+0000 7f3085ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T02:58:46.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.752+0000 7f3085ffb640 1 -- 172.21.15.5:0/1364222591 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3090034030 con 0x55d4826c4680 2024-09-12T02:58:46.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.752+0000 7f3085ffb640 1 -- 172.21.15.5:0/1364222591 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f30900040f0 con 0x55d4826c4680 2024-09-12T02:58:46.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.752+0000 7f30aaac4740 1 -- 172.21.15.5:0/1364222591 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3090003b70 con 0x55d4826c4680 2024-09-12T02:58:46.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.752+0000 7f3085ffb640 1 -- 172.21.15.5:0/1364222591 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f30900042f0 con 0x55d4826c4680 2024-09-12T02:58:46.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.752+0000 7f30aaac4740 10 client.15468 fetch_fsmap learned FSMap version 26 2024-09-12T02:58:46.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.752+0000 7f30aaac4740 1 -- 172.21.15.5:0/1364222591 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d4827429a0 con 0x55d4826c4680 2024-09-12T02:58:46.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.752+0000 7f3085ffb640 1 -- 172.21.15.5:0/1364222591 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3090004490 con 0x55d4826c4680 2024-09-12T02:58:46.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.752+0000 7f3085ffb640 1 -- 172.21.15.5:0/1364222591 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f30900495f0 con 0x55d4826c4680 2024-09-12T02:58:46.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.752+0000 7f30aaac4740 10 client.15468 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T02:58:46.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.752+0000 7f30aaac4740 10 client.15468 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T02:58:46.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.752+0000 7f30aaac4740 20 client.15468 populate_metadata read hostname 'smithi005' 2024-09-12T02:58:46.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.752+0000 7f30aaac4740 1 --2- 172.21.15.5:0/1364222591 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55d482743060 0x55d482745450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:46.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.753+0000 7f30aaac4740 4 client.15468 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T02:58:46.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.753+0000 7f30aaac4740 1 -- 172.21.15.5:0/1364222591 --> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f3090003b70 con 0x55d482743060 2024-09-12T02:58:46.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.753+0000 7f30a6487640 1 --2- 172.21.15.5:0/1364222591 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55d482743060 0x55d482745450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:46.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.753+0000 7f3085ffb640 10 client.15468 ms_handle_connect on v2:172.21.15.148:6838/2317600000 2024-09-12T02:58:46.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.753+0000 7f30a6487640 1 --2- 172.21.15.5:0/1364222591 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55d482743060 0x55d482745450 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9bed30e7f299468c server_cookie=8783e3ca29b28cf0 in_seq=0 out_seq=0 2024-09-12T02:58:46.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.753+0000 7f3085ffb640 1 -- 172.21.15.5:0/1364222591 <== mds.0 v2:172.21.15.148:6838/2317600000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7f3090003b70 con 0x55d482743060 2024-09-12T02:58:46.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.753+0000 7f3085ffb640 10 client.15468 handle_command_reply: tid=0 2024-09-12T02:58:46.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.753+0000 7f30aaac4740 1 client.15468 shutdown 2024-09-12T02:58:46.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.754+0000 7f30aaac4740 1 -- 172.21.15.5:0/1364222591 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4826c4680 msgr2=0x55d4826db270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:46.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.754+0000 7f30aaac4740 1 --2- 172.21.15.5:0/1364222591 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4826c4680 0x55d4826db270 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f3090005c90 tx=0x7f309002fa40 comp rx=0 tx=0).stop 2024-09-12T02:58:46.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.754+0000 7f30aaac4740 1 -- 172.21.15.5:0/1364222591 shutdown_connections 2024-09-12T02:58:46.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.754+0000 7f30aaac4740 1 --2- 172.21.15.5:0/1364222591 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55d482743060 0x55d482745450 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:46.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.754+0000 7f30aaac4740 1 --2- 172.21.15.5:0/1364222591 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4826c4680 0x55d4826db270 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:46.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.754+0000 7f30aaac4740 1 --2- 172.21.15.5:0/1364222591 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d4826b9dc0 0x55d4826dad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:46.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.754+0000 7f30aaac4740 1 --2- 172.21.15.5:0/1364222591 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d4826b94a0 0x55d4826dd990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:46.759 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.754+0000 7f30aaac4740 1 -- 172.21.15.5:0/1364222591 >> 172.21.15.5:0/1364222591 conn(0x55d4826b5a30 msgr2=0x55d4826c2a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:46.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.754+0000 7f3085ffb640 0 client.15468 ms_handle_reset on v2:172.21.15.148:6838/2317600000 2024-09-12T02:58:46.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.754+0000 7f30aaac4740 1 -- 172.21.15.5:0/1364222591 shutdown_connections 2024-09-12T02:58:46.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.754+0000 7f30aaac4740 1 -- 172.21.15.5:0/1364222591 wait complete. 2024-09-12T02:58:46.760 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.754+0000 7f30aaac4740 20 client.15468 trim_cache size 0 max 16384 2024-09-12T02:58:46.760 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-12T02:58:46.760 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-09-12T02:58:46.760 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-09-12T02:58:46.760 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-12T02:58:46.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.754+0000 7f30a86eb640 1 -- 172.21.15.5:0/2480083567 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f30880a2c10 msgr2=0x7f30880a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:46.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.754+0000 7f30a86eb640 1 --2- 172.21.15.5:0/2480083567 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f30880a2c10 0x7f30880a50d0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f3090004af0 tx=0x7f3090005d80 comp rx=0 tx=0).stop 2024-09-12T02:58:46.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.754+0000 7f30a86eb640 1 -- 172.21.15.5:0/2480083567 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f30a0100e20 msgr2=0x7f30a01993b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:46.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.754+0000 7f30a86eb640 1 --2- 172.21.15.5:0/2480083567 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f30a0100e20 0x7f30a01993b0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f309800d950 tx=0x7f309800de10 comp rx=0 tx=0).stop 2024-09-12T02:58:46.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.755+0000 7f30a86eb640 1 -- 172.21.15.5:0/2480083567 shutdown_connections 2024-09-12T02:58:46.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.755+0000 7f30a86eb640 1 --2- 172.21.15.5:0/2480083567 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f30a0102ef0 0x7f30a0199e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:46.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.755+0000 7f30a86eb640 1 --2- 172.21.15.5:0/2480083567 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f30a0102570 0x7f30a01998f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:46.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.755+0000 7f30a86eb640 1 --2- 172.21.15.5:0/2480083567 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f30880a2c10 0x7f30880a50d0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:46.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.755+0000 7f30a86eb640 1 --2- 172.21.15.5:0/2480083567 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f30a0100e20 0x7f30a01993b0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:46.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.755+0000 7f30a86eb640 1 -- 172.21.15.5:0/2480083567 >> 172.21.15.5:0/2480083567 conn(0x7f30a00fc7d0 msgr2=0x7f30a01068d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:46.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.755+0000 7f30a86eb640 1 -- 172.21.15.5:0/2480083567 shutdown_connections 2024-09-12T02:58:46.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:46.755+0000 7f30a86eb640 1 -- 172.21.15.5:0/2480083567 wait complete. 2024-09-12T02:58:47.165 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:58:46 smithi148 ceph-mon[35090]: pgmap v351: 97 pgs: 97 active+clean; 658 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-12T02:58:47.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:58:46 smithi146 ceph-mon[35011]: pgmap v351: 97 pgs: 97 active+clean; 658 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-12T02:58:47.222 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:58:46 smithi005 ceph-mon[31197]: pgmap v351: 97 pgs: 97 active+clean; 658 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-12T02:58:47.438 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T02:58:47.439 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:58:47.704 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:58:47.992 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 24.5 2024-09-12T02:58:47.992 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:58:48.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:48.174+0000 7fd534ba3640 1 -- 172.21.15.5:0/1095800657 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd530106a70 msgr2=0x7fd530106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:48.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:48.174+0000 7fd534ba3640 1 --2- 172.21.15.5:0/1095800657 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd530106a70 0x7fd530106e50 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fd520009930 tx=0x7fd52002a420 comp rx=0 tx=0).stop 2024-09-12T02:58:48.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:48.174+0000 7fd534ba3640 1 -- 172.21.15.5:0/1095800657 shutdown_connections 2024-09-12T02:58:48.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:48.174+0000 7fd534ba3640 1 --2- 172.21.15.5:0/1095800657 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd530101410 0x7fd53010d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:48.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:48.174+0000 7fd534ba3640 1 --2- 172.21.15.5:0/1095800657 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd530100a70 0x7fd530100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:48.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:48.174+0000 7fd534ba3640 1 --2- 172.21.15.5:0/1095800657 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd530106a70 0x7fd530106e50 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:48.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:48.174+0000 7fd534ba3640 1 -- 172.21.15.5:0/1095800657 >> 172.21.15.5:0/1095800657 conn(0x7fd5300fc790 msgr2=0x7fd5300febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:48.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:48.174+0000 7fd534ba3640 1 -- 172.21.15.5:0/1095800657 shutdown_connections 2024-09-12T02:58:48.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:48.174+0000 7fd534ba3640 1 -- 172.21.15.5:0/1095800657 wait complete. 2024-09-12T02:58:48.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:48.175+0000 7fd534ba3640 1 Processor -- start 2024-09-12T02:58:48.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:48.175+0000 7fd534ba3640 1 -- start start 2024-09-12T02:58:48.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:48.175+0000 7fd534ba3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd530100a70 0x7fd53019a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:48.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:48.175+0000 7fd534ba3640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd530101410 0x7fd53019abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:48.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:48.175+0000 7fd534ba3640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd530106a70 0x7fd53019ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:48.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:48.175+0000 7fd534ba3640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd530110e70 con 0x7fd530100a70 2024-09-12T02:58:48.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:48.175+0000 7fd534ba3640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fd530110cf0 con 0x7fd530106a70 2024-09-12T02:58:48.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:48.175+0000 7fd534ba3640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd530110ff0 con 0x7fd530101410 2024-09-12T02:58:48.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:48.175+0000 7fd52e59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd530100a70 0x7fd53019a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:48.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:48.175+0000 7fd52e59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd530100a70 0x7fd53019a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39800/0 (socket says 172.21.15.5:39800) 2024-09-12T02:58:48.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:48.175+0000 7fd52e59c640 1 -- 172.21.15.5:0/1441272720 learned_addr learned my addr 172.21.15.5:0/1441272720 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:58:48.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:48.176+0000 7fd52dd9b640 1 --2- 172.21.15.5:0/1441272720 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd530101410 0x7fd53019abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:48.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:48.176+0000 7fd52ed9d640 1 --2- 172.21.15.5:0/1441272720 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd530106a70 0x7fd53019ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:48.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:48.176+0000 7fd52e59c640 1 -- 172.21.15.5:0/1441272720 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd530101410 msgr2=0x7fd53019abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:48.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:48.176+0000 7fd52e59c640 1 --2- 172.21.15.5:0/1441272720 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd530101410 0x7fd53019abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:48.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:48.176+0000 7fd52e59c640 1 -- 172.21.15.5:0/1441272720 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd530106a70 msgr2=0x7fd53019ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:48.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:48.176+0000 7fd52e59c640 1 --2- 172.21.15.5:0/1441272720 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd530106a70 0x7fd53019ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:48.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:48.176+0000 7fd52e59c640 1 -- 172.21.15.5:0/1441272720 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd53019f630 con 0x7fd530100a70 2024-09-12T02:58:48.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:48.176+0000 7fd52e59c640 1 --2- 172.21.15.5:0/1441272720 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd530100a70 0x7fd53019a680 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fd520009230 tx=0x7fd520004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:48.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:48.176+0000 7fd51f7fe640 1 -- 172.21.15.5:0/1441272720 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd52003f030 con 0x7fd530100a70 2024-09-12T02:58:48.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:48.176+0000 7fd534ba3640 1 -- 172.21.15.5:0/1441272720 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd53019f8c0 con 0x7fd530100a70 2024-09-12T02:58:48.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:48.176+0000 7fd534ba3640 1 -- 172.21.15.5:0/1441272720 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5301a7200 con 0x7fd530100a70 2024-09-12T02:58:48.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:48.177+0000 7fd51f7fe640 1 -- 172.21.15.5:0/1441272720 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd520003ad0 con 0x7fd530100a70 2024-09-12T02:58:48.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:48.177+0000 7fd51f7fe640 1 -- 172.21.15.5:0/1441272720 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd520035070 con 0x7fd530100a70 2024-09-12T02:58:48.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:48.178+0000 7fd534ba3640 1 -- 172.21.15.5:0/1441272720 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd510005290 con 0x7fd530100a70 2024-09-12T02:58:48.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:48.179+0000 7fd51f7fe640 1 -- 172.21.15.5:0/1441272720 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7fd520042030 con 0x7fd530100a70 2024-09-12T02:58:48.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:48.180+0000 7fd51f7fe640 1 --2- 172.21.15.5:0/1441272720 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fd5180a2c90 0x7fd5180a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:48.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:48.180+0000 7fd51f7fe640 1 -- 172.21.15.5:0/1441272720 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fd5200ef900 con 0x7fd530100a70 2024-09-12T02:58:48.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:48.180+0000 7fd52dd9b640 1 --2- 172.21.15.5:0/1441272720 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fd5180a2c90 0x7fd5180a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:48.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:48.181+0000 7fd52dd9b640 1 --2- 172.21.15.5:0/1441272720 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fd5180a2c90 0x7fd5180a5150 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fd5300ffc80 tx=0x7fd52400b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:48.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:48.185+0000 7fd51f7fe640 1 -- 172.21.15.5:0/1441272720 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd5200bc710 con 0x7fd530100a70 2024-09-12T02:58:48.286 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:58:48.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:48.433+0000 7fd534ba3640 1 -- 172.21.15.5:0/1441272720 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd510005580 con 0x7fd530100a70 2024-09-12T02:58:48.435 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:48.434+0000 7fd51f7fe640 1 -- 172.21.15.5:0/1441272720 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11966 (secure 0 0 0) 0x7fd5200bc010 con 0x7fd530100a70 2024-09-12T02:58:48.436 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-12T02:58:48.436 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":26,"btime":"2024-09-12T02:58:03:323671+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/4135232521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":4135232521},{"type":"v1","addr":"172.21.15.5:6835","nonce":4135232521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/4290221940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":4290221940},{"type":"v1","addr":"172.21.15.5:6837","nonce":4290221940}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T02:56:16.595562+0000","modified":"2024-09-12T02:58:02.463400+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24502,"mds_1":24487,"mds_2":24481,"mds_3":24479,"mds_4":24458},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/26447341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":26447341},{"type":"v1","addr":"172.21.15.5:6839","nonce":26447341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2393443295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2393443295},{"type":"v1","addr":"172.21.15.5:6841","nonce":2393443295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6833/3686970137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3686970137},{"type":"v1","addr":"172.21.15.146:6833","nonce":3686970137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6833/568512209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":568512209},{"type":"v1","addr":"172.21.15.148:6833","nonce":568512209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6835/2540740420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2540740420},{"type":"v1","addr":"172.21.15.146:6835","nonce":2540740420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.146:6837/1673370937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":1673370937},{"type":"v1","addr":"172.21.15.146:6837","nonce":1673370937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":3,"incarnation":22,"state":"up:active","state_seq":6,"addr":"172.21.15.146:6839/3036767228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":3036767228},{"type":"v1","addr":"172.21.15.146:6839","nonce":3036767228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":2,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.148:6835/4195086427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":4195086427},{"type":"v1","addr":"172.21.15.148:6835","nonce":4195086427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.148:6837/2606873922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2606873922},{"type":"v1","addr":"172.21.15.148:6837","nonce":2606873922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6839/2317600000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":2317600000},{"type":"v1","addr":"172.21.15.148:6839","nonce":2317600000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24502,"qdb_cluster":[24502,24487,24481,24479,24458]},"id":1}]} 2024-09-12T02:58:48.436 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 26 2024-09-12T02:58:48.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:48.436+0000 7fd534ba3640 1 -- 172.21.15.5:0/1441272720 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fd5180a2c90 msgr2=0x7fd5180a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:48.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:48.436+0000 7fd534ba3640 1 --2- 172.21.15.5:0/1441272720 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fd5180a2c90 0x7fd5180a5150 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fd5300ffc80 tx=0x7fd52400b410 comp rx=0 tx=0).stop 2024-09-12T02:58:48.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:48.436+0000 7fd534ba3640 1 -- 172.21.15.5:0/1441272720 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd530100a70 msgr2=0x7fd53019a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:48.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:48.436+0000 7fd534ba3640 1 --2- 172.21.15.5:0/1441272720 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd530100a70 0x7fd53019a680 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fd520009230 tx=0x7fd520004270 comp rx=0 tx=0).stop 2024-09-12T02:58:48.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:48.436+0000 7fd534ba3640 1 -- 172.21.15.5:0/1441272720 shutdown_connections 2024-09-12T02:58:48.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:48.436+0000 7fd534ba3640 1 --2- 172.21.15.5:0/1441272720 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd530106a70 0x7fd53019ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:48.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:48.436+0000 7fd534ba3640 1 --2- 172.21.15.5:0/1441272720 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd530101410 0x7fd53019abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:48.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:48.436+0000 7fd534ba3640 1 --2- 172.21.15.5:0/1441272720 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fd5180a2c90 0x7fd5180a5150 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:48.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:48.436+0000 7fd534ba3640 1 --2- 172.21.15.5:0/1441272720 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd530100a70 0x7fd53019a680 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:48.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:48.436+0000 7fd534ba3640 1 -- 172.21.15.5:0/1441272720 >> 172.21.15.5:0/1441272720 conn(0x7fd5300fc790 msgr2=0x7fd53006a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:48.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:48.436+0000 7fd534ba3640 1 -- 172.21.15.5:0/1441272720 shutdown_connections 2024-09-12T02:58:48.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:48.436+0000 7fd534ba3640 1 -- 172.21.15.5:0/1441272720 wait complete. 2024-09-12T02:58:48.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:48.929+0000 7f702b24c640 1 -- 172.21.15.5:0/574752071 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7024100e20 msgr2=0x7f7024101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:48.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:48.929+0000 7f702b24c640 1 --2- 172.21.15.5:0/574752071 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7024100e20 0x7f7024101200 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f7018009900 tx=0x7f701802a480 comp rx=0 tx=0).stop 2024-09-12T02:58:48.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:48.929+0000 7f702b24c640 1 -- 172.21.15.5:0/574752071 shutdown_connections 2024-09-12T02:58:48.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:48.929+0000 7f702b24c640 1 --2- 172.21.15.5:0/574752071 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7024102ef0 0x7f702410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:48.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:48.929+0000 7f702b24c640 1 --2- 172.21.15.5:0/574752071 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7024102570 0x7f70241029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:48.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:48.929+0000 7f702b24c640 1 --2- 172.21.15.5:0/574752071 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7024100e20 0x7f7024101200 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:48.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:48.929+0000 7f702b24c640 1 -- 172.21.15.5:0/574752071 >> 172.21.15.5:0/574752071 conn(0x7f70240fc7d0 msgr2=0x7f70240febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:48.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:48.929+0000 7f702b24c640 1 -- 172.21.15.5:0/574752071 shutdown_connections 2024-09-12T02:58:48.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:48.929+0000 7f702b24c640 1 -- 172.21.15.5:0/574752071 wait complete. 2024-09-12T02:58:48.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:48.930+0000 7f702b24c640 1 Processor -- start 2024-09-12T02:58:48.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:48.930+0000 7f702b24c640 1 -- start start 2024-09-12T02:58:48.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:48.930+0000 7f702b24c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7024100e20 0x7f702419a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:48.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:48.930+0000 7f702b24c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7024102570 0x7f702419abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:48.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:48.930+0000 7f702b24c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7024102ef0 0x7f702419ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:48.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:48.930+0000 7f702b24c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f702410fe80 con 0x7f7024102ef0 2024-09-12T02:58:48.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:48.930+0000 7f702b24c640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f702410fd00 con 0x7f7024102570 2024-09-12T02:58:48.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:48.930+0000 7f702b24c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f7024110000 con 0x7f7024100e20 2024-09-12T02:58:48.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:48.930+0000 7f70297e9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7024102ef0 0x7f702419ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:48.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:48.930+0000 7f70297e9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7024102ef0 0x7f702419ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39814/0 (socket says 172.21.15.5:39814) 2024-09-12T02:58:48.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:48.930+0000 7f70297e9640 1 -- 172.21.15.5:0/2724486318 learned_addr learned my addr 172.21.15.5:0/2724486318 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:58:48.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:48.930+0000 7f7028fe8640 1 --2- 172.21.15.5:0/2724486318 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7024100e20 0x7f702419a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:48.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:48.930+0000 7f7023fff640 1 --2- 172.21.15.5:0/2724486318 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7024102570 0x7f702419abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:48.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:48.930+0000 7f70297e9640 1 -- 172.21.15.5:0/2724486318 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7024100e20 msgr2=0x7f702419a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:48.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:48.931+0000 7f70297e9640 1 --2- 172.21.15.5:0/2724486318 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7024100e20 0x7f702419a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:48.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:48.931+0000 7f70297e9640 1 -- 172.21.15.5:0/2724486318 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7024102570 msgr2=0x7f702419abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:48.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:48.931+0000 7f70297e9640 1 --2- 172.21.15.5:0/2724486318 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7024102570 0x7f702419abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:48.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:48.931+0000 7f70297e9640 1 -- 172.21.15.5:0/2724486318 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f702419f620 con 0x7f7024102ef0 2024-09-12T02:58:48.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:48.931+0000 7f7023fff640 1 --2- 172.21.15.5:0/2724486318 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7024102570 0x7f702419abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:58:48.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:48.931+0000 7f7028fe8640 1 --2- 172.21.15.5:0/2724486318 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7024100e20 0x7f702419a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:58:48.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:48.931+0000 7f70297e9640 1 --2- 172.21.15.5:0/2724486318 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7024102ef0 0x7f702419ef40 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f701c00abd0 tx=0x7f701c007590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:48.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:48.931+0000 7f7021ffb640 1 -- 172.21.15.5:0/2724486318 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f701c007ca0 con 0x7f7024102ef0 2024-09-12T02:58:48.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:48.931+0000 7f7021ffb640 1 -- 172.21.15.5:0/2724486318 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f701c007e40 con 0x7f7024102ef0 2024-09-12T02:58:48.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:48.931+0000 7f702b24c640 1 -- 172.21.15.5:0/2724486318 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f702419f910 con 0x7f7024102ef0 2024-09-12T02:58:48.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:48.931+0000 7f7021ffb640 1 -- 172.21.15.5:0/2724486318 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f701c014670 con 0x7f7024102ef0 2024-09-12T02:58:48.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:48.931+0000 7f702b24c640 1 -- 172.21.15.5:0/2724486318 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70241a71f0 con 0x7f7024102ef0 2024-09-12T02:58:48.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:48.932+0000 7f702b24c640 1 -- 172.21.15.5:0/2724486318 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7024101200 con 0x7f7024102ef0 2024-09-12T02:58:48.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:48.933+0000 7f7021ffb640 1 -- 172.21.15.5:0/2724486318 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7f701c004a40 con 0x7f7024102ef0 2024-09-12T02:58:48.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:48.934+0000 7f7021ffb640 1 --2- 172.21.15.5:0/2724486318 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f70140a2c10 0x7f70140a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:48.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:48.935+0000 7f7028fe8640 1 --2- 172.21.15.5:0/2724486318 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f70140a2c10 0x7f70140a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:48.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:48.935+0000 7f7021ffb640 1 -- 172.21.15.5:0/2724486318 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f701c0ced20 con 0x7f7024102ef0 2024-09-12T02:58:48.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:48.935+0000 7f7028fe8640 1 --2- 172.21.15.5:0/2724486318 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f70140a2c10 0x7f70140a50d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f7018005ab0 tx=0x7f7018005a20 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:48.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:48.939+0000 7f7021ffb640 1 -- 172.21.15.5:0/2724486318 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f701c09bad0 con 0x7f7024102ef0 2024-09-12T02:58:49.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:49.164+0000 7f702b24c640 1 -- 172.21.15.5:0/2724486318 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f70241a73c0 con 0x7f7024102ef0 2024-09-12T02:58:49.165 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:58:48 smithi005 ceph-mon[31197]: pgmap v352: 97 pgs: 97 active+clean; 658 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-12T02:58:49.165 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:58:48 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/1441272720' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:58:49.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:49.165+0000 7f7021ffb640 1 -- 172.21.15.5:0/2724486318 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11966 (secure 0 0 0) 0x7f701c09b3d0 con 0x7f7024102ef0 2024-09-12T02:58:49.166 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-12T02:58:49.166 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":26,"btime":"2024-09-12T02:58:03:323671+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/4135232521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":4135232521},{"type":"v1","addr":"172.21.15.5:6835","nonce":4135232521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/4290221940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":4290221940},{"type":"v1","addr":"172.21.15.5:6837","nonce":4290221940}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T02:56:16.595562+0000","modified":"2024-09-12T02:58:02.463400+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24502,"mds_1":24487,"mds_2":24481,"mds_3":24479,"mds_4":24458},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/26447341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":26447341},{"type":"v1","addr":"172.21.15.5:6839","nonce":26447341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2393443295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2393443295},{"type":"v1","addr":"172.21.15.5:6841","nonce":2393443295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6833/3686970137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3686970137},{"type":"v1","addr":"172.21.15.146:6833","nonce":3686970137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6833/568512209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":568512209},{"type":"v1","addr":"172.21.15.148:6833","nonce":568512209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6835/2540740420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2540740420},{"type":"v1","addr":"172.21.15.146:6835","nonce":2540740420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.146:6837/1673370937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":1673370937},{"type":"v1","addr":"172.21.15.146:6837","nonce":1673370937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":3,"incarnation":22,"state":"up:active","state_seq":6,"addr":"172.21.15.146:6839/3036767228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":3036767228},{"type":"v1","addr":"172.21.15.146:6839","nonce":3036767228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":2,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.148:6835/4195086427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":4195086427},{"type":"v1","addr":"172.21.15.148:6835","nonce":4195086427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.148:6837/2606873922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2606873922},{"type":"v1","addr":"172.21.15.148:6837","nonce":2606873922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6839/2317600000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":2317600000},{"type":"v1","addr":"172.21.15.148:6839","nonce":2317600000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24502,"qdb_cluster":[24502,24487,24481,24479,24458]},"id":1}]} 2024-09-12T02:58:49.166 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 26 2024-09-12T02:58:49.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:58:48 smithi148 ceph-mon[35090]: pgmap v352: 97 pgs: 97 active+clean; 658 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-12T02:58:49.167 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:58:48 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/1441272720' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:58:49.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:49.167+0000 7f702b24c640 1 -- 172.21.15.5:0/2724486318 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f70140a2c10 msgr2=0x7f70140a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:49.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:49.167+0000 7f702b24c640 1 --2- 172.21.15.5:0/2724486318 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f70140a2c10 0x7f70140a50d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f7018005ab0 tx=0x7f7018005a20 comp rx=0 tx=0).stop 2024-09-12T02:58:49.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:49.167+0000 7f702b24c640 1 -- 172.21.15.5:0/2724486318 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7024102ef0 msgr2=0x7f702419ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:49.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:49.167+0000 7f702b24c640 1 --2- 172.21.15.5:0/2724486318 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7024102ef0 0x7f702419ef40 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f701c00abd0 tx=0x7f701c007590 comp rx=0 tx=0).stop 2024-09-12T02:58:49.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:49.167+0000 7f702b24c640 1 -- 172.21.15.5:0/2724486318 shutdown_connections 2024-09-12T02:58:49.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:49.167+0000 7f702b24c640 1 --2- 172.21.15.5:0/2724486318 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f70140a2c10 0x7f70140a50d0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:49.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:49.167+0000 7f702b24c640 1 --2- 172.21.15.5:0/2724486318 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7024102ef0 0x7f702419ef40 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:49.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:49.167+0000 7f702b24c640 1 --2- 172.21.15.5:0/2724486318 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7024102570 0x7f702419abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:49.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:49.168+0000 7f702b24c640 1 --2- 172.21.15.5:0/2724486318 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7024100e20 0x7f702419a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:49.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:49.168+0000 7f702b24c640 1 -- 172.21.15.5:0/2724486318 >> 172.21.15.5:0/2724486318 conn(0x7f70240fc7d0 msgr2=0x7f70241056e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:49.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:49.168+0000 7f702b24c640 1 -- 172.21.15.5:0/2724486318 shutdown_connections 2024-09-12T02:58:49.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:49.168+0000 7f702b24c640 1 -- 172.21.15.5:0/2724486318 wait complete. 2024-09-12T02:58:49.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:58:48 smithi146 ceph-mon[35011]: pgmap v352: 97 pgs: 97 active+clean; 658 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-12T02:58:49.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:58:48 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/1441272720' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:58:49.510 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-12T02:58:50.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:58:49 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/2724486318' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:58:50.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:58:49 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/2724486318' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:58:50.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:58:49 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/2724486318' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:58:51.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:58:50 smithi148 ceph-mon[35090]: pgmap v353: 97 pgs: 97 active+clean; 658 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-12T02:58:51.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:58:50 smithi146 ceph-mon[35011]: pgmap v353: 97 pgs: 97 active+clean; 658 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-12T02:58:51.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:58:50 smithi005 ceph-mon[31197]: pgmap v353: 97 pgs: 97 active+clean; 658 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-12T02:58:52.491 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24502 quiesce db / --timeout 90 --expiration 144.5 --await-for 45.0' 2024-09-12T02:58:52.491 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24502 quiesce db / --timeout 90 --expiration 144.5 --await-for 45.0 2024-09-12T02:58:52.560 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:58:53.165 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:58:52 smithi148 ceph-mon[35090]: pgmap v354: 97 pgs: 97 active+clean; 658 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T02:58:53.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:58:52 smithi146 ceph-mon[35011]: pgmap v354: 97 pgs: 97 active+clean; 658 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T02:58:53.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:58:52 smithi005 ceph-mon[31197]: pgmap v354: 97 pgs: 97 active+clean; 658 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T02:58:54.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.895+0000 7f6688138640 1 -- 172.21.15.5:0/1339717693 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6680102570 msgr2=0x7f66801029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:54.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.895+0000 7f6688138640 1 --2- 172.21.15.5:0/1339717693 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6680102570 0x7f66801029b0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f6670009900 tx=0x7f667002a480 comp rx=0 tx=0).stop 2024-09-12T02:58:54.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.895+0000 7f6688138640 1 -- 172.21.15.5:0/1339717693 shutdown_connections 2024-09-12T02:58:54.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.895+0000 7f6688138640 1 --2- 172.21.15.5:0/1339717693 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6680102570 0x7f66801029b0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:54.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.895+0000 7f6688138640 1 --2- 172.21.15.5:0/1339717693 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6680102ef0 0x7f668010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:54.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.895+0000 7f6688138640 1 --2- 172.21.15.5:0/1339717693 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6680100e20 0x7f6680101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:54.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.895+0000 7f6688138640 1 -- 172.21.15.5:0/1339717693 >> 172.21.15.5:0/1339717693 conn(0x7f66800fc7d0 msgr2=0x7f66800febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:54.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.895+0000 7f6688138640 1 -- 172.21.15.5:0/1339717693 shutdown_connections 2024-09-12T02:58:54.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.895+0000 7f6688138640 1 -- 172.21.15.5:0/1339717693 wait complete. 2024-09-12T02:58:54.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.896+0000 7f6688138640 1 Processor -- start 2024-09-12T02:58:54.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.896+0000 7f6688138640 1 -- start start 2024-09-12T02:58:54.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.896+0000 7f6688138640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6680100e20 0x7f668010e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:54.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.896+0000 7f6688138640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6680102570 0x7f668010ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:54.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.896+0000 7f6688138640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6680102ef0 0x7f668010f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:54.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.897+0000 7f6688138640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6680112040 con 0x7f6680102ef0 2024-09-12T02:58:54.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.897+0000 7f6688138640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f6680111ec0 con 0x7f6680100e20 2024-09-12T02:58:54.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.897+0000 7f6688138640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f66801121c0 con 0x7f6680102570 2024-09-12T02:58:54.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.897+0000 7f66866d5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6680102ef0 0x7f668010f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:54.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.897+0000 7f6685ed4640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6680100e20 0x7f668010e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:54.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.897+0000 7f6685ed4640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6680100e20 0x7f668010e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.5:52666/0 (socket says 172.21.15.5:52666) 2024-09-12T02:58:54.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.897+0000 7f6685ed4640 1 -- 172.21.15.5:0/1054794798 learned_addr learned my addr 172.21.15.5:0/1054794798 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:58:54.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.897+0000 7f66856d3640 1 --2- 172.21.15.5:0/1054794798 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6680102570 0x7f668010ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:54.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.897+0000 7f66866d5640 1 -- 172.21.15.5:0/1054794798 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6680102570 msgr2=0x7f668010ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:54.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.897+0000 7f66866d5640 1 --2- 172.21.15.5:0/1054794798 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6680102570 0x7f668010ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:54.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.897+0000 7f66866d5640 1 -- 172.21.15.5:0/1054794798 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6680100e20 msgr2=0x7f668010e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:54.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.897+0000 7f66866d5640 1 --2- 172.21.15.5:0/1054794798 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6680100e20 0x7f668010e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:54.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.897+0000 7f66866d5640 1 -- 172.21.15.5:0/1054794798 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66801a6980 con 0x7f6680102ef0 2024-09-12T02:58:54.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.897+0000 7f66866d5640 1 --2- 172.21.15.5:0/1054794798 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6680102ef0 0x7f668010f180 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f667800e940 tx=0x7f667800ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:54.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.898+0000 7f6676ffd640 1 -- 172.21.15.5:0/1054794798 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6678004d20 con 0x7f6680102ef0 2024-09-12T02:58:54.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.898+0000 7f6676ffd640 1 -- 172.21.15.5:0/1054794798 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6678005020 con 0x7f6680102ef0 2024-09-12T02:58:54.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.898+0000 7f6676ffd640 1 -- 172.21.15.5:0/1054794798 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66780116b0 con 0x7f6680102ef0 2024-09-12T02:58:54.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.898+0000 7f6688138640 1 -- 172.21.15.5:0/1054794798 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66801a6c70 con 0x7f6680102ef0 2024-09-12T02:58:54.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.898+0000 7f6688138640 1 -- 172.21.15.5:0/1054794798 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66801a7130 con 0x7f6680102ef0 2024-09-12T02:58:54.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.901+0000 7f668a511740 1 Processor -- start 2024-09-12T02:58:54.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.902+0000 7f6676ffd640 1 -- 172.21.15.5:0/1054794798 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7f66780054d0 con 0x7f6680102ef0 2024-09-12T02:58:54.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.902+0000 7f668a511740 1 -- start start 2024-09-12T02:58:54.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.902+0000 7f668a511740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556cc26d1b40 0x556cc26d0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:54.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.902+0000 7f668a511740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556cc26d05d0 0x556cc26d0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:54.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.902+0000 7f668a511740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cc26db370 0x556cc26dd760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:54.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.902+0000 7f668a511740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556cc26362f0 con 0x556cc26db370 2024-09-12T02:58:54.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.902+0000 7f668a511740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x556cc25ddcf0 con 0x556cc26d1b40 2024-09-12T02:58:54.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.902+0000 7f668a511740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x556cc26322b0 con 0x556cc26d05d0 2024-09-12T02:58:54.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.902+0000 7f66856d3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cc26db370 0x556cc26dd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:54.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.903+0000 7f66856d3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cc26db370 0x556cc26dd760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48742/0 (socket says 172.21.15.5:48742) 2024-09-12T02:58:54.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.903+0000 7f66856d3640 1 -- 172.21.15.5:0/2685277285 learned_addr learned my addr 172.21.15.5:0/2685277285 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:58:54.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.903+0000 7f6685ed4640 1 --2- 172.21.15.5:0/2685277285 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556cc26d05d0 0x556cc26d0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:54.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.903+0000 7f6676ffd640 1 --2- 172.21.15.5:0/1054794798 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f666c0a2c90 0x7f666c0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:54.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.903+0000 7f66856d3640 1 --2- 172.21.15.5:0/2685277285 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556cc26d1b40 0x556cc26d0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:54.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.903+0000 7f6676ffd640 1 -- 172.21.15.5:0/1054794798 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f66780cf170 con 0x7f6680102ef0 2024-09-12T02:58:54.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.903+0000 7f6685ed4640 1 -- 172.21.15.5:0/2685277285 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556cc26d1b40 msgr2=0x556cc26d0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:54.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.903+0000 7f6685ed4640 1 --2- 172.21.15.5:0/2685277285 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556cc26d1b40 0x556cc26d0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:54.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.903+0000 7f6685ed4640 1 -- 172.21.15.5:0/2685277285 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cc26db370 msgr2=0x556cc26dd760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:54.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.903+0000 7f6685ed4640 1 --2- 172.21.15.5:0/2685277285 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cc26db370 0x556cc26dd760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:54.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.903+0000 7f66866d5640 1 --2- 172.21.15.5:0/1054794798 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f666c0a2c90 0x7f666c0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:54.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.903+0000 7f6685ed4640 1 -- 172.21.15.5:0/2685277285 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556cc25c23a0 con 0x556cc26d05d0 2024-09-12T02:58:54.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.903+0000 7f6685ed4640 1 --2- 172.21.15.5:0/2685277285 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556cc26d05d0 0x556cc26d0a10 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f667c004590 tx=0x7f667c00bbb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9be52695d2fe54d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:54.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.904+0000 7f66866d5640 1 --2- 172.21.15.5:0/1054794798 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f666c0a2c90 0x7f666c0a5150 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f66780a3fc0 tx=0x7f6678005f50 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:54.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.904+0000 7f666bfff640 1 -- 172.21.15.5:0/2685277285 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f667c003bd0 con 0x556cc26d05d0 2024-09-12T02:58:54.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.904+0000 7f666bfff640 1 -- 172.21.15.5:0/2685277285 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f667c015070 con 0x556cc26d05d0 2024-09-12T02:58:54.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.904+0000 7f666bfff640 1 -- 172.21.15.5:0/2685277285 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f667c004e40 con 0x556cc26d05d0 2024-09-12T02:58:54.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.904+0000 7f668a511740 1 -- 172.21.15.5:0/2685277285 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556cc26d05d0 msgr2=0x556cc26d0a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:54.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.904+0000 7f668a511740 1 --2- 172.21.15.5:0/2685277285 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556cc26d05d0 0x556cc26d0a10 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f667c004590 tx=0x7f667c00bbb0 comp rx=0 tx=0).stop 2024-09-12T02:58:54.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.905+0000 7f668a511740 1 -- 172.21.15.5:0/2685277285 shutdown_connections 2024-09-12T02:58:54.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.905+0000 7f668a511740 1 --2- 172.21.15.5:0/2685277285 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cc26db370 0x556cc26dd760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:54.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.905+0000 7f668a511740 1 --2- 172.21.15.5:0/2685277285 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556cc26d05d0 0x556cc26d0a10 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:54.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.905+0000 7f668a511740 1 --2- 172.21.15.5:0/2685277285 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556cc26d1b40 0x556cc26d0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:54.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.905+0000 7f668a511740 1 -- 172.21.15.5:0/2685277285 >> 172.21.15.5:0/2685277285 conn(0x556cc2699310 msgr2=0x556cc26ccc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:54.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.905+0000 7f668a511740 1 -- 172.21.15.5:0/2685277285 shutdown_connections 2024-09-12T02:58:54.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.905+0000 7f668a511740 1 -- 172.21.15.5:0/2685277285 wait complete. 2024-09-12T02:58:54.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.905+0000 7f668a511740 1 -- start start 2024-09-12T02:58:54.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.906+0000 7f668a511740 1 Processor -- start 2024-09-12T02:58:54.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.906+0000 7f668a511740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556cc26d05d0 0x556cc26dad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:54.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.906+0000 7f668a511740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cc26d1b40 0x556cc26d7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:54.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.906+0000 7f668a511740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556cc26db370 0x556cc26d85d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:54.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.906+0000 7f668a511740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556cc26de070 con 0x556cc26d1b40 2024-09-12T02:58:54.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.906+0000 7f668a511740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x556cc25f1060 con 0x556cc26db370 2024-09-12T02:58:54.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.906+0000 7f668a511740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x556cc26de1f0 con 0x556cc26d05d0 2024-09-12T02:58:54.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.906+0000 7f668a511740 10 client.0 _ll_register_callbacks cb 0x556cc23448c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:58:54.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.906+0000 7f6685ed4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cc26d1b40 0x556cc26d7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:54.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.906+0000 7f6685ed4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cc26d1b40 0x556cc26d7fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48758/0 (socket says 172.21.15.5:48758) 2024-09-12T02:58:54.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.906+0000 7f6685ed4640 1 -- 172.21.15.5:0/3475695349 learned_addr learned my addr 172.21.15.5:0/3475695349 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:58:54.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.906+0000 7f66856d3640 1 --2- 172.21.15.5:0/3475695349 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556cc26d05d0 0x556cc26dad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:54.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.906+0000 7f66856d3640 1 --2- 172.21.15.5:0/3475695349 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556cc26db370 0x556cc26d85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:54.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.907+0000 7f6685ed4640 1 -- 172.21.15.5:0/3475695349 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556cc26d05d0 msgr2=0x556cc26dad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:54.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.907+0000 7f6685ed4640 1 --2- 172.21.15.5:0/3475695349 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556cc26d05d0 0x556cc26dad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:54.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.907+0000 7f6685ed4640 1 -- 172.21.15.5:0/3475695349 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556cc26db370 msgr2=0x556cc26d85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:54.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.907+0000 7f6685ed4640 1 --2- 172.21.15.5:0/3475695349 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556cc26db370 0x556cc26d85d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:54.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.907+0000 7f6685ed4640 1 -- 172.21.15.5:0/3475695349 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556cc23a73c0 con 0x556cc26d1b40 2024-09-12T02:58:54.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.907+0000 7f66856d3640 1 --2- 172.21.15.5:0/3475695349 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556cc26db370 0x556cc26d85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:58:54.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.907+0000 7f6685ed4640 1 --2- 172.21.15.5:0/3475695349 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cc26d1b40 0x556cc26d7fe0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f667c000c00 tx=0x7f667c004260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ddc1bde03e36aec server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:54.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.907+0000 7f66697fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T02:58:54.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.907+0000 7f66697fa640 1 -- 172.21.15.5:0/3475695349 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f667c028060 con 0x556cc26d1b40 2024-09-12T02:58:54.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.907+0000 7f66697fa640 1 -- 172.21.15.5:0/3475695349 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f667c004b80 con 0x556cc26d1b40 2024-09-12T02:58:54.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.907+0000 7f668a511740 1 -- 172.21.15.5:0/3475695349 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f667c015070 con 0x556cc26d1b40 2024-09-12T02:58:54.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.908+0000 7f66697fa640 1 -- 172.21.15.5:0/3475695349 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f667c004da0 con 0x556cc26d1b40 2024-09-12T02:58:54.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.908+0000 7f668a511740 10 client.15492 fetch_fsmap learned FSMap version 26 2024-09-12T02:58:54.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.908+0000 7f668a511740 1 -- 172.21.15.5:0/3475695349 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556cc26e9ad0 con 0x556cc26d1b40 2024-09-12T02:58:54.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.908+0000 7f66697fa640 1 -- 172.21.15.5:0/3475695349 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f667c015070 con 0x556cc26d1b40 2024-09-12T02:58:54.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.908+0000 7f66697fa640 1 -- 172.21.15.5:0/3475695349 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f667c022070 con 0x556cc26d1b40 2024-09-12T02:58:54.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.909+0000 7f668a511740 10 client.15492 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T02:58:54.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.909+0000 7f668a511740 10 client.15492 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-12T02:58:54.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.909+0000 7f668a511740 20 client.15492 populate_metadata read hostname 'smithi005' 2024-09-12T02:58:54.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.909+0000 7f668a511740 1 --2- 172.21.15.5:0/3475695349 >> [v2:172.21.15.146:6836/1673370937,v1:172.21.15.146:6837/1673370937] conn(0x556cc26eae20 0x556cc26ed210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:54.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.909+0000 7f668a511740 4 client.15492 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:58:54.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.909+0000 7f668a511740 1 -- 172.21.15.5:0/3475695349 --> [v2:172.21.15.146:6836/1673370937,v1:172.21.15.146:6837/1673370937] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556cc2643860 con 0x556cc26eae20 2024-09-12T02:58:54.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.909+0000 7f66856d3640 1 --2- 172.21.15.5:0/3475695349 >> [v2:172.21.15.146:6836/1673370937,v1:172.21.15.146:6837/1673370937] conn(0x556cc26eae20 0x556cc26ed210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:54.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.910+0000 7f66856d3640 1 --2- 172.21.15.5:0/3475695349 >> [v2:172.21.15.146:6836/1673370937,v1:172.21.15.146:6837/1673370937] conn(0x556cc26eae20 0x556cc26ed210 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8d07745194afb043 server_cookie=2b0cb144f0623c67 in_seq=0 out_seq=0 2024-09-12T02:58:54.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.910+0000 7f66697fa640 10 client.15492 ms_handle_connect on v2:172.21.15.146:6836/1673370937 2024-09-12T02:58:54.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.911+0000 7f66697fa640 1 -- 172.21.15.5:0/3475695349 <== mds.4 v2:172.21.15.146:6836/1673370937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556cc2643860 con 0x556cc26eae20 2024-09-12T02:58:54.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.911+0000 7f66697fa640 10 client.15492 handle_command_reply: tid=0 2024-09-12T02:58:54.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.911+0000 7f668a511740 1 client.15492 shutdown 2024-09-12T02:58:54.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.912+0000 7f668a511740 1 -- 172.21.15.5:0/3475695349 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cc26d1b40 msgr2=0x556cc26d7fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:54.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.912+0000 7f668a511740 1 --2- 172.21.15.5:0/3475695349 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cc26d1b40 0x556cc26d7fe0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f667c000c00 tx=0x7f667c004260 comp rx=0 tx=0).stop 2024-09-12T02:58:54.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.913+0000 7f668a511740 1 -- 172.21.15.5:0/3475695349 shutdown_connections 2024-09-12T02:58:54.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.913+0000 7f668a511740 1 --2- 172.21.15.5:0/3475695349 >> [v2:172.21.15.146:6836/1673370937,v1:172.21.15.146:6837/1673370937] conn(0x556cc26eae20 0x556cc26ed210 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:54.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.913+0000 7f668a511740 1 --2- 172.21.15.5:0/3475695349 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cc26d1b40 0x556cc26d7fe0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:54.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.913+0000 7f668a511740 1 --2- 172.21.15.5:0/3475695349 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556cc26db370 0x556cc26d85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:54.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.913+0000 7f668a511740 1 --2- 172.21.15.5:0/3475695349 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556cc26d05d0 0x556cc26dad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:54.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.913+0000 7f668a511740 1 -- 172.21.15.5:0/3475695349 >> 172.21.15.5:0/3475695349 conn(0x556cc2699310 msgr2=0x556cc26d2960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:54.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.913+0000 7f66697fa640 0 client.15492 ms_handle_reset on v2:172.21.15.146:6836/1673370937 2024-09-12T02:58:54.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.914+0000 7f668a511740 1 -- 172.21.15.5:0/3475695349 shutdown_connections 2024-09-12T02:58:54.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.914+0000 7f668a511740 1 -- 172.21.15.5:0/3475695349 wait complete. 2024-09-12T02:58:54.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.914+0000 7f668a511740 20 client.15492 trim_cache size 0 max 16384 2024-09-12T02:58:54.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.938+0000 7f668a511740 1 Processor -- start 2024-09-12T02:58:54.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.938+0000 7f668a511740 1 -- start start 2024-09-12T02:58:54.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.938+0000 7f668a511740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556cc26d05d0 0x556cc26e7310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:54.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.939+0000 7f668a511740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cc26d1b40 0x556cc26d3310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:54.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.939+0000 7f6685ed4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cc26d1b40 0x556cc26d3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:54.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.939+0000 7f6685ed4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cc26d1b40 0x556cc26d3310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48762/0 (socket says 172.21.15.5:48762) 2024-09-12T02:58:54.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.939+0000 7f6685ed4640 1 -- 172.21.15.5:0/3696388259 learned_addr learned my addr 172.21.15.5:0/3696388259 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:58:54.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.939+0000 7f668a511740 1 --2- 172.21.15.5:0/3696388259 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556cc26db370 0x556cc26d3850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:54.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.939+0000 7f66856d3640 1 --2- 172.21.15.5:0/3696388259 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556cc26d05d0 0x556cc26e7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:54.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.939+0000 7f668a511740 1 -- 172.21.15.5:0/3696388259 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f66801af7b0 con 0x556cc26d1b40 2024-09-12T02:58:54.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.939+0000 7f668a511740 1 -- 172.21.15.5:0/3696388259 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f66801b12f0 con 0x556cc26db370 2024-09-12T02:58:54.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.939+0000 7f668a511740 1 -- 172.21.15.5:0/3696388259 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f66801afe80 con 0x556cc26d05d0 2024-09-12T02:58:54.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.940+0000 7f6685ed4640 1 -- 172.21.15.5:0/3696388259 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556cc26d05d0 msgr2=0x556cc26e7310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:54.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.940+0000 7f6685ed4640 1 --2- 172.21.15.5:0/3696388259 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556cc26d05d0 0x556cc26e7310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:54.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.940+0000 7f6685ed4640 1 -- 172.21.15.5:0/3696388259 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556cc26db370 msgr2=0x556cc26d3850 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T02:58:54.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.940+0000 7f6685ed4640 1 --2- 172.21.15.5:0/3696388259 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556cc26db370 0x556cc26d3850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:54.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.940+0000 7f6685ed4640 1 -- 172.21.15.5:0/3696388259 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556cc26cced0 con 0x556cc26d1b40 2024-09-12T02:58:54.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.940+0000 7f6685ed4640 1 --2- 172.21.15.5:0/3696388259 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cc26d1b40 0x556cc26d3310 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f667c00b630 tx=0x7f667c011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fd8119745770534 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:54.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.941+0000 7f666b7fe640 1 -- 172.21.15.5:0/3696388259 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f667c028070 con 0x556cc26d1b40 2024-09-12T02:58:54.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.941+0000 7f666b7fe640 1 -- 172.21.15.5:0/3696388259 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f667c015030 con 0x556cc26d1b40 2024-09-12T02:58:54.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.941+0000 7f666b7fe640 1 -- 172.21.15.5:0/3696388259 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f667c032080 con 0x556cc26d1b40 2024-09-12T02:58:54.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.942+0000 7f668a511740 1 -- 172.21.15.5:0/3696388259 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cc26d1b40 msgr2=0x556cc26d3310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:54.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.942+0000 7f668a511740 1 --2- 172.21.15.5:0/3696388259 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cc26d1b40 0x556cc26d3310 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f667c00b630 tx=0x7f667c011e90 comp rx=0 tx=0).stop 2024-09-12T02:58:54.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.942+0000 7f668a511740 1 -- 172.21.15.5:0/3696388259 shutdown_connections 2024-09-12T02:58:54.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.942+0000 7f668a511740 1 --2- 172.21.15.5:0/3696388259 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cc26d1b40 0x556cc26d3310 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:54.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.942+0000 7f668a511740 1 --2- 172.21.15.5:0/3696388259 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556cc26db370 0x556cc26d3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:54.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.942+0000 7f668a511740 1 --2- 172.21.15.5:0/3696388259 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556cc26d05d0 0x556cc26e7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:54.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.942+0000 7f668a511740 1 -- 172.21.15.5:0/3696388259 >> 172.21.15.5:0/3696388259 conn(0x556cc2699310 msgr2=0x556cc26dd370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:54.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.942+0000 7f668a511740 1 -- 172.21.15.5:0/3696388259 shutdown_connections 2024-09-12T02:58:54.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.943+0000 7f668a511740 1 -- 172.21.15.5:0/3696388259 wait complete. 2024-09-12T02:58:54.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.943+0000 7f668a511740 1 -- start start 2024-09-12T02:58:54.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.943+0000 7f668a511740 1 Processor -- start 2024-09-12T02:58:54.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.944+0000 7f668a511740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556cc26d05d0 0x556cc26dbed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:54.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.944+0000 7f668a511740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cc26d1b40 0x556cc26dc410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:54.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.944+0000 7f668a511740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556cc26db370 0x556cc26dc950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:54.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.944+0000 7f668a511740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f66801b0c20 con 0x556cc26d1b40 2024-09-12T02:58:54.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.944+0000 7f668a511740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f66801b0550 con 0x556cc26db370 2024-09-12T02:58:54.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.944+0000 7f668a511740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f66801ad790 con 0x556cc26d05d0 2024-09-12T02:58:54.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.944+0000 7f668a511740 10 client.0 _ll_register_callbacks cb 0x7f6670005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:58:54.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.944+0000 7f66856d3640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556cc26db370 0x556cc26dc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:54.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.944+0000 7f6685ed4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cc26d1b40 0x556cc26dc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:54.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.944+0000 7f6685ed4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cc26d1b40 0x556cc26dc410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48768/0 (socket says 172.21.15.5:48768) 2024-09-12T02:58:54.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.944+0000 7f66856d3640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556cc26db370 0x556cc26dc950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.5:52692/0 (socket says 172.21.15.5:52692) 2024-09-12T02:58:54.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.944+0000 7f6685ed4640 1 -- 172.21.15.5:0/285222269 learned_addr learned my addr 172.21.15.5:0/285222269 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:58:54.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.944+0000 7f66856d3640 1 --2- 172.21.15.5:0/285222269 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556cc26d05d0 0x556cc26dbed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:54.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.944+0000 7f66856d3640 1 -- 172.21.15.5:0/285222269 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556cc26d05d0 msgr2=0x556cc26dbed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:54.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.944+0000 7f66856d3640 1 --2- 172.21.15.5:0/285222269 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556cc26d05d0 0x556cc26dbed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:54.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.944+0000 7f66856d3640 1 -- 172.21.15.5:0/285222269 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cc26d1b40 msgr2=0x556cc26dc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:54.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.944+0000 7f66856d3640 1 --2- 172.21.15.5:0/285222269 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cc26d1b40 0x556cc26dc410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:54.949 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.944+0000 7f66856d3640 1 -- 172.21.15.5:0/285222269 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556cc26e26f0 con 0x556cc26db370 2024-09-12T02:58:54.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.945+0000 7f6685ed4640 1 --2- 172.21.15.5:0/285222269 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cc26d1b40 0x556cc26dc410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:58:54.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.945+0000 7f66856d3640 1 --2- 172.21.15.5:0/285222269 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556cc26db370 0x556cc26dc950 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6670031470 tx=0x7f6670040040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1caff9b257ded8c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:54.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.945+0000 7f66697fa640 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2024-09-12T02:58:54.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.945+0000 7f66697fa640 1 -- 172.21.15.5:0/285222269 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6670050030 con 0x556cc26db370 2024-09-12T02:58:54.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.945+0000 7f668a511740 1 -- 172.21.15.5:0/285222269 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f667c015030 con 0x556cc26db370 2024-09-12T02:58:54.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.945+0000 7f66697fa640 1 -- 172.21.15.5:0/285222269 <== mon.2 v2:172.21.15.146:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f6670003e40 con 0x556cc26db370 2024-09-12T02:58:54.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.945+0000 7f66697fa640 1 -- 172.21.15.5:0/285222269 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6670035040 con 0x556cc26db370 2024-09-12T02:58:54.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.945+0000 7f668a511740 10 client.25028 fetch_fsmap learned FSMap version 26 2024-09-12T02:58:54.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.945+0000 7f668a511740 1 -- 172.21.15.5:0/285222269 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556cc26e39e0 con 0x556cc26db370 2024-09-12T02:58:54.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.946+0000 7f66697fa640 1 -- 172.21.15.5:0/285222269 <== mon.2 v2:172.21.15.146:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f667c015030 con 0x556cc26db370 2024-09-12T02:58:54.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.946+0000 7f66697fa640 1 -- 172.21.15.5:0/285222269 <== mon.2 v2:172.21.15.146:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f6670056070 con 0x556cc26db370 2024-09-12T02:58:54.950 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.946+0000 7f668a511740 10 client.25028 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T02:58:54.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.946+0000 7f668a511740 10 client.25028 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-12T02:58:54.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.946+0000 7f668a511740 20 client.25028 populate_metadata read hostname 'smithi005' 2024-09-12T02:58:54.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.946+0000 7f668a511740 1 --2- 172.21.15.5:0/285222269 >> [v2:172.21.15.146:6836/1673370937,v1:172.21.15.146:6837/1673370937] conn(0x556cc275dfc0 0x556cc27603b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:54.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.946+0000 7f668a511740 4 client.25028 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:58:54.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.946+0000 7f668a511740 1 -- 172.21.15.5:0/285222269 --> [v2:172.21.15.146:6836/1673370937,v1:172.21.15.146:6837/1673370937] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556cc263dd70 con 0x556cc275dfc0 2024-09-12T02:58:54.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.946+0000 7f6685ed4640 1 --2- 172.21.15.5:0/285222269 >> [v2:172.21.15.146:6836/1673370937,v1:172.21.15.146:6837/1673370937] conn(0x556cc275dfc0 0x556cc27603b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:54.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.947+0000 7f6685ed4640 1 --2- 172.21.15.5:0/285222269 >> [v2:172.21.15.146:6836/1673370937,v1:172.21.15.146:6837/1673370937] conn(0x556cc275dfc0 0x556cc27603b0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c16007a509aefb8a server_cookie=cc80864661e04623 in_seq=0 out_seq=0 2024-09-12T02:58:54.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.947+0000 7f66697fa640 10 client.25028 ms_handle_connect on v2:172.21.15.146:6836/1673370937 2024-09-12T02:58:54.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.947+0000 7f66697fa640 1 -- 172.21.15.5:0/285222269 <== mds.4 v2:172.21.15.146:6836/1673370937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556cc263dd70 con 0x556cc275dfc0 2024-09-12T02:58:54.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.947+0000 7f66697fa640 10 client.25028 handle_command_reply: tid=0 2024-09-12T02:58:54.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.947+0000 7f668a511740 1 client.25028 shutdown 2024-09-12T02:58:54.951 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.947+0000 7f668a511740 1 -- 172.21.15.5:0/285222269 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556cc26db370 msgr2=0x556cc26dc950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:54.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.947+0000 7f668a511740 1 --2- 172.21.15.5:0/285222269 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556cc26db370 0x556cc26dc950 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6670031470 tx=0x7f6670040040 comp rx=0 tx=0).stop 2024-09-12T02:58:54.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.947+0000 7f668a511740 1 -- 172.21.15.5:0/285222269 shutdown_connections 2024-09-12T02:58:54.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.947+0000 7f668a511740 1 --2- 172.21.15.5:0/285222269 >> [v2:172.21.15.146:6836/1673370937,v1:172.21.15.146:6837/1673370937] conn(0x556cc275dfc0 0x556cc27603b0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:54.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.947+0000 7f668a511740 1 --2- 172.21.15.5:0/285222269 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556cc26d1b40 0x556cc26dc410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:54.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.947+0000 7f668a511740 1 --2- 172.21.15.5:0/285222269 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556cc26db370 0x556cc26dc950 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:54.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.947+0000 7f668a511740 1 --2- 172.21.15.5:0/285222269 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556cc26d05d0 0x556cc26dbed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:54.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.947+0000 7f668a511740 1 -- 172.21.15.5:0/285222269 >> 172.21.15.5:0/285222269 conn(0x556cc2699310 msgr2=0x556cc26d70d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:54.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.947+0000 7f66697fa640 0 client.25028 ms_handle_reset on v2:172.21.15.146:6836/1673370937 2024-09-12T02:58:54.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.947+0000 7f668a511740 1 -- 172.21.15.5:0/285222269 shutdown_connections 2024-09-12T02:58:54.952 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.947+0000 7f668a511740 1 -- 172.21.15.5:0/285222269 wait complete. 2024-09-12T02:58:54.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.948+0000 7f668a511740 20 client.25028 trim_cache size 0 max 16384 2024-09-12T02:58:54.953 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-12T02:58:54.953 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.949+0000 7f6688138640 1 -- 172.21.15.5:0/1054794798 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f666c0a2c90 msgr2=0x7f666c0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:54.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.949+0000 7f6688138640 1 --2- 172.21.15.5:0/1054794798 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f666c0a2c90 0x7f666c0a5150 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f66780a3fc0 tx=0x7f6678005f50 comp rx=0 tx=0).stop 2024-09-12T02:58:54.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.949+0000 7f6688138640 1 -- 172.21.15.5:0/1054794798 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6680102ef0 msgr2=0x7f668010f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:54.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.949+0000 7f6688138640 1 --2- 172.21.15.5:0/1054794798 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6680102ef0 0x7f668010f180 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f667800e940 tx=0x7f667800ee00 comp rx=0 tx=0).stop 2024-09-12T02:58:54.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.949+0000 7f6688138640 1 -- 172.21.15.5:0/1054794798 shutdown_connections 2024-09-12T02:58:54.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.949+0000 7f6688138640 1 --2- 172.21.15.5:0/1054794798 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f666c0a2c90 0x7f666c0a5150 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:54.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.949+0000 7f6688138640 1 --2- 172.21.15.5:0/1054794798 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6680102ef0 0x7f668010f180 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:54.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.949+0000 7f6688138640 1 --2- 172.21.15.5:0/1054794798 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6680102570 0x7f668010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:54.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.949+0000 7f6688138640 1 --2- 172.21.15.5:0/1054794798 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6680100e20 0x7f668010e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:54.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.949+0000 7f6688138640 1 -- 172.21.15.5:0/1054794798 >> 172.21.15.5:0/1054794798 conn(0x7f66800fc7d0 msgr2=0x7f66800febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:54.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.949+0000 7f6688138640 1 -- 172.21.15.5:0/1054794798 shutdown_connections 2024-09-12T02:58:54.954 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:54.950+0000 7f6688138640 1 -- 172.21.15.5:0/1054794798 wait complete. 2024-09-12T02:58:55.165 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:58:54 smithi148 ceph-mon[35090]: pgmap v355: 97 pgs: 97 active+clean; 658 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-12T02:58:55.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:58:54 smithi146 ceph-mon[35011]: pgmap v355: 97 pgs: 97 active+clean; 658 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-12T02:58:55.283 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:58:54 smithi005 ceph-mon[31197]: pgmap v355: 97 pgs: 97 active+clean; 658 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-12T02:58:55.651 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-12T02:58:55.685 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:58:56.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.177+0000 7f3d64208640 1 -- 172.21.15.5:0/2271199026 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d5c100aa0 msgr2=0x7f3d5c10ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:56.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.177+0000 7f3d64208640 1 --2- 172.21.15.5:0/2271199026 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d5c100aa0 0x7f3d5c10ea20 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f3d54009ca0 tx=0x7f3d5402a8c0 comp rx=0 tx=0).stop 2024-09-12T02:58:56.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.177+0000 7f3d64208640 1 -- 172.21.15.5:0/2271199026 shutdown_connections 2024-09-12T02:58:56.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.177+0000 7f3d64208640 1 --2- 172.21.15.5:0/2271199026 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d5c100aa0 0x7f3d5c10ea20 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:56.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.177+0000 7f3d64208640 1 --2- 172.21.15.5:0/2271199026 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3d5c100100 0x7f3d5c100560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:56.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.177+0000 7f3d64208640 1 --2- 172.21.15.5:0/2271199026 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d5c106b80 0x7f3d5c106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:56.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.177+0000 7f3d64208640 1 -- 172.21.15.5:0/2271199026 >> 172.21.15.5:0/2271199026 conn(0x7f3d5c077230 msgr2=0x7f3d5c0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:56.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.177+0000 7f3d64208640 1 -- 172.21.15.5:0/2271199026 shutdown_connections 2024-09-12T02:58:56.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.177+0000 7f3d64208640 1 -- 172.21.15.5:0/2271199026 wait complete. 2024-09-12T02:58:56.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.179+0000 7f3d64208640 1 Processor -- start 2024-09-12T02:58:56.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.179+0000 7f3d64208640 1 -- start start 2024-09-12T02:58:56.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.180+0000 7f3d64208640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3d5c100100 0x7f3d5c198250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:56.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.180+0000 7f3d64208640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d5c100aa0 0x7f3d5c198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:56.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.180+0000 7f3d61fa4640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3d5c100100 0x7f3d5c198250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:56.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.180+0000 7f3d617a3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d5c100aa0 0x7f3d5c198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:56.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.180+0000 7f3d617a3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d5c100aa0 0x7f3d5c198790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48774/0 (socket says 172.21.15.5:48774) 2024-09-12T02:58:56.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.180+0000 7f3d617a3640 1 -- 172.21.15.5:0/593014124 learned_addr learned my addr 172.21.15.5:0/593014124 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:58:56.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.180+0000 7f3d64208640 1 --2- 172.21.15.5:0/593014124 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d5c106b80 0x7f3d5c19cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:56.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.180+0000 7f3d64208640 1 -- 172.21.15.5:0/593014124 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3d5c112050 con 0x7f3d5c100aa0 2024-09-12T02:58:56.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.180+0000 7f3d64208640 1 -- 172.21.15.5:0/593014124 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f3d5c111ed0 con 0x7f3d5c100100 2024-09-12T02:58:56.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.180+0000 7f3d64208640 1 -- 172.21.15.5:0/593014124 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3d5c1121d0 con 0x7f3d5c106b80 2024-09-12T02:58:56.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.181+0000 7f3d627a5640 1 --2- 172.21.15.5:0/593014124 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d5c106b80 0x7f3d5c19cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:56.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.181+0000 7f3d617a3640 1 -- 172.21.15.5:0/593014124 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d5c106b80 msgr2=0x7f3d5c19cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:56.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.181+0000 7f3d617a3640 1 --2- 172.21.15.5:0/593014124 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d5c106b80 0x7f3d5c19cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:56.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.181+0000 7f3d617a3640 1 -- 172.21.15.5:0/593014124 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3d5c100100 msgr2=0x7f3d5c198250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:56.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.181+0000 7f3d617a3640 1 --2- 172.21.15.5:0/593014124 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3d5c100100 0x7f3d5c198250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:56.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.181+0000 7f3d617a3640 1 -- 172.21.15.5:0/593014124 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d5c19d200 con 0x7f3d5c100aa0 2024-09-12T02:58:56.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.181+0000 7f3d617a3640 1 --2- 172.21.15.5:0/593014124 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d5c100aa0 0x7f3d5c198790 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c00e9f0 tx=0x7f3d4c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:56.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.182+0000 7f3d52ffd640 1 -- 172.21.15.5:0/593014124 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d4c00cd80 con 0x7f3d5c100aa0 2024-09-12T02:58:56.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.182+0000 7f3d52ffd640 1 -- 172.21.15.5:0/593014124 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d4c010910 con 0x7f3d5c100aa0 2024-09-12T02:58:56.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.182+0000 7f3d52ffd640 1 -- 172.21.15.5:0/593014124 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d4c018980 con 0x7f3d5c100aa0 2024-09-12T02:58:56.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.182+0000 7f3d64208640 1 -- 172.21.15.5:0/593014124 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d5c19d4f0 con 0x7f3d5c100aa0 2024-09-12T02:58:56.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.182+0000 7f3d64208640 1 -- 172.21.15.5:0/593014124 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d5c1a4dd0 con 0x7f3d5c100aa0 2024-09-12T02:58:56.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.185+0000 7f3d665e1740 1 Processor -- start 2024-09-12T02:58:56.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.185+0000 7f3d665e1740 1 -- start start 2024-09-12T02:58:56.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.186+0000 7f3d665e1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0c07d09a0 0x55f0c07d0d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:56.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.186+0000 7f3d665e1740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f0c07d12c0 0x55f0c07cb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:56.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.186+0000 7f3d665e1740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f0c07cb960 0x55f0c07cbde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:56.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.186+0000 7f3d52ffd640 1 -- 172.21.15.5:0/593014124 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7f3d4c018b20 con 0x7f3d5c100aa0 2024-09-12T02:58:56.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.186+0000 7f3d665e1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f0c07272a0 con 0x55f0c07d09a0 2024-09-12T02:58:56.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.186+0000 7f3d665e1740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55f0c06ceb10 con 0x55f0c07d12c0 2024-09-12T02:58:56.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.186+0000 7f3d665e1740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55f0c0723260 con 0x55f0c07cb960 2024-09-12T02:58:56.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.186+0000 7f3d627a5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0c07d09a0 0x55f0c07d0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:56.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.186+0000 7f3d627a5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0c07d09a0 0x55f0c07d0d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48778/0 (socket says 172.21.15.5:48778) 2024-09-12T02:58:56.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.186+0000 7f3d627a5640 1 -- 172.21.15.5:0/939301622 learned_addr learned my addr 172.21.15.5:0/939301622 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:58:56.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.187+0000 7f3d627a5640 1 -- 172.21.15.5:0/939301622 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f0c07cb960 msgr2=0x55f0c07cbde0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T02:58:56.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.187+0000 7f3d627a5640 1 --2- 172.21.15.5:0/939301622 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f0c07cb960 0x55f0c07cbde0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:56.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.187+0000 7f3d627a5640 1 -- 172.21.15.5:0/939301622 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f0c07d12c0 msgr2=0x55f0c07cb2f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T02:58:56.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.187+0000 7f3d627a5640 1 --2- 172.21.15.5:0/939301622 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f0c07d12c0 0x55f0c07cb2f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:56.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.187+0000 7f3d627a5640 1 -- 172.21.15.5:0/939301622 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0c0498b90 con 0x55f0c07d09a0 2024-09-12T02:58:56.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.187+0000 7f3d627a5640 1 --2- 172.21.15.5:0/939301622 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0c07d09a0 0x55f0c07d0d80 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c1077c0 tx=0x7f3d5402f410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2586fba2ebf2d7b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:56.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.187+0000 7f3d52ffd640 1 --2- 172.21.15.5:0/593014124 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f3d480a2d60 0x7f3d480a5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:56.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.188+0000 7f3d52ffd640 1 -- 172.21.15.5:0/593014124 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f3d4c0cf550 con 0x7f3d5c100aa0 2024-09-12T02:58:56.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.188+0000 7f3d61fa4640 1 --2- 172.21.15.5:0/593014124 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f3d480a2d60 0x7f3d480a5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:56.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.188+0000 7f3d47fff640 1 -- 172.21.15.5:0/939301622 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d54031db0 con 0x55f0c07d09a0 2024-09-12T02:58:56.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.188+0000 7f3d47fff640 1 -- 172.21.15.5:0/939301622 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d54036030 con 0x55f0c07d09a0 2024-09-12T02:58:56.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.188+0000 7f3d47fff640 1 -- 172.21.15.5:0/939301622 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d540042a0 con 0x55f0c07d09a0 2024-09-12T02:58:56.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.189+0000 7f3d665e1740 1 -- 172.21.15.5:0/939301622 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0c07d09a0 msgr2=0x55f0c07d0d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:56.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.189+0000 7f3d665e1740 1 --2- 172.21.15.5:0/939301622 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0c07d09a0 0x55f0c07d0d80 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c1077c0 tx=0x7f3d5402f410 comp rx=0 tx=0).stop 2024-09-12T02:58:56.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.190+0000 7f3d61fa4640 1 --2- 172.21.15.5:0/593014124 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f3d480a2d60 0x7f3d480a5220 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f3d58005fd0 tx=0x7f3d58005850 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:56.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.190+0000 7f3d665e1740 1 -- 172.21.15.5:0/939301622 shutdown_connections 2024-09-12T02:58:56.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.190+0000 7f3d665e1740 1 --2- 172.21.15.5:0/939301622 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f0c07cb960 0x55f0c07cbde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:56.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.190+0000 7f3d665e1740 1 --2- 172.21.15.5:0/939301622 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f0c07d12c0 0x55f0c07cb2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:56.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.190+0000 7f3d665e1740 1 --2- 172.21.15.5:0/939301622 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0c07d09a0 0x55f0c07d0d80 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:56.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.190+0000 7f3d665e1740 1 -- 172.21.15.5:0/939301622 >> 172.21.15.5:0/939301622 conn(0x55f0c07bd7a0 msgr2=0x55f0c07bdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:56.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.190+0000 7f3d665e1740 1 -- 172.21.15.5:0/939301622 shutdown_connections 2024-09-12T02:58:56.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.190+0000 7f3d665e1740 1 -- 172.21.15.5:0/939301622 wait complete. 2024-09-12T02:58:56.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.190+0000 7f3d665e1740 1 -- start start 2024-09-12T02:58:56.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.191+0000 7f3d665e1740 1 Processor -- start 2024-09-12T02:58:56.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.191+0000 7f3d665e1740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f0c07cb960 0x55f0c07d4850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:56.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.192+0000 7f3d665e1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0c07d09a0 0x55f0c07d4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:56.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.192+0000 7f3d665e1740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f0c07d12c0 0x55f0c07d52d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:56.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.192+0000 7f3d665e1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55f0c07e8b60 con 0x55f0c07d09a0 2024-09-12T02:58:56.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.192+0000 7f3d665e1740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55f0c06af6e0 con 0x55f0c07cb960 2024-09-12T02:58:56.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.192+0000 7f3d665e1740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55f0c07e8ce0 con 0x55f0c07d12c0 2024-09-12T02:58:56.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.192+0000 7f3d665e1740 10 client.0 _ll_register_callbacks cb 0x55f0c06d3a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:58:56.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.192+0000 7f3d61fa4640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f0c07cb960 0x55f0c07d4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:56.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.192+0000 7f3d617a3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0c07d09a0 0x55f0c07d4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:56.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.192+0000 7f3d617a3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0c07d09a0 0x55f0c07d4d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48780/0 (socket says 172.21.15.5:48780) 2024-09-12T02:58:56.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.192+0000 7f3d617a3640 1 -- 172.21.15.5:0/2520231179 learned_addr learned my addr 172.21.15.5:0/2520231179 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:58:56.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.192+0000 7f3d627a5640 1 --2- 172.21.15.5:0/2520231179 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f0c07d12c0 0x55f0c07d52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:56.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.193+0000 7f3d617a3640 1 -- 172.21.15.5:0/2520231179 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f0c07d12c0 msgr2=0x55f0c07d52d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:56.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.193+0000 7f3d617a3640 1 --2- 172.21.15.5:0/2520231179 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f0c07d12c0 0x55f0c07d52d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:56.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.193+0000 7f3d617a3640 1 -- 172.21.15.5:0/2520231179 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f0c07cb960 msgr2=0x55f0c07d4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:56.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.193+0000 7f3d617a3640 1 --2- 172.21.15.5:0/2520231179 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f0c07cb960 0x55f0c07d4850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:56.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.193+0000 7f3d617a3640 1 -- 172.21.15.5:0/2520231179 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0c07f21b0 con 0x55f0c07d09a0 2024-09-12T02:58:56.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.193+0000 7f3d617a3640 1 --2- 172.21.15.5:0/2520231179 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0c07d09a0 0x55f0c07d4d90 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c00c6a0 tx=0x7f3d4c09b650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc8746a4384367a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:56.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.193+0000 7f3d457fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T02:58:56.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.193+0000 7f3d457fa640 1 -- 172.21.15.5:0/2520231179 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d4c0a4870 con 0x55f0c07d09a0 2024-09-12T02:58:56.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.193+0000 7f3d457fa640 1 -- 172.21.15.5:0/2520231179 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f3d4c0cf0f0 con 0x55f0c07d09a0 2024-09-12T02:58:56.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.193+0000 7f3d665e1740 1 -- 172.21.15.5:0/2520231179 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d54036030 con 0x55f0c07d09a0 2024-09-12T02:58:56.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.193+0000 7f3d457fa640 1 -- 172.21.15.5:0/2520231179 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3d4c0cf310 con 0x55f0c07d09a0 2024-09-12T02:58:56.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.193+0000 7f3d665e1740 10 client.15510 fetch_fsmap learned FSMap version 26 2024-09-12T02:58:56.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.193+0000 7f3d665e1740 1 -- 172.21.15.5:0/2520231179 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f0c07f4b40 con 0x55f0c07d09a0 2024-09-12T02:58:56.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.194+0000 7f3d457fa640 1 -- 172.21.15.5:0/2520231179 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d54036030 con 0x55f0c07d09a0 2024-09-12T02:58:56.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.194+0000 7f3d457fa640 1 -- 172.21.15.5:0/2520231179 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f3d4c097a70 con 0x55f0c07d09a0 2024-09-12T02:58:56.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.194+0000 7f3d665e1740 10 client.15510 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T02:58:56.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.194+0000 7f3d665e1740 10 client.15510 resolve_mds: validated gid 24502 aka daemon mds.i 2024-09-12T02:58:56.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.194+0000 7f3d665e1740 20 client.15510 populate_metadata read hostname 'smithi005' 2024-09-12T02:58:56.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.194+0000 7f3d665e1740 1 --2- 172.21.15.5:0/2520231179 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55f0c07f5f50 0x55f0c07f8340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:56.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.194+0000 7f3d665e1740 4 client.15510 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:58:56.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.194+0000 7f3d665e1740 1 -- 172.21.15.5:0/2520231179 --> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f0c07349d0 con 0x55f0c07f5f50 2024-09-12T02:58:56.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.194+0000 7f3d61fa4640 1 --2- 172.21.15.5:0/2520231179 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55f0c07f5f50 0x55f0c07f8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:56.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.195+0000 7f3d61fa4640 1 --2- 172.21.15.5:0/2520231179 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55f0c07f5f50 0x55f0c07f8340 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f8c530ead127a6a server_cookie=a260b9a66efb639 in_seq=0 out_seq=0 2024-09-12T02:58:56.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.195+0000 7f3d457fa640 10 client.15510 ms_handle_connect on v2:172.21.15.148:6838/2317600000 2024-09-12T02:58:56.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.196+0000 7f3d457fa640 1 -- 172.21.15.5:0/2520231179 <== mds.0 v2:172.21.15.148:6838/2317600000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f0c07349d0 con 0x55f0c07f5f50 2024-09-12T02:58:56.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.196+0000 7f3d457fa640 10 client.15510 handle_command_reply: tid=0 2024-09-12T02:58:56.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.196+0000 7f3d665e1740 1 client.15510 shutdown 2024-09-12T02:58:56.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.196+0000 7f3d665e1740 1 -- 172.21.15.5:0/2520231179 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0c07d09a0 msgr2=0x55f0c07d4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:56.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.196+0000 7f3d665e1740 1 --2- 172.21.15.5:0/2520231179 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0c07d09a0 0x55f0c07d4d90 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c00c6a0 tx=0x7f3d4c09b650 comp rx=0 tx=0).stop 2024-09-12T02:58:56.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.196+0000 7f3d665e1740 1 -- 172.21.15.5:0/2520231179 shutdown_connections 2024-09-12T02:58:56.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.196+0000 7f3d665e1740 1 --2- 172.21.15.5:0/2520231179 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55f0c07f5f50 0x55f0c07f8340 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:56.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.196+0000 7f3d665e1740 1 --2- 172.21.15.5:0/2520231179 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0c07d09a0 0x55f0c07d4d90 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:56.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.196+0000 7f3d665e1740 1 --2- 172.21.15.5:0/2520231179 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f0c07d12c0 0x55f0c07d52d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:56.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.196+0000 7f3d665e1740 1 --2- 172.21.15.5:0/2520231179 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f0c07cb960 0x55f0c07d4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:56.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.196+0000 7f3d665e1740 1 -- 172.21.15.5:0/2520231179 >> 172.21.15.5:0/2520231179 conn(0x55f0c07bd7a0 msgr2=0x55f0c07bce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:56.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.196+0000 7f3d457fa640 0 client.15510 ms_handle_reset on v2:172.21.15.148:6838/2317600000 2024-09-12T02:58:56.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.196+0000 7f3d665e1740 1 -- 172.21.15.5:0/2520231179 shutdown_connections 2024-09-12T02:58:56.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.196+0000 7f3d665e1740 1 -- 172.21.15.5:0/2520231179 wait complete. 2024-09-12T02:58:56.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.196+0000 7f3d665e1740 20 client.15510 trim_cache size 0 max 16384 2024-09-12T02:58:56.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.224+0000 7f3d665e1740 1 Processor -- start 2024-09-12T02:58:56.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.224+0000 7f3d665e1740 1 -- start start 2024-09-12T02:58:56.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.225+0000 7f3d665e1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0c07cb960 0x55f0c07d0260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:56.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.225+0000 7f3d61fa4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0c07cb960 0x55f0c07d0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:56.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.225+0000 7f3d61fa4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0c07cb960 0x55f0c07d0260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48786/0 (socket says 172.21.15.5:48786) 2024-09-12T02:58:56.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.225+0000 7f3d61fa4640 1 -- 172.21.15.5:0/3608166866 learned_addr learned my addr 172.21.15.5:0/3608166866 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:58:56.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.225+0000 7f3d665e1740 1 --2- 172.21.15.5:0/3608166866 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f0c07d09a0 0x55f0c07cb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:56.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.225+0000 7f3d665e1740 1 --2- 172.21.15.5:0/3608166866 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f0c07d12c0 0x55f0c07d5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:56.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.225+0000 7f3d665e1740 1 -- 172.21.15.5:0/3608166866 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3d5c071fb0 con 0x55f0c07cb960 2024-09-12T02:58:56.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.225+0000 7f3d665e1740 1 -- 172.21.15.5:0/3608166866 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f3d5c073af0 con 0x55f0c07d12c0 2024-09-12T02:58:56.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.225+0000 7f3d665e1740 1 -- 172.21.15.5:0/3608166866 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3d5c072680 con 0x55f0c07d09a0 2024-09-12T02:58:56.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.226+0000 7f3d627a5640 1 --2- 172.21.15.5:0/3608166866 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f0c07d12c0 0x55f0c07d5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:56.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.226+0000 7f3d617a3640 1 --2- 172.21.15.5:0/3608166866 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f0c07d09a0 0x55f0c07cb2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:56.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.226+0000 7f3d61fa4640 1 -- 172.21.15.5:0/3608166866 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f0c07d09a0 msgr2=0x55f0c07cb2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:56.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.226+0000 7f3d61fa4640 1 --2- 172.21.15.5:0/3608166866 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f0c07d09a0 0x55f0c07cb2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:56.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.226+0000 7f3d61fa4640 1 -- 172.21.15.5:0/3608166866 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f0c07d12c0 msgr2=0x55f0c07d5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:56.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.226+0000 7f3d61fa4640 1 --2- 172.21.15.5:0/3608166866 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f0c07d12c0 0x55f0c07d5120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:56.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.226+0000 7f3d61fa4640 1 -- 172.21.15.5:0/3608166866 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0c07b8110 con 0x55f0c07cb960 2024-09-12T02:58:56.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.226+0000 7f3d61fa4640 1 --2- 172.21.15.5:0/3608166866 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0c07cb960 0x55f0c07d0260 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x55f0c07beeb0 tx=0x7f3d58015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2640baee2e5f00b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:56.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.227+0000 7f3d477fe640 1 -- 172.21.15.5:0/3608166866 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d580285b0 con 0x55f0c07cb960 2024-09-12T02:58:56.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.227+0000 7f3d477fe640 1 -- 172.21.15.5:0/3608166866 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d5802e070 con 0x55f0c07cb960 2024-09-12T02:58:56.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.227+0000 7f3d477fe640 1 -- 172.21.15.5:0/3608166866 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d58011b90 con 0x55f0c07cb960 2024-09-12T02:58:56.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.228+0000 7f3d665e1740 1 -- 172.21.15.5:0/3608166866 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0c07cb960 msgr2=0x55f0c07d0260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:56.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.228+0000 7f3d665e1740 1 --2- 172.21.15.5:0/3608166866 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0c07cb960 0x55f0c07d0260 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x55f0c07beeb0 tx=0x7f3d58015000 comp rx=0 tx=0).stop 2024-09-12T02:58:56.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.228+0000 7f3d665e1740 1 -- 172.21.15.5:0/3608166866 shutdown_connections 2024-09-12T02:58:56.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.228+0000 7f3d665e1740 1 --2- 172.21.15.5:0/3608166866 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f0c07d12c0 0x55f0c07d5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:56.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.228+0000 7f3d665e1740 1 --2- 172.21.15.5:0/3608166866 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f0c07d09a0 0x55f0c07cb2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:56.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.228+0000 7f3d665e1740 1 --2- 172.21.15.5:0/3608166866 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0c07cb960 0x55f0c07d0260 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:56.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.228+0000 7f3d665e1740 1 -- 172.21.15.5:0/3608166866 >> 172.21.15.5:0/3608166866 conn(0x55f0c07bd7a0 msgr2=0x55f0c07f0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:56.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.228+0000 7f3d665e1740 1 -- 172.21.15.5:0/3608166866 shutdown_connections 2024-09-12T02:58:56.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.228+0000 7f3d665e1740 1 -- 172.21.15.5:0/3608166866 wait complete. 2024-09-12T02:58:56.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.229+0000 7f3d665e1740 1 -- start start 2024-09-12T02:58:56.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.229+0000 7f3d665e1740 1 Processor -- start 2024-09-12T02:58:56.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.229+0000 7f3d665e1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0c07cb960 0x55f0c07edf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:56.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.229+0000 7f3d665e1740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f0c07d09a0 0x55f0c07ee4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:56.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.229+0000 7f3d665e1740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f0c07d12c0 0x55f0c07eaf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:56.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.229+0000 7f3d665e1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3d5c073420 con 0x55f0c07cb960 2024-09-12T02:58:56.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.229+0000 7f3d665e1740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f3d5c072d50 con 0x55f0c07d09a0 2024-09-12T02:58:56.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.229+0000 7f3d665e1740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3d5c06ff90 con 0x55f0c07d12c0 2024-09-12T02:58:56.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.229+0000 7f3d665e1740 10 client.0 _ll_register_callbacks cb 0x7f3d4c09b7d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:58:56.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.230+0000 7f3d61fa4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0c07cb960 0x55f0c07edf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:56.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.230+0000 7f3d61fa4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0c07cb960 0x55f0c07edf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48790/0 (socket says 172.21.15.5:48790) 2024-09-12T02:58:56.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.230+0000 7f3d61fa4640 1 -- 172.21.15.5:0/2207905310 learned_addr learned my addr 172.21.15.5:0/2207905310 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:58:56.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.230+0000 7f3d627a5640 1 --2- 172.21.15.5:0/2207905310 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f0c07d12c0 0x55f0c07eaf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:56.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.230+0000 7f3d61fa4640 1 -- 172.21.15.5:0/2207905310 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f0c07d12c0 msgr2=0x55f0c07eaf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:56.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.230+0000 7f3d617a3640 1 --2- 172.21.15.5:0/2207905310 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f0c07d09a0 0x55f0c07ee4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:56.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.230+0000 7f3d61fa4640 1 --2- 172.21.15.5:0/2207905310 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f0c07d12c0 0x55f0c07eaf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:56.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.230+0000 7f3d61fa4640 1 -- 172.21.15.5:0/2207905310 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f0c07d09a0 msgr2=0x55f0c07ee4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:56.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.230+0000 7f3d61fa4640 1 --2- 172.21.15.5:0/2207905310 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f0c07d09a0 0x55f0c07ee4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:56.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.230+0000 7f3d61fa4640 1 -- 172.21.15.5:0/2207905310 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0c07b9260 con 0x55f0c07cb960 2024-09-12T02:58:56.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.230+0000 7f3d61fa4640 1 --2- 172.21.15.5:0/2207905310 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0c07cb960 0x55f0c07edf70 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f3d580114f0 tx=0x7f3d58012b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62d73da536c4ee53 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:56.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.230+0000 7f3d457fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T02:58:56.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.231+0000 7f3d457fa640 1 -- 172.21.15.5:0/2207905310 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d58038070 con 0x55f0c07cb960 2024-09-12T02:58:56.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.231+0000 7f3d457fa640 1 -- 172.21.15.5:0/2207905310 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f3d58005040 con 0x55f0c07cb960 2024-09-12T02:58:56.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.231+0000 7f3d665e1740 1 -- 172.21.15.5:0/2207905310 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d5802e070 con 0x55f0c07cb960 2024-09-12T02:58:56.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.231+0000 7f3d457fa640 1 -- 172.21.15.5:0/2207905310 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3d58041050 con 0x55f0c07cb960 2024-09-12T02:58:56.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.231+0000 7f3d665e1740 10 client.15516 fetch_fsmap learned FSMap version 26 2024-09-12T02:58:56.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.231+0000 7f3d665e1740 1 -- 172.21.15.5:0/2207905310 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f0c08611c0 con 0x55f0c07cb960 2024-09-12T02:58:56.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.231+0000 7f3d457fa640 1 -- 172.21.15.5:0/2207905310 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3d5802e070 con 0x55f0c07cb960 2024-09-12T02:58:56.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.232+0000 7f3d457fa640 1 -- 172.21.15.5:0/2207905310 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f3d580283b0 con 0x55f0c07cb960 2024-09-12T02:58:56.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.233+0000 7f3d665e1740 10 client.15516 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T02:58:56.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.233+0000 7f3d665e1740 10 client.15516 resolve_mds: validated gid 24502 aka daemon mds.i 2024-09-12T02:58:56.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.233+0000 7f3d665e1740 20 client.15516 populate_metadata read hostname 'smithi005' 2024-09-12T02:58:56.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.233+0000 7f3d665e1740 1 --2- 172.21.15.5:0/2207905310 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55f0c0861850 0x55f0c0863c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:56.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.233+0000 7f3d665e1740 4 client.15516 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.5, "await_for": 45.0}] 2024-09-12T02:58:56.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.233+0000 7f3d665e1740 1 -- 172.21.15.5:0/2207905310 --> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.5, "await_for": 45.0}) -- 0x55f0c072ed20 con 0x55f0c0861850 2024-09-12T02:58:56.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.233+0000 7f3d617a3640 1 --2- 172.21.15.5:0/2207905310 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55f0c0861850 0x55f0c0863c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:56.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.233+0000 7f3d617a3640 1 --2- 172.21.15.5:0/2207905310 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55f0c0861850 0x55f0c0863c70 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=730b97cd30098682 server_cookie=5d84132d1dff3a2d in_seq=0 out_seq=0 2024-09-12T02:58:56.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.233+0000 7f3d457fa640 10 client.15516 ms_handle_connect on v2:172.21.15.148:6838/2317600000 2024-09-12T02:58:56.237 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:58:56.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.249+0000 7f3d457fa640 1 -- 172.21.15.5:0/2207905310 <== mds.0 v2:172.21.15.148:6838/2317600000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x55f0c072ed20 con 0x55f0c0861850 2024-09-12T02:58:56.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.249+0000 7f3d457fa640 10 client.15516 handle_command_reply: tid=0 2024-09-12T02:58:56.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.249+0000 7f3d665e1740 1 client.15516 shutdown 2024-09-12T02:58:56.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.249+0000 7f3d665e1740 1 -- 172.21.15.5:0/2207905310 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0c07cb960 msgr2=0x55f0c07edf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:56.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.249+0000 7f3d665e1740 1 --2- 172.21.15.5:0/2207905310 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0c07cb960 0x55f0c07edf70 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f3d580114f0 tx=0x7f3d58012b50 comp rx=0 tx=0).stop 2024-09-12T02:58:56.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.249+0000 7f3d665e1740 1 -- 172.21.15.5:0/2207905310 shutdown_connections 2024-09-12T02:58:56.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.249+0000 7f3d665e1740 1 --2- 172.21.15.5:0/2207905310 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55f0c07d12c0 0x55f0c07eaf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:56.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.249+0000 7f3d665e1740 1 --2- 172.21.15.5:0/2207905310 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55f0c07d09a0 0x55f0c07ee4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:56.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.249+0000 7f3d665e1740 1 --2- 172.21.15.5:0/2207905310 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55f0c0861850 0x55f0c0863c70 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:56.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.249+0000 7f3d665e1740 1 --2- 172.21.15.5:0/2207905310 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0c07cb960 0x55f0c07edf70 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:56.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.249+0000 7f3d665e1740 1 -- 172.21.15.5:0/2207905310 >> 172.21.15.5:0/2207905310 conn(0x55f0c07bd7a0 msgr2=0x55f0c07f1610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:56.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.249+0000 7f3d457fa640 0 client.15516 ms_handle_reset on v2:172.21.15.148:6838/2317600000 2024-09-12T02:58:56.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.250+0000 7f3d665e1740 1 -- 172.21.15.5:0/2207905310 shutdown_connections 2024-09-12T02:58:56.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.250+0000 7f3d665e1740 1 -- 172.21.15.5:0/2207905310 wait complete. 2024-09-12T02:58:56.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.250+0000 7f3d665e1740 20 client.15516 trim_cache size 0 max 16384 2024-09-12T02:58:56.254 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-12T02:58:56.254 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 26, 2024-09-12T02:58:56.254 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24502, 2024-09-12T02:58:56.254 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 2, 2024-09-12T02:58:56.255 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-09-12T02:58:56.255 INFO:teuthology.orchestra.run.smithi005.stdout: "26c61f34": { 2024-09-12T02:58:56.255 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 2, 2024-09-12T02:58:56.255 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-09-12T02:58:56.255 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-12T02:58:56.255 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-09-12T02:58:56.256 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-12T02:58:56.256 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T02:58:56.256 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 90.0, 2024-09-12T02:58:56.256 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 144.5, 2024-09-12T02:58:56.256 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-09-12T02:58:56.256 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-09-12T02:58:56.256 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-09-12T02:58:56.256 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-12T02:58:56.257 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-09-12T02:58:56.257 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-12T02:58:56.257 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T02:58:56.257 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T02:58:56.257 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T02:58:56.257 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T02:58:56.257 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T02:58:56.258 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-12T02:58:56.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.250+0000 7f3d64208640 1 -- 172.21.15.5:0/593014124 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f3d480a2d60 msgr2=0x7f3d480a5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:56.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.250+0000 7f3d64208640 1 --2- 172.21.15.5:0/593014124 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f3d480a2d60 0x7f3d480a5220 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f3d58005fd0 tx=0x7f3d58005850 comp rx=0 tx=0).stop 2024-09-12T02:58:56.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.251+0000 7f3d64208640 1 -- 172.21.15.5:0/593014124 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d5c100aa0 msgr2=0x7f3d5c198790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:56.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.251+0000 7f3d64208640 1 --2- 172.21.15.5:0/593014124 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d5c100aa0 0x7f3d5c198790 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c00e9f0 tx=0x7f3d4c00eeb0 comp rx=0 tx=0).stop 2024-09-12T02:58:56.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.251+0000 7f3d64208640 1 -- 172.21.15.5:0/593014124 shutdown_connections 2024-09-12T02:58:56.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.251+0000 7f3d64208640 1 --2- 172.21.15.5:0/593014124 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f3d480a2d60 0x7f3d480a5220 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:56.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.251+0000 7f3d64208640 1 --2- 172.21.15.5:0/593014124 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3d5c100aa0 0x7f3d5c198790 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:56.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.251+0000 7f3d64208640 1 --2- 172.21.15.5:0/593014124 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3d5c106b80 0x7f3d5c19cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:56.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.251+0000 7f3d64208640 1 --2- 172.21.15.5:0/593014124 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3d5c100100 0x7f3d5c198250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:56.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.251+0000 7f3d64208640 1 -- 172.21.15.5:0/593014124 >> 172.21.15.5:0/593014124 conn(0x7f3d5c077230 msgr2=0x7f3d5c0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:56.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.252+0000 7f3d64208640 1 -- 172.21.15.5:0/593014124 shutdown_connections 2024-09-12T02:58:56.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:56.252+0000 7f3d64208640 1 -- 172.21.15.5:0/593014124 wait complete. 2024-09-12T02:58:57.153 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '26c61f34', quiesce took 9.2 seconds. Will release after: 15.3 2024-09-12T02:58:57.165 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:58:56 smithi148 ceph-mon[35090]: pgmap v356: 97 pgs: 97 active+clean; 658 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-12T02:58:57.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:58:56 smithi146 ceph-mon[35011]: pgmap v356: 97 pgs: 97 active+clean; 658 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-12T02:58:57.309 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:58:56 smithi005 ceph-mon[31197]: pgmap v356: 97 pgs: 97 active+clean; 658 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-12T02:58:57.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.396+0000 7f96a68be640 1 -- 172.21.15.5:0/3250250300 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96a0100e20 msgr2=0x7f96a0101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:57.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.396+0000 7f96a68be640 1 --2- 172.21.15.5:0/3250250300 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96a0100e20 0x7f96a0101200 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f9690009900 tx=0x7f969002a480 comp rx=0 tx=0).stop 2024-09-12T02:58:57.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.396+0000 7f96a68be640 1 -- 172.21.15.5:0/3250250300 shutdown_connections 2024-09-12T02:58:57.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.396+0000 7f96a68be640 1 --2- 172.21.15.5:0/3250250300 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96a0102ef0 0x7f96a010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:57.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.396+0000 7f96a68be640 1 --2- 172.21.15.5:0/3250250300 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f96a0102570 0x7f96a01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:57.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.396+0000 7f96a68be640 1 --2- 172.21.15.5:0/3250250300 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96a0100e20 0x7f96a0101200 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:57.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.396+0000 7f96a68be640 1 -- 172.21.15.5:0/3250250300 >> 172.21.15.5:0/3250250300 conn(0x7f96a00fc7d0 msgr2=0x7f96a00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:57.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.397+0000 7f96a68be640 1 -- 172.21.15.5:0/3250250300 shutdown_connections 2024-09-12T02:58:57.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.397+0000 7f96a68be640 1 -- 172.21.15.5:0/3250250300 wait complete. 2024-09-12T02:58:57.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.397+0000 7f96a68be640 1 Processor -- start 2024-09-12T02:58:57.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.397+0000 7f96a68be640 1 -- start start 2024-09-12T02:58:57.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.398+0000 7f96a68be640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96a0100e20 0x7f96a0199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:57.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.398+0000 7f96a68be640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f96a0102570 0x7f96a01998a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:57.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.398+0000 7f96a68be640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96a0102ef0 0x7f96a01a43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:57.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.398+0000 7f96a68be640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f96a0110f80 con 0x7f96a0102ef0 2024-09-12T02:58:57.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.398+0000 7f969ffff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96a0100e20 0x7f96a0199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:57.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.398+0000 7f96a68be640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f96a0110e00 con 0x7f96a0100e20 2024-09-12T02:58:57.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.398+0000 7f96a68be640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f96a0111100 con 0x7f96a0102570 2024-09-12T02:58:57.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.398+0000 7f969ffff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96a0100e20 0x7f96a0199360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.5:52758/0 (socket says 172.21.15.5:52758) 2024-09-12T02:58:57.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.398+0000 7f969ffff640 1 -- 172.21.15.5:0/1791741959 learned_addr learned my addr 172.21.15.5:0/1791741959 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:58:57.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.398+0000 7f96a4e5b640 1 --2- 172.21.15.5:0/1791741959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96a0102ef0 0x7f96a01a43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:57.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.398+0000 7f969f7fe640 1 --2- 172.21.15.5:0/1791741959 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f96a0102570 0x7f96a01998a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:57.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.399+0000 7f969ffff640 1 -- 172.21.15.5:0/1791741959 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f96a0102570 msgr2=0x7f96a01998a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:57.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.399+0000 7f969ffff640 1 --2- 172.21.15.5:0/1791741959 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f96a0102570 0x7f96a01998a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:57.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.399+0000 7f969ffff640 1 -- 172.21.15.5:0/1791741959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96a0102ef0 msgr2=0x7f96a01a43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:57.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.399+0000 7f969ffff640 1 --2- 172.21.15.5:0/1791741959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96a0102ef0 0x7f96a01a43e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:57.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.399+0000 7f969ffff640 1 -- 172.21.15.5:0/1791741959 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96a0197150 con 0x7f96a0100e20 2024-09-12T02:58:57.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.399+0000 7f969ffff640 1 --2- 172.21.15.5:0/1791741959 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96a0100e20 0x7f96a0199360 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9690006010 tx=0x7f9690003c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:57.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.399+0000 7f969d7fa640 1 -- 172.21.15.5:0/1791741959 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f969003f070 con 0x7f96a0100e20 2024-09-12T02:58:57.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.399+0000 7f969d7fa640 1 -- 172.21.15.5:0/1791741959 <== mon.2 v2:172.21.15.146:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9690031e30 con 0x7f96a0100e20 2024-09-12T02:58:57.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.399+0000 7f96a68be640 1 -- 172.21.15.5:0/1791741959 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96a01973e0 con 0x7f96a0100e20 2024-09-12T02:58:57.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.399+0000 7f969d7fa640 1 -- 172.21.15.5:0/1791741959 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9690035030 con 0x7f96a0100e20 2024-09-12T02:58:57.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.400+0000 7f96a68be640 1 -- 172.21.15.5:0/1791741959 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96a0197910 con 0x7f96a0100e20 2024-09-12T02:58:57.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.403+0000 7f96a8c97740 1 Processor -- start 2024-09-12T02:58:57.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.403+0000 7f96a8c97740 1 -- start start 2024-09-12T02:58:57.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.403+0000 7f96a8c97740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561cea6c6b40 0x561cea6c5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:57.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.403+0000 7f96a8c97740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561cea6c55d0 0x561cea6c5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:57.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.403+0000 7f96a8c97740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561cea6d0370 0x561cea6d2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:57.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.403+0000 7f96a8c97740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561cea62b2f0 con 0x561cea6d0370 2024-09-12T02:58:57.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.403+0000 7f96a8c97740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x561cea5d2cf0 con 0x561cea6c55d0 2024-09-12T02:58:57.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.403+0000 7f96a8c97740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x561cea6272b0 con 0x561cea6c6b40 2024-09-12T02:58:57.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.404+0000 7f96a4e5b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561cea6c6b40 0x561cea6c5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:57.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.404+0000 7f96a4e5b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561cea6c6b40 0x561cea6c5090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.5:43994/0 (socket says 172.21.15.5:43994) 2024-09-12T02:58:57.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.404+0000 7f96a4e5b640 1 -- 172.21.15.5:0/2766552526 learned_addr learned my addr 172.21.15.5:0/2766552526 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:58:57.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.404+0000 7f969d7fa640 1 -- 172.21.15.5:0/1791741959 <== mon.2 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7f9690042070 con 0x7f96a0100e20 2024-09-12T02:58:57.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.404+0000 7f96a4e5b640 1 -- 172.21.15.5:0/2766552526 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561cea6c55d0 msgr2=0x561cea6c5a10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T02:58:57.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.404+0000 7f96a4e5b640 1 --2- 172.21.15.5:0/2766552526 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561cea6c55d0 0x561cea6c5a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:57.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.404+0000 7f96a4e5b640 1 -- 172.21.15.5:0/2766552526 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561cea6d0370 msgr2=0x561cea6d2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:57.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.404+0000 7f96a4e5b640 1 --2- 172.21.15.5:0/2766552526 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561cea6d0370 0x561cea6d2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:57.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.404+0000 7f96a4e5b640 1 -- 172.21.15.5:0/2766552526 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561cea5b73a0 con 0x561cea6c6b40 2024-09-12T02:58:57.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.404+0000 7f96a4e5b640 1 --2- 172.21.15.5:0/2766552526 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561cea6c6b40 0x561cea6c5090 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9698008d40 tx=0x7f969800e720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7eb6bb1ea294be2f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:57.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.405+0000 7f96867fc640 1 -- 172.21.15.5:0/2766552526 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9698014070 con 0x561cea6c6b40 2024-09-12T02:58:57.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.405+0000 7f96867fc640 1 -- 172.21.15.5:0/2766552526 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9698003b70 con 0x561cea6c6b40 2024-09-12T02:58:57.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.405+0000 7f96867fc640 1 -- 172.21.15.5:0/2766552526 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96980107b0 con 0x561cea6c6b40 2024-09-12T02:58:57.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.405+0000 7f969d7fa640 1 --2- 172.21.15.5:0/1791741959 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f96880a2c90 0x7f96880a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:57.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.405+0000 7f969ffff640 1 --2- 172.21.15.5:0/1791741959 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f96880a2c90 0x7f96880a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:57.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.405+0000 7f969d7fa640 1 -- 172.21.15.5:0/1791741959 <== mon.2 v2:172.21.15.146:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f96900ef5d0 con 0x7f96a0100e20 2024-09-12T02:58:57.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.405+0000 7f96a8c97740 1 -- 172.21.15.5:0/2766552526 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561cea6c6b40 msgr2=0x561cea6c5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:57.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.405+0000 7f96a8c97740 1 --2- 172.21.15.5:0/2766552526 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561cea6c6b40 0x561cea6c5090 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9698008d40 tx=0x7f969800e720 comp rx=0 tx=0).stop 2024-09-12T02:58:57.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.406+0000 7f96a8c97740 1 -- 172.21.15.5:0/2766552526 shutdown_connections 2024-09-12T02:58:57.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.406+0000 7f96a8c97740 1 --2- 172.21.15.5:0/2766552526 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561cea6d0370 0x561cea6d2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:57.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.406+0000 7f96a8c97740 1 --2- 172.21.15.5:0/2766552526 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561cea6c55d0 0x561cea6c5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:57.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.406+0000 7f96a8c97740 1 --2- 172.21.15.5:0/2766552526 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561cea6c6b40 0x561cea6c5090 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:57.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.406+0000 7f96a8c97740 1 -- 172.21.15.5:0/2766552526 >> 172.21.15.5:0/2766552526 conn(0x561cea68e310 msgr2=0x561cea6c1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:57.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.406+0000 7f96a8c97740 1 -- 172.21.15.5:0/2766552526 shutdown_connections 2024-09-12T02:58:57.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.406+0000 7f969ffff640 1 --2- 172.21.15.5:0/1791741959 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f96880a2c90 0x7f96880a5150 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f96900bba10 tx=0x7f96900e6810 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:57.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.406+0000 7f96a8c97740 1 -- 172.21.15.5:0/2766552526 wait complete. 2024-09-12T02:58:57.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.406+0000 7f96a8c97740 1 -- start start 2024-09-12T02:58:57.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.406+0000 7f96a8c97740 1 Processor -- start 2024-09-12T02:58:57.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.407+0000 7f96a8c97740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561cea6c55d0 0x561cea6cfcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:57.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.407+0000 7f96a8c97740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561cea6c6b40 0x561cea6ccfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:57.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.407+0000 7f96a4e5b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561cea6c55d0 0x561cea6cfcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:57.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.407+0000 7f96a4e5b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561cea6c55d0 0x561cea6cfcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48818/0 (socket says 172.21.15.5:48818) 2024-09-12T02:58:57.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.407+0000 7f96a8c97740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561cea6d0370 0x561cea6cd520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:57.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.407+0000 7f96a4e5b640 1 -- 172.21.15.5:0/3100178713 learned_addr learned my addr 172.21.15.5:0/3100178713 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:58:57.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.407+0000 7f96a8c97740 1 -- 172.21.15.5:0/3100178713 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561cea6d3070 con 0x561cea6c55d0 2024-09-12T02:58:57.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.407+0000 7f96a8c97740 1 -- 172.21.15.5:0/3100178713 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x561cea5e6060 con 0x561cea6d0370 2024-09-12T02:58:57.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.407+0000 7f969f7fe640 1 --2- 172.21.15.5:0/3100178713 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561cea6c6b40 0x561cea6ccfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:57.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.407+0000 7f96a8c97740 1 -- 172.21.15.5:0/3100178713 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x561cea6d31f0 con 0x561cea6c6b40 2024-09-12T02:58:57.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.408+0000 7f96a8c97740 10 client.0 _ll_register_callbacks cb 0x561cea3398c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:58:57.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.408+0000 7f96a4e5b640 1 --2- 172.21.15.5:0/3100178713 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561cea6d0370 0x561cea6cd520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:57.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.408+0000 7f96a4e5b640 1 -- 172.21.15.5:0/3100178713 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561cea6c6b40 msgr2=0x561cea6ccfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:57.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.408+0000 7f96a4e5b640 1 --2- 172.21.15.5:0/3100178713 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561cea6c6b40 0x561cea6ccfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:57.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.408+0000 7f96a4e5b640 1 -- 172.21.15.5:0/3100178713 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561cea6d0370 msgr2=0x561cea6cd520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:57.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.408+0000 7f96a4e5b640 1 --2- 172.21.15.5:0/3100178713 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561cea6d0370 0x561cea6cd520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:57.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.408+0000 7f96a4e5b640 1 -- 172.21.15.5:0/3100178713 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561cea39c3c0 con 0x561cea6c55d0 2024-09-12T02:58:57.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.408+0000 7f96a4e5b640 1 --2- 172.21.15.5:0/3100178713 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561cea6c55d0 0x561cea6cfcf0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f9698005b40 tx=0x7f9698005a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2490f684c04664f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:57.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.408+0000 7f967f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T02:58:57.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.408+0000 7f967f7fe640 1 -- 172.21.15.5:0/3100178713 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9698014070 con 0x561cea6c55d0 2024-09-12T02:58:57.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.409+0000 7f967f7fe640 1 -- 172.21.15.5:0/3100178713 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f9698010760 con 0x561cea6c55d0 2024-09-12T02:58:57.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.409+0000 7f96a8c97740 1 -- 172.21.15.5:0/3100178713 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9698003b70 con 0x561cea6c55d0 2024-09-12T02:58:57.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.409+0000 7f967f7fe640 1 -- 172.21.15.5:0/3100178713 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f96980043a0 con 0x561cea6c55d0 2024-09-12T02:58:57.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.409+0000 7f96a8c97740 10 client.15525 fetch_fsmap learned FSMap version 26 2024-09-12T02:58:57.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.409+0000 7f96a8c97740 1 -- 172.21.15.5:0/3100178713 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x561cea6deae0 con 0x561cea6c55d0 2024-09-12T02:58:57.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.409+0000 7f967f7fe640 1 -- 172.21.15.5:0/3100178713 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9698003b70 con 0x561cea6c55d0 2024-09-12T02:58:57.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.410+0000 7f967f7fe640 1 -- 172.21.15.5:0/3100178713 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f9698018970 con 0x561cea6c55d0 2024-09-12T02:58:57.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.410+0000 7f96a8c97740 10 client.15525 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T02:58:57.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.410+0000 7f96a8c97740 10 client.15525 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.b 2024-09-12T02:58:57.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.410+0000 7f96a8c97740 20 client.15525 populate_metadata read hostname 'smithi005' 2024-09-12T02:58:57.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.410+0000 7f96a8c97740 1 --2- 172.21.15.5:0/3100178713 >> [v2:172.21.15.146:6838/3036767228,v1:172.21.15.146:6839/3036767228] conn(0x561cea6dfe30 0x561cea6e2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:57.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.410+0000 7f96a8c97740 4 client.15525 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:58:57.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.410+0000 7f96a8c97740 1 -- 172.21.15.5:0/3100178713 --> [v2:172.21.15.146:6838/3036767228,v1:172.21.15.146:6839/3036767228] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561cea638860 con 0x561cea6dfe30 2024-09-12T02:58:57.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.410+0000 7f969f7fe640 1 --2- 172.21.15.5:0/3100178713 >> [v2:172.21.15.146:6838/3036767228,v1:172.21.15.146:6839/3036767228] conn(0x561cea6dfe30 0x561cea6e2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:57.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.411+0000 7f969f7fe640 1 --2- 172.21.15.5:0/3100178713 >> [v2:172.21.15.146:6838/3036767228,v1:172.21.15.146:6839/3036767228] conn(0x561cea6dfe30 0x561cea6e2220 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=dbfcfca80c6e72b4 server_cookie=2e3f94a86773e4f2 in_seq=0 out_seq=0 2024-09-12T02:58:57.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.411+0000 7f967f7fe640 10 client.15525 ms_handle_connect on v2:172.21.15.146:6838/3036767228 2024-09-12T02:58:57.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.412+0000 7f967f7fe640 1 -- 172.21.15.5:0/3100178713 <== mds.3 v2:172.21.15.146:6838/3036767228 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561cea638860 con 0x561cea6dfe30 2024-09-12T02:58:57.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.412+0000 7f967f7fe640 10 client.15525 handle_command_reply: tid=0 2024-09-12T02:58:57.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.412+0000 7f96a8c97740 1 client.15525 shutdown 2024-09-12T02:58:57.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.412+0000 7f96a8c97740 1 -- 172.21.15.5:0/3100178713 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561cea6c55d0 msgr2=0x561cea6cfcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:57.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.412+0000 7f96a8c97740 1 --2- 172.21.15.5:0/3100178713 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561cea6c55d0 0x561cea6cfcf0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f9698005b40 tx=0x7f9698005a50 comp rx=0 tx=0).stop 2024-09-12T02:58:57.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.412+0000 7f96a8c97740 1 -- 172.21.15.5:0/3100178713 shutdown_connections 2024-09-12T02:58:57.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.412+0000 7f96a8c97740 1 --2- 172.21.15.5:0/3100178713 >> [v2:172.21.15.146:6838/3036767228,v1:172.21.15.146:6839/3036767228] conn(0x561cea6dfe30 0x561cea6e2220 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:57.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.412+0000 7f96a8c97740 1 --2- 172.21.15.5:0/3100178713 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561cea6d0370 0x561cea6cd520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:57.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.412+0000 7f96a8c97740 1 --2- 172.21.15.5:0/3100178713 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561cea6c6b40 0x561cea6ccfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:57.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.412+0000 7f967f7fe640 0 client.15525 ms_handle_reset on v2:172.21.15.146:6838/3036767228 2024-09-12T02:58:57.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.412+0000 7f96a8c97740 1 --2- 172.21.15.5:0/3100178713 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561cea6c55d0 0x561cea6cfcf0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:57.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.412+0000 7f96a8c97740 1 -- 172.21.15.5:0/3100178713 >> 172.21.15.5:0/3100178713 conn(0x561cea68e310 msgr2=0x561cea6c7960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:57.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.412+0000 7f96a8c97740 1 -- 172.21.15.5:0/3100178713 shutdown_connections 2024-09-12T02:58:57.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.412+0000 7f96a8c97740 1 -- 172.21.15.5:0/3100178713 wait complete. 2024-09-12T02:58:57.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.413+0000 7f96a8c97740 20 client.15525 trim_cache size 0 max 16384 2024-09-12T02:58:57.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.438+0000 7f96a8c97740 1 Processor -- start 2024-09-12T02:58:57.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.438+0000 7f96a8c97740 1 -- start start 2024-09-12T02:58:57.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.438+0000 7f96a8c97740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561cea6c55d0 0x561cea6cbee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:57.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.438+0000 7f96a8c97740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561cea6c6b40 0x561cea6c0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:57.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.438+0000 7f96a8c97740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561cea6d0370 0x561cea6c8120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:57.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.438+0000 7f96a8c97740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f96a01af7e0 con 0x561cea6c6b40 2024-09-12T02:58:57.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.438+0000 7f96a8c97740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f96a01b1320 con 0x561cea6d0370 2024-09-12T02:58:57.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.438+0000 7f96a8c97740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f96a01afeb0 con 0x561cea6c55d0 2024-09-12T02:58:57.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.438+0000 7f969f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561cea6c6b40 0x561cea6c0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:57.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.438+0000 7f969f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561cea6c6b40 0x561cea6c0bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48824/0 (socket says 172.21.15.5:48824) 2024-09-12T02:58:57.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.439+0000 7f969f7fe640 1 -- 172.21.15.5:0/1594496352 learned_addr learned my addr 172.21.15.5:0/1594496352 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:58:57.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.439+0000 7f96a4e5b640 1 --2- 172.21.15.5:0/1594496352 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561cea6c55d0 0x561cea6cbee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:57.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.439+0000 7f96a4e5b640 1 --2- 172.21.15.5:0/1594496352 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561cea6d0370 0x561cea6c8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:57.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.439+0000 7f969f7fe640 1 -- 172.21.15.5:0/1594496352 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561cea6c55d0 msgr2=0x561cea6cbee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:57.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.439+0000 7f969f7fe640 1 --2- 172.21.15.5:0/1594496352 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561cea6c55d0 0x561cea6cbee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:57.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.439+0000 7f969f7fe640 1 -- 172.21.15.5:0/1594496352 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561cea6d0370 msgr2=0x561cea6c8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:57.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.439+0000 7f969f7fe640 1 --2- 172.21.15.5:0/1594496352 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561cea6d0370 0x561cea6c8120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:57.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.439+0000 7f969f7fe640 1 -- 172.21.15.5:0/1594496352 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561cea6cb4f0 con 0x561cea6c6b40 2024-09-12T02:58:57.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.439+0000 7f969f7fe640 1 --2- 172.21.15.5:0/1594496352 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561cea6c6b40 0x561cea6c0bf0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f96a006aac0 tx=0x7f9694012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5509d84a1e87c137 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:57.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.439+0000 7f9685ffb640 1 -- 172.21.15.5:0/1594496352 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9694021030 con 0x561cea6c6b40 2024-09-12T02:58:57.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.439+0000 7f9685ffb640 1 -- 172.21.15.5:0/1594496352 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9694012e00 con 0x561cea6c6b40 2024-09-12T02:58:57.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.439+0000 7f9685ffb640 1 -- 172.21.15.5:0/1594496352 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9694003c50 con 0x561cea6c6b40 2024-09-12T02:58:57.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.440+0000 7f96a8c97740 1 -- 172.21.15.5:0/1594496352 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561cea6c6b40 msgr2=0x561cea6c0bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:57.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.440+0000 7f96a8c97740 1 --2- 172.21.15.5:0/1594496352 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561cea6c6b40 0x561cea6c0bf0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f96a006aac0 tx=0x7f9694012820 comp rx=0 tx=0).stop 2024-09-12T02:58:57.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.440+0000 7f96a8c97740 1 -- 172.21.15.5:0/1594496352 shutdown_connections 2024-09-12T02:58:57.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.440+0000 7f96a8c97740 1 --2- 172.21.15.5:0/1594496352 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561cea6c6b40 0x561cea6c0bf0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:57.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.440+0000 7f96a8c97740 1 --2- 172.21.15.5:0/1594496352 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561cea6d0370 0x561cea6c8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:57.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.440+0000 7f96a8c97740 1 --2- 172.21.15.5:0/1594496352 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561cea6c55d0 0x561cea6cbee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:57.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.440+0000 7f96a8c97740 1 -- 172.21.15.5:0/1594496352 >> 172.21.15.5:0/1594496352 conn(0x561cea68e310 msgr2=0x561cea6c04e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:57.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.441+0000 7f96a8c97740 1 -- 172.21.15.5:0/1594496352 shutdown_connections 2024-09-12T02:58:57.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.441+0000 7f96a8c97740 1 -- 172.21.15.5:0/1594496352 wait complete. 2024-09-12T02:58:57.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.441+0000 7f96a8c97740 1 -- start start 2024-09-12T02:58:57.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.441+0000 7f96a8c97740 1 Processor -- start 2024-09-12T02:58:57.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.442+0000 7f96a8c97740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561cea6c55d0 0x561cea6d88c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:57.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.442+0000 7f96a8c97740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561cea6c6b40 0x561cea6d8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:57.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.442+0000 7f96a8c97740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561cea6d0370 0x561cea6d3e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:57.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.442+0000 7f96a8c97740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f96a01b0c50 con 0x561cea6d0370 2024-09-12T02:58:57.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.442+0000 7f96a8c97740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f96a01b0580 con 0x561cea6c55d0 2024-09-12T02:58:57.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.442+0000 7f96a8c97740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f96a01ad7c0 con 0x561cea6c6b40 2024-09-12T02:58:57.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.442+0000 7f96a8c97740 10 client.0 _ll_register_callbacks cb 0x7f9698008b90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:58:57.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.442+0000 7f969f7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561cea6c6b40 0x561cea6d8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:57.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.442+0000 7f96a4e5b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561cea6d0370 0x561cea6d3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:57.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.442+0000 7f969f7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561cea6c6b40 0x561cea6d8e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.5:44028/0 (socket says 172.21.15.5:44028) 2024-09-12T02:58:57.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.442+0000 7f969f7fe640 1 -- 172.21.15.5:0/3120910645 learned_addr learned my addr 172.21.15.5:0/3120910645 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:58:57.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.442+0000 7f969f7fe640 1 -- 172.21.15.5:0/3120910645 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561cea6c55d0 msgr2=0x561cea6d88c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T02:58:57.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.442+0000 7f969f7fe640 1 --2- 172.21.15.5:0/3120910645 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561cea6c55d0 0x561cea6d88c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:57.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.442+0000 7f969f7fe640 1 -- 172.21.15.5:0/3120910645 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561cea6d0370 msgr2=0x561cea6d3e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:57.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.443+0000 7f969f7fe640 1 --2- 172.21.15.5:0/3120910645 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561cea6d0370 0x561cea6d3e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:57.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.443+0000 7f969f7fe640 1 -- 172.21.15.5:0/3120910645 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561cea6d46c0 con 0x561cea6c6b40 2024-09-12T02:58:57.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.443+0000 7f96a4e5b640 1 --2- 172.21.15.5:0/3120910645 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561cea6d0370 0x561cea6d3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:58:57.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.443+0000 7f969f7fe640 1 --2- 172.21.15.5:0/3120910645 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561cea6c6b40 0x561cea6d8e00 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f969400a310 tx=0x7f9694012380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af835d6674c7345a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:57.447 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.443+0000 7f967f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-09-12T02:58:57.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.443+0000 7f967f7fe640 1 -- 172.21.15.5:0/3120910645 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9694021070 con 0x561cea6c6b40 2024-09-12T02:58:57.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.443+0000 7f967f7fe640 1 -- 172.21.15.5:0/3120910645 <== mon.1 v2:172.21.15.148:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f9694004430 con 0x561cea6c6b40 2024-09-12T02:58:57.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.443+0000 7f96a8c97740 1 -- 172.21.15.5:0/3120910645 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9694012e00 con 0x561cea6c6b40 2024-09-12T02:58:57.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.443+0000 7f967f7fe640 1 -- 172.21.15.5:0/3120910645 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f969400aba0 con 0x561cea6c6b40 2024-09-12T02:58:57.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.443+0000 7f96a8c97740 10 client.25135 fetch_fsmap learned FSMap version 26 2024-09-12T02:58:57.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.443+0000 7f96a8c97740 1 -- 172.21.15.5:0/3120910645 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x561cea752c80 con 0x561cea6c6b40 2024-09-12T02:58:57.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.444+0000 7f967f7fe640 1 -- 172.21.15.5:0/3120910645 <== mon.1 v2:172.21.15.148:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9694012e00 con 0x561cea6c6b40 2024-09-12T02:58:57.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.444+0000 7f967f7fe640 1 -- 172.21.15.5:0/3120910645 <== mon.1 v2:172.21.15.148:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f969402e030 con 0x561cea6c6b40 2024-09-12T02:58:57.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.444+0000 7f96a8c97740 10 client.25135 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T02:58:57.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.444+0000 7f96a8c97740 10 client.25135 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.b 2024-09-12T02:58:57.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.444+0000 7f96a8c97740 20 client.25135 populate_metadata read hostname 'smithi005' 2024-09-12T02:58:57.448 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.444+0000 7f96a8c97740 1 --2- 172.21.15.5:0/3120910645 >> [v2:172.21.15.146:6838/3036767228,v1:172.21.15.146:6839/3036767228] conn(0x561cea753380 0x561cea7557e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:57.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.444+0000 7f96a8c97740 4 client.25135 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:58:57.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.444+0000 7f96a8c97740 1 -- 172.21.15.5:0/3120910645 --> [v2:172.21.15.146:6838/3036767228,v1:172.21.15.146:6839/3036767228] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561cea632d70 con 0x561cea753380 2024-09-12T02:58:57.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.444+0000 7f96a4e5b640 1 --2- 172.21.15.5:0/3120910645 >> [v2:172.21.15.146:6838/3036767228,v1:172.21.15.146:6839/3036767228] conn(0x561cea753380 0x561cea7557e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:57.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.445+0000 7f96a4e5b640 1 --2- 172.21.15.5:0/3120910645 >> [v2:172.21.15.146:6838/3036767228,v1:172.21.15.146:6839/3036767228] conn(0x561cea753380 0x561cea7557e0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ab7316baf3e9001c server_cookie=63e37284435a54dc in_seq=0 out_seq=0 2024-09-12T02:58:57.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.445+0000 7f967f7fe640 10 client.25135 ms_handle_connect on v2:172.21.15.146:6838/3036767228 2024-09-12T02:58:57.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.445+0000 7f967f7fe640 1 -- 172.21.15.5:0/3120910645 <== mds.3 v2:172.21.15.146:6838/3036767228 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561cea632d70 con 0x561cea753380 2024-09-12T02:58:57.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.445+0000 7f967f7fe640 10 client.25135 handle_command_reply: tid=0 2024-09-12T02:58:57.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.445+0000 7f96a8c97740 1 client.25135 shutdown 2024-09-12T02:58:57.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.445+0000 7f96a8c97740 1 -- 172.21.15.5:0/3120910645 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561cea6c6b40 msgr2=0x561cea6d8e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:57.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.445+0000 7f96a8c97740 1 --2- 172.21.15.5:0/3120910645 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561cea6c6b40 0x561cea6d8e00 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f969400a310 tx=0x7f9694012380 comp rx=0 tx=0).stop 2024-09-12T02:58:57.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.445+0000 7f96a8c97740 1 -- 172.21.15.5:0/3120910645 shutdown_connections 2024-09-12T02:58:57.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.445+0000 7f96a8c97740 1 --2- 172.21.15.5:0/3120910645 >> [v2:172.21.15.146:6838/3036767228,v1:172.21.15.146:6839/3036767228] conn(0x561cea753380 0x561cea7557e0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:57.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.445+0000 7f96a8c97740 1 --2- 172.21.15.5:0/3120910645 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561cea6d0370 0x561cea6d3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:57.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.445+0000 7f967f7fe640 0 client.25135 ms_handle_reset on v2:172.21.15.146:6838/3036767228 2024-09-12T02:58:57.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.445+0000 7f96a8c97740 1 --2- 172.21.15.5:0/3120910645 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561cea6c6b40 0x561cea6d8e00 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:57.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.445+0000 7f96a8c97740 1 --2- 172.21.15.5:0/3120910645 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561cea6c55d0 0x561cea6d88c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:57.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.445+0000 7f96a8c97740 1 -- 172.21.15.5:0/3120910645 >> 172.21.15.5:0/3120910645 conn(0x561cea68e310 msgr2=0x561cea6c0d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:57.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.446+0000 7f96a8c97740 1 -- 172.21.15.5:0/3120910645 shutdown_connections 2024-09-12T02:58:57.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.446+0000 7f96a8c97740 1 -- 172.21.15.5:0/3120910645 wait complete. 2024-09-12T02:58:57.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.446+0000 7f96a8c97740 20 client.25135 trim_cache size 0 max 16384 2024-09-12T02:58:57.452 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-12T02:58:57.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.446+0000 7f96a68be640 1 -- 172.21.15.5:0/1791741959 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f96880a2c90 msgr2=0x7f96880a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:57.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.446+0000 7f96a68be640 1 --2- 172.21.15.5:0/1791741959 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f96880a2c90 0x7f96880a5150 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f96900bba10 tx=0x7f96900e6810 comp rx=0 tx=0).stop 2024-09-12T02:58:57.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.446+0000 7f96a68be640 1 -- 172.21.15.5:0/1791741959 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96a0100e20 msgr2=0x7f96a0199360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:57.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.446+0000 7f96a68be640 1 --2- 172.21.15.5:0/1791741959 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96a0100e20 0x7f96a0199360 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9690006010 tx=0x7f9690003c80 comp rx=0 tx=0).stop 2024-09-12T02:58:57.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.446+0000 7f96a68be640 1 -- 172.21.15.5:0/1791741959 shutdown_connections 2024-09-12T02:58:57.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.446+0000 7f96a68be640 1 --2- 172.21.15.5:0/1791741959 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f96880a2c90 0x7f96880a5150 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:57.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.446+0000 7f96a68be640 1 --2- 172.21.15.5:0/1791741959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f96a0102ef0 0x7f96a01a43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:57.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.446+0000 7f96a68be640 1 --2- 172.21.15.5:0/1791741959 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f96a0102570 0x7f96a01998a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:57.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.446+0000 7f96a68be640 1 --2- 172.21.15.5:0/1791741959 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f96a0100e20 0x7f96a0199360 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:57.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.446+0000 7f96a68be640 1 -- 172.21.15.5:0/1791741959 >> 172.21.15.5:0/1791741959 conn(0x7f96a00fc7d0 msgr2=0x7f96a00fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:57.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.447+0000 7f96a68be640 1 -- 172.21.15.5:0/1791741959 shutdown_connections 2024-09-12T02:58:57.453 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:57.447+0000 7f96a68be640 1 -- 172.21.15.5:0/1791741959 wait complete. 2024-09-12T02:58:58.124 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T02:58:58.476 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:58:58.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:58.992+0000 7f733b945640 1 -- 172.21.15.5:0/740173507 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7334102db0 msgr2=0x7f7334103190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:58.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:58.992+0000 7f733b945640 1 --2- 172.21.15.5:0/740173507 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7334102db0 0x7f7334103190 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f7330009930 tx=0x7f733002a420 comp rx=0 tx=0).stop 2024-09-12T02:58:58.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:58.993+0000 7f733b945640 1 -- 172.21.15.5:0/740173507 shutdown_connections 2024-09-12T02:58:58.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:58.993+0000 7f733b945640 1 --2- 172.21.15.5:0/740173507 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f73341082e0 0x7f733410ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:58.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:58.993+0000 7f733b945640 1 --2- 172.21.15.5:0/740173507 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7334103760 0x7f7334107ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:58.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:58.993+0000 7f733b945640 1 --2- 172.21.15.5:0/740173507 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7334102db0 0x7f7334103190 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:58.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:58.993+0000 7f733b945640 1 -- 172.21.15.5:0/740173507 >> 172.21.15.5:0/740173507 conn(0x7f73340fc7d0 msgr2=0x7f73340febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:58.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:58.993+0000 7f733b945640 1 -- 172.21.15.5:0/740173507 shutdown_connections 2024-09-12T02:58:58.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:58.993+0000 7f733b945640 1 -- 172.21.15.5:0/740173507 wait complete. 2024-09-12T02:58:58.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:58.993+0000 7f733b945640 1 Processor -- start 2024-09-12T02:58:58.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:58.994+0000 7f733b945640 1 -- start start 2024-09-12T02:58:58.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:58.994+0000 7f733b945640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7334102db0 0x7f7334100a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:58.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:58.994+0000 7f733b945640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7334103760 0x7f7334100f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:58.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:58.994+0000 7f733b945640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73341082e0 0x7f7334101490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:58.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:58.994+0000 7f733b945640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f73341120b0 con 0x7f73341082e0 2024-09-12T02:58:58.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:58.994+0000 7f733b945640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f7334111f30 con 0x7f7334103760 2024-09-12T02:58:58.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:58.994+0000 7f733b945640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f7334112230 con 0x7f7334102db0 2024-09-12T02:58:58.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:58.994+0000 7f7339ee2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73341082e0 0x7f7334101490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:58.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:58.994+0000 7f7339ee2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73341082e0 0x7f7334101490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48856/0 (socket says 172.21.15.5:48856) 2024-09-12T02:58:58.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:58.994+0000 7f7339ee2640 1 -- 172.21.15.5:0/3036742663 learned_addr learned my addr 172.21.15.5:0/3036742663 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:58:58.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:58.994+0000 7f73396e1640 1 --2- 172.21.15.5:0/3036742663 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7334102db0 0x7f7334100a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:58.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:58.994+0000 7f7338ee0640 1 --2- 172.21.15.5:0/3036742663 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7334103760 0x7f7334100f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:58.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:58.994+0000 7f7339ee2640 1 -- 172.21.15.5:0/3036742663 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7334102db0 msgr2=0x7f7334100a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:58.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:58.994+0000 7f7339ee2640 1 --2- 172.21.15.5:0/3036742663 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7334102db0 0x7f7334100a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:58.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:58.994+0000 7f7339ee2640 1 -- 172.21.15.5:0/3036742663 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7334103760 msgr2=0x7f7334100f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:58.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:58.994+0000 7f7339ee2640 1 --2- 172.21.15.5:0/3036742663 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7334103760 0x7f7334100f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:58.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:58.995+0000 7f7339ee2640 1 -- 172.21.15.5:0/3036742663 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73341a6a90 con 0x7f73341082e0 2024-09-12T02:58:58.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:58.995+0000 7f7339ee2640 1 --2- 172.21.15.5:0/3036742663 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73341082e0 0x7f7334101490 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f732c00eaa0 tx=0x7f732c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:58.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:58.995+0000 7f732a7fc640 1 -- 172.21.15.5:0/3036742663 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f732c00cdb0 con 0x7f73341082e0 2024-09-12T02:58:58.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:58.995+0000 7f732a7fc640 1 -- 172.21.15.5:0/3036742663 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f732c005020 con 0x7f73341082e0 2024-09-12T02:58:58.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:58.995+0000 7f733b945640 1 -- 172.21.15.5:0/3036742663 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73341a6d80 con 0x7f73341082e0 2024-09-12T02:58:58.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:58.995+0000 7f732a7fc640 1 -- 172.21.15.5:0/3036742663 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f732c0116b0 con 0x7f73341082e0 2024-09-12T02:58:58.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:58.995+0000 7f733b945640 1 -- 172.21.15.5:0/3036742663 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73341a7260 con 0x7f73341082e0 2024-09-12T02:58:58.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:58.999+0000 7f732a7fc640 1 -- 172.21.15.5:0/3036742663 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7f732c005560 con 0x7f73341082e0 2024-09-12T02:58:59.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:58.999+0000 7f733dd1e740 1 Processor -- start 2024-09-12T02:58:59.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:58.999+0000 7f733dd1e740 1 -- start start 2024-09-12T02:58:59.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.000+0000 7f733dd1e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558071054b40 0x558071053090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:59.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.000+0000 7f733dd1e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5580710535d0 0x558071053a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:59.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.000+0000 7f733dd1e740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55807105e370 0x558071060760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:59.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.000+0000 7f733dd1e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558070fb92f0 con 0x558071054b40 2024-09-12T02:58:59.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.000+0000 7f733dd1e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x558070f60cf0 con 0x5580710535d0 2024-09-12T02:58:59.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.000+0000 7f733dd1e740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x558070fb52b0 con 0x55807105e370 2024-09-12T02:58:59.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.000+0000 7f7338ee0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558071054b40 0x558071053090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:59.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.000+0000 7f73396e1640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5580710535d0 0x558071053a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:59.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.000+0000 7f7338ee0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558071054b40 0x558071053090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48866/0 (socket says 172.21.15.5:48866) 2024-09-12T02:58:59.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.000+0000 7f7338ee0640 1 -- 172.21.15.5:0/1893669989 learned_addr learned my addr 172.21.15.5:0/1893669989 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:58:59.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.000+0000 7f7338ee0640 1 --2- 172.21.15.5:0/1893669989 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55807105e370 0x558071060760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:59.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.000+0000 7f732a7fc640 1 --2- 172.21.15.5:0/3036742663 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f73200a2c90 0x7f73200a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:59.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.000+0000 7f7338ee0640 1 -- 172.21.15.5:0/1893669989 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55807105e370 msgr2=0x558071060760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:59.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.000+0000 7f7338ee0640 1 --2- 172.21.15.5:0/1893669989 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55807105e370 0x558071060760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:59.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.000+0000 7f7338ee0640 1 -- 172.21.15.5:0/1893669989 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5580710535d0 msgr2=0x558071053a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:59.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.000+0000 7f7338ee0640 1 --2- 172.21.15.5:0/1893669989 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5580710535d0 0x558071053a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:59.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.000+0000 7f7338ee0640 1 -- 172.21.15.5:0/1893669989 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558070f453a0 con 0x558071054b40 2024-09-12T02:58:59.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.001+0000 7f7339ee2640 1 --2- 172.21.15.5:0/3036742663 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f73200a2c90 0x7f73200a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:59.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.001+0000 7f732a7fc640 1 -- 172.21.15.5:0/3036742663 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f732c0cf2a0 con 0x7f73341082e0 2024-09-12T02:58:59.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.001+0000 7f7338ee0640 1 --2- 172.21.15.5:0/1893669989 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558071054b40 0x558071053090 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x558071054070 tx=0x7f7324005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5b8d2c1db499dcb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:59.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.001+0000 7f731f7fe640 1 -- 172.21.15.5:0/1893669989 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7324004050 con 0x558071054b40 2024-09-12T02:58:59.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.001+0000 7f731f7fe640 1 -- 172.21.15.5:0/1893669989 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73240041f0 con 0x558071054b40 2024-09-12T02:58:59.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.001+0000 7f731f7fe640 1 -- 172.21.15.5:0/1893669989 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f732400f7d0 con 0x558071054b40 2024-09-12T02:58:59.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.002+0000 7f733dd1e740 1 -- 172.21.15.5:0/1893669989 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558071054b40 msgr2=0x558071053090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:59.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.002+0000 7f7339ee2640 1 --2- 172.21.15.5:0/3036742663 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f73200a2c90 0x7f73200a5150 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f732c09b760 tx=0x7f732c0666e0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:59.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.002+0000 7f733dd1e740 1 --2- 172.21.15.5:0/1893669989 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558071054b40 0x558071053090 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x558071054070 tx=0x7f7324005850 comp rx=0 tx=0).stop 2024-09-12T02:58:59.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.002+0000 7f733dd1e740 1 -- 172.21.15.5:0/1893669989 shutdown_connections 2024-09-12T02:58:59.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.002+0000 7f733dd1e740 1 --2- 172.21.15.5:0/1893669989 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55807105e370 0x558071060760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:59.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.002+0000 7f733dd1e740 1 --2- 172.21.15.5:0/1893669989 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5580710535d0 0x558071053a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:59.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.002+0000 7f733dd1e740 1 --2- 172.21.15.5:0/1893669989 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558071054b40 0x558071053090 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:59.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.002+0000 7f733dd1e740 1 -- 172.21.15.5:0/1893669989 >> 172.21.15.5:0/1893669989 conn(0x55807101c310 msgr2=0x55807104fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:59.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.002+0000 7f733dd1e740 1 -- 172.21.15.5:0/1893669989 shutdown_connections 2024-09-12T02:58:59.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.002+0000 7f733dd1e740 1 -- 172.21.15.5:0/1893669989 wait complete. 2024-09-12T02:58:59.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.003+0000 7f733dd1e740 1 -- start start 2024-09-12T02:58:59.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.003+0000 7f733dd1e740 1 Processor -- start 2024-09-12T02:58:59.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.003+0000 7f733dd1e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5580710535d0 0x55807105b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:59.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.003+0000 7f733dd1e740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55807105e370 0x55807105b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:59.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.003+0000 7f733dd1e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55807105bcc0 0x55807106a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:59.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.003+0000 7f733dd1e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558071061070 con 0x55807105bcc0 2024-09-12T02:58:59.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.003+0000 7f733dd1e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x558070f74060 con 0x5580710535d0 2024-09-12T02:58:59.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.003+0000 7f733dd1e740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5580710611f0 con 0x55807105e370 2024-09-12T02:58:59.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.003+0000 7f733dd1e740 10 client.0 _ll_register_callbacks cb 0x558070cc78c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:58:59.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.003+0000 7f7338ee0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55807105bcc0 0x55807106a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:59.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.003+0000 7f7338ee0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55807105bcc0 0x55807106a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48882/0 (socket says 172.21.15.5:48882) 2024-09-12T02:58:59.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.003+0000 7f7338ee0640 1 -- 172.21.15.5:0/3512806722 learned_addr learned my addr 172.21.15.5:0/3512806722 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:58:59.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.004+0000 7f73396e1640 1 --2- 172.21.15.5:0/3512806722 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55807105e370 0x55807105b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:59.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.004+0000 7f7338ee0640 1 --2- 172.21.15.5:0/3512806722 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5580710535d0 0x55807105b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:59.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.004+0000 7f73396e1640 1 -- 172.21.15.5:0/3512806722 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5580710535d0 msgr2=0x55807105b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:59.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.004+0000 7f73396e1640 1 --2- 172.21.15.5:0/3512806722 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5580710535d0 0x55807105b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:59.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.004+0000 7f73396e1640 1 -- 172.21.15.5:0/3512806722 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55807105bcc0 msgr2=0x55807106a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:59.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.004+0000 7f73396e1640 1 --2- 172.21.15.5:0/3512806722 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55807105bcc0 0x55807106a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:59.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.004+0000 7f73396e1640 1 -- 172.21.15.5:0/3512806722 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558070d2a3c0 con 0x55807105e370 2024-09-12T02:58:59.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.004+0000 7f73396e1640 1 --2- 172.21.15.5:0/3512806722 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55807105e370 0x55807105b780 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7330005ab0 tx=0x7f73300043b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1bc47da113165352 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:59.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.004+0000 7f731cff9640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-09-12T02:58:59.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.004+0000 7f731cff9640 1 -- 172.21.15.5:0/3512806722 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7330035040 con 0x55807105e370 2024-09-12T02:58:59.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.005+0000 7f733dd1e740 1 -- 172.21.15.5:0/3512806722 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73240041f0 con 0x55807105e370 2024-09-12T02:58:59.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.005+0000 7f731cff9640 1 -- 172.21.15.5:0/3512806722 <== mon.1 v2:172.21.15.148:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f7330005440 con 0x55807105e370 2024-09-12T02:58:59.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.005+0000 7f731cff9640 1 -- 172.21.15.5:0/3512806722 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f73300046c0 con 0x55807105e370 2024-09-12T02:58:59.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.005+0000 7f733dd1e740 10 client.25147 fetch_fsmap learned FSMap version 26 2024-09-12T02:58:59.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.005+0000 7f733dd1e740 1 -- 172.21.15.5:0/3512806722 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55807106d160 con 0x55807105e370 2024-09-12T02:58:59.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.005+0000 7f731cff9640 1 -- 172.21.15.5:0/3512806722 <== mon.1 v2:172.21.15.148:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73240041f0 con 0x55807105e370 2024-09-12T02:58:59.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.005+0000 7f731cff9640 1 -- 172.21.15.5:0/3512806722 <== mon.1 v2:172.21.15.148:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f7330003b60 con 0x55807105e370 2024-09-12T02:58:59.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.005+0000 7f733dd1e740 10 client.25147 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T02:58:59.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.005+0000 7f733dd1e740 10 client.25147 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-09-12T02:58:59.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.005+0000 7f733dd1e740 20 client.25147 populate_metadata read hostname 'smithi005' 2024-09-12T02:58:59.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.005+0000 7f733dd1e740 1 --2- 172.21.15.5:0/3512806722 >> [v2:172.21.15.148:6834/4195086427,v1:172.21.15.148:6835/4195086427] conn(0x55807106ea40 0x558071070e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:59.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.006+0000 7f733dd1e740 4 client.25147 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:58:59.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.006+0000 7f733dd1e740 1 -- 172.21.15.5:0/3512806722 --> [v2:172.21.15.148:6834/4195086427,v1:172.21.15.148:6835/4195086427] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558070fc6860 con 0x55807106ea40 2024-09-12T02:58:59.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.006+0000 7f7338ee0640 1 --2- 172.21.15.5:0/3512806722 >> [v2:172.21.15.148:6834/4195086427,v1:172.21.15.148:6835/4195086427] conn(0x55807106ea40 0x558071070e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:59.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.006+0000 7f731cff9640 10 client.25147 ms_handle_connect on v2:172.21.15.148:6834/4195086427 2024-09-12T02:58:59.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.007+0000 7f7338ee0640 1 --2- 172.21.15.5:0/3512806722 >> [v2:172.21.15.148:6834/4195086427,v1:172.21.15.148:6835/4195086427] conn(0x55807106ea40 0x558071070e60 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1c24ec7d07b0b1a7 server_cookie=72fb39d1cb36eae4 in_seq=0 out_seq=0 2024-09-12T02:58:59.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.007+0000 7f731cff9640 1 -- 172.21.15.5:0/3512806722 <== mds.2 v2:172.21.15.148:6834/4195086427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558070fc6860 con 0x55807106ea40 2024-09-12T02:58:59.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.007+0000 7f731cff9640 10 client.25147 handle_command_reply: tid=0 2024-09-12T02:58:59.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.007+0000 7f733dd1e740 1 client.25147 shutdown 2024-09-12T02:58:59.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.008+0000 7f733dd1e740 1 -- 172.21.15.5:0/3512806722 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55807105e370 msgr2=0x55807105b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:59.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.008+0000 7f733dd1e740 1 --2- 172.21.15.5:0/3512806722 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55807105e370 0x55807105b780 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7330005ab0 tx=0x7f73300043b0 comp rx=0 tx=0).stop 2024-09-12T02:58:59.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.008+0000 7f733dd1e740 1 -- 172.21.15.5:0/3512806722 shutdown_connections 2024-09-12T02:58:59.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.008+0000 7f733dd1e740 1 --2- 172.21.15.5:0/3512806722 >> [v2:172.21.15.148:6834/4195086427,v1:172.21.15.148:6835/4195086427] conn(0x55807106ea40 0x558071070e60 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:59.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.008+0000 7f733dd1e740 1 --2- 172.21.15.5:0/3512806722 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55807105bcc0 0x55807106a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:59.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.008+0000 7f733dd1e740 1 --2- 172.21.15.5:0/3512806722 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55807105e370 0x55807105b780 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:59.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.008+0000 7f733dd1e740 1 --2- 172.21.15.5:0/3512806722 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5580710535d0 0x55807105b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:59.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.008+0000 7f733dd1e740 1 -- 172.21.15.5:0/3512806722 >> 172.21.15.5:0/3512806722 conn(0x55807101c310 msgr2=0x558071055930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:59.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.008+0000 7f731cff9640 0 client.25147 ms_handle_reset on v2:172.21.15.148:6834/4195086427 2024-09-12T02:58:59.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.008+0000 7f733dd1e740 1 -- 172.21.15.5:0/3512806722 shutdown_connections 2024-09-12T02:58:59.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.008+0000 7f733dd1e740 1 -- 172.21.15.5:0/3512806722 wait complete. 2024-09-12T02:58:59.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.008+0000 7f733dd1e740 20 client.25147 trim_cache size 0 max 16384 2024-09-12T02:58:59.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.021+0000 7f733dd1e740 1 Processor -- start 2024-09-12T02:58:59.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.021+0000 7f733dd1e740 1 -- start start 2024-09-12T02:58:59.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.022+0000 7f733dd1e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55807101c310 0x558071056a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:59.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.022+0000 7f733dd1e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5580710535d0 0x55807105b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:59.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.022+0000 7f733dd1e740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55807105bcc0 0x55807106a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:59.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.022+0000 7f733dd1e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f73341af6d0 con 0x55807101c310 2024-09-12T02:58:59.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.022+0000 7f733dd1e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f73341b1210 con 0x5580710535d0 2024-09-12T02:58:59.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.022+0000 7f733dd1e740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f73341afda0 con 0x55807105bcc0 2024-09-12T02:58:59.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.022+0000 7f7338ee0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55807101c310 0x558071056a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:59.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.022+0000 7f7338ee0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55807101c310 0x558071056a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48896/0 (socket says 172.21.15.5:48896) 2024-09-12T02:58:59.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.022+0000 7f7338ee0640 1 -- 172.21.15.5:0/1399319000 learned_addr learned my addr 172.21.15.5:0/1399319000 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:58:59.024 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.022+0000 7f73396e1640 1 --2- 172.21.15.5:0/1399319000 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5580710535d0 0x55807105b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:59.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.022+0000 7f7338ee0640 1 -- 172.21.15.5:0/1399319000 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55807105bcc0 msgr2=0x55807106a310 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T02:58:59.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.022+0000 7f7338ee0640 1 --2- 172.21.15.5:0/1399319000 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55807105bcc0 0x55807106a310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:59.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.022+0000 7f7338ee0640 1 -- 172.21.15.5:0/1399319000 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5580710535d0 msgr2=0x55807105b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:59.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.022+0000 7f7338ee0640 1 --2- 172.21.15.5:0/1399319000 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5580710535d0 0x55807105b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:59.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.022+0000 7f7338ee0640 1 -- 172.21.15.5:0/1399319000 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55807105c1e0 con 0x55807101c310 2024-09-12T02:58:59.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.023+0000 7f7338ee0640 1 --2- 172.21.15.5:0/1399319000 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55807101c310 0x558071056a20 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f7324003f20 tx=0x7f7324005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79563699b0de5b39 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:59.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.023+0000 7f731effd640 1 -- 172.21.15.5:0/1399319000 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7324013170 con 0x55807101c310 2024-09-12T02:58:59.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.023+0000 7f731effd640 1 -- 172.21.15.5:0/1399319000 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7324005520 con 0x55807101c310 2024-09-12T02:58:59.026 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.023+0000 7f731effd640 1 -- 172.21.15.5:0/1399319000 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7324003850 con 0x55807101c310 2024-09-12T02:58:59.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.023+0000 7f733dd1e740 1 -- 172.21.15.5:0/1399319000 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55807101c310 msgr2=0x558071056a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:59.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.023+0000 7f733dd1e740 1 --2- 172.21.15.5:0/1399319000 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55807101c310 0x558071056a20 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f7324003f20 tx=0x7f7324005f90 comp rx=0 tx=0).stop 2024-09-12T02:58:59.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.024+0000 7f733dd1e740 1 -- 172.21.15.5:0/1399319000 shutdown_connections 2024-09-12T02:58:59.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.024+0000 7f733dd1e740 1 --2- 172.21.15.5:0/1399319000 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55807105bcc0 0x55807106a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:59.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.024+0000 7f733dd1e740 1 --2- 172.21.15.5:0/1399319000 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5580710535d0 0x55807105b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:59.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.024+0000 7f733dd1e740 1 --2- 172.21.15.5:0/1399319000 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55807101c310 0x558071056a20 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:59.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.024+0000 7f733dd1e740 1 -- 172.21.15.5:0/1399319000 >> 172.21.15.5:0/1399319000 conn(0x55807105b8e0 msgr2=0x558071056310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:59.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.024+0000 7f733dd1e740 1 -- 172.21.15.5:0/1399319000 shutdown_connections 2024-09-12T02:58:59.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.024+0000 7f733dd1e740 1 -- 172.21.15.5:0/1399319000 wait complete. 2024-09-12T02:58:59.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.024+0000 7f733dd1e740 1 -- start start 2024-09-12T02:58:59.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.024+0000 7f733dd1e740 1 Processor -- start 2024-09-12T02:58:59.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.025+0000 7f733dd1e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55807101c310 0x5580710668e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:59.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.025+0000 7f733dd1e740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5580710535d0 0x5580710585e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:59.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.025+0000 7f733dd1e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55807105bcc0 0x558071058b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:59.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.025+0000 7f733dd1e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f73341b0b40 con 0x55807101c310 2024-09-12T02:58:59.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.025+0000 7f733dd1e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f73341b0470 con 0x55807105bcc0 2024-09-12T02:58:59.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.025+0000 7f7338ee0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55807101c310 0x5580710668e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:59.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.025+0000 7f733dd1e740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f73341ad6d0 con 0x5580710535d0 2024-09-12T02:58:59.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.025+0000 7f7338ee0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55807101c310 0x5580710668e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48908/0 (socket says 172.21.15.5:48908) 2024-09-12T02:58:59.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.025+0000 7f7338ee0640 1 -- 172.21.15.5:0/3351549493 learned_addr learned my addr 172.21.15.5:0/3351549493 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:58:59.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.025+0000 7f733dd1e740 10 client.0 _ll_register_callbacks cb 0x7f7330009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:58:59.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.025+0000 7f7338ee0640 1 -- 172.21.15.5:0/3351549493 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5580710535d0 msgr2=0x5580710585e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:59.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.025+0000 7f7338ee0640 1 --2- 172.21.15.5:0/3351549493 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5580710535d0 0x5580710585e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:59.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.025+0000 7f7338ee0640 1 -- 172.21.15.5:0/3351549493 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55807105bcc0 msgr2=0x558071058b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:58:59.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.025+0000 7f7338ee0640 1 --2- 172.21.15.5:0/3351549493 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55807105bcc0 0x558071058b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:59.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.025+0000 7f7338ee0640 1 -- 172.21.15.5:0/3351549493 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558071056120 con 0x55807101c310 2024-09-12T02:58:59.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.025+0000 7f7338ee0640 1 --2- 172.21.15.5:0/3351549493 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55807101c310 0x5580710668e0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f7324004190 tx=0x7f73240096b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36184aa411383a31 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:58:59.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.025+0000 7f731cff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T02:58:59.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.025+0000 7f731cff9640 1 -- 172.21.15.5:0/3351549493 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7324013170 con 0x55807101c310 2024-09-12T02:58:59.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.026+0000 7f731cff9640 1 -- 172.21.15.5:0/3351549493 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f73240035f0 con 0x55807101c310 2024-09-12T02:58:59.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.026+0000 7f733dd1e740 1 -- 172.21.15.5:0/3351549493 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7324005520 con 0x55807101c310 2024-09-12T02:58:59.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.026+0000 7f731cff9640 1 -- 172.21.15.5:0/3351549493 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f73240037f0 con 0x55807101c310 2024-09-12T02:58:59.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.026+0000 7f733dd1e740 10 client.15549 fetch_fsmap learned FSMap version 26 2024-09-12T02:58:59.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.026+0000 7f733dd1e740 1 -- 172.21.15.5:0/3351549493 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55807105fc30 con 0x55807101c310 2024-09-12T02:58:59.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.026+0000 7f731cff9640 1 -- 172.21.15.5:0/3351549493 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7324005520 con 0x55807101c310 2024-09-12T02:58:59.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.026+0000 7f731cff9640 1 -- 172.21.15.5:0/3351549493 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f732402e070 con 0x55807101c310 2024-09-12T02:58:59.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.027+0000 7f733dd1e740 10 client.15549 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T02:58:59.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.027+0000 7f733dd1e740 10 client.15549 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-09-12T02:58:59.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.027+0000 7f733dd1e740 20 client.15549 populate_metadata read hostname 'smithi005' 2024-09-12T02:58:59.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.027+0000 7f733dd1e740 1 --2- 172.21.15.5:0/3351549493 >> [v2:172.21.15.148:6834/4195086427,v1:172.21.15.148:6835/4195086427] conn(0x55807105e370 0x5580710db860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:58:59.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.027+0000 7f733dd1e740 4 client.15549 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:58:59.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.027+0000 7f733dd1e740 1 -- 172.21.15.5:0/3351549493 --> [v2:172.21.15.148:6834/4195086427,v1:172.21.15.148:6835/4195086427] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558070fc0d70 con 0x55807105e370 2024-09-12T02:58:59.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.027+0000 7f73396e1640 1 --2- 172.21.15.5:0/3351549493 >> [v2:172.21.15.148:6834/4195086427,v1:172.21.15.148:6835/4195086427] conn(0x55807105e370 0x5580710db860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:58:59.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.027+0000 7f73396e1640 1 --2- 172.21.15.5:0/3351549493 >> [v2:172.21.15.148:6834/4195086427,v1:172.21.15.148:6835/4195086427] conn(0x55807105e370 0x5580710db860 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=71c08cf2e9cad42b server_cookie=5f97b4eec37202bd in_seq=0 out_seq=0 2024-09-12T02:58:59.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.027+0000 7f731cff9640 10 client.15549 ms_handle_connect on v2:172.21.15.148:6834/4195086427 2024-09-12T02:58:59.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.027+0000 7f731cff9640 1 -- 172.21.15.5:0/3351549493 <== mds.2 v2:172.21.15.148:6834/4195086427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558070fc0d70 con 0x55807105e370 2024-09-12T02:58:59.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.028+0000 7f731cff9640 10 client.15549 handle_command_reply: tid=0 2024-09-12T02:58:59.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.028+0000 7f733dd1e740 1 client.15549 shutdown 2024-09-12T02:58:59.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.028+0000 7f733dd1e740 1 -- 172.21.15.5:0/3351549493 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55807101c310 msgr2=0x5580710668e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:59.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.028+0000 7f733dd1e740 1 --2- 172.21.15.5:0/3351549493 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55807101c310 0x5580710668e0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f7324004190 tx=0x7f73240096b0 comp rx=0 tx=0).stop 2024-09-12T02:58:59.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.028+0000 7f733dd1e740 1 -- 172.21.15.5:0/3351549493 shutdown_connections 2024-09-12T02:58:59.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.028+0000 7f733dd1e740 1 --2- 172.21.15.5:0/3351549493 >> [v2:172.21.15.148:6834/4195086427,v1:172.21.15.148:6835/4195086427] conn(0x55807105e370 0x5580710db860 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:59.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.028+0000 7f733dd1e740 1 --2- 172.21.15.5:0/3351549493 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55807105bcc0 0x558071058b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:59.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.028+0000 7f733dd1e740 1 --2- 172.21.15.5:0/3351549493 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5580710535d0 0x5580710585e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:59.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.028+0000 7f733dd1e740 1 --2- 172.21.15.5:0/3351549493 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55807101c310 0x5580710668e0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:59.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.028+0000 7f733dd1e740 1 -- 172.21.15.5:0/3351549493 >> 172.21.15.5:0/3351549493 conn(0x55807105b8e0 msgr2=0x55807104f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:59.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.028+0000 7f731cff9640 0 client.15549 ms_handle_reset on v2:172.21.15.148:6834/4195086427 2024-09-12T02:58:59.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.028+0000 7f733dd1e740 1 -- 172.21.15.5:0/3351549493 shutdown_connections 2024-09-12T02:58:59.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.028+0000 7f733dd1e740 1 -- 172.21.15.5:0/3351549493 wait complete. 2024-09-12T02:58:59.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.028+0000 7f733dd1e740 20 client.15549 trim_cache size 0 max 16384 2024-09-12T02:58:59.035 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-12T02:58:59.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.029+0000 7f733b945640 1 -- 172.21.15.5:0/3036742663 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f73200a2c90 msgr2=0x7f73200a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:59.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.029+0000 7f733b945640 1 --2- 172.21.15.5:0/3036742663 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f73200a2c90 0x7f73200a5150 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f732c09b760 tx=0x7f732c0666e0 comp rx=0 tx=0).stop 2024-09-12T02:58:59.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.029+0000 7f733b945640 1 -- 172.21.15.5:0/3036742663 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73341082e0 msgr2=0x7f7334101490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:58:59.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.029+0000 7f733b945640 1 --2- 172.21.15.5:0/3036742663 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73341082e0 0x7f7334101490 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f732c00eaa0 tx=0x7f732c00ef60 comp rx=0 tx=0).stop 2024-09-12T02:58:59.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.029+0000 7f733b945640 1 -- 172.21.15.5:0/3036742663 shutdown_connections 2024-09-12T02:58:59.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.029+0000 7f733b945640 1 --2- 172.21.15.5:0/3036742663 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f73200a2c90 0x7f73200a5150 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:59.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.029+0000 7f733b945640 1 --2- 172.21.15.5:0/3036742663 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f73341082e0 0x7f7334101490 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:59.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.029+0000 7f733b945640 1 --2- 172.21.15.5:0/3036742663 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7334103760 0x7f7334100f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:59.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.029+0000 7f733b945640 1 --2- 172.21.15.5:0/3036742663 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7334102db0 0x7f7334100a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:58:59.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.029+0000 7f733b945640 1 -- 172.21.15.5:0/3036742663 >> 172.21.15.5:0/3036742663 conn(0x7f73340fc7d0 msgr2=0x7f7334105d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:58:59.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.029+0000 7f733b945640 1 -- 172.21.15.5:0/3036742663 shutdown_connections 2024-09-12T02:58:59.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:58:59.029+0000 7f733b945640 1 -- 172.21.15.5:0/3036742663 wait complete. 2024-09-12T02:58:59.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:58:58 smithi146 ceph-mon[35011]: pgmap v357: 97 pgs: 97 active+clean; 658 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-12T02:58:59.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:58:58 smithi005 ceph-mon[31197]: pgmap v357: 97 pgs: 97 active+clean; 658 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-12T02:58:59.415 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:58:58 smithi148 ceph-mon[35090]: pgmap v357: 97 pgs: 97 active+clean; 658 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-12T02:58:59.688 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T02:58:59.980 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:59:00.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.490+0000 7f159147f640 1 -- 172.21.15.5:0/3999184701 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f158c102d90 msgr2=0x7f158c103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:00.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.490+0000 7f159147f640 1 --2- 172.21.15.5:0/3999184701 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f158c102d90 0x7f158c103170 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f157c009930 tx=0x7f157c02a420 comp rx=0 tx=0).stop 2024-09-12T02:59:00.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.490+0000 7f159147f640 1 -- 172.21.15.5:0/3999184701 shutdown_connections 2024-09-12T02:59:00.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.490+0000 7f159147f640 1 --2- 172.21.15.5:0/3999184701 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f158c108290 0x7f158c10ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:00.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.490+0000 7f159147f640 1 --2- 172.21.15.5:0/3999184701 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f158c103740 0x7f158c107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:00.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.490+0000 7f159147f640 1 --2- 172.21.15.5:0/3999184701 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f158c102d90 0x7f158c103170 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:00.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.490+0000 7f159147f640 1 -- 172.21.15.5:0/3999184701 >> 172.21.15.5:0/3999184701 conn(0x7f158c0fc7d0 msgr2=0x7f158c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:59:00.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.490+0000 7f159147f640 1 -- 172.21.15.5:0/3999184701 shutdown_connections 2024-09-12T02:59:00.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.490+0000 7f159147f640 1 -- 172.21.15.5:0/3999184701 wait complete. 2024-09-12T02:59:00.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.491+0000 7f159147f640 1 Processor -- start 2024-09-12T02:59:00.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.491+0000 7f159147f640 1 -- start start 2024-09-12T02:59:00.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.491+0000 7f159147f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f158c102d90 0x7f158c1009f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:00.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.491+0000 7f159147f640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f158c103740 0x7f158c100f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:00.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.492+0000 7f159147f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f158c108290 0x7f158c101470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:00.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.492+0000 7f159147f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f158c112090 con 0x7f158c108290 2024-09-12T02:59:00.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.492+0000 7f159147f640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f158c111f10 con 0x7f158c103740 2024-09-12T02:59:00.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.492+0000 7f159147f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f158c112210 con 0x7f158c102d90 2024-09-12T02:59:00.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.492+0000 7f158b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f158c108290 0x7f158c101470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:00.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.492+0000 7f158b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f158c108290 0x7f158c101470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48918/0 (socket says 172.21.15.5:48918) 2024-09-12T02:59:00.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.492+0000 7f158b7fe640 1 -- 172.21.15.5:0/391857336 learned_addr learned my addr 172.21.15.5:0/391857336 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:59:00.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.492+0000 7f158affd640 1 --2- 172.21.15.5:0/391857336 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f158c102d90 0x7f158c1009f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:00.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.492+0000 7f158a7fc640 1 --2- 172.21.15.5:0/391857336 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f158c103740 0x7f158c100f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:00.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.492+0000 7f158b7fe640 1 -- 172.21.15.5:0/391857336 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f158c102d90 msgr2=0x7f158c1009f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:00.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.492+0000 7f158b7fe640 1 --2- 172.21.15.5:0/391857336 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f158c102d90 0x7f158c1009f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:00.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.492+0000 7f158b7fe640 1 -- 172.21.15.5:0/391857336 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f158c103740 msgr2=0x7f158c100f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:00.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.492+0000 7f158b7fe640 1 --2- 172.21.15.5:0/391857336 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f158c103740 0x7f158c100f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:00.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.492+0000 7f158b7fe640 1 -- 172.21.15.5:0/391857336 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f158c1a6930 con 0x7f158c108290 2024-09-12T02:59:00.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.492+0000 7f158b7fe640 1 --2- 172.21.15.5:0/391857336 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f158c108290 0x7f158c101470 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f158400cce0 tx=0x7f158400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:59:00.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.493+0000 7f1573fff640 1 -- 172.21.15.5:0/391857336 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f158400f450 con 0x7f158c108290 2024-09-12T02:59:00.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.493+0000 7f1573fff640 1 -- 172.21.15.5:0/391857336 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f158400fad0 con 0x7f158c108290 2024-09-12T02:59:00.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.493+0000 7f1573fff640 1 -- 172.21.15.5:0/391857336 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1584018b60 con 0x7f158c108290 2024-09-12T02:59:00.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.493+0000 7f159147f640 1 -- 172.21.15.5:0/391857336 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f158c1a6c20 con 0x7f158c108290 2024-09-12T02:59:00.495 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.493+0000 7f159147f640 1 -- 172.21.15.5:0/391857336 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f158c1a7110 con 0x7f158c108290 2024-09-12T02:59:00.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.496+0000 7f1573fff640 1 -- 172.21.15.5:0/391857336 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7f1584018d00 con 0x7f158c108290 2024-09-12T02:59:00.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.496+0000 7f1593858740 1 Processor -- start 2024-09-12T02:59:00.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.496+0000 7f1593858740 1 -- start start 2024-09-12T02:59:00.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.497+0000 7f1593858740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562ce0a03b40 0x562ce0a02090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:00.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.497+0000 7f1593858740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ce0a025d0 0x562ce0a02a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:00.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.497+0000 7f1593858740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562ce0a0d370 0x562ce0a0f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:00.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.497+0000 7f1593858740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562ce09682f0 con 0x562ce0a025d0 2024-09-12T02:59:00.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.497+0000 7f1593858740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x562ce090fcf0 con 0x562ce0a03b40 2024-09-12T02:59:00.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.497+0000 7f1593858740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x562ce09642b0 con 0x562ce0a0d370 2024-09-12T02:59:00.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.497+0000 7f158affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ce0a025d0 0x562ce0a02a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:00.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.497+0000 7f158affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ce0a025d0 0x562ce0a02a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48928/0 (socket says 172.21.15.5:48928) 2024-09-12T02:59:00.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.497+0000 7f158affd640 1 -- 172.21.15.5:0/1214024816 learned_addr learned my addr 172.21.15.5:0/1214024816 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:59:00.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.497+0000 7f1573fff640 1 --2- 172.21.15.5:0/391857336 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f15740a2c90 0x7f15740a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:00.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.497+0000 7f158affd640 1 -- 172.21.15.5:0/1214024816 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562ce0a0d370 msgr2=0x562ce0a0f760 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T02:59:00.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.497+0000 7f158affd640 1 --2- 172.21.15.5:0/1214024816 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562ce0a0d370 0x562ce0a0f760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:00.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.497+0000 7f158affd640 1 -- 172.21.15.5:0/1214024816 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562ce0a03b40 msgr2=0x562ce0a02090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:00.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.497+0000 7f158affd640 1 --2- 172.21.15.5:0/1214024816 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562ce0a03b40 0x562ce0a02090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:00.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.497+0000 7f158affd640 1 -- 172.21.15.5:0/1214024816 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ce08f43a0 con 0x562ce0a025d0 2024-09-12T02:59:00.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.497+0000 7f1573fff640 1 -- 172.21.15.5:0/391857336 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f15840cf460 con 0x7f158c108290 2024-09-12T02:59:00.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.498+0000 7f158b7fe640 1 --2- 172.21.15.5:0/391857336 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f15740a2c90 0x7f15740a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:00.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.498+0000 7f158affd640 1 --2- 172.21.15.5:0/1214024816 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ce0a025d0 0x562ce0a02a10 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f157c009930 tx=0x7f157c005220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98519db6cb4ea3af server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:59:00.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.498+0000 7f158b7fe640 1 --2- 172.21.15.5:0/391857336 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f15740a2c90 0x7f15740a5150 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f1584098540 tx=0x7f15840077a0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:59:00.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.498+0000 7f15717fa640 1 -- 172.21.15.5:0/1214024816 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f157c035020 con 0x562ce0a025d0 2024-09-12T02:59:00.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.498+0000 7f15717fa640 1 -- 172.21.15.5:0/1214024816 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f157c041070 con 0x562ce0a025d0 2024-09-12T02:59:00.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.498+0000 7f15717fa640 1 -- 172.21.15.5:0/1214024816 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f157c0047a0 con 0x562ce0a025d0 2024-09-12T02:59:00.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.499+0000 7f1593858740 1 -- 172.21.15.5:0/1214024816 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ce0a025d0 msgr2=0x562ce0a02a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:00.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.499+0000 7f1593858740 1 --2- 172.21.15.5:0/1214024816 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ce0a025d0 0x562ce0a02a10 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f157c009930 tx=0x7f157c005220 comp rx=0 tx=0).stop 2024-09-12T02:59:00.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.500+0000 7f1593858740 1 -- 172.21.15.5:0/1214024816 shutdown_connections 2024-09-12T02:59:00.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.500+0000 7f1593858740 1 --2- 172.21.15.5:0/1214024816 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ce0a025d0 0x562ce0a02a10 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:00.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.500+0000 7f1593858740 1 --2- 172.21.15.5:0/1214024816 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562ce0a0d370 0x562ce0a0f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:00.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.500+0000 7f1593858740 1 --2- 172.21.15.5:0/1214024816 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562ce0a03b40 0x562ce0a02090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:00.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.500+0000 7f1593858740 1 -- 172.21.15.5:0/1214024816 >> 172.21.15.5:0/1214024816 conn(0x562ce09cb310 msgr2=0x562ce09fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:59:00.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.500+0000 7f1593858740 1 -- 172.21.15.5:0/1214024816 shutdown_connections 2024-09-12T02:59:00.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.501+0000 7f1593858740 1 -- 172.21.15.5:0/1214024816 wait complete. 2024-09-12T02:59:00.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.501+0000 7f1593858740 1 -- start start 2024-09-12T02:59:00.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.501+0000 7f1593858740 1 Processor -- start 2024-09-12T02:59:00.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.501+0000 7f1593858740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562ce0a025d0 0x562ce0a0cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:00.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.501+0000 7f1593858740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ce0a03b40 0x562ce0a0a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:00.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.501+0000 7f1593858740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562ce0a0d370 0x562ce0a0a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:00.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.501+0000 7f1593858740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562ce0a10010 con 0x562ce0a03b40 2024-09-12T02:59:00.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.501+0000 7f1593858740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x562ce0923060 con 0x562ce0a0d370 2024-09-12T02:59:00.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.501+0000 7f1593858740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x562ce0a10190 con 0x562ce0a025d0 2024-09-12T02:59:00.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.501+0000 7f1593858740 10 client.0 _ll_register_callbacks cb 0x562ce06768c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:59:00.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.502+0000 7f158affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ce0a03b40 0x562ce0a0a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:00.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.502+0000 7f158a7fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562ce0a025d0 0x562ce0a0cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:00.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.502+0000 7f158affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ce0a03b40 0x562ce0a0a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48930/0 (socket says 172.21.15.5:48930) 2024-09-12T02:59:00.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.502+0000 7f158affd640 1 -- 172.21.15.5:0/2801612715 learned_addr learned my addr 172.21.15.5:0/2801612715 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:59:00.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.502+0000 7f158a7fc640 1 --2- 172.21.15.5:0/2801612715 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562ce0a0d370 0x562ce0a0a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:00.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.502+0000 7f158affd640 1 -- 172.21.15.5:0/2801612715 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562ce0a025d0 msgr2=0x562ce0a0cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:00.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.502+0000 7f158affd640 1 --2- 172.21.15.5:0/2801612715 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562ce0a025d0 0x562ce0a0cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:00.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.502+0000 7f158affd640 1 -- 172.21.15.5:0/2801612715 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562ce0a0d370 msgr2=0x562ce0a0a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:00.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.502+0000 7f158affd640 1 --2- 172.21.15.5:0/2801612715 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562ce0a0d370 0x562ce0a0a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:00.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.502+0000 7f158affd640 1 -- 172.21.15.5:0/2801612715 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ce06d93c0 con 0x562ce0a03b40 2024-09-12T02:59:00.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.502+0000 7f158a7fc640 1 --2- 172.21.15.5:0/2801612715 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562ce0a025d0 0x562ce0a0cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:59:00.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.502+0000 7f158affd640 1 --2- 172.21.15.5:0/2801612715 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ce0a03b40 0x562ce0a0a020 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f157c02a460 tx=0x7f157c031420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25c1f16c289ab73f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:59:00.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.502+0000 7f156eff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T02:59:00.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.503+0000 7f156eff5640 1 -- 172.21.15.5:0/2801612715 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f157c035070 con 0x562ce0a03b40 2024-09-12T02:59:00.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.503+0000 7f1593858740 1 -- 172.21.15.5:0/2801612715 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f157c041070 con 0x562ce0a03b40 2024-09-12T02:59:00.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.503+0000 7f156eff5640 1 -- 172.21.15.5:0/2801612715 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f157c004bb0 con 0x562ce0a03b40 2024-09-12T02:59:00.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.503+0000 7f156eff5640 1 -- 172.21.15.5:0/2801612715 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f157c032e40 con 0x562ce0a03b40 2024-09-12T02:59:00.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.504+0000 7f1593858740 10 client.15561 fetch_fsmap learned FSMap version 26 2024-09-12T02:59:00.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.504+0000 7f1593858740 1 -- 172.21.15.5:0/2801612715 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562ce0a104f0 con 0x562ce0a03b40 2024-09-12T02:59:00.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.504+0000 7f156eff5640 1 -- 172.21.15.5:0/2801612715 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f157c041070 con 0x562ce0a03b40 2024-09-12T02:59:00.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.504+0000 7f156eff5640 1 -- 172.21.15.5:0/2801612715 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f157c03a040 con 0x562ce0a03b40 2024-09-12T02:59:00.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.504+0000 7f1593858740 10 client.15561 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T02:59:00.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.504+0000 7f1593858740 10 client.15561 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-12T02:59:00.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.504+0000 7f1593858740 20 client.15561 populate_metadata read hostname 'smithi005' 2024-09-12T02:59:00.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.504+0000 7f1593858740 1 --2- 172.21.15.5:0/2801612715 >> [v2:172.21.15.148:6836/2606873922,v1:172.21.15.148:6837/2606873922] conn(0x562ce0a11a80 0x562ce0a13ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:00.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.504+0000 7f1593858740 4 client.15561 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:59:00.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.504+0000 7f1593858740 1 -- 172.21.15.5:0/2801612715 --> [v2:172.21.15.148:6836/2606873922,v1:172.21.15.148:6837/2606873922] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562ce0975860 con 0x562ce0a11a80 2024-09-12T02:59:00.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.504+0000 7f158a7fc640 1 --2- 172.21.15.5:0/2801612715 >> [v2:172.21.15.148:6836/2606873922,v1:172.21.15.148:6837/2606873922] conn(0x562ce0a11a80 0x562ce0a13ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:00.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.505+0000 7f156eff5640 10 client.15561 ms_handle_connect on v2:172.21.15.148:6836/2606873922 2024-09-12T02:59:00.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.505+0000 7f158a7fc640 1 --2- 172.21.15.5:0/2801612715 >> [v2:172.21.15.148:6836/2606873922,v1:172.21.15.148:6837/2606873922] conn(0x562ce0a11a80 0x562ce0a13ea0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cadb3900d18d1c36 server_cookie=eccae726c6f349a7 in_seq=0 out_seq=0 2024-09-12T02:59:00.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.506+0000 7f156eff5640 1 -- 172.21.15.5:0/2801612715 <== mds.1 v2:172.21.15.148:6836/2606873922 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562ce0975860 con 0x562ce0a11a80 2024-09-12T02:59:00.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.506+0000 7f156eff5640 10 client.15561 handle_command_reply: tid=0 2024-09-12T02:59:00.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.506+0000 7f1593858740 1 client.15561 shutdown 2024-09-12T02:59:00.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.506+0000 7f1593858740 1 -- 172.21.15.5:0/2801612715 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ce0a03b40 msgr2=0x562ce0a0a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:00.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.506+0000 7f1593858740 1 --2- 172.21.15.5:0/2801612715 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ce0a03b40 0x562ce0a0a020 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f157c02a460 tx=0x7f157c031420 comp rx=0 tx=0).stop 2024-09-12T02:59:00.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.506+0000 7f1593858740 1 -- 172.21.15.5:0/2801612715 shutdown_connections 2024-09-12T02:59:00.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.506+0000 7f1593858740 1 --2- 172.21.15.5:0/2801612715 >> [v2:172.21.15.148:6836/2606873922,v1:172.21.15.148:6837/2606873922] conn(0x562ce0a11a80 0x562ce0a13ea0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:00.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.506+0000 7f1593858740 1 --2- 172.21.15.5:0/2801612715 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ce0a03b40 0x562ce0a0a020 secure :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f157c02a460 tx=0x7f157c031420 comp rx=0 tx=0).stop 2024-09-12T02:59:00.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.506+0000 7f1593858740 1 --2- 172.21.15.5:0/2801612715 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562ce0a0d370 0x562ce0a0a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:00.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.506+0000 7f1593858740 1 --2- 172.21.15.5:0/2801612715 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562ce0a025d0 0x562ce0a0cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:00.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.506+0000 7f1593858740 1 -- 172.21.15.5:0/2801612715 >> 172.21.15.5:0/2801612715 conn(0x562ce09cb310 msgr2=0x562ce0a04930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:59:00.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.506+0000 7f156eff5640 0 client.15561 ms_handle_reset on v2:172.21.15.148:6836/2606873922 2024-09-12T02:59:00.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.506+0000 7f1593858740 1 -- 172.21.15.5:0/2801612715 shutdown_connections 2024-09-12T02:59:00.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.506+0000 7f1593858740 1 -- 172.21.15.5:0/2801612715 wait complete. 2024-09-12T02:59:00.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.506+0000 7f1593858740 20 client.15561 trim_cache size 0 max 16384 2024-09-12T02:59:00.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.532+0000 7f1593858740 1 Processor -- start 2024-09-12T02:59:00.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.532+0000 7f1593858740 1 -- start start 2024-09-12T02:59:00.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.532+0000 7f1593858740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562ce0a025d0 0x562ce0a09ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:00.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.532+0000 7f1593858740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ce0a0d370 0x562ce0a21790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:00.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.532+0000 7f1593858740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562ce09fef20 0x562ce0a07280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:00.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.532+0000 7f1593858740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f157800e400 con 0x562ce0a0d370 2024-09-12T02:59:00.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.532+0000 7f1593858740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f157800ff40 con 0x562ce09fef20 2024-09-12T02:59:00.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.532+0000 7f1593858740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f157800ead0 con 0x562ce0a025d0 2024-09-12T02:59:00.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.532+0000 7f158affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ce0a0d370 0x562ce0a21790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:00.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.532+0000 7f158affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ce0a0d370 0x562ce0a21790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48942/0 (socket says 172.21.15.5:48942) 2024-09-12T02:59:00.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.532+0000 7f158affd640 1 -- 172.21.15.5:0/1346952385 learned_addr learned my addr 172.21.15.5:0/1346952385 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:59:00.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.532+0000 7f158a7fc640 1 --2- 172.21.15.5:0/1346952385 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562ce0a025d0 0x562ce0a09ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:00.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.533+0000 7f158a7fc640 1 --2- 172.21.15.5:0/1346952385 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562ce09fef20 0x562ce0a07280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:00.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.533+0000 7f158affd640 1 -- 172.21.15.5:0/1346952385 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562ce0a025d0 msgr2=0x562ce0a09ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:00.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.533+0000 7f158affd640 1 --2- 172.21.15.5:0/1346952385 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562ce0a025d0 0x562ce0a09ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:00.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.533+0000 7f158affd640 1 -- 172.21.15.5:0/1346952385 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562ce09fef20 msgr2=0x562ce0a07280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:00.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.533+0000 7f158affd640 1 --2- 172.21.15.5:0/1346952385 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562ce09fef20 0x562ce0a07280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:00.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.533+0000 7f158affd640 1 -- 172.21.15.5:0/1346952385 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ce09ff960 con 0x562ce0a0d370 2024-09-12T02:59:00.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.533+0000 7f158affd640 1 --2- 172.21.15.5:0/1346952385 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ce0a0d370 0x562ce0a21790 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f157c02a940 tx=0x7f157c046f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43ede1965d3443a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:59:00.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.533+0000 7f1570ff9640 1 -- 172.21.15.5:0/1346952385 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f157c06f070 con 0x562ce0a0d370 2024-09-12T02:59:00.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.533+0000 7f1570ff9640 1 -- 172.21.15.5:0/1346952385 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f157c041070 con 0x562ce0a0d370 2024-09-12T02:59:00.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.533+0000 7f1570ff9640 1 -- 172.21.15.5:0/1346952385 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f157c035070 con 0x562ce0a0d370 2024-09-12T02:59:00.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.534+0000 7f1593858740 1 -- 172.21.15.5:0/1346952385 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ce0a0d370 msgr2=0x562ce0a21790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:00.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.534+0000 7f1593858740 1 --2- 172.21.15.5:0/1346952385 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ce0a0d370 0x562ce0a21790 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f157c02a940 tx=0x7f157c046f50 comp rx=0 tx=0).stop 2024-09-12T02:59:00.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.534+0000 7f1593858740 1 -- 172.21.15.5:0/1346952385 shutdown_connections 2024-09-12T02:59:00.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.534+0000 7f1593858740 1 --2- 172.21.15.5:0/1346952385 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ce0a0d370 0x562ce0a21790 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:00.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.534+0000 7f1593858740 1 --2- 172.21.15.5:0/1346952385 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562ce09fef20 0x562ce0a07280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:00.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.534+0000 7f1593858740 1 --2- 172.21.15.5:0/1346952385 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562ce0a025d0 0x562ce0a09ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:00.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.534+0000 7f1593858740 1 -- 172.21.15.5:0/1346952385 >> 172.21.15.5:0/1346952385 conn(0x562ce09cb310 msgr2=0x562ce0a098e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:59:00.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.535+0000 7f1593858740 1 -- 172.21.15.5:0/1346952385 shutdown_connections 2024-09-12T02:59:00.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.535+0000 7f1593858740 1 -- 172.21.15.5:0/1346952385 wait complete. 2024-09-12T02:59:00.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.535+0000 7f1593858740 1 -- start start 2024-09-12T02:59:00.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.535+0000 7f1593858740 1 Processor -- start 2024-09-12T02:59:00.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.535+0000 7f1593858740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562ce09fef20 0x562ce0a7f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:00.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.536+0000 7f1593858740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562ce0a025d0 0x562ce0a7fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:00.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.536+0000 7f1593858740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ce0a0d370 0x562ce0a258d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:00.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.536+0000 7f1593858740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f157800f870 con 0x562ce0a0d370 2024-09-12T02:59:00.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.536+0000 7f1593858740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f157800f1a0 con 0x562ce09fef20 2024-09-12T02:59:00.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.536+0000 7f1593858740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f157800c3e0 con 0x562ce0a025d0 2024-09-12T02:59:00.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.536+0000 7f1593858740 10 client.0 _ll_register_callbacks cb 0x7f157c031840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:59:00.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.536+0000 7f158a7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ce0a0d370 0x562ce0a258d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:00.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.536+0000 7f158affd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562ce0a025d0 0x562ce0a7fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:00.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.536+0000 7f158affd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562ce0a025d0 0x562ce0a7fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.5:44116/0 (socket says 172.21.15.5:44116) 2024-09-12T02:59:00.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.536+0000 7f158affd640 1 -- 172.21.15.5:0/3767781939 learned_addr learned my addr 172.21.15.5:0/3767781939 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:59:00.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.536+0000 7f158a7fc640 1 --2- 172.21.15.5:0/3767781939 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562ce09fef20 0x562ce0a7f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:00.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.536+0000 7f158a7fc640 1 -- 172.21.15.5:0/3767781939 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562ce0a025d0 msgr2=0x562ce0a7fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:00.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.536+0000 7f158a7fc640 1 --2- 172.21.15.5:0/3767781939 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562ce0a025d0 0x562ce0a7fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:00.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.536+0000 7f158a7fc640 1 -- 172.21.15.5:0/3767781939 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562ce09fef20 msgr2=0x562ce0a7f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:00.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.536+0000 7f158a7fc640 1 --2- 172.21.15.5:0/3767781939 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562ce09fef20 0x562ce0a7f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:00.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.536+0000 7f158a7fc640 1 -- 172.21.15.5:0/3767781939 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ce0a26100 con 0x562ce0a0d370 2024-09-12T02:59:00.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.537+0000 7f158a7fc640 1 --2- 172.21.15.5:0/3767781939 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ce0a0d370 0x562ce0a258d0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x562ce0a146c0 tx=0x7f1580013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99dd20512177756e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:59:00.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.537+0000 7f156eff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T02:59:00.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.537+0000 7f156eff5640 1 -- 172.21.15.5:0/3767781939 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f158000fd70 con 0x562ce0a0d370 2024-09-12T02:59:00.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.537+0000 7f156eff5640 1 -- 172.21.15.5:0/3767781939 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f158001fe80 con 0x562ce0a0d370 2024-09-12T02:59:00.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.537+0000 7f1593858740 1 -- 172.21.15.5:0/3767781939 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f157c041070 con 0x562ce0a0d370 2024-09-12T02:59:00.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.537+0000 7f156eff5640 1 -- 172.21.15.5:0/3767781939 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f158001f350 con 0x562ce0a0d370 2024-09-12T02:59:00.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.537+0000 7f1593858740 10 client.15567 fetch_fsmap learned FSMap version 26 2024-09-12T02:59:00.541 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.537+0000 7f1593858740 1 -- 172.21.15.5:0/3767781939 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562ce0a86e90 con 0x562ce0a0d370 2024-09-12T02:59:00.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.538+0000 7f156eff5640 1 -- 172.21.15.5:0/3767781939 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f157c041070 con 0x562ce0a0d370 2024-09-12T02:59:00.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.538+0000 7f156eff5640 1 -- 172.21.15.5:0/3767781939 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f158002b070 con 0x562ce0a0d370 2024-09-12T02:59:00.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.538+0000 7f1593858740 10 client.15567 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T02:59:00.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.538+0000 7f1593858740 10 client.15567 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-12T02:59:00.543 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.538+0000 7f1593858740 20 client.15567 populate_metadata read hostname 'smithi005' 2024-09-12T02:59:00.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.538+0000 7f1593858740 1 --2- 172.21.15.5:0/3767781939 >> [v2:172.21.15.148:6836/2606873922,v1:172.21.15.148:6837/2606873922] conn(0x562ce0a87560 0x562ce0a89950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:00.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.538+0000 7f1593858740 4 client.15567 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:59:00.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.538+0000 7f1593858740 1 -- 172.21.15.5:0/3767781939 --> [v2:172.21.15.148:6836/2606873922,v1:172.21.15.148:6837/2606873922] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562ce096fd70 con 0x562ce0a87560 2024-09-12T02:59:00.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.539+0000 7f158affd640 1 --2- 172.21.15.5:0/3767781939 >> [v2:172.21.15.148:6836/2606873922,v1:172.21.15.148:6837/2606873922] conn(0x562ce0a87560 0x562ce0a89950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:00.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.539+0000 7f156eff5640 10 client.15567 ms_handle_connect on v2:172.21.15.148:6836/2606873922 2024-09-12T02:59:00.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.539+0000 7f158affd640 1 --2- 172.21.15.5:0/3767781939 >> [v2:172.21.15.148:6836/2606873922,v1:172.21.15.148:6837/2606873922] conn(0x562ce0a87560 0x562ce0a89950 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=52f16d8284d4cae6 server_cookie=3eaf7b856aefc8b0 in_seq=0 out_seq=0 2024-09-12T02:59:00.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.539+0000 7f156eff5640 1 -- 172.21.15.5:0/3767781939 <== mds.1 v2:172.21.15.148:6836/2606873922 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562ce096fd70 con 0x562ce0a87560 2024-09-12T02:59:00.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.539+0000 7f156eff5640 10 client.15567 handle_command_reply: tid=0 2024-09-12T02:59:00.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.539+0000 7f1593858740 1 client.15567 shutdown 2024-09-12T02:59:00.544 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.539+0000 7f1593858740 1 -- 172.21.15.5:0/3767781939 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ce0a0d370 msgr2=0x562ce0a258d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:00.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.540+0000 7f1593858740 1 --2- 172.21.15.5:0/3767781939 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ce0a0d370 0x562ce0a258d0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x562ce0a146c0 tx=0x7f1580013040 comp rx=0 tx=0).stop 2024-09-12T02:59:00.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.540+0000 7f1593858740 1 -- 172.21.15.5:0/3767781939 shutdown_connections 2024-09-12T02:59:00.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.540+0000 7f1593858740 1 --2- 172.21.15.5:0/3767781939 >> [v2:172.21.15.148:6836/2606873922,v1:172.21.15.148:6837/2606873922] conn(0x562ce0a87560 0x562ce0a89950 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:00.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.540+0000 7f1593858740 1 --2- 172.21.15.5:0/3767781939 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562ce0a0d370 0x562ce0a258d0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:00.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.540+0000 7f1593858740 1 --2- 172.21.15.5:0/3767781939 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562ce0a025d0 0x562ce0a7fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:00.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.540+0000 7f1593858740 1 --2- 172.21.15.5:0/3767781939 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562ce09fef20 0x562ce0a7f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:00.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.540+0000 7f1593858740 1 -- 172.21.15.5:0/3767781939 >> 172.21.15.5:0/3767781939 conn(0x562ce09cb310 msgr2=0x562ce0a04110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:59:00.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.540+0000 7f156eff5640 0 client.15567 ms_handle_reset on v2:172.21.15.148:6836/2606873922 2024-09-12T02:59:00.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.540+0000 7f1593858740 1 -- 172.21.15.5:0/3767781939 shutdown_connections 2024-09-12T02:59:00.545 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.540+0000 7f1593858740 1 -- 172.21.15.5:0/3767781939 wait complete. 2024-09-12T02:59:00.546 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.540+0000 7f1593858740 20 client.15567 trim_cache size 0 max 16384 2024-09-12T02:59:00.546 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-12T02:59:00.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.541+0000 7f159147f640 1 -- 172.21.15.5:0/391857336 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f15740a2c90 msgr2=0x7f15740a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:00.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.541+0000 7f159147f640 1 --2- 172.21.15.5:0/391857336 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f15740a2c90 0x7f15740a5150 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f1584098540 tx=0x7f15840077a0 comp rx=0 tx=0).stop 2024-09-12T02:59:00.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.541+0000 7f159147f640 1 -- 172.21.15.5:0/391857336 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f158c108290 msgr2=0x7f158c101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:00.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.541+0000 7f159147f640 1 --2- 172.21.15.5:0/391857336 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f158c108290 0x7f158c101470 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f158400cce0 tx=0x7f158400a620 comp rx=0 tx=0).stop 2024-09-12T02:59:00.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.541+0000 7f159147f640 1 -- 172.21.15.5:0/391857336 shutdown_connections 2024-09-12T02:59:00.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.541+0000 7f159147f640 1 --2- 172.21.15.5:0/391857336 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f15740a2c90 0x7f15740a5150 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:00.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.541+0000 7f159147f640 1 --2- 172.21.15.5:0/391857336 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f158c108290 0x7f158c101470 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:00.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.541+0000 7f159147f640 1 --2- 172.21.15.5:0/391857336 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f158c103740 0x7f158c100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:00.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.541+0000 7f159147f640 1 --2- 172.21.15.5:0/391857336 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f158c102d90 0x7f158c1009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:00.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.541+0000 7f159147f640 1 -- 172.21.15.5:0/391857336 >> 172.21.15.5:0/391857336 conn(0x7f158c0fc7d0 msgr2=0x7f158c0fe390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:59:00.547 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.541+0000 7f159147f640 1 -- 172.21.15.5:0/391857336 shutdown_connections 2024-09-12T02:59:00.548 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:00.542+0000 7f159147f640 1 -- 172.21.15.5:0/391857336 wait complete. 2024-09-12T02:59:01.155 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:59:00 smithi005 ceph-mon[31197]: pgmap v358: 97 pgs: 97 active+clean; 663 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-12T02:59:01.195 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T02:59:01.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:59:00 smithi146 ceph-mon[35011]: pgmap v358: 97 pgs: 97 active+clean; 663 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-12T02:59:01.415 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:59:00 smithi148 ceph-mon[35090]: pgmap v358: 97 pgs: 97 active+clean; 663 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-12T02:59:01.465 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:59:01.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:01.989+0000 7f4419508640 1 -- 172.21.15.5:0/2107094520 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4414100100 msgr2=0x7f4414100560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:01.995 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:01.989+0000 7f4419508640 1 --2- 172.21.15.5:0/2107094520 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4414100100 0x7f4414100560 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f4408009900 tx=0x7f440802a410 comp rx=0 tx=0).stop 2024-09-12T02:59:01.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:01.989+0000 7f4419508640 1 -- 172.21.15.5:0/2107094520 shutdown_connections 2024-09-12T02:59:01.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:01.989+0000 7f4419508640 1 --2- 172.21.15.5:0/2107094520 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4414100100 0x7f4414100560 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:01.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:01.989+0000 7f4419508640 1 --2- 172.21.15.5:0/2107094520 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4414100aa0 0x7f441410ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:01.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:01.989+0000 7f4419508640 1 --2- 172.21.15.5:0/2107094520 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4414106b80 0x7f4414106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:01.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:01.989+0000 7f4419508640 1 -- 172.21.15.5:0/2107094520 >> 172.21.15.5:0/2107094520 conn(0x7f4414077230 msgr2=0x7f44140fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:59:01.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:01.989+0000 7f4419508640 1 -- 172.21.15.5:0/2107094520 shutdown_connections 2024-09-12T02:59:01.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:01.989+0000 7f4419508640 1 -- 172.21.15.5:0/2107094520 wait complete. 2024-09-12T02:59:01.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:01.990+0000 7f4419508640 1 Processor -- start 2024-09-12T02:59:01.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:01.990+0000 7f4419508640 1 -- start start 2024-09-12T02:59:01.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:01.990+0000 7f4419508640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4414100100 0x7f44141012e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:01.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:01.990+0000 7f4419508640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4414100aa0 0x7f4414101820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:01.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:01.990+0000 7f4419508640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4414106b80 0x7f4414101d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:01.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:01.990+0000 7f4419508640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f44141120a0 con 0x7f4414106b80 2024-09-12T02:59:01.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:01.990+0000 7f4419508640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f4414111f20 con 0x7f4414100100 2024-09-12T02:59:01.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:01.990+0000 7f4419508640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4414112220 con 0x7f4414100aa0 2024-09-12T02:59:01.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:01.991+0000 7f44127fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4414100aa0 0x7f4414101820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:01.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:01.991+0000 7f4412ffd640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4414100100 0x7f44141012e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:01.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:01.991+0000 7f44127fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4414100aa0 0x7f4414101820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.5:44138/0 (socket says 172.21.15.5:44138) 2024-09-12T02:59:01.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:01.991+0000 7f44127fc640 1 -- 172.21.15.5:0/1552382776 learned_addr learned my addr 172.21.15.5:0/1552382776 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:59:01.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:01.991+0000 7f44137fe640 1 --2- 172.21.15.5:0/1552382776 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4414106b80 0x7f4414101d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:01.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:01.991+0000 7f4412ffd640 1 -- 172.21.15.5:0/1552382776 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4414100aa0 msgr2=0x7f4414101820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:01.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:01.991+0000 7f4412ffd640 1 --2- 172.21.15.5:0/1552382776 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4414100aa0 0x7f4414101820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:01.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:01.991+0000 7f4412ffd640 1 -- 172.21.15.5:0/1552382776 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4414106b80 msgr2=0x7f4414101d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:01.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:01.991+0000 7f4412ffd640 1 --2- 172.21.15.5:0/1552382776 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4414106b80 0x7f4414101d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:01.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:01.991+0000 7f4412ffd640 1 -- 172.21.15.5:0/1552382776 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44141aaeb0 con 0x7f4414100100 2024-09-12T02:59:01.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:01.992+0000 7f44137fe640 1 --2- 172.21.15.5:0/1552382776 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4414106b80 0x7f4414101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:59:01.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:01.992+0000 7f4412ffd640 1 --2- 172.21.15.5:0/1552382776 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4414100100 0x7f44141012e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f440400e9f0 tx=0x7f440400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:59:01.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:01.992+0000 7f43fbfff640 1 -- 172.21.15.5:0/1552382776 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f440400ccc0 con 0x7f4414100100 2024-09-12T02:59:01.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:01.992+0000 7f4419508640 1 -- 172.21.15.5:0/1552382776 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44141ab0e0 con 0x7f4414100100 2024-09-12T02:59:01.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:01.992+0000 7f4419508640 1 -- 172.21.15.5:0/1552382776 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44141ab640 con 0x7f4414100100 2024-09-12T02:59:01.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:01.992+0000 7f43fbfff640 1 -- 172.21.15.5:0/1552382776 <== mon.2 v2:172.21.15.146:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4404010910 con 0x7f4414100100 2024-09-12T02:59:01.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:01.992+0000 7f43fbfff640 1 -- 172.21.15.5:0/1552382776 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4404018960 con 0x7f4414100100 2024-09-12T02:59:02.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:01.995+0000 7f43fbfff640 1 -- 172.21.15.5:0/1552382776 <== mon.2 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7f4404010430 con 0x7f4414100100 2024-09-12T02:59:02.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:01.996+0000 7f441b8e1740 1 Processor -- start 2024-09-12T02:59:02.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:01.996+0000 7f441b8e1740 1 -- start start 2024-09-12T02:59:02.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:01.997+0000 7f441b8e1740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561a4cafab40 0x561a4caf9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:02.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:01.997+0000 7f441b8e1740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561a4caf95d0 0x561a4caf9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:02.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:01.997+0000 7f44137fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561a4cafab40 0x561a4caf9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:02.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:01.997+0000 7f44137fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561a4cafab40 0x561a4caf9090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.5:44144/0 (socket says 172.21.15.5:44144) 2024-09-12T02:59:02.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:01.997+0000 7f4412ffd640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561a4caf95d0 0x561a4caf9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:02.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:01.997+0000 7f4412ffd640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561a4caf95d0 0x561a4caf9a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.5:52910/0 (socket says 172.21.15.5:52910) 2024-09-12T02:59:02.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:01.998+0000 7f441b8e1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a4cb04370 0x561a4cb06760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:02.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:01.998+0000 7f44137fe640 1 -- 172.21.15.5:0/2475148085 learned_addr learned my addr 172.21.15.5:0/2475148085 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:59:02.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:01.998+0000 7f441b8e1740 1 -- 172.21.15.5:0/2475148085 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561a4ca5f2f0 con 0x561a4cb04370 2024-09-12T02:59:02.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:01.998+0000 7f441b8e1740 1 -- 172.21.15.5:0/2475148085 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x561a4ca06cf0 con 0x561a4caf95d0 2024-09-12T02:59:02.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:01.998+0000 7f441b8e1740 1 -- 172.21.15.5:0/2475148085 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x561a4ca5b2b0 con 0x561a4cafab40 2024-09-12T02:59:02.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:01.998+0000 7f44127fc640 1 --2- 172.21.15.5:0/2475148085 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a4cb04370 0x561a4cb06760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:02.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:01.998+0000 7f43fbfff640 1 --2- 172.21.15.5:0/1552382776 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f43fc0a2c90 0x7f43fc0a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:02.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:01.998+0000 7f44137fe640 1 -- 172.21.15.5:0/2475148085 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561a4caf95d0 msgr2=0x561a4caf9a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:02.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:01.998+0000 7f44137fe640 1 --2- 172.21.15.5:0/2475148085 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561a4caf95d0 0x561a4caf9a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:02.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:01.998+0000 7f44137fe640 1 -- 172.21.15.5:0/2475148085 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a4cb04370 msgr2=0x561a4cb06760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:02.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:01.998+0000 7f44137fe640 1 --2- 172.21.15.5:0/2475148085 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a4cb04370 0x561a4cb06760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:02.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:01.998+0000 7f44137fe640 1 -- 172.21.15.5:0/2475148085 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a4c9eb3a0 con 0x561a4cafab40 2024-09-12T02:59:02.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:01.998+0000 7f44127fc640 1 --2- 172.21.15.5:0/2475148085 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a4cb04370 0x561a4cb06760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:59:02.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:01.998+0000 7f44137fe640 1 --2- 172.21.15.5:0/1552382776 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f43fc0a2c90 0x7f43fc0a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:02.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:01.999+0000 7f44137fe640 1 --2- 172.21.15.5:0/2475148085 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561a4cafab40 0x561a4caf9090 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f440c00f480 tx=0x7f440c00f940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3cacdabbfe243ed server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:59:02.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:01.999+0000 7f43f97fa640 1 -- 172.21.15.5:0/2475148085 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f440c018070 con 0x561a4cafab40 2024-09-12T02:59:02.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:01.999+0000 7f43f97fa640 1 -- 172.21.15.5:0/2475148085 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f440c021070 con 0x561a4cafab40 2024-09-12T02:59:02.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:01.999+0000 7f43f97fa640 1 -- 172.21.15.5:0/2475148085 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f440c010af0 con 0x561a4cafab40 2024-09-12T02:59:02.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:01.999+0000 7f43fbfff640 1 -- 172.21.15.5:0/1552382776 <== mon.2 v2:172.21.15.146:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f44040cf420 con 0x7f4414100100 2024-09-12T02:59:02.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.000+0000 7f441b8e1740 1 -- 172.21.15.5:0/2475148085 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561a4cafab40 msgr2=0x561a4caf9090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:02.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.000+0000 7f441b8e1740 1 --2- 172.21.15.5:0/2475148085 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561a4cafab40 0x561a4caf9090 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f440c00f480 tx=0x7f440c00f940 comp rx=0 tx=0).stop 2024-09-12T02:59:02.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.000+0000 7f44137fe640 1 --2- 172.21.15.5:0/1552382776 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f43fc0a2c90 0x7f43fc0a5150 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f440c020000 tx=0x7f440c01e000 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:59:02.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.000+0000 7f441b8e1740 1 -- 172.21.15.5:0/2475148085 shutdown_connections 2024-09-12T02:59:02.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.000+0000 7f441b8e1740 1 --2- 172.21.15.5:0/2475148085 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a4cb04370 0x561a4cb06760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:02.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.000+0000 7f441b8e1740 1 --2- 172.21.15.5:0/2475148085 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561a4caf95d0 0x561a4caf9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:02.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.000+0000 7f441b8e1740 1 --2- 172.21.15.5:0/2475148085 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561a4cafab40 0x561a4caf9090 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:02.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.000+0000 7f441b8e1740 1 -- 172.21.15.5:0/2475148085 >> 172.21.15.5:0/2475148085 conn(0x561a4cac2310 msgr2=0x561a4caf5c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:59:02.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.000+0000 7f441b8e1740 1 -- 172.21.15.5:0/2475148085 shutdown_connections 2024-09-12T02:59:02.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.000+0000 7f441b8e1740 1 -- 172.21.15.5:0/2475148085 wait complete. 2024-09-12T02:59:02.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.001+0000 7f441b8e1740 1 -- start start 2024-09-12T02:59:02.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.001+0000 7f441b8e1740 1 Processor -- start 2024-09-12T02:59:02.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.001+0000 7f441b8e1740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561a4caf95d0 0x561a4cb03ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:02.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.001+0000 7f441b8e1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a4cafab40 0x561a4cb00f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:02.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.001+0000 7f441b8e1740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561a4cb04370 0x561a4cb014f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:02.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.001+0000 7f441b8e1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x561a4cb07070 con 0x561a4cafab40 2024-09-12T02:59:02.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.001+0000 7f441b8e1740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x561a4ca1a060 con 0x561a4cb04370 2024-09-12T02:59:02.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.001+0000 7f441b8e1740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x561a4cb071f0 con 0x561a4caf95d0 2024-09-12T02:59:02.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.001+0000 7f441b8e1740 10 client.0 _ll_register_callbacks cb 0x561a4c76d8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:59:02.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.001+0000 7f44137fe640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561a4cb04370 0x561a4cb014f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:02.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.001+0000 7f44127fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a4cafab40 0x561a4cb00f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:02.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.001+0000 7f44137fe640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561a4cb04370 0x561a4cb014f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.5:52912/0 (socket says 172.21.15.5:52912) 2024-09-12T02:59:02.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.001+0000 7f44137fe640 1 -- 172.21.15.5:0/4252078887 learned_addr learned my addr 172.21.15.5:0/4252078887 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:59:02.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.002+0000 7f4412ffd640 1 --2- 172.21.15.5:0/4252078887 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561a4caf95d0 0x561a4cb03ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:02.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.002+0000 7f44137fe640 1 -- 172.21.15.5:0/4252078887 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561a4caf95d0 msgr2=0x561a4cb03ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:02.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.002+0000 7f44137fe640 1 --2- 172.21.15.5:0/4252078887 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561a4caf95d0 0x561a4cb03ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:02.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.002+0000 7f44137fe640 1 -- 172.21.15.5:0/4252078887 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a4cafab40 msgr2=0x561a4cb00f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:02.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.002+0000 7f44137fe640 1 --2- 172.21.15.5:0/4252078887 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a4cafab40 0x561a4cb00f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:02.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.002+0000 7f44137fe640 1 -- 172.21.15.5:0/4252078887 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a4c7d03c0 con 0x561a4cb04370 2024-09-12T02:59:02.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.002+0000 7f44127fc640 1 --2- 172.21.15.5:0/4252078887 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a4cafab40 0x561a4cb00f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:59:02.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.002+0000 7f44137fe640 1 --2- 172.21.15.5:0/4252078887 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561a4cb04370 0x561a4cb014f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f440c00fe40 tx=0x7f440c0126e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b2444dc67ba2153 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:59:02.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.002+0000 7f43f6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2024-09-12T02:59:02.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.003+0000 7f43f6ff5640 1 -- 172.21.15.5:0/4252078887 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f440c018040 con 0x561a4cb04370 2024-09-12T02:59:02.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.003+0000 7f43f6ff5640 1 -- 172.21.15.5:0/4252078887 <== mon.2 v2:172.21.15.146:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f440c010a70 con 0x561a4cb04370 2024-09-12T02:59:02.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.003+0000 7f441b8e1740 1 -- 172.21.15.5:0/4252078887 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f440c021070 con 0x561a4cb04370 2024-09-12T02:59:02.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.003+0000 7f43f6ff5640 1 -- 172.21.15.5:0/4252078887 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f440c003970 con 0x561a4cb04370 2024-09-12T02:59:02.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.003+0000 7f441b8e1740 10 client.25088 fetch_fsmap learned FSMap version 26 2024-09-12T02:59:02.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.003+0000 7f441b8e1740 1 -- 172.21.15.5:0/4252078887 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x561a4cb07650 con 0x561a4cb04370 2024-09-12T02:59:02.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.003+0000 7f43f6ff5640 1 -- 172.21.15.5:0/4252078887 <== mon.2 v2:172.21.15.146:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f440c029040 con 0x561a4cb04370 2024-09-12T02:59:02.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.004+0000 7f43f6ff5640 1 -- 172.21.15.5:0/4252078887 <== mon.2 v2:172.21.15.146:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f440c0051f0 con 0x561a4cb04370 2024-09-12T02:59:02.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.004+0000 7f441b8e1740 10 client.25088 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T02:59:02.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.004+0000 7f441b8e1740 10 client.25088 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T02:59:02.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.004+0000 7f441b8e1740 20 client.25088 populate_metadata read hostname 'smithi005' 2024-09-12T02:59:02.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.004+0000 7f441b8e1740 1 --2- 172.21.15.5:0/4252078887 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x561a4cb08b00 0x561a4cb0af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:02.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.005+0000 7f441b8e1740 4 client.25088 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:59:02.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.005+0000 7f441b8e1740 1 -- 172.21.15.5:0/4252078887 --> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f440c021070 con 0x561a4cb08b00 2024-09-12T02:59:02.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.005+0000 7f4412ffd640 1 --2- 172.21.15.5:0/4252078887 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x561a4cb08b00 0x561a4cb0af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:02.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.005+0000 7f43f6ff5640 10 client.25088 ms_handle_connect on v2:172.21.15.148:6838/2317600000 2024-09-12T02:59:02.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.005+0000 7f4412ffd640 1 --2- 172.21.15.5:0/4252078887 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x561a4cb08b00 0x561a4cb0af40 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=167ab24fc8b6d3d6 server_cookie=f75ecc2503e493c in_seq=0 out_seq=0 2024-09-12T02:59:02.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.006+0000 7f43f6ff5640 1 -- 172.21.15.5:0/4252078887 <== mds.0 v2:172.21.15.148:6838/2317600000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f440c021070 con 0x561a4cb08b00 2024-09-12T02:59:02.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.006+0000 7f43f6ff5640 10 client.25088 handle_command_reply: tid=0 2024-09-12T02:59:02.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.006+0000 7f441b8e1740 1 client.25088 shutdown 2024-09-12T02:59:02.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.006+0000 7f441b8e1740 1 -- 172.21.15.5:0/4252078887 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561a4cb04370 msgr2=0x561a4cb014f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:02.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.006+0000 7f441b8e1740 1 --2- 172.21.15.5:0/4252078887 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561a4cb04370 0x561a4cb014f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f440c00fe40 tx=0x7f440c0126e0 comp rx=0 tx=0).stop 2024-09-12T02:59:02.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.006+0000 7f441b8e1740 1 -- 172.21.15.5:0/4252078887 shutdown_connections 2024-09-12T02:59:02.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.006+0000 7f441b8e1740 1 --2- 172.21.15.5:0/4252078887 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x561a4cb08b00 0x561a4cb0af40 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:02.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.006+0000 7f441b8e1740 1 --2- 172.21.15.5:0/4252078887 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a4cafab40 0x561a4cb00f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:02.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.006+0000 7f441b8e1740 1 --2- 172.21.15.5:0/4252078887 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561a4cb04370 0x561a4cb014f0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:02.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.006+0000 7f441b8e1740 1 --2- 172.21.15.5:0/4252078887 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561a4caf95d0 0x561a4cb03ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:02.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.006+0000 7f441b8e1740 1 -- 172.21.15.5:0/4252078887 >> 172.21.15.5:0/4252078887 conn(0x561a4cac2310 msgr2=0x561a4cafb930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:59:02.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.006+0000 7f43f6ff5640 0 client.25088 ms_handle_reset on v2:172.21.15.148:6838/2317600000 2024-09-12T02:59:02.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.006+0000 7f441b8e1740 1 -- 172.21.15.5:0/4252078887 shutdown_connections 2024-09-12T02:59:02.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.006+0000 7f441b8e1740 1 -- 172.21.15.5:0/4252078887 wait complete. 2024-09-12T02:59:02.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.007+0000 7f441b8e1740 20 client.25088 trim_cache size 0 max 16384 2024-09-12T02:59:02.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.032+0000 7f441b8e1740 1 Processor -- start 2024-09-12T02:59:02.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.033+0000 7f441b8e1740 1 -- start start 2024-09-12T02:59:02.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.033+0000 7f441b8e1740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561a4caf95d0 0x561a4cb105d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:02.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.033+0000 7f441b8e1740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561a4cafab40 0x561a4cb10b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:02.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.033+0000 7f441b8e1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a4cb04370 0x561a4cafe210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:02.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.033+0000 7f441b8e1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f440000e4c0 con 0x561a4cb04370 2024-09-12T02:59:02.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.033+0000 7f441b8e1740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f4400010000 con 0x561a4caf95d0 2024-09-12T02:59:02.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.033+0000 7f441b8e1740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f440000eb90 con 0x561a4cafab40 2024-09-12T02:59:02.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.033+0000 7f4412ffd640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561a4caf95d0 0x561a4cb105d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:02.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.033+0000 7f4412ffd640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561a4caf95d0 0x561a4cb105d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.5:52918/0 (socket says 172.21.15.5:52918) 2024-09-12T02:59:02.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.033+0000 7f4412ffd640 1 -- 172.21.15.5:0/163753044 learned_addr learned my addr 172.21.15.5:0/163753044 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:59:02.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.033+0000 7f44137fe640 1 --2- 172.21.15.5:0/163753044 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a4cb04370 0x561a4cafe210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:02.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.034+0000 7f4412ffd640 1 -- 172.21.15.5:0/163753044 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561a4cafab40 msgr2=0x561a4cb10b10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T02:59:02.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.034+0000 7f4412ffd640 1 --2- 172.21.15.5:0/163753044 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561a4cafab40 0x561a4cb10b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:02.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.034+0000 7f4412ffd640 1 -- 172.21.15.5:0/163753044 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a4cb04370 msgr2=0x561a4cafe210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:02.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.034+0000 7f4412ffd640 1 --2- 172.21.15.5:0/163753044 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a4cb04370 0x561a4cafe210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:02.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.034+0000 7f4412ffd640 1 -- 172.21.15.5:0/163753044 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a4cb04050 con 0x561a4caf95d0 2024-09-12T02:59:02.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.034+0000 7f4412ffd640 1 --2- 172.21.15.5:0/163753044 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561a4caf95d0 0x561a4cb105d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4404000c00 tx=0x7f44040cf6f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc82e70b9a150e17 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:59:02.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.034+0000 7f43f8ff9640 1 -- 172.21.15.5:0/163753044 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f440409ce60 con 0x561a4caf95d0 2024-09-12T02:59:02.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.034+0000 7f43f8ff9640 1 -- 172.21.15.5:0/163753044 <== mon.2 v2:172.21.15.146:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f440409bb30 con 0x561a4caf95d0 2024-09-12T02:59:02.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.034+0000 7f43f8ff9640 1 -- 172.21.15.5:0/163753044 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44040cd030 con 0x561a4caf95d0 2024-09-12T02:59:02.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.035+0000 7f441b8e1740 1 -- 172.21.15.5:0/163753044 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561a4caf95d0 msgr2=0x561a4cb105d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:02.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.035+0000 7f441b8e1740 1 --2- 172.21.15.5:0/163753044 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561a4caf95d0 0x561a4cb105d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4404000c00 tx=0x7f44040cf6f0 comp rx=0 tx=0).stop 2024-09-12T02:59:02.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.035+0000 7f441b8e1740 1 -- 172.21.15.5:0/163753044 shutdown_connections 2024-09-12T02:59:02.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.035+0000 7f441b8e1740 1 --2- 172.21.15.5:0/163753044 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a4cb04370 0x561a4cafe210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:02.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.035+0000 7f441b8e1740 1 --2- 172.21.15.5:0/163753044 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561a4cafab40 0x561a4cb10b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:02.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.035+0000 7f441b8e1740 1 --2- 172.21.15.5:0/163753044 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561a4caf95d0 0x561a4cb105d0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:02.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.035+0000 7f441b8e1740 1 -- 172.21.15.5:0/163753044 >> 172.21.15.5:0/163753044 conn(0x561a4cac2310 msgr2=0x561a4cb0aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:59:02.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.035+0000 7f441b8e1740 1 -- 172.21.15.5:0/163753044 shutdown_connections 2024-09-12T02:59:02.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.035+0000 7f441b8e1740 1 -- 172.21.15.5:0/163753044 wait complete. 2024-09-12T02:59:02.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.036+0000 7f441b8e1740 1 -- start start 2024-09-12T02:59:02.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.036+0000 7f441b8e1740 1 Processor -- start 2024-09-12T02:59:02.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.036+0000 7f441b8e1740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a4caf95d0 0x561a4cb1c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:02.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.036+0000 7f441b8e1740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561a4cafab40 0x561a4cb1cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:02.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.036+0000 7f441b8e1740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561a4cb04370 0x561a4cb193f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:02.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.036+0000 7f441b8e1740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f440000f930 con 0x561a4caf95d0 2024-09-12T02:59:02.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.036+0000 7f441b8e1740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f440000f260 con 0x561a4cb04370 2024-09-12T02:59:02.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.036+0000 7f441b8e1740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f440000c4a0 con 0x561a4cafab40 2024-09-12T02:59:02.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.036+0000 7f441b8e1740 10 client.0 _ll_register_callbacks cb 0x7f440c0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:59:02.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.036+0000 7f4412ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a4caf95d0 0x561a4cb1c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:02.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.036+0000 7f4412ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a4caf95d0 0x561a4cb1c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48994/0 (socket says 172.21.15.5:48994) 2024-09-12T02:59:02.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.036+0000 7f4412ffd640 1 -- 172.21.15.5:0/3311507874 learned_addr learned my addr 172.21.15.5:0/3311507874 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:59:02.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.036+0000 7f44137fe640 1 --2- 172.21.15.5:0/3311507874 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561a4cb04370 0x561a4cb193f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:02.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.037+0000 7f44127fc640 1 --2- 172.21.15.5:0/3311507874 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561a4cafab40 0x561a4cb1cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:02.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.037+0000 7f4412ffd640 1 -- 172.21.15.5:0/3311507874 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561a4cafab40 msgr2=0x561a4cb1cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:02.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.037+0000 7f4412ffd640 1 --2- 172.21.15.5:0/3311507874 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561a4cafab40 0x561a4cb1cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:02.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.037+0000 7f4412ffd640 1 -- 172.21.15.5:0/3311507874 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561a4cb04370 msgr2=0x561a4cb193f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:02.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.037+0000 7f4412ffd640 1 --2- 172.21.15.5:0/3311507874 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561a4cb04370 0x561a4cb193f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:02.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.037+0000 7f4412ffd640 1 -- 172.21.15.5:0/3311507874 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a4cb19bf0 con 0x561a4caf95d0 2024-09-12T02:59:02.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.037+0000 7f44127fc640 1 --2- 172.21.15.5:0/3311507874 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561a4cafab40 0x561a4cb1cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:59:02.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.037+0000 7f4412ffd640 1 --2- 172.21.15.5:0/3311507874 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a4caf95d0 0x561a4cb1c620 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f440409bac0 tx=0x7f440409c2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b993c4344240f1a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:59:02.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.037+0000 7f43f6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T02:59:02.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.037+0000 7f43f6ff5640 1 -- 172.21.15.5:0/3311507874 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f440409ce60 con 0x561a4caf95d0 2024-09-12T02:59:02.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.037+0000 7f43f6ff5640 1 -- 172.21.15.5:0/3311507874 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f44040cd030 con 0x561a4caf95d0 2024-09-12T02:59:02.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.037+0000 7f441b8e1740 1 -- 172.21.15.5:0/3311507874 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f440409bb30 con 0x561a4caf95d0 2024-09-12T02:59:02.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.037+0000 7f43f6ff5640 1 -- 172.21.15.5:0/3311507874 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f44040a47a0 con 0x561a4caf95d0 2024-09-12T02:59:02.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.037+0000 7f441b8e1740 10 client.15582 fetch_fsmap learned FSMap version 26 2024-09-12T02:59:02.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.037+0000 7f441b8e1740 1 -- 172.21.15.5:0/3311507874 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x561a4cba0810 con 0x561a4caf95d0 2024-09-12T02:59:02.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.038+0000 7f43f6ff5640 1 -- 172.21.15.5:0/3311507874 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f440409bb30 con 0x561a4caf95d0 2024-09-12T02:59:02.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.038+0000 7f43f6ff5640 1 -- 172.21.15.5:0/3311507874 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f4404010b50 con 0x561a4caf95d0 2024-09-12T02:59:02.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.038+0000 7f441b8e1740 10 client.15582 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T02:59:02.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.038+0000 7f441b8e1740 10 client.15582 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T02:59:02.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.038+0000 7f441b8e1740 20 client.15582 populate_metadata read hostname 'smithi005' 2024-09-12T02:59:02.045 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-12T02:59:02.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.038+0000 7f441b8e1740 1 --2- 172.21.15.5:0/3311507874 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x561a4cba0ee0 0x561a4cba3300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:02.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.038+0000 7f441b8e1740 4 client.15582 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:59:02.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.038+0000 7f441b8e1740 1 -- 172.21.15.5:0/3311507874 --> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561a4ca6c860 con 0x561a4cba0ee0 2024-09-12T02:59:02.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.039+0000 7f44127fc640 1 --2- 172.21.15.5:0/3311507874 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x561a4cba0ee0 0x561a4cba3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:02.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.039+0000 7f43f6ff5640 10 client.15582 ms_handle_connect on v2:172.21.15.148:6838/2317600000 2024-09-12T02:59:02.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.039+0000 7f44127fc640 1 --2- 172.21.15.5:0/3311507874 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x561a4cba0ee0 0x561a4cba3300 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e2282cc1eea3e3e server_cookie=811c5652525f39ae in_seq=0 out_seq=0 2024-09-12T02:59:02.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.039+0000 7f43f6ff5640 1 -- 172.21.15.5:0/3311507874 <== mds.0 v2:172.21.15.148:6838/2317600000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561a4ca6c860 con 0x561a4cba0ee0 2024-09-12T02:59:02.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.039+0000 7f43f6ff5640 10 client.15582 handle_command_reply: tid=0 2024-09-12T02:59:02.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.039+0000 7f441b8e1740 1 client.15582 shutdown 2024-09-12T02:59:02.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.039+0000 7f441b8e1740 1 -- 172.21.15.5:0/3311507874 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a4caf95d0 msgr2=0x561a4cb1c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:02.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.039+0000 7f441b8e1740 1 --2- 172.21.15.5:0/3311507874 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a4caf95d0 0x561a4cb1c620 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f440409bac0 tx=0x7f440409c2f0 comp rx=0 tx=0).stop 2024-09-12T02:59:02.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.039+0000 7f441b8e1740 1 -- 172.21.15.5:0/3311507874 shutdown_connections 2024-09-12T02:59:02.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.039+0000 7f441b8e1740 1 --2- 172.21.15.5:0/3311507874 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x561a4cb04370 0x561a4cb193f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:02.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.039+0000 7f441b8e1740 1 --2- 172.21.15.5:0/3311507874 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x561a4cafab40 0x561a4cb1cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:02.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.039+0000 7f441b8e1740 1 --2- 172.21.15.5:0/3311507874 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x561a4cba0ee0 0x561a4cba3300 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:02.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.039+0000 7f441b8e1740 1 --2- 172.21.15.5:0/3311507874 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a4caf95d0 0x561a4cb1c620 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:02.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.039+0000 7f441b8e1740 1 -- 172.21.15.5:0/3311507874 >> 172.21.15.5:0/3311507874 conn(0x561a4cac2310 msgr2=0x561a4cb027f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:59:02.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.039+0000 7f43f6ff5640 0 client.15582 ms_handle_reset on v2:172.21.15.148:6838/2317600000 2024-09-12T02:59:02.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.040+0000 7f441b8e1740 1 -- 172.21.15.5:0/3311507874 shutdown_connections 2024-09-12T02:59:02.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.040+0000 7f441b8e1740 1 -- 172.21.15.5:0/3311507874 wait complete. 2024-09-12T02:59:02.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.040+0000 7f441b8e1740 20 client.15582 trim_cache size 0 max 16384 2024-09-12T02:59:02.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.040+0000 7f4419508640 1 -- 172.21.15.5:0/1552382776 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f43fc0a2c90 msgr2=0x7f43fc0a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:02.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.040+0000 7f4419508640 1 --2- 172.21.15.5:0/1552382776 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f43fc0a2c90 0x7f43fc0a5150 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f440c020000 tx=0x7f440c01e000 comp rx=0 tx=0).stop 2024-09-12T02:59:02.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.040+0000 7f4419508640 1 -- 172.21.15.5:0/1552382776 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4414100100 msgr2=0x7f44141012e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:02.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.040+0000 7f4419508640 1 --2- 172.21.15.5:0/1552382776 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4414100100 0x7f44141012e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f440400e9f0 tx=0x7f440400eeb0 comp rx=0 tx=0).stop 2024-09-12T02:59:02.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.041+0000 7f4419508640 1 -- 172.21.15.5:0/1552382776 shutdown_connections 2024-09-12T02:59:02.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.041+0000 7f4419508640 1 --2- 172.21.15.5:0/1552382776 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f43fc0a2c90 0x7f43fc0a5150 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:02.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.041+0000 7f4419508640 1 --2- 172.21.15.5:0/1552382776 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4414106b80 0x7f4414101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:02.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.041+0000 7f4419508640 1 --2- 172.21.15.5:0/1552382776 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4414100aa0 0x7f4414101820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:02.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.041+0000 7f4419508640 1 --2- 172.21.15.5:0/1552382776 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4414100100 0x7f44141012e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:02.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.041+0000 7f4419508640 1 -- 172.21.15.5:0/1552382776 >> 172.21.15.5:0/1552382776 conn(0x7f4414077230 msgr2=0x7f44140fc810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:59:02.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.041+0000 7f4419508640 1 -- 172.21.15.5:0/1552382776 shutdown_connections 2024-09-12T02:59:02.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:02.041+0000 7f4419508640 1 -- 172.21.15.5:0/1552382776 wait complete. 2024-09-12T02:59:03.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:59:02 smithi005 ceph-mon[31197]: pgmap v359: 97 pgs: 97 active+clean; 663 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 597 B/s wr, 9 op/s 2024-09-12T02:59:03.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:59:02 smithi146 ceph-mon[35011]: pgmap v359: 97 pgs: 97 active+clean; 663 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 597 B/s wr, 9 op/s 2024-09-12T02:59:03.415 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:59:02 smithi148 ceph-mon[35090]: pgmap v359: 97 pgs: 97 active+clean; 663 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 597 B/s wr, 9 op/s 2024-09-12T02:59:03.733 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T02:59:03.733 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T02:59:04.102 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:59:04.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.589+0000 7f05ab212640 1 -- 172.21.15.5:0/241029668 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05a4106f20 msgr2=0x7f05a410eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:04.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.589+0000 7f05ab212640 1 --2- 172.21.15.5:0/241029668 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05a4106f20 0x7f05a410eaa0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f059c009ca0 tx=0x7f059c02a8c0 comp rx=0 tx=0).stop 2024-09-12T02:59:04.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.589+0000 7f05ab212640 1 -- 172.21.15.5:0/241029668 shutdown_connections 2024-09-12T02:59:04.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.589+0000 7f05ab212640 1 --2- 172.21.15.5:0/241029668 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05a4106f20 0x7f05a410eaa0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:04.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.589+0000 7f05ab212640 1 --2- 172.21.15.5:0/241029668 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f05a41027a0 0x7f05a41067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:04.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.589+0000 7f05ab212640 1 --2- 172.21.15.5:0/241029668 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f05a4101df0 0x7f05a41021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:04.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.589+0000 7f05ab212640 1 -- 172.21.15.5:0/241029668 >> 172.21.15.5:0/241029668 conn(0x7f05a40fd8b0 msgr2=0x7f05a40ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:59:04.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.589+0000 7f05ab212640 1 -- 172.21.15.5:0/241029668 shutdown_connections 2024-09-12T02:59:04.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.589+0000 7f05ab212640 1 -- 172.21.15.5:0/241029668 wait complete. 2024-09-12T02:59:04.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.590+0000 7f05ab212640 1 Processor -- start 2024-09-12T02:59:04.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.590+0000 7f05ab212640 1 -- start start 2024-09-12T02:59:04.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.590+0000 7f05ab212640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05a4101df0 0x7f05a410dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:04.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.590+0000 7f05ab212640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f05a41027a0 0x7f05a410e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:04.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.590+0000 7f05ab212640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f05a4106f20 0x7f05a419aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:04.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.590+0000 7f05ab212640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f05a4111f80 con 0x7f05a4101df0 2024-09-12T02:59:04.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.590+0000 7f05ab212640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f05a4111e00 con 0x7f05a4106f20 2024-09-12T02:59:04.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.590+0000 7f05ab212640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f05a4112100 con 0x7f05a41027a0 2024-09-12T02:59:04.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.590+0000 7f05a8fae640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05a4101df0 0x7f05a410dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:04.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.590+0000 7f05a8fae640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05a4101df0 0x7f05a410dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52710/0 (socket says 172.21.15.5:52710) 2024-09-12T02:59:04.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.590+0000 7f05a8fae640 1 -- 172.21.15.5:0/2988542772 learned_addr learned my addr 172.21.15.5:0/2988542772 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:59:04.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.590+0000 7f05a97af640 1 --2- 172.21.15.5:0/2988542772 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f05a4106f20 0x7f05a419aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:04.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.590+0000 7f05a3fff640 1 --2- 172.21.15.5:0/2988542772 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f05a41027a0 0x7f05a410e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:04.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.591+0000 7f05a8fae640 1 -- 172.21.15.5:0/2988542772 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f05a41027a0 msgr2=0x7f05a410e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:04.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.591+0000 7f05a8fae640 1 --2- 172.21.15.5:0/2988542772 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f05a41027a0 0x7f05a410e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:04.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.591+0000 7f05a8fae640 1 -- 172.21.15.5:0/2988542772 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f05a4106f20 msgr2=0x7f05a419aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:04.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.591+0000 7f05a8fae640 1 --2- 172.21.15.5:0/2988542772 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f05a4106f20 0x7f05a419aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:04.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.591+0000 7f05a8fae640 1 -- 172.21.15.5:0/2988542772 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05a419b120 con 0x7f05a4101df0 2024-09-12T02:59:04.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.591+0000 7f05a8fae640 1 --2- 172.21.15.5:0/2988542772 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05a4101df0 0x7f05a410dd60 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f059800c920 tx=0x7f059800cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:59:04.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.591+0000 7f05a1ffb640 1 -- 172.21.15.5:0/2988542772 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0598007bb0 con 0x7f05a4101df0 2024-09-12T02:59:04.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.591+0000 7f05ab212640 1 -- 172.21.15.5:0/2988542772 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05a419b410 con 0x7f05a4101df0 2024-09-12T02:59:04.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.591+0000 7f05a1ffb640 1 -- 172.21.15.5:0/2988542772 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0598007d50 con 0x7f05a4101df0 2024-09-12T02:59:04.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.591+0000 7f05a1ffb640 1 -- 172.21.15.5:0/2988542772 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f059800f450 con 0x7f05a4101df0 2024-09-12T02:59:04.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.591+0000 7f05ab212640 1 -- 172.21.15.5:0/2988542772 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05a4197970 con 0x7f05a4101df0 2024-09-12T02:59:04.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.595+0000 7f05a1ffb640 1 -- 172.21.15.5:0/2988542772 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7f0598004ec0 con 0x7f05a4101df0 2024-09-12T02:59:04.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.596+0000 7f05ad5eb740 1 Processor -- start 2024-09-12T02:59:04.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.596+0000 7f05ad5eb740 1 -- start start 2024-09-12T02:59:04.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.596+0000 7f05a1ffb640 1 --2- 172.21.15.5:0/2988542772 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f05940a2c90 0x7f05940a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:04.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.596+0000 7f05ad5eb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e70c0bb60 0x556e70c0bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:04.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.596+0000 7f05ad5eb740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556e70c0c480 0x556e70c16800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:04.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.596+0000 7f05ad5eb740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556e70c16d40 0x556e70c19130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:04.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.596+0000 7f05ad5eb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556e70b71280 con 0x556e70c0bb60 2024-09-12T02:59:04.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.596+0000 7f05ad5eb740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x556e70b18b10 con 0x556e70c0c480 2024-09-12T02:59:04.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.596+0000 7f05ad5eb740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x556e70b6d240 con 0x556e70c16d40 2024-09-12T02:59:04.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.596+0000 7f05a97af640 1 --2- 172.21.15.5:0/2988542772 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f05940a2c90 0x7f05940a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:04.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.596+0000 7f05a1ffb640 1 -- 172.21.15.5:0/2988542772 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f05980cf250 con 0x7f05a4101df0 2024-09-12T02:59:04.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.596+0000 7f05a8fae640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e70c0bb60 0x556e70c0bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:04.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.596+0000 7f05a8fae640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e70c0bb60 0x556e70c0bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52716/0 (socket says 172.21.15.5:52716) 2024-09-12T02:59:04.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.596+0000 7f05a8fae640 1 -- 172.21.15.5:0/1369759213 learned_addr learned my addr 172.21.15.5:0/1369759213 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:59:04.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.596+0000 7f05a97af640 1 --2- 172.21.15.5:0/1369759213 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556e70c16d40 0x556e70c19130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:04.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.597+0000 7f05a3fff640 1 --2- 172.21.15.5:0/1369759213 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556e70c0c480 0x556e70c16800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:04.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.597+0000 7f05a8fae640 1 -- 172.21.15.5:0/1369759213 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556e70c16d40 msgr2=0x556e70c19130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:04.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.597+0000 7f05a8fae640 1 --2- 172.21.15.5:0/1369759213 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556e70c16d40 0x556e70c19130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:04.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.597+0000 7f05a8fae640 1 -- 172.21.15.5:0/1369759213 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556e70c0c480 msgr2=0x556e70c16800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:04.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.597+0000 7f05a97af640 1 --2- 172.21.15.5:0/2988542772 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f05940a2c90 0x7f05940a5150 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f059c009430 tx=0x7f059c031db0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:59:04.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.597+0000 7f05a8fae640 1 --2- 172.21.15.5:0/1369759213 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556e70c0c480 0x556e70c16800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:04.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.597+0000 7f05a8fae640 1 -- 172.21.15.5:0/1369759213 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e70b251f0 con 0x556e70c0bb60 2024-09-12T02:59:04.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.597+0000 7f05a3fff640 1 --2- 172.21.15.5:0/1369759213 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556e70c0c480 0x556e70c16800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:59:04.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.597+0000 7f05a8fae640 1 --2- 172.21.15.5:0/1369759213 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e70c0bb60 0x556e70c0bf40 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f0598098330 tx=0x7f059809b3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ac5a0fefe84bc7a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:59:04.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.598+0000 7f058effd640 1 -- 172.21.15.5:0/1369759213 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f059809fd10 con 0x556e70c0bb60 2024-09-12T02:59:04.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.598+0000 7f058effd640 1 -- 172.21.15.5:0/1369759213 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05980a45d0 con 0x556e70c0bb60 2024-09-12T02:59:04.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.598+0000 7f058effd640 1 -- 172.21.15.5:0/1369759213 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0598059540 con 0x556e70c0bb60 2024-09-12T02:59:04.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.599+0000 7f05ad5eb740 1 -- 172.21.15.5:0/1369759213 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e70c0bb60 msgr2=0x556e70c0bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:04.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.599+0000 7f05ad5eb740 1 --2- 172.21.15.5:0/1369759213 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e70c0bb60 0x556e70c0bf40 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f0598098330 tx=0x7f059809b3f0 comp rx=0 tx=0).stop 2024-09-12T02:59:04.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.599+0000 7f05ad5eb740 1 -- 172.21.15.5:0/1369759213 shutdown_connections 2024-09-12T02:59:04.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.599+0000 7f05ad5eb740 1 --2- 172.21.15.5:0/1369759213 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556e70c16d40 0x556e70c19130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:04.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.599+0000 7f05ad5eb740 1 --2- 172.21.15.5:0/1369759213 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556e70c0c480 0x556e70c16800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:04.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.599+0000 7f05ad5eb740 1 --2- 172.21.15.5:0/1369759213 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e70c0bb60 0x556e70c0bf40 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:04.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.599+0000 7f05ad5eb740 1 -- 172.21.15.5:0/1369759213 >> 172.21.15.5:0/1369759213 conn(0x556e70c080f0 msgr2=0x556e70c084d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:59:04.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.599+0000 7f05ad5eb740 1 -- 172.21.15.5:0/1369759213 shutdown_connections 2024-09-12T02:59:04.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.599+0000 7f05ad5eb740 1 -- 172.21.15.5:0/1369759213 wait complete. 2024-09-12T02:59:04.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.599+0000 7f05ad5eb740 1 -- start start 2024-09-12T02:59:04.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.600+0000 7f05ad5eb740 1 Processor -- start 2024-09-12T02:59:04.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.600+0000 7f05ad5eb740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556e70c0bb60 0x556e70c13be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:04.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.600+0000 7f05ad5eb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e70c0c480 0x556e70c14120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:04.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.600+0000 7f05ad5eb740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556e70c16d40 0x556e70c14660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:04.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.600+0000 7f05ad5eb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556e70c19910 con 0x556e70c0c480 2024-09-12T02:59:04.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.600+0000 7f05ad5eb740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x556e70b2bf10 con 0x556e70c0bb60 2024-09-12T02:59:04.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.600+0000 7f05ad5eb740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x556e70c19a90 con 0x556e70c16d40 2024-09-12T02:59:04.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.601+0000 7f05a3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e70c0c480 0x556e70c14120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:04.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.601+0000 7f05a3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e70c0c480 0x556e70c14120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52732/0 (socket says 172.21.15.5:52732) 2024-09-12T02:59:04.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.601+0000 7f05a3fff640 1 -- 172.21.15.5:0/2055720602 learned_addr learned my addr 172.21.15.5:0/2055720602 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:59:04.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.601+0000 7f05a97af640 1 --2- 172.21.15.5:0/2055720602 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556e70c16d40 0x556e70c14660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:04.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.601+0000 7f05a8fae640 1 --2- 172.21.15.5:0/2055720602 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556e70c0bb60 0x556e70c13be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:04.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.601+0000 7f05a3fff640 1 -- 172.21.15.5:0/2055720602 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556e70c16d40 msgr2=0x556e70c14660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:04.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.601+0000 7f05a3fff640 1 --2- 172.21.15.5:0/2055720602 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556e70c16d40 0x556e70c14660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:04.613 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.601+0000 7f05a3fff640 1 -- 172.21.15.5:0/2055720602 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556e70c0bb60 msgr2=0x556e70c13be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:04.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.601+0000 7f05a3fff640 1 --2- 172.21.15.5:0/2055720602 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556e70c0bb60 0x556e70c13be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:04.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.601+0000 7f05a3fff640 1 -- 172.21.15.5:0/2055720602 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e708e26a0 con 0x556e70c0c480 2024-09-12T02:59:04.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.601+0000 7f05ad5eb740 10 client.0 _ll_register_callbacks cb 0x556e70af4a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:59:04.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.601+0000 7f05a97af640 1 --2- 172.21.15.5:0/2055720602 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556e70c16d40 0x556e70c14660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:59:04.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.601+0000 7f05ad5eb740 1 -- 172.21.15.5:0/2055720602 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05980a45d0 con 0x556e70c0c480 2024-09-12T02:59:04.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.601+0000 7f05a3fff640 1 --2- 172.21.15.5:0/2055720602 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e70c0c480 0x556e70c14120 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f0590011710 tx=0x7f0590011b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d59af6d80b37690 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:59:04.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.601+0000 7f0583fff640 10 client.15594 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T02:59:04.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.602+0000 7f0583fff640 1 -- 172.21.15.5:0/2055720602 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f059001f060 con 0x556e70c0c480 2024-09-12T02:59:04.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.602+0000 7f0583fff640 1 -- 172.21.15.5:0/2055720602 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f059000fb40 con 0x556e70c0c480 2024-09-12T02:59:04.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.602+0000 7f0583fff640 1 -- 172.21.15.5:0/2055720602 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f059000fda0 con 0x556e70c0c480 2024-09-12T02:59:04.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.602+0000 7f05ad5eb740 10 client.15594 fetch_fsmap learned FSMap version 26 2024-09-12T02:59:04.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.602+0000 7f05ad5eb740 1 -- 172.21.15.5:0/2055720602 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({fsmap=26}) -- 0x556e70c254c0 con 0x556e70c0c480 2024-09-12T02:59:04.614 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.602+0000 7f0583fff640 1 -- 172.21.15.5:0/2055720602 <== mon.0 v2:172.21.15.5:3300/0 4 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f0590012450 con 0x556e70c0c480 2024-09-12T02:59:04.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.603+0000 7f05ad5eb740 10 client.15594 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T02:59:04.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.603+0000 7f05ad5eb740 10 client.15594 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T02:59:04.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.603+0000 7f05ad5eb740 20 client.15594 populate_metadata read hostname 'smithi005' 2024-09-12T02:59:04.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.603+0000 7f05ad5eb740 1 --2- 172.21.15.5:0/2055720602 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x556e70c26810 0x556e70c28c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:04.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.603+0000 7f05ad5eb740 4 client.15594 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:59:04.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.603+0000 7f05ad5eb740 1 -- 172.21.15.5:0/2055720602 --> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556e70b7e7f0 con 0x556e70c26810 2024-09-12T02:59:04.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.603+0000 7f05a8fae640 1 --2- 172.21.15.5:0/2055720602 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x556e70c26810 0x556e70c28c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:04.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.604+0000 7f05a8fae640 1 --2- 172.21.15.5:0/2055720602 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x556e70c26810 0x556e70c28c20 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2726bc1c2376e98 server_cookie=23b3a615bfebaaa2 in_seq=0 out_seq=0 2024-09-12T02:59:04.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.604+0000 7f0583fff640 10 client.15594 ms_handle_connect on v2:172.21.15.148:6838/2317600000 2024-09-12T02:59:04.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.604+0000 7f0583fff640 1 -- 172.21.15.5:0/2055720602 <== mds.0 v2:172.21.15.148:6838/2317600000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556e70b7e7f0 con 0x556e70c26810 2024-09-12T02:59:04.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.604+0000 7f0583fff640 10 client.15594 handle_command_reply: tid=0 2024-09-12T02:59:04.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.605+0000 7f05ad5eb740 1 client.15594 shutdown 2024-09-12T02:59:04.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.605+0000 7f05ad5eb740 1 -- 172.21.15.5:0/2055720602 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e70c0c480 msgr2=0x556e70c14120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:04.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.605+0000 7f05ad5eb740 1 --2- 172.21.15.5:0/2055720602 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e70c0c480 0x556e70c14120 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f0590011710 tx=0x7f0590011b10 comp rx=0 tx=0).stop 2024-09-12T02:59:04.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.605+0000 7f05ad5eb740 1 -- 172.21.15.5:0/2055720602 shutdown_connections 2024-09-12T02:59:04.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.605+0000 7f05ad5eb740 1 --2- 172.21.15.5:0/2055720602 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x556e70c26810 0x556e70c28c20 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:04.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.605+0000 7f05ad5eb740 1 --2- 172.21.15.5:0/2055720602 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e70c0c480 0x556e70c14120 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:04.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.605+0000 7f05ad5eb740 1 --2- 172.21.15.5:0/2055720602 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556e70c16d40 0x556e70c14660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:04.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.605+0000 7f0583fff640 0 client.15594 ms_handle_reset on v2:172.21.15.148:6838/2317600000 2024-09-12T02:59:04.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.605+0000 7f05ad5eb740 1 --2- 172.21.15.5:0/2055720602 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556e70c0bb60 0x556e70c13be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:04.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.605+0000 7f05ad5eb740 1 -- 172.21.15.5:0/2055720602 >> 172.21.15.5:0/2055720602 conn(0x556e70c080f0 msgr2=0x556e70c07270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:59:04.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.605+0000 7f05ad5eb740 1 -- 172.21.15.5:0/2055720602 shutdown_connections 2024-09-12T02:59:04.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.605+0000 7f05ad5eb740 1 -- 172.21.15.5:0/2055720602 wait complete. 2024-09-12T02:59:04.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.605+0000 7f05ad5eb740 20 client.15594 trim_cache size 0 max 16384 2024-09-12T02:59:04.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.623+0000 7f05ad5eb740 1 Processor -- start 2024-09-12T02:59:04.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.624+0000 7f05ad5eb740 1 -- start start 2024-09-12T02:59:04.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.624+0000 7f05ad5eb740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556e70c0bb60 0x556e70c2e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:04.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.624+0000 7f05ad5eb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e70c0c480 0x556e70c2ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:04.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.624+0000 7f05ad5eb740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556e70c16d40 0x556e70c15330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:04.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.624+0000 7f05ad5eb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f058800e3f0 con 0x556e70c0c480 2024-09-12T02:59:04.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.624+0000 7f05ad5eb740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f058800ff30 con 0x556e70c16d40 2024-09-12T02:59:04.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.624+0000 7f05ad5eb740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f058800eac0 con 0x556e70c0bb60 2024-09-12T02:59:04.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.624+0000 7f05a3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e70c0c480 0x556e70c2ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:04.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.624+0000 7f05a3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e70c0c480 0x556e70c2ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52738/0 (socket says 172.21.15.5:52738) 2024-09-12T02:59:04.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.624+0000 7f05a3fff640 1 -- 172.21.15.5:0/657059189 learned_addr learned my addr 172.21.15.5:0/657059189 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:59:04.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.625+0000 7f05a8fae640 1 --2- 172.21.15.5:0/657059189 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556e70c0bb60 0x556e70c2e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:04.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.625+0000 7f05a97af640 1 --2- 172.21.15.5:0/657059189 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556e70c16d40 0x556e70c15330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:04.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.625+0000 7f05a3fff640 1 -- 172.21.15.5:0/657059189 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556e70c0bb60 msgr2=0x556e70c2e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:04.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.625+0000 7f05a3fff640 1 --2- 172.21.15.5:0/657059189 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556e70c0bb60 0x556e70c2e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:04.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.625+0000 7f05a3fff640 1 -- 172.21.15.5:0/657059189 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556e70c16d40 msgr2=0x556e70c15330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:04.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.625+0000 7f05a3fff640 1 --2- 172.21.15.5:0/657059189 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556e70c16d40 0x556e70c15330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:04.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.625+0000 7f05a3fff640 1 -- 172.21.15.5:0/657059189 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e70c164b0 con 0x556e70c0c480 2024-09-12T02:59:04.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.625+0000 7f05a3fff640 1 --2- 172.21.15.5:0/657059189 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e70c0c480 0x556e70c2ea30 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f0590000c00 tx=0x7f0590014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f92edf9104141ba server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:59:04.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.625+0000 7f058e7fc640 1 -- 172.21.15.5:0/657059189 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f059001f040 con 0x556e70c0c480 2024-09-12T02:59:04.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.625+0000 7f058e7fc640 1 -- 172.21.15.5:0/657059189 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05980a45d0 con 0x556e70c0c480 2024-09-12T02:59:04.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.626+0000 7f058e7fc640 1 -- 172.21.15.5:0/657059189 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f059001a440 con 0x556e70c0c480 2024-09-12T02:59:04.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.626+0000 7f05ad5eb740 1 -- 172.21.15.5:0/657059189 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e70c0c480 msgr2=0x556e70c2ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:04.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.626+0000 7f05ad5eb740 1 --2- 172.21.15.5:0/657059189 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e70c0c480 0x556e70c2ea30 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f0590000c00 tx=0x7f0590014000 comp rx=0 tx=0).stop 2024-09-12T02:59:04.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.627+0000 7f05ad5eb740 1 -- 172.21.15.5:0/657059189 shutdown_connections 2024-09-12T02:59:04.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.627+0000 7f05ad5eb740 1 --2- 172.21.15.5:0/657059189 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e70c0c480 0x556e70c2ea30 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:04.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.627+0000 7f05ad5eb740 1 --2- 172.21.15.5:0/657059189 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556e70c16d40 0x556e70c15330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:04.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.627+0000 7f05ad5eb740 1 --2- 172.21.15.5:0/657059189 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556e70c0bb60 0x556e70c2e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:04.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.627+0000 7f05ad5eb740 1 -- 172.21.15.5:0/657059189 >> 172.21.15.5:0/657059189 conn(0x556e70c080f0 msgr2=0x556e70c12cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:59:04.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.627+0000 7f05ad5eb740 1 -- 172.21.15.5:0/657059189 shutdown_connections 2024-09-12T02:59:04.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.627+0000 7f05ad5eb740 1 -- 172.21.15.5:0/657059189 wait complete. 2024-09-12T02:59:04.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.627+0000 7f05ad5eb740 1 -- start start 2024-09-12T02:59:04.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.627+0000 7f05ad5eb740 1 Processor -- start 2024-09-12T02:59:04.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.628+0000 7f05ad5eb740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556e70c0bb60 0x556e70c1f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:04.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.628+0000 7f05ad5eb740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556e70c0c480 0x556e70c1a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:04.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.628+0000 7f05ad5eb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e70c16d40 0x556e70c1ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:04.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.628+0000 7f05ad5eb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f058800f860 con 0x556e70c16d40 2024-09-12T02:59:04.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.628+0000 7f05ad5eb740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f058800f190 con 0x556e70c0bb60 2024-09-12T02:59:04.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.628+0000 7f05ad5eb740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f058800c3d0 con 0x556e70c0c480 2024-09-12T02:59:04.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.628+0000 7f05ad5eb740 10 client.0 _ll_register_callbacks cb 0x7f059000af10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:59:04.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.628+0000 7f05a97af640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e70c16d40 0x556e70c1ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:04.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.628+0000 7f05a97af640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e70c16d40 0x556e70c1ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52750/0 (socket says 172.21.15.5:52750) 2024-09-12T02:59:04.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.628+0000 7f05a97af640 1 -- 172.21.15.5:0/1919622091 learned_addr learned my addr 172.21.15.5:0/1919622091 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:59:04.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.628+0000 7f05a8fae640 1 --2- 172.21.15.5:0/1919622091 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556e70c0bb60 0x556e70c1f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:04.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.628+0000 7f05a3fff640 1 --2- 172.21.15.5:0/1919622091 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556e70c0c480 0x556e70c1a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:04.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.628+0000 7f05a97af640 1 -- 172.21.15.5:0/1919622091 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556e70c0c480 msgr2=0x556e70c1a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:04.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.628+0000 7f05a97af640 1 --2- 172.21.15.5:0/1919622091 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556e70c0c480 0x556e70c1a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:04.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.628+0000 7f05a97af640 1 -- 172.21.15.5:0/1919622091 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556e70c0bb60 msgr2=0x556e70c1f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:04.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.628+0000 7f05a97af640 1 --2- 172.21.15.5:0/1919622091 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556e70c0bb60 0x556e70c1f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:04.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.628+0000 7f05a97af640 1 -- 172.21.15.5:0/1919622091 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e70c1b480 con 0x556e70c16d40 2024-09-12T02:59:04.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.629+0000 7f05a97af640 1 --2- 172.21.15.5:0/1919622091 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e70c16d40 0x556e70c1ac50 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f059c009940 tx=0x7f059c046000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d745d662f6fea00 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:59:04.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.629+0000 7f0583fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T02:59:04.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.629+0000 7f0583fff640 1 -- 172.21.15.5:0/1919622091 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f059c02f610 con 0x556e70c16d40 2024-09-12T02:59:04.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.629+0000 7f0583fff640 1 -- 172.21.15.5:0/1919622091 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f059c02f7b0 con 0x556e70c16d40 2024-09-12T02:59:04.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.629+0000 7f05ad5eb740 1 -- 172.21.15.5:0/1919622091 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05980a45d0 con 0x556e70c16d40 2024-09-12T02:59:04.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.629+0000 7f0583fff640 1 -- 172.21.15.5:0/1919622091 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f059c02f9b0 con 0x556e70c16d40 2024-09-12T02:59:04.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.629+0000 7f05ad5eb740 10 client.15600 fetch_fsmap learned FSMap version 26 2024-09-12T02:59:04.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.629+0000 7f05ad5eb740 1 -- 172.21.15.5:0/1919622091 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556e70c91400 con 0x556e70c16d40 2024-09-12T02:59:04.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.630+0000 7f0583fff640 1 -- 172.21.15.5:0/1919622091 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05980a45d0 con 0x556e70c16d40 2024-09-12T02:59:04.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.630+0000 7f0583fff640 1 -- 172.21.15.5:0/1919622091 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f059c04b450 con 0x556e70c16d40 2024-09-12T02:59:04.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.630+0000 7f05ad5eb740 10 client.15600 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T02:59:04.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.630+0000 7f05ad5eb740 10 client.15600 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T02:59:04.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.630+0000 7f05ad5eb740 20 client.15600 populate_metadata read hostname 'smithi005' 2024-09-12T02:59:04.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.630+0000 7f05ad5eb740 1 --2- 172.21.15.5:0/1919622091 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x556e70c91b30 0x556e70c93f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:04.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.630+0000 7f05ad5eb740 4 client.15600 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T02:59:04.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.630+0000 7f05ad5eb740 1 -- 172.21.15.5:0/1919622091 --> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x556e70b78d00 con 0x556e70c91b30 2024-09-12T02:59:04.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.630+0000 7f05a8fae640 1 --2- 172.21.15.5:0/1919622091 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x556e70c91b30 0x556e70c93f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:04.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.631+0000 7f0583fff640 10 client.15600 ms_handle_connect on v2:172.21.15.148:6838/2317600000 2024-09-12T02:59:04.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.631+0000 7f05a8fae640 1 --2- 172.21.15.5:0/1919622091 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x556e70c91b30 0x556e70c93f50 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c8a584b8b2c8740 server_cookie=7bee5ead4d5bee05 in_seq=0 out_seq=0 2024-09-12T02:59:04.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.632+0000 7f0583fff640 1 -- 172.21.15.5:0/1919622091 <== mds.0 v2:172.21.15.148:6838/2317600000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x556e70b78d00 con 0x556e70c91b30 2024-09-12T02:59:04.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.632+0000 7f0583fff640 10 client.15600 handle_command_reply: tid=0 2024-09-12T02:59:04.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.632+0000 7f05ad5eb740 1 client.15600 shutdown 2024-09-12T02:59:04.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.632+0000 7f05ad5eb740 1 -- 172.21.15.5:0/1919622091 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e70c16d40 msgr2=0x556e70c1ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:04.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.632+0000 7f05ad5eb740 1 --2- 172.21.15.5:0/1919622091 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e70c16d40 0x556e70c1ac50 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f059c009940 tx=0x7f059c046000 comp rx=0 tx=0).stop 2024-09-12T02:59:04.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.632+0000 7f05ad5eb740 1 -- 172.21.15.5:0/1919622091 shutdown_connections 2024-09-12T02:59:04.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.632+0000 7f05ad5eb740 1 --2- 172.21.15.5:0/1919622091 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x556e70c91b30 0x556e70c93f50 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:04.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.632+0000 7f05ad5eb740 1 --2- 172.21.15.5:0/1919622091 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556e70c16d40 0x556e70c1ac50 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:04.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.632+0000 7f05ad5eb740 1 --2- 172.21.15.5:0/1919622091 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556e70c0c480 0x556e70c1a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:04.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.632+0000 7f05ad5eb740 1 --2- 172.21.15.5:0/1919622091 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556e70c0bb60 0x556e70c1f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:04.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.632+0000 7f05ad5eb740 1 -- 172.21.15.5:0/1919622091 >> 172.21.15.5:0/1919622091 conn(0x556e70c080f0 msgr2=0x556e70c13ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:59:04.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.632+0000 7f0583fff640 0 client.15600 ms_handle_reset on v2:172.21.15.148:6838/2317600000 2024-09-12T02:59:04.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.633+0000 7f05ad5eb740 1 -- 172.21.15.5:0/1919622091 shutdown_connections 2024-09-12T02:59:04.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.633+0000 7f05ad5eb740 1 -- 172.21.15.5:0/1919622091 wait complete. 2024-09-12T02:59:04.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.633+0000 7f05ad5eb740 20 client.15600 trim_cache size 0 max 16384 2024-09-12T02:59:04.637 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-12T02:59:04.638 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-09-12T02:59:04.638 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "8649705c-756e-4aef-9450-4d400d073b5c", 2024-09-12T02:59:04.638 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-09-12T02:59:04.638 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-12T02:59:04.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.634+0000 7f05ab212640 1 -- 172.21.15.5:0/2988542772 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f05940a2c90 msgr2=0x7f05940a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:04.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.634+0000 7f05ab212640 1 --2- 172.21.15.5:0/2988542772 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f05940a2c90 0x7f05940a5150 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f059c009430 tx=0x7f059c031db0 comp rx=0 tx=0).stop 2024-09-12T02:59:04.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.634+0000 7f05ab212640 1 -- 172.21.15.5:0/2988542772 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05a4101df0 msgr2=0x7f05a410dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:04.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.634+0000 7f05ab212640 1 --2- 172.21.15.5:0/2988542772 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05a4101df0 0x7f05a410dd60 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f059800c920 tx=0x7f059800cd20 comp rx=0 tx=0).stop 2024-09-12T02:59:04.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.635+0000 7f05ab212640 1 -- 172.21.15.5:0/2988542772 shutdown_connections 2024-09-12T02:59:04.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.635+0000 7f05ab212640 1 --2- 172.21.15.5:0/2988542772 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f05a4106f20 0x7f05a419aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:04.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.635+0000 7f05ab212640 1 --2- 172.21.15.5:0/2988542772 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f05a41027a0 0x7f05a410e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:04.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.635+0000 7f05ab212640 1 --2- 172.21.15.5:0/2988542772 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f05940a2c90 0x7f05940a5150 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:04.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.635+0000 7f05ab212640 1 --2- 172.21.15.5:0/2988542772 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05a4101df0 0x7f05a410dd60 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:04.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.635+0000 7f05ab212640 1 -- 172.21.15.5:0/2988542772 >> 172.21.15.5:0/2988542772 conn(0x7f05a40fd8b0 msgr2=0x7f05a40fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:59:04.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.635+0000 7f05ab212640 1 -- 172.21.15.5:0/2988542772 shutdown_connections 2024-09-12T02:59:04.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:04.635+0000 7f05ab212640 1 -- 172.21.15.5:0/2988542772 wait complete. 2024-09-12T02:59:05.185 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:59:04 smithi005 ceph-mon[31197]: pgmap v360: 97 pgs: 97 active+clean; 663 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-12T02:59:05.185 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:59:04 smithi005 ceph-mon[31197]: scrub queued for path: / 2024-09-12T02:59:05.185 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:59:04 smithi005 ceph-mon[31197]: scrub summary: idle+waiting paths [/] 2024-09-12T02:59:05.185 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:59:04 smithi005 ceph-mon[31197]: scrub summary: active paths [/] 2024-09-12T02:59:05.186 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:59:04 smithi005 ceph-mon[31197]: scrub summary: idle+waiting paths [/] 2024-09-12T02:59:05.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:59:04 smithi146 ceph-mon[35011]: pgmap v360: 97 pgs: 97 active+clean; 663 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-12T02:59:05.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:59:04 smithi146 ceph-mon[35011]: scrub queued for path: / 2024-09-12T02:59:05.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:59:04 smithi146 ceph-mon[35011]: scrub summary: idle+waiting paths [/] 2024-09-12T02:59:05.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:59:04 smithi146 ceph-mon[35011]: scrub summary: active paths [/] 2024-09-12T02:59:05.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:59:04 smithi146 ceph-mon[35011]: scrub summary: idle+waiting paths [/] 2024-09-12T02:59:05.223 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T02:59:05.415 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:59:04 smithi148 ceph-mon[35090]: pgmap v360: 97 pgs: 97 active+clean; 663 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-12T02:59:05.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:59:04 smithi148 ceph-mon[35090]: scrub queued for path: / 2024-09-12T02:59:05.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:59:04 smithi148 ceph-mon[35090]: scrub summary: idle+waiting paths [/] 2024-09-12T02:59:05.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:59:04 smithi148 ceph-mon[35090]: scrub summary: active paths [/] 2024-09-12T02:59:05.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:59:04 smithi148 ceph-mon[35090]: scrub summary: idle+waiting paths [/] 2024-09-12T02:59:05.598 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:59:06.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.122+0000 7f1cf1304640 1 -- 172.21.15.5:0/2926311886 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1cec103740 msgr2=0x7f1cec107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:06.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.122+0000 7f1cf1304640 1 --2- 172.21.15.5:0/2926311886 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1cec103740 0x7f1cec107b80 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f1ce0009930 tx=0x7f1ce002a420 comp rx=0 tx=0).stop 2024-09-12T02:59:06.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.122+0000 7f1cf1304640 1 -- 172.21.15.5:0/2926311886 shutdown_connections 2024-09-12T02:59:06.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.122+0000 7f1cf1304640 1 --2- 172.21.15.5:0/2926311886 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1cec103740 0x7f1cec107b80 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:06.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.122+0000 7f1cf1304640 1 --2- 172.21.15.5:0/2926311886 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1cec108290 0x7f1cec10ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:06.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.122+0000 7f1cf1304640 1 --2- 172.21.15.5:0/2926311886 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1cec102d90 0x7f1cec103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:06.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.122+0000 7f1cf1304640 1 -- 172.21.15.5:0/2926311886 >> 172.21.15.5:0/2926311886 conn(0x7f1cec0fc7d0 msgr2=0x7f1cec0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:59:06.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.122+0000 7f1cf1304640 1 -- 172.21.15.5:0/2926311886 shutdown_connections 2024-09-12T02:59:06.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.122+0000 7f1cf1304640 1 -- 172.21.15.5:0/2926311886 wait complete. 2024-09-12T02:59:06.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.123+0000 7f1cf1304640 1 Processor -- start 2024-09-12T02:59:06.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.123+0000 7f1cf1304640 1 -- start start 2024-09-12T02:59:06.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.123+0000 7f1cf1304640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1cec102d90 0x7f1cec1009f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:06.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.123+0000 7f1cf1304640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1cec103740 0x7f1cec100f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:06.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.123+0000 7f1cf1304640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1cec108290 0x7f1cec101470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:06.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.124+0000 7f1cf1304640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1cec112080 con 0x7f1cec103740 2024-09-12T02:59:06.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.124+0000 7f1cf1304640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f1cec111f00 con 0x7f1cec102d90 2024-09-12T02:59:06.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.124+0000 7f1cea7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1cec103740 0x7f1cec100f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:06.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.124+0000 7f1cf1304640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1cec112200 con 0x7f1cec108290 2024-09-12T02:59:06.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.124+0000 7f1cea7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1cec103740 0x7f1cec100f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52768/0 (socket says 172.21.15.5:52768) 2024-09-12T02:59:06.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.124+0000 7f1cea7fc640 1 -- 172.21.15.5:0/375698171 learned_addr learned my addr 172.21.15.5:0/375698171 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:59:06.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.124+0000 7f1ceaffd640 1 --2- 172.21.15.5:0/375698171 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1cec102d90 0x7f1cec1009f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:06.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.124+0000 7f1ceb7fe640 1 --2- 172.21.15.5:0/375698171 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1cec108290 0x7f1cec101470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:06.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.124+0000 7f1cea7fc640 1 -- 172.21.15.5:0/375698171 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1cec108290 msgr2=0x7f1cec101470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:06.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.124+0000 7f1cea7fc640 1 --2- 172.21.15.5:0/375698171 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1cec108290 0x7f1cec101470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:06.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.124+0000 7f1cea7fc640 1 -- 172.21.15.5:0/375698171 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1cec102d90 msgr2=0x7f1cec1009f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:06.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.124+0000 7f1cea7fc640 1 --2- 172.21.15.5:0/375698171 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1cec102d90 0x7f1cec1009f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:06.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.124+0000 7f1cea7fc640 1 -- 172.21.15.5:0/375698171 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cec1aaf70 con 0x7f1cec103740 2024-09-12T02:59:06.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.124+0000 7f1cea7fc640 1 --2- 172.21.15.5:0/375698171 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1cec103740 0x7f1cec100f30 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f1ce002a940 tx=0x7f1ce0003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:59:06.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.125+0000 7f1cd3fff640 1 -- 172.21.15.5:0/375698171 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce003f070 con 0x7f1cec103740 2024-09-12T02:59:06.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.125+0000 7f1cd3fff640 1 -- 172.21.15.5:0/375698171 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ce0031c90 con 0x7f1cec103740 2024-09-12T02:59:06.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.125+0000 7f1cd3fff640 1 -- 172.21.15.5:0/375698171 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce0035030 con 0x7f1cec103740 2024-09-12T02:59:06.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.125+0000 7f1cf1304640 1 -- 172.21.15.5:0/375698171 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cec1ab140 con 0x7f1cec103740 2024-09-12T02:59:06.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.125+0000 7f1cf1304640 1 -- 172.21.15.5:0/375698171 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cec1ab630 con 0x7f1cec103740 2024-09-12T02:59:06.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.128+0000 7f1cf36dd740 1 Processor -- start 2024-09-12T02:59:06.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.128+0000 7f1cf36dd740 1 -- start start 2024-09-12T02:59:06.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.128+0000 7f1cf36dd740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5632219d64a0 0x5632219d6880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:06.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.128+0000 7f1cf36dd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632219d6dc0 0x5632219e1140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:06.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.128+0000 7f1cf36dd740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5632219e1680 0x5632219e3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:06.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.128+0000 7f1cf36dd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56322193c2e0 con 0x5632219d6dc0 2024-09-12T02:59:06.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.128+0000 7f1cf36dd740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5632218e3ce0 con 0x5632219e1680 2024-09-12T02:59:06.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.128+0000 7f1cf36dd740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5632219382a0 con 0x5632219d64a0 2024-09-12T02:59:06.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.128+0000 7f1ceaffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632219d6dc0 0x5632219e1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:06.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.128+0000 7f1ceaffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632219d6dc0 0x5632219e1140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52780/0 (socket says 172.21.15.5:52780) 2024-09-12T02:59:06.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.128+0000 7f1ceaffd640 1 -- 172.21.15.5:0/2498387997 learned_addr learned my addr 172.21.15.5:0/2498387997 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:59:06.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.128+0000 7f1ceb7fe640 1 --2- 172.21.15.5:0/2498387997 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5632219d64a0 0x5632219d6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:06.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.129+0000 7f1ceaffd640 1 -- 172.21.15.5:0/2498387997 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5632219d64a0 msgr2=0x5632219d6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:06.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.129+0000 7f1ceaffd640 1 --2- 172.21.15.5:0/2498387997 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5632219d64a0 0x5632219d6880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:06.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.129+0000 7f1ceaffd640 1 -- 172.21.15.5:0/2498387997 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5632219e1680 msgr2=0x5632219e3a70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T02:59:06.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.129+0000 7f1ceaffd640 1 --2- 172.21.15.5:0/2498387997 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5632219e1680 0x5632219e3a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:06.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.129+0000 7f1ceaffd640 1 -- 172.21.15.5:0/2498387997 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632218c8440 con 0x5632219d6dc0 2024-09-12T02:59:06.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.129+0000 7f1ceaffd640 1 --2- 172.21.15.5:0/2498387997 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632219d6dc0 0x5632219e1140 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc009cf0 tx=0x7f1cdc011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3be2f40a72b65e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:59:06.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.129+0000 7f1cd17fa640 1 -- 172.21.15.5:0/2498387997 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cdc004230 con 0x5632219d6dc0 2024-09-12T02:59:06.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.129+0000 7f1cd17fa640 1 -- 172.21.15.5:0/2498387997 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cdc0043d0 con 0x5632219d6dc0 2024-09-12T02:59:06.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.129+0000 7f1cd17fa640 1 -- 172.21.15.5:0/2498387997 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cdc019890 con 0x5632219d6dc0 2024-09-12T02:59:06.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.129+0000 7f1cd3fff640 1 -- 172.21.15.5:0/375698171 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7f1ce0042070 con 0x7f1cec103740 2024-09-12T02:59:06.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.130+0000 7f1cf36dd740 1 -- 172.21.15.5:0/2498387997 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632219d6dc0 msgr2=0x5632219e1140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:06.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.130+0000 7f1cf36dd740 1 --2- 172.21.15.5:0/2498387997 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632219d6dc0 0x5632219e1140 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc009cf0 tx=0x7f1cdc011810 comp rx=0 tx=0).stop 2024-09-12T02:59:06.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.130+0000 7f1cf36dd740 1 -- 172.21.15.5:0/2498387997 shutdown_connections 2024-09-12T02:59:06.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.130+0000 7f1cf36dd740 1 --2- 172.21.15.5:0/2498387997 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632219d6dc0 0x5632219e1140 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:06.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.130+0000 7f1cf36dd740 1 --2- 172.21.15.5:0/2498387997 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5632219e1680 0x5632219e3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:06.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.130+0000 7f1cf36dd740 1 --2- 172.21.15.5:0/2498387997 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5632219d64a0 0x5632219d6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:06.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.130+0000 7f1cf36dd740 1 -- 172.21.15.5:0/2498387997 >> 172.21.15.5:0/2498387997 conn(0x5632219d2a30 msgr2=0x5632219d2e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:59:06.135 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.131+0000 7f1cf36dd740 1 -- 172.21.15.5:0/2498387997 shutdown_connections 2024-09-12T02:59:06.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.131+0000 7f1cf36dd740 1 -- 172.21.15.5:0/2498387997 wait complete. 2024-09-12T02:59:06.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.131+0000 7f1cd3fff640 1 --2- 172.21.15.5:0/375698171 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f1cd40a2ba0 0x7f1cd40a5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:06.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.131+0000 7f1cd3fff640 1 -- 172.21.15.5:0/375698171 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f1ce00ef470 con 0x7f1cec103740 2024-09-12T02:59:06.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.131+0000 7f1cf36dd740 1 -- start start 2024-09-12T02:59:06.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.131+0000 7f1ceaffd640 1 --2- 172.21.15.5:0/375698171 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f1cd40a2ba0 0x7f1cd40a5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:06.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.131+0000 7f1cf36dd740 1 Processor -- start 2024-09-12T02:59:06.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.132+0000 7f1cf36dd740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5632219d64a0 0x5632219e1050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:06.139 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.132+0000 7f1cf36dd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632219d6dc0 0x5632219de300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:06.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.132+0000 7f1cf36dd740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5632219e1680 0x5632219de8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:06.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.132+0000 7f1cf36dd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5632219e4250 con 0x5632219d6dc0 2024-09-12T02:59:06.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.132+0000 7f1cf36dd740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5632218f7050 con 0x5632219d64a0 2024-09-12T02:59:06.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.132+0000 7f1cf36dd740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5632219e43d0 con 0x5632219e1680 2024-09-12T02:59:06.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.132+0000 7f1cf36dd740 10 client.0 _ll_register_callbacks cb 0x563221937160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:59:06.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.132+0000 7f1cea7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632219d6dc0 0x5632219de300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:06.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.132+0000 7f1cea7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632219d6dc0 0x5632219de300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52790/0 (socket says 172.21.15.5:52790) 2024-09-12T02:59:06.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.132+0000 7f1cea7fc640 1 -- 172.21.15.5:0/4145025842 learned_addr learned my addr 172.21.15.5:0/4145025842 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:59:06.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.132+0000 7f1ceb7fe640 1 --2- 172.21.15.5:0/4145025842 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5632219e1680 0x5632219de8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:06.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.132+0000 7f1ceaffd640 1 --2- 172.21.15.5:0/375698171 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f1cd40a2ba0 0x7f1cd40a5060 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc01b040 tx=0x7f1cdc003d70 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:59:06.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.132+0000 7f1cea7fc640 1 -- 172.21.15.5:0/4145025842 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5632219e1680 msgr2=0x5632219de8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:06.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.132+0000 7f1cea7fc640 1 --2- 172.21.15.5:0/4145025842 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5632219e1680 0x5632219de8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:06.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.132+0000 7f1cea7fc640 1 -- 172.21.15.5:0/4145025842 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5632219d64a0 msgr2=0x5632219e1050 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-12T02:59:06.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.132+0000 7f1cea7fc640 1 --2- 172.21.15.5:0/4145025842 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5632219d64a0 0x5632219e1050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:06.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.132+0000 7f1cea7fc640 1 -- 172.21.15.5:0/4145025842 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632216ad3c0 con 0x5632219d6dc0 2024-09-12T02:59:06.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.133+0000 7f1cea7fc640 1 --2- 172.21.15.5:0/4145025842 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632219d6dc0 0x5632219de300 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f1ce00bb930 tx=0x7f1ce0005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f306ebf94ac4ae0b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:59:06.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.133+0000 7f1cceff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T02:59:06.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.133+0000 7f1cceff5640 1 -- 172.21.15.5:0/4145025842 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce0049070 con 0x5632219d6dc0 2024-09-12T02:59:06.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.133+0000 7f1cceff5640 1 -- 172.21.15.5:0/4145025842 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f1ce00b82a0 con 0x5632219d6dc0 2024-09-12T02:59:06.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.133+0000 7f1cf36dd740 1 -- 172.21.15.5:0/4145025842 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1cdc0043d0 con 0x5632219d6dc0 2024-09-12T02:59:06.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.133+0000 7f1cceff5640 1 -- 172.21.15.5:0/4145025842 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1ce00d9050 con 0x5632219d6dc0 2024-09-12T02:59:06.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.133+0000 7f1cf36dd740 10 client.15612 fetch_fsmap learned FSMap version 26 2024-09-12T02:59:06.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.133+0000 7f1cf36dd740 1 -- 172.21.15.5:0/4145025842 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5632219ed6f0 con 0x5632219d6dc0 2024-09-12T02:59:06.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.133+0000 7f1cceff5640 1 -- 172.21.15.5:0/4145025842 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ce00d3030 con 0x5632219d6dc0 2024-09-12T02:59:06.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.134+0000 7f1cceff5640 1 -- 172.21.15.5:0/4145025842 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f1ce00c4360 con 0x5632219d6dc0 2024-09-12T02:59:06.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.134+0000 7f1cf36dd740 10 client.15612 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T02:59:06.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.134+0000 7f1cf36dd740 10 client.15612 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T02:59:06.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.134+0000 7f1cf36dd740 20 client.15612 populate_metadata read hostname 'smithi005' 2024-09-12T02:59:06.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.134+0000 7f1cf36dd740 1 --2- 172.21.15.5:0/4145025842 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x5632219eec00 0x5632219f0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:06.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.134+0000 7f1cf36dd740 4 client.15612 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:59:06.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.134+0000 7f1cf36dd740 1 -- 172.21.15.5:0/4145025842 --> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563221949850 con 0x5632219eec00 2024-09-12T02:59:06.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.134+0000 7f1ceaffd640 1 --2- 172.21.15.5:0/4145025842 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x5632219eec00 0x5632219f0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:06.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.134+0000 7f1ceaffd640 1 --2- 172.21.15.5:0/4145025842 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x5632219eec00 0x5632219f0ff0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41c584da16153f14 server_cookie=822120561121abe2 in_seq=0 out_seq=0 2024-09-12T02:59:06.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.134+0000 7f1cceff5640 10 client.15612 ms_handle_connect on v2:172.21.15.148:6838/2317600000 2024-09-12T02:59:06.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.135+0000 7f1cceff5640 1 -- 172.21.15.5:0/4145025842 <== mds.0 v2:172.21.15.148:6838/2317600000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563221949850 con 0x5632219eec00 2024-09-12T02:59:06.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.135+0000 7f1cceff5640 10 client.15612 handle_command_reply: tid=0 2024-09-12T02:59:06.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.135+0000 7f1cf36dd740 1 client.15612 shutdown 2024-09-12T02:59:06.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.135+0000 7f1cf36dd740 1 -- 172.21.15.5:0/4145025842 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632219d6dc0 msgr2=0x5632219de300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:06.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.135+0000 7f1cf36dd740 1 --2- 172.21.15.5:0/4145025842 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632219d6dc0 0x5632219de300 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f1ce00bb930 tx=0x7f1ce0005e50 comp rx=0 tx=0).stop 2024-09-12T02:59:06.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.135+0000 7f1cf36dd740 1 -- 172.21.15.5:0/4145025842 shutdown_connections 2024-09-12T02:59:06.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.135+0000 7f1cf36dd740 1 --2- 172.21.15.5:0/4145025842 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x5632219eec00 0x5632219f0ff0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:06.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.135+0000 7f1cf36dd740 1 --2- 172.21.15.5:0/4145025842 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632219d6dc0 0x5632219de300 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:06.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.135+0000 7f1cf36dd740 1 --2- 172.21.15.5:0/4145025842 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5632219e1680 0x5632219de8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:06.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.135+0000 7f1cf36dd740 1 --2- 172.21.15.5:0/4145025842 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5632219d64a0 0x5632219e1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:06.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.135+0000 7f1cf36dd740 1 -- 172.21.15.5:0/4145025842 >> 172.21.15.5:0/4145025842 conn(0x5632219d2a30 msgr2=0x5632219d8710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:59:06.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.135+0000 7f1cceff5640 0 client.15612 ms_handle_reset on v2:172.21.15.148:6838/2317600000 2024-09-12T02:59:06.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.135+0000 7f1cf36dd740 1 -- 172.21.15.5:0/4145025842 shutdown_connections 2024-09-12T02:59:06.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.135+0000 7f1cf36dd740 1 -- 172.21.15.5:0/4145025842 wait complete. 2024-09-12T02:59:06.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.136+0000 7f1cf36dd740 20 client.15612 trim_cache size 0 max 16384 2024-09-12T02:59:06.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.153+0000 7f1cf36dd740 1 Processor -- start 2024-09-12T02:59:06.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.153+0000 7f1cf36dd740 1 -- start start 2024-09-12T02:59:06.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.153+0000 7f1cf36dd740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5632219d64a0 0x5632219fe560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:06.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.153+0000 7f1cf36dd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632219d6dc0 0x5632219d2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:06.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.153+0000 7f1cf36dd740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5632219e1680 0x5632219f6860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:06.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.153+0000 7f1cf36dd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1cec1a4a90 con 0x5632219d6dc0 2024-09-12T02:59:06.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.153+0000 7f1cf36dd740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f1cec06d1a0 con 0x5632219d64a0 2024-09-12T02:59:06.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.153+0000 7f1cf36dd740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1cec06bd30 con 0x5632219e1680 2024-09-12T02:59:06.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.153+0000 7f1cea7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632219d6dc0 0x5632219d2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:06.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.153+0000 7f1cea7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632219d6dc0 0x5632219d2e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52796/0 (socket says 172.21.15.5:52796) 2024-09-12T02:59:06.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.153+0000 7f1cea7fc640 1 -- 172.21.15.5:0/3257881008 learned_addr learned my addr 172.21.15.5:0/3257881008 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:59:06.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.154+0000 7f1ceaffd640 1 --2- 172.21.15.5:0/3257881008 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5632219d64a0 0x5632219fe560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:06.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.154+0000 7f1ceb7fe640 1 --2- 172.21.15.5:0/3257881008 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5632219e1680 0x5632219f6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:06.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.154+0000 7f1cea7fc640 1 -- 172.21.15.5:0/3257881008 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5632219e1680 msgr2=0x5632219f6860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:06.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.154+0000 7f1cea7fc640 1 --2- 172.21.15.5:0/3257881008 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5632219e1680 0x5632219f6860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:06.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.154+0000 7f1cea7fc640 1 -- 172.21.15.5:0/3257881008 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5632219d64a0 msgr2=0x5632219fe560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:06.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.154+0000 7f1cea7fc640 1 --2- 172.21.15.5:0/3257881008 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5632219d64a0 0x5632219fe560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:06.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.154+0000 7f1cea7fc640 1 -- 172.21.15.5:0/3257881008 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632219f15d0 con 0x5632219d6dc0 2024-09-12T02:59:06.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.154+0000 7f1cea7fc640 1 --2- 172.21.15.5:0/3257881008 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632219d6dc0 0x5632219d2e10 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f1ce00bbbc0 tx=0x7f1ce00bbdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23900c8d7e10b12e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:59:06.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.154+0000 7f1cd0ff9640 1 -- 172.21.15.5:0/3257881008 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce0049070 con 0x5632219d6dc0 2024-09-12T02:59:06.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.154+0000 7f1cd0ff9640 1 -- 172.21.15.5:0/3257881008 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ce00d3070 con 0x5632219d6dc0 2024-09-12T02:59:06.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.154+0000 7f1cd0ff9640 1 -- 172.21.15.5:0/3257881008 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce00b82a0 con 0x5632219d6dc0 2024-09-12T02:59:06.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.155+0000 7f1cf36dd740 1 -- 172.21.15.5:0/3257881008 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632219d6dc0 msgr2=0x5632219d2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:06.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.155+0000 7f1cf36dd740 1 --2- 172.21.15.5:0/3257881008 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632219d6dc0 0x5632219d2e10 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f1ce00bbbc0 tx=0x7f1ce00bbdb0 comp rx=0 tx=0).stop 2024-09-12T02:59:06.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.155+0000 7f1cf36dd740 1 -- 172.21.15.5:0/3257881008 shutdown_connections 2024-09-12T02:59:06.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.155+0000 7f1cf36dd740 1 --2- 172.21.15.5:0/3257881008 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632219d6dc0 0x5632219d2e10 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:06.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.155+0000 7f1cf36dd740 1 --2- 172.21.15.5:0/3257881008 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5632219e1680 0x5632219f6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:06.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.155+0000 7f1cf36dd740 1 --2- 172.21.15.5:0/3257881008 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5632219d64a0 0x5632219fe560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:06.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.155+0000 7f1cf36dd740 1 -- 172.21.15.5:0/3257881008 >> 172.21.15.5:0/3257881008 conn(0x5632219d2a30 msgr2=0x5632219e0380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:59:06.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.155+0000 7f1cf36dd740 1 -- 172.21.15.5:0/3257881008 shutdown_connections 2024-09-12T02:59:06.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.155+0000 7f1cf36dd740 1 -- 172.21.15.5:0/3257881008 wait complete. 2024-09-12T02:59:06.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.156+0000 7f1cf36dd740 1 -- start start 2024-09-12T02:59:06.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.156+0000 7f1cf36dd740 1 Processor -- start 2024-09-12T02:59:06.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.156+0000 7f1cf36dd740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5632219d64a0 0x5632219e49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:06.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.156+0000 7f1cf36dd740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5632219e1680 0x5632219e4f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:06.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.156+0000 7f1cf36dd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563221a58ec0 0x563221a5b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:06.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.156+0000 7f1cf36dd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1cec06cad0 con 0x563221a58ec0 2024-09-12T02:59:06.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.156+0000 7f1cf36dd740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f1cec06c400 con 0x5632219e1680 2024-09-12T02:59:06.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.156+0000 7f1cf36dd740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1cec1a2a70 con 0x5632219d64a0 2024-09-12T02:59:06.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.156+0000 7f1cf36dd740 10 client.0 _ll_register_callbacks cb 0x7f1ce0031e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:59:06.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.156+0000 7f1ceb7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563221a58ec0 0x563221a5b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:06.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.156+0000 7f1ceaffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5632219d64a0 0x5632219e49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:06.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.156+0000 7f1ceb7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563221a58ec0 0x563221a5b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52798/0 (socket says 172.21.15.5:52798) 2024-09-12T02:59:06.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.156+0000 7f1ceb7fe640 1 -- 172.21.15.5:0/2982510744 learned_addr learned my addr 172.21.15.5:0/2982510744 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:59:06.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.157+0000 7f1ceaffd640 1 -- 172.21.15.5:0/2982510744 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5632219e1680 msgr2=0x5632219e4f10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T02:59:06.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.157+0000 7f1ceaffd640 1 --2- 172.21.15.5:0/2982510744 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5632219e1680 0x5632219e4f10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:06.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.157+0000 7f1ceaffd640 1 -- 172.21.15.5:0/2982510744 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563221a58ec0 msgr2=0x563221a5b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:06.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.157+0000 7f1ceaffd640 1 --2- 172.21.15.5:0/2982510744 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563221a58ec0 0x563221a5b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:06.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.157+0000 7f1ceaffd640 1 -- 172.21.15.5:0/2982510744 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563221a5b7f0 con 0x5632219d64a0 2024-09-12T02:59:06.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.157+0000 7f1ceaffd640 1 --2- 172.21.15.5:0/2982510744 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5632219d64a0 0x5632219e49d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc009ab0 tx=0x7f1cdc011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab97d4164b6d4d8f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:59:06.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.157+0000 7f1cceff5640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-09-12T02:59:06.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.157+0000 7f1cceff5640 1 -- 172.21.15.5:0/2982510744 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cdc03d020 con 0x5632219d64a0 2024-09-12T02:59:06.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.157+0000 7f1cf36dd740 1 -- 172.21.15.5:0/2982510744 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ce00d3070 con 0x5632219d64a0 2024-09-12T02:59:06.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.157+0000 7f1cceff5640 1 -- 172.21.15.5:0/2982510744 <== mon.1 v2:172.21.15.148:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f1cdc004580 con 0x5632219d64a0 2024-09-12T02:59:06.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.157+0000 7f1cceff5640 1 -- 172.21.15.5:0/2982510744 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1cdc004780 con 0x5632219d64a0 2024-09-12T02:59:06.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.157+0000 7f1cf36dd740 10 client.25201 fetch_fsmap learned FSMap version 26 2024-09-12T02:59:06.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.157+0000 7f1cf36dd740 1 -- 172.21.15.5:0/2982510744 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563221a7eb10 con 0x5632219d64a0 2024-09-12T02:59:06.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.158+0000 7f1cceff5640 1 -- 172.21.15.5:0/2982510744 <== mon.1 v2:172.21.15.148:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1ce00d3070 con 0x5632219d64a0 2024-09-12T02:59:06.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.158+0000 7f1cceff5640 1 -- 172.21.15.5:0/2982510744 <== mon.1 v2:172.21.15.148:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f1cdc038070 con 0x5632219d64a0 2024-09-12T02:59:06.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.158+0000 7f1cf36dd740 10 client.25201 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T02:59:06.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.158+0000 7f1cf36dd740 10 client.25201 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T02:59:06.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.158+0000 7f1cf36dd740 20 client.25201 populate_metadata read hostname 'smithi005' 2024-09-12T02:59:06.164 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-12T02:59:06.164 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-12T02:59:06.164 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-09-12T02:59:06.164 INFO:teuthology.orchestra.run.smithi005.stdout: "8649705c-756e-4aef-9450-4d400d073b5c": { 2024-09-12T02:59:06.164 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-09-12T02:59:06.164 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "8649705c-756e-4aef-9450-4d400d073b5c", 2024-09-12T02:59:06.164 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-09-12T02:59:06.164 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T02:59:06.164 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T02:59:06.164 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-12T02:59:06.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.158+0000 7f1cf36dd740 1 --2- 172.21.15.5:0/2982510744 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x563221a7f0a0 0x563221a81490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:06.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.158+0000 7f1cf36dd740 4 client.25201 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T02:59:06.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.158+0000 7f1cf36dd740 1 -- 172.21.15.5:0/2982510744 --> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563221943d60 con 0x563221a7f0a0 2024-09-12T02:59:06.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.159+0000 7f1cea7fc640 1 --2- 172.21.15.5:0/2982510744 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x563221a7f0a0 0x563221a81490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:06.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.159+0000 7f1cea7fc640 1 --2- 172.21.15.5:0/2982510744 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x563221a7f0a0 0x563221a81490 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3246f42bd775cf97 server_cookie=3f680c5f12693672 in_seq=0 out_seq=0 2024-09-12T02:59:06.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.159+0000 7f1cceff5640 10 client.25201 ms_handle_connect on v2:172.21.15.148:6838/2317600000 2024-09-12T02:59:06.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.159+0000 7f1cceff5640 1 -- 172.21.15.5:0/2982510744 <== mds.0 v2:172.21.15.148:6838/2317600000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x563221943d60 con 0x563221a7f0a0 2024-09-12T02:59:06.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.159+0000 7f1cceff5640 10 client.25201 handle_command_reply: tid=0 2024-09-12T02:59:06.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.159+0000 7f1cf36dd740 1 client.25201 shutdown 2024-09-12T02:59:06.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.159+0000 7f1cf36dd740 1 -- 172.21.15.5:0/2982510744 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5632219d64a0 msgr2=0x5632219e49d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:06.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.159+0000 7f1cf36dd740 1 --2- 172.21.15.5:0/2982510744 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5632219d64a0 0x5632219e49d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc009ab0 tx=0x7f1cdc011810 comp rx=0 tx=0).stop 2024-09-12T02:59:06.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.159+0000 7f1cf36dd740 1 -- 172.21.15.5:0/2982510744 shutdown_connections 2024-09-12T02:59:06.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.159+0000 7f1cf36dd740 1 --2- 172.21.15.5:0/2982510744 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x563221a7f0a0 0x563221a81490 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:06.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.159+0000 7f1cf36dd740 1 --2- 172.21.15.5:0/2982510744 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563221a58ec0 0x563221a5b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:06.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.159+0000 7f1cf36dd740 1 --2- 172.21.15.5:0/2982510744 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5632219e1680 0x5632219e4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:06.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.159+0000 7f1cf36dd740 1 --2- 172.21.15.5:0/2982510744 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5632219d64a0 0x5632219e49d0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:06.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.159+0000 7f1cf36dd740 1 -- 172.21.15.5:0/2982510744 >> 172.21.15.5:0/2982510744 conn(0x5632219d2a30 msgr2=0x5632219d7ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:59:06.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.159+0000 7f1cceff5640 0 client.25201 ms_handle_reset on v2:172.21.15.148:6838/2317600000 2024-09-12T02:59:06.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.159+0000 7f1cf36dd740 1 -- 172.21.15.5:0/2982510744 shutdown_connections 2024-09-12T02:59:06.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.159+0000 7f1cf36dd740 1 -- 172.21.15.5:0/2982510744 wait complete. 2024-09-12T02:59:06.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.160+0000 7f1cf36dd740 20 client.25201 trim_cache size 0 max 16384 2024-09-12T02:59:06.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.160+0000 7f1cf1304640 1 -- 172.21.15.5:0/375698171 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f1cd40a2ba0 msgr2=0x7f1cd40a5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:06.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.160+0000 7f1cf1304640 1 --2- 172.21.15.5:0/375698171 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f1cd40a2ba0 0x7f1cd40a5060 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc01b040 tx=0x7f1cdc003d70 comp rx=0 tx=0).stop 2024-09-12T02:59:06.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.160+0000 7f1cf1304640 1 -- 172.21.15.5:0/375698171 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1cec103740 msgr2=0x7f1cec100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:06.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.160+0000 7f1cf1304640 1 --2- 172.21.15.5:0/375698171 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1cec103740 0x7f1cec100f30 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f1ce002a940 tx=0x7f1ce0003b90 comp rx=0 tx=0).stop 2024-09-12T02:59:06.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.160+0000 7f1cf1304640 1 -- 172.21.15.5:0/375698171 shutdown_connections 2024-09-12T02:59:06.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.160+0000 7f1cf1304640 1 --2- 172.21.15.5:0/375698171 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f1cd40a2ba0 0x7f1cd40a5060 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:06.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.160+0000 7f1cf1304640 1 --2- 172.21.15.5:0/375698171 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1cec103740 0x7f1cec100f30 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:06.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.160+0000 7f1cf1304640 1 --2- 172.21.15.5:0/375698171 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1cec108290 0x7f1cec101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:06.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.160+0000 7f1cf1304640 1 --2- 172.21.15.5:0/375698171 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1cec102d90 0x7f1cec1009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:06.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.160+0000 7f1cf1304640 1 -- 172.21.15.5:0/375698171 >> 172.21.15.5:0/375698171 conn(0x7f1cec0fc7d0 msgr2=0x7f1cec105d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:59:06.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.160+0000 7f1cf1304640 1 -- 172.21.15.5:0/375698171 shutdown_connections 2024-09-12T02:59:06.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:06.160+0000 7f1cf1304640 1 -- 172.21.15.5:0/375698171 wait complete. 2024-09-12T02:59:06.884 INFO:tasks.cephfs.filesystem:scrub status for tag:8649705c-756e-4aef-9450-4d400d073b5c - {'path': '/', 'tag': '8649705c-756e-4aef-9450-4d400d073b5c', 'options': 'recursive,force'} 2024-09-12T02:59:07.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:59:06 smithi005 ceph-mon[31197]: pgmap v361: 97 pgs: 97 active+clean; 663 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 682 B/s wr, 9 op/s 2024-09-12T02:59:07.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:59:06 smithi146 ceph-mon[35011]: pgmap v361: 97 pgs: 97 active+clean; 663 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 682 B/s wr, 9 op/s 2024-09-12T02:59:07.415 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:59:06 smithi148 ceph-mon[35090]: pgmap v361: 97 pgs: 97 active+clean; 663 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 682 B/s wr, 9 op/s 2024-09-12T02:59:08.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:59:07 smithi146 ceph-mon[35011]: scrub summary: idle 2024-09-12T02:59:08.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:59:07 smithi005 ceph-mon[31197]: scrub summary: idle 2024-09-12T02:59:08.415 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:59:07 smithi148 ceph-mon[35090]: scrub summary: idle 2024-09-12T02:59:09.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:59:08 smithi146 ceph-mon[35011]: pgmap v362: 97 pgs: 97 active+clean; 663 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 9 op/s 2024-09-12T02:59:09.211 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:59:08 smithi005 ceph-mon[31197]: pgmap v362: 97 pgs: 97 active+clean; 663 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 9 op/s 2024-09-12T02:59:09.415 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:59:08 smithi148 ceph-mon[35090]: pgmap v362: 97 pgs: 97 active+clean; 663 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 9 op/s 2024-09-12T02:59:11.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:59:10 smithi005 ceph-mon[31197]: pgmap v363: 97 pgs: 97 active+clean; 663 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-12T02:59:11.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:59:10 smithi146 ceph-mon[35011]: pgmap v363: 97 pgs: 97 active+clean; 663 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-12T02:59:11.217 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:59:10 smithi148 ceph-mon[35090]: pgmap v363: 97 pgs: 97 active+clean; 663 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 682 B/s wr, 11 op/s 2024-09-12T02:59:12.455 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:59:12.743 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:59:13.164 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:59:12 smithi005 ceph-mon[31197]: pgmap v364: 97 pgs: 97 active+clean; 663 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-12T02:59:13.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:59:12 smithi146 ceph-mon[35011]: pgmap v364: 97 pgs: 97 active+clean; 663 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-12T02:59:13.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:13.251+0000 7f1c6f0b1640 1 -- 172.21.15.5:0/1226239755 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c68108300 msgr2=0x7f1c6810ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:13.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:13.251+0000 7f1c6f0b1640 1 --2- 172.21.15.5:0/1226239755 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c68108300 0x7f1c6810ec70 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f1c60009cd0 tx=0x7f1c6002a8d0 comp rx=0 tx=0).stop 2024-09-12T02:59:13.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:13.251+0000 7f1c6f0b1640 1 -- 172.21.15.5:0/1226239755 shutdown_connections 2024-09-12T02:59:13.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:13.251+0000 7f1c6f0b1640 1 --2- 172.21.15.5:0/1226239755 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c68108300 0x7f1c6810ec70 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:13.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:13.251+0000 7f1c6f0b1640 1 --2- 172.21.15.5:0/1226239755 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c68103780 0x7f1c68107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:13.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:13.251+0000 7f1c6f0b1640 1 --2- 172.21.15.5:0/1226239755 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1c68102dd0 0x7f1c681031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:13.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:13.251+0000 7f1c6f0b1640 1 -- 172.21.15.5:0/1226239755 >> 172.21.15.5:0/1226239755 conn(0x7f1c680fc7d0 msgr2=0x7f1c680febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:59:13.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:13.251+0000 7f1c6f0b1640 1 -- 172.21.15.5:0/1226239755 shutdown_connections 2024-09-12T02:59:13.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:13.251+0000 7f1c6f0b1640 1 -- 172.21.15.5:0/1226239755 wait complete. 2024-09-12T02:59:13.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:13.252+0000 7f1c6f0b1640 1 Processor -- start 2024-09-12T02:59:13.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:13.252+0000 7f1c6f0b1640 1 -- start start 2024-09-12T02:59:13.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:13.253+0000 7f1c6f0b1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c68102dd0 0x7f1c68100a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:13.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:13.253+0000 7f1c6f0b1640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1c68103780 0x7f1c68100f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:13.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:13.253+0000 7f1c6f0b1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c68108300 0x7f1c681014b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:13.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:13.253+0000 7f1c6ce4d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c68102dd0 0x7f1c68100a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:13.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:13.253+0000 7f1c6f0b1640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1c681120d0 con 0x7f1c68102dd0 2024-09-12T02:59:13.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:13.253+0000 7f1c6f0b1640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f1c68111f50 con 0x7f1c68103780 2024-09-12T02:59:13.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:13.253+0000 7f1c6f0b1640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1c68112250 con 0x7f1c68108300 2024-09-12T02:59:13.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:13.253+0000 7f1c67fff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1c68103780 0x7f1c68100f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:13.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:13.253+0000 7f1c67fff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1c68103780 0x7f1c68100f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.5:36804/0 (socket says 172.21.15.5:36804) 2024-09-12T02:59:13.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:13.253+0000 7f1c67fff640 1 -- 172.21.15.5:0/3407324473 learned_addr learned my addr 172.21.15.5:0/3407324473 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:59:13.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:13.253+0000 7f1c6d64e640 1 --2- 172.21.15.5:0/3407324473 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c68108300 0x7f1c681014b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:13.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:13.253+0000 7f1c67fff640 1 -- 172.21.15.5:0/3407324473 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c68108300 msgr2=0x7f1c681014b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:13.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:13.253+0000 7f1c67fff640 1 --2- 172.21.15.5:0/3407324473 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c68108300 0x7f1c681014b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:13.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:13.253+0000 7f1c67fff640 1 -- 172.21.15.5:0/3407324473 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c68102dd0 msgr2=0x7f1c68100a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:13.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:13.253+0000 7f1c67fff640 1 --2- 172.21.15.5:0/3407324473 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c68102dd0 0x7f1c68100a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:13.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:13.253+0000 7f1c67fff640 1 -- 172.21.15.5:0/3407324473 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c681aaf30 con 0x7f1c68103780 2024-09-12T02:59:13.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:13.254+0000 7f1c67fff640 1 --2- 172.21.15.5:0/3407324473 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1c68103780 0x7f1c68100f70 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1c5400d950 tx=0x7f1c5400de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:59:13.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:13.254+0000 7f1c65ffb640 1 -- 172.21.15.5:0/3407324473 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c54017070 con 0x7f1c68103780 2024-09-12T02:59:13.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:13.254+0000 7f1c6f0b1640 1 -- 172.21.15.5:0/3407324473 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c681ab160 con 0x7f1c68103780 2024-09-12T02:59:13.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:13.254+0000 7f1c65ffb640 1 -- 172.21.15.5:0/3407324473 <== mon.2 v2:172.21.15.146:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c54009c30 con 0x7f1c68103780 2024-09-12T02:59:13.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:13.254+0000 7f1c65ffb640 1 -- 172.21.15.5:0/3407324473 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c540047a0 con 0x7f1c68103780 2024-09-12T02:59:13.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:13.254+0000 7f1c6f0b1640 1 -- 172.21.15.5:0/3407324473 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c681ab6f0 con 0x7f1c68103780 2024-09-12T02:59:13.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:13.255+0000 7f1c6f0b1640 1 -- 172.21.15.5:0/3407324473 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c6806a7b0 con 0x7f1c68103780 2024-09-12T02:59:13.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:13.257+0000 7f1c65ffb640 1 -- 172.21.15.5:0/3407324473 <== mon.2 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7f1c54004020 con 0x7f1c68103780 2024-09-12T02:59:13.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:13.257+0000 7f1c65ffb640 1 --2- 172.21.15.5:0/3407324473 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f1c580a2c90 0x7f1c580a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:13.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:13.258+0000 7f1c65ffb640 1 -- 172.21.15.5:0/3407324473 <== mon.2 v2:172.21.15.146:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f1c540cf060 con 0x7f1c68103780 2024-09-12T02:59:13.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:13.258+0000 7f1c6ce4d640 1 --2- 172.21.15.5:0/3407324473 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f1c580a2c90 0x7f1c580a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:13.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:13.258+0000 7f1c6ce4d640 1 --2- 172.21.15.5:0/3407324473 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f1c580a2c90 0x7f1c580a5150 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c009b60 tx=0x7f1c5c005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:59:13.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:13.261+0000 7f1c65ffb640 1 -- 172.21.15.5:0/3407324473 <== mon.2 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1c54020020 con 0x7f1c68103780 2024-09-12T02:59:13.415 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:59:12 smithi148 ceph-mon[35090]: pgmap v364: 97 pgs: 97 active+clean; 663 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-12T02:59:13.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:13.472+0000 7f1c6f0b1640 1 -- 172.21.15.5:0/3407324473 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1c681031b0 con 0x7f1c68103780 2024-09-12T02:59:13.473 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-12T02:59:13.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:13.473+0000 7f1c65ffb640 1 -- 172.21.15.5:0/3407324473 <== mon.2 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11966 (secure 0 0 0) 0x7f1c681031b0 con 0x7f1c68103780 2024-09-12T02:59:13.474 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":26,"btime":"2024-09-12T02:58:03:323671+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/4135232521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":4135232521},{"type":"v1","addr":"172.21.15.5:6835","nonce":4135232521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/4290221940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":4290221940},{"type":"v1","addr":"172.21.15.5:6837","nonce":4290221940}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T02:56:16.595562+0000","modified":"2024-09-12T02:58:02.463400+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24502,"mds_1":24487,"mds_2":24481,"mds_3":24479,"mds_4":24458},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/26447341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":26447341},{"type":"v1","addr":"172.21.15.5:6839","nonce":26447341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2393443295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2393443295},{"type":"v1","addr":"172.21.15.5:6841","nonce":2393443295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6833/3686970137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3686970137},{"type":"v1","addr":"172.21.15.146:6833","nonce":3686970137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6833/568512209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":568512209},{"type":"v1","addr":"172.21.15.148:6833","nonce":568512209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6835/2540740420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2540740420},{"type":"v1","addr":"172.21.15.146:6835","nonce":2540740420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.146:6837/1673370937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":1673370937},{"type":"v1","addr":"172.21.15.146:6837","nonce":1673370937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":3,"incarnation":22,"state":"up:active","state_seq":6,"addr":"172.21.15.146:6839/3036767228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":3036767228},{"type":"v1","addr":"172.21.15.146:6839","nonce":3036767228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":2,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.148:6835/4195086427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":4195086427},{"type":"v1","addr":"172.21.15.148:6835","nonce":4195086427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.148:6837/2606873922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2606873922},{"type":"v1","addr":"172.21.15.148:6837","nonce":2606873922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6839/2317600000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":2317600000},{"type":"v1","addr":"172.21.15.148:6839","nonce":2317600000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24502,"qdb_cluster":[24502,24487,24481,24479,24458]},"id":1}]} 2024-09-12T02:59:13.474 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 26 2024-09-12T02:59:13.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:13.476+0000 7f1c6f0b1640 1 -- 172.21.15.5:0/3407324473 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f1c580a2c90 msgr2=0x7f1c580a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:13.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:13.476+0000 7f1c6f0b1640 1 --2- 172.21.15.5:0/3407324473 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f1c580a2c90 0x7f1c580a5150 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c009b60 tx=0x7f1c5c005f90 comp rx=0 tx=0).stop 2024-09-12T02:59:13.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:13.476+0000 7f1c6f0b1640 1 -- 172.21.15.5:0/3407324473 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1c68103780 msgr2=0x7f1c68100f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:13.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:13.476+0000 7f1c6f0b1640 1 --2- 172.21.15.5:0/3407324473 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1c68103780 0x7f1c68100f70 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1c5400d950 tx=0x7f1c5400de10 comp rx=0 tx=0).stop 2024-09-12T02:59:13.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:13.476+0000 7f1c6f0b1640 1 -- 172.21.15.5:0/3407324473 shutdown_connections 2024-09-12T02:59:13.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:13.476+0000 7f1c6f0b1640 1 --2- 172.21.15.5:0/3407324473 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1c68108300 0x7f1c681014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:13.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:13.476+0000 7f1c6f0b1640 1 --2- 172.21.15.5:0/3407324473 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1c68103780 0x7f1c68100f70 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:13.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:13.476+0000 7f1c6f0b1640 1 --2- 172.21.15.5:0/3407324473 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f1c580a2c90 0x7f1c580a5150 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:13.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:13.476+0000 7f1c6f0b1640 1 --2- 172.21.15.5:0/3407324473 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c68102dd0 0x7f1c68100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:13.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:13.476+0000 7f1c6f0b1640 1 -- 172.21.15.5:0/3407324473 >> 172.21.15.5:0/3407324473 conn(0x7f1c680fc7d0 msgr2=0x7f1c68105e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:59:13.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:13.476+0000 7f1c6f0b1640 1 -- 172.21.15.5:0/3407324473 shutdown_connections 2024-09-12T02:59:13.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:13.476+0000 7f1c6f0b1640 1 -- 172.21.15.5:0/3407324473 wait complete. 2024-09-12T02:59:14.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:59:13 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/3407324473' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:59:14.207 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:59:13 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/3407324473' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:59:14.242 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24502 quiesce db --set-id 26c61f34 --release --await' 2024-09-12T02:59:14.242 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24502 quiesce db --set-id 26c61f34 --release --await 2024-09-12T02:59:14.415 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:59:13 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/3407324473' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:59:14.535 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:59:15.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.043+0000 7f9dcbf45640 1 -- 172.21.15.5:0/3142517390 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9dc410c180 msgr2=0x7f9dc410e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:15.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.043+0000 7f9dcbf45640 1 --2- 172.21.15.5:0/3142517390 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9dc410c180 0x7f9dc410e620 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc009ca0 tx=0x7f9dbc02a950 comp rx=0 tx=0).stop 2024-09-12T02:59:15.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.043+0000 7f9dcbf45640 1 -- 172.21.15.5:0/3142517390 shutdown_connections 2024-09-12T02:59:15.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.043+0000 7f9dcbf45640 1 --2- 172.21.15.5:0/3142517390 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9dc410c180 0x7f9dc410e620 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:15.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.043+0000 7f9dcbf45640 1 --2- 172.21.15.5:0/3142517390 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9dc4103670 0x7f9dc410bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:15.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.043+0000 7f9dcbf45640 1 --2- 172.21.15.5:0/3142517390 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9dc4102d50 0x7f9dc4103130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:15.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.043+0000 7f9dcbf45640 1 -- 172.21.15.5:0/3142517390 >> 172.21.15.5:0/3142517390 conn(0x7f9dc40fc7d0 msgr2=0x7f9dc40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:59:15.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.043+0000 7f9dcbf45640 1 -- 172.21.15.5:0/3142517390 shutdown_connections 2024-09-12T02:59:15.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.043+0000 7f9dcbf45640 1 -- 172.21.15.5:0/3142517390 wait complete. 2024-09-12T02:59:15.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.044+0000 7f9dcbf45640 1 Processor -- start 2024-09-12T02:59:15.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.044+0000 7f9dcbf45640 1 -- start start 2024-09-12T02:59:15.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.044+0000 7f9dcbf45640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9dc4102d50 0x7f9dc4198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:15.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.044+0000 7f9dcbf45640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9dc4103670 0x7f9dc41989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:15.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.044+0000 7f9dcbf45640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9dc410c180 0x7f9dc419cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:15.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.045+0000 7f9dcbf45640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9dc4111f40 con 0x7f9dc4103670 2024-09-12T02:59:15.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.045+0000 7f9dcbf45640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f9dc4111dc0 con 0x7f9dc410c180 2024-09-12T02:59:15.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.045+0000 7f9dcbf45640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f9dc41120c0 con 0x7f9dc4102d50 2024-09-12T02:59:15.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.045+0000 7f9dc94e0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9dc4103670 0x7f9dc41989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:15.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.045+0000 7f9dc94e0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9dc4103670 0x7f9dc41989a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40748/0 (socket says 172.21.15.5:40748) 2024-09-12T02:59:15.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.045+0000 7f9dc94e0640 1 -- 172.21.15.5:0/946638625 learned_addr learned my addr 172.21.15.5:0/946638625 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:59:15.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.045+0000 7f9dc9ce1640 1 --2- 172.21.15.5:0/946638625 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9dc4102d50 0x7f9dc4198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:15.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.045+0000 7f9dc94e0640 1 -- 172.21.15.5:0/946638625 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9dc4102d50 msgr2=0x7f9dc4198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:15.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.045+0000 7f9dc94e0640 1 --2- 172.21.15.5:0/946638625 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9dc4102d50 0x7f9dc4198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:15.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.045+0000 7f9dc94e0640 1 -- 172.21.15.5:0/946638625 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9dc410c180 msgr2=0x7f9dc419cd30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T02:59:15.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.045+0000 7f9dc94e0640 1 --2- 172.21.15.5:0/946638625 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9dc410c180 0x7f9dc419cd30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:15.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.045+0000 7f9dc94e0640 1 -- 172.21.15.5:0/946638625 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dc419d410 con 0x7f9dc4103670 2024-09-12T02:59:15.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.045+0000 7f9dc94e0640 1 --2- 172.21.15.5:0/946638625 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9dc4103670 0x7f9dc41989a0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f9db400d740 tx=0x7f9db400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:59:15.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.045+0000 7f9dbaffd640 1 -- 172.21.15.5:0/946638625 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9db4017070 con 0x7f9dc4103670 2024-09-12T02:59:15.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.045+0000 7f9dcbf45640 1 -- 172.21.15.5:0/946638625 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9dc419d700 con 0x7f9dc4103670 2024-09-12T02:59:15.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.045+0000 7f9dbaffd640 1 -- 172.21.15.5:0/946638625 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9db400de50 con 0x7f9dc4103670 2024-09-12T02:59:15.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.045+0000 7f9dbaffd640 1 -- 172.21.15.5:0/946638625 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9db400f620 con 0x7f9dc4103670 2024-09-12T02:59:15.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.046+0000 7f9dcbf45640 1 -- 172.21.15.5:0/946638625 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9dc41008e0 con 0x7f9dc4103670 2024-09-12T02:59:15.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.049+0000 7f9dbaffd640 1 -- 172.21.15.5:0/946638625 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 19) ==== 137883+0+0 (secure 0 0 0) 0x7f9db4009c30 con 0x7f9dc4103670 2024-09-12T02:59:15.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.049+0000 7f9dce31e740 1 Processor -- start 2024-09-12T02:59:15.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.049+0000 7f9dce31e740 1 -- start start 2024-09-12T02:59:15.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.050+0000 7f9dce31e740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559bd5e30870 0x559bd5e30c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:15.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.050+0000 7f9dce31e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559bd5e31190 0x559bd5e3b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:15.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.050+0000 7f9dce31e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559bd5e3ba50 0x559bd5e3de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:15.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.050+0000 7f9dce31e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559bd5d8d620 con 0x559bd5e31190 2024-09-12T02:59:15.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.050+0000 7f9dce31e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x559bd5d34e50 con 0x559bd5e3ba50 2024-09-12T02:59:15.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.050+0000 7f9dce31e740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x559bd5d895e0 con 0x559bd5e30870 2024-09-12T02:59:15.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.050+0000 7f9dc9ce1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559bd5e31190 0x559bd5e3b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:15.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.050+0000 7f9dc9ce1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559bd5e31190 0x559bd5e3b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40752/0 (socket says 172.21.15.5:40752) 2024-09-12T02:59:15.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.050+0000 7f9dc9ce1640 1 -- 172.21.15.5:0/3894766359 learned_addr learned my addr 172.21.15.5:0/3894766359 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:59:15.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.050+0000 7f9dca4e2640 1 --2- 172.21.15.5:0/3894766359 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559bd5e30870 0x559bd5e30c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:15.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.050+0000 7f9dc94e0640 1 --2- 172.21.15.5:0/3894766359 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559bd5e3ba50 0x559bd5e3de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:15.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.051+0000 7f9dca4e2640 1 -- 172.21.15.5:0/3894766359 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559bd5e3ba50 msgr2=0x559bd5e3de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:15.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.051+0000 7f9dca4e2640 1 --2- 172.21.15.5:0/3894766359 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559bd5e3ba50 0x559bd5e3de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:15.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.051+0000 7f9dca4e2640 1 -- 172.21.15.5:0/3894766359 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559bd5e31190 msgr2=0x559bd5e3b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:15.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.051+0000 7f9dca4e2640 1 --2- 172.21.15.5:0/3894766359 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559bd5e31190 0x559bd5e3b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:15.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.051+0000 7f9dca4e2640 1 -- 172.21.15.5:0/3894766359 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559bd5d25d40 con 0x559bd5e30870 2024-09-12T02:59:15.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.051+0000 7f9dbaffd640 1 --2- 172.21.15.5:0/946638625 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f9db00a2c90 0x7f9db00a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:15.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.051+0000 7f9dc9ce1640 1 --2- 172.21.15.5:0/3894766359 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559bd5e31190 0x559bd5e3b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:59:15.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.051+0000 7f9dbaffd640 1 -- 172.21.15.5:0/946638625 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f9db40cee00 con 0x7f9dc4103670 2024-09-12T02:59:15.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.051+0000 7f9dc9ce1640 1 --2- 172.21.15.5:0/946638625 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f9db00a2c90 0x7f9db00a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:15.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.051+0000 7f9dca4e2640 1 --2- 172.21.15.5:0/3894766359 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559bd5e30870 0x559bd5e30c50 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc005cc0 tx=0x7f9dbc005fa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=476ce7f96033596 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:59:15.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.052+0000 7f9daffff640 1 -- 172.21.15.5:0/3894766359 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dbc034040 con 0x559bd5e30870 2024-09-12T02:59:15.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.052+0000 7f9daffff640 1 -- 172.21.15.5:0/3894766359 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9dbc02fb40 con 0x559bd5e30870 2024-09-12T02:59:15.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.052+0000 7f9daffff640 1 -- 172.21.15.5:0/3894766359 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dbc0042d0 con 0x559bd5e30870 2024-09-12T02:59:15.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.053+0000 7f9dce31e740 1 -- 172.21.15.5:0/3894766359 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559bd5e30870 msgr2=0x559bd5e30c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:15.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.053+0000 7f9dce31e740 1 --2- 172.21.15.5:0/3894766359 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559bd5e30870 0x559bd5e30c50 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc005cc0 tx=0x7f9dbc005fa0 comp rx=0 tx=0).stop 2024-09-12T02:59:15.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.053+0000 7f9dc9ce1640 1 --2- 172.21.15.5:0/946638625 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f9db00a2c90 0x7f9db00a5150 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x559bd5e2dc60 tx=0x7f9dc000d5a0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:59:15.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.053+0000 7f9dce31e740 1 -- 172.21.15.5:0/3894766359 shutdown_connections 2024-09-12T02:59:15.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.053+0000 7f9dce31e740 1 --2- 172.21.15.5:0/3894766359 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559bd5e31190 0x559bd5e3b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:15.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.053+0000 7f9dce31e740 1 --2- 172.21.15.5:0/3894766359 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559bd5e3ba50 0x559bd5e3de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:15.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.053+0000 7f9dce31e740 1 --2- 172.21.15.5:0/3894766359 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559bd5e30870 0x559bd5e30c50 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:15.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.053+0000 7f9dce31e740 1 -- 172.21.15.5:0/3894766359 >> 172.21.15.5:0/3894766359 conn(0x559bd5e2ce00 msgr2=0x559bd5e2d1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:59:15.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.053+0000 7f9dce31e740 1 -- 172.21.15.5:0/3894766359 shutdown_connections 2024-09-12T02:59:15.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.053+0000 7f9dce31e740 1 -- 172.21.15.5:0/3894766359 wait complete. 2024-09-12T02:59:15.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.054+0000 7f9dce31e740 1 -- start start 2024-09-12T02:59:15.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.054+0000 7f9dce31e740 1 Processor -- start 2024-09-12T02:59:15.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.054+0000 7f9dce31e740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559bd5e30870 0x559bd5e388f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:15.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.054+0000 7f9dce31e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559bd5e31190 0x559bd5e38e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:15.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.054+0000 7f9dce31e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559bd5e3ba50 0x559bd5e39370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:15.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.054+0000 7f9dce31e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559bd5e3e670 con 0x559bd5e31190 2024-09-12T02:59:15.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.054+0000 7f9dce31e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x559bd5d15f10 con 0x559bd5e3ba50 2024-09-12T02:59:15.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.054+0000 7f9dce31e740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x559bd5e3e7f0 con 0x559bd5e30870 2024-09-12T02:59:15.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.054+0000 7f9dce31e740 10 client.0 _ll_register_callbacks cb 0x559bd5d39db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:59:15.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.054+0000 7f9dc94e0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559bd5e31190 0x559bd5e38e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:15.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.054+0000 7f9dc94e0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559bd5e31190 0x559bd5e38e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40760/0 (socket says 172.21.15.5:40760) 2024-09-12T02:59:15.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.055+0000 7f9dc94e0640 1 -- 172.21.15.5:0/2266370086 learned_addr learned my addr 172.21.15.5:0/2266370086 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:59:15.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.055+0000 7f9dc9ce1640 1 --2- 172.21.15.5:0/2266370086 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559bd5e30870 0x559bd5e388f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:15.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.055+0000 7f9dca4e2640 1 --2- 172.21.15.5:0/2266370086 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559bd5e3ba50 0x559bd5e39370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:15.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.055+0000 7f9dc94e0640 1 -- 172.21.15.5:0/2266370086 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559bd5e30870 msgr2=0x559bd5e388f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:15.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.055+0000 7f9dc94e0640 1 --2- 172.21.15.5:0/2266370086 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559bd5e30870 0x559bd5e388f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:15.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.055+0000 7f9dc94e0640 1 -- 172.21.15.5:0/2266370086 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559bd5e3ba50 msgr2=0x559bd5e39370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:15.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.055+0000 7f9dc94e0640 1 --2- 172.21.15.5:0/2266370086 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559bd5e3ba50 0x559bd5e39370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:15.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.055+0000 7f9dc94e0640 1 -- 172.21.15.5:0/2266370086 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559bd5a9c580 con 0x559bd5e31190 2024-09-12T02:59:15.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.055+0000 7f9dc9ce1640 1 --2- 172.21.15.5:0/2266370086 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559bd5e30870 0x559bd5e388f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:59:15.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.055+0000 7f9dc94e0640 1 --2- 172.21.15.5:0/2266370086 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559bd5e31190 0x559bd5e38e30 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f9db4000c00 tx=0x7f9db409b3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efe7c20b714dc20b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:59:15.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.055+0000 7f9dad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T02:59:15.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.055+0000 7f9dad7fa640 1 -- 172.21.15.5:0/2266370086 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9db4020070 con 0x559bd5e31190 2024-09-12T02:59:15.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.055+0000 7f9dce31e740 1 -- 172.21.15.5:0/2266370086 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9dbc02fb40 con 0x559bd5e31190 2024-09-12T02:59:15.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.055+0000 7f9dad7fa640 1 -- 172.21.15.5:0/2266370086 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f9db40d2030 con 0x559bd5e31190 2024-09-12T02:59:15.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.056+0000 7f9dad7fa640 1 -- 172.21.15.5:0/2266370086 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9db409b510 con 0x559bd5e31190 2024-09-12T02:59:15.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.056+0000 7f9dce31e740 10 client.15636 fetch_fsmap learned FSMap version 26 2024-09-12T02:59:15.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.056+0000 7f9dce31e740 1 -- 172.21.15.5:0/2266370086 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559bd5b66960 con 0x559bd5e31190 2024-09-12T02:59:15.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.056+0000 7f9dad7fa640 1 -- 172.21.15.5:0/2266370086 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9dbc02fb40 con 0x559bd5e31190 2024-09-12T02:59:15.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.056+0000 7f9dad7fa640 1 -- 172.21.15.5:0/2266370086 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f9db409a960 con 0x559bd5e31190 2024-09-12T02:59:15.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.056+0000 7f9dce31e740 10 client.15636 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T02:59:15.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.056+0000 7f9dce31e740 10 client.15636 resolve_mds: validated gid 24502 aka daemon mds.i 2024-09-12T02:59:15.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.056+0000 7f9dce31e740 20 client.15636 populate_metadata read hostname 'smithi005' 2024-09-12T02:59:15.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.056+0000 7f9dce31e740 1 --2- 172.21.15.5:0/2266370086 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x559bd5e4b450 0x559bd5e4d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:15.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.056+0000 7f9dce31e740 4 client.15636 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:59:15.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.056+0000 7f9dce31e740 1 -- 172.21.15.5:0/2266370086 --> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559bd5d9ab90 con 0x559bd5e4b450 2024-09-12T02:59:15.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.057+0000 7f9dc9ce1640 1 --2- 172.21.15.5:0/2266370086 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x559bd5e4b450 0x559bd5e4d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:15.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.057+0000 7f9dad7fa640 10 client.15636 ms_handle_connect on v2:172.21.15.148:6838/2317600000 2024-09-12T02:59:15.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.057+0000 7f9dc9ce1640 1 --2- 172.21.15.5:0/2266370086 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x559bd5e4b450 0x559bd5e4d8c0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b2ab90b170b71e1 server_cookie=e067d96f024c5f68 in_seq=0 out_seq=0 2024-09-12T02:59:15.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.058+0000 7f9dad7fa640 1 -- 172.21.15.5:0/2266370086 <== mds.0 v2:172.21.15.148:6838/2317600000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559bd5d9ab90 con 0x559bd5e4b450 2024-09-12T02:59:15.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.058+0000 7f9dad7fa640 10 client.15636 handle_command_reply: tid=0 2024-09-12T02:59:15.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.058+0000 7f9dce31e740 1 client.15636 shutdown 2024-09-12T02:59:15.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.058+0000 7f9dce31e740 1 -- 172.21.15.5:0/2266370086 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559bd5e31190 msgr2=0x559bd5e38e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:15.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.058+0000 7f9dce31e740 1 --2- 172.21.15.5:0/2266370086 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559bd5e31190 0x559bd5e38e30 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f9db4000c00 tx=0x7f9db409b3d0 comp rx=0 tx=0).stop 2024-09-12T02:59:15.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.058+0000 7f9dce31e740 1 -- 172.21.15.5:0/2266370086 shutdown_connections 2024-09-12T02:59:15.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.058+0000 7f9dce31e740 1 --2- 172.21.15.5:0/2266370086 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x559bd5e4b450 0x559bd5e4d8c0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:15.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.058+0000 7f9dce31e740 1 --2- 172.21.15.5:0/2266370086 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559bd5e31190 0x559bd5e38e30 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:15.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.058+0000 7f9dce31e740 1 --2- 172.21.15.5:0/2266370086 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559bd5e3ba50 0x559bd5e39370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:15.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.058+0000 7f9dad7fa640 0 client.15636 ms_handle_reset on v2:172.21.15.148:6838/2317600000 2024-09-12T02:59:15.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.058+0000 7f9dce31e740 1 --2- 172.21.15.5:0/2266370086 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559bd5e30870 0x559bd5e388f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:15.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.058+0000 7f9dce31e740 1 -- 172.21.15.5:0/2266370086 >> 172.21.15.5:0/2266370086 conn(0x559bd5e2ce00 msgr2=0x559bd5e32b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:59:15.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.059+0000 7f9dce31e740 1 -- 172.21.15.5:0/2266370086 shutdown_connections 2024-09-12T02:59:15.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.059+0000 7f9dce31e740 1 -- 172.21.15.5:0/2266370086 wait complete. 2024-09-12T02:59:15.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.059+0000 7f9dce31e740 20 client.15636 trim_cache size 0 max 16384 2024-09-12T02:59:15.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.076+0000 7f9dce31e740 1 Processor -- start 2024-09-12T02:59:15.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.076+0000 7f9dce31e740 1 -- start start 2024-09-12T02:59:15.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.077+0000 7f9dce31e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559bd5e30870 0x559bd5e332d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:15.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.077+0000 7f9dce31e740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559bd5e31190 0x559bd5e23640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:15.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.077+0000 7f9dce31e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559bd5e3ba50 0x559bd5e53190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:15.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.077+0000 7f9dce31e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9dc41ab2b0 con 0x559bd5e3ba50 2024-09-12T02:59:15.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.077+0000 7f9dce31e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f9dc41acdf0 con 0x559bd5e30870 2024-09-12T02:59:15.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.077+0000 7f9dce31e740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f9dc41ab980 con 0x559bd5e31190 2024-09-12T02:59:15.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.077+0000 7f9dc9ce1640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559bd5e30870 0x559bd5e332d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:15.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.077+0000 7f9dc94e0640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559bd5e31190 0x559bd5e23640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:15.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.077+0000 7f9dc94e0640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559bd5e31190 0x559bd5e23640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.5:34034/0 (socket says 172.21.15.5:34034) 2024-09-12T02:59:15.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.077+0000 7f9dc9ce1640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559bd5e30870 0x559bd5e332d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.5:49476/0 (socket says 172.21.15.5:49476) 2024-09-12T02:59:15.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.077+0000 7f9dc94e0640 1 -- 172.21.15.5:0/1002317922 learned_addr learned my addr 172.21.15.5:0/1002317922 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:59:15.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.077+0000 7f9dc94e0640 1 -- 172.21.15.5:0/1002317922 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559bd5e30870 msgr2=0x559bd5e332d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:15.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.077+0000 7f9dc94e0640 1 --2- 172.21.15.5:0/1002317922 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559bd5e30870 0x559bd5e332d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:15.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.077+0000 7f9dc94e0640 1 -- 172.21.15.5:0/1002317922 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559bd5e3ba50 msgr2=0x559bd5e53190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:15.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.078+0000 7f9dca4e2640 1 --2- 172.21.15.5:0/1002317922 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559bd5e3ba50 0x559bd5e53190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:15.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.078+0000 7f9dc94e0640 1 --2- 172.21.15.5:0/1002317922 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559bd5e3ba50 0x559bd5e53190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:15.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.078+0000 7f9dc94e0640 1 -- 172.21.15.5:0/1002317922 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559bd5e2cc30 con 0x559bd5e31190 2024-09-12T02:59:15.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.078+0000 7f9dca4e2640 1 --2- 172.21.15.5:0/1002317922 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559bd5e3ba50 0x559bd5e53190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:59:15.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.078+0000 7f9dc94e0640 1 --2- 172.21.15.5:0/1002317922 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559bd5e31190 0x559bd5e23640 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9db409b210 tx=0x7f9db4005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e53fa29fd4a05de4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:59:15.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.078+0000 7f9daf7fe640 1 -- 172.21.15.5:0/1002317922 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9db4020070 con 0x559bd5e31190 2024-09-12T02:59:15.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.078+0000 7f9daf7fe640 1 -- 172.21.15.5:0/1002317922 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9db409f8c0 con 0x559bd5e31190 2024-09-12T02:59:15.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.078+0000 7f9daf7fe640 1 -- 172.21.15.5:0/1002317922 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9db409a6c0 con 0x559bd5e31190 2024-09-12T02:59:15.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.079+0000 7f9dce31e740 1 -- 172.21.15.5:0/1002317922 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559bd5e31190 msgr2=0x559bd5e23640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:15.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.079+0000 7f9dce31e740 1 --2- 172.21.15.5:0/1002317922 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559bd5e31190 0x559bd5e23640 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9db409b210 tx=0x7f9db4005880 comp rx=0 tx=0).stop 2024-09-12T02:59:15.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.079+0000 7f9dce31e740 1 -- 172.21.15.5:0/1002317922 shutdown_connections 2024-09-12T02:59:15.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.079+0000 7f9dce31e740 1 --2- 172.21.15.5:0/1002317922 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559bd5e3ba50 0x559bd5e53190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:15.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.079+0000 7f9dce31e740 1 --2- 172.21.15.5:0/1002317922 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559bd5e31190 0x559bd5e23640 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:15.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.079+0000 7f9dce31e740 1 --2- 172.21.15.5:0/1002317922 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559bd5e30870 0x559bd5e332d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:15.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.079+0000 7f9dce31e740 1 -- 172.21.15.5:0/1002317922 >> 172.21.15.5:0/1002317922 conn(0x559bd5e2ce00 msgr2=0x559bd5e22f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:59:15.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.079+0000 7f9dce31e740 1 -- 172.21.15.5:0/1002317922 shutdown_connections 2024-09-12T02:59:15.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.079+0000 7f9dce31e740 1 -- 172.21.15.5:0/1002317922 wait complete. 2024-09-12T02:59:15.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.080+0000 7f9dce31e740 1 -- start start 2024-09-12T02:59:15.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.080+0000 7f9dce31e740 1 Processor -- start 2024-09-12T02:59:15.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.080+0000 7f9dce31e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559bd5e30870 0x559bd5e40400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:15.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.080+0000 7f9dce31e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559bd5e31190 0x559bd5e40940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:15.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.080+0000 7f9dce31e740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559bd5e3ba50 0x559bd5eae120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:15.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.080+0000 7f9dce31e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9dc41ac720 con 0x559bd5e31190 2024-09-12T02:59:15.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.080+0000 7f9dce31e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f9dc41ac050 con 0x559bd5e30870 2024-09-12T02:59:15.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.080+0000 7f9dce31e740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f9dc41a9290 con 0x559bd5e3ba50 2024-09-12T02:59:15.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.080+0000 7f9dce31e740 10 client.0 _ll_register_callbacks cb 0x7f9db4097820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:59:15.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.081+0000 7f9dc94e0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559bd5e31190 0x559bd5e40940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:15.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.081+0000 7f9dc94e0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559bd5e31190 0x559bd5e40940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40768/0 (socket says 172.21.15.5:40768) 2024-09-12T02:59:15.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.081+0000 7f9dc94e0640 1 -- 172.21.15.5:0/4044689977 learned_addr learned my addr 172.21.15.5:0/4044689977 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:59:15.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.081+0000 7f9dc9ce1640 1 --2- 172.21.15.5:0/4044689977 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559bd5e30870 0x559bd5e40400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:15.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.081+0000 7f9dc94e0640 1 -- 172.21.15.5:0/4044689977 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559bd5e3ba50 msgr2=0x559bd5eae120 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T02:59:15.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.081+0000 7f9dc94e0640 1 --2- 172.21.15.5:0/4044689977 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559bd5e3ba50 0x559bd5eae120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:15.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.081+0000 7f9dc94e0640 1 -- 172.21.15.5:0/4044689977 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559bd5e30870 msgr2=0x559bd5e40400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:15.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.081+0000 7f9dc94e0640 1 --2- 172.21.15.5:0/4044689977 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559bd5e30870 0x559bd5e40400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:15.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.081+0000 7f9dc94e0640 1 -- 172.21.15.5:0/4044689977 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559bd5eae930 con 0x559bd5e31190 2024-09-12T02:59:15.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.081+0000 7f9dc94e0640 1 --2- 172.21.15.5:0/4044689977 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559bd5e31190 0x559bd5e40940 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f9db40ccd40 tx=0x7f9db40ccc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5847c54bbeb64d1f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:59:15.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.081+0000 7f9dad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T02:59:15.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.082+0000 7f9dad7fa640 1 -- 172.21.15.5:0/4044689977 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9db4020070 con 0x559bd5e31190 2024-09-12T02:59:15.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.082+0000 7f9dad7fa640 1 -- 172.21.15.5:0/4044689977 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f9db4097260 con 0x559bd5e31190 2024-09-12T02:59:15.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.082+0000 7f9dce31e740 1 -- 172.21.15.5:0/4044689977 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9db409f8c0 con 0x559bd5e31190 2024-09-12T02:59:15.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.082+0000 7f9dad7fa640 1 -- 172.21.15.5:0/4044689977 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9db40b4040 con 0x559bd5e31190 2024-09-12T02:59:15.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.082+0000 7f9dce31e740 10 client.15639 fetch_fsmap learned FSMap version 26 2024-09-12T02:59:15.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.082+0000 7f9dce31e740 1 -- 172.21.15.5:0/4044689977 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x559bd5eb6c30 con 0x559bd5e31190 2024-09-12T02:59:15.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.082+0000 7f9dad7fa640 1 -- 172.21.15.5:0/4044689977 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9db40b9030 con 0x559bd5e31190 2024-09-12T02:59:15.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.082+0000 7f9dad7fa640 1 -- 172.21.15.5:0/4044689977 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) ==== 8362+0+0 (secure 0 0 0) 0x7f9db40a0a50 con 0x559bd5e31190 2024-09-12T02:59:15.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.082+0000 7f9dce31e740 10 client.15639 fetch_fsmap finished waiting for FSMap version 26 2024-09-12T02:59:15.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.082+0000 7f9dce31e740 10 client.15639 resolve_mds: validated gid 24502 aka daemon mds.i 2024-09-12T02:59:15.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.082+0000 7f9dce31e740 20 client.15639 populate_metadata read hostname 'smithi005' 2024-09-12T02:59:15.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.082+0000 7f9dce31e740 1 --2- 172.21.15.5:0/4044689977 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x559bd5eb7140 0x559bd5eb9560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:15.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.083+0000 7f9dce31e740 4 client.15639 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "26c61f34", "release": true, "await": true}] 2024-09-12T02:59:15.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.083+0000 7f9dce31e740 1 -- 172.21.15.5:0/4044689977 --> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] -- command(tid 0: {"prefix": "quiesce db", "set_id": "26c61f34", "release": true, "await": true}) -- 0x559bd5d950a0 con 0x559bd5eb7140 2024-09-12T02:59:15.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.083+0000 7f9dc9ce1640 1 --2- 172.21.15.5:0/4044689977 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x559bd5eb7140 0x559bd5eb9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:15.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.083+0000 7f9dc9ce1640 1 --2- 172.21.15.5:0/4044689977 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x559bd5eb7140 0x559bd5eb9560 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e8d44853ee79716 server_cookie=371335f8402c7f20 in_seq=0 out_seq=0 2024-09-12T02:59:15.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.083+0000 7f9dad7fa640 10 client.15639 ms_handle_connect on v2:172.21.15.148:6838/2317600000 2024-09-12T02:59:15.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.085+0000 7f9dad7fa640 1 -- 172.21.15.5:0/4044689977 <== mds.0 v2:172.21.15.148:6838/2317600000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x559bd5d950a0 con 0x559bd5eb7140 2024-09-12T02:59:15.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.085+0000 7f9dad7fa640 10 client.15639 handle_command_reply: tid=0 2024-09-12T02:59:15.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.085+0000 7f9dce31e740 1 client.15639 shutdown 2024-09-12T02:59:15.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.085+0000 7f9dce31e740 1 -- 172.21.15.5:0/4044689977 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559bd5e31190 msgr2=0x559bd5e40940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:15.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.085+0000 7f9dce31e740 1 --2- 172.21.15.5:0/4044689977 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559bd5e31190 0x559bd5e40940 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f9db40ccd40 tx=0x7f9db40ccc20 comp rx=0 tx=0).stop 2024-09-12T02:59:15.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.086+0000 7f9dce31e740 1 -- 172.21.15.5:0/4044689977 shutdown_connections 2024-09-12T02:59:15.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.086+0000 7f9dce31e740 1 --2- 172.21.15.5:0/4044689977 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x559bd5eb7140 0x559bd5eb9560 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:15.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.086+0000 7f9dce31e740 1 --2- 172.21.15.5:0/4044689977 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559bd5e31190 0x559bd5e40940 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:15.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.086+0000 7f9dce31e740 1 --2- 172.21.15.5:0/4044689977 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559bd5e3ba50 0x559bd5eae120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:15.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.086+0000 7f9dce31e740 1 --2- 172.21.15.5:0/4044689977 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559bd5e30870 0x559bd5e40400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:15.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.086+0000 7f9dce31e740 1 -- 172.21.15.5:0/4044689977 >> 172.21.15.5:0/4044689977 conn(0x559bd5e2ce00 msgr2=0x559bd5e23750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:59:15.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.086+0000 7f9dad7fa640 0 client.15639 ms_handle_reset on v2:172.21.15.148:6838/2317600000 2024-09-12T02:59:15.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.086+0000 7f9dce31e740 1 -- 172.21.15.5:0/4044689977 shutdown_connections 2024-09-12T02:59:15.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.086+0000 7f9dce31e740 1 -- 172.21.15.5:0/4044689977 wait complete. 2024-09-12T02:59:15.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.086+0000 7f9dce31e740 20 client.15639 trim_cache size 0 max 16384 2024-09-12T02:59:15.089 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-12T02:59:15.089 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 26, 2024-09-12T02:59:15.089 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24502, 2024-09-12T02:59:15.089 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 4, 2024-09-12T02:59:15.089 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-09-12T02:59:15.089 INFO:teuthology.orchestra.run.smithi005.stdout: "26c61f34": { 2024-09-12T02:59:15.089 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 4, 2024-09-12T02:59:15.089 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-09-12T02:59:15.090 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-12T02:59:15.090 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-09-12T02:59:15.090 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-12T02:59:15.090 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T02:59:15.090 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 90.0, 2024-09-12T02:59:15.090 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 144.5, 2024-09-12T02:59:15.090 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-09-12T02:59:15.090 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-09-12T02:59:15.090 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-09-12T02:59:15.091 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-12T02:59:15.091 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-09-12T02:59:15.091 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-12T02:59:15.091 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T02:59:15.091 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T02:59:15.091 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T02:59:15.091 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T02:59:15.091 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T02:59:15.091 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-12T02:59:15.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.087+0000 7f9dcbf45640 1 -- 172.21.15.5:0/946638625 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f9db00a2c90 msgr2=0x7f9db00a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:15.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.087+0000 7f9dcbf45640 1 --2- 172.21.15.5:0/946638625 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f9db00a2c90 0x7f9db00a5150 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x559bd5e2dc60 tx=0x7f9dc000d5a0 comp rx=0 tx=0).stop 2024-09-12T02:59:15.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.087+0000 7f9dcbf45640 1 -- 172.21.15.5:0/946638625 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9dc4103670 msgr2=0x7f9dc41989a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:15.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.087+0000 7f9dcbf45640 1 --2- 172.21.15.5:0/946638625 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9dc4103670 0x7f9dc41989a0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f9db400d740 tx=0x7f9db400dc00 comp rx=0 tx=0).stop 2024-09-12T02:59:15.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.088+0000 7f9dcbf45640 1 -- 172.21.15.5:0/946638625 shutdown_connections 2024-09-12T02:59:15.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.088+0000 7f9dcbf45640 1 --2- 172.21.15.5:0/946638625 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f9db00a2c90 0x7f9db00a5150 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:15.092 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.088+0000 7f9dcbf45640 1 --2- 172.21.15.5:0/946638625 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9dc4103670 0x7f9dc41989a0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:15.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.088+0000 7f9dcbf45640 1 --2- 172.21.15.5:0/946638625 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9dc410c180 0x7f9dc419cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:15.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.088+0000 7f9dcbf45640 1 --2- 172.21.15.5:0/946638625 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9dc4102d50 0x7f9dc4198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:15.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.088+0000 7f9dcbf45640 1 -- 172.21.15.5:0/946638625 >> 172.21.15.5:0/946638625 conn(0x7f9dc40fc7d0 msgr2=0x7f9dc40fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:59:15.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.088+0000 7f9dcbf45640 1 -- 172.21.15.5:0/946638625 shutdown_connections 2024-09-12T02:59:15.093 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:15.088+0000 7f9dcbf45640 1 -- 172.21.15.5:0/946638625 wait complete. 2024-09-12T02:59:15.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:59:14 smithi146 ceph-mon[35011]: pgmap v365: 97 pgs: 97 active+clean; 663 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 85 B/s wr, 11 op/s 2024-09-12T02:59:15.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:59:14 smithi005 ceph-mon[31197]: pgmap v365: 97 pgs: 97 active+clean; 663 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 85 B/s wr, 11 op/s 2024-09-12T02:59:15.415 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:59:14 smithi148 ceph-mon[35090]: pgmap v365: 97 pgs: 97 active+clean; 663 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 85 B/s wr, 11 op/s 2024-09-12T02:59:15.918 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '26c61f34', total seconds elapsed: 27.9 2024-09-12T02:59:17.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:59:16 smithi146 ceph-mon[35011]: pgmap v366: 97 pgs: 97 active+clean; 663 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-12T02:59:17.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:59:16 smithi005 ceph-mon[31197]: pgmap v366: 97 pgs: 97 active+clean; 663 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-12T02:59:17.415 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:59:16 smithi148 ceph-mon[35090]: pgmap v366: 97 pgs: 97 active+clean; 663 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-12T02:59:19.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:59:18 smithi146 ceph-mon[35011]: pgmap v367: 97 pgs: 97 active+clean; 663 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s 2024-09-12T02:59:19.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:59:18 smithi005 ceph-mon[31197]: pgmap v367: 97 pgs: 97 active+clean; 663 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s 2024-09-12T02:59:19.415 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:59:18 smithi148 ceph-mon[35090]: pgmap v367: 97 pgs: 97 active+clean; 663 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 9 op/s 2024-09-12T02:59:21.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:59:20 smithi146 ceph-mon[35011]: pgmap v368: 97 pgs: 97 active+clean; 663 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 10 op/s 2024-09-12T02:59:21.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:59:20 smithi005 ceph-mon[31197]: pgmap v368: 97 pgs: 97 active+clean; 663 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 10 op/s 2024-09-12T02:59:21.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:59:20 smithi148 ceph-mon[35090]: pgmap v368: 97 pgs: 97 active+clean; 663 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 10 op/s 2024-09-12T02:59:23.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:59:22 smithi146 ceph-mon[35011]: pgmap v369: 97 pgs: 97 active+clean; 663 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T02:59:23.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:59:22 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:59:23.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:59:22 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:59:23.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:59:22 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:59:23.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:59:22 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:59:23.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:59:22 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:59:23.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:59:22 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:59:23.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:59:22 smithi005 ceph-mon[31197]: pgmap v369: 97 pgs: 97 active+clean; 663 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T02:59:23.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:59:22 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:59:23.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:59:22 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:59:23.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:59:22 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:59:23.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:59:22 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:59:23.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:59:22 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:59:23.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:59:22 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:59:23.415 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:59:22 smithi148 ceph-mon[35090]: pgmap v369: 97 pgs: 97 active+clean; 663 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T02:59:23.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:59:22 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T02:59:23.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:59:22 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:59:23.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:59:22 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:59:23.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:59:22 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T02:59:23.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:59:22 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T02:59:23.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:59:22 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T02:59:24.179 INFO:tasks.workunit.client.0.smithi005.stderr:Updating files: 93% (12120/12903) Updating files: 94% (12129/12903) Updating files: 95% (12258/12903) Updating files: 96% (12387/12903) Updating files: 97% (12516/12903) Updating files: 98% (12645/12903) Updating files: 99% (12774/12903) Updating files: 100% (12903/12903) Updating files: 100% (12903/12903), done. 2024-09-12T02:59:24.642 INFO:tasks.workunit.client.0.smithi005.stderr:Note: switching to '086c737fa75c22d521a3932c59b31c5e1d1c9ef4'. 2024-09-12T02:59:24.642 INFO:tasks.workunit.client.0.smithi005.stderr: 2024-09-12T02:59:24.642 INFO:tasks.workunit.client.0.smithi005.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-12T02:59:24.642 INFO:tasks.workunit.client.0.smithi005.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-12T02:59:24.642 INFO:tasks.workunit.client.0.smithi005.stderr:state without impacting any branches by switching back to a branch. 2024-09-12T02:59:24.642 INFO:tasks.workunit.client.0.smithi005.stderr: 2024-09-12T02:59:24.642 INFO:tasks.workunit.client.0.smithi005.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-12T02:59:24.643 INFO:tasks.workunit.client.0.smithi005.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-12T02:59:24.643 INFO:tasks.workunit.client.0.smithi005.stderr: 2024-09-12T02:59:24.643 INFO:tasks.workunit.client.0.smithi005.stderr: git switch -c 2024-09-12T02:59:24.643 INFO:tasks.workunit.client.0.smithi005.stderr: 2024-09-12T02:59:24.643 INFO:tasks.workunit.client.0.smithi005.stderr:Or undo this operation with: 2024-09-12T02:59:24.643 INFO:tasks.workunit.client.0.smithi005.stderr: 2024-09-12T02:59:24.643 INFO:tasks.workunit.client.0.smithi005.stderr: git switch - 2024-09-12T02:59:24.643 INFO:tasks.workunit.client.0.smithi005.stderr: 2024-09-12T02:59:24.643 INFO:tasks.workunit.client.0.smithi005.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-12T02:59:24.643 INFO:tasks.workunit.client.0.smithi005.stderr: 2024-09-12T02:59:24.643 INFO:tasks.workunit.client.0.smithi005.stderr:HEAD is now at 086c737fa75 Merge pull request #59733 from zdover23/wip-doc-2024-09-11-backport-59635-to-squid 2024-09-12T02:59:24.648 DEBUG:teuthology.orchestra.run.smithi005:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-12T02:59:24.671 INFO:tasks.workunit.client.0.smithi005.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-12T02:59:24.674 INFO:tasks.workunit.client.0.smithi005.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-12T02:59:24.674 INFO:tasks.workunit.client.0.smithi005.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-12T02:59:25.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:59:24 smithi005 ceph-mon[31197]: pgmap v370: 97 pgs: 97 active+clean; 663 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-12T02:59:25.355 INFO:tasks.workunit.client.0.smithi005.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-12T02:59:25.415 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:59:24 smithi148 ceph-mon[35090]: pgmap v370: 97 pgs: 97 active+clean; 663 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-12T02:59:25.439 INFO:tasks.workunit.client.0.smithi005.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-12T02:59:25.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:59:24 smithi146 ceph-mon[35011]: pgmap v370: 97 pgs: 97 active+clean; 663 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-12T02:59:25.479 INFO:tasks.workunit.client.0.smithi005.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-12T02:59:25.480 INFO:tasks.workunit.client.0.smithi005.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-12T02:59:25.481 INFO:tasks.workunit.client.0.smithi005.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-12T02:59:25.520 INFO:tasks.workunit.client.0.smithi005.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-12T02:59:25.524 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-09-12T02:59:25.524 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-12T02:59:25.580 INFO:tasks.workunit:Running workunits matching suites/fsx.sh on client.0... 2024-09-12T02:59:25.582 INFO:tasks.workunit:Running workunit suites/fsx.sh... 2024-09-12T02:59:25.582 DEBUG:teuthology.orchestra.run.smithi005:workunit test suites/fsx.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=086c737fa75c22d521a3932c59b31c5e1d1c9ef4 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsx.sh 2024-09-12T02:59:25.643 INFO:tasks.workunit.client.0.smithi005.stderr:+ set -e 2024-09-12T02:59:25.644 INFO:tasks.workunit.client.0.smithi005.stderr:+ git clone https://git.ceph.com/xfstests-dev.git 2024-09-12T02:59:25.645 INFO:tasks.workunit.client.0.smithi005.stderr:Cloning into 'xfstests-dev'... 2024-09-12T02:59:27.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:59:26 smithi005 ceph-mon[31197]: pgmap v371: 97 pgs: 97 active+clean; 663 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-12T02:59:27.415 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:59:26 smithi148 ceph-mon[35090]: pgmap v371: 97 pgs: 97 active+clean; 663 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-12T02:59:27.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:59:26 smithi146 ceph-mon[35011]: pgmap v371: 97 pgs: 97 active+clean; 663 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-12T02:59:29.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:59:28 smithi005 ceph-mon[31197]: pgmap v372: 97 pgs: 97 active+clean; 663 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-12T02:59:29.415 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:59:28 smithi148 ceph-mon[35090]: pgmap v372: 97 pgs: 97 active+clean; 663 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-12T02:59:29.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:59:28 smithi146 ceph-mon[35011]: pgmap v372: 97 pgs: 97 active+clean; 663 KiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-12T02:59:31.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:59:30 smithi005 ceph-mon[31197]: pgmap v373: 97 pgs: 97 active+clean; 827 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-09-12T02:59:31.415 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:59:30 smithi148 ceph-mon[35090]: pgmap v373: 97 pgs: 97 active+clean; 827 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-09-12T02:59:31.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:59:30 smithi146 ceph-mon[35011]: pgmap v373: 97 pgs: 97 active+clean; 827 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-09-12T02:59:33.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:59:32 smithi005 ceph-mon[31197]: pgmap v374: 97 pgs: 97 active+clean; 2.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 151 KiB/s wr, 11 op/s 2024-09-12T02:59:33.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:59:32 smithi005 ceph-mon[31197]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-09-12T02:59:33.415 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:59:32 smithi148 ceph-mon[35090]: pgmap v374: 97 pgs: 97 active+clean; 2.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 151 KiB/s wr, 11 op/s 2024-09-12T02:59:33.415 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:59:32 smithi148 ceph-mon[35090]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-09-12T02:59:33.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:59:32 smithi146 ceph-mon[35011]: pgmap v374: 97 pgs: 97 active+clean; 2.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 151 KiB/s wr, 11 op/s 2024-09-12T02:59:33.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:59:32 smithi146 ceph-mon[35011]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-09-12T02:59:34.101 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:59:33 smithi005 ceph-mon[31197]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T02:59:34.415 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:59:33 smithi148 ceph-mon[35090]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T02:59:34.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:59:33 smithi146 ceph-mon[35011]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T02:59:35.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:59:34 smithi005 ceph-mon[31197]: pgmap v375: 97 pgs: 97 active+clean; 16 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 1.3 MiB/s wr, 17 op/s 2024-09-12T02:59:35.415 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:59:34 smithi148 ceph-mon[35090]: pgmap v375: 97 pgs: 97 active+clean; 16 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 1.3 MiB/s wr, 17 op/s 2024-09-12T02:59:35.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:59:34 smithi146 ceph-mon[35011]: pgmap v375: 97 pgs: 97 active+clean; 16 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 1.3 MiB/s wr, 17 op/s 2024-09-12T02:59:36.885 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T02:59:37.163 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:59:37.284 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:59:36 smithi005 ceph-mon[31197]: pgmap v376: 97 pgs: 97 active+clean; 16 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 1.3 MiB/s wr, 17 op/s 2024-09-12T02:59:37.415 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:59:36 smithi148 ceph-mon[35090]: pgmap v376: 97 pgs: 97 active+clean; 16 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 1.3 MiB/s wr, 17 op/s 2024-09-12T02:59:37.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:59:36 smithi146 ceph-mon[35011]: pgmap v376: 97 pgs: 97 active+clean; 16 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 1.3 MiB/s wr, 17 op/s 2024-09-12T02:59:37.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.611+0000 7fa5c38ba640 1 -- 172.21.15.5:0/1156848154 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5bc100b20 msgr2=0x7fa5bc100f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:37.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.611+0000 7fa5c38ba640 1 --2- 172.21.15.5:0/1156848154 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5bc100b20 0x7fa5bc100f00 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fa5b8009900 tx=0x7fa5b802a410 comp rx=0 tx=0).stop 2024-09-12T02:59:37.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.611+0000 7fa5c38ba640 1 -- 172.21.15.5:0/1156848154 shutdown_connections 2024-09-12T02:59:37.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.611+0000 7fa5c38ba640 1 --2- 172.21.15.5:0/1156848154 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa5bc10e500 0x7fa5bc110910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:37.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.611+0000 7fa5c38ba640 1 --2- 172.21.15.5:0/1156848154 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5bc101440 0x7fa5bc10df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:37.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.611+0000 7fa5c38ba640 1 --2- 172.21.15.5:0/1156848154 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5bc100b20 0x7fa5bc100f00 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:37.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.611+0000 7fa5c38ba640 1 -- 172.21.15.5:0/1156848154 >> 172.21.15.5:0/1156848154 conn(0x7fa5bc0fc7d0 msgr2=0x7fa5bc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:59:37.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.611+0000 7fa5c38ba640 1 -- 172.21.15.5:0/1156848154 shutdown_connections 2024-09-12T02:59:37.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.611+0000 7fa5c38ba640 1 -- 172.21.15.5:0/1156848154 wait complete. 2024-09-12T02:59:37.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.611+0000 7fa5c38ba640 1 Processor -- start 2024-09-12T02:59:37.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.611+0000 7fa5c38ba640 1 -- start start 2024-09-12T02:59:37.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.611+0000 7fa5c38ba640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5bc100b20 0x7fa5bc1a0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:37.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.612+0000 7fa5c38ba640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa5bc101440 0x7fa5bc1a0950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:37.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.612+0000 7fa5c38ba640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5bc10e500 0x7fa5bc19a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:37.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.612+0000 7fa5c38ba640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa5bc114280 con 0x7fa5bc100b20 2024-09-12T02:59:37.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.612+0000 7fa5c38ba640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fa5bc114100 con 0x7fa5bc101440 2024-09-12T02:59:37.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.612+0000 7fa5c38ba640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa5bc114400 con 0x7fa5bc10e500 2024-09-12T02:59:37.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.612+0000 7fa5c1656640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5bc100b20 0x7fa5bc1a0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:37.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.612+0000 7fa5c1656640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5bc100b20 0x7fa5bc1a0410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50168/0 (socket says 172.21.15.5:50168) 2024-09-12T02:59:37.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.612+0000 7fa5c1656640 1 -- 172.21.15.5:0/1192765913 learned_addr learned my addr 172.21.15.5:0/1192765913 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:59:37.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.612+0000 7fa5c0e55640 1 --2- 172.21.15.5:0/1192765913 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa5bc101440 0x7fa5bc1a0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:37.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.612+0000 7fa5c1e57640 1 --2- 172.21.15.5:0/1192765913 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5bc10e500 0x7fa5bc19a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:37.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.612+0000 7fa5c1656640 1 -- 172.21.15.5:0/1192765913 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5bc10e500 msgr2=0x7fa5bc19a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:37.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.612+0000 7fa5c1656640 1 --2- 172.21.15.5:0/1192765913 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5bc10e500 0x7fa5bc19a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:37.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.612+0000 7fa5c1656640 1 -- 172.21.15.5:0/1192765913 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa5bc101440 msgr2=0x7fa5bc1a0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:37.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.612+0000 7fa5c1656640 1 --2- 172.21.15.5:0/1192765913 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa5bc101440 0x7fa5bc1a0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:37.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.612+0000 7fa5c1656640 1 -- 172.21.15.5:0/1192765913 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5bc19ad00 con 0x7fa5bc100b20 2024-09-12T02:59:37.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.612+0000 7fa5c1e57640 1 --2- 172.21.15.5:0/1192765913 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5bc10e500 0x7fa5bc19a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:59:37.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.612+0000 7fa5c1656640 1 --2- 172.21.15.5:0/1192765913 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5bc100b20 0x7fa5bc1a0410 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fa5b8006010 tx=0x7fa5b80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:59:37.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.612+0000 7fa5ae7fc640 1 -- 172.21.15.5:0/1192765913 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b803f030 con 0x7fa5bc100b20 2024-09-12T02:59:37.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.612+0000 7fa5c38ba640 1 -- 172.21.15.5:0/1192765913 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5bc19af90 con 0x7fa5bc100b20 2024-09-12T02:59:37.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.612+0000 7fa5ae7fc640 1 -- 172.21.15.5:0/1192765913 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5b8003ce0 con 0x7fa5bc100b20 2024-09-12T02:59:37.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.612+0000 7fa5ae7fc640 1 -- 172.21.15.5:0/1192765913 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b8035070 con 0x7fa5bc100b20 2024-09-12T02:59:37.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.612+0000 7fa5c38ba640 1 -- 172.21.15.5:0/1192765913 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5bc1a72e0 con 0x7fa5bc100b20 2024-09-12T02:59:37.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.615+0000 7fa5ae7fc640 1 -- 172.21.15.5:0/1192765913 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fa5b8042030 con 0x7fa5bc100b20 2024-09-12T02:59:37.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.615+0000 7fa5c5c93740 1 Processor -- start 2024-09-12T02:59:37.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.615+0000 7fa5c5c93740 1 -- start start 2024-09-12T02:59:37.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.615+0000 7fa5ae7fc640 1 --2- 172.21.15.5:0/1192765913 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fa5a80a2b80 0x7fa5a80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:37.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.616+0000 7fa5c5c93740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555d7db8d4a0 0x555d7db8d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:37.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.616+0000 7fa5c5c93740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d7db8ddc0 0x555d7db98140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:37.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.616+0000 7fa5c5c93740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555d7db98680 0x555d7db9aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:37.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.616+0000 7fa5c5c93740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555d7daf32e0 con 0x555d7db8ddc0 2024-09-12T02:59:37.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.616+0000 7fa5c5c93740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x555d7da9ace0 con 0x555d7db8d4a0 2024-09-12T02:59:37.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.616+0000 7fa5c5c93740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x555d7daef2a0 con 0x555d7db98680 2024-09-12T02:59:37.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.616+0000 7fa5ae7fc640 1 -- 172.21.15.5:0/1192765913 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fa5b80ef750 con 0x7fa5bc100b20 2024-09-12T02:59:37.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.616+0000 7fa5c1e57640 1 --2- 172.21.15.5:0/1192765913 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fa5a80a2b80 0x7fa5a80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:37.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.616+0000 7fa5c0e55640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d7db8ddc0 0x555d7db98140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:37.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.616+0000 7fa5c0e55640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d7db8ddc0 0x555d7db98140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50172/0 (socket says 172.21.15.5:50172) 2024-09-12T02:59:37.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.616+0000 7fa5c0e55640 1 -- 172.21.15.5:0/2948493328 learned_addr learned my addr 172.21.15.5:0/2948493328 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:59:37.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.616+0000 7fa5c1656640 1 --2- 172.21.15.5:0/2948493328 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555d7db8d4a0 0x555d7db8d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:37.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.616+0000 7fa5c1e57640 1 --2- 172.21.15.5:0/2948493328 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555d7db98680 0x555d7db9aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:37.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.616+0000 7fa5c0e55640 1 -- 172.21.15.5:0/2948493328 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555d7db98680 msgr2=0x555d7db9aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:37.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.616+0000 7fa5c0e55640 1 --2- 172.21.15.5:0/2948493328 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555d7db98680 0x555d7db9aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:37.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.616+0000 7fa5c0e55640 1 -- 172.21.15.5:0/2948493328 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555d7db8d4a0 msgr2=0x555d7db8d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:37.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.616+0000 7fa5c0e55640 1 --2- 172.21.15.5:0/2948493328 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555d7db8d4a0 0x555d7db8d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:37.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.616+0000 7fa5c0e55640 1 -- 172.21.15.5:0/2948493328 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d7da7f440 con 0x555d7db8ddc0 2024-09-12T02:59:37.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.616+0000 7fa5c1e57640 1 --2- 172.21.15.5:0/1192765913 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fa5a80a2b80 0x7fa5a80a5040 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc069760 tx=0x7fa5b4012a30 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:59:37.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.616+0000 7fa5c0e55640 1 --2- 172.21.15.5:0/2948493328 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d7db8ddc0 0x555d7db98140 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fa5b000bc30 tx=0x7fa5b00107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92196d393e406087 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:59:37.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.616+0000 7fa5a77fe640 1 -- 172.21.15.5:0/2948493328 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b001c480 con 0x555d7db8ddc0 2024-09-12T02:59:37.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.617+0000 7fa5a77fe640 1 -- 172.21.15.5:0/2948493328 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5b0011920 con 0x555d7db8ddc0 2024-09-12T02:59:37.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.617+0000 7fa5a77fe640 1 -- 172.21.15.5:0/2948493328 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b001a810 con 0x555d7db8ddc0 2024-09-12T02:59:37.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.617+0000 7fa5c5c93740 1 -- 172.21.15.5:0/2948493328 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d7db8ddc0 msgr2=0x555d7db98140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:37.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.617+0000 7fa5c5c93740 1 --2- 172.21.15.5:0/2948493328 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d7db8ddc0 0x555d7db98140 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fa5b000bc30 tx=0x7fa5b00107a0 comp rx=0 tx=0).stop 2024-09-12T02:59:37.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.617+0000 7fa5c5c93740 1 -- 172.21.15.5:0/2948493328 shutdown_connections 2024-09-12T02:59:37.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.617+0000 7fa5c5c93740 1 --2- 172.21.15.5:0/2948493328 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d7db8ddc0 0x555d7db98140 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:37.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.617+0000 7fa5c5c93740 1 --2- 172.21.15.5:0/2948493328 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555d7db98680 0x555d7db9aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:37.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.617+0000 7fa5c5c93740 1 --2- 172.21.15.5:0/2948493328 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555d7db8d4a0 0x555d7db8d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:37.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.617+0000 7fa5c5c93740 1 -- 172.21.15.5:0/2948493328 >> 172.21.15.5:0/2948493328 conn(0x555d7db89a30 msgr2=0x555d7db89e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:59:37.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.617+0000 7fa5c5c93740 1 -- 172.21.15.5:0/2948493328 shutdown_connections 2024-09-12T02:59:37.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.617+0000 7fa5c5c93740 1 -- 172.21.15.5:0/2948493328 wait complete. 2024-09-12T02:59:37.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.618+0000 7fa5c5c93740 1 -- start start 2024-09-12T02:59:37.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.618+0000 7fa5c5c93740 1 Processor -- start 2024-09-12T02:59:37.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.618+0000 7fa5c5c93740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555d7db8d4a0 0x555d7db97f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:37.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.618+0000 7fa5c5c93740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555d7db8ddc0 0x555d7db951f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:37.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.618+0000 7fa5c5c93740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d7db98680 0x555d7db95730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:37.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.618+0000 7fa5c5c93740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555d7db9b250 con 0x555d7db98680 2024-09-12T02:59:37.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.618+0000 7fa5c5c93740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x555d7daae050 con 0x555d7db8ddc0 2024-09-12T02:59:37.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.618+0000 7fa5c5c93740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x555d7db9b3d0 con 0x555d7db8d4a0 2024-09-12T02:59:37.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.618+0000 7fa5c5c93740 10 client.0 _ll_register_callbacks cb 0x555d7daee160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:59:37.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.618+0000 7fa5c1e57640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d7db98680 0x555d7db95730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:37.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.618+0000 7fa5c1e57640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d7db98680 0x555d7db95730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50176/0 (socket says 172.21.15.5:50176) 2024-09-12T02:59:37.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.618+0000 7fa5c1e57640 1 -- 172.21.15.5:0/338018808 learned_addr learned my addr 172.21.15.5:0/338018808 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:59:37.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.618+0000 7fa5c1656640 1 --2- 172.21.15.5:0/338018808 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555d7db8d4a0 0x555d7db97f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:37.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.618+0000 7fa5c0e55640 1 --2- 172.21.15.5:0/338018808 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555d7db8ddc0 0x555d7db951f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:37.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.619+0000 7fa5c1e57640 1 -- 172.21.15.5:0/338018808 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555d7db8d4a0 msgr2=0x555d7db97f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:37.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.619+0000 7fa5c1e57640 1 --2- 172.21.15.5:0/338018808 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555d7db8d4a0 0x555d7db97f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:37.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.619+0000 7fa5c1e57640 1 -- 172.21.15.5:0/338018808 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555d7db8ddc0 msgr2=0x555d7db951f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:37.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.619+0000 7fa5c1e57640 1 --2- 172.21.15.5:0/338018808 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555d7db8ddc0 0x555d7db951f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:37.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.619+0000 7fa5c1e57640 1 -- 172.21.15.5:0/338018808 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d7d8643c0 con 0x555d7db98680 2024-09-12T02:59:37.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.619+0000 7fa5c1e57640 1 --2- 172.21.15.5:0/338018808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d7db98680 0x555d7db95730 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fa5b40108f0 tx=0x7fa5b4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc2cf52e067b5bc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:59:37.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.619+0000 7fa5a4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T02:59:37.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.619+0000 7fa5a4ff9640 1 -- 172.21.15.5:0/338018808 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b4024020 con 0x555d7db98680 2024-09-12T02:59:37.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.619+0000 7fa5a4ff9640 1 -- 172.21.15.5:0/338018808 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fa5b40047c0 con 0x555d7db98680 2024-09-12T02:59:37.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.619+0000 7fa5c5c93740 1 -- 172.21.15.5:0/338018808 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5b0011920 con 0x555d7db98680 2024-09-12T02:59:37.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.619+0000 7fa5a4ff9640 1 -- 172.21.15.5:0/338018808 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa5b401f480 con 0x555d7db98680 2024-09-12T02:59:37.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.619+0000 7fa5c5c93740 10 client.15651 fetch_fsmap learned FSMap version 27 2024-09-12T02:59:37.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.619+0000 7fa5c5c93740 1 -- 172.21.15.5:0/338018808 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555d7dba6ba0 con 0x555d7db98680 2024-09-12T02:59:37.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.619+0000 7fa5a4ff9640 1 -- 172.21.15.5:0/338018808 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5b0011920 con 0x555d7db98680 2024-09-12T02:59:37.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.620+0000 7fa5a4ff9640 1 -- 172.21.15.5:0/338018808 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7fa5b402b5d0 con 0x555d7db98680 2024-09-12T02:59:37.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.620+0000 7fa5c5c93740 10 client.15651 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T02:59:37.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.620+0000 7fa5c5c93740 10 client.15651 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T02:59:37.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.620+0000 7fa5c5c93740 20 client.15651 populate_metadata read hostname 'smithi005' 2024-09-12T02:59:37.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.620+0000 7fa5c5c93740 1 --2- 172.21.15.5:0/338018808 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x555d7dba80b0 0x555d7dbaa550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:37.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.620+0000 7fa5c5c93740 4 client.15651 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:59:37.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.620+0000 7fa5c5c93740 1 -- 172.21.15.5:0/338018808 --> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555d7db00850 con 0x555d7dba80b0 2024-09-12T02:59:37.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.620+0000 7fa5c1656640 1 --2- 172.21.15.5:0/338018808 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x555d7dba80b0 0x555d7dbaa550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:37.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.620+0000 7fa5c1656640 1 --2- 172.21.15.5:0/338018808 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x555d7dba80b0 0x555d7dbaa550 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac9c2919ce2eedc3 server_cookie=57873229fbc15e85 in_seq=0 out_seq=0 2024-09-12T02:59:37.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.620+0000 7fa5a4ff9640 10 client.15651 ms_handle_connect on v2:172.21.15.148:6838/2317600000 2024-09-12T02:59:37.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.621+0000 7fa5a4ff9640 1 -- 172.21.15.5:0/338018808 <== mds.0 v2:172.21.15.148:6838/2317600000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555d7db00850 con 0x555d7dba80b0 2024-09-12T02:59:37.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.621+0000 7fa5a4ff9640 10 client.15651 handle_command_reply: tid=0 2024-09-12T02:59:37.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.621+0000 7fa5c5c93740 1 client.15651 shutdown 2024-09-12T02:59:37.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.621+0000 7fa5c5c93740 1 -- 172.21.15.5:0/338018808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d7db98680 msgr2=0x555d7db95730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:37.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.621+0000 7fa5c5c93740 1 --2- 172.21.15.5:0/338018808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d7db98680 0x555d7db95730 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fa5b40108f0 tx=0x7fa5b4004ab0 comp rx=0 tx=0).stop 2024-09-12T02:59:37.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.621+0000 7fa5c5c93740 1 -- 172.21.15.5:0/338018808 shutdown_connections 2024-09-12T02:59:37.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.621+0000 7fa5c5c93740 1 --2- 172.21.15.5:0/338018808 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x555d7dba80b0 0x555d7dbaa550 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:37.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.621+0000 7fa5c5c93740 1 --2- 172.21.15.5:0/338018808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d7db98680 0x555d7db95730 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:37.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.621+0000 7fa5c5c93740 1 --2- 172.21.15.5:0/338018808 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555d7db8ddc0 0x555d7db951f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:37.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.621+0000 7fa5c5c93740 1 --2- 172.21.15.5:0/338018808 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555d7db8d4a0 0x555d7db97f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:37.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.621+0000 7fa5a4ff9640 0 client.15651 ms_handle_reset on v2:172.21.15.148:6838/2317600000 2024-09-12T02:59:37.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.621+0000 7fa5c5c93740 1 -- 172.21.15.5:0/338018808 >> 172.21.15.5:0/338018808 conn(0x555d7db89a30 msgr2=0x555d7db8f760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:59:37.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.621+0000 7fa5c5c93740 1 -- 172.21.15.5:0/338018808 shutdown_connections 2024-09-12T02:59:37.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.621+0000 7fa5c5c93740 1 -- 172.21.15.5:0/338018808 wait complete. 2024-09-12T02:59:37.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.621+0000 7fa5c5c93740 20 client.15651 trim_cache size 0 max 16384 2024-09-12T02:59:37.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.639+0000 7fa5c5c93740 1 Processor -- start 2024-09-12T02:59:37.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.639+0000 7fa5c5c93740 1 -- start start 2024-09-12T02:59:37.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.639+0000 7fa5c5c93740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555d7db89a30 0x555d7db97160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:37.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.639+0000 7fa5c5c93740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555d7db8d4a0 0x555d7dbafe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:37.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.639+0000 7fa5c5c93740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d7db8ddc0 0x555d7dbb0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:37.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.639+0000 7fa5c1e57640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d7db8ddc0 0x555d7dbb0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:37.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.639+0000 7fa5c1e57640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d7db8ddc0 0x555d7dbb0360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50178/0 (socket says 172.21.15.5:50178) 2024-09-12T02:59:37.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.640+0000 7fa5c1e57640 1 -- 172.21.15.5:0/3326010791 learned_addr learned my addr 172.21.15.5:0/3326010791 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:59:37.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.640+0000 7fa5c5c93740 1 -- 172.21.15.5:0/3326010791 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa5bc1af640 con 0x555d7db8ddc0 2024-09-12T02:59:37.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.640+0000 7fa5c5c93740 1 -- 172.21.15.5:0/3326010791 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fa5bc1b11e0 con 0x555d7db89a30 2024-09-12T02:59:37.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.640+0000 7fa5c5c93740 1 -- 172.21.15.5:0/3326010791 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa5bc1afd10 con 0x555d7db8d4a0 2024-09-12T02:59:37.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.640+0000 7fa5c1656640 1 --2- 172.21.15.5:0/3326010791 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555d7db89a30 0x555d7db97160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:37.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.640+0000 7fa5c0e55640 1 --2- 172.21.15.5:0/3326010791 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555d7db8d4a0 0x555d7dbafe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:37.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.640+0000 7fa5c1e57640 1 -- 172.21.15.5:0/3326010791 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555d7db8d4a0 msgr2=0x555d7dbafe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:37.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.640+0000 7fa5c1e57640 1 --2- 172.21.15.5:0/3326010791 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555d7db8d4a0 0x555d7dbafe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:37.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.640+0000 7fa5c1e57640 1 -- 172.21.15.5:0/3326010791 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555d7db89a30 msgr2=0x555d7db97160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:37.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.640+0000 7fa5c1e57640 1 --2- 172.21.15.5:0/3326010791 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555d7db89a30 0x555d7db97160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:37.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.640+0000 7fa5c1e57640 1 -- 172.21.15.5:0/3326010791 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d7db9add0 con 0x555d7db8ddc0 2024-09-12T02:59:37.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.640+0000 7fa5c1656640 1 --2- 172.21.15.5:0/3326010791 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555d7db89a30 0x555d7db97160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:59:37.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.640+0000 7fa5c1e57640 1 --2- 172.21.15.5:0/3326010791 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d7db8ddc0 0x555d7dbb0360 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fa5b402bc30 tx=0x7fa5b4003f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d7f6082b5375bf7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:59:37.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.640+0000 7fa5a6ffd640 1 -- 172.21.15.5:0/3326010791 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b4039070 con 0x555d7db8ddc0 2024-09-12T02:59:37.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.640+0000 7fa5a6ffd640 1 -- 172.21.15.5:0/3326010791 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5b4004410 con 0x555d7db8ddc0 2024-09-12T02:59:37.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.640+0000 7fa5a6ffd640 1 -- 172.21.15.5:0/3326010791 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b4024070 con 0x555d7db8ddc0 2024-09-12T02:59:37.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.641+0000 7fa5c5c93740 1 -- 172.21.15.5:0/3326010791 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d7db8ddc0 msgr2=0x555d7dbb0360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:37.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.641+0000 7fa5c5c93740 1 --2- 172.21.15.5:0/3326010791 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d7db8ddc0 0x555d7dbb0360 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fa5b402bc30 tx=0x7fa5b4003f00 comp rx=0 tx=0).stop 2024-09-12T02:59:37.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.641+0000 7fa5c5c93740 1 -- 172.21.15.5:0/3326010791 shutdown_connections 2024-09-12T02:59:37.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.641+0000 7fa5c5c93740 1 --2- 172.21.15.5:0/3326010791 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d7db8ddc0 0x555d7dbb0360 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:37.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.641+0000 7fa5c5c93740 1 --2- 172.21.15.5:0/3326010791 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555d7db8d4a0 0x555d7dbafe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:37.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.641+0000 7fa5c5c93740 1 --2- 172.21.15.5:0/3326010791 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555d7db89a30 0x555d7db97160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:37.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.641+0000 7fa5c5c93740 1 -- 172.21.15.5:0/3326010791 >> 172.21.15.5:0/3326010791 conn(0x555d7db95890 msgr2=0x555d7db96a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:59:37.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.641+0000 7fa5c5c93740 1 -- 172.21.15.5:0/3326010791 shutdown_connections 2024-09-12T02:59:37.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.641+0000 7fa5c5c93740 1 -- 172.21.15.5:0/3326010791 wait complete. 2024-09-12T02:59:37.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.641+0000 7fa5c5c93740 1 -- start start 2024-09-12T02:59:37.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.642+0000 7fa5c5c93740 1 Processor -- start 2024-09-12T02:59:37.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.642+0000 7fa5c5c93740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555d7db89a30 0x555d7db9e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:37.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.642+0000 7fa5c5c93740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555d7db8d4a0 0x555d7db9ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:37.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.642+0000 7fa5c5c93740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d7db8ddc0 0x555d7db9ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:37.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.642+0000 7fa5c5c93740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa5bc1b0ab0 con 0x555d7db8ddc0 2024-09-12T02:59:37.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.642+0000 7fa5c5c93740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fa5bc1b03e0 con 0x555d7db8d4a0 2024-09-12T02:59:37.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.642+0000 7fa5c5c93740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa5bc1ad620 con 0x555d7db89a30 2024-09-12T02:59:37.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.642+0000 7fa5c5c93740 10 client.0 _ll_register_callbacks cb 0x7fa5b4007f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:59:37.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.642+0000 7fa5c1e57640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d7db8ddc0 0x555d7db9ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:37.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.642+0000 7fa5c1e57640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d7db8ddc0 0x555d7db9ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50190/0 (socket says 172.21.15.5:50190) 2024-09-12T02:59:37.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.642+0000 7fa5c1e57640 1 -- 172.21.15.5:0/147049091 learned_addr learned my addr 172.21.15.5:0/147049091 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:59:37.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.642+0000 7fa5c1656640 1 --2- 172.21.15.5:0/147049091 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555d7db89a30 0x555d7db9e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:37.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.642+0000 7fa5c0e55640 1 --2- 172.21.15.5:0/147049091 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555d7db8d4a0 0x555d7db9ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:37.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.642+0000 7fa5c1e57640 1 -- 172.21.15.5:0/147049091 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555d7db89a30 msgr2=0x555d7db9e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:37.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.642+0000 7fa5c1e57640 1 --2- 172.21.15.5:0/147049091 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555d7db89a30 0x555d7db9e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:37.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.642+0000 7fa5c1e57640 1 -- 172.21.15.5:0/147049091 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555d7db8d4a0 msgr2=0x555d7db9ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:37.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.642+0000 7fa5c1e57640 1 --2- 172.21.15.5:0/147049091 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555d7db8d4a0 0x555d7db9ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:37.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.642+0000 7fa5c1e57640 1 -- 172.21.15.5:0/147049091 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d7dc11ed0 con 0x555d7db8ddc0 2024-09-12T02:59:37.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.642+0000 7fa5c0e55640 1 --2- 172.21.15.5:0/147049091 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555d7db8d4a0 0x555d7db9ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:59:37.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.643+0000 7fa5c1e57640 1 --2- 172.21.15.5:0/147049091 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d7db8ddc0 0x555d7db9ef40 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fa5b40047f0 tx=0x7fa5b4004820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f593e5a3ab5be10 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:59:37.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.643+0000 7fa5a4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T02:59:37.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.643+0000 7fa5a4ff9640 1 -- 172.21.15.5:0/147049091 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5b4024070 con 0x555d7db8ddc0 2024-09-12T02:59:37.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.643+0000 7fa5a4ff9640 1 -- 172.21.15.5:0/147049091 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fa5b402b420 con 0x555d7db8ddc0 2024-09-12T02:59:37.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.643+0000 7fa5c5c93740 1 -- 172.21.15.5:0/147049091 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5b4004410 con 0x555d7db8ddc0 2024-09-12T02:59:37.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.643+0000 7fa5a4ff9640 1 -- 172.21.15.5:0/147049091 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa5b4039070 con 0x555d7db8ddc0 2024-09-12T02:59:37.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.643+0000 7fa5c5c93740 10 client.15657 fetch_fsmap learned FSMap version 27 2024-09-12T02:59:37.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.643+0000 7fa5c5c93740 1 -- 172.21.15.5:0/147049091 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x555d7dc131c0 con 0x555d7db8ddc0 2024-09-12T02:59:37.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.643+0000 7fa5a4ff9640 1 -- 172.21.15.5:0/147049091 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa5b4004410 con 0x555d7db8ddc0 2024-09-12T02:59:37.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.643+0000 7fa5a4ff9640 1 -- 172.21.15.5:0/147049091 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7fa5b4005040 con 0x555d7db8ddc0 2024-09-12T02:59:37.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.643+0000 7fa5c5c93740 10 client.15657 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T02:59:37.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.643+0000 7fa5c5c93740 10 client.15657 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T02:59:37.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.643+0000 7fa5c5c93740 20 client.15657 populate_metadata read hostname 'smithi005' 2024-09-12T02:59:37.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.643+0000 7fa5c5c93740 1 --2- 172.21.15.5:0/147049091 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x555d7db98680 0x555d7dc15920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:37.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.643+0000 7fa5c5c93740 4 client.15657 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T02:59:37.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.643+0000 7fa5c5c93740 1 -- 172.21.15.5:0/147049091 --> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555d7dafad60 con 0x555d7db98680 2024-09-12T02:59:37.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.644+0000 7fa5c1656640 1 --2- 172.21.15.5:0/147049091 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x555d7db98680 0x555d7dc15920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:37.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.644+0000 7fa5c1656640 1 --2- 172.21.15.5:0/147049091 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x555d7db98680 0x555d7dc15920 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1166a23f8cb1b85b server_cookie=26e61667fbfa5d43 in_seq=0 out_seq=0 2024-09-12T02:59:37.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.644+0000 7fa5a4ff9640 10 client.15657 ms_handle_connect on v2:172.21.15.148:6838/2317600000 2024-09-12T02:59:37.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.644+0000 7fa5a4ff9640 1 -- 172.21.15.5:0/147049091 <== mds.0 v2:172.21.15.148:6838/2317600000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x555d7dafad60 con 0x555d7db98680 2024-09-12T02:59:37.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.644+0000 7fa5a4ff9640 10 client.15657 handle_command_reply: tid=0 2024-09-12T02:59:37.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.644+0000 7fa5c5c93740 1 client.15657 shutdown 2024-09-12T02:59:37.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.644+0000 7fa5c5c93740 1 -- 172.21.15.5:0/147049091 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d7db8ddc0 msgr2=0x555d7db9ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:37.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.644+0000 7fa5c5c93740 1 --2- 172.21.15.5:0/147049091 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d7db8ddc0 0x555d7db9ef40 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fa5b40047f0 tx=0x7fa5b4004820 comp rx=0 tx=0).stop 2024-09-12T02:59:37.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.644+0000 7fa5c5c93740 1 -- 172.21.15.5:0/147049091 shutdown_connections 2024-09-12T02:59:37.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.644+0000 7fa5c5c93740 1 --2- 172.21.15.5:0/147049091 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x555d7db98680 0x555d7dc15920 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:37.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.644+0000 7fa5c5c93740 1 --2- 172.21.15.5:0/147049091 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d7db8ddc0 0x555d7db9ef40 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:37.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.644+0000 7fa5c5c93740 1 --2- 172.21.15.5:0/147049091 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555d7db8d4a0 0x555d7db9ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:37.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.644+0000 7fa5c5c93740 1 --2- 172.21.15.5:0/147049091 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555d7db89a30 0x555d7db9e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:37.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.644+0000 7fa5c5c93740 1 -- 172.21.15.5:0/147049091 >> 172.21.15.5:0/147049091 conn(0x555d7db95890 msgr2=0x555d7db93680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:59:37.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.644+0000 7fa5a4ff9640 0 client.15657 ms_handle_reset on v2:172.21.15.148:6838/2317600000 2024-09-12T02:59:37.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.645+0000 7fa5c5c93740 1 -- 172.21.15.5:0/147049091 shutdown_connections 2024-09-12T02:59:37.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.645+0000 7fa5c5c93740 1 -- 172.21.15.5:0/147049091 wait complete. 2024-09-12T02:59:37.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.645+0000 7fa5c5c93740 20 client.15657 trim_cache size 0 max 16384 2024-09-12T02:59:37.655 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-12T02:59:37.656 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-09-12T02:59:37.656 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-09-12T02:59:37.656 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-12T02:59:37.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.645+0000 7fa5c38ba640 1 -- 172.21.15.5:0/1192765913 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fa5a80a2b80 msgr2=0x7fa5a80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:37.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.645+0000 7fa5c38ba640 1 --2- 172.21.15.5:0/1192765913 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fa5a80a2b80 0x7fa5a80a5040 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc069760 tx=0x7fa5b4012a30 comp rx=0 tx=0).stop 2024-09-12T02:59:37.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.645+0000 7fa5c38ba640 1 -- 172.21.15.5:0/1192765913 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5bc100b20 msgr2=0x7fa5bc1a0410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:37.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.645+0000 7fa5c38ba640 1 --2- 172.21.15.5:0/1192765913 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5bc100b20 0x7fa5bc1a0410 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fa5b8006010 tx=0x7fa5b80042f0 comp rx=0 tx=0).stop 2024-09-12T02:59:37.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.645+0000 7fa5c38ba640 1 -- 172.21.15.5:0/1192765913 shutdown_connections 2024-09-12T02:59:37.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.645+0000 7fa5c38ba640 1 --2- 172.21.15.5:0/1192765913 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa5bc10e500 0x7fa5bc19a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:37.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.645+0000 7fa5c38ba640 1 --2- 172.21.15.5:0/1192765913 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa5bc101440 0x7fa5bc1a0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:37.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.645+0000 7fa5c38ba640 1 --2- 172.21.15.5:0/1192765913 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fa5a80a2b80 0x7fa5a80a5040 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:37.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.645+0000 7fa5c38ba640 1 --2- 172.21.15.5:0/1192765913 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5bc100b20 0x7fa5bc1a0410 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:37.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.645+0000 7fa5c38ba640 1 -- 172.21.15.5:0/1192765913 >> 172.21.15.5:0/1192765913 conn(0x7fa5bc0fc7d0 msgr2=0x7fa5bc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:59:37.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.646+0000 7fa5c38ba640 1 -- 172.21.15.5:0/1192765913 shutdown_connections 2024-09-12T02:59:37.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:37.646+0000 7fa5c38ba640 1 -- 172.21.15.5:0/1192765913 wait complete. 2024-09-12T02:59:38.474 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T02:59:38.474 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T02:59:38.738 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:59:39.126 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:59:38 smithi005 ceph-mon[31197]: pgmap v377: 97 pgs: 97 active+clean; 16 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 1.3 MiB/s wr, 19 op/s 2024-09-12T02:59:39.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:39.211+0000 7f53ec3e0640 1 -- 172.21.15.5:0/3235839679 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53e410c170 msgr2=0x7f53e410e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:39.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:39.211+0000 7f53ec3e0640 1 --2- 172.21.15.5:0/3235839679 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53e410c170 0x7f53e410e610 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f53dc009900 tx=0x7f53dc02a480 comp rx=0 tx=0).stop 2024-09-12T02:59:39.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:39.211+0000 7f53ec3e0640 1 -- 172.21.15.5:0/3235839679 shutdown_connections 2024-09-12T02:59:39.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:39.211+0000 7f53ec3e0640 1 --2- 172.21.15.5:0/3235839679 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53e410c170 0x7f53e410e610 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:39.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:39.211+0000 7f53ec3e0640 1 --2- 172.21.15.5:0/3235839679 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f53e4103660 0x7f53e410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:39.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:39.211+0000 7f53ec3e0640 1 --2- 172.21.15.5:0/3235839679 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f53e4102d40 0x7f53e4103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:39.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:39.211+0000 7f53ec3e0640 1 -- 172.21.15.5:0/3235839679 >> 172.21.15.5:0/3235839679 conn(0x7f53e40fc7a0 msgr2=0x7f53e40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:59:39.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:39.211+0000 7f53ec3e0640 1 -- 172.21.15.5:0/3235839679 shutdown_connections 2024-09-12T02:59:39.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:39.211+0000 7f53ec3e0640 1 -- 172.21.15.5:0/3235839679 wait complete. 2024-09-12T02:59:39.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:39.212+0000 7f53ec3e0640 1 Processor -- start 2024-09-12T02:59:39.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:39.212+0000 7f53ec3e0640 1 -- start start 2024-09-12T02:59:39.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:39.212+0000 7f53ec3e0640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f53e4102d40 0x7f53e41009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:39.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:39.212+0000 7f53ec3e0640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f53e4103660 0x7f53e4100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:39.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:39.212+0000 7f53ec3e0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53e410c170 0x7f53e4101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:39.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:39.212+0000 7f53ec3e0640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f53e4111f70 con 0x7f53e410c170 2024-09-12T02:59:39.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:39.212+0000 7f53ec3e0640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f53e4111df0 con 0x7f53e4103660 2024-09-12T02:59:39.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:39.212+0000 7f53ec3e0640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f53e41120f0 con 0x7f53e4102d40 2024-09-12T02:59:39.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:39.212+0000 7f53ea17c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f53e4102d40 0x7f53e41009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:39.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:39.212+0000 7f53ea97d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53e410c170 0x7f53e4101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:39.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:39.212+0000 7f53ea17c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f53e4102d40 0x7f53e41009a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.5:42620/0 (socket says 172.21.15.5:42620) 2024-09-12T02:59:39.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:39.212+0000 7f53ea17c640 1 -- 172.21.15.5:0/2495028260 learned_addr learned my addr 172.21.15.5:0/2495028260 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:59:39.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:39.212+0000 7f53e997b640 1 --2- 172.21.15.5:0/2495028260 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f53e4103660 0x7f53e4100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:39.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:39.213+0000 7f53ea17c640 1 -- 172.21.15.5:0/2495028260 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f53e4103660 msgr2=0x7f53e4100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:39.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:39.213+0000 7f53ea17c640 1 --2- 172.21.15.5:0/2495028260 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f53e4103660 0x7f53e4100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:39.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:39.213+0000 7f53ea17c640 1 -- 172.21.15.5:0/2495028260 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53e410c170 msgr2=0x7f53e4101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:39.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:39.213+0000 7f53ea17c640 1 --2- 172.21.15.5:0/2495028260 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53e410c170 0x7f53e4101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:39.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:39.213+0000 7f53ea17c640 1 -- 172.21.15.5:0/2495028260 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53e41a6830 con 0x7f53e4102d40 2024-09-12T02:59:39.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:39.213+0000 7f53ea17c640 1 --2- 172.21.15.5:0/2495028260 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f53e4102d40 0x7f53e41009a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f53e000eab0 tx=0x7f53e000ef70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:59:39.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:39.213+0000 7f53db7fe640 1 -- 172.21.15.5:0/2495028260 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e000ce40 con 0x7f53e4102d40 2024-09-12T02:59:39.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:39.213+0000 7f53ec3e0640 1 -- 172.21.15.5:0/2495028260 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53e41a6b20 con 0x7f53e4102d40 2024-09-12T02:59:39.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:39.213+0000 7f53db7fe640 1 -- 172.21.15.5:0/2495028260 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53e0011950 con 0x7f53e4102d40 2024-09-12T02:59:39.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:39.213+0000 7f53db7fe640 1 -- 172.21.15.5:0/2495028260 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e0010ae0 con 0x7f53e4102d40 2024-09-12T02:59:39.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:39.213+0000 7f53ec3e0640 1 -- 172.21.15.5:0/2495028260 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53e41a7060 con 0x7f53e4102d40 2024-09-12T02:59:39.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:39.214+0000 7f53ec3e0640 1 -- 172.21.15.5:0/2495028260 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53e4103120 con 0x7f53e4102d40 2024-09-12T02:59:39.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:39.215+0000 7f53db7fe640 1 -- 172.21.15.5:0/2495028260 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f53e0011b00 con 0x7f53e4102d40 2024-09-12T02:59:39.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:39.216+0000 7f53db7fe640 1 --2- 172.21.15.5:0/2495028260 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f53d00a2b80 0x7f53d00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:39.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:39.216+0000 7f53db7fe640 1 -- 172.21.15.5:0/2495028260 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f53e00cf650 con 0x7f53e4102d40 2024-09-12T02:59:39.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:39.216+0000 7f53e997b640 1 --2- 172.21.15.5:0/2495028260 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f53d00a2b80 0x7f53d00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:39.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:39.217+0000 7f53e997b640 1 --2- 172.21.15.5:0/2495028260 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f53d00a2b80 0x7f53d00a5040 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f53e4102430 tx=0x7f53d400a250 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:59:39.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:39.221+0000 7f53db7fe640 1 -- 172.21.15.5:0/2495028260 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f53e0016030 con 0x7f53e4102d40 2024-09-12T02:59:39.259 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:59:38 smithi146 ceph-mon[35011]: pgmap v377: 97 pgs: 97 active+clean; 16 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 1.3 MiB/s wr, 19 op/s 2024-09-12T02:59:39.415 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:59:38 smithi148 ceph-mon[35090]: pgmap v377: 97 pgs: 97 active+clean; 16 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 1.3 MiB/s wr, 19 op/s 2024-09-12T02:59:39.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:39.441+0000 7f53ec3e0640 1 -- 172.21.15.5:0/2495028260 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f53e4101ed0 con 0x7f53e4102d40 2024-09-12T02:59:39.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:39.442+0000 7f53db7fe640 1 -- 172.21.15.5:0/2495028260 <== mon.1 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11969 (secure 0 0 0) 0x7f53e009c5c0 con 0x7f53e4102d40 2024-09-12T02:59:39.443 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-12T02:59:39.443 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":27,"btime":"2024-09-12T02:59:32:973410+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/4135232521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":4135232521},{"type":"v1","addr":"172.21.15.5:6835","nonce":4135232521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/4290221940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":4290221940},{"type":"v1","addr":"172.21.15.5:6837","nonce":4290221940}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T02:56:16.595562+0000","modified":"2024-09-12T02:59:32.675420+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24502,"mds_1":24487,"mds_2":24481,"mds_3":24479,"mds_4":24458},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/26447341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":26447341},{"type":"v1","addr":"172.21.15.5:6839","nonce":26447341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2393443295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2393443295},{"type":"v1","addr":"172.21.15.5:6841","nonce":2393443295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6833/3686970137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3686970137},{"type":"v1","addr":"172.21.15.146:6833","nonce":3686970137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6833/568512209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":568512209},{"type":"v1","addr":"172.21.15.148:6833","nonce":568512209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6835/2540740420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2540740420},{"type":"v1","addr":"172.21.15.146:6835","nonce":2540740420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.146:6837/1673370937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":1673370937},{"type":"v1","addr":"172.21.15.146:6837","nonce":1673370937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":3,"incarnation":22,"state":"up:active","state_seq":6,"addr":"172.21.15.146:6839/3036767228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":3036767228},{"type":"v1","addr":"172.21.15.146:6839","nonce":3036767228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":2,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.148:6835/4195086427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":4195086427},{"type":"v1","addr":"172.21.15.148:6835","nonce":4195086427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.148:6837/2606873922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2606873922},{"type":"v1","addr":"172.21.15.148:6837","nonce":2606873922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6839/2317600000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":2317600000},{"type":"v1","addr":"172.21.15.148:6839","nonce":2317600000}]},"join_fscid":-1,"export_targets":[1,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24502,"qdb_cluster":[24458,24479,24481,24487,24502]},"id":1}]} 2024-09-12T02:59:39.443 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 27 2024-09-12T02:59:39.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:39.444+0000 7f53ec3e0640 1 -- 172.21.15.5:0/2495028260 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f53d00a2b80 msgr2=0x7f53d00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:39.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:39.444+0000 7f53ec3e0640 1 --2- 172.21.15.5:0/2495028260 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f53d00a2b80 0x7f53d00a5040 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f53e4102430 tx=0x7f53d400a250 comp rx=0 tx=0).stop 2024-09-12T02:59:39.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:39.444+0000 7f53ec3e0640 1 -- 172.21.15.5:0/2495028260 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f53e4102d40 msgr2=0x7f53e41009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:39.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:39.444+0000 7f53ec3e0640 1 --2- 172.21.15.5:0/2495028260 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f53e4102d40 0x7f53e41009a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f53e000eab0 tx=0x7f53e000ef70 comp rx=0 tx=0).stop 2024-09-12T02:59:39.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:39.444+0000 7f53ec3e0640 1 -- 172.21.15.5:0/2495028260 shutdown_connections 2024-09-12T02:59:39.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:39.444+0000 7f53ec3e0640 1 --2- 172.21.15.5:0/2495028260 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f53d00a2b80 0x7f53d00a5040 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:39.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:39.444+0000 7f53ec3e0640 1 --2- 172.21.15.5:0/2495028260 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53e410c170 0x7f53e4101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:39.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:39.444+0000 7f53ec3e0640 1 --2- 172.21.15.5:0/2495028260 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f53e4103660 0x7f53e4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:39.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:39.444+0000 7f53ec3e0640 1 --2- 172.21.15.5:0/2495028260 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f53e4102d40 0x7f53e41009a0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:39.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:39.444+0000 7f53ec3e0640 1 -- 172.21.15.5:0/2495028260 >> 172.21.15.5:0/2495028260 conn(0x7f53e40fc7a0 msgr2=0x7f53e40fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:59:39.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:39.445+0000 7f53ec3e0640 1 -- 172.21.15.5:0/2495028260 shutdown_connections 2024-09-12T02:59:39.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:39.445+0000 7f53ec3e0640 1 -- 172.21.15.5:0/2495028260 wait complete. 2024-09-12T02:59:40.049 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-12T02:59:40.113 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:59:39 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/2495028260' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:59:40.346 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:59:40.415 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:59:39 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/2495028260' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:59:40.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:59:39 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/2495028260' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T02:59:40.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.759+0000 7f8113124640 1 -- 172.21.15.5:0/3159059161 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f810c100a80 msgr2=0x7f810c100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:40.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.759+0000 7f8113124640 1 --2- 172.21.15.5:0/3159059161 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f810c100a80 0x7f810c100ee0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f80f8009930 tx=0x7f80f802a420 comp rx=0 tx=0).stop 2024-09-12T02:59:40.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.759+0000 7f8113124640 1 -- 172.21.15.5:0/3159059161 shutdown_connections 2024-09-12T02:59:40.761 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.759+0000 7f8113124640 1 --2- 172.21.15.5:0/3159059161 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f810c100a80 0x7f810c100ee0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:40.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.759+0000 7f8113124640 1 --2- 172.21.15.5:0/3159059161 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f810c101420 0x7f810c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:40.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.759+0000 7f8113124640 1 --2- 172.21.15.5:0/3159059161 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f810c106a80 0x7f810c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:40.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.759+0000 7f8113124640 1 -- 172.21.15.5:0/3159059161 >> 172.21.15.5:0/3159059161 conn(0x7f810c0fc7a0 msgr2=0x7f810c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:59:40.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.759+0000 7f8113124640 1 -- 172.21.15.5:0/3159059161 shutdown_connections 2024-09-12T02:59:40.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.760+0000 7f8113124640 1 -- 172.21.15.5:0/3159059161 wait complete. 2024-09-12T02:59:40.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.760+0000 7f8113124640 1 Processor -- start 2024-09-12T02:59:40.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.760+0000 7f8113124640 1 -- start start 2024-09-12T02:59:40.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.760+0000 7f8113124640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f810c100a80 0x7f810c197070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:40.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.760+0000 7f8113124640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f810c101420 0x7f810c1975b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:40.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.760+0000 7f8113124640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f810c106a80 0x7f810c197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:40.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.761+0000 7f8113124640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f810c110ed0 con 0x7f810c100a80 2024-09-12T02:59:40.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.761+0000 7f8113124640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f810c110d50 con 0x7f810c106a80 2024-09-12T02:59:40.762 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.761+0000 7f8113124640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f810c111050 con 0x7f810c101420 2024-09-12T02:59:40.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.761+0000 7f8110ec0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f810c100a80 0x7f810c197070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:40.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.761+0000 7f8110ec0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f810c100a80 0x7f810c197070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50230/0 (socket says 172.21.15.5:50230) 2024-09-12T02:59:40.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.761+0000 7f8110ec0640 1 -- 172.21.15.5:0/291486061 learned_addr learned my addr 172.21.15.5:0/291486061 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:59:40.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.761+0000 7f81116c1640 1 --2- 172.21.15.5:0/291486061 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f810c106a80 0x7f810c197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:40.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.761+0000 7f810bfff640 1 --2- 172.21.15.5:0/291486061 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f810c101420 0x7f810c1975b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:40.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.761+0000 7f8110ec0640 1 -- 172.21.15.5:0/291486061 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f810c101420 msgr2=0x7f810c1975b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:40.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.761+0000 7f8110ec0640 1 --2- 172.21.15.5:0/291486061 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f810c101420 0x7f810c1975b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:40.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.761+0000 7f8110ec0640 1 -- 172.21.15.5:0/291486061 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f810c106a80 msgr2=0x7f810c197af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:40.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.761+0000 7f8110ec0640 1 --2- 172.21.15.5:0/291486061 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f810c106a80 0x7f810c197af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:40.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.761+0000 7f8110ec0640 1 -- 172.21.15.5:0/291486061 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f810c1a68e0 con 0x7f810c100a80 2024-09-12T02:59:40.763 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.761+0000 7f8110ec0640 1 --2- 172.21.15.5:0/291486061 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f810c100a80 0x7f810c197070 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f810000d950 tx=0x7f810000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:59:40.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.761+0000 7f8109ffb640 1 -- 172.21.15.5:0/291486061 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8100017070 con 0x7f810c100a80 2024-09-12T02:59:40.764 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.761+0000 7f8109ffb640 1 -- 172.21.15.5:0/291486061 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8100009c30 con 0x7f810c100a80 2024-09-12T02:59:40.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.761+0000 7f8113124640 1 -- 172.21.15.5:0/291486061 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f810c1a6bd0 con 0x7f810c100a80 2024-09-12T02:59:40.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.761+0000 7f8109ffb640 1 -- 172.21.15.5:0/291486061 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f810000f6b0 con 0x7f810c100a80 2024-09-12T02:59:40.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.762+0000 7f8113124640 1 -- 172.21.15.5:0/291486061 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f810c1a70e0 con 0x7f810c100a80 2024-09-12T02:59:40.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.764+0000 7f8109ffb640 1 -- 172.21.15.5:0/291486061 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f81000109d0 con 0x7f810c100a80 2024-09-12T02:59:40.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.764+0000 7f81154fd740 1 Processor -- start 2024-09-12T02:59:40.765 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.765+0000 7f81154fd740 1 -- start start 2024-09-12T02:59:40.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.765+0000 7f81154fd740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55800d06db40 0x55800d06c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:40.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.765+0000 7f81154fd740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55800d06c5d0 0x55800d06ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:40.767 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.765+0000 7f81154fd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55800d077370 0x55800d079760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:40.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.765+0000 7f81154fd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55800cfd22f0 con 0x55800d077370 2024-09-12T02:59:40.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.765+0000 7f81154fd740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55800cf79cf0 con 0x55800d06c5d0 2024-09-12T02:59:40.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.765+0000 7f81154fd740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55800cfce2b0 con 0x55800d06db40 2024-09-12T02:59:40.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.765+0000 7f8109ffb640 1 --2- 172.21.15.5:0/291486061 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f80fc0a2b80 0x7f80fc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:40.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.765+0000 7f810bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55800d077370 0x55800d079760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:40.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.765+0000 7f810bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55800d077370 0x55800d079760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50232/0 (socket says 172.21.15.5:50232) 2024-09-12T02:59:40.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.765+0000 7f810bfff640 1 -- 172.21.15.5:0/1213217994 learned_addr learned my addr 172.21.15.5:0/1213217994 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:59:40.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.765+0000 7f81116c1640 1 --2- 172.21.15.5:0/1213217994 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55800d06db40 0x55800d06c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:40.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.765+0000 7f8110ec0640 1 --2- 172.21.15.5:0/1213217994 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55800d06c5d0 0x55800d06ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:40.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.765+0000 7f8109ffb640 1 -- 172.21.15.5:0/291486061 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f81000ceb20 con 0x7f810c100a80 2024-09-12T02:59:40.768 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.765+0000 7f81116c1640 1 --2- 172.21.15.5:0/291486061 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f80fc0a2b80 0x7f80fc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:40.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.765+0000 7f810bfff640 1 -- 172.21.15.5:0/1213217994 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55800d06db40 msgr2=0x55800d06c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:40.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.765+0000 7f810bfff640 1 --2- 172.21.15.5:0/1213217994 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55800d06db40 0x55800d06c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:40.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.765+0000 7f810bfff640 1 -- 172.21.15.5:0/1213217994 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55800d06c5d0 msgr2=0x55800d06ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:40.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.765+0000 7f810bfff640 1 --2- 172.21.15.5:0/1213217994 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55800d06c5d0 0x55800d06ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:40.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.765+0000 7f810bfff640 1 -- 172.21.15.5:0/1213217994 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55800cf5e3a0 con 0x55800d077370 2024-09-12T02:59:40.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.765+0000 7f8110ec0640 1 --2- 172.21.15.5:0/1213217994 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55800d06c5d0 0x55800d06ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:59:40.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.766+0000 7f810bfff640 1 --2- 172.21.15.5:0/1213217994 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55800d077370 0x55800d079760 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f80f8009930 tx=0x7f80f8004180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4605b3c428a49114 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:59:40.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.766+0000 7f80f6ffd640 1 -- 172.21.15.5:0/1213217994 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80f8035040 con 0x55800d077370 2024-09-12T02:59:40.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.766+0000 7f81116c1640 1 --2- 172.21.15.5:0/291486061 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f80fc0a2b80 0x7f80fc0a5040 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f8104014950 tx=0x7f8104007520 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:59:40.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.766+0000 7f80f6ffd640 1 -- 172.21.15.5:0/1213217994 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80f8005520 con 0x55800d077370 2024-09-12T02:59:40.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.766+0000 7f80f6ffd640 1 -- 172.21.15.5:0/1213217994 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80f8003850 con 0x55800d077370 2024-09-12T02:59:40.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.767+0000 7f81154fd740 1 -- 172.21.15.5:0/1213217994 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55800d077370 msgr2=0x55800d079760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:40.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.767+0000 7f81154fd740 1 --2- 172.21.15.5:0/1213217994 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55800d077370 0x55800d079760 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f80f8009930 tx=0x7f80f8004180 comp rx=0 tx=0).stop 2024-09-12T02:59:40.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.767+0000 7f81154fd740 1 -- 172.21.15.5:0/1213217994 shutdown_connections 2024-09-12T02:59:40.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.767+0000 7f81154fd740 1 --2- 172.21.15.5:0/1213217994 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55800d077370 0x55800d079760 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:40.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.767+0000 7f81154fd740 1 --2- 172.21.15.5:0/1213217994 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55800d06c5d0 0x55800d06ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:40.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.767+0000 7f81154fd740 1 --2- 172.21.15.5:0/1213217994 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55800d06db40 0x55800d06c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:40.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.767+0000 7f81154fd740 1 -- 172.21.15.5:0/1213217994 >> 172.21.15.5:0/1213217994 conn(0x55800d035310 msgr2=0x55800d068c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:59:40.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.767+0000 7f81154fd740 1 -- 172.21.15.5:0/1213217994 shutdown_connections 2024-09-12T02:59:40.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.767+0000 7f81154fd740 1 -- 172.21.15.5:0/1213217994 wait complete. 2024-09-12T02:59:40.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.767+0000 7f81154fd740 1 -- start start 2024-09-12T02:59:40.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.767+0000 7f81154fd740 1 Processor -- start 2024-09-12T02:59:40.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.768+0000 7f81154fd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55800d06c5d0 0x55800d076d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:40.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.768+0000 7f81154fd740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55800d06db40 0x55800d073fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:40.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.768+0000 7f81154fd740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55800d077370 0x55800d074530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:40.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.768+0000 7f81154fd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55800d07a010 con 0x55800d06c5d0 2024-09-12T02:59:40.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.768+0000 7f81154fd740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55800cf8d060 con 0x55800d077370 2024-09-12T02:59:40.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.768+0000 7f81154fd740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55800d07a190 con 0x55800d06db40 2024-09-12T02:59:40.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.768+0000 7f81154fd740 10 client.0 _ll_register_callbacks cb 0x55800cce08c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:59:40.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.768+0000 7f8110ec0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55800d06c5d0 0x55800d076d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:40.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.768+0000 7f8110ec0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55800d06c5d0 0x55800d076d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50236/0 (socket says 172.21.15.5:50236) 2024-09-12T02:59:40.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.768+0000 7f810bfff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55800d06db40 0x55800d073fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:40.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.768+0000 7f8110ec0640 1 -- 172.21.15.5:0/3732264829 learned_addr learned my addr 172.21.15.5:0/3732264829 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:59:40.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.768+0000 7f81116c1640 1 --2- 172.21.15.5:0/3732264829 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55800d077370 0x55800d074530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:40.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.769+0000 7f810bfff640 1 -- 172.21.15.5:0/3732264829 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55800d077370 msgr2=0x55800d074530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:40.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.769+0000 7f810bfff640 1 --2- 172.21.15.5:0/3732264829 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55800d077370 0x55800d074530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:40.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.769+0000 7f810bfff640 1 -- 172.21.15.5:0/3732264829 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55800d06c5d0 msgr2=0x55800d076d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:40.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.769+0000 7f810bfff640 1 --2- 172.21.15.5:0/3732264829 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55800d06c5d0 0x55800d076d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:40.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.769+0000 7f810bfff640 1 -- 172.21.15.5:0/3732264829 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55800cd433c0 con 0x55800d06db40 2024-09-12T02:59:40.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.769+0000 7f8110ec0640 1 --2- 172.21.15.5:0/3732264829 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55800d06c5d0 0x55800d076d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:59:40.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.769+0000 7f810bfff640 1 --2- 172.21.15.5:0/3732264829 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55800d06db40 0x55800d073fb0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f80f802a940 tx=0x7f80f8004d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d840df55b516bdd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:59:40.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.769+0000 7f80ebfff640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-09-12T02:59:40.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.769+0000 7f80ebfff640 1 -- 172.21.15.5:0/3732264829 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80f8052060 con 0x55800d06db40 2024-09-12T02:59:40.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.769+0000 7f81154fd740 1 -- 172.21.15.5:0/3732264829 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80f8005520 con 0x55800d06db40 2024-09-12T02:59:40.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.769+0000 7f80ebfff640 1 -- 172.21.15.5:0/3732264829 <== mon.1 v2:172.21.15.148:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f80f80048d0 con 0x55800d06db40 2024-09-12T02:59:40.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.769+0000 7f80ebfff640 1 -- 172.21.15.5:0/3732264829 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f80f8035050 con 0x55800d06db40 2024-09-12T02:59:40.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.769+0000 7f81154fd740 10 client.25252 fetch_fsmap learned FSMap version 27 2024-09-12T02:59:40.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.769+0000 7f81154fd740 1 -- 172.21.15.5:0/3732264829 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55800d07a010 con 0x55800d06db40 2024-09-12T02:59:40.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.770+0000 7f80ebfff640 1 -- 172.21.15.5:0/3732264829 <== mon.1 v2:172.21.15.148:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80f8005520 con 0x55800d06db40 2024-09-12T02:59:40.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.770+0000 7f80ebfff640 1 -- 172.21.15.5:0/3732264829 <== mon.1 v2:172.21.15.148:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7f80f80315b0 con 0x55800d06db40 2024-09-12T02:59:40.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.770+0000 7f81154fd740 10 client.25252 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T02:59:40.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.770+0000 7f81154fd740 10 client.25252 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-12T02:59:40.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.770+0000 7f81154fd740 20 client.25252 populate_metadata read hostname 'smithi005' 2024-09-12T02:59:40.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.770+0000 7f81154fd740 1 --2- 172.21.15.5:0/3732264829 >> [v2:172.21.15.146:6836/1673370937,v1:172.21.15.146:6837/1673370937] conn(0x55800d086f10 0x55800d089350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:40.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.770+0000 7f81154fd740 4 client.25252 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:59:40.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.770+0000 7f81154fd740 1 -- 172.21.15.5:0/3732264829 --> [v2:172.21.15.146:6836/1673370937,v1:172.21.15.146:6837/1673370937] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55800cfdf860 con 0x55800d086f10 2024-09-12T02:59:40.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.770+0000 7f8110ec0640 1 --2- 172.21.15.5:0/3732264829 >> [v2:172.21.15.146:6836/1673370937,v1:172.21.15.146:6837/1673370937] conn(0x55800d086f10 0x55800d089350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:40.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.771+0000 7f80ebfff640 10 client.25252 ms_handle_connect on v2:172.21.15.146:6836/1673370937 2024-09-12T02:59:40.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.771+0000 7f8110ec0640 1 --2- 172.21.15.5:0/3732264829 >> [v2:172.21.15.146:6836/1673370937,v1:172.21.15.146:6837/1673370937] conn(0x55800d086f10 0x55800d089350 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=23e029b614255c1 server_cookie=b31077b13004f107 in_seq=0 out_seq=0 2024-09-12T02:59:40.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.771+0000 7f80ebfff640 1 -- 172.21.15.5:0/3732264829 <== mds.4 v2:172.21.15.146:6836/1673370937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55800cfdf860 con 0x55800d086f10 2024-09-12T02:59:40.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.771+0000 7f80ebfff640 10 client.25252 handle_command_reply: tid=0 2024-09-12T02:59:40.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.771+0000 7f81154fd740 1 client.25252 shutdown 2024-09-12T02:59:40.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.772+0000 7f81154fd740 1 -- 172.21.15.5:0/3732264829 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55800d06db40 msgr2=0x55800d073fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:40.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.772+0000 7f81154fd740 1 --2- 172.21.15.5:0/3732264829 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55800d06db40 0x55800d073fb0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f80f802a940 tx=0x7f80f8004d50 comp rx=0 tx=0).stop 2024-09-12T02:59:40.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.772+0000 7f81154fd740 1 -- 172.21.15.5:0/3732264829 shutdown_connections 2024-09-12T02:59:40.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.772+0000 7f81154fd740 1 --2- 172.21.15.5:0/3732264829 >> [v2:172.21.15.146:6836/1673370937,v1:172.21.15.146:6837/1673370937] conn(0x55800d086f10 0x55800d089350 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:40.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.772+0000 7f81154fd740 1 --2- 172.21.15.5:0/3732264829 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55800d077370 0x55800d074530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:40.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.772+0000 7f81154fd740 1 --2- 172.21.15.5:0/3732264829 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55800d06db40 0x55800d073fb0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:40.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.772+0000 7f80ebfff640 0 client.25252 ms_handle_reset on v2:172.21.15.146:6836/1673370937 2024-09-12T02:59:40.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.772+0000 7f81154fd740 1 --2- 172.21.15.5:0/3732264829 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55800d06c5d0 0x55800d076d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:40.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.772+0000 7f81154fd740 1 -- 172.21.15.5:0/3732264829 >> 172.21.15.5:0/3732264829 conn(0x55800d035310 msgr2=0x55800d06e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:59:40.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.772+0000 7f81154fd740 1 -- 172.21.15.5:0/3732264829 shutdown_connections 2024-09-12T02:59:40.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.772+0000 7f81154fd740 1 -- 172.21.15.5:0/3732264829 wait complete. 2024-09-12T02:59:40.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.772+0000 7f81154fd740 20 client.25252 trim_cache size 0 max 16384 2024-09-12T02:59:40.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.785+0000 7f81154fd740 1 Processor -- start 2024-09-12T02:59:40.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.785+0000 7f81154fd740 1 -- start start 2024-09-12T02:59:40.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.785+0000 7f81154fd740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55800d06c5d0 0x55800d0832b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:40.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.785+0000 7f81154fd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55800d06db40 0x55800d072f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:40.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.785+0000 7f81154fd740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55800d077370 0x55800d08ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:40.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.785+0000 7f81154fd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f810c1af6b0 con 0x55800d06db40 2024-09-12T02:59:40.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.785+0000 7f81154fd740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f810c1b1250 con 0x55800d06c5d0 2024-09-12T02:59:40.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.785+0000 7f81154fd740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f810c1afd80 con 0x55800d077370 2024-09-12T02:59:40.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.785+0000 7f810bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55800d06db40 0x55800d072f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:40.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.785+0000 7f810bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55800d06db40 0x55800d072f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50238/0 (socket says 172.21.15.5:50238) 2024-09-12T02:59:40.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.785+0000 7f810bfff640 1 -- 172.21.15.5:0/787953902 learned_addr learned my addr 172.21.15.5:0/787953902 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:59:40.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.785+0000 7f8110ec0640 1 --2- 172.21.15.5:0/787953902 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55800d06c5d0 0x55800d0832b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:40.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.785+0000 7f81116c1640 1 --2- 172.21.15.5:0/787953902 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55800d077370 0x55800d08ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:40.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.785+0000 7f810bfff640 1 -- 172.21.15.5:0/787953902 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55800d077370 msgr2=0x55800d08ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:40.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.785+0000 7f810bfff640 1 --2- 172.21.15.5:0/787953902 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55800d077370 0x55800d08ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:40.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.785+0000 7f810bfff640 1 -- 172.21.15.5:0/787953902 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55800d06c5d0 msgr2=0x55800d0832b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:40.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.785+0000 7f810bfff640 1 --2- 172.21.15.5:0/787953902 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55800d06c5d0 0x55800d0832b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:40.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.785+0000 7f810bfff640 1 -- 172.21.15.5:0/787953902 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55800d077070 con 0x55800d06db40 2024-09-12T02:59:40.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.785+0000 7f8110ec0640 1 --2- 172.21.15.5:0/787953902 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55800d06c5d0 0x55800d0832b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:59:40.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.786+0000 7f81116c1640 1 --2- 172.21.15.5:0/787953902 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55800d077370 0x55800d08ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:59:40.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.786+0000 7f810bfff640 1 --2- 172.21.15.5:0/787953902 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55800d06db40 0x55800d072f90 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f80f8031950 tx=0x7f80f802a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da4dfff10487bf14 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:59:40.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.786+0000 7f80f67fc640 1 -- 172.21.15.5:0/787953902 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80f8052040 con 0x55800d06db40 2024-09-12T02:59:40.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.786+0000 7f80f67fc640 1 -- 172.21.15.5:0/787953902 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80f80048d0 con 0x55800d06db40 2024-09-12T02:59:40.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.786+0000 7f80f67fc640 1 -- 172.21.15.5:0/787953902 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80f80035f0 con 0x55800d06db40 2024-09-12T02:59:40.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.786+0000 7f81154fd740 1 -- 172.21.15.5:0/787953902 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55800d06db40 msgr2=0x55800d072f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:40.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.786+0000 7f81154fd740 1 --2- 172.21.15.5:0/787953902 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55800d06db40 0x55800d072f90 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f80f8031950 tx=0x7f80f802a420 comp rx=0 tx=0).stop 2024-09-12T02:59:40.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.786+0000 7f81154fd740 1 -- 172.21.15.5:0/787953902 shutdown_connections 2024-09-12T02:59:40.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.786+0000 7f81154fd740 1 --2- 172.21.15.5:0/787953902 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55800d06db40 0x55800d072f90 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:40.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.786+0000 7f81154fd740 1 --2- 172.21.15.5:0/787953902 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55800d077370 0x55800d08ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:40.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.786+0000 7f81154fd740 1 --2- 172.21.15.5:0/787953902 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55800d06c5d0 0x55800d0832b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:40.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.786+0000 7f81154fd740 1 -- 172.21.15.5:0/787953902 >> 172.21.15.5:0/787953902 conn(0x55800d035310 msgr2=0x55800d072880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:59:40.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.787+0000 7f81154fd740 1 -- 172.21.15.5:0/787953902 shutdown_connections 2024-09-12T02:59:40.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.787+0000 7f81154fd740 1 -- 172.21.15.5:0/787953902 wait complete. 2024-09-12T02:59:40.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.787+0000 7f81154fd740 1 -- start start 2024-09-12T02:59:40.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.787+0000 7f81154fd740 1 Processor -- start 2024-09-12T02:59:40.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.787+0000 7f81154fd740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55800d06c5d0 0x55800d07a980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:40.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.787+0000 7f81154fd740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55800d06db40 0x55800d07aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:40.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.787+0000 7f81154fd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55800d077370 0x55800d0efc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:40.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.787+0000 7f81154fd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f810c1b0b20 con 0x55800d077370 2024-09-12T02:59:40.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.787+0000 7f81154fd740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f810c1b0450 con 0x55800d06db40 2024-09-12T02:59:40.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.787+0000 7f81154fd740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f810c1ad690 con 0x55800d06c5d0 2024-09-12T02:59:40.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.787+0000 7f81154fd740 10 client.0 _ll_register_callbacks cb 0x7f80f80040d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:59:40.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.787+0000 7f81116c1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55800d077370 0x55800d0efc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:40.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.788+0000 7f81116c1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55800d077370 0x55800d0efc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50240/0 (socket says 172.21.15.5:50240) 2024-09-12T02:59:40.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.788+0000 7f81116c1640 1 -- 172.21.15.5:0/991947831 learned_addr learned my addr 172.21.15.5:0/991947831 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:59:40.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.788+0000 7f810bfff640 1 --2- 172.21.15.5:0/991947831 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55800d06db40 0x55800d07aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:40.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.788+0000 7f8110ec0640 1 --2- 172.21.15.5:0/991947831 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55800d06c5d0 0x55800d07a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:40.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.788+0000 7f81116c1640 1 -- 172.21.15.5:0/991947831 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55800d06c5d0 msgr2=0x55800d07a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:40.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.788+0000 7f81116c1640 1 --2- 172.21.15.5:0/991947831 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55800d06c5d0 0x55800d07a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:40.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.788+0000 7f81116c1640 1 -- 172.21.15.5:0/991947831 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55800d06db40 msgr2=0x55800d07aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:40.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.788+0000 7f81116c1640 1 --2- 172.21.15.5:0/991947831 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55800d06db40 0x55800d07aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:40.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.788+0000 7f81116c1640 1 -- 172.21.15.5:0/991947831 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55800d0f0160 con 0x55800d077370 2024-09-12T02:59:40.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.788+0000 7f810bfff640 1 --2- 172.21.15.5:0/991947831 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55800d06db40 0x55800d07aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:59:40.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.788+0000 7f81116c1640 1 --2- 172.21.15.5:0/991947831 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55800d077370 0x55800d0efc20 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f8104014600 tx=0x7f8104004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c746fbb76d570671 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:59:40.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.788+0000 7f80ebfff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T02:59:40.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.788+0000 7f80ebfff640 1 -- 172.21.15.5:0/991947831 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f810401fb60 con 0x55800d077370 2024-09-12T02:59:40.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.788+0000 7f80ebfff640 1 -- 172.21.15.5:0/991947831 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f810401fd00 con 0x55800d077370 2024-09-12T02:59:40.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.788+0000 7f81154fd740 1 -- 172.21.15.5:0/991947831 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80f80048d0 con 0x55800d077370 2024-09-12T02:59:40.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.788+0000 7f80ebfff640 1 -- 172.21.15.5:0/991947831 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8104004b20 con 0x55800d077370 2024-09-12T02:59:40.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.789+0000 7f81154fd740 10 client.15681 fetch_fsmap learned FSMap version 27 2024-09-12T02:59:40.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.789+0000 7f81154fd740 1 -- 172.21.15.5:0/991947831 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55800d0f1340 con 0x55800d077370 2024-09-12T02:59:40.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.789+0000 7f80ebfff640 1 -- 172.21.15.5:0/991947831 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f80f80048d0 con 0x55800d077370 2024-09-12T02:59:40.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.789+0000 7f80ebfff640 1 -- 172.21.15.5:0/991947831 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7f8104027070 con 0x55800d077370 2024-09-12T02:59:40.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.789+0000 7f81154fd740 10 client.15681 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T02:59:40.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.789+0000 7f81154fd740 10 client.15681 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-12T02:59:40.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.789+0000 7f81154fd740 20 client.15681 populate_metadata read hostname 'smithi005' 2024-09-12T02:59:40.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.789+0000 7f81154fd740 1 --2- 172.21.15.5:0/991947831 >> [v2:172.21.15.146:6836/1673370937,v1:172.21.15.146:6837/1673370937] conn(0x55800d0f1b10 0x55800d0f3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:40.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.789+0000 7f81154fd740 4 client.15681 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:59:40.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.789+0000 7f81154fd740 1 -- 172.21.15.5:0/991947831 --> [v2:172.21.15.146:6836/1673370937,v1:172.21.15.146:6837/1673370937] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55800cfd9d70 con 0x55800d0f1b10 2024-09-12T02:59:40.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.789+0000 7f8110ec0640 1 --2- 172.21.15.5:0/991947831 >> [v2:172.21.15.146:6836/1673370937,v1:172.21.15.146:6837/1673370937] conn(0x55800d0f1b10 0x55800d0f3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:40.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.790+0000 7f80ebfff640 10 client.15681 ms_handle_connect on v2:172.21.15.146:6836/1673370937 2024-09-12T02:59:40.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.790+0000 7f8110ec0640 1 --2- 172.21.15.5:0/991947831 >> [v2:172.21.15.146:6836/1673370937,v1:172.21.15.146:6837/1673370937] conn(0x55800d0f1b10 0x55800d0f3f30 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=20cd23ca749d5b55 server_cookie=1313618dee863244 in_seq=0 out_seq=0 2024-09-12T02:59:40.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.790+0000 7f80ebfff640 1 -- 172.21.15.5:0/991947831 <== mds.4 v2:172.21.15.146:6836/1673370937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55800cfd9d70 con 0x55800d0f1b10 2024-09-12T02:59:40.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.790+0000 7f80ebfff640 10 client.15681 handle_command_reply: tid=0 2024-09-12T02:59:40.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.790+0000 7f81154fd740 1 client.15681 shutdown 2024-09-12T02:59:40.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.790+0000 7f81154fd740 1 -- 172.21.15.5:0/991947831 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55800d077370 msgr2=0x55800d0efc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:40.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.790+0000 7f81154fd740 1 --2- 172.21.15.5:0/991947831 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55800d077370 0x55800d0efc20 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f8104014600 tx=0x7f8104004ab0 comp rx=0 tx=0).stop 2024-09-12T02:59:40.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.790+0000 7f81154fd740 1 -- 172.21.15.5:0/991947831 shutdown_connections 2024-09-12T02:59:40.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.790+0000 7f81154fd740 1 --2- 172.21.15.5:0/991947831 >> [v2:172.21.15.146:6836/1673370937,v1:172.21.15.146:6837/1673370937] conn(0x55800d0f1b10 0x55800d0f3f30 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:40.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.790+0000 7f81154fd740 1 --2- 172.21.15.5:0/991947831 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55800d077370 0x55800d0efc20 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:40.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.790+0000 7f81154fd740 1 --2- 172.21.15.5:0/991947831 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55800d06db40 0x55800d07aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:40.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.790+0000 7f81154fd740 1 --2- 172.21.15.5:0/991947831 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55800d06c5d0 0x55800d07a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:40.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.790+0000 7f81154fd740 1 -- 172.21.15.5:0/991947831 >> 172.21.15.5:0/991947831 conn(0x55800d035310 msgr2=0x55800d0730a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:59:40.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.790+0000 7f80ebfff640 0 client.15681 ms_handle_reset on v2:172.21.15.146:6836/1673370937 2024-09-12T02:59:40.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.790+0000 7f81154fd740 1 -- 172.21.15.5:0/991947831 shutdown_connections 2024-09-12T02:59:40.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.790+0000 7f81154fd740 1 -- 172.21.15.5:0/991947831 wait complete. 2024-09-12T02:59:40.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.790+0000 7f81154fd740 20 client.15681 trim_cache size 0 max 16384 2024-09-12T02:59:40.796 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-12T02:59:40.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.791+0000 7f8113124640 1 -- 172.21.15.5:0/291486061 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f80fc0a2b80 msgr2=0x7f80fc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:40.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.791+0000 7f8113124640 1 --2- 172.21.15.5:0/291486061 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f80fc0a2b80 0x7f80fc0a5040 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f8104014950 tx=0x7f8104007520 comp rx=0 tx=0).stop 2024-09-12T02:59:40.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.791+0000 7f8113124640 1 -- 172.21.15.5:0/291486061 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f810c100a80 msgr2=0x7f810c197070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:40.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.791+0000 7f8113124640 1 --2- 172.21.15.5:0/291486061 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f810c100a80 0x7f810c197070 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f810000d950 tx=0x7f810000de10 comp rx=0 tx=0).stop 2024-09-12T02:59:40.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.791+0000 7f8113124640 1 -- 172.21.15.5:0/291486061 shutdown_connections 2024-09-12T02:59:40.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.791+0000 7f8113124640 1 --2- 172.21.15.5:0/291486061 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f810c106a80 0x7f810c197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:40.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.791+0000 7f8113124640 1 --2- 172.21.15.5:0/291486061 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f810c101420 0x7f810c1975b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:40.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.791+0000 7f8113124640 1 --2- 172.21.15.5:0/291486061 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f80fc0a2b80 0x7f80fc0a5040 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:40.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.791+0000 7f8113124640 1 --2- 172.21.15.5:0/291486061 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f810c100a80 0x7f810c197070 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:40.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.791+0000 7f8113124640 1 -- 172.21.15.5:0/291486061 >> 172.21.15.5:0/291486061 conn(0x7f810c0fc7a0 msgr2=0x7f810c0fce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:59:40.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.791+0000 7f8113124640 1 -- 172.21.15.5:0/291486061 shutdown_connections 2024-09-12T02:59:40.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:40.791+0000 7f8113124640 1 -- 172.21.15.5:0/291486061 wait complete. 2024-09-12T02:59:41.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:59:40 smithi005 ceph-mon[31197]: pgmap v378: 97 pgs: 97 active+clean; 17 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 1.5 MiB/s wr, 27 op/s 2024-09-12T02:59:41.317 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:59:40 smithi148 ceph-mon[35090]: pgmap v378: 97 pgs: 97 active+clean; 17 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 1.5 MiB/s wr, 27 op/s 2024-09-12T02:59:41.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:59:40 smithi146 ceph-mon[35011]: pgmap v378: 97 pgs: 97 active+clean; 17 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 1.5 MiB/s wr, 27 op/s 2024-09-12T02:59:41.465 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-12T02:59:41.762 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:59:42.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.225+0000 7f54a1e8e640 1 -- 172.21.15.5:0/242479030 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f549c101de0 msgr2=0x7f549c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:42.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.225+0000 7f54a1e8e640 1 --2- 172.21.15.5:0/242479030 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f549c101de0 0x7f549c1021c0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f548c009900 tx=0x7f548c02a410 comp rx=0 tx=0).stop 2024-09-12T02:59:42.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.225+0000 7f54a1e8e640 1 -- 172.21.15.5:0/242479030 shutdown_connections 2024-09-12T02:59:42.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.225+0000 7f54a1e8e640 1 --2- 172.21.15.5:0/242479030 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f549c106f10 0x7f549c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:42.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.225+0000 7f54a1e8e640 1 --2- 172.21.15.5:0/242479030 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f549c102790 0x7f549c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:42.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.225+0000 7f54a1e8e640 1 --2- 172.21.15.5:0/242479030 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f549c101de0 0x7f549c1021c0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:42.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.225+0000 7f54a1e8e640 1 -- 172.21.15.5:0/242479030 >> 172.21.15.5:0/242479030 conn(0x7f549c0fd8c0 msgr2=0x7f549c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:59:42.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.225+0000 7f54a1e8e640 1 -- 172.21.15.5:0/242479030 shutdown_connections 2024-09-12T02:59:42.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.225+0000 7f54a1e8e640 1 -- 172.21.15.5:0/242479030 wait complete. 2024-09-12T02:59:42.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.226+0000 7f54a1e8e640 1 Processor -- start 2024-09-12T02:59:42.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.226+0000 7f54a1e8e640 1 -- start start 2024-09-12T02:59:42.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.226+0000 7f54a1e8e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f549c101de0 0x7f549c1984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:42.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.226+0000 7f54a1e8e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f549c102790 0x7f549c1989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:42.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.226+0000 7f54a1e8e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f549c106f10 0x7f549c19cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:42.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.226+0000 7f54a1e8e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f549c111f90 con 0x7f549c101de0 2024-09-12T02:59:42.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.226+0000 7f54a1e8e640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f549c111e10 con 0x7f549c106f10 2024-09-12T02:59:42.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.226+0000 7f54a1e8e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f549c112110 con 0x7f549c102790 2024-09-12T02:59:42.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.226+0000 7f549b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f549c101de0 0x7f549c1984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:42.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.226+0000 7f549b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f549c101de0 0x7f549c1984b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50248/0 (socket says 172.21.15.5:50248) 2024-09-12T02:59:42.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.226+0000 7f549b7fe640 1 -- 172.21.15.5:0/70085087 learned_addr learned my addr 172.21.15.5:0/70085087 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:59:42.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.227+0000 7f549affd640 1 --2- 172.21.15.5:0/70085087 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f549c102790 0x7f549c1989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:42.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.227+0000 7f549bfff640 1 --2- 172.21.15.5:0/70085087 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f549c106f10 0x7f549c19cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:42.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.227+0000 7f549b7fe640 1 -- 172.21.15.5:0/70085087 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f549c102790 msgr2=0x7f549c1989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:42.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.227+0000 7f549b7fe640 1 --2- 172.21.15.5:0/70085087 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f549c102790 0x7f549c1989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:42.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.227+0000 7f549b7fe640 1 -- 172.21.15.5:0/70085087 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f549c106f10 msgr2=0x7f549c19cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:42.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.227+0000 7f549b7fe640 1 --2- 172.21.15.5:0/70085087 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f549c106f10 0x7f549c19cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:42.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.227+0000 7f549b7fe640 1 -- 172.21.15.5:0/70085087 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f549c19d460 con 0x7f549c101de0 2024-09-12T02:59:42.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.227+0000 7f549b7fe640 1 --2- 172.21.15.5:0/70085087 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f549c101de0 0x7f549c1984b0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f548c005ab0 tx=0x7f548c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:59:42.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.227+0000 7f5498ff9640 1 -- 172.21.15.5:0/70085087 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f548c03f030 con 0x7f549c101de0 2024-09-12T02:59:42.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.227+0000 7f5498ff9640 1 -- 172.21.15.5:0/70085087 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f548c003ce0 con 0x7f549c101de0 2024-09-12T02:59:42.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.227+0000 7f5498ff9640 1 -- 172.21.15.5:0/70085087 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f548c035070 con 0x7f549c101de0 2024-09-12T02:59:42.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.227+0000 7f54a1e8e640 1 -- 172.21.15.5:0/70085087 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f549c19d6f0 con 0x7f549c101de0 2024-09-12T02:59:42.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.227+0000 7f54a1e8e640 1 -- 172.21.15.5:0/70085087 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f549c1a5030 con 0x7f549c101de0 2024-09-12T02:59:42.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.230+0000 7f5498ff9640 1 -- 172.21.15.5:0/70085087 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f548c042030 con 0x7f549c101de0 2024-09-12T02:59:42.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.230+0000 7f54a4267740 1 Processor -- start 2024-09-12T02:59:42.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.230+0000 7f54a4267740 1 -- start start 2024-09-12T02:59:42.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.230+0000 7f54a4267740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56005ebf1f50 0x56005ebf04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:42.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.230+0000 7f54a4267740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56005ebf09e0 0x56005ebf0e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:42.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.230+0000 7f54a4267740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56005ebfb780 0x56005ebfdb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:42.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.230+0000 7f54a4267740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56005eb522f0 con 0x56005ebfb780 2024-09-12T02:59:42.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.230+0000 7f54a4267740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x56005eaf9cf0 con 0x56005ebf1f50 2024-09-12T02:59:42.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.230+0000 7f54a4267740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x56005eb4e2b0 con 0x56005ebf09e0 2024-09-12T02:59:42.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.230+0000 7f549affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56005ebfb780 0x56005ebfdb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:42.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.230+0000 7f549affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56005ebfb780 0x56005ebfdb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50254/0 (socket says 172.21.15.5:50254) 2024-09-12T02:59:42.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.230+0000 7f549affd640 1 -- 172.21.15.5:0/4056736519 learned_addr learned my addr 172.21.15.5:0/4056736519 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:59:42.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.230+0000 7f549bfff640 1 --2- 172.21.15.5:0/4056736519 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56005ebf1f50 0x56005ebf04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:42.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.230+0000 7f549b7fe640 1 --2- 172.21.15.5:0/4056736519 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56005ebf09e0 0x56005ebf0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:42.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.231+0000 7f5498ff9640 1 --2- 172.21.15.5:0/70085087 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f54880a2b80 0x7f54880a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:42.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.231+0000 7f549affd640 1 -- 172.21.15.5:0/4056736519 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56005ebf09e0 msgr2=0x56005ebf0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:42.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.231+0000 7f549affd640 1 --2- 172.21.15.5:0/4056736519 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56005ebf09e0 0x56005ebf0e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:42.241 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.231+0000 7f549affd640 1 -- 172.21.15.5:0/4056736519 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56005ebf1f50 msgr2=0x56005ebf04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:42.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.231+0000 7f549affd640 1 --2- 172.21.15.5:0/4056736519 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56005ebf1f50 0x56005ebf04a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:42.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.231+0000 7f549affd640 1 -- 172.21.15.5:0/4056736519 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56005eade3a0 con 0x56005ebfb780 2024-09-12T02:59:42.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.231+0000 7f549b7fe640 1 --2- 172.21.15.5:0/4056736519 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56005ebf09e0 0x56005ebf0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:59:42.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.231+0000 7f549bfff640 1 --2- 172.21.15.5:0/70085087 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f54880a2b80 0x7f54880a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:42.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.231+0000 7f5498ff9640 1 -- 172.21.15.5:0/70085087 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f548c0ef9d0 con 0x7f549c101de0 2024-09-12T02:59:42.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.231+0000 7f549affd640 1 --2- 172.21.15.5:0/4056736519 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56005ebfb780 0x56005ebfdb70 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f548400b7f0 tx=0x7f548400bcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30bec695b6c49d03 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:59:42.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.231+0000 7f5491ffb640 1 -- 172.21.15.5:0/4056736519 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f548400d950 con 0x56005ebfb780 2024-09-12T02:59:42.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.231+0000 7f5491ffb640 1 -- 172.21.15.5:0/4056736519 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5484018070 con 0x56005ebfb780 2024-09-12T02:59:42.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.231+0000 7f5491ffb640 1 -- 172.21.15.5:0/4056736519 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54840123e0 con 0x56005ebfb780 2024-09-12T02:59:42.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.232+0000 7f54a4267740 1 -- 172.21.15.5:0/4056736519 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56005ebfb780 msgr2=0x56005ebfdb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:42.242 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.232+0000 7f549bfff640 1 --2- 172.21.15.5:0/70085087 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f54880a2b80 0x7f54880a5040 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f5494007f30 tx=0x7f5494004a40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:59:42.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.232+0000 7f54a4267740 1 --2- 172.21.15.5:0/4056736519 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56005ebfb780 0x56005ebfdb70 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f548400b7f0 tx=0x7f548400bcb0 comp rx=0 tx=0).stop 2024-09-12T02:59:42.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.232+0000 7f54a4267740 1 -- 172.21.15.5:0/4056736519 shutdown_connections 2024-09-12T02:59:42.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.232+0000 7f54a4267740 1 --2- 172.21.15.5:0/4056736519 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56005ebfb780 0x56005ebfdb70 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:42.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.232+0000 7f54a4267740 1 --2- 172.21.15.5:0/4056736519 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56005ebf09e0 0x56005ebf0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:42.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.232+0000 7f54a4267740 1 --2- 172.21.15.5:0/4056736519 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56005ebf1f50 0x56005ebf04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:42.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.232+0000 7f54a4267740 1 -- 172.21.15.5:0/4056736519 >> 172.21.15.5:0/4056736519 conn(0x56005ebb5310 msgr2=0x56005ebed070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:59:42.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.232+0000 7f54a4267740 1 -- 172.21.15.5:0/4056736519 shutdown_connections 2024-09-12T02:59:42.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.232+0000 7f54a4267740 1 -- 172.21.15.5:0/4056736519 wait complete. 2024-09-12T02:59:42.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.232+0000 7f54a4267740 1 -- start start 2024-09-12T02:59:42.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.233+0000 7f54a4267740 1 Processor -- start 2024-09-12T02:59:42.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.233+0000 7f54a4267740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56005ebf09e0 0x56005ebfb150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:42.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.233+0000 7f54a4267740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56005ebf1f50 0x56005ebf8400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:42.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.233+0000 7f54a4267740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56005ebfb780 0x56005ebf89f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:42.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.233+0000 7f54a4267740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56005ebfe480 con 0x56005ebf1f50 2024-09-12T02:59:42.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.233+0000 7f54a4267740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x56005eb0d060 con 0x56005ebf09e0 2024-09-12T02:59:42.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.233+0000 7f54a4267740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x56005ebfe600 con 0x56005ebfb780 2024-09-12T02:59:42.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.233+0000 7f54a4267740 10 client.0 _ll_register_callbacks cb 0x56005e8608c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:59:42.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.233+0000 7f549affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56005ebf1f50 0x56005ebf8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:42.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.233+0000 7f549affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56005ebf1f50 0x56005ebf8400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50270/0 (socket says 172.21.15.5:50270) 2024-09-12T02:59:42.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.233+0000 7f549affd640 1 -- 172.21.15.5:0/2459258278 learned_addr learned my addr 172.21.15.5:0/2459258278 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:59:42.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.233+0000 7f549affd640 1 -- 172.21.15.5:0/2459258278 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56005ebfb780 msgr2=0x56005ebf89f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:42.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.233+0000 7f549affd640 1 --2- 172.21.15.5:0/2459258278 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56005ebfb780 0x56005ebf89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:42.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.233+0000 7f549affd640 1 -- 172.21.15.5:0/2459258278 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56005ebf09e0 msgr2=0x56005ebfb150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:42.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.233+0000 7f549affd640 1 --2- 172.21.15.5:0/2459258278 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56005ebf09e0 0x56005ebfb150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:42.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.233+0000 7f549affd640 1 -- 172.21.15.5:0/2459258278 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56005e8c33c0 con 0x56005ebf1f50 2024-09-12T02:59:42.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.234+0000 7f549affd640 1 --2- 172.21.15.5:0/2459258278 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56005ebf1f50 0x56005ebf8400 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f5484013bf0 tx=0x7f54840136b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6fc3a58d6c6df8e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:59:42.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.234+0000 7f547affd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T02:59:42.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.234+0000 7f547affd640 1 -- 172.21.15.5:0/2459258278 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5484023070 con 0x56005ebf1f50 2024-09-12T02:59:42.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.234+0000 7f54a4267740 1 -- 172.21.15.5:0/2459258278 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5484018070 con 0x56005ebf1f50 2024-09-12T02:59:42.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.234+0000 7f547affd640 1 -- 172.21.15.5:0/2459258278 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f548400dd60 con 0x56005ebf1f50 2024-09-12T02:59:42.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.235+0000 7f547affd640 1 -- 172.21.15.5:0/2459258278 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f548402e050 con 0x56005ebf1f50 2024-09-12T02:59:42.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.235+0000 7f54a4267740 10 client.15693 fetch_fsmap learned FSMap version 27 2024-09-12T02:59:42.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.235+0000 7f54a4267740 1 -- 172.21.15.5:0/2459258278 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56005ec09ef0 con 0x56005ebf1f50 2024-09-12T02:59:42.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.235+0000 7f547affd640 1 -- 172.21.15.5:0/2459258278 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5484018070 con 0x56005ebf1f50 2024-09-12T02:59:42.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.235+0000 7f547affd640 1 -- 172.21.15.5:0/2459258278 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7f548401b380 con 0x56005ebf1f50 2024-09-12T02:59:42.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.235+0000 7f54a4267740 10 client.15693 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T02:59:42.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.235+0000 7f54a4267740 10 client.15693 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.b 2024-09-12T02:59:42.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.235+0000 7f54a4267740 20 client.15693 populate_metadata read hostname 'smithi005' 2024-09-12T02:59:42.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.235+0000 7f54a4267740 1 --2- 172.21.15.5:0/2459258278 >> [v2:172.21.15.146:6838/3036767228,v1:172.21.15.146:6839/3036767228] conn(0x56005ec0b240 0x56005ec0d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:42.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.235+0000 7f54a4267740 4 client.15693 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:59:42.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.235+0000 7f54a4267740 1 -- 172.21.15.5:0/2459258278 --> [v2:172.21.15.146:6838/3036767228,v1:172.21.15.146:6839/3036767228] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56005eb5f860 con 0x56005ec0b240 2024-09-12T02:59:42.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.235+0000 7f549b7fe640 1 --2- 172.21.15.5:0/2459258278 >> [v2:172.21.15.146:6838/3036767228,v1:172.21.15.146:6839/3036767228] conn(0x56005ec0b240 0x56005ec0d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:42.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.236+0000 7f547affd640 10 client.15693 ms_handle_connect on v2:172.21.15.146:6838/3036767228 2024-09-12T02:59:42.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.236+0000 7f549b7fe640 1 --2- 172.21.15.5:0/2459258278 >> [v2:172.21.15.146:6838/3036767228,v1:172.21.15.146:6839/3036767228] conn(0x56005ec0b240 0x56005ec0d630 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=eb2d8d8c4a9d51d6 server_cookie=165dddf7ffb86a48 in_seq=0 out_seq=0 2024-09-12T02:59:42.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.237+0000 7f547affd640 1 -- 172.21.15.5:0/2459258278 <== mds.3 v2:172.21.15.146:6838/3036767228 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56005eb5f860 con 0x56005ec0b240 2024-09-12T02:59:42.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.237+0000 7f547affd640 10 client.15693 handle_command_reply: tid=0 2024-09-12T02:59:42.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.237+0000 7f54a4267740 1 client.15693 shutdown 2024-09-12T02:59:42.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.237+0000 7f54a4267740 1 -- 172.21.15.5:0/2459258278 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56005ebf1f50 msgr2=0x56005ebf8400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:42.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.237+0000 7f54a4267740 1 --2- 172.21.15.5:0/2459258278 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56005ebf1f50 0x56005ebf8400 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f5484013bf0 tx=0x7f54840136b0 comp rx=0 tx=0).stop 2024-09-12T02:59:42.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.237+0000 7f54a4267740 1 -- 172.21.15.5:0/2459258278 shutdown_connections 2024-09-12T02:59:42.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.237+0000 7f54a4267740 1 --2- 172.21.15.5:0/2459258278 >> [v2:172.21.15.146:6838/3036767228,v1:172.21.15.146:6839/3036767228] conn(0x56005ec0b240 0x56005ec0d630 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:42.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.237+0000 7f54a4267740 1 --2- 172.21.15.5:0/2459258278 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56005ebf1f50 0x56005ebf8400 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:42.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.237+0000 7f54a4267740 1 --2- 172.21.15.5:0/2459258278 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56005ebfb780 0x56005ebf89f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:42.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.237+0000 7f54a4267740 1 --2- 172.21.15.5:0/2459258278 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56005ebf09e0 0x56005ebfb150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:42.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.237+0000 7f54a4267740 1 -- 172.21.15.5:0/2459258278 >> 172.21.15.5:0/2459258278 conn(0x56005ebb5310 msgr2=0x56005ebf2d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:59:42.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.237+0000 7f547affd640 0 client.15693 ms_handle_reset on v2:172.21.15.146:6838/3036767228 2024-09-12T02:59:42.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.237+0000 7f54a4267740 1 -- 172.21.15.5:0/2459258278 shutdown_connections 2024-09-12T02:59:42.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.237+0000 7f54a4267740 1 -- 172.21.15.5:0/2459258278 wait complete. 2024-09-12T02:59:42.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.237+0000 7f54a4267740 20 client.15693 trim_cache size 0 max 16384 2024-09-12T02:59:42.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.250+0000 7f54a4267740 1 Processor -- start 2024-09-12T02:59:42.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.250+0000 7f54a4267740 1 -- start start 2024-09-12T02:59:42.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.250+0000 7f54a4267740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56005ebf09e0 0x56005ec07720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:42.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.250+0000 7f54a4267740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56005ebf1f50 0x56005ebecb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:42.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.250+0000 7f54a4267740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56005ebfb780 0x56005ec12c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:42.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.250+0000 7f54a4267740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f549c1ab2f0 con 0x56005ebfb780 2024-09-12T02:59:42.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.250+0000 7f54a4267740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f549c1ace90 con 0x56005ebf09e0 2024-09-12T02:59:42.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.250+0000 7f54a4267740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f549c1ab9c0 con 0x56005ebf1f50 2024-09-12T02:59:42.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.250+0000 7f549bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56005ebfb780 0x56005ec12c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:42.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.251+0000 7f549bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56005ebfb780 0x56005ec12c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50284/0 (socket says 172.21.15.5:50284) 2024-09-12T02:59:42.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.251+0000 7f549bfff640 1 -- 172.21.15.5:0/423771659 learned_addr learned my addr 172.21.15.5:0/423771659 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:59:42.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.251+0000 7f549b7fe640 1 --2- 172.21.15.5:0/423771659 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56005ebf09e0 0x56005ec07720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:42.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.251+0000 7f549affd640 1 --2- 172.21.15.5:0/423771659 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56005ebf1f50 0x56005ebecb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:42.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.251+0000 7f549bfff640 1 -- 172.21.15.5:0/423771659 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56005ebf1f50 msgr2=0x56005ebecb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:42.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.251+0000 7f549bfff640 1 --2- 172.21.15.5:0/423771659 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56005ebf1f50 0x56005ebecb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:42.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.251+0000 7f549bfff640 1 -- 172.21.15.5:0/423771659 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56005ebf09e0 msgr2=0x56005ec07720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:42.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.251+0000 7f549bfff640 1 --2- 172.21.15.5:0/423771659 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56005ebf09e0 0x56005ec07720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:42.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.251+0000 7f549bfff640 1 -- 172.21.15.5:0/423771659 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56005ebed1f0 con 0x56005ebfb780 2024-09-12T02:59:42.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.251+0000 7f549b7fe640 1 --2- 172.21.15.5:0/423771659 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56005ebf09e0 0x56005ec07720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:59:42.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.251+0000 7f549bfff640 1 --2- 172.21.15.5:0/423771659 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56005ebfb780 0x56005ec12c30 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f54940127d0 tx=0x7f5494014070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5301e07bf46bb396 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:59:42.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.251+0000 7f54917fa640 1 -- 172.21.15.5:0/423771659 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f549401dae0 con 0x56005ebfb780 2024-09-12T02:59:42.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.251+0000 7f54917fa640 1 -- 172.21.15.5:0/423771659 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5494022070 con 0x56005ebfb780 2024-09-12T02:59:42.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.251+0000 7f54917fa640 1 -- 172.21.15.5:0/423771659 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f549401e6b0 con 0x56005ebfb780 2024-09-12T02:59:42.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.252+0000 7f54a4267740 1 -- 172.21.15.5:0/423771659 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56005ebfb780 msgr2=0x56005ec12c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:42.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.252+0000 7f54a4267740 1 --2- 172.21.15.5:0/423771659 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56005ebfb780 0x56005ec12c30 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f54940127d0 tx=0x7f5494014070 comp rx=0 tx=0).stop 2024-09-12T02:59:42.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.252+0000 7f54a4267740 1 -- 172.21.15.5:0/423771659 shutdown_connections 2024-09-12T02:59:42.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.252+0000 7f54a4267740 1 --2- 172.21.15.5:0/423771659 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56005ebfb780 0x56005ec12c30 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:42.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.252+0000 7f54a4267740 1 --2- 172.21.15.5:0/423771659 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56005ebf1f50 0x56005ebecb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:42.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.252+0000 7f54a4267740 1 --2- 172.21.15.5:0/423771659 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56005ebf09e0 0x56005ec07720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:42.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.252+0000 7f54a4267740 1 -- 172.21.15.5:0/423771659 >> 172.21.15.5:0/423771659 conn(0x56005ebb5310 msgr2=0x56005ebf74f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:59:42.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.252+0000 7f54a4267740 1 -- 172.21.15.5:0/423771659 shutdown_connections 2024-09-12T02:59:42.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.252+0000 7f54a4267740 1 -- 172.21.15.5:0/423771659 wait complete. 2024-09-12T02:59:42.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.252+0000 7f54a4267740 1 -- start start 2024-09-12T02:59:42.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.253+0000 7f54a4267740 1 Processor -- start 2024-09-12T02:59:42.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.253+0000 7f54a4267740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56005ebf09e0 0x56005ebf55b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:42.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.253+0000 7f54a4267740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56005ebf1f50 0x56005ebf5af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:42.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.253+0000 7f54a4267740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56005ebfb780 0x56005ebf6030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:42.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.253+0000 7f54a4267740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f549c1ac760 con 0x56005ebfb780 2024-09-12T02:59:42.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.253+0000 7f54a4267740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f549c1ac090 con 0x56005ebf09e0 2024-09-12T02:59:42.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.253+0000 7f54a4267740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f549c1a92d0 con 0x56005ebf1f50 2024-09-12T02:59:42.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.253+0000 7f54a4267740 10 client.0 _ll_register_callbacks cb 0x7f5494007520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:59:42.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.253+0000 7f549bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56005ebfb780 0x56005ebf6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:42.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.253+0000 7f549bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56005ebfb780 0x56005ebf6030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50290/0 (socket says 172.21.15.5:50290) 2024-09-12T02:59:42.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.253+0000 7f549bfff640 1 -- 172.21.15.5:0/242557352 learned_addr learned my addr 172.21.15.5:0/242557352 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:59:42.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.253+0000 7f549affd640 1 --2- 172.21.15.5:0/242557352 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56005ebf1f50 0x56005ebf5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:42.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.253+0000 7f549b7fe640 1 --2- 172.21.15.5:0/242557352 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56005ebf09e0 0x56005ebf55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:42.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.253+0000 7f549bfff640 1 -- 172.21.15.5:0/242557352 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56005ebf1f50 msgr2=0x56005ebf5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:42.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.253+0000 7f549bfff640 1 --2- 172.21.15.5:0/242557352 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56005ebf1f50 0x56005ebf5af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:42.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.253+0000 7f549bfff640 1 -- 172.21.15.5:0/242557352 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56005ebf09e0 msgr2=0x56005ebf55b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:42.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.253+0000 7f549bfff640 1 --2- 172.21.15.5:0/242557352 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56005ebf09e0 0x56005ebf55b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:42.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.253+0000 7f549bfff640 1 -- 172.21.15.5:0/242557352 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56005ec74db0 con 0x56005ebfb780 2024-09-12T02:59:42.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.253+0000 7f549affd640 1 --2- 172.21.15.5:0/242557352 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56005ebf1f50 0x56005ebf5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:59:42.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.253+0000 7f549b7fe640 1 --2- 172.21.15.5:0/242557352 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56005ebf09e0 0x56005ebf55b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:59:42.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.253+0000 7f549bfff640 1 --2- 172.21.15.5:0/242557352 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56005ebfb780 0x56005ebf6030 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f54940276a0 tx=0x7f54940276d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fc3897e9de3d913 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:59:42.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.253+0000 7f547affd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T02:59:42.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.254+0000 7f547affd640 1 -- 172.21.15.5:0/242557352 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54940278f0 con 0x56005ebfb780 2024-09-12T02:59:42.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.254+0000 7f54a4267740 1 -- 172.21.15.5:0/242557352 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5494022070 con 0x56005ebfb780 2024-09-12T02:59:42.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.254+0000 7f547affd640 1 -- 172.21.15.5:0/242557352 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f549401daf0 con 0x56005ebfb780 2024-09-12T02:59:42.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.254+0000 7f547affd640 1 -- 172.21.15.5:0/242557352 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5494007cb0 con 0x56005ebfb780 2024-09-12T02:59:42.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.254+0000 7f54a4267740 10 client.15699 fetch_fsmap learned FSMap version 27 2024-09-12T02:59:42.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.254+0000 7f54a4267740 1 -- 172.21.15.5:0/242557352 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56005ec76070 con 0x56005ebfb780 2024-09-12T02:59:42.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.254+0000 7f547affd640 1 -- 172.21.15.5:0/242557352 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5494022070 con 0x56005ebfb780 2024-09-12T02:59:42.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.254+0000 7f547affd640 1 -- 172.21.15.5:0/242557352 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7f549401e450 con 0x56005ebfb780 2024-09-12T02:59:42.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.254+0000 7f54a4267740 10 client.15699 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T02:59:42.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.254+0000 7f54a4267740 10 client.15699 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.b 2024-09-12T02:59:42.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.254+0000 7f54a4267740 20 client.15699 populate_metadata read hostname 'smithi005' 2024-09-12T02:59:42.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.254+0000 7f54a4267740 1 --2- 172.21.15.5:0/242557352 >> [v2:172.21.15.146:6838/3036767228,v1:172.21.15.146:6839/3036767228] conn(0x56005ec766f0 0x56005ec78b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:42.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.254+0000 7f54a4267740 4 client.15699 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:59:42.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.254+0000 7f54a4267740 1 -- 172.21.15.5:0/242557352 --> [v2:172.21.15.146:6838/3036767228,v1:172.21.15.146:6839/3036767228] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56005eb59d70 con 0x56005ec766f0 2024-09-12T02:59:42.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.255+0000 7f549b7fe640 1 --2- 172.21.15.5:0/242557352 >> [v2:172.21.15.146:6838/3036767228,v1:172.21.15.146:6839/3036767228] conn(0x56005ec766f0 0x56005ec78b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:42.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.255+0000 7f549b7fe640 1 --2- 172.21.15.5:0/242557352 >> [v2:172.21.15.146:6838/3036767228,v1:172.21.15.146:6839/3036767228] conn(0x56005ec766f0 0x56005ec78b10 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2970e5373fee14b4 server_cookie=f43e1c7ee28d0661 in_seq=0 out_seq=0 2024-09-12T02:59:42.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.255+0000 7f547affd640 10 client.15699 ms_handle_connect on v2:172.21.15.146:6838/3036767228 2024-09-12T02:59:42.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.255+0000 7f547affd640 1 -- 172.21.15.5:0/242557352 <== mds.3 v2:172.21.15.146:6838/3036767228 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56005eb59d70 con 0x56005ec766f0 2024-09-12T02:59:42.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.255+0000 7f547affd640 10 client.15699 handle_command_reply: tid=0 2024-09-12T02:59:42.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.255+0000 7f54a4267740 1 client.15699 shutdown 2024-09-12T02:59:42.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.256+0000 7f54a4267740 1 -- 172.21.15.5:0/242557352 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56005ebfb780 msgr2=0x56005ebf6030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:42.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.256+0000 7f54a4267740 1 --2- 172.21.15.5:0/242557352 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56005ebfb780 0x56005ebf6030 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f54940276a0 tx=0x7f54940276d0 comp rx=0 tx=0).stop 2024-09-12T02:59:42.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.256+0000 7f54a4267740 1 -- 172.21.15.5:0/242557352 shutdown_connections 2024-09-12T02:59:42.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.256+0000 7f54a4267740 1 --2- 172.21.15.5:0/242557352 >> [v2:172.21.15.146:6838/3036767228,v1:172.21.15.146:6839/3036767228] conn(0x56005ec766f0 0x56005ec78b10 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:42.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.256+0000 7f54a4267740 1 --2- 172.21.15.5:0/242557352 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56005ebfb780 0x56005ebf6030 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:42.262 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.256+0000 7f54a4267740 1 --2- 172.21.15.5:0/242557352 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56005ebf1f50 0x56005ebf5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:42.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.256+0000 7f547affd640 0 client.15699 ms_handle_reset on v2:172.21.15.146:6838/3036767228 2024-09-12T02:59:42.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.256+0000 7f54a4267740 1 --2- 172.21.15.5:0/242557352 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56005ebf09e0 0x56005ebf55b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:42.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.256+0000 7f54a4267740 1 -- 172.21.15.5:0/242557352 >> 172.21.15.5:0/242557352 conn(0x56005ebb5310 msgr2=0x56005ebf74f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:59:42.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.256+0000 7f54a4267740 1 -- 172.21.15.5:0/242557352 shutdown_connections 2024-09-12T02:59:42.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.256+0000 7f54a4267740 1 -- 172.21.15.5:0/242557352 wait complete. 2024-09-12T02:59:42.263 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.256+0000 7f54a4267740 20 client.15699 trim_cache size 0 max 16384 2024-09-12T02:59:42.263 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-12T02:59:42.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.256+0000 7f54a1e8e640 1 -- 172.21.15.5:0/70085087 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f54880a2b80 msgr2=0x7f54880a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:42.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.256+0000 7f54a1e8e640 1 --2- 172.21.15.5:0/70085087 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f54880a2b80 0x7f54880a5040 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f5494007f30 tx=0x7f5494004a40 comp rx=0 tx=0).stop 2024-09-12T02:59:42.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.256+0000 7f54a1e8e640 1 -- 172.21.15.5:0/70085087 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f549c101de0 msgr2=0x7f549c1984b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:42.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.256+0000 7f54a1e8e640 1 --2- 172.21.15.5:0/70085087 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f549c101de0 0x7f549c1984b0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f548c005ab0 tx=0x7f548c0042f0 comp rx=0 tx=0).stop 2024-09-12T02:59:42.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.256+0000 7f54a1e8e640 1 -- 172.21.15.5:0/70085087 shutdown_connections 2024-09-12T02:59:42.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.256+0000 7f54a1e8e640 1 --2- 172.21.15.5:0/70085087 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f549c106f10 0x7f549c19cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:42.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.256+0000 7f54a1e8e640 1 --2- 172.21.15.5:0/70085087 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f549c102790 0x7f549c1989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:42.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.256+0000 7f54a1e8e640 1 --2- 172.21.15.5:0/70085087 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f54880a2b80 0x7f54880a5040 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:42.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.256+0000 7f54a1e8e640 1 --2- 172.21.15.5:0/70085087 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f549c101de0 0x7f549c1984b0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:42.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.256+0000 7f54a1e8e640 1 -- 172.21.15.5:0/70085087 >> 172.21.15.5:0/70085087 conn(0x7f549c0fd8c0 msgr2=0x7f549c0feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:59:42.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.257+0000 7f54a1e8e640 1 -- 172.21.15.5:0/70085087 shutdown_connections 2024-09-12T02:59:42.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:42.257+0000 7f54a1e8e640 1 -- 172.21.15.5:0/70085087 wait complete. 2024-09-12T02:59:43.001 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T02:59:43.258 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:59:43.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:59:42 smithi005 ceph-mon[31197]: pgmap v379: 97 pgs: 97 active+clean; 17 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 1.5 MiB/s wr, 26 op/s 2024-09-12T02:59:43.415 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:59:42 smithi148 ceph-mon[35090]: pgmap v379: 97 pgs: 97 active+clean; 17 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 1.5 MiB/s wr, 26 op/s 2024-09-12T02:59:43.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:59:42 smithi146 ceph-mon[35011]: pgmap v379: 97 pgs: 97 active+clean; 17 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 1.5 MiB/s wr, 26 op/s 2024-09-12T02:59:43.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.759+0000 7f670e28b640 1 -- 172.21.15.5:0/3400018974 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6708102de0 msgr2=0x7f67081031c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:43.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.759+0000 7f670e28b640 1 --2- 172.21.15.5:0/3400018974 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6708102de0 0x7f67081031c0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f66f8009900 tx=0x7f66f802a480 comp rx=0 tx=0).stop 2024-09-12T02:59:43.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.760+0000 7f670e28b640 1 -- 172.21.15.5:0/3400018974 shutdown_connections 2024-09-12T02:59:43.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.760+0000 7f670e28b640 1 --2- 172.21.15.5:0/3400018974 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f670810c180 0x7f670810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:43.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.760+0000 7f670e28b640 1 --2- 172.21.15.5:0/3400018974 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6708103700 0x7f670810bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:43.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.760+0000 7f670e28b640 1 --2- 172.21.15.5:0/3400018974 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6708102de0 0x7f67081031c0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:43.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.760+0000 7f670e28b640 1 -- 172.21.15.5:0/3400018974 >> 172.21.15.5:0/3400018974 conn(0x7f67080fc7a0 msgr2=0x7f67080febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:59:43.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.760+0000 7f670e28b640 1 -- 172.21.15.5:0/3400018974 shutdown_connections 2024-09-12T02:59:43.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.760+0000 7f670e28b640 1 -- 172.21.15.5:0/3400018974 wait complete. 2024-09-12T02:59:43.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.760+0000 7f670e28b640 1 Processor -- start 2024-09-12T02:59:43.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.760+0000 7f670e28b640 1 -- start start 2024-09-12T02:59:43.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.760+0000 7f670e28b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6708102de0 0x7f6708100a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:43.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.760+0000 7f670e28b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6708103700 0x7f6708100f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:43.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.760+0000 7f670e28b640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f670810c180 0x7f67081014c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:43.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.760+0000 7f670e28b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6708111f70 con 0x7f6708103700 2024-09-12T02:59:43.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.761+0000 7f670e28b640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f6708111df0 con 0x7f670810c180 2024-09-12T02:59:43.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.761+0000 7f670e28b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f67081120f0 con 0x7f6708102de0 2024-09-12T02:59:43.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.761+0000 7f66fffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6708103700 0x7f6708100f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:43.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.761+0000 7f66fffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6708103700 0x7f6708100f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39506/0 (socket says 172.21.15.5:39506) 2024-09-12T02:59:43.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.761+0000 7f66fffff640 1 -- 172.21.15.5:0/4261501918 learned_addr learned my addr 172.21.15.5:0/4261501918 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:59:43.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.761+0000 7f6707fff640 1 --2- 172.21.15.5:0/4261501918 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6708102de0 0x7f6708100a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:43.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.761+0000 7f670c828640 1 --2- 172.21.15.5:0/4261501918 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f670810c180 0x7f67081014c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:43.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.761+0000 7f66fffff640 1 -- 172.21.15.5:0/4261501918 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6708102de0 msgr2=0x7f6708100a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:43.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.761+0000 7f66fffff640 1 --2- 172.21.15.5:0/4261501918 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6708102de0 0x7f6708100a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:43.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.761+0000 7f66fffff640 1 -- 172.21.15.5:0/4261501918 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f670810c180 msgr2=0x7f67081014c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:43.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.761+0000 7f66fffff640 1 --2- 172.21.15.5:0/4261501918 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f670810c180 0x7f67081014c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:43.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.761+0000 7f66fffff640 1 -- 172.21.15.5:0/4261501918 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67081a6930 con 0x7f6708103700 2024-09-12T02:59:43.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.761+0000 7f670c828640 1 --2- 172.21.15.5:0/4261501918 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f670810c180 0x7f67081014c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:59:43.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.761+0000 7f66fffff640 1 --2- 172.21.15.5:0/4261501918 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6708103700 0x7f6708100f80 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f66f000d770 tx=0x7f66f000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:59:43.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.761+0000 7f6705ffb640 1 -- 172.21.15.5:0/4261501918 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66f0017070 con 0x7f6708103700 2024-09-12T02:59:43.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.761+0000 7f6705ffb640 1 -- 172.21.15.5:0/4261501918 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66f0005500 con 0x7f6708103700 2024-09-12T02:59:43.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.761+0000 7f670e28b640 1 -- 172.21.15.5:0/4261501918 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67081a6c20 con 0x7f6708103700 2024-09-12T02:59:43.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.761+0000 7f6705ffb640 1 -- 172.21.15.5:0/4261501918 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66f0004e00 con 0x7f6708103700 2024-09-12T02:59:43.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.761+0000 7f670e28b640 1 -- 172.21.15.5:0/4261501918 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67081a7110 con 0x7f6708103700 2024-09-12T02:59:43.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.764+0000 7f6705ffb640 1 -- 172.21.15.5:0/4261501918 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f66f0004fa0 con 0x7f6708103700 2024-09-12T02:59:43.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.764+0000 7f6710664740 1 Processor -- start 2024-09-12T02:59:43.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.764+0000 7f6710664740 1 -- start start 2024-09-12T02:59:43.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.764+0000 7f6710664740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5623b1df2b40 0x5623b1df1090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:43.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.764+0000 7f6710664740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5623b1df15d0 0x5623b1df1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:43.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.764+0000 7f6710664740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623b1dfc370 0x5623b1dfe760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:43.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.764+0000 7f6710664740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5623b1d572f0 con 0x5623b1dfc370 2024-09-12T02:59:43.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.764+0000 7f6710664740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5623b1cfecf0 con 0x5623b1df2b40 2024-09-12T02:59:43.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.764+0000 7f6710664740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5623b1d532b0 con 0x5623b1df15d0 2024-09-12T02:59:43.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.764+0000 7f66fffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623b1dfc370 0x5623b1dfe760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:43.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.764+0000 7f66fffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623b1dfc370 0x5623b1dfe760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39510/0 (socket says 172.21.15.5:39510) 2024-09-12T02:59:43.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.764+0000 7f66fffff640 1 -- 172.21.15.5:0/2638085298 learned_addr learned my addr 172.21.15.5:0/2638085298 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:59:43.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.764+0000 7f6707fff640 1 --2- 172.21.15.5:0/2638085298 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5623b1df15d0 0x5623b1df1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:43.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.765+0000 7f6705ffb640 1 --2- 172.21.15.5:0/4261501918 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f66f40a2b80 0x7f66f40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:43.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.765+0000 7f670c828640 1 --2- 172.21.15.5:0/2638085298 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5623b1df2b40 0x5623b1df1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:43.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.765+0000 7f670c828640 1 --2- 172.21.15.5:0/4261501918 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f66f40a2b80 0x7f66f40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:43.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.765+0000 7f6705ffb640 1 -- 172.21.15.5:0/4261501918 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f66f00ceab0 con 0x7f6708103700 2024-09-12T02:59:43.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.765+0000 7f66fffff640 1 -- 172.21.15.5:0/2638085298 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5623b1df15d0 msgr2=0x5623b1df1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:43.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.765+0000 7f66fffff640 1 --2- 172.21.15.5:0/2638085298 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5623b1df15d0 0x5623b1df1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:43.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.765+0000 7f66fffff640 1 -- 172.21.15.5:0/2638085298 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5623b1df2b40 msgr2=0x5623b1df1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:43.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.765+0000 7f66fffff640 1 --2- 172.21.15.5:0/2638085298 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5623b1df2b40 0x5623b1df1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:43.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.765+0000 7f66fffff640 1 -- 172.21.15.5:0/2638085298 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623b1ce33a0 con 0x5623b1dfc370 2024-09-12T02:59:43.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.765+0000 7f66fffff640 1 --2- 172.21.15.5:0/2638085298 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623b1dfc370 0x5623b1dfe760 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f66f00cc500 tx=0x7f66f0009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0af008dffe1850f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:59:43.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.765+0000 7f670c828640 1 --2- 172.21.15.5:0/4261501918 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f66f40a2b80 0x7f66f40a5040 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f6700005c40 tx=0x7f6700009280 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:59:43.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.765+0000 7f66fe7fc640 1 -- 172.21.15.5:0/2638085298 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66f0020070 con 0x5623b1dfc370 2024-09-12T02:59:43.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.765+0000 7f66fe7fc640 1 -- 172.21.15.5:0/2638085298 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66f00046c0 con 0x5623b1dfc370 2024-09-12T02:59:43.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.765+0000 7f66fe7fc640 1 -- 172.21.15.5:0/2638085298 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66f00a3f90 con 0x5623b1dfc370 2024-09-12T02:59:43.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.766+0000 7f6710664740 1 -- 172.21.15.5:0/2638085298 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623b1dfc370 msgr2=0x5623b1dfe760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:43.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.766+0000 7f6710664740 1 --2- 172.21.15.5:0/2638085298 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623b1dfc370 0x5623b1dfe760 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f66f00cc500 tx=0x7f66f0009c30 comp rx=0 tx=0).stop 2024-09-12T02:59:43.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.766+0000 7f6710664740 1 -- 172.21.15.5:0/2638085298 shutdown_connections 2024-09-12T02:59:43.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.766+0000 7f6710664740 1 --2- 172.21.15.5:0/2638085298 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623b1dfc370 0x5623b1dfe760 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:43.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.766+0000 7f6710664740 1 --2- 172.21.15.5:0/2638085298 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5623b1df15d0 0x5623b1df1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:43.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.766+0000 7f6710664740 1 --2- 172.21.15.5:0/2638085298 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5623b1df2b40 0x5623b1df1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:43.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.766+0000 7f6710664740 1 -- 172.21.15.5:0/2638085298 >> 172.21.15.5:0/2638085298 conn(0x5623b1dba310 msgr2=0x5623b1dedc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:59:43.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.766+0000 7f6710664740 1 -- 172.21.15.5:0/2638085298 shutdown_connections 2024-09-12T02:59:43.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.766+0000 7f6710664740 1 -- 172.21.15.5:0/2638085298 wait complete. 2024-09-12T02:59:43.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.766+0000 7f6710664740 1 -- start start 2024-09-12T02:59:43.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.767+0000 7f6710664740 1 Processor -- start 2024-09-12T02:59:43.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.767+0000 7f6710664740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5623b1df15d0 0x5623b1dfbd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:43.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.767+0000 7f6710664740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5623b1df2b40 0x5623b1df9020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:43.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.767+0000 7f6710664740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623b1dfc370 0x5623b1df9610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:43.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.767+0000 7f6710664740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5623b1dff010 con 0x5623b1dfc370 2024-09-12T02:59:43.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.767+0000 7f6710664740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5623b1d12060 con 0x5623b1df2b40 2024-09-12T02:59:43.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.767+0000 7f6710664740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5623b1dff190 con 0x5623b1df15d0 2024-09-12T02:59:43.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.767+0000 7f6710664740 10 client.0 _ll_register_callbacks cb 0x5623b1a658c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:59:43.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.767+0000 7f670c828640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623b1dfc370 0x5623b1df9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:43.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.767+0000 7f670c828640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623b1dfc370 0x5623b1df9610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39522/0 (socket says 172.21.15.5:39522) 2024-09-12T02:59:43.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.767+0000 7f670c828640 1 -- 172.21.15.5:0/1038491074 learned_addr learned my addr 172.21.15.5:0/1038491074 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:59:43.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.767+0000 7f66fffff640 1 --2- 172.21.15.5:0/1038491074 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5623b1df2b40 0x5623b1df9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:43.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.767+0000 7f6707fff640 1 --2- 172.21.15.5:0/1038491074 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5623b1df15d0 0x5623b1dfbd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:43.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.767+0000 7f670c828640 1 -- 172.21.15.5:0/1038491074 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5623b1df15d0 msgr2=0x5623b1dfbd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:43.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.767+0000 7f670c828640 1 --2- 172.21.15.5:0/1038491074 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5623b1df15d0 0x5623b1dfbd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:43.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.767+0000 7f670c828640 1 -- 172.21.15.5:0/1038491074 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5623b1df2b40 msgr2=0x5623b1df9020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:43.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.767+0000 7f670c828640 1 --2- 172.21.15.5:0/1038491074 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5623b1df2b40 0x5623b1df9020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:43.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.767+0000 7f670c828640 1 -- 172.21.15.5:0/1038491074 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623b1ac83c0 con 0x5623b1dfc370 2024-09-12T02:59:43.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.767+0000 7f66fffff640 1 --2- 172.21.15.5:0/1038491074 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5623b1df2b40 0x5623b1df9020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:59:43.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.768+0000 7f670c828640 1 --2- 172.21.15.5:0/1038491074 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623b1dfc370 0x5623b1df9610 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f6700012fb0 tx=0x7f6700004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeb52b6db59b5d24 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:59:43.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.768+0000 7f66e77fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T02:59:43.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.768+0000 7f66e77fe640 1 -- 172.21.15.5:0/1038491074 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6700025070 con 0x5623b1dfc370 2024-09-12T02:59:43.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.768+0000 7f66e77fe640 1 -- 172.21.15.5:0/1038491074 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f6700010ce0 con 0x5623b1dfc370 2024-09-12T02:59:43.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.768+0000 7f6710664740 1 -- 172.21.15.5:0/1038491074 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5623b1d64860 con 0x5623b1dfc370 2024-09-12T02:59:43.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.768+0000 7f66e77fe640 1 -- 172.21.15.5:0/1038491074 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f670002a460 con 0x5623b1dfc370 2024-09-12T02:59:43.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.768+0000 7f6710664740 10 client.15711 fetch_fsmap learned FSMap version 27 2024-09-12T02:59:43.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.768+0000 7f6710664740 1 -- 172.21.15.5:0/1038491074 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5623b1e0ab10 con 0x5623b1dfc370 2024-09-12T02:59:43.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.768+0000 7f66e77fe640 1 -- 172.21.15.5:0/1038491074 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5623b1d64860 con 0x5623b1dfc370 2024-09-12T02:59:43.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.768+0000 7f66e77fe640 1 -- 172.21.15.5:0/1038491074 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7f670001e040 con 0x5623b1dfc370 2024-09-12T02:59:43.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.768+0000 7f6710664740 10 client.15711 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T02:59:43.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.768+0000 7f6710664740 10 client.15711 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-09-12T02:59:43.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.768+0000 7f6710664740 20 client.15711 populate_metadata read hostname 'smithi005' 2024-09-12T02:59:43.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.768+0000 7f6710664740 1 --2- 172.21.15.5:0/1038491074 >> [v2:172.21.15.148:6834/4195086427,v1:172.21.15.148:6835/4195086427] conn(0x5623b1e0bfd0 0x5623b1e0e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:43.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.768+0000 7f6710664740 4 client.15711 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:59:43.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.768+0000 7f6710664740 1 -- 172.21.15.5:0/1038491074 --> [v2:172.21.15.148:6834/4195086427,v1:172.21.15.148:6835/4195086427] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5623b1d5ed70 con 0x5623b1e0bfd0 2024-09-12T02:59:43.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.769+0000 7f6707fff640 1 --2- 172.21.15.5:0/1038491074 >> [v2:172.21.15.148:6834/4195086427,v1:172.21.15.148:6835/4195086427] conn(0x5623b1e0bfd0 0x5623b1e0e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:43.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.769+0000 7f6707fff640 1 --2- 172.21.15.5:0/1038491074 >> [v2:172.21.15.148:6834/4195086427,v1:172.21.15.148:6835/4195086427] conn(0x5623b1e0bfd0 0x5623b1e0e470 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9deedf2f63443fc6 server_cookie=f88ce741f315702a in_seq=0 out_seq=0 2024-09-12T02:59:43.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.769+0000 7f66e77fe640 10 client.15711 ms_handle_connect on v2:172.21.15.148:6834/4195086427 2024-09-12T02:59:43.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.770+0000 7f66e77fe640 1 -- 172.21.15.5:0/1038491074 <== mds.2 v2:172.21.15.148:6834/4195086427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5623b1d5ed70 con 0x5623b1e0bfd0 2024-09-12T02:59:43.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.770+0000 7f66e77fe640 10 client.15711 handle_command_reply: tid=0 2024-09-12T02:59:43.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.770+0000 7f6710664740 1 client.15711 shutdown 2024-09-12T02:59:43.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.770+0000 7f6710664740 1 -- 172.21.15.5:0/1038491074 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623b1dfc370 msgr2=0x5623b1df9610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:43.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.770+0000 7f6710664740 1 --2- 172.21.15.5:0/1038491074 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623b1dfc370 0x5623b1df9610 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f6700012fb0 tx=0x7f6700004a40 comp rx=0 tx=0).stop 2024-09-12T02:59:43.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.770+0000 7f6710664740 1 -- 172.21.15.5:0/1038491074 shutdown_connections 2024-09-12T02:59:43.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.770+0000 7f6710664740 1 --2- 172.21.15.5:0/1038491074 >> [v2:172.21.15.148:6834/4195086427,v1:172.21.15.148:6835/4195086427] conn(0x5623b1e0bfd0 0x5623b1e0e470 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:43.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.770+0000 7f66e77fe640 0 client.15711 ms_handle_reset on v2:172.21.15.148:6834/4195086427 2024-09-12T02:59:43.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.770+0000 7f6710664740 1 --2- 172.21.15.5:0/1038491074 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623b1dfc370 0x5623b1df9610 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:43.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.770+0000 7f6710664740 1 --2- 172.21.15.5:0/1038491074 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5623b1df2b40 0x5623b1df9020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:43.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.770+0000 7f6710664740 1 --2- 172.21.15.5:0/1038491074 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5623b1df15d0 0x5623b1dfbd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:43.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.770+0000 7f6710664740 1 -- 172.21.15.5:0/1038491074 >> 172.21.15.5:0/1038491074 conn(0x5623b1dba310 msgr2=0x5623b1df3990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:59:43.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.770+0000 7f6710664740 1 -- 172.21.15.5:0/1038491074 shutdown_connections 2024-09-12T02:59:43.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.770+0000 7f6710664740 1 -- 172.21.15.5:0/1038491074 wait complete. 2024-09-12T02:59:43.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.770+0000 7f6710664740 20 client.15711 trim_cache size 0 max 16384 2024-09-12T02:59:43.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.784+0000 7f6710664740 1 Processor -- start 2024-09-12T02:59:43.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.784+0000 7f6710664740 1 -- start start 2024-09-12T02:59:43.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.784+0000 7f6710664740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5623b1df15d0 0x5623b1e082b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:43.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.784+0000 7f6710664740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623b1df2b40 0x5623b1df4180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:43.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.784+0000 7f6710664740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5623b1dfc370 0x5623b1df46c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:43.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.784+0000 7f6710664740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f67081af710 con 0x5623b1df2b40 2024-09-12T02:59:43.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.784+0000 7f6710664740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f67081b12b0 con 0x5623b1df15d0 2024-09-12T02:59:43.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.784+0000 7f6710664740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f67081afde0 con 0x5623b1dfc370 2024-09-12T02:59:43.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.784+0000 7f66fffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623b1df2b40 0x5623b1df4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:43.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.784+0000 7f66fffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623b1df2b40 0x5623b1df4180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39530/0 (socket says 172.21.15.5:39530) 2024-09-12T02:59:43.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.784+0000 7f66fffff640 1 -- 172.21.15.5:0/3888729484 learned_addr learned my addr 172.21.15.5:0/3888729484 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:59:43.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.785+0000 7f6707fff640 1 --2- 172.21.15.5:0/3888729484 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5623b1df15d0 0x5623b1e082b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:43.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.785+0000 7f66fffff640 1 -- 172.21.15.5:0/3888729484 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5623b1dfc370 msgr2=0x5623b1df46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:43.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.785+0000 7f66fffff640 1 --2- 172.21.15.5:0/3888729484 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5623b1dfc370 0x5623b1df46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:43.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.785+0000 7f66fffff640 1 -- 172.21.15.5:0/3888729484 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5623b1df15d0 msgr2=0x5623b1e082b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:43.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.785+0000 7f66fffff640 1 --2- 172.21.15.5:0/3888729484 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5623b1df15d0 0x5623b1e082b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:43.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.785+0000 7f66fffff640 1 -- 172.21.15.5:0/3888729484 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623b1dee960 con 0x5623b1df2b40 2024-09-12T02:59:43.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.785+0000 7f6707fff640 1 --2- 172.21.15.5:0/3888729484 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5623b1df15d0 0x5623b1e082b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:59:43.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.785+0000 7f66fffff640 1 --2- 172.21.15.5:0/3888729484 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623b1df2b40 0x5623b1df4180 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f66f0097780 tx=0x7f66f00cc550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edcfa9020186a6a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:59:43.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.785+0000 7f66fdffb640 1 -- 172.21.15.5:0/3888729484 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66f0020050 con 0x5623b1df2b40 2024-09-12T02:59:43.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.785+0000 7f66fdffb640 1 -- 172.21.15.5:0/3888729484 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66f009ba20 con 0x5623b1df2b40 2024-09-12T02:59:43.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.785+0000 7f66fdffb640 1 -- 172.21.15.5:0/3888729484 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66f0097990 con 0x5623b1df2b40 2024-09-12T02:59:43.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.786+0000 7f6710664740 1 -- 172.21.15.5:0/3888729484 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623b1df2b40 msgr2=0x5623b1df4180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:43.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.786+0000 7f6710664740 1 --2- 172.21.15.5:0/3888729484 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623b1df2b40 0x5623b1df4180 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f66f0097780 tx=0x7f66f00cc550 comp rx=0 tx=0).stop 2024-09-12T02:59:43.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.786+0000 7f6710664740 1 -- 172.21.15.5:0/3888729484 shutdown_connections 2024-09-12T02:59:43.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.786+0000 7f6710664740 1 --2- 172.21.15.5:0/3888729484 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623b1df2b40 0x5623b1df4180 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:43.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.786+0000 7f6710664740 1 --2- 172.21.15.5:0/3888729484 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5623b1dfc370 0x5623b1df46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:43.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.786+0000 7f6710664740 1 --2- 172.21.15.5:0/3888729484 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5623b1df15d0 0x5623b1e082b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:43.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.786+0000 7f6710664740 1 -- 172.21.15.5:0/3888729484 >> 172.21.15.5:0/3888729484 conn(0x5623b1dba310 msgr2=0x5623b1df8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:59:43.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.786+0000 7f6710664740 1 -- 172.21.15.5:0/3888729484 shutdown_connections 2024-09-12T02:59:43.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.786+0000 7f6710664740 1 -- 172.21.15.5:0/3888729484 wait complete. 2024-09-12T02:59:43.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.786+0000 7f6710664740 1 -- start start 2024-09-12T02:59:43.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.786+0000 7f6710664740 1 Processor -- start 2024-09-12T02:59:43.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.787+0000 7f6710664740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5623b1df15d0 0x5623b1e049d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:43.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.787+0000 7f6710664740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5623b1df2b40 0x5623b1df5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:43.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.787+0000 7f6710664740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623b1dfc370 0x5623b1df6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:43.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.787+0000 7f6710664740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f67081b0b80 con 0x5623b1dfc370 2024-09-12T02:59:43.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.787+0000 7f6710664740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f67081b04b0 con 0x5623b1df15d0 2024-09-12T02:59:43.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.787+0000 7f6710664740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f67081ad6f0 con 0x5623b1df2b40 2024-09-12T02:59:43.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.787+0000 7f6710664740 10 client.0 _ll_register_callbacks cb 0x7f67000128c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:59:43.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.787+0000 7f670c828640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623b1dfc370 0x5623b1df6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:43.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.787+0000 7f670c828640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623b1dfc370 0x5623b1df6100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39542/0 (socket says 172.21.15.5:39542) 2024-09-12T02:59:43.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.787+0000 7f670c828640 1 -- 172.21.15.5:0/2905685985 learned_addr learned my addr 172.21.15.5:0/2905685985 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:59:43.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.787+0000 7f6707fff640 1 --2- 172.21.15.5:0/2905685985 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5623b1df15d0 0x5623b1e049d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:43.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.787+0000 7f66fffff640 1 --2- 172.21.15.5:0/2905685985 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5623b1df2b40 0x5623b1df5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:43.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.787+0000 7f670c828640 1 -- 172.21.15.5:0/2905685985 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5623b1df2b40 msgr2=0x5623b1df5bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:43.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.787+0000 7f670c828640 1 --2- 172.21.15.5:0/2905685985 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5623b1df2b40 0x5623b1df5bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:43.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.787+0000 7f670c828640 1 -- 172.21.15.5:0/2905685985 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5623b1df15d0 msgr2=0x5623b1e049d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:43.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.787+0000 7f670c828640 1 --2- 172.21.15.5:0/2905685985 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5623b1df15d0 0x5623b1e049d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:43.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.787+0000 7f670c828640 1 -- 172.21.15.5:0/2905685985 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623b1df6770 con 0x5623b1dfc370 2024-09-12T02:59:43.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.787+0000 7f670c828640 1 --2- 172.21.15.5:0/2905685985 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623b1dfc370 0x5623b1df6100 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f67000092f0 tx=0x7f67000094e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=518049d6b4df75b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:59:43.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.787+0000 7f66e77fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T02:59:43.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.788+0000 7f66e77fe640 1 -- 172.21.15.5:0/2905685985 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6700039070 con 0x5623b1dfc370 2024-09-12T02:59:43.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.788+0000 7f66e77fe640 1 -- 172.21.15.5:0/2905685985 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f6700010ce0 con 0x5623b1dfc370 2024-09-12T02:59:43.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.788+0000 7f6710664740 1 -- 172.21.15.5:0/2905685985 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66f009ba20 con 0x5623b1dfc370 2024-09-12T02:59:43.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.788+0000 7f66e77fe640 1 -- 172.21.15.5:0/2905685985 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6700045050 con 0x5623b1dfc370 2024-09-12T02:59:43.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.788+0000 7f6710664740 10 client.15717 fetch_fsmap learned FSMap version 27 2024-09-12T02:59:43.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.788+0000 7f6710664740 1 -- 172.21.15.5:0/2905685985 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5623b1e76b80 con 0x5623b1dfc370 2024-09-12T02:59:43.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.788+0000 7f66e77fe640 1 -- 172.21.15.5:0/2905685985 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f66f009ba20 con 0x5623b1dfc370 2024-09-12T02:59:43.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.788+0000 7f66e77fe640 1 -- 172.21.15.5:0/2905685985 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7f670002a720 con 0x5623b1dfc370 2024-09-12T02:59:43.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.788+0000 7f6710664740 10 client.15717 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T02:59:43.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.788+0000 7f6710664740 10 client.15717 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-09-12T02:59:43.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.788+0000 7f6710664740 20 client.15717 populate_metadata read hostname 'smithi005' 2024-09-12T02:59:43.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.788+0000 7f6710664740 1 --2- 172.21.15.5:0/2905685985 >> [v2:172.21.15.148:6834/4195086427,v1:172.21.15.148:6835/4195086427] conn(0x5623b1e772d0 0x5623b1e796f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:43.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.788+0000 7f6710664740 4 client.15717 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:59:43.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.788+0000 7f6710664740 1 -- 172.21.15.5:0/2905685985 --> [v2:172.21.15.148:6834/4195086427,v1:172.21.15.148:6835/4195086427] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5623b1cdb170 con 0x5623b1e772d0 2024-09-12T02:59:43.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.788+0000 7f6707fff640 1 --2- 172.21.15.5:0/2905685985 >> [v2:172.21.15.148:6834/4195086427,v1:172.21.15.148:6835/4195086427] conn(0x5623b1e772d0 0x5623b1e796f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:43.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.789+0000 7f6707fff640 1 --2- 172.21.15.5:0/2905685985 >> [v2:172.21.15.148:6834/4195086427,v1:172.21.15.148:6835/4195086427] conn(0x5623b1e772d0 0x5623b1e796f0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7e7683cbb3b546ef server_cookie=7abeec242e08b634 in_seq=0 out_seq=0 2024-09-12T02:59:43.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.789+0000 7f66e77fe640 10 client.15717 ms_handle_connect on v2:172.21.15.148:6834/4195086427 2024-09-12T02:59:43.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.789+0000 7f66e77fe640 1 -- 172.21.15.5:0/2905685985 <== mds.2 v2:172.21.15.148:6834/4195086427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5623b1cdb170 con 0x5623b1e772d0 2024-09-12T02:59:43.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.789+0000 7f66e77fe640 10 client.15717 handle_command_reply: tid=0 2024-09-12T02:59:43.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.789+0000 7f6710664740 1 client.15717 shutdown 2024-09-12T02:59:43.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.789+0000 7f6710664740 1 -- 172.21.15.5:0/2905685985 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623b1dfc370 msgr2=0x5623b1df6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:43.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.789+0000 7f6710664740 1 --2- 172.21.15.5:0/2905685985 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623b1dfc370 0x5623b1df6100 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f67000092f0 tx=0x7f67000094e0 comp rx=0 tx=0).stop 2024-09-12T02:59:43.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.789+0000 7f6710664740 1 -- 172.21.15.5:0/2905685985 shutdown_connections 2024-09-12T02:59:43.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.789+0000 7f6710664740 1 --2- 172.21.15.5:0/2905685985 >> [v2:172.21.15.148:6834/4195086427,v1:172.21.15.148:6835/4195086427] conn(0x5623b1e772d0 0x5623b1e796f0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:43.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.789+0000 7f6710664740 1 --2- 172.21.15.5:0/2905685985 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5623b1dfc370 0x5623b1df6100 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:43.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.789+0000 7f6710664740 1 --2- 172.21.15.5:0/2905685985 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5623b1df2b40 0x5623b1df5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:43.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.789+0000 7f6710664740 1 --2- 172.21.15.5:0/2905685985 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5623b1df15d0 0x5623b1e049d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:43.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.789+0000 7f6710664740 1 -- 172.21.15.5:0/2905685985 >> 172.21.15.5:0/2905685985 conn(0x5623b1dba310 msgr2=0x5623b1dfa880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:59:43.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.789+0000 7f66e77fe640 0 client.15717 ms_handle_reset on v2:172.21.15.148:6834/4195086427 2024-09-12T02:59:43.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.789+0000 7f6710664740 1 -- 172.21.15.5:0/2905685985 shutdown_connections 2024-09-12T02:59:43.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.789+0000 7f6710664740 1 -- 172.21.15.5:0/2905685985 wait complete. 2024-09-12T02:59:43.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.790+0000 7f6710664740 20 client.15717 trim_cache size 0 max 16384 2024-09-12T02:59:43.803 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-12T02:59:43.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.790+0000 7f670e28b640 1 -- 172.21.15.5:0/4261501918 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f66f40a2b80 msgr2=0x7f66f40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:43.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.790+0000 7f670e28b640 1 --2- 172.21.15.5:0/4261501918 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f66f40a2b80 0x7f66f40a5040 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f6700005c40 tx=0x7f6700009280 comp rx=0 tx=0).stop 2024-09-12T02:59:43.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.790+0000 7f670e28b640 1 -- 172.21.15.5:0/4261501918 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6708103700 msgr2=0x7f6708100f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:43.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.790+0000 7f670e28b640 1 --2- 172.21.15.5:0/4261501918 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6708103700 0x7f6708100f80 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f66f000d770 tx=0x7f66f000dc30 comp rx=0 tx=0).stop 2024-09-12T02:59:43.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.790+0000 7f670e28b640 1 -- 172.21.15.5:0/4261501918 shutdown_connections 2024-09-12T02:59:43.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.790+0000 7f670e28b640 1 --2- 172.21.15.5:0/4261501918 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f66f40a2b80 0x7f66f40a5040 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:43.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.790+0000 7f670e28b640 1 --2- 172.21.15.5:0/4261501918 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6708103700 0x7f6708100f80 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:43.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.790+0000 7f670e28b640 1 --2- 172.21.15.5:0/4261501918 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f670810c180 0x7f67081014c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:43.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.790+0000 7f670e28b640 1 --2- 172.21.15.5:0/4261501918 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6708102de0 0x7f6708100a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:43.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.790+0000 7f670e28b640 1 -- 172.21.15.5:0/4261501918 >> 172.21.15.5:0/4261501918 conn(0x7f67080fc7a0 msgr2=0x7f67080fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:59:43.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.790+0000 7f670e28b640 1 -- 172.21.15.5:0/4261501918 shutdown_connections 2024-09-12T02:59:43.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:43.790+0000 7f670e28b640 1 -- 172.21.15.5:0/4261501918 wait complete. 2024-09-12T02:59:44.544 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T02:59:44.840 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:59:45.235 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:59:45 smithi005 ceph-mon[31197]: pgmap v380: 97 pgs: 97 active+clean; 20 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 1.6 MiB/s wr, 36 op/s 2024-09-12T02:59:45.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.319+0000 7fc6b7083640 1 -- 172.21.15.5:0/717122224 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc6b0102ef0 msgr2=0x7fc6b010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:45.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.319+0000 7fc6b7083640 1 --2- 172.21.15.5:0/717122224 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc6b0102ef0 0x7fc6b010da40 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fc6a8009ca0 tx=0x7fc6a802a950 comp rx=0 tx=0).stop 2024-09-12T02:59:45.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.319+0000 7fc6b7083640 1 -- 172.21.15.5:0/717122224 shutdown_connections 2024-09-12T02:59:45.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.319+0000 7fc6b7083640 1 --2- 172.21.15.5:0/717122224 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc6b0102ef0 0x7fc6b010da40 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:45.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.319+0000 7fc6b7083640 1 --2- 172.21.15.5:0/717122224 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc6b0102570 0x7fc6b01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:45.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.319+0000 7fc6b7083640 1 --2- 172.21.15.5:0/717122224 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc6b0100e20 0x7fc6b0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:45.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.319+0000 7fc6b7083640 1 -- 172.21.15.5:0/717122224 >> 172.21.15.5:0/717122224 conn(0x7fc6b00fc7d0 msgr2=0x7fc6b00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:59:45.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.319+0000 7fc6b7083640 1 -- 172.21.15.5:0/717122224 shutdown_connections 2024-09-12T02:59:45.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.319+0000 7fc6b7083640 1 -- 172.21.15.5:0/717122224 wait complete. 2024-09-12T02:59:45.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.319+0000 7fc6b7083640 1 Processor -- start 2024-09-12T02:59:45.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.320+0000 7fc6b7083640 1 -- start start 2024-09-12T02:59:45.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.320+0000 7fc6b7083640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc6b0100e20 0x7fc6b0198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:45.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.320+0000 7fc6b7083640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc6b0102570 0x7fc6b0198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:45.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.320+0000 7fc6b7083640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc6b0102ef0 0x7fc6b019ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:45.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.320+0000 7fc6b7083640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc6b0110fa0 con 0x7fc6b0102570 2024-09-12T02:59:45.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.320+0000 7fc6b7083640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fc6b0110e20 con 0x7fc6b0102ef0 2024-09-12T02:59:45.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.320+0000 7fc6b7083640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc6b0111120 con 0x7fc6b0100e20 2024-09-12T02:59:45.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.320+0000 7fc6a7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc6b0102570 0x7fc6b0198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:45.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.320+0000 7fc6a7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc6b0102570 0x7fc6b0198a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39560/0 (socket says 172.21.15.5:39560) 2024-09-12T02:59:45.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.320+0000 7fc6a7fff640 1 -- 172.21.15.5:0/1685042280 learned_addr learned my addr 172.21.15.5:0/1685042280 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:59:45.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.320+0000 7fc6b4e1f640 1 --2- 172.21.15.5:0/1685042280 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc6b0100e20 0x7fc6b0198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:45.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.320+0000 7fc6b5620640 1 --2- 172.21.15.5:0/1685042280 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc6b0102ef0 0x7fc6b019ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:45.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.320+0000 7fc6a7fff640 1 -- 172.21.15.5:0/1685042280 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc6b0100e20 msgr2=0x7fc6b0198550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:45.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.320+0000 7fc6a7fff640 1 --2- 172.21.15.5:0/1685042280 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc6b0100e20 0x7fc6b0198550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:45.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.320+0000 7fc6a7fff640 1 -- 172.21.15.5:0/1685042280 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc6b0102ef0 msgr2=0x7fc6b019ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:45.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.320+0000 7fc6a7fff640 1 --2- 172.21.15.5:0/1685042280 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc6b0102ef0 0x7fc6b019ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:45.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.320+0000 7fc6a7fff640 1 -- 172.21.15.5:0/1685042280 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6b019d500 con 0x7fc6b0102570 2024-09-12T02:59:45.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.321+0000 7fc6a7fff640 1 --2- 172.21.15.5:0/1685042280 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc6b0102570 0x7fc6b0198a90 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fc6a000b4a0 tx=0x7fc6a000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:59:45.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.321+0000 7fc6a5ffb640 1 -- 172.21.15.5:0/1685042280 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6a0014070 con 0x7fc6b0102570 2024-09-12T02:59:45.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.321+0000 7fc6a5ffb640 1 -- 172.21.15.5:0/1685042280 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6a0010af0 con 0x7fc6b0102570 2024-09-12T02:59:45.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.321+0000 7fc6a5ffb640 1 -- 172.21.15.5:0/1685042280 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6a0017d10 con 0x7fc6b0102570 2024-09-12T02:59:45.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.321+0000 7fc6b7083640 1 -- 172.21.15.5:0/1685042280 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6b019d7f0 con 0x7fc6b0102570 2024-09-12T02:59:45.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.321+0000 7fc6b7083640 1 -- 172.21.15.5:0/1685042280 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6b01a50d0 con 0x7fc6b0102570 2024-09-12T02:59:45.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.323+0000 7fc6a5ffb640 1 -- 172.21.15.5:0/1685042280 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fc6a0017070 con 0x7fc6b0102570 2024-09-12T02:59:45.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.324+0000 7fc6b945c740 1 Processor -- start 2024-09-12T02:59:45.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.324+0000 7fc6b945c740 1 -- start start 2024-09-12T02:59:45.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.324+0000 7fc6b945c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557938da8b40 0x557938da7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:45.326 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.324+0000 7fc6b945c740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557938da75d0 0x557938da7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:45.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.324+0000 7fc6b945c740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557938db2370 0x557938db4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:45.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.324+0000 7fc6b945c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557938d0d2f0 con 0x557938da8b40 2024-09-12T02:59:45.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.324+0000 7fc6b945c740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x557938cb4cf0 con 0x557938db2370 2024-09-12T02:59:45.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.324+0000 7fc6a5ffb640 1 --2- 172.21.15.5:0/1685042280 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fc6980a2c00 0x7fc6980a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:45.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.324+0000 7fc6b945c740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x557938d092b0 con 0x557938da75d0 2024-09-12T02:59:45.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.324+0000 7fc6b5620640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557938da8b40 0x557938da7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:45.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.324+0000 7fc6b5620640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557938da8b40 0x557938da7090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39576/0 (socket says 172.21.15.5:39576) 2024-09-12T02:59:45.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.324+0000 7fc6b5620640 1 -- 172.21.15.5:0/1582039492 learned_addr learned my addr 172.21.15.5:0/1582039492 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:59:45.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.324+0000 7fc6b4e1f640 1 --2- 172.21.15.5:0/1582039492 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557938da75d0 0x557938da7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:45.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.324+0000 7fc6b5620640 1 --2- 172.21.15.5:0/1685042280 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fc6980a2c00 0x7fc6980a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:45.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.324+0000 7fc6a5ffb640 1 -- 172.21.15.5:0/1685042280 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fc6a00ce390 con 0x7fc6b0102570 2024-09-12T02:59:45.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.324+0000 7fc6a7fff640 1 --2- 172.21.15.5:0/1582039492 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557938db2370 0x557938db4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:45.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.324+0000 7fc6b4e1f640 1 -- 172.21.15.5:0/1582039492 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557938db2370 msgr2=0x557938db4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:45.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.325+0000 7fc6b4e1f640 1 --2- 172.21.15.5:0/1582039492 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557938db2370 0x557938db4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:45.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.325+0000 7fc6b4e1f640 1 -- 172.21.15.5:0/1582039492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557938da8b40 msgr2=0x557938da7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:45.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.325+0000 7fc6b4e1f640 1 --2- 172.21.15.5:0/1582039492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557938da8b40 0x557938da7090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:45.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.325+0000 7fc6b4e1f640 1 -- 172.21.15.5:0/1582039492 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557938c993a0 con 0x557938da75d0 2024-09-12T02:59:45.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.325+0000 7fc6a7fff640 1 --2- 172.21.15.5:0/1582039492 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557938db2370 0x557938db4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:59:45.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.325+0000 7fc6b4e1f640 1 --2- 172.21.15.5:0/1582039492 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557938da75d0 0x557938da7a10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc6ac00bbf0 tx=0x7fc6ac0107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5dea488b08d9c1c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:59:45.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.325+0000 7fc6b5620640 1 --2- 172.21.15.5:0/1685042280 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fc6980a2c00 0x7fc6980a50c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fc6a8009a60 tx=0x7fc6a80095c0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:59:45.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.325+0000 7fc696ffd640 1 -- 172.21.15.5:0/1582039492 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6ac004030 con 0x557938da75d0 2024-09-12T02:59:45.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.325+0000 7fc696ffd640 1 -- 172.21.15.5:0/1582039492 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6ac0041d0 con 0x557938da75d0 2024-09-12T02:59:45.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.325+0000 7fc696ffd640 1 -- 172.21.15.5:0/1582039492 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6ac011640 con 0x557938da75d0 2024-09-12T02:59:45.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.326+0000 7fc6b945c740 1 -- 172.21.15.5:0/1582039492 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557938da75d0 msgr2=0x557938da7a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:45.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.326+0000 7fc6b945c740 1 --2- 172.21.15.5:0/1582039492 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557938da75d0 0x557938da7a10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc6ac00bbf0 tx=0x7fc6ac0107a0 comp rx=0 tx=0).stop 2024-09-12T02:59:45.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.326+0000 7fc6b945c740 1 -- 172.21.15.5:0/1582039492 shutdown_connections 2024-09-12T02:59:45.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.326+0000 7fc6b945c740 1 --2- 172.21.15.5:0/1582039492 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557938db2370 0x557938db4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:45.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.326+0000 7fc6b945c740 1 --2- 172.21.15.5:0/1582039492 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557938da75d0 0x557938da7a10 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:45.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.326+0000 7fc6b945c740 1 --2- 172.21.15.5:0/1582039492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557938da8b40 0x557938da7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:45.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.326+0000 7fc6b945c740 1 -- 172.21.15.5:0/1582039492 >> 172.21.15.5:0/1582039492 conn(0x557938d70310 msgr2=0x557938da3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:59:45.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.326+0000 7fc6b945c740 1 -- 172.21.15.5:0/1582039492 shutdown_connections 2024-09-12T02:59:45.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.326+0000 7fc6b945c740 1 -- 172.21.15.5:0/1582039492 wait complete. 2024-09-12T02:59:45.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.326+0000 7fc6b945c740 1 -- start start 2024-09-12T02:59:45.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.326+0000 7fc6b945c740 1 Processor -- start 2024-09-12T02:59:45.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.327+0000 7fc6b945c740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557938da75d0 0x557938db1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:45.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.327+0000 7fc6b945c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557938da8b40 0x557938daeff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:45.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.327+0000 7fc6b945c740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557938db2370 0x557938daf5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:45.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.327+0000 7fc6b945c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557938db5070 con 0x557938da8b40 2024-09-12T02:59:45.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.327+0000 7fc6b945c740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x557938cc8060 con 0x557938da75d0 2024-09-12T02:59:45.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.327+0000 7fc6b945c740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x557938db51f0 con 0x557938db2370 2024-09-12T02:59:45.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.327+0000 7fc6b945c740 10 client.0 _ll_register_callbacks cb 0x557938a1b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:59:45.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.327+0000 7fc6a7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557938da8b40 0x557938daeff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:45.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.327+0000 7fc6a7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557938da8b40 0x557938daeff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39586/0 (socket says 172.21.15.5:39586) 2024-09-12T02:59:45.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.327+0000 7fc6a7fff640 1 -- 172.21.15.5:0/3316347751 learned_addr learned my addr 172.21.15.5:0/3316347751 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:59:45.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.327+0000 7fc6b5620640 1 --2- 172.21.15.5:0/3316347751 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557938db2370 0x557938daf5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:45.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.327+0000 7fc6b4e1f640 1 --2- 172.21.15.5:0/3316347751 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557938da75d0 0x557938db1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:45.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.327+0000 7fc6a7fff640 1 -- 172.21.15.5:0/3316347751 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557938db2370 msgr2=0x557938daf5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:45.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.327+0000 7fc6a7fff640 1 --2- 172.21.15.5:0/3316347751 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557938db2370 0x557938daf5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:45.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.327+0000 7fc6a7fff640 1 -- 172.21.15.5:0/3316347751 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557938da75d0 msgr2=0x557938db1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:45.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.327+0000 7fc6a7fff640 1 --2- 172.21.15.5:0/3316347751 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557938da75d0 0x557938db1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:45.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.327+0000 7fc6a7fff640 1 -- 172.21.15.5:0/3316347751 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557938a7e3c0 con 0x557938da8b40 2024-09-12T02:59:45.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.327+0000 7fc6b4e1f640 1 --2- 172.21.15.5:0/3316347751 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557938da75d0 0x557938db1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:59:45.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.327+0000 7fc6a7fff640 1 --2- 172.21.15.5:0/3316347751 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557938da8b40 0x557938daeff0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fc6a000be60 tx=0x7fc6a00a0230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5a1c55388c03042 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:59:45.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.327+0000 7fc68ffff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T02:59:45.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.328+0000 7fc68ffff640 1 -- 172.21.15.5:0/3316347751 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6a001e040 con 0x557938da8b40 2024-09-12T02:59:45.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.328+0000 7fc68ffff640 1 -- 172.21.15.5:0/3316347751 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fc6a00a0420 con 0x557938da8b40 2024-09-12T02:59:45.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.328+0000 7fc6b945c740 1 -- 172.21.15.5:0/3316347751 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6ac0041d0 con 0x557938da8b40 2024-09-12T02:59:45.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.328+0000 7fc68ffff640 1 -- 172.21.15.5:0/3316347751 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc6a009ac00 con 0x557938da8b40 2024-09-12T02:59:45.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.328+0000 7fc6b945c740 10 client.15729 fetch_fsmap learned FSMap version 27 2024-09-12T02:59:45.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.328+0000 7fc6b945c740 1 -- 172.21.15.5:0/3316347751 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557938dc0a50 con 0x557938da8b40 2024-09-12T02:59:45.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.328+0000 7fc68ffff640 1 -- 172.21.15.5:0/3316347751 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6ac0041d0 con 0x557938da8b40 2024-09-12T02:59:45.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.328+0000 7fc68ffff640 1 -- 172.21.15.5:0/3316347751 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7fc6a0097270 con 0x557938da8b40 2024-09-12T02:59:45.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.328+0000 7fc6b945c740 10 client.15729 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T02:59:45.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.328+0000 7fc6b945c740 10 client.15729 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-12T02:59:45.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.328+0000 7fc6b945c740 20 client.15729 populate_metadata read hostname 'smithi005' 2024-09-12T02:59:45.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.328+0000 7fc6b945c740 1 --2- 172.21.15.5:0/3316347751 >> [v2:172.21.15.148:6836/2606873922,v1:172.21.15.148:6837/2606873922] conn(0x557938dc1df0 0x557938dc4220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:45.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.328+0000 7fc6b945c740 4 client.15729 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:59:45.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.328+0000 7fc6b945c740 1 -- 172.21.15.5:0/3316347751 --> [v2:172.21.15.148:6836/2606873922,v1:172.21.15.148:6837/2606873922] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557938d1a860 con 0x557938dc1df0 2024-09-12T02:59:45.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.328+0000 7fc6b4e1f640 1 --2- 172.21.15.5:0/3316347751 >> [v2:172.21.15.148:6836/2606873922,v1:172.21.15.148:6837/2606873922] conn(0x557938dc1df0 0x557938dc4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:45.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.329+0000 7fc6b4e1f640 1 --2- 172.21.15.5:0/3316347751 >> [v2:172.21.15.148:6836/2606873922,v1:172.21.15.148:6837/2606873922] conn(0x557938dc1df0 0x557938dc4220 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=38738c20721be019 server_cookie=7652dbc61a6b9ad9 in_seq=0 out_seq=0 2024-09-12T02:59:45.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.329+0000 7fc68ffff640 10 client.15729 ms_handle_connect on v2:172.21.15.148:6836/2606873922 2024-09-12T02:59:45.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.330+0000 7fc68ffff640 1 -- 172.21.15.5:0/3316347751 <== mds.1 v2:172.21.15.148:6836/2606873922 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557938d1a860 con 0x557938dc1df0 2024-09-12T02:59:45.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.330+0000 7fc68ffff640 10 client.15729 handle_command_reply: tid=0 2024-09-12T02:59:45.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.330+0000 7fc6b945c740 1 client.15729 shutdown 2024-09-12T02:59:45.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.330+0000 7fc6b945c740 1 -- 172.21.15.5:0/3316347751 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557938da8b40 msgr2=0x557938daeff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:45.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.330+0000 7fc6b945c740 1 --2- 172.21.15.5:0/3316347751 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557938da8b40 0x557938daeff0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fc6a000be60 tx=0x7fc6a00a0230 comp rx=0 tx=0).stop 2024-09-12T02:59:45.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.330+0000 7fc6b945c740 1 -- 172.21.15.5:0/3316347751 shutdown_connections 2024-09-12T02:59:45.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.330+0000 7fc6b945c740 1 --2- 172.21.15.5:0/3316347751 >> [v2:172.21.15.148:6836/2606873922,v1:172.21.15.148:6837/2606873922] conn(0x557938dc1df0 0x557938dc4220 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:45.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.330+0000 7fc6b945c740 1 --2- 172.21.15.5:0/3316347751 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557938da8b40 0x557938daeff0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:45.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.330+0000 7fc6b945c740 1 --2- 172.21.15.5:0/3316347751 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557938db2370 0x557938daf5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:45.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.330+0000 7fc68ffff640 0 client.15729 ms_handle_reset on v2:172.21.15.148:6836/2606873922 2024-09-12T02:59:45.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.330+0000 7fc6b945c740 1 --2- 172.21.15.5:0/3316347751 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557938da75d0 0x557938db1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:45.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.330+0000 7fc6b945c740 1 -- 172.21.15.5:0/3316347751 >> 172.21.15.5:0/3316347751 conn(0x557938d70310 msgr2=0x557938da9960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:59:45.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.330+0000 7fc6b945c740 1 -- 172.21.15.5:0/3316347751 shutdown_connections 2024-09-12T02:59:45.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.330+0000 7fc6b945c740 1 -- 172.21.15.5:0/3316347751 wait complete. 2024-09-12T02:59:45.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.330+0000 7fc6b945c740 20 client.15729 trim_cache size 0 max 16384 2024-09-12T02:59:45.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.343+0000 7fc6b945c740 1 Processor -- start 2024-09-12T02:59:45.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.343+0000 7fc6b945c740 1 -- start start 2024-09-12T02:59:45.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.343+0000 7fc6b945c740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557938da75d0 0x557938dbe310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:45.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.343+0000 7fc6b945c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557938da8b40 0x557938daa120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:45.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.343+0000 7fc6b945c740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557938db2370 0x557938daa660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:45.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.343+0000 7fc6b945c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc6b01ad520 con 0x557938da8b40 2024-09-12T02:59:45.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.343+0000 7fc6b945c740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fc6b01af0c0 con 0x557938db2370 2024-09-12T02:59:45.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.343+0000 7fc6b945c740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc6b01adbf0 con 0x557938da75d0 2024-09-12T02:59:45.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.343+0000 7fc6a7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557938da8b40 0x557938daa120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:45.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.343+0000 7fc6b4e1f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557938da75d0 0x557938dbe310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:45.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.343+0000 7fc6a7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557938da8b40 0x557938daa120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39594/0 (socket says 172.21.15.5:39594) 2024-09-12T02:59:45.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.343+0000 7fc6a7fff640 1 -- 172.21.15.5:0/1274467741 learned_addr learned my addr 172.21.15.5:0/1274467741 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:59:45.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.344+0000 7fc6b5620640 1 --2- 172.21.15.5:0/1274467741 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557938db2370 0x557938daa660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:45.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.344+0000 7fc6a7fff640 1 -- 172.21.15.5:0/1274467741 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557938da75d0 msgr2=0x557938dbe310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:45.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.344+0000 7fc6a7fff640 1 --2- 172.21.15.5:0/1274467741 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557938da75d0 0x557938dbe310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:45.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.344+0000 7fc6a7fff640 1 -- 172.21.15.5:0/1274467741 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557938db2370 msgr2=0x557938daa660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:45.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.344+0000 7fc6a7fff640 1 --2- 172.21.15.5:0/1274467741 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557938db2370 0x557938daa660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:45.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.344+0000 7fc6a7fff640 1 -- 172.21.15.5:0/1274467741 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557938db1960 con 0x557938da8b40 2024-09-12T02:59:45.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.344+0000 7fc6b5620640 1 --2- 172.21.15.5:0/1274467741 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557938db2370 0x557938daa660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:59:45.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.344+0000 7fc6a7fff640 1 --2- 172.21.15.5:0/1274467741 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557938da8b40 0x557938daa120 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fc6a000b470 tx=0x7fc6a0099110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5495e055254732d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:59:45.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.344+0000 7fc6967fc640 1 -- 172.21.15.5:0/1274467741 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6a001e070 con 0x557938da8b40 2024-09-12T02:59:45.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.344+0000 7fc6967fc640 1 -- 172.21.15.5:0/1274467741 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6a00b9020 con 0x557938da8b40 2024-09-12T02:59:45.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.344+0000 7fc6967fc640 1 -- 172.21.15.5:0/1274467741 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6a0097270 con 0x557938da8b40 2024-09-12T02:59:45.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.345+0000 7fc6b945c740 1 -- 172.21.15.5:0/1274467741 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557938da8b40 msgr2=0x557938daa120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:45.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.345+0000 7fc6b945c740 1 --2- 172.21.15.5:0/1274467741 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557938da8b40 0x557938daa120 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fc6a000b470 tx=0x7fc6a0099110 comp rx=0 tx=0).stop 2024-09-12T02:59:45.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.345+0000 7fc6b945c740 1 -- 172.21.15.5:0/1274467741 shutdown_connections 2024-09-12T02:59:45.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.345+0000 7fc6b945c740 1 --2- 172.21.15.5:0/1274467741 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557938da8b40 0x557938daa120 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:45.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.345+0000 7fc6b945c740 1 --2- 172.21.15.5:0/1274467741 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557938db2370 0x557938daa660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:45.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.345+0000 7fc6b945c740 1 --2- 172.21.15.5:0/1274467741 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557938da75d0 0x557938dbe310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:45.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.345+0000 7fc6b945c740 1 -- 172.21.15.5:0/1274467741 >> 172.21.15.5:0/1274467741 conn(0x557938d70310 msgr2=0x557938dae0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:59:45.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.345+0000 7fc6b945c740 1 -- 172.21.15.5:0/1274467741 shutdown_connections 2024-09-12T02:59:45.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.345+0000 7fc6b945c740 1 -- 172.21.15.5:0/1274467741 wait complete. 2024-09-12T02:59:45.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.345+0000 7fc6b945c740 1 -- start start 2024-09-12T02:59:45.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.345+0000 7fc6b945c740 1 Processor -- start 2024-09-12T02:59:45.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.346+0000 7fc6b945c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557938da75d0 0x557938e28460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:45.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.346+0000 7fc6b945c740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557938da8b40 0x557938dba480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:45.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.346+0000 7fc6b945c740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557938db2370 0x557938dbaa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:45.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.346+0000 7fc6b945c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc6b01ae990 con 0x557938da75d0 2024-09-12T02:59:45.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.346+0000 7fc6b945c740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fc6b01ae2c0 con 0x557938db2370 2024-09-12T02:59:45.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.346+0000 7fc6b945c740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc6b01ab500 con 0x557938da8b40 2024-09-12T02:59:45.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.346+0000 7fc6b945c740 10 client.0 _ll_register_callbacks cb 0x7fc6a0096ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:59:45.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.346+0000 7fc6b4e1f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557938da75d0 0x557938e28460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:45.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.346+0000 7fc6b4e1f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557938da75d0 0x557938e28460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39598/0 (socket says 172.21.15.5:39598) 2024-09-12T02:59:45.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.346+0000 7fc6b4e1f640 1 -- 172.21.15.5:0/1570672392 learned_addr learned my addr 172.21.15.5:0/1570672392 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:59:45.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.346+0000 7fc6a7fff640 1 --2- 172.21.15.5:0/1570672392 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557938da8b40 0x557938dba480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:45.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.346+0000 7fc6b5620640 1 --2- 172.21.15.5:0/1570672392 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557938db2370 0x557938dbaa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:45.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.346+0000 7fc6b4e1f640 1 -- 172.21.15.5:0/1570672392 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557938da8b40 msgr2=0x557938dba480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:45.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.346+0000 7fc6b4e1f640 1 --2- 172.21.15.5:0/1570672392 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557938da8b40 0x557938dba480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:45.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.346+0000 7fc6b4e1f640 1 -- 172.21.15.5:0/1570672392 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557938db2370 msgr2=0x557938dbaa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:45.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.346+0000 7fc6b4e1f640 1 --2- 172.21.15.5:0/1570672392 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557938db2370 0x557938dbaa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:45.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.346+0000 7fc6b4e1f640 1 -- 172.21.15.5:0/1570672392 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557938e257b0 con 0x557938da75d0 2024-09-12T02:59:45.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.346+0000 7fc6b4e1f640 1 --2- 172.21.15.5:0/1570672392 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557938da75d0 0x557938e28460 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x557938da4ac0 tx=0x7fc6ac003c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=681d359c0ae20622 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:59:45.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.346+0000 7fc68ffff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T02:59:45.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.347+0000 7fc68ffff640 1 -- 172.21.15.5:0/1570672392 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6ac0037f0 con 0x557938da75d0 2024-09-12T02:59:45.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.347+0000 7fc6b945c740 1 -- 172.21.15.5:0/1570672392 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6a00b9020 con 0x557938da75d0 2024-09-12T02:59:45.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.347+0000 7fc68ffff640 1 -- 172.21.15.5:0/1570672392 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fc6ac003990 con 0x557938da75d0 2024-09-12T02:59:45.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.347+0000 7fc68ffff640 1 -- 172.21.15.5:0/1570672392 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc6ac005040 con 0x557938da75d0 2024-09-12T02:59:45.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.347+0000 7fc6b945c740 10 client.15735 fetch_fsmap learned FSMap version 27 2024-09-12T02:59:45.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.347+0000 7fc6b945c740 1 -- 172.21.15.5:0/1570672392 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x557938e310b0 con 0x557938da75d0 2024-09-12T02:59:45.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.347+0000 7fc68ffff640 1 -- 172.21.15.5:0/1570672392 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6a00b9020 con 0x557938da75d0 2024-09-12T02:59:45.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.347+0000 7fc68ffff640 1 -- 172.21.15.5:0/1570672392 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7fc6ac0288b0 con 0x557938da75d0 2024-09-12T02:59:45.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.347+0000 7fc6b945c740 10 client.15735 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T02:59:45.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.347+0000 7fc6b945c740 10 client.15735 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-12T02:59:45.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.347+0000 7fc6b945c740 20 client.15735 populate_metadata read hostname 'smithi005' 2024-09-12T02:59:45.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.347+0000 7fc6b945c740 1 --2- 172.21.15.5:0/1570672392 >> [v2:172.21.15.148:6836/2606873922,v1:172.21.15.148:6837/2606873922] conn(0x557938e31500 0x557938e338f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:45.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.347+0000 7fc6b945c740 4 client.15735 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:59:45.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.347+0000 7fc6b945c740 1 -- 172.21.15.5:0/1570672392 --> [v2:172.21.15.148:6836/2606873922,v1:172.21.15.148:6837/2606873922] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557938d14d70 con 0x557938e31500 2024-09-12T02:59:45.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.347+0000 7fc6a7fff640 1 --2- 172.21.15.5:0/1570672392 >> [v2:172.21.15.148:6836/2606873922,v1:172.21.15.148:6837/2606873922] conn(0x557938e31500 0x557938e338f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:45.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.348+0000 7fc6a7fff640 1 --2- 172.21.15.5:0/1570672392 >> [v2:172.21.15.148:6836/2606873922,v1:172.21.15.148:6837/2606873922] conn(0x557938e31500 0x557938e338f0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=79c96755441a647 server_cookie=639a696bce9629fc in_seq=0 out_seq=0 2024-09-12T02:59:45.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.348+0000 7fc68ffff640 10 client.15735 ms_handle_connect on v2:172.21.15.148:6836/2606873922 2024-09-12T02:59:45.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.348+0000 7fc68ffff640 1 -- 172.21.15.5:0/1570672392 <== mds.1 v2:172.21.15.148:6836/2606873922 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557938d14d70 con 0x557938e31500 2024-09-12T02:59:45.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.348+0000 7fc68ffff640 10 client.15735 handle_command_reply: tid=0 2024-09-12T02:59:45.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.348+0000 7fc6b945c740 1 client.15735 shutdown 2024-09-12T02:59:45.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.348+0000 7fc6b945c740 1 -- 172.21.15.5:0/1570672392 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557938da75d0 msgr2=0x557938e28460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:45.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.348+0000 7fc6b945c740 1 --2- 172.21.15.5:0/1570672392 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557938da75d0 0x557938e28460 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x557938da4ac0 tx=0x7fc6ac003c90 comp rx=0 tx=0).stop 2024-09-12T02:59:45.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.348+0000 7fc6b945c740 1 -- 172.21.15.5:0/1570672392 shutdown_connections 2024-09-12T02:59:45.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.348+0000 7fc6b945c740 1 --2- 172.21.15.5:0/1570672392 >> [v2:172.21.15.148:6836/2606873922,v1:172.21.15.148:6837/2606873922] conn(0x557938e31500 0x557938e338f0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:45.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.348+0000 7fc6b945c740 1 --2- 172.21.15.5:0/1570672392 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557938db2370 0x557938dbaa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:45.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.348+0000 7fc6b945c740 1 --2- 172.21.15.5:0/1570672392 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557938da8b40 0x557938dba480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:45.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.348+0000 7fc68ffff640 0 client.15735 ms_handle_reset on v2:172.21.15.148:6836/2606873922 2024-09-12T02:59:45.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.348+0000 7fc6b945c740 1 --2- 172.21.15.5:0/1570672392 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557938da75d0 0x557938e28460 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:45.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.348+0000 7fc6b945c740 1 -- 172.21.15.5:0/1570672392 >> 172.21.15.5:0/1570672392 conn(0x557938d70310 msgr2=0x557938da24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:59:45.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.348+0000 7fc6b945c740 1 -- 172.21.15.5:0/1570672392 shutdown_connections 2024-09-12T02:59:45.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.349+0000 7fc6b945c740 1 -- 172.21.15.5:0/1570672392 wait complete. 2024-09-12T02:59:45.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.349+0000 7fc6b945c740 20 client.15735 trim_cache size 0 max 16384 2024-09-12T02:59:45.355 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-12T02:59:45.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.349+0000 7fc6b7083640 1 -- 172.21.15.5:0/1685042280 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fc6980a2c00 msgr2=0x7fc6980a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:45.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.349+0000 7fc6b7083640 1 --2- 172.21.15.5:0/1685042280 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fc6980a2c00 0x7fc6980a50c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fc6a8009a60 tx=0x7fc6a80095c0 comp rx=0 tx=0).stop 2024-09-12T02:59:45.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.349+0000 7fc6b7083640 1 -- 172.21.15.5:0/1685042280 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc6b0102570 msgr2=0x7fc6b0198a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:45.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.349+0000 7fc6b7083640 1 --2- 172.21.15.5:0/1685042280 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc6b0102570 0x7fc6b0198a90 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fc6a000b4a0 tx=0x7fc6a000b960 comp rx=0 tx=0).stop 2024-09-12T02:59:45.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.349+0000 7fc6b7083640 1 -- 172.21.15.5:0/1685042280 shutdown_connections 2024-09-12T02:59:45.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.349+0000 7fc6b7083640 1 --2- 172.21.15.5:0/1685042280 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fc6980a2c00 0x7fc6980a50c0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:45.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.349+0000 7fc6b7083640 1 --2- 172.21.15.5:0/1685042280 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc6b0102570 0x7fc6b0198a90 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:45.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.349+0000 7fc6b7083640 1 --2- 172.21.15.5:0/1685042280 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc6b0102ef0 0x7fc6b019ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:45.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.349+0000 7fc6b7083640 1 --2- 172.21.15.5:0/1685042280 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc6b0100e20 0x7fc6b0198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:45.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.349+0000 7fc6b7083640 1 -- 172.21.15.5:0/1685042280 >> 172.21.15.5:0/1685042280 conn(0x7fc6b00fc7d0 msgr2=0x7fc6b006a210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:59:45.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.349+0000 7fc6b7083640 1 -- 172.21.15.5:0/1685042280 shutdown_connections 2024-09-12T02:59:45.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:45.349+0000 7fc6b7083640 1 -- 172.21.15.5:0/1685042280 wait complete. 2024-09-12T02:59:45.415 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:59:45 smithi148 ceph-mon[35090]: pgmap v380: 97 pgs: 97 active+clean; 20 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 1.6 MiB/s wr, 36 op/s 2024-09-12T02:59:45.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:59:45 smithi146 ceph-mon[35011]: pgmap v380: 97 pgs: 97 active+clean; 20 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 1.6 MiB/s wr, 36 op/s 2024-09-12T02:59:46.007 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T02:59:46.269 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:59:46.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.834+0000 7f6c2ebc4640 1 -- 172.21.15.5:0/152378791 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c28106a60 msgr2=0x7f6c28106e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:46.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.834+0000 7f6c2ebc4640 1 --2- 172.21.15.5:0/152378791 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c28106a60 0x7f6c28106e40 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f6c24009930 tx=0x7f6c2402a420 comp rx=0 tx=0).stop 2024-09-12T02:59:46.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.834+0000 7f6c2ebc4640 1 -- 172.21.15.5:0/152378791 shutdown_connections 2024-09-12T02:59:46.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.834+0000 7f6c2ebc4640 1 --2- 172.21.15.5:0/152378791 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6c28101400 0x7f6c2810d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:46.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.834+0000 7f6c2ebc4640 1 --2- 172.21.15.5:0/152378791 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6c28100a60 0x7f6c28100ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:46.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.834+0000 7f6c2ebc4640 1 --2- 172.21.15.5:0/152378791 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c28106a60 0x7f6c28106e40 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:46.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.834+0000 7f6c2ebc4640 1 -- 172.21.15.5:0/152378791 >> 172.21.15.5:0/152378791 conn(0x7f6c280fc780 msgr2=0x7f6c280feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:59:46.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.834+0000 7f6c2ebc4640 1 -- 172.21.15.5:0/152378791 shutdown_connections 2024-09-12T02:59:46.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.834+0000 7f6c2ebc4640 1 -- 172.21.15.5:0/152378791 wait complete. 2024-09-12T02:59:46.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.834+0000 7f6c2ebc4640 1 Processor -- start 2024-09-12T02:59:46.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.835+0000 7f6c2ebc4640 1 -- start start 2024-09-12T02:59:46.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.835+0000 7f6c2ebc4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6c28100a60 0x7f6c28198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:46.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.835+0000 7f6c2ebc4640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6c28101400 0x7f6c28198b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:46.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.835+0000 7f6c2ebc4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c28106a60 0x7f6c2819cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:46.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.835+0000 7f6c2ebc4640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6c28110f00 con 0x7f6c28106a60 2024-09-12T02:59:46.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.835+0000 7f6c2ebc4640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f6c28110d80 con 0x7f6c28101400 2024-09-12T02:59:46.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.835+0000 7f6c2ebc4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f6c28111080 con 0x7f6c28100a60 2024-09-12T02:59:46.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.835+0000 7f6c2d161640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c28106a60 0x7f6c2819cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:46.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.835+0000 7f6c2d161640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c28106a60 0x7f6c2819cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39614/0 (socket says 172.21.15.5:39614) 2024-09-12T02:59:46.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.835+0000 7f6c2d161640 1 -- 172.21.15.5:0/3680012396 learned_addr learned my addr 172.21.15.5:0/3680012396 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:59:46.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.835+0000 7f6c2c960640 1 --2- 172.21.15.5:0/3680012396 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6c28100a60 0x7f6c28198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:46.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.835+0000 7f6c1ffff640 1 --2- 172.21.15.5:0/3680012396 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6c28101400 0x7f6c28198b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:46.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.835+0000 7f6c2d161640 1 -- 172.21.15.5:0/3680012396 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6c28100a60 msgr2=0x7f6c28198630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:46.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.835+0000 7f6c2d161640 1 --2- 172.21.15.5:0/3680012396 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6c28100a60 0x7f6c28198630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:46.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.835+0000 7f6c2d161640 1 -- 172.21.15.5:0/3680012396 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6c28101400 msgr2=0x7f6c28198b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:46.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.835+0000 7f6c2d161640 1 --2- 172.21.15.5:0/3680012396 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6c28101400 0x7f6c28198b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:46.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.835+0000 7f6c2d161640 1 -- 172.21.15.5:0/3680012396 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c2819d5e0 con 0x7f6c28106a60 2024-09-12T02:59:46.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.835+0000 7f6c1ffff640 1 --2- 172.21.15.5:0/3680012396 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6c28101400 0x7f6c28198b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:59:46.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.836+0000 7f6c2d161640 1 --2- 172.21.15.5:0/3680012396 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c28106a60 0x7f6c2819cf00 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f6c2000b970 tx=0x7f6c2000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:59:46.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.836+0000 7f6c1dffb640 1 -- 172.21.15.5:0/3680012396 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c20019070 con 0x7f6c28106a60 2024-09-12T02:59:46.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.836+0000 7f6c1dffb640 1 -- 172.21.15.5:0/3680012396 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c2000cb40 con 0x7f6c28106a60 2024-09-12T02:59:46.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.836+0000 7f6c2ebc4640 1 -- 172.21.15.5:0/3680012396 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c2819d8d0 con 0x7f6c28106a60 2024-09-12T02:59:46.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.836+0000 7f6c1dffb640 1 -- 172.21.15.5:0/3680012396 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c20007650 con 0x7f6c28106a60 2024-09-12T02:59:46.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.836+0000 7f6c2ebc4640 1 -- 172.21.15.5:0/3680012396 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c281a51b0 con 0x7f6c28106a60 2024-09-12T02:59:46.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.838+0000 7f6c1dffb640 1 -- 172.21.15.5:0/3680012396 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f6c2001c070 con 0x7f6c28106a60 2024-09-12T02:59:46.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.839+0000 7f6c30f9d740 1 Processor -- start 2024-09-12T02:59:46.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.839+0000 7f6c30f9d740 1 -- start start 2024-09-12T02:59:46.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.839+0000 7f6c30f9d740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c19fc3bb40 0x55c19fc3a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:46.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.839+0000 7f6c30f9d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c19fc3a5d0 0x55c19fc3aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:46.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.839+0000 7f6c30f9d740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c19fc45370 0x55c19fc47760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:46.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.839+0000 7f6c30f9d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55c19fba02f0 con 0x55c19fc3a5d0 2024-09-12T02:59:46.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.839+0000 7f6c30f9d740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55c19fb47cf0 con 0x55c19fc45370 2024-09-12T02:59:46.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.839+0000 7f6c30f9d740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55c19fb9c2b0 con 0x55c19fc3bb40 2024-09-12T02:59:46.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.839+0000 7f6c1dffb640 1 --2- 172.21.15.5:0/3680012396 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f6c100a2b80 0x7f6c100a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:46.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.839+0000 7f6c2c960640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c19fc3a5d0 0x55c19fc3aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:46.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.839+0000 7f6c2c960640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c19fc3a5d0 0x55c19fc3aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39630/0 (socket says 172.21.15.5:39630) 2024-09-12T02:59:46.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.839+0000 7f6c2c960640 1 -- 172.21.15.5:0/2893402016 learned_addr learned my addr 172.21.15.5:0/2893402016 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:59:46.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.839+0000 7f6c1ffff640 1 --2- 172.21.15.5:0/2893402016 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c19fc3bb40 0x55c19fc3a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:46.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.839+0000 7f6c2d161640 1 --2- 172.21.15.5:0/3680012396 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f6c100a2b80 0x7f6c100a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:46.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.839+0000 7f6c1ffff640 1 --2- 172.21.15.5:0/2893402016 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c19fc45370 0x55c19fc47760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:46.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.839+0000 7f6c1dffb640 1 -- 172.21.15.5:0/3680012396 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f6c200ce680 con 0x7f6c28106a60 2024-09-12T02:59:46.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.839+0000 7f6c2c960640 1 -- 172.21.15.5:0/2893402016 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c19fc3bb40 msgr2=0x55c19fc3a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:46.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.839+0000 7f6c2c960640 1 --2- 172.21.15.5:0/2893402016 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c19fc3bb40 0x55c19fc3a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:46.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.839+0000 7f6c2c960640 1 -- 172.21.15.5:0/2893402016 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c19fc45370 msgr2=0x55c19fc47760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:46.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.839+0000 7f6c2c960640 1 --2- 172.21.15.5:0/2893402016 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c19fc45370 0x55c19fc47760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:46.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.840+0000 7f6c2c960640 1 -- 172.21.15.5:0/2893402016 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c19fb2c3a0 con 0x55c19fc3a5d0 2024-09-12T02:59:46.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.840+0000 7f6c1ffff640 1 --2- 172.21.15.5:0/2893402016 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c19fc3bb40 0x55c19fc3a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:59:46.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.840+0000 7f6c2c960640 1 --2- 172.21.15.5:0/2893402016 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c19fc3a5d0 0x55c19fc3aa10 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f6c2402a940 tx=0x7f6c240316e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ae2863e302f33fc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:59:46.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.840+0000 7f6c2d161640 1 --2- 172.21.15.5:0/3680012396 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f6c100a2b80 0x7f6c100a5040 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f6c20016000 tx=0x7f6c20065bc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:59:46.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.840+0000 7f6c0effd640 1 -- 172.21.15.5:0/2893402016 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c24040070 con 0x55c19fc3a5d0 2024-09-12T02:59:46.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.840+0000 7f6c0effd640 1 -- 172.21.15.5:0/2893402016 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c24005020 con 0x55c19fc3a5d0 2024-09-12T02:59:46.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.840+0000 7f6c0effd640 1 -- 172.21.15.5:0/2893402016 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c24035040 con 0x55c19fc3a5d0 2024-09-12T02:59:46.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.841+0000 7f6c30f9d740 1 -- 172.21.15.5:0/2893402016 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c19fc3a5d0 msgr2=0x55c19fc3aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:46.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.841+0000 7f6c30f9d740 1 --2- 172.21.15.5:0/2893402016 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c19fc3a5d0 0x55c19fc3aa10 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f6c2402a940 tx=0x7f6c240316e0 comp rx=0 tx=0).stop 2024-09-12T02:59:46.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.841+0000 7f6c30f9d740 1 -- 172.21.15.5:0/2893402016 shutdown_connections 2024-09-12T02:59:46.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.841+0000 7f6c30f9d740 1 --2- 172.21.15.5:0/2893402016 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c19fc3a5d0 0x55c19fc3aa10 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:46.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.841+0000 7f6c30f9d740 1 --2- 172.21.15.5:0/2893402016 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c19fc45370 0x55c19fc47760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:46.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.841+0000 7f6c30f9d740 1 --2- 172.21.15.5:0/2893402016 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c19fc3bb40 0x55c19fc3a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:46.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.841+0000 7f6c30f9d740 1 -- 172.21.15.5:0/2893402016 >> 172.21.15.5:0/2893402016 conn(0x55c19fc03310 msgr2=0x55c19fc36c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:59:46.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.841+0000 7f6c30f9d740 1 -- 172.21.15.5:0/2893402016 shutdown_connections 2024-09-12T02:59:46.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.841+0000 7f6c30f9d740 1 -- 172.21.15.5:0/2893402016 wait complete. 2024-09-12T02:59:46.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.841+0000 7f6c30f9d740 1 -- start start 2024-09-12T02:59:46.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.841+0000 7f6c30f9d740 1 Processor -- start 2024-09-12T02:59:46.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.842+0000 7f6c30f9d740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c19fc3a5d0 0x55c19fc496b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:46.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.842+0000 7f6c30f9d740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c19fc3bb40 0x55c19fc49bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:46.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.842+0000 7f6c30f9d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c19fc45370 0x55c19fc4dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:46.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.842+0000 7f6c30f9d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55c19fc48010 con 0x55c19fc45370 2024-09-12T02:59:46.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.842+0000 7f6c30f9d740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55c19fb5b060 con 0x55c19fc3a5d0 2024-09-12T02:59:46.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.842+0000 7f6c30f9d740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55c19fc48190 con 0x55c19fc3bb40 2024-09-12T02:59:46.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.842+0000 7f6c30f9d740 10 client.0 _ll_register_callbacks cb 0x55c19f8ae8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:59:46.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.842+0000 7f6c1ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c19fc45370 0x55c19fc4dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:46.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.842+0000 7f6c1ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c19fc45370 0x55c19fc4dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39636/0 (socket says 172.21.15.5:39636) 2024-09-12T02:59:46.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.842+0000 7f6c1ffff640 1 -- 172.21.15.5:0/2732936478 learned_addr learned my addr 172.21.15.5:0/2732936478 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:59:46.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.842+0000 7f6c1ffff640 1 --2- 172.21.15.5:0/2732936478 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c19fc3a5d0 0x55c19fc496b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:46.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.842+0000 7f6c2c960640 1 --2- 172.21.15.5:0/2732936478 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c19fc3bb40 0x55c19fc49bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:46.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.842+0000 7f6c1ffff640 1 -- 172.21.15.5:0/2732936478 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c19fc3bb40 msgr2=0x55c19fc49bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:46.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.842+0000 7f6c1ffff640 1 --2- 172.21.15.5:0/2732936478 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c19fc3bb40 0x55c19fc49bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:46.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.842+0000 7f6c1ffff640 1 -- 172.21.15.5:0/2732936478 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c19fc3a5d0 msgr2=0x55c19fc496b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:46.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.842+0000 7f6c1ffff640 1 --2- 172.21.15.5:0/2732936478 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c19fc3a5d0 0x55c19fc496b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:46.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.842+0000 7f6c1ffff640 1 -- 172.21.15.5:0/2732936478 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c19f9113c0 con 0x55c19fc45370 2024-09-12T02:59:46.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.843+0000 7f6c1ffff640 1 --2- 172.21.15.5:0/2732936478 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c19fc45370 0x55c19fc4dd60 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f6c18017da0 tx=0x7f6c180047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9192eeef44c00099 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:59:46.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.843+0000 7f6c07fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T02:59:46.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.843+0000 7f6c07fff640 1 -- 172.21.15.5:0/2732936478 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c1801d060 con 0x55c19fc45370 2024-09-12T02:59:46.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.843+0000 7f6c07fff640 1 -- 172.21.15.5:0/2732936478 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f6c1800ac50 con 0x55c19fc45370 2024-09-12T02:59:46.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.843+0000 7f6c30f9d740 1 -- 172.21.15.5:0/2732936478 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c24005020 con 0x55c19fc45370 2024-09-12T02:59:46.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.843+0000 7f6c07fff640 1 -- 172.21.15.5:0/2732936478 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6c1800ae50 con 0x55c19fc45370 2024-09-12T02:59:46.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.843+0000 7f6c30f9d740 10 client.15747 fetch_fsmap learned FSMap version 27 2024-09-12T02:59:46.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.843+0000 7f6c30f9d740 1 -- 172.21.15.5:0/2732936478 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c19fc57ec0 con 0x55c19fc45370 2024-09-12T02:59:46.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.843+0000 7f6c07fff640 1 -- 172.21.15.5:0/2732936478 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c24005020 con 0x55c19fc45370 2024-09-12T02:59:46.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.843+0000 7f6c07fff640 1 -- 172.21.15.5:0/2732936478 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7f6c18015040 con 0x55c19fc45370 2024-09-12T02:59:46.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.843+0000 7f6c30f9d740 10 client.15747 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T02:59:46.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.843+0000 7f6c30f9d740 10 client.15747 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T02:59:46.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.843+0000 7f6c30f9d740 20 client.15747 populate_metadata read hostname 'smithi005' 2024-09-12T02:59:46.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.843+0000 7f6c30f9d740 1 --2- 172.21.15.5:0/2732936478 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55c19fc59380 0x55c19fc5b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:46.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.843+0000 7f6c30f9d740 4 client.15747 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:59:46.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.844+0000 7f6c30f9d740 1 -- 172.21.15.5:0/2732936478 --> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c19fbad860 con 0x55c19fc59380 2024-09-12T02:59:46.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.844+0000 7f6c2c960640 1 --2- 172.21.15.5:0/2732936478 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55c19fc59380 0x55c19fc5b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:46.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.844+0000 7f6c2c960640 1 --2- 172.21.15.5:0/2732936478 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55c19fc59380 0x55c19fc5b770 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a6e4ba434987ed2 server_cookie=528aed4873960bb9 in_seq=0 out_seq=0 2024-09-12T02:59:46.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.844+0000 7f6c07fff640 10 client.15747 ms_handle_connect on v2:172.21.15.148:6838/2317600000 2024-09-12T02:59:46.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.845+0000 7f6c07fff640 1 -- 172.21.15.5:0/2732936478 <== mds.0 v2:172.21.15.148:6838/2317600000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c19fbad860 con 0x55c19fc59380 2024-09-12T02:59:46.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.845+0000 7f6c07fff640 10 client.15747 handle_command_reply: tid=0 2024-09-12T02:59:46.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.845+0000 7f6c30f9d740 1 client.15747 shutdown 2024-09-12T02:59:46.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.845+0000 7f6c30f9d740 1 -- 172.21.15.5:0/2732936478 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c19fc45370 msgr2=0x55c19fc4dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:46.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.845+0000 7f6c30f9d740 1 --2- 172.21.15.5:0/2732936478 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c19fc45370 0x55c19fc4dd60 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f6c18017da0 tx=0x7f6c180047d0 comp rx=0 tx=0).stop 2024-09-12T02:59:46.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.845+0000 7f6c30f9d740 1 -- 172.21.15.5:0/2732936478 shutdown_connections 2024-09-12T02:59:46.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.845+0000 7f6c30f9d740 1 --2- 172.21.15.5:0/2732936478 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55c19fc59380 0x55c19fc5b770 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:46.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.845+0000 7f6c30f9d740 1 --2- 172.21.15.5:0/2732936478 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c19fc45370 0x55c19fc4dd60 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:46.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.845+0000 7f6c30f9d740 1 --2- 172.21.15.5:0/2732936478 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c19fc3bb40 0x55c19fc49bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:46.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.845+0000 7f6c07fff640 0 client.15747 ms_handle_reset on v2:172.21.15.148:6838/2317600000 2024-09-12T02:59:46.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.845+0000 7f6c30f9d740 1 --2- 172.21.15.5:0/2732936478 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c19fc3a5d0 0x55c19fc496b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:46.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.845+0000 7f6c30f9d740 1 -- 172.21.15.5:0/2732936478 >> 172.21.15.5:0/2732936478 conn(0x55c19fc03310 msgr2=0x55c19fc3c960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:59:46.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.845+0000 7f6c30f9d740 1 -- 172.21.15.5:0/2732936478 shutdown_connections 2024-09-12T02:59:46.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.845+0000 7f6c30f9d740 1 -- 172.21.15.5:0/2732936478 wait complete. 2024-09-12T02:59:46.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.845+0000 7f6c30f9d740 20 client.15747 trim_cache size 0 max 16384 2024-09-12T02:59:46.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.858+0000 7f6c30f9d740 1 Processor -- start 2024-09-12T02:59:46.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.858+0000 7f6c30f9d740 1 -- start start 2024-09-12T02:59:46.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.858+0000 7f6c30f9d740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c19fc3a5d0 0x55c19fc4dd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:46.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.858+0000 7f6c30f9d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c19fc3bb40 0x55c19fc39d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:46.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.858+0000 7f6c30f9d740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c19fc45370 0x55c19fc46760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:46.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.858+0000 7f6c30f9d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6c281ab480 con 0x55c19fc3bb40 2024-09-12T02:59:46.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.858+0000 7f6c30f9d740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f6c281ad020 con 0x55c19fc45370 2024-09-12T02:59:46.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.858+0000 7f6c30f9d740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f6c281abb50 con 0x55c19fc3a5d0 2024-09-12T02:59:46.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.859+0000 7f6c2c960640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c19fc3bb40 0x55c19fc39d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:46.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.859+0000 7f6c2c960640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c19fc3bb40 0x55c19fc39d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39644/0 (socket says 172.21.15.5:39644) 2024-09-12T02:59:46.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.859+0000 7f6c2c960640 1 -- 172.21.15.5:0/3192317549 learned_addr learned my addr 172.21.15.5:0/3192317549 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:59:46.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.859+0000 7f6c1ffff640 1 --2- 172.21.15.5:0/3192317549 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c19fc3a5d0 0x55c19fc4dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:46.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.859+0000 7f6c2c960640 1 -- 172.21.15.5:0/3192317549 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c19fc3a5d0 msgr2=0x55c19fc4dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:46.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.859+0000 7f6c2c960640 1 --2- 172.21.15.5:0/3192317549 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c19fc3a5d0 0x55c19fc4dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:46.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.859+0000 7f6c2c960640 1 -- 172.21.15.5:0/3192317549 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c19fc45370 msgr2=0x55c19fc46760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:46.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.859+0000 7f6c1ffff640 1 --2- 172.21.15.5:0/3192317549 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c19fc45370 0x55c19fc46760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:46.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.859+0000 7f6c2c960640 1 --2- 172.21.15.5:0/3192317549 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c19fc45370 0x55c19fc46760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:46.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.859+0000 7f6c2c960640 1 -- 172.21.15.5:0/3192317549 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c19fc36510 con 0x55c19fc3bb40 2024-09-12T02:59:46.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.859+0000 7f6c1ffff640 1 --2- 172.21.15.5:0/3192317549 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c19fc45370 0x55c19fc46760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T02:59:46.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.859+0000 7f6c2c960640 1 --2- 172.21.15.5:0/3192317549 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c19fc3bb40 0x55c19fc39d00 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f6c24004810 tx=0x7f6c24005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65cbb6b32b621967 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:59:46.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.859+0000 7f6c0e7fc640 1 -- 172.21.15.5:0/3192317549 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c24054070 con 0x55c19fc3bb40 2024-09-12T02:59:46.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.859+0000 7f6c0e7fc640 1 -- 172.21.15.5:0/3192317549 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c2403d6b0 con 0x55c19fc3bb40 2024-09-12T02:59:46.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.859+0000 7f6c0e7fc640 1 -- 172.21.15.5:0/3192317549 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c24040040 con 0x55c19fc3bb40 2024-09-12T02:59:46.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.860+0000 7f6c30f9d740 1 -- 172.21.15.5:0/3192317549 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c19fc3bb40 msgr2=0x55c19fc39d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:46.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.860+0000 7f6c30f9d740 1 --2- 172.21.15.5:0/3192317549 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c19fc3bb40 0x55c19fc39d00 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f6c24004810 tx=0x7f6c24005fd0 comp rx=0 tx=0).stop 2024-09-12T02:59:46.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.860+0000 7f6c30f9d740 1 -- 172.21.15.5:0/3192317549 shutdown_connections 2024-09-12T02:59:46.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.860+0000 7f6c30f9d740 1 --2- 172.21.15.5:0/3192317549 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c19fc3bb40 0x55c19fc39d00 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:46.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.860+0000 7f6c30f9d740 1 --2- 172.21.15.5:0/3192317549 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c19fc45370 0x55c19fc46760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:46.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.860+0000 7f6c30f9d740 1 --2- 172.21.15.5:0/3192317549 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c19fc3a5d0 0x55c19fc4dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:46.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.860+0000 7f6c30f9d740 1 -- 172.21.15.5:0/3192317549 >> 172.21.15.5:0/3192317549 conn(0x55c19fc03310 msgr2=0x55c19fc496b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:59:46.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.860+0000 7f6c30f9d740 1 -- 172.21.15.5:0/3192317549 shutdown_connections 2024-09-12T02:59:46.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.860+0000 7f6c30f9d740 1 -- 172.21.15.5:0/3192317549 wait complete. 2024-09-12T02:59:46.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.860+0000 7f6c30f9d740 1 -- start start 2024-09-12T02:59:46.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.860+0000 7f6c30f9d740 1 Processor -- start 2024-09-12T02:59:46.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.861+0000 7f6c30f9d740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c19fc3a5d0 0x55c19fc42f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:46.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.861+0000 7f6c30f9d740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c19fc3bb40 0x55c19fc434a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:46.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.861+0000 7f6c30f9d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c19fc45370 0x55c19fc439e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:46.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.861+0000 7f6c30f9d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6c281ac8f0 con 0x55c19fc45370 2024-09-12T02:59:46.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.861+0000 7f6c30f9d740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f6c281ac220 con 0x55c19fc3a5d0 2024-09-12T02:59:46.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.861+0000 7f6c30f9d740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f6c281a9480 con 0x55c19fc3bb40 2024-09-12T02:59:46.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.861+0000 7f6c30f9d740 10 client.0 _ll_register_callbacks cb 0x7f6c1800a7c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:59:46.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.861+0000 7f6c1ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c19fc45370 0x55c19fc439e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:46.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.861+0000 7f6c1ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c19fc45370 0x55c19fc439e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39656/0 (socket says 172.21.15.5:39656) 2024-09-12T02:59:46.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.861+0000 7f6c1ffff640 1 -- 172.21.15.5:0/153155568 learned_addr learned my addr 172.21.15.5:0/153155568 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:59:46.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.861+0000 7f6c2c960640 1 --2- 172.21.15.5:0/153155568 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c19fc3bb40 0x55c19fc434a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:46.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.861+0000 7f6c1ffff640 1 --2- 172.21.15.5:0/153155568 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c19fc3a5d0 0x55c19fc42f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:46.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.861+0000 7f6c1ffff640 1 -- 172.21.15.5:0/153155568 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c19fc3bb40 msgr2=0x55c19fc434a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:46.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.861+0000 7f6c1ffff640 1 --2- 172.21.15.5:0/153155568 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c19fc3bb40 0x55c19fc434a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:46.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.861+0000 7f6c1ffff640 1 -- 172.21.15.5:0/153155568 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c19fc3a5d0 msgr2=0x55c19fc42f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:46.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.861+0000 7f6c1ffff640 1 --2- 172.21.15.5:0/153155568 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c19fc3a5d0 0x55c19fc42f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:46.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.861+0000 7f6c1ffff640 1 -- 172.21.15.5:0/153155568 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c19fc52d80 con 0x55c19fc45370 2024-09-12T02:59:46.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.862+0000 7f6c1ffff640 1 --2- 172.21.15.5:0/153155568 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c19fc45370 0x55c19fc439e0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f6c18008000 tx=0x7f6c180047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8df71d1a7d7d578 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:59:46.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.862+0000 7f6c07fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T02:59:46.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.862+0000 7f6c07fff640 1 -- 172.21.15.5:0/153155568 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c18035060 con 0x55c19fc45370 2024-09-12T02:59:46.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.862+0000 7f6c07fff640 1 -- 172.21.15.5:0/153155568 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f6c1800ac50 con 0x55c19fc45370 2024-09-12T02:59:46.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.862+0000 7f6c30f9d740 1 -- 172.21.15.5:0/153155568 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c2403d6b0 con 0x55c19fc45370 2024-09-12T02:59:46.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.862+0000 7f6c07fff640 1 -- 172.21.15.5:0/153155568 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6c18027040 con 0x55c19fc45370 2024-09-12T02:59:46.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.862+0000 7f6c30f9d740 10 client.15753 fetch_fsmap learned FSMap version 27 2024-09-12T02:59:46.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.862+0000 7f6c30f9d740 1 -- 172.21.15.5:0/153155568 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c19fcc4430 con 0x55c19fc45370 2024-09-12T02:59:46.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.862+0000 7f6c07fff640 1 -- 172.21.15.5:0/153155568 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c2403d6b0 con 0x55c19fc45370 2024-09-12T02:59:46.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.862+0000 7f6c07fff640 1 -- 172.21.15.5:0/153155568 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7f6c1802d040 con 0x55c19fc45370 2024-09-12T02:59:46.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.862+0000 7f6c30f9d740 10 client.15753 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T02:59:46.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.862+0000 7f6c30f9d740 10 client.15753 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T02:59:46.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.862+0000 7f6c30f9d740 20 client.15753 populate_metadata read hostname 'smithi005' 2024-09-12T02:59:46.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.862+0000 7f6c30f9d740 1 --2- 172.21.15.5:0/153155568 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55c19fc59380 0x55c19fcc6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:46.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.862+0000 7f6c30f9d740 4 client.15753 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T02:59:46.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.862+0000 7f6c30f9d740 1 -- 172.21.15.5:0/153155568 --> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c19fba7d70 con 0x55c19fc59380 2024-09-12T02:59:46.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.863+0000 7f6c2c960640 1 --2- 172.21.15.5:0/153155568 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55c19fc59380 0x55c19fcc6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:46.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.863+0000 7f6c2c960640 1 --2- 172.21.15.5:0/153155568 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55c19fc59380 0x55c19fcc6b90 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=913b812c6e4f7ce1 server_cookie=9b691ee53734c9cc in_seq=0 out_seq=0 2024-09-12T02:59:46.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.863+0000 7f6c07fff640 10 client.15753 ms_handle_connect on v2:172.21.15.148:6838/2317600000 2024-09-12T02:59:46.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.863+0000 7f6c07fff640 1 -- 172.21.15.5:0/153155568 <== mds.0 v2:172.21.15.148:6838/2317600000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c19fba7d70 con 0x55c19fc59380 2024-09-12T02:59:46.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.863+0000 7f6c07fff640 10 client.15753 handle_command_reply: tid=0 2024-09-12T02:59:46.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.863+0000 7f6c30f9d740 1 client.15753 shutdown 2024-09-12T02:59:46.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.863+0000 7f6c30f9d740 1 -- 172.21.15.5:0/153155568 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c19fc45370 msgr2=0x55c19fc439e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:46.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.863+0000 7f6c30f9d740 1 --2- 172.21.15.5:0/153155568 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c19fc45370 0x55c19fc439e0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f6c18008000 tx=0x7f6c180047d0 comp rx=0 tx=0).stop 2024-09-12T02:59:46.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.863+0000 7f6c30f9d740 1 -- 172.21.15.5:0/153155568 shutdown_connections 2024-09-12T02:59:46.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.863+0000 7f6c30f9d740 1 --2- 172.21.15.5:0/153155568 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55c19fc59380 0x55c19fcc6b90 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:46.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.863+0000 7f6c07fff640 0 client.15753 ms_handle_reset on v2:172.21.15.148:6838/2317600000 2024-09-12T02:59:46.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.863+0000 7f6c30f9d740 1 --2- 172.21.15.5:0/153155568 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c19fc45370 0x55c19fc439e0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:46.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.863+0000 7f6c30f9d740 1 --2- 172.21.15.5:0/153155568 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c19fc3bb40 0x55c19fc434a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:46.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.863+0000 7f6c30f9d740 1 --2- 172.21.15.5:0/153155568 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c19fc3a5d0 0x55c19fc42f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:46.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.863+0000 7f6c30f9d740 1 -- 172.21.15.5:0/153155568 >> 172.21.15.5:0/153155568 conn(0x55c19fc03310 msgr2=0x55c19fc35500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:59:46.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.864+0000 7f6c30f9d740 1 -- 172.21.15.5:0/153155568 shutdown_connections 2024-09-12T02:59:46.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.864+0000 7f6c30f9d740 1 -- 172.21.15.5:0/153155568 wait complete. 2024-09-12T02:59:46.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.864+0000 7f6c30f9d740 20 client.15753 trim_cache size 0 max 16384 2024-09-12T02:59:46.872 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-12T02:59:46.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.864+0000 7f6c2ebc4640 1 -- 172.21.15.5:0/3680012396 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f6c100a2b80 msgr2=0x7f6c100a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:46.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.864+0000 7f6c2ebc4640 1 --2- 172.21.15.5:0/3680012396 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f6c100a2b80 0x7f6c100a5040 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f6c20016000 tx=0x7f6c20065bc0 comp rx=0 tx=0).stop 2024-09-12T02:59:46.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.864+0000 7f6c2ebc4640 1 -- 172.21.15.5:0/3680012396 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c28106a60 msgr2=0x7f6c2819cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:46.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.864+0000 7f6c2ebc4640 1 --2- 172.21.15.5:0/3680012396 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c28106a60 0x7f6c2819cf00 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f6c2000b970 tx=0x7f6c2000be30 comp rx=0 tx=0).stop 2024-09-12T02:59:46.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.864+0000 7f6c2ebc4640 1 -- 172.21.15.5:0/3680012396 shutdown_connections 2024-09-12T02:59:46.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.864+0000 7f6c2ebc4640 1 --2- 172.21.15.5:0/3680012396 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f6c100a2b80 0x7f6c100a5040 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:46.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.864+0000 7f6c2ebc4640 1 --2- 172.21.15.5:0/3680012396 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c28106a60 0x7f6c2819cf00 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:46.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.864+0000 7f6c2ebc4640 1 --2- 172.21.15.5:0/3680012396 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6c28101400 0x7f6c28198b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:46.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.864+0000 7f6c2ebc4640 1 --2- 172.21.15.5:0/3680012396 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6c28100a60 0x7f6c28198630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:46.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.864+0000 7f6c2ebc4640 1 -- 172.21.15.5:0/3680012396 >> 172.21.15.5:0/3680012396 conn(0x7f6c280fc780 msgr2=0x7f6c281049b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:59:46.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.864+0000 7f6c2ebc4640 1 -- 172.21.15.5:0/3680012396 shutdown_connections 2024-09-12T02:59:46.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:46.864+0000 7f6c2ebc4640 1 -- 172.21.15.5:0/3680012396 wait complete. 2024-09-12T02:59:47.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:59:47 smithi005 ceph-mon[31197]: pgmap v381: 97 pgs: 97 active+clean; 20 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 483 KiB/s wr, 33 op/s 2024-09-12T02:59:47.415 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:59:47 smithi148 ceph-mon[35090]: pgmap v381: 97 pgs: 97 active+clean; 20 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 483 KiB/s wr, 33 op/s 2024-09-12T02:59:47.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:59:47 smithi146 ceph-mon[35011]: pgmap v381: 97 pgs: 97 active+clean; 20 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 400 KiB/s rd, 483 KiB/s wr, 33 op/s 2024-09-12T02:59:48.426 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T02:59:48.426 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T02:59:48.690 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:59:49.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:59:49 smithi005 ceph-mon[31197]: pgmap v382: 97 pgs: 97 active+clean; 20 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 523 KiB/s wr, 33 op/s 2024-09-12T02:59:49.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.153+0000 7fd5bdd20640 1 -- 172.21.15.5:0/2021369073 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd5b8101dd0 msgr2=0x7fd5b81021b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:49.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.153+0000 7fd5bdd20640 1 --2- 172.21.15.5:0/2021369073 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd5b8101dd0 0x7fd5b81021b0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fd5a8009900 tx=0x7fd5a802a410 comp rx=0 tx=0).stop 2024-09-12T02:59:49.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.153+0000 7fd5bdd20640 1 -- 172.21.15.5:0/2021369073 shutdown_connections 2024-09-12T02:59:49.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.153+0000 7fd5bdd20640 1 --2- 172.21.15.5:0/2021369073 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd5b8106f00 0x7fd5b810ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:49.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.153+0000 7fd5bdd20640 1 --2- 172.21.15.5:0/2021369073 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5b8102780 0x7fd5b81067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:49.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.153+0000 7fd5bdd20640 1 --2- 172.21.15.5:0/2021369073 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd5b8101dd0 0x7fd5b81021b0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:49.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.153+0000 7fd5bdd20640 1 -- 172.21.15.5:0/2021369073 >> 172.21.15.5:0/2021369073 conn(0x7fd5b80fd8b0 msgr2=0x7fd5b80ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:59:49.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.153+0000 7fd5bdd20640 1 -- 172.21.15.5:0/2021369073 shutdown_connections 2024-09-12T02:59:49.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.153+0000 7fd5bdd20640 1 -- 172.21.15.5:0/2021369073 wait complete. 2024-09-12T02:59:49.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.153+0000 7fd5bdd20640 1 Processor -- start 2024-09-12T02:59:49.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.154+0000 7fd5bdd20640 1 -- start start 2024-09-12T02:59:49.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.154+0000 7fd5bdd20640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5b8101dd0 0x7fd5b80744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:49.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.154+0000 7fd5bdd20640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd5b8102780 0x7fd5b8074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:49.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.154+0000 7fd5bdd20640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd5b8106f00 0x7fd5b8074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:49.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.154+0000 7fd5bdd20640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd5b8111f60 con 0x7fd5b8106f00 2024-09-12T02:59:49.157 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.154+0000 7fd5bdd20640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fd5b8111de0 con 0x7fd5b8102780 2024-09-12T02:59:49.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.154+0000 7fd5bdd20640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd5b81120e0 con 0x7fd5b8101dd0 2024-09-12T02:59:49.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.154+0000 7fd5b77fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5b8101dd0 0x7fd5b80744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:49.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.154+0000 7fd5b77fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5b8101dd0 0x7fd5b80744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.5:41424/0 (socket says 172.21.15.5:41424) 2024-09-12T02:59:49.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.154+0000 7fd5b77fe640 1 -- 172.21.15.5:0/2888710842 learned_addr learned my addr 172.21.15.5:0/2888710842 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:59:49.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.154+0000 7fd5b7fff640 1 --2- 172.21.15.5:0/2888710842 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd5b8106f00 0x7fd5b8074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:49.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.154+0000 7fd5b6ffd640 1 --2- 172.21.15.5:0/2888710842 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd5b8102780 0x7fd5b8074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:49.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.155+0000 7fd5b77fe640 1 -- 172.21.15.5:0/2888710842 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd5b8102780 msgr2=0x7fd5b8074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:49.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.155+0000 7fd5b77fe640 1 --2- 172.21.15.5:0/2888710842 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd5b8102780 0x7fd5b8074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:49.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.155+0000 7fd5b77fe640 1 -- 172.21.15.5:0/2888710842 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd5b8106f00 msgr2=0x7fd5b8074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:49.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.155+0000 7fd5b77fe640 1 --2- 172.21.15.5:0/2888710842 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd5b8106f00 0x7fd5b8074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:49.158 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.155+0000 7fd5b77fe640 1 -- 172.21.15.5:0/2888710842 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5b81a6930 con 0x7fd5b8101dd0 2024-09-12T02:59:49.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.155+0000 7fd5b6ffd640 1 --2- 172.21.15.5:0/2888710842 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd5b8102780 0x7fd5b8074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T02:59:49.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.155+0000 7fd5b77fe640 1 --2- 172.21.15.5:0/2888710842 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5b8101dd0 0x7fd5b80744f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd5a8006010 tx=0x7fd5a8004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:59:49.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.155+0000 7fd5b4ff9640 1 -- 172.21.15.5:0/2888710842 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5a803f030 con 0x7fd5b8101dd0 2024-09-12T02:59:49.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.155+0000 7fd5b4ff9640 1 -- 172.21.15.5:0/2888710842 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5a8004440 con 0x7fd5b8101dd0 2024-09-12T02:59:49.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.155+0000 7fd5b4ff9640 1 -- 172.21.15.5:0/2888710842 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5a8035070 con 0x7fd5b8101dd0 2024-09-12T02:59:49.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.155+0000 7fd5bdd20640 1 -- 172.21.15.5:0/2888710842 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5b81a6bc0 con 0x7fd5b8101dd0 2024-09-12T02:59:49.159 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.156+0000 7fd5bdd20640 1 -- 172.21.15.5:0/2888710842 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5b81a70f0 con 0x7fd5b8101dd0 2024-09-12T02:59:49.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.158+0000 7fd5b4ff9640 1 -- 172.21.15.5:0/2888710842 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fd5a8042030 con 0x7fd5b8101dd0 2024-09-12T02:59:49.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.159+0000 7fd5b4ff9640 1 --2- 172.21.15.5:0/2888710842 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fd5a00a2c00 0x7fd5a00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:49.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.159+0000 7fd5b6ffd640 1 --2- 172.21.15.5:0/2888710842 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fd5a00a2c00 0x7fd5a00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:49.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.159+0000 7fd5b4ff9640 1 -- 172.21.15.5:0/2888710842 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fd5a80ef3a0 con 0x7fd5b8101dd0 2024-09-12T02:59:49.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.159+0000 7fd5c00f9740 1 Processor -- start 2024-09-12T02:59:49.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.159+0000 7fd5c00f9740 1 -- start start 2024-09-12T02:59:49.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.160+0000 7fd5b6ffd640 1 --2- 172.21.15.5:0/2888710842 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fd5a00a2c00 0x7fd5a00a50c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fd5b8075f80 tx=0x7fd5ac005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:59:49.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.160+0000 7fd5c00f9740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d20110cb60 0x55d20110cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:49.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.160+0000 7fd5c00f9740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d20110d480 0x55d201117800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:49.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.160+0000 7fd5c00f9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d201117d40 0x55d20111a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:49.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.160+0000 7fd5c00f9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d201072280 con 0x55d201117d40 2024-09-12T02:59:49.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.160+0000 7fd5c00f9740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55d201019b10 con 0x55d20110cb60 2024-09-12T02:59:49.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.160+0000 7fd5c00f9740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55d20106e240 con 0x55d20110d480 2024-09-12T02:59:49.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.160+0000 7fd5b6ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d20110d480 0x55d201117800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:49.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.160+0000 7fd5b7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d201117d40 0x55d20111a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:49.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.160+0000 7fd5b6ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d20110d480 0x55d201117800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.5:41432/0 (socket says 172.21.15.5:41432) 2024-09-12T02:59:49.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.160+0000 7fd5b6ffd640 1 -- 172.21.15.5:0/3797149799 learned_addr learned my addr 172.21.15.5:0/3797149799 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:59:49.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.160+0000 7fd5b7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d201117d40 0x55d20111a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39696/0 (socket says 172.21.15.5:39696) 2024-09-12T02:59:49.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.160+0000 7fd5b77fe640 1 --2- 172.21.15.5:0/3797149799 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d20110cb60 0x55d20110cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:49.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.160+0000 7fd5b6ffd640 1 -- 172.21.15.5:0/3797149799 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d20110cb60 msgr2=0x55d20110cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:49.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.160+0000 7fd5b6ffd640 1 --2- 172.21.15.5:0/3797149799 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d20110cb60 0x55d20110cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:49.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.160+0000 7fd5b6ffd640 1 -- 172.21.15.5:0/3797149799 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d201117d40 msgr2=0x55d20111a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:49.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.160+0000 7fd5b6ffd640 1 --2- 172.21.15.5:0/3797149799 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d201117d40 0x55d20111a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:49.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.160+0000 7fd5b6ffd640 1 -- 172.21.15.5:0/3797149799 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2010261f0 con 0x55d20110d480 2024-09-12T02:59:49.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.160+0000 7fd5b7fff640 1 --2- 172.21.15.5:0/3797149799 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d201117d40 0x55d20111a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:59:49.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.161+0000 7fd5b6ffd640 1 --2- 172.21.15.5:0/3797149799 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d20110d480 0x55d201117800 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac004130 tx=0x7fd5ac014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ddba68a27b87ad2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:59:49.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.161+0000 7fd59dffb640 1 -- 172.21.15.5:0/3797149799 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5ac003db0 con 0x55d20110d480 2024-09-12T02:59:49.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.161+0000 7fd59dffb640 1 -- 172.21.15.5:0/3797149799 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5ac025070 con 0x55d20110d480 2024-09-12T02:59:49.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.161+0000 7fd59dffb640 1 -- 172.21.15.5:0/3797149799 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5ac0207c0 con 0x55d20110d480 2024-09-12T02:59:49.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.162+0000 7fd5c00f9740 1 -- 172.21.15.5:0/3797149799 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d20110d480 msgr2=0x55d201117800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:49.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.162+0000 7fd5c00f9740 1 --2- 172.21.15.5:0/3797149799 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d20110d480 0x55d201117800 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac004130 tx=0x7fd5ac014000 comp rx=0 tx=0).stop 2024-09-12T02:59:49.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.162+0000 7fd5c00f9740 1 -- 172.21.15.5:0/3797149799 shutdown_connections 2024-09-12T02:59:49.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.162+0000 7fd5c00f9740 1 --2- 172.21.15.5:0/3797149799 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d201117d40 0x55d20111a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:49.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.162+0000 7fd5c00f9740 1 --2- 172.21.15.5:0/3797149799 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d20110d480 0x55d201117800 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:49.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.162+0000 7fd5c00f9740 1 --2- 172.21.15.5:0/3797149799 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d20110cb60 0x55d20110cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:49.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.162+0000 7fd5c00f9740 1 -- 172.21.15.5:0/3797149799 >> 172.21.15.5:0/3797149799 conn(0x55d2011090f0 msgr2=0x55d2011094d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:59:49.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.162+0000 7fd5c00f9740 1 -- 172.21.15.5:0/3797149799 shutdown_connections 2024-09-12T02:59:49.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.162+0000 7fd5c00f9740 1 -- 172.21.15.5:0/3797149799 wait complete. 2024-09-12T02:59:49.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.163+0000 7fd5c00f9740 1 -- start start 2024-09-12T02:59:49.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.163+0000 7fd5c00f9740 1 Processor -- start 2024-09-12T02:59:49.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.163+0000 7fd5c00f9740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d20110cb60 0x55d201117770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:49.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.163+0000 7fd5c00f9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d20110d480 0x55d201114ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:49.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.163+0000 7fd5c00f9740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d201117d40 0x55d201115250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:49.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.163+0000 7fd5c00f9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d20111a910 con 0x55d20110d480 2024-09-12T02:59:49.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.163+0000 7fd5c00f9740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55d20102cf10 con 0x55d201117d40 2024-09-12T02:59:49.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.163+0000 7fd5c00f9740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55d20111aa90 con 0x55d20110cb60 2024-09-12T02:59:49.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.163+0000 7fd5c00f9740 10 client.0 _ll_register_callbacks cb 0x55d200ff5a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:59:49.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.163+0000 7fd5b77fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d20110cb60 0x55d201117770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:49.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.163+0000 7fd5b77fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d20110cb60 0x55d201117770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.5:41448/0 (socket says 172.21.15.5:41448) 2024-09-12T02:59:49.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.163+0000 7fd5b77fe640 1 -- 172.21.15.5:0/539787379 learned_addr learned my addr 172.21.15.5:0/539787379 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:59:49.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.163+0000 7fd5b6ffd640 1 --2- 172.21.15.5:0/539787379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d20110d480 0x55d201114ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:49.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.163+0000 7fd5b7fff640 1 --2- 172.21.15.5:0/539787379 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d201117d40 0x55d201115250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:49.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.164+0000 7fd5b6ffd640 1 -- 172.21.15.5:0/539787379 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d20110cb60 msgr2=0x55d201117770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:49.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.164+0000 7fd5b6ffd640 1 --2- 172.21.15.5:0/539787379 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d20110cb60 0x55d201117770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:49.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.164+0000 7fd5b6ffd640 1 -- 172.21.15.5:0/539787379 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d201117d40 msgr2=0x55d201115250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:49.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.164+0000 7fd5b6ffd640 1 --2- 172.21.15.5:0/539787379 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d201117d40 0x55d201115250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:49.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.164+0000 7fd5b6ffd640 1 -- 172.21.15.5:0/539787379 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d200de36a0 con 0x55d20110d480 2024-09-12T02:59:49.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.164+0000 7fd5b7fff640 1 --2- 172.21.15.5:0/539787379 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d201117d40 0x55d201115250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:59:49.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.164+0000 7fd5b6ffd640 1 --2- 172.21.15.5:0/539787379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d20110d480 0x55d201114ac0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac012f10 tx=0x7fd5ac012c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=720cd75609e8fe17 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:59:49.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.164+0000 7fd596ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T02:59:49.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.164+0000 7fd596ffd640 1 -- 172.21.15.5:0/539787379 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5ac005050 con 0x55d20110d480 2024-09-12T02:59:49.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.164+0000 7fd596ffd640 1 -- 172.21.15.5:0/539787379 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fd5ac0051f0 con 0x55d20110d480 2024-09-12T02:59:49.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.164+0000 7fd5c00f9740 1 -- 172.21.15.5:0/539787379 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5ac025070 con 0x55d20110d480 2024-09-12T02:59:49.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.165+0000 7fd596ffd640 1 -- 172.21.15.5:0/539787379 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd5ac03c050 con 0x55d20110d480 2024-09-12T02:59:49.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.165+0000 7fd5c00f9740 10 client.15765 fetch_fsmap learned FSMap version 27 2024-09-12T02:59:49.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.165+0000 7fd5c00f9740 1 -- 172.21.15.5:0/539787379 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d2011263e0 con 0x55d20110d480 2024-09-12T02:59:49.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.165+0000 7fd596ffd640 1 -- 172.21.15.5:0/539787379 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5ac025070 con 0x55d20110d480 2024-09-12T02:59:49.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.165+0000 7fd596ffd640 1 -- 172.21.15.5:0/539787379 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7fd5ac020540 con 0x55d20110d480 2024-09-12T02:59:49.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.165+0000 7fd5c00f9740 10 client.15765 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T02:59:49.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.165+0000 7fd5c00f9740 10 client.15765 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T02:59:49.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.165+0000 7fd5c00f9740 20 client.15765 populate_metadata read hostname 'smithi005' 2024-09-12T02:59:49.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.165+0000 7fd5c00f9740 1 --2- 172.21.15.5:0/539787379 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55d201127840 0x55d201129c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:49.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.165+0000 7fd5c00f9740 4 client.15765 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:59:49.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.165+0000 7fd5c00f9740 1 -- 172.21.15.5:0/539787379 --> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d20107f7f0 con 0x55d201127840 2024-09-12T02:59:49.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.165+0000 7fd5b77fe640 1 --2- 172.21.15.5:0/539787379 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55d201127840 0x55d201129c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:49.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.166+0000 7fd5b77fe640 1 --2- 172.21.15.5:0/539787379 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55d201127840 0x55d201129c30 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0964c783496e922 server_cookie=8adc98d305fa7ff9 in_seq=0 out_seq=0 2024-09-12T02:59:49.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.166+0000 7fd596ffd640 10 client.15765 ms_handle_connect on v2:172.21.15.148:6838/2317600000 2024-09-12T02:59:49.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.168+0000 7fd596ffd640 1 -- 172.21.15.5:0/539787379 <== mds.0 v2:172.21.15.148:6838/2317600000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d20107f7f0 con 0x55d201127840 2024-09-12T02:59:49.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.168+0000 7fd596ffd640 10 client.15765 handle_command_reply: tid=0 2024-09-12T02:59:49.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.168+0000 7fd5c00f9740 1 client.15765 shutdown 2024-09-12T02:59:49.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.168+0000 7fd5c00f9740 1 -- 172.21.15.5:0/539787379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d20110d480 msgr2=0x55d201114ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:49.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.168+0000 7fd5c00f9740 1 --2- 172.21.15.5:0/539787379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d20110d480 0x55d201114ac0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac012f10 tx=0x7fd5ac012c90 comp rx=0 tx=0).stop 2024-09-12T02:59:49.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.168+0000 7fd5c00f9740 1 -- 172.21.15.5:0/539787379 shutdown_connections 2024-09-12T02:59:49.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.168+0000 7fd5c00f9740 1 --2- 172.21.15.5:0/539787379 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55d201127840 0x55d201129c30 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:49.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.168+0000 7fd596ffd640 0 client.15765 ms_handle_reset on v2:172.21.15.148:6838/2317600000 2024-09-12T02:59:49.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.168+0000 7fd5c00f9740 1 --2- 172.21.15.5:0/539787379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d20110d480 0x55d201114ac0 secure :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac012f10 tx=0x7fd5ac012c90 comp rx=0 tx=0).stop 2024-09-12T02:59:49.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.168+0000 7fd5c00f9740 1 --2- 172.21.15.5:0/539787379 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d201117d40 0x55d201115250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:49.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.168+0000 7fd5c00f9740 1 --2- 172.21.15.5:0/539787379 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d20110cb60 0x55d201117770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:49.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.168+0000 7fd5c00f9740 1 -- 172.21.15.5:0/539787379 >> 172.21.15.5:0/539787379 conn(0x55d2011090f0 msgr2=0x55d201108270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:59:49.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.169+0000 7fd5c00f9740 1 -- 172.21.15.5:0/539787379 shutdown_connections 2024-09-12T02:59:49.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.169+0000 7fd5c00f9740 1 -- 172.21.15.5:0/539787379 wait complete. 2024-09-12T02:59:49.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.169+0000 7fd5c00f9740 20 client.15765 trim_cache size 0 max 16384 2024-09-12T02:59:49.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.189+0000 7fd5c00f9740 1 Processor -- start 2024-09-12T02:59:49.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.189+0000 7fd5c00f9740 1 -- start start 2024-09-12T02:59:49.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.189+0000 7fd5c00f9740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d20110cb60 0x55d201116920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:49.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.189+0000 7fd5c00f9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d201117d40 0x55d20112f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:49.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.189+0000 7fd5c00f9740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d201116e60 0x55d20112f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:49.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.189+0000 7fd5c00f9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd5b81af670 con 0x55d201117d40 2024-09-12T02:59:49.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.189+0000 7fd5c00f9740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fd5b81b1210 con 0x55d20110cb60 2024-09-12T02:59:49.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.189+0000 7fd5c00f9740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd5b81afd40 con 0x55d201116e60 2024-09-12T02:59:49.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.189+0000 7fd5b6ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d201117d40 0x55d20112f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:49.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.189+0000 7fd5b6ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d201117d40 0x55d20112f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39712/0 (socket says 172.21.15.5:39712) 2024-09-12T02:59:49.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.189+0000 7fd5b6ffd640 1 -- 172.21.15.5:0/1295444445 learned_addr learned my addr 172.21.15.5:0/1295444445 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:59:49.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.189+0000 7fd5b7fff640 1 --2- 172.21.15.5:0/1295444445 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d201116e60 0x55d20112f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:49.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.189+0000 7fd5b77fe640 1 --2- 172.21.15.5:0/1295444445 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d20110cb60 0x55d201116920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:49.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.189+0000 7fd5b6ffd640 1 -- 172.21.15.5:0/1295444445 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d201116e60 msgr2=0x55d20112f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:49.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.189+0000 7fd5b6ffd640 1 --2- 172.21.15.5:0/1295444445 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d201116e60 0x55d20112f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:49.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.189+0000 7fd5b6ffd640 1 -- 172.21.15.5:0/1295444445 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d20110cb60 msgr2=0x55d201116920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:49.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.189+0000 7fd5b6ffd640 1 --2- 172.21.15.5:0/1295444445 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d20110cb60 0x55d201116920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:49.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.189+0000 7fd5b6ffd640 1 -- 172.21.15.5:0/1295444445 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2011155c0 con 0x55d201117d40 2024-09-12T02:59:49.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.190+0000 7fd5b6ffd640 1 --2- 172.21.15.5:0/1295444445 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d201117d40 0x55d20112f430 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac020930 tx=0x7fd5ac0040b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c7584c6dceb9bf6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:59:49.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.190+0000 7fd59d7fa640 1 -- 172.21.15.5:0/1295444445 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5ac039070 con 0x55d201117d40 2024-09-12T02:59:49.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.190+0000 7fd59d7fa640 1 -- 172.21.15.5:0/1295444445 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5ac025070 con 0x55d201117d40 2024-09-12T02:59:49.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.190+0000 7fd59d7fa640 1 -- 172.21.15.5:0/1295444445 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5ac029b60 con 0x55d201117d40 2024-09-12T02:59:49.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.191+0000 7fd5c00f9740 1 -- 172.21.15.5:0/1295444445 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d201117d40 msgr2=0x55d20112f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:49.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.191+0000 7fd5c00f9740 1 --2- 172.21.15.5:0/1295444445 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d201117d40 0x55d20112f430 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac020930 tx=0x7fd5ac0040b0 comp rx=0 tx=0).stop 2024-09-12T02:59:49.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.191+0000 7fd5c00f9740 1 -- 172.21.15.5:0/1295444445 shutdown_connections 2024-09-12T02:59:49.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.191+0000 7fd5c00f9740 1 --2- 172.21.15.5:0/1295444445 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d201117d40 0x55d20112f430 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:49.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.191+0000 7fd5c00f9740 1 --2- 172.21.15.5:0/1295444445 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d201116e60 0x55d20112f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:49.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.191+0000 7fd5c00f9740 1 --2- 172.21.15.5:0/1295444445 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d20110cb60 0x55d201116920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:49.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.191+0000 7fd5c00f9740 1 -- 172.21.15.5:0/1295444445 >> 172.21.15.5:0/1295444445 conn(0x55d2011090f0 msgr2=0x55d201116210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:59:49.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.191+0000 7fd5c00f9740 1 -- 172.21.15.5:0/1295444445 shutdown_connections 2024-09-12T02:59:49.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.191+0000 7fd5c00f9740 1 -- 172.21.15.5:0/1295444445 wait complete. 2024-09-12T02:59:49.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.191+0000 7fd5c00f9740 1 -- start start 2024-09-12T02:59:49.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.191+0000 7fd5c00f9740 1 Processor -- start 2024-09-12T02:59:49.197 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.192+0000 7fd5c00f9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d20110cb60 0x55d201120090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:49.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.192+0000 7fd5c00f9740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d201116e60 0x55d2011205d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:49.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.192+0000 7fd5c00f9740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d201117d40 0x55d20111af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:49.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.192+0000 7fd5c00f9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd5b81b0ae0 con 0x55d20110cb60 2024-09-12T02:59:49.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.192+0000 7fd5c00f9740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fd5b81b0410 con 0x55d201116e60 2024-09-12T02:59:49.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.192+0000 7fd5c00f9740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd5b81ad650 con 0x55d201117d40 2024-09-12T02:59:49.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.192+0000 7fd5c00f9740 10 client.0 _ll_register_callbacks cb 0x7fd5ac003f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:59:49.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.192+0000 7fd5b7fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d201117d40 0x55d20111af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:49.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.192+0000 7fd5b77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d20110cb60 0x55d201120090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:49.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.192+0000 7fd5b7fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d201117d40 0x55d20111af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.5:41460/0 (socket says 172.21.15.5:41460) 2024-09-12T02:59:49.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.192+0000 7fd5b77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d20110cb60 0x55d201120090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39728/0 (socket says 172.21.15.5:39728) 2024-09-12T02:59:49.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.192+0000 7fd5b7fff640 1 -- 172.21.15.5:0/965386461 learned_addr learned my addr 172.21.15.5:0/965386461 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:59:49.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.192+0000 7fd5b6ffd640 1 --2- 172.21.15.5:0/965386461 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d201116e60 0x55d2011205d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:49.198 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.192+0000 7fd5b7fff640 1 -- 172.21.15.5:0/965386461 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d201116e60 msgr2=0x55d2011205d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:49.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.192+0000 7fd5b7fff640 1 --2- 172.21.15.5:0/965386461 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d201116e60 0x55d2011205d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:49.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.192+0000 7fd5b7fff640 1 -- 172.21.15.5:0/965386461 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d20110cb60 msgr2=0x55d201120090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:49.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.192+0000 7fd5b7fff640 1 --2- 172.21.15.5:0/965386461 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d20110cb60 0x55d201120090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:49.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.192+0000 7fd5b7fff640 1 -- 172.21.15.5:0/965386461 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d20110d2b0 con 0x55d201117d40 2024-09-12T02:59:49.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.193+0000 7fd5b7fff640 1 --2- 172.21.15.5:0/965386461 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d201117d40 0x55d20111af20 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd5b0012990 tx=0x7fd5b0012e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e38dfaca6e6397f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:59:49.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.193+0000 7fd596ffd640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-09-12T02:59:49.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.193+0000 7fd596ffd640 1 -- 172.21.15.5:0/965386461 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5b0021030 con 0x55d201117d40 2024-09-12T02:59:49.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.193+0000 7fd5c00f9740 1 -- 172.21.15.5:0/965386461 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5ac025070 con 0x55d201117d40 2024-09-12T02:59:49.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.193+0000 7fd596ffd640 1 -- 172.21.15.5:0/965386461 <== mon.1 v2:172.21.15.148:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fd5b0004b20 con 0x55d201117d40 2024-09-12T02:59:49.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.193+0000 7fd596ffd640 1 -- 172.21.15.5:0/965386461 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd5b0004d20 con 0x55d201117d40 2024-09-12T02:59:49.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.193+0000 7fd5c00f9740 10 client.25324 fetch_fsmap learned FSMap version 27 2024-09-12T02:59:49.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.193+0000 7fd5c00f9740 1 -- 172.21.15.5:0/965386461 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d201193010 con 0x55d201117d40 2024-09-12T02:59:49.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.193+0000 7fd596ffd640 1 -- 172.21.15.5:0/965386461 <== mon.1 v2:172.21.15.148:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5ac025070 con 0x55d201117d40 2024-09-12T02:59:49.199 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.194+0000 7fd596ffd640 1 -- 172.21.15.5:0/965386461 <== mon.1 v2:172.21.15.148:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7fd5b001d5e0 con 0x55d201117d40 2024-09-12T02:59:49.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.194+0000 7fd5c00f9740 10 client.25324 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T02:59:49.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.194+0000 7fd5c00f9740 10 client.25324 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T02:59:49.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.194+0000 7fd5c00f9740 20 client.25324 populate_metadata read hostname 'smithi005' 2024-09-12T02:59:49.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.194+0000 7fd5c00f9740 1 --2- 172.21.15.5:0/965386461 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55d2011937f0 0x55d201195c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:49.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.194+0000 7fd5c00f9740 4 client.25324 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T02:59:49.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.194+0000 7fd5c00f9740 1 -- 172.21.15.5:0/965386461 --> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55d201079d00 con 0x55d2011937f0 2024-09-12T02:59:49.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.194+0000 7fd5b77fe640 1 --2- 172.21.15.5:0/965386461 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55d2011937f0 0x55d201195c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:49.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.194+0000 7fd596ffd640 10 client.25324 ms_handle_connect on v2:172.21.15.148:6838/2317600000 2024-09-12T02:59:49.200 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.194+0000 7fd5b77fe640 1 --2- 172.21.15.5:0/965386461 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55d2011937f0 0x55d201195c10 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc4fc11afee21547 server_cookie=c5efe153f7a879ce in_seq=0 out_seq=0 2024-09-12T02:59:49.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.196+0000 7fd596ffd640 1 -- 172.21.15.5:0/965386461 <== mds.0 v2:172.21.15.148:6838/2317600000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55d201079d00 con 0x55d2011937f0 2024-09-12T02:59:49.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.196+0000 7fd596ffd640 10 client.25324 handle_command_reply: tid=0 2024-09-12T02:59:49.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.196+0000 7fd5c00f9740 1 client.25324 shutdown 2024-09-12T02:59:49.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.196+0000 7fd5c00f9740 1 -- 172.21.15.5:0/965386461 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d201117d40 msgr2=0x55d20111af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:49.201 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.196+0000 7fd5c00f9740 1 --2- 172.21.15.5:0/965386461 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d201117d40 0x55d20111af20 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd5b0012990 tx=0x7fd5b0012e50 comp rx=0 tx=0).stop 2024-09-12T02:59:49.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.196+0000 7fd5c00f9740 1 -- 172.21.15.5:0/965386461 shutdown_connections 2024-09-12T02:59:49.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.196+0000 7fd5c00f9740 1 --2- 172.21.15.5:0/965386461 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d201117d40 0x55d20111af20 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:49.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.196+0000 7fd5c00f9740 1 --2- 172.21.15.5:0/965386461 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d201116e60 0x55d2011205d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:49.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.196+0000 7fd5c00f9740 1 --2- 172.21.15.5:0/965386461 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55d2011937f0 0x55d201195c10 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:49.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.196+0000 7fd5c00f9740 1 --2- 172.21.15.5:0/965386461 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d20110cb60 0x55d201120090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:49.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.196+0000 7fd596ffd640 0 client.25324 ms_handle_reset on v2:172.21.15.148:6838/2317600000 2024-09-12T02:59:49.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.196+0000 7fd5c00f9740 1 -- 172.21.15.5:0/965386461 >> 172.21.15.5:0/965386461 conn(0x55d2011090f0 msgr2=0x55d201107a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:59:49.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.196+0000 7fd5c00f9740 1 -- 172.21.15.5:0/965386461 shutdown_connections 2024-09-12T02:59:49.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.196+0000 7fd5c00f9740 1 -- 172.21.15.5:0/965386461 wait complete. 2024-09-12T02:59:49.202 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.196+0000 7fd5c00f9740 20 client.25324 trim_cache size 0 max 16384 2024-09-12T02:59:49.202 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-12T02:59:49.203 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-09-12T02:59:49.203 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "40e408a1-22e4-41a2-8089-adb28e80a8ae", 2024-09-12T02:59:49.203 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-09-12T02:59:49.203 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-12T02:59:49.203 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.197+0000 7fd5bdd20640 1 -- 172.21.15.5:0/2888710842 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fd5a00a2c00 msgr2=0x7fd5a00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:49.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.197+0000 7fd5bdd20640 1 --2- 172.21.15.5:0/2888710842 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fd5a00a2c00 0x7fd5a00a50c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fd5b8075f80 tx=0x7fd5ac005f90 comp rx=0 tx=0).stop 2024-09-12T02:59:49.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.197+0000 7fd5bdd20640 1 -- 172.21.15.5:0/2888710842 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5b8101dd0 msgr2=0x7fd5b80744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:49.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.197+0000 7fd5bdd20640 1 --2- 172.21.15.5:0/2888710842 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5b8101dd0 0x7fd5b80744f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd5a8006010 tx=0x7fd5a8004040 comp rx=0 tx=0).stop 2024-09-12T02:59:49.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.197+0000 7fd5bdd20640 1 -- 172.21.15.5:0/2888710842 shutdown_connections 2024-09-12T02:59:49.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.197+0000 7fd5bdd20640 1 --2- 172.21.15.5:0/2888710842 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fd5a00a2c00 0x7fd5a00a50c0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:49.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.197+0000 7fd5bdd20640 1 --2- 172.21.15.5:0/2888710842 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd5b8106f00 0x7fd5b8074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:49.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.197+0000 7fd5bdd20640 1 --2- 172.21.15.5:0/2888710842 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd5b8102780 0x7fd5b8074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:49.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.197+0000 7fd5bdd20640 1 --2- 172.21.15.5:0/2888710842 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd5b8101dd0 0x7fd5b80744f0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:49.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.197+0000 7fd5bdd20640 1 -- 172.21.15.5:0/2888710842 >> 172.21.15.5:0/2888710842 conn(0x7fd5b80fd8b0 msgr2=0x7fd5b80fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:59:49.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.197+0000 7fd5bdd20640 1 -- 172.21.15.5:0/2888710842 shutdown_connections 2024-09-12T02:59:49.204 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:49.197+0000 7fd5bdd20640 1 -- 172.21.15.5:0/2888710842 wait complete. 2024-09-12T02:59:49.415 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:59:49 smithi148 ceph-mon[35090]: pgmap v382: 97 pgs: 97 active+clean; 20 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 523 KiB/s wr, 33 op/s 2024-09-12T02:59:49.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:59:49 smithi146 ceph-mon[35011]: pgmap v382: 97 pgs: 97 active+clean; 20 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 523 KiB/s wr, 33 op/s 2024-09-12T02:59:49.686 INFO:tasks.workunit.client.0.smithi005.stderr:Updating files: 7% (322/4351) Updating files: 8% (349/4351) Updating files: 9% (392/4351) Updating files: 10% (436/4351) Updating files: 11% (479/4351) Updating files: 11% (493/4351) Updating files: 12% (523/4351) Updating files: 13% (566/4351) Updating files: 14% (610/4351) Updating files: 15% (653/4351) Updating files: 16% (697/4351) Updating files: 16% (736/4351) Updating files: 17% (740/4351) Updating files: 18% (784/4351) Updating files: 19% (827/4351) Updating files: 20% (871/4351) Updating files: 21% (914/4351) Updating files: 22% (958/4351) Updating files: 23% (1001/4351) Updating files: 24% (1045/4351) Updating files: 25% (1088/4351) Updating files: 26% (1132/4351) Updating files: 27% (1175/4351) Updating files: 28% (1219/4351) Updating files: 29% (1262/4351) Updating files: 29% (1303/4351) Updating files: 30% (1306/4351) Updating files: 31% (1349/4351) Updating files: 32% (1393/4351) Updating files: 33% (1436/4351) Updating files: 33% (1477/4351) Updating files: 34% (1480/4351) Updating files: 35% (1523/4351) Updating files: 36% (1567/4351) Updating files: 37% (1610/4351) Updating files: 38% (1654/4351) Updating files: 39% (1697/4351) Updating files: 40% (1741/4351) Updating files: 41% (1784/4351) Updating files: 42% (1828/4351) Updating files: 43% (1871/4351) Updating files: 44% (1915/4351) Updating files: 45% (1958/4351) Updating files: 46% (2002/4351) Updating files: 47% (2045/4351) Updating files: 48% (2089/4351) Updating files: 49% (2132/4351) Updating files: 50% (2176/4351) Updating files: 51% (2220/4351) Updating files: 52% (2263/4351) Updating files: 52% (2265/4351) Updating files: 53% (2307/4351) Updating files: 54% (2350/4351) Updating files: 55% (2394/4351) Updating files: 56% (2437/4351) Updating files: 57% (2481/4351) Updating files: 58% (2524/4351) Updating files: 59% (2568/4351) Updating files: 60% (2611/4351) Updating files: 61% (2655/4351) Updating files: 62% (2698/4351) Updating files: 63% (2742/4351) Updating files: 64% (2785/4351) Updating files: 64% (2809/4351) Updating files: 65% (2829/4351) Updating files: 66% (2872/4351) Updating files: 67% (2916/4351) Updating files: 68% (2959/4351) Updating files: 69% (3003/4351) Updating files: 69% (3017/4351) Updating files: 69% (3040/4351) Updating files: 70% (3046/4351) Updating files: 71% (3090/4351) Updating files: 72% (3133/4351) Updating files: 73% (3177/4351) Updating files: 74% (3220/4351) Updating files: 75% (3264/4351) Updating files: 75% (3266/4351) Updating files: 76% (3307/4351) Updating files: 77% (3351/4351) Updating files: 78% (3394/4351) Updating files: 79% (3438/4351) Updating files: 80% (3481/4351) Updating files: 81% (3525/4351) Updating files: 82% (3568/4351) Updating files: 83% (3612/4351) Updating files: 84% (3655/4351) Updating files: 85% (3699/4351) Updating files: 86% (3742/4351) Updating files: 87% (3786/4351) Updating files: 88% (3829/4351) Updating files: 89% (3873/4351) Updating files: 90% (3916/4351) Updating files: 91% (3960/4351) Updating files: 91% (3987/4351) Updating files: 92% (4003/4351) Updating files: 93% (4047/4351) Updating files: 94% (4090/4351) Updating files: 95% (4134/4351) Updating files: 96% (4177/4351) Updating files: 97% (4221/4351) Updating files: 98% (4264/4351) Updating files: 99% (4308/4351) Updating files: 100% (4351/4351) Updating files: 100% (4351/4351), done. 2024-09-12T02:59:49.693 INFO:tasks.workunit.client.0.smithi005.stderr:+ cd xfstests-dev 2024-09-12T02:59:49.693 INFO:tasks.workunit.client.0.smithi005.stderr:+ git checkout 0e5c12dfd008efc2848c98108c9237487e91ef35 2024-09-12T02:59:50.004 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T02:59:50.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:59:50 smithi005 ceph-mon[31197]: scrub queued for path: / 2024-09-12T02:59:50.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:59:50 smithi005 ceph-mon[31197]: scrub summary: idle+waiting paths [/] 2024-09-12T02:59:50.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:59:50 smithi005 ceph-mon[31197]: scrub summary: active paths [/] 2024-09-12T02:59:50.315 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T02:59:50.415 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:59:50 smithi148 ceph-mon[35090]: scrub queued for path: / 2024-09-12T02:59:50.415 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:59:50 smithi148 ceph-mon[35090]: scrub summary: idle+waiting paths [/] 2024-09-12T02:59:50.415 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:59:50 smithi148 ceph-mon[35090]: scrub summary: active paths [/] 2024-09-12T02:59:50.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:59:50 smithi146 ceph-mon[35011]: scrub queued for path: / 2024-09-12T02:59:50.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:59:50 smithi146 ceph-mon[35011]: scrub summary: idle+waiting paths [/] 2024-09-12T02:59:50.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:59:50 smithi146 ceph-mon[35011]: scrub summary: active paths [/] 2024-09-12T02:59:50.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.806+0000 7f4f4d051640 1 -- 172.21.15.5:0/3456916059 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f481008d0 msgr2=0x7f4f48100cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:50.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.806+0000 7f4f4d051640 1 --2- 172.21.15.5:0/3456916059 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f481008d0 0x7f4f48100cb0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f4f38009ca0 tx=0x7f4f3802a8c0 comp rx=0 tx=0).stop 2024-09-12T02:59:50.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.806+0000 7f4f4d051640 1 -- 172.21.15.5:0/3456916059 shutdown_connections 2024-09-12T02:59:50.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.806+0000 7f4f4d051640 1 --2- 172.21.15.5:0/3456916059 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4f4810e480 0x7f4f48110870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:50.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.806+0000 7f4f4d051640 1 --2- 172.21.15.5:0/3456916059 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4f48101280 0x7f4f4810df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:50.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.806+0000 7f4f4d051640 1 --2- 172.21.15.5:0/3456916059 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f481008d0 0x7f4f48100cb0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:50.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.806+0000 7f4f4d051640 1 -- 172.21.15.5:0/3456916059 >> 172.21.15.5:0/3456916059 conn(0x7f4f480fc7a0 msgr2=0x7f4f480febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:59:50.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.806+0000 7f4f4d051640 1 -- 172.21.15.5:0/3456916059 shutdown_connections 2024-09-12T02:59:50.808 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.806+0000 7f4f4d051640 1 -- 172.21.15.5:0/3456916059 wait complete. 2024-09-12T02:59:50.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.807+0000 7f4f4d051640 1 Processor -- start 2024-09-12T02:59:50.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.807+0000 7f4f4d051640 1 -- start start 2024-09-12T02:59:50.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.808+0000 7f4f4d051640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f481008d0 0x7f4f4819a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:50.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.808+0000 7f4f4d051640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4f48101280 0x7f4f4819abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:50.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.808+0000 7f4f4d051640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4f4810e480 0x7f4f4819ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:50.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.808+0000 7f4f4d051640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4f48114190 con 0x7f4f481008d0 2024-09-12T02:59:50.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.808+0000 7f4f46d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f481008d0 0x7f4f4819a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:50.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.808+0000 7f4f4d051640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f4f48114010 con 0x7f4f48101280 2024-09-12T02:59:50.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.808+0000 7f4f4d051640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4f48114310 con 0x7f4f4810e480 2024-09-12T02:59:50.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.808+0000 7f4f46d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f481008d0 0x7f4f4819a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39734/0 (socket says 172.21.15.5:39734) 2024-09-12T02:59:50.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.808+0000 7f4f46d9d640 1 -- 172.21.15.5:0/2706800192 learned_addr learned my addr 172.21.15.5:0/2706800192 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:59:50.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.808+0000 7f4f4659c640 1 --2- 172.21.15.5:0/2706800192 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4f48101280 0x7f4f4819abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:50.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.808+0000 7f4f4759e640 1 --2- 172.21.15.5:0/2706800192 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4f4810e480 0x7f4f4819ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:50.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.808+0000 7f4f46d9d640 1 -- 172.21.15.5:0/2706800192 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4f4810e480 msgr2=0x7f4f4819ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:50.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.808+0000 7f4f46d9d640 1 --2- 172.21.15.5:0/2706800192 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4f4810e480 0x7f4f4819ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:50.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.808+0000 7f4f46d9d640 1 -- 172.21.15.5:0/2706800192 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4f48101280 msgr2=0x7f4f4819abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:50.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.808+0000 7f4f46d9d640 1 --2- 172.21.15.5:0/2706800192 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4f48101280 0x7f4f4819abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:50.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.808+0000 7f4f46d9d640 1 -- 172.21.15.5:0/2706800192 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f4819f620 con 0x7f4f481008d0 2024-09-12T02:59:50.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.808+0000 7f4f4659c640 1 --2- 172.21.15.5:0/2706800192 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4f48101280 0x7f4f4819abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:59:50.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.808+0000 7f4f46d9d640 1 --2- 172.21.15.5:0/2706800192 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f481008d0 0x7f4f4819a670 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f4f3802ade0 tx=0x7f4f380319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:59:50.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.809+0000 7f4f2ffff640 1 -- 172.21.15.5:0/2706800192 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f3802fa20 con 0x7f4f481008d0 2024-09-12T02:59:50.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.809+0000 7f4f4d051640 1 -- 172.21.15.5:0/2706800192 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f4819f8b0 con 0x7f4f481008d0 2024-09-12T02:59:50.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.809+0000 7f4f4d051640 1 -- 172.21.15.5:0/2706800192 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f481a71f0 con 0x7f4f481008d0 2024-09-12T02:59:50.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.809+0000 7f4f2ffff640 1 -- 172.21.15.5:0/2706800192 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f3802fbc0 con 0x7f4f481008d0 2024-09-12T02:59:50.811 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.809+0000 7f4f2ffff640 1 -- 172.21.15.5:0/2706800192 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f380466e0 con 0x7f4f481008d0 2024-09-12T02:59:50.812 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.812+0000 7f4f2ffff640 1 -- 172.21.15.5:0/2706800192 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f4f3802fd70 con 0x7f4f481008d0 2024-09-12T02:59:50.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.813+0000 7f4f4f42a740 1 Processor -- start 2024-09-12T02:59:50.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.813+0000 7f4f2ffff640 1 --2- 172.21.15.5:0/2706800192 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f4f300a2c00 0x7f4f300a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:50.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.813+0000 7f4f4f42a740 1 -- start start 2024-09-12T02:59:50.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.813+0000 7f4f4759e640 1 --2- 172.21.15.5:0/2706800192 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f4f300a2c00 0x7f4f300a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:50.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.813+0000 7f4f2ffff640 1 -- 172.21.15.5:0/2706800192 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f4f380f0220 con 0x7f4f481008d0 2024-09-12T02:59:50.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.813+0000 7f4f4f42a740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558c117064a0 0x558c11706880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:50.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.813+0000 7f4f4f42a740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558c11706dc0 0x558c11711140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:50.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.813+0000 7f4f4f42a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c11711680 0x558c11713a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:50.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.813+0000 7f4f4f42a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558c1166c2e0 con 0x558c11711680 2024-09-12T02:59:50.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.813+0000 7f4f4f42a740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x558c11613ce0 con 0x558c11706dc0 2024-09-12T02:59:50.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.813+0000 7f4f4f42a740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x558c116682a0 con 0x558c117064a0 2024-09-12T02:59:50.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.813+0000 7f4f46d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558c117064a0 0x558c11706880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:50.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.813+0000 7f4f46d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558c117064a0 0x558c11706880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.5:41494/0 (socket says 172.21.15.5:41494) 2024-09-12T02:59:50.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.813+0000 7f4f46d9d640 1 -- 172.21.15.5:0/2311967285 learned_addr learned my addr 172.21.15.5:0/2311967285 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:59:50.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.813+0000 7f4f4659c640 1 --2- 172.21.15.5:0/2311967285 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558c11706dc0 0x558c11711140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:50.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.813+0000 7f4f4759e640 1 --2- 172.21.15.5:0/2311967285 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c11711680 0x558c11713a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:50.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.813+0000 7f4f4759e640 1 --2- 172.21.15.5:0/2706800192 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f4f300a2c00 0x7f4f300a50c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f4f4806a0f0 tx=0x7f4f40009f70 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:59:50.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.813+0000 7f4f46d9d640 1 -- 172.21.15.5:0/2311967285 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558c11706dc0 msgr2=0x558c11711140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:50.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.813+0000 7f4f46d9d640 1 --2- 172.21.15.5:0/2311967285 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558c11706dc0 0x558c11711140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:50.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.813+0000 7f4f46d9d640 1 -- 172.21.15.5:0/2311967285 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c11711680 msgr2=0x558c11713a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:50.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.813+0000 7f4f46d9d640 1 --2- 172.21.15.5:0/2311967285 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c11711680 0x558c11713a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:50.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.813+0000 7f4f46d9d640 1 -- 172.21.15.5:0/2311967285 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c115f8440 con 0x558c117064a0 2024-09-12T02:59:50.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.813+0000 7f4f4659c640 1 --2- 172.21.15.5:0/2311967285 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558c11706dc0 0x558c11711140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:59:50.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.814+0000 7f4f46d9d640 1 --2- 172.21.15.5:0/2311967285 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558c117064a0 0x558c11706880 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4f380c5170 tx=0x7f4f380bd130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d9bee925a284ddf server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:59:50.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.814+0000 7f4f2d7fa640 1 -- 172.21.15.5:0/2311967285 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f380c0de0 con 0x558c117064a0 2024-09-12T02:59:50.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.814+0000 7f4f2d7fa640 1 -- 172.21.15.5:0/2311967285 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f38036070 con 0x558c117064a0 2024-09-12T02:59:50.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.814+0000 7f4f2d7fa640 1 -- 172.21.15.5:0/2311967285 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f380bbe80 con 0x558c117064a0 2024-09-12T02:59:50.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.815+0000 7f4f4f42a740 1 -- 172.21.15.5:0/2311967285 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558c117064a0 msgr2=0x558c11706880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:50.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.815+0000 7f4f4f42a740 1 --2- 172.21.15.5:0/2311967285 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558c117064a0 0x558c11706880 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4f380c5170 tx=0x7f4f380bd130 comp rx=0 tx=0).stop 2024-09-12T02:59:50.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.815+0000 7f4f4f42a740 1 -- 172.21.15.5:0/2311967285 shutdown_connections 2024-09-12T02:59:50.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.815+0000 7f4f4f42a740 1 --2- 172.21.15.5:0/2311967285 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c11711680 0x558c11713a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:50.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.815+0000 7f4f4f42a740 1 --2- 172.21.15.5:0/2311967285 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558c11706dc0 0x558c11711140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:50.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.815+0000 7f4f4f42a740 1 --2- 172.21.15.5:0/2311967285 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558c117064a0 0x558c11706880 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:50.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.815+0000 7f4f4f42a740 1 -- 172.21.15.5:0/2311967285 >> 172.21.15.5:0/2311967285 conn(0x558c11702a30 msgr2=0x558c11702e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:59:50.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.815+0000 7f4f4f42a740 1 -- 172.21.15.5:0/2311967285 shutdown_connections 2024-09-12T02:59:50.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.815+0000 7f4f4f42a740 1 -- 172.21.15.5:0/2311967285 wait complete. 2024-09-12T02:59:50.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.815+0000 7f4f4f42a740 1 -- start start 2024-09-12T02:59:50.820 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.815+0000 7f4f4f42a740 1 Processor -- start 2024-09-12T02:59:50.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.815+0000 7f4f4f42a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c117064a0 0x558c11710fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:50.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.815+0000 7f4f4f42a740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558c11706dc0 0x558c1170e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:50.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.815+0000 7f4f4f42a740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558c11711680 0x558c1170e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:50.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.815+0000 7f4f4f42a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558c116682a0 con 0x558c117064a0 2024-09-12T02:59:50.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.815+0000 7f4f4f42a740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x558c11627050 con 0x558c11711680 2024-09-12T02:59:50.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.815+0000 7f4f4f42a740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x558c11714010 con 0x558c11706dc0 2024-09-12T02:59:50.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.816+0000 7f4f4f42a740 10 client.0 _ll_register_callbacks cb 0x558c11667160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:59:50.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.816+0000 7f4f46d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c117064a0 0x558c11710fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:50.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.816+0000 7f4f46d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c117064a0 0x558c11710fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39748/0 (socket says 172.21.15.5:39748) 2024-09-12T02:59:50.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.816+0000 7f4f46d9d640 1 -- 172.21.15.5:0/3813051711 learned_addr learned my addr 172.21.15.5:0/3813051711 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:59:50.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.816+0000 7f4f4659c640 1 --2- 172.21.15.5:0/3813051711 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558c11706dc0 0x558c1170e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:50.821 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.816+0000 7f4f4759e640 1 --2- 172.21.15.5:0/3813051711 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558c11711680 0x558c1170e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:50.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.816+0000 7f4f46d9d640 1 -- 172.21.15.5:0/3813051711 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558c11706dc0 msgr2=0x558c1170e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:50.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.816+0000 7f4f46d9d640 1 --2- 172.21.15.5:0/3813051711 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558c11706dc0 0x558c1170e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:50.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.816+0000 7f4f46d9d640 1 -- 172.21.15.5:0/3813051711 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558c11711680 msgr2=0x558c1170e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:50.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.816+0000 7f4f46d9d640 1 --2- 172.21.15.5:0/3813051711 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558c11711680 0x558c1170e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:50.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.816+0000 7f4f46d9d640 1 -- 172.21.15.5:0/3813051711 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c113dd3c0 con 0x558c117064a0 2024-09-12T02:59:50.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.816+0000 7f4f4759e640 1 --2- 172.21.15.5:0/3813051711 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558c11711680 0x558c1170e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:59:50.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.816+0000 7f4f46d9d640 1 --2- 172.21.15.5:0/3813051711 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c117064a0 0x558c11710fa0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f4f380bc9c0 tx=0x7f4f38009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ce942f238afd0e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:59:50.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.816+0000 7f4f2aff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T02:59:50.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.816+0000 7f4f2aff5640 1 -- 172.21.15.5:0/3813051711 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f380da060 con 0x558c117064a0 2024-09-12T02:59:50.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.816+0000 7f4f2aff5640 1 -- 172.21.15.5:0/3813051711 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f4f380b8780 con 0x558c117064a0 2024-09-12T02:59:50.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.816+0000 7f4f4f42a740 1 -- 172.21.15.5:0/3813051711 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f38036070 con 0x558c117064a0 2024-09-12T02:59:50.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.817+0000 7f4f2aff5640 1 -- 172.21.15.5:0/3813051711 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4f380b8980 con 0x558c117064a0 2024-09-12T02:59:50.822 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.817+0000 7f4f4f42a740 10 client.15783 fetch_fsmap learned FSMap version 27 2024-09-12T02:59:50.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.817+0000 7f4f4f42a740 1 -- 172.21.15.5:0/3813051711 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558c1171f7d0 con 0x558c117064a0 2024-09-12T02:59:50.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.817+0000 7f4f2aff5640 1 -- 172.21.15.5:0/3813051711 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f38036070 con 0x558c117064a0 2024-09-12T02:59:50.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.817+0000 7f4f2aff5640 1 -- 172.21.15.5:0/3813051711 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7f4f3804e030 con 0x558c117064a0 2024-09-12T02:59:50.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.817+0000 7f4f4f42a740 10 client.15783 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T02:59:50.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.817+0000 7f4f4f42a740 10 client.15783 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T02:59:50.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.817+0000 7f4f4f42a740 20 client.15783 populate_metadata read hostname 'smithi005' 2024-09-12T02:59:50.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.817+0000 7f4f4f42a740 1 --2- 172.21.15.5:0/3813051711 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x558c11720bc0 0x558c11722fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:50.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.817+0000 7f4f4f42a740 4 client.15783 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T02:59:50.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.817+0000 7f4f4f42a740 1 -- 172.21.15.5:0/3813051711 --> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558c11679850 con 0x558c11720bc0 2024-09-12T02:59:50.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.817+0000 7f4f4659c640 1 --2- 172.21.15.5:0/3813051711 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x558c11720bc0 0x558c11722fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:50.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.818+0000 7f4f4659c640 1 --2- 172.21.15.5:0/3813051711 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x558c11720bc0 0x558c11722fd0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6abe3d6a34a5e40 server_cookie=6d645f80fbe456be in_seq=0 out_seq=0 2024-09-12T02:59:50.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.818+0000 7f4f2aff5640 10 client.15783 ms_handle_connect on v2:172.21.15.148:6838/2317600000 2024-09-12T02:59:50.823 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.819+0000 7f4f2aff5640 1 -- 172.21.15.5:0/3813051711 <== mds.0 v2:172.21.15.148:6838/2317600000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558c11679850 con 0x558c11720bc0 2024-09-12T02:59:50.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.819+0000 7f4f2aff5640 10 client.15783 handle_command_reply: tid=0 2024-09-12T02:59:50.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.819+0000 7f4f4f42a740 1 client.15783 shutdown 2024-09-12T02:59:50.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.819+0000 7f4f4f42a740 1 -- 172.21.15.5:0/3813051711 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c117064a0 msgr2=0x558c11710fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:50.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.819+0000 7f4f4f42a740 1 --2- 172.21.15.5:0/3813051711 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c117064a0 0x558c11710fa0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f4f380bc9c0 tx=0x7f4f38009680 comp rx=0 tx=0).stop 2024-09-12T02:59:50.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.819+0000 7f4f4f42a740 1 -- 172.21.15.5:0/3813051711 shutdown_connections 2024-09-12T02:59:50.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.819+0000 7f4f4f42a740 1 --2- 172.21.15.5:0/3813051711 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558c11711680 0x558c1170e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:50.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.819+0000 7f4f4f42a740 1 --2- 172.21.15.5:0/3813051711 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558c11706dc0 0x558c1170e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:50.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.819+0000 7f4f4f42a740 1 --2- 172.21.15.5:0/3813051711 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x558c11720bc0 0x558c11722fd0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:50.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.819+0000 7f4f4f42a740 1 --2- 172.21.15.5:0/3813051711 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c117064a0 0x558c11710fa0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:50.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.819+0000 7f4f2aff5640 0 client.15783 ms_handle_reset on v2:172.21.15.148:6838/2317600000 2024-09-12T02:59:50.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.819+0000 7f4f4f42a740 1 -- 172.21.15.5:0/3813051711 >> 172.21.15.5:0/3813051711 conn(0x558c11702a30 msgr2=0x558c11701b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:59:50.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.819+0000 7f4f4f42a740 1 -- 172.21.15.5:0/3813051711 shutdown_connections 2024-09-12T02:59:50.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.819+0000 7f4f4f42a740 1 -- 172.21.15.5:0/3813051711 wait complete. 2024-09-12T02:59:50.824 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.819+0000 7f4f4f42a740 20 client.15783 trim_cache size 0 max 16384 2024-09-12T02:59:50.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.838+0000 7f4f4f42a740 1 Processor -- start 2024-09-12T02:59:50.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.838+0000 7f4f4f42a740 1 -- start start 2024-09-12T02:59:50.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.838+0000 7f4f4f42a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c117064a0 0x558c1171d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:50.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.838+0000 7f4f4f42a740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558c11706dc0 0x558c11722bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:50.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.838+0000 7f4f4f42a740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558c11711680 0x558c117097f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:50.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.838+0000 7f4f4f42a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4f3400e400 con 0x558c117064a0 2024-09-12T02:59:50.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.838+0000 7f4f4f42a740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f4f3400ffa0 con 0x558c11711680 2024-09-12T02:59:50.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.838+0000 7f4f4f42a740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4f3400ead0 con 0x558c11706dc0 2024-09-12T02:59:50.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.838+0000 7f4f46d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c117064a0 0x558c1171d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:50.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.838+0000 7f4f46d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c117064a0 0x558c1171d0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39752/0 (socket says 172.21.15.5:39752) 2024-09-12T02:59:50.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.838+0000 7f4f46d9d640 1 -- 172.21.15.5:0/220859656 learned_addr learned my addr 172.21.15.5:0/220859656 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:59:50.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.838+0000 7f4f4659c640 1 --2- 172.21.15.5:0/220859656 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558c11706dc0 0x558c11722bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:50.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.838+0000 7f4f4759e640 1 --2- 172.21.15.5:0/220859656 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558c11711680 0x558c117097f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:50.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.838+0000 7f4f46d9d640 1 -- 172.21.15.5:0/220859656 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558c11706dc0 msgr2=0x558c11722bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:50.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.838+0000 7f4f46d9d640 1 --2- 172.21.15.5:0/220859656 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558c11706dc0 0x558c11722bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:50.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.838+0000 7f4f46d9d640 1 -- 172.21.15.5:0/220859656 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558c11711680 msgr2=0x558c117097f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:50.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.838+0000 7f4f46d9d640 1 --2- 172.21.15.5:0/220859656 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558c11711680 0x558c117097f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:50.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.838+0000 7f4f46d9d640 1 -- 172.21.15.5:0/220859656 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c1170d200 con 0x558c117064a0 2024-09-12T02:59:50.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.839+0000 7f4f46d9d640 1 --2- 172.21.15.5:0/220859656 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c117064a0 0x558c1171d0f0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f4f380bcbf0 tx=0x7f4f38005f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bca6ca75e1ddf8c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:59:50.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.839+0000 7f4f2cff9640 1 -- 172.21.15.5:0/220859656 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f380da070 con 0x558c117064a0 2024-09-12T02:59:50.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.839+0000 7f4f2cff9640 1 -- 172.21.15.5:0/220859656 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f38036070 con 0x558c117064a0 2024-09-12T02:59:50.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.839+0000 7f4f2cff9640 1 -- 172.21.15.5:0/220859656 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f380b91c0 con 0x558c117064a0 2024-09-12T02:59:50.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.839+0000 7f4f4f42a740 1 -- 172.21.15.5:0/220859656 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c117064a0 msgr2=0x558c1171d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:50.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.839+0000 7f4f4f42a740 1 --2- 172.21.15.5:0/220859656 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c117064a0 0x558c1171d0f0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f4f380bcbf0 tx=0x7f4f38005f60 comp rx=0 tx=0).stop 2024-09-12T02:59:50.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.839+0000 7f4f4f42a740 1 -- 172.21.15.5:0/220859656 shutdown_connections 2024-09-12T02:59:50.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.840+0000 7f4f4f42a740 1 --2- 172.21.15.5:0/220859656 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558c11711680 0x558c117097f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:50.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.840+0000 7f4f4f42a740 1 --2- 172.21.15.5:0/220859656 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558c11706dc0 0x558c11722bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:50.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.840+0000 7f4f4f42a740 1 --2- 172.21.15.5:0/220859656 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c117064a0 0x558c1171d0f0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:50.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.840+0000 7f4f4f42a740 1 -- 172.21.15.5:0/220859656 >> 172.21.15.5:0/220859656 conn(0x558c11702a30 msgr2=0x558c117090e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:59:50.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.840+0000 7f4f4f42a740 1 -- 172.21.15.5:0/220859656 shutdown_connections 2024-09-12T02:59:50.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.840+0000 7f4f4f42a740 1 -- 172.21.15.5:0/220859656 wait complete. 2024-09-12T02:59:50.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.840+0000 7f4f4f42a740 1 -- start start 2024-09-12T02:59:50.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.840+0000 7f4f4f42a740 1 Processor -- start 2024-09-12T02:59:50.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.840+0000 7f4f4f42a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c117064a0 0x558c11783a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:50.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.840+0000 7f4f4f42a740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558c11706dc0 0x558c11783f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:50.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.840+0000 7f4f4f42a740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558c11711680 0x558c11718ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:50.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.840+0000 7f4f4f42a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4f3400f870 con 0x558c117064a0 2024-09-12T02:59:50.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.840+0000 7f4f4f42a740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f4f3400f1a0 con 0x558c11706dc0 2024-09-12T02:59:50.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.840+0000 7f4f4f42a740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4f3400c3e0 con 0x558c11711680 2024-09-12T02:59:50.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.840+0000 7f4f46d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c117064a0 0x558c11783a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:50.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.841+0000 7f4f4659c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558c11706dc0 0x558c11783f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:50.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.841+0000 7f4f4759e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558c11711680 0x558c11718ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:50.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.841+0000 7f4f4f42a740 10 client.0 _ll_register_callbacks cb 0x7f4f380d0f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T02:59:50.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.841+0000 7f4f4659c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558c11706dc0 0x558c11783f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.5:44456/0 (socket says 172.21.15.5:44456) 2024-09-12T02:59:50.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.841+0000 7f4f4659c640 1 -- 172.21.15.5:0/1618639168 learned_addr learned my addr 172.21.15.5:0/1618639168 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T02:59:50.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.841+0000 7f4f46d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c117064a0 0x558c11783a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39754/0 (socket says 172.21.15.5:39754) 2024-09-12T02:59:50.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.841+0000 7f4f4759e640 1 -- 172.21.15.5:0/1618639168 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558c11706dc0 msgr2=0x558c11783f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:50.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.841+0000 7f4f4759e640 1 --2- 172.21.15.5:0/1618639168 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558c11706dc0 0x558c11783f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:50.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.841+0000 7f4f4759e640 1 -- 172.21.15.5:0/1618639168 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c117064a0 msgr2=0x558c11783a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T02:59:50.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.841+0000 7f4f4759e640 1 --2- 172.21.15.5:0/1618639168 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c117064a0 0x558c11783a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:50.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.841+0000 7f4f4759e640 1 -- 172.21.15.5:0/1618639168 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c11711310 con 0x558c11711680 2024-09-12T02:59:50.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.841+0000 7f4f46d9d640 1 --2- 172.21.15.5:0/1618639168 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c117064a0 0x558c11783a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:59:50.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.841+0000 7f4f4659c640 1 --2- 172.21.15.5:0/1618639168 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558c11706dc0 0x558c11783f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T02:59:50.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.841+0000 7f4f4759e640 1 --2- 172.21.15.5:0/1618639168 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558c11711680 0x558c11718ef0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x558c11703db0 tx=0x7f4f40004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ccefabb9e22bb7d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T02:59:50.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.841+0000 7f4f2aff5640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-09-12T02:59:50.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.841+0000 7f4f2aff5640 1 -- 172.21.15.5:0/1618639168 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f40017040 con 0x558c11711680 2024-09-12T02:59:50.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.841+0000 7f4f2aff5640 1 -- 172.21.15.5:0/1618639168 <== mon.1 v2:172.21.15.148:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f4f40005020 con 0x558c11711680 2024-09-12T02:59:50.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.841+0000 7f4f4f42a740 1 -- 172.21.15.5:0/1618639168 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f38036070 con 0x558c11711680 2024-09-12T02:59:50.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.842+0000 7f4f2aff5640 1 -- 172.21.15.5:0/1618639168 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4f40005220 con 0x558c11711680 2024-09-12T02:59:50.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.842+0000 7f4f4f42a740 10 client.25342 fetch_fsmap learned FSMap version 27 2024-09-12T02:59:50.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.842+0000 7f4f4f42a740 1 -- 172.21.15.5:0/1618639168 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x558c1178b130 con 0x558c11711680 2024-09-12T02:59:50.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.842+0000 7f4f2aff5640 1 -- 172.21.15.5:0/1618639168 <== mon.1 v2:172.21.15.148:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f38036070 con 0x558c11711680 2024-09-12T02:59:50.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.842+0000 7f4f2aff5640 1 -- 172.21.15.5:0/1618639168 <== mon.1 v2:172.21.15.148:3300/0 5 ==== fsmap(e 27) ==== 8370+0+0 (secure 0 0 0) 0x7f4f40027070 con 0x558c11711680 2024-09-12T02:59:50.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.842+0000 7f4f4f42a740 10 client.25342 fetch_fsmap finished waiting for FSMap version 27 2024-09-12T02:59:50.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.842+0000 7f4f4f42a740 10 client.25342 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T02:59:50.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.842+0000 7f4f4f42a740 20 client.25342 populate_metadata read hostname 'smithi005' 2024-09-12T02:59:50.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.842+0000 7f4f4f42a740 1 --2- 172.21.15.5:0/1618639168 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x558c1178b990 0x558c1178ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T02:59:50.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.842+0000 7f4f4f42a740 4 client.25342 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T02:59:50.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.842+0000 7f4f4f42a740 1 -- 172.21.15.5:0/1618639168 --> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558c11673d60 con 0x558c1178b990 2024-09-12T02:59:50.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.842+0000 7f4f46d9d640 1 --2- 172.21.15.5:0/1618639168 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x558c1178b990 0x558c1178ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T02:59:50.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.843+0000 7f4f2aff5640 10 client.25342 ms_handle_connect on v2:172.21.15.148:6838/2317600000 2024-09-12T02:59:50.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.843+0000 7f4f46d9d640 1 --2- 172.21.15.5:0/1618639168 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x558c1178b990 0x558c1178ddb0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66fb397ed30c0bf1 server_cookie=53ea11510886d116 in_seq=0 out_seq=0 2024-09-12T02:59:50.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.846+0000 7f4f2aff5640 1 -- 172.21.15.5:0/1618639168 <== mds.0 v2:172.21.15.148:6838/2317600000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x558c11673d60 con 0x558c1178b990 2024-09-12T02:59:50.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.846+0000 7f4f2aff5640 10 client.25342 handle_command_reply: tid=0 2024-09-12T02:59:50.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.846+0000 7f4f4f42a740 1 client.25342 shutdown 2024-09-12T02:59:50.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.846+0000 7f4f4f42a740 1 -- 172.21.15.5:0/1618639168 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558c11711680 msgr2=0x558c11718ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:50.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.846+0000 7f4f4f42a740 1 --2- 172.21.15.5:0/1618639168 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558c11711680 0x558c11718ef0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x558c11703db0 tx=0x7f4f40004730 comp rx=0 tx=0).stop 2024-09-12T02:59:50.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.846+0000 7f4f4f42a740 1 -- 172.21.15.5:0/1618639168 shutdown_connections 2024-09-12T02:59:50.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.846+0000 7f4f4f42a740 1 --2- 172.21.15.5:0/1618639168 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558c11711680 0x558c11718ef0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:50.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.846+0000 7f4f4f42a740 1 --2- 172.21.15.5:0/1618639168 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558c11706dc0 0x558c11783f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:50.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.846+0000 7f4f4f42a740 1 --2- 172.21.15.5:0/1618639168 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x558c1178b990 0x558c1178ddb0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:50.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.846+0000 7f4f4f42a740 1 --2- 172.21.15.5:0/1618639168 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c117064a0 0x558c11783a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:50.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.846+0000 7f4f2aff5640 0 client.25342 ms_handle_reset on v2:172.21.15.148:6838/2317600000 2024-09-12T02:59:50.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.846+0000 7f4f4f42a740 1 -- 172.21.15.5:0/1618639168 >> 172.21.15.5:0/1618639168 conn(0x558c11702a30 msgr2=0x558c11709830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:59:50.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.846+0000 7f4f4f42a740 1 -- 172.21.15.5:0/1618639168 shutdown_connections 2024-09-12T02:59:50.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.846+0000 7f4f4f42a740 1 -- 172.21.15.5:0/1618639168 wait complete. 2024-09-12T02:59:50.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.846+0000 7f4f4f42a740 20 client.25342 trim_cache size 0 max 16384 2024-09-12T02:59:50.850 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-12T02:59:50.850 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "scrub active (509 inodes in the stack)", 2024-09-12T02:59:50.850 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-09-12T02:59:50.850 INFO:teuthology.orchestra.run.smithi005.stdout: "40e408a1-22e4-41a2-8089-adb28e80a8ae": { 2024-09-12T02:59:50.850 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-09-12T02:59:50.850 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "40e408a1-22e4-41a2-8089-adb28e80a8ae", 2024-09-12T02:59:50.850 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-09-12T02:59:50.850 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T02:59:50.850 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T02:59:50.850 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-12T02:59:50.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.847+0000 7f4f4d051640 1 -- 172.21.15.5:0/2706800192 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f4f300a2c00 msgr2=0x7f4f300a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:50.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.847+0000 7f4f4d051640 1 --2- 172.21.15.5:0/2706800192 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f4f300a2c00 0x7f4f300a50c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f4f4806a0f0 tx=0x7f4f40009f70 comp rx=0 tx=0).stop 2024-09-12T02:59:50.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.847+0000 7f4f4d051640 1 -- 172.21.15.5:0/2706800192 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f481008d0 msgr2=0x7f4f4819a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T02:59:50.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.847+0000 7f4f4d051640 1 --2- 172.21.15.5:0/2706800192 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f481008d0 0x7f4f4819a670 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f4f3802ade0 tx=0x7f4f380319f0 comp rx=0 tx=0).stop 2024-09-12T02:59:50.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.847+0000 7f4f4d051640 1 -- 172.21.15.5:0/2706800192 shutdown_connections 2024-09-12T02:59:50.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.847+0000 7f4f4d051640 1 --2- 172.21.15.5:0/2706800192 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4f4810e480 0x7f4f4819ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:50.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.847+0000 7f4f4d051640 1 --2- 172.21.15.5:0/2706800192 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4f48101280 0x7f4f4819abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:50.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.847+0000 7f4f4d051640 1 --2- 172.21.15.5:0/2706800192 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f4f300a2c00 0x7f4f300a50c0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:50.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.847+0000 7f4f4d051640 1 --2- 172.21.15.5:0/2706800192 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f481008d0 0x7f4f4819a670 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T02:59:50.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.847+0000 7f4f4d051640 1 -- 172.21.15.5:0/2706800192 >> 172.21.15.5:0/2706800192 conn(0x7f4f480fc7a0 msgr2=0x7f4f480feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T02:59:50.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.847+0000 7f4f4d051640 1 -- 172.21.15.5:0/2706800192 shutdown_connections 2024-09-12T02:59:50.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T02:59:50.847+0000 7f4f4d051640 1 -- 172.21.15.5:0/2706800192 wait complete. 2024-09-12T02:59:51.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:59:51 smithi005 ceph-mon[31197]: scrub summary: active paths [/] 2024-09-12T02:59:51.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:59:51 smithi005 ceph-mon[31197]: scrub summary: idle+waiting paths [/] 2024-09-12T02:59:51.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:59:51 smithi005 ceph-mon[31197]: pgmap v383: 97 pgs: 97 active+clean; 21 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 527 KiB/s rd, 627 KiB/s wr, 37 op/s 2024-09-12T02:59:51.415 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:59:51 smithi148 ceph-mon[35090]: scrub summary: active paths [/] 2024-09-12T02:59:51.415 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:59:51 smithi148 ceph-mon[35090]: scrub summary: idle+waiting paths [/] 2024-09-12T02:59:51.415 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:59:51 smithi148 ceph-mon[35090]: pgmap v383: 97 pgs: 97 active+clean; 21 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 527 KiB/s rd, 627 KiB/s wr, 37 op/s 2024-09-12T02:59:51.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:59:51 smithi146 ceph-mon[35011]: scrub summary: active paths [/] 2024-09-12T02:59:51.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:59:51 smithi146 ceph-mon[35011]: scrub summary: idle+waiting paths [/] 2024-09-12T02:59:51.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:59:51 smithi146 ceph-mon[35011]: pgmap v383: 97 pgs: 97 active+clean; 21 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 527 KiB/s rd, 627 KiB/s wr, 37 op/s 2024-09-12T02:59:51.704 INFO:tasks.cephfs.filesystem:scrub status for tag:40e408a1-22e4-41a2-8089-adb28e80a8ae - {'path': '/', 'tag': '40e408a1-22e4-41a2-8089-adb28e80a8ae', 'options': 'recursive,force'} 2024-09-12T02:59:53.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:59:53 smithi005 ceph-mon[31197]: pgmap v384: 97 pgs: 97 active+clean; 21 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 424 KiB/s rd, 452 KiB/s wr, 31 op/s 2024-09-12T02:59:53.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:59:53 smithi005 ceph-mon[31197]: scrub summary: active paths [/] 2024-09-12T02:59:53.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:59:53 smithi005 ceph-mon[31197]: scrub summary: idle+waiting paths [/] 2024-09-12T02:59:53.415 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:59:53 smithi148 ceph-mon[35090]: pgmap v384: 97 pgs: 97 active+clean; 21 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 424 KiB/s rd, 452 KiB/s wr, 31 op/s 2024-09-12T02:59:53.415 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:59:53 smithi148 ceph-mon[35090]: scrub summary: active paths [/] 2024-09-12T02:59:53.415 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:59:53 smithi148 ceph-mon[35090]: scrub summary: idle+waiting paths [/] 2024-09-12T02:59:53.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:59:53 smithi146 ceph-mon[35011]: pgmap v384: 97 pgs: 97 active+clean; 21 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 424 KiB/s rd, 452 KiB/s wr, 31 op/s 2024-09-12T02:59:53.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:59:53 smithi146 ceph-mon[35011]: scrub summary: active paths [/] 2024-09-12T02:59:53.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:59:53 smithi146 ceph-mon[35011]: scrub summary: idle+waiting paths [/] 2024-09-12T02:59:54.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:59:54 smithi005 ceph-mon[31197]: scrub summary: active paths [/] 2024-09-12T02:59:54.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:59:54 smithi005 ceph-mon[31197]: scrub summary: idle+waiting paths [/] 2024-09-12T02:59:54.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:59:54 smithi005 ceph-mon[31197]: scrub summary: idle+waiting paths [/] 2024-09-12T02:59:54.415 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:59:54 smithi148 ceph-mon[35090]: scrub summary: active paths [/] 2024-09-12T02:59:54.415 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:59:54 smithi148 ceph-mon[35090]: scrub summary: idle+waiting paths [/] 2024-09-12T02:59:54.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:59:54 smithi148 ceph-mon[35090]: scrub summary: idle+waiting paths [/] 2024-09-12T02:59:54.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:59:54 smithi146 ceph-mon[35011]: scrub summary: active paths [/] 2024-09-12T02:59:54.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:59:54 smithi146 ceph-mon[35011]: scrub summary: idle+waiting paths [/] 2024-09-12T02:59:54.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:59:54 smithi146 ceph-mon[35011]: scrub summary: idle+waiting paths [/] 2024-09-12T02:59:55.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:59:55 smithi005 ceph-mon[31197]: pgmap v385: 97 pgs: 97 active+clean; 23 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 578 KiB/s rd, 636 KiB/s wr, 38 op/s 2024-09-12T02:59:55.415 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:59:55 smithi148 ceph-mon[35090]: pgmap v385: 97 pgs: 97 active+clean; 23 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 578 KiB/s rd, 636 KiB/s wr, 38 op/s 2024-09-12T02:59:55.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:59:55 smithi146 ceph-mon[35011]: pgmap v385: 97 pgs: 97 active+clean; 23 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 578 KiB/s rd, 636 KiB/s wr, 38 op/s 2024-09-12T02:59:57.171 INFO:tasks.workunit.client.0.smithi005.stderr:Note: switching to '0e5c12dfd008efc2848c98108c9237487e91ef35'. 2024-09-12T02:59:57.171 INFO:tasks.workunit.client.0.smithi005.stderr: 2024-09-12T02:59:57.171 INFO:tasks.workunit.client.0.smithi005.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-12T02:59:57.171 INFO:tasks.workunit.client.0.smithi005.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-12T02:59:57.171 INFO:tasks.workunit.client.0.smithi005.stderr:state without impacting any branches by switching back to a branch. 2024-09-12T02:59:57.171 INFO:tasks.workunit.client.0.smithi005.stderr: 2024-09-12T02:59:57.171 INFO:tasks.workunit.client.0.smithi005.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-12T02:59:57.171 INFO:tasks.workunit.client.0.smithi005.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-12T02:59:57.172 INFO:tasks.workunit.client.0.smithi005.stderr: 2024-09-12T02:59:57.172 INFO:tasks.workunit.client.0.smithi005.stderr: git switch -c 2024-09-12T02:59:57.172 INFO:tasks.workunit.client.0.smithi005.stderr: 2024-09-12T02:59:57.172 INFO:tasks.workunit.client.0.smithi005.stderr:Or undo this operation with: 2024-09-12T02:59:57.172 INFO:tasks.workunit.client.0.smithi005.stderr: 2024-09-12T02:59:57.172 INFO:tasks.workunit.client.0.smithi005.stderr: git switch - 2024-09-12T02:59:57.172 INFO:tasks.workunit.client.0.smithi005.stderr: 2024-09-12T02:59:57.172 INFO:tasks.workunit.client.0.smithi005.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-12T02:59:57.172 INFO:tasks.workunit.client.0.smithi005.stderr: 2024-09-12T02:59:57.172 INFO:tasks.workunit.client.0.smithi005.stderr:HEAD is now at 0e5c12df xfs/522: use reflink instead of crc as test feature 2024-09-12T02:59:57.173 INFO:tasks.workunit.client.0.smithi005.stderr:+ make -j4 2024-09-12T02:59:57.182 INFO:tasks.workunit.client.0.smithi005.stdout:libtoolize -cfi 2024-09-12T02:59:57.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:59:57 smithi005 ceph-mon[31197]: pgmap v386: 97 pgs: 97 active+clean; 30 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 889 KiB/s rd, 995 KiB/s wr, 32 op/s 2024-09-12T02:59:57.415 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:59:57 smithi148 ceph-mon[35090]: pgmap v386: 97 pgs: 97 active+clean; 30 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 889 KiB/s rd, 995 KiB/s wr, 32 op/s 2024-09-12T02:59:57.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:59:57 smithi146 ceph-mon[35011]: pgmap v386: 97 pgs: 97 active+clean; 30 MiB data, 455 MiB used, 1.0 TiB / 1.0 TiB avail; 889 KiB/s rd, 995 KiB/s wr, 32 op/s 2024-09-12T02:59:58.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:59:58 smithi005 ceph-mon[31197]: scrub summary: idle 2024-09-12T02:59:58.316 INFO:tasks.workunit.client.0.smithi005.stdout:libtoolize: putting auxiliary files in '.'. 2024-09-12T02:59:58.316 INFO:tasks.workunit.client.0.smithi005.stdout:libtoolize: copying file './config.guess' 2024-09-12T02:59:58.321 INFO:tasks.workunit.client.0.smithi005.stdout:libtoolize: copying file './config.sub' 2024-09-12T02:59:58.327 INFO:tasks.workunit.client.0.smithi005.stdout:libtoolize: copying file './install-sh' 2024-09-12T02:59:58.332 INFO:tasks.workunit.client.0.smithi005.stdout:libtoolize: copying file './ltmain.sh' 2024-09-12T02:59:58.357 INFO:tasks.workunit.client.0.smithi005.stdout:libtoolize: putting macros in AC_CONFIG_MACRO_DIRS, 'm4'. 2024-09-12T02:59:58.357 INFO:tasks.workunit.client.0.smithi005.stdout:libtoolize: copying file 'm4/libtool.m4' 2024-09-12T02:59:58.362 INFO:tasks.workunit.client.0.smithi005.stdout:libtoolize: copying file 'm4/ltoptions.m4' 2024-09-12T02:59:58.369 INFO:tasks.workunit.client.0.smithi005.stdout:libtoolize: copying file 'm4/ltsugar.m4' 2024-09-12T02:59:58.375 INFO:tasks.workunit.client.0.smithi005.stdout:libtoolize: copying file 'm4/ltversion.m4' 2024-09-12T02:59:58.381 INFO:tasks.workunit.client.0.smithi005.stdout:libtoolize: copying file 'm4/lt~obsolete.m4' 2024-09-12T02:59:58.388 INFO:tasks.workunit.client.0.smithi005.stdout:libtoolize: Consider adding '-I m4' to ACLOCAL_AMFLAGS in Makefile.am. 2024-09-12T02:59:58.389 INFO:tasks.workunit.client.0.smithi005.stdout:cp include/install-sh . 2024-09-12T02:59:58.400 INFO:tasks.workunit.client.0.smithi005.stdout:aclocal -I m4 2024-09-12T02:59:58.415 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:59:58 smithi148 ceph-mon[35090]: scrub summary: idle 2024-09-12T02:59:58.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:59:58 smithi146 ceph-mon[35011]: scrub summary: idle 2024-09-12T02:59:59.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:59:59 smithi005 ceph-mon[31197]: pgmap v387: 97 pgs: 97 active+clean; 34 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 33 op/s 2024-09-12T02:59:59.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:59:59 smithi005 ceph-mon[31197]: scrub summary: idle 2024-09-12T02:59:59.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 02:59:59 smithi005 ceph-mon[31197]: scrub summary: idle 2024-09-12T02:59:59.415 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:59:59 smithi148 ceph-mon[35090]: pgmap v387: 97 pgs: 97 active+clean; 34 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 33 op/s 2024-09-12T02:59:59.415 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:59:59 smithi148 ceph-mon[35090]: scrub summary: idle 2024-09-12T02:59:59.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 02:59:59 smithi148 ceph-mon[35090]: scrub summary: idle 2024-09-12T02:59:59.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:59:59 smithi146 ceph-mon[35011]: pgmap v387: 97 pgs: 97 active+clean; 34 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 33 op/s 2024-09-12T02:59:59.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:59:59 smithi146 ceph-mon[35011]: scrub summary: idle 2024-09-12T02:59:59.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 02:59:59 smithi146 ceph-mon[35011]: scrub summary: idle 2024-09-12T02:59:59.574 INFO:tasks.workunit.client.0.smithi005.stdout:autoheader 2024-09-12T02:59:59.911 INFO:tasks.workunit.client.0.smithi005.stdout:autoconf 2024-09-12T03:00:00.101 INFO:tasks.workunit.client.0.smithi005.stdout:./configure \ 2024-09-12T03:00:00.101 INFO:tasks.workunit.client.0.smithi005.stdout: --libexecdir=/usr/lib \ 2024-09-12T03:00:00.101 INFO:tasks.workunit.client.0.smithi005.stdout: --exec_prefix=/var/lib 2024-09-12T03:00:00.244 INFO:tasks.workunit.client.0.smithi005.stdout:checking build system type... x86_64-pc-linux-gnu 2024-09-12T03:00:00.245 INFO:tasks.workunit.client.0.smithi005.stdout:checking host system type... x86_64-pc-linux-gnu 2024-09-12T03:00:00.245 INFO:tasks.workunit.client.0.smithi005.stdout:checking how to print strings... printf 2024-09-12T03:00:00.246 INFO:tasks.workunit.client.0.smithi005.stdout:checking for gcc... gcc 2024-09-12T03:00:00.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:00:00 smithi005 ceph-mon[31197]: overall HEALTH_OK 2024-09-12T03:00:00.328 INFO:tasks.workunit.client.0.smithi005.stdout:checking whether the C compiler works... yes 2024-09-12T03:00:00.328 INFO:tasks.workunit.client.0.smithi005.stdout:checking for C compiler default output file name... a.out 2024-09-12T03:00:00.365 INFO:tasks.workunit.client.0.smithi005.stdout:checking for suffix of executables... 2024-09-12T03:00:00.403 INFO:tasks.workunit.client.0.smithi005.stdout:checking whether we are cross compiling... no 2024-09-12T03:00:00.415 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:00:00 smithi148 ceph-mon[35090]: overall HEALTH_OK 2024-09-12T03:00:00.439 INFO:tasks.workunit.client.0.smithi005.stdout:checking for suffix of object files... o 2024-09-12T03:00:00.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:00:00 smithi146 ceph-mon[35011]: overall HEALTH_OK 2024-09-12T03:00:00.463 INFO:tasks.workunit.client.0.smithi005.stdout:checking whether we are using the GNU C compiler... yes 2024-09-12T03:00:00.487 INFO:tasks.workunit.client.0.smithi005.stdout:checking whether gcc accepts -g... yes 2024-09-12T03:00:00.560 INFO:tasks.workunit.client.0.smithi005.stdout:checking for gcc option to accept ISO C89... none needed 2024-09-12T03:00:00.564 INFO:tasks.workunit.client.0.smithi005.stdout:checking for a sed that does not truncate output... /usr/bin/sed 2024-09-12T03:00:00.567 INFO:tasks.workunit.client.0.smithi005.stdout:checking for grep that handles long lines and -e... /usr/bin/grep 2024-09-12T03:00:00.569 INFO:tasks.workunit.client.0.smithi005.stdout:checking for egrep... /usr/bin/grep -E 2024-09-12T03:00:00.570 INFO:tasks.workunit.client.0.smithi005.stdout:checking for fgrep... /usr/bin/grep -F 2024-09-12T03:00:00.573 INFO:tasks.workunit.client.0.smithi005.stdout:checking for ld used by gcc... /usr/bin/ld 2024-09-12T03:00:00.575 INFO:tasks.workunit.client.0.smithi005.stdout:checking if the linker (/usr/bin/ld) is GNU ld... yes 2024-09-12T03:00:00.589 INFO:tasks.workunit.client.0.smithi005.stdout:checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B 2024-09-12T03:00:01.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:00:01 smithi005 ceph-mon[31197]: pgmap v388: 97 pgs: 97 active+clean; 39 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 36 op/s 2024-09-12T03:00:01.415 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:00:01 smithi148 ceph-mon[35090]: pgmap v388: 97 pgs: 97 active+clean; 39 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 36 op/s 2024-09-12T03:00:01.419 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 13.4 2024-09-12T03:00:01.419 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:00:01.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:00:01 smithi146 ceph-mon[35011]: pgmap v388: 97 pgs: 97 active+clean; 39 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 36 op/s 2024-09-12T03:00:01.904 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:00:02.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:02.569+0000 7fa2e8585640 1 -- 172.21.15.5:0/1016907430 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2e0101280 msgr2=0x7fa2e010df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:02.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:02.569+0000 7fa2e8585640 1 --2- 172.21.15.5:0/1016907430 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2e0101280 0x7fa2e010df40 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fa2d0009900 tx=0x7fa2d002a410 comp rx=0 tx=0).stop 2024-09-12T03:00:02.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:02.569+0000 7fa2e8585640 1 -- 172.21.15.5:0/1016907430 shutdown_connections 2024-09-12T03:00:02.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:02.569+0000 7fa2e8585640 1 --2- 172.21.15.5:0/1016907430 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2e0101280 0x7fa2e010df40 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:02.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:02.569+0000 7fa2e8585640 1 --2- 172.21.15.5:0/1016907430 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa2e010e480 0x7fa2e0110870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:02.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:02.569+0000 7fa2e8585640 1 --2- 172.21.15.5:0/1016907430 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2e01008d0 0x7fa2e0100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:02.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:02.569+0000 7fa2e8585640 1 -- 172.21.15.5:0/1016907430 >> 172.21.15.5:0/1016907430 conn(0x7fa2e00fc7a0 msgr2=0x7fa2e00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:02.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:02.569+0000 7fa2e8585640 1 -- 172.21.15.5:0/1016907430 shutdown_connections 2024-09-12T03:00:02.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:02.569+0000 7fa2e8585640 1 -- 172.21.15.5:0/1016907430 wait complete. 2024-09-12T03:00:02.599 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:02.569+0000 7fa2e8585640 1 Processor -- start 2024-09-12T03:00:02.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:02.570+0000 7fa2e8585640 1 -- start start 2024-09-12T03:00:02.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:02.570+0000 7fa2e8585640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa2e01008d0 0x7fa2e019a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:02.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:02.570+0000 7fa2e8585640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2e0101280 0x7fa2e019ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:02.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:02.570+0000 7fa2e8585640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2e010e480 0x7fa2e019efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:02.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:02.570+0000 7fa2e8585640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e0114200 con 0x7fa2e010e480 2024-09-12T03:00:02.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:02.570+0000 7fa2e8585640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e0114080 con 0x7fa2e01008d0 2024-09-12T03:00:02.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:02.570+0000 7fa2e8585640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e0114380 con 0x7fa2e0101280 2024-09-12T03:00:02.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:02.570+0000 7fa2e6b22640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2e010e480 0x7fa2e019efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:02.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:02.570+0000 7fa2e6b22640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2e010e480 0x7fa2e019efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33096/0 (socket says 172.21.15.5:33096) 2024-09-12T03:00:02.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:02.570+0000 7fa2e6b22640 1 -- 172.21.15.5:0/2047826164 learned_addr learned my addr 172.21.15.5:0/2047826164 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:00:02.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:02.570+0000 7fa2e5b20640 1 --2- 172.21.15.5:0/2047826164 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2e0101280 0x7fa2e019ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:02.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:02.570+0000 7fa2e6321640 1 --2- 172.21.15.5:0/2047826164 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa2e01008d0 0x7fa2e019a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:02.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:02.570+0000 7fa2e6b22640 1 -- 172.21.15.5:0/2047826164 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2e0101280 msgr2=0x7fa2e019ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:02.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:02.570+0000 7fa2e6b22640 1 --2- 172.21.15.5:0/2047826164 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2e0101280 0x7fa2e019ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:02.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:02.570+0000 7fa2e6b22640 1 -- 172.21.15.5:0/2047826164 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa2e01008d0 msgr2=0x7fa2e019a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:02.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:02.570+0000 7fa2e6b22640 1 --2- 172.21.15.5:0/2047826164 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa2e01008d0 0x7fa2e019a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:02.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:02.570+0000 7fa2e6b22640 1 -- 172.21.15.5:0/2047826164 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2e019f6b0 con 0x7fa2e010e480 2024-09-12T03:00:02.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:02.570+0000 7fa2e5b20640 1 --2- 172.21.15.5:0/2047826164 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2e0101280 0x7fa2e019ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:00:02.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:02.570+0000 7fa2e6321640 1 --2- 172.21.15.5:0/2047826164 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa2e01008d0 0x7fa2e019a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:00:02.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:02.571+0000 7fa2e6b22640 1 --2- 172.21.15.5:0/2047826164 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2e010e480 0x7fa2e019efd0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fa2d8009fd0 tx=0x7fa2d800ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:02.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:02.571+0000 7fa2d77fe640 1 -- 172.21.15.5:0/2047826164 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d8004d20 con 0x7fa2e010e480 2024-09-12T03:00:02.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:02.571+0000 7fa2d77fe640 1 -- 172.21.15.5:0/2047826164 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2d8005020 con 0x7fa2e010e480 2024-09-12T03:00:02.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:02.571+0000 7fa2e8585640 1 -- 172.21.15.5:0/2047826164 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2e019f9a0 con 0x7fa2e010e480 2024-09-12T03:00:02.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:02.571+0000 7fa2d77fe640 1 -- 172.21.15.5:0/2047826164 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d80116d0 con 0x7fa2e010e480 2024-09-12T03:00:02.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:02.571+0000 7fa2e8585640 1 -- 172.21.15.5:0/2047826164 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2e01a7280 con 0x7fa2e010e480 2024-09-12T03:00:02.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:02.572+0000 7fa2e8585640 1 -- 172.21.15.5:0/2047826164 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2e0100cb0 con 0x7fa2e010e480 2024-09-12T03:00:02.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:02.573+0000 7fa2d77fe640 1 -- 172.21.15.5:0/2047826164 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fa2d80051d0 con 0x7fa2e010e480 2024-09-12T03:00:02.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:02.574+0000 7fa2d77fe640 1 --2- 172.21.15.5:0/2047826164 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fa2cc0a2c00 0x7fa2cc0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:02.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:02.574+0000 7fa2e6321640 1 --2- 172.21.15.5:0/2047826164 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fa2cc0a2c00 0x7fa2cc0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:02.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:02.574+0000 7fa2d77fe640 1 -- 172.21.15.5:0/2047826164 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fa2d80cf510 con 0x7fa2e010e480 2024-09-12T03:00:02.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:02.574+0000 7fa2e6321640 1 --2- 172.21.15.5:0/2047826164 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fa2cc0a2c00 0x7fa2cc0a50c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc009c20 tx=0x7fa2dc005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:02.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:02.578+0000 7fa2d77fe640 1 -- 172.21.15.5:0/2047826164 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa2d8016030 con 0x7fa2e010e480 2024-09-12T03:00:02.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:02.795+0000 7fa2e8585640 1 -- 172.21.15.5:0/2047826164 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa2e0109610 con 0x7fa2e010e480 2024-09-12T03:00:02.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:02.796+0000 7fa2d77fe640 1 -- 172.21.15.5:0/2047826164 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11969 (secure 0 0 0) 0x7fa2d809c480 con 0x7fa2e010e480 2024-09-12T03:00:02.797 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-12T03:00:02.797 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":27,"btime":"2024-09-12T02:59:32:973410+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/4135232521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":4135232521},{"type":"v1","addr":"172.21.15.5:6835","nonce":4135232521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/4290221940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":4290221940},{"type":"v1","addr":"172.21.15.5:6837","nonce":4290221940}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T02:56:16.595562+0000","modified":"2024-09-12T02:59:32.675420+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24502,"mds_1":24487,"mds_2":24481,"mds_3":24479,"mds_4":24458},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/26447341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":26447341},{"type":"v1","addr":"172.21.15.5:6839","nonce":26447341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2393443295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2393443295},{"type":"v1","addr":"172.21.15.5:6841","nonce":2393443295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6833/3686970137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3686970137},{"type":"v1","addr":"172.21.15.146:6833","nonce":3686970137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6833/568512209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":568512209},{"type":"v1","addr":"172.21.15.148:6833","nonce":568512209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6835/2540740420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2540740420},{"type":"v1","addr":"172.21.15.146:6835","nonce":2540740420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.146:6837/1673370937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":1673370937},{"type":"v1","addr":"172.21.15.146:6837","nonce":1673370937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":3,"incarnation":22,"state":"up:active","state_seq":6,"addr":"172.21.15.146:6839/3036767228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":3036767228},{"type":"v1","addr":"172.21.15.146:6839","nonce":3036767228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":2,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.148:6835/4195086427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":4195086427},{"type":"v1","addr":"172.21.15.148:6835","nonce":4195086427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.148:6837/2606873922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2606873922},{"type":"v1","addr":"172.21.15.148:6837","nonce":2606873922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6839/2317600000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":2317600000},{"type":"v1","addr":"172.21.15.148:6839","nonce":2317600000}]},"join_fscid":-1,"export_targets":[1,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24502,"qdb_cluster":[24458,24479,24481,24487,24502]},"id":1}]} 2024-09-12T03:00:02.797 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 27 2024-09-12T03:00:02.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:02.798+0000 7fa2e8585640 1 -- 172.21.15.5:0/2047826164 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fa2cc0a2c00 msgr2=0x7fa2cc0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:02.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:02.798+0000 7fa2e8585640 1 --2- 172.21.15.5:0/2047826164 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fa2cc0a2c00 0x7fa2cc0a50c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fa2dc009c20 tx=0x7fa2dc005f90 comp rx=0 tx=0).stop 2024-09-12T03:00:02.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:02.798+0000 7fa2e8585640 1 -- 172.21.15.5:0/2047826164 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2e010e480 msgr2=0x7fa2e019efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:02.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:02.798+0000 7fa2e8585640 1 --2- 172.21.15.5:0/2047826164 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2e010e480 0x7fa2e019efd0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fa2d8009fd0 tx=0x7fa2d800ee60 comp rx=0 tx=0).stop 2024-09-12T03:00:02.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:02.798+0000 7fa2e8585640 1 -- 172.21.15.5:0/2047826164 shutdown_connections 2024-09-12T03:00:02.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:02.798+0000 7fa2e8585640 1 --2- 172.21.15.5:0/2047826164 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fa2cc0a2c00 0x7fa2cc0a50c0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:02.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:02.798+0000 7fa2e8585640 1 --2- 172.21.15.5:0/2047826164 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2e010e480 0x7fa2e019efd0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:02.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:02.798+0000 7fa2e8585640 1 --2- 172.21.15.5:0/2047826164 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2e0101280 0x7fa2e019ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:02.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:02.798+0000 7fa2e8585640 1 --2- 172.21.15.5:0/2047826164 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa2e01008d0 0x7fa2e019a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:02.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:02.798+0000 7fa2e8585640 1 -- 172.21.15.5:0/2047826164 >> 172.21.15.5:0/2047826164 conn(0x7fa2e00fc7a0 msgr2=0x7fa2e00feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:02.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:02.798+0000 7fa2e8585640 1 -- 172.21.15.5:0/2047826164 shutdown_connections 2024-09-12T03:00:02.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:02.799+0000 7fa2e8585640 1 -- 172.21.15.5:0/2047826164 wait complete. 2024-09-12T03:00:03.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:00:03 smithi005 ceph-mon[31197]: pgmap v389: 97 pgs: 97 active+clean; 39 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 31 op/s 2024-09-12T03:00:03.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:00:03 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/2047826164' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:00:03.370 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24502 quiesce db / --timeout 90 --expiration 133.4 --await-for 45.0' 2024-09-12T03:00:03.370 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24502 quiesce db / --timeout 90 --expiration 133.4 --await-for 45.0 2024-09-12T03:00:03.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:00:03 smithi148 ceph-mon[35090]: pgmap v389: 97 pgs: 97 active+clean; 39 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 31 op/s 2024-09-12T03:00:03.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:00:03 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/2047826164' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:00:03.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:00:03 smithi146 ceph-mon[35011]: pgmap v389: 97 pgs: 97 active+clean; 39 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 31 op/s 2024-09-12T03:00:03.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:00:03 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/2047826164' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:00:03.677 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:00:04.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:00:04 smithi005 ceph-mon[31197]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:00:04.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.405+0000 7fa2dffc4640 1 -- 172.21.15.5:0/2983572260 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2d8102d30 msgr2=0x7fa2d8103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:04.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.405+0000 7fa2dffc4640 1 --2- 172.21.15.5:0/2983572260 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2d8102d30 0x7fa2d8103110 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fa2d4009930 tx=0x7fa2d402a420 comp rx=0 tx=0).stop 2024-09-12T03:00:04.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.405+0000 7fa2dffc4640 1 -- 172.21.15.5:0/2983572260 shutdown_connections 2024-09-12T03:00:04.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.405+0000 7fa2dffc4640 1 --2- 172.21.15.5:0/2983572260 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa2d81082a0 0x7fa2d810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:04.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.405+0000 7fa2dffc4640 1 --2- 172.21.15.5:0/2983572260 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2d81036e0 0x7fa2d8107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:04.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.405+0000 7fa2dffc4640 1 --2- 172.21.15.5:0/2983572260 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2d8102d30 0x7fa2d8103110 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:04.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.405+0000 7fa2dffc4640 1 -- 172.21.15.5:0/2983572260 >> 172.21.15.5:0/2983572260 conn(0x7fa2d80fc7d0 msgr2=0x7fa2d80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:04.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.405+0000 7fa2dffc4640 1 -- 172.21.15.5:0/2983572260 shutdown_connections 2024-09-12T03:00:04.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.405+0000 7fa2dffc4640 1 -- 172.21.15.5:0/2983572260 wait complete. 2024-09-12T03:00:04.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.406+0000 7fa2dffc4640 1 Processor -- start 2024-09-12T03:00:04.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.406+0000 7fa2dffc4640 1 -- start start 2024-09-12T03:00:04.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.406+0000 7fa2dffc4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2d8102d30 0x7fa2d81985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:04.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.406+0000 7fa2dffc4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2d81036e0 0x7fa2d8198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:04.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.406+0000 7fa2dffc4640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa2d81082a0 0x7fa2d819ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:04.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.406+0000 7fa2dffc4640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d81120a0 con 0x7fa2d81036e0 2024-09-12T03:00:04.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.406+0000 7fa2dffc4640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d8111f20 con 0x7fa2d81082a0 2024-09-12T03:00:04.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.406+0000 7fa2dffc4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d8112220 con 0x7fa2d8102d30 2024-09-12T03:00:04.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.406+0000 7fa2ddd60640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2d8102d30 0x7fa2d81985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:04.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.406+0000 7fa2ddd60640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2d8102d30 0x7fa2d81985a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.5:55386/0 (socket says 172.21.15.5:55386) 2024-09-12T03:00:04.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.406+0000 7fa2de561640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa2d81082a0 0x7fa2d819ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:04.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.406+0000 7fa2ddd60640 1 -- 172.21.15.5:0/3264660897 learned_addr learned my addr 172.21.15.5:0/3264660897 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:00:04.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.406+0000 7fa2dd55f640 1 --2- 172.21.15.5:0/3264660897 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2d81036e0 0x7fa2d8198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:04.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.407+0000 7fa2ddd60640 1 -- 172.21.15.5:0/3264660897 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa2d81082a0 msgr2=0x7fa2d819ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:04.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.407+0000 7fa2ddd60640 1 --2- 172.21.15.5:0/3264660897 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa2d81082a0 0x7fa2d819ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:04.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.407+0000 7fa2ddd60640 1 -- 172.21.15.5:0/3264660897 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2d81036e0 msgr2=0x7fa2d8198ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:04.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.407+0000 7fa2ddd60640 1 --2- 172.21.15.5:0/3264660897 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2d81036e0 0x7fa2d8198ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:04.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.407+0000 7fa2ddd60640 1 -- 172.21.15.5:0/3264660897 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2d819d550 con 0x7fa2d8102d30 2024-09-12T03:00:04.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.407+0000 7fa2de561640 1 --2- 172.21.15.5:0/3264660897 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa2d81082a0 0x7fa2d819ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:00:04.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.407+0000 7fa2ddd60640 1 --2- 172.21.15.5:0/3264660897 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2d8102d30 0x7fa2d81985a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa2d4009230 tx=0x7fa2d4004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:04.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.407+0000 7fa2ceffd640 1 -- 172.21.15.5:0/3264660897 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d403f030 con 0x7fa2d8102d30 2024-09-12T03:00:04.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.407+0000 7fa2ceffd640 1 -- 172.21.15.5:0/3264660897 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2d4004440 con 0x7fa2d8102d30 2024-09-12T03:00:04.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.407+0000 7fa2dffc4640 1 -- 172.21.15.5:0/3264660897 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2d819d7e0 con 0x7fa2d8102d30 2024-09-12T03:00:04.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.407+0000 7fa2ceffd640 1 -- 172.21.15.5:0/3264660897 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d4035070 con 0x7fa2d8102d30 2024-09-12T03:00:04.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.407+0000 7fa2dffc4640 1 -- 172.21.15.5:0/3264660897 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2d81a5120 con 0x7fa2d8102d30 2024-09-12T03:00:04.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.410+0000 7fa2e239d740 1 Processor -- start 2024-09-12T03:00:04.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.410+0000 7fa2e239d740 1 -- start start 2024-09-12T03:00:04.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.410+0000 7fa2ceffd640 1 -- 172.21.15.5:0/3264660897 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fa2d4042030 con 0x7fa2d8102d30 2024-09-12T03:00:04.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.410+0000 7fa2e239d740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fd5cd87640 0x55fd5cd87a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:04.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.410+0000 7fa2e239d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd5cd87f60 0x55fd5cd96810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:04.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.410+0000 7fa2e239d740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fd5cd96d50 0x55fd5cd99140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:04.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.410+0000 7fa2e239d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55fd5cced2a0 con 0x55fd5cd87f60 2024-09-12T03:00:04.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.410+0000 7fa2e239d740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55fd5cc94b10 con 0x55fd5cd87640 2024-09-12T03:00:04.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.410+0000 7fa2e239d740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55fd5cce9260 con 0x55fd5cd96d50 2024-09-12T03:00:04.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.410+0000 7fa2ddd60640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd5cd87f60 0x55fd5cd96810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:04.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.410+0000 7fa2ddd60640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd5cd87f60 0x55fd5cd96810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60270/0 (socket says 172.21.15.5:60270) 2024-09-12T03:00:04.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.410+0000 7fa2ddd60640 1 -- 172.21.15.5:0/1246563736 learned_addr learned my addr 172.21.15.5:0/1246563736 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:00:04.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.410+0000 7fa2dd55f640 1 --2- 172.21.15.5:0/1246563736 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fd5cd96d50 0x55fd5cd99140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:04.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.410+0000 7fa2de561640 1 --2- 172.21.15.5:0/1246563736 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fd5cd87640 0x55fd5cd87a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:04.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.411+0000 7fa2dd55f640 1 -- 172.21.15.5:0/1246563736 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fd5cd87640 msgr2=0x55fd5cd87a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:00:04.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.411+0000 7fa2dd55f640 1 --2- 172.21.15.5:0/1246563736 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fd5cd87640 0x55fd5cd87a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:04.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.411+0000 7fa2dd55f640 1 -- 172.21.15.5:0/1246563736 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd5cd87f60 msgr2=0x55fd5cd96810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:00:04.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.411+0000 7fa2dd55f640 1 --2- 172.21.15.5:0/1246563736 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd5cd87f60 0x55fd5cd96810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:04.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.411+0000 7fa2dd55f640 1 -- 172.21.15.5:0/1246563736 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd5ca5eb90 con 0x55fd5cd96d50 2024-09-12T03:00:04.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.411+0000 7fa2ddd60640 1 --2- 172.21.15.5:0/1246563736 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd5cd87f60 0x55fd5cd96810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:00:04.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.411+0000 7fa2ceffd640 1 --2- 172.21.15.5:0/3264660897 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fa2c40a2b80 0x7fa2c40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:04.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.411+0000 7fa2dd55f640 1 --2- 172.21.15.5:0/1246563736 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fd5cd96d50 0x55fd5cd99140 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa2d806a370 tx=0x7fa2c8011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12d9ad14567dda6a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:04.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.411+0000 7fa2de561640 1 --2- 172.21.15.5:0/3264660897 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fa2c40a2b80 0x7fa2c40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:04.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.411+0000 7fa2ceffd640 1 -- 172.21.15.5:0/3264660897 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fa2d40ef350 con 0x7fa2d8102d30 2024-09-12T03:00:04.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.411+0000 7fa2c3fff640 1 -- 172.21.15.5:0/1246563736 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c8004050 con 0x55fd5cd96d50 2024-09-12T03:00:04.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.411+0000 7fa2c3fff640 1 -- 172.21.15.5:0/1246563736 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2c8005500 con 0x55fd5cd96d50 2024-09-12T03:00:04.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.411+0000 7fa2c3fff640 1 -- 172.21.15.5:0/1246563736 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c8004e00 con 0x55fd5cd96d50 2024-09-12T03:00:04.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.412+0000 7fa2e239d740 1 -- 172.21.15.5:0/1246563736 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fd5cd96d50 msgr2=0x55fd5cd99140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:04.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.412+0000 7fa2e239d740 1 --2- 172.21.15.5:0/1246563736 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fd5cd96d50 0x55fd5cd99140 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa2d806a370 tx=0x7fa2c8011810 comp rx=0 tx=0).stop 2024-09-12T03:00:04.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.412+0000 7fa2de561640 1 --2- 172.21.15.5:0/3264660897 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fa2c40a2b80 0x7fa2c40a5040 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa2d0009fd0 tx=0x7fa2d0009500 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:04.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.412+0000 7fa2e239d740 1 -- 172.21.15.5:0/1246563736 shutdown_connections 2024-09-12T03:00:04.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.412+0000 7fa2e239d740 1 --2- 172.21.15.5:0/1246563736 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd5cd87f60 0x55fd5cd96810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:04.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.412+0000 7fa2e239d740 1 --2- 172.21.15.5:0/1246563736 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fd5cd96d50 0x55fd5cd99140 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:04.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.412+0000 7fa2e239d740 1 --2- 172.21.15.5:0/1246563736 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fd5cd87640 0x55fd5cd87a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:04.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.412+0000 7fa2e239d740 1 -- 172.21.15.5:0/1246563736 >> 172.21.15.5:0/1246563736 conn(0x55fd5cd83cf0 msgr2=0x55fd5cd840d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:04.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.412+0000 7fa2e239d740 1 -- 172.21.15.5:0/1246563736 shutdown_connections 2024-09-12T03:00:04.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.412+0000 7fa2e239d740 1 -- 172.21.15.5:0/1246563736 wait complete. 2024-09-12T03:00:04.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.412+0000 7fa2e239d740 1 -- start start 2024-09-12T03:00:04.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.412+0000 7fa2e239d740 1 Processor -- start 2024-09-12T03:00:04.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.413+0000 7fa2e239d740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fd5cd87640 0x55fd5cd93c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:04.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.413+0000 7fa2e239d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd5cd87f60 0x55fd5cd941d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:04.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.413+0000 7fa2e239d740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fd5cd96d50 0x55fd5cd94710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:04.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.413+0000 7fa2e239d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55fd5cd99920 con 0x55fd5cd87f60 2024-09-12T03:00:04.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.413+0000 7fa2e239d740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55fd5cc756e0 con 0x55fd5cd96d50 2024-09-12T03:00:04.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.413+0000 7fa2e239d740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55fd5cd99aa0 con 0x55fd5cd87640 2024-09-12T03:00:04.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.413+0000 7fa2e239d740 10 client.0 _ll_register_callbacks cb 0x55fd5cc99a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:00:04.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.413+0000 7fa2dd55f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd5cd87f60 0x55fd5cd941d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:04.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.413+0000 7fa2ddd60640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fd5cd87640 0x55fd5cd93c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:04.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.413+0000 7fa2dd55f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd5cd87f60 0x55fd5cd941d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60272/0 (socket says 172.21.15.5:60272) 2024-09-12T03:00:04.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.413+0000 7fa2dd55f640 1 -- 172.21.15.5:0/3862141606 learned_addr learned my addr 172.21.15.5:0/3862141606 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:00:04.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.413+0000 7fa2de561640 1 --2- 172.21.15.5:0/3862141606 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fd5cd96d50 0x55fd5cd94710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:04.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.413+0000 7fa2ddd60640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fd5cd87640 0x55fd5cd93c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.5:55418/0 (socket says 172.21.15.5:55418) 2024-09-12T03:00:04.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.413+0000 7fa2dd55f640 1 -- 172.21.15.5:0/3862141606 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fd5cd87640 msgr2=0x55fd5cd93c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:00:04.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.413+0000 7fa2dd55f640 1 --2- 172.21.15.5:0/3862141606 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fd5cd87640 0x55fd5cd93c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:04.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.413+0000 7fa2dd55f640 1 -- 172.21.15.5:0/3862141606 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fd5cd96d50 msgr2=0x55fd5cd94710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:00:04.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.413+0000 7fa2dd55f640 1 --2- 172.21.15.5:0/3862141606 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fd5cd96d50 0x55fd5cd94710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:04.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.413+0000 7fa2dd55f640 1 -- 172.21.15.5:0/3862141606 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd5cda2ea0 con 0x55fd5cd87f60 2024-09-12T03:00:04.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.414+0000 7fa2dd55f640 1 --2- 172.21.15.5:0/3862141606 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd5cd87f60 0x55fd5cd941d0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fa2c80044d0 tx=0x7fa2c8005d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f611baa933a8800a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:04.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.414+0000 7fa2c17fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T03:00:04.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.414+0000 7fa2c17fa640 1 -- 172.21.15.5:0/3862141606 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c8028070 con 0x55fd5cd87f60 2024-09-12T03:00:04.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.414+0000 7fa2c17fa640 1 -- 172.21.15.5:0/3862141606 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fa2c8004b80 con 0x55fd5cd87f60 2024-09-12T03:00:04.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.414+0000 7fa2e239d740 1 -- 172.21.15.5:0/3862141606 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2c8005500 con 0x55fd5cd87f60 2024-09-12T03:00:04.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.414+0000 7fa2c17fa640 1 -- 172.21.15.5:0/3862141606 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa2c8034050 con 0x55fd5cd87f60 2024-09-12T03:00:04.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.414+0000 7fa2e239d740 10 client.15807 fetch_fsmap learned FSMap version 28 2024-09-12T03:00:04.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.414+0000 7fa2e239d740 1 -- 172.21.15.5:0/3862141606 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55fd5cda5830 con 0x55fd5cd87f60 2024-09-12T03:00:04.419 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:00:04 smithi148 ceph-mon[35090]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:00:04.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.414+0000 7fa2c17fa640 1 -- 172.21.15.5:0/3862141606 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2c8005500 con 0x55fd5cd87f60 2024-09-12T03:00:04.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.415+0000 7fa2c17fa640 1 -- 172.21.15.5:0/3862141606 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8374+0+0 (secure 0 0 0) 0x7fa2c8003ae0 con 0x55fd5cd87f60 2024-09-12T03:00:04.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.415+0000 7fa2e239d740 10 client.15807 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T03:00:04.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.415+0000 7fa2e239d740 10 client.15807 resolve_mds: validated gid 24502 aka daemon mds.i 2024-09-12T03:00:04.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.415+0000 7fa2e239d740 20 client.15807 populate_metadata read hostname 'smithi005' 2024-09-12T03:00:04.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.415+0000 7fa2e239d740 1 --2- 172.21.15.5:0/3862141606 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55fd5cda6be0 0x55fd5cda8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:04.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.415+0000 7fa2e239d740 4 client.15807 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:00:04.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.415+0000 7fa2e239d740 1 -- 172.21.15.5:0/3862141606 --> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fd5ccfa9d0 con 0x55fd5cda6be0 2024-09-12T03:00:04.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.415+0000 7fa2ddd60640 1 --2- 172.21.15.5:0/3862141606 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55fd5cda6be0 0x55fd5cda8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:04.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.415+0000 7fa2c17fa640 10 client.15807 ms_handle_connect on v2:172.21.15.148:6838/2317600000 2024-09-12T03:00:04.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.415+0000 7fa2ddd60640 1 --2- 172.21.15.5:0/3862141606 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55fd5cda6be0 0x55fd5cda8fd0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e0c640826c5b4d8 server_cookie=f8d7123b7c838915 in_seq=0 out_seq=0 2024-09-12T03:00:04.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.417+0000 7fa2c17fa640 1 -- 172.21.15.5:0/3862141606 <== mds.0 v2:172.21.15.148:6838/2317600000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fd5ccfa9d0 con 0x55fd5cda6be0 2024-09-12T03:00:04.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.417+0000 7fa2c17fa640 10 client.15807 handle_command_reply: tid=0 2024-09-12T03:00:04.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.417+0000 7fa2e239d740 1 client.15807 shutdown 2024-09-12T03:00:04.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.417+0000 7fa2e239d740 1 -- 172.21.15.5:0/3862141606 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd5cd87f60 msgr2=0x55fd5cd941d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:04.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.417+0000 7fa2e239d740 1 --2- 172.21.15.5:0/3862141606 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd5cd87f60 0x55fd5cd941d0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fa2c80044d0 tx=0x7fa2c8005d30 comp rx=0 tx=0).stop 2024-09-12T03:00:04.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.417+0000 7fa2e239d740 1 -- 172.21.15.5:0/3862141606 shutdown_connections 2024-09-12T03:00:04.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.417+0000 7fa2e239d740 1 --2- 172.21.15.5:0/3862141606 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55fd5cda6be0 0x55fd5cda8fd0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:04.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.417+0000 7fa2e239d740 1 --2- 172.21.15.5:0/3862141606 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd5cd87f60 0x55fd5cd941d0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:04.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.417+0000 7fa2e239d740 1 --2- 172.21.15.5:0/3862141606 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fd5cd96d50 0x55fd5cd94710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:04.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.418+0000 7fa2c17fa640 0 client.15807 ms_handle_reset on v2:172.21.15.148:6838/2317600000 2024-09-12T03:00:04.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.418+0000 7fa2e239d740 1 --2- 172.21.15.5:0/3862141606 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fd5cd87640 0x55fd5cd93c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:04.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.418+0000 7fa2e239d740 1 -- 172.21.15.5:0/3862141606 >> 172.21.15.5:0/3862141606 conn(0x55fd5cd83cf0 msgr2=0x55fd5cd89a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:04.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.418+0000 7fa2e239d740 1 -- 172.21.15.5:0/3862141606 shutdown_connections 2024-09-12T03:00:04.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.418+0000 7fa2e239d740 1 -- 172.21.15.5:0/3862141606 wait complete. 2024-09-12T03:00:04.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.418+0000 7fa2e239d740 20 client.15807 trim_cache size 0 max 16384 2024-09-12T03:00:04.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.435+0000 7fa2e239d740 1 Processor -- start 2024-09-12T03:00:04.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.435+0000 7fa2e239d740 1 -- start start 2024-09-12T03:00:04.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.435+0000 7fa2e239d740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fd5cd87640 0x55fd5cd82d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:04.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.435+0000 7fa2e239d740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fd5cd87f60 0x55fd5cdae710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:04.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.435+0000 7fa2e239d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd5cd96d50 0x55fd5cdaec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:04.436 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.435+0000 7fa2e239d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d81ab380 con 0x55fd5cd96d50 2024-09-12T03:00:04.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.435+0000 7fa2e239d740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d81acf50 con 0x55fd5cd87f60 2024-09-12T03:00:04.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.435+0000 7fa2e239d740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d81aba50 con 0x55fd5cd87640 2024-09-12T03:00:04.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.435+0000 7fa2de561640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd5cd96d50 0x55fd5cdaec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:04.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.435+0000 7fa2ddd60640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fd5cd87640 0x55fd5cd82d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:04.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.435+0000 7fa2de561640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd5cd96d50 0x55fd5cdaec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60282/0 (socket says 172.21.15.5:60282) 2024-09-12T03:00:04.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.435+0000 7fa2de561640 1 -- 172.21.15.5:0/2738548076 learned_addr learned my addr 172.21.15.5:0/2738548076 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:00:04.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.435+0000 7fa2dd55f640 1 --2- 172.21.15.5:0/2738548076 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fd5cd87f60 0x55fd5cdae710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:04.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.436+0000 7fa2de561640 1 -- 172.21.15.5:0/2738548076 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fd5cd87640 msgr2=0x55fd5cd82d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:00:04.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.436+0000 7fa2de561640 1 --2- 172.21.15.5:0/2738548076 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fd5cd87640 0x55fd5cd82d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:04.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.436+0000 7fa2de561640 1 -- 172.21.15.5:0/2738548076 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fd5cd87f60 msgr2=0x55fd5cdae710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:00:04.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.436+0000 7fa2de561640 1 --2- 172.21.15.5:0/2738548076 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fd5cd87f60 0x55fd5cdae710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:04.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.436+0000 7fa2de561640 1 -- 172.21.15.5:0/2738548076 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd5cd7f260 con 0x55fd5cd96d50 2024-09-12T03:00:04.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.436+0000 7fa2ddd60640 1 --2- 172.21.15.5:0/2738548076 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fd5cd87640 0x55fd5cd82d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:00:04.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.436+0000 7fa2de561640 1 --2- 172.21.15.5:0/2738548076 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd5cd96d50 0x55fd5cdaec50 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fa2d0003e10 tx=0x7fa2d001b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae89ef448951f786 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:04.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.436+0000 7fa2c37fe640 1 -- 172.21.15.5:0/2738548076 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d000fda0 con 0x55fd5cd96d50 2024-09-12T03:00:04.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.436+0000 7fa2c37fe640 1 -- 172.21.15.5:0/2738548076 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2d00045c0 con 0x55fd5cd96d50 2024-09-12T03:00:04.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.436+0000 7fa2c37fe640 1 -- 172.21.15.5:0/2738548076 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2d001f690 con 0x55fd5cd96d50 2024-09-12T03:00:04.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.437+0000 7fa2e239d740 1 -- 172.21.15.5:0/2738548076 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd5cd96d50 msgr2=0x55fd5cdaec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:04.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.437+0000 7fa2e239d740 1 --2- 172.21.15.5:0/2738548076 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd5cd96d50 0x55fd5cdaec50 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fa2d0003e10 tx=0x7fa2d001b000 comp rx=0 tx=0).stop 2024-09-12T03:00:04.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.437+0000 7fa2e239d740 1 -- 172.21.15.5:0/2738548076 shutdown_connections 2024-09-12T03:00:04.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.437+0000 7fa2e239d740 1 --2- 172.21.15.5:0/2738548076 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd5cd96d50 0x55fd5cdaec50 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:04.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.437+0000 7fa2e239d740 1 --2- 172.21.15.5:0/2738548076 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fd5cd87f60 0x55fd5cdae710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:04.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.437+0000 7fa2e239d740 1 --2- 172.21.15.5:0/2738548076 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fd5cd87640 0x55fd5cd82d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:04.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.437+0000 7fa2e239d740 1 -- 172.21.15.5:0/2738548076 >> 172.21.15.5:0/2738548076 conn(0x55fd5cd83cf0 msgr2=0x55fd5cd82650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:04.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.437+0000 7fa2e239d740 1 -- 172.21.15.5:0/2738548076 shutdown_connections 2024-09-12T03:00:04.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.437+0000 7fa2e239d740 1 -- 172.21.15.5:0/2738548076 wait complete. 2024-09-12T03:00:04.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.437+0000 7fa2e239d740 1 -- start start 2024-09-12T03:00:04.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.437+0000 7fa2e239d740 1 Processor -- start 2024-09-12T03:00:04.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.438+0000 7fa2e239d740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fd5cd87640 0x55fd5cd9b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:04.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.438+0000 7fa2e239d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd5cd87f60 0x55fd5cd9b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:04.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.438+0000 7fa2e239d740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fd5cd96d50 0x55fd5ce0a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:04.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.438+0000 7fa2e239d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d81ac7f0 con 0x55fd5cd87f60 2024-09-12T03:00:04.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.438+0000 7fa2e239d740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d81ac120 con 0x55fd5cd87640 2024-09-12T03:00:04.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.438+0000 7fa2e239d740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa2d81a9360 con 0x55fd5cd96d50 2024-09-12T03:00:04.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.438+0000 7fa2e239d740 10 client.0 _ll_register_callbacks cb 0x7fa2c8009a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:00:04.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.438+0000 7fa2dd55f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd5cd87f60 0x55fd5cd9b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:04.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.438+0000 7fa2dd55f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd5cd87f60 0x55fd5cd9b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60284/0 (socket says 172.21.15.5:60284) 2024-09-12T03:00:04.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.438+0000 7fa2dd55f640 1 -- 172.21.15.5:0/1953780 learned_addr learned my addr 172.21.15.5:0/1953780 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:00:04.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.438+0000 7fa2de561640 1 --2- 172.21.15.5:0/1953780 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fd5cd96d50 0x55fd5ce0a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:04.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.438+0000 7fa2ddd60640 1 --2- 172.21.15.5:0/1953780 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fd5cd87640 0x55fd5cd9b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:04.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.438+0000 7fa2dd55f640 1 -- 172.21.15.5:0/1953780 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fd5cd96d50 msgr2=0x55fd5ce0a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:00:04.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.438+0000 7fa2dd55f640 1 --2- 172.21.15.5:0/1953780 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fd5cd96d50 0x55fd5ce0a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:04.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.438+0000 7fa2dd55f640 1 -- 172.21.15.5:0/1953780 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fd5cd87640 msgr2=0x55fd5cd9b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:00:04.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.438+0000 7fa2dd55f640 1 --2- 172.21.15.5:0/1953780 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fd5cd87640 0x55fd5cd9b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:04.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.438+0000 7fa2dd55f640 1 -- 172.21.15.5:0/1953780 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd5cd88a20 con 0x55fd5cd87f60 2024-09-12T03:00:04.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.438+0000 7fa2dd55f640 1 --2- 172.21.15.5:0/1953780 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd5cd87f60 0x55fd5cd9b8b0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fa2c8000c00 tx=0x7fa2c80235e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90cdecf5bbca7f8a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:04.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.438+0000 7fa2c17fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T03:00:04.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.438+0000 7fa2c17fa640 1 -- 172.21.15.5:0/1953780 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2c803d060 con 0x55fd5cd87f60 2024-09-12T03:00:04.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.438+0000 7fa2c17fa640 1 -- 172.21.15.5:0/1953780 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fa2c80035f0 con 0x55fd5cd87f60 2024-09-12T03:00:04.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.438+0000 7fa2e239d740 1 -- 172.21.15.5:0/1953780 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2d00045c0 con 0x55fd5cd87f60 2024-09-12T03:00:04.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.439+0000 7fa2c17fa640 1 -- 172.21.15.5:0/1953780 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa2c8028040 con 0x55fd5cd87f60 2024-09-12T03:00:04.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.439+0000 7fa2e239d740 10 client.15813 fetch_fsmap learned FSMap version 28 2024-09-12T03:00:04.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.439+0000 7fa2e239d740 1 -- 172.21.15.5:0/1953780 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55fd5ce11eb0 con 0x55fd5cd87f60 2024-09-12T03:00:04.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.439+0000 7fa2c17fa640 1 -- 172.21.15.5:0/1953780 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2d00045c0 con 0x55fd5cd87f60 2024-09-12T03:00:04.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.439+0000 7fa2c17fa640 1 -- 172.21.15.5:0/1953780 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8374+0+0 (secure 0 0 0) 0x7fa2c8037030 con 0x55fd5cd87f60 2024-09-12T03:00:04.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.439+0000 7fa2e239d740 10 client.15813 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T03:00:04.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.439+0000 7fa2e239d740 10 client.15813 resolve_mds: validated gid 24502 aka daemon mds.i 2024-09-12T03:00:04.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.439+0000 7fa2e239d740 20 client.15813 populate_metadata read hostname 'smithi005' 2024-09-12T03:00:04.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.439+0000 7fa2e239d740 1 --2- 172.21.15.5:0/1953780 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55fd5ce12640 0x55fd5ce14a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:04.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.439+0000 7fa2e239d740 4 client.15813 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 133.4, "await_for": 45.0}] 2024-09-12T03:00:04.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.439+0000 7fa2e239d740 1 -- 172.21.15.5:0/1953780 --> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 133.4, "await_for": 45.0}) -- 0x55fd5ccf4d20 con 0x55fd5ce12640 2024-09-12T03:00:04.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.439+0000 7fa2ddd60640 1 --2- 172.21.15.5:0/1953780 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55fd5ce12640 0x55fd5ce14a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:04.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.440+0000 7fa2ddd60640 1 --2- 172.21.15.5:0/1953780 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55fd5ce12640 0x55fd5ce14a60 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d6ea516c076ede7 server_cookie=d70f86479c36d441 in_seq=0 out_seq=0 2024-09-12T03:00:04.444 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:04.440+0000 7fa2c17fa640 10 client.15813 ms_handle_connect on v2:172.21.15.148:6838/2317600000 2024-09-12T03:00:04.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:00:04 smithi146 ceph-mon[35011]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:00:05.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:00:05 smithi005 ceph-mon[31197]: pgmap v390: 97 pgs: 97 active+clean; 45 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 250 op/s 2024-09-12T03:00:05.415 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:00:05 smithi148 ceph-mon[35090]: pgmap v390: 97 pgs: 97 active+clean; 45 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 250 op/s 2024-09-12T03:00:05.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:00:05 smithi146 ceph-mon[35011]: pgmap v390: 97 pgs: 97 active+clean; 45 MiB data, 500 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.2 MiB/s wr, 250 op/s 2024-09-12T03:00:07.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:00:07 smithi005 ceph-mon[31197]: pgmap v391: 97 pgs: 97 active+clean; 56 MiB data, 544 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.1 MiB/s wr, 426 op/s 2024-09-12T03:00:07.415 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:00:07 smithi148 ceph-mon[35090]: pgmap v391: 97 pgs: 97 active+clean; 56 MiB data, 544 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.1 MiB/s wr, 426 op/s 2024-09-12T03:00:07.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:00:07 smithi146 ceph-mon[35011]: pgmap v391: 97 pgs: 97 active+clean; 56 MiB data, 544 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.1 MiB/s wr, 426 op/s 2024-09-12T03:00:09.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:00:09 smithi005 ceph-mon[31197]: pgmap v392: 97 pgs: 97 active+clean; 58 MiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.9 MiB/s wr, 632 op/s 2024-09-12T03:00:09.415 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:00:09 smithi148 ceph-mon[35090]: pgmap v392: 97 pgs: 97 active+clean; 58 MiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.9 MiB/s wr, 632 op/s 2024-09-12T03:00:09.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:00:09 smithi146 ceph-mon[35011]: pgmap v392: 97 pgs: 97 active+clean; 58 MiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.9 MiB/s wr, 632 op/s 2024-09-12T03:00:09.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:09.846+0000 7fa2c17fa640 1 -- 172.21.15.5:0/1953780 <== mds.0 v2:172.21.15.148:6838/2317600000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x55fd5ccf4d20 con 0x55fd5ce12640 2024-09-12T03:00:09.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:09.846+0000 7fa2c17fa640 10 client.15813 handle_command_reply: tid=0 2024-09-12T03:00:09.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:09.846+0000 7fa2e239d740 1 client.15813 shutdown 2024-09-12T03:00:09.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:09.846+0000 7fa2e239d740 1 -- 172.21.15.5:0/1953780 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd5cd87f60 msgr2=0x55fd5cd9b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:09.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:09.846+0000 7fa2e239d740 1 --2- 172.21.15.5:0/1953780 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd5cd87f60 0x55fd5cd9b8b0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fa2c8000c00 tx=0x7fa2c80235e0 comp rx=0 tx=0).stop 2024-09-12T03:00:09.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:09.846+0000 7fa2e239d740 1 -- 172.21.15.5:0/1953780 shutdown_connections 2024-09-12T03:00:09.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:09.846+0000 7fa2e239d740 1 --2- 172.21.15.5:0/1953780 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55fd5ce12640 0x55fd5ce14a60 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:09.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:09.846+0000 7fa2e239d740 1 --2- 172.21.15.5:0/1953780 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fd5cd87f60 0x55fd5cd9b8b0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:09.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:09.846+0000 7fa2e239d740 1 --2- 172.21.15.5:0/1953780 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fd5cd96d50 0x55fd5ce0a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:09.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:09.846+0000 7fa2e239d740 1 --2- 172.21.15.5:0/1953780 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fd5cd87640 0x55fd5cd9b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:09.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:09.846+0000 7fa2e239d740 1 -- 172.21.15.5:0/1953780 >> 172.21.15.5:0/1953780 conn(0x55fd5cd83cf0 msgr2=0x55fd5cd94400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:09.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:09.846+0000 7fa2c17fa640 0 client.15813 ms_handle_reset on v2:172.21.15.148:6838/2317600000 2024-09-12T03:00:09.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:09.846+0000 7fa2e239d740 1 -- 172.21.15.5:0/1953780 shutdown_connections 2024-09-12T03:00:09.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:09.847+0000 7fa2e239d740 1 -- 172.21.15.5:0/1953780 wait complete. 2024-09-12T03:00:09.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:09.847+0000 7fa2e239d740 20 client.15813 trim_cache size 0 max 16384 2024-09-12T03:00:09.848 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-12T03:00:09.848 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 28, 2024-09-12T03:00:09.848 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24502, 2024-09-12T03:00:09.849 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 6, 2024-09-12T03:00:09.849 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-09-12T03:00:09.849 INFO:teuthology.orchestra.run.smithi005.stdout: "83b717e4": { 2024-09-12T03:00:09.849 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 6, 2024-09-12T03:00:09.849 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-09-12T03:00:09.849 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-12T03:00:09.849 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-09-12T03:00:09.849 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-12T03:00:09.849 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:00:09.849 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 90.0, 2024-09-12T03:00:09.849 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 133.4, 2024-09-12T03:00:09.850 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-09-12T03:00:09.850 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-09-12T03:00:09.850 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-09-12T03:00:09.850 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-12T03:00:09.850 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-09-12T03:00:09.850 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-12T03:00:09.850 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:00:09.850 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:00:09.850 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:00:09.850 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:00:09.851 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:00:09.851 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-12T03:00:09.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:09.847+0000 7fa2dffc4640 1 -- 172.21.15.5:0/3264660897 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fa2c40a2b80 msgr2=0x7fa2c40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:09.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:09.847+0000 7fa2dffc4640 1 --2- 172.21.15.5:0/3264660897 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fa2c40a2b80 0x7fa2c40a5040 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa2d0009fd0 tx=0x7fa2d0009500 comp rx=0 tx=0).stop 2024-09-12T03:00:09.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:09.847+0000 7fa2dffc4640 1 -- 172.21.15.5:0/3264660897 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2d8102d30 msgr2=0x7fa2d81985a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:09.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:09.847+0000 7fa2dffc4640 1 --2- 172.21.15.5:0/3264660897 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2d8102d30 0x7fa2d81985a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa2d4009230 tx=0x7fa2d4004040 comp rx=0 tx=0).stop 2024-09-12T03:00:09.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:09.847+0000 7fa2dffc4640 1 -- 172.21.15.5:0/3264660897 shutdown_connections 2024-09-12T03:00:09.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:09.847+0000 7fa2dffc4640 1 --2- 172.21.15.5:0/3264660897 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fa2c40a2b80 0x7fa2c40a5040 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:09.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:09.847+0000 7fa2dffc4640 1 --2- 172.21.15.5:0/3264660897 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2d81036e0 0x7fa2d8198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:09.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:09.847+0000 7fa2dffc4640 1 --2- 172.21.15.5:0/3264660897 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa2d81082a0 0x7fa2d819ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:09.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:09.847+0000 7fa2dffc4640 1 --2- 172.21.15.5:0/3264660897 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa2d8102d30 0x7fa2d81985a0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:09.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:09.847+0000 7fa2dffc4640 1 -- 172.21.15.5:0/3264660897 >> 172.21.15.5:0/3264660897 conn(0x7fa2d80fc7d0 msgr2=0x7fa2d8105d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:09.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:09.848+0000 7fa2dffc4640 1 -- 172.21.15.5:0/3264660897 shutdown_connections 2024-09-12T03:00:09.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:09.848+0000 7fa2dffc4640 1 -- 172.21.15.5:0/3264660897 wait complete. 2024-09-12T03:00:10.674 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '83b717e4', quiesce took 9.3 seconds. Will release after: 4.1 2024-09-12T03:00:11.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:00:11 smithi005 ceph-mon[31197]: pgmap v393: 97 pgs: 97 active+clean; 62 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.1 MiB/s wr, 717 op/s 2024-09-12T03:00:11.415 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:00:11 smithi148 ceph-mon[35090]: pgmap v393: 97 pgs: 97 active+clean; 62 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.1 MiB/s wr, 717 op/s 2024-09-12T03:00:11.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:00:11 smithi146 ceph-mon[35011]: pgmap v393: 97 pgs: 97 active+clean; 62 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.1 MiB/s wr, 717 op/s 2024-09-12T03:00:13.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:00:13 smithi005 ceph-mon[31197]: pgmap v394: 97 pgs: 97 active+clean; 62 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 987 KiB/s rd, 2.7 MiB/s wr, 735 op/s 2024-09-12T03:00:13.415 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:00:13 smithi148 ceph-mon[35090]: pgmap v394: 97 pgs: 97 active+clean; 62 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 987 KiB/s rd, 2.7 MiB/s wr, 735 op/s 2024-09-12T03:00:13.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:00:13 smithi146 ceph-mon[35011]: pgmap v394: 97 pgs: 97 active+clean; 62 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 987 KiB/s rd, 2.7 MiB/s wr, 735 op/s 2024-09-12T03:00:14.775 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:00:15.048 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:00:15.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:00:15 smithi005 ceph-mon[31197]: pgmap v395: 97 pgs: 97 active+clean; 65 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.0 MiB/s wr, 748 op/s 2024-09-12T03:00:15.415 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:00:15 smithi148 ceph-mon[35090]: pgmap v395: 97 pgs: 97 active+clean; 65 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.0 MiB/s wr, 748 op/s 2024-09-12T03:00:15.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:00:15 smithi146 ceph-mon[35011]: pgmap v395: 97 pgs: 97 active+clean; 65 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.0 MiB/s wr, 748 op/s 2024-09-12T03:00:15.533 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:15.533+0000 7f8d97def640 1 -- 172.21.15.5:0/1773149938 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8d90101de0 msgr2=0x7f8d901021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:15.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:15.533+0000 7f8d97def640 1 --2- 172.21.15.5:0/1773149938 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8d90101de0 0x7f8d901021c0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c009900 tx=0x7f8d8c02a410 comp rx=0 tx=0).stop 2024-09-12T03:00:15.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:15.533+0000 7f8d97def640 1 -- 172.21.15.5:0/1773149938 shutdown_connections 2024-09-12T03:00:15.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:15.533+0000 7f8d97def640 1 --2- 172.21.15.5:0/1773149938 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8d90106f10 0x7f8d9010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:15.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:15.533+0000 7f8d97def640 1 --2- 172.21.15.5:0/1773149938 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d90102790 0x7f8d901067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:15.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:15.533+0000 7f8d97def640 1 --2- 172.21.15.5:0/1773149938 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8d90101de0 0x7f8d901021c0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:15.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:15.533+0000 7f8d97def640 1 -- 172.21.15.5:0/1773149938 >> 172.21.15.5:0/1773149938 conn(0x7f8d900fd8c0 msgr2=0x7f8d900ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:15.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:15.533+0000 7f8d97def640 1 -- 172.21.15.5:0/1773149938 shutdown_connections 2024-09-12T03:00:15.534 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:15.533+0000 7f8d97def640 1 -- 172.21.15.5:0/1773149938 wait complete. 2024-09-12T03:00:15.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:15.534+0000 7f8d97def640 1 Processor -- start 2024-09-12T03:00:15.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:15.534+0000 7f8d97def640 1 -- start start 2024-09-12T03:00:15.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:15.534+0000 7f8d97def640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8d90101de0 0x7f8d900744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:15.536 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:15.534+0000 7f8d97def640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8d90102790 0x7f8d90074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:15.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:15.534+0000 7f8d97def640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d90106f10 0x7f8d90074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:15.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:15.534+0000 7f8d97def640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8d90111f90 con 0x7f8d90102790 2024-09-12T03:00:15.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:15.534+0000 7f8d97def640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f8d90111e10 con 0x7f8d90101de0 2024-09-12T03:00:15.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:15.534+0000 7f8d97def640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8d90112110 con 0x7f8d90106f10 2024-09-12T03:00:15.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:15.534+0000 7f8d9538a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8d90102790 0x7f8d90074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:15.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:15.534+0000 7f8d9538a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8d90102790 0x7f8d90074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56914/0 (socket says 172.21.15.5:56914) 2024-09-12T03:00:15.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:15.534+0000 7f8d9538a640 1 -- 172.21.15.5:0/1388434016 learned_addr learned my addr 172.21.15.5:0/1388434016 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:00:15.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:15.534+0000 7f8d9638c640 1 --2- 172.21.15.5:0/1388434016 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d90106f10 0x7f8d90074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:15.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:15.535+0000 7f8d95b8b640 1 --2- 172.21.15.5:0/1388434016 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8d90101de0 0x7f8d900744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:15.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:15.535+0000 7f8d9538a640 1 -- 172.21.15.5:0/1388434016 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d90106f10 msgr2=0x7f8d90074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:15.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:15.535+0000 7f8d9538a640 1 --2- 172.21.15.5:0/1388434016 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d90106f10 0x7f8d90074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:15.537 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:15.535+0000 7f8d9538a640 1 -- 172.21.15.5:0/1388434016 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8d90101de0 msgr2=0x7f8d900744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:15.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:15.535+0000 7f8d9538a640 1 --2- 172.21.15.5:0/1388434016 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8d90101de0 0x7f8d900744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:15.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:15.535+0000 7f8d9538a640 1 -- 172.21.15.5:0/1388434016 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d901a6950 con 0x7f8d90102790 2024-09-12T03:00:15.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:15.535+0000 7f8d95b8b640 1 --2- 172.21.15.5:0/1388434016 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8d90101de0 0x7f8d900744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:00:15.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:15.535+0000 7f8d9638c640 1 --2- 172.21.15.5:0/1388434016 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d90106f10 0x7f8d90074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:00:15.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:15.535+0000 7f8d9538a640 1 --2- 172.21.15.5:0/1388434016 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8d90102790 0x7f8d90074a30 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f8d8000d950 tx=0x7f8d8000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:15.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:15.535+0000 7f8d86ffd640 1 -- 172.21.15.5:0/1388434016 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d80017070 con 0x7f8d90102790 2024-09-12T03:00:15.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:15.535+0000 7f8d86ffd640 1 -- 172.21.15.5:0/1388434016 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d80009c30 con 0x7f8d90102790 2024-09-12T03:00:15.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:15.535+0000 7f8d86ffd640 1 -- 172.21.15.5:0/1388434016 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d8000f6b0 con 0x7f8d90102790 2024-09-12T03:00:15.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:15.535+0000 7f8d97def640 1 -- 172.21.15.5:0/1388434016 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d901a6c40 con 0x7f8d90102790 2024-09-12T03:00:15.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:15.535+0000 7f8d97def640 1 -- 172.21.15.5:0/1388434016 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d901a7180 con 0x7f8d90102790 2024-09-12T03:00:15.538 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:15.536+0000 7f8d97def640 1 -- 172.21.15.5:0/1388434016 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d901021c0 con 0x7f8d90102790 2024-09-12T03:00:15.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:15.537+0000 7f8d86ffd640 1 -- 172.21.15.5:0/1388434016 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f8d800109d0 con 0x7f8d90102790 2024-09-12T03:00:15.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:15.538+0000 7f8d86ffd640 1 --2- 172.21.15.5:0/1388434016 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f8d7c0a2c00 0x7f8d7c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:15.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:15.538+0000 7f8d95b8b640 1 --2- 172.21.15.5:0/1388434016 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f8d7c0a2c00 0x7f8d7c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:15.539 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:15.539+0000 7f8d86ffd640 1 -- 172.21.15.5:0/1388434016 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f8d800cec70 con 0x7f8d90102790 2024-09-12T03:00:15.540 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:15.539+0000 7f8d95b8b640 1 --2- 172.21.15.5:0/1388434016 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f8d7c0a2c00 0x7f8d7c0a50c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c005ab0 tx=0x7f8d8c005a20 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:15.542 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:15.542+0000 7f8d86ffd640 1 -- 172.21.15.5:0/1388434016 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8d80009de0 con 0x7f8d90102790 2024-09-12T03:00:15.754 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:15.753+0000 7f8d97def640 1 -- 172.21.15.5:0/1388434016 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8d90075a20 con 0x7f8d90102790 2024-09-12T03:00:15.755 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:15.754+0000 7f8d86ffd640 1 -- 172.21.15.5:0/1388434016 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11971 (secure 0 0 0) 0x7f8d90075a20 con 0x7f8d90102790 2024-09-12T03:00:15.755 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-12T03:00:15.755 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":28,"btime":"2024-09-12T03:00:03:036015+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/4135232521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":4135232521},{"type":"v1","addr":"172.21.15.5:6835","nonce":4135232521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/4290221940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":4290221940},{"type":"v1","addr":"172.21.15.5:6837","nonce":4290221940}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T02:56:16.595562+0000","modified":"2024-09-12T03:00:02.675878+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24502,"mds_1":24487,"mds_2":24481,"mds_3":24479,"mds_4":24458},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/26447341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":26447341},{"type":"v1","addr":"172.21.15.5:6839","nonce":26447341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2393443295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2393443295},{"type":"v1","addr":"172.21.15.5:6841","nonce":2393443295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6833/3686970137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3686970137},{"type":"v1","addr":"172.21.15.146:6833","nonce":3686970137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6833/568512209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":568512209},{"type":"v1","addr":"172.21.15.148:6833","nonce":568512209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6835/2540740420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2540740420},{"type":"v1","addr":"172.21.15.146:6835","nonce":2540740420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.146:6837/1673370937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":1673370937},{"type":"v1","addr":"172.21.15.146:6837","nonce":1673370937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":3,"incarnation":22,"state":"up:active","state_seq":6,"addr":"172.21.15.146:6839/3036767228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":3036767228},{"type":"v1","addr":"172.21.15.146:6839","nonce":3036767228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":2,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.148:6835/4195086427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":4195086427},{"type":"v1","addr":"172.21.15.148:6835","nonce":4195086427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.148:6837/2606873922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2606873922},{"type":"v1","addr":"172.21.15.148:6837","nonce":2606873922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6839/2317600000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":2317600000},{"type":"v1","addr":"172.21.15.148:6839","nonce":2317600000}]},"join_fscid":-1,"export_targets":[1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24502,"qdb_cluster":[24502,24487,24481,24479,24458]},"id":1}]} 2024-09-12T03:00:15.755 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 28 2024-09-12T03:00:15.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:15.756+0000 7f8d97def640 1 -- 172.21.15.5:0/1388434016 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f8d7c0a2c00 msgr2=0x7f8d7c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:15.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:15.756+0000 7f8d97def640 1 --2- 172.21.15.5:0/1388434016 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f8d7c0a2c00 0x7f8d7c0a50c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c005ab0 tx=0x7f8d8c005a20 comp rx=0 tx=0).stop 2024-09-12T03:00:15.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:15.756+0000 7f8d97def640 1 -- 172.21.15.5:0/1388434016 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8d90102790 msgr2=0x7f8d90074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:15.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:15.756+0000 7f8d97def640 1 --2- 172.21.15.5:0/1388434016 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8d90102790 0x7f8d90074a30 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f8d8000d950 tx=0x7f8d8000de10 comp rx=0 tx=0).stop 2024-09-12T03:00:15.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:15.756+0000 7f8d97def640 1 -- 172.21.15.5:0/1388434016 shutdown_connections 2024-09-12T03:00:15.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:15.756+0000 7f8d97def640 1 --2- 172.21.15.5:0/1388434016 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f8d7c0a2c00 0x7f8d7c0a50c0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:15.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:15.756+0000 7f8d97def640 1 --2- 172.21.15.5:0/1388434016 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8d90102790 0x7f8d90074a30 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:15.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:15.756+0000 7f8d97def640 1 --2- 172.21.15.5:0/1388434016 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8d90106f10 0x7f8d90074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:15.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:15.756+0000 7f8d97def640 1 --2- 172.21.15.5:0/1388434016 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8d90101de0 0x7f8d900744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:15.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:15.756+0000 7f8d97def640 1 -- 172.21.15.5:0/1388434016 >> 172.21.15.5:0/1388434016 conn(0x7f8d900fd8c0 msgr2=0x7f8d900fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:15.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:15.757+0000 7f8d97def640 1 -- 172.21.15.5:0/1388434016 shutdown_connections 2024-09-12T03:00:15.758 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:15.757+0000 7f8d97def640 1 -- 172.21.15.5:0/1388434016 wait complete. 2024-09-12T03:00:16.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:00:16 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/1388434016' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:00:16.342 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24502 quiesce db --set-id 83b717e4 --release --await' 2024-09-12T03:00:16.343 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24502 quiesce db --set-id 83b717e4 --release --await 2024-09-12T03:00:16.415 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:00:16 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/1388434016' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:00:16.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:00:16 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/1388434016' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:00:16.600 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:00:17.114 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.110+0000 7fe6d41e4640 1 -- 172.21.15.5:0/69522661 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6cc102ef0 msgr2=0x7fe6cc10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:17.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.110+0000 7fe6d41e4640 1 --2- 172.21.15.5:0/69522661 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6cc102ef0 0x7fe6cc10da40 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fe6c4009ca0 tx=0x7fe6c402a950 comp rx=0 tx=0).stop 2024-09-12T03:00:17.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.110+0000 7fe6d41e4640 1 -- 172.21.15.5:0/69522661 shutdown_connections 2024-09-12T03:00:17.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.110+0000 7fe6d41e4640 1 --2- 172.21.15.5:0/69522661 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6cc102ef0 0x7fe6cc10da40 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:17.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.110+0000 7fe6d41e4640 1 --2- 172.21.15.5:0/69522661 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe6cc102570 0x7fe6cc1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:17.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.110+0000 7fe6d41e4640 1 --2- 172.21.15.5:0/69522661 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe6cc100e20 0x7fe6cc101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:17.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.110+0000 7fe6d41e4640 1 -- 172.21.15.5:0/69522661 >> 172.21.15.5:0/69522661 conn(0x7fe6cc0fc7d0 msgr2=0x7fe6cc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:17.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.110+0000 7fe6d41e4640 1 -- 172.21.15.5:0/69522661 shutdown_connections 2024-09-12T03:00:17.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.110+0000 7fe6d41e4640 1 -- 172.21.15.5:0/69522661 wait complete. 2024-09-12T03:00:17.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.110+0000 7fe6d41e4640 1 Processor -- start 2024-09-12T03:00:17.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.111+0000 7fe6d41e4640 1 -- start start 2024-09-12T03:00:17.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.111+0000 7fe6d41e4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6cc100e20 0x7fe6cc0fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:17.115 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.111+0000 7fe6d41e4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe6cc102570 0x7fe6cc0fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:17.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.111+0000 7fe6d41e4640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe6cc102ef0 0x7fe6cc0fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:17.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.111+0000 7fe6d41e4640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe6cc110fa0 con 0x7fe6cc100e20 2024-09-12T03:00:17.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.111+0000 7fe6d41e4640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fe6cc110e20 con 0x7fe6cc102ef0 2024-09-12T03:00:17.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.111+0000 7fe6d41e4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe6cc111120 con 0x7fe6cc102570 2024-09-12T03:00:17.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.111+0000 7fe6d1f80640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6cc100e20 0x7fe6cc0fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:17.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.111+0000 7fe6d1f80640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6cc100e20 0x7fe6cc0fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56922/0 (socket says 172.21.15.5:56922) 2024-09-12T03:00:17.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.111+0000 7fe6d1f80640 1 -- 172.21.15.5:0/806431255 learned_addr learned my addr 172.21.15.5:0/806431255 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:00:17.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.111+0000 7fe6d177f640 1 --2- 172.21.15.5:0/806431255 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe6cc102570 0x7fe6cc0fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:17.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.111+0000 7fe6d2781640 1 --2- 172.21.15.5:0/806431255 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe6cc102ef0 0x7fe6cc0fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:17.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.111+0000 7fe6d1f80640 1 -- 172.21.15.5:0/806431255 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe6cc102570 msgr2=0x7fe6cc0fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:17.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.111+0000 7fe6d1f80640 1 --2- 172.21.15.5:0/806431255 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe6cc102570 0x7fe6cc0fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:17.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.111+0000 7fe6d1f80640 1 -- 172.21.15.5:0/806431255 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe6cc102ef0 msgr2=0x7fe6cc0fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:17.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.111+0000 7fe6d1f80640 1 --2- 172.21.15.5:0/806431255 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe6cc102ef0 0x7fe6cc0fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:17.116 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.111+0000 7fe6d1f80640 1 -- 172.21.15.5:0/806431255 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6cc1a6990 con 0x7fe6cc100e20 2024-09-12T03:00:17.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.112+0000 7fe6d1f80640 1 --2- 172.21.15.5:0/806431255 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6cc100e20 0x7fe6cc0fcc80 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fe6c800b4a0 tx=0x7fe6c800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:17.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.112+0000 7fe6c2ffd640 1 -- 172.21.15.5:0/806431255 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c8014070 con 0x7fe6cc100e20 2024-09-12T03:00:17.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.112+0000 7fe6d41e4640 1 -- 172.21.15.5:0/806431255 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6cc1a6c80 con 0x7fe6cc100e20 2024-09-12T03:00:17.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.112+0000 7fe6c2ffd640 1 -- 172.21.15.5:0/806431255 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6c8010af0 con 0x7fe6cc100e20 2024-09-12T03:00:17.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.112+0000 7fe6c2ffd640 1 -- 172.21.15.5:0/806431255 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c8017d10 con 0x7fe6cc100e20 2024-09-12T03:00:17.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.112+0000 7fe6d41e4640 1 -- 172.21.15.5:0/806431255 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6cc1a71c0 con 0x7fe6cc100e20 2024-09-12T03:00:17.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.115+0000 7fe6c2ffd640 1 -- 172.21.15.5:0/806431255 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fe6c8017070 con 0x7fe6cc100e20 2024-09-12T03:00:17.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.115+0000 7fe6d65bd740 1 Processor -- start 2024-09-12T03:00:17.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.115+0000 7fe6d65bd740 1 -- start start 2024-09-12T03:00:17.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.115+0000 7fe6d65bd740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5643df775060 0x5643df775440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:17.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.115+0000 7fe6d65bd740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5643df775980 0x5643df77fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:17.121 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.115+0000 7fe6d65bd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5643df780240 0x5643df782630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:17.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.116+0000 7fe6d65bd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5643df6da620 con 0x5643df780240 2024-09-12T03:00:17.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.116+0000 7fe6d65bd740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5643df681e50 con 0x5643df775060 2024-09-12T03:00:17.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.116+0000 7fe6d65bd740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5643df6d65e0 con 0x5643df775980 2024-09-12T03:00:17.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.116+0000 7fe6d177f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5643df780240 0x5643df782630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:17.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.116+0000 7fe6d177f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5643df780240 0x5643df782630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56930/0 (socket says 172.21.15.5:56930) 2024-09-12T03:00:17.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.116+0000 7fe6d177f640 1 -- 172.21.15.5:0/3243943646 learned_addr learned my addr 172.21.15.5:0/3243943646 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:00:17.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.116+0000 7fe6d2781640 1 --2- 172.21.15.5:0/3243943646 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5643df775060 0x5643df775440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:17.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.116+0000 7fe6d1f80640 1 --2- 172.21.15.5:0/3243943646 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5643df775980 0x5643df77fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:17.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.116+0000 7fe6c2ffd640 1 --2- 172.21.15.5:0/806431255 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fe6b80a2b80 0x7fe6b80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:17.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.116+0000 7fe6d177f640 1 -- 172.21.15.5:0/3243943646 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5643df775980 msgr2=0x5643df77fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:00:17.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.116+0000 7fe6d2781640 1 --2- 172.21.15.5:0/806431255 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fe6b80a2b80 0x7fe6b80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:17.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.116+0000 7fe6d177f640 1 --2- 172.21.15.5:0/3243943646 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5643df775980 0x5643df77fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:17.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.116+0000 7fe6d177f640 1 -- 172.21.15.5:0/3243943646 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5643df775060 msgr2=0x5643df775440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:00:17.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.116+0000 7fe6d177f640 1 --2- 172.21.15.5:0/3243943646 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5643df775060 0x5643df775440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:17.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.116+0000 7fe6d177f640 1 -- 172.21.15.5:0/3243943646 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643df672d40 con 0x5643df780240 2024-09-12T03:00:17.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.116+0000 7fe6c2ffd640 1 -- 172.21.15.5:0/806431255 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fe6c80ce450 con 0x7fe6cc100e20 2024-09-12T03:00:17.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.116+0000 7fe6d177f640 1 --2- 172.21.15.5:0/3243943646 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5643df780240 0x5643df782630 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc00cc40 tx=0x7fe6bc00e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1dc16e709599767 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:17.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.117+0000 7fe6b7fff640 1 -- 172.21.15.5:0/3243943646 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6bc014070 con 0x5643df780240 2024-09-12T03:00:17.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.117+0000 7fe6d2781640 1 --2- 172.21.15.5:0/806431255 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fe6b80a2b80 0x7fe6b80a5040 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe6c4009430 tx=0x7fe6c402a950 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:17.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.117+0000 7fe6b7fff640 1 -- 172.21.15.5:0/3243943646 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6bc0047a0 con 0x5643df780240 2024-09-12T03:00:17.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.117+0000 7fe6b7fff640 1 -- 172.21.15.5:0/3243943646 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6bc010790 con 0x5643df780240 2024-09-12T03:00:17.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.117+0000 7fe6d65bd740 1 -- 172.21.15.5:0/3243943646 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5643df780240 msgr2=0x5643df782630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:17.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.117+0000 7fe6d65bd740 1 --2- 172.21.15.5:0/3243943646 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5643df780240 0x5643df782630 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc00cc40 tx=0x7fe6bc00e720 comp rx=0 tx=0).stop 2024-09-12T03:00:17.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.117+0000 7fe6d65bd740 1 -- 172.21.15.5:0/3243943646 shutdown_connections 2024-09-12T03:00:17.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.117+0000 7fe6d65bd740 1 --2- 172.21.15.5:0/3243943646 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5643df780240 0x5643df782630 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:17.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.117+0000 7fe6d65bd740 1 --2- 172.21.15.5:0/3243943646 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5643df775980 0x5643df77fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:17.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.117+0000 7fe6d65bd740 1 --2- 172.21.15.5:0/3243943646 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5643df775060 0x5643df775440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:17.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.117+0000 7fe6d65bd740 1 -- 172.21.15.5:0/3243943646 >> 172.21.15.5:0/3243943646 conn(0x5643df7715f0 msgr2=0x5643df7719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:17.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.118+0000 7fe6d65bd740 1 -- 172.21.15.5:0/3243943646 shutdown_connections 2024-09-12T03:00:17.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.118+0000 7fe6d65bd740 1 -- 172.21.15.5:0/3243943646 wait complete. 2024-09-12T03:00:17.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.118+0000 7fe6d65bd740 1 -- start start 2024-09-12T03:00:17.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.118+0000 7fe6d65bd740 1 Processor -- start 2024-09-12T03:00:17.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.118+0000 7fe6d65bd740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5643df775060 0x5643df77d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:17.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.118+0000 7fe6d65bd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5643df775980 0x5643df77d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:17.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.118+0000 7fe6d65bd740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5643df780240 0x5643df77db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:17.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.118+0000 7fe6d65bd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5643df782e10 con 0x5643df775980 2024-09-12T03:00:17.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.118+0000 7fe6d65bd740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5643df662f10 con 0x5643df775060 2024-09-12T03:00:17.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.118+0000 7fe6d65bd740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5643df782f90 con 0x5643df780240 2024-09-12T03:00:17.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.118+0000 7fe6d65bd740 10 client.0 _ll_register_callbacks cb 0x5643df686db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:00:17.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.119+0000 7fe6d177f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5643df775980 0x5643df77d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:17.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.119+0000 7fe6d177f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5643df775980 0x5643df77d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56932/0 (socket says 172.21.15.5:56932) 2024-09-12T03:00:17.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.119+0000 7fe6d177f640 1 -- 172.21.15.5:0/4017532099 learned_addr learned my addr 172.21.15.5:0/4017532099 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:00:17.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.119+0000 7fe6d1f80640 1 --2- 172.21.15.5:0/4017532099 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5643df775060 0x5643df77d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:17.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.119+0000 7fe6d2781640 1 --2- 172.21.15.5:0/4017532099 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5643df780240 0x5643df77db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:17.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.119+0000 7fe6d177f640 1 -- 172.21.15.5:0/4017532099 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5643df780240 msgr2=0x5643df77db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:00:17.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.119+0000 7fe6d177f640 1 --2- 172.21.15.5:0/4017532099 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5643df780240 0x5643df77db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:17.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.119+0000 7fe6d177f640 1 -- 172.21.15.5:0/4017532099 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5643df775060 msgr2=0x5643df77d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:00:17.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.119+0000 7fe6d177f640 1 --2- 172.21.15.5:0/4017532099 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5643df775060 0x5643df77d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:17.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.119+0000 7fe6d177f640 1 -- 172.21.15.5:0/4017532099 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643df3e9580 con 0x5643df775980 2024-09-12T03:00:17.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.119+0000 7fe6d177f640 1 --2- 172.21.15.5:0/4017532099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5643df775980 0x5643df77d640 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc000c00 tx=0x7fe6bc005210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a022212f40db5608 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:17.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.119+0000 7fe6b57fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T03:00:17.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.119+0000 7fe6b57fa640 1 -- 172.21.15.5:0/4017532099 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6bc014030 con 0x5643df775980 2024-09-12T03:00:17.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.119+0000 7fe6b57fa640 1 -- 172.21.15.5:0/4017532099 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fe6bc004050 con 0x5643df775980 2024-09-12T03:00:17.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.119+0000 7fe6d65bd740 1 -- 172.21.15.5:0/4017532099 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6bc0047a0 con 0x5643df775980 2024-09-12T03:00:17.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.119+0000 7fe6b57fa640 1 -- 172.21.15.5:0/4017532099 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe6bc004e10 con 0x5643df775980 2024-09-12T03:00:17.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.120+0000 7fe6d65bd740 10 client.15831 fetch_fsmap learned FSMap version 28 2024-09-12T03:00:17.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.120+0000 7fe6d65bd740 1 -- 172.21.15.5:0/4017532099 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5643df4b3960 con 0x5643df775980 2024-09-12T03:00:17.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.120+0000 7fe6b57fa640 1 -- 172.21.15.5:0/4017532099 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6bc0047a0 con 0x5643df775980 2024-09-12T03:00:17.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.120+0000 7fe6b57fa640 1 -- 172.21.15.5:0/4017532099 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8374+0+0 (secure 0 0 0) 0x7fe6bc003bf0 con 0x5643df775980 2024-09-12T03:00:17.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.120+0000 7fe6d65bd740 10 client.15831 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T03:00:17.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.120+0000 7fe6d65bd740 10 client.15831 resolve_mds: validated gid 24502 aka daemon mds.i 2024-09-12T03:00:17.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.120+0000 7fe6d65bd740 20 client.15831 populate_metadata read hostname 'smithi005' 2024-09-12T03:00:17.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.120+0000 7fe6d65bd740 1 --2- 172.21.15.5:0/4017532099 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x5643df78fcd0 0x5643df7920c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:17.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.120+0000 7fe6d65bd740 4 client.15831 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:00:17.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.120+0000 7fe6d65bd740 1 -- 172.21.15.5:0/4017532099 --> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5643df6e7b90 con 0x5643df78fcd0 2024-09-12T03:00:17.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.120+0000 7fe6d1f80640 1 --2- 172.21.15.5:0/4017532099 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x5643df78fcd0 0x5643df7920c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:17.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.121+0000 7fe6d1f80640 1 --2- 172.21.15.5:0/4017532099 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x5643df78fcd0 0x5643df7920c0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4762869c0d5e0cea server_cookie=8d93fc2a26d2bee8 in_seq=0 out_seq=0 2024-09-12T03:00:17.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.121+0000 7fe6b57fa640 10 client.15831 ms_handle_connect on v2:172.21.15.148:6838/2317600000 2024-09-12T03:00:17.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.121+0000 7fe6b57fa640 1 -- 172.21.15.5:0/4017532099 <== mds.0 v2:172.21.15.148:6838/2317600000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5643df6e7b90 con 0x5643df78fcd0 2024-09-12T03:00:17.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.121+0000 7fe6b57fa640 10 client.15831 handle_command_reply: tid=0 2024-09-12T03:00:17.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.122+0000 7fe6d65bd740 1 client.15831 shutdown 2024-09-12T03:00:17.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.122+0000 7fe6d65bd740 1 -- 172.21.15.5:0/4017532099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5643df775980 msgr2=0x5643df77d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:17.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.122+0000 7fe6d65bd740 1 --2- 172.21.15.5:0/4017532099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5643df775980 0x5643df77d640 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc000c00 tx=0x7fe6bc005210 comp rx=0 tx=0).stop 2024-09-12T03:00:17.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.122+0000 7fe6d65bd740 1 -- 172.21.15.5:0/4017532099 shutdown_connections 2024-09-12T03:00:17.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.122+0000 7fe6d65bd740 1 --2- 172.21.15.5:0/4017532099 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x5643df78fcd0 0x5643df7920c0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:17.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.122+0000 7fe6d65bd740 1 --2- 172.21.15.5:0/4017532099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5643df775980 0x5643df77d640 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:17.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.122+0000 7fe6d65bd740 1 --2- 172.21.15.5:0/4017532099 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5643df780240 0x5643df77db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:17.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.122+0000 7fe6d65bd740 1 --2- 172.21.15.5:0/4017532099 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5643df775060 0x5643df77d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:17.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.122+0000 7fe6b57fa640 0 client.15831 ms_handle_reset on v2:172.21.15.148:6838/2317600000 2024-09-12T03:00:17.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.122+0000 7fe6d65bd740 1 -- 172.21.15.5:0/4017532099 >> 172.21.15.5:0/4017532099 conn(0x5643df7715f0 msgr2=0x5643df7772d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:17.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.122+0000 7fe6d65bd740 1 -- 172.21.15.5:0/4017532099 shutdown_connections 2024-09-12T03:00:17.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.122+0000 7fe6d65bd740 1 -- 172.21.15.5:0/4017532099 wait complete. 2024-09-12T03:00:17.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.122+0000 7fe6d65bd740 20 client.15831 trim_cache size 0 max 16384 2024-09-12T03:00:17.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.139+0000 7fe6d65bd740 1 Processor -- start 2024-09-12T03:00:17.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.139+0000 7fe6d65bd740 1 -- start start 2024-09-12T03:00:17.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.139+0000 7fe6d65bd740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5643df775060 0x5643df7771c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:17.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.139+0000 7fe6d65bd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5643df775980 0x5643df7978c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:17.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.139+0000 7fe6d65bd740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5643df780240 0x5643df797e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:17.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.139+0000 7fe6d65bd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe6cc1af770 con 0x5643df775980 2024-09-12T03:00:17.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.139+0000 7fe6d65bd740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fe6cc1b1340 con 0x5643df775060 2024-09-12T03:00:17.140 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.139+0000 7fe6d65bd740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe6cc1afe40 con 0x5643df780240 2024-09-12T03:00:17.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.139+0000 7fe6d177f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5643df775980 0x5643df7978c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:17.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.139+0000 7fe6d177f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5643df775980 0x5643df7978c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56936/0 (socket says 172.21.15.5:56936) 2024-09-12T03:00:17.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.139+0000 7fe6d177f640 1 -- 172.21.15.5:0/441816186 learned_addr learned my addr 172.21.15.5:0/441816186 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:00:17.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.140+0000 7fe6d2781640 1 --2- 172.21.15.5:0/441816186 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5643df780240 0x5643df797e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:17.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.140+0000 7fe6d177f640 1 -- 172.21.15.5:0/441816186 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5643df780240 msgr2=0x5643df797e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:00:17.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.140+0000 7fe6d177f640 1 --2- 172.21.15.5:0/441816186 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5643df780240 0x5643df797e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:17.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.140+0000 7fe6d177f640 1 -- 172.21.15.5:0/441816186 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5643df775060 msgr2=0x5643df7771c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:00:17.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.140+0000 7fe6d177f640 1 --2- 172.21.15.5:0/441816186 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5643df775060 0x5643df7771c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:17.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.140+0000 7fe6d177f640 1 -- 172.21.15.5:0/441816186 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643df782990 con 0x5643df775980 2024-09-12T03:00:17.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.140+0000 7fe6d177f640 1 --2- 172.21.15.5:0/441816186 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5643df775980 0x5643df7978c0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc00cb60 tx=0x7fe6bc005d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e931686230b613a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:17.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.140+0000 7fe6b77fe640 1 -- 172.21.15.5:0/441816186 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6bc014040 con 0x5643df775980 2024-09-12T03:00:17.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.140+0000 7fe6b77fe640 1 -- 172.21.15.5:0/441816186 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6bc029030 con 0x5643df775980 2024-09-12T03:00:17.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.140+0000 7fe6b77fe640 1 -- 172.21.15.5:0/441816186 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6bc003f90 con 0x5643df775980 2024-09-12T03:00:17.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.141+0000 7fe6d65bd740 1 -- 172.21.15.5:0/441816186 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5643df775980 msgr2=0x5643df7978c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:17.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.141+0000 7fe6d65bd740 1 --2- 172.21.15.5:0/441816186 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5643df775980 0x5643df7978c0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc00cb60 tx=0x7fe6bc005d60 comp rx=0 tx=0).stop 2024-09-12T03:00:17.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.141+0000 7fe6d65bd740 1 -- 172.21.15.5:0/441816186 shutdown_connections 2024-09-12T03:00:17.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.141+0000 7fe6d65bd740 1 --2- 172.21.15.5:0/441816186 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5643df775980 0x5643df7978c0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:17.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.141+0000 7fe6d65bd740 1 --2- 172.21.15.5:0/441816186 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5643df780240 0x5643df797e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:17.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.141+0000 7fe6d65bd740 1 --2- 172.21.15.5:0/441816186 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5643df775060 0x5643df7771c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:17.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.141+0000 7fe6d65bd740 1 -- 172.21.15.5:0/441816186 >> 172.21.15.5:0/441816186 conn(0x5643df7715f0 msgr2=0x5643df776ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:17.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.141+0000 7fe6d65bd740 1 -- 172.21.15.5:0/441816186 shutdown_connections 2024-09-12T03:00:17.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.141+0000 7fe6d65bd740 1 -- 172.21.15.5:0/441816186 wait complete. 2024-09-12T03:00:17.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.141+0000 7fe6d65bd740 1 -- start start 2024-09-12T03:00:17.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.141+0000 7fe6d65bd740 1 Processor -- start 2024-09-12T03:00:17.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.141+0000 7fe6d65bd740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5643df775060 0x5643df7881e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:17.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.141+0000 7fe6d65bd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5643df775980 0x5643df788720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:17.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.141+0000 7fe6d65bd740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5643df780240 0x5643df7851b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:17.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.142+0000 7fe6d65bd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe6cc1b0be0 con 0x5643df775980 2024-09-12T03:00:17.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.142+0000 7fe6d65bd740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fe6cc1b0510 con 0x5643df780240 2024-09-12T03:00:17.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.142+0000 7fe6d65bd740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe6cc1ad750 con 0x5643df775060 2024-09-12T03:00:17.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.142+0000 7fe6d65bd740 10 client.0 _ll_register_callbacks cb 0x7fe6bc00ca90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:00:17.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.142+0000 7fe6d177f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5643df775980 0x5643df788720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:17.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.142+0000 7fe6d177f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5643df775980 0x5643df788720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56938/0 (socket says 172.21.15.5:56938) 2024-09-12T03:00:17.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.142+0000 7fe6d177f640 1 -- 172.21.15.5:0/4151573438 learned_addr learned my addr 172.21.15.5:0/4151573438 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:00:17.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.142+0000 7fe6d2781640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5643df780240 0x5643df7851b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:17.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.142+0000 7fe6d1f80640 1 --2- 172.21.15.5:0/4151573438 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5643df775060 0x5643df7881e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:17.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.142+0000 7fe6d177f640 1 -- 172.21.15.5:0/4151573438 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5643df775060 msgr2=0x5643df7881e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:00:17.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.142+0000 7fe6d177f640 1 --2- 172.21.15.5:0/4151573438 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5643df775060 0x5643df7881e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:17.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.142+0000 7fe6d177f640 1 -- 172.21.15.5:0/4151573438 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5643df780240 msgr2=0x5643df7851b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:00:17.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.142+0000 7fe6d177f640 1 --2- 172.21.15.5:0/4151573438 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5643df780240 0x5643df7851b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:17.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.142+0000 7fe6d177f640 1 -- 172.21.15.5:0/4151573438 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643df7856f0 con 0x5643df775980 2024-09-12T03:00:17.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.142+0000 7fe6d177f640 1 --2- 172.21.15.5:0/4151573438 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5643df775980 0x5643df788720 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc00c590 tx=0x7fe6bc00c640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8fdff90f336e474 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:17.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.142+0000 7fe6b57fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T03:00:17.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.142+0000 7fe6b57fa640 1 -- 172.21.15.5:0/4151573438 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6bc014040 con 0x5643df775980 2024-09-12T03:00:17.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.142+0000 7fe6b57fa640 1 -- 172.21.15.5:0/4151573438 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fe6bc0105b0 con 0x5643df775980 2024-09-12T03:00:17.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.142+0000 7fe6d65bd740 1 -- 172.21.15.5:0/4151573438 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6bc029030 con 0x5643df775980 2024-09-12T03:00:17.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.143+0000 7fe6b57fa640 1 -- 172.21.15.5:0/4151573438 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe6bc004c60 con 0x5643df775980 2024-09-12T03:00:17.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.143+0000 7fe6d65bd740 10 client.15837 fetch_fsmap learned FSMap version 28 2024-09-12T03:00:17.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.143+0000 7fe6d65bd740 1 -- 172.21.15.5:0/4151573438 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5643df7fb040 con 0x5643df775980 2024-09-12T03:00:17.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.143+0000 7fe6b57fa640 1 -- 172.21.15.5:0/4151573438 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6bc029030 con 0x5643df775980 2024-09-12T03:00:17.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.143+0000 7fe6b57fa640 1 -- 172.21.15.5:0/4151573438 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8374+0+0 (secure 0 0 0) 0x7fe6bc0035f0 con 0x5643df775980 2024-09-12T03:00:17.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.143+0000 7fe6d65bd740 10 client.15837 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T03:00:17.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.143+0000 7fe6d65bd740 10 client.15837 resolve_mds: validated gid 24502 aka daemon mds.i 2024-09-12T03:00:17.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.143+0000 7fe6d65bd740 20 client.15837 populate_metadata read hostname 'smithi005' 2024-09-12T03:00:17.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.143+0000 7fe6d65bd740 1 --2- 172.21.15.5:0/4151573438 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x5643df7fb770 0x5643df7fdb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:17.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.143+0000 7fe6d65bd740 4 client.15837 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "83b717e4", "release": true, "await": true}] 2024-09-12T03:00:17.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.143+0000 7fe6d65bd740 1 -- 172.21.15.5:0/4151573438 --> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] -- command(tid 0: {"prefix": "quiesce db", "set_id": "83b717e4", "release": true, "await": true}) -- 0x5643df76c950 con 0x5643df7fb770 2024-09-12T03:00:17.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.143+0000 7fe6d1f80640 1 --2- 172.21.15.5:0/4151573438 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x5643df7fb770 0x5643df7fdb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:17.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.144+0000 7fe6b57fa640 10 client.15837 ms_handle_connect on v2:172.21.15.148:6838/2317600000 2024-09-12T03:00:17.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.144+0000 7fe6d1f80640 1 --2- 172.21.15.5:0/4151573438 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x5643df7fb770 0x5643df7fdb90 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0895fc893bcafda server_cookie=2f5ffa744ee8b712 in_seq=0 out_seq=0 2024-09-12T03:00:17.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:00:17 smithi005 ceph-mon[31197]: pgmap v396: 97 pgs: 97 active+clean; 65 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 529 op/s 2024-09-12T03:00:17.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.326+0000 7fe6b57fa640 1 -- 172.21.15.5:0/4151573438 <== mds.0 v2:172.21.15.148:6838/2317600000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x5643df76c950 con 0x5643df7fb770 2024-09-12T03:00:17.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.326+0000 7fe6b57fa640 10 client.15837 handle_command_reply: tid=0 2024-09-12T03:00:17.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.326+0000 7fe6d65bd740 1 client.15837 shutdown 2024-09-12T03:00:17.327 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.327+0000 7fe6d65bd740 1 -- 172.21.15.5:0/4151573438 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5643df775980 msgr2=0x5643df788720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:17.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.327+0000 7fe6d65bd740 1 --2- 172.21.15.5:0/4151573438 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5643df775980 0x5643df788720 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc00c590 tx=0x7fe6bc00c640 comp rx=0 tx=0).stop 2024-09-12T03:00:17.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.327+0000 7fe6d65bd740 1 -- 172.21.15.5:0/4151573438 shutdown_connections 2024-09-12T03:00:17.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.327+0000 7fe6d65bd740 1 --2- 172.21.15.5:0/4151573438 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x5643df7fb770 0x5643df7fdb90 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:17.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.327+0000 7fe6d65bd740 1 --2- 172.21.15.5:0/4151573438 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5643df775980 0x5643df788720 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:17.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.327+0000 7fe6d65bd740 1 --2- 172.21.15.5:0/4151573438 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5643df780240 0x5643df7851b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:17.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.327+0000 7fe6d65bd740 1 --2- 172.21.15.5:0/4151573438 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5643df775060 0x5643df7881e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:17.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.327+0000 7fe6b57fa640 0 client.15837 ms_handle_reset on v2:172.21.15.148:6838/2317600000 2024-09-12T03:00:17.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.327+0000 7fe6d65bd740 1 -- 172.21.15.5:0/4151573438 >> 172.21.15.5:0/4151573438 conn(0x5643df7715f0 msgr2=0x5643df77d8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:17.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.327+0000 7fe6d65bd740 1 -- 172.21.15.5:0/4151573438 shutdown_connections 2024-09-12T03:00:17.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.327+0000 7fe6d65bd740 1 -- 172.21.15.5:0/4151573438 wait complete. 2024-09-12T03:00:17.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.327+0000 7fe6d65bd740 20 client.15837 trim_cache size 0 max 16384 2024-09-12T03:00:17.329 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-12T03:00:17.329 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 28, 2024-09-12T03:00:17.329 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24502, 2024-09-12T03:00:17.329 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 8, 2024-09-12T03:00:17.329 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-09-12T03:00:17.329 INFO:teuthology.orchestra.run.smithi005.stdout: "83b717e4": { 2024-09-12T03:00:17.329 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 8, 2024-09-12T03:00:17.329 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-09-12T03:00:17.329 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-12T03:00:17.329 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-09-12T03:00:17.329 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-12T03:00:17.330 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:00:17.330 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 90.0, 2024-09-12T03:00:17.330 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 133.4, 2024-09-12T03:00:17.330 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-09-12T03:00:17.330 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-09-12T03:00:17.330 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-09-12T03:00:17.330 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-12T03:00:17.330 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-09-12T03:00:17.330 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-12T03:00:17.330 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:00:17.330 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:00:17.330 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:00:17.330 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:00:17.331 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:00:17.331 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-12T03:00:17.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.328+0000 7fe6d41e4640 1 -- 172.21.15.5:0/806431255 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fe6b80a2b80 msgr2=0x7fe6b80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:17.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.328+0000 7fe6d41e4640 1 --2- 172.21.15.5:0/806431255 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fe6b80a2b80 0x7fe6b80a5040 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe6c4009430 tx=0x7fe6c402a950 comp rx=0 tx=0).stop 2024-09-12T03:00:17.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.328+0000 7fe6d41e4640 1 -- 172.21.15.5:0/806431255 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6cc100e20 msgr2=0x7fe6cc0fcc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:17.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.328+0000 7fe6d41e4640 1 --2- 172.21.15.5:0/806431255 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6cc100e20 0x7fe6cc0fcc80 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fe6c800b4a0 tx=0x7fe6c800b960 comp rx=0 tx=0).stop 2024-09-12T03:00:17.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.328+0000 7fe6d41e4640 1 -- 172.21.15.5:0/806431255 shutdown_connections 2024-09-12T03:00:17.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.328+0000 7fe6d41e4640 1 --2- 172.21.15.5:0/806431255 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe6cc102ef0 0x7fe6cc0fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:17.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.328+0000 7fe6d41e4640 1 --2- 172.21.15.5:0/806431255 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe6cc102570 0x7fe6cc0fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:17.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.328+0000 7fe6d41e4640 1 --2- 172.21.15.5:0/806431255 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fe6b80a2b80 0x7fe6b80a5040 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:17.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.328+0000 7fe6d41e4640 1 --2- 172.21.15.5:0/806431255 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6cc100e20 0x7fe6cc0fcc80 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:17.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.328+0000 7fe6d41e4640 1 -- 172.21.15.5:0/806431255 >> 172.21.15.5:0/806431255 conn(0x7fe6cc0fc7d0 msgr2=0x7fe6cc06a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:17.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.328+0000 7fe6d41e4640 1 -- 172.21.15.5:0/806431255 shutdown_connections 2024-09-12T03:00:17.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:17.328+0000 7fe6d41e4640 1 -- 172.21.15.5:0/806431255 wait complete. 2024-09-12T03:00:17.333 INFO:tasks.workunit.client.0.smithi005.stdout:checking the name lister (/usr/bin/nm -B) interface... BSD nm 2024-09-12T03:00:17.333 INFO:tasks.workunit.client.0.smithi005.stdout:checking whether ln -s works... yes 2024-09-12T03:00:17.355 INFO:tasks.workunit.client.0.smithi005.stdout:checking the maximum length of command line arguments... 1572864 2024-09-12T03:00:17.359 INFO:tasks.workunit.client.0.smithi005.stdout:checking how to convert x86_64-pc-linux-gnu file names to x86_64-pc-linux-gnu format... func_convert_file_noop 2024-09-12T03:00:17.359 INFO:tasks.workunit.client.0.smithi005.stdout:checking how to convert x86_64-pc-linux-gnu file names to toolchain format... func_convert_file_noop 2024-09-12T03:00:17.359 INFO:tasks.workunit.client.0.smithi005.stdout:checking for /usr/bin/ld option to reload object files... -r 2024-09-12T03:00:17.359 INFO:tasks.workunit.client.0.smithi005.stdout:checking for objdump... objdump 2024-09-12T03:00:17.360 INFO:tasks.workunit.client.0.smithi005.stdout:checking how to recognize dependent libraries... pass_all 2024-09-12T03:00:17.360 INFO:tasks.workunit.client.0.smithi005.stdout:checking for dlltool... no 2024-09-12T03:00:17.360 INFO:tasks.workunit.client.0.smithi005.stdout:checking how to associate runtime and link libraries... printf %s\n 2024-09-12T03:00:17.360 INFO:tasks.workunit.client.0.smithi005.stdout:checking for ar... ar 2024-09-12T03:00:17.415 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:00:17 smithi148 ceph-mon[35090]: pgmap v396: 97 pgs: 97 active+clean; 65 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 529 op/s 2024-09-12T03:00:17.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:00:17 smithi146 ceph-mon[35011]: pgmap v396: 97 pgs: 97 active+clean; 65 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 529 op/s 2024-09-12T03:00:17.479 INFO:tasks.workunit.client.0.smithi005.stdout:checking for archiver @FILE support... @ 2024-09-12T03:00:17.479 INFO:tasks.workunit.client.0.smithi005.stdout:checking for strip... strip 2024-09-12T03:00:17.480 INFO:tasks.workunit.client.0.smithi005.stdout:checking for ranlib... ranlib 2024-09-12T03:00:17.480 INFO:tasks.workunit.client.0.smithi005.stdout:checking for gawk... gawk 2024-09-12T03:00:17.576 INFO:tasks.workunit.client.0.smithi005.stdout:checking command to parse /usr/bin/nm -B output from gcc object... ok 2024-09-12T03:00:17.578 INFO:tasks.workunit.client.0.smithi005.stdout:checking for sysroot... no 2024-09-12T03:00:17.588 INFO:tasks.workunit.client.0.smithi005.stdout:checking for a working dd... /usr/bin/dd 2024-09-12T03:00:17.598 INFO:tasks.workunit.client.0.smithi005.stdout:checking how to truncate binary pipes... /usr/bin/dd bs=4096 count=1 2024-09-12T03:00:17.633 INFO:tasks.workunit.client.0.smithi005.stdout:checking for mt... no 2024-09-12T03:00:17.645 INFO:tasks.workunit.client.0.smithi005.stdout:checking if : is a manifest tool... no 2024-09-12T03:00:17.794 INFO:tasks.workunit.client.0.smithi005.stdout:checking how to run the C preprocessor... gcc -E 2024-09-12T03:00:18.014 INFO:tasks.workunit.client.0.smithi005.stdout:checking for ANSI C header files... yes 2024-09-12T03:00:18.066 INFO:tasks.workunit.client.0.smithi005.stdout:checking for sys/types.h... yes 2024-09-12T03:00:18.105 INFO:tasks.workunit.client.0.smithi005.stdout:checking for sys/stat.h... yes 2024-09-12T03:00:18.107 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '83b717e4', total seconds elapsed: 16.7 2024-09-12T03:00:18.143 INFO:tasks.workunit.client.0.smithi005.stdout:checking for stdlib.h... yes 2024-09-12T03:00:18.183 INFO:tasks.workunit.client.0.smithi005.stdout:checking for string.h... yes 2024-09-12T03:00:18.223 INFO:tasks.workunit.client.0.smithi005.stdout:checking for memory.h... yes 2024-09-12T03:00:18.262 INFO:tasks.workunit.client.0.smithi005.stdout:checking for strings.h... yes 2024-09-12T03:00:18.305 INFO:tasks.workunit.client.0.smithi005.stdout:checking for inttypes.h... yes 2024-09-12T03:00:18.345 INFO:tasks.workunit.client.0.smithi005.stdout:checking for stdint.h... yes 2024-09-12T03:00:18.388 INFO:tasks.workunit.client.0.smithi005.stdout:checking for unistd.h... yes 2024-09-12T03:00:18.453 INFO:tasks.workunit.client.0.smithi005.stdout:checking for dlfcn.h... yes 2024-09-12T03:00:18.460 INFO:tasks.workunit.client.0.smithi005.stdout:checking for objdir... .libs 2024-09-12T03:00:18.578 INFO:tasks.workunit.client.0.smithi005.stdout:checking if gcc supports -fno-rtti -fno-exceptions... no 2024-09-12T03:00:18.578 INFO:tasks.workunit.client.0.smithi005.stdout:checking for gcc option to produce PIC... -fPIC -DPIC 2024-09-12T03:00:18.614 INFO:tasks.workunit.client.0.smithi005.stdout:checking if gcc PIC flag -fPIC -DPIC works... yes 2024-09-12T03:00:18.660 INFO:tasks.workunit.client.0.smithi005.stdout:checking if gcc static flag -static works... no 2024-09-12T03:00:18.717 INFO:tasks.workunit.client.0.smithi005.stdout:checking if gcc supports -c -o file.o... yes 2024-09-12T03:00:18.717 INFO:tasks.workunit.client.0.smithi005.stdout:checking if gcc supports -c -o file.o... (cached) yes 2024-09-12T03:00:18.723 INFO:tasks.workunit.client.0.smithi005.stdout:checking whether the gcc linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes 2024-09-12T03:00:18.785 INFO:tasks.workunit.client.0.smithi005.stdout:checking whether -lc should be explicitly linked in... no 2024-09-12T03:00:18.843 INFO:tasks.workunit.client.0.smithi005.stdout:checking dynamic linker characteristics... GNU/Linux ld.so 2024-09-12T03:00:18.843 INFO:tasks.workunit.client.0.smithi005.stdout:checking how to hardcode library paths into programs... immediate 2024-09-12T03:00:18.844 INFO:tasks.workunit.client.0.smithi005.stdout:checking whether stripping libraries is possible... yes 2024-09-12T03:00:18.845 INFO:tasks.workunit.client.0.smithi005.stdout:checking if libtool supports shared libraries... yes 2024-09-12T03:00:18.845 INFO:tasks.workunit.client.0.smithi005.stdout:checking whether to build shared libraries... yes 2024-09-12T03:00:18.845 INFO:tasks.workunit.client.0.smithi005.stdout:checking whether to build static libraries... yes 2024-09-12T03:00:18.851 INFO:tasks.workunit.client.0.smithi005.stdout:checking for gcc... (cached) gcc 2024-09-12T03:00:18.886 INFO:tasks.workunit.client.0.smithi005.stdout:checking whether we are using the GNU C compiler... (cached) yes 2024-09-12T03:00:18.886 INFO:tasks.workunit.client.0.smithi005.stdout:checking whether gcc accepts -g... (cached) yes 2024-09-12T03:00:18.886 INFO:tasks.workunit.client.0.smithi005.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2024-09-12T03:00:18.887 INFO:tasks.workunit.client.0.smithi005.stdout:checking for gmake... /usr/bin/gmake 2024-09-12T03:00:18.887 INFO:tasks.workunit.client.0.smithi005.stderr:./configure: line 12082: test: $(SHELL): binary operator expected 2024-09-12T03:00:18.888 INFO:tasks.workunit.client.0.smithi005.stdout:checking for tar... /bin/tar 2024-09-12T03:00:18.888 INFO:tasks.workunit.client.0.smithi005.stdout:checking for gzip... /bin/gzip 2024-09-12T03:00:18.913 INFO:tasks.workunit.client.0.smithi005.stdout:checking whether gcc -MM is supported... yes 2024-09-12T03:00:18.913 INFO:tasks.workunit.client.0.smithi005.stdout:checking for sort... /bin/sort 2024-09-12T03:00:18.913 INFO:tasks.workunit.client.0.smithi005.stdout:checking whether ln -s works... yes 2024-09-12T03:00:18.914 INFO:tasks.workunit.client.0.smithi005.stdout:checking for rpm... /bin/rpm 2024-09-12T03:00:18.916 INFO:tasks.workunit.client.0.smithi005.stdout:checking for rpmbuild... no 2024-09-12T03:00:18.966 INFO:tasks.workunit.client.0.smithi005.stdout:checking assert.h usability... yes 2024-09-12T03:00:18.983 INFO:tasks.workunit.client.0.smithi005.stdout:checking assert.h presence... yes 2024-09-12T03:00:18.983 INFO:tasks.workunit.client.0.smithi005.stdout:checking for assert.h... yes 2024-09-12T03:00:19.032 INFO:tasks.workunit.client.0.smithi005.stdout:checking bstring.h usability... no 2024-09-12T03:00:19.060 INFO:tasks.workunit.client.0.smithi005.stdout:checking bstring.h presence... no 2024-09-12T03:00:19.060 INFO:tasks.workunit.client.0.smithi005.stdout:checking for bstring.h... no 2024-09-12T03:00:19.099 INFO:tasks.workunit.client.0.smithi005.stdout:checking libgen.h usability... yes 2024-09-12T03:00:19.116 INFO:tasks.workunit.client.0.smithi005.stdout:checking libgen.h presence... yes 2024-09-12T03:00:19.116 INFO:tasks.workunit.client.0.smithi005.stdout:checking for libgen.h... yes 2024-09-12T03:00:19.171 INFO:tasks.workunit.client.0.smithi005.stdout:checking dirent.h usability... yes 2024-09-12T03:00:19.187 INFO:tasks.workunit.client.0.smithi005.stdout:checking dirent.h presence... yes 2024-09-12T03:00:19.188 INFO:tasks.workunit.client.0.smithi005.stdout:checking for dirent.h... yes 2024-09-12T03:00:19.232 INFO:tasks.workunit.client.0.smithi005.stdout:checking errno.h usability... yes 2024-09-12T03:00:19.248 INFO:tasks.workunit.client.0.smithi005.stdout:checking errno.h presence... yes 2024-09-12T03:00:19.248 INFO:tasks.workunit.client.0.smithi005.stdout:checking for errno.h... yes 2024-09-12T03:00:19.294 INFO:tasks.workunit.client.0.smithi005.stdout:checking malloc.h usability... yes 2024-09-12T03:00:19.311 INFO:tasks.workunit.client.0.smithi005.stdout:checking malloc.h presence... yes 2024-09-12T03:00:19.311 INFO:tasks.workunit.client.0.smithi005.stdout:checking for malloc.h... yes 2024-09-12T03:00:19.363 INFO:tasks.workunit.client.0.smithi005.stdout:checking uuid.h usability... no 2024-09-12T03:00:19.389 INFO:tasks.workunit.client.0.smithi005.stdout:checking uuid.h presence... no 2024-09-12T03:00:19.389 INFO:tasks.workunit.client.0.smithi005.stdout:checking for uuid.h... no 2024-09-12T03:00:19.415 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:00:19 smithi148 ceph-mon[35090]: pgmap v397: 97 pgs: 97 active+clean; 65 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 612 KiB/s rd, 1.3 MiB/s wr, 345 op/s 2024-09-12T03:00:19.434 INFO:tasks.workunit.client.0.smithi005.stdout:checking uuid/uuid.h usability... yes 2024-09-12T03:00:19.452 INFO:tasks.workunit.client.0.smithi005.stdout:checking uuid/uuid.h presence... yes 2024-09-12T03:00:19.452 INFO:tasks.workunit.client.0.smithi005.stdout:checking for uuid/uuid.h... yes 2024-09-12T03:00:19.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:00:19 smithi146 ceph-mon[35011]: pgmap v397: 97 pgs: 97 active+clean; 65 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 612 KiB/s rd, 1.3 MiB/s wr, 345 op/s 2024-09-12T03:00:19.501 INFO:tasks.workunit.client.0.smithi005.stdout:checking sys/uuid.h usability... no 2024-09-12T03:00:19.529 INFO:tasks.workunit.client.0.smithi005.stdout:checking sys/uuid.h presence... no 2024-09-12T03:00:19.529 INFO:tasks.workunit.client.0.smithi005.stdout:checking for sys/uuid.h... no 2024-09-12T03:00:19.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:00:19 smithi005 ceph-mon[31197]: pgmap v397: 97 pgs: 97 active+clean; 65 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 612 KiB/s rd, 1.3 MiB/s wr, 345 op/s 2024-09-12T03:00:19.578 INFO:tasks.workunit.client.0.smithi005.stdout:checking sys/file.h usability... yes 2024-09-12T03:00:19.595 INFO:tasks.workunit.client.0.smithi005.stdout:checking sys/file.h presence... yes 2024-09-12T03:00:19.595 INFO:tasks.workunit.client.0.smithi005.stdout:checking for sys/file.h... yes 2024-09-12T03:00:19.641 INFO:tasks.workunit.client.0.smithi005.stdout:checking sys/fcntl.h usability... yes 2024-09-12T03:00:19.658 INFO:tasks.workunit.client.0.smithi005.stdout:checking sys/fcntl.h presence... yes 2024-09-12T03:00:19.658 INFO:tasks.workunit.client.0.smithi005.stdout:checking for sys/fcntl.h... yes 2024-09-12T03:00:19.707 INFO:tasks.workunit.client.0.smithi005.stdout:checking sys/syssgi.h usability... no 2024-09-12T03:00:19.735 INFO:tasks.workunit.client.0.smithi005.stdout:checking sys/syssgi.h presence... no 2024-09-12T03:00:19.735 INFO:tasks.workunit.client.0.smithi005.stdout:checking for sys/syssgi.h... no 2024-09-12T03:00:19.804 INFO:tasks.workunit.client.0.smithi005.stdout:checking sys/param.h usability... yes 2024-09-12T03:00:19.824 INFO:tasks.workunit.client.0.smithi005.stdout:checking sys/param.h presence... yes 2024-09-12T03:00:19.824 INFO:tasks.workunit.client.0.smithi005.stdout:checking for sys/param.h... yes 2024-09-12T03:00:19.837 INFO:tasks.workunit.client.0.smithi005.stdout:checking for sys/stat.h... (cached) yes 2024-09-12T03:00:19.881 INFO:tasks.workunit.client.0.smithi005.stdout:checking sys/statvfs.h usability... yes 2024-09-12T03:00:19.897 INFO:tasks.workunit.client.0.smithi005.stdout:checking sys/statvfs.h presence... yes 2024-09-12T03:00:19.897 INFO:tasks.workunit.client.0.smithi005.stdout:checking for sys/statvfs.h... yes 2024-09-12T03:00:19.943 INFO:tasks.workunit.client.0.smithi005.stdout:checking sys/time.h usability... yes 2024-09-12T03:00:19.962 INFO:tasks.workunit.client.0.smithi005.stdout:checking sys/time.h presence... yes 2024-09-12T03:00:19.962 INFO:tasks.workunit.client.0.smithi005.stdout:checking for sys/time.h... yes 2024-09-12T03:00:20.007 INFO:tasks.workunit.client.0.smithi005.stdout:checking sys/ioctl.h usability... yes 2024-09-12T03:00:20.026 INFO:tasks.workunit.client.0.smithi005.stdout:checking sys/ioctl.h presence... yes 2024-09-12T03:00:20.026 INFO:tasks.workunit.client.0.smithi005.stdout:checking for sys/ioctl.h... yes 2024-09-12T03:00:20.088 INFO:tasks.workunit.client.0.smithi005.stdout:checking sys/wait.h usability... yes 2024-09-12T03:00:20.110 INFO:tasks.workunit.client.0.smithi005.stdout:checking sys/wait.h presence... yes 2024-09-12T03:00:20.111 INFO:tasks.workunit.client.0.smithi005.stdout:checking for sys/wait.h... yes 2024-09-12T03:00:20.125 INFO:tasks.workunit.client.0.smithi005.stdout:checking for sys/types.h... (cached) yes 2024-09-12T03:00:20.129 INFO:tasks.workunit.client.0.smithi005.stdout:checking for strings.h... (cached) yes 2024-09-12T03:00:20.166 INFO:tasks.workunit.client.0.smithi005.stdout:checking err.h usability... yes 2024-09-12T03:00:20.183 INFO:tasks.workunit.client.0.smithi005.stdout:checking err.h presence... yes 2024-09-12T03:00:20.183 INFO:tasks.workunit.client.0.smithi005.stdout:checking for err.h... yes 2024-09-12T03:00:20.228 INFO:tasks.workunit.client.0.smithi005.stdout:checking linux/falloc.h usability... yes 2024-09-12T03:00:20.244 INFO:tasks.workunit.client.0.smithi005.stdout:checking linux/falloc.h presence... yes 2024-09-12T03:00:20.244 INFO:tasks.workunit.client.0.smithi005.stdout:checking for linux/falloc.h... yes 2024-09-12T03:00:20.297 INFO:tasks.workunit.client.0.smithi005.stdout:checking sys/fs/xfs_fsops.h usability... no 2024-09-12T03:00:20.331 INFO:tasks.workunit.client.0.smithi005.stdout:checking sys/fs/xfs_fsops.h presence... no 2024-09-12T03:00:20.331 INFO:tasks.workunit.client.0.smithi005.stdout:checking for sys/fs/xfs_fsops.h... no 2024-09-12T03:00:20.383 INFO:tasks.workunit.client.0.smithi005.stdout:checking sys/fs/xfs_itable.h usability... no 2024-09-12T03:00:20.416 INFO:tasks.workunit.client.0.smithi005.stdout:checking sys/fs/xfs_itable.h presence... no 2024-09-12T03:00:20.416 INFO:tasks.workunit.client.0.smithi005.stdout:checking for sys/fs/xfs_itable.h... no 2024-09-12T03:00:20.461 INFO:tasks.workunit.client.0.smithi005.stdout:checking xfs/platform_defs.h usability... no 2024-09-12T03:00:20.491 INFO:tasks.workunit.client.0.smithi005.stdout:checking xfs/platform_defs.h presence... no 2024-09-12T03:00:20.492 INFO:tasks.workunit.client.0.smithi005.stdout:checking for xfs/platform_defs.h... no 2024-09-12T03:00:20.538 INFO:tasks.workunit.client.0.smithi005.stdout:checking btrfs/ioctl.h usability... no 2024-09-12T03:00:20.569 INFO:tasks.workunit.client.0.smithi005.stdout:checking btrfs/ioctl.h presence... no 2024-09-12T03:00:20.569 INFO:tasks.workunit.client.0.smithi005.stdout:checking for btrfs/ioctl.h... no 2024-09-12T03:00:20.614 INFO:tasks.workunit.client.0.smithi005.stdout:checking cifs/ioctl.h usability... no 2024-09-12T03:00:20.641 INFO:tasks.workunit.client.0.smithi005.stdout:checking cifs/ioctl.h presence... no 2024-09-12T03:00:20.642 INFO:tasks.workunit.client.0.smithi005.stdout:checking for cifs/ioctl.h... no 2024-09-12T03:00:20.689 INFO:tasks.workunit.client.0.smithi005.stdout:checking sys/mman.h usability... yes 2024-09-12T03:00:20.706 INFO:tasks.workunit.client.0.smithi005.stdout:checking sys/mman.h presence... yes 2024-09-12T03:00:20.706 INFO:tasks.workunit.client.0.smithi005.stdout:checking for sys/mman.h... yes 2024-09-12T03:00:20.761 INFO:tasks.workunit.client.0.smithi005.stdout:checking linux/ext4.h usability... no 2024-09-12T03:00:20.793 INFO:tasks.workunit.client.0.smithi005.stdout:checking linux/ext4.h presence... no 2024-09-12T03:00:20.793 INFO:tasks.workunit.client.0.smithi005.stdout:checking for linux/ext4.h... no 2024-09-12T03:00:20.823 INFO:tasks.workunit.client.0.smithi005.stdout:checking for xfs/xfs_log_format.h... no 2024-09-12T03:00:20.882 INFO:tasks.workunit.client.0.smithi005.stdout:checking for uuid_compare... no 2024-09-12T03:00:20.922 INFO:tasks.workunit.client.0.smithi005.stdout:checking for uuid_compare in -luuid... yes 2024-09-12T03:00:21.000 INFO:tasks.workunit.client.0.smithi005.stdout:checking for xfs/xfs.h... yes 2024-09-12T03:00:21.033 INFO:tasks.workunit.client.0.smithi005.stdout:checking for xfs/libxfs.h... no 2024-09-12T03:00:21.062 INFO:tasks.workunit.client.0.smithi005.stdout:checking whether xlog_assign_lsn is declared... no 2024-09-12T03:00:21.101 INFO:tasks.workunit.client.0.smithi005.stdout:checking for xfs/xqm.h... yes 2024-09-12T03:00:21.154 INFO:tasks.workunit.client.0.smithi005.stdout:checking xfsctl from xfs/xfs.h... ok 2024-09-12T03:00:21.191 INFO:tasks.workunit.client.0.smithi005.stdout:checking xfs/handle.h usability... yes 2024-09-12T03:00:21.207 INFO:tasks.workunit.client.0.smithi005.stdout:checking xfs/handle.h presence... yes 2024-09-12T03:00:21.207 INFO:tasks.workunit.client.0.smithi005.stdout:checking for xfs/handle.h... yes 2024-09-12T03:00:21.258 INFO:tasks.workunit.client.0.smithi005.stdout:checking for attr_list_by_handle in -lhandle... yes 2024-09-12T03:00:21.298 INFO:tasks.workunit.client.0.smithi005.stdout:checking sys/acl.h usability... yes 2024-09-12T03:00:21.316 INFO:tasks.workunit.client.0.smithi005.stdout:checking sys/acl.h presence... yes 2024-09-12T03:00:21.316 INFO:tasks.workunit.client.0.smithi005.stdout:checking for sys/acl.h... yes 2024-09-12T03:00:21.358 INFO:tasks.workunit.client.0.smithi005.stdout:checking acl/libacl.h usability... yes 2024-09-12T03:00:21.377 INFO:tasks.workunit.client.0.smithi005.stdout:checking acl/libacl.h presence... yes 2024-09-12T03:00:21.377 INFO:tasks.workunit.client.0.smithi005.stdout:checking for acl/libacl.h... yes 2024-09-12T03:00:21.415 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:00:21 smithi148 ceph-mon[35090]: pgmap v398: 97 pgs: 97 active+clean; 65 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 614 KiB/s rd, 875 KiB/s wr, 137 op/s 2024-09-12T03:00:21.418 INFO:tasks.workunit.client.0.smithi005.stdout:checking for acl_init in -lacl... yes 2024-09-12T03:00:21.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:00:21 smithi146 ceph-mon[35011]: pgmap v398: 97 pgs: 97 active+clean; 65 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 614 KiB/s rd, 875 KiB/s wr, 137 op/s 2024-09-12T03:00:21.485 INFO:tasks.workunit.client.0.smithi005.stdout:checking gdbm-ndbm.h usability... no 2024-09-12T03:00:21.514 INFO:tasks.workunit.client.0.smithi005.stdout:checking gdbm-ndbm.h presence... no 2024-09-12T03:00:21.514 INFO:tasks.workunit.client.0.smithi005.stdout:checking for gdbm-ndbm.h... no 2024-09-12T03:00:21.551 INFO:tasks.workunit.client.0.smithi005.stdout:checking gdbm/ndbm.h usability... no 2024-09-12T03:00:21.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:00:21 smithi005 ceph-mon[31197]: pgmap v398: 97 pgs: 97 active+clean; 65 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 614 KiB/s rd, 875 KiB/s wr, 137 op/s 2024-09-12T03:00:21.579 INFO:tasks.workunit.client.0.smithi005.stdout:checking gdbm/ndbm.h presence... no 2024-09-12T03:00:21.579 INFO:tasks.workunit.client.0.smithi005.stdout:checking for gdbm/ndbm.h... no 2024-09-12T03:00:21.616 INFO:tasks.workunit.client.0.smithi005.stdout:checking ndbm.h usability... no 2024-09-12T03:00:21.644 INFO:tasks.workunit.client.0.smithi005.stdout:checking ndbm.h presence... no 2024-09-12T03:00:21.645 INFO:tasks.workunit.client.0.smithi005.stdout:checking for ndbm.h... no 2024-09-12T03:00:21.677 INFO:tasks.workunit.client.0.smithi005.stdout:checking libaio.h usability... yes 2024-09-12T03:00:21.695 INFO:tasks.workunit.client.0.smithi005.stdout:checking libaio.h presence... yes 2024-09-12T03:00:21.696 INFO:tasks.workunit.client.0.smithi005.stdout:checking for libaio.h... yes 2024-09-12T03:00:21.705 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T03:00:21.740 INFO:tasks.workunit.client.0.smithi005.stdout:checking liburing.h usability... no 2024-09-12T03:00:21.768 INFO:tasks.workunit.client.0.smithi005.stdout:checking liburing.h presence... no 2024-09-12T03:00:21.768 INFO:tasks.workunit.client.0.smithi005.stdout:checking for liburing.h... no 2024-09-12T03:00:21.805 INFO:tasks.workunit.client.0.smithi005.stdout:checking sys/capability.h usability... no 2024-09-12T03:00:21.832 INFO:tasks.workunit.client.0.smithi005.stdout:checking sys/capability.h presence... no 2024-09-12T03:00:21.832 INFO:tasks.workunit.client.0.smithi005.stdout:checking for sys/capability.h... no 2024-09-12T03:00:21.865 INFO:tasks.workunit.client.0.smithi005.stdout:checking linux/fiemap.h usability... yes 2024-09-12T03:00:21.881 INFO:tasks.workunit.client.0.smithi005.stdout:checking linux/fiemap.h presence... yes 2024-09-12T03:00:21.881 INFO:tasks.workunit.client.0.smithi005.stdout:checking for linux/fiemap.h... yes 2024-09-12T03:00:21.920 INFO:tasks.workunit.client.0.smithi005.stdout:checking for fallocate... yes 2024-09-12T03:00:21.971 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:00:21.977 INFO:tasks.workunit.client.0.smithi005.stdout:checking for open_by_handle_at... yes 2024-09-12T03:00:22.021 INFO:tasks.workunit.client.0.smithi005.stdout:checking linux/fs.h usability... yes 2024-09-12T03:00:22.034 INFO:tasks.workunit.client.0.smithi005.stdout:checking linux/fs.h presence... yes 2024-09-12T03:00:22.034 INFO:tasks.workunit.client.0.smithi005.stdout:checking for linux/fs.h... yes 2024-09-12T03:00:22.097 INFO:tasks.workunit.client.0.smithi005.stdout:checking btrfsutil.h usability... no 2024-09-12T03:00:22.125 INFO:tasks.workunit.client.0.smithi005.stdout:checking btrfsutil.h presence... no 2024-09-12T03:00:22.125 INFO:tasks.workunit.client.0.smithi005.stdout:checking for btrfsutil.h... no 2024-09-12T03:00:22.185 INFO:tasks.workunit.client.0.smithi005.stdout:checking for copy_file_range... yes 2024-09-12T03:00:22.232 INFO:tasks.workunit.client.0.smithi005.stdout:checking for SEEK_DATA... yes 2024-09-12T03:00:22.328 INFO:tasks.workunit.client.0.smithi005.stdout:checking for BMV_OF_SHARED... yes 2024-09-12T03:00:22.380 INFO:tasks.workunit.client.0.smithi005.stdout:checking for nftw... yes 2024-09-12T03:00:22.454 INFO:tasks.workunit.client.0.smithi005.stdout:checking for RLIMIT_NOFILE... yes 2024-09-12T03:00:22.601 INFO:tasks.workunit.client.0.smithi005.stdout:checking for XFS_IOC_EXCHANGE_RANGE... no 2024-09-12T03:00:22.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.443+0000 7ff30d245640 1 -- 172.21.15.5:0/2094068127 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff308102dd0 msgr2=0x7ff3081031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:22.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.443+0000 7ff30d245640 1 --2- 172.21.15.5:0/2094068127 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff308102dd0 0x7ff3081031b0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7ff2f8009cd0 tx=0x7ff2f802a8d0 comp rx=0 tx=0).stop 2024-09-12T03:00:22.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.443+0000 7ff30d245640 1 -- 172.21.15.5:0/2094068127 shutdown_connections 2024-09-12T03:00:22.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.443+0000 7ff30d245640 1 --2- 172.21.15.5:0/2094068127 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff308108300 0x7ff30810ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:22.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.443+0000 7ff30d245640 1 --2- 172.21.15.5:0/2094068127 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff308103780 0x7ff308107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:22.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.443+0000 7ff30d245640 1 --2- 172.21.15.5:0/2094068127 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff308102dd0 0x7ff3081031b0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:22.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.443+0000 7ff30d245640 1 -- 172.21.15.5:0/2094068127 >> 172.21.15.5:0/2094068127 conn(0x7ff3080fc7d0 msgr2=0x7ff3080febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:22.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.443+0000 7ff30d245640 1 -- 172.21.15.5:0/2094068127 shutdown_connections 2024-09-12T03:00:22.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.443+0000 7ff30d245640 1 -- 172.21.15.5:0/2094068127 wait complete. 2024-09-12T03:00:22.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.444+0000 7ff30d245640 1 Processor -- start 2024-09-12T03:00:22.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.444+0000 7ff30d245640 1 -- start start 2024-09-12T03:00:22.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.444+0000 7ff30d245640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff308102dd0 0x7ff308198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:22.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.444+0000 7ff30d245640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff308103780 0x7ff308198b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:22.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.444+0000 7ff30d245640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff308108300 0x7ff30819cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:22.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.444+0000 7ff30d245640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff308112090 con 0x7ff308108300 2024-09-12T03:00:22.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.444+0000 7ff30d245640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ff308111f10 con 0x7ff308102dd0 2024-09-12T03:00:22.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.444+0000 7ff30d245640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff308112210 con 0x7ff308103780 2024-09-12T03:00:22.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.445+0000 7ff30759e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff308108300 0x7ff30819cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:22.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.445+0000 7ff30759e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff308108300 0x7ff30819cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56966/0 (socket says 172.21.15.5:56966) 2024-09-12T03:00:22.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.445+0000 7ff30759e640 1 -- 172.21.15.5:0/2252047734 learned_addr learned my addr 172.21.15.5:0/2252047734 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:00:22.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.445+0000 7ff306d9d640 1 --2- 172.21.15.5:0/2252047734 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff308102dd0 0x7ff308198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:22.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.445+0000 7ff30659c640 1 --2- 172.21.15.5:0/2252047734 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff308103780 0x7ff308198b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:22.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.445+0000 7ff30759e640 1 -- 172.21.15.5:0/2252047734 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff308103780 msgr2=0x7ff308198b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:22.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.445+0000 7ff30759e640 1 --2- 172.21.15.5:0/2252047734 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff308103780 0x7ff308198b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:22.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.445+0000 7ff30759e640 1 -- 172.21.15.5:0/2252047734 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff308102dd0 msgr2=0x7ff308198650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:22.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.445+0000 7ff30759e640 1 --2- 172.21.15.5:0/2252047734 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff308102dd0 0x7ff308198650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:22.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.445+0000 7ff30759e640 1 -- 172.21.15.5:0/2252047734 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff30819d600 con 0x7ff308108300 2024-09-12T03:00:22.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.445+0000 7ff30759e640 1 --2- 172.21.15.5:0/2252047734 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff308108300 0x7ff30819cf20 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7ff30000d950 tx=0x7ff30000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:22.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.445+0000 7ff2effff640 1 -- 172.21.15.5:0/2252047734 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff300017030 con 0x7ff308108300 2024-09-12T03:00:22.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.445+0000 7ff30d245640 1 -- 172.21.15.5:0/2252047734 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff30819d8f0 con 0x7ff308108300 2024-09-12T03:00:22.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.445+0000 7ff2effff640 1 -- 172.21.15.5:0/2252047734 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff300004020 con 0x7ff308108300 2024-09-12T03:00:22.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.445+0000 7ff2effff640 1 -- 172.21.15.5:0/2252047734 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff300003c00 con 0x7ff308108300 2024-09-12T03:00:22.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.446+0000 7ff30d245640 1 -- 172.21.15.5:0/2252047734 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3081a51d0 con 0x7ff308108300 2024-09-12T03:00:22.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.448+0000 7ff2effff640 1 -- 172.21.15.5:0/2252047734 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7ff30001a070 con 0x7ff308108300 2024-09-12T03:00:22.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.448+0000 7ff30f61e740 1 Processor -- start 2024-09-12T03:00:22.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.448+0000 7ff30f61e740 1 -- start start 2024-09-12T03:00:22.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.448+0000 7ff30f61e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e20c4fa4a0 0x55e20c4fa880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:22.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.449+0000 7ff30f61e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e20c4fadc0 0x55e20c505140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:22.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.449+0000 7ff30f61e740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e20c505680 0x55e20c507a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:22.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.449+0000 7ff30f61e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e20c4602e0 con 0x55e20c4fadc0 2024-09-12T03:00:22.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.449+0000 7ff30f61e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55e20c407ce0 con 0x55e20c4fa4a0 2024-09-12T03:00:22.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.449+0000 7ff30f61e740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55e20c45c2a0 con 0x55e20c505680 2024-09-12T03:00:22.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.449+0000 7ff2effff640 1 --2- 172.21.15.5:0/2252047734 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7ff2f00a2b80 0x7ff2f00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:22.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.449+0000 7ff306d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e20c4fadc0 0x55e20c505140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:22.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.449+0000 7ff30659c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e20c4fa4a0 0x55e20c4fa880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:22.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.449+0000 7ff306d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e20c4fadc0 0x55e20c505140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56970/0 (socket says 172.21.15.5:56970) 2024-09-12T03:00:22.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.449+0000 7ff306d9d640 1 -- 172.21.15.5:0/1100737066 learned_addr learned my addr 172.21.15.5:0/1100737066 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:00:22.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.449+0000 7ff30759e640 1 --2- 172.21.15.5:0/2252047734 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7ff2f00a2b80 0x7ff2f00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:22.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.449+0000 7ff30659c640 1 --2- 172.21.15.5:0/1100737066 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e20c505680 0x55e20c507a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:22.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.449+0000 7ff2effff640 1 -- 172.21.15.5:0/2252047734 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7ff3000ce830 con 0x7ff308108300 2024-09-12T03:00:22.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.449+0000 7ff306d9d640 1 -- 172.21.15.5:0/1100737066 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e20c505680 msgr2=0x55e20c507a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:00:22.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.449+0000 7ff306d9d640 1 --2- 172.21.15.5:0/1100737066 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e20c505680 0x55e20c507a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:22.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.449+0000 7ff306d9d640 1 -- 172.21.15.5:0/1100737066 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e20c4fa4a0 msgr2=0x55e20c4fa880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:00:22.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.449+0000 7ff306d9d640 1 --2- 172.21.15.5:0/1100737066 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e20c4fa4a0 0x55e20c4fa880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:22.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.449+0000 7ff306d9d640 1 -- 172.21.15.5:0/1100737066 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e20c3ec440 con 0x55e20c4fadc0 2024-09-12T03:00:22.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.449+0000 7ff30659c640 1 --2- 172.21.15.5:0/1100737066 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e20c505680 0x55e20c507a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:00:22.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.449+0000 7ff306d9d640 1 --2- 172.21.15.5:0/1100737066 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e20c4fadc0 0x55e20c505140 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7ff2f802adf0 tx=0x7ff2f8009880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f261ca911af08bac server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:22.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.449+0000 7ff30759e640 1 --2- 172.21.15.5:0/2252047734 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7ff2f00a2b80 0x7ff2f00a5040 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7ff3000036a0 tx=0x7ff3000035f0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:22.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.450+0000 7ff2ed7fa640 1 -- 172.21.15.5:0/1100737066 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f8031d70 con 0x55e20c4fadc0 2024-09-12T03:00:22.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.450+0000 7ff2ed7fa640 1 -- 172.21.15.5:0/1100737066 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2f8036030 con 0x55e20c4fadc0 2024-09-12T03:00:22.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.450+0000 7ff2ed7fa640 1 -- 172.21.15.5:0/1100737066 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f8004380 con 0x55e20c4fadc0 2024-09-12T03:00:22.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.450+0000 7ff30f61e740 1 -- 172.21.15.5:0/1100737066 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e20c4fadc0 msgr2=0x55e20c505140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:22.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.450+0000 7ff30f61e740 1 --2- 172.21.15.5:0/1100737066 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e20c4fadc0 0x55e20c505140 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7ff2f802adf0 tx=0x7ff2f8009880 comp rx=0 tx=0).stop 2024-09-12T03:00:22.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.450+0000 7ff30f61e740 1 -- 172.21.15.5:0/1100737066 shutdown_connections 2024-09-12T03:00:22.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.450+0000 7ff30f61e740 1 --2- 172.21.15.5:0/1100737066 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e20c4fadc0 0x55e20c505140 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:22.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.450+0000 7ff30f61e740 1 --2- 172.21.15.5:0/1100737066 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e20c505680 0x55e20c507a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:22.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.450+0000 7ff30f61e740 1 --2- 172.21.15.5:0/1100737066 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e20c4fa4a0 0x55e20c4fa880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:22.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.450+0000 7ff30f61e740 1 -- 172.21.15.5:0/1100737066 >> 172.21.15.5:0/1100737066 conn(0x55e20c4f6a30 msgr2=0x55e20c4f6e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:22.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.451+0000 7ff30f61e740 1 -- 172.21.15.5:0/1100737066 shutdown_connections 2024-09-12T03:00:22.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.451+0000 7ff30f61e740 1 -- 172.21.15.5:0/1100737066 wait complete. 2024-09-12T03:00:22.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.451+0000 7ff30f61e740 1 -- start start 2024-09-12T03:00:22.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.451+0000 7ff30f61e740 1 Processor -- start 2024-09-12T03:00:22.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.451+0000 7ff30f61e740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e20c4fa4a0 0x55e20c504f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:22.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.451+0000 7ff30f61e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e20c4fadc0 0x55e20c5021d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:22.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.451+0000 7ff30f61e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e20c505680 0x55e20c502710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:22.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.451+0000 7ff30f61e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e20c508250 con 0x55e20c4fadc0 2024-09-12T03:00:22.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.451+0000 7ff30f61e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55e20c41b050 con 0x55e20c505680 2024-09-12T03:00:22.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.451+0000 7ff30f61e740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55e20c5083d0 con 0x55e20c4fa4a0 2024-09-12T03:00:22.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.451+0000 7ff30f61e740 10 client.0 _ll_register_callbacks cb 0x55e20c45b160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:00:22.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.451+0000 7ff306d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e20c4fadc0 0x55e20c5021d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:22.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.451+0000 7ff306d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e20c4fadc0 0x55e20c5021d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56972/0 (socket says 172.21.15.5:56972) 2024-09-12T03:00:22.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.451+0000 7ff306d9d640 1 -- 172.21.15.5:0/3926611668 learned_addr learned my addr 172.21.15.5:0/3926611668 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:00:22.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.451+0000 7ff30659c640 1 --2- 172.21.15.5:0/3926611668 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e20c505680 0x55e20c502710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:22.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.452+0000 7ff30659c640 1 --2- 172.21.15.5:0/3926611668 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e20c4fa4a0 0x55e20c504f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:22.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.452+0000 7ff306d9d640 1 -- 172.21.15.5:0/3926611668 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e20c4fa4a0 msgr2=0x55e20c504f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:00:22.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.452+0000 7ff306d9d640 1 --2- 172.21.15.5:0/3926611668 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e20c4fa4a0 0x55e20c504f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:22.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.452+0000 7ff306d9d640 1 -- 172.21.15.5:0/3926611668 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e20c505680 msgr2=0x55e20c502710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:00:22.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.452+0000 7ff306d9d640 1 --2- 172.21.15.5:0/3926611668 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e20c505680 0x55e20c502710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:22.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.452+0000 7ff306d9d640 1 -- 172.21.15.5:0/3926611668 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e20c1d13c0 con 0x55e20c4fadc0 2024-09-12T03:00:22.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.452+0000 7ff30659c640 1 --2- 172.21.15.5:0/3926611668 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e20c4fa4a0 0x55e20c504f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:00:22.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.452+0000 7ff306d9d640 1 --2- 172.21.15.5:0/3926611668 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e20c4fadc0 0x55e20c5021d0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7ff2f8004080 tx=0x7ff2f80095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=195235d0e34ddd0a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:22.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.452+0000 7ff2eaff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T03:00:22.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.452+0000 7ff2eaff5640 1 -- 172.21.15.5:0/3926611668 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f804a040 con 0x55e20c4fadc0 2024-09-12T03:00:22.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.452+0000 7ff2eaff5640 1 -- 172.21.15.5:0/3926611668 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7ff2f8003b70 con 0x55e20c4fadc0 2024-09-12T03:00:22.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.452+0000 7ff30f61e740 1 -- 172.21.15.5:0/3926611668 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2f8036030 con 0x55e20c4fadc0 2024-09-12T03:00:22.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.452+0000 7ff2eaff5640 1 -- 172.21.15.5:0/3926611668 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff2f8031420 con 0x55e20c4fadc0 2024-09-12T03:00:22.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.452+0000 7ff30f61e740 10 client.15849 fetch_fsmap learned FSMap version 28 2024-09-12T03:00:22.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.452+0000 7ff30f61e740 1 -- 172.21.15.5:0/3926611668 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e20c508250 con 0x55e20c4fadc0 2024-09-12T03:00:22.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.452+0000 7ff2eaff5640 1 -- 172.21.15.5:0/3926611668 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2f8036030 con 0x55e20c4fadc0 2024-09-12T03:00:22.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.453+0000 7ff2eaff5640 1 -- 172.21.15.5:0/3926611668 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8374+0+0 (secure 0 0 0) 0x7ff2f804d030 con 0x55e20c4fadc0 2024-09-12T03:00:22.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.453+0000 7ff30f61e740 10 client.15849 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T03:00:22.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.453+0000 7ff30f61e740 10 client.15849 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:00:22.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.453+0000 7ff30f61e740 20 client.15849 populate_metadata read hostname 'smithi005' 2024-09-12T03:00:22.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.453+0000 7ff30f61e740 1 --2- 172.21.15.5:0/3926611668 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55e20c514cf0 0x55e20c5170e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:22.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.453+0000 7ff30f61e740 4 client.15849 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:00:22.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.453+0000 7ff30f61e740 1 -- 172.21.15.5:0/3926611668 --> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e20c46d850 con 0x55e20c514cf0 2024-09-12T03:00:22.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.453+0000 7ff30659c640 1 --2- 172.21.15.5:0/3926611668 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55e20c514cf0 0x55e20c5170e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:22.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.453+0000 7ff30659c640 1 --2- 172.21.15.5:0/3926611668 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55e20c514cf0 0x55e20c5170e0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1898ac443c49342f server_cookie=626cb6a014e25774 in_seq=0 out_seq=0 2024-09-12T03:00:22.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.453+0000 7ff2eaff5640 10 client.15849 ms_handle_connect on v2:172.21.15.148:6838/2317600000 2024-09-12T03:00:22.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.454+0000 7ff2eaff5640 1 -- 172.21.15.5:0/3926611668 <== mds.0 v2:172.21.15.148:6838/2317600000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e20c46d850 con 0x55e20c514cf0 2024-09-12T03:00:22.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.454+0000 7ff2eaff5640 10 client.15849 handle_command_reply: tid=0 2024-09-12T03:00:22.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.454+0000 7ff30f61e740 1 client.15849 shutdown 2024-09-12T03:00:22.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.454+0000 7ff30f61e740 1 -- 172.21.15.5:0/3926611668 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e20c4fadc0 msgr2=0x55e20c5021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:22.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.454+0000 7ff30f61e740 1 --2- 172.21.15.5:0/3926611668 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e20c4fadc0 0x55e20c5021d0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7ff2f8004080 tx=0x7ff2f80095c0 comp rx=0 tx=0).stop 2024-09-12T03:00:22.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.454+0000 7ff30f61e740 1 -- 172.21.15.5:0/3926611668 shutdown_connections 2024-09-12T03:00:22.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.454+0000 7ff30f61e740 1 --2- 172.21.15.5:0/3926611668 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55e20c514cf0 0x55e20c5170e0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:22.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.454+0000 7ff30f61e740 1 --2- 172.21.15.5:0/3926611668 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e20c4fadc0 0x55e20c5021d0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:22.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.454+0000 7ff30f61e740 1 --2- 172.21.15.5:0/3926611668 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e20c505680 0x55e20c502710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:22.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.454+0000 7ff30f61e740 1 --2- 172.21.15.5:0/3926611668 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e20c4fa4a0 0x55e20c504f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:22.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.454+0000 7ff30f61e740 1 -- 172.21.15.5:0/3926611668 >> 172.21.15.5:0/3926611668 conn(0x55e20c4f6a30 msgr2=0x55e20c4f5b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:22.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.454+0000 7ff2eaff5640 0 client.15849 ms_handle_reset on v2:172.21.15.148:6838/2317600000 2024-09-12T03:00:22.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.455+0000 7ff30f61e740 1 -- 172.21.15.5:0/3926611668 shutdown_connections 2024-09-12T03:00:22.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.455+0000 7ff30f61e740 1 -- 172.21.15.5:0/3926611668 wait complete. 2024-09-12T03:00:22.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.455+0000 7ff30f61e740 20 client.15849 trim_cache size 0 max 16384 2024-09-12T03:00:22.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.472+0000 7ff30f61e740 1 Processor -- start 2024-09-12T03:00:22.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.472+0000 7ff30f61e740 1 -- start start 2024-09-12T03:00:22.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.472+0000 7ff30f61e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e20c4fa4a0 0x55e20c5114b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:22.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.472+0000 7ff30f61e740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e20c4fadc0 0x55e20c501080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:22.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.472+0000 7ff30f61e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e20c505680 0x55e20c51c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:22.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.472+0000 7ff30f61e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff3081ab2c0 con 0x55e20c505680 2024-09-12T03:00:22.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.472+0000 7ff30f61e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ff3081ace90 con 0x55e20c4fa4a0 2024-09-12T03:00:22.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.472+0000 7ff30f61e740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff3081ab990 con 0x55e20c4fadc0 2024-09-12T03:00:22.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.472+0000 7ff30659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e20c505680 0x55e20c51c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:22.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.472+0000 7ff30659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e20c505680 0x55e20c51c8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56976/0 (socket says 172.21.15.5:56976) 2024-09-12T03:00:22.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.472+0000 7ff30659c640 1 -- 172.21.15.5:0/573594767 learned_addr learned my addr 172.21.15.5:0/573594767 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:00:22.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.472+0000 7ff30659c640 1 --2- 172.21.15.5:0/573594767 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e20c4fa4a0 0x55e20c5114b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:22.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.472+0000 7ff306d9d640 1 --2- 172.21.15.5:0/573594767 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e20c4fadc0 0x55e20c501080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:22.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.473+0000 7ff30659c640 1 -- 172.21.15.5:0/573594767 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e20c4fadc0 msgr2=0x55e20c501080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:00:22.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.473+0000 7ff30659c640 1 --2- 172.21.15.5:0/573594767 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e20c4fadc0 0x55e20c501080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:22.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.473+0000 7ff30659c640 1 -- 172.21.15.5:0/573594767 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e20c4fa4a0 msgr2=0x55e20c5114b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:00:22.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.473+0000 7ff30659c640 1 --2- 172.21.15.5:0/573594767 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e20c4fa4a0 0x55e20c5114b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:22.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.473+0000 7ff30659c640 1 -- 172.21.15.5:0/573594767 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e20c4f76f0 con 0x55e20c505680 2024-09-12T03:00:22.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.473+0000 7ff30659c640 1 --2- 172.21.15.5:0/573594767 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e20c505680 0x55e20c51c8f0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc005940 tx=0x7ff2fc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d4c0a784157c492 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:22.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.473+0000 7ff2ecff9640 1 -- 172.21.15.5:0/573594767 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2fc0268e0 con 0x55e20c505680 2024-09-12T03:00:22.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.473+0000 7ff2ecff9640 1 -- 172.21.15.5:0/573594767 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2fc028070 con 0x55e20c505680 2024-09-12T03:00:22.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.473+0000 7ff2ecff9640 1 -- 172.21.15.5:0/573594767 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2fc004a90 con 0x55e20c505680 2024-09-12T03:00:22.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.474+0000 7ff30f61e740 1 -- 172.21.15.5:0/573594767 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e20c505680 msgr2=0x55e20c51c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:22.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.474+0000 7ff30f61e740 1 --2- 172.21.15.5:0/573594767 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e20c505680 0x55e20c51c8f0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc005940 tx=0x7ff2fc004730 comp rx=0 tx=0).stop 2024-09-12T03:00:22.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.474+0000 7ff30f61e740 1 -- 172.21.15.5:0/573594767 shutdown_connections 2024-09-12T03:00:22.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.474+0000 7ff30f61e740 1 --2- 172.21.15.5:0/573594767 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e20c505680 0x55e20c51c8f0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:22.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.474+0000 7ff30f61e740 1 --2- 172.21.15.5:0/573594767 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e20c4fadc0 0x55e20c501080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:22.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.474+0000 7ff30f61e740 1 --2- 172.21.15.5:0/573594767 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e20c4fa4a0 0x55e20c5114b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:22.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.474+0000 7ff30f61e740 1 -- 172.21.15.5:0/573594767 >> 172.21.15.5:0/573594767 conn(0x55e20c4f6a30 msgr2=0x55e20c500970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:22.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.474+0000 7ff30f61e740 1 -- 172.21.15.5:0/573594767 shutdown_connections 2024-09-12T03:00:22.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.474+0000 7ff30f61e740 1 -- 172.21.15.5:0/573594767 wait complete. 2024-09-12T03:00:22.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.474+0000 7ff30f61e740 1 -- start start 2024-09-12T03:00:22.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.474+0000 7ff30f61e740 1 Processor -- start 2024-09-12T03:00:22.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.474+0000 7ff30f61e740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e20c4fa4a0 0x55e20c50ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:22.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.474+0000 7ff30f61e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e20c4fadc0 0x55e20c509120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:22.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.474+0000 7ff30f61e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e20c505680 0x55e20c509680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:22.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.474+0000 7ff30f61e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff3081ac730 con 0x55e20c505680 2024-09-12T03:00:22.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.474+0000 7ff30f61e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ff3081ac060 con 0x55e20c4fadc0 2024-09-12T03:00:22.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.474+0000 7ff30f61e740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff3081a92c0 con 0x55e20c4fa4a0 2024-09-12T03:00:22.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.474+0000 7ff30f61e740 10 client.0 _ll_register_callbacks cb 0x7ff2fc00ba20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:00:22.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.475+0000 7ff30659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e20c505680 0x55e20c509680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:22.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.475+0000 7ff30659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e20c505680 0x55e20c509680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56984/0 (socket says 172.21.15.5:56984) 2024-09-12T03:00:22.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.475+0000 7ff30659c640 1 -- 172.21.15.5:0/418304216 learned_addr learned my addr 172.21.15.5:0/418304216 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:00:22.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.475+0000 7ff30659c640 1 --2- 172.21.15.5:0/418304216 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e20c4fa4a0 0x55e20c50ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:22.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.475+0000 7ff306d9d640 1 --2- 172.21.15.5:0/418304216 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e20c4fadc0 0x55e20c509120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:22.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.475+0000 7ff30659c640 1 -- 172.21.15.5:0/418304216 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e20c4fa4a0 msgr2=0x55e20c50ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:00:22.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.475+0000 7ff30659c640 1 --2- 172.21.15.5:0/418304216 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e20c4fa4a0 0x55e20c50ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:22.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.475+0000 7ff30659c640 1 -- 172.21.15.5:0/418304216 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e20c4fadc0 msgr2=0x55e20c509120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:00:22.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.475+0000 7ff30659c640 1 --2- 172.21.15.5:0/418304216 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e20c4fadc0 0x55e20c509120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:22.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.475+0000 7ff30659c640 1 -- 172.21.15.5:0/418304216 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e20c509eb0 con 0x55e20c505680 2024-09-12T03:00:22.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.475+0000 7ff30659c640 1 --2- 172.21.15.5:0/418304216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e20c505680 0x55e20c509680 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc00bb30 tx=0x7ff2fc00b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d32315d7fc554180 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:22.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.475+0000 7ff2eaff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T03:00:22.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.475+0000 7ff2eaff5640 1 -- 172.21.15.5:0/418304216 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2fc026b10 con 0x55e20c505680 2024-09-12T03:00:22.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.475+0000 7ff2eaff5640 1 -- 172.21.15.5:0/418304216 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7ff2fc01d040 con 0x55e20c505680 2024-09-12T03:00:22.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.475+0000 7ff30f61e740 1 -- 172.21.15.5:0/418304216 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2fc028070 con 0x55e20c505680 2024-09-12T03:00:22.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.476+0000 7ff2eaff5640 1 -- 172.21.15.5:0/418304216 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff2fc013020 con 0x55e20c505680 2024-09-12T03:00:22.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.476+0000 7ff30f61e740 10 client.15855 fetch_fsmap learned FSMap version 28 2024-09-12T03:00:22.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.476+0000 7ff30f61e740 1 -- 172.21.15.5:0/418304216 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e20c57f960 con 0x55e20c505680 2024-09-12T03:00:22.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.476+0000 7ff2eaff5640 1 -- 172.21.15.5:0/418304216 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2fc028070 con 0x55e20c505680 2024-09-12T03:00:22.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.476+0000 7ff2eaff5640 1 -- 172.21.15.5:0/418304216 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8374+0+0 (secure 0 0 0) 0x7ff2fc03d020 con 0x55e20c505680 2024-09-12T03:00:22.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.476+0000 7ff30f61e740 10 client.15855 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T03:00:22.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.476+0000 7ff30f61e740 10 client.15855 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:00:22.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.476+0000 7ff30f61e740 20 client.15855 populate_metadata read hostname 'smithi005' 2024-09-12T03:00:22.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.476+0000 7ff30f61e740 1 --2- 172.21.15.5:0/418304216 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55e20c580000 0x55e20c582420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:22.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.476+0000 7ff30f61e740 4 client.15855 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T03:00:22.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.476+0000 7ff30f61e740 1 -- 172.21.15.5:0/418304216 --> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e20c503320 con 0x55e20c580000 2024-09-12T03:00:22.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.476+0000 7ff306d9d640 1 --2- 172.21.15.5:0/418304216 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55e20c580000 0x55e20c582420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:22.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.477+0000 7ff2eaff5640 10 client.15855 ms_handle_connect on v2:172.21.15.148:6838/2317600000 2024-09-12T03:00:22.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.477+0000 7ff306d9d640 1 --2- 172.21.15.5:0/418304216 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55e20c580000 0x55e20c582420 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=807563e4b40f1cdc server_cookie=c1644308b54c10d1 in_seq=0 out_seq=0 2024-09-12T03:00:22.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.477+0000 7ff2eaff5640 1 -- 172.21.15.5:0/418304216 <== mds.0 v2:172.21.15.148:6838/2317600000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55e20c503320 con 0x55e20c580000 2024-09-12T03:00:22.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.477+0000 7ff2eaff5640 10 client.15855 handle_command_reply: tid=0 2024-09-12T03:00:22.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.477+0000 7ff30f61e740 1 client.15855 shutdown 2024-09-12T03:00:22.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.477+0000 7ff30f61e740 1 -- 172.21.15.5:0/418304216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e20c505680 msgr2=0x55e20c509680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:22.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.477+0000 7ff30f61e740 1 --2- 172.21.15.5:0/418304216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e20c505680 0x55e20c509680 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc00bb30 tx=0x7ff2fc00b390 comp rx=0 tx=0).stop 2024-09-12T03:00:22.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.477+0000 7ff30f61e740 1 -- 172.21.15.5:0/418304216 shutdown_connections 2024-09-12T03:00:22.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.477+0000 7ff30f61e740 1 --2- 172.21.15.5:0/418304216 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55e20c580000 0x55e20c582420 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:22.641 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-12T03:00:22.641 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-09-12T03:00:22.641 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-09-12T03:00:22.641 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-12T03:00:22.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.477+0000 7ff30f61e740 1 --2- 172.21.15.5:0/418304216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e20c505680 0x55e20c509680 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:22.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.477+0000 7ff2eaff5640 0 client.15855 ms_handle_reset on v2:172.21.15.148:6838/2317600000 2024-09-12T03:00:22.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.477+0000 7ff30f61e740 1 --2- 172.21.15.5:0/418304216 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e20c4fadc0 0x55e20c509120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:22.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.477+0000 7ff30f61e740 1 --2- 172.21.15.5:0/418304216 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e20c4fa4a0 0x55e20c50ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:22.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.477+0000 7ff30f61e740 1 -- 172.21.15.5:0/418304216 >> 172.21.15.5:0/418304216 conn(0x55e20c4f6a30 msgr2=0x55e20c501190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:22.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.477+0000 7ff30f61e740 1 -- 172.21.15.5:0/418304216 shutdown_connections 2024-09-12T03:00:22.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.477+0000 7ff30f61e740 1 -- 172.21.15.5:0/418304216 wait complete. 2024-09-12T03:00:22.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.477+0000 7ff30f61e740 20 client.15855 trim_cache size 0 max 16384 2024-09-12T03:00:22.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.478+0000 7ff30d245640 1 -- 172.21.15.5:0/2252047734 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7ff2f00a2b80 msgr2=0x7ff2f00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:22.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.478+0000 7ff30d245640 1 --2- 172.21.15.5:0/2252047734 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7ff2f00a2b80 0x7ff2f00a5040 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7ff3000036a0 tx=0x7ff3000035f0 comp rx=0 tx=0).stop 2024-09-12T03:00:22.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.478+0000 7ff30d245640 1 -- 172.21.15.5:0/2252047734 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff308108300 msgr2=0x7ff30819cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:22.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.478+0000 7ff30d245640 1 --2- 172.21.15.5:0/2252047734 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff308108300 0x7ff30819cf20 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7ff30000d950 tx=0x7ff30000de10 comp rx=0 tx=0).stop 2024-09-12T03:00:22.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.478+0000 7ff30d245640 1 -- 172.21.15.5:0/2252047734 shutdown_connections 2024-09-12T03:00:22.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.478+0000 7ff30d245640 1 --2- 172.21.15.5:0/2252047734 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7ff2f00a2b80 0x7ff2f00a5040 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:22.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.478+0000 7ff30d245640 1 --2- 172.21.15.5:0/2252047734 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff308108300 0x7ff30819cf20 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:22.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.478+0000 7ff30d245640 1 --2- 172.21.15.5:0/2252047734 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff308103780 0x7ff308198b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:22.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.478+0000 7ff30d245640 1 --2- 172.21.15.5:0/2252047734 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff308102dd0 0x7ff308198650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:22.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.478+0000 7ff30d245640 1 -- 172.21.15.5:0/2252047734 >> 172.21.15.5:0/2252047734 conn(0x7ff3080fc7d0 msgr2=0x7ff308105ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:22.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.478+0000 7ff30d245640 1 -- 172.21.15.5:0/2252047734 shutdown_connections 2024-09-12T03:00:22.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:22.478+0000 7ff30d245640 1 -- 172.21.15.5:0/2252047734 wait complete. 2024-09-12T03:00:22.664 INFO:tasks.workunit.client.0.smithi005.stdout:checking for FICLONE... yes 2024-09-12T03:00:22.711 INFO:tasks.workunit.client.0.smithi005.stdout:checking for renameat2... yes 2024-09-12T03:00:22.754 INFO:tasks.workunit.client.0.smithi005.stdout:checking for reallocarray... yes 2024-09-12T03:00:22.818 INFO:tasks.workunit.client.0.smithi005.stdout:checking for struct mount_attr... yes 2024-09-12T03:00:22.906 INFO:tasks.workunit.client.0.smithi005.stdout:checking for struct btrfs_qgroup_limit... yes 2024-09-12T03:00:22.976 INFO:tasks.workunit.client.0.smithi005.stdout:checking for struct btrfs_qgroup_inherit... yes 2024-09-12T03:00:23.038 INFO:tasks.workunit.client.0.smithi005.stdout:checking for struct btrfs_ioctl_vol_args... yes 2024-09-12T03:00:23.097 INFO:tasks.workunit.client.0.smithi005.stdout:checking for struct btrfs_ioctl_vol_args_v2... yes 2024-09-12T03:00:23.163 INFO:tasks.workunit.client.0.smithi005.stdout:checking for struct btrfs_ioctl_ino_lookup_args... yes 2024-09-12T03:00:23.221 INFO:tasks.workunit.client.0.smithi005.stdout:checking for struct btrfs_ioctl_ino_lookup_user_args... yes 2024-09-12T03:00:23.298 INFO:tasks.workunit.client.0.smithi005.stdout:checking for struct btrfs_ioctl_get_subvol_rootref_args... yes 2024-09-12T03:00:23.339 INFO:tasks.workunit.client.0.smithi005.stdout:checking linux/btrfs.h usability... yes 2024-09-12T03:00:23.354 INFO:tasks.workunit.client.0.smithi005.stdout:checking linux/btrfs.h presence... yes 2024-09-12T03:00:23.355 INFO:tasks.workunit.client.0.smithi005.stdout:checking for linux/btrfs.h... yes 2024-09-12T03:00:23.415 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:00:23 smithi148 ceph-mon[35090]: pgmap v399: 97 pgs: 97 active+clean; 65 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 337 KiB/s wr, 50 op/s 2024-09-12T03:00:23.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:00:23 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:00:23.421 INFO:tasks.workunit.client.0.smithi005.stdout:checking linux/btrfs_tree.h usability... yes 2024-09-12T03:00:23.444 INFO:tasks.workunit.client.0.smithi005.stdout:checking linux/btrfs_tree.h presence... yes 2024-09-12T03:00:23.444 INFO:tasks.workunit.client.0.smithi005.stdout:checking for linux/btrfs_tree.h... yes 2024-09-12T03:00:23.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:00:23 smithi146 ceph-mon[35011]: pgmap v399: 97 pgs: 97 active+clean; 65 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 337 KiB/s wr, 50 op/s 2024-09-12T03:00:23.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:00:23 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:00:23.492 INFO:tasks.workunit.client.0.smithi005.stdout:checking for struct btrfs_ioctl_vol_args_v2.subvolid... yes 2024-09-12T03:00:23.500 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:00:23 smithi005 ceph-mon[31197]: pgmap v399: 97 pgs: 97 active+clean; 65 MiB data, 644 MiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 337 KiB/s wr, 50 op/s 2024-09-12T03:00:23.500 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:00:23 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:00:23.528 INFO:tasks.workunit.client.0.smithi005.stdout:checking whether BTRFS_IOC_SNAP_DESTROY_V2 is declared... yes 2024-09-12T03:00:23.533 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T03:00:23.533 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:00:23.550 INFO:tasks.workunit.client.0.smithi005.stdout:configure: creating ./config.status 2024-09-12T03:00:23.822 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:00:23.909 INFO:tasks.workunit.client.0.smithi005.stdout:config.status: creating include/builddefs 2024-09-12T03:00:23.923 INFO:tasks.workunit.client.0.smithi005.stdout:config.status: creating include/config.h 2024-09-12T03:00:23.935 INFO:tasks.workunit.client.0.smithi005.stdout:config.status: executing libtool commands 2024-09-12T03:00:24.019 INFO:tasks.workunit.client.0.smithi005.stdout:Building include 2024-09-12T03:00:24.019 INFO:tasks.workunit.client.0.smithi005.stdout:Building lib 2024-09-12T03:00:24.019 INFO:tasks.workunit.client.0.smithi005.stdout:Building tests 2024-09-12T03:00:24.024 INFO:tasks.workunit.client.0.smithi005.stdout: [LTDEP] 2024-09-12T03:00:24.027 INFO:tasks.workunit.client.0.smithi005.stdout:Building btrfs 2024-09-12T03:00:24.027 INFO:tasks.workunit.client.0.smithi005.stdout:Building ceph 2024-09-12T03:00:24.027 INFO:tasks.workunit.client.0.smithi005.stdout:Building cifs 2024-09-12T03:00:24.037 INFO:tasks.workunit.client.0.smithi005.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/ceph/group.list 2024-09-12T03:00:24.037 INFO:tasks.workunit.client.0.smithi005.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/cifs/group.list 2024-09-12T03:00:24.053 INFO:tasks.workunit.client.0.smithi005.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/btrfs/group.list 2024-09-12T03:00:24.063 INFO:tasks.workunit.client.0.smithi005.stdout:Building ext4 2024-09-12T03:00:24.070 INFO:tasks.workunit.client.0.smithi005.stdout:Building f2fs 2024-09-12T03:00:24.081 INFO:tasks.workunit.client.0.smithi005.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/ext4/group.list 2024-09-12T03:00:24.081 INFO:tasks.workunit.client.0.smithi005.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/f2fs/group.list 2024-09-12T03:00:24.137 INFO:tasks.workunit.client.0.smithi005.stdout:Building generic 2024-09-12T03:00:24.217 INFO:tasks.workunit.client.0.smithi005.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/generic/group.list 2024-09-12T03:00:24.228 INFO:tasks.workunit.client.0.smithi005.stdout:Building nfs 2024-09-12T03:00:24.236 INFO:tasks.workunit.client.0.smithi005.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/nfs/group.list 2024-09-12T03:00:24.267 INFO:tasks.workunit.client.0.smithi005.stdout:Building ocfs2 2024-09-12T03:00:24.275 INFO:tasks.workunit.client.0.smithi005.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/ocfs2/group.list 2024-09-12T03:00:24.276 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] dataascii.lo 2024-09-12T03:00:24.296 INFO:tasks.workunit.client.0.smithi005.stdout:Building overlay 2024-09-12T03:00:24.313 INFO:tasks.workunit.client.0.smithi005.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/overlay/group.list 2024-09-12T03:00:24.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:24.312+0000 7fca88778640 1 -- 172.21.15.5:0/1735806340 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fca80102ef0 msgr2=0x7fca8010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:24.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:24.312+0000 7fca88778640 1 --2- 172.21.15.5:0/1735806340 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fca80102ef0 0x7fca8010da40 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fca78009c10 tx=0x7fca7802a770 comp rx=0 tx=0).stop 2024-09-12T03:00:24.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:24.312+0000 7fca88778640 1 -- 172.21.15.5:0/1735806340 shutdown_connections 2024-09-12T03:00:24.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:24.312+0000 7fca88778640 1 --2- 172.21.15.5:0/1735806340 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca80102570 0x7fca801029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:24.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:24.312+0000 7fca88778640 1 --2- 172.21.15.5:0/1735806340 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fca80102ef0 0x7fca8010da40 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:24.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:24.312+0000 7fca88778640 1 --2- 172.21.15.5:0/1735806340 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fca80100e20 0x7fca80101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:24.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:24.312+0000 7fca88778640 1 -- 172.21.15.5:0/1735806340 >> 172.21.15.5:0/1735806340 conn(0x7fca800fc7d0 msgr2=0x7fca800febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:24.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:24.312+0000 7fca88778640 1 -- 172.21.15.5:0/1735806340 shutdown_connections 2024-09-12T03:00:24.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:24.313+0000 7fca88778640 1 -- 172.21.15.5:0/1735806340 wait complete. 2024-09-12T03:00:24.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:24.313+0000 7fca88778640 1 Processor -- start 2024-09-12T03:00:24.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:24.313+0000 7fca88778640 1 -- start start 2024-09-12T03:00:24.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:24.314+0000 7fca88778640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fca80100e20 0x7fca801992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:24.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:24.314+0000 7fca88778640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fca80102570 0x7fca80199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:24.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:24.314+0000 7fca88778640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca80102ef0 0x7fca801a4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:24.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:24.314+0000 7fca88778640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fca80110f30 con 0x7fca80102ef0 2024-09-12T03:00:24.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:24.314+0000 7fca88778640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fca80110db0 con 0x7fca80100e20 2024-09-12T03:00:24.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:24.314+0000 7fca88778640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fca801110b0 con 0x7fca80102570 2024-09-12T03:00:24.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:24.314+0000 7fca85d13640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fca80102570 0x7fca80199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:24.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:24.314+0000 7fca86514640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fca80100e20 0x7fca801992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:24.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:24.314+0000 7fca85d13640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fca80102570 0x7fca80199830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.5:33100/0 (socket says 172.21.15.5:33100) 2024-09-12T03:00:24.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:24.314+0000 7fca85d13640 1 -- 172.21.15.5:0/1153339229 learned_addr learned my addr 172.21.15.5:0/1153339229 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:00:24.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:24.314+0000 7fca86d15640 1 --2- 172.21.15.5:0/1153339229 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca80102ef0 0x7fca801a4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:24.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:24.314+0000 7fca85d13640 1 -- 172.21.15.5:0/1153339229 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fca80100e20 msgr2=0x7fca801992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:24.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:24.314+0000 7fca85d13640 1 --2- 172.21.15.5:0/1153339229 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fca80100e20 0x7fca801992f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:24.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:24.314+0000 7fca85d13640 1 -- 172.21.15.5:0/1153339229 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca80102ef0 msgr2=0x7fca801a4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:24.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:24.314+0000 7fca85d13640 1 --2- 172.21.15.5:0/1153339229 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca80102ef0 0x7fca801a4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:24.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:24.314+0000 7fca85d13640 1 -- 172.21.15.5:0/1153339229 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca801a48b0 con 0x7fca80102570 2024-09-12T03:00:24.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:24.314+0000 7fca85d13640 1 --2- 172.21.15.5:0/1153339229 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fca80102570 0x7fca80199830 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fca7000e9c0 tx=0x7fca7000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:24.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:24.315+0000 7fca777fe640 1 -- 172.21.15.5:0/1153339229 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca7000ccc0 con 0x7fca80102570 2024-09-12T03:00:24.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:24.315+0000 7fca777fe640 1 -- 172.21.15.5:0/1153339229 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca70005500 con 0x7fca80102570 2024-09-12T03:00:24.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:24.315+0000 7fca88778640 1 -- 172.21.15.5:0/1153339229 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca801970e0 con 0x7fca80102570 2024-09-12T03:00:24.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:24.315+0000 7fca777fe640 1 -- 172.21.15.5:0/1153339229 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca70010430 con 0x7fca80102570 2024-09-12T03:00:24.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:24.315+0000 7fca88778640 1 -- 172.21.15.5:0/1153339229 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca80197670 con 0x7fca80102570 2024-09-12T03:00:24.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:24.317+0000 7fca88778640 1 -- 172.21.15.5:0/1153339229 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca80101200 con 0x7fca80102570 2024-09-12T03:00:24.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:24.318+0000 7fca777fe640 1 -- 172.21.15.5:0/1153339229 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fca700046c0 con 0x7fca80102570 2024-09-12T03:00:24.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:24.319+0000 7fca777fe640 1 --2- 172.21.15.5:0/1153339229 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fca6c0a2c00 0x7fca6c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:24.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:24.319+0000 7fca777fe640 1 -- 172.21.15.5:0/1153339229 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fca700cf250 con 0x7fca80102570 2024-09-12T03:00:24.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:24.319+0000 7fca86514640 1 --2- 172.21.15.5:0/1153339229 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fca6c0a2c00 0x7fca6c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:24.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:24.320+0000 7fca86514640 1 --2- 172.21.15.5:0/1153339229 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fca6c0a2c00 0x7fca6c0a50c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fca7c010d80 tx=0x7fca7c012040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:24.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:24.323+0000 7fca777fe640 1 -- 172.21.15.5:0/1153339229 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fca7009c160 con 0x7fca80102570 2024-09-12T03:00:24.434 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] databin.lo 2024-09-12T03:00:24.474 INFO:tasks.workunit.client.0.smithi005.stdout:Building perf 2024-09-12T03:00:24.483 INFO:tasks.workunit.client.0.smithi005.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/perf/group.list 2024-09-12T03:00:24.506 INFO:tasks.workunit.client.0.smithi005.stdout:Building selftest 2024-09-12T03:00:24.514 INFO:tasks.workunit.client.0.smithi005.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/selftest/group.list 2024-09-12T03:00:24.515 INFO:tasks.workunit.client.0.smithi005.stdout:Building shared 2024-09-12T03:00:24.525 INFO:tasks.workunit.client.0.smithi005.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/shared/group.list 2024-09-12T03:00:24.540 INFO:tasks.workunit.client.0.smithi005.stdout:Building tmpfs 2024-09-12T03:00:24.547 INFO:tasks.workunit.client.0.smithi005.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/tmpfs/group.list 2024-09-12T03:00:24.553 INFO:tasks.workunit.client.0.smithi005.stdout:Building udf 2024-09-12T03:00:24.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:00:24 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T03:00:24.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:00:24 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T03:00:24.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:00:24 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T03:00:24.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:00:24 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T03:00:24.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:00:24 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:00:24.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:00:24 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:00:24.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:00:24 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T03:00:24.563 INFO:tasks.workunit.client.0.smithi005.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/udf/group.list 2024-09-12T03:00:24.567 INFO:tasks.workunit.client.0.smithi005.stdout:Building xfs 2024-09-12T03:00:24.581 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] datapid.lo 2024-09-12T03:00:24.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:24.619+0000 7fca88778640 1 -- 172.21.15.5:0/1153339229 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fca80109090 con 0x7fca80102570 2024-09-12T03:00:24.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:24.620+0000 7fca777fe640 1 -- 172.21.15.5:0/1153339229 <== mon.1 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11971 (secure 0 0 0) 0x7fca7009ba60 con 0x7fca80102570 2024-09-12T03:00:24.621 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-12T03:00:24.621 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":28,"btime":"2024-09-12T03:00:03:036015+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/4135232521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":4135232521},{"type":"v1","addr":"172.21.15.5:6835","nonce":4135232521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/4290221940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":4290221940},{"type":"v1","addr":"172.21.15.5:6837","nonce":4290221940}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T02:56:16.595562+0000","modified":"2024-09-12T03:00:02.675878+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24502,"mds_1":24487,"mds_2":24481,"mds_3":24479,"mds_4":24458},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/26447341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":26447341},{"type":"v1","addr":"172.21.15.5:6839","nonce":26447341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2393443295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2393443295},{"type":"v1","addr":"172.21.15.5:6841","nonce":2393443295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6833/3686970137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3686970137},{"type":"v1","addr":"172.21.15.146:6833","nonce":3686970137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6833/568512209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":568512209},{"type":"v1","addr":"172.21.15.148:6833","nonce":568512209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6835/2540740420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2540740420},{"type":"v1","addr":"172.21.15.146:6835","nonce":2540740420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.146:6837/1673370937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":1673370937},{"type":"v1","addr":"172.21.15.146:6837","nonce":1673370937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":3,"incarnation":22,"state":"up:active","state_seq":6,"addr":"172.21.15.146:6839/3036767228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":3036767228},{"type":"v1","addr":"172.21.15.146:6839","nonce":3036767228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":2,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.148:6835/4195086427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":4195086427},{"type":"v1","addr":"172.21.15.148:6835","nonce":4195086427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.148:6837/2606873922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2606873922},{"type":"v1","addr":"172.21.15.148:6837","nonce":2606873922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6839/2317600000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":2317600000},{"type":"v1","addr":"172.21.15.148:6839","nonce":2317600000}]},"join_fscid":-1,"export_targets":[1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24502,"qdb_cluster":[24502,24487,24481,24479,24458]},"id":1}]} 2024-09-12T03:00:24.621 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 28 2024-09-12T03:00:24.622 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] file_lock.lo 2024-09-12T03:00:24.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:24.623+0000 7fca88778640 1 -- 172.21.15.5:0/1153339229 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fca6c0a2c00 msgr2=0x7fca6c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:24.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:24.623+0000 7fca88778640 1 --2- 172.21.15.5:0/1153339229 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fca6c0a2c00 0x7fca6c0a50c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fca7c010d80 tx=0x7fca7c012040 comp rx=0 tx=0).stop 2024-09-12T03:00:24.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:24.623+0000 7fca88778640 1 -- 172.21.15.5:0/1153339229 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fca80102570 msgr2=0x7fca80199830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:24.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:24.623+0000 7fca88778640 1 --2- 172.21.15.5:0/1153339229 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fca80102570 0x7fca80199830 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fca7000e9c0 tx=0x7fca7000ee80 comp rx=0 tx=0).stop 2024-09-12T03:00:24.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:24.623+0000 7fca88778640 1 -- 172.21.15.5:0/1153339229 shutdown_connections 2024-09-12T03:00:24.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:24.623+0000 7fca88778640 1 --2- 172.21.15.5:0/1153339229 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fca6c0a2c00 0x7fca6c0a50c0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:24.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:24.623+0000 7fca88778640 1 --2- 172.21.15.5:0/1153339229 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca80102ef0 0x7fca801a4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:24.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:24.623+0000 7fca88778640 1 --2- 172.21.15.5:0/1153339229 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fca80102570 0x7fca80199830 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:24.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:24.623+0000 7fca88778640 1 --2- 172.21.15.5:0/1153339229 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fca80100e20 0x7fca801992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:24.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:24.623+0000 7fca88778640 1 -- 172.21.15.5:0/1153339229 >> 172.21.15.5:0/1153339229 conn(0x7fca800fc7d0 msgr2=0x7fca801048f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:24.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:24.623+0000 7fca88778640 1 -- 172.21.15.5:0/1153339229 shutdown_connections 2024-09-12T03:00:24.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:24.623+0000 7fca88778640 1 -- 172.21.15.5:0/1153339229 wait complete. 2024-09-12T03:00:24.644 INFO:tasks.workunit.client.0.smithi005.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/xfs/group.list 2024-09-12T03:00:24.665 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:00:24 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T03:00:24.665 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:00:24 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T03:00:24.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:00:24 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T03:00:24.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:00:24 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T03:00:24.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:00:24 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:00:24.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:00:24 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:00:24.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:00:24 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T03:00:24.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:00:24 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T03:00:24.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:00:24 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T03:00:24.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:00:24 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T03:00:24.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:00:24 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T03:00:24.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:00:24 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:00:24.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:00:24 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:00:24.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:00:24 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T03:00:24.710 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] forker.lo 2024-09-12T03:00:24.804 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] pattern.lo 2024-09-12T03:00:24.875 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] open_flags.lo 2024-09-12T03:00:24.950 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] random_range.lo 2024-09-12T03:00:25.054 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] string_to_tokens.lo 2024-09-12T03:00:25.140 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] str_to_bytes.lo 2024-09-12T03:00:25.185 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] tlibio.lo 2024-09-12T03:00:25.222 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] write_log.lo 2024-09-12T03:00:25.348 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] random.lo 2024-09-12T03:00:25.412 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-12T03:00:25.503 INFO:tasks.workunit.client.0.smithi005.stdout: [LD] 2024-09-12T03:00:25.645 INFO:tasks.workunit.client.0.smithi005.stdout:Building src 2024-09-12T03:00:25.645 INFO:tasks.workunit.client.0.smithi005.stdout:Building ltp 2024-09-12T03:00:25.645 INFO:tasks.workunit.client.0.smithi005.stdout:Building m4 2024-09-12T03:00:25.645 INFO:tasks.workunit.client.0.smithi005.stdout:Building common 2024-09-12T03:00:25.653 INFO:tasks.workunit.client.0.smithi005.stdout: [DEP] 2024-09-12T03:00:25.664 INFO:tasks.workunit.client.0.smithi005.stdout: [DEP] 2024-09-12T03:00:25.665 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:00:25 smithi148 ceph-mon[35090]: pgmap v400: 97 pgs: 97 active+clean; 66 MiB data, 652 MiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 369 KiB/s wr, 32 op/s 2024-09-12T03:00:25.665 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:00:25 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/1153339229' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:00:25.695 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:00:25.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:00:25 smithi146 ceph-mon[35011]: pgmap v400: 97 pgs: 97 active+clean; 66 MiB data, 652 MiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 369 KiB/s wr, 32 op/s 2024-09-12T03:00:25.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:00:25 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/1153339229' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:00:25.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:00:25 smithi005 ceph-mon[31197]: pgmap v400: 97 pgs: 97 active+clean; 66 MiB data, 652 MiB used, 1.0 TiB / 1.0 TiB avail; 348 KiB/s rd, 369 KiB/s wr, 32 op/s 2024-09-12T03:00:25.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:00:25 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/1153339229' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:00:25.992 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] doio 2024-09-12T03:00:25.992 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] fsstress 2024-09-12T03:00:25.993 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] fsx 2024-09-12T03:00:26.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.264+0000 7fe8f8655640 1 -- 172.21.15.5:0/3938320392 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8f0100a90 msgr2=0x7fe8f0100ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:26.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.264+0000 7fe8f8655640 1 --2- 172.21.15.5:0/3938320392 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8f0100a90 0x7fe8f0100ef0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fe8e0009930 tx=0x7fe8e002a420 comp rx=0 tx=0).stop 2024-09-12T03:00:26.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.264+0000 7fe8f8655640 1 -- 172.21.15.5:0/3938320392 shutdown_connections 2024-09-12T03:00:26.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.264+0000 7fe8f8655640 1 --2- 172.21.15.5:0/3938320392 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8f0100a90 0x7fe8f0100ef0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:26.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.264+0000 7fe8f8655640 1 --2- 172.21.15.5:0/3938320392 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe8f0101430 0x7fe8f010d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:26.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.264+0000 7fe8f8655640 1 --2- 172.21.15.5:0/3938320392 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe8f0106a90 0x7fe8f0106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:26.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.264+0000 7fe8f8655640 1 -- 172.21.15.5:0/3938320392 >> 172.21.15.5:0/3938320392 conn(0x7fe8f00fc7d0 msgr2=0x7fe8f00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:26.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.264+0000 7fe8f8655640 1 -- 172.21.15.5:0/3938320392 shutdown_connections 2024-09-12T03:00:26.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.264+0000 7fe8f8655640 1 -- 172.21.15.5:0/3938320392 wait complete. 2024-09-12T03:00:26.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.265+0000 7fe8f8655640 1 Processor -- start 2024-09-12T03:00:26.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.265+0000 7fe8f8655640 1 -- start start 2024-09-12T03:00:26.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.265+0000 7fe8f8655640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8f0100a90 0x7fe8f0196290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:26.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.265+0000 7fe8f8655640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe8f0101430 0x7fe8f01967d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:26.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.265+0000 7fe8f8655640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe8f0106a90 0x7fe8f019ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:26.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.265+0000 7fe8f8655640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f010fd90 con 0x7fe8f0100a90 2024-09-12T03:00:26.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.265+0000 7fe8f8655640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f010fc10 con 0x7fe8f0106a90 2024-09-12T03:00:26.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.265+0000 7fe8f8655640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f010ff10 con 0x7fe8f0101430 2024-09-12T03:00:26.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.265+0000 7fe8f63f1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8f0100a90 0x7fe8f0196290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:26.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.265+0000 7fe8f63f1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8f0100a90 0x7fe8f0196290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48246/0 (socket says 172.21.15.5:48246) 2024-09-12T03:00:26.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.266+0000 7fe8f63f1640 1 -- 172.21.15.5:0/1368302913 learned_addr learned my addr 172.21.15.5:0/1368302913 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:00:26.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.266+0000 7fe8f5bf0640 1 --2- 172.21.15.5:0/1368302913 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe8f0101430 0x7fe8f01967d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:26.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.266+0000 7fe8f6bf2640 1 --2- 172.21.15.5:0/1368302913 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe8f0106a90 0x7fe8f019ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:26.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.266+0000 7fe8f63f1640 1 -- 172.21.15.5:0/1368302913 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe8f0101430 msgr2=0x7fe8f01967d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:26.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.266+0000 7fe8f63f1640 1 --2- 172.21.15.5:0/1368302913 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe8f0101430 0x7fe8f01967d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:26.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.266+0000 7fe8f63f1640 1 -- 172.21.15.5:0/1368302913 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe8f0106a90 msgr2=0x7fe8f019ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:26.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.266+0000 7fe8f63f1640 1 --2- 172.21.15.5:0/1368302913 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe8f0106a90 0x7fe8f019ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:26.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.266+0000 7fe8f63f1640 1 -- 172.21.15.5:0/1368302913 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8f019b240 con 0x7fe8f0100a90 2024-09-12T03:00:26.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.266+0000 7fe8f63f1640 1 --2- 172.21.15.5:0/1368302913 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8f0100a90 0x7fe8f0196290 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec009be0 tx=0x7fe8ec0071c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:26.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.266+0000 7fe8e77fe640 1 -- 172.21.15.5:0/1368302913 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8ec007e00 con 0x7fe8f0100a90 2024-09-12T03:00:26.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.267+0000 7fe8f8655640 1 -- 172.21.15.5:0/1368302913 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8f019b530 con 0x7fe8f0100a90 2024-09-12T03:00:26.391 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.267+0000 7fe8e77fe640 1 -- 172.21.15.5:0/1368302913 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8ec01e070 con 0x7fe8f0100a90 2024-09-12T03:00:26.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.267+0000 7fe8e77fe640 1 -- 172.21.15.5:0/1368302913 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8ec018890 con 0x7fe8f0100a90 2024-09-12T03:00:26.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.267+0000 7fe8f8655640 1 -- 172.21.15.5:0/1368302913 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8f01a2e10 con 0x7fe8f0100a90 2024-09-12T03:00:26.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.269+0000 7fe8e77fe640 1 -- 172.21.15.5:0/1368302913 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fe8ec019b60 con 0x7fe8f0100a90 2024-09-12T03:00:26.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.270+0000 7fe8faa2e740 1 Processor -- start 2024-09-12T03:00:26.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.271+0000 7fe8e77fe640 1 --2- 172.21.15.5:0/1368302913 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fe8dc0a2b80 0x7fe8dc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:26.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.271+0000 7fe8faa2e740 1 -- start start 2024-09-12T03:00:26.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.271+0000 7fe8e77fe640 1 -- 172.21.15.5:0/1368302913 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fe8ec0d2930 con 0x7fe8f0100a90 2024-09-12T03:00:26.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.271+0000 7fe8f6bf2640 1 --2- 172.21.15.5:0/1368302913 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fe8dc0a2b80 0x7fe8dc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:26.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.271+0000 7fe8faa2e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559923174b40 0x559923173090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:26.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.271+0000 7fe8faa2e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5599231735d0 0x559923173a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:26.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.271+0000 7fe8faa2e740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55992317e370 0x559923180760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:26.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.271+0000 7fe8faa2e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5599230d92f0 con 0x559923174b40 2024-09-12T03:00:26.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.272+0000 7fe8faa2e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x559923080cf0 con 0x5599231735d0 2024-09-12T03:00:26.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.272+0000 7fe8f5bf0640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5599231735d0 0x559923173a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:26.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.272+0000 7fe8f5bf0640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5599231735d0 0x559923173a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.5:54030/0 (socket says 172.21.15.5:54030) 2024-09-12T03:00:26.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.272+0000 7fe8f5bf0640 1 -- 172.21.15.5:0/400060479 learned_addr learned my addr 172.21.15.5:0/400060479 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:00:26.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.272+0000 7fe8faa2e740 1 -- 172.21.15.5:0/400060479 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5599230d52b0 con 0x55992317e370 2024-09-12T03:00:26.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.272+0000 7fe8f63f1640 1 --2- 172.21.15.5:0/400060479 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559923174b40 0x559923173090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:26.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.272+0000 7fe8f6bf2640 1 --2- 172.21.15.5:0/1368302913 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fe8dc0a2b80 0x7fe8dc0a5040 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fe8e8005fd0 tx=0x7fe8e80096a0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:26.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.272+0000 7fe8f5bf0640 1 -- 172.21.15.5:0/400060479 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55992317e370 msgr2=0x559923180760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:00:26.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.272+0000 7fe8f5bf0640 1 --2- 172.21.15.5:0/400060479 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55992317e370 0x559923180760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:26.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.272+0000 7fe8f5bf0640 1 -- 172.21.15.5:0/400060479 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559923174b40 msgr2=0x559923173090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:00:26.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.272+0000 7fe8f5bf0640 1 --2- 172.21.15.5:0/400060479 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559923174b40 0x559923173090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:26.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.272+0000 7fe8f5bf0640 1 -- 172.21.15.5:0/400060479 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599230653a0 con 0x5599231735d0 2024-09-12T03:00:26.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.272+0000 7fe8f5bf0640 1 --2- 172.21.15.5:0/400060479 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5599231735d0 0x559923173a10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe8e002a940 tx=0x7fe8e0005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c0759c8dd143ab1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:26.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.273+0000 7fe8e4ff9640 1 -- 172.21.15.5:0/400060479 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e0040070 con 0x5599231735d0 2024-09-12T03:00:26.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.273+0000 7fe8e4ff9640 1 -- 172.21.15.5:0/400060479 <== mon.2 v2:172.21.15.146:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8e00318f0 con 0x5599231735d0 2024-09-12T03:00:26.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.273+0000 7fe8e4ff9640 1 -- 172.21.15.5:0/400060479 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e0035040 con 0x5599231735d0 2024-09-12T03:00:26.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.273+0000 7fe8faa2e740 1 -- 172.21.15.5:0/400060479 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5599231735d0 msgr2=0x559923173a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:26.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.273+0000 7fe8faa2e740 1 --2- 172.21.15.5:0/400060479 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5599231735d0 0x559923173a10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe8e002a940 tx=0x7fe8e0005c00 comp rx=0 tx=0).stop 2024-09-12T03:00:26.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.274+0000 7fe8faa2e740 1 -- 172.21.15.5:0/400060479 shutdown_connections 2024-09-12T03:00:26.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.274+0000 7fe8faa2e740 1 --2- 172.21.15.5:0/400060479 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55992317e370 0x559923180760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:26.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.274+0000 7fe8faa2e740 1 --2- 172.21.15.5:0/400060479 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5599231735d0 0x559923173a10 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:26.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.274+0000 7fe8faa2e740 1 --2- 172.21.15.5:0/400060479 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559923174b40 0x559923173090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:26.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.274+0000 7fe8faa2e740 1 -- 172.21.15.5:0/400060479 >> 172.21.15.5:0/400060479 conn(0x55992313c310 msgr2=0x55992316fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:26.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.274+0000 7fe8faa2e740 1 -- 172.21.15.5:0/400060479 shutdown_connections 2024-09-12T03:00:26.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.274+0000 7fe8faa2e740 1 -- 172.21.15.5:0/400060479 wait complete. 2024-09-12T03:00:26.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.274+0000 7fe8faa2e740 1 -- start start 2024-09-12T03:00:26.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.274+0000 7fe8faa2e740 1 Processor -- start 2024-09-12T03:00:26.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.275+0000 7fe8faa2e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5599231735d0 0x55992317dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:26.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.275+0000 7fe8faa2e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559923174b40 0x55992317afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:26.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.275+0000 7fe8faa2e740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55992317e370 0x55992317b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:26.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.275+0000 7fe8faa2e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559923181010 con 0x559923174b40 2024-09-12T03:00:26.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.275+0000 7fe8faa2e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x559923094060 con 0x5599231735d0 2024-09-12T03:00:26.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.275+0000 7fe8faa2e740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x559923181190 con 0x55992317e370 2024-09-12T03:00:26.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.275+0000 7fe8f5bf0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559923174b40 0x55992317afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:26.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.275+0000 7fe8f5bf0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559923174b40 0x55992317afe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48262/0 (socket says 172.21.15.5:48262) 2024-09-12T03:00:26.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.275+0000 7fe8f63f1640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5599231735d0 0x55992317dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:26.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.275+0000 7fe8f5bf0640 1 -- 172.21.15.5:0/2860118427 learned_addr learned my addr 172.21.15.5:0/2860118427 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:00:26.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.275+0000 7fe8f5bf0640 1 -- 172.21.15.5:0/2860118427 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55992317e370 msgr2=0x55992317b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:00:26.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.275+0000 7fe8f6bf2640 1 --2- 172.21.15.5:0/2860118427 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55992317e370 0x55992317b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:26.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.275+0000 7fe8f5bf0640 1 --2- 172.21.15.5:0/2860118427 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55992317e370 0x55992317b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:26.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.275+0000 7fe8f5bf0640 1 -- 172.21.15.5:0/2860118427 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5599231735d0 msgr2=0x55992317dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:00:26.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.275+0000 7fe8f5bf0640 1 --2- 172.21.15.5:0/2860118427 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5599231735d0 0x55992317dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:26.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.275+0000 7fe8f5bf0640 1 -- 172.21.15.5:0/2860118427 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559922e4a3c0 con 0x559923174b40 2024-09-12T03:00:26.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.275+0000 7fe8faa2e740 10 client.0 _ll_register_callbacks cb 0x559922de78c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:00:26.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.276+0000 7fe8faa2e740 1 -- 172.21.15.5:0/2860118427 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8e00318f0 con 0x559923174b40 2024-09-12T03:00:26.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.276+0000 7fe8f5bf0640 1 --2- 172.21.15.5:0/2860118427 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559923174b40 0x55992317afe0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fe8e00052d0 tx=0x7fe8e00054b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=424deb0f89d02b97 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:26.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.276+0000 7fe8d9ffb640 10 client.15873 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T03:00:26.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.276+0000 7fe8f6bf2640 1 --2- 172.21.15.5:0/2860118427 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55992317e370 0x55992317b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:00:26.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.276+0000 7fe8f63f1640 1 --2- 172.21.15.5:0/2860118427 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5599231735d0 0x55992317dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:00:26.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.276+0000 7fe8d9ffb640 1 -- 172.21.15.5:0/2860118427 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e0040030 con 0x559923174b40 2024-09-12T03:00:26.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.276+0000 7fe8d9ffb640 1 -- 172.21.15.5:0/2860118427 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fe8e0004260 con 0x559923174b40 2024-09-12T03:00:26.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.276+0000 7fe8d9ffb640 1 -- 172.21.15.5:0/2860118427 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe8e0035040 con 0x559923174b40 2024-09-12T03:00:26.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.277+0000 7fe8faa2e740 10 client.15873 fetch_fsmap learned FSMap version 28 2024-09-12T03:00:26.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.277+0000 7fe8faa2e740 1 -- 172.21.15.5:0/2860118427 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({fsmap=28}) -- 0x55992318c880 con 0x559923174b40 2024-09-12T03:00:26.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.277+0000 7fe8d9ffb640 1 -- 172.21.15.5:0/2860118427 <== mon.0 v2:172.21.15.5:3300/0 4 ==== fsmap(e 28) ==== 8374+0+0 (secure 0 0 0) 0x7fe8e0004770 con 0x559923174b40 2024-09-12T03:00:26.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.277+0000 7fe8faa2e740 10 client.15873 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T03:00:26.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.277+0000 7fe8faa2e740 10 client.15873 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-12T03:00:26.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.277+0000 7fe8faa2e740 20 client.15873 populate_metadata read hostname 'smithi005' 2024-09-12T03:00:26.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.277+0000 7fe8faa2e740 1 --2- 172.21.15.5:0/2860118427 >> [v2:172.21.15.146:6836/1673370937,v1:172.21.15.146:6837/1673370937] conn(0x55992318dbd0 0x55992318ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:26.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.277+0000 7fe8faa2e740 4 client.15873 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:00:26.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.277+0000 7fe8faa2e740 1 -- 172.21.15.5:0/2860118427 --> [v2:172.21.15.146:6836/1673370937,v1:172.21.15.146:6837/1673370937] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5599230e6860 con 0x55992318dbd0 2024-09-12T03:00:26.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.278+0000 7fe8f63f1640 1 --2- 172.21.15.5:0/2860118427 >> [v2:172.21.15.146:6836/1673370937,v1:172.21.15.146:6837/1673370937] conn(0x55992318dbd0 0x55992318ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:26.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.278+0000 7fe8d9ffb640 10 client.15873 ms_handle_connect on v2:172.21.15.146:6836/1673370937 2024-09-12T03:00:26.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.278+0000 7fe8f63f1640 1 --2- 172.21.15.5:0/2860118427 >> [v2:172.21.15.146:6836/1673370937,v1:172.21.15.146:6837/1673370937] conn(0x55992318dbd0 0x55992318ffc0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9b3b8a8a571d564f server_cookie=5801845bb72cb777 in_seq=0 out_seq=0 2024-09-12T03:00:26.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.279+0000 7fe8d9ffb640 1 -- 172.21.15.5:0/2860118427 <== mds.4 v2:172.21.15.146:6836/1673370937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5599230e6860 con 0x55992318dbd0 2024-09-12T03:00:26.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.279+0000 7fe8d9ffb640 10 client.15873 handle_command_reply: tid=0 2024-09-12T03:00:26.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.279+0000 7fe8faa2e740 1 client.15873 shutdown 2024-09-12T03:00:26.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.279+0000 7fe8faa2e740 1 -- 172.21.15.5:0/2860118427 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559923174b40 msgr2=0x55992317afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:26.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.279+0000 7fe8faa2e740 1 --2- 172.21.15.5:0/2860118427 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559923174b40 0x55992317afe0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fe8e00052d0 tx=0x7fe8e00054b0 comp rx=0 tx=0).stop 2024-09-12T03:00:26.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.279+0000 7fe8faa2e740 1 -- 172.21.15.5:0/2860118427 shutdown_connections 2024-09-12T03:00:26.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.279+0000 7fe8faa2e740 1 --2- 172.21.15.5:0/2860118427 >> [v2:172.21.15.146:6836/1673370937,v1:172.21.15.146:6837/1673370937] conn(0x55992318dbd0 0x55992318ffc0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:26.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.279+0000 7fe8faa2e740 1 --2- 172.21.15.5:0/2860118427 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559923174b40 0x55992317afe0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:26.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.279+0000 7fe8d9ffb640 0 client.15873 ms_handle_reset on v2:172.21.15.146:6836/1673370937 2024-09-12T03:00:26.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.279+0000 7fe8faa2e740 1 --2- 172.21.15.5:0/2860118427 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55992317e370 0x55992317b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:26.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.279+0000 7fe8faa2e740 1 --2- 172.21.15.5:0/2860118427 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5599231735d0 0x55992317dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:26.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.279+0000 7fe8faa2e740 1 -- 172.21.15.5:0/2860118427 >> 172.21.15.5:0/2860118427 conn(0x55992313c310 msgr2=0x559923175930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:26.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.279+0000 7fe8faa2e740 1 -- 172.21.15.5:0/2860118427 shutdown_connections 2024-09-12T03:00:26.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.280+0000 7fe8faa2e740 1 -- 172.21.15.5:0/2860118427 wait complete. 2024-09-12T03:00:26.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.280+0000 7fe8faa2e740 20 client.15873 trim_cache size 0 max 16384 2024-09-12T03:00:26.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.308+0000 7fe8faa2e740 1 Processor -- start 2024-09-12T03:00:26.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.308+0000 7fe8faa2e740 1 -- start start 2024-09-12T03:00:26.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.308+0000 7fe8faa2e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5599231735d0 0x55992318a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:26.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.308+0000 7fe8faa2e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559923174b40 0x559923195590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:26.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.308+0000 7fe8faa2e740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55992317e370 0x559923195ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:26.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.308+0000 7fe8faa2e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f01ab390 con 0x559923174b40 2024-09-12T03:00:26.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.308+0000 7fe8faa2e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f01acf60 con 0x5599231735d0 2024-09-12T03:00:26.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.308+0000 7fe8faa2e740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f01aba60 con 0x55992317e370 2024-09-12T03:00:26.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.308+0000 7fe8f5bf0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559923174b40 0x559923195590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:26.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.308+0000 7fe8f5bf0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559923174b40 0x559923195590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48266/0 (socket says 172.21.15.5:48266) 2024-09-12T03:00:26.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.309+0000 7fe8f5bf0640 1 -- 172.21.15.5:0/2132952485 learned_addr learned my addr 172.21.15.5:0/2132952485 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:00:26.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.309+0000 7fe8f63f1640 1 --2- 172.21.15.5:0/2132952485 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5599231735d0 0x55992318a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:26.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.309+0000 7fe8f6bf2640 1 --2- 172.21.15.5:0/2132952485 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55992317e370 0x559923195ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:26.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.309+0000 7fe8f6bf2640 1 -- 172.21.15.5:0/2132952485 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5599231735d0 msgr2=0x55992318a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:00:26.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.309+0000 7fe8f6bf2640 1 --2- 172.21.15.5:0/2132952485 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5599231735d0 0x55992318a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:26.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.309+0000 7fe8f6bf2640 1 -- 172.21.15.5:0/2132952485 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559923174b40 msgr2=0x559923195590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:00:26.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.309+0000 7fe8f6bf2640 1 --2- 172.21.15.5:0/2132952485 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559923174b40 0x559923195590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:26.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.309+0000 7fe8f6bf2640 1 -- 172.21.15.5:0/2132952485 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55992316e310 con 0x55992317e370 2024-09-12T03:00:26.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.309+0000 7fe8f63f1640 1 --2- 172.21.15.5:0/2132952485 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5599231735d0 0x55992318a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:00:26.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.309+0000 7fe8f6bf2640 1 --2- 172.21.15.5:0/2132952485 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55992317e370 0x559923195ad0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe8e8004410 tx=0x7fe8e8013070 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e015d46bf2062190 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:26.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.310+0000 7fe8dbfff640 1 -- 172.21.15.5:0/2132952485 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e8027470 con 0x55992317e370 2024-09-12T03:00:26.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.310+0000 7fe8dbfff640 1 -- 172.21.15.5:0/2132952485 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8e8022070 con 0x55992317e370 2024-09-12T03:00:26.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.310+0000 7fe8dbfff640 1 -- 172.21.15.5:0/2132952485 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e801e9e0 con 0x55992317e370 2024-09-12T03:00:26.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.311+0000 7fe8faa2e740 1 -- 172.21.15.5:0/2132952485 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55992317e370 msgr2=0x559923195ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:26.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.311+0000 7fe8faa2e740 1 --2- 172.21.15.5:0/2132952485 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55992317e370 0x559923195ad0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe8e8004410 tx=0x7fe8e8013070 comp rx=0 tx=0).stop 2024-09-12T03:00:26.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.311+0000 7fe8faa2e740 1 -- 172.21.15.5:0/2132952485 shutdown_connections 2024-09-12T03:00:26.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.311+0000 7fe8faa2e740 1 --2- 172.21.15.5:0/2132952485 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559923174b40 0x559923195590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:26.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.311+0000 7fe8faa2e740 1 --2- 172.21.15.5:0/2132952485 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55992317e370 0x559923195ad0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:26.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.311+0000 7fe8faa2e740 1 --2- 172.21.15.5:0/2132952485 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5599231735d0 0x55992318a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:26.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.311+0000 7fe8faa2e740 1 -- 172.21.15.5:0/2132952485 >> 172.21.15.5:0/2132952485 conn(0x55992313c310 msgr2=0x55992317d060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:26.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.311+0000 7fe8faa2e740 1 -- 172.21.15.5:0/2132952485 shutdown_connections 2024-09-12T03:00:26.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.311+0000 7fe8faa2e740 1 -- 172.21.15.5:0/2132952485 wait complete. 2024-09-12T03:00:26.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.311+0000 7fe8faa2e740 1 -- start start 2024-09-12T03:00:26.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.312+0000 7fe8faa2e740 1 Processor -- start 2024-09-12T03:00:26.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.312+0000 7fe8faa2e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5599231735d0 0x559923182ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:26.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.312+0000 7fe8faa2e740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559923174b40 0x559923182ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:26.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.312+0000 7fe8faa2e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55992317e370 0x5599231f1530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:26.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.312+0000 7fe8faa2e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f01ac800 con 0x55992317e370 2024-09-12T03:00:26.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.312+0000 7fe8faa2e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f01ac130 con 0x5599231735d0 2024-09-12T03:00:26.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.312+0000 7fe8faa2e740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f01a9370 con 0x559923174b40 2024-09-12T03:00:26.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.312+0000 7fe8faa2e740 10 client.0 _ll_register_callbacks cb 0x7fe8e0005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:00:26.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.312+0000 7fe8f6bf2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55992317e370 0x5599231f1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:26.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.312+0000 7fe8f6bf2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55992317e370 0x5599231f1530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48268/0 (socket says 172.21.15.5:48268) 2024-09-12T03:00:26.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.312+0000 7fe8f6bf2640 1 -- 172.21.15.5:0/2981364036 learned_addr learned my addr 172.21.15.5:0/2981364036 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:00:26.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.312+0000 7fe8f63f1640 1 --2- 172.21.15.5:0/2981364036 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5599231735d0 0x559923182ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:26.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.312+0000 7fe8f5bf0640 1 --2- 172.21.15.5:0/2981364036 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559923174b40 0x559923182ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:26.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.313+0000 7fe8f6bf2640 1 -- 172.21.15.5:0/2981364036 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559923174b40 msgr2=0x559923182ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:00:26.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.313+0000 7fe8f6bf2640 1 --2- 172.21.15.5:0/2981364036 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559923174b40 0x559923182ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:26.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.313+0000 7fe8f6bf2640 1 -- 172.21.15.5:0/2981364036 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5599231735d0 msgr2=0x559923182ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:00:26.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.313+0000 7fe8f6bf2640 1 --2- 172.21.15.5:0/2981364036 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5599231735d0 0x559923182ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:26.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.313+0000 7fe8f6bf2640 1 -- 172.21.15.5:0/2981364036 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599231f1bd0 con 0x55992317e370 2024-09-12T03:00:26.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.313+0000 7fe8f63f1640 1 --2- 172.21.15.5:0/2981364036 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5599231735d0 0x559923182ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:00:26.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.313+0000 7fe8f5bf0640 1 --2- 172.21.15.5:0/2981364036 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559923174b40 0x559923182ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:00:26.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.313+0000 7fe8f6bf2640 1 --2- 172.21.15.5:0/2981364036 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55992317e370 0x5599231f1530 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fe8e8009c00 tx=0x7fe8e8009d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2facf8c8cb4f794e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:26.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.313+0000 7fe8d9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T03:00:26.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.313+0000 7fe8d9ffb640 1 -- 172.21.15.5:0/2981364036 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e8037060 con 0x55992317e370 2024-09-12T03:00:26.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.313+0000 7fe8d9ffb640 1 -- 172.21.15.5:0/2981364036 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fe8e801e870 con 0x55992317e370 2024-09-12T03:00:26.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.313+0000 7fe8faa2e740 1 -- 172.21.15.5:0/2981364036 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8e8022070 con 0x55992317e370 2024-09-12T03:00:26.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.314+0000 7fe8d9ffb640 1 -- 172.21.15.5:0/2981364036 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe8e801ea70 con 0x55992317e370 2024-09-12T03:00:26.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.314+0000 7fe8faa2e740 10 client.15879 fetch_fsmap learned FSMap version 28 2024-09-12T03:00:26.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.314+0000 7fe8faa2e740 1 -- 172.21.15.5:0/2981364036 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5599231f8f80 con 0x55992317e370 2024-09-12T03:00:26.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.314+0000 7fe8d9ffb640 1 -- 172.21.15.5:0/2981364036 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe8e8022070 con 0x55992317e370 2024-09-12T03:00:26.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.314+0000 7fe8d9ffb640 1 -- 172.21.15.5:0/2981364036 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8374+0+0 (secure 0 0 0) 0x7fe8e8032410 con 0x55992317e370 2024-09-12T03:00:26.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.314+0000 7fe8faa2e740 10 client.15879 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T03:00:26.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.314+0000 7fe8faa2e740 10 client.15879 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-12T03:00:26.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.314+0000 7fe8faa2e740 20 client.15879 populate_metadata read hostname 'smithi005' 2024-09-12T03:00:26.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.314+0000 7fe8faa2e740 1 --2- 172.21.15.5:0/2981364036 >> [v2:172.21.15.146:6836/1673370937,v1:172.21.15.146:6837/1673370937] conn(0x55992318dbd0 0x5599231fb550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:26.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.314+0000 7fe8faa2e740 4 client.15879 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:00:26.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.314+0000 7fe8faa2e740 1 -- 172.21.15.5:0/2981364036 --> [v2:172.21.15.146:6836/1673370937,v1:172.21.15.146:6837/1673370937] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5599230e0d70 con 0x55992318dbd0 2024-09-12T03:00:26.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.315+0000 7fe8f63f1640 1 --2- 172.21.15.5:0/2981364036 >> [v2:172.21.15.146:6836/1673370937,v1:172.21.15.146:6837/1673370937] conn(0x55992318dbd0 0x5599231fb550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:26.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.315+0000 7fe8d9ffb640 10 client.15879 ms_handle_connect on v2:172.21.15.146:6836/1673370937 2024-09-12T03:00:26.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.315+0000 7fe8f63f1640 1 --2- 172.21.15.5:0/2981364036 >> [v2:172.21.15.146:6836/1673370937,v1:172.21.15.146:6837/1673370937] conn(0x55992318dbd0 0x5599231fb550 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=36e1f025fb339894 server_cookie=3fe9c7f59ae773f8 in_seq=0 out_seq=0 2024-09-12T03:00:26.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.316+0000 7fe8d9ffb640 1 -- 172.21.15.5:0/2981364036 <== mds.4 v2:172.21.15.146:6836/1673370937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5599230e0d70 con 0x55992318dbd0 2024-09-12T03:00:26.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.316+0000 7fe8d9ffb640 10 client.15879 handle_command_reply: tid=0 2024-09-12T03:00:26.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.316+0000 7fe8faa2e740 1 client.15879 shutdown 2024-09-12T03:00:26.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.317+0000 7fe8faa2e740 1 -- 172.21.15.5:0/2981364036 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55992317e370 msgr2=0x5599231f1530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:26.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.317+0000 7fe8faa2e740 1 --2- 172.21.15.5:0/2981364036 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55992317e370 0x5599231f1530 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fe8e8009c00 tx=0x7fe8e8009d90 comp rx=0 tx=0).stop 2024-09-12T03:00:26.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.318+0000 7fe8faa2e740 1 -- 172.21.15.5:0/2981364036 shutdown_connections 2024-09-12T03:00:26.405 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-12T03:00:26.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.318+0000 7fe8faa2e740 1 --2- 172.21.15.5:0/2981364036 >> [v2:172.21.15.146:6836/1673370937,v1:172.21.15.146:6837/1673370937] conn(0x55992318dbd0 0x5599231fb550 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:26.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.318+0000 7fe8faa2e740 1 --2- 172.21.15.5:0/2981364036 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55992317e370 0x5599231f1530 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:26.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.318+0000 7fe8faa2e740 1 --2- 172.21.15.5:0/2981364036 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559923174b40 0x559923182ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:26.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.318+0000 7fe8d9ffb640 0 client.15879 ms_handle_reset on v2:172.21.15.146:6836/1673370937 2024-09-12T03:00:26.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.318+0000 7fe8faa2e740 1 --2- 172.21.15.5:0/2981364036 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5599231735d0 0x559923182ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:26.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.318+0000 7fe8faa2e740 1 -- 172.21.15.5:0/2981364036 >> 172.21.15.5:0/2981364036 conn(0x55992313c310 msgr2=0x559923175110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:26.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.318+0000 7fe8faa2e740 1 -- 172.21.15.5:0/2981364036 shutdown_connections 2024-09-12T03:00:26.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.318+0000 7fe8faa2e740 1 -- 172.21.15.5:0/2981364036 wait complete. 2024-09-12T03:00:26.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.318+0000 7fe8faa2e740 20 client.15879 trim_cache size 0 max 16384 2024-09-12T03:00:26.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.319+0000 7fe8f8655640 1 -- 172.21.15.5:0/1368302913 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fe8dc0a2b80 msgr2=0x7fe8dc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:26.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.319+0000 7fe8f8655640 1 --2- 172.21.15.5:0/1368302913 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fe8dc0a2b80 0x7fe8dc0a5040 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fe8e8005fd0 tx=0x7fe8e80096a0 comp rx=0 tx=0).stop 2024-09-12T03:00:26.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.319+0000 7fe8f8655640 1 -- 172.21.15.5:0/1368302913 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8f0100a90 msgr2=0x7fe8f0196290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:26.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.319+0000 7fe8f8655640 1 --2- 172.21.15.5:0/1368302913 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8f0100a90 0x7fe8f0196290 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec009be0 tx=0x7fe8ec0071c0 comp rx=0 tx=0).stop 2024-09-12T03:00:26.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.319+0000 7fe8f8655640 1 -- 172.21.15.5:0/1368302913 shutdown_connections 2024-09-12T03:00:26.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.319+0000 7fe8f8655640 1 --2- 172.21.15.5:0/1368302913 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe8f0106a90 0x7fe8f019ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:26.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.319+0000 7fe8f8655640 1 --2- 172.21.15.5:0/1368302913 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe8f0101430 0x7fe8f01967d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:26.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.319+0000 7fe8f8655640 1 --2- 172.21.15.5:0/1368302913 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fe8dc0a2b80 0x7fe8dc0a5040 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:26.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.319+0000 7fe8f8655640 1 --2- 172.21.15.5:0/1368302913 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe8f0100a90 0x7fe8f0196290 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:26.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.319+0000 7fe8f8655640 1 -- 172.21.15.5:0/1368302913 >> 172.21.15.5:0/1368302913 conn(0x7fe8f00fc7d0 msgr2=0x7fe8f00fd060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:26.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.319+0000 7fe8f8655640 1 -- 172.21.15.5:0/1368302913 shutdown_connections 2024-09-12T03:00:26.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:26.319+0000 7fe8f8655640 1 -- 172.21.15.5:0/1368302913 wait complete. 2024-09-12T03:00:26.835 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] growfiles 2024-09-12T03:00:26.872 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] iogen 2024-09-12T03:00:27.192 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-12T03:00:27.281 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] aio-stress 2024-09-12T03:00:27.511 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:00:27.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:00:27 smithi005 ceph-mon[31197]: pgmap v401: 97 pgs: 97 active+clean; 68 MiB data, 657 MiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 322 KiB/s wr, 83 op/s 2024-09-12T03:00:27.665 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:00:27 smithi148 ceph-mon[35090]: pgmap v401: 97 pgs: 97 active+clean; 68 MiB data, 657 MiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 322 KiB/s wr, 83 op/s 2024-09-12T03:00:27.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:00:27 smithi146 ceph-mon[35011]: pgmap v401: 97 pgs: 97 active+clean; 68 MiB data, 657 MiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 322 KiB/s wr, 83 op/s 2024-09-12T03:00:28.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.046+0000 7fa811f81640 1 -- 172.21.15.5:0/2204676368 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa80c102570 msgr2=0x7fa80c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:28.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.046+0000 7fa811f81640 1 --2- 172.21.15.5:0/2204676368 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa80c102570 0x7fa80c1029b0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fa7f4009900 tx=0x7fa7f402a480 comp rx=0 tx=0).stop 2024-09-12T03:00:28.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.046+0000 7fa811f81640 1 -- 172.21.15.5:0/2204676368 shutdown_connections 2024-09-12T03:00:28.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.046+0000 7fa811f81640 1 --2- 172.21.15.5:0/2204676368 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa80c102570 0x7fa80c1029b0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:28.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.046+0000 7fa811f81640 1 --2- 172.21.15.5:0/2204676368 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa80c102ef0 0x7fa80c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:28.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.046+0000 7fa811f81640 1 --2- 172.21.15.5:0/2204676368 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa80c100e20 0x7fa80c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:28.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.046+0000 7fa811f81640 1 -- 172.21.15.5:0/2204676368 >> 172.21.15.5:0/2204676368 conn(0x7fa80c0fc7d0 msgr2=0x7fa80c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:28.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.046+0000 7fa811f81640 1 -- 172.21.15.5:0/2204676368 shutdown_connections 2024-09-12T03:00:28.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.046+0000 7fa811f81640 1 -- 172.21.15.5:0/2204676368 wait complete. 2024-09-12T03:00:28.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.047+0000 7fa811f81640 1 Processor -- start 2024-09-12T03:00:28.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.047+0000 7fa811f81640 1 -- start start 2024-09-12T03:00:28.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.047+0000 7fa811f81640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa80c100e20 0x7fa80c198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:28.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.047+0000 7fa811f81640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa80c102570 0x7fa80c198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:28.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.047+0000 7fa811f81640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa80c102ef0 0x7fa80c19cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:28.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.047+0000 7fa811f81640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa80c110f50 con 0x7fa80c102570 2024-09-12T03:00:28.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.047+0000 7fa811f81640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fa80c110dd0 con 0x7fa80c102ef0 2024-09-12T03:00:28.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.047+0000 7fa811f81640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa80c1110d0 con 0x7fa80c100e20 2024-09-12T03:00:28.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.047+0000 7fa80affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa80c102570 0x7fa80c198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:28.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.047+0000 7fa80affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa80c102570 0x7fa80c198a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48286/0 (socket says 172.21.15.5:48286) 2024-09-12T03:00:28.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.047+0000 7fa80affd640 1 -- 172.21.15.5:0/792321071 learned_addr learned my addr 172.21.15.5:0/792321071 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:00:28.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.047+0000 7fa80b7fe640 1 --2- 172.21.15.5:0/792321071 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa80c100e20 0x7fa80c198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:28.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.048+0000 7fa80bfff640 1 --2- 172.21.15.5:0/792321071 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa80c102ef0 0x7fa80c19cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:28.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.048+0000 7fa80affd640 1 -- 172.21.15.5:0/792321071 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa80c100e20 msgr2=0x7fa80c198520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:28.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.048+0000 7fa80affd640 1 --2- 172.21.15.5:0/792321071 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa80c100e20 0x7fa80c198520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:28.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.048+0000 7fa80affd640 1 -- 172.21.15.5:0/792321071 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa80c102ef0 msgr2=0x7fa80c19cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:28.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.048+0000 7fa80affd640 1 --2- 172.21.15.5:0/792321071 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa80c102ef0 0x7fa80c19cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:28.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.048+0000 7fa80affd640 1 -- 172.21.15.5:0/792321071 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa80c19d4d0 con 0x7fa80c102570 2024-09-12T03:00:28.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.048+0000 7fa80bfff640 1 --2- 172.21.15.5:0/792321071 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa80c102ef0 0x7fa80c19cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:00:28.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.048+0000 7fa80affd640 1 --2- 172.21.15.5:0/792321071 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa80c102570 0x7fa80c198a60 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fa7f4006010 tx=0x7fa7f4003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:28.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.048+0000 7fa808ff9640 1 -- 172.21.15.5:0/792321071 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f403f070 con 0x7fa80c102570 2024-09-12T03:00:28.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.048+0000 7fa811f81640 1 -- 172.21.15.5:0/792321071 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa80c19d760 con 0x7fa80c102570 2024-09-12T03:00:28.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.048+0000 7fa808ff9640 1 -- 172.21.15.5:0/792321071 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7f4031c30 con 0x7fa80c102570 2024-09-12T03:00:28.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.048+0000 7fa808ff9640 1 -- 172.21.15.5:0/792321071 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f4035030 con 0x7fa80c102570 2024-09-12T03:00:28.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.048+0000 7fa811f81640 1 -- 172.21.15.5:0/792321071 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa80c1a50a0 con 0x7fa80c102570 2024-09-12T03:00:28.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.051+0000 7fa81435a740 1 Processor -- start 2024-09-12T03:00:28.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.051+0000 7fa81435a740 1 -- start start 2024-09-12T03:00:28.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.051+0000 7fa808ff9640 1 -- 172.21.15.5:0/792321071 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fa7f4042070 con 0x7fa80c102570 2024-09-12T03:00:28.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.051+0000 7fa81435a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558caba5bb40 0x558caba5a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:28.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.051+0000 7fa81435a740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558caba5a5d0 0x558caba5aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:28.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.051+0000 7fa81435a740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558caba65370 0x558caba67760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:28.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.051+0000 7fa81435a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558cab9c02f0 con 0x558caba5bb40 2024-09-12T03:00:28.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.051+0000 7fa81435a740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x558cab967cf0 con 0x558caba65370 2024-09-12T03:00:28.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.051+0000 7fa81435a740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x558cab9bc2b0 con 0x558caba5a5d0 2024-09-12T03:00:28.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.051+0000 7fa80bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558caba5bb40 0x558caba5a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:28.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.051+0000 7fa80bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558caba5bb40 0x558caba5a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48288/0 (socket says 172.21.15.5:48288) 2024-09-12T03:00:28.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.051+0000 7fa80bfff640 1 -- 172.21.15.5:0/2059830075 learned_addr learned my addr 172.21.15.5:0/2059830075 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:00:28.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.051+0000 7fa80b7fe640 1 --2- 172.21.15.5:0/2059830075 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558caba5a5d0 0x558caba5aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:28.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.051+0000 7fa80affd640 1 --2- 172.21.15.5:0/2059830075 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558caba65370 0x558caba67760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:28.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.051+0000 7fa80bfff640 1 -- 172.21.15.5:0/2059830075 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558caba5a5d0 msgr2=0x558caba5aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:00:28.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.051+0000 7fa80bfff640 1 --2- 172.21.15.5:0/2059830075 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558caba5a5d0 0x558caba5aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:28.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.051+0000 7fa80bfff640 1 -- 172.21.15.5:0/2059830075 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558caba65370 msgr2=0x558caba67760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:00:28.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.051+0000 7fa80bfff640 1 --2- 172.21.15.5:0/2059830075 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558caba65370 0x558caba67760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:28.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.052+0000 7fa80bfff640 1 -- 172.21.15.5:0/2059830075 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558cab94c3a0 con 0x558caba5bb40 2024-09-12T03:00:28.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.052+0000 7fa80affd640 1 --2- 172.21.15.5:0/2059830075 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558caba65370 0x558caba67760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:00:28.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.052+0000 7fa80bfff640 1 --2- 172.21.15.5:0/2059830075 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558caba5bb40 0x558caba5a090 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fa804009e40 tx=0x7fa80400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d0c297e5ec07bfc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:28.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.052+0000 7fa801ffb640 1 -- 172.21.15.5:0/2059830075 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa804004d20 con 0x558caba5bb40 2024-09-12T03:00:28.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.052+0000 7fa808ff9640 1 --2- 172.21.15.5:0/792321071 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fa7f80a2b80 0x7fa7f80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:28.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.052+0000 7fa801ffb640 1 -- 172.21.15.5:0/2059830075 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa804005500 con 0x558caba5bb40 2024-09-12T03:00:28.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.052+0000 7fa801ffb640 1 -- 172.21.15.5:0/2059830075 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa804011860 con 0x558caba5bb40 2024-09-12T03:00:28.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.052+0000 7fa80b7fe640 1 --2- 172.21.15.5:0/792321071 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fa7f80a2b80 0x7fa7f80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:28.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.052+0000 7fa808ff9640 1 -- 172.21.15.5:0/792321071 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fa7f40ef690 con 0x7fa80c102570 2024-09-12T03:00:28.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.053+0000 7fa80b7fe640 1 --2- 172.21.15.5:0/792321071 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fa7f80a2b80 0x7fa7f80a5040 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x558caba57ac0 tx=0x7fa7fc00a2d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:28.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.053+0000 7fa81435a740 1 -- 172.21.15.5:0/2059830075 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558caba5bb40 msgr2=0x558caba5a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:28.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.053+0000 7fa81435a740 1 --2- 172.21.15.5:0/2059830075 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558caba5bb40 0x558caba5a090 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fa804009e40 tx=0x7fa80400f730 comp rx=0 tx=0).stop 2024-09-12T03:00:28.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.053+0000 7fa81435a740 1 -- 172.21.15.5:0/2059830075 shutdown_connections 2024-09-12T03:00:28.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.053+0000 7fa81435a740 1 --2- 172.21.15.5:0/2059830075 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558caba65370 0x558caba67760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:28.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.053+0000 7fa81435a740 1 --2- 172.21.15.5:0/2059830075 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558caba5a5d0 0x558caba5aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:28.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.053+0000 7fa81435a740 1 --2- 172.21.15.5:0/2059830075 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558caba5bb40 0x558caba5a090 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:28.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.053+0000 7fa81435a740 1 -- 172.21.15.5:0/2059830075 >> 172.21.15.5:0/2059830075 conn(0x558caba23310 msgr2=0x558caba56c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:28.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.053+0000 7fa81435a740 1 -- 172.21.15.5:0/2059830075 shutdown_connections 2024-09-12T03:00:28.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.053+0000 7fa81435a740 1 -- 172.21.15.5:0/2059830075 wait complete. 2024-09-12T03:00:28.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.053+0000 7fa81435a740 1 -- start start 2024-09-12T03:00:28.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.054+0000 7fa81435a740 1 Processor -- start 2024-09-12T03:00:28.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.054+0000 7fa81435a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558caba5a5d0 0x558caba64d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:28.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.054+0000 7fa81435a740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558caba5bb40 0x558caba62040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:28.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.054+0000 7fa81435a740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558caba65370 0x558caba62650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:28.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.054+0000 7fa81435a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558caba68070 con 0x558caba5a5d0 2024-09-12T03:00:28.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.054+0000 7fa81435a740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x558cab97b060 con 0x558caba65370 2024-09-12T03:00:28.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.054+0000 7fa81435a740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x558caba681f0 con 0x558caba5bb40 2024-09-12T03:00:28.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.054+0000 7fa81435a740 10 client.0 _ll_register_callbacks cb 0x558cab6ce8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:00:28.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.054+0000 7fa80b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558caba5a5d0 0x558caba64d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:28.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.054+0000 7fa80b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558caba5a5d0 0x558caba64d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48302/0 (socket says 172.21.15.5:48302) 2024-09-12T03:00:28.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.054+0000 7fa80b7fe640 1 -- 172.21.15.5:0/4195673081 learned_addr learned my addr 172.21.15.5:0/4195673081 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:00:28.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.054+0000 7fa80bfff640 1 --2- 172.21.15.5:0/4195673081 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558caba65370 0x558caba62650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:28.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.054+0000 7fa80affd640 1 --2- 172.21.15.5:0/4195673081 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558caba5bb40 0x558caba62040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:28.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.054+0000 7fa80b7fe640 1 -- 172.21.15.5:0/4195673081 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558caba5bb40 msgr2=0x558caba62040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:00:28.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.054+0000 7fa80b7fe640 1 --2- 172.21.15.5:0/4195673081 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558caba5bb40 0x558caba62040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:28.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.054+0000 7fa80b7fe640 1 -- 172.21.15.5:0/4195673081 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558caba65370 msgr2=0x558caba62650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:00:28.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.054+0000 7fa80b7fe640 1 --2- 172.21.15.5:0/4195673081 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558caba65370 0x558caba62650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:28.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.054+0000 7fa80b7fe640 1 -- 172.21.15.5:0/4195673081 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558cab7313c0 con 0x558caba5a5d0 2024-09-12T03:00:28.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.054+0000 7fa80affd640 1 --2- 172.21.15.5:0/4195673081 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558caba5bb40 0x558caba62040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:00:28.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.055+0000 7fa80b7fe640 1 --2- 172.21.15.5:0/4195673081 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558caba5a5d0 0x558caba64d90 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fa7fc00a7a0 tx=0x7fa7fc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29cdb2ead8046576 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:28.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.055+0000 7fa7eaffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T03:00:28.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.055+0000 7fa7eaffd640 1 -- 172.21.15.5:0/4195673081 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7fc00f8c0 con 0x558caba5a5d0 2024-09-12T03:00:28.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.055+0000 7fa7eaffd640 1 -- 172.21.15.5:0/4195673081 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fa7fc00fa60 con 0x558caba5a5d0 2024-09-12T03:00:28.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.055+0000 7fa81435a740 1 -- 172.21.15.5:0/4195673081 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa804005500 con 0x558caba5a5d0 2024-09-12T03:00:28.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.055+0000 7fa7eaffd640 1 -- 172.21.15.5:0/4195673081 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa7fc00fc60 con 0x558caba5a5d0 2024-09-12T03:00:28.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.055+0000 7fa81435a740 10 client.15891 fetch_fsmap learned FSMap version 28 2024-09-12T03:00:28.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.055+0000 7fa81435a740 1 -- 172.21.15.5:0/4195673081 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558caba73b10 con 0x558caba5a5d0 2024-09-12T03:00:28.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.055+0000 7fa7eaffd640 1 -- 172.21.15.5:0/4195673081 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa804005500 con 0x558caba5a5d0 2024-09-12T03:00:28.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.056+0000 7fa7eaffd640 1 -- 172.21.15.5:0/4195673081 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8374+0+0 (secure 0 0 0) 0x7fa7fc01d4a0 con 0x558caba5a5d0 2024-09-12T03:00:28.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.056+0000 7fa81435a740 10 client.15891 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T03:00:28.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.056+0000 7fa81435a740 10 client.15891 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.b 2024-09-12T03:00:28.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.056+0000 7fa81435a740 20 client.15891 populate_metadata read hostname 'smithi005' 2024-09-12T03:00:28.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.056+0000 7fa81435a740 1 --2- 172.21.15.5:0/4195673081 >> [v2:172.21.15.146:6838/3036767228,v1:172.21.15.146:6839/3036767228] conn(0x558caba74f20 0x558caba77350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:28.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.056+0000 7fa81435a740 4 client.15891 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:00:28.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.056+0000 7fa81435a740 1 -- 172.21.15.5:0/4195673081 --> [v2:172.21.15.146:6838/3036767228,v1:172.21.15.146:6839/3036767228] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558cab9cd860 con 0x558caba74f20 2024-09-12T03:00:28.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.056+0000 7fa80affd640 1 --2- 172.21.15.5:0/4195673081 >> [v2:172.21.15.146:6838/3036767228,v1:172.21.15.146:6839/3036767228] conn(0x558caba74f20 0x558caba77350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:28.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.056+0000 7fa80affd640 1 --2- 172.21.15.5:0/4195673081 >> [v2:172.21.15.146:6838/3036767228,v1:172.21.15.146:6839/3036767228] conn(0x558caba74f20 0x558caba77350 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=92fa3da089e3617 server_cookie=4bc555b061e56ad7 in_seq=0 out_seq=0 2024-09-12T03:00:28.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.056+0000 7fa7eaffd640 10 client.15891 ms_handle_connect on v2:172.21.15.146:6838/3036767228 2024-09-12T03:00:28.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.057+0000 7fa7eaffd640 1 -- 172.21.15.5:0/4195673081 <== mds.3 v2:172.21.15.146:6838/3036767228 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558cab9cd860 con 0x558caba74f20 2024-09-12T03:00:28.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.057+0000 7fa7eaffd640 10 client.15891 handle_command_reply: tid=0 2024-09-12T03:00:28.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.057+0000 7fa81435a740 1 client.15891 shutdown 2024-09-12T03:00:28.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.057+0000 7fa81435a740 1 -- 172.21.15.5:0/4195673081 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558caba5a5d0 msgr2=0x558caba64d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:28.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.057+0000 7fa81435a740 1 --2- 172.21.15.5:0/4195673081 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558caba5a5d0 0x558caba64d90 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fa7fc00a7a0 tx=0x7fa7fc0046c0 comp rx=0 tx=0).stop 2024-09-12T03:00:28.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.058+0000 7fa81435a740 1 -- 172.21.15.5:0/4195673081 shutdown_connections 2024-09-12T03:00:28.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.058+0000 7fa81435a740 1 --2- 172.21.15.5:0/4195673081 >> [v2:172.21.15.146:6838/3036767228,v1:172.21.15.146:6839/3036767228] conn(0x558caba74f20 0x558caba77350 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:28.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.058+0000 7fa7eaffd640 0 client.15891 ms_handle_reset on v2:172.21.15.146:6838/3036767228 2024-09-12T03:00:28.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.058+0000 7fa81435a740 1 --2- 172.21.15.5:0/4195673081 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558caba65370 0x558caba62650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:28.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.058+0000 7fa81435a740 1 --2- 172.21.15.5:0/4195673081 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558caba5bb40 0x558caba62040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:28.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.058+0000 7fa81435a740 1 --2- 172.21.15.5:0/4195673081 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558caba5a5d0 0x558caba64d90 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:28.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.058+0000 7fa81435a740 1 -- 172.21.15.5:0/4195673081 >> 172.21.15.5:0/4195673081 conn(0x558caba23310 msgr2=0x558caba5c960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:28.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.058+0000 7fa81435a740 1 -- 172.21.15.5:0/4195673081 shutdown_connections 2024-09-12T03:00:28.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.059+0000 7fa81435a740 1 -- 172.21.15.5:0/4195673081 wait complete. 2024-09-12T03:00:28.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.059+0000 7fa81435a740 20 client.15891 trim_cache size 0 max 16384 2024-09-12T03:00:28.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.072+0000 7fa81435a740 1 Processor -- start 2024-09-12T03:00:28.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.072+0000 7fa81435a740 1 -- start start 2024-09-12T03:00:28.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.072+0000 7fa81435a740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558caba5a5d0 0x558caba55bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:28.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.072+0000 7fa81435a740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558caba5bb40 0x558caba56130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:28.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.072+0000 7fa81435a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558caba65370 0x558caba5d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:28.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.072+0000 7fa81435a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa80c1ad5d0 con 0x558caba65370 2024-09-12T03:00:28.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.072+0000 7fa81435a740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fa80c1af1a0 con 0x558caba5a5d0 2024-09-12T03:00:28.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.072+0000 7fa81435a740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa80c1adca0 con 0x558caba5bb40 2024-09-12T03:00:28.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.072+0000 7fa80bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558caba65370 0x558caba5d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:28.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.072+0000 7fa80bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558caba65370 0x558caba5d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48304/0 (socket says 172.21.15.5:48304) 2024-09-12T03:00:28.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.072+0000 7fa80bfff640 1 -- 172.21.15.5:0/1263869060 learned_addr learned my addr 172.21.15.5:0/1263869060 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:00:28.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.072+0000 7fa80affd640 1 --2- 172.21.15.5:0/1263869060 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558caba5bb40 0x558caba56130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:28.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.072+0000 7fa80b7fe640 1 --2- 172.21.15.5:0/1263869060 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558caba5a5d0 0x558caba55bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:28.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.073+0000 7fa80affd640 1 -- 172.21.15.5:0/1263869060 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558caba5a5d0 msgr2=0x558caba55bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:00:28.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.073+0000 7fa80affd640 1 --2- 172.21.15.5:0/1263869060 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558caba5a5d0 0x558caba55bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:28.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.073+0000 7fa80affd640 1 -- 172.21.15.5:0/1263869060 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558caba65370 msgr2=0x558caba5d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:00:28.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.073+0000 7fa80affd640 1 --2- 172.21.15.5:0/1263869060 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558caba65370 0x558caba5d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:28.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.073+0000 7fa80affd640 1 -- 172.21.15.5:0/1263869060 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558caba629c0 con 0x558caba5bb40 2024-09-12T03:00:28.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.073+0000 7fa80b7fe640 1 --2- 172.21.15.5:0/1263869060 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558caba5a5d0 0x558caba55bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:00:28.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.073+0000 7fa80affd640 1 --2- 172.21.15.5:0/1263869060 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558caba5bb40 0x558caba56130 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa7f40ed1f0 tx=0x7fa7f402a480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bde1fd758809346d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:28.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.073+0000 7fa8017fa640 1 -- 172.21.15.5:0/1263869060 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f4049040 con 0x558caba5bb40 2024-09-12T03:00:28.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.073+0000 7fa8017fa640 1 -- 172.21.15.5:0/1263869060 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7f40c02b0 con 0x558caba5bb40 2024-09-12T03:00:28.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.073+0000 7fa8017fa640 1 -- 172.21.15.5:0/1263869060 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f403b4f0 con 0x558caba5bb40 2024-09-12T03:00:28.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.074+0000 7fa81435a740 1 -- 172.21.15.5:0/1263869060 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558caba5bb40 msgr2=0x558caba56130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:28.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.074+0000 7fa81435a740 1 --2- 172.21.15.5:0/1263869060 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558caba5bb40 0x558caba56130 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa7f40ed1f0 tx=0x7fa7f402a480 comp rx=0 tx=0).stop 2024-09-12T03:00:28.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.074+0000 7fa81435a740 1 -- 172.21.15.5:0/1263869060 shutdown_connections 2024-09-12T03:00:28.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.074+0000 7fa81435a740 1 --2- 172.21.15.5:0/1263869060 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558caba65370 0x558caba5d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:28.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.074+0000 7fa81435a740 1 --2- 172.21.15.5:0/1263869060 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558caba5bb40 0x558caba56130 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:28.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.074+0000 7fa81435a740 1 --2- 172.21.15.5:0/1263869060 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558caba5a5d0 0x558caba55bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:28.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.074+0000 7fa81435a740 1 -- 172.21.15.5:0/1263869060 >> 172.21.15.5:0/1263869060 conn(0x558caba23310 msgr2=0x558caba554e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:28.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.074+0000 7fa81435a740 1 -- 172.21.15.5:0/1263869060 shutdown_connections 2024-09-12T03:00:28.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.074+0000 7fa81435a740 1 -- 172.21.15.5:0/1263869060 wait complete. 2024-09-12T03:00:28.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.074+0000 7fa81435a740 1 -- start start 2024-09-12T03:00:28.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.075+0000 7fa81435a740 1 Processor -- start 2024-09-12T03:00:28.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.075+0000 7fa81435a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558caba5a5d0 0x558caba6d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:28.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.075+0000 7fa81435a740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558caba5bb40 0x558caba5eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:28.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.075+0000 7fa81435a740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558caba65370 0x558caba5f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:28.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.075+0000 7fa81435a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa80c1aea40 con 0x558caba5a5d0 2024-09-12T03:00:28.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.075+0000 7fa81435a740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fa80c1ae370 con 0x558caba65370 2024-09-12T03:00:28.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.075+0000 7fa81435a740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa80c1ab5b0 con 0x558caba5bb40 2024-09-12T03:00:28.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.075+0000 7fa81435a740 10 client.0 _ll_register_callbacks cb 0x7fa7fc00a990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:00:28.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.075+0000 7fa80b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558caba5a5d0 0x558caba6d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:28.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.075+0000 7fa80b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558caba5a5d0 0x558caba6d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48320/0 (socket says 172.21.15.5:48320) 2024-09-12T03:00:28.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.075+0000 7fa80b7fe640 1 -- 172.21.15.5:0/3731425853 learned_addr learned my addr 172.21.15.5:0/3731425853 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:00:28.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.075+0000 7fa80affd640 1 --2- 172.21.15.5:0/3731425853 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558caba5bb40 0x558caba5eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:28.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.075+0000 7fa80bfff640 1 --2- 172.21.15.5:0/3731425853 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558caba65370 0x558caba5f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:28.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.075+0000 7fa80b7fe640 1 -- 172.21.15.5:0/3731425853 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558caba5bb40 msgr2=0x558caba5eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:00:28.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.075+0000 7fa80b7fe640 1 --2- 172.21.15.5:0/3731425853 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558caba5bb40 0x558caba5eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:28.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.075+0000 7fa80b7fe640 1 -- 172.21.15.5:0/3731425853 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558caba65370 msgr2=0x558caba5f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:00:28.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.075+0000 7fa80b7fe640 1 --2- 172.21.15.5:0/3731425853 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558caba65370 0x558caba5f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:28.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.075+0000 7fa80b7fe640 1 -- 172.21.15.5:0/3731425853 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558caba5f6b0 con 0x558caba5a5d0 2024-09-12T03:00:28.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.075+0000 7fa80bfff640 1 --2- 172.21.15.5:0/3731425853 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558caba65370 0x558caba5f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:00:28.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.076+0000 7fa80b7fe640 1 --2- 172.21.15.5:0/3731425853 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558caba5a5d0 0x558caba6d950 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fa7fc01d870 tx=0x7fa7fc00ae80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfbc8f3109cadcbd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:28.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.076+0000 7fa7eaffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T03:00:28.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.076+0000 7fa7eaffd640 1 -- 172.21.15.5:0/3731425853 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7fc01dce0 con 0x558caba5a5d0 2024-09-12T03:00:28.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.076+0000 7fa7eaffd640 1 -- 172.21.15.5:0/3731425853 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fa7fc01d410 con 0x558caba5a5d0 2024-09-12T03:00:28.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.076+0000 7fa81435a740 1 -- 172.21.15.5:0/3731425853 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7f40c02b0 con 0x558caba5a5d0 2024-09-12T03:00:28.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.076+0000 7fa7eaffd640 1 -- 172.21.15.5:0/3731425853 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa7fc01d610 con 0x558caba5a5d0 2024-09-12T03:00:28.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.076+0000 7fa81435a740 10 client.15897 fetch_fsmap learned FSMap version 28 2024-09-12T03:00:28.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.076+0000 7fa81435a740 1 -- 172.21.15.5:0/3731425853 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558cabadfaa0 con 0x558caba5a5d0 2024-09-12T03:00:28.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.076+0000 7fa7eaffd640 1 -- 172.21.15.5:0/3731425853 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7f40c02b0 con 0x558caba5a5d0 2024-09-12T03:00:28.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.076+0000 7fa7eaffd640 1 -- 172.21.15.5:0/3731425853 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8374+0+0 (secure 0 0 0) 0x7fa7fc038020 con 0x558caba5a5d0 2024-09-12T03:00:28.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.076+0000 7fa81435a740 10 client.15897 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T03:00:28.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.076+0000 7fa81435a740 10 client.15897 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.b 2024-09-12T03:00:28.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.077+0000 7fa81435a740 20 client.15897 populate_metadata read hostname 'smithi005' 2024-09-12T03:00:28.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.077+0000 7fa81435a740 1 --2- 172.21.15.5:0/3731425853 >> [v2:172.21.15.146:6838/3036767228,v1:172.21.15.146:6839/3036767228] conn(0x558cabae01f0 0x558cabae2610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:28.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.077+0000 7fa81435a740 4 client.15897 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:00:28.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.077+0000 7fa81435a740 1 -- 172.21.15.5:0/3731425853 --> [v2:172.21.15.146:6838/3036767228,v1:172.21.15.146:6839/3036767228] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558cab9c7d70 con 0x558cabae01f0 2024-09-12T03:00:28.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.077+0000 7fa80affd640 1 --2- 172.21.15.5:0/3731425853 >> [v2:172.21.15.146:6838/3036767228,v1:172.21.15.146:6839/3036767228] conn(0x558cabae01f0 0x558cabae2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:28.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.077+0000 7fa80affd640 1 --2- 172.21.15.5:0/3731425853 >> [v2:172.21.15.146:6838/3036767228,v1:172.21.15.146:6839/3036767228] conn(0x558cabae01f0 0x558cabae2610 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=eed4187d11dad23f server_cookie=bbb3cd0143e3d4a9 in_seq=0 out_seq=0 2024-09-12T03:00:28.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.077+0000 7fa7eaffd640 10 client.15897 ms_handle_connect on v2:172.21.15.146:6838/3036767228 2024-09-12T03:00:28.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.077+0000 7fa7eaffd640 1 -- 172.21.15.5:0/3731425853 <== mds.3 v2:172.21.15.146:6838/3036767228 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558cab9c7d70 con 0x558cabae01f0 2024-09-12T03:00:28.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.077+0000 7fa7eaffd640 10 client.15897 handle_command_reply: tid=0 2024-09-12T03:00:28.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.077+0000 7fa81435a740 1 client.15897 shutdown 2024-09-12T03:00:28.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.078+0000 7fa81435a740 1 -- 172.21.15.5:0/3731425853 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558caba5a5d0 msgr2=0x558caba6d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:28.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.078+0000 7fa81435a740 1 --2- 172.21.15.5:0/3731425853 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558caba5a5d0 0x558caba6d950 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fa7fc01d870 tx=0x7fa7fc00ae80 comp rx=0 tx=0).stop 2024-09-12T03:00:28.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.078+0000 7fa81435a740 1 -- 172.21.15.5:0/3731425853 shutdown_connections 2024-09-12T03:00:28.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.078+0000 7fa81435a740 1 --2- 172.21.15.5:0/3731425853 >> [v2:172.21.15.146:6838/3036767228,v1:172.21.15.146:6839/3036767228] conn(0x558cabae01f0 0x558cabae2610 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:28.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.078+0000 7fa81435a740 1 --2- 172.21.15.5:0/3731425853 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558caba65370 0x558caba5f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:28.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.078+0000 7fa81435a740 1 --2- 172.21.15.5:0/3731425853 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558caba5bb40 0x558caba5eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:28.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.078+0000 7fa81435a740 1 --2- 172.21.15.5:0/3731425853 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558caba5a5d0 0x558caba6d950 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:28.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.078+0000 7fa81435a740 1 -- 172.21.15.5:0/3731425853 >> 172.21.15.5:0/3731425853 conn(0x558caba23310 msgr2=0x558caba55d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:28.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.078+0000 7fa7eaffd640 0 client.15897 ms_handle_reset on v2:172.21.15.146:6838/3036767228 2024-09-12T03:00:28.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.078+0000 7fa81435a740 1 -- 172.21.15.5:0/3731425853 shutdown_connections 2024-09-12T03:00:28.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.078+0000 7fa81435a740 1 -- 172.21.15.5:0/3731425853 wait complete. 2024-09-12T03:00:28.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.078+0000 7fa81435a740 20 client.15897 trim_cache size 0 max 16384 2024-09-12T03:00:28.084 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-12T03:00:28.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.079+0000 7fa811f81640 1 -- 172.21.15.5:0/792321071 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fa7f80a2b80 msgr2=0x7fa7f80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:28.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.079+0000 7fa811f81640 1 --2- 172.21.15.5:0/792321071 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fa7f80a2b80 0x7fa7f80a5040 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x558caba57ac0 tx=0x7fa7fc00a2d0 comp rx=0 tx=0).stop 2024-09-12T03:00:28.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.079+0000 7fa811f81640 1 -- 172.21.15.5:0/792321071 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa80c102570 msgr2=0x7fa80c198a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:28.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.079+0000 7fa811f81640 1 --2- 172.21.15.5:0/792321071 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa80c102570 0x7fa80c198a60 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fa7f4006010 tx=0x7fa7f4003c10 comp rx=0 tx=0).stop 2024-09-12T03:00:28.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.079+0000 7fa811f81640 1 -- 172.21.15.5:0/792321071 shutdown_connections 2024-09-12T03:00:28.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.079+0000 7fa811f81640 1 --2- 172.21.15.5:0/792321071 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fa7f80a2b80 0x7fa7f80a5040 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:28.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.079+0000 7fa811f81640 1 --2- 172.21.15.5:0/792321071 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa80c102570 0x7fa80c198a60 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:28.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.079+0000 7fa811f81640 1 --2- 172.21.15.5:0/792321071 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa80c102ef0 0x7fa80c19cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:28.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.079+0000 7fa811f81640 1 --2- 172.21.15.5:0/792321071 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa80c100e20 0x7fa80c198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:28.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.079+0000 7fa811f81640 1 -- 172.21.15.5:0/792321071 >> 172.21.15.5:0/792321071 conn(0x7fa80c0fc7d0 msgr2=0x7fa80c06a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:28.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.079+0000 7fa811f81640 1 -- 172.21.15.5:0/792321071 shutdown_connections 2024-09-12T03:00:28.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:28.079+0000 7fa811f81640 1 -- 172.21.15.5:0/792321071 wait complete. 2024-09-12T03:00:28.102 INFO:tasks.workunit.client.0.smithi005.stdout: [DEP] 2024-09-12T03:00:28.102 INFO:tasks.workunit.client.0.smithi005.stdout: [DEP] 2024-09-12T03:00:28.103 INFO:tasks.workunit.client.0.smithi005.stdout: [DEP] 2024-09-12T03:00:28.103 INFO:tasks.workunit.client.0.smithi005.stdout: [DEP] 2024-09-12T03:00:28.355 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] dirstress 2024-09-12T03:00:28.355 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] fill 2024-09-12T03:00:28.357 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] fill2 2024-09-12T03:00:28.359 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] getpagesize 2024-09-12T03:00:28.508 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] holes 2024-09-12T03:00:28.518 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] lstat64 2024-09-12T03:00:28.602 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] nametest 2024-09-12T03:00:28.607 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] permname 2024-09-12T03:00:28.724 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] randholes 2024-09-12T03:00:28.724 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] runas 2024-09-12T03:00:28.810 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] truncfile 2024-09-12T03:00:28.871 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] usemem 2024-09-12T03:00:28.918 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] mmapcat 2024-09-12T03:00:29.005 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T03:00:29.010 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] append_reader 2024-09-12T03:00:29.023 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] append_writer 2024-09-12T03:00:29.028 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] dirperf 2024-09-12T03:00:29.079 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] metaperf 2024-09-12T03:00:29.189 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] devzero 2024-09-12T03:00:29.193 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] feature 2024-09-12T03:00:29.289 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] alloc 2024-09-12T03:00:29.341 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:00:29.412 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] fault 2024-09-12T03:00:29.446 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] fstest 2024-09-12T03:00:29.470 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] t_access_root 2024-09-12T03:00:29.509 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] godown 2024-09-12T03:00:29.577 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] resvtest 2024-09-12T03:00:29.640 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] writemod 2024-09-12T03:00:29.665 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:00:29 smithi148 ceph-mon[35090]: pgmap v402: 97 pgs: 97 active+clean; 68 MiB data, 659 MiB used, 1.0 TiB / 1.0 TiB avail; 836 KiB/s rd, 322 KiB/s wr, 253 op/s 2024-09-12T03:00:29.688 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:00:29 smithi005 ceph-mon[31197]: pgmap v402: 97 pgs: 97 active+clean; 68 MiB data, 659 MiB used, 1.0 TiB / 1.0 TiB avail; 836 KiB/s rd, 322 KiB/s wr, 253 op/s 2024-09-12T03:00:29.697 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] writev_on_pagefault 2024-09-12T03:00:29.705 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] makeextents 2024-09-12T03:00:29.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:00:29 smithi146 ceph-mon[35011]: pgmap v402: 97 pgs: 97 active+clean; 68 MiB data, 659 MiB used, 1.0 TiB / 1.0 TiB avail; 836 KiB/s rd, 322 KiB/s wr, 253 op/s 2024-09-12T03:00:29.784 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] itrash 2024-09-12T03:00:29.798 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] rename 2024-09-12T03:00:29.870 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] multi_open_unlink 2024-09-12T03:00:29.900 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] unwritten_sync 2024-09-12T03:00:29.949 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] genhashnames 2024-09-12T03:00:29.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.955+0000 7f9e5949a640 1 -- 172.21.15.5:0/1555599175 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9e54100aa0 msgr2=0x7f9e5410ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:29.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.955+0000 7f9e5949a640 1 --2- 172.21.15.5:0/1555599175 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9e54100aa0 0x7f9e5410ea20 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c009ca0 tx=0x7f9e4c02a8c0 comp rx=0 tx=0).stop 2024-09-12T03:00:29.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.955+0000 7f9e5949a640 1 -- 172.21.15.5:0/1555599175 shutdown_connections 2024-09-12T03:00:29.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.955+0000 7f9e5949a640 1 --2- 172.21.15.5:0/1555599175 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9e54100aa0 0x7f9e5410ea20 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:29.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.955+0000 7f9e5949a640 1 --2- 172.21.15.5:0/1555599175 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9e54100100 0x7f9e54100560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:29.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.955+0000 7f9e5949a640 1 --2- 172.21.15.5:0/1555599175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e54106b80 0x7f9e54106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:29.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.955+0000 7f9e5949a640 1 -- 172.21.15.5:0/1555599175 >> 172.21.15.5:0/1555599175 conn(0x7f9e54077230 msgr2=0x7f9e540fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:29.959 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.955+0000 7f9e5949a640 1 -- 172.21.15.5:0/1555599175 shutdown_connections 2024-09-12T03:00:29.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.955+0000 7f9e5949a640 1 -- 172.21.15.5:0/1555599175 wait complete. 2024-09-12T03:00:29.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.956+0000 7f9e5949a640 1 Processor -- start 2024-09-12T03:00:29.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.956+0000 7f9e5949a640 1 -- start start 2024-09-12T03:00:29.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.956+0000 7f9e5949a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9e54100100 0x7f9e54073930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:29.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.956+0000 7f9e5949a640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9e54100aa0 0x7f9e54073e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:29.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.957+0000 7f9e5949a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e54106b80 0x7f9e5406cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:29.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.957+0000 7f9e5949a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9e541120a0 con 0x7f9e54106b80 2024-09-12T03:00:29.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.957+0000 7f9e5949a640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f9e54111f20 con 0x7f9e54100aa0 2024-09-12T03:00:29.960 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.957+0000 7f9e5949a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f9e54112220 con 0x7f9e54100100 2024-09-12T03:00:29.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.957+0000 7f9e52ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9e54100100 0x7f9e54073930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:29.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.957+0000 7f9e52ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9e54100100 0x7f9e54073930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.5:33178/0 (socket says 172.21.15.5:33178) 2024-09-12T03:00:29.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.957+0000 7f9e52ffd640 1 -- 172.21.15.5:0/2788659881 learned_addr learned my addr 172.21.15.5:0/2788659881 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:00:29.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.957+0000 7f9e537fe640 1 --2- 172.21.15.5:0/2788659881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e54106b80 0x7f9e5406cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:29.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.957+0000 7f9e52ffd640 1 -- 172.21.15.5:0/2788659881 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9e54100aa0 msgr2=0x7f9e54073e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:29.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.957+0000 7f9e527fc640 1 --2- 172.21.15.5:0/2788659881 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9e54100aa0 0x7f9e54073e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:29.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.958+0000 7f9e52ffd640 1 --2- 172.21.15.5:0/2788659881 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9e54100aa0 0x7f9e54073e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:29.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.958+0000 7f9e52ffd640 1 -- 172.21.15.5:0/2788659881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e54106b80 msgr2=0x7f9e5406cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:29.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.958+0000 7f9e52ffd640 1 --2- 172.21.15.5:0/2788659881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e54106b80 0x7f9e5406cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:29.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.958+0000 7f9e52ffd640 1 -- 172.21.15.5:0/2788659881 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e5406d460 con 0x7f9e54100100 2024-09-12T03:00:29.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.958+0000 7f9e527fc640 1 --2- 172.21.15.5:0/2788659881 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9e54100aa0 0x7f9e54073e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:00:29.961 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.958+0000 7f9e537fe640 1 --2- 172.21.15.5:0/2788659881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e54106b80 0x7f9e5406cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:00:29.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.958+0000 7f9e52ffd640 1 --2- 172.21.15.5:0/2788659881 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9e54100100 0x7f9e54073930 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9e4400d950 tx=0x7f9e4400de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:29.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.958+0000 7f9e3bfff640 1 -- 172.21.15.5:0/2788659881 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e44017030 con 0x7f9e54100100 2024-09-12T03:00:29.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.958+0000 7f9e3bfff640 1 -- 172.21.15.5:0/2788659881 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e44004020 con 0x7f9e54100100 2024-09-12T03:00:29.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.958+0000 7f9e5949a640 1 -- 172.21.15.5:0/2788659881 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e5406d750 con 0x7f9e54100100 2024-09-12T03:00:29.962 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.958+0000 7f9e3bfff640 1 -- 172.21.15.5:0/2788659881 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e440047a0 con 0x7f9e54100100 2024-09-12T03:00:29.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.958+0000 7f9e5949a640 1 -- 172.21.15.5:0/2788659881 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e54071740 con 0x7f9e54100100 2024-09-12T03:00:29.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.962+0000 7f9e5b873740 1 Processor -- start 2024-09-12T03:00:29.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.962+0000 7f9e3bfff640 1 -- 172.21.15.5:0/2788659881 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f9e440041d0 con 0x7f9e54100100 2024-09-12T03:00:29.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.962+0000 7f9e5b873740 1 -- start start 2024-09-12T03:00:29.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.962+0000 7f9e5b873740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55937afd6b40 0x55937afd5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:29.963 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.962+0000 7f9e5b873740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55937afd55d0 0x55937afd5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:29.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.962+0000 7f9e5b873740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55937afe0370 0x55937afe2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:29.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.962+0000 7f9e5b873740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55937af3b2f0 con 0x55937afd6b40 2024-09-12T03:00:29.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.962+0000 7f9e5b873740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55937aee2cf0 con 0x55937afd55d0 2024-09-12T03:00:29.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.962+0000 7f9e5b873740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55937af372b0 con 0x55937afe0370 2024-09-12T03:00:29.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.963+0000 7f9e52ffd640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55937afd55d0 0x55937afd5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:29.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.963+0000 7f9e537fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55937afd6b40 0x55937afd5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:29.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.963+0000 7f9e52ffd640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55937afd55d0 0x55937afd5a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.5:54122/0 (socket says 172.21.15.5:54122) 2024-09-12T03:00:29.964 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.963+0000 7f9e52ffd640 1 -- 172.21.15.5:0/1450767725 learned_addr learned my addr 172.21.15.5:0/1450767725 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:00:29.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.963+0000 7f9e527fc640 1 --2- 172.21.15.5:0/1450767725 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55937afe0370 0x55937afe2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:29.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.963+0000 7f9e52ffd640 1 -- 172.21.15.5:0/1450767725 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55937afe0370 msgr2=0x55937afe2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:00:29.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.963+0000 7f9e52ffd640 1 --2- 172.21.15.5:0/1450767725 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55937afe0370 0x55937afe2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:29.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.963+0000 7f9e52ffd640 1 -- 172.21.15.5:0/1450767725 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55937afd6b40 msgr2=0x55937afd5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:00:29.965 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.963+0000 7f9e52ffd640 1 --2- 172.21.15.5:0/1450767725 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55937afd6b40 0x55937afd5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:29.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.963+0000 7f9e3bfff640 1 --2- 172.21.15.5:0/2788659881 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f9e3c0a2c00 0x7f9e3c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:29.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.964+0000 7f9e527fc640 1 --2- 172.21.15.5:0/2788659881 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f9e3c0a2c00 0x7f9e3c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:29.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.963+0000 7f9e52ffd640 1 -- 172.21.15.5:0/1450767725 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55937aec73a0 con 0x55937afd55d0 2024-09-12T03:00:29.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.964+0000 7f9e537fe640 1 --2- 172.21.15.5:0/1450767725 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55937afd6b40 0x55937afd5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:00:29.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.964+0000 7f9e3bfff640 1 -- 172.21.15.5:0/2788659881 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f9e440ce9a0 con 0x7f9e54100100 2024-09-12T03:00:29.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.964+0000 7f9e527fc640 1 --2- 172.21.15.5:0/2788659881 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f9e3c0a2c00 0x7f9e3c0a50c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x55937afd6070 tx=0x7f9e48005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:29.966 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.965+0000 7f9e52ffd640 1 --2- 172.21.15.5:0/1450767725 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55937afd55d0 0x55937afd5a10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9e44074950 tx=0x7f9e44000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33e94aa662a3802d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:29.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.965+0000 7f9e397fa640 1 -- 172.21.15.5:0/1450767725 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e4409af80 con 0x55937afd55d0 2024-09-12T03:00:29.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.966+0000 7f9e397fa640 1 -- 172.21.15.5:0/1450767725 <== mon.2 v2:172.21.15.146:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e440741b0 con 0x55937afd55d0 2024-09-12T03:00:29.967 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.966+0000 7f9e397fa640 1 -- 172.21.15.5:0/1450767725 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e44074780 con 0x55937afd55d0 2024-09-12T03:00:29.967 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] t_holes 2024-09-12T03:00:29.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.966+0000 7f9e5b873740 1 -- 172.21.15.5:0/1450767725 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55937afd55d0 msgr2=0x55937afd5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:29.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.966+0000 7f9e5b873740 1 --2- 172.21.15.5:0/1450767725 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55937afd55d0 0x55937afd5a10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9e44074950 tx=0x7f9e44000bc0 comp rx=0 tx=0).stop 2024-09-12T03:00:29.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.967+0000 7f9e5b873740 1 -- 172.21.15.5:0/1450767725 shutdown_connections 2024-09-12T03:00:29.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.967+0000 7f9e5b873740 1 --2- 172.21.15.5:0/1450767725 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55937afe0370 0x55937afe2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:29.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.967+0000 7f9e5b873740 1 --2- 172.21.15.5:0/1450767725 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55937afd55d0 0x55937afd5a10 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:29.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.967+0000 7f9e5b873740 1 --2- 172.21.15.5:0/1450767725 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55937afd6b40 0x55937afd5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:29.970 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.967+0000 7f9e5b873740 1 -- 172.21.15.5:0/1450767725 >> 172.21.15.5:0/1450767725 conn(0x55937af9e310 msgr2=0x55937afd1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:29.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.967+0000 7f9e5b873740 1 -- 172.21.15.5:0/1450767725 shutdown_connections 2024-09-12T03:00:29.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.967+0000 7f9e5b873740 1 -- 172.21.15.5:0/1450767725 wait complete. 2024-09-12T03:00:29.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.967+0000 7f9e5b873740 1 -- start start 2024-09-12T03:00:29.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.967+0000 7f9e5b873740 1 Processor -- start 2024-09-12T03:00:29.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.968+0000 7f9e5b873740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55937afd55d0 0x55937afdfd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:29.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.968+0000 7f9e5b873740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55937afd6b40 0x55937afdcfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:29.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.968+0000 7f9e5b873740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55937afe0370 0x55937afdd570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:29.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.968+0000 7f9e5b873740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55937afe2fb0 con 0x55937afd55d0 2024-09-12T03:00:29.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.968+0000 7f9e5b873740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55937aef6060 con 0x55937afe0370 2024-09-12T03:00:29.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.968+0000 7f9e5b873740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55937afe3130 con 0x55937afd6b40 2024-09-12T03:00:29.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.968+0000 7f9e5b873740 10 client.0 _ll_register_callbacks cb 0x55937ac498c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:00:29.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.968+0000 7f9e52ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55937afd55d0 0x55937afdfd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:29.971 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.968+0000 7f9e52ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55937afd55d0 0x55937afdfd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48340/0 (socket says 172.21.15.5:48340) 2024-09-12T03:00:29.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.968+0000 7f9e52ffd640 1 -- 172.21.15.5:0/3651379023 learned_addr learned my addr 172.21.15.5:0/3651379023 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:00:29.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.968+0000 7f9e527fc640 1 --2- 172.21.15.5:0/3651379023 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55937afd6b40 0x55937afdcfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:29.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.968+0000 7f9e537fe640 1 --2- 172.21.15.5:0/3651379023 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55937afe0370 0x55937afdd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:29.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.969+0000 7f9e52ffd640 1 -- 172.21.15.5:0/3651379023 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55937afd6b40 msgr2=0x55937afdcfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:00:29.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.969+0000 7f9e52ffd640 1 --2- 172.21.15.5:0/3651379023 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55937afd6b40 0x55937afdcfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:29.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.969+0000 7f9e52ffd640 1 -- 172.21.15.5:0/3651379023 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55937afe0370 msgr2=0x55937afdd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:00:29.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.969+0000 7f9e52ffd640 1 --2- 172.21.15.5:0/3651379023 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55937afe0370 0x55937afdd570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:29.972 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.969+0000 7f9e52ffd640 1 -- 172.21.15.5:0/3651379023 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55937acac3c0 con 0x55937afd55d0 2024-09-12T03:00:29.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.969+0000 7f9e52ffd640 1 --2- 172.21.15.5:0/3651379023 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55937afd55d0 0x55937afdfd20 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f9e440a38f0 tx=0x7f9e44097550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=151fc5d9dd6e1656 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:29.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.969+0000 7f9e36ff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T03:00:29.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.969+0000 7f9e36ff5640 1 -- 172.21.15.5:0/3651379023 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e4409ab10 con 0x55937afd55d0 2024-09-12T03:00:29.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.969+0000 7f9e36ff5640 1 -- 172.21.15.5:0/3651379023 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f9e44066220 con 0x55937afd55d0 2024-09-12T03:00:29.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.970+0000 7f9e5b873740 1 -- 172.21.15.5:0/3651379023 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55937af48860 con 0x55937afd55d0 2024-09-12T03:00:29.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.970+0000 7f9e36ff5640 1 -- 172.21.15.5:0/3651379023 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9e4409acf0 con 0x55937afd55d0 2024-09-12T03:00:29.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.970+0000 7f9e5b873740 10 client.15909 fetch_fsmap learned FSMap version 28 2024-09-12T03:00:29.973 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.970+0000 7f9e5b873740 1 -- 172.21.15.5:0/3651379023 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55937afe2fb0 con 0x55937afd55d0 2024-09-12T03:00:29.974 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.970+0000 7f9e36ff5640 1 -- 172.21.15.5:0/3651379023 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e440b4180 con 0x55937afd55d0 2024-09-12T03:00:29.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.970+0000 7f9e36ff5640 1 -- 172.21.15.5:0/3651379023 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8374+0+0 (secure 0 0 0) 0x7f9e440bb110 con 0x55937afd55d0 2024-09-12T03:00:29.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.970+0000 7f9e5b873740 10 client.15909 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T03:00:29.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.970+0000 7f9e5b873740 10 client.15909 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-09-12T03:00:29.975 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.970+0000 7f9e5b873740 20 client.15909 populate_metadata read hostname 'smithi005' 2024-09-12T03:00:29.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.970+0000 7f9e5b873740 1 --2- 172.21.15.5:0/3651379023 >> [v2:172.21.15.148:6834/4195086427,v1:172.21.15.148:6835/4195086427] conn(0x55937afefb80 0x55937aff1f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:29.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.970+0000 7f9e5b873740 4 client.15909 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:00:29.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.970+0000 7f9e5b873740 1 -- 172.21.15.5:0/3651379023 --> [v2:172.21.15.148:6834/4195086427,v1:172.21.15.148:6835/4195086427] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55937af48860 con 0x55937afefb80 2024-09-12T03:00:29.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.971+0000 7f9e527fc640 1 --2- 172.21.15.5:0/3651379023 >> [v2:172.21.15.148:6834/4195086427,v1:172.21.15.148:6835/4195086427] conn(0x55937afefb80 0x55937aff1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:29.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.971+0000 7f9e527fc640 1 --2- 172.21.15.5:0/3651379023 >> [v2:172.21.15.148:6834/4195086427,v1:172.21.15.148:6835/4195086427] conn(0x55937afefb80 0x55937aff1f70 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=823b522980334da0 server_cookie=41929c6b7a6d836f in_seq=0 out_seq=0 2024-09-12T03:00:29.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.971+0000 7f9e36ff5640 10 client.15909 ms_handle_connect on v2:172.21.15.148:6834/4195086427 2024-09-12T03:00:29.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.972+0000 7f9e36ff5640 1 -- 172.21.15.5:0/3651379023 <== mds.2 v2:172.21.15.148:6834/4195086427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55937af48860 con 0x55937afefb80 2024-09-12T03:00:29.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.972+0000 7f9e36ff5640 10 client.15909 handle_command_reply: tid=0 2024-09-12T03:00:29.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.972+0000 7f9e5b873740 1 client.15909 shutdown 2024-09-12T03:00:29.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.972+0000 7f9e5b873740 1 -- 172.21.15.5:0/3651379023 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55937afd55d0 msgr2=0x55937afdfd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:29.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.972+0000 7f9e5b873740 1 --2- 172.21.15.5:0/3651379023 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55937afd55d0 0x55937afdfd20 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f9e440a38f0 tx=0x7f9e44097550 comp rx=0 tx=0).stop 2024-09-12T03:00:29.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.973+0000 7f9e5b873740 1 -- 172.21.15.5:0/3651379023 shutdown_connections 2024-09-12T03:00:29.976 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.973+0000 7f9e5b873740 1 --2- 172.21.15.5:0/3651379023 >> [v2:172.21.15.148:6834/4195086427,v1:172.21.15.148:6835/4195086427] conn(0x55937afefb80 0x55937aff1f70 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:29.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.973+0000 7f9e5b873740 1 --2- 172.21.15.5:0/3651379023 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55937afe0370 0x55937afdd570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:29.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.973+0000 7f9e5b873740 1 --2- 172.21.15.5:0/3651379023 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55937afd6b40 0x55937afdcfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:29.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.973+0000 7f9e5b873740 1 --2- 172.21.15.5:0/3651379023 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55937afd55d0 0x55937afdfd20 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:29.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.973+0000 7f9e5b873740 1 -- 172.21.15.5:0/3651379023 >> 172.21.15.5:0/3651379023 conn(0x55937af9e310 msgr2=0x55937afd7930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:29.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.973+0000 7f9e36ff5640 0 client.15909 ms_handle_reset on v2:172.21.15.148:6834/4195086427 2024-09-12T03:00:29.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.973+0000 7f9e5b873740 1 -- 172.21.15.5:0/3651379023 shutdown_connections 2024-09-12T03:00:29.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.973+0000 7f9e5b873740 1 -- 172.21.15.5:0/3651379023 wait complete. 2024-09-12T03:00:29.977 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.973+0000 7f9e5b873740 20 client.15909 trim_cache size 0 max 16384 2024-09-12T03:00:29.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.997+0000 7f9e5b873740 1 Processor -- start 2024-09-12T03:00:29.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.998+0000 7f9e5b873740 1 -- start start 2024-09-12T03:00:29.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.998+0000 7f9e5b873740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55937afd6b40 0x55937afec250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:29.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.998+0000 7f9e5b873740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55937afe0370 0x55937afdbfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:29.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.998+0000 7f9e5b873740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55937afd1e80 0x55937aff7660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:29.998 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.998+0000 7f9e5b873740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9e541b3c30 con 0x55937afd6b40 2024-09-12T03:00:29.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.998+0000 7f9e5b873740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f9e541b5800 con 0x55937afe0370 2024-09-12T03:00:29.999 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.998+0000 7f9e5b873740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f9e541b4300 con 0x55937afd1e80 2024-09-12T03:00:30.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.999+0000 7f9e537fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55937afd1e80 0x55937aff7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:30.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.999+0000 7f9e527fc640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55937afe0370 0x55937afdbfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:30.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.999+0000 7f9e537fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55937afd1e80 0x55937aff7660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.5:33202/0 (socket says 172.21.15.5:33202) 2024-09-12T03:00:30.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.999+0000 7f9e527fc640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55937afe0370 0x55937afdbfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.5:54152/0 (socket says 172.21.15.5:54152) 2024-09-12T03:00:30.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.999+0000 7f9e537fe640 1 -- 172.21.15.5:0/3332091756 learned_addr learned my addr 172.21.15.5:0/3332091756 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:00:30.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.999+0000 7f9e52ffd640 1 --2- 172.21.15.5:0/3332091756 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55937afd6b40 0x55937afec250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:30.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.999+0000 7f9e527fc640 1 -- 172.21.15.5:0/3332091756 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55937afd1e80 msgr2=0x55937aff7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:00:30.000 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.999+0000 7f9e527fc640 1 --2- 172.21.15.5:0/3332091756 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55937afd1e80 0x55937aff7660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:30.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.999+0000 7f9e527fc640 1 -- 172.21.15.5:0/3332091756 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55937afd6b40 msgr2=0x55937afec250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:00:30.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.999+0000 7f9e527fc640 1 --2- 172.21.15.5:0/3332091756 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55937afd6b40 0x55937afec250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:30.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.999+0000 7f9e527fc640 1 -- 172.21.15.5:0/3332091756 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55937afd6750 con 0x55937afe0370 2024-09-12T03:00:30.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.999+0000 7f9e537fe640 1 --2- 172.21.15.5:0/3332091756 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55937afd1e80 0x55937aff7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:00:30.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.999+0000 7f9e52ffd640 1 --2- 172.21.15.5:0/3332091756 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55937afd6b40 0x55937afec250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:00:30.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:29.999+0000 7f9e527fc640 1 --2- 172.21.15.5:0/3332091756 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55937afe0370 0x55937afdbfd0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x55937afde610 tx=0x7f9e48010000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e2ae724a4aeca20b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:30.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:30.000+0000 7f9e38ff9640 1 -- 172.21.15.5:0/3332091756 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e48004220 con 0x55937afe0370 2024-09-12T03:00:30.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:30.000+0000 7f9e38ff9640 1 -- 172.21.15.5:0/3332091756 <== mon.2 v2:172.21.15.146:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e4801ce40 con 0x55937afe0370 2024-09-12T03:00:30.001 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:30.000+0000 7f9e38ff9640 1 -- 172.21.15.5:0/3332091756 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e48003d40 con 0x55937afe0370 2024-09-12T03:00:30.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:30.001+0000 7f9e5b873740 1 -- 172.21.15.5:0/3332091756 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55937afe0370 msgr2=0x55937afdbfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:30.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:30.001+0000 7f9e5b873740 1 --2- 172.21.15.5:0/3332091756 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55937afe0370 0x55937afdbfd0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x55937afde610 tx=0x7f9e48010000 comp rx=0 tx=0).stop 2024-09-12T03:00:30.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:30.001+0000 7f9e5b873740 1 -- 172.21.15.5:0/3332091756 shutdown_connections 2024-09-12T03:00:30.002 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:30.001+0000 7f9e5b873740 1 --2- 172.21.15.5:0/3332091756 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55937afd1e80 0x55937aff7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:30.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:30.001+0000 7f9e5b873740 1 --2- 172.21.15.5:0/3332091756 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55937afe0370 0x55937afdbfd0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:30.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:30.001+0000 7f9e5b873740 1 --2- 172.21.15.5:0/3332091756 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55937afd6b40 0x55937afec250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:30.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:30.001+0000 7f9e5b873740 1 -- 172.21.15.5:0/3332091756 >> 172.21.15.5:0/3332091756 conn(0x55937af9e310 msgr2=0x55937afdb8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:30.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:30.001+0000 7f9e5b873740 1 -- 172.21.15.5:0/3332091756 shutdown_connections 2024-09-12T03:00:30.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:30.001+0000 7f9e5b873740 1 -- 172.21.15.5:0/3332091756 wait complete. 2024-09-12T03:00:30.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:30.002+0000 7f9e5b873740 1 -- start start 2024-09-12T03:00:30.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:30.002+0000 7f9e5b873740 1 Processor -- start 2024-09-12T03:00:30.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:30.002+0000 7f9e5b873740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55937afd1e80 0x55937afe83c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:30.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:30.002+0000 7f9e5b873740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55937afd6b40 0x55937afe8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:30.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:30.002+0000 7f9e5b873740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55937afe0370 0x55937afe5390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:30.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:30.002+0000 7f9e5b873740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9e541b50a0 con 0x55937afd1e80 2024-09-12T03:00:30.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:30.002+0000 7f9e5b873740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f9e541b49d0 con 0x55937afe0370 2024-09-12T03:00:30.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:30.002+0000 7f9e5b873740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f9e541b1c10 con 0x55937afd6b40 2024-09-12T03:00:30.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:30.002+0000 7f9e5b873740 10 client.0 _ll_register_callbacks cb 0x7f9e4c009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:00:30.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:30.002+0000 7f9e52ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55937afd1e80 0x55937afe83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:30.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:30.003+0000 7f9e52ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55937afd1e80 0x55937afe83c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48358/0 (socket says 172.21.15.5:48358) 2024-09-12T03:00:30.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:30.003+0000 7f9e52ffd640 1 -- 172.21.15.5:0/3835639639 learned_addr learned my addr 172.21.15.5:0/3835639639 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:00:30.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:30.003+0000 7f9e527fc640 1 --2- 172.21.15.5:0/3835639639 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55937afd6b40 0x55937afe8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:30.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:30.003+0000 7f9e537fe640 1 --2- 172.21.15.5:0/3835639639 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55937afe0370 0x55937afe5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:30.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:30.003+0000 7f9e52ffd640 1 -- 172.21.15.5:0/3835639639 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55937afd6b40 msgr2=0x55937afe8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:00:30.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:30.003+0000 7f9e52ffd640 1 --2- 172.21.15.5:0/3835639639 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55937afd6b40 0x55937afe8900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:30.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:30.003+0000 7f9e52ffd640 1 -- 172.21.15.5:0/3835639639 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55937afe0370 msgr2=0x55937afe5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:00:30.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:30.003+0000 7f9e52ffd640 1 --2- 172.21.15.5:0/3835639639 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55937afe0370 0x55937afe5390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:30.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:30.003+0000 7f9e52ffd640 1 -- 172.21.15.5:0/3835639639 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55937afe58d0 con 0x55937afd1e80 2024-09-12T03:00:30.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:30.003+0000 7f9e52ffd640 1 --2- 172.21.15.5:0/3835639639 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55937afd1e80 0x55937afe83c0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f9e44009610 tx=0x7f9e44005cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2064e0bbfc298d39 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:30.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:30.003+0000 7f9e36ff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T03:00:30.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:30.004+0000 7f9e36ff5640 1 -- 172.21.15.5:0/3835639639 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e4409ab10 con 0x55937afd1e80 2024-09-12T03:00:30.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:30.004+0000 7f9e36ff5640 1 -- 172.21.15.5:0/3835639639 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f9e440bb110 con 0x55937afd1e80 2024-09-12T03:00:30.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:30.004+0000 7f9e5b873740 1 -- 172.21.15.5:0/3835639639 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55937af42d70 con 0x55937afd1e80 2024-09-12T03:00:30.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:30.004+0000 7f9e36ff5640 1 -- 172.21.15.5:0/3835639639 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9e4409af80 con 0x55937afd1e80 2024-09-12T03:00:30.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:30.004+0000 7f9e5b873740 10 client.15915 fetch_fsmap learned FSMap version 28 2024-09-12T03:00:30.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:30.004+0000 7f9e5b873740 1 -- 172.21.15.5:0/3835639639 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55937b05acb0 con 0x55937afd1e80 2024-09-12T03:00:30.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:30.004+0000 7f9e36ff5640 1 -- 172.21.15.5:0/3835639639 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55937af42d70 con 0x55937afd1e80 2024-09-12T03:00:30.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:30.005+0000 7f9e36ff5640 1 -- 172.21.15.5:0/3835639639 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8374+0+0 (secure 0 0 0) 0x7f9e4409a600 con 0x55937afd1e80 2024-09-12T03:00:30.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:30.005+0000 7f9e5b873740 10 client.15915 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T03:00:30.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:30.005+0000 7f9e5b873740 10 client.15915 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-09-12T03:00:30.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:30.005+0000 7f9e5b873740 20 client.15915 populate_metadata read hostname 'smithi005' 2024-09-12T03:00:30.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:30.005+0000 7f9e5b873740 1 --2- 172.21.15.5:0/3835639639 >> [v2:172.21.15.148:6834/4195086427,v1:172.21.15.148:6835/4195086427] conn(0x55937b05b1e0 0x55937b05d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:30.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:30.005+0000 7f9e5b873740 4 client.15915 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:00:30.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:30.005+0000 7f9e5b873740 1 -- 172.21.15.5:0/3835639639 --> [v2:172.21.15.148:6834/4195086427,v1:172.21.15.148:6835/4195086427] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55937aebf170 con 0x55937b05b1e0 2024-09-12T03:00:30.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:30.005+0000 7f9e527fc640 1 --2- 172.21.15.5:0/3835639639 >> [v2:172.21.15.148:6834/4195086427,v1:172.21.15.148:6835/4195086427] conn(0x55937b05b1e0 0x55937b05d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:30.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:30.005+0000 7f9e36ff5640 10 client.15915 ms_handle_connect on v2:172.21.15.148:6834/4195086427 2024-09-12T03:00:30.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:30.005+0000 7f9e527fc640 1 --2- 172.21.15.5:0/3835639639 >> [v2:172.21.15.148:6834/4195086427,v1:172.21.15.148:6835/4195086427] conn(0x55937b05b1e0 0x55937b05d5d0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=37204dbcf71eeb7a server_cookie=6a32b848a6f813a2 in_seq=0 out_seq=0 2024-09-12T03:00:30.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:30.006+0000 7f9e36ff5640 1 -- 172.21.15.5:0/3835639639 <== mds.2 v2:172.21.15.148:6834/4195086427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55937aebf170 con 0x55937b05b1e0 2024-09-12T03:00:30.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:30.006+0000 7f9e36ff5640 10 client.15915 handle_command_reply: tid=0 2024-09-12T03:00:30.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:30.006+0000 7f9e5b873740 1 client.15915 shutdown 2024-09-12T03:00:30.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:30.006+0000 7f9e5b873740 1 -- 172.21.15.5:0/3835639639 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55937afd1e80 msgr2=0x55937afe83c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:30.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:30.006+0000 7f9e5b873740 1 --2- 172.21.15.5:0/3835639639 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55937afd1e80 0x55937afe83c0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f9e44009610 tx=0x7f9e44005cd0 comp rx=0 tx=0).stop 2024-09-12T03:00:30.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:30.006+0000 7f9e5b873740 1 -- 172.21.15.5:0/3835639639 shutdown_connections 2024-09-12T03:00:30.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:30.006+0000 7f9e5b873740 1 --2- 172.21.15.5:0/3835639639 >> [v2:172.21.15.148:6834/4195086427,v1:172.21.15.148:6835/4195086427] conn(0x55937b05b1e0 0x55937b05d5d0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:30.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:30.006+0000 7f9e5b873740 1 --2- 172.21.15.5:0/3835639639 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55937afe0370 0x55937afe5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:30.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:30.006+0000 7f9e5b873740 1 --2- 172.21.15.5:0/3835639639 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55937afd6b40 0x55937afe8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:30.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:30.006+0000 7f9e5b873740 1 --2- 172.21.15.5:0/3835639639 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55937afd1e80 0x55937afe83c0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:30.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:30.006+0000 7f9e5b873740 1 -- 172.21.15.5:0/3835639639 >> 172.21.15.5:0/3835639639 conn(0x55937af9e310 msgr2=0x55937afdc0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:30.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:30.006+0000 7f9e36ff5640 0 client.15915 ms_handle_reset on v2:172.21.15.148:6834/4195086427 2024-09-12T03:00:30.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:30.006+0000 7f9e5b873740 1 -- 172.21.15.5:0/3835639639 shutdown_connections 2024-09-12T03:00:30.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:30.006+0000 7f9e5b873740 1 -- 172.21.15.5:0/3835639639 wait complete. 2024-09-12T03:00:30.011 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:30.007+0000 7f9e5b873740 20 client.15915 trim_cache size 0 max 16384 2024-09-12T03:00:30.011 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-12T03:00:30.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:30.007+0000 7f9e5949a640 1 -- 172.21.15.5:0/2788659881 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f9e3c0a2c00 msgr2=0x7f9e3c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:30.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:30.007+0000 7f9e5949a640 1 --2- 172.21.15.5:0/2788659881 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f9e3c0a2c00 0x7f9e3c0a50c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x55937afd6070 tx=0x7f9e48005f90 comp rx=0 tx=0).stop 2024-09-12T03:00:30.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:30.007+0000 7f9e5949a640 1 -- 172.21.15.5:0/2788659881 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9e54100100 msgr2=0x7f9e54073930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:30.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:30.007+0000 7f9e5949a640 1 --2- 172.21.15.5:0/2788659881 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9e54100100 0x7f9e54073930 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9e4400d950 tx=0x7f9e4400de10 comp rx=0 tx=0).stop 2024-09-12T03:00:30.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:30.008+0000 7f9e5949a640 1 -- 172.21.15.5:0/2788659881 shutdown_connections 2024-09-12T03:00:30.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:30.008+0000 7f9e5949a640 1 --2- 172.21.15.5:0/2788659881 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f9e3c0a2c00 0x7f9e3c0a50c0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:30.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:30.008+0000 7f9e5949a640 1 --2- 172.21.15.5:0/2788659881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e54106b80 0x7f9e5406cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:30.012 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:30.008+0000 7f9e5949a640 1 --2- 172.21.15.5:0/2788659881 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9e54100aa0 0x7f9e54073e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:30.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:30.008+0000 7f9e5949a640 1 --2- 172.21.15.5:0/2788659881 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9e54100100 0x7f9e54073930 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:30.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:30.008+0000 7f9e5949a640 1 -- 172.21.15.5:0/2788659881 >> 172.21.15.5:0/2788659881 conn(0x7f9e54077230 msgr2=0x7f9e54104aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:30.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:30.008+0000 7f9e5949a640 1 -- 172.21.15.5:0/2788659881 shutdown_connections 2024-09-12T03:00:30.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:30.008+0000 7f9e5949a640 1 -- 172.21.15.5:0/2788659881 wait complete. 2024-09-12T03:00:30.063 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] t_mmap_writev 2024-09-12T03:00:30.132 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] t_truncate_cmtime 2024-09-12T03:00:30.160 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] dirhash_collide 2024-09-12T03:00:30.161 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] t_rename_overwrite 2024-09-12T03:00:30.225 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] holetest 2024-09-12T03:00:30.301 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] t_truncate_self 2024-09-12T03:00:30.302 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] af_unix 2024-09-12T03:00:30.366 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] t_mmap_stale_pmd 2024-09-12T03:00:30.439 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] t_mmap_cow_race 2024-09-12T03:00:30.462 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] t_mmap_fallocate 2024-09-12T03:00:30.499 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] fsync-err 2024-09-12T03:00:30.547 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] t_mmap_write_ro 2024-09-12T03:00:30.608 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] t_ext4_dax_journal_corruption 2024-09-12T03:00:30.618 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] t_ext4_dax_inline_corruption 2024-09-12T03:00:30.693 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] t_ofd_locks 2024-09-12T03:00:30.705 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] t_mmap_collision 2024-09-12T03:00:30.784 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] mmap-write-concurrent 2024-09-12T03:00:30.784 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] t_get_file_time 2024-09-12T03:00:30.908 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] t_create_short_dirs 2024-09-12T03:00:30.936 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] t_create_long_dirs 2024-09-12T03:00:30.952 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] t_enospc 2024-09-12T03:00:30.972 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] t_mmap_writev_overlap 2024-09-12T03:00:31.096 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] checkpoint_journal 2024-09-12T03:00:31.104 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T03:00:31.112 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] mmap-rw-fault 2024-09-12T03:00:31.157 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] allocstale 2024-09-12T03:00:31.167 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] t_mmap_cow_memory_failure 2024-09-12T03:00:31.282 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] fake-dump-rootino 2024-09-12T03:00:31.319 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] dio-buf-fault 2024-09-12T03:00:31.351 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] rewinddir-test 2024-09-12T03:00:31.362 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] readdir-while-renames 2024-09-12T03:00:31.461 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:00:31.487 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] xfsctl 2024-09-12T03:00:31.510 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] bstat 2024-09-12T03:00:31.538 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] t_mtab 2024-09-12T03:00:31.556 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] getdevicesize 2024-09-12T03:00:31.636 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:00:31 smithi005 ceph-mon[31197]: pgmap v403: 97 pgs: 97 active+clean; 69 MiB data, 659 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 344 KiB/s wr, 369 op/s 2024-09-12T03:00:31.665 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:00:31 smithi148 ceph-mon[35090]: pgmap v403: 97 pgs: 97 active+clean; 69 MiB data, 659 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 344 KiB/s wr, 369 op/s 2024-09-12T03:00:31.684 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] preallo_rw_pattern_reader 2024-09-12T03:00:31.698 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] preallo_rw_pattern_writer 2024-09-12T03:00:31.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:00:31 smithi146 ceph-mon[35011]: pgmap v403: 97 pgs: 97 active+clean; 69 MiB data, 659 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 344 KiB/s wr, 369 op/s 2024-09-12T03:00:31.727 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] ftrunc 2024-09-12T03:00:31.729 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] trunc 2024-09-12T03:00:31.945 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] fs_perms 2024-09-12T03:00:31.945 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] testx 2024-09-12T03:00:31.945 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] looptest 2024-09-12T03:00:31.945 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] locktest 2024-09-12T03:00:32.093 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] unwritten_mmap 2024-09-12T03:00:32.142 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] bulkstat_unlink_test 2024-09-12T03:00:32.156 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] deduperace 2024-09-12T03:00:32.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.172+0000 7f5bd2cb1640 1 -- 172.21.15.5:0/3057868789 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5bcc100a70 msgr2=0x7f5bcc100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:32.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.172+0000 7f5bd2cb1640 1 --2- 172.21.15.5:0/3057868789 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5bcc100a70 0x7f5bcc100ed0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f5bb8009930 tx=0x7f5bb802a420 comp rx=0 tx=0).stop 2024-09-12T03:00:32.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.172+0000 7f5bd2cb1640 1 -- 172.21.15.5:0/3057868789 shutdown_connections 2024-09-12T03:00:32.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.172+0000 7f5bd2cb1640 1 --2- 172.21.15.5:0/3057868789 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5bcc100a70 0x7f5bcc100ed0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:32.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.172+0000 7f5bd2cb1640 1 --2- 172.21.15.5:0/3057868789 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5bcc101410 0x7f5bcc10d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:32.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.172+0000 7f5bd2cb1640 1 --2- 172.21.15.5:0/3057868789 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5bcc106a70 0x7f5bcc106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:32.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.172+0000 7f5bd2cb1640 1 -- 172.21.15.5:0/3057868789 >> 172.21.15.5:0/3057868789 conn(0x7f5bcc0fc7d0 msgr2=0x7f5bcc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:32.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.173+0000 7f5bd2cb1640 1 -- 172.21.15.5:0/3057868789 shutdown_connections 2024-09-12T03:00:32.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.173+0000 7f5bd2cb1640 1 -- 172.21.15.5:0/3057868789 wait complete. 2024-09-12T03:00:32.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.173+0000 7f5bd2cb1640 1 Processor -- start 2024-09-12T03:00:32.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.173+0000 7f5bd2cb1640 1 -- start start 2024-09-12T03:00:32.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.174+0000 7f5bd2cb1640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5bcc100a70 0x7f5bcc198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:32.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.174+0000 7f5bd2cb1640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5bcc101410 0x7f5bcc1989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:32.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.174+0000 7f5bd2cb1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5bcc106a70 0x7f5bcc19cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:32.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.174+0000 7f5bd2cb1640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5bcc110ec0 con 0x7f5bcc106a70 2024-09-12T03:00:32.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.174+0000 7f5bd2cb1640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f5bcc110d40 con 0x7f5bcc101410 2024-09-12T03:00:32.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.174+0000 7f5bd2cb1640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5bcc111040 con 0x7f5bcc100a70 2024-09-12T03:00:32.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.174+0000 7f5bd124e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5bcc106a70 0x7f5bcc19cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:32.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.174+0000 7f5bd124e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5bcc106a70 0x7f5bcc19cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48380/0 (socket says 172.21.15.5:48380) 2024-09-12T03:00:32.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.174+0000 7f5bd124e640 1 -- 172.21.15.5:0/4265821340 learned_addr learned my addr 172.21.15.5:0/4265821340 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:00:32.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.174+0000 7f5bd0a4d640 1 --2- 172.21.15.5:0/4265821340 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5bcc100a70 0x7f5bcc198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:32.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.174+0000 7f5bd124e640 1 -- 172.21.15.5:0/4265821340 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5bcc100a70 msgr2=0x7f5bcc198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:32.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.174+0000 7f5bd124e640 1 --2- 172.21.15.5:0/4265821340 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5bcc100a70 0x7f5bcc198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:32.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.174+0000 7f5bd124e640 1 -- 172.21.15.5:0/4265821340 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5bcc101410 msgr2=0x7f5bcc1989b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T03:00:32.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.174+0000 7f5bd124e640 1 --2- 172.21.15.5:0/4265821340 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5bcc101410 0x7f5bcc1989b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:32.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.174+0000 7f5bd124e640 1 -- 172.21.15.5:0/4265821340 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bcc19d420 con 0x7f5bcc106a70 2024-09-12T03:00:32.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.175+0000 7f5bd124e640 1 --2- 172.21.15.5:0/4265821340 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5bcc106a70 0x7f5bcc19cd40 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f5bc400eb10 tx=0x7f5bc400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:32.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.175+0000 7f5bc9ffb640 1 -- 172.21.15.5:0/4265821340 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bc4004b20 con 0x7f5bcc106a70 2024-09-12T03:00:32.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.175+0000 7f5bc9ffb640 1 -- 172.21.15.5:0/4265821340 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5bc4004cc0 con 0x7f5bcc106a70 2024-09-12T03:00:32.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.175+0000 7f5bc9ffb640 1 -- 172.21.15.5:0/4265821340 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bc40107b0 con 0x7f5bcc106a70 2024-09-12T03:00:32.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.175+0000 7f5bd2cb1640 1 -- 172.21.15.5:0/4265821340 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bcc19d710 con 0x7f5bcc106a70 2024-09-12T03:00:32.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.175+0000 7f5bd2cb1640 1 -- 172.21.15.5:0/4265821340 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bcc0fcca0 con 0x7f5bcc106a70 2024-09-12T03:00:32.211 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.179+0000 7f5bd508a740 1 Processor -- start 2024-09-12T03:00:32.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.179+0000 7f5bc9ffb640 1 -- 172.21.15.5:0/4265821340 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f5bc4005020 con 0x7f5bcc106a70 2024-09-12T03:00:32.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.179+0000 7f5bd508a740 1 -- start start 2024-09-12T03:00:32.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.180+0000 7f5bd508a740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c2bfb5ab40 0x55c2bfb59090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:32.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.180+0000 7f5bd508a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c2bfb595d0 0x55c2bfb59a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:32.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.180+0000 7f5bd508a740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c2bfb64370 0x55c2bfb66760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:32.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.180+0000 7f5bd508a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55c2bfabf2f0 con 0x55c2bfb595d0 2024-09-12T03:00:32.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.180+0000 7f5bd508a740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55c2bfa66cf0 con 0x55c2bfb64370 2024-09-12T03:00:32.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.180+0000 7f5bd508a740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55c2bfabb2b0 con 0x55c2bfb5ab40 2024-09-12T03:00:32.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.180+0000 7f5bd0a4d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c2bfb595d0 0x55c2bfb59a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:32.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.180+0000 7f5bd0a4d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c2bfb595d0 0x55c2bfb59a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48386/0 (socket says 172.21.15.5:48386) 2024-09-12T03:00:32.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.180+0000 7f5bd0a4d640 1 -- 172.21.15.5:0/2002062114 learned_addr learned my addr 172.21.15.5:0/2002062114 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:00:32.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.180+0000 7f5bc9ffb640 1 --2- 172.21.15.5:0/4265821340 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f5bbc0a2c00 0x7f5bbc0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:32.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.181+0000 7f5bcbfff640 1 --2- 172.21.15.5:0/2002062114 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c2bfb5ab40 0x55c2bfb59090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:32.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.181+0000 7f5bd0a4d640 1 -- 172.21.15.5:0/2002062114 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c2bfb5ab40 msgr2=0x55c2bfb59090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:00:32.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.181+0000 7f5bc9ffb640 1 -- 172.21.15.5:0/4265821340 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f5bc40cf770 con 0x7f5bcc106a70 2024-09-12T03:00:32.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.181+0000 7f5bd0a4d640 1 --2- 172.21.15.5:0/2002062114 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c2bfb5ab40 0x55c2bfb59090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:32.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.181+0000 7f5bd0a4d640 1 -- 172.21.15.5:0/2002062114 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c2bfb64370 msgr2=0x55c2bfb66760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:00:32.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.181+0000 7f5bd0a4d640 1 --2- 172.21.15.5:0/2002062114 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c2bfb64370 0x55c2bfb66760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:32.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.181+0000 7f5bd0a4d640 1 -- 172.21.15.5:0/2002062114 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2bfa4b3a0 con 0x55c2bfb595d0 2024-09-12T03:00:32.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.181+0000 7f5bcbfff640 1 --2- 172.21.15.5:0/2002062114 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c2bfb5ab40 0x55c2bfb59090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:00:32.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.181+0000 7f5bd0a4d640 1 --2- 172.21.15.5:0/2002062114 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c2bfb595d0 0x55c2bfb59a10 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f5bc0009660 tx=0x7f5bc0009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da4a888fbd709a07 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:32.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.181+0000 7f5bd124e640 1 --2- 172.21.15.5:0/4265821340 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f5bbc0a2c00 0x7f5bbc0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:32.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.181+0000 7f5bb6ffd640 1 -- 172.21.15.5:0/2002062114 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bc0024070 con 0x55c2bfb595d0 2024-09-12T03:00:32.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.182+0000 7f5bb6ffd640 1 -- 172.21.15.5:0/2002062114 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5bc000bd80 con 0x55c2bfb595d0 2024-09-12T03:00:32.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.182+0000 7f5bd124e640 1 --2- 172.21.15.5:0/4265821340 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f5bbc0a2c00 0x7f5bbc0a50c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f5bc4098440 tx=0x7f5bc400efd0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:32.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.182+0000 7f5bb6ffd640 1 -- 172.21.15.5:0/2002062114 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bc001e3a0 con 0x55c2bfb595d0 2024-09-12T03:00:32.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.183+0000 7f5bd508a740 1 -- 172.21.15.5:0/2002062114 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c2bfb595d0 msgr2=0x55c2bfb59a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:32.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.183+0000 7f5bd508a740 1 --2- 172.21.15.5:0/2002062114 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c2bfb595d0 0x55c2bfb59a10 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f5bc0009660 tx=0x7f5bc0009b20 comp rx=0 tx=0).stop 2024-09-12T03:00:32.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.183+0000 7f5bd508a740 1 -- 172.21.15.5:0/2002062114 shutdown_connections 2024-09-12T03:00:32.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.183+0000 7f5bd508a740 1 --2- 172.21.15.5:0/2002062114 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c2bfb595d0 0x55c2bfb59a10 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:32.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.183+0000 7f5bd508a740 1 --2- 172.21.15.5:0/2002062114 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c2bfb64370 0x55c2bfb66760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:32.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.183+0000 7f5bd508a740 1 --2- 172.21.15.5:0/2002062114 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c2bfb5ab40 0x55c2bfb59090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:32.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.183+0000 7f5bd508a740 1 -- 172.21.15.5:0/2002062114 >> 172.21.15.5:0/2002062114 conn(0x55c2bfb22310 msgr2=0x55c2bfb55c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:32.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.183+0000 7f5bd508a740 1 -- 172.21.15.5:0/2002062114 shutdown_connections 2024-09-12T03:00:32.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.183+0000 7f5bd508a740 1 -- 172.21.15.5:0/2002062114 wait complete. 2024-09-12T03:00:32.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.183+0000 7f5bd508a740 1 -- start start 2024-09-12T03:00:32.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.184+0000 7f5bd508a740 1 Processor -- start 2024-09-12T03:00:32.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.184+0000 7f5bd508a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c2bfb595d0 0x55c2bfb63da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:32.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.185+0000 7f5bd508a740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c2bfb5ab40 0x55c2bfb61050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:32.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.185+0000 7f5bd508a740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c2bfb64370 0x55c2bfb617e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:32.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.185+0000 7f5bd508a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55c2bfb67070 con 0x55c2bfb595d0 2024-09-12T03:00:32.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.185+0000 7f5bcbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c2bfb595d0 0x55c2bfb63da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:32.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.185+0000 7f5bcbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c2bfb595d0 0x55c2bfb63da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48402/0 (socket says 172.21.15.5:48402) 2024-09-12T03:00:32.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.185+0000 7f5bcbfff640 1 -- 172.21.15.5:0/3085125215 learned_addr learned my addr 172.21.15.5:0/3085125215 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:00:32.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.185+0000 7f5bd508a740 1 -- 172.21.15.5:0/3085125215 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55c2bfa7a060 con 0x55c2bfb64370 2024-09-12T03:00:32.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.185+0000 7f5bd508a740 1 -- 172.21.15.5:0/3085125215 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55c2bfb671f0 con 0x55c2bfb5ab40 2024-09-12T03:00:32.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.185+0000 7f5bd0a4d640 1 --2- 172.21.15.5:0/3085125215 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c2bfb5ab40 0x55c2bfb61050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:32.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.185+0000 7f5bd508a740 10 client.0 _ll_register_callbacks cb 0x55c2bf7cd8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:00:32.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.186+0000 7f5bd0a4d640 1 -- 172.21.15.5:0/3085125215 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c2bfb64370 msgr2=0x55c2bfb617e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-12T03:00:32.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.186+0000 7f5bd0a4d640 1 --2- 172.21.15.5:0/3085125215 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c2bfb64370 0x55c2bfb617e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:32.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.186+0000 7f5bd0a4d640 1 -- 172.21.15.5:0/3085125215 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c2bfb595d0 msgr2=0x55c2bfb63da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:00:32.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.186+0000 7f5bd0a4d640 1 --2- 172.21.15.5:0/3085125215 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c2bfb595d0 0x55c2bfb63da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:32.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.186+0000 7f5bd0a4d640 1 -- 172.21.15.5:0/3085125215 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2bf8303c0 con 0x55c2bfb5ab40 2024-09-12T03:00:32.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.186+0000 7f5bd0a4d640 1 --2- 172.21.15.5:0/3085125215 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c2bfb5ab40 0x55c2bfb61050 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5bc0022040 tx=0x7f5bc001f8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=610a6a9ee1939919 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:32.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.186+0000 7f5babfff640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-09-12T03:00:32.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.186+0000 7f5babfff640 1 -- 172.21.15.5:0/3085125215 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bc002c020 con 0x55c2bfb5ab40 2024-09-12T03:00:32.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.186+0000 7f5bd508a740 1 -- 172.21.15.5:0/3085125215 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5bc000bd80 con 0x55c2bfb5ab40 2024-09-12T03:00:32.216 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.186+0000 7f5babfff640 1 -- 172.21.15.5:0/3085125215 <== mon.1 v2:172.21.15.148:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f5bc000baa0 con 0x55c2bfb5ab40 2024-09-12T03:00:32.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.187+0000 7f5babfff640 1 -- 172.21.15.5:0/3085125215 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5bc0024040 con 0x55c2bfb5ab40 2024-09-12T03:00:32.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.187+0000 7f5bd508a740 10 client.25459 fetch_fsmap learned FSMap version 28 2024-09-12T03:00:32.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.187+0000 7f5bd508a740 1 -- 172.21.15.5:0/3085125215 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c2bfb72aa0 con 0x55c2bfb5ab40 2024-09-12T03:00:32.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.187+0000 7f5babfff640 1 -- 172.21.15.5:0/3085125215 <== mon.1 v2:172.21.15.148:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5bc000bd80 con 0x55c2bfb5ab40 2024-09-12T03:00:32.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.187+0000 7f5babfff640 1 -- 172.21.15.5:0/3085125215 <== mon.1 v2:172.21.15.148:3300/0 5 ==== fsmap(e 28) ==== 8374+0+0 (secure 0 0 0) 0x7f5bc0004bf0 con 0x55c2bfb5ab40 2024-09-12T03:00:32.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.187+0000 7f5bd508a740 10 client.25459 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T03:00:32.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.187+0000 7f5bd508a740 10 client.25459 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-12T03:00:32.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.187+0000 7f5bd508a740 20 client.25459 populate_metadata read hostname 'smithi005' 2024-09-12T03:00:32.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.187+0000 7f5bd508a740 1 --2- 172.21.15.5:0/3085125215 >> [v2:172.21.15.148:6836/2606873922,v1:172.21.15.148:6837/2606873922] conn(0x55c2bfb73e50 0x55c2bfb76240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:32.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.187+0000 7f5bd508a740 4 client.25459 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:00:32.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.187+0000 7f5bd508a740 1 -- 172.21.15.5:0/3085125215 --> [v2:172.21.15.148:6836/2606873922,v1:172.21.15.148:6837/2606873922] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c2bfacc860 con 0x55c2bfb73e50 2024-09-12T03:00:32.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.188+0000 7f5bcbfff640 1 --2- 172.21.15.5:0/3085125215 >> [v2:172.21.15.148:6836/2606873922,v1:172.21.15.148:6837/2606873922] conn(0x55c2bfb73e50 0x55c2bfb76240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:32.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.188+0000 7f5babfff640 10 client.25459 ms_handle_connect on v2:172.21.15.148:6836/2606873922 2024-09-12T03:00:32.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.188+0000 7f5bcbfff640 1 --2- 172.21.15.5:0/3085125215 >> [v2:172.21.15.148:6836/2606873922,v1:172.21.15.148:6837/2606873922] conn(0x55c2bfb73e50 0x55c2bfb76240 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3f321f5fea8bf5df server_cookie=559e02919afe90fa in_seq=0 out_seq=0 2024-09-12T03:00:32.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.189+0000 7f5babfff640 1 -- 172.21.15.5:0/3085125215 <== mds.1 v2:172.21.15.148:6836/2606873922 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c2bfacc860 con 0x55c2bfb73e50 2024-09-12T03:00:32.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.189+0000 7f5babfff640 10 client.25459 handle_command_reply: tid=0 2024-09-12T03:00:32.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.189+0000 7f5bd508a740 1 client.25459 shutdown 2024-09-12T03:00:32.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.189+0000 7f5bd508a740 1 -- 172.21.15.5:0/3085125215 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c2bfb5ab40 msgr2=0x55c2bfb61050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:32.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.189+0000 7f5bd508a740 1 --2- 172.21.15.5:0/3085125215 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c2bfb5ab40 0x55c2bfb61050 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5bc0022040 tx=0x7f5bc001f8b0 comp rx=0 tx=0).stop 2024-09-12T03:00:32.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.189+0000 7f5bd508a740 1 -- 172.21.15.5:0/3085125215 shutdown_connections 2024-09-12T03:00:32.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.189+0000 7f5bd508a740 1 --2- 172.21.15.5:0/3085125215 >> [v2:172.21.15.148:6836/2606873922,v1:172.21.15.148:6837/2606873922] conn(0x55c2bfb73e50 0x55c2bfb76240 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:32.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.189+0000 7f5babfff640 0 client.25459 ms_handle_reset on v2:172.21.15.148:6836/2606873922 2024-09-12T03:00:32.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.189+0000 7f5bd508a740 1 --2- 172.21.15.5:0/3085125215 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c2bfb64370 0x55c2bfb617e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:32.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.189+0000 7f5bd508a740 1 --2- 172.21.15.5:0/3085125215 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c2bfb5ab40 0x55c2bfb61050 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:32.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.189+0000 7f5bd508a740 1 --2- 172.21.15.5:0/3085125215 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c2bfb595d0 0x55c2bfb63da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:32.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.189+0000 7f5bd508a740 1 -- 172.21.15.5:0/3085125215 >> 172.21.15.5:0/3085125215 conn(0x55c2bfb22310 msgr2=0x55c2bfb5b960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:32.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.190+0000 7f5bd508a740 1 -- 172.21.15.5:0/3085125215 shutdown_connections 2024-09-12T03:00:32.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.190+0000 7f5bd508a740 1 -- 172.21.15.5:0/3085125215 wait complete. 2024-09-12T03:00:32.219 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.190+0000 7f5bd508a740 20 client.25459 trim_cache size 0 max 16384 2024-09-12T03:00:32.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.217+0000 7f5bd508a740 1 Processor -- start 2024-09-12T03:00:32.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.217+0000 7f5bd508a740 1 -- start start 2024-09-12T03:00:32.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.217+0000 7f5bd508a740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c2bfb595d0 0x55c2bfb60030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:32.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.217+0000 7f5bd508a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c2bfb5ab40 0x55c2bfb7b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:32.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.218+0000 7f5bd508a740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c2bfb64370 0x55c2bfb7be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:32.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.218+0000 7f5bd508a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5bcc1ab420 con 0x55c2bfb5ab40 2024-09-12T03:00:32.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.218+0000 7f5bd508a740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f5bcc1acff0 con 0x55c2bfb595d0 2024-09-12T03:00:32.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.218+0000 7f5bd508a740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5bcc1abaf0 con 0x55c2bfb64370 2024-09-12T03:00:32.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.218+0000 7f5bd0a4d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c2bfb5ab40 0x55c2bfb7b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:32.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.218+0000 7f5bd0a4d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c2bfb5ab40 0x55c2bfb7b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48410/0 (socket says 172.21.15.5:48410) 2024-09-12T03:00:32.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.218+0000 7f5bd0a4d640 1 -- 172.21.15.5:0/1724126368 learned_addr learned my addr 172.21.15.5:0/1724126368 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:00:32.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.218+0000 7f5bcbfff640 1 --2- 172.21.15.5:0/1724126368 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c2bfb595d0 0x55c2bfb60030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:32.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.218+0000 7f5bcbfff640 1 --2- 172.21.15.5:0/1724126368 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c2bfb64370 0x55c2bfb7be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:32.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.218+0000 7f5bd0a4d640 1 -- 172.21.15.5:0/1724126368 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c2bfb64370 msgr2=0x55c2bfb7be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:00:32.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.218+0000 7f5bd0a4d640 1 --2- 172.21.15.5:0/1724126368 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c2bfb64370 0x55c2bfb7be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:32.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.218+0000 7f5bd0a4d640 1 -- 172.21.15.5:0/1724126368 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c2bfb595d0 msgr2=0x55c2bfb60030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:00:32.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.218+0000 7f5bd0a4d640 1 --2- 172.21.15.5:0/1724126368 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c2bfb595d0 0x55c2bfb60030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:32.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.218+0000 7f5bd0a4d640 1 -- 172.21.15.5:0/1724126368 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2bfb56960 con 0x55c2bfb5ab40 2024-09-12T03:00:32.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.218+0000 7f5bcbfff640 1 --2- 172.21.15.5:0/1724126368 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c2bfb595d0 0x55c2bfb60030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:00:32.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.219+0000 7f5bd0a4d640 1 --2- 172.21.15.5:0/1724126368 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c2bfb5ab40 0x55c2bfb7b930 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f5bc0005850 tx=0x7f5bc0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ffb1897bc2b4665 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:32.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.219+0000 7f5bb67fc640 1 -- 172.21.15.5:0/1724126368 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bc003e070 con 0x55c2bfb5ab40 2024-09-12T03:00:32.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.219+0000 7f5bb67fc640 1 -- 172.21.15.5:0/1724126368 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5bc001f920 con 0x55c2bfb5ab40 2024-09-12T03:00:32.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.219+0000 7f5bb67fc640 1 -- 172.21.15.5:0/1724126368 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bc002c070 con 0x55c2bfb5ab40 2024-09-12T03:00:32.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.220+0000 7f5bd508a740 1 -- 172.21.15.5:0/1724126368 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c2bfb5ab40 msgr2=0x55c2bfb7b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:32.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.220+0000 7f5bd508a740 1 --2- 172.21.15.5:0/1724126368 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c2bfb5ab40 0x55c2bfb7b930 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f5bc0005850 tx=0x7f5bc0005880 comp rx=0 tx=0).stop 2024-09-12T03:00:32.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.220+0000 7f5bd508a740 1 -- 172.21.15.5:0/1724126368 shutdown_connections 2024-09-12T03:00:32.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.220+0000 7f5bd508a740 1 --2- 172.21.15.5:0/1724126368 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c2bfb5ab40 0x55c2bfb7b930 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:32.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.220+0000 7f5bd508a740 1 --2- 172.21.15.5:0/1724126368 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c2bfb64370 0x55c2bfb7be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:32.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.220+0000 7f5bd508a740 1 --2- 172.21.15.5:0/1724126368 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c2bfb595d0 0x55c2bfb60030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:32.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.220+0000 7f5bd508a740 1 -- 172.21.15.5:0/1724126368 >> 172.21.15.5:0/1724126368 conn(0x55c2bfb22310 msgr2=0x55c2bfb5f920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:32.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.220+0000 7f5bd508a740 1 -- 172.21.15.5:0/1724126368 shutdown_connections 2024-09-12T03:00:32.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.220+0000 7f5bd508a740 1 -- 172.21.15.5:0/1724126368 wait complete. 2024-09-12T03:00:32.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.221+0000 7f5bd508a740 1 -- start start 2024-09-12T03:00:32.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.221+0000 7f5bd508a740 1 Processor -- start 2024-09-12T03:00:32.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.221+0000 7f5bd508a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c2bfb595d0 0x55c2bfb6c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:32.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.221+0000 7f5bd508a740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c2bfb64370 0x55c2bfb5da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:32.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.221+0000 7f5bd508a740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c2bfb73e50 0x55c2bfb5dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:32.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.222+0000 7f5bcbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c2bfb595d0 0x55c2bfb6c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:32.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.222+0000 7f5bd508a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5bcc1ac890 con 0x55c2bfb595d0 2024-09-12T03:00:32.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.222+0000 7f5bcbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c2bfb595d0 0x55c2bfb6c8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48414/0 (socket says 172.21.15.5:48414) 2024-09-12T03:00:32.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.222+0000 7f5bcbfff640 1 -- 172.21.15.5:0/566850843 learned_addr learned my addr 172.21.15.5:0/566850843 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:00:32.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.222+0000 7f5bd0a4d640 1 --2- 172.21.15.5:0/566850843 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c2bfb64370 0x55c2bfb5da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:32.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.222+0000 7f5bd508a740 1 -- 172.21.15.5:0/566850843 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f5bcc1ac1c0 con 0x55c2bfb73e50 2024-09-12T03:00:32.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.222+0000 7f5bd508a740 1 -- 172.21.15.5:0/566850843 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5bcc1a9400 con 0x55c2bfb64370 2024-09-12T03:00:32.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.222+0000 7f5bd508a740 10 client.0 _ll_register_callbacks cb 0x7f5bc0022070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:00:32.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.222+0000 7f5bcbfff640 1 -- 172.21.15.5:0/566850843 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c2bfb64370 msgr2=0x55c2bfb5da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:00:32.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.222+0000 7f5bcbfff640 1 --2- 172.21.15.5:0/566850843 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c2bfb64370 0x55c2bfb5da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:32.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.222+0000 7f5bcbfff640 1 -- 172.21.15.5:0/566850843 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c2bfb73e50 msgr2=0x55c2bfb5dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:00:32.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.222+0000 7f5bcbfff640 1 --2- 172.21.15.5:0/566850843 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c2bfb73e50 0x55c2bfb5dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:32.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.222+0000 7f5bcbfff640 1 -- 172.21.15.5:0/566850843 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2bfb5c120 con 0x55c2bfb595d0 2024-09-12T03:00:32.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.222+0000 7f5bd0a4d640 1 --2- 172.21.15.5:0/566850843 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c2bfb64370 0x55c2bfb5da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:00:32.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.222+0000 7f5bcbfff640 1 --2- 172.21.15.5:0/566850843 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c2bfb595d0 0x55c2bfb6c8a0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f5bb8005fd0 tx=0x7f5bb8005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90ce55b671838a55 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:32.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.222+0000 7f5babfff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T03:00:32.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.223+0000 7f5babfff640 1 -- 172.21.15.5:0/566850843 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb8050060 con 0x55c2bfb595d0 2024-09-12T03:00:32.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.223+0000 7f5babfff640 1 -- 172.21.15.5:0/566850843 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f5bb8005150 con 0x55c2bfb595d0 2024-09-12T03:00:32.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.223+0000 7f5bd508a740 1 -- 172.21.15.5:0/566850843 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5bc001f920 con 0x55c2bfb595d0 2024-09-12T03:00:32.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.223+0000 7f5babfff640 1 -- 172.21.15.5:0/566850843 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5bb8046040 con 0x55c2bfb595d0 2024-09-12T03:00:32.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.223+0000 7f5bd508a740 10 client.15933 fetch_fsmap learned FSMap version 28 2024-09-12T03:00:32.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.223+0000 7f5bd508a740 1 -- 172.21.15.5:0/566850843 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c2bfbde4e0 con 0x55c2bfb595d0 2024-09-12T03:00:32.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.224+0000 7f5babfff640 1 -- 172.21.15.5:0/566850843 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5bb8004490 con 0x55c2bfb595d0 2024-09-12T03:00:32.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.224+0000 7f5babfff640 1 -- 172.21.15.5:0/566850843 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8374+0+0 (secure 0 0 0) 0x7f5bb8003680 con 0x55c2bfb595d0 2024-09-12T03:00:32.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.224+0000 7f5bd508a740 10 client.15933 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T03:00:32.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.224+0000 7f5bd508a740 10 client.15933 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-12T03:00:32.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.224+0000 7f5bd508a740 20 client.15933 populate_metadata read hostname 'smithi005' 2024-09-12T03:00:32.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.224+0000 7f5bd508a740 1 --2- 172.21.15.5:0/566850843 >> [v2:172.21.15.148:6836/2606873922,v1:172.21.15.148:6837/2606873922] conn(0x55c2bfbdeb90 0x55c2bfbe0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:32.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.224+0000 7f5bd508a740 4 client.15933 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:00:32.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.224+0000 7f5bd508a740 1 -- 172.21.15.5:0/566850843 --> [v2:172.21.15.148:6836/2606873922,v1:172.21.15.148:6837/2606873922] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f5bc001f920 con 0x55c2bfbdeb90 2024-09-12T03:00:32.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.224+0000 7f5bd0a4d640 1 --2- 172.21.15.5:0/566850843 >> [v2:172.21.15.148:6836/2606873922,v1:172.21.15.148:6837/2606873922] conn(0x55c2bfbdeb90 0x55c2bfbe0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:32.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.225+0000 7f5bd0a4d640 1 --2- 172.21.15.5:0/566850843 >> [v2:172.21.15.148:6836/2606873922,v1:172.21.15.148:6837/2606873922] conn(0x55c2bfbdeb90 0x55c2bfbe0fb0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=17b0b7f3473720d7 server_cookie=64cd26789dfe77f9 in_seq=0 out_seq=0 2024-09-12T03:00:32.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.225+0000 7f5babfff640 10 client.15933 ms_handle_connect on v2:172.21.15.148:6836/2606873922 2024-09-12T03:00:32.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.225+0000 7f5babfff640 1 -- 172.21.15.5:0/566850843 <== mds.1 v2:172.21.15.148:6836/2606873922 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f5bc001f920 con 0x55c2bfbdeb90 2024-09-12T03:00:32.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.225+0000 7f5babfff640 10 client.15933 handle_command_reply: tid=0 2024-09-12T03:00:32.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.225+0000 7f5bd508a740 1 client.15933 shutdown 2024-09-12T03:00:32.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.225+0000 7f5bd508a740 1 -- 172.21.15.5:0/566850843 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c2bfb595d0 msgr2=0x55c2bfb6c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:32.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.225+0000 7f5bd508a740 1 --2- 172.21.15.5:0/566850843 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c2bfb595d0 0x55c2bfb6c8a0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f5bb8005fd0 tx=0x7f5bb8005040 comp rx=0 tx=0).stop 2024-09-12T03:00:32.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.226+0000 7f5bd508a740 1 -- 172.21.15.5:0/566850843 shutdown_connections 2024-09-12T03:00:32.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.226+0000 7f5bd508a740 1 --2- 172.21.15.5:0/566850843 >> [v2:172.21.15.148:6836/2606873922,v1:172.21.15.148:6837/2606873922] conn(0x55c2bfbdeb90 0x55c2bfbe0fb0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:32.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.226+0000 7f5bd508a740 1 --2- 172.21.15.5:0/566850843 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c2bfb73e50 0x55c2bfb5dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:32.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.226+0000 7f5babfff640 0 client.15933 ms_handle_reset on v2:172.21.15.148:6836/2606873922 2024-09-12T03:00:32.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.226+0000 7f5bd508a740 1 --2- 172.21.15.5:0/566850843 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c2bfb64370 0x55c2bfb5da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:32.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.226+0000 7f5bd508a740 1 --2- 172.21.15.5:0/566850843 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c2bfb595d0 0x55c2bfb6c8a0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:32.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.226+0000 7f5bd508a740 1 -- 172.21.15.5:0/566850843 >> 172.21.15.5:0/566850843 conn(0x55c2bfb22310 msgr2=0x55c2bfb60140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:32.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.226+0000 7f5bd508a740 1 -- 172.21.15.5:0/566850843 shutdown_connections 2024-09-12T03:00:32.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.226+0000 7f5bd508a740 1 -- 172.21.15.5:0/566850843 wait complete. 2024-09-12T03:00:32.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.226+0000 7f5bd508a740 20 client.15933 trim_cache size 0 max 16384 2024-09-12T03:00:32.234 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-12T03:00:32.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.227+0000 7f5bd2cb1640 1 -- 172.21.15.5:0/4265821340 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f5bbc0a2c00 msgr2=0x7f5bbc0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:32.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.227+0000 7f5bd2cb1640 1 --2- 172.21.15.5:0/4265821340 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f5bbc0a2c00 0x7f5bbc0a50c0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f5bc4098440 tx=0x7f5bc400efd0 comp rx=0 tx=0).stop 2024-09-12T03:00:32.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.227+0000 7f5bd2cb1640 1 -- 172.21.15.5:0/4265821340 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5bcc106a70 msgr2=0x7f5bcc19cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:32.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.227+0000 7f5bd2cb1640 1 --2- 172.21.15.5:0/4265821340 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5bcc106a70 0x7f5bcc19cd40 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f5bc400eb10 tx=0x7f5bc400c2d0 comp rx=0 tx=0).stop 2024-09-12T03:00:32.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.227+0000 7f5bd2cb1640 1 -- 172.21.15.5:0/4265821340 shutdown_connections 2024-09-12T03:00:32.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.227+0000 7f5bd2cb1640 1 --2- 172.21.15.5:0/4265821340 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f5bbc0a2c00 0x7f5bbc0a50c0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:32.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.227+0000 7f5bd2cb1640 1 --2- 172.21.15.5:0/4265821340 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5bcc106a70 0x7f5bcc19cd40 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:32.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.227+0000 7f5bd2cb1640 1 --2- 172.21.15.5:0/4265821340 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5bcc101410 0x7f5bcc1989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:32.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.227+0000 7f5bd2cb1640 1 --2- 172.21.15.5:0/4265821340 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5bcc100a70 0x7f5bcc198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:32.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.227+0000 7f5bd2cb1640 1 -- 172.21.15.5:0/4265821340 >> 172.21.15.5:0/4265821340 conn(0x7f5bcc0fc7d0 msgr2=0x7f5bcc06aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:32.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.227+0000 7f5bd2cb1640 1 -- 172.21.15.5:0/4265821340 shutdown_connections 2024-09-12T03:00:32.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:32.227+0000 7f5bd2cb1640 1 -- 172.21.15.5:0/4265821340 wait complete. 2024-09-12T03:00:32.293 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] bulkstat_unlink_test_modified 2024-09-12T03:00:32.308 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] t_dir_offset 2024-09-12T03:00:32.372 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] t_futimens 2024-09-12T03:00:32.377 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] t_immutable 2024-09-12T03:00:32.467 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] stale_handle 2024-09-12T03:00:32.499 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] pwrite_mmap_blocked 2024-09-12T03:00:32.524 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] t_dir_offset2 2024-09-12T03:00:32.662 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] seek_sanity_test 2024-09-12T03:00:32.670 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] seek_copy_test 2024-09-12T03:00:32.722 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] t_readdir_1 2024-09-12T03:00:33.062 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] t_readdir_2 2024-09-12T03:00:33.093 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] fsync-tester 2024-09-12T03:00:33.157 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] nsexec 2024-09-12T03:00:33.238 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] cloner 2024-09-12T03:00:33.241 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] renameat2 2024-09-12T03:00:33.313 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T03:00:33.358 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] t_getcwd 2024-09-12T03:00:33.442 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] e4compact 2024-09-12T03:00:33.449 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] test-nextquota 2024-09-12T03:00:33.451 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] punch-alternating 2024-09-12T03:00:33.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:00:33 smithi005 ceph-mon[31197]: pgmap v404: 97 pgs: 97 active+clean; 69 MiB data, 660 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 322 KiB/s wr, 371 op/s 2024-09-12T03:00:33.576 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] attr-list-by-handle-cursor-test 2024-09-12T03:00:33.665 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:00:33 smithi148 ceph-mon[35090]: pgmap v404: 97 pgs: 97 active+clean; 69 MiB data, 660 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 322 KiB/s wr, 371 op/s 2024-09-12T03:00:33.697 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] listxattr 2024-09-12T03:00:33.700 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:00:33.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:00:33 smithi146 ceph-mon[35011]: pgmap v404: 97 pgs: 97 active+clean; 69 MiB data, 660 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 322 KiB/s wr, 371 op/s 2024-09-12T03:00:33.708 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] dio-interleaved 2024-09-12T03:00:33.737 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] t_dir_type 2024-09-12T03:00:33.799 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] dio-invalidate-cache 2024-09-12T03:00:33.917 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] stat_test 2024-09-12T03:00:33.917 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] t_encrypted_d_revalidate 2024-09-12T03:00:33.917 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] attr_replace_test 2024-09-12T03:00:34.021 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] swapon 2024-09-12T03:00:34.097 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] mkswap 2024-09-12T03:00:34.103 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] t_attr_corruption 2024-09-12T03:00:34.199 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] t_open_tmpfiles 2024-09-12T03:00:34.254 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] fscrypt-crypt-util 2024-09-12T03:00:34.270 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] bulkstat_null_ocount 2024-09-12T03:00:34.273 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] splice-test 2024-09-12T03:00:34.412 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] chprojid_fail 2024-09-12T03:00:34.437 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.437+0000 7fbb6dda2640 1 -- 172.21.15.5:0/1691769977 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb68106f20 msgr2=0x7fbb6810eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:34.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.437+0000 7fbb6dda2640 1 --2- 172.21.15.5:0/1691769977 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb68106f20 0x7fbb6810eaa0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fbb60009ca0 tx=0x7fbb6002a8c0 comp rx=0 tx=0).stop 2024-09-12T03:00:34.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.437+0000 7fbb6dda2640 1 -- 172.21.15.5:0/1691769977 shutdown_connections 2024-09-12T03:00:34.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.437+0000 7fbb6dda2640 1 --2- 172.21.15.5:0/1691769977 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb68106f20 0x7fbb6810eaa0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:34.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.437+0000 7fbb6dda2640 1 --2- 172.21.15.5:0/1691769977 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb681027a0 0x7fbb681067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:34.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.437+0000 7fbb6dda2640 1 --2- 172.21.15.5:0/1691769977 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbb68101df0 0x7fbb681021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:34.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.437+0000 7fbb6dda2640 1 -- 172.21.15.5:0/1691769977 >> 172.21.15.5:0/1691769977 conn(0x7fbb680fd8b0 msgr2=0x7fbb680ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:34.438 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.437+0000 7fbb6dda2640 1 -- 172.21.15.5:0/1691769977 shutdown_connections 2024-09-12T03:00:34.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.437+0000 7fbb6dda2640 1 -- 172.21.15.5:0/1691769977 wait complete. 2024-09-12T03:00:34.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.438+0000 7fbb6dda2640 1 Processor -- start 2024-09-12T03:00:34.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.439+0000 7fbb6dda2640 1 -- start start 2024-09-12T03:00:34.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.439+0000 7fbb6dda2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb68101df0 0x7fbb680744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:34.439 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.439+0000 7fbb6dda2640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb681027a0 0x7fbb68074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:34.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.439+0000 7fbb6dda2640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbb68106f20 0x7fbb68074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:34.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.439+0000 7fbb6dda2640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbb68111f80 con 0x7fbb68101df0 2024-09-12T03:00:34.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.439+0000 7fbb6dda2640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fbb68111e00 con 0x7fbb68106f20 2024-09-12T03:00:34.440 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.439+0000 7fbb6dda2640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fbb68112100 con 0x7fbb681027a0 2024-09-12T03:00:34.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.439+0000 7fbb66ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb681027a0 0x7fbb68074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:34.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.439+0000 7fbb677fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb68101df0 0x7fbb680744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:34.441 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.439+0000 7fbb67fff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbb68106f20 0x7fbb68074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:34.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.439+0000 7fbb67fff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbb68106f20 0x7fbb68074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.5:39954/0 (socket says 172.21.15.5:39954) 2024-09-12T03:00:34.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.439+0000 7fbb677fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb68101df0 0x7fbb680744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56788/0 (socket says 172.21.15.5:56788) 2024-09-12T03:00:34.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.439+0000 7fbb67fff640 1 -- 172.21.15.5:0/610213905 learned_addr learned my addr 172.21.15.5:0/610213905 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:00:34.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.440+0000 7fbb677fe640 1 -- 172.21.15.5:0/610213905 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb681027a0 msgr2=0x7fbb68074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:34.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.440+0000 7fbb677fe640 1 --2- 172.21.15.5:0/610213905 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb681027a0 0x7fbb68074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:34.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.440+0000 7fbb677fe640 1 -- 172.21.15.5:0/610213905 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbb68106f20 msgr2=0x7fbb68074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:34.442 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.440+0000 7fbb677fe640 1 --2- 172.21.15.5:0/610213905 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbb68106f20 0x7fbb68074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:34.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.440+0000 7fbb677fe640 1 -- 172.21.15.5:0/610213905 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb681aad10 con 0x7fbb68101df0 2024-09-12T03:00:34.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.440+0000 7fbb66ffd640 1 --2- 172.21.15.5:0/610213905 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb681027a0 0x7fbb68074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:00:34.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.440+0000 7fbb677fe640 1 --2- 172.21.15.5:0/610213905 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb68101df0 0x7fbb680744f0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fbb5800e9f0 tx=0x7fbb5800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:34.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.440+0000 7fbb64ff9640 1 -- 172.21.15.5:0/610213905 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb5800ccc0 con 0x7fbb68101df0 2024-09-12T03:00:34.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.441+0000 7fbb64ff9640 1 -- 172.21.15.5:0/610213905 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb58010910 con 0x7fbb68101df0 2024-09-12T03:00:34.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.441+0000 7fbb6dda2640 1 -- 172.21.15.5:0/610213905 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb681aaf40 con 0x7fbb68101df0 2024-09-12T03:00:34.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.441+0000 7fbb6dda2640 1 -- 172.21.15.5:0/610213905 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb681ab3f0 con 0x7fbb68101df0 2024-09-12T03:00:34.443 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.441+0000 7fbb64ff9640 1 -- 172.21.15.5:0/610213905 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb58018960 con 0x7fbb68101df0 2024-09-12T03:00:34.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.444+0000 7fbb7017b740 1 Processor -- start 2024-09-12T03:00:34.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.444+0000 7fbb7017b740 1 -- start start 2024-09-12T03:00:34.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.444+0000 7fbb7017b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cd9a42b40 0x558cd9a41090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:34.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.444+0000 7fbb7017b740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558cd9a415d0 0x558cd9a41a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:34.445 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.444+0000 7fbb7017b740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558cd9a4c370 0x558cd9a4e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:34.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.444+0000 7fbb7017b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558cd99a72f0 con 0x558cd9a42b40 2024-09-12T03:00:34.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.444+0000 7fbb7017b740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x558cd994ecf0 con 0x558cd9a4c370 2024-09-12T03:00:34.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.444+0000 7fbb7017b740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x558cd99a32b0 con 0x558cd9a415d0 2024-09-12T03:00:34.446 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.445+0000 7fbb67fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cd9a42b40 0x558cd9a41090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:34.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.445+0000 7fbb64ff9640 1 -- 172.21.15.5:0/610213905 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fbb58010ac0 con 0x7fbb68101df0 2024-09-12T03:00:34.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.445+0000 7fbb67fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cd9a42b40 0x558cd9a41090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56798/0 (socket says 172.21.15.5:56798) 2024-09-12T03:00:34.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.445+0000 7fbb67fff640 1 -- 172.21.15.5:0/453773153 learned_addr learned my addr 172.21.15.5:0/453773153 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:00:34.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.445+0000 7fbb66ffd640 1 --2- 172.21.15.5:0/453773153 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558cd9a4c370 0x558cd9a4e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:34.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.445+0000 7fbb677fe640 1 --2- 172.21.15.5:0/453773153 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558cd9a415d0 0x558cd9a41a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:34.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.445+0000 7fbb67fff640 1 -- 172.21.15.5:0/453773153 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558cd9a415d0 msgr2=0x558cd9a41a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:00:34.449 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.445+0000 7fbb67fff640 1 --2- 172.21.15.5:0/453773153 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558cd9a415d0 0x558cd9a41a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:34.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.445+0000 7fbb67fff640 1 -- 172.21.15.5:0/453773153 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558cd9a4c370 msgr2=0x558cd9a4e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:00:34.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.445+0000 7fbb67fff640 1 --2- 172.21.15.5:0/453773153 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558cd9a4c370 0x558cd9a4e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:34.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.445+0000 7fbb67fff640 1 -- 172.21.15.5:0/453773153 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558cd99333a0 con 0x558cd9a42b40 2024-09-12T03:00:34.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.445+0000 7fbb677fe640 1 --2- 172.21.15.5:0/453773153 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558cd9a415d0 0x558cd9a41a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:00:34.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.446+0000 7fbb67fff640 1 --2- 172.21.15.5:0/453773153 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cd9a42b40 0x558cd9a41090 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fbb600097b0 tx=0x7fbb60009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49733abb325f86f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:34.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.446+0000 7fbb4dffb640 1 -- 172.21.15.5:0/453773153 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb60031db0 con 0x558cd9a42b40 2024-09-12T03:00:34.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.446+0000 7fbb4dffb640 1 -- 172.21.15.5:0/453773153 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb60036040 con 0x558cd9a42b40 2024-09-12T03:00:34.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.446+0000 7fbb4dffb640 1 -- 172.21.15.5:0/453773153 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb60031670 con 0x558cd9a42b40 2024-09-12T03:00:34.450 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.446+0000 7fbb64ff9640 1 --2- 172.21.15.5:0/610213905 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fbb500a2c00 0x7fbb500a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:34.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.447+0000 7fbb64ff9640 1 -- 172.21.15.5:0/610213905 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fbb580cf320 con 0x7fbb68101df0 2024-09-12T03:00:34.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.447+0000 7fbb677fe640 1 --2- 172.21.15.5:0/610213905 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fbb500a2c00 0x7fbb500a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:34.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.447+0000 7fbb7017b740 1 -- 172.21.15.5:0/453773153 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cd9a42b40 msgr2=0x558cd9a41090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:34.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.447+0000 7fbb7017b740 1 --2- 172.21.15.5:0/453773153 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cd9a42b40 0x558cd9a41090 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fbb600097b0 tx=0x7fbb60009a20 comp rx=0 tx=0).stop 2024-09-12T03:00:34.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.447+0000 7fbb7017b740 1 -- 172.21.15.5:0/453773153 shutdown_connections 2024-09-12T03:00:34.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.447+0000 7fbb7017b740 1 --2- 172.21.15.5:0/453773153 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558cd9a4c370 0x558cd9a4e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:34.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.447+0000 7fbb7017b740 1 --2- 172.21.15.5:0/453773153 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558cd9a415d0 0x558cd9a41a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:34.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.447+0000 7fbb7017b740 1 --2- 172.21.15.5:0/453773153 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cd9a42b40 0x558cd9a41090 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:34.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.448+0000 7fbb7017b740 1 -- 172.21.15.5:0/453773153 >> 172.21.15.5:0/453773153 conn(0x558cd9a0a310 msgr2=0x558cd9a3dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:34.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.448+0000 7fbb7017b740 1 -- 172.21.15.5:0/453773153 shutdown_connections 2024-09-12T03:00:34.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.448+0000 7fbb7017b740 1 -- 172.21.15.5:0/453773153 wait complete. 2024-09-12T03:00:34.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.448+0000 7fbb677fe640 1 --2- 172.21.15.5:0/610213905 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fbb500a2c00 0x7fbb500a50c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fbb580cd1a0 tx=0x7fbb58009700 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:34.451 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.448+0000 7fbb7017b740 1 -- start start 2024-09-12T03:00:34.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.449+0000 7fbb7017b740 1 Processor -- start 2024-09-12T03:00:34.452 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.449+0000 7fbb7017b740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558cd9a415d0 0x558cd9a4bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:34.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.449+0000 7fbb7017b740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558cd9a42b40 0x558cd9a49040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:34.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.449+0000 7fbb7017b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cd9a4c370 0x558cd9a49650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:34.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.449+0000 7fbb7017b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558cd9a4ee90 con 0x558cd9a4c370 2024-09-12T03:00:34.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.449+0000 7fbb7017b740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x558cd9962060 con 0x558cd9a415d0 2024-09-12T03:00:34.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.449+0000 7fbb7017b740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x558cd9a4f010 con 0x558cd9a42b40 2024-09-12T03:00:34.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.449+0000 7fbb7017b740 10 client.0 _ll_register_callbacks cb 0x558cd96b58c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:00:34.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.449+0000 7fbb67fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cd9a4c370 0x558cd9a49650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:34.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.449+0000 7fbb66ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558cd9a42b40 0x558cd9a49040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:34.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.449+0000 7fbb67fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cd9a4c370 0x558cd9a49650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56812/0 (socket says 172.21.15.5:56812) 2024-09-12T03:00:34.454 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.449+0000 7fbb66ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558cd9a42b40 0x558cd9a49040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.5:54332/0 (socket says 172.21.15.5:54332) 2024-09-12T03:00:34.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.449+0000 7fbb67fff640 1 -- 172.21.15.5:0/1927466647 learned_addr learned my addr 172.21.15.5:0/1927466647 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:00:34.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.449+0000 7fbb67fff640 1 --2- 172.21.15.5:0/1927466647 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558cd9a415d0 0x558cd9a4bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:34.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.450+0000 7fbb67fff640 1 -- 172.21.15.5:0/1927466647 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558cd9a42b40 msgr2=0x558cd9a49040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:00:34.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.450+0000 7fbb67fff640 1 --2- 172.21.15.5:0/1927466647 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558cd9a42b40 0x558cd9a49040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:34.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.450+0000 7fbb67fff640 1 -- 172.21.15.5:0/1927466647 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cd9a4c370 msgr2=0x558cd9a49650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:00:34.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.450+0000 7fbb67fff640 1 --2- 172.21.15.5:0/1927466647 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cd9a4c370 0x558cd9a49650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:34.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.450+0000 7fbb67fff640 1 -- 172.21.15.5:0/1927466647 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558cd97183c0 con 0x558cd9a415d0 2024-09-12T03:00:34.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.450+0000 7fbb67fff640 1 --2- 172.21.15.5:0/1927466647 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558cd9a415d0 0x558cd9a4bd90 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fbb60009620 tx=0x7fbb6002a8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d716974ebc1b773 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:34.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.450+0000 7fbb46ffd640 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2024-09-12T03:00:34.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.450+0000 7fbb46ffd640 1 -- 172.21.15.5:0/1927466647 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb60058060 con 0x558cd9a415d0 2024-09-12T03:00:34.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.450+0000 7fbb46ffd640 1 -- 172.21.15.5:0/1927466647 <== mon.2 v2:172.21.15.146:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fbb60030c10 con 0x558cd9a415d0 2024-09-12T03:00:34.455 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.450+0000 7fbb7017b740 1 -- 172.21.15.5:0/1927466647 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb60036040 con 0x558cd9a415d0 2024-09-12T03:00:34.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.451+0000 7fbb46ffd640 1 -- 172.21.15.5:0/1927466647 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbb60030e10 con 0x558cd9a415d0 2024-09-12T03:00:34.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.451+0000 7fbb7017b740 10 client.25376 fetch_fsmap learned FSMap version 28 2024-09-12T03:00:34.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.451+0000 7fbb7017b740 1 -- 172.21.15.5:0/1927466647 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558cd9a4ee90 con 0x558cd9a415d0 2024-09-12T03:00:34.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.451+0000 7fbb46ffd640 1 -- 172.21.15.5:0/1927466647 <== mon.2 v2:172.21.15.146:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb60036040 con 0x558cd9a415d0 2024-09-12T03:00:34.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.452+0000 7fbb46ffd640 1 -- 172.21.15.5:0/1927466647 <== mon.2 v2:172.21.15.146:3300/0 5 ==== fsmap(e 28) ==== 8374+0+0 (secure 0 0 0) 0x7fbb6004d070 con 0x558cd9a415d0 2024-09-12T03:00:34.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.452+0000 7fbb7017b740 10 client.25376 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T03:00:34.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.452+0000 7fbb7017b740 10 client.25376 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:00:34.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.452+0000 7fbb7017b740 20 client.25376 populate_metadata read hostname 'smithi005' 2024-09-12T03:00:34.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.452+0000 7fbb7017b740 1 --2- 172.21.15.5:0/1927466647 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x558cd9a5ba80 0x558cd9a5de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:34.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.452+0000 7fbb7017b740 4 client.25376 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:00:34.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.452+0000 7fbb7017b740 1 -- 172.21.15.5:0/1927466647 --> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558cd99b4860 con 0x558cd9a5ba80 2024-09-12T03:00:34.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.453+0000 7fbb66ffd640 1 --2- 172.21.15.5:0/1927466647 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x558cd9a5ba80 0x558cd9a5de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:34.456 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.453+0000 7fbb46ffd640 10 client.25376 ms_handle_connect on v2:172.21.15.148:6838/2317600000 2024-09-12T03:00:34.457 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.453+0000 7fbb66ffd640 1 --2- 172.21.15.5:0/1927466647 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x558cd9a5ba80 0x558cd9a5de70 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d137f9321ec1d16 server_cookie=1bd11d3a8c67c098 in_seq=0 out_seq=0 2024-09-12T03:00:34.457 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] detached_mounts_propagation 2024-09-12T03:00:34.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.454+0000 7fbb46ffd640 1 -- 172.21.15.5:0/1927466647 <== mds.0 v2:172.21.15.148:6838/2317600000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558cd99b4860 con 0x558cd9a5ba80 2024-09-12T03:00:34.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.454+0000 7fbb46ffd640 10 client.25376 handle_command_reply: tid=0 2024-09-12T03:00:34.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.454+0000 7fbb7017b740 1 client.25376 shutdown 2024-09-12T03:00:34.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.454+0000 7fbb7017b740 1 -- 172.21.15.5:0/1927466647 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558cd9a415d0 msgr2=0x558cd9a4bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:34.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.454+0000 7fbb7017b740 1 --2- 172.21.15.5:0/1927466647 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558cd9a415d0 0x558cd9a4bd90 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fbb60009620 tx=0x7fbb6002a8c0 comp rx=0 tx=0).stop 2024-09-12T03:00:34.458 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.454+0000 7fbb7017b740 1 -- 172.21.15.5:0/1927466647 shutdown_connections 2024-09-12T03:00:34.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.454+0000 7fbb7017b740 1 --2- 172.21.15.5:0/1927466647 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x558cd9a5ba80 0x558cd9a5de70 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:34.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.455+0000 7fbb46ffd640 0 client.25376 ms_handle_reset on v2:172.21.15.148:6838/2317600000 2024-09-12T03:00:34.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.455+0000 7fbb7017b740 1 --2- 172.21.15.5:0/1927466647 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cd9a4c370 0x558cd9a49650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:34.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.455+0000 7fbb7017b740 1 --2- 172.21.15.5:0/1927466647 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558cd9a42b40 0x558cd9a49040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:34.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.455+0000 7fbb7017b740 1 --2- 172.21.15.5:0/1927466647 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558cd9a415d0 0x558cd9a4bd90 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:34.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.455+0000 7fbb7017b740 1 -- 172.21.15.5:0/1927466647 >> 172.21.15.5:0/1927466647 conn(0x558cd9a0a310 msgr2=0x558cd9a43930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:34.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.455+0000 7fbb7017b740 1 -- 172.21.15.5:0/1927466647 shutdown_connections 2024-09-12T03:00:34.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.455+0000 7fbb7017b740 1 -- 172.21.15.5:0/1927466647 wait complete. 2024-09-12T03:00:34.459 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.455+0000 7fbb7017b740 20 client.25376 trim_cache size 0 max 16384 2024-09-12T03:00:34.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.475+0000 7fbb7017b740 1 Processor -- start 2024-09-12T03:00:34.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.475+0000 7fbb7017b740 1 -- start start 2024-09-12T03:00:34.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.475+0000 7fbb7017b740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558cd9a415d0 0x558cd9a58160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:34.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.475+0000 7fbb7017b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cd9a42b40 0x558cd9a636c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:34.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.476+0000 7fbb66ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cd9a42b40 0x558cd9a636c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:34.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.476+0000 7fbb66ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cd9a42b40 0x558cd9a636c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56826/0 (socket says 172.21.15.5:56826) 2024-09-12T03:00:34.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.476+0000 7fbb66ffd640 1 -- 172.21.15.5:0/3284969965 learned_addr learned my addr 172.21.15.5:0/3284969965 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:00:34.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.476+0000 7fbb7017b740 1 --2- 172.21.15.5:0/3284969965 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558cd9a4c370 0x558cd9a63c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:34.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.476+0000 7fbb7017b740 1 -- 172.21.15.5:0/3284969965 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbb681b3b80 con 0x558cd9a42b40 2024-09-12T03:00:34.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.476+0000 7fbb7017b740 1 -- 172.21.15.5:0/3284969965 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fbb681b5750 con 0x558cd9a415d0 2024-09-12T03:00:34.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.476+0000 7fbb67fff640 1 --2- 172.21.15.5:0/3284969965 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558cd9a415d0 0x558cd9a58160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:34.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.476+0000 7fbb7017b740 1 -- 172.21.15.5:0/3284969965 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fbb681b4250 con 0x558cd9a4c370 2024-09-12T03:00:34.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.476+0000 7fbb67fff640 1 --2- 172.21.15.5:0/3284969965 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558cd9a4c370 0x558cd9a63c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:34.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.476+0000 7fbb67fff640 1 -- 172.21.15.5:0/3284969965 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558cd9a4c370 msgr2=0x558cd9a63c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:00:34.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.476+0000 7fbb67fff640 1 --2- 172.21.15.5:0/3284969965 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558cd9a4c370 0x558cd9a63c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:34.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.476+0000 7fbb67fff640 1 -- 172.21.15.5:0/3284969965 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cd9a42b40 msgr2=0x558cd9a636c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:00:34.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.476+0000 7fbb67fff640 1 --2- 172.21.15.5:0/3284969965 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cd9a42b40 0x558cd9a636c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:34.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.476+0000 7fbb67fff640 1 -- 172.21.15.5:0/3284969965 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558cd9a4c0c0 con 0x558cd9a415d0 2024-09-12T03:00:34.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.477+0000 7fbb67fff640 1 --2- 172.21.15.5:0/3284969965 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558cd9a415d0 0x558cd9a58160 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fbb60005e30 tx=0x7fbb60004990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f82266e9549f900 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:34.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.477+0000 7fbb4d7fa640 1 -- 172.21.15.5:0/3284969965 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb60058040 con 0x558cd9a415d0 2024-09-12T03:00:34.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.477+0000 7fbb4d7fa640 1 -- 172.21.15.5:0/3284969965 <== mon.2 v2:172.21.15.146:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb60036070 con 0x558cd9a415d0 2024-09-12T03:00:34.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.477+0000 7fbb4d7fa640 1 -- 172.21.15.5:0/3284969965 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb60004b40 con 0x558cd9a415d0 2024-09-12T03:00:34.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.478+0000 7fbb7017b740 1 -- 172.21.15.5:0/3284969965 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558cd9a415d0 msgr2=0x558cd9a58160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:34.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.478+0000 7fbb7017b740 1 --2- 172.21.15.5:0/3284969965 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558cd9a415d0 0x558cd9a58160 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fbb60005e30 tx=0x7fbb60004990 comp rx=0 tx=0).stop 2024-09-12T03:00:34.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.478+0000 7fbb7017b740 1 -- 172.21.15.5:0/3284969965 shutdown_connections 2024-09-12T03:00:34.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.478+0000 7fbb7017b740 1 --2- 172.21.15.5:0/3284969965 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cd9a42b40 0x558cd9a636c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:34.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.478+0000 7fbb7017b740 1 --2- 172.21.15.5:0/3284969965 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558cd9a4c370 0x558cd9a63c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:34.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.478+0000 7fbb7017b740 1 --2- 172.21.15.5:0/3284969965 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558cd9a415d0 0x558cd9a58160 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:34.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.478+0000 7fbb7017b740 1 -- 172.21.15.5:0/3284969965 >> 172.21.15.5:0/3284969965 conn(0x558cd9a0a310 msgr2=0x558cd9a47930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:34.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.479+0000 7fbb7017b740 1 -- 172.21.15.5:0/3284969965 shutdown_connections 2024-09-12T03:00:34.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.479+0000 7fbb7017b740 1 -- 172.21.15.5:0/3284969965 wait complete. 2024-09-12T03:00:34.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.479+0000 7fbb7017b740 1 -- start start 2024-09-12T03:00:34.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.480+0000 7fbb7017b740 1 Processor -- start 2024-09-12T03:00:34.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.480+0000 7fbb7017b740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558cd9a415d0 0x558cd9a542d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:34.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.480+0000 7fbb7017b740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558cd9a42b40 0x558cd9a54810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:34.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.480+0000 7fbb7017b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cd9a4c370 0x558cd9a512a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:34.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.480+0000 7fbb7017b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fbb681b4ff0 con 0x558cd9a4c370 2024-09-12T03:00:34.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.480+0000 7fbb7017b740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fbb681b4920 con 0x558cd9a42b40 2024-09-12T03:00:34.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.480+0000 7fbb7017b740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fbb681b1b60 con 0x558cd9a415d0 2024-09-12T03:00:34.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.480+0000 7fbb7017b740 10 client.0 _ll_register_callbacks cb 0x7fbb60009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:00:34.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.480+0000 7fbb67fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558cd9a415d0 0x558cd9a542d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:34.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.480+0000 7fbb67fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558cd9a415d0 0x558cd9a542d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.5:54336/0 (socket says 172.21.15.5:54336) 2024-09-12T03:00:34.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.480+0000 7fbb67fff640 1 -- 172.21.15.5:0/4188079752 learned_addr learned my addr 172.21.15.5:0/4188079752 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:00:34.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.480+0000 7fbb67fff640 1 --2- 172.21.15.5:0/4188079752 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cd9a4c370 0x558cd9a512a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:34.486 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.481+0000 7fbb67fff640 1 -- 172.21.15.5:0/4188079752 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558cd9a42b40 msgr2=0x558cd9a54810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:00:34.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.481+0000 7fbb67fff640 1 --2- 172.21.15.5:0/4188079752 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558cd9a42b40 0x558cd9a54810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:34.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.481+0000 7fbb67fff640 1 -- 172.21.15.5:0/4188079752 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cd9a4c370 msgr2=0x558cd9a512a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:00:34.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.481+0000 7fbb67fff640 1 --2- 172.21.15.5:0/4188079752 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cd9a4c370 0x558cd9a512a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:34.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.481+0000 7fbb67fff640 1 -- 172.21.15.5:0/4188079752 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558cd9a517e0 con 0x558cd9a415d0 2024-09-12T03:00:34.487 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.481+0000 7fbb67fff640 1 --2- 172.21.15.5:0/4188079752 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558cd9a415d0 0x558cd9a542d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fbb60030920 tx=0x7fbb60031ae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c40458a862a2f21d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:34.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.482+0000 7fbb46ffd640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-09-12T03:00:34.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.482+0000 7fbb46ffd640 1 -- 172.21.15.5:0/4188079752 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb60058070 con 0x558cd9a415d0 2024-09-12T03:00:34.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.482+0000 7fbb46ffd640 1 -- 172.21.15.5:0/4188079752 <== mon.1 v2:172.21.15.148:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fbb60030c10 con 0x558cd9a415d0 2024-09-12T03:00:34.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.482+0000 7fbb7017b740 1 -- 172.21.15.5:0/4188079752 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb60036070 con 0x558cd9a415d0 2024-09-12T03:00:34.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.482+0000 7fbb46ffd640 1 -- 172.21.15.5:0/4188079752 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbb60070050 con 0x558cd9a415d0 2024-09-12T03:00:34.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.482+0000 7fbb7017b740 10 client.25480 fetch_fsmap learned FSMap version 28 2024-09-12T03:00:34.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.482+0000 7fbb7017b740 1 -- 172.21.15.5:0/4188079752 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558cd9ac6db0 con 0x558cd9a415d0 2024-09-12T03:00:34.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.483+0000 7fbb46ffd640 1 -- 172.21.15.5:0/4188079752 <== mon.1 v2:172.21.15.148:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb60003db0 con 0x558cd9a415d0 2024-09-12T03:00:34.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.483+0000 7fbb46ffd640 1 -- 172.21.15.5:0/4188079752 <== mon.1 v2:172.21.15.148:3300/0 5 ==== fsmap(e 28) ==== 8374+0+0 (secure 0 0 0) 0x7fbb6004d070 con 0x558cd9a415d0 2024-09-12T03:00:34.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.483+0000 7fbb7017b740 10 client.25480 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T03:00:34.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.483+0000 7fbb7017b740 10 client.25480 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:00:34.488 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.483+0000 7fbb7017b740 20 client.25480 populate_metadata read hostname 'smithi005' 2024-09-12T03:00:34.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.483+0000 7fbb7017b740 1 --2- 172.21.15.5:0/4188079752 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x558cd9ac7400 0x558cd9ac9820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:34.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.483+0000 7fbb7017b740 4 client.25480 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:00:34.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.483+0000 7fbb7017b740 1 -- 172.21.15.5:0/4188079752 --> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fbb60036070 con 0x558cd9ac7400 2024-09-12T03:00:34.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.483+0000 7fbb66ffd640 1 --2- 172.21.15.5:0/4188079752 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x558cd9ac7400 0x558cd9ac9820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:34.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.484+0000 7fbb46ffd640 10 client.25480 ms_handle_connect on v2:172.21.15.148:6838/2317600000 2024-09-12T03:00:34.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.484+0000 7fbb66ffd640 1 --2- 172.21.15.5:0/4188079752 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x558cd9ac7400 0x558cd9ac9820 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dabaaa343f118187 server_cookie=ef32ea64b2acf60b in_seq=0 out_seq=0 2024-09-12T03:00:34.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.484+0000 7fbb46ffd640 1 -- 172.21.15.5:0/4188079752 <== mds.0 v2:172.21.15.148:6838/2317600000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fbb60036070 con 0x558cd9ac7400 2024-09-12T03:00:34.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.484+0000 7fbb46ffd640 10 client.25480 handle_command_reply: tid=0 2024-09-12T03:00:34.489 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.484+0000 7fbb7017b740 1 client.25480 shutdown 2024-09-12T03:00:34.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.485+0000 7fbb7017b740 1 -- 172.21.15.5:0/4188079752 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558cd9a415d0 msgr2=0x558cd9a542d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:34.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.485+0000 7fbb7017b740 1 --2- 172.21.15.5:0/4188079752 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558cd9a415d0 0x558cd9a542d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fbb60030920 tx=0x7fbb60031ae0 comp rx=0 tx=0).stop 2024-09-12T03:00:34.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.485+0000 7fbb7017b740 1 -- 172.21.15.5:0/4188079752 shutdown_connections 2024-09-12T03:00:34.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.485+0000 7fbb7017b740 1 --2- 172.21.15.5:0/4188079752 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x558cd9ac7400 0x558cd9ac9820 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:34.490 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.485+0000 7fbb7017b740 1 --2- 172.21.15.5:0/4188079752 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cd9a4c370 0x558cd9a512a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:34.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.485+0000 7fbb7017b740 1 --2- 172.21.15.5:0/4188079752 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558cd9a42b40 0x558cd9a54810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:34.491 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-12T03:00:34.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.485+0000 7fbb46ffd640 0 client.25480 ms_handle_reset on v2:172.21.15.148:6838/2317600000 2024-09-12T03:00:34.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.485+0000 7fbb7017b740 1 --2- 172.21.15.5:0/4188079752 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558cd9a415d0 0x558cd9a542d0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:34.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.485+0000 7fbb7017b740 1 -- 172.21.15.5:0/4188079752 >> 172.21.15.5:0/4188079752 conn(0x558cd9a0a310 msgr2=0x558cd9a3cd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:34.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.485+0000 7fbb7017b740 1 -- 172.21.15.5:0/4188079752 shutdown_connections 2024-09-12T03:00:34.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.485+0000 7fbb7017b740 1 -- 172.21.15.5:0/4188079752 wait complete. 2024-09-12T03:00:34.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.485+0000 7fbb7017b740 20 client.25480 trim_cache size 0 max 16384 2024-09-12T03:00:34.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.486+0000 7fbb6dda2640 1 -- 172.21.15.5:0/610213905 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fbb500a2c00 msgr2=0x7fbb500a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:34.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.486+0000 7fbb6dda2640 1 --2- 172.21.15.5:0/610213905 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fbb500a2c00 0x7fbb500a50c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fbb580cd1a0 tx=0x7fbb58009700 comp rx=0 tx=0).stop 2024-09-12T03:00:34.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.486+0000 7fbb6dda2640 1 -- 172.21.15.5:0/610213905 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb68101df0 msgr2=0x7fbb680744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:34.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.486+0000 7fbb6dda2640 1 --2- 172.21.15.5:0/610213905 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb68101df0 0x7fbb680744f0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fbb5800e9f0 tx=0x7fbb5800eeb0 comp rx=0 tx=0).stop 2024-09-12T03:00:34.493 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.487+0000 7fbb6dda2640 1 -- 172.21.15.5:0/610213905 shutdown_connections 2024-09-12T03:00:34.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.487+0000 7fbb6dda2640 1 --2- 172.21.15.5:0/610213905 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fbb68106f20 0x7fbb68074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:34.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.487+0000 7fbb6dda2640 1 --2- 172.21.15.5:0/610213905 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fbb681027a0 0x7fbb68074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:34.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.487+0000 7fbb6dda2640 1 --2- 172.21.15.5:0/610213905 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fbb500a2c00 0x7fbb500a50c0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:34.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.487+0000 7fbb6dda2640 1 --2- 172.21.15.5:0/610213905 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb68101df0 0x7fbb680744f0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:34.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.487+0000 7fbb6dda2640 1 -- 172.21.15.5:0/610213905 >> 172.21.15.5:0/610213905 conn(0x7fbb680fd8b0 msgr2=0x7fbb680feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:34.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.487+0000 7fbb6dda2640 1 -- 172.21.15.5:0/610213905 shutdown_connections 2024-09-12T03:00:34.494 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:34.487+0000 7fbb6dda2640 1 -- 172.21.15.5:0/610213905 wait complete. 2024-09-12T03:00:34.495 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] ext4_resize 2024-09-12T03:00:34.621 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] t_readdir_3 2024-09-12T03:00:34.644 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] splice2pipe 2024-09-12T03:00:34.653 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] uuid_ioctl 2024-09-12T03:00:34.807 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] t_snapshot_deleted_subvolume 2024-09-12T03:00:34.808 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] fiemap-fault 2024-09-12T03:00:34.850 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] fiemap-tester 2024-09-12T03:00:34.973 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] t_stripealign 2024-09-12T03:00:34.973 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] open_by_handle 2024-09-12T03:00:35.060 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] t_mmap_dio 2024-09-12T03:00:35.144 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] xfsfind 2024-09-12T03:00:35.147 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] t_reflink_read_race 2024-09-12T03:00:35.229 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] fssum 2024-09-12T03:00:35.251 INFO:tasks.workunit.client.0.smithi005.stdout:Building log-writes 2024-09-12T03:00:35.257 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] replay-log 2024-09-12T03:00:35.362 INFO:tasks.workunit.client.0.smithi005.stdout:Building perf 2024-09-12T03:00:35.366 INFO:tasks.workunit.client.0.smithi005.stdout:Building aio-dio-regress 2024-09-12T03:00:35.373 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] aiocp 2024-09-12T03:00:35.378 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] aio-dio-append-write-fallocate-race 2024-09-12T03:00:35.548 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] aio-dio-append-write-read-race 2024-09-12T03:00:35.608 INFO:tasks.workunit.client.0.smithi005.stdout:Building vfs 2024-09-12T03:00:35.614 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] vfstest 2024-09-12T03:00:35.665 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:00:35 smithi148 ceph-mon[35090]: pgmap v405: 97 pgs: 97 active+clean; 69 MiB data, 660 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 386 KiB/s wr, 379 op/s 2024-09-12T03:00:35.669 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] mount-idmapped 2024-09-12T03:00:35.671 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:00:35 smithi005 ceph-mon[31197]: pgmap v405: 97 pgs: 97 active+clean; 69 MiB data, 660 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 386 KiB/s wr, 379 op/s 2024-09-12T03:00:35.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:00:35 smithi146 ceph-mon[35011]: pgmap v405: 97 pgs: 97 active+clean; 69 MiB data, 660 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 386 KiB/s wr, 379 op/s 2024-09-12T03:00:35.734 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] aio-dio-cow-race 2024-09-12T03:00:35.764 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] aio-dio-cycle-write 2024-09-12T03:00:35.891 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] aio-dio-eof-race 2024-09-12T03:00:35.945 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] aio-dio-extend-stat 2024-09-12T03:00:36.073 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] aio-dio-fcntl-race 2024-09-12T03:00:36.135 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] aio-dio-hole-filling-race 2024-09-12T03:00:36.136 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] aio-dio-invalidate-failure 2024-09-12T03:00:36.258 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] aio-dio-invalidate-readahead 2024-09-12T03:00:36.299 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] aiodio_sparse2 2024-09-12T03:00:36.309 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] aio-dio-subblock-eof-read 2024-09-12T03:00:36.434 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] aio-dio-write-verify 2024-09-12T03:00:36.471 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] aio-free-ring-with-bogus-nr-pages 2024-09-12T03:00:36.513 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] aio-io-setup-with-nonwritable-context-pointer 2024-09-12T03:00:36.612 INFO:tasks.workunit.client.0.smithi005.stdout: [CC] aio-last-ref-held-by-io 2024-09-12T03:00:36.713 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T03:00:36.713 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T03:00:36.988 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:00:37.365 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:00:37 smithi005 ceph-mon[31197]: pgmap v406: 97 pgs: 97 active+clean; 72 MiB data, 670 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 563 KiB/s wr, 402 op/s 2024-09-12T03:00:37.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.449+0000 7fb7d0b9f640 1 -- 172.21.15.5:0/2761584994 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb7cc102c10 msgr2=0x7fb7cc102ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:37.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.449+0000 7fb7d0b9f640 1 --2- 172.21.15.5:0/2761584994 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb7cc102c10 0x7fb7cc102ff0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc009900 tx=0x7fb7bc02a480 comp rx=0 tx=0).stop 2024-09-12T03:00:37.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.449+0000 7fb7d0b9f640 1 -- 172.21.15.5:0/2761584994 shutdown_connections 2024-09-12T03:00:37.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.449+0000 7fb7d0b9f640 1 --2- 172.21.15.5:0/2761584994 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7cc10bff0 0x7fb7cc10e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:37.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.449+0000 7fb7d0b9f640 1 --2- 172.21.15.5:0/2761584994 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb7cc103530 0x7fb7cc10ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:37.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.449+0000 7fb7d0b9f640 1 --2- 172.21.15.5:0/2761584994 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb7cc102c10 0x7fb7cc102ff0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:37.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.449+0000 7fb7d0b9f640 1 -- 172.21.15.5:0/2761584994 >> 172.21.15.5:0/2761584994 conn(0x7fb7cc0fc610 msgr2=0x7fb7cc0fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:37.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.449+0000 7fb7d0b9f640 1 -- 172.21.15.5:0/2761584994 shutdown_connections 2024-09-12T03:00:37.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.449+0000 7fb7d0b9f640 1 -- 172.21.15.5:0/2761584994 wait complete. 2024-09-12T03:00:37.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.450+0000 7fb7d0b9f640 1 Processor -- start 2024-09-12T03:00:37.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.450+0000 7fb7d0b9f640 1 -- start start 2024-09-12T03:00:37.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.450+0000 7fb7d0b9f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb7cc102c10 0x7fb7cc198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:37.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.450+0000 7fb7d0b9f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7cc103530 0x7fb7cc1987c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:37.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.450+0000 7fb7d0b9f640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb7cc10bff0 0x7fb7cc19cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:37.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.450+0000 7fb7d0b9f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb7cc111dc0 con 0x7fb7cc102c10 2024-09-12T03:00:37.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.450+0000 7fb7d0b9f640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fb7cc111c40 con 0x7fb7cc10bff0 2024-09-12T03:00:37.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.450+0000 7fb7d0b9f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb7cc111f40 con 0x7fb7cc103530 2024-09-12T03:00:37.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.450+0000 7fb7cb7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb7cc102c10 0x7fb7cc198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:37.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.450+0000 7fb7cb7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb7cc102c10 0x7fb7cc198280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56862/0 (socket says 172.21.15.5:56862) 2024-09-12T03:00:37.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.450+0000 7fb7cb7fe640 1 -- 172.21.15.5:0/1870676542 learned_addr learned my addr 172.21.15.5:0/1870676542 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:00:37.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.451+0000 7fb7caffd640 1 --2- 172.21.15.5:0/1870676542 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7cc103530 0x7fb7cc1987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:37.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.451+0000 7fb7cbfff640 1 --2- 172.21.15.5:0/1870676542 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb7cc10bff0 0x7fb7cc19cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:37.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.451+0000 7fb7cb7fe640 1 -- 172.21.15.5:0/1870676542 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7cc103530 msgr2=0x7fb7cc1987c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:37.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.451+0000 7fb7cb7fe640 1 --2- 172.21.15.5:0/1870676542 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7cc103530 0x7fb7cc1987c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:37.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.451+0000 7fb7cb7fe640 1 -- 172.21.15.5:0/1870676542 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb7cc10bff0 msgr2=0x7fb7cc19cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:37.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.451+0000 7fb7cb7fe640 1 --2- 172.21.15.5:0/1870676542 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb7cc10bff0 0x7fb7cc19cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:37.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.451+0000 7fb7cb7fe640 1 -- 172.21.15.5:0/1870676542 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7cc19d230 con 0x7fb7cc102c10 2024-09-12T03:00:37.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.451+0000 7fb7cbfff640 1 --2- 172.21.15.5:0/1870676542 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb7cc10bff0 0x7fb7cc19cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:00:37.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.451+0000 7fb7cb7fe640 1 --2- 172.21.15.5:0/1870676542 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb7cc102c10 0x7fb7cc198280 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc006010 tx=0x7fb7bc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:37.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.451+0000 7fb7c8ff9640 1 -- 172.21.15.5:0/1870676542 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7bc03f030 con 0x7fb7cc102c10 2024-09-12T03:00:37.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.451+0000 7fb7c8ff9640 1 -- 172.21.15.5:0/1870676542 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7bc003ce0 con 0x7fb7cc102c10 2024-09-12T03:00:37.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.451+0000 7fb7c8ff9640 1 -- 172.21.15.5:0/1870676542 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7bc035070 con 0x7fb7cc102c10 2024-09-12T03:00:37.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.451+0000 7fb7d0b9f640 1 -- 172.21.15.5:0/1870676542 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7cc19d4c0 con 0x7fb7cc102c10 2024-09-12T03:00:37.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.451+0000 7fb7d0b9f640 1 -- 172.21.15.5:0/1870676542 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7cc19d950 con 0x7fb7cc102c10 2024-09-12T03:00:37.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.454+0000 7fb7d41da740 1 Processor -- start 2024-09-12T03:00:37.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.454+0000 7fb7d41da740 1 -- start start 2024-09-12T03:00:37.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.454+0000 7fb7d41da740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559155b7cc70 0x559155b7d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:37.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.454+0000 7fb7d41da740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559155b7d590 0x559155b8be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:37.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.454+0000 7fb7d41da740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559155b8c360 0x559155b8e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:37.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.454+0000 7fb7d41da740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559155ae2280 con 0x559155b7cc70 2024-09-12T03:00:37.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.454+0000 7fb7d41da740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x559155a89b10 con 0x559155b7d590 2024-09-12T03:00:37.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.454+0000 7fb7d41da740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x559155ade240 con 0x559155b8c360 2024-09-12T03:00:37.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.454+0000 7fb7cbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559155b7cc70 0x559155b7d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:37.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.454+0000 7fb7c8ff9640 1 -- 172.21.15.5:0/1870676542 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fb7bc042030 con 0x7fb7cc102c10 2024-09-12T03:00:37.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.454+0000 7fb7cbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559155b7cc70 0x559155b7d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56878/0 (socket says 172.21.15.5:56878) 2024-09-12T03:00:37.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.454+0000 7fb7cbfff640 1 -- 172.21.15.5:0/3896205482 learned_addr learned my addr 172.21.15.5:0/3896205482 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:00:37.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.455+0000 7fb7caffd640 1 --2- 172.21.15.5:0/3896205482 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559155b8c360 0x559155b8e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:37.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.455+0000 7fb7cbfff640 1 -- 172.21.15.5:0/3896205482 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559155b8c360 msgr2=0x559155b8e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:00:37.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.455+0000 7fb7cbfff640 1 --2- 172.21.15.5:0/3896205482 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559155b8c360 0x559155b8e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:37.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.455+0000 7fb7cbfff640 1 -- 172.21.15.5:0/3896205482 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559155b7d590 msgr2=0x559155b8be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:00:37.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.455+0000 7fb7cb7fe640 1 --2- 172.21.15.5:0/3896205482 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559155b7d590 0x559155b8be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:37.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.455+0000 7fb7cbfff640 1 --2- 172.21.15.5:0/3896205482 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559155b7d590 0x559155b8be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:37.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.455+0000 7fb7cbfff640 1 -- 172.21.15.5:0/3896205482 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559155a961f0 con 0x559155b7cc70 2024-09-12T03:00:37.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.455+0000 7fb7cb7fe640 1 --2- 172.21.15.5:0/3896205482 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559155b7d590 0x559155b8be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:00:37.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.455+0000 7fb7cbfff640 1 --2- 172.21.15.5:0/3896205482 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559155b7cc70 0x559155b7d050 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fb7c4009f30 tx=0x7fb7c4010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83ee66a521726d91 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:37.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.455+0000 7fb7b1ffb640 1 -- 172.21.15.5:0/3896205482 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7c4016070 con 0x559155b7cc70 2024-09-12T03:00:37.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.455+0000 7fb7c8ff9640 1 --2- 172.21.15.5:0/1870676542 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fb7b40a2b80 0x7fb7b40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:37.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.455+0000 7fb7b1ffb640 1 -- 172.21.15.5:0/3896205482 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7c401bb40 con 0x559155b7cc70 2024-09-12T03:00:37.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.455+0000 7fb7b1ffb640 1 -- 172.21.15.5:0/3896205482 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7c4012c50 con 0x559155b7cc70 2024-09-12T03:00:37.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.456+0000 7fb7cb7fe640 1 --2- 172.21.15.5:0/1870676542 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fb7b40a2b80 0x7fb7b40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:37.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.456+0000 7fb7c8ff9640 1 -- 172.21.15.5:0/1870676542 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fb7bc0ef240 con 0x7fb7cc102c10 2024-09-12T03:00:37.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.456+0000 7fb7d41da740 1 -- 172.21.15.5:0/3896205482 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559155b7cc70 msgr2=0x559155b7d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:37.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.456+0000 7fb7cb7fe640 1 --2- 172.21.15.5:0/1870676542 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fb7b40a2b80 0x7fb7b40a5040 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc0ed0c0 tx=0x7fb7bc02a480 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:37.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.456+0000 7fb7d41da740 1 --2- 172.21.15.5:0/3896205482 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559155b7cc70 0x559155b7d050 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fb7c4009f30 tx=0x7fb7c4010800 comp rx=0 tx=0).stop 2024-09-12T03:00:37.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.456+0000 7fb7d41da740 1 -- 172.21.15.5:0/3896205482 shutdown_connections 2024-09-12T03:00:37.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.456+0000 7fb7d41da740 1 --2- 172.21.15.5:0/3896205482 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559155b8c360 0x559155b8e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:37.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.456+0000 7fb7d41da740 1 --2- 172.21.15.5:0/3896205482 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559155b7d590 0x559155b8be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:37.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.456+0000 7fb7d41da740 1 --2- 172.21.15.5:0/3896205482 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559155b7cc70 0x559155b7d050 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:37.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.456+0000 7fb7d41da740 1 -- 172.21.15.5:0/3896205482 >> 172.21.15.5:0/3896205482 conn(0x559155b79300 msgr2=0x559155b796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:37.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.456+0000 7fb7d41da740 1 -- 172.21.15.5:0/3896205482 shutdown_connections 2024-09-12T03:00:37.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.456+0000 7fb7d41da740 1 -- 172.21.15.5:0/3896205482 wait complete. 2024-09-12T03:00:37.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.457+0000 7fb7d41da740 1 -- start start 2024-09-12T03:00:37.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.457+0000 7fb7d41da740 1 Processor -- start 2024-09-12T03:00:37.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.457+0000 7fb7d41da740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559155b7cc70 0x559155b89290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:37.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.457+0000 7fb7d41da740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559155b7d590 0x559155b897d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:37.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.457+0000 7fb7d41da740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559155b8c360 0x559155b89d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:37.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.457+0000 7fb7d41da740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559155b8ef90 con 0x559155b8c360 2024-09-12T03:00:37.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.457+0000 7fb7d41da740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x559155a9cf10 con 0x559155b7cc70 2024-09-12T03:00:37.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.457+0000 7fb7d41da740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x559155b8f110 con 0x559155b7d590 2024-09-12T03:00:37.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.457+0000 7fb7d41da740 10 client.0 _ll_register_callbacks cb 0x559155a65a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:00:37.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.457+0000 7fb7cbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559155b8c360 0x559155b89d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:37.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.457+0000 7fb7cbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559155b8c360 0x559155b89d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56888/0 (socket says 172.21.15.5:56888) 2024-09-12T03:00:37.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.457+0000 7fb7cbfff640 1 -- 172.21.15.5:0/591476014 learned_addr learned my addr 172.21.15.5:0/591476014 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:00:37.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.457+0000 7fb7caffd640 1 --2- 172.21.15.5:0/591476014 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559155b7d590 0x559155b897d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:37.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.458+0000 7fb7cbfff640 1 --2- 172.21.15.5:0/591476014 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559155b7cc70 0x559155b89290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:37.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.458+0000 7fb7cbfff640 1 -- 172.21.15.5:0/591476014 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559155b7d590 msgr2=0x559155b897d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:00:37.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.458+0000 7fb7cbfff640 1 --2- 172.21.15.5:0/591476014 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559155b7d590 0x559155b897d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:37.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.458+0000 7fb7cbfff640 1 -- 172.21.15.5:0/591476014 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559155b7cc70 msgr2=0x559155b89290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:00:37.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.458+0000 7fb7cbfff640 1 --2- 172.21.15.5:0/591476014 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559155b7cc70 0x559155b89290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:37.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.458+0000 7fb7cbfff640 1 -- 172.21.15.5:0/591476014 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591558536a0 con 0x559155b8c360 2024-09-12T03:00:37.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.458+0000 7fb7cbfff640 1 --2- 172.21.15.5:0/591476014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559155b8c360 0x559155b89d10 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fb7c401bf20 tx=0x7fb7c4012890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4881e7c36f2759fc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:37.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.458+0000 7fb7aaffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T03:00:37.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.458+0000 7fb7aaffd640 1 -- 172.21.15.5:0/591476014 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7c4016040 con 0x559155b8c360 2024-09-12T03:00:37.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.458+0000 7fb7d41da740 1 -- 172.21.15.5:0/591476014 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7c401bb40 con 0x559155b8c360 2024-09-12T03:00:37.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.458+0000 7fb7aaffd640 1 -- 172.21.15.5:0/591476014 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fb7c4004a40 con 0x559155b8c360 2024-09-12T03:00:37.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.458+0000 7fb7aaffd640 1 -- 172.21.15.5:0/591476014 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb7c4031050 con 0x559155b8c360 2024-09-12T03:00:37.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.458+0000 7fb7d41da740 10 client.15963 fetch_fsmap learned FSMap version 28 2024-09-12T03:00:37.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.458+0000 7fb7d41da740 1 -- 172.21.15.5:0/591476014 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559155b9abb0 con 0x559155b8c360 2024-09-12T03:00:37.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.459+0000 7fb7aaffd640 1 -- 172.21.15.5:0/591476014 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7c401bb40 con 0x559155b8c360 2024-09-12T03:00:37.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.459+0000 7fb7aaffd640 1 -- 172.21.15.5:0/591476014 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8374+0+0 (secure 0 0 0) 0x7fb7c400f040 con 0x559155b8c360 2024-09-12T03:00:37.510 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.459+0000 7fb7d41da740 10 client.15963 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T03:00:37.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.459+0000 7fb7d41da740 10 client.15963 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:00:37.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.459+0000 7fb7d41da740 20 client.15963 populate_metadata read hostname 'smithi005' 2024-09-12T03:00:37.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.459+0000 7fb7d41da740 1 --2- 172.21.15.5:0/591476014 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x559155b9bf00 0x559155b9e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:37.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.459+0000 7fb7d41da740 4 client.15963 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:00:37.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.459+0000 7fb7d41da740 1 -- 172.21.15.5:0/591476014 --> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559155aef7f0 con 0x559155b9bf00 2024-09-12T03:00:37.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.459+0000 7fb7caffd640 1 --2- 172.21.15.5:0/591476014 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x559155b9bf00 0x559155b9e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:37.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.460+0000 7fb7caffd640 1 --2- 172.21.15.5:0/591476014 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x559155b9bf00 0x559155b9e2f0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5786d270caaa080b server_cookie=fe2b5970275042df in_seq=0 out_seq=0 2024-09-12T03:00:37.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.460+0000 7fb7aaffd640 10 client.15963 ms_handle_connect on v2:172.21.15.148:6838/2317600000 2024-09-12T03:00:37.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.460+0000 7fb7aaffd640 1 -- 172.21.15.5:0/591476014 <== mds.0 v2:172.21.15.148:6838/2317600000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559155aef7f0 con 0x559155b9bf00 2024-09-12T03:00:37.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.460+0000 7fb7aaffd640 10 client.15963 handle_command_reply: tid=0 2024-09-12T03:00:37.511 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.460+0000 7fb7d41da740 1 client.15963 shutdown 2024-09-12T03:00:37.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.460+0000 7fb7d41da740 1 -- 172.21.15.5:0/591476014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559155b8c360 msgr2=0x559155b89d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:37.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.460+0000 7fb7d41da740 1 --2- 172.21.15.5:0/591476014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559155b8c360 0x559155b89d10 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fb7c401bf20 tx=0x7fb7c4012890 comp rx=0 tx=0).stop 2024-09-12T03:00:37.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.460+0000 7fb7d41da740 1 -- 172.21.15.5:0/591476014 shutdown_connections 2024-09-12T03:00:37.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.461+0000 7fb7d41da740 1 --2- 172.21.15.5:0/591476014 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x559155b9bf00 0x559155b9e2f0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:37.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.461+0000 7fb7aaffd640 0 client.15963 ms_handle_reset on v2:172.21.15.148:6838/2317600000 2024-09-12T03:00:37.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.461+0000 7fb7d41da740 1 --2- 172.21.15.5:0/591476014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559155b8c360 0x559155b89d10 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:37.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.461+0000 7fb7d41da740 1 --2- 172.21.15.5:0/591476014 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559155b7d590 0x559155b897d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:37.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.461+0000 7fb7d41da740 1 --2- 172.21.15.5:0/591476014 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559155b7cc70 0x559155b89290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:37.512 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.461+0000 7fb7d41da740 1 -- 172.21.15.5:0/591476014 >> 172.21.15.5:0/591476014 conn(0x559155b79300 msgr2=0x559155b7f060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:37.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.461+0000 7fb7d41da740 1 -- 172.21.15.5:0/591476014 shutdown_connections 2024-09-12T03:00:37.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.461+0000 7fb7d41da740 1 -- 172.21.15.5:0/591476014 wait complete. 2024-09-12T03:00:37.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.461+0000 7fb7d41da740 20 client.15963 trim_cache size 0 max 16384 2024-09-12T03:00:37.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.488+0000 7fb7d41da740 1 Processor -- start 2024-09-12T03:00:37.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.489+0000 7fb7d41da740 1 -- start start 2024-09-12T03:00:37.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.489+0000 7fb7d41da740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559155b7cc70 0x559155b7ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:37.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.489+0000 7fb7d41da740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559155b7d590 0x559155ba3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:37.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.489+0000 7fb7d41da740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559155b8c360 0x559155ba3e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:37.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.489+0000 7fb7d41da740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb7cc1ab1d0 con 0x559155b8c360 2024-09-12T03:00:37.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.489+0000 7fb7d41da740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fb7cc1acda0 con 0x559155b7cc70 2024-09-12T03:00:37.513 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.489+0000 7fb7d41da740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb7cc1ab8a0 con 0x559155b7d590 2024-09-12T03:00:37.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.489+0000 7fb7cbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559155b8c360 0x559155ba3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:37.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.489+0000 7fb7cbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559155b8c360 0x559155ba3e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56892/0 (socket says 172.21.15.5:56892) 2024-09-12T03:00:37.514 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.489+0000 7fb7caffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559155b7d590 0x559155ba3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:37.514 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-12T03:00:37.514 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-09-12T03:00:37.514 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "e6a90fb8-e6ec-4ee2-959a-e5e280e9b9cf", 2024-09-12T03:00:37.514 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-09-12T03:00:37.515 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-12T03:00:37.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.489+0000 7fb7cbfff640 1 -- 172.21.15.5:0/1230617147 learned_addr learned my addr 172.21.15.5:0/1230617147 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:00:37.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.489+0000 7fb7cbfff640 1 --2- 172.21.15.5:0/1230617147 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559155b7cc70 0x559155b7ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:37.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.489+0000 7fb7cbfff640 1 -- 172.21.15.5:0/1230617147 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559155b7d590 msgr2=0x559155ba3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:00:37.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.489+0000 7fb7cbfff640 1 --2- 172.21.15.5:0/1230617147 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559155b7d590 0x559155ba3920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:37.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.489+0000 7fb7cbfff640 1 -- 172.21.15.5:0/1230617147 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559155b7cc70 msgr2=0x559155b7ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:00:37.515 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.489+0000 7fb7cbfff640 1 --2- 172.21.15.5:0/1230617147 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559155b7cc70 0x559155b7ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:37.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.489+0000 7fb7cbfff640 1 -- 172.21.15.5:0/1230617147 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559155b877e0 con 0x559155b8c360 2024-09-12T03:00:37.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.490+0000 7fb7cbfff640 1 --2- 172.21.15.5:0/1230617147 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559155b8c360 0x559155ba3e60 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fb7c4005d70 tx=0x7fb7c4010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd5c8dfdd96d78f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:37.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.490+0000 7fb7b17fa640 1 -- 172.21.15.5:0/1230617147 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7c4016040 con 0x559155b8c360 2024-09-12T03:00:37.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.490+0000 7fb7b17fa640 1 -- 172.21.15.5:0/1230617147 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7c40058e0 con 0x559155b8c360 2024-09-12T03:00:37.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.490+0000 7fb7b17fa640 1 -- 172.21.15.5:0/1230617147 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7c4004e40 con 0x559155b8c360 2024-09-12T03:00:37.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.491+0000 7fb7d41da740 1 -- 172.21.15.5:0/1230617147 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559155b8c360 msgr2=0x559155ba3e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:37.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.491+0000 7fb7d41da740 1 --2- 172.21.15.5:0/1230617147 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559155b8c360 0x559155ba3e60 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fb7c4005d70 tx=0x7fb7c4010800 comp rx=0 tx=0).stop 2024-09-12T03:00:37.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.491+0000 7fb7d41da740 1 -- 172.21.15.5:0/1230617147 shutdown_connections 2024-09-12T03:00:37.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.491+0000 7fb7d41da740 1 --2- 172.21.15.5:0/1230617147 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559155b8c360 0x559155ba3e60 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:37.516 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.491+0000 7fb7d41da740 1 --2- 172.21.15.5:0/1230617147 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559155b7d590 0x559155ba3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:37.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.491+0000 7fb7d41da740 1 --2- 172.21.15.5:0/1230617147 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559155b7cc70 0x559155b7ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:37.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.491+0000 7fb7d41da740 1 -- 172.21.15.5:0/1230617147 >> 172.21.15.5:0/1230617147 conn(0x559155b79300 msgr2=0x559155b7e840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:37.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.491+0000 7fb7d41da740 1 -- 172.21.15.5:0/1230617147 shutdown_connections 2024-09-12T03:00:37.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.491+0000 7fb7d41da740 1 -- 172.21.15.5:0/1230617147 wait complete. 2024-09-12T03:00:37.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.491+0000 7fb7d41da740 1 -- start start 2024-09-12T03:00:37.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.491+0000 7fb7d41da740 1 Processor -- start 2024-09-12T03:00:37.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.492+0000 7fb7d41da740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559155b7cc70 0x559155b903b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:37.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.492+0000 7fb7d41da740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559155b7d590 0x559155b908f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:37.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.492+0000 7fb7d41da740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559155b8c360 0x559155b90e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:37.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.492+0000 7fb7d41da740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb7cc1ac640 con 0x559155b8c360 2024-09-12T03:00:37.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.492+0000 7fb7d41da740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fb7cc1abf70 con 0x559155b7d590 2024-09-12T03:00:37.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.492+0000 7fb7d41da740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb7cc1a91b0 con 0x559155b7cc70 2024-09-12T03:00:37.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.492+0000 7fb7d41da740 10 client.0 _ll_register_callbacks cb 0x7fb7c000b770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:00:37.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.492+0000 7fb7cbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559155b8c360 0x559155b90e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:37.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.492+0000 7fb7cbfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559155b8c360 0x559155b90e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56904/0 (socket says 172.21.15.5:56904) 2024-09-12T03:00:37.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.492+0000 7fb7cbfff640 1 -- 172.21.15.5:0/2125718235 learned_addr learned my addr 172.21.15.5:0/2125718235 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:00:37.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.492+0000 7fb7cbfff640 1 --2- 172.21.15.5:0/2125718235 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559155b7cc70 0x559155b903b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:37.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.492+0000 7fb7caffd640 1 --2- 172.21.15.5:0/2125718235 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559155b7d590 0x559155b908f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:37.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.492+0000 7fb7cbfff640 1 -- 172.21.15.5:0/2125718235 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559155b7cc70 msgr2=0x559155b903b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:00:37.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.492+0000 7fb7cbfff640 1 --2- 172.21.15.5:0/2125718235 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559155b7cc70 0x559155b903b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:37.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.492+0000 7fb7cbfff640 1 -- 172.21.15.5:0/2125718235 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559155b7d590 msgr2=0x559155b908f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:00:37.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.492+0000 7fb7cbfff640 1 --2- 172.21.15.5:0/2125718235 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559155b7d590 0x559155b908f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:37.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.492+0000 7fb7cbfff640 1 -- 172.21.15.5:0/2125718235 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559155c05be0 con 0x559155b8c360 2024-09-12T03:00:37.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.492+0000 7fb7cbfff640 1 --2- 172.21.15.5:0/2125718235 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559155b8c360 0x559155b90e30 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fb7c4009920 tx=0x7fb7c4009bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=128376a51fb70f56 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:37.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.492+0000 7fb7aaffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T03:00:37.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.493+0000 7fb7aaffd640 1 -- 172.21.15.5:0/2125718235 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7c4016030 con 0x559155b8c360 2024-09-12T03:00:37.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.493+0000 7fb7d41da740 1 -- 172.21.15.5:0/2125718235 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7c40058e0 con 0x559155b8c360 2024-09-12T03:00:37.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.493+0000 7fb7aaffd640 1 -- 172.21.15.5:0/2125718235 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fb7c4013dd0 con 0x559155b8c360 2024-09-12T03:00:37.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.493+0000 7fb7aaffd640 1 -- 172.21.15.5:0/2125718235 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb7c404c050 con 0x559155b8c360 2024-09-12T03:00:37.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.493+0000 7fb7d41da740 10 client.15969 fetch_fsmap learned FSMap version 28 2024-09-12T03:00:37.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.493+0000 7fb7d41da740 1 -- 172.21.15.5:0/2125718235 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559155c06ea0 con 0x559155b8c360 2024-09-12T03:00:37.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.493+0000 7fb7aaffd640 1 -- 172.21.15.5:0/2125718235 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7c40058e0 con 0x559155b8c360 2024-09-12T03:00:37.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.493+0000 7fb7aaffd640 1 -- 172.21.15.5:0/2125718235 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8374+0+0 (secure 0 0 0) 0x7fb7c4012760 con 0x559155b8c360 2024-09-12T03:00:37.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.493+0000 7fb7d41da740 10 client.15969 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T03:00:37.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.493+0000 7fb7d41da740 10 client.15969 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:00:37.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.493+0000 7fb7d41da740 20 client.15969 populate_metadata read hostname 'smithi005' 2024-09-12T03:00:37.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.493+0000 7fb7d41da740 1 --2- 172.21.15.5:0/2125718235 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x559155b9bf00 0x559155c095e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:37.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.493+0000 7fb7d41da740 4 client.15969 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T03:00:37.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.493+0000 7fb7d41da740 1 -- 172.21.15.5:0/2125718235 --> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x559155ae9d00 con 0x559155b9bf00 2024-09-12T03:00:37.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.494+0000 7fb7caffd640 1 --2- 172.21.15.5:0/2125718235 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x559155b9bf00 0x559155c095e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:37.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.494+0000 7fb7aaffd640 10 client.15969 ms_handle_connect on v2:172.21.15.148:6838/2317600000 2024-09-12T03:00:37.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.494+0000 7fb7caffd640 1 --2- 172.21.15.5:0/2125718235 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x559155b9bf00 0x559155c095e0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f744c5adab167b1 server_cookie=8641594ebf995172 in_seq=0 out_seq=0 2024-09-12T03:00:37.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.495+0000 7fb7aaffd640 1 -- 172.21.15.5:0/2125718235 <== mds.0 v2:172.21.15.148:6838/2317600000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x559155ae9d00 con 0x559155b9bf00 2024-09-12T03:00:37.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.495+0000 7fb7aaffd640 10 client.15969 handle_command_reply: tid=0 2024-09-12T03:00:37.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.495+0000 7fb7d41da740 1 client.15969 shutdown 2024-09-12T03:00:37.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.495+0000 7fb7d41da740 1 -- 172.21.15.5:0/2125718235 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559155b8c360 msgr2=0x559155b90e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:37.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.495+0000 7fb7d41da740 1 --2- 172.21.15.5:0/2125718235 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559155b8c360 0x559155b90e30 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fb7c4009920 tx=0x7fb7c4009bf0 comp rx=0 tx=0).stop 2024-09-12T03:00:37.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.495+0000 7fb7d41da740 1 -- 172.21.15.5:0/2125718235 shutdown_connections 2024-09-12T03:00:37.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.495+0000 7fb7d41da740 1 --2- 172.21.15.5:0/2125718235 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x559155b9bf00 0x559155c095e0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:37.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.495+0000 7fb7d41da740 1 --2- 172.21.15.5:0/2125718235 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559155b8c360 0x559155b90e30 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:37.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.495+0000 7fb7d41da740 1 --2- 172.21.15.5:0/2125718235 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559155b7d590 0x559155b908f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:37.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.495+0000 7fb7d41da740 1 --2- 172.21.15.5:0/2125718235 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559155b7cc70 0x559155b903b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:37.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.495+0000 7fb7aaffd640 0 client.15969 ms_handle_reset on v2:172.21.15.148:6838/2317600000 2024-09-12T03:00:37.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.496+0000 7fb7d41da740 1 -- 172.21.15.5:0/2125718235 >> 172.21.15.5:0/2125718235 conn(0x559155b79300 msgr2=0x559155b89a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:37.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.496+0000 7fb7d41da740 1 -- 172.21.15.5:0/2125718235 shutdown_connections 2024-09-12T03:00:37.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.496+0000 7fb7d41da740 1 -- 172.21.15.5:0/2125718235 wait complete. 2024-09-12T03:00:37.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.496+0000 7fb7d41da740 20 client.15969 trim_cache size 0 max 16384 2024-09-12T03:00:37.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.496+0000 7fb7d0b9f640 1 -- 172.21.15.5:0/1870676542 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fb7b40a2b80 msgr2=0x7fb7b40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:37.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.496+0000 7fb7d0b9f640 1 --2- 172.21.15.5:0/1870676542 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fb7b40a2b80 0x7fb7b40a5040 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc0ed0c0 tx=0x7fb7bc02a480 comp rx=0 tx=0).stop 2024-09-12T03:00:37.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.496+0000 7fb7d0b9f640 1 -- 172.21.15.5:0/1870676542 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb7cc102c10 msgr2=0x7fb7cc198280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:37.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.496+0000 7fb7d0b9f640 1 --2- 172.21.15.5:0/1870676542 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb7cc102c10 0x7fb7cc198280 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc006010 tx=0x7fb7bc0042f0 comp rx=0 tx=0).stop 2024-09-12T03:00:37.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.496+0000 7fb7d0b9f640 1 -- 172.21.15.5:0/1870676542 shutdown_connections 2024-09-12T03:00:37.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.496+0000 7fb7d0b9f640 1 --2- 172.21.15.5:0/1870676542 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb7cc10bff0 0x7fb7cc19cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:37.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.496+0000 7fb7d0b9f640 1 --2- 172.21.15.5:0/1870676542 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb7cc103530 0x7fb7cc1987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:37.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.496+0000 7fb7d0b9f640 1 --2- 172.21.15.5:0/1870676542 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fb7b40a2b80 0x7fb7b40a5040 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:37.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.496+0000 7fb7d0b9f640 1 --2- 172.21.15.5:0/1870676542 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb7cc102c10 0x7fb7cc198280 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:37.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.496+0000 7fb7d0b9f640 1 -- 172.21.15.5:0/1870676542 >> 172.21.15.5:0/1870676542 conn(0x7fb7cc0fc610 msgr2=0x7fb7cc0fd2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:37.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.497+0000 7fb7d0b9f640 1 -- 172.21.15.5:0/1870676542 shutdown_connections 2024-09-12T03:00:37.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:37.497+0000 7fb7d0b9f640 1 -- 172.21.15.5:0/1870676542 wait complete. 2024-09-12T03:00:37.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:00:37 smithi148 ceph-mon[35090]: pgmap v406: 97 pgs: 97 active+clean; 72 MiB data, 670 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 563 KiB/s wr, 402 op/s 2024-09-12T03:00:37.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:00:37 smithi146 ceph-mon[35011]: pgmap v406: 97 pgs: 97 active+clean; 72 MiB data, 670 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 563 KiB/s wr, 402 op/s 2024-09-12T03:00:37.852 INFO:tasks.workunit.client.0.smithi005.stderr:+ cd .. 2024-09-12T03:00:37.853 INFO:tasks.workunit.client.0.smithi005.stderr:+ cp xfstests-dev/ltp/fsx . 2024-09-12T03:00:37.856 INFO:tasks.workunit.client.0.smithi005.stderr:+ OPTIONS=-z 2024-09-12T03:00:37.856 INFO:tasks.workunit.client.0.smithi005.stderr:+ ./fsx -z 1MB -N 50000 -p 10000 -l 1048576 2024-09-12T03:00:37.856 INFO:tasks.workunit.client.0.smithi005.stdout:Seed set to 1 2024-09-12T03:00:37.880 INFO:tasks.workunit.client.0.smithi005.stderr:main: filesystem does not support fallocate mode 0, disabling! 2024-09-12T03:00:37.880 INFO:tasks.workunit.client.0.smithi005.stderr:main: filesystem does not support fallocate mode FALLOC_FL_KEEP_SIZE, disabling! 2024-09-12T03:00:37.882 INFO:tasks.workunit.client.0.smithi005.stderr:main: filesystem does not support fallocate mode FALLOC_FL_COLLAPSE_RANGE, disabling! 2024-09-12T03:00:37.882 INFO:tasks.workunit.client.0.smithi005.stderr:main: filesystem does not support fallocate mode FALLOC_FL_INSERT_RANGE, disabling! 2024-09-12T03:00:37.882 INFO:tasks.workunit.client.0.smithi005.stderr:main: filesystem does not support clone range, disabling! 2024-09-12T03:00:37.885 INFO:tasks.workunit.client.0.smithi005.stderr:main: filesystem does not support dedupe range, disabling! 2024-09-12T03:00:37.887 INFO:tasks.workunit.client.0.smithi005.stdout:skipping zero size read 2024-09-12T03:00:37.888 INFO:tasks.workunit.client.0.smithi005.stderr:main: filesystem does not support exchange range, disabling! 2024-09-12T03:00:37.888 INFO:tasks.workunit.client.0.smithi005.stdout:skipping zero size read 2024-09-12T03:00:37.888 INFO:tasks.workunit.client.0.smithi005.stdout:truncating to largest ever: 0x81d54 2024-09-12T03:00:37.893 INFO:tasks.workunit.client.0.smithi005.stdout:copying to largest ever: 0xb86d7 2024-09-12T03:00:37.900 INFO:tasks.workunit.client.0.smithi005.stdout:copying to largest ever: 0xe627e 2024-09-12T03:00:37.940 INFO:tasks.workunit.client.0.smithi005.stdout:truncating to largest ever: 0xf3194 2024-09-12T03:00:37.964 INFO:tasks.workunit.client.0.smithi005.stdout:copying to largest ever: 0xf3ceb 2024-09-12T03:00:37.970 INFO:tasks.workunit.client.0.smithi005.stdout:truncating to largest ever: 0xf47e4 2024-09-12T03:00:38.064 INFO:tasks.workunit.client.0.smithi005.stdout:copying to largest ever: 0xfaccc 2024-09-12T03:00:38.247 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T03:00:38.418 INFO:tasks.workunit.client.0.smithi005.stdout:copying to largest ever: 0xfe423 2024-09-12T03:00:38.519 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:00:38.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:00:38 smithi005 ceph-mon[31197]: scrub queued for path: / 2024-09-12T03:00:38.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:00:38 smithi005 ceph-mon[31197]: scrub summary: idle+waiting paths [/] 2024-09-12T03:00:38.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:00:38 smithi005 ceph-mon[31197]: scrub summary: active paths [/] 2024-09-12T03:00:38.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:00:38 smithi005 ceph-mon[31197]: scrub summary: active paths [/] 2024-09-12T03:00:38.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:00:38 smithi005 ceph-mon[31197]: scrub summary: idle+waiting paths [/] 2024-09-12T03:00:38.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:00:38 smithi005 ceph-mon[31197]: scrub summary: active paths [/] 2024-09-12T03:00:38.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:00:38 smithi005 ceph-mon[31197]: scrub summary: idle+waiting paths [/] 2024-09-12T03:00:38.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:00:38 smithi005 ceph-mon[31197]: scrub summary: active paths [/] 2024-09-12T03:00:38.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:00:38 smithi005 ceph-mon[31197]: scrub summary: idle+waiting paths [/] 2024-09-12T03:00:38.665 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:00:38 smithi148 ceph-mon[35090]: scrub queued for path: / 2024-09-12T03:00:38.665 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:00:38 smithi148 ceph-mon[35090]: scrub summary: idle+waiting paths [/] 2024-09-12T03:00:38.665 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:00:38 smithi148 ceph-mon[35090]: scrub summary: active paths [/] 2024-09-12T03:00:38.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:00:38 smithi148 ceph-mon[35090]: scrub summary: active paths [/] 2024-09-12T03:00:38.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:00:38 smithi148 ceph-mon[35090]: scrub summary: idle+waiting paths [/] 2024-09-12T03:00:38.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:00:38 smithi148 ceph-mon[35090]: scrub summary: active paths [/] 2024-09-12T03:00:38.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:00:38 smithi148 ceph-mon[35090]: scrub summary: idle+waiting paths [/] 2024-09-12T03:00:38.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:00:38 smithi148 ceph-mon[35090]: scrub summary: active paths [/] 2024-09-12T03:00:38.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:00:38 smithi148 ceph-mon[35090]: scrub summary: idle+waiting paths [/] 2024-09-12T03:00:38.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:00:38 smithi146 ceph-mon[35011]: scrub queued for path: / 2024-09-12T03:00:38.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:00:38 smithi146 ceph-mon[35011]: scrub summary: idle+waiting paths [/] 2024-09-12T03:00:38.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:00:38 smithi146 ceph-mon[35011]: scrub summary: active paths [/] 2024-09-12T03:00:38.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:00:38 smithi146 ceph-mon[35011]: scrub summary: active paths [/] 2024-09-12T03:00:38.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:00:38 smithi146 ceph-mon[35011]: scrub summary: idle+waiting paths [/] 2024-09-12T03:00:38.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:00:38 smithi146 ceph-mon[35011]: scrub summary: active paths [/] 2024-09-12T03:00:38.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:00:38 smithi146 ceph-mon[35011]: scrub summary: idle+waiting paths [/] 2024-09-12T03:00:38.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:00:38 smithi146 ceph-mon[35011]: scrub summary: active paths [/] 2024-09-12T03:00:38.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:00:38 smithi146 ceph-mon[35011]: scrub summary: idle+waiting paths [/] 2024-09-12T03:00:39.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.026+0000 7fb170af4640 1 -- 172.21.15.5:0/1863629819 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb16c1082a0 msgr2=0x7fb16c10ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:39.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.026+0000 7fb170af4640 1 --2- 172.21.15.5:0/1863629819 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb16c1082a0 0x7fb16c10ec40 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fb164009cd0 tx=0x7fb16402a8e0 comp rx=0 tx=0).stop 2024-09-12T03:00:39.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.026+0000 7fb170af4640 1 -- 172.21.15.5:0/1863629819 shutdown_connections 2024-09-12T03:00:39.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.026+0000 7fb170af4640 1 --2- 172.21.15.5:0/1863629819 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb16c1082a0 0x7fb16c10ec40 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:39.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.026+0000 7fb170af4640 1 --2- 172.21.15.5:0/1863629819 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb16c1036e0 0x7fb16c107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:39.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.026+0000 7fb170af4640 1 --2- 172.21.15.5:0/1863629819 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb16c102d30 0x7fb16c103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:39.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.026+0000 7fb170af4640 1 -- 172.21.15.5:0/1863629819 >> 172.21.15.5:0/1863629819 conn(0x7fb16c0fc7d0 msgr2=0x7fb16c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:39.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.026+0000 7fb170af4640 1 -- 172.21.15.5:0/1863629819 shutdown_connections 2024-09-12T03:00:39.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.026+0000 7fb170af4640 1 -- 172.21.15.5:0/1863629819 wait complete. 2024-09-12T03:00:39.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.027+0000 7fb170af4640 1 Processor -- start 2024-09-12T03:00:39.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.027+0000 7fb170af4640 1 -- start start 2024-09-12T03:00:39.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.027+0000 7fb170af4640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb16c102d30 0x7fb16c198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:39.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.027+0000 7fb170af4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb16c1036e0 0x7fb16c198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:39.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.027+0000 7fb170af4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb16c1082a0 0x7fb16c19ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:39.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.027+0000 7fb170af4640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb16c1120a0 con 0x7fb16c1036e0 2024-09-12T03:00:39.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.027+0000 7fb170af4640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fb16c111f20 con 0x7fb16c102d30 2024-09-12T03:00:39.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.027+0000 7fb170af4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb16c112220 con 0x7fb16c1082a0 2024-09-12T03:00:39.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.027+0000 7fb163fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb16c1036e0 0x7fb16c198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:39.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.027+0000 7fb163fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb16c1036e0 0x7fb16c198a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56924/0 (socket says 172.21.15.5:56924) 2024-09-12T03:00:39.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.027+0000 7fb163fff640 1 -- 172.21.15.5:0/3325441151 learned_addr learned my addr 172.21.15.5:0/3325441151 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:00:39.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.027+0000 7fb16a59c640 1 --2- 172.21.15.5:0/3325441151 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb16c102d30 0x7fb16c198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:39.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.027+0000 7fb16ad9d640 1 --2- 172.21.15.5:0/3325441151 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb16c1082a0 0x7fb16c19ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:39.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.028+0000 7fb163fff640 1 -- 172.21.15.5:0/3325441151 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb16c1082a0 msgr2=0x7fb16c19ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:39.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.028+0000 7fb163fff640 1 --2- 172.21.15.5:0/3325441151 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb16c1082a0 0x7fb16c19ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:39.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.028+0000 7fb163fff640 1 -- 172.21.15.5:0/3325441151 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb16c102d30 msgr2=0x7fb16c198530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:39.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.028+0000 7fb163fff640 1 --2- 172.21.15.5:0/3325441151 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb16c102d30 0x7fb16c198530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:39.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.028+0000 7fb163fff640 1 -- 172.21.15.5:0/3325441151 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb16c19d4e0 con 0x7fb16c1036e0 2024-09-12T03:00:39.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.028+0000 7fb16a59c640 1 --2- 172.21.15.5:0/3325441151 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb16c102d30 0x7fb16c198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:00:39.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.028+0000 7fb163fff640 1 --2- 172.21.15.5:0/3325441151 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb16c1036e0 0x7fb16c198a70 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fb15400e9f0 tx=0x7fb15400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:39.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.028+0000 7fb1637fe640 1 -- 172.21.15.5:0/3325441151 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb15400ccc0 con 0x7fb16c1036e0 2024-09-12T03:00:39.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.028+0000 7fb1637fe640 1 -- 172.21.15.5:0/3325441151 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb154010910 con 0x7fb16c1036e0 2024-09-12T03:00:39.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.028+0000 7fb170af4640 1 -- 172.21.15.5:0/3325441151 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb16c19d7d0 con 0x7fb16c1036e0 2024-09-12T03:00:39.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.028+0000 7fb1637fe640 1 -- 172.21.15.5:0/3325441151 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb154018920 con 0x7fb16c1036e0 2024-09-12T03:00:39.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.028+0000 7fb170af4640 1 -- 172.21.15.5:0/3325441151 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb16c1a50b0 con 0x7fb16c1036e0 2024-09-12T03:00:39.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.031+0000 7fb172ecd740 1 Processor -- start 2024-09-12T03:00:39.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.031+0000 7fb1637fe640 1 -- 172.21.15.5:0/3325441151 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fb154010ac0 con 0x7fb16c1036e0 2024-09-12T03:00:39.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.031+0000 7fb172ecd740 1 -- start start 2024-09-12T03:00:39.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.031+0000 7fb172ecd740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559e141d1340 0x559e141d1720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:39.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.031+0000 7fb172ecd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e141d1c60 0x559e141e0550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:39.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.031+0000 7fb172ecd740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559e141e0a90 0x559e141e2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:39.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.032+0000 7fb172ecd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559e141372e0 con 0x559e141d1c60 2024-09-12T03:00:39.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.032+0000 7fb172ecd740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x559e140dece0 con 0x559e141d1340 2024-09-12T03:00:39.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.032+0000 7fb172ecd740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x559e141332a0 con 0x559e141e0a90 2024-09-12T03:00:39.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.032+0000 7fb16a59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e141d1c60 0x559e141e0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:39.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.032+0000 7fb16a59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e141d1c60 0x559e141e0550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56940/0 (socket says 172.21.15.5:56940) 2024-09-12T03:00:39.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.032+0000 7fb163fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559e141e0a90 0x559e141e2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:39.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.032+0000 7fb16a59c640 1 -- 172.21.15.5:0/690686773 learned_addr learned my addr 172.21.15.5:0/690686773 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:00:39.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.032+0000 7fb16ad9d640 1 --2- 172.21.15.5:0/690686773 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559e141d1340 0x559e141d1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:39.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.032+0000 7fb16a59c640 1 -- 172.21.15.5:0/690686773 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559e141e0a90 msgr2=0x559e141e2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:00:39.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.032+0000 7fb16a59c640 1 --2- 172.21.15.5:0/690686773 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559e141e0a90 0x559e141e2e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:39.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.032+0000 7fb16a59c640 1 -- 172.21.15.5:0/690686773 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559e141d1340 msgr2=0x559e141d1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:00:39.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.032+0000 7fb16a59c640 1 --2- 172.21.15.5:0/690686773 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559e141d1340 0x559e141d1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:39.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.032+0000 7fb16a59c640 1 -- 172.21.15.5:0/690686773 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e140c3440 con 0x559e141d1c60 2024-09-12T03:00:39.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.032+0000 7fb1637fe640 1 --2- 172.21.15.5:0/3325441151 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fb1580a2c00 0x7fb1580a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:39.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.032+0000 7fb16a59c640 1 --2- 172.21.15.5:0/690686773 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e141d1c60 0x559e141e0550 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fb15c0115f0 tx=0x7fb15c011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e0a7de94d78ab87 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:39.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.032+0000 7fb16ad9d640 1 --2- 172.21.15.5:0/3325441151 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fb1580a2c00 0x7fb1580a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:39.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.032+0000 7fb1637fe640 1 -- 172.21.15.5:0/3325441151 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fb1540cf200 con 0x7fb16c1036e0 2024-09-12T03:00:39.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.033+0000 7fb160ff9640 1 -- 172.21.15.5:0/690686773 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb15c01e070 con 0x559e141d1c60 2024-09-12T03:00:39.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.033+0000 7fb160ff9640 1 -- 172.21.15.5:0/690686773 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb15c00fb20 con 0x559e141d1c60 2024-09-12T03:00:39.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.033+0000 7fb160ff9640 1 -- 172.21.15.5:0/690686773 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb15c019c40 con 0x559e141d1c60 2024-09-12T03:00:39.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.033+0000 7fb16ad9d640 1 --2- 172.21.15.5:0/3325441151 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fb1580a2c00 0x7fb1580a50c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fb164004ab0 tx=0x7fb164004a40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:39.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.033+0000 7fb172ecd740 1 -- 172.21.15.5:0/690686773 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e141d1c60 msgr2=0x559e141e0550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:39.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.033+0000 7fb172ecd740 1 --2- 172.21.15.5:0/690686773 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e141d1c60 0x559e141e0550 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fb15c0115f0 tx=0x7fb15c011ab0 comp rx=0 tx=0).stop 2024-09-12T03:00:39.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.033+0000 7fb172ecd740 1 -- 172.21.15.5:0/690686773 shutdown_connections 2024-09-12T03:00:39.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.033+0000 7fb172ecd740 1 --2- 172.21.15.5:0/690686773 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e141d1c60 0x559e141e0550 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:39.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.033+0000 7fb172ecd740 1 --2- 172.21.15.5:0/690686773 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559e141e0a90 0x559e141e2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:39.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.033+0000 7fb172ecd740 1 --2- 172.21.15.5:0/690686773 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559e141d1340 0x559e141d1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:39.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.033+0000 7fb172ecd740 1 -- 172.21.15.5:0/690686773 >> 172.21.15.5:0/690686773 conn(0x559e141cda30 msgr2=0x559e141cde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:39.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.034+0000 7fb172ecd740 1 -- 172.21.15.5:0/690686773 shutdown_connections 2024-09-12T03:00:39.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.034+0000 7fb172ecd740 1 -- 172.21.15.5:0/690686773 wait complete. 2024-09-12T03:00:39.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.034+0000 7fb172ecd740 1 -- start start 2024-09-12T03:00:39.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.034+0000 7fb172ecd740 1 Processor -- start 2024-09-12T03:00:39.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.034+0000 7fb172ecd740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559e141d1340 0x559e141e0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:39.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.034+0000 7fb172ecd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e141d1c60 0x559e141dd6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:39.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.034+0000 7fb172ecd740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559e141e0a90 0x559e141ddc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:39.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.034+0000 7fb172ecd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559e141e3660 con 0x559e141d1c60 2024-09-12T03:00:39.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.034+0000 7fb172ecd740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x559e140f2050 con 0x559e141d1340 2024-09-12T03:00:39.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.034+0000 7fb172ecd740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x559e141e37e0 con 0x559e141e0a90 2024-09-12T03:00:39.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.034+0000 7fb172ecd740 10 client.0 _ll_register_callbacks cb 0x559e14132160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:00:39.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.034+0000 7fb163fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e141d1c60 0x559e141dd6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:39.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.034+0000 7fb16ad9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559e141e0a90 0x559e141ddc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:39.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.034+0000 7fb163fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e141d1c60 0x559e141dd6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56942/0 (socket says 172.21.15.5:56942) 2024-09-12T03:00:39.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.034+0000 7fb163fff640 1 -- 172.21.15.5:0/231705796 learned_addr learned my addr 172.21.15.5:0/231705796 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:00:39.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.034+0000 7fb16a59c640 1 --2- 172.21.15.5:0/231705796 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559e141d1340 0x559e141e0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:39.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.035+0000 7fb163fff640 1 -- 172.21.15.5:0/231705796 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559e141e0a90 msgr2=0x559e141ddc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:00:39.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.035+0000 7fb163fff640 1 --2- 172.21.15.5:0/231705796 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559e141e0a90 0x559e141ddc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:39.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.035+0000 7fb163fff640 1 -- 172.21.15.5:0/231705796 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559e141d1340 msgr2=0x559e141e0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:00:39.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.035+0000 7fb163fff640 1 --2- 172.21.15.5:0/231705796 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559e141d1340 0x559e141e0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:39.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.035+0000 7fb163fff640 1 -- 172.21.15.5:0/231705796 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e13ea83c0 con 0x559e141d1c60 2024-09-12T03:00:39.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.035+0000 7fb163fff640 1 --2- 172.21.15.5:0/231705796 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e141d1c60 0x559e141dd6e0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fb15400c6a0 tx=0x7fb1540d28d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0907f7abb03ef55 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:39.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.035+0000 7fb151ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T03:00:39.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.035+0000 7fb151ffb640 1 -- 172.21.15.5:0/231705796 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb15409b660 con 0x559e141d1c60 2024-09-12T03:00:39.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.035+0000 7fb151ffb640 1 -- 172.21.15.5:0/231705796 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fb15409af70 con 0x559e141d1c60 2024-09-12T03:00:39.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.035+0000 7fb172ecd740 1 -- 172.21.15.5:0/231705796 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb15c00fb20 con 0x559e141d1c60 2024-09-12T03:00:39.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.035+0000 7fb151ffb640 1 -- 172.21.15.5:0/231705796 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb15409b480 con 0x559e141d1c60 2024-09-12T03:00:39.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.035+0000 7fb172ecd740 10 client.15981 fetch_fsmap learned FSMap version 28 2024-09-12T03:00:39.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.035+0000 7fb172ecd740 1 -- 172.21.15.5:0/231705796 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559e141ef100 con 0x559e141d1c60 2024-09-12T03:00:39.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.035+0000 7fb151ffb640 1 -- 172.21.15.5:0/231705796 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb15c00fb20 con 0x559e141d1c60 2024-09-12T03:00:39.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.036+0000 7fb151ffb640 1 -- 172.21.15.5:0/231705796 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8374+0+0 (secure 0 0 0) 0x7fb154097900 con 0x559e141d1c60 2024-09-12T03:00:39.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.036+0000 7fb172ecd740 10 client.15981 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T03:00:39.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.036+0000 7fb172ecd740 10 client.15981 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:00:39.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.036+0000 7fb172ecd740 20 client.15981 populate_metadata read hostname 'smithi005' 2024-09-12T03:00:39.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.036+0000 7fb172ecd740 1 --2- 172.21.15.5:0/231705796 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x559e141f0560 0x559e141f29c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:39.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.036+0000 7fb172ecd740 4 client.15981 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:00:39.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.036+0000 7fb172ecd740 1 -- 172.21.15.5:0/231705796 --> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559e14144850 con 0x559e141f0560 2024-09-12T03:00:39.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.036+0000 7fb16a59c640 1 --2- 172.21.15.5:0/231705796 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x559e141f0560 0x559e141f29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:39.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.036+0000 7fb151ffb640 10 client.15981 ms_handle_connect on v2:172.21.15.148:6838/2317600000 2024-09-12T03:00:39.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.036+0000 7fb16a59c640 1 --2- 172.21.15.5:0/231705796 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x559e141f0560 0x559e141f29c0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=468e63d64240acd4 server_cookie=74777835cd0e2152 in_seq=0 out_seq=0 2024-09-12T03:00:39.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.037+0000 7fb151ffb640 1 -- 172.21.15.5:0/231705796 <== mds.0 v2:172.21.15.148:6838/2317600000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559e14144850 con 0x559e141f0560 2024-09-12T03:00:39.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.037+0000 7fb151ffb640 10 client.15981 handle_command_reply: tid=0 2024-09-12T03:00:39.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.037+0000 7fb172ecd740 1 client.15981 shutdown 2024-09-12T03:00:39.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.037+0000 7fb172ecd740 1 -- 172.21.15.5:0/231705796 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e141d1c60 msgr2=0x559e141dd6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:39.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.037+0000 7fb172ecd740 1 --2- 172.21.15.5:0/231705796 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e141d1c60 0x559e141dd6e0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fb15400c6a0 tx=0x7fb1540d28d0 comp rx=0 tx=0).stop 2024-09-12T03:00:39.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.037+0000 7fb172ecd740 1 -- 172.21.15.5:0/231705796 shutdown_connections 2024-09-12T03:00:39.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.037+0000 7fb172ecd740 1 --2- 172.21.15.5:0/231705796 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x559e141f0560 0x559e141f29c0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:39.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.037+0000 7fb172ecd740 1 --2- 172.21.15.5:0/231705796 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e141d1c60 0x559e141dd6e0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:39.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.037+0000 7fb172ecd740 1 --2- 172.21.15.5:0/231705796 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559e141e0a90 0x559e141ddc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:39.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.037+0000 7fb151ffb640 0 client.15981 ms_handle_reset on v2:172.21.15.148:6838/2317600000 2024-09-12T03:00:39.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.037+0000 7fb172ecd740 1 --2- 172.21.15.5:0/231705796 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559e141d1340 0x559e141e0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:39.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.037+0000 7fb172ecd740 1 -- 172.21.15.5:0/231705796 >> 172.21.15.5:0/231705796 conn(0x559e141cda30 msgr2=0x559e141d3790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:39.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.037+0000 7fb172ecd740 1 -- 172.21.15.5:0/231705796 shutdown_connections 2024-09-12T03:00:39.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.037+0000 7fb172ecd740 1 -- 172.21.15.5:0/231705796 wait complete. 2024-09-12T03:00:39.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.038+0000 7fb172ecd740 20 client.15981 trim_cache size 0 max 16384 2024-09-12T03:00:39.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.056+0000 7fb172ecd740 1 Processor -- start 2024-09-12T03:00:39.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.056+0000 7fb172ecd740 1 -- start start 2024-09-12T03:00:39.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.056+0000 7fb172ecd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e141d1340 0x559e141ec900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:39.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.056+0000 7fb172ecd740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559e141d1c60 0x559e141f8290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:39.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.056+0000 7fb172ecd740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559e141e0a90 0x559e141f87d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:39.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.056+0000 7fb172ecd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb16c1ab320 con 0x559e141d1340 2024-09-12T03:00:39.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.056+0000 7fb172ecd740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fb16c1acef0 con 0x559e141d1c60 2024-09-12T03:00:39.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.056+0000 7fb172ecd740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb16c1ab9f0 con 0x559e141e0a90 2024-09-12T03:00:39.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.056+0000 7fb16a59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e141d1340 0x559e141ec900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:39.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.056+0000 7fb16a59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e141d1340 0x559e141ec900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56948/0 (socket says 172.21.15.5:56948) 2024-09-12T03:00:39.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.056+0000 7fb16a59c640 1 -- 172.21.15.5:0/278799219 learned_addr learned my addr 172.21.15.5:0/278799219 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:00:39.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.056+0000 7fb163fff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559e141d1c60 0x559e141f8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:39.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.056+0000 7fb16ad9d640 1 --2- 172.21.15.5:0/278799219 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559e141e0a90 0x559e141f87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:39.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.056+0000 7fb16a59c640 1 -- 172.21.15.5:0/278799219 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559e141e0a90 msgr2=0x559e141f87d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:00:39.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.056+0000 7fb16a59c640 1 --2- 172.21.15.5:0/278799219 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559e141e0a90 0x559e141f87d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:39.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.056+0000 7fb16a59c640 1 -- 172.21.15.5:0/278799219 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559e141d1c60 msgr2=0x559e141f8290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:00:39.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.056+0000 7fb16a59c640 1 --2- 172.21.15.5:0/278799219 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559e141d1c60 0x559e141f8290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:39.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.056+0000 7fb16a59c640 1 -- 172.21.15.5:0/278799219 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e141ddff0 con 0x559e141d1340 2024-09-12T03:00:39.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.056+0000 7fb163fff640 1 --2- 172.21.15.5:0/278799219 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559e141d1c60 0x559e141f8290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:00:39.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.056+0000 7fb16a59c640 1 --2- 172.21.15.5:0/278799219 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e141d1340 0x559e141ec900 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fb15c0115f0 tx=0x7fb15c019820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12e793a5777eb362 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:39.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.057+0000 7fb153fff640 1 -- 172.21.15.5:0/278799219 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb15c028070 con 0x559e141d1340 2024-09-12T03:00:39.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.057+0000 7fb153fff640 1 -- 172.21.15.5:0/278799219 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb15c00fb20 con 0x559e141d1340 2024-09-12T03:00:39.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.057+0000 7fb153fff640 1 -- 172.21.15.5:0/278799219 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb15c01e040 con 0x559e141d1340 2024-09-12T03:00:39.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.057+0000 7fb172ecd740 1 -- 172.21.15.5:0/278799219 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e141d1340 msgr2=0x559e141ec900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:39.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.057+0000 7fb172ecd740 1 --2- 172.21.15.5:0/278799219 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e141d1340 0x559e141ec900 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fb15c0115f0 tx=0x7fb15c019820 comp rx=0 tx=0).stop 2024-09-12T03:00:39.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.057+0000 7fb172ecd740 1 -- 172.21.15.5:0/278799219 shutdown_connections 2024-09-12T03:00:39.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.057+0000 7fb172ecd740 1 --2- 172.21.15.5:0/278799219 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559e141e0a90 0x559e141f87d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:39.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.057+0000 7fb172ecd740 1 --2- 172.21.15.5:0/278799219 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559e141d1c60 0x559e141f8290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:39.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.057+0000 7fb172ecd740 1 --2- 172.21.15.5:0/278799219 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e141d1340 0x559e141ec900 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:39.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.057+0000 7fb172ecd740 1 -- 172.21.15.5:0/278799219 >> 172.21.15.5:0/278799219 conn(0x559e141cda30 msgr2=0x559e141d8d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:39.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.058+0000 7fb172ecd740 1 -- 172.21.15.5:0/278799219 shutdown_connections 2024-09-12T03:00:39.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.058+0000 7fb172ecd740 1 -- 172.21.15.5:0/278799219 wait complete. 2024-09-12T03:00:39.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.058+0000 7fb172ecd740 1 -- start start 2024-09-12T03:00:39.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.058+0000 7fb172ecd740 1 Processor -- start 2024-09-12T03:00:39.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.058+0000 7fb172ecd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e141d1340 0x559e141e4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:39.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.058+0000 7fb172ecd740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559e141d1c60 0x559e141e5040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:39.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.058+0000 7fb172ecd740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559e141e0a90 0x559e1425a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:39.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.058+0000 7fb172ecd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb16c1ac790 con 0x559e141d1340 2024-09-12T03:00:39.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.058+0000 7fb172ecd740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fb16c1ac0c0 con 0x559e141d1c60 2024-09-12T03:00:39.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.058+0000 7fb172ecd740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb16c1a9300 con 0x559e141e0a90 2024-09-12T03:00:39.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.058+0000 7fb172ecd740 10 client.0 _ll_register_callbacks cb 0x7fb1540d29d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:00:39.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.058+0000 7fb16a59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e141d1340 0x559e141e4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:39.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.058+0000 7fb16a59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e141d1340 0x559e141e4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56956/0 (socket says 172.21.15.5:56956) 2024-09-12T03:00:39.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.058+0000 7fb16a59c640 1 -- 172.21.15.5:0/2664080245 learned_addr learned my addr 172.21.15.5:0/2664080245 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:00:39.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.058+0000 7fb163fff640 1 --2- 172.21.15.5:0/2664080245 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559e141d1c60 0x559e141e5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:39.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.058+0000 7fb16ad9d640 1 --2- 172.21.15.5:0/2664080245 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559e141e0a90 0x559e1425a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:39.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.059+0000 7fb163fff640 1 -- 172.21.15.5:0/2664080245 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559e141e0a90 msgr2=0x559e1425a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:00:39.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.059+0000 7fb163fff640 1 --2- 172.21.15.5:0/2664080245 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559e141e0a90 0x559e1425a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:39.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.059+0000 7fb163fff640 1 -- 172.21.15.5:0/2664080245 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e141d1340 msgr2=0x559e141e4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:00:39.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.059+0000 7fb163fff640 1 --2- 172.21.15.5:0/2664080245 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e141d1340 0x559e141e4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:39.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.059+0000 7fb163fff640 1 -- 172.21.15.5:0/2664080245 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e141e58d0 con 0x559e141d1c60 2024-09-12T03:00:39.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.059+0000 7fb16a59c640 1 --2- 172.21.15.5:0/2664080245 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e141d1340 0x559e141e4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:00:39.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.059+0000 7fb163fff640 1 --2- 172.21.15.5:0/2664080245 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559e141d1c60 0x559e141e5040 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb15409bab0 tx=0x7fb15409b1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cdf089f61e2426d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:39.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.059+0000 7fb151ffb640 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2024-09-12T03:00:39.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.059+0000 7fb151ffb640 1 -- 172.21.15.5:0/2664080245 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb154018e60 con 0x559e141d1c60 2024-09-12T03:00:39.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.059+0000 7fb151ffb640 1 -- 172.21.15.5:0/2664080245 <== mon.2 v2:172.21.15.146:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fb154010640 con 0x559e141d1c60 2024-09-12T03:00:39.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.059+0000 7fb172ecd740 1 -- 172.21.15.5:0/2664080245 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb15c00fb20 con 0x559e141d1c60 2024-09-12T03:00:39.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.059+0000 7fb151ffb640 1 -- 172.21.15.5:0/2664080245 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb15409bb20 con 0x559e141d1c60 2024-09-12T03:00:39.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.059+0000 7fb172ecd740 10 client.25412 fetch_fsmap learned FSMap version 28 2024-09-12T03:00:39.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.059+0000 7fb172ecd740 1 -- 172.21.15.5:0/2664080245 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559e1425b880 con 0x559e141d1c60 2024-09-12T03:00:39.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.060+0000 7fb151ffb640 1 -- 172.21.15.5:0/2664080245 <== mon.2 v2:172.21.15.146:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb15c00fb20 con 0x559e141d1c60 2024-09-12T03:00:39.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.060+0000 7fb151ffb640 1 -- 172.21.15.5:0/2664080245 <== mon.2 v2:172.21.15.146:3300/0 5 ==== fsmap(e 28) ==== 8374+0+0 (secure 0 0 0) 0x7fb1540a0fb0 con 0x559e141d1c60 2024-09-12T03:00:39.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.060+0000 7fb172ecd740 10 client.25412 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T03:00:39.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.060+0000 7fb172ecd740 10 client.25412 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:00:39.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.060+0000 7fb172ecd740 20 client.25412 populate_metadata read hostname 'smithi005' 2024-09-12T03:00:39.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.060+0000 7fb172ecd740 1 --2- 172.21.15.5:0/2664080245 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x559e1425c090 0x559e1425e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:39.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.060+0000 7fb172ecd740 4 client.25412 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T03:00:39.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.060+0000 7fb172ecd740 1 -- 172.21.15.5:0/2664080245 --> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559e1413ed60 con 0x559e1425c090 2024-09-12T03:00:39.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.060+0000 7fb16a59c640 1 --2- 172.21.15.5:0/2664080245 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x559e1425c090 0x559e1425e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:39.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.061+0000 7fb16a59c640 1 --2- 172.21.15.5:0/2664080245 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x559e1425c090 0x559e1425e4b0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1aae6126721f25a6 server_cookie=366f182215771af6 in_seq=0 out_seq=0 2024-09-12T03:00:39.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.061+0000 7fb151ffb640 10 client.25412 ms_handle_connect on v2:172.21.15.148:6838/2317600000 2024-09-12T03:00:39.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.061+0000 7fb151ffb640 1 -- 172.21.15.5:0/2664080245 <== mds.0 v2:172.21.15.148:6838/2317600000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x559e1413ed60 con 0x559e1425c090 2024-09-12T03:00:39.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.061+0000 7fb151ffb640 10 client.25412 handle_command_reply: tid=0 2024-09-12T03:00:39.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.061+0000 7fb172ecd740 1 client.25412 shutdown 2024-09-12T03:00:39.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.061+0000 7fb172ecd740 1 -- 172.21.15.5:0/2664080245 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559e141d1c60 msgr2=0x559e141e5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:39.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.061+0000 7fb172ecd740 1 --2- 172.21.15.5:0/2664080245 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559e141d1c60 0x559e141e5040 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb15409bab0 tx=0x7fb15409b1a0 comp rx=0 tx=0).stop 2024-09-12T03:00:39.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.061+0000 7fb172ecd740 1 -- 172.21.15.5:0/2664080245 shutdown_connections 2024-09-12T03:00:39.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.061+0000 7fb172ecd740 1 --2- 172.21.15.5:0/2664080245 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559e141e0a90 0x559e1425a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:39.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.061+0000 7fb172ecd740 1 --2- 172.21.15.5:0/2664080245 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559e141d1c60 0x559e141e5040 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:39.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.061+0000 7fb172ecd740 1 --2- 172.21.15.5:0/2664080245 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x559e1425c090 0x559e1425e4b0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:39.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.061+0000 7fb172ecd740 1 --2- 172.21.15.5:0/2664080245 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559e141d1340 0x559e141e4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:39.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.061+0000 7fb151ffb640 0 client.25412 ms_handle_reset on v2:172.21.15.148:6838/2317600000 2024-09-12T03:00:39.067 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-12T03:00:39.067 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-12T03:00:39.067 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-09-12T03:00:39.067 INFO:teuthology.orchestra.run.smithi005.stdout: "e6a90fb8-e6ec-4ee2-959a-e5e280e9b9cf": { 2024-09-12T03:00:39.067 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-09-12T03:00:39.067 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "e6a90fb8-e6ec-4ee2-959a-e5e280e9b9cf", 2024-09-12T03:00:39.067 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-09-12T03:00:39.068 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:00:39.068 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:00:39.068 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-12T03:00:39.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.061+0000 7fb172ecd740 1 -- 172.21.15.5:0/2664080245 >> 172.21.15.5:0/2664080245 conn(0x559e141cda30 msgr2=0x559e141def40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:39.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.061+0000 7fb172ecd740 1 -- 172.21.15.5:0/2664080245 shutdown_connections 2024-09-12T03:00:39.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.061+0000 7fb172ecd740 1 -- 172.21.15.5:0/2664080245 wait complete. 2024-09-12T03:00:39.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.061+0000 7fb172ecd740 20 client.25412 trim_cache size 0 max 16384 2024-09-12T03:00:39.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.062+0000 7fb170af4640 1 -- 172.21.15.5:0/3325441151 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fb1580a2c00 msgr2=0x7fb1580a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:39.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.062+0000 7fb170af4640 1 --2- 172.21.15.5:0/3325441151 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fb1580a2c00 0x7fb1580a50c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fb164004ab0 tx=0x7fb164004a40 comp rx=0 tx=0).stop 2024-09-12T03:00:39.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.062+0000 7fb170af4640 1 -- 172.21.15.5:0/3325441151 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb16c1036e0 msgr2=0x7fb16c198a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:39.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.062+0000 7fb170af4640 1 --2- 172.21.15.5:0/3325441151 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb16c1036e0 0x7fb16c198a70 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fb15400e9f0 tx=0x7fb15400eeb0 comp rx=0 tx=0).stop 2024-09-12T03:00:39.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.062+0000 7fb170af4640 1 -- 172.21.15.5:0/3325441151 shutdown_connections 2024-09-12T03:00:39.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.062+0000 7fb170af4640 1 --2- 172.21.15.5:0/3325441151 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fb1580a2c00 0x7fb1580a50c0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:39.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.062+0000 7fb170af4640 1 --2- 172.21.15.5:0/3325441151 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb16c1036e0 0x7fb16c198a70 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:39.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.062+0000 7fb170af4640 1 --2- 172.21.15.5:0/3325441151 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb16c1082a0 0x7fb16c19ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:39.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.062+0000 7fb170af4640 1 --2- 172.21.15.5:0/3325441151 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb16c102d30 0x7fb16c198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:39.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.062+0000 7fb170af4640 1 -- 172.21.15.5:0/3325441151 >> 172.21.15.5:0/3325441151 conn(0x7fb16c0fc7d0 msgr2=0x7fb16c105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:39.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.062+0000 7fb170af4640 1 -- 172.21.15.5:0/3325441151 shutdown_connections 2024-09-12T03:00:39.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:39.062+0000 7fb170af4640 1 -- 172.21.15.5:0/3325441151 wait complete. 2024-09-12T03:00:39.670 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:00:39 smithi005 ceph-mon[31197]: pgmap v407: 97 pgs: 97 active+clean; 74 MiB data, 679 MiB used, 1.0 TiB / 1.0 TiB avail; 891 KiB/s rd, 578 KiB/s wr, 376 op/s 2024-09-12T03:00:39.670 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:00:39 smithi005 ceph-mon[31197]: scrub summary: active paths [/] 2024-09-12T03:00:39.670 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:00:39 smithi005 ceph-mon[31197]: scrub summary: idle+waiting paths [/] 2024-09-12T03:00:39.670 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:00:39 smithi005 ceph-mon[31197]: scrub summary: idle+waiting paths [/] 2024-09-12T03:00:39.671 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:00:39 smithi148 ceph-mon[35090]: pgmap v407: 97 pgs: 97 active+clean; 74 MiB data, 679 MiB used, 1.0 TiB / 1.0 TiB avail; 891 KiB/s rd, 578 KiB/s wr, 376 op/s 2024-09-12T03:00:39.671 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:00:39 smithi148 ceph-mon[35090]: scrub summary: active paths [/] 2024-09-12T03:00:39.671 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:00:39 smithi148 ceph-mon[35090]: scrub summary: idle+waiting paths [/] 2024-09-12T03:00:39.671 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:00:39 smithi148 ceph-mon[35090]: scrub summary: idle+waiting paths [/] 2024-09-12T03:00:39.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:00:39 smithi146 ceph-mon[35011]: pgmap v407: 97 pgs: 97 active+clean; 74 MiB data, 679 MiB used, 1.0 TiB / 1.0 TiB avail; 891 KiB/s rd, 578 KiB/s wr, 376 op/s 2024-09-12T03:00:39.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:00:39 smithi146 ceph-mon[35011]: scrub summary: active paths [/] 2024-09-12T03:00:39.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:00:39 smithi146 ceph-mon[35011]: scrub summary: idle+waiting paths [/] 2024-09-12T03:00:39.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:00:39 smithi146 ceph-mon[35011]: scrub summary: idle+waiting paths [/] 2024-09-12T03:00:39.865 INFO:tasks.cephfs.filesystem:scrub status for tag:e6a90fb8-e6ec-4ee2-959a-e5e280e9b9cf - {'path': '/', 'tag': 'e6a90fb8-e6ec-4ee2-959a-e5e280e9b9cf', 'options': 'recursive,force'} 2024-09-12T03:00:40.446 INFO:tasks.workunit.client.0.smithi005.stdout:copying to largest ever: 0xfe492 2024-09-12T03:00:41.665 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:00:41 smithi148 ceph-mon[35090]: pgmap v408: 97 pgs: 97 active+clean; 76 MiB data, 679 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 331 op/s 2024-09-12T03:00:41.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:00:41 smithi146 ceph-mon[35011]: pgmap v408: 97 pgs: 97 active+clean; 76 MiB data, 679 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 331 op/s 2024-09-12T03:00:41.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:00:41 smithi005 ceph-mon[31197]: pgmap v408: 97 pgs: 97 active+clean; 76 MiB data, 679 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 331 op/s 2024-09-12T03:00:42.170 INFO:tasks.workunit.client.0.smithi005.stdout:copying to largest ever: 0xff038 2024-09-12T03:00:42.944 INFO:tasks.workunit.client.0.smithi005.stdout:truncating to largest ever: 0xffcac 2024-09-12T03:00:42.994 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 24.7 2024-09-12T03:00:42.994 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:00:43.054 INFO:tasks.workunit.client.0.smithi005.stdout:10000 copy from 0x2d237 to 0x30552, (0x331b bytes) at 0x82aed 2024-09-12T03:00:43.270 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:00:43.656 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:00:43 smithi005 ceph-mon[31197]: pgmap v409: 97 pgs: 97 active+clean; 76 MiB data, 679 MiB used, 1.0 TiB / 1.0 TiB avail; 749 KiB/s rd, 1.3 MiB/s wr, 214 op/s 2024-09-12T03:00:43.665 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:00:43 smithi148 ceph-mon[35090]: pgmap v409: 97 pgs: 97 active+clean; 76 MiB data, 679 MiB used, 1.0 TiB / 1.0 TiB avail; 749 KiB/s rd, 1.3 MiB/s wr, 214 op/s 2024-09-12T03:00:43.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:00:43 smithi146 ceph-mon[35011]: pgmap v409: 97 pgs: 97 active+clean; 76 MiB data, 679 MiB used, 1.0 TiB / 1.0 TiB avail; 749 KiB/s rd, 1.3 MiB/s wr, 214 op/s 2024-09-12T03:00:43.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:43.746+0000 7f1bd0b7c640 1 -- 172.21.15.5:0/368423408 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1bcc100e20 msgr2=0x7f1bcc101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:43.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:43.746+0000 7f1bd0b7c640 1 --2- 172.21.15.5:0/368423408 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1bcc100e20 0x7f1bcc101200 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f1bb4009900 tx=0x7f1bb402a480 comp rx=0 tx=0).stop 2024-09-12T03:00:43.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:43.746+0000 7f1bd0b7c640 1 -- 172.21.15.5:0/368423408 shutdown_connections 2024-09-12T03:00:43.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:43.746+0000 7f1bd0b7c640 1 --2- 172.21.15.5:0/368423408 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1bcc102ef0 0x7f1bcc10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:43.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:43.746+0000 7f1bd0b7c640 1 --2- 172.21.15.5:0/368423408 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1bcc102570 0x7f1bcc1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:43.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:43.746+0000 7f1bd0b7c640 1 --2- 172.21.15.5:0/368423408 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1bcc100e20 0x7f1bcc101200 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:43.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:43.746+0000 7f1bd0b7c640 1 -- 172.21.15.5:0/368423408 >> 172.21.15.5:0/368423408 conn(0x7f1bcc0fc7d0 msgr2=0x7f1bcc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:43.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:43.746+0000 7f1bd0b7c640 1 -- 172.21.15.5:0/368423408 shutdown_connections 2024-09-12T03:00:43.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:43.746+0000 7f1bd0b7c640 1 -- 172.21.15.5:0/368423408 wait complete. 2024-09-12T03:00:43.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:43.747+0000 7f1bd0b7c640 1 Processor -- start 2024-09-12T03:00:43.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:43.747+0000 7f1bd0b7c640 1 -- start start 2024-09-12T03:00:43.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:43.748+0000 7f1bd0b7c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1bcc100e20 0x7f1bcc1984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:43.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:43.748+0000 7f1bd0b7c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1bcc102570 0x7f1bcc198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:43.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:43.748+0000 7f1bd0b7c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1bcc102ef0 0x7f1bcc19cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:43.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:43.748+0000 7f1bd0b7c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1bcc110fa0 con 0x7f1bcc100e20 2024-09-12T03:00:43.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:43.748+0000 7f1bd0b7c640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f1bcc110e20 con 0x7f1bcc102570 2024-09-12T03:00:43.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:43.748+0000 7f1bd0b7c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1bcc111120 con 0x7f1bcc102ef0 2024-09-12T03:00:43.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:43.748+0000 7f1bc9d9b640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1bcc102570 0x7f1bcc198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:43.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:43.748+0000 7f1bca59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1bcc100e20 0x7f1bcc1984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:43.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:43.748+0000 7f1bc9d9b640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1bcc102570 0x7f1bcc198a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.5:52268/0 (socket says 172.21.15.5:52268) 2024-09-12T03:00:43.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:43.748+0000 7f1bc9d9b640 1 -- 172.21.15.5:0/274889876 learned_addr learned my addr 172.21.15.5:0/274889876 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:00:43.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:43.748+0000 7f1bcad9d640 1 --2- 172.21.15.5:0/274889876 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1bcc102ef0 0x7f1bcc19cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:43.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:43.748+0000 7f1bc9d9b640 1 -- 172.21.15.5:0/274889876 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1bcc102ef0 msgr2=0x7f1bcc19cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:43.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:43.748+0000 7f1bc9d9b640 1 --2- 172.21.15.5:0/274889876 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1bcc102ef0 0x7f1bcc19cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:43.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:43.748+0000 7f1bc9d9b640 1 -- 172.21.15.5:0/274889876 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1bcc100e20 msgr2=0x7f1bcc1984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:43.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:43.748+0000 7f1bc9d9b640 1 --2- 172.21.15.5:0/274889876 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1bcc100e20 0x7f1bcc1984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:43.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:43.748+0000 7f1bc9d9b640 1 -- 172.21.15.5:0/274889876 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bcc19d470 con 0x7f1bcc102570 2024-09-12T03:00:43.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:43.748+0000 7f1bca59c640 1 --2- 172.21.15.5:0/274889876 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1bcc100e20 0x7f1bcc1984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:00:43.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:43.748+0000 7f1bcad9d640 1 --2- 172.21.15.5:0/274889876 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1bcc102ef0 0x7f1bcc19cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:00:43.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:43.748+0000 7f1bc9d9b640 1 --2- 172.21.15.5:0/274889876 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1bcc102570 0x7f1bcc198a00 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc00ea60 tx=0x7f1bbc00ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:43.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:43.749+0000 7f1bc37fe640 1 -- 172.21.15.5:0/274889876 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bbc00cde0 con 0x7f1bcc102570 2024-09-12T03:00:43.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:43.749+0000 7f1bc37fe640 1 -- 172.21.15.5:0/274889876 <== mon.2 v2:172.21.15.146:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1bbc0035f0 con 0x7f1bcc102570 2024-09-12T03:00:43.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:43.749+0000 7f1bd0b7c640 1 -- 172.21.15.5:0/274889876 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bcc19d760 con 0x7f1bcc102570 2024-09-12T03:00:43.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:43.749+0000 7f1bc37fe640 1 -- 172.21.15.5:0/274889876 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bbc010670 con 0x7f1bcc102570 2024-09-12T03:00:43.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:43.749+0000 7f1bd0b7c640 1 -- 172.21.15.5:0/274889876 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bcc1a5040 con 0x7f1bcc102570 2024-09-12T03:00:43.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:43.750+0000 7f1bd0b7c640 1 -- 172.21.15.5:0/274889876 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1bac005290 con 0x7f1bcc102570 2024-09-12T03:00:43.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:43.751+0000 7f1bc37fe640 1 -- 172.21.15.5:0/274889876 <== mon.2 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f1bbc043ae0 con 0x7f1bcc102570 2024-09-12T03:00:43.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:43.752+0000 7f1bc37fe640 1 --2- 172.21.15.5:0/274889876 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f1bb80a2c00 0x7f1bb80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:43.752 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:43.752+0000 7f1bc37fe640 1 -- 172.21.15.5:0/274889876 <== mon.2 v2:172.21.15.146:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f1bbc0cf130 con 0x7f1bcc102570 2024-09-12T03:00:43.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:43.752+0000 7f1bca59c640 1 --2- 172.21.15.5:0/274889876 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f1bb80a2c00 0x7f1bb80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:43.753 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:43.753+0000 7f1bca59c640 1 --2- 172.21.15.5:0/274889876 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f1bb80a2c00 0x7f1bb80a50c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f1bb4000c00 tx=0x7f1bb4005b50 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:43.756 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:43.756+0000 7f1bc37fe640 1 -- 172.21.15.5:0/274889876 <== mon.2 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1bbc09c080 con 0x7f1bcc102570 2024-09-12T03:00:43.993 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:43.993+0000 7f1bd0b7c640 1 -- 172.21.15.5:0/274889876 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1bac005580 con 0x7f1bcc102570 2024-09-12T03:00:43.994 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:43.994+0000 7f1bc37fe640 1 -- 172.21.15.5:0/274889876 <== mon.2 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11971 (secure 0 0 0) 0x7f1bbc09b980 con 0x7f1bcc102570 2024-09-12T03:00:43.994 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-12T03:00:43.994 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":28,"btime":"2024-09-12T03:00:03:036015+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/4135232521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":4135232521},{"type":"v1","addr":"172.21.15.5:6835","nonce":4135232521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/4290221940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":4290221940},{"type":"v1","addr":"172.21.15.5:6837","nonce":4290221940}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T02:56:16.595562+0000","modified":"2024-09-12T03:00:02.675878+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24502,"mds_1":24487,"mds_2":24481,"mds_3":24479,"mds_4":24458},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/26447341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":26447341},{"type":"v1","addr":"172.21.15.5:6839","nonce":26447341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2393443295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2393443295},{"type":"v1","addr":"172.21.15.5:6841","nonce":2393443295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6833/3686970137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3686970137},{"type":"v1","addr":"172.21.15.146:6833","nonce":3686970137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6833/568512209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":568512209},{"type":"v1","addr":"172.21.15.148:6833","nonce":568512209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6835/2540740420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2540740420},{"type":"v1","addr":"172.21.15.146:6835","nonce":2540740420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.146:6837/1673370937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":1673370937},{"type":"v1","addr":"172.21.15.146:6837","nonce":1673370937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":3,"incarnation":22,"state":"up:active","state_seq":6,"addr":"172.21.15.146:6839/3036767228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":3036767228},{"type":"v1","addr":"172.21.15.146:6839","nonce":3036767228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":2,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.148:6835/4195086427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":4195086427},{"type":"v1","addr":"172.21.15.148:6835","nonce":4195086427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.148:6837/2606873922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2606873922},{"type":"v1","addr":"172.21.15.148:6837","nonce":2606873922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6839/2317600000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":2317600000},{"type":"v1","addr":"172.21.15.148:6839","nonce":2317600000}]},"join_fscid":-1,"export_targets":[1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24502,"qdb_cluster":[24502,24487,24481,24479,24458]},"id":1}]} 2024-09-12T03:00:43.995 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 28 2024-09-12T03:00:43.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:43.996+0000 7f1bd0b7c640 1 -- 172.21.15.5:0/274889876 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f1bb80a2c00 msgr2=0x7f1bb80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:43.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:43.996+0000 7f1bd0b7c640 1 --2- 172.21.15.5:0/274889876 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f1bb80a2c00 0x7f1bb80a50c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f1bb4000c00 tx=0x7f1bb4005b50 comp rx=0 tx=0).stop 2024-09-12T03:00:43.996 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:43.996+0000 7f1bd0b7c640 1 -- 172.21.15.5:0/274889876 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1bcc102570 msgr2=0x7f1bcc198a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:43.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:43.996+0000 7f1bd0b7c640 1 --2- 172.21.15.5:0/274889876 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1bcc102570 0x7f1bcc198a00 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc00ea60 tx=0x7f1bbc00ef20 comp rx=0 tx=0).stop 2024-09-12T03:00:43.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:43.996+0000 7f1bd0b7c640 1 -- 172.21.15.5:0/274889876 shutdown_connections 2024-09-12T03:00:43.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:43.996+0000 7f1bd0b7c640 1 --2- 172.21.15.5:0/274889876 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1bcc102ef0 0x7f1bcc19cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:43.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:43.996+0000 7f1bd0b7c640 1 --2- 172.21.15.5:0/274889876 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1bcc102570 0x7f1bcc198a00 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:43.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:43.996+0000 7f1bd0b7c640 1 --2- 172.21.15.5:0/274889876 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f1bb80a2c00 0x7f1bb80a50c0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:43.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:43.996+0000 7f1bd0b7c640 1 --2- 172.21.15.5:0/274889876 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1bcc100e20 0x7f1bcc1984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:43.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:43.996+0000 7f1bd0b7c640 1 -- 172.21.15.5:0/274889876 >> 172.21.15.5:0/274889876 conn(0x7f1bcc0fc7d0 msgr2=0x7f1bcc06a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:43.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:43.996+0000 7f1bd0b7c640 1 -- 172.21.15.5:0/274889876 shutdown_connections 2024-09-12T03:00:43.997 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:43.996+0000 7f1bd0b7c640 1 -- 172.21.15.5:0/274889876 wait complete. 2024-09-12T03:00:44.665 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:00:44 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/274889876' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:00:44.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:00:44 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/274889876' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:00:44.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:00:44 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/274889876' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:00:45.019 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24502 quiesce db / --timeout 90 --expiration 144.7 --await-for 45.0' 2024-09-12T03:00:45.019 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24502 quiesce db / --timeout 90 --expiration 144.7 --await-for 45.0 2024-09-12T03:00:45.306 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:00:45.665 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:00:45 smithi148 ceph-mon[35090]: pgmap v410: 97 pgs: 97 active+clean; 76 MiB data, 694 MiB used, 1.0 TiB / 1.0 TiB avail; 723 KiB/s rd, 1.3 MiB/s wr, 211 op/s 2024-09-12T03:00:45.699 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:00:45 smithi005 ceph-mon[31197]: pgmap v410: 97 pgs: 97 active+clean; 76 MiB data, 694 MiB used, 1.0 TiB / 1.0 TiB avail; 723 KiB/s rd, 1.3 MiB/s wr, 211 op/s 2024-09-12T03:00:45.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:00:45 smithi146 ceph-mon[35011]: pgmap v410: 97 pgs: 97 active+clean; 76 MiB data, 694 MiB used, 1.0 TiB / 1.0 TiB avail; 723 KiB/s rd, 1.3 MiB/s wr, 211 op/s 2024-09-12T03:00:45.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.801+0000 7f0ae7222640 1 -- 172.21.15.5:0/4064161255 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ae0102570 msgr2=0x7f0ae01029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:45.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.801+0000 7f0ae7222640 1 --2- 172.21.15.5:0/4064161255 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ae0102570 0x7f0ae01029b0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f0acc009900 tx=0x7f0acc02a480 comp rx=0 tx=0).stop 2024-09-12T03:00:45.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.801+0000 7f0ae7222640 1 -- 172.21.15.5:0/4064161255 shutdown_connections 2024-09-12T03:00:45.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.801+0000 7f0ae7222640 1 --2- 172.21.15.5:0/4064161255 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ae0102570 0x7f0ae01029b0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:45.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.801+0000 7f0ae7222640 1 --2- 172.21.15.5:0/4064161255 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0ae0102ef0 0x7f0ae010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:45.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.801+0000 7f0ae7222640 1 --2- 172.21.15.5:0/4064161255 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0ae0100e20 0x7f0ae0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:45.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.801+0000 7f0ae7222640 1 -- 172.21.15.5:0/4064161255 >> 172.21.15.5:0/4064161255 conn(0x7f0ae00fc7d0 msgr2=0x7f0ae00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:45.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.801+0000 7f0ae7222640 1 -- 172.21.15.5:0/4064161255 shutdown_connections 2024-09-12T03:00:45.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.801+0000 7f0ae7222640 1 -- 172.21.15.5:0/4064161255 wait complete. 2024-09-12T03:00:45.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.802+0000 7f0ae7222640 1 Processor -- start 2024-09-12T03:00:45.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.802+0000 7f0ae7222640 1 -- start start 2024-09-12T03:00:45.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.802+0000 7f0ae7222640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ae0100e20 0x7f0ae0199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:45.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.802+0000 7f0ae7222640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0ae0102570 0x7f0ae0199840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:45.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.802+0000 7f0ae7222640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0ae0102ef0 0x7f0ae01a4380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:45.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.802+0000 7f0ae7222640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae0110fd0 con 0x7f0ae0100e20 2024-09-12T03:00:45.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.802+0000 7f0ae7222640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae0110e50 con 0x7f0ae0102570 2024-09-12T03:00:45.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.802+0000 7f0ae7222640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae0111150 con 0x7f0ae0102ef0 2024-09-12T03:00:45.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.802+0000 7f0ae4fbe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ae0100e20 0x7f0ae0199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:45.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.802+0000 7f0ae4fbe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ae0100e20 0x7f0ae0199300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39540/0 (socket says 172.21.15.5:39540) 2024-09-12T03:00:45.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.802+0000 7f0ae4fbe640 1 -- 172.21.15.5:0/2631069164 learned_addr learned my addr 172.21.15.5:0/2631069164 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:00:45.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.802+0000 7f0adffff640 1 --2- 172.21.15.5:0/2631069164 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0ae0102570 0x7f0ae0199840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:45.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.802+0000 7f0ae57bf640 1 --2- 172.21.15.5:0/2631069164 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0ae0102ef0 0x7f0ae01a4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:45.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.803+0000 7f0ae4fbe640 1 -- 172.21.15.5:0/2631069164 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0ae0102ef0 msgr2=0x7f0ae01a4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:45.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.803+0000 7f0ae4fbe640 1 --2- 172.21.15.5:0/2631069164 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0ae0102ef0 0x7f0ae01a4380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:45.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.803+0000 7f0ae4fbe640 1 -- 172.21.15.5:0/2631069164 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0ae0102570 msgr2=0x7f0ae0199840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:45.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.803+0000 7f0ae4fbe640 1 --2- 172.21.15.5:0/2631069164 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0ae0102570 0x7f0ae0199840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:45.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.803+0000 7f0ae4fbe640 1 -- 172.21.15.5:0/2631069164 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ae01a48c0 con 0x7f0ae0100e20 2024-09-12T03:00:45.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.803+0000 7f0adffff640 1 --2- 172.21.15.5:0/2631069164 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0ae0102570 0x7f0ae0199840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:00:45.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.803+0000 7f0ae4fbe640 1 --2- 172.21.15.5:0/2631069164 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ae0100e20 0x7f0ae0199300 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f0ad400e9f0 tx=0x7f0ad400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:45.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.803+0000 7f0addffb640 1 -- 172.21.15.5:0/2631069164 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad400cd80 con 0x7f0ae0100e20 2024-09-12T03:00:45.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.803+0000 7f0ae7222640 1 -- 172.21.15.5:0/2631069164 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ae01970f0 con 0x7f0ae0100e20 2024-09-12T03:00:45.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.803+0000 7f0addffb640 1 -- 172.21.15.5:0/2631069164 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ad4010910 con 0x7f0ae0100e20 2024-09-12T03:00:45.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.803+0000 7f0addffb640 1 -- 172.21.15.5:0/2631069164 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad40188f0 con 0x7f0ae0100e20 2024-09-12T03:00:45.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.803+0000 7f0ae7222640 1 -- 172.21.15.5:0/2631069164 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ae0197650 con 0x7f0ae0100e20 2024-09-12T03:00:45.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.806+0000 7f0addffb640 1 -- 172.21.15.5:0/2631069164 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f0ad4010ac0 con 0x7f0ae0100e20 2024-09-12T03:00:45.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.807+0000 7f0ae95fb740 1 Processor -- start 2024-09-12T03:00:45.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.807+0000 7f0ae95fb740 1 -- start start 2024-09-12T03:00:45.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.807+0000 7f0ae95fb740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bf96072760 0x55bf96072b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:45.807 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.807+0000 7f0ae95fb740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bf96073080 0x55bf9607d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:45.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.807+0000 7f0ae95fb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf9607d940 0x55bf9607fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:45.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.807+0000 7f0ae95fb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55bf95fd82a0 con 0x55bf9607d940 2024-09-12T03:00:45.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.807+0000 7f0ae95fb740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55bf95f7fb10 con 0x55bf96073080 2024-09-12T03:00:45.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.807+0000 7f0ae95fb740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55bf95fd4260 con 0x55bf96072760 2024-09-12T03:00:45.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.807+0000 7f0ae57bf640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bf96072760 0x55bf96072b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:45.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.807+0000 7f0ae57bf640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bf96072760 0x55bf96072b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.5:36480/0 (socket says 172.21.15.5:36480) 2024-09-12T03:00:45.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.807+0000 7f0ae57bf640 1 -- 172.21.15.5:0/2018454118 learned_addr learned my addr 172.21.15.5:0/2018454118 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:00:45.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.807+0000 7f0ae4fbe640 1 --2- 172.21.15.5:0/2018454118 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bf96073080 0x55bf9607d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:45.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.807+0000 7f0adffff640 1 --2- 172.21.15.5:0/2018454118 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf9607d940 0x55bf9607fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:45.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.808+0000 7f0ae57bf640 1 -- 172.21.15.5:0/2018454118 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bf96073080 msgr2=0x55bf9607d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:00:45.809 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.808+0000 7f0ae57bf640 1 --2- 172.21.15.5:0/2018454118 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bf96073080 0x55bf9607d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:45.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.808+0000 7f0ae57bf640 1 -- 172.21.15.5:0/2018454118 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf9607d940 msgr2=0x55bf9607fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:00:45.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.808+0000 7f0ae57bf640 1 --2- 172.21.15.5:0/2018454118 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf9607d940 0x55bf9607fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:45.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.808+0000 7f0ae57bf640 1 -- 172.21.15.5:0/2018454118 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf95d49b90 con 0x55bf96072760 2024-09-12T03:00:45.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.808+0000 7f0ae4fbe640 1 --2- 172.21.15.5:0/2018454118 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bf96073080 0x55bf9607d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:00:45.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.808+0000 7f0addffb640 1 --2- 172.21.15.5:0/2631069164 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f0ad00a2b80 0x7f0ad00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:45.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.808+0000 7f0ae57bf640 1 --2- 172.21.15.5:0/2018454118 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bf96072760 0x55bf96072b40 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0ad8009f50 tx=0x7f0ad8009500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8df721267f68a5cb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:45.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.808+0000 7f0ae4fbe640 1 --2- 172.21.15.5:0/2631069164 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f0ad00a2b80 0x7f0ad00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:45.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.808+0000 7f0addffb640 1 -- 172.21.15.5:0/2631069164 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f0ad40cf3e0 con 0x7f0ae0100e20 2024-09-12T03:00:45.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.808+0000 7f0acaffd640 1 -- 172.21.15.5:0/2018454118 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad8003c30 con 0x55bf96072760 2024-09-12T03:00:45.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.808+0000 7f0acaffd640 1 -- 172.21.15.5:0/2018454118 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ad801f070 con 0x55bf96072760 2024-09-12T03:00:45.810 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.808+0000 7f0acaffd640 1 -- 172.21.15.5:0/2018454118 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad800ba60 con 0x55bf96072760 2024-09-12T03:00:45.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.809+0000 7f0ae95fb740 1 -- 172.21.15.5:0/2018454118 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bf96072760 msgr2=0x55bf96072b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:45.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.809+0000 7f0ae95fb740 1 --2- 172.21.15.5:0/2018454118 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bf96072760 0x55bf96072b40 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0ad8009f50 tx=0x7f0ad8009500 comp rx=0 tx=0).stop 2024-09-12T03:00:45.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.809+0000 7f0ae4fbe640 1 --2- 172.21.15.5:0/2631069164 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f0ad00a2b80 0x7f0ad00a5040 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f0ad40cd260 tx=0x7f0ad40d2ec0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:45.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.809+0000 7f0ae95fb740 1 -- 172.21.15.5:0/2018454118 shutdown_connections 2024-09-12T03:00:45.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.809+0000 7f0ae95fb740 1 --2- 172.21.15.5:0/2018454118 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf9607d940 0x55bf9607fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:45.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.809+0000 7f0ae95fb740 1 --2- 172.21.15.5:0/2018454118 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bf96073080 0x55bf9607d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:45.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.809+0000 7f0ae95fb740 1 --2- 172.21.15.5:0/2018454118 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bf96072760 0x55bf96072b40 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:45.813 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.809+0000 7f0ae95fb740 1 -- 172.21.15.5:0/2018454118 >> 172.21.15.5:0/2018454118 conn(0x55bf9606ecf0 msgr2=0x55bf9606f0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:45.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.810+0000 7f0ae95fb740 1 -- 172.21.15.5:0/2018454118 shutdown_connections 2024-09-12T03:00:45.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.810+0000 7f0ae95fb740 1 -- 172.21.15.5:0/2018454118 wait complete. 2024-09-12T03:00:45.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.810+0000 7f0ae95fb740 1 -- start start 2024-09-12T03:00:45.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.810+0000 7f0ae95fb740 1 Processor -- start 2024-09-12T03:00:45.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.811+0000 7f0ae95fb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf96072760 0x55bf9607a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:45.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.811+0000 7f0ae95fb740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bf96073080 0x55bf9607ad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:45.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.811+0000 7f0ae95fb740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bf9607d940 0x55bf9607b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:45.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.811+0000 7f0ae95fb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55bf96080510 con 0x55bf96072760 2024-09-12T03:00:45.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.811+0000 7f0ae95fb740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55bf95f606e0 con 0x55bf96073080 2024-09-12T03:00:45.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.811+0000 7f0ae95fb740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55bf96080690 con 0x55bf9607d940 2024-09-12T03:00:45.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.811+0000 7f0ae95fb740 10 client.0 _ll_register_callbacks cb 0x55bf95f84a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:00:45.814 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.811+0000 7f0ae57bf640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf96072760 0x55bf9607a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:45.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.811+0000 7f0ae57bf640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf96072760 0x55bf9607a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39562/0 (socket says 172.21.15.5:39562) 2024-09-12T03:00:45.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.811+0000 7f0adffff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bf96073080 0x55bf9607ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:45.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.811+0000 7f0ae57bf640 1 -- 172.21.15.5:0/1408901690 learned_addr learned my addr 172.21.15.5:0/1408901690 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:00:45.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.811+0000 7f0ae57bf640 1 --2- 172.21.15.5:0/1408901690 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bf9607d940 0x55bf9607b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:45.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.811+0000 7f0adffff640 1 -- 172.21.15.5:0/1408901690 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bf9607d940 msgr2=0x55bf9607b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:00:45.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.811+0000 7f0adffff640 1 --2- 172.21.15.5:0/1408901690 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bf9607d940 0x55bf9607b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:45.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.811+0000 7f0adffff640 1 -- 172.21.15.5:0/1408901690 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf96072760 msgr2=0x55bf9607a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:00:45.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.811+0000 7f0adffff640 1 --2- 172.21.15.5:0/1408901690 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf96072760 0x55bf9607a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:45.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.811+0000 7f0adffff640 1 -- 172.21.15.5:0/1408901690 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf96089a30 con 0x55bf96073080 2024-09-12T03:00:45.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.811+0000 7f0ae57bf640 1 --2- 172.21.15.5:0/1408901690 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf96072760 0x55bf9607a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:00:45.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.811+0000 7f0adffff640 1 --2- 172.21.15.5:0/1408901690 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bf96073080 0x55bf9607ad80 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0acc02a9a0 tx=0x7f0acc005270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=476b1f7876131230 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:45.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.811+0000 7f0abffff640 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2024-09-12T03:00:45.815 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.812+0000 7f0abffff640 1 -- 172.21.15.5:0/1408901690 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0acc035040 con 0x55bf96073080 2024-09-12T03:00:45.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.812+0000 7f0abffff640 1 -- 172.21.15.5:0/1408901690 <== mon.2 v2:172.21.15.146:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f0acc004760 con 0x55bf96073080 2024-09-12T03:00:45.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.812+0000 7f0ae95fb740 1 -- 172.21.15.5:0/1408901690 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ad801f070 con 0x55bf96073080 2024-09-12T03:00:45.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.812+0000 7f0abffff640 1 -- 172.21.15.5:0/1408901690 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0acc004960 con 0x55bf96073080 2024-09-12T03:00:45.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.812+0000 7f0ae95fb740 10 client.25430 fetch_fsmap learned FSMap version 28 2024-09-12T03:00:45.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.812+0000 7f0ae95fb740 1 -- 172.21.15.5:0/1408901690 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55bf9608c3a0 con 0x55bf96073080 2024-09-12T03:00:45.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.812+0000 7f0abffff640 1 -- 172.21.15.5:0/1408901690 <== mon.2 v2:172.21.15.146:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ad801f070 con 0x55bf96073080 2024-09-12T03:00:45.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.812+0000 7f0abffff640 1 -- 172.21.15.5:0/1408901690 <== mon.2 v2:172.21.15.146:3300/0 5 ==== fsmap(e 28) ==== 8374+0+0 (secure 0 0 0) 0x7f0acc031540 con 0x55bf96073080 2024-09-12T03:00:45.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.812+0000 7f0ae95fb740 10 client.25430 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T03:00:45.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.813+0000 7f0ae95fb740 10 client.25430 resolve_mds: validated gid 24502 aka daemon mds.i 2024-09-12T03:00:45.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.813+0000 7f0ae95fb740 20 client.25430 populate_metadata read hostname 'smithi005' 2024-09-12T03:00:45.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.813+0000 7f0ae95fb740 1 --2- 172.21.15.5:0/1408901690 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55bf9608dcb0 0x55bf96090130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:45.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.813+0000 7f0ae95fb740 4 client.25430 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:00:45.816 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.813+0000 7f0ae95fb740 1 -- 172.21.15.5:0/1408901690 --> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bf95fe59d0 con 0x55bf9608dcb0 2024-09-12T03:00:45.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.813+0000 7f0ae57bf640 1 --2- 172.21.15.5:0/1408901690 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55bf9608dcb0 0x55bf96090130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:45.817 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.813+0000 7f0ae57bf640 1 --2- 172.21.15.5:0/1408901690 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55bf9608dcb0 0x55bf96090130 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5961564aa54cb4f6 server_cookie=528968192bfe3d7c in_seq=0 out_seq=0 2024-09-12T03:00:45.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.813+0000 7f0abffff640 10 client.25430 ms_handle_connect on v2:172.21.15.148:6838/2317600000 2024-09-12T03:00:45.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.815+0000 7f0abffff640 1 -- 172.21.15.5:0/1408901690 <== mds.0 v2:172.21.15.148:6838/2317600000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bf95fe59d0 con 0x55bf9608dcb0 2024-09-12T03:00:45.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.815+0000 7f0abffff640 10 client.25430 handle_command_reply: tid=0 2024-09-12T03:00:45.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.815+0000 7f0ae95fb740 1 client.25430 shutdown 2024-09-12T03:00:45.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.815+0000 7f0ae95fb740 1 -- 172.21.15.5:0/1408901690 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bf96073080 msgr2=0x55bf9607ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:45.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.815+0000 7f0ae95fb740 1 --2- 172.21.15.5:0/1408901690 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bf96073080 0x55bf9607ad80 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0acc02a9a0 tx=0x7f0acc005270 comp rx=0 tx=0).stop 2024-09-12T03:00:45.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.815+0000 7f0ae95fb740 1 -- 172.21.15.5:0/1408901690 shutdown_connections 2024-09-12T03:00:45.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.815+0000 7f0ae95fb740 1 --2- 172.21.15.5:0/1408901690 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bf9607d940 0x55bf9607b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:45.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.815+0000 7f0ae95fb740 1 --2- 172.21.15.5:0/1408901690 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bf96073080 0x55bf9607ad80 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:45.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.815+0000 7f0ae95fb740 1 --2- 172.21.15.5:0/1408901690 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55bf9608dcb0 0x55bf96090130 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:45.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.815+0000 7f0ae95fb740 1 --2- 172.21.15.5:0/1408901690 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf96072760 0x55bf9607a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:45.818 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.815+0000 7f0abffff640 0 client.25430 ms_handle_reset on v2:172.21.15.148:6838/2317600000 2024-09-12T03:00:45.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.815+0000 7f0ae95fb740 1 -- 172.21.15.5:0/1408901690 >> 172.21.15.5:0/1408901690 conn(0x55bf9606ecf0 msgr2=0x55bf9606de70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:45.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.815+0000 7f0ae95fb740 1 -- 172.21.15.5:0/1408901690 shutdown_connections 2024-09-12T03:00:45.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.815+0000 7f0ae95fb740 1 -- 172.21.15.5:0/1408901690 wait complete. 2024-09-12T03:00:45.819 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.815+0000 7f0ae95fb740 20 client.25430 trim_cache size 0 max 16384 2024-09-12T03:00:45.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.833+0000 7f0ae95fb740 1 Processor -- start 2024-09-12T03:00:45.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.833+0000 7f0ae95fb740 1 -- start start 2024-09-12T03:00:45.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.833+0000 7f0ae95fb740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bf96072760 0x55bf96095a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:45.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.833+0000 7f0ae95fb740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bf96073080 0x55bf96095f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:45.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.833+0000 7f0ae95fb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf9607d940 0x55bf96077ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:45.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.833+0000 7f0ae95fb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae01ab3b0 con 0x55bf9607d940 2024-09-12T03:00:45.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.833+0000 7f0ae95fb740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae01acf80 con 0x55bf96072760 2024-09-12T03:00:45.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.833+0000 7f0ae95fb740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae01aba80 con 0x55bf96073080 2024-09-12T03:00:45.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.834+0000 7f0ae57bf640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bf96072760 0x55bf96095a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:45.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.834+0000 7f0adffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bf96073080 0x55bf96095f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:45.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.834+0000 7f0ae57bf640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bf96072760 0x55bf96095a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.5:52306/0 (socket says 172.21.15.5:52306) 2024-09-12T03:00:45.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.834+0000 7f0adffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bf96073080 0x55bf96095f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.5:36498/0 (socket says 172.21.15.5:36498) 2024-09-12T03:00:45.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.834+0000 7f0ae57bf640 1 -- 172.21.15.5:0/1889667269 learned_addr learned my addr 172.21.15.5:0/1889667269 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:00:45.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.834+0000 7f0ae57bf640 1 --2- 172.21.15.5:0/1889667269 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf9607d940 0x55bf96077ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:45.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.834+0000 7f0ae57bf640 1 -- 172.21.15.5:0/1889667269 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bf96073080 msgr2=0x55bf96095f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:00:45.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.834+0000 7f0ae57bf640 1 --2- 172.21.15.5:0/1889667269 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bf96073080 0x55bf96095f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:45.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.834+0000 7f0ae57bf640 1 -- 172.21.15.5:0/1889667269 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf9607d940 msgr2=0x55bf96077ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:00:45.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.834+0000 7f0ae57bf640 1 --2- 172.21.15.5:0/1889667269 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf9607d940 0x55bf96077ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:45.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.834+0000 7f0ae57bf640 1 -- 172.21.15.5:0/1889667269 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf9606eb20 con 0x55bf96072760 2024-09-12T03:00:45.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.834+0000 7f0adffff640 1 --2- 172.21.15.5:0/1889667269 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bf96073080 0x55bf96095f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:00:45.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.834+0000 7f0ae57bf640 1 --2- 172.21.15.5:0/1889667269 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bf96072760 0x55bf96095a40 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0ad8009ef0 tx=0x7f0ad800b680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c91feee6fa7ea291 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:45.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.834+0000 7f0aca7fc640 1 -- 172.21.15.5:0/1889667269 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad8038070 con 0x55bf96072760 2024-09-12T03:00:45.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.834+0000 7f0aca7fc640 1 -- 172.21.15.5:0/1889667269 <== mon.2 v2:172.21.15.146:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ad801f030 con 0x55bf96072760 2024-09-12T03:00:45.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.835+0000 7f0aca7fc640 1 -- 172.21.15.5:0/1889667269 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad8004320 con 0x55bf96072760 2024-09-12T03:00:45.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.835+0000 7f0ae95fb740 1 -- 172.21.15.5:0/1889667269 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bf96072760 msgr2=0x55bf96095a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:45.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.835+0000 7f0ae95fb740 1 --2- 172.21.15.5:0/1889667269 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bf96072760 0x55bf96095a40 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0ad8009ef0 tx=0x7f0ad800b680 comp rx=0 tx=0).stop 2024-09-12T03:00:45.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.835+0000 7f0ae95fb740 1 -- 172.21.15.5:0/1889667269 shutdown_connections 2024-09-12T03:00:45.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.835+0000 7f0ae95fb740 1 --2- 172.21.15.5:0/1889667269 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf9607d940 0x55bf96077ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:45.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.835+0000 7f0ae95fb740 1 --2- 172.21.15.5:0/1889667269 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bf96073080 0x55bf96095f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:45.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.835+0000 7f0ae95fb740 1 --2- 172.21.15.5:0/1889667269 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bf96072760 0x55bf96095a40 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:45.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.835+0000 7f0ae95fb740 1 -- 172.21.15.5:0/1889667269 >> 172.21.15.5:0/1889667269 conn(0x55bf9606ecf0 msgr2=0x55bf9607c8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:45.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.835+0000 7f0ae95fb740 1 -- 172.21.15.5:0/1889667269 shutdown_connections 2024-09-12T03:00:45.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.835+0000 7f0ae95fb740 1 -- 172.21.15.5:0/1889667269 wait complete. 2024-09-12T03:00:45.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.836+0000 7f0ae95fb740 1 -- start start 2024-09-12T03:00:45.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.836+0000 7f0ae95fb740 1 Processor -- start 2024-09-12T03:00:45.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.836+0000 7f0ae95fb740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bf96072760 0x55bf96085ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:45.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.836+0000 7f0ae95fb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf96073080 0x55bf96086000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:45.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.836+0000 7f0ae95fb740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bf9607d940 0x55bf960828b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:45.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.836+0000 7f0ae95fb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae01ac820 con 0x55bf96073080 2024-09-12T03:00:45.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.836+0000 7f0ae95fb740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae01ac150 con 0x55bf96072760 2024-09-12T03:00:45.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.836+0000 7f0ae95fb740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae01a93b0 con 0x55bf9607d940 2024-09-12T03:00:45.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.836+0000 7f0ae95fb740 10 client.0 _ll_register_callbacks cb 0x7f0acc0043f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:00:45.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.836+0000 7f0adffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf96073080 0x55bf96086000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:45.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.836+0000 7f0adffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf96073080 0x55bf96086000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39574/0 (socket says 172.21.15.5:39574) 2024-09-12T03:00:45.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.837+0000 7f0adffff640 1 -- 172.21.15.5:0/1726486192 learned_addr learned my addr 172.21.15.5:0/1726486192 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:00:45.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.837+0000 7f0ae57bf640 1 --2- 172.21.15.5:0/1726486192 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bf96072760 0x55bf96085ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:45.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.837+0000 7f0ae57bf640 1 --2- 172.21.15.5:0/1726486192 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bf9607d940 0x55bf960828b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:45.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.837+0000 7f0ae57bf640 1 -- 172.21.15.5:0/1726486192 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bf9607d940 msgr2=0x55bf960828b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:00:45.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.837+0000 7f0ae57bf640 1 --2- 172.21.15.5:0/1726486192 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bf9607d940 0x55bf960828b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:45.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.837+0000 7f0ae57bf640 1 -- 172.21.15.5:0/1726486192 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf96073080 msgr2=0x55bf96086000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:00:45.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.837+0000 7f0ae57bf640 1 --2- 172.21.15.5:0/1726486192 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf96073080 0x55bf96086000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:45.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.837+0000 7f0ae57bf640 1 -- 172.21.15.5:0/1726486192 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf96082f40 con 0x55bf96072760 2024-09-12T03:00:45.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.837+0000 7f0adffff640 1 --2- 172.21.15.5:0/1726486192 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf96073080 0x55bf96086000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:00:45.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.837+0000 7f0ae57bf640 1 --2- 172.21.15.5:0/1726486192 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bf96072760 0x55bf96085ac0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0ad8006010 tx=0x7f0ad8033f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e24200c12d9c4d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:00:45.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.837+0000 7f0abffff640 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2024-09-12T03:00:45.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.837+0000 7f0abffff640 1 -- 172.21.15.5:0/1726486192 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad8038070 con 0x55bf96072760 2024-09-12T03:00:45.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.837+0000 7f0abffff640 1 -- 172.21.15.5:0/1726486192 <== mon.2 v2:172.21.15.146:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f0ad80043b0 con 0x55bf96072760 2024-09-12T03:00:45.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.837+0000 7f0ae95fb740 1 -- 172.21.15.5:0/1726486192 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ad801f030 con 0x55bf96072760 2024-09-12T03:00:45.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.838+0000 7f0abffff640 1 -- 172.21.15.5:0/1726486192 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0ad80045b0 con 0x55bf96072760 2024-09-12T03:00:45.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.838+0000 7f0ae95fb740 10 client.25436 fetch_fsmap learned FSMap version 28 2024-09-12T03:00:45.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.838+0000 7f0ae95fb740 1 -- 172.21.15.5:0/1726486192 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55bf960f9380 con 0x55bf96072760 2024-09-12T03:00:45.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.838+0000 7f0abffff640 1 -- 172.21.15.5:0/1726486192 <== mon.2 v2:172.21.15.146:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ad801f030 con 0x55bf96072760 2024-09-12T03:00:45.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.838+0000 7f0abffff640 1 -- 172.21.15.5:0/1726486192 <== mon.2 v2:172.21.15.146:3300/0 5 ==== fsmap(e 28) ==== 8374+0+0 (secure 0 0 0) 0x7f0ad8025530 con 0x55bf96072760 2024-09-12T03:00:45.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.838+0000 7f0ae95fb740 10 client.25436 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T03:00:45.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.838+0000 7f0ae95fb740 10 client.25436 resolve_mds: validated gid 24502 aka daemon mds.i 2024-09-12T03:00:45.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.838+0000 7f0ae95fb740 20 client.25436 populate_metadata read hostname 'smithi005' 2024-09-12T03:00:45.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.838+0000 7f0ae95fb740 1 --2- 172.21.15.5:0/1726486192 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55bf960f98d0 0x55bf960fbcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:00:45.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.838+0000 7f0ae95fb740 4 client.25436 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.7, "await_for": 45.0}] 2024-09-12T03:00:45.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.838+0000 7f0ae95fb740 1 -- 172.21.15.5:0/1726486192 --> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.7, "await_for": 45.0}) -- 0x55bf95fdfd20 con 0x55bf960f98d0 2024-09-12T03:00:45.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.838+0000 7f0adffff640 1 --2- 172.21.15.5:0/1726486192 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55bf960f98d0 0x55bf960fbcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:00:45.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.839+0000 7f0abffff640 10 client.25436 ms_handle_connect on v2:172.21.15.148:6838/2317600000 2024-09-12T03:00:45.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:45.839+0000 7f0adffff640 1 --2- 172.21.15.5:0/1726486192 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55bf960f98d0 0x55bf960fbcf0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=949be36dba736bc7 server_cookie=5215e0dc6c653009 in_seq=0 out_seq=0 2024-09-12T03:00:47.665 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:00:47 smithi148 ceph-mon[35090]: pgmap v411: 97 pgs: 97 active+clean; 79 MiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.5 MiB/s wr, 533 op/s 2024-09-12T03:00:47.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:00:47 smithi146 ceph-mon[35011]: pgmap v411: 97 pgs: 97 active+clean; 79 MiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.5 MiB/s wr, 533 op/s 2024-09-12T03:00:47.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:00:47 smithi005 ceph-mon[31197]: pgmap v411: 97 pgs: 97 active+clean; 79 MiB data, 691 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.5 MiB/s wr, 533 op/s 2024-09-12T03:00:47.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:47.933+0000 7f0abffff640 1 -- 172.21.15.5:0/1726486192 <== mds.0 v2:172.21.15.148:6838/2317600000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55bf95fdfd20 con 0x55bf960f98d0 2024-09-12T03:00:47.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:47.933+0000 7f0abffff640 10 client.25436 handle_command_reply: tid=0 2024-09-12T03:00:47.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:47.933+0000 7f0ae95fb740 1 client.25436 shutdown 2024-09-12T03:00:47.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:47.933+0000 7f0ae95fb740 1 -- 172.21.15.5:0/1726486192 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bf96072760 msgr2=0x55bf96085ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:47.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:47.933+0000 7f0ae95fb740 1 --2- 172.21.15.5:0/1726486192 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bf96072760 0x55bf96085ac0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0ad8006010 tx=0x7f0ad8033f90 comp rx=0 tx=0).stop 2024-09-12T03:00:47.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:47.933+0000 7f0ae95fb740 1 -- 172.21.15.5:0/1726486192 shutdown_connections 2024-09-12T03:00:47.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:47.933+0000 7f0ae95fb740 1 --2- 172.21.15.5:0/1726486192 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55bf960f98d0 0x55bf960fbcf0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:47.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:47.933+0000 7f0ae95fb740 1 --2- 172.21.15.5:0/1726486192 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf96073080 0x55bf96086000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:47.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:47.933+0000 7f0ae95fb740 1 --2- 172.21.15.5:0/1726486192 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bf9607d940 0x55bf960828b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:47.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:47.933+0000 7f0abffff640 0 client.25436 ms_handle_reset on v2:172.21.15.148:6838/2317600000 2024-09-12T03:00:47.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:47.933+0000 7f0ae95fb740 1 --2- 172.21.15.5:0/1726486192 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bf96072760 0x55bf96085ac0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:47.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:47.933+0000 7f0ae95fb740 1 -- 172.21.15.5:0/1726486192 >> 172.21.15.5:0/1726486192 conn(0x55bf9606ecf0 msgr2=0x55bf9607c0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:47.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:47.933+0000 7f0ae95fb740 1 -- 172.21.15.5:0/1726486192 shutdown_connections 2024-09-12T03:00:47.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:47.933+0000 7f0ae95fb740 1 -- 172.21.15.5:0/1726486192 wait complete. 2024-09-12T03:00:47.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:47.933+0000 7f0ae95fb740 20 client.25436 trim_cache size 0 max 16384 2024-09-12T03:00:47.935 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-12T03:00:47.935 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 28, 2024-09-12T03:00:47.935 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24502, 2024-09-12T03:00:47.936 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 10, 2024-09-12T03:00:47.936 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-09-12T03:00:47.936 INFO:teuthology.orchestra.run.smithi005.stdout: "4d66795": { 2024-09-12T03:00:47.936 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 10, 2024-09-12T03:00:47.936 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-09-12T03:00:47.936 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-12T03:00:47.936 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-09-12T03:00:47.936 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-12T03:00:47.936 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:00:47.936 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 90.0, 2024-09-12T03:00:47.936 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 144.7, 2024-09-12T03:00:47.936 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-09-12T03:00:47.937 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-09-12T03:00:47.937 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-09-12T03:00:47.937 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-12T03:00:47.937 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-09-12T03:00:47.937 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-12T03:00:47.937 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:00:47.937 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:00:47.937 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:00:47.937 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:00:47.937 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:00:47.937 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-12T03:00:47.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:47.934+0000 7f0ae7222640 1 -- 172.21.15.5:0/2631069164 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f0ad00a2b80 msgr2=0x7f0ad00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:47.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:47.934+0000 7f0ae7222640 1 --2- 172.21.15.5:0/2631069164 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f0ad00a2b80 0x7f0ad00a5040 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f0ad40cd260 tx=0x7f0ad40d2ec0 comp rx=0 tx=0).stop 2024-09-12T03:00:47.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:47.934+0000 7f0ae7222640 1 -- 172.21.15.5:0/2631069164 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ae0100e20 msgr2=0x7f0ae0199300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:00:47.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:47.934+0000 7f0ae7222640 1 --2- 172.21.15.5:0/2631069164 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ae0100e20 0x7f0ae0199300 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f0ad400e9f0 tx=0x7f0ad400eeb0 comp rx=0 tx=0).stop 2024-09-12T03:00:47.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:47.934+0000 7f0ae7222640 1 -- 172.21.15.5:0/2631069164 shutdown_connections 2024-09-12T03:00:47.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:47.934+0000 7f0ae7222640 1 --2- 172.21.15.5:0/2631069164 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0ae0102ef0 0x7f0ae01a4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:47.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:47.934+0000 7f0ae7222640 1 --2- 172.21.15.5:0/2631069164 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0ae0102570 0x7f0ae0199840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:47.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:47.934+0000 7f0ae7222640 1 --2- 172.21.15.5:0/2631069164 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f0ad00a2b80 0x7f0ad00a5040 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:47.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:47.934+0000 7f0ae7222640 1 --2- 172.21.15.5:0/2631069164 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ae0100e20 0x7f0ae0199300 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:00:47.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:47.934+0000 7f0ae7222640 1 -- 172.21.15.5:0/2631069164 >> 172.21.15.5:0/2631069164 conn(0x7f0ae00fc7d0 msgr2=0x7f0ae0106840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:00:47.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:47.935+0000 7f0ae7222640 1 -- 172.21.15.5:0/2631069164 shutdown_connections 2024-09-12T03:00:47.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:00:47.935+0000 7f0ae7222640 1 -- 172.21.15.5:0/2631069164 wait complete. 2024-09-12T03:00:48.665 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:00:48 smithi148 ceph-mon[35090]: scrub summary: idle 2024-09-12T03:00:48.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:00:48 smithi148 ceph-mon[35090]: scrub summary: idle 2024-09-12T03:00:48.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:00:48 smithi148 ceph-mon[35090]: scrub summary: idle 2024-09-12T03:00:48.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:00:48 smithi146 ceph-mon[35011]: scrub summary: idle 2024-09-12T03:00:48.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:00:48 smithi146 ceph-mon[35011]: scrub summary: idle 2024-09-12T03:00:48.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:00:48 smithi146 ceph-mon[35011]: scrub summary: idle 2024-09-12T03:00:48.763 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:00:48 smithi005 ceph-mon[31197]: scrub summary: idle 2024-09-12T03:00:48.763 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:00:48 smithi005 ceph-mon[31197]: scrub summary: idle 2024-09-12T03:00:48.764 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:00:48 smithi005 ceph-mon[31197]: scrub summary: idle 2024-09-12T03:00:48.800 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '4d66795', quiesce took 5.8 seconds. Will release after: 18.9 2024-09-12T03:00:49.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:00:49 smithi148 ceph-mon[35090]: pgmap v412: 97 pgs: 97 active+clean; 82 MiB data, 681 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.6 MiB/s wr, 580 op/s 2024-09-12T03:00:49.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:00:49 smithi148 ceph-mon[35090]: scrub summary: idle 2024-09-12T03:00:49.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:00:49 smithi146 ceph-mon[35011]: pgmap v412: 97 pgs: 97 active+clean; 82 MiB data, 681 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.6 MiB/s wr, 580 op/s 2024-09-12T03:00:49.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:00:49 smithi146 ceph-mon[35011]: scrub summary: idle 2024-09-12T03:00:49.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:00:49 smithi005 ceph-mon[31197]: pgmap v412: 97 pgs: 97 active+clean; 82 MiB data, 681 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.6 MiB/s wr, 580 op/s 2024-09-12T03:00:49.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:00:49 smithi005 ceph-mon[31197]: scrub summary: idle 2024-09-12T03:00:51.665 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:00:51 smithi148 ceph-mon[35090]: pgmap v413: 97 pgs: 97 active+clean; 82 MiB data, 694 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.9 MiB/s wr, 608 op/s 2024-09-12T03:00:51.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:00:51 smithi146 ceph-mon[35011]: pgmap v413: 97 pgs: 97 active+clean; 82 MiB data, 694 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.9 MiB/s wr, 608 op/s 2024-09-12T03:00:51.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:00:51 smithi005 ceph-mon[31197]: pgmap v413: 97 pgs: 97 active+clean; 82 MiB data, 694 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.9 MiB/s wr, 608 op/s 2024-09-12T03:00:53.665 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:00:53 smithi148 ceph-mon[35090]: pgmap v414: 97 pgs: 97 active+clean; 82 MiB data, 695 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.1 MiB/s wr, 481 op/s 2024-09-12T03:00:53.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:00:53 smithi146 ceph-mon[35011]: pgmap v414: 97 pgs: 97 active+clean; 82 MiB data, 695 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.1 MiB/s wr, 481 op/s 2024-09-12T03:00:53.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:00:53 smithi005 ceph-mon[31197]: pgmap v414: 97 pgs: 97 active+clean; 82 MiB data, 695 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.1 MiB/s wr, 481 op/s 2024-09-12T03:00:55.665 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:00:55 smithi148 ceph-mon[35090]: pgmap v415: 97 pgs: 97 active+clean; 82 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.1 MiB/s wr, 484 op/s 2024-09-12T03:00:55.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:00:55 smithi146 ceph-mon[35011]: pgmap v415: 97 pgs: 97 active+clean; 82 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.1 MiB/s wr, 484 op/s 2024-09-12T03:00:55.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:00:55 smithi005 ceph-mon[31197]: pgmap v415: 97 pgs: 97 active+clean; 82 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.1 MiB/s wr, 484 op/s 2024-09-12T03:00:57.665 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:00:57 smithi148 ceph-mon[35090]: pgmap v416: 97 pgs: 97 active+clean; 82 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.1 MiB/s wr, 484 op/s 2024-09-12T03:00:57.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:00:57 smithi146 ceph-mon[35011]: pgmap v416: 97 pgs: 97 active+clean; 82 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.1 MiB/s wr, 484 op/s 2024-09-12T03:00:57.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:00:57 smithi005 ceph-mon[31197]: pgmap v416: 97 pgs: 97 active+clean; 82 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.1 MiB/s wr, 484 op/s 2024-09-12T03:00:59.665 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:00:59 smithi148 ceph-mon[35090]: pgmap v417: 97 pgs: 97 active+clean; 82 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 753 KiB/s rd, 913 KiB/s wr, 154 op/s 2024-09-12T03:00:59.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:00:59 smithi146 ceph-mon[35011]: pgmap v417: 97 pgs: 97 active+clean; 82 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 753 KiB/s rd, 913 KiB/s wr, 154 op/s 2024-09-12T03:00:59.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:00:59 smithi005 ceph-mon[31197]: pgmap v417: 97 pgs: 97 active+clean; 82 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 753 KiB/s rd, 913 KiB/s wr, 154 op/s 2024-09-12T03:01:01.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:01:01 smithi148 ceph-mon[35090]: pgmap v418: 97 pgs: 97 active+clean; 82 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 609 KiB/s rd, 609 KiB/s wr, 80 op/s 2024-09-12T03:01:01.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:01:01 smithi146 ceph-mon[35011]: pgmap v418: 97 pgs: 97 active+clean; 82 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 609 KiB/s rd, 609 KiB/s wr, 80 op/s 2024-09-12T03:01:01.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:01:01 smithi005 ceph-mon[31197]: pgmap v418: 97 pgs: 97 active+clean; 82 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 609 KiB/s rd, 609 KiB/s wr, 80 op/s 2024-09-12T03:01:03.665 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:01:03 smithi148 ceph-mon[35090]: pgmap v419: 97 pgs: 97 active+clean; 82 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 32 KiB/s wr, 12 op/s 2024-09-12T03:01:03.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:01:03 smithi146 ceph-mon[35011]: pgmap v419: 97 pgs: 97 active+clean; 82 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 32 KiB/s wr, 12 op/s 2024-09-12T03:01:03.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:01:03 smithi005 ceph-mon[31197]: pgmap v419: 97 pgs: 97 active+clean; 82 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 32 KiB/s wr, 12 op/s 2024-09-12T03:01:05.665 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:01:05 smithi148 ceph-mon[35090]: pgmap v420: 97 pgs: 97 active+clean; 82 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 10 KiB/s wr, 13 op/s 2024-09-12T03:01:05.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:01:05 smithi146 ceph-mon[35011]: pgmap v420: 97 pgs: 97 active+clean; 82 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 10 KiB/s wr, 13 op/s 2024-09-12T03:01:05.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:01:05 smithi005 ceph-mon[31197]: pgmap v420: 97 pgs: 97 active+clean; 82 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 10 KiB/s wr, 13 op/s 2024-09-12T03:01:07.665 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:01:07 smithi148 ceph-mon[35090]: pgmap v421: 97 pgs: 97 active+clean; 82 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.9 KiB/s wr, 12 op/s 2024-09-12T03:01:07.701 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:01:07.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:01:07 smithi146 ceph-mon[35011]: pgmap v421: 97 pgs: 97 active+clean; 82 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.9 KiB/s wr, 12 op/s 2024-09-12T03:01:07.732 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:01:07 smithi005 ceph-mon[31197]: pgmap v421: 97 pgs: 97 active+clean; 82 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.9 KiB/s wr, 12 op/s 2024-09-12T03:01:07.976 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:01:08.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:08.475+0000 7f5efa6fa640 1 -- 172.21.15.5:0/536523062 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ef410c170 msgr2=0x7f5ef410e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:08.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:08.475+0000 7f5efa6fa640 1 --2- 172.21.15.5:0/536523062 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ef410c170 0x7f5ef410e610 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f5eec009ca0 tx=0x7f5eec02a950 comp rx=0 tx=0).stop 2024-09-12T03:01:08.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:08.475+0000 7f5efa6fa640 1 -- 172.21.15.5:0/536523062 shutdown_connections 2024-09-12T03:01:08.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:08.475+0000 7f5efa6fa640 1 --2- 172.21.15.5:0/536523062 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ef410c170 0x7f5ef410e610 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:08.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:08.475+0000 7f5efa6fa640 1 --2- 172.21.15.5:0/536523062 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5ef4103660 0x7f5ef410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:08.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:08.475+0000 7f5efa6fa640 1 --2- 172.21.15.5:0/536523062 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5ef4102d40 0x7f5ef4103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:08.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:08.475+0000 7f5efa6fa640 1 -- 172.21.15.5:0/536523062 >> 172.21.15.5:0/536523062 conn(0x7f5ef40fc7a0 msgr2=0x7f5ef40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:08.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:08.475+0000 7f5efa6fa640 1 -- 172.21.15.5:0/536523062 shutdown_connections 2024-09-12T03:01:08.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:08.475+0000 7f5efa6fa640 1 -- 172.21.15.5:0/536523062 wait complete. 2024-09-12T03:01:08.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:08.476+0000 7f5efa6fa640 1 Processor -- start 2024-09-12T03:01:08.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:08.476+0000 7f5efa6fa640 1 -- start start 2024-09-12T03:01:08.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:08.476+0000 7f5efa6fa640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5ef4102d40 0x7f5ef41009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:08.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:08.476+0000 7f5efa6fa640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ef4103660 0x7f5ef4100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:08.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:08.476+0000 7f5efa6fa640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5ef410c170 0x7f5ef4101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:08.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:08.476+0000 7f5efa6fa640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5ef4111fb0 con 0x7f5ef4103660 2024-09-12T03:01:08.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:08.476+0000 7f5efa6fa640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f5ef4111e30 con 0x7f5ef410c170 2024-09-12T03:01:08.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:08.476+0000 7f5efa6fa640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5ef4112130 con 0x7f5ef4102d40 2024-09-12T03:01:08.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:08.476+0000 7f5ef37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ef4103660 0x7f5ef4100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:08.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:08.476+0000 7f5ef37fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ef4103660 0x7f5ef4100ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38940/0 (socket says 172.21.15.5:38940) 2024-09-12T03:01:08.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:08.476+0000 7f5ef37fe640 1 -- 172.21.15.5:0/1704019590 learned_addr learned my addr 172.21.15.5:0/1704019590 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:01:08.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:08.476+0000 7f5ef3fff640 1 --2- 172.21.15.5:0/1704019590 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5ef4102d40 0x7f5ef41009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:08.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:08.476+0000 7f5ef8c97640 1 --2- 172.21.15.5:0/1704019590 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5ef410c170 0x7f5ef4101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:08.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:08.477+0000 7f5ef37fe640 1 -- 172.21.15.5:0/1704019590 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5ef4102d40 msgr2=0x7f5ef41009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:08.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:08.477+0000 7f5ef37fe640 1 --2- 172.21.15.5:0/1704019590 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5ef4102d40 0x7f5ef41009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:08.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:08.477+0000 7f5ef37fe640 1 -- 172.21.15.5:0/1704019590 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5ef410c170 msgr2=0x7f5ef4101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:08.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:08.477+0000 7f5ef37fe640 1 --2- 172.21.15.5:0/1704019590 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5ef410c170 0x7f5ef4101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:08.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:08.477+0000 7f5ef37fe640 1 -- 172.21.15.5:0/1704019590 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ef41a6890 con 0x7f5ef4103660 2024-09-12T03:01:08.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:08.477+0000 7f5ef37fe640 1 --2- 172.21.15.5:0/1704019590 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ef4103660 0x7f5ef4100ee0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f5ee800c970 tx=0x7f5ee800cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:08.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:08.477+0000 7f5ef17fa640 1 -- 172.21.15.5:0/1704019590 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ee8007bb0 con 0x7f5ef4103660 2024-09-12T03:01:08.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:08.477+0000 7f5efa6fa640 1 -- 172.21.15.5:0/1704019590 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ef41a6b80 con 0x7f5ef4103660 2024-09-12T03:01:08.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:08.477+0000 7f5ef17fa640 1 -- 172.21.15.5:0/1704019590 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ee8007d50 con 0x7f5ef4103660 2024-09-12T03:01:08.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:08.477+0000 7f5ef17fa640 1 -- 172.21.15.5:0/1704019590 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ee8018730 con 0x7f5ef4103660 2024-09-12T03:01:08.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:08.477+0000 7f5efa6fa640 1 -- 172.21.15.5:0/1704019590 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ef41a7110 con 0x7f5ef4103660 2024-09-12T03:01:08.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:08.478+0000 7f5efa6fa640 1 -- 172.21.15.5:0/1704019590 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ef4103120 con 0x7f5ef4103660 2024-09-12T03:01:08.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:08.479+0000 7f5ef17fa640 1 -- 172.21.15.5:0/1704019590 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f5ee800f450 con 0x7f5ef4103660 2024-09-12T03:01:08.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:08.480+0000 7f5ef17fa640 1 --2- 172.21.15.5:0/1704019590 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f5edc0a2b80 0x7f5edc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:08.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:08.480+0000 7f5ef3fff640 1 --2- 172.21.15.5:0/1704019590 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f5edc0a2b80 0x7f5edc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:08.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:08.480+0000 7f5ef17fa640 1 -- 172.21.15.5:0/1704019590 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f5ee80cf050 con 0x7f5ef4103660 2024-09-12T03:01:08.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:08.481+0000 7f5ef3fff640 1 --2- 172.21.15.5:0/1704019590 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f5edc0a2b80 0x7f5edc0a5040 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f5ee4009990 tx=0x7f5ee4009110 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:08.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:08.484+0000 7f5ef17fa640 1 -- 172.21.15.5:0/1704019590 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5ee8014030 con 0x7f5ef4103660 2024-09-12T03:01:08.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:08.696+0000 7f5efa6fa640 1 -- 172.21.15.5:0/1704019590 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5ef41a72e0 con 0x7f5ef4103660 2024-09-12T03:01:08.697 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:08.697+0000 7f5ef17fa640 1 -- 172.21.15.5:0/1704019590 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11971 (secure 0 0 0) 0x7f5ee809bf60 con 0x7f5ef4103660 2024-09-12T03:01:08.697 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-12T03:01:08.698 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":28,"btime":"2024-09-12T03:00:03:036015+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/4135232521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":4135232521},{"type":"v1","addr":"172.21.15.5:6835","nonce":4135232521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/4290221940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":4290221940},{"type":"v1","addr":"172.21.15.5:6837","nonce":4290221940}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T02:56:16.595562+0000","modified":"2024-09-12T03:00:02.675878+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24502,"mds_1":24487,"mds_2":24481,"mds_3":24479,"mds_4":24458},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/26447341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":26447341},{"type":"v1","addr":"172.21.15.5:6839","nonce":26447341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2393443295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2393443295},{"type":"v1","addr":"172.21.15.5:6841","nonce":2393443295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6833/3686970137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3686970137},{"type":"v1","addr":"172.21.15.146:6833","nonce":3686970137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6833/568512209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":568512209},{"type":"v1","addr":"172.21.15.148:6833","nonce":568512209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6835/2540740420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2540740420},{"type":"v1","addr":"172.21.15.146:6835","nonce":2540740420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.146:6837/1673370937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":1673370937},{"type":"v1","addr":"172.21.15.146:6837","nonce":1673370937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":3,"incarnation":22,"state":"up:active","state_seq":6,"addr":"172.21.15.146:6839/3036767228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":3036767228},{"type":"v1","addr":"172.21.15.146:6839","nonce":3036767228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":2,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.148:6835/4195086427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":4195086427},{"type":"v1","addr":"172.21.15.148:6835","nonce":4195086427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.148:6837/2606873922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2606873922},{"type":"v1","addr":"172.21.15.148:6837","nonce":2606873922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6839/2317600000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":2317600000},{"type":"v1","addr":"172.21.15.148:6839","nonce":2317600000}]},"join_fscid":-1,"export_targets":[1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24502,"qdb_cluster":[24502,24487,24481,24479,24458]},"id":1}]} 2024-09-12T03:01:08.698 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 28 2024-09-12T03:01:08.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:08.699+0000 7f5efa6fa640 1 -- 172.21.15.5:0/1704019590 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f5edc0a2b80 msgr2=0x7f5edc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:08.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:08.699+0000 7f5efa6fa640 1 --2- 172.21.15.5:0/1704019590 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f5edc0a2b80 0x7f5edc0a5040 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f5ee4009990 tx=0x7f5ee4009110 comp rx=0 tx=0).stop 2024-09-12T03:01:08.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:08.699+0000 7f5efa6fa640 1 -- 172.21.15.5:0/1704019590 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ef4103660 msgr2=0x7f5ef4100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:08.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:08.699+0000 7f5efa6fa640 1 --2- 172.21.15.5:0/1704019590 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ef4103660 0x7f5ef4100ee0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f5ee800c970 tx=0x7f5ee800cd70 comp rx=0 tx=0).stop 2024-09-12T03:01:08.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:08.699+0000 7f5efa6fa640 1 -- 172.21.15.5:0/1704019590 shutdown_connections 2024-09-12T03:01:08.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:08.699+0000 7f5efa6fa640 1 --2- 172.21.15.5:0/1704019590 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f5edc0a2b80 0x7f5edc0a5040 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:08.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:08.699+0000 7f5efa6fa640 1 --2- 172.21.15.5:0/1704019590 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ef4103660 0x7f5ef4100ee0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:08.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:08.699+0000 7f5efa6fa640 1 --2- 172.21.15.5:0/1704019590 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5ef410c170 0x7f5ef4101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:08.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:08.699+0000 7f5efa6fa640 1 --2- 172.21.15.5:0/1704019590 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5ef4102d40 0x7f5ef41009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:08.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:08.699+0000 7f5efa6fa640 1 -- 172.21.15.5:0/1704019590 >> 172.21.15.5:0/1704019590 conn(0x7f5ef40fc7a0 msgr2=0x7f5ef40fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:08.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:08.699+0000 7f5efa6fa640 1 -- 172.21.15.5:0/1704019590 shutdown_connections 2024-09-12T03:01:08.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:08.699+0000 7f5efa6fa640 1 -- 172.21.15.5:0/1704019590 wait complete. 2024-09-12T03:01:09.414 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24502 quiesce db --set-id 4d66795 --release --await' 2024-09-12T03:01:09.414 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24502 quiesce db --set-id 4d66795 --release --await 2024-09-12T03:01:09.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:01:09 smithi146 ceph-mon[35011]: pgmap v422: 97 pgs: 97 active+clean; 82 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 2.7 KiB/s wr, 11 op/s 2024-09-12T03:01:09.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:01:09 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/1704019590' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:01:09.717 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:01:09.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:01:09 smithi005 ceph-mon[31197]: pgmap v422: 97 pgs: 97 active+clean; 82 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 2.7 KiB/s wr, 11 op/s 2024-09-12T03:01:09.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:01:09 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/1704019590' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:01:09.866 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T03:01:09.915 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:01:09 smithi148 ceph-mon[35090]: pgmap v422: 97 pgs: 97 active+clean; 82 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 2.7 KiB/s wr, 11 op/s 2024-09-12T03:01:09.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:01:09 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/1704019590' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:01:10.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.198+0000 7fa9e3aeb640 1 -- 172.21.15.5:0/1835548021 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9dc102790 msgr2=0x7fa9dc1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:10.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.198+0000 7fa9e3aeb640 1 --2- 172.21.15.5:0/1835548021 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9dc102790 0x7fa9dc1067e0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc009900 tx=0x7fa9cc02a410 comp rx=0 tx=0).stop 2024-09-12T03:01:10.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.198+0000 7fa9e3aeb640 1 -- 172.21.15.5:0/1835548021 shutdown_connections 2024-09-12T03:01:10.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.198+0000 7fa9e3aeb640 1 --2- 172.21.15.5:0/1835548021 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9dc102790 0x7fa9dc1067e0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:10.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.198+0000 7fa9e3aeb640 1 --2- 172.21.15.5:0/1835548021 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa9dc106f10 0x7fa9dc10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:10.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.198+0000 7fa9e3aeb640 1 --2- 172.21.15.5:0/1835548021 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa9dc101de0 0x7fa9dc1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:10.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.198+0000 7fa9e3aeb640 1 -- 172.21.15.5:0/1835548021 >> 172.21.15.5:0/1835548021 conn(0x7fa9dc0fd8c0 msgr2=0x7fa9dc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:10.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.198+0000 7fa9e3aeb640 1 -- 172.21.15.5:0/1835548021 shutdown_connections 2024-09-12T03:01:10.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.198+0000 7fa9e3aeb640 1 -- 172.21.15.5:0/1835548021 wait complete. 2024-09-12T03:01:10.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.199+0000 7fa9e3aeb640 1 Processor -- start 2024-09-12T03:01:10.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.199+0000 7fa9e3aeb640 1 -- start start 2024-09-12T03:01:10.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.199+0000 7fa9e3aeb640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa9dc101de0 0x7fa9dc0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:10.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.199+0000 7fa9e3aeb640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa9dc102790 0x7fa9dc074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:10.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.199+0000 7fa9e3aeb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9dc106f10 0x7fa9dc074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:10.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.199+0000 7fa9e3aeb640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa9dc111f30 con 0x7fa9dc106f10 2024-09-12T03:01:10.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.199+0000 7fa9e3aeb640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fa9dc111db0 con 0x7fa9dc101de0 2024-09-12T03:01:10.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.199+0000 7fa9e3aeb640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa9dc1120b0 con 0x7fa9dc102790 2024-09-12T03:01:10.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.199+0000 7fa9e2088640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9dc106f10 0x7fa9dc074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:10.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.199+0000 7fa9e2088640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9dc106f10 0x7fa9dc074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38950/0 (socket says 172.21.15.5:38950) 2024-09-12T03:01:10.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.199+0000 7fa9e1887640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa9dc101de0 0x7fa9dc0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:10.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.199+0000 7fa9e2088640 1 -- 172.21.15.5:0/3222303665 learned_addr learned my addr 172.21.15.5:0/3222303665 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:01:10.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.199+0000 7fa9e1086640 1 --2- 172.21.15.5:0/3222303665 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa9dc102790 0x7fa9dc074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:10.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.199+0000 7fa9e2088640 1 -- 172.21.15.5:0/3222303665 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa9dc102790 msgr2=0x7fa9dc074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:10.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.199+0000 7fa9e2088640 1 --2- 172.21.15.5:0/3222303665 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa9dc102790 0x7fa9dc074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:10.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.199+0000 7fa9e2088640 1 -- 172.21.15.5:0/3222303665 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa9dc101de0 msgr2=0x7fa9dc0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:10.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.199+0000 7fa9e2088640 1 --2- 172.21.15.5:0/3222303665 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa9dc101de0 0x7fa9dc0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:10.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.199+0000 7fa9e2088640 1 -- 172.21.15.5:0/3222303665 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9dc1a6850 con 0x7fa9dc106f10 2024-09-12T03:01:10.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.200+0000 7fa9e2088640 1 --2- 172.21.15.5:0/3222303665 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9dc106f10 0x7fa9dc074f70 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fa9d400eaa0 tx=0x7fa9d400eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:10.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.200+0000 7fa9d2ffd640 1 -- 172.21.15.5:0/3222303665 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d4004d20 con 0x7fa9dc106f10 2024-09-12T03:01:10.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.200+0000 7fa9d2ffd640 1 -- 172.21.15.5:0/3222303665 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9d4005020 con 0x7fa9dc106f10 2024-09-12T03:01:10.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.200+0000 7fa9e3aeb640 1 -- 172.21.15.5:0/3222303665 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9dc1a6b40 con 0x7fa9dc106f10 2024-09-12T03:01:10.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.200+0000 7fa9d2ffd640 1 -- 172.21.15.5:0/3222303665 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d40116d0 con 0x7fa9dc106f10 2024-09-12T03:01:10.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.200+0000 7fa9e3aeb640 1 -- 172.21.15.5:0/3222303665 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9dc1a7080 con 0x7fa9dc106f10 2024-09-12T03:01:10.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.202+0000 7fa9d2ffd640 1 -- 172.21.15.5:0/3222303665 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fa9d40051d0 con 0x7fa9dc106f10 2024-09-12T03:01:10.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.203+0000 7fa9e5ec4740 1 Processor -- start 2024-09-12T03:01:10.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.203+0000 7fa9e5ec4740 1 -- start start 2024-09-12T03:01:10.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.203+0000 7fa9e5ec4740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5651520aa060 0x5651520aa440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:10.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.203+0000 7fa9e5ec4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651520aa980 0x5651520b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:10.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.203+0000 7fa9e5ec4740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5651520b5240 0x5651520b7630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:10.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.203+0000 7fa9e5ec4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56515200f620 con 0x5651520aa980 2024-09-12T03:01:10.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.203+0000 7fa9e5ec4740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x565151fb6e50 con 0x5651520aa060 2024-09-12T03:01:10.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.203+0000 7fa9e5ec4740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x56515200b5e0 con 0x5651520b5240 2024-09-12T03:01:10.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.203+0000 7fa9e1887640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651520aa980 0x5651520b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:10.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.203+0000 7fa9e1887640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651520aa980 0x5651520b4d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38962/0 (socket says 172.21.15.5:38962) 2024-09-12T03:01:10.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.203+0000 7fa9e1887640 1 -- 172.21.15.5:0/2573784376 learned_addr learned my addr 172.21.15.5:0/2573784376 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:01:10.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.203+0000 7fa9e1086640 1 --2- 172.21.15.5:0/2573784376 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5651520aa060 0x5651520aa440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:10.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.203+0000 7fa9d2ffd640 1 --2- 172.21.15.5:0/3222303665 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fa9c80a2c00 0x7fa9c80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:10.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.204+0000 7fa9e1086640 1 --2- 172.21.15.5:0/2573784376 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5651520b5240 0x5651520b7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:10.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.204+0000 7fa9e2088640 1 --2- 172.21.15.5:0/3222303665 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fa9c80a2c00 0x7fa9c80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:10.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.204+0000 7fa9e1887640 1 -- 172.21.15.5:0/2573784376 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5651520b5240 msgr2=0x5651520b7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:01:10.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.204+0000 7fa9e1887640 1 --2- 172.21.15.5:0/2573784376 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5651520b5240 0x5651520b7630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:10.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.204+0000 7fa9e1887640 1 -- 172.21.15.5:0/2573784376 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5651520aa060 msgr2=0x5651520aa440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:01:10.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.204+0000 7fa9e1887640 1 --2- 172.21.15.5:0/2573784376 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5651520aa060 0x5651520aa440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:10.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.204+0000 7fa9e1887640 1 -- 172.21.15.5:0/2573784376 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565151fa7d40 con 0x5651520aa980 2024-09-12T03:01:10.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.204+0000 7fa9e1086640 1 --2- 172.21.15.5:0/2573784376 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5651520aa060 0x5651520aa440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:01:10.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.204+0000 7fa9d2ffd640 1 -- 172.21.15.5:0/3222303665 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fa9d40cf220 con 0x7fa9dc106f10 2024-09-12T03:01:10.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.204+0000 7fa9e1887640 1 --2- 172.21.15.5:0/2573784376 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651520aa980 0x5651520b4d00 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fa9d800ad00 tx=0x7fa9d800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7782e2f785fc04e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:10.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.204+0000 7fa9c7fff640 1 -- 172.21.15.5:0/2573784376 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d801d070 con 0x5651520aa980 2024-09-12T03:01:10.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.204+0000 7fa9c7fff640 1 -- 172.21.15.5:0/2573784376 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9d8018b20 con 0x5651520aa980 2024-09-12T03:01:10.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.204+0000 7fa9c7fff640 1 -- 172.21.15.5:0/2573784376 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d8010bc0 con 0x5651520aa980 2024-09-12T03:01:10.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.204+0000 7fa9e2088640 1 --2- 172.21.15.5:0/3222303665 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fa9c80a2c00 0x7fa9c80a50c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa9d400efd0 tx=0x7fa9d4066600 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:10.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.205+0000 7fa9e5ec4740 1 -- 172.21.15.5:0/2573784376 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651520aa980 msgr2=0x5651520b4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:10.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.205+0000 7fa9e5ec4740 1 --2- 172.21.15.5:0/2573784376 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651520aa980 0x5651520b4d00 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fa9d800ad00 tx=0x7fa9d800f790 comp rx=0 tx=0).stop 2024-09-12T03:01:10.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.205+0000 7fa9e5ec4740 1 -- 172.21.15.5:0/2573784376 shutdown_connections 2024-09-12T03:01:10.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.205+0000 7fa9e5ec4740 1 --2- 172.21.15.5:0/2573784376 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651520aa980 0x5651520b4d00 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:10.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.205+0000 7fa9e5ec4740 1 --2- 172.21.15.5:0/2573784376 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5651520b5240 0x5651520b7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:10.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.205+0000 7fa9e5ec4740 1 --2- 172.21.15.5:0/2573784376 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5651520aa060 0x5651520aa440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:10.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.205+0000 7fa9e5ec4740 1 -- 172.21.15.5:0/2573784376 >> 172.21.15.5:0/2573784376 conn(0x5651520a65f0 msgr2=0x5651520a69d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:10.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.205+0000 7fa9e5ec4740 1 -- 172.21.15.5:0/2573784376 shutdown_connections 2024-09-12T03:01:10.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.205+0000 7fa9e5ec4740 1 -- 172.21.15.5:0/2573784376 wait complete. 2024-09-12T03:01:10.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.205+0000 7fa9e5ec4740 1 -- start start 2024-09-12T03:01:10.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.205+0000 7fa9e5ec4740 1 Processor -- start 2024-09-12T03:01:10.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.206+0000 7fa9e5ec4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651520aa060 0x5651520b2120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:10.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.206+0000 7fa9e5ec4740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5651520aa980 0x5651520b2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:10.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.206+0000 7fa9e5ec4740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5651520b5240 0x5651520b2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:10.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.206+0000 7fa9e5ec4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5651520b7e10 con 0x5651520aa060 2024-09-12T03:01:10.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.206+0000 7fa9e5ec4740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x565151f97f10 con 0x5651520aa980 2024-09-12T03:01:10.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.206+0000 7fa9e5ec4740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5651520b7f90 con 0x5651520b5240 2024-09-12T03:01:10.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.206+0000 7fa9e5ec4740 10 client.0 _ll_register_callbacks cb 0x565151fbbdb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:01:10.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.206+0000 7fa9e1086640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651520aa060 0x5651520b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:10.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.206+0000 7fa9e1086640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651520aa060 0x5651520b2120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38964/0 (socket says 172.21.15.5:38964) 2024-09-12T03:01:10.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.206+0000 7fa9e1086640 1 -- 172.21.15.5:0/1944531324 learned_addr learned my addr 172.21.15.5:0/1944531324 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:01:10.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.206+0000 7fa9e1887640 1 --2- 172.21.15.5:0/1944531324 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5651520aa980 0x5651520b2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:10.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.206+0000 7fa9e1086640 1 --2- 172.21.15.5:0/1944531324 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5651520b5240 0x5651520b2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:10.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.206+0000 7fa9e1086640 1 -- 172.21.15.5:0/1944531324 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5651520b5240 msgr2=0x5651520b2ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:01:10.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.206+0000 7fa9e1086640 1 --2- 172.21.15.5:0/1944531324 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5651520b5240 0x5651520b2ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:10.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.206+0000 7fa9e1086640 1 -- 172.21.15.5:0/1944531324 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5651520aa980 msgr2=0x5651520b2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:01:10.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.206+0000 7fa9e1086640 1 --2- 172.21.15.5:0/1944531324 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5651520aa980 0x5651520b2660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:10.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.206+0000 7fa9e1086640 1 -- 172.21.15.5:0/1944531324 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565151d1e580 con 0x5651520aa060 2024-09-12T03:01:10.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.206+0000 7fa9e1887640 1 --2- 172.21.15.5:0/1944531324 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5651520aa980 0x5651520b2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:01:10.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.207+0000 7fa9e1086640 1 --2- 172.21.15.5:0/1944531324 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651520aa060 0x5651520b2120 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc0096c0 tx=0x7fa9cc0058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86e869076cef8898 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:10.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.207+0000 7fa9c57fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T03:01:10.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.207+0000 7fa9c57fa640 1 -- 172.21.15.5:0/1944531324 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9cc035040 con 0x5651520aa060 2024-09-12T03:01:10.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.207+0000 7fa9c57fa640 1 -- 172.21.15.5:0/1944531324 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fa9cc004460 con 0x5651520aa060 2024-09-12T03:01:10.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.207+0000 7fa9e5ec4740 1 -- 172.21.15.5:0/1944531324 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9d8018b20 con 0x5651520aa060 2024-09-12T03:01:10.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.207+0000 7fa9c57fa640 1 -- 172.21.15.5:0/1944531324 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa9cc0035f0 con 0x5651520aa060 2024-09-12T03:01:10.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.207+0000 7fa9e5ec4740 10 client.16029 fetch_fsmap learned FSMap version 28 2024-09-12T03:01:10.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.207+0000 7fa9e5ec4740 1 -- 172.21.15.5:0/1944531324 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x565151de8960 con 0x5651520aa060 2024-09-12T03:01:10.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.207+0000 7fa9c57fa640 1 -- 172.21.15.5:0/1944531324 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9d8018b20 con 0x5651520aa060 2024-09-12T03:01:10.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.207+0000 7fa9c57fa640 1 -- 172.21.15.5:0/1944531324 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8374+0+0 (secure 0 0 0) 0x7fa9cc049060 con 0x5651520aa060 2024-09-12T03:01:10.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.208+0000 7fa9e5ec4740 10 client.16029 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T03:01:10.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.208+0000 7fa9e5ec4740 10 client.16029 resolve_mds: validated gid 24502 aka daemon mds.i 2024-09-12T03:01:10.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.208+0000 7fa9e5ec4740 20 client.16029 populate_metadata read hostname 'smithi005' 2024-09-12T03:01:10.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.208+0000 7fa9e5ec4740 1 --2- 172.21.15.5:0/1944531324 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x5651520c4d50 0x5651520c7190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:10.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.208+0000 7fa9e5ec4740 4 client.16029 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:01:10.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.208+0000 7fa9e5ec4740 1 -- 172.21.15.5:0/1944531324 --> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56515201cb90 con 0x5651520c4d50 2024-09-12T03:01:10.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.208+0000 7fa9e1887640 1 --2- 172.21.15.5:0/1944531324 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x5651520c4d50 0x5651520c7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:10.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.208+0000 7fa9e1887640 1 --2- 172.21.15.5:0/1944531324 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x5651520c4d50 0x5651520c7190 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97913f5a236a06ad server_cookie=f4b4cd5a415c78ce in_seq=0 out_seq=0 2024-09-12T03:01:10.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.208+0000 7fa9c57fa640 10 client.16029 ms_handle_connect on v2:172.21.15.148:6838/2317600000 2024-09-12T03:01:10.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.209+0000 7fa9c57fa640 1 -- 172.21.15.5:0/1944531324 <== mds.0 v2:172.21.15.148:6838/2317600000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56515201cb90 con 0x5651520c4d50 2024-09-12T03:01:10.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.209+0000 7fa9c57fa640 10 client.16029 handle_command_reply: tid=0 2024-09-12T03:01:10.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.209+0000 7fa9e5ec4740 1 client.16029 shutdown 2024-09-12T03:01:10.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.209+0000 7fa9e5ec4740 1 -- 172.21.15.5:0/1944531324 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651520aa060 msgr2=0x5651520b2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:10.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.209+0000 7fa9e5ec4740 1 --2- 172.21.15.5:0/1944531324 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651520aa060 0x5651520b2120 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc0096c0 tx=0x7fa9cc0058b0 comp rx=0 tx=0).stop 2024-09-12T03:01:10.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.209+0000 7fa9e5ec4740 1 -- 172.21.15.5:0/1944531324 shutdown_connections 2024-09-12T03:01:10.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.209+0000 7fa9e5ec4740 1 --2- 172.21.15.5:0/1944531324 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5651520b5240 0x5651520b2ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:10.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.209+0000 7fa9e5ec4740 1 --2- 172.21.15.5:0/1944531324 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5651520aa980 0x5651520b2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:10.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.209+0000 7fa9e5ec4740 1 --2- 172.21.15.5:0/1944531324 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x5651520c4d50 0x5651520c7190 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:10.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.209+0000 7fa9e5ec4740 1 --2- 172.21.15.5:0/1944531324 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651520aa060 0x5651520b2120 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:10.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.209+0000 7fa9c57fa640 0 client.16029 ms_handle_reset on v2:172.21.15.148:6838/2317600000 2024-09-12T03:01:10.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.209+0000 7fa9e5ec4740 1 -- 172.21.15.5:0/1944531324 >> 172.21.15.5:0/1944531324 conn(0x5651520a65f0 msgr2=0x5651520a5750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:10.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.209+0000 7fa9e5ec4740 1 -- 172.21.15.5:0/1944531324 shutdown_connections 2024-09-12T03:01:10.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.209+0000 7fa9e5ec4740 1 -- 172.21.15.5:0/1944531324 wait complete. 2024-09-12T03:01:10.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.209+0000 7fa9e5ec4740 20 client.16029 trim_cache size 0 max 16384 2024-09-12T03:01:10.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.226+0000 7fa9e5ec4740 1 Processor -- start 2024-09-12T03:01:10.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.226+0000 7fa9e5ec4740 1 -- start start 2024-09-12T03:01:10.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.226+0000 7fa9e5ec4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651520aa060 0x5651520acb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:10.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.226+0000 7fa9e5ec4740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5651520aa980 0x5651520b3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:10.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.227+0000 7fa9e5ec4740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5651520b5240 0x5651520cca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:10.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.227+0000 7fa9e5ec4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa9dc1af6d0 con 0x5651520aa060 2024-09-12T03:01:10.254 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.227+0000 7fa9e5ec4740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fa9dc1b12a0 con 0x5651520b5240 2024-09-12T03:01:10.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.227+0000 7fa9e5ec4740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa9dc1afda0 con 0x5651520aa980 2024-09-12T03:01:10.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.227+0000 7fa9e1086640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651520aa060 0x5651520acb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:10.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.227+0000 7fa9e1086640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651520aa060 0x5651520acb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38972/0 (socket says 172.21.15.5:38972) 2024-09-12T03:01:10.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.227+0000 7fa9e1086640 1 -- 172.21.15.5:0/4188894567 learned_addr learned my addr 172.21.15.5:0/4188894567 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:01:10.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.227+0000 7fa9e1086640 1 --2- 172.21.15.5:0/4188894567 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5651520b5240 0x5651520cca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:10.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.227+0000 7fa9e1887640 1 --2- 172.21.15.5:0/4188894567 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5651520aa980 0x5651520b3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:10.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.227+0000 7fa9e1086640 1 -- 172.21.15.5:0/4188894567 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5651520aa980 msgr2=0x5651520b3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:01:10.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.227+0000 7fa9e1086640 1 --2- 172.21.15.5:0/4188894567 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5651520aa980 0x5651520b3ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:10.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.227+0000 7fa9e1086640 1 -- 172.21.15.5:0/4188894567 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5651520b5240 msgr2=0x5651520cca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:01:10.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.227+0000 7fa9e1086640 1 --2- 172.21.15.5:0/4188894567 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5651520b5240 0x5651520cca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:10.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.227+0000 7fa9e1086640 1 -- 172.21.15.5:0/4188894567 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651520a6090 con 0x5651520aa060 2024-09-12T03:01:10.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.227+0000 7fa9e1887640 1 --2- 172.21.15.5:0/4188894567 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5651520aa980 0x5651520b3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:01:10.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.227+0000 7fa9e1086640 1 --2- 172.21.15.5:0/4188894567 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651520aa060 0x5651520acb10 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc0316c0 tx=0x7fa9cc001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ab628cfcf8b4db6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:10.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.227+0000 7fa9c77fe640 1 -- 172.21.15.5:0/4188894567 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9cc035070 con 0x5651520aa060 2024-09-12T03:01:10.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.227+0000 7fa9c77fe640 1 -- 172.21.15.5:0/4188894567 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9cc041070 con 0x5651520aa060 2024-09-12T03:01:10.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.227+0000 7fa9c77fe640 1 -- 172.21.15.5:0/4188894567 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9cc0044c0 con 0x5651520aa060 2024-09-12T03:01:10.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.228+0000 7fa9e5ec4740 1 -- 172.21.15.5:0/4188894567 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651520aa060 msgr2=0x5651520acb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:10.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.228+0000 7fa9e5ec4740 1 --2- 172.21.15.5:0/4188894567 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651520aa060 0x5651520acb10 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc0316c0 tx=0x7fa9cc001e50 comp rx=0 tx=0).stop 2024-09-12T03:01:10.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.228+0000 7fa9e5ec4740 1 -- 172.21.15.5:0/4188894567 shutdown_connections 2024-09-12T03:01:10.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.228+0000 7fa9e5ec4740 1 --2- 172.21.15.5:0/4188894567 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5651520b5240 0x5651520cca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:10.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.228+0000 7fa9e5ec4740 1 --2- 172.21.15.5:0/4188894567 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5651520aa980 0x5651520b3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:10.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.228+0000 7fa9e5ec4740 1 --2- 172.21.15.5:0/4188894567 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651520aa060 0x5651520acb10 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:10.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.228+0000 7fa9e5ec4740 1 -- 172.21.15.5:0/4188894567 >> 172.21.15.5:0/4188894567 conn(0x5651520a65f0 msgr2=0x5651520a5750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:10.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.228+0000 7fa9e5ec4740 1 -- 172.21.15.5:0/4188894567 shutdown_connections 2024-09-12T03:01:10.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.228+0000 7fa9e5ec4740 1 -- 172.21.15.5:0/4188894567 wait complete. 2024-09-12T03:01:10.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.228+0000 7fa9e5ec4740 1 -- start start 2024-09-12T03:01:10.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.228+0000 7fa9e5ec4740 1 Processor -- start 2024-09-12T03:01:10.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.229+0000 7fa9e5ec4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651520aa060 0x5651520bd830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:10.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.229+0000 7fa9e5ec4740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5651520aa980 0x5651520b9b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:10.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.229+0000 7fa9e5ec4740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5651520b5240 0x5651520ba0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:10.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.229+0000 7fa9e5ec4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa9dc1b0b40 con 0x5651520aa060 2024-09-12T03:01:10.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.229+0000 7fa9e5ec4740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fa9dc1b0470 con 0x5651520b5240 2024-09-12T03:01:10.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.229+0000 7fa9e5ec4740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa9dc1ad6b0 con 0x5651520aa980 2024-09-12T03:01:10.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.229+0000 7fa9e5ec4740 10 client.0 _ll_register_callbacks cb 0x7fa9cc0315b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:01:10.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.229+0000 7fa9e1086640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651520aa060 0x5651520bd830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:10.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.229+0000 7fa9e1086640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651520aa060 0x5651520bd830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38980/0 (socket says 172.21.15.5:38980) 2024-09-12T03:01:10.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.229+0000 7fa9e1086640 1 -- 172.21.15.5:0/3591115009 learned_addr learned my addr 172.21.15.5:0/3591115009 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:01:10.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.229+0000 7fa9e1086640 1 --2- 172.21.15.5:0/3591115009 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5651520b5240 0x5651520ba0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:10.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.229+0000 7fa9e1887640 1 --2- 172.21.15.5:0/3591115009 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5651520aa980 0x5651520b9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:10.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.229+0000 7fa9e1086640 1 -- 172.21.15.5:0/3591115009 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5651520aa980 msgr2=0x5651520b9b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:01:10.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.229+0000 7fa9e1086640 1 --2- 172.21.15.5:0/3591115009 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5651520aa980 0x5651520b9b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:10.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.229+0000 7fa9e1086640 1 -- 172.21.15.5:0/3591115009 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5651520b5240 msgr2=0x5651520ba0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:01:10.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.229+0000 7fa9e1086640 1 --2- 172.21.15.5:0/3591115009 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5651520b5240 0x5651520ba0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:10.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.229+0000 7fa9e1086640 1 -- 172.21.15.5:0/3591115009 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651520ba8e0 con 0x5651520aa060 2024-09-12T03:01:10.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.229+0000 7fa9e1887640 1 --2- 172.21.15.5:0/3591115009 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5651520aa980 0x5651520b9b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:01:10.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.230+0000 7fa9e1086640 1 --2- 172.21.15.5:0/3591115009 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651520aa060 0x5651520bd830 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc02a930 tx=0x7fa9cc0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c5723d568874cff server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:10.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.230+0000 7fa9c57fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T03:01:10.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.230+0000 7fa9c57fa640 1 -- 172.21.15.5:0/3591115009 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9cc035040 con 0x5651520aa060 2024-09-12T03:01:10.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.230+0000 7fa9c57fa640 1 -- 172.21.15.5:0/3591115009 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fa9cc003790 con 0x5651520aa060 2024-09-12T03:01:10.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.230+0000 7fa9e5ec4740 1 -- 172.21.15.5:0/3591115009 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9cc041070 con 0x5651520aa060 2024-09-12T03:01:10.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.230+0000 7fa9c57fa640 1 -- 172.21.15.5:0/3591115009 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa9cc04ba50 con 0x5651520aa060 2024-09-12T03:01:10.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.230+0000 7fa9e5ec4740 10 client.16035 fetch_fsmap learned FSMap version 28 2024-09-12T03:01:10.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.230+0000 7fa9e5ec4740 1 -- 172.21.15.5:0/3591115009 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5651521308f0 con 0x5651520aa060 2024-09-12T03:01:10.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.230+0000 7fa9c57fa640 1 -- 172.21.15.5:0/3591115009 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9cc041070 con 0x5651520aa060 2024-09-12T03:01:10.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.230+0000 7fa9c57fa640 1 -- 172.21.15.5:0/3591115009 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8374+0+0 (secure 0 0 0) 0x7fa9cc049040 con 0x5651520aa060 2024-09-12T03:01:10.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.230+0000 7fa9e5ec4740 10 client.16035 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T03:01:10.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.230+0000 7fa9e5ec4740 10 client.16035 resolve_mds: validated gid 24502 aka daemon mds.i 2024-09-12T03:01:10.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.230+0000 7fa9e5ec4740 20 client.16035 populate_metadata read hostname 'smithi005' 2024-09-12T03:01:10.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.230+0000 7fa9e5ec4740 1 --2- 172.21.15.5:0/3591115009 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x565152130df0 0x565152133210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:10.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.230+0000 7fa9e5ec4740 4 client.16035 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "4d66795", "release": true, "await": true}] 2024-09-12T03:01:10.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.230+0000 7fa9e5ec4740 1 -- 172.21.15.5:0/3591115009 --> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] -- command(tid 0: {"prefix": "quiesce db", "set_id": "4d66795", "release": true, "await": true}) -- 0x5651520170a0 con 0x565152130df0 2024-09-12T03:01:10.260 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.231+0000 7fa9e1887640 1 --2- 172.21.15.5:0/3591115009 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x565152130df0 0x565152133210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:10.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.231+0000 7fa9e1887640 1 --2- 172.21.15.5:0/3591115009 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x565152130df0 0x565152133210 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6a4c8959e4cdb6 server_cookie=afc5555349b72620 in_seq=0 out_seq=0 2024-09-12T03:01:10.261 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.231+0000 7fa9c57fa640 10 client.16035 ms_handle_connect on v2:172.21.15.148:6838/2317600000 2024-09-12T03:01:10.355 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:01:10.421 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.420+0000 7fa9c57fa640 1 -- 172.21.15.5:0/3591115009 <== mds.0 v2:172.21.15.148:6838/2317600000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x5651520170a0 con 0x565152130df0 2024-09-12T03:01:10.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.420+0000 7fa9c57fa640 10 client.16035 handle_command_reply: tid=0 2024-09-12T03:01:10.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.420+0000 7fa9e5ec4740 1 client.16035 shutdown 2024-09-12T03:01:10.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.420+0000 7fa9e5ec4740 1 -- 172.21.15.5:0/3591115009 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651520aa060 msgr2=0x5651520bd830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:10.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.421+0000 7fa9e5ec4740 1 --2- 172.21.15.5:0/3591115009 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651520aa060 0x5651520bd830 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc02a930 tx=0x7fa9cc0095c0 comp rx=0 tx=0).stop 2024-09-12T03:01:10.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.421+0000 7fa9e5ec4740 1 -- 172.21.15.5:0/3591115009 shutdown_connections 2024-09-12T03:01:10.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.421+0000 7fa9e5ec4740 1 --2- 172.21.15.5:0/3591115009 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5651520b5240 0x5651520ba0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:10.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.421+0000 7fa9e5ec4740 1 --2- 172.21.15.5:0/3591115009 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5651520aa980 0x5651520b9b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:10.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.421+0000 7fa9e5ec4740 1 --2- 172.21.15.5:0/3591115009 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x565152130df0 0x565152133210 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:10.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.421+0000 7fa9e5ec4740 1 --2- 172.21.15.5:0/3591115009 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651520aa060 0x5651520bd830 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:10.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.421+0000 7fa9e5ec4740 1 -- 172.21.15.5:0/3591115009 >> 172.21.15.5:0/3591115009 conn(0x5651520a65f0 msgr2=0x5651520a4f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:10.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.421+0000 7fa9c57fa640 0 client.16035 ms_handle_reset on v2:172.21.15.148:6838/2317600000 2024-09-12T03:01:10.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.421+0000 7fa9e5ec4740 1 -- 172.21.15.5:0/3591115009 shutdown_connections 2024-09-12T03:01:10.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.421+0000 7fa9e5ec4740 1 -- 172.21.15.5:0/3591115009 wait complete. 2024-09-12T03:01:10.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.421+0000 7fa9e5ec4740 20 client.16035 trim_cache size 0 max 16384 2024-09-12T03:01:10.423 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-12T03:01:10.423 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 28, 2024-09-12T03:01:10.423 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24502, 2024-09-12T03:01:10.423 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 12, 2024-09-12T03:01:10.423 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-09-12T03:01:10.423 INFO:teuthology.orchestra.run.smithi005.stdout: "4d66795": { 2024-09-12T03:01:10.423 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 12, 2024-09-12T03:01:10.424 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-09-12T03:01:10.424 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-12T03:01:10.424 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-09-12T03:01:10.424 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-12T03:01:10.424 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:01:10.424 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 90.0, 2024-09-12T03:01:10.424 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 144.7, 2024-09-12T03:01:10.424 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-09-12T03:01:10.424 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-09-12T03:01:10.424 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-09-12T03:01:10.424 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-12T03:01:10.424 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-09-12T03:01:10.424 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-12T03:01:10.425 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:01:10.425 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:01:10.425 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:01:10.425 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:01:10.425 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:01:10.425 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-12T03:01:10.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.422+0000 7fa9e3aeb640 1 -- 172.21.15.5:0/3222303665 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fa9c80a2c00 msgr2=0x7fa9c80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:10.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.422+0000 7fa9e3aeb640 1 --2- 172.21.15.5:0/3222303665 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fa9c80a2c00 0x7fa9c80a50c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa9d400efd0 tx=0x7fa9d4066600 comp rx=0 tx=0).stop 2024-09-12T03:01:10.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.422+0000 7fa9e3aeb640 1 -- 172.21.15.5:0/3222303665 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9dc106f10 msgr2=0x7fa9dc074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:10.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.422+0000 7fa9e3aeb640 1 --2- 172.21.15.5:0/3222303665 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9dc106f10 0x7fa9dc074f70 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fa9d400eaa0 tx=0x7fa9d400eea0 comp rx=0 tx=0).stop 2024-09-12T03:01:10.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.422+0000 7fa9e3aeb640 1 -- 172.21.15.5:0/3222303665 shutdown_connections 2024-09-12T03:01:10.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.422+0000 7fa9e3aeb640 1 --2- 172.21.15.5:0/3222303665 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fa9c80a2c00 0x7fa9c80a50c0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:10.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.422+0000 7fa9e3aeb640 1 --2- 172.21.15.5:0/3222303665 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9dc106f10 0x7fa9dc074f70 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:10.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.422+0000 7fa9e3aeb640 1 --2- 172.21.15.5:0/3222303665 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa9dc102790 0x7fa9dc074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:10.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.422+0000 7fa9e3aeb640 1 --2- 172.21.15.5:0/3222303665 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa9dc101de0 0x7fa9dc0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:10.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.422+0000 7fa9e3aeb640 1 -- 172.21.15.5:0/3222303665 >> 172.21.15.5:0/3222303665 conn(0x7fa9dc0fd8c0 msgr2=0x7fa9dc0fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:10.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.422+0000 7fa9e3aeb640 1 -- 172.21.15.5:0/3222303665 shutdown_connections 2024-09-12T03:01:10.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:10.422+0000 7fa9e3aeb640 1 -- 172.21.15.5:0/3222303665 wait complete. 2024-09-12T03:01:11.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.029+0000 7fb89fb24640 1 -- 172.21.15.5:0/3777781194 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb898102790 msgr2=0x7fb8981067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:11.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.029+0000 7fb89fb24640 1 --2- 172.21.15.5:0/3777781194 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb898102790 0x7fb8981067e0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fb888009900 tx=0x7fb88802a410 comp rx=0 tx=0).stop 2024-09-12T03:01:11.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.029+0000 7fb89fb24640 1 -- 172.21.15.5:0/3777781194 shutdown_connections 2024-09-12T03:01:11.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.029+0000 7fb89fb24640 1 --2- 172.21.15.5:0/3777781194 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb898102790 0x7fb8981067e0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:11.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.029+0000 7fb89fb24640 1 --2- 172.21.15.5:0/3777781194 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb898106f10 0x7fb89810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:11.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.029+0000 7fb89fb24640 1 --2- 172.21.15.5:0/3777781194 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb898101de0 0x7fb8981021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:11.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.029+0000 7fb89fb24640 1 -- 172.21.15.5:0/3777781194 >> 172.21.15.5:0/3777781194 conn(0x7fb8980fd8c0 msgr2=0x7fb8980ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:11.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.029+0000 7fb89fb24640 1 -- 172.21.15.5:0/3777781194 shutdown_connections 2024-09-12T03:01:11.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.029+0000 7fb89fb24640 1 -- 172.21.15.5:0/3777781194 wait complete. 2024-09-12T03:01:11.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.029+0000 7fb89fb24640 1 Processor -- start 2024-09-12T03:01:11.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.030+0000 7fb89fb24640 1 -- start start 2024-09-12T03:01:11.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.030+0000 7fb89fb24640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb898101de0 0x7fb8980744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:11.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.030+0000 7fb89fb24640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb898102790 0x7fb898074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:11.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.030+0000 7fb89fb24640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb898106f10 0x7fb898074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:11.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.030+0000 7fb89fb24640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb898111fc0 con 0x7fb898106f10 2024-09-12T03:01:11.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.030+0000 7fb89fb24640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fb898111e40 con 0x7fb898101de0 2024-09-12T03:01:11.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.030+0000 7fb89fb24640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb898112140 con 0x7fb898102790 2024-09-12T03:01:11.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.030+0000 7fb89d0bf640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb898102790 0x7fb898074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:11.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.030+0000 7fb89d0bf640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb898102790 0x7fb898074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.5:60466/0 (socket says 172.21.15.5:60466) 2024-09-12T03:01:11.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.030+0000 7fb89d0bf640 1 -- 172.21.15.5:0/493630802 learned_addr learned my addr 172.21.15.5:0/493630802 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:01:11.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.030+0000 7fb89d8c0640 1 --2- 172.21.15.5:0/493630802 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb898101de0 0x7fb8980744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:11.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.030+0000 7fb89e0c1640 1 --2- 172.21.15.5:0/493630802 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb898106f10 0x7fb898074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:11.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.030+0000 7fb89d0bf640 1 -- 172.21.15.5:0/493630802 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb898101de0 msgr2=0x7fb8980744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:11.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.030+0000 7fb89d0bf640 1 --2- 172.21.15.5:0/493630802 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb898101de0 0x7fb8980744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:11.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.030+0000 7fb89d0bf640 1 -- 172.21.15.5:0/493630802 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb898106f10 msgr2=0x7fb898074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:11.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.030+0000 7fb89d0bf640 1 --2- 172.21.15.5:0/493630802 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb898106f10 0x7fb898074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:11.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.030+0000 7fb89d0bf640 1 -- 172.21.15.5:0/493630802 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8981a68f0 con 0x7fb898102790 2024-09-12T03:01:11.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.030+0000 7fb89d8c0640 1 --2- 172.21.15.5:0/493630802 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb898101de0 0x7fb8980744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:01:11.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.030+0000 7fb89e0c1640 1 --2- 172.21.15.5:0/493630802 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb898106f10 0x7fb898074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:01:11.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.031+0000 7fb89d0bf640 1 --2- 172.21.15.5:0/493630802 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb898102790 0x7fb898074a30 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb888004770 tx=0x7fb888003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:11.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.031+0000 7fb88effd640 1 -- 172.21.15.5:0/493630802 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb88803f070 con 0x7fb898102790 2024-09-12T03:01:11.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.031+0000 7fb88effd640 1 -- 172.21.15.5:0/493630802 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb888031ca0 con 0x7fb898102790 2024-09-12T03:01:11.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.031+0000 7fb89fb24640 1 -- 172.21.15.5:0/493630802 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8981a6b80 con 0x7fb898102790 2024-09-12T03:01:11.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.031+0000 7fb88effd640 1 -- 172.21.15.5:0/493630802 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb888035030 con 0x7fb898102790 2024-09-12T03:01:11.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.031+0000 7fb89fb24640 1 -- 172.21.15.5:0/493630802 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8981a70b0 con 0x7fb898102790 2024-09-12T03:01:11.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.033+0000 7fb88effd640 1 -- 172.21.15.5:0/493630802 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fb888042070 con 0x7fb898102790 2024-09-12T03:01:11.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.034+0000 7fb8a1efd740 1 Processor -- start 2024-09-12T03:01:11.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.034+0000 7fb8a1efd740 1 -- start start 2024-09-12T03:01:11.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.034+0000 7fb88effd640 1 --2- 172.21.15.5:0/493630802 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fb8840a2c00 0x7fb8840a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:11.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.035+0000 7fb8a1efd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d91cc7d4a0 0x55d91cc7d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:11.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.035+0000 7fb8a1efd740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d91cc7ddc0 0x55d91cc88140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:11.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.035+0000 7fb8a1efd740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d91cc88680 0x55d91cc8aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:11.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.035+0000 7fb8a1efd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d91cbe32e0 con 0x55d91cc7d4a0 2024-09-12T03:01:11.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.035+0000 7fb8a1efd740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55d91cb8ace0 con 0x55d91cc7ddc0 2024-09-12T03:01:11.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.035+0000 7fb8a1efd740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55d91cbdf2a0 con 0x55d91cc88680 2024-09-12T03:01:11.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.035+0000 7fb88effd640 1 -- 172.21.15.5:0/493630802 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fb8880ef540 con 0x7fb898102790 2024-09-12T03:01:11.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.035+0000 7fb89d0bf640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d91cc7ddc0 0x55d91cc88140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:11.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.035+0000 7fb89d8c0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d91cc7d4a0 0x55d91cc7d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:11.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.035+0000 7fb89d0bf640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d91cc7ddc0 0x55d91cc88140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.5:45612/0 (socket says 172.21.15.5:45612) 2024-09-12T03:01:11.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.035+0000 7fb89d0bf640 1 -- 172.21.15.5:0/2034539173 learned_addr learned my addr 172.21.15.5:0/2034539173 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:01:11.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.035+0000 7fb89e0c1640 1 --2- 172.21.15.5:0/2034539173 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d91cc88680 0x55d91cc8aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:11.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.035+0000 7fb89e0c1640 1 --2- 172.21.15.5:0/493630802 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fb8840a2c00 0x7fb8840a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:11.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.035+0000 7fb89d0bf640 1 -- 172.21.15.5:0/2034539173 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d91cc88680 msgr2=0x55d91cc8aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:01:11.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.035+0000 7fb89d0bf640 1 --2- 172.21.15.5:0/2034539173 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d91cc88680 0x55d91cc8aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:11.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.035+0000 7fb89d0bf640 1 -- 172.21.15.5:0/2034539173 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d91cc7d4a0 msgr2=0x55d91cc7d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:01:11.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.035+0000 7fb89d0bf640 1 --2- 172.21.15.5:0/2034539173 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d91cc7d4a0 0x55d91cc7d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:11.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.035+0000 7fb89d0bf640 1 -- 172.21.15.5:0/2034539173 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d91cb6f440 con 0x55d91cc7ddc0 2024-09-12T03:01:11.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.035+0000 7fb89d0bf640 1 --2- 172.21.15.5:0/2034539173 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d91cc7ddc0 0x55d91cc88140 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb8880c44f0 tx=0x7fb8880b8110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c6912033a98c75f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:11.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.036+0000 7fb89e0c1640 1 --2- 172.21.15.5:0/493630802 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fb8840a2c00 0x7fb8840a50c0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x55d91cc7adb0 tx=0x7fb890009500 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:11.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.036+0000 7fb883fff640 1 -- 172.21.15.5:0/2034539173 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb888049070 con 0x55d91cc7ddc0 2024-09-12T03:01:11.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.036+0000 7fb883fff640 1 -- 172.21.15.5:0/2034539173 <== mon.2 v2:172.21.15.146:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8880bc4b0 con 0x55d91cc7ddc0 2024-09-12T03:01:11.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.036+0000 7fb883fff640 1 -- 172.21.15.5:0/2034539173 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8880c4a20 con 0x55d91cc7ddc0 2024-09-12T03:01:11.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.036+0000 7fb8a1efd740 1 -- 172.21.15.5:0/2034539173 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d91cc7ddc0 msgr2=0x55d91cc88140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:11.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.036+0000 7fb8a1efd740 1 --2- 172.21.15.5:0/2034539173 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d91cc7ddc0 0x55d91cc88140 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb8880c44f0 tx=0x7fb8880b8110 comp rx=0 tx=0).stop 2024-09-12T03:01:11.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.037+0000 7fb8a1efd740 1 -- 172.21.15.5:0/2034539173 shutdown_connections 2024-09-12T03:01:11.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.037+0000 7fb8a1efd740 1 --2- 172.21.15.5:0/2034539173 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d91cc88680 0x55d91cc8aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:11.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.037+0000 7fb8a1efd740 1 --2- 172.21.15.5:0/2034539173 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d91cc7ddc0 0x55d91cc88140 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:11.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.037+0000 7fb8a1efd740 1 --2- 172.21.15.5:0/2034539173 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d91cc7d4a0 0x55d91cc7d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:11.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.037+0000 7fb8a1efd740 1 -- 172.21.15.5:0/2034539173 >> 172.21.15.5:0/2034539173 conn(0x55d91cc79a30 msgr2=0x55d91cc79e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:11.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.037+0000 7fb8a1efd740 1 -- 172.21.15.5:0/2034539173 shutdown_connections 2024-09-12T03:01:11.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.037+0000 7fb8a1efd740 1 -- 172.21.15.5:0/2034539173 wait complete. 2024-09-12T03:01:11.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.037+0000 7fb8a1efd740 1 -- start start 2024-09-12T03:01:11.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.037+0000 7fb8a1efd740 1 Processor -- start 2024-09-12T03:01:11.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.038+0000 7fb8a1efd740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d91cc7d4a0 0x55d91cc88050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:11.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.038+0000 7fb8a1efd740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d91cc7ddc0 0x55d91cc85300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:11.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.038+0000 7fb8a1efd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d91cc88680 0x55d91cc858f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:11.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.038+0000 7fb8a1efd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d91cc8b010 con 0x55d91cc88680 2024-09-12T03:01:11.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.038+0000 7fb8a1efd740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55d91cb9e050 con 0x55d91cc7d4a0 2024-09-12T03:01:11.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.038+0000 7fb8a1efd740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55d91cc8b190 con 0x55d91cc7ddc0 2024-09-12T03:01:11.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.038+0000 7fb8a1efd740 10 client.0 _ll_register_callbacks cb 0x55d91cbde160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:01:11.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.038+0000 7fb89d8c0640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d91cc7d4a0 0x55d91cc88050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:11.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.038+0000 7fb89d0bf640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d91cc7ddc0 0x55d91cc85300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:11.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.038+0000 7fb89d8c0640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d91cc7d4a0 0x55d91cc88050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.5:45620/0 (socket says 172.21.15.5:45620) 2024-09-12T03:01:11.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.038+0000 7fb89d8c0640 1 -- 172.21.15.5:0/3767989714 learned_addr learned my addr 172.21.15.5:0/3767989714 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:01:11.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.038+0000 7fb89e0c1640 1 --2- 172.21.15.5:0/3767989714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d91cc88680 0x55d91cc858f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:11.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.038+0000 7fb89d8c0640 1 -- 172.21.15.5:0/3767989714 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d91cc7ddc0 msgr2=0x55d91cc85300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:01:11.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.038+0000 7fb89d8c0640 1 --2- 172.21.15.5:0/3767989714 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d91cc7ddc0 0x55d91cc85300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:11.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.038+0000 7fb89d8c0640 1 -- 172.21.15.5:0/3767989714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d91cc88680 msgr2=0x55d91cc858f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:01:11.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.038+0000 7fb89d8c0640 1 --2- 172.21.15.5:0/3767989714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d91cc88680 0x55d91cc858f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:11.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.038+0000 7fb89d8c0640 1 -- 172.21.15.5:0/3767989714 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d91c9543c0 con 0x55d91cc7d4a0 2024-09-12T03:01:11.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.038+0000 7fb89e0c1640 1 --2- 172.21.15.5:0/3767989714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d91cc88680 0x55d91cc858f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:01:11.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.039+0000 7fb89d8c0640 1 --2- 172.21.15.5:0/3767989714 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d91cc7d4a0 0x55d91cc88050 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb894012d20 tx=0x7fb89400f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4acdc9e2ee0ae6e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:11.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.039+0000 7fb8817fa640 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2024-09-12T03:01:11.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.039+0000 7fb8817fa640 1 -- 172.21.15.5:0/3767989714 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb89400fe20 con 0x55d91cc7d4a0 2024-09-12T03:01:11.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.039+0000 7fb8817fa640 1 -- 172.21.15.5:0/3767989714 <== mon.2 v2:172.21.15.146:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fb894010430 con 0x55d91cc7d4a0 2024-09-12T03:01:11.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.039+0000 7fb8a1efd740 1 -- 172.21.15.5:0/3767989714 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8880bc4b0 con 0x55d91cc7d4a0 2024-09-12T03:01:11.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.039+0000 7fb8817fa640 1 -- 172.21.15.5:0/3767989714 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb894010630 con 0x55d91cc7d4a0 2024-09-12T03:01:11.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.039+0000 7fb8a1efd740 10 client.25469 fetch_fsmap learned FSMap version 28 2024-09-12T03:01:11.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.039+0000 7fb8a1efd740 1 -- 172.21.15.5:0/3767989714 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d91cc96a10 con 0x55d91cc7d4a0 2024-09-12T03:01:11.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.039+0000 7fb8817fa640 1 -- 172.21.15.5:0/3767989714 <== mon.2 v2:172.21.15.146:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8880bc4b0 con 0x55d91cc7d4a0 2024-09-12T03:01:11.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.039+0000 7fb8817fa640 1 -- 172.21.15.5:0/3767989714 <== mon.2 v2:172.21.15.146:3300/0 5 ==== fsmap(e 28) ==== 8374+0+0 (secure 0 0 0) 0x7fb89401c860 con 0x55d91cc7d4a0 2024-09-12T03:01:11.057 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.040+0000 7fb8a1efd740 10 client.25469 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T03:01:11.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.040+0000 7fb8a1efd740 10 client.25469 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:01:11.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.040+0000 7fb8a1efd740 20 client.25469 populate_metadata read hostname 'smithi005' 2024-09-12T03:01:11.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.040+0000 7fb8a1efd740 1 --2- 172.21.15.5:0/3767989714 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55d91cc97ee0 0x55d91cc9a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:11.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.040+0000 7fb8a1efd740 4 client.25469 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:01:11.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.040+0000 7fb8a1efd740 1 -- 172.21.15.5:0/3767989714 --> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d91cbf0850 con 0x55d91cc97ee0 2024-09-12T03:01:11.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.040+0000 7fb89d0bf640 1 --2- 172.21.15.5:0/3767989714 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55d91cc97ee0 0x55d91cc9a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:11.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.040+0000 7fb89d0bf640 1 --2- 172.21.15.5:0/3767989714 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55d91cc97ee0 0x55d91cc9a360 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ad0d6309dfa60eb server_cookie=69f8daa5f9d3daee in_seq=0 out_seq=0 2024-09-12T03:01:11.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.040+0000 7fb8817fa640 10 client.25469 ms_handle_connect on v2:172.21.15.148:6838/2317600000 2024-09-12T03:01:11.058 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.041+0000 7fb8817fa640 1 -- 172.21.15.5:0/3767989714 <== mds.0 v2:172.21.15.148:6838/2317600000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d91cbf0850 con 0x55d91cc97ee0 2024-09-12T03:01:11.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.041+0000 7fb8817fa640 10 client.25469 handle_command_reply: tid=0 2024-09-12T03:01:11.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.041+0000 7fb8a1efd740 1 client.25469 shutdown 2024-09-12T03:01:11.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.041+0000 7fb8a1efd740 1 -- 172.21.15.5:0/3767989714 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d91cc7d4a0 msgr2=0x55d91cc88050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:11.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.041+0000 7fb8a1efd740 1 --2- 172.21.15.5:0/3767989714 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d91cc7d4a0 0x55d91cc88050 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb894012d20 tx=0x7fb89400f790 comp rx=0 tx=0).stop 2024-09-12T03:01:11.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.041+0000 7fb8a1efd740 1 -- 172.21.15.5:0/3767989714 shutdown_connections 2024-09-12T03:01:11.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.041+0000 7fb8a1efd740 1 --2- 172.21.15.5:0/3767989714 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55d91cc97ee0 0x55d91cc9a360 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:11.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.041+0000 7fb8a1efd740 1 --2- 172.21.15.5:0/3767989714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d91cc88680 0x55d91cc858f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:11.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.041+0000 7fb8a1efd740 1 --2- 172.21.15.5:0/3767989714 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d91cc7ddc0 0x55d91cc85300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:11.059 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.041+0000 7fb8a1efd740 1 --2- 172.21.15.5:0/3767989714 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d91cc7d4a0 0x55d91cc88050 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:11.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.041+0000 7fb8817fa640 0 client.25469 ms_handle_reset on v2:172.21.15.148:6838/2317600000 2024-09-12T03:01:11.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.041+0000 7fb8a1efd740 1 -- 172.21.15.5:0/3767989714 >> 172.21.15.5:0/3767989714 conn(0x55d91cc79a30 msgr2=0x55d91cc7f760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:11.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.041+0000 7fb8a1efd740 1 -- 172.21.15.5:0/3767989714 shutdown_connections 2024-09-12T03:01:11.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.041+0000 7fb8a1efd740 1 -- 172.21.15.5:0/3767989714 wait complete. 2024-09-12T03:01:11.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.042+0000 7fb8a1efd740 20 client.25469 trim_cache size 0 max 16384 2024-09-12T03:01:11.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.061+0000 7fb8a1efd740 1 Processor -- start 2024-09-12T03:01:11.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.061+0000 7fb8a1efd740 1 -- start start 2024-09-12T03:01:11.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.061+0000 7fb8a1efd740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d91cc7d4a0 0x55d91cc83bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:11.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.061+0000 7fb8a1efd740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d91cc7ddc0 0x55d91cc9fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:11.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.061+0000 7fb8a1efd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d91cc88680 0x55d91cca0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:11.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.061+0000 7fb8a1efd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb8981af6a0 con 0x55d91cc88680 2024-09-12T03:01:11.063 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.061+0000 7fb8a1efd740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fb8981b1270 con 0x55d91cc7ddc0 2024-09-12T03:01:11.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.061+0000 7fb8a1efd740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb8981afd70 con 0x55d91cc7d4a0 2024-09-12T03:01:11.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.061+0000 7fb89d8c0640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d91cc7d4a0 0x55d91cc83bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:11.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.061+0000 7fb89d0bf640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d91cc7ddc0 0x55d91cc9fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:11.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.061+0000 7fb89d8c0640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d91cc7d4a0 0x55d91cc83bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.5:60502/0 (socket says 172.21.15.5:60502) 2024-09-12T03:01:11.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.061+0000 7fb89d8c0640 1 -- 172.21.15.5:0/3881098312 learned_addr learned my addr 172.21.15.5:0/3881098312 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:01:11.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.062+0000 7fb89e0c1640 1 --2- 172.21.15.5:0/3881098312 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d91cc88680 0x55d91cca0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:11.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.062+0000 7fb89d0bf640 1 -- 172.21.15.5:0/3881098312 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d91cc7d4a0 msgr2=0x55d91cc83bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:01:11.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.062+0000 7fb89d0bf640 1 --2- 172.21.15.5:0/3881098312 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d91cc7d4a0 0x55d91cc83bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:11.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.062+0000 7fb89d0bf640 1 -- 172.21.15.5:0/3881098312 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d91cc88680 msgr2=0x55d91cca0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:01:11.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.062+0000 7fb89d0bf640 1 --2- 172.21.15.5:0/3881098312 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d91cc88680 0x55d91cca0170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:11.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.062+0000 7fb89d0bf640 1 -- 172.21.15.5:0/3881098312 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d91cc7d2d0 con 0x55d91cc7ddc0 2024-09-12T03:01:11.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.062+0000 7fb89d8c0640 1 --2- 172.21.15.5:0/3881098312 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d91cc7d4a0 0x55d91cc83bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:01:11.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.062+0000 7fb89d0bf640 1 --2- 172.21.15.5:0/3881098312 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d91cc7ddc0 0x55d91cc9fc30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x55d91cc7e3e0 tx=0x7fb8880bb9f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6a351b15e063a57 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:11.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.062+0000 7fb8837fe640 1 -- 172.21.15.5:0/3881098312 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8880d9070 con 0x55d91cc7ddc0 2024-09-12T03:01:11.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.062+0000 7fb8837fe640 1 -- 172.21.15.5:0/3881098312 <== mon.2 v2:172.21.15.146:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8880bf040 con 0x55d91cc7ddc0 2024-09-12T03:01:11.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.062+0000 7fb8837fe640 1 -- 172.21.15.5:0/3881098312 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb888049040 con 0x55d91cc7ddc0 2024-09-12T03:01:11.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.063+0000 7fb8a1efd740 1 -- 172.21.15.5:0/3881098312 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d91cc7ddc0 msgr2=0x55d91cc9fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:11.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.063+0000 7fb8a1efd740 1 --2- 172.21.15.5:0/3881098312 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d91cc7ddc0 0x55d91cc9fc30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x55d91cc7e3e0 tx=0x7fb8880bb9f0 comp rx=0 tx=0).stop 2024-09-12T03:01:11.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.063+0000 7fb8a1efd740 1 -- 172.21.15.5:0/3881098312 shutdown_connections 2024-09-12T03:01:11.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.063+0000 7fb8a1efd740 1 --2- 172.21.15.5:0/3881098312 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d91cc88680 0x55d91cca0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:11.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.063+0000 7fb8a1efd740 1 --2- 172.21.15.5:0/3881098312 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d91cc7ddc0 0x55d91cc9fc30 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:11.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.063+0000 7fb8a1efd740 1 --2- 172.21.15.5:0/3881098312 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d91cc7d4a0 0x55d91cc83bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:11.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.063+0000 7fb8a1efd740 1 -- 172.21.15.5:0/3881098312 >> 172.21.15.5:0/3881098312 conn(0x55d91cc79a30 msgr2=0x55d91cc87380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:11.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.063+0000 7fb8a1efd740 1 -- 172.21.15.5:0/3881098312 shutdown_connections 2024-09-12T03:01:11.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.063+0000 7fb8a1efd740 1 -- 172.21.15.5:0/3881098312 wait complete. 2024-09-12T03:01:11.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.064+0000 7fb8a1efd740 1 -- start start 2024-09-12T03:01:11.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.064+0000 7fb8a1efd740 1 Processor -- start 2024-09-12T03:01:11.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.064+0000 7fb8a1efd740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d91cc7d4a0 0x55d91cc8e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:11.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.064+0000 7fb8a1efd740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d91cc7ddc0 0x55d91cc8e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:11.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.064+0000 7fb8a1efd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d91cc88680 0x55d91cc8edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:11.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.064+0000 7fb8a1efd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb8981b0b10 con 0x55d91cc88680 2024-09-12T03:01:11.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.064+0000 7fb8a1efd740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fb8981b0440 con 0x55d91cc7d4a0 2024-09-12T03:01:11.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.064+0000 7fb8a1efd740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb8981ad680 con 0x55d91cc7ddc0 2024-09-12T03:01:11.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.064+0000 7fb8a1efd740 10 client.0 _ll_register_callbacks cb 0x7fb894009cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:01:11.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.064+0000 7fb89d8c0640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d91cc7d4a0 0x55d91cc8e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:11.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.064+0000 7fb89d0bf640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d91cc7ddc0 0x55d91cc8e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:11.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.064+0000 7fb89e0c1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d91cc88680 0x55d91cc8edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:11.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.064+0000 7fb89d0bf640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d91cc7ddc0 0x55d91cc8e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.5:60508/0 (socket says 172.21.15.5:60508) 2024-09-12T03:01:11.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.064+0000 7fb89d8c0640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d91cc7d4a0 0x55d91cc8e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.5:45634/0 (socket says 172.21.15.5:45634) 2024-09-12T03:01:11.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.065+0000 7fb89d0bf640 1 -- 172.21.15.5:0/520858477 learned_addr learned my addr 172.21.15.5:0/520858477 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:01:11.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.065+0000 7fb89d0bf640 1 -- 172.21.15.5:0/520858477 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d91cc7d4a0 msgr2=0x55d91cc8e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:01:11.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.065+0000 7fb89d0bf640 1 --2- 172.21.15.5:0/520858477 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d91cc7d4a0 0x55d91cc8e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:11.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.065+0000 7fb89d0bf640 1 -- 172.21.15.5:0/520858477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d91cc88680 msgr2=0x55d91cc8edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:01:11.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.065+0000 7fb89d0bf640 1 --2- 172.21.15.5:0/520858477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d91cc88680 0x55d91cc8edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:11.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.065+0000 7fb89d0bf640 1 -- 172.21.15.5:0/520858477 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d91cc80f00 con 0x55d91cc7ddc0 2024-09-12T03:01:11.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.065+0000 7fb89d8c0640 1 --2- 172.21.15.5:0/520858477 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d91cc7d4a0 0x55d91cc8e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:01:11.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.065+0000 7fb89e0c1640 1 --2- 172.21.15.5:0/520858477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d91cc88680 0x55d91cc8edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:01:11.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.065+0000 7fb89d0bf640 1 --2- 172.21.15.5:0/520858477 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d91cc7ddc0 0x55d91cc8e870 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb8880c44f0 tx=0x7fb8880b8130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40427ceb9925e716 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:11.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.065+0000 7fb8817fa640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-09-12T03:01:11.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.065+0000 7fb8817fa640 1 -- 172.21.15.5:0/520858477 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb88809e030 con 0x55d91cc7ddc0 2024-09-12T03:01:11.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.065+0000 7fb8817fa640 1 -- 172.21.15.5:0/520858477 <== mon.1 v2:172.21.15.148:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fb88803b460 con 0x55d91cc7ddc0 2024-09-12T03:01:11.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.065+0000 7fb8a1efd740 1 -- 172.21.15.5:0/520858477 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8880bf040 con 0x55d91cc7ddc0 2024-09-12T03:01:11.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.066+0000 7fb8817fa640 1 -- 172.21.15.5:0/520858477 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fb8880d9050 con 0x55d91cc7ddc0 2024-09-12T03:01:11.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.066+0000 7fb8a1efd740 10 client.25570 fetch_fsmap learned FSMap version 28 2024-09-12T03:01:11.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.066+0000 7fb8a1efd740 1 -- 172.21.15.5:0/520858477 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d91cd02fd0 con 0x55d91cc7ddc0 2024-09-12T03:01:11.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.066+0000 7fb8817fa640 1 -- 172.21.15.5:0/520858477 <== mon.1 v2:172.21.15.148:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8880bf040 con 0x55d91cc7ddc0 2024-09-12T03:01:11.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.066+0000 7fb8817fa640 1 -- 172.21.15.5:0/520858477 <== mon.1 v2:172.21.15.148:3300/0 5 ==== fsmap(e 28) ==== 8374+0+0 (secure 0 0 0) 0x7fb8880bc010 con 0x55d91cc7ddc0 2024-09-12T03:01:11.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.066+0000 7fb8a1efd740 10 client.25570 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T03:01:11.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.066+0000 7fb8a1efd740 10 client.25570 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:01:11.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.066+0000 7fb8a1efd740 20 client.25570 populate_metadata read hostname 'smithi005' 2024-09-12T03:01:11.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.066+0000 7fb8a1efd740 1 --2- 172.21.15.5:0/520858477 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55d91cd03300 0x55d91cd05720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:11.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.066+0000 7fb8a1efd740 4 client.25570 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T03:01:11.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.066+0000 7fb8a1efd740 1 -- 172.21.15.5:0/520858477 --> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d91cbead60 con 0x55d91cd03300 2024-09-12T03:01:11.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.066+0000 7fb89d8c0640 1 --2- 172.21.15.5:0/520858477 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55d91cd03300 0x55d91cd05720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:11.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.067+0000 7fb89d8c0640 1 --2- 172.21.15.5:0/520858477 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55d91cd03300 0x55d91cd05720 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2191bf3c48fda113 server_cookie=1560c3b5321eca8c in_seq=0 out_seq=0 2024-09-12T03:01:11.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.067+0000 7fb8817fa640 10 client.25570 ms_handle_connect on v2:172.21.15.148:6838/2317600000 2024-09-12T03:01:11.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.067+0000 7fb8817fa640 1 -- 172.21.15.5:0/520858477 <== mds.0 v2:172.21.15.148:6838/2317600000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55d91cbead60 con 0x55d91cd03300 2024-09-12T03:01:11.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.067+0000 7fb8817fa640 10 client.25570 handle_command_reply: tid=0 2024-09-12T03:01:11.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.067+0000 7fb8a1efd740 1 client.25570 shutdown 2024-09-12T03:01:11.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.067+0000 7fb8a1efd740 1 -- 172.21.15.5:0/520858477 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d91cc7ddc0 msgr2=0x55d91cc8e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:11.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.067+0000 7fb8a1efd740 1 --2- 172.21.15.5:0/520858477 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d91cc7ddc0 0x55d91cc8e870 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb8880c44f0 tx=0x7fb8880b8130 comp rx=0 tx=0).stop 2024-09-12T03:01:11.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.067+0000 7fb8a1efd740 1 -- 172.21.15.5:0/520858477 shutdown_connections 2024-09-12T03:01:11.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.067+0000 7fb8a1efd740 1 --2- 172.21.15.5:0/520858477 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55d91cd03300 0x55d91cd05720 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:11.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.067+0000 7fb8a1efd740 1 --2- 172.21.15.5:0/520858477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d91cc88680 0x55d91cc8edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:11.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.067+0000 7fb8a1efd740 1 --2- 172.21.15.5:0/520858477 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d91cc7ddc0 0x55d91cc8e870 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:11.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.067+0000 7fb8817fa640 0 client.25570 ms_handle_reset on v2:172.21.15.148:6838/2317600000 2024-09-12T03:01:11.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.067+0000 7fb8a1efd740 1 --2- 172.21.15.5:0/520858477 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d91cc7d4a0 0x55d91cc8e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:11.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.068+0000 7fb8a1efd740 1 -- 172.21.15.5:0/520858477 >> 172.21.15.5:0/520858477 conn(0x55d91cc79a30 msgr2=0x55d91cc86b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:11.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.068+0000 7fb8a1efd740 1 -- 172.21.15.5:0/520858477 shutdown_connections 2024-09-12T03:01:11.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.068+0000 7fb8a1efd740 1 -- 172.21.15.5:0/520858477 wait complete. 2024-09-12T03:01:11.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.068+0000 7fb8a1efd740 20 client.25570 trim_cache size 0 max 16384 2024-09-12T03:01:11.081 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-12T03:01:11.081 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-09-12T03:01:11.081 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-09-12T03:01:11.082 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-12T03:01:11.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.068+0000 7fb89fb24640 1 -- 172.21.15.5:0/493630802 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fb8840a2c00 msgr2=0x7fb8840a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:11.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.068+0000 7fb89fb24640 1 --2- 172.21.15.5:0/493630802 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fb8840a2c00 0x7fb8840a50c0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x55d91cc7adb0 tx=0x7fb890009500 comp rx=0 tx=0).stop 2024-09-12T03:01:11.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.068+0000 7fb89fb24640 1 -- 172.21.15.5:0/493630802 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb898102790 msgr2=0x7fb898074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:11.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.068+0000 7fb89fb24640 1 --2- 172.21.15.5:0/493630802 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb898102790 0x7fb898074a30 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb888004770 tx=0x7fb888003c10 comp rx=0 tx=0).stop 2024-09-12T03:01:11.083 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.068+0000 7fb89fb24640 1 -- 172.21.15.5:0/493630802 shutdown_connections 2024-09-12T03:01:11.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.068+0000 7fb89fb24640 1 --2- 172.21.15.5:0/493630802 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fb8840a2c00 0x7fb8840a50c0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:11.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.068+0000 7fb89fb24640 1 --2- 172.21.15.5:0/493630802 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb898106f10 0x7fb898074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:11.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.068+0000 7fb89fb24640 1 --2- 172.21.15.5:0/493630802 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb898102790 0x7fb898074a30 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:11.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.068+0000 7fb89fb24640 1 --2- 172.21.15.5:0/493630802 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb898101de0 0x7fb8980744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:11.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.068+0000 7fb89fb24640 1 -- 172.21.15.5:0/493630802 >> 172.21.15.5:0/493630802 conn(0x7fb8980fd8c0 msgr2=0x7fb8980fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:11.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.069+0000 7fb89fb24640 1 -- 172.21.15.5:0/493630802 shutdown_connections 2024-09-12T03:01:11.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:11.069+0000 7fb89fb24640 1 -- 172.21.15.5:0/493630802 wait complete. 2024-09-12T03:01:11.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:01:11 smithi146 ceph-mon[35011]: pgmap v423: 97 pgs: 97 active+clean; 82 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-09-12T03:01:11.717 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:01:11 smithi148 ceph-mon[35090]: pgmap v423: 97 pgs: 97 active+clean; 82 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-09-12T03:01:11.788 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:01:11 smithi005 ceph-mon[31197]: pgmap v423: 97 pgs: 97 active+clean; 82 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-09-12T03:01:11.818 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '4d66795', total seconds elapsed: 28.8 2024-09-12T03:01:12.541 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T03:01:12.542 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:01:12.822 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:01:13.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:13.407+0000 7f9266058640 1 -- 172.21.15.5:0/3729961701 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f92601027c0 msgr2=0x7f9260106810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:13.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:13.407+0000 7f9266058640 1 --2- 172.21.15.5:0/3729961701 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f92601027c0 0x7f9260106810 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f9254009900 tx=0x7f925402a410 comp rx=0 tx=0).stop 2024-09-12T03:01:13.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:13.407+0000 7f9266058640 1 -- 172.21.15.5:0/3729961701 shutdown_connections 2024-09-12T03:01:13.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:13.407+0000 7f9266058640 1 --2- 172.21.15.5:0/3729961701 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f92601027c0 0x7f9260106810 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:13.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:13.407+0000 7f9266058640 1 --2- 172.21.15.5:0/3729961701 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9260106f40 0x7f926010eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:13.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:13.407+0000 7f9266058640 1 --2- 172.21.15.5:0/3729961701 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9260101e10 0x7f92601021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:13.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:13.407+0000 7f9266058640 1 -- 172.21.15.5:0/3729961701 >> 172.21.15.5:0/3729961701 conn(0x7f92600fd8d0 msgr2=0x7f92600ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:13.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:13.407+0000 7f9266058640 1 -- 172.21.15.5:0/3729961701 shutdown_connections 2024-09-12T03:01:13.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:13.407+0000 7f9266058640 1 -- 172.21.15.5:0/3729961701 wait complete. 2024-09-12T03:01:13.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:13.408+0000 7f9266058640 1 Processor -- start 2024-09-12T03:01:13.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:13.408+0000 7f9266058640 1 -- start start 2024-09-12T03:01:13.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:13.408+0000 7f9266058640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9260101e10 0x7f9260198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:13.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:13.409+0000 7f9266058640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f92601027c0 0x7f9260198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:13.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:13.409+0000 7f9266058640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9260106f40 0x7f926019cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:13.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:13.409+0000 7f9266058640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9260111fa0 con 0x7f9260106f40 2024-09-12T03:01:13.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:13.409+0000 7f9266058640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f9260111e20 con 0x7f92601027c0 2024-09-12T03:01:13.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:13.409+0000 7f9266058640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f9260112120 con 0x7f9260101e10 2024-09-12T03:01:13.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:13.409+0000 7f925ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9260106f40 0x7f926019cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:13.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:13.409+0000 7f925f7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9260101e10 0x7f9260198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:13.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:13.409+0000 7f925f7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9260101e10 0x7f9260198440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.5:60524/0 (socket says 172.21.15.5:60524) 2024-09-12T03:01:13.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:13.409+0000 7f925ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9260106f40 0x7f926019cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39068/0 (socket says 172.21.15.5:39068) 2024-09-12T03:01:13.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:13.409+0000 7f925f7fe640 1 -- 172.21.15.5:0/2210686394 learned_addr learned my addr 172.21.15.5:0/2210686394 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:01:13.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:13.409+0000 7f925effd640 1 --2- 172.21.15.5:0/2210686394 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f92601027c0 0x7f9260198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:13.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:13.409+0000 7f925ffff640 1 -- 172.21.15.5:0/2210686394 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9260101e10 msgr2=0x7f9260198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:13.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:13.409+0000 7f925ffff640 1 --2- 172.21.15.5:0/2210686394 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9260101e10 0x7f9260198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:13.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:13.409+0000 7f925ffff640 1 -- 172.21.15.5:0/2210686394 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f92601027c0 msgr2=0x7f9260198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:13.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:13.409+0000 7f925ffff640 1 --2- 172.21.15.5:0/2210686394 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f92601027c0 0x7f9260198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:13.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:13.409+0000 7f925ffff640 1 -- 172.21.15.5:0/2210686394 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f926019d3f0 con 0x7f9260106f40 2024-09-12T03:01:13.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:13.409+0000 7f925f7fe640 1 --2- 172.21.15.5:0/2210686394 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9260101e10 0x7f9260198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:01:13.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:13.409+0000 7f925effd640 1 --2- 172.21.15.5:0/2210686394 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f92601027c0 0x7f9260198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:01:13.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:13.409+0000 7f925ffff640 1 --2- 172.21.15.5:0/2210686394 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9260106f40 0x7f926019cd10 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f9258009fd0 tx=0x7f925800ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:13.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:13.410+0000 7f925cff9640 1 -- 172.21.15.5:0/2210686394 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f925800ce20 con 0x7f9260106f40 2024-09-12T03:01:13.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:13.410+0000 7f925cff9640 1 -- 172.21.15.5:0/2210686394 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9258005020 con 0x7f9260106f40 2024-09-12T03:01:13.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:13.410+0000 7f9266058640 1 -- 172.21.15.5:0/2210686394 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f926019d6e0 con 0x7f9260106f40 2024-09-12T03:01:13.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:13.410+0000 7f925cff9640 1 -- 172.21.15.5:0/2210686394 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9258010720 con 0x7f9260106f40 2024-09-12T03:01:13.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:13.410+0000 7f9266058640 1 -- 172.21.15.5:0/2210686394 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92601a4fc0 con 0x7f9260106f40 2024-09-12T03:01:13.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:13.411+0000 7f9266058640 1 -- 172.21.15.5:0/2210686394 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92601021f0 con 0x7f9260106f40 2024-09-12T03:01:13.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:13.412+0000 7f925cff9640 1 -- 172.21.15.5:0/2210686394 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f9258005540 con 0x7f9260106f40 2024-09-12T03:01:13.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:13.413+0000 7f925cff9640 1 --2- 172.21.15.5:0/2210686394 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f92480a2c00 0x7f92480a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:13.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:13.413+0000 7f925f7fe640 1 --2- 172.21.15.5:0/2210686394 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f92480a2c00 0x7f92480a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:13.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:13.413+0000 7f925cff9640 1 -- 172.21.15.5:0/2210686394 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f92580cf550 con 0x7f9260106f40 2024-09-12T03:01:13.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:13.414+0000 7f925f7fe640 1 --2- 172.21.15.5:0/2210686394 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f92480a2c00 0x7f92480a50c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f9250011550 tx=0x7f9250005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:13.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:13.418+0000 7f925cff9640 1 -- 172.21.15.5:0/2210686394 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f925809c4c0 con 0x7f9260106f40 2024-09-12T03:01:13.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:13.663+0000 7f9266058640 1 -- 172.21.15.5:0/2210686394 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f926002d7c0 con 0x7f9260106f40 2024-09-12T03:01:13.664 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:01:13 smithi005 ceph-mon[31197]: pgmap v424: 97 pgs: 97 active+clean; 82 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.6 KiB/s wr, 10 op/s 2024-09-12T03:01:13.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:13.664+0000 7f925cff9640 1 -- 172.21.15.5:0/2210686394 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11971 (secure 0 0 0) 0x7f925809bdc0 con 0x7f9260106f40 2024-09-12T03:01:13.664 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-12T03:01:13.664 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":28,"btime":"2024-09-12T03:00:03:036015+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/4135232521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":4135232521},{"type":"v1","addr":"172.21.15.5:6835","nonce":4135232521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/4290221940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":4290221940},{"type":"v1","addr":"172.21.15.5:6837","nonce":4290221940}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T02:56:16.595562+0000","modified":"2024-09-12T03:00:02.675878+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24502,"mds_1":24487,"mds_2":24481,"mds_3":24479,"mds_4":24458},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/26447341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":26447341},{"type":"v1","addr":"172.21.15.5:6839","nonce":26447341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2393443295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2393443295},{"type":"v1","addr":"172.21.15.5:6841","nonce":2393443295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6833/3686970137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3686970137},{"type":"v1","addr":"172.21.15.146:6833","nonce":3686970137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6833/568512209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":568512209},{"type":"v1","addr":"172.21.15.148:6833","nonce":568512209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6835/2540740420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2540740420},{"type":"v1","addr":"172.21.15.146:6835","nonce":2540740420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.146:6837/1673370937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":1673370937},{"type":"v1","addr":"172.21.15.146:6837","nonce":1673370937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":3,"incarnation":22,"state":"up:active","state_seq":6,"addr":"172.21.15.146:6839/3036767228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":3036767228},{"type":"v1","addr":"172.21.15.146:6839","nonce":3036767228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":2,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.148:6835/4195086427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":4195086427},{"type":"v1","addr":"172.21.15.148:6835","nonce":4195086427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.148:6837/2606873922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2606873922},{"type":"v1","addr":"172.21.15.148:6837","nonce":2606873922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6839/2317600000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":2317600000},{"type":"v1","addr":"172.21.15.148:6839","nonce":2317600000}]},"join_fscid":-1,"export_targets":[1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24502,"qdb_cluster":[24502,24487,24481,24479,24458]},"id":1}]} 2024-09-12T03:01:13.665 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 28 2024-09-12T03:01:13.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:13.666+0000 7f9266058640 1 -- 172.21.15.5:0/2210686394 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f92480a2c00 msgr2=0x7f92480a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:13.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:13.666+0000 7f9266058640 1 --2- 172.21.15.5:0/2210686394 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f92480a2c00 0x7f92480a50c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f9250011550 tx=0x7f9250005f90 comp rx=0 tx=0).stop 2024-09-12T03:01:13.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:13.666+0000 7f9266058640 1 -- 172.21.15.5:0/2210686394 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9260106f40 msgr2=0x7f926019cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:13.666 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:13.666+0000 7f9266058640 1 --2- 172.21.15.5:0/2210686394 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9260106f40 0x7f926019cd10 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f9258009fd0 tx=0x7f925800ef10 comp rx=0 tx=0).stop 2024-09-12T03:01:13.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:13.666+0000 7f9266058640 1 -- 172.21.15.5:0/2210686394 shutdown_connections 2024-09-12T03:01:13.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:13.666+0000 7f9266058640 1 --2- 172.21.15.5:0/2210686394 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f92480a2c00 0x7f92480a50c0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:13.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:13.666+0000 7f9266058640 1 --2- 172.21.15.5:0/2210686394 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9260106f40 0x7f926019cd10 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:13.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:13.666+0000 7f9266058640 1 --2- 172.21.15.5:0/2210686394 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f92601027c0 0x7f9260198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:13.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:13.666+0000 7f9266058640 1 --2- 172.21.15.5:0/2210686394 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9260101e10 0x7f9260198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:13.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:13.666+0000 7f9266058640 1 -- 172.21.15.5:0/2210686394 >> 172.21.15.5:0/2210686394 conn(0x7f92600fd8d0 msgr2=0x7f92600fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:13.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:13.666+0000 7f9266058640 1 -- 172.21.15.5:0/2210686394 shutdown_connections 2024-09-12T03:01:13.667 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:13.666+0000 7f9266058640 1 -- 172.21.15.5:0/2210686394 wait complete. 2024-09-12T03:01:13.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:01:13 smithi146 ceph-mon[35011]: pgmap v424: 97 pgs: 97 active+clean; 82 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.6 KiB/s wr, 10 op/s 2024-09-12T03:01:13.915 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:01:13 smithi148 ceph-mon[35090]: pgmap v424: 97 pgs: 97 active+clean; 82 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.6 KiB/s wr, 10 op/s 2024-09-12T03:01:14.380 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-12T03:01:14.658 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:01:14.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:01:14 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/2210686394' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:01:14.780 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:01:14 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/2210686394' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:01:14.915 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:01:14 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/2210686394' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:01:15.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.156+0000 7fd14521f640 1 -- 172.21.15.5:0/2970692517 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd140102ef0 msgr2=0x7fd14010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:15.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.156+0000 7fd14521f640 1 --2- 172.21.15.5:0/2970692517 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd140102ef0 0x7fd14010da40 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fd138009ca0 tx=0x7fd13802a950 comp rx=0 tx=0).stop 2024-09-12T03:01:15.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.156+0000 7fd14521f640 1 -- 172.21.15.5:0/2970692517 shutdown_connections 2024-09-12T03:01:15.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.156+0000 7fd14521f640 1 --2- 172.21.15.5:0/2970692517 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd140102ef0 0x7fd14010da40 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:15.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.156+0000 7fd14521f640 1 --2- 172.21.15.5:0/2970692517 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd140102570 0x7fd1401029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:15.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.156+0000 7fd14521f640 1 --2- 172.21.15.5:0/2970692517 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd140100e20 0x7fd140101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:15.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.156+0000 7fd14521f640 1 -- 172.21.15.5:0/2970692517 >> 172.21.15.5:0/2970692517 conn(0x7fd1400fc7d0 msgr2=0x7fd1400febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:15.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.156+0000 7fd14521f640 1 -- 172.21.15.5:0/2970692517 shutdown_connections 2024-09-12T03:01:15.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.156+0000 7fd14521f640 1 -- 172.21.15.5:0/2970692517 wait complete. 2024-09-12T03:01:15.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.157+0000 7fd14521f640 1 Processor -- start 2024-09-12T03:01:15.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.157+0000 7fd14521f640 1 -- start start 2024-09-12T03:01:15.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.157+0000 7fd14521f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd140100e20 0x7fd1400fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:15.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.157+0000 7fd14521f640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd140102570 0x7fd1400fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:15.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.157+0000 7fd14521f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd140102ef0 0x7fd1400fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:15.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.157+0000 7fd14521f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd140110f80 con 0x7fd140100e20 2024-09-12T03:01:15.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.157+0000 7fd14521f640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fd140110e00 con 0x7fd140102570 2024-09-12T03:01:15.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.157+0000 7fd14521f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd140111100 con 0x7fd140102ef0 2024-09-12T03:01:15.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.157+0000 7fd13ed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd140100e20 0x7fd1400fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:15.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.157+0000 7fd13ed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd140100e20 0x7fd1400fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40966/0 (socket says 172.21.15.5:40966) 2024-09-12T03:01:15.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.157+0000 7fd13ed9d640 1 -- 172.21.15.5:0/2493060942 learned_addr learned my addr 172.21.15.5:0/2493060942 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:01:15.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.157+0000 7fd13e59c640 1 --2- 172.21.15.5:0/2493060942 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd140102570 0x7fd1400fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:15.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.157+0000 7fd13f59e640 1 --2- 172.21.15.5:0/2493060942 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd140102ef0 0x7fd1400fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:15.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.158+0000 7fd13ed9d640 1 -- 172.21.15.5:0/2493060942 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd140102ef0 msgr2=0x7fd1400fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:15.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.158+0000 7fd13ed9d640 1 --2- 172.21.15.5:0/2493060942 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd140102ef0 0x7fd1400fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:15.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.158+0000 7fd13ed9d640 1 -- 172.21.15.5:0/2493060942 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd140102570 msgr2=0x7fd1400fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:15.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.158+0000 7fd13ed9d640 1 --2- 172.21.15.5:0/2493060942 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd140102570 0x7fd1400fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:15.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.158+0000 7fd13ed9d640 1 -- 172.21.15.5:0/2493060942 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1401a69c0 con 0x7fd140100e20 2024-09-12T03:01:15.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.158+0000 7fd13e59c640 1 --2- 172.21.15.5:0/2493060942 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd140102570 0x7fd1400fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:01:15.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.158+0000 7fd13ed9d640 1 --2- 172.21.15.5:0/2493060942 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd140100e20 0x7fd1400fcc80 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fd13000e9f0 tx=0x7fd13000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:15.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.158+0000 7fd127fff640 1 -- 172.21.15.5:0/2493060942 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd13000cd80 con 0x7fd140100e20 2024-09-12T03:01:15.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.158+0000 7fd14521f640 1 -- 172.21.15.5:0/2493060942 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1401a6cb0 con 0x7fd140100e20 2024-09-12T03:01:15.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.158+0000 7fd127fff640 1 -- 172.21.15.5:0/2493060942 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd130010910 con 0x7fd140100e20 2024-09-12T03:01:15.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.158+0000 7fd127fff640 1 -- 172.21.15.5:0/2493060942 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1300188f0 con 0x7fd140100e20 2024-09-12T03:01:15.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.158+0000 7fd14521f640 1 -- 172.21.15.5:0/2493060942 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1401a71a0 con 0x7fd140100e20 2024-09-12T03:01:15.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.161+0000 7fd127fff640 1 -- 172.21.15.5:0/2493060942 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fd130010ac0 con 0x7fd140100e20 2024-09-12T03:01:15.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.161+0000 7fd1475f8740 1 Processor -- start 2024-09-12T03:01:15.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.161+0000 7fd1475f8740 1 -- start start 2024-09-12T03:01:15.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.161+0000 7fd127fff640 1 --2- 172.21.15.5:0/2493060942 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fd1280a2b80 0x7fd1280a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:15.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.162+0000 7fd1475f8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559af963db40 0x559af963c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:15.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.162+0000 7fd1475f8740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559af963c5d0 0x559af963ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:15.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.162+0000 7fd1475f8740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559af9647370 0x559af9649760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:15.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.162+0000 7fd1475f8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559af95a22f0 con 0x559af963db40 2024-09-12T03:01:15.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.162+0000 7fd1475f8740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x559af9549cf0 con 0x559af9647370 2024-09-12T03:01:15.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.162+0000 7fd1475f8740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x559af959e2b0 con 0x559af963c5d0 2024-09-12T03:01:15.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.162+0000 7fd127fff640 1 -- 172.21.15.5:0/2493060942 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fd1300cf410 con 0x7fd140100e20 2024-09-12T03:01:15.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.162+0000 7fd13f59e640 1 --2- 172.21.15.5:0/2493060942 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fd1280a2b80 0x7fd1280a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:15.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.162+0000 7fd13ed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559af963db40 0x559af963c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:15.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.162+0000 7fd13ed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559af963db40 0x559af963c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40968/0 (socket says 172.21.15.5:40968) 2024-09-12T03:01:15.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.162+0000 7fd13ed9d640 1 -- 172.21.15.5:0/551099306 learned_addr learned my addr 172.21.15.5:0/551099306 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:01:15.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.162+0000 7fd13f59e640 1 --2- 172.21.15.5:0/551099306 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559af9647370 0x559af9649760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:15.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.162+0000 7fd13e59c640 1 --2- 172.21.15.5:0/551099306 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559af963c5d0 0x559af963ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:15.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.162+0000 7fd13ed9d640 1 -- 172.21.15.5:0/551099306 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559af963c5d0 msgr2=0x559af963ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:01:15.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.162+0000 7fd13ed9d640 1 --2- 172.21.15.5:0/551099306 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559af963c5d0 0x559af963ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:15.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.162+0000 7fd13ed9d640 1 -- 172.21.15.5:0/551099306 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559af9647370 msgr2=0x559af9649760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:01:15.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.162+0000 7fd13ed9d640 1 --2- 172.21.15.5:0/551099306 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559af9647370 0x559af9649760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:15.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.162+0000 7fd13ed9d640 1 -- 172.21.15.5:0/551099306 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559af952e3a0 con 0x559af963db40 2024-09-12T03:01:15.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.162+0000 7fd13ed9d640 1 --2- 172.21.15.5:0/551099306 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559af963db40 0x559af963c090 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fd13009bcc0 tx=0x7fd13009bdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24f84465bac45f0f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:15.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.162+0000 7fd13f59e640 1 --2- 172.21.15.5:0/2493060942 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fd1280a2b80 0x7fd1280a5040 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fd138005e30 tx=0x7fd13802ff80 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:15.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.163+0000 7fd1257fa640 1 -- 172.21.15.5:0/551099306 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd13009a030 con 0x559af963db40 2024-09-12T03:01:15.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.163+0000 7fd1257fa640 1 -- 172.21.15.5:0/551099306 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1300983a0 con 0x559af963db40 2024-09-12T03:01:15.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.163+0000 7fd1257fa640 1 -- 172.21.15.5:0/551099306 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd130097d70 con 0x559af963db40 2024-09-12T03:01:15.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.163+0000 7fd1475f8740 1 -- 172.21.15.5:0/551099306 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559af963db40 msgr2=0x559af963c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:15.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.163+0000 7fd1475f8740 1 --2- 172.21.15.5:0/551099306 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559af963db40 0x559af963c090 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fd13009bcc0 tx=0x7fd13009bdf0 comp rx=0 tx=0).stop 2024-09-12T03:01:15.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.163+0000 7fd1475f8740 1 -- 172.21.15.5:0/551099306 shutdown_connections 2024-09-12T03:01:15.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.163+0000 7fd1475f8740 1 --2- 172.21.15.5:0/551099306 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559af9647370 0x559af9649760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:15.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.163+0000 7fd1475f8740 1 --2- 172.21.15.5:0/551099306 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559af963c5d0 0x559af963ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:15.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.163+0000 7fd1475f8740 1 --2- 172.21.15.5:0/551099306 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559af963db40 0x559af963c090 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:15.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.163+0000 7fd1475f8740 1 -- 172.21.15.5:0/551099306 >> 172.21.15.5:0/551099306 conn(0x559af9605310 msgr2=0x559af9638c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:15.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.164+0000 7fd1475f8740 1 -- 172.21.15.5:0/551099306 shutdown_connections 2024-09-12T03:01:15.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.164+0000 7fd1475f8740 1 -- 172.21.15.5:0/551099306 wait complete. 2024-09-12T03:01:15.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.164+0000 7fd1475f8740 1 -- start start 2024-09-12T03:01:15.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.164+0000 7fd1475f8740 1 Processor -- start 2024-09-12T03:01:15.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.164+0000 7fd1475f8740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559af963c5d0 0x559af9646d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:15.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.164+0000 7fd1475f8740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559af963db40 0x559af9643fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:15.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.164+0000 7fd1475f8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559af9647370 0x559af9644570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:15.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.164+0000 7fd1475f8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559af964a070 con 0x559af9647370 2024-09-12T03:01:15.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.164+0000 7fd1475f8740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x559af955d060 con 0x559af963db40 2024-09-12T03:01:15.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.164+0000 7fd1475f8740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x559af964a1f0 con 0x559af963c5d0 2024-09-12T03:01:15.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.164+0000 7fd1475f8740 10 client.0 _ll_register_callbacks cb 0x559af92b08c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:01:15.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.164+0000 7fd13ed9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559af963c5d0 0x559af9646d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:15.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.165+0000 7fd13ed9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559af963c5d0 0x559af9646d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.5:50254/0 (socket says 172.21.15.5:50254) 2024-09-12T03:01:15.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.165+0000 7fd13f59e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559af9647370 0x559af9644570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:15.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.165+0000 7fd13ed9d640 1 -- 172.21.15.5:0/3970096013 learned_addr learned my addr 172.21.15.5:0/3970096013 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:01:15.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.165+0000 7fd13e59c640 1 --2- 172.21.15.5:0/3970096013 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559af963db40 0x559af9643fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:15.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.165+0000 7fd13ed9d640 1 -- 172.21.15.5:0/3970096013 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559af963db40 msgr2=0x559af9643fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:01:15.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.165+0000 7fd13ed9d640 1 --2- 172.21.15.5:0/3970096013 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559af963db40 0x559af9643fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:15.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.165+0000 7fd13ed9d640 1 -- 172.21.15.5:0/3970096013 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559af9647370 msgr2=0x559af9644570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:01:15.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.165+0000 7fd13ed9d640 1 --2- 172.21.15.5:0/3970096013 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559af9647370 0x559af9644570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:15.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.165+0000 7fd13ed9d640 1 -- 172.21.15.5:0/3970096013 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559af93133c0 con 0x559af963c5d0 2024-09-12T03:01:15.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.165+0000 7fd13ed9d640 1 --2- 172.21.15.5:0/3970096013 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559af963c5d0 0x559af9646d20 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd13000c6a0 tx=0x7fd130097ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4b402d9398093dc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:15.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.165+0000 7fd122ff5640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-09-12T03:01:15.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.165+0000 7fd122ff5640 1 -- 172.21.15.5:0/3970096013 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1300b8030 con 0x559af963c5d0 2024-09-12T03:01:15.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.165+0000 7fd122ff5640 1 -- 172.21.15.5:0/3970096013 <== mon.1 v2:172.21.15.148:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fd130099dd0 con 0x559af963c5d0 2024-09-12T03:01:15.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.165+0000 7fd1475f8740 1 -- 172.21.15.5:0/3970096013 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1300983a0 con 0x559af963c5d0 2024-09-12T03:01:15.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.165+0000 7fd122ff5640 1 -- 172.21.15.5:0/3970096013 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd13009b390 con 0x559af963c5d0 2024-09-12T03:01:15.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.165+0000 7fd1475f8740 10 client.25588 fetch_fsmap learned FSMap version 28 2024-09-12T03:01:15.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.165+0000 7fd1475f8740 1 -- 172.21.15.5:0/3970096013 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559af964a070 con 0x559af963c5d0 2024-09-12T03:01:15.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.166+0000 7fd122ff5640 1 -- 172.21.15.5:0/3970096013 <== mon.1 v2:172.21.15.148:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1300983a0 con 0x559af963c5d0 2024-09-12T03:01:15.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.166+0000 7fd122ff5640 1 -- 172.21.15.5:0/3970096013 <== mon.1 v2:172.21.15.148:3300/0 5 ==== fsmap(e 28) ==== 8374+0+0 (secure 0 0 0) 0x7fd1300b4020 con 0x559af963c5d0 2024-09-12T03:01:15.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.166+0000 7fd1475f8740 10 client.25588 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T03:01:15.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.166+0000 7fd1475f8740 10 client.25588 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-12T03:01:15.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.166+0000 7fd1475f8740 20 client.25588 populate_metadata read hostname 'smithi005' 2024-09-12T03:01:15.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.166+0000 7fd1475f8740 1 --2- 172.21.15.5:0/3970096013 >> [v2:172.21.15.146:6836/1673370937,v1:172.21.15.146:6837/1673370937] conn(0x559af9656c70 0x559af9659060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:15.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.166+0000 7fd1475f8740 4 client.25588 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:01:15.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.166+0000 7fd1475f8740 1 -- 172.21.15.5:0/3970096013 --> [v2:172.21.15.146:6836/1673370937,v1:172.21.15.146:6837/1673370937] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559af95af860 con 0x559af9656c70 2024-09-12T03:01:15.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.166+0000 7fd13e59c640 1 --2- 172.21.15.5:0/3970096013 >> [v2:172.21.15.146:6836/1673370937,v1:172.21.15.146:6837/1673370937] conn(0x559af9656c70 0x559af9659060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:15.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.167+0000 7fd13e59c640 1 --2- 172.21.15.5:0/3970096013 >> [v2:172.21.15.146:6836/1673370937,v1:172.21.15.146:6837/1673370937] conn(0x559af9656c70 0x559af9659060 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=df1294007d1e7ef3 server_cookie=6c40d869f97e83ed in_seq=0 out_seq=0 2024-09-12T03:01:15.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.167+0000 7fd122ff5640 10 client.25588 ms_handle_connect on v2:172.21.15.146:6836/1673370937 2024-09-12T03:01:15.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.167+0000 7fd122ff5640 1 -- 172.21.15.5:0/3970096013 <== mds.4 v2:172.21.15.146:6836/1673370937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559af95af860 con 0x559af9656c70 2024-09-12T03:01:15.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.167+0000 7fd122ff5640 10 client.25588 handle_command_reply: tid=0 2024-09-12T03:01:15.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.167+0000 7fd1475f8740 1 client.25588 shutdown 2024-09-12T03:01:15.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.167+0000 7fd1475f8740 1 -- 172.21.15.5:0/3970096013 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559af963c5d0 msgr2=0x559af9646d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:15.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.167+0000 7fd1475f8740 1 --2- 172.21.15.5:0/3970096013 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559af963c5d0 0x559af9646d20 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd13000c6a0 tx=0x7fd130097ed0 comp rx=0 tx=0).stop 2024-09-12T03:01:15.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.167+0000 7fd1475f8740 1 -- 172.21.15.5:0/3970096013 shutdown_connections 2024-09-12T03:01:15.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.167+0000 7fd1475f8740 1 --2- 172.21.15.5:0/3970096013 >> [v2:172.21.15.146:6836/1673370937,v1:172.21.15.146:6837/1673370937] conn(0x559af9656c70 0x559af9659060 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:15.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.167+0000 7fd1475f8740 1 --2- 172.21.15.5:0/3970096013 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559af9647370 0x559af9644570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:15.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.167+0000 7fd1475f8740 1 --2- 172.21.15.5:0/3970096013 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559af963db40 0x559af9643fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:15.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.167+0000 7fd122ff5640 0 client.25588 ms_handle_reset on v2:172.21.15.146:6836/1673370937 2024-09-12T03:01:15.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.167+0000 7fd1475f8740 1 --2- 172.21.15.5:0/3970096013 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559af963c5d0 0x559af9646d20 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:15.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.167+0000 7fd1475f8740 1 -- 172.21.15.5:0/3970096013 >> 172.21.15.5:0/3970096013 conn(0x559af9605310 msgr2=0x559af963e930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:15.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.168+0000 7fd1475f8740 1 -- 172.21.15.5:0/3970096013 shutdown_connections 2024-09-12T03:01:15.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.168+0000 7fd1475f8740 1 -- 172.21.15.5:0/3970096013 wait complete. 2024-09-12T03:01:15.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.168+0000 7fd1475f8740 20 client.25588 trim_cache size 0 max 16384 2024-09-12T03:01:15.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.183+0000 7fd1475f8740 1 Processor -- start 2024-09-12T03:01:15.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.183+0000 7fd1475f8740 1 -- start start 2024-09-12T03:01:15.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.183+0000 7fd1475f8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559af963c5d0 0x559af9653340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:15.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.183+0000 7fd1475f8740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559af963db40 0x559af965e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:15.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.183+0000 7fd1475f8740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559af9647370 0x559af965eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:15.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.183+0000 7fd1475f8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd1401af8c0 con 0x559af963c5d0 2024-09-12T03:01:15.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.183+0000 7fd1475f8740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fd1401b1490 con 0x559af9647370 2024-09-12T03:01:15.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.183+0000 7fd1475f8740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd1401aff90 con 0x559af963db40 2024-09-12T03:01:15.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.183+0000 7fd13ed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559af963c5d0 0x559af9653340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:15.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.183+0000 7fd13ed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559af963c5d0 0x559af9653340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40986/0 (socket says 172.21.15.5:40986) 2024-09-12T03:01:15.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.183+0000 7fd13e59c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559af963db40 0x559af965e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:15.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.183+0000 7fd13f59e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559af9647370 0x559af965eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:15.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.183+0000 7fd13ed9d640 1 -- 172.21.15.5:0/268677432 learned_addr learned my addr 172.21.15.5:0/268677432 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:01:15.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.184+0000 7fd13ed9d640 1 -- 172.21.15.5:0/268677432 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559af963db40 msgr2=0x559af965e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:01:15.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.184+0000 7fd13ed9d640 1 --2- 172.21.15.5:0/268677432 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559af963db40 0x559af965e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:15.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.184+0000 7fd13ed9d640 1 -- 172.21.15.5:0/268677432 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559af9647370 msgr2=0x559af965eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:01:15.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.184+0000 7fd13ed9d640 1 --2- 172.21.15.5:0/268677432 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559af9647370 0x559af965eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:15.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.184+0000 7fd13ed9d640 1 -- 172.21.15.5:0/268677432 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559af963f120 con 0x559af963c5d0 2024-09-12T03:01:15.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.184+0000 7fd13f59e640 1 --2- 172.21.15.5:0/268677432 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559af9647370 0x559af965eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:01:15.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.184+0000 7fd13ed9d640 1 --2- 172.21.15.5:0/268677432 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559af963c5d0 0x559af9653340 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fd13009bcc0 tx=0x7fd13009bdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d850a135c9d0a9fc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:15.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.184+0000 7fd124ff9640 1 -- 172.21.15.5:0/268677432 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd130018a90 con 0x559af963c5d0 2024-09-12T03:01:15.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.184+0000 7fd124ff9640 1 -- 172.21.15.5:0/268677432 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1300aa070 con 0x559af963c5d0 2024-09-12T03:01:15.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.184+0000 7fd124ff9640 1 -- 172.21.15.5:0/268677432 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1300a4da0 con 0x559af963c5d0 2024-09-12T03:01:15.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.185+0000 7fd1475f8740 1 -- 172.21.15.5:0/268677432 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559af963c5d0 msgr2=0x559af9653340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:15.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.185+0000 7fd1475f8740 1 --2- 172.21.15.5:0/268677432 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559af963c5d0 0x559af9653340 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fd13009bcc0 tx=0x7fd13009bdf0 comp rx=0 tx=0).stop 2024-09-12T03:01:15.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.185+0000 7fd1475f8740 1 -- 172.21.15.5:0/268677432 shutdown_connections 2024-09-12T03:01:15.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.185+0000 7fd1475f8740 1 --2- 172.21.15.5:0/268677432 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559af9647370 0x559af965eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:15.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.185+0000 7fd1475f8740 1 --2- 172.21.15.5:0/268677432 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559af963db40 0x559af965e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:15.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.185+0000 7fd1475f8740 1 --2- 172.21.15.5:0/268677432 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559af963c5d0 0x559af9653340 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:15.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.185+0000 7fd1475f8740 1 -- 172.21.15.5:0/268677432 >> 172.21.15.5:0/268677432 conn(0x559af9605310 msgr2=0x559af96430c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:15.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.185+0000 7fd1475f8740 1 -- 172.21.15.5:0/268677432 shutdown_connections 2024-09-12T03:01:15.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.185+0000 7fd1475f8740 1 -- 172.21.15.5:0/268677432 wait complete. 2024-09-12T03:01:15.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.185+0000 7fd1475f8740 1 -- start start 2024-09-12T03:01:15.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.185+0000 7fd1475f8740 1 Processor -- start 2024-09-12T03:01:15.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.186+0000 7fd1475f8740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559af963c5d0 0x559af964f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:15.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.186+0000 7fd1475f8740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559af963db40 0x559af964ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:15.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.186+0000 7fd1475f8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559af9647370 0x559af964b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:15.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.186+0000 7fd1475f8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fd1401b0d30 con 0x559af9647370 2024-09-12T03:01:15.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.186+0000 7fd1475f8740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fd1401b0660 con 0x559af963c5d0 2024-09-12T03:01:15.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.186+0000 7fd1475f8740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fd1401ad8a0 con 0x559af963db40 2024-09-12T03:01:15.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.186+0000 7fd1475f8740 10 client.0 _ll_register_callbacks cb 0x7fd134005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:01:15.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.186+0000 7fd13ed9d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559af963c5d0 0x559af964f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:15.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.186+0000 7fd13f59e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559af9647370 0x559af964b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:15.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.186+0000 7fd13e59c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559af963db40 0x559af964ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:15.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.186+0000 7fd13ed9d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559af963c5d0 0x559af964f940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.5:47016/0 (socket says 172.21.15.5:47016) 2024-09-12T03:01:15.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.186+0000 7fd13ed9d640 1 -- 172.21.15.5:0/3758149956 learned_addr learned my addr 172.21.15.5:0/3758149956 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:01:15.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.186+0000 7fd13f59e640 1 -- 172.21.15.5:0/3758149956 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559af963db40 msgr2=0x559af964ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:01:15.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.186+0000 7fd13f59e640 1 --2- 172.21.15.5:0/3758149956 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559af963db40 0x559af964ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:15.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.186+0000 7fd13f59e640 1 -- 172.21.15.5:0/3758149956 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559af963c5d0 msgr2=0x559af964f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:01:15.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.186+0000 7fd13f59e640 1 --2- 172.21.15.5:0/3758149956 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559af963c5d0 0x559af964f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:15.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.186+0000 7fd13f59e640 1 -- 172.21.15.5:0/3758149956 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559af964b6f0 con 0x559af9647370 2024-09-12T03:01:15.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.186+0000 7fd13f59e640 1 --2- 172.21.15.5:0/3758149956 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559af9647370 0x559af964b1b0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fd138005cc0 tx=0x7fd13802a950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f7ec363f8a108dd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:15.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.186+0000 7fd122ff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T03:01:15.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.186+0000 7fd122ff5640 1 -- 172.21.15.5:0/3758149956 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd138034040 con 0x559af9647370 2024-09-12T03:01:15.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.187+0000 7fd122ff5640 1 -- 172.21.15.5:0/3758149956 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fd138004310 con 0x559af9647370 2024-09-12T03:01:15.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.187+0000 7fd1475f8740 1 -- 172.21.15.5:0/3758149956 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1300aa070 con 0x559af9647370 2024-09-12T03:01:15.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.187+0000 7fd122ff5640 1 -- 172.21.15.5:0/3758149956 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd138004510 con 0x559af9647370 2024-09-12T03:01:15.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.187+0000 7fd1475f8740 10 client.16077 fetch_fsmap learned FSMap version 28 2024-09-12T03:01:15.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.187+0000 7fd1475f8740 1 -- 172.21.15.5:0/3758149956 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559af96c1580 con 0x559af9647370 2024-09-12T03:01:15.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.187+0000 7fd122ff5640 1 -- 172.21.15.5:0/3758149956 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1300aa070 con 0x559af9647370 2024-09-12T03:01:15.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.187+0000 7fd122ff5640 1 -- 172.21.15.5:0/3758149956 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8374+0+0 (secure 0 0 0) 0x7fd138052020 con 0x559af9647370 2024-09-12T03:01:15.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.187+0000 7fd1475f8740 10 client.16077 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T03:01:15.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.187+0000 7fd1475f8740 10 client.16077 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-12T03:01:15.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.187+0000 7fd1475f8740 20 client.16077 populate_metadata read hostname 'smithi005' 2024-09-12T03:01:15.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.187+0000 7fd1475f8740 1 --2- 172.21.15.5:0/3758149956 >> [v2:172.21.15.146:6836/1673370937,v1:172.21.15.146:6837/1673370937] conn(0x559af96c1d20 0x559af96c4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:15.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.187+0000 7fd1475f8740 4 client.16077 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:01:15.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.187+0000 7fd1475f8740 1 -- 172.21.15.5:0/3758149956 --> [v2:172.21.15.146:6836/1673370937,v1:172.21.15.146:6837/1673370937] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559af95a9d70 con 0x559af96c1d20 2024-09-12T03:01:15.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.187+0000 7fd13ed9d640 1 --2- 172.21.15.5:0/3758149956 >> [v2:172.21.15.146:6836/1673370937,v1:172.21.15.146:6837/1673370937] conn(0x559af96c1d20 0x559af96c4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:15.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.188+0000 7fd122ff5640 10 client.16077 ms_handle_connect on v2:172.21.15.146:6836/1673370937 2024-09-12T03:01:15.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.188+0000 7fd13ed9d640 1 --2- 172.21.15.5:0/3758149956 >> [v2:172.21.15.146:6836/1673370937,v1:172.21.15.146:6837/1673370937] conn(0x559af96c1d20 0x559af96c4140 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=544b48ff641f356b server_cookie=8bbbd32a015be304 in_seq=0 out_seq=0 2024-09-12T03:01:15.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.188+0000 7fd122ff5640 1 -- 172.21.15.5:0/3758149956 <== mds.4 v2:172.21.15.146:6836/1673370937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559af95a9d70 con 0x559af96c1d20 2024-09-12T03:01:15.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.188+0000 7fd122ff5640 10 client.16077 handle_command_reply: tid=0 2024-09-12T03:01:15.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.188+0000 7fd1475f8740 1 client.16077 shutdown 2024-09-12T03:01:15.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.188+0000 7fd1475f8740 1 -- 172.21.15.5:0/3758149956 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559af9647370 msgr2=0x559af964b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:15.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.188+0000 7fd1475f8740 1 --2- 172.21.15.5:0/3758149956 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559af9647370 0x559af964b1b0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fd138005cc0 tx=0x7fd13802a950 comp rx=0 tx=0).stop 2024-09-12T03:01:15.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.188+0000 7fd1475f8740 1 -- 172.21.15.5:0/3758149956 shutdown_connections 2024-09-12T03:01:15.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.188+0000 7fd1475f8740 1 --2- 172.21.15.5:0/3758149956 >> [v2:172.21.15.146:6836/1673370937,v1:172.21.15.146:6837/1673370937] conn(0x559af96c1d20 0x559af96c4140 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:15.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.188+0000 7fd1475f8740 1 --2- 172.21.15.5:0/3758149956 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559af9647370 0x559af964b1b0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:15.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.188+0000 7fd1475f8740 1 --2- 172.21.15.5:0/3758149956 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559af963db40 0x559af964ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:15.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.188+0000 7fd1475f8740 1 --2- 172.21.15.5:0/3758149956 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559af963c5d0 0x559af964f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:15.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.188+0000 7fd1475f8740 1 -- 172.21.15.5:0/3758149956 >> 172.21.15.5:0/3758149956 conn(0x559af9605310 msgr2=0x559af9645830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:15.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.188+0000 7fd122ff5640 0 client.16077 ms_handle_reset on v2:172.21.15.146:6836/1673370937 2024-09-12T03:01:15.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.188+0000 7fd1475f8740 1 -- 172.21.15.5:0/3758149956 shutdown_connections 2024-09-12T03:01:15.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.188+0000 7fd1475f8740 1 -- 172.21.15.5:0/3758149956 wait complete. 2024-09-12T03:01:15.194 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.188+0000 7fd1475f8740 20 client.16077 trim_cache size 0 max 16384 2024-09-12T03:01:15.195 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-12T03:01:15.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.189+0000 7fd14521f640 1 -- 172.21.15.5:0/2493060942 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fd1280a2b80 msgr2=0x7fd1280a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:15.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.189+0000 7fd14521f640 1 --2- 172.21.15.5:0/2493060942 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fd1280a2b80 0x7fd1280a5040 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fd138005e30 tx=0x7fd13802ff80 comp rx=0 tx=0).stop 2024-09-12T03:01:15.195 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.189+0000 7fd14521f640 1 -- 172.21.15.5:0/2493060942 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd140100e20 msgr2=0x7fd1400fcc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:15.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.189+0000 7fd14521f640 1 --2- 172.21.15.5:0/2493060942 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd140100e20 0x7fd1400fcc80 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fd13000e9f0 tx=0x7fd13000eeb0 comp rx=0 tx=0).stop 2024-09-12T03:01:15.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.189+0000 7fd14521f640 1 -- 172.21.15.5:0/2493060942 shutdown_connections 2024-09-12T03:01:15.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.189+0000 7fd14521f640 1 --2- 172.21.15.5:0/2493060942 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fd140102ef0 0x7fd1400fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:15.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.189+0000 7fd14521f640 1 --2- 172.21.15.5:0/2493060942 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fd140102570 0x7fd1400fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:15.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.189+0000 7fd14521f640 1 --2- 172.21.15.5:0/2493060942 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fd1280a2b80 0x7fd1280a5040 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:15.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.189+0000 7fd14521f640 1 --2- 172.21.15.5:0/2493060942 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd140100e20 0x7fd1400fcc80 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:15.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.189+0000 7fd14521f640 1 -- 172.21.15.5:0/2493060942 >> 172.21.15.5:0/2493060942 conn(0x7fd1400fc7d0 msgr2=0x7fd14006a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:15.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.189+0000 7fd14521f640 1 -- 172.21.15.5:0/2493060942 shutdown_connections 2024-09-12T03:01:15.196 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:15.189+0000 7fd14521f640 1 -- 172.21.15.5:0/2493060942 wait complete. 2024-09-12T03:01:15.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:01:15 smithi146 ceph-mon[35011]: pgmap v425: 97 pgs: 97 active+clean; 83 MiB data, 692 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 132 KiB/s wr, 51 op/s 2024-09-12T03:01:15.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:01:15 smithi005 ceph-mon[31197]: pgmap v425: 97 pgs: 97 active+clean; 83 MiB data, 692 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 132 KiB/s wr, 51 op/s 2024-09-12T03:01:15.910 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-12T03:01:15.915 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:01:15 smithi148 ceph-mon[35090]: pgmap v425: 97 pgs: 97 active+clean; 83 MiB data, 692 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 132 KiB/s wr, 51 op/s 2024-09-12T03:01:16.185 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:01:16.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.707+0000 7f5f4e0cf640 1 -- 172.21.15.5:0/3084501714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f48102790 msgr2=0x7f5f481067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:16.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.707+0000 7f5f4e0cf640 1 --2- 172.21.15.5:0/3084501714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f48102790 0x7f5f481067e0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c009900 tx=0x7f5f3c02a410 comp rx=0 tx=0).stop 2024-09-12T03:01:16.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.707+0000 7f5f4e0cf640 1 -- 172.21.15.5:0/3084501714 shutdown_connections 2024-09-12T03:01:16.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.707+0000 7f5f4e0cf640 1 --2- 172.21.15.5:0/3084501714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f48102790 0x7f5f481067e0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:16.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.707+0000 7f5f4e0cf640 1 --2- 172.21.15.5:0/3084501714 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f48106f10 0x7f5f4810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:16.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.707+0000 7f5f4e0cf640 1 --2- 172.21.15.5:0/3084501714 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5f48101de0 0x7f5f481021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:16.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.707+0000 7f5f4e0cf640 1 -- 172.21.15.5:0/3084501714 >> 172.21.15.5:0/3084501714 conn(0x7f5f480fd8c0 msgr2=0x7f5f480ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:16.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.707+0000 7f5f4e0cf640 1 -- 172.21.15.5:0/3084501714 shutdown_connections 2024-09-12T03:01:16.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.707+0000 7f5f4e0cf640 1 -- 172.21.15.5:0/3084501714 wait complete. 2024-09-12T03:01:16.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.708+0000 7f5f4e0cf640 1 Processor -- start 2024-09-12T03:01:16.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.708+0000 7f5f4e0cf640 1 -- start start 2024-09-12T03:01:16.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.708+0000 7f5f4e0cf640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f48101de0 0x7f5f48198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:16.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.708+0000 7f5f4e0cf640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5f48102790 0x7f5f481989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:16.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.708+0000 7f5f4e0cf640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f48106f10 0x7f5f4819cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:16.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.708+0000 7f5f4e0cf640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5f48111f70 con 0x7f5f48101de0 2024-09-12T03:01:16.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.708+0000 7f5f4e0cf640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f5f48111df0 con 0x7f5f48102790 2024-09-12T03:01:16.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.708+0000 7f5f4e0cf640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5f481120f0 con 0x7f5f48106f10 2024-09-12T03:01:16.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.708+0000 7f5f477fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f48101de0 0x7f5f48198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:16.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.709+0000 7f5f46ffd640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5f48102790 0x7f5f481989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:16.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.709+0000 7f5f477fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f48101de0 0x7f5f48198490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41026/0 (socket says 172.21.15.5:41026) 2024-09-12T03:01:16.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.709+0000 7f5f477fe640 1 -- 172.21.15.5:0/2852753962 learned_addr learned my addr 172.21.15.5:0/2852753962 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:01:16.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.709+0000 7f5f47fff640 1 --2- 172.21.15.5:0/2852753962 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f48106f10 0x7f5f4819cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:16.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.709+0000 7f5f46ffd640 1 -- 172.21.15.5:0/2852753962 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f48106f10 msgr2=0x7f5f4819cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:16.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.709+0000 7f5f46ffd640 1 --2- 172.21.15.5:0/2852753962 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f48106f10 0x7f5f4819cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:16.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.709+0000 7f5f46ffd640 1 -- 172.21.15.5:0/2852753962 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f48101de0 msgr2=0x7f5f48198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:16.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.709+0000 7f5f46ffd640 1 --2- 172.21.15.5:0/2852753962 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f48101de0 0x7f5f48198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:16.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.709+0000 7f5f46ffd640 1 -- 172.21.15.5:0/2852753962 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f4819d440 con 0x7f5f48102790 2024-09-12T03:01:16.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.709+0000 7f5f47fff640 1 --2- 172.21.15.5:0/2852753962 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f48106f10 0x7f5f4819cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:01:16.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.709+0000 7f5f46ffd640 1 --2- 172.21.15.5:0/2852753962 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5f48102790 0x7f5f481989d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c005940 tx=0x7f5f3c0040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:16.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.709+0000 7f5f44ff9640 1 -- 172.21.15.5:0/2852753962 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f3c03f030 con 0x7f5f48102790 2024-09-12T03:01:16.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.709+0000 7f5f44ff9640 1 -- 172.21.15.5:0/2852753962 <== mon.2 v2:172.21.15.146:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f3c0044c0 con 0x7f5f48102790 2024-09-12T03:01:16.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.709+0000 7f5f4e0cf640 1 -- 172.21.15.5:0/2852753962 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f4819d6d0 con 0x7f5f48102790 2024-09-12T03:01:16.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.709+0000 7f5f44ff9640 1 -- 172.21.15.5:0/2852753962 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f3c035070 con 0x7f5f48102790 2024-09-12T03:01:16.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.709+0000 7f5f4e0cf640 1 -- 172.21.15.5:0/2852753962 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f481a5010 con 0x7f5f48102790 2024-09-12T03:01:16.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.712+0000 7f5f44ff9640 1 -- 172.21.15.5:0/2852753962 <== mon.2 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f5f3c042030 con 0x7f5f48102790 2024-09-12T03:01:16.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.712+0000 7f5f504a8740 1 Processor -- start 2024-09-12T03:01:16.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.712+0000 7f5f504a8740 1 -- start start 2024-09-12T03:01:16.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.712+0000 7f5f504a8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bfe4c47b40 0x55bfe4c46090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:16.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.712+0000 7f5f504a8740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bfe4c465d0 0x55bfe4c46a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:16.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.712+0000 7f5f504a8740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bfe4c55780 0x55bfe4c57b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:16.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.712+0000 7f5f504a8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55bfe4bac2f0 con 0x55bfe4c47b40 2024-09-12T03:01:16.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.712+0000 7f5f504a8740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55bfe4b53cf0 con 0x55bfe4c465d0 2024-09-12T03:01:16.714 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.712+0000 7f5f504a8740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55bfe4ba82b0 con 0x55bfe4c55780 2024-09-12T03:01:16.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.712+0000 7f5f44ff9640 1 --2- 172.21.15.5:0/2852753962 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f5f300a2b80 0x7f5f300a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:16.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.712+0000 7f5f46ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bfe4c55780 0x55bfe4c57b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:16.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.713+0000 7f5f47fff640 1 --2- 172.21.15.5:0/2852753962 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f5f300a2b80 0x7f5f300a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:16.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.713+0000 7f5f46ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bfe4c55780 0x55bfe4c57b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.5:50310/0 (socket says 172.21.15.5:50310) 2024-09-12T03:01:16.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.713+0000 7f5f46ffd640 1 -- 172.21.15.5:0/3346423926 learned_addr learned my addr 172.21.15.5:0/3346423926 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:01:16.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.713+0000 7f5f477fe640 1 --2- 172.21.15.5:0/3346423926 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bfe4c465d0 0x55bfe4c46a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:16.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.713+0000 7f5f44ff9640 1 -- 172.21.15.5:0/2852753962 <== mon.2 v2:172.21.15.146:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f5f3c0ef380 con 0x7f5f48102790 2024-09-12T03:01:16.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.713+0000 7f5f47fff640 1 --2- 172.21.15.5:0/3346423926 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bfe4c47b40 0x55bfe4c46090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:16.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.713+0000 7f5f46ffd640 1 -- 172.21.15.5:0/3346423926 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bfe4c465d0 msgr2=0x55bfe4c46a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:01:16.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.713+0000 7f5f46ffd640 1 --2- 172.21.15.5:0/3346423926 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bfe4c465d0 0x55bfe4c46a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:16.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.713+0000 7f5f46ffd640 1 -- 172.21.15.5:0/3346423926 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bfe4c47b40 msgr2=0x55bfe4c46090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:01:16.715 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.713+0000 7f5f46ffd640 1 --2- 172.21.15.5:0/3346423926 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bfe4c47b40 0x55bfe4c46090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:16.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.713+0000 7f5f46ffd640 1 -- 172.21.15.5:0/3346423926 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bfe4b383a0 con 0x55bfe4c55780 2024-09-12T03:01:16.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.713+0000 7f5f46ffd640 1 --2- 172.21.15.5:0/3346423926 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bfe4c55780 0x55bfe4c57b70 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c0b85c0 tx=0x7f5f3c08ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=abffc0af1789fd34 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:16.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.713+0000 7f5f47fff640 1 --2- 172.21.15.5:0/2852753962 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f5f300a2b80 0x7f5f300a5040 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f5f40009270 tx=0x7f5f400091e0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:16.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.713+0000 7f5f2dffb640 1 -- 172.21.15.5:0/3346423926 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f3c049070 con 0x55bfe4c55780 2024-09-12T03:01:16.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.713+0000 7f5f2dffb640 1 -- 172.21.15.5:0/3346423926 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f3c0a6040 con 0x55bfe4c55780 2024-09-12T03:01:16.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.713+0000 7f5f2dffb640 1 -- 172.21.15.5:0/3346423926 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f3c0bbe50 con 0x55bfe4c55780 2024-09-12T03:01:16.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.714+0000 7f5f504a8740 1 -- 172.21.15.5:0/3346423926 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bfe4c55780 msgr2=0x55bfe4c57b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:16.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.714+0000 7f5f504a8740 1 --2- 172.21.15.5:0/3346423926 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bfe4c55780 0x55bfe4c57b70 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c0b85c0 tx=0x7f5f3c08ecb0 comp rx=0 tx=0).stop 2024-09-12T03:01:16.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.714+0000 7f5f504a8740 1 -- 172.21.15.5:0/3346423926 shutdown_connections 2024-09-12T03:01:16.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.714+0000 7f5f504a8740 1 --2- 172.21.15.5:0/3346423926 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bfe4c55780 0x55bfe4c57b70 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:16.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.714+0000 7f5f504a8740 1 --2- 172.21.15.5:0/3346423926 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bfe4c465d0 0x55bfe4c46a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:16.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.714+0000 7f5f504a8740 1 --2- 172.21.15.5:0/3346423926 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bfe4c47b40 0x55bfe4c46090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:16.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.714+0000 7f5f504a8740 1 -- 172.21.15.5:0/3346423926 >> 172.21.15.5:0/3346423926 conn(0x55bfe4c0f310 msgr2=0x55bfe4c42c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:16.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.714+0000 7f5f504a8740 1 -- 172.21.15.5:0/3346423926 shutdown_connections 2024-09-12T03:01:16.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.714+0000 7f5f504a8740 1 -- 172.21.15.5:0/3346423926 wait complete. 2024-09-12T03:01:16.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.715+0000 7f5f504a8740 1 -- start start 2024-09-12T03:01:16.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.715+0000 7f5f504a8740 1 Processor -- start 2024-09-12T03:01:16.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.715+0000 7f5f504a8740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bfe4c465d0 0x55bfe4c55170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:16.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.715+0000 7f5f504a8740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bfe4c47b40 0x55bfe4c52420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:16.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.715+0000 7f5f504a8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bfe4c55780 0x55bfe4c52a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:16.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.715+0000 7f5f504a8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55bfe4ba82b0 con 0x55bfe4c55780 2024-09-12T03:01:16.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.715+0000 7f5f504a8740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55bfe4b67060 con 0x55bfe4c47b40 2024-09-12T03:01:16.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.715+0000 7f5f504a8740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55bfe4c58240 con 0x55bfe4c465d0 2024-09-12T03:01:16.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.715+0000 7f5f504a8740 10 client.0 _ll_register_callbacks cb 0x55bfe48ba8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:01:16.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.715+0000 7f5f46ffd640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bfe4c47b40 0x55bfe4c52420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:16.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.715+0000 7f5f477fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bfe4c465d0 0x55bfe4c55170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:16.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.715+0000 7f5f46ffd640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bfe4c47b40 0x55bfe4c52420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.5:47050/0 (socket says 172.21.15.5:47050) 2024-09-12T03:01:16.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.715+0000 7f5f47fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bfe4c55780 0x55bfe4c52a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:16.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.715+0000 7f5f46ffd640 1 -- 172.21.15.5:0/1683577242 learned_addr learned my addr 172.21.15.5:0/1683577242 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:01:16.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.716+0000 7f5f477fe640 1 -- 172.21.15.5:0/1683577242 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bfe4c47b40 msgr2=0x55bfe4c52420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:01:16.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.716+0000 7f5f477fe640 1 --2- 172.21.15.5:0/1683577242 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bfe4c47b40 0x55bfe4c52420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:16.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.716+0000 7f5f477fe640 1 -- 172.21.15.5:0/1683577242 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bfe4c55780 msgr2=0x55bfe4c52a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:01:16.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.716+0000 7f5f477fe640 1 --2- 172.21.15.5:0/1683577242 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bfe4c55780 0x55bfe4c52a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:16.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.716+0000 7f5f477fe640 1 -- 172.21.15.5:0/1683577242 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bfe491d3c0 con 0x55bfe4c465d0 2024-09-12T03:01:16.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.716+0000 7f5f46ffd640 1 --2- 172.21.15.5:0/1683577242 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bfe4c47b40 0x55bfe4c52420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:01:16.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.716+0000 7f5f477fe640 1 --2- 172.21.15.5:0/1683577242 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bfe4c465d0 0x55bfe4c55170 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5f38009620 tx=0x7f5f38012bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1703efa12ac34272 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:16.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.716+0000 7f5f26ffd640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-09-12T03:01:16.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.716+0000 7f5f26ffd640 1 -- 172.21.15.5:0/1683577242 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f38004050 con 0x55bfe4c465d0 2024-09-12T03:01:16.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.716+0000 7f5f26ffd640 1 -- 172.21.15.5:0/1683577242 <== mon.1 v2:172.21.15.148:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f5f380041f0 con 0x55bfe4c465d0 2024-09-12T03:01:16.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.716+0000 7f5f504a8740 1 -- 172.21.15.5:0/1683577242 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f3c0a6040 con 0x55bfe4c465d0 2024-09-12T03:01:16.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.716+0000 7f5f26ffd640 1 -- 172.21.15.5:0/1683577242 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5f380043f0 con 0x55bfe4c465d0 2024-09-12T03:01:16.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.716+0000 7f5f504a8740 10 client.25606 fetch_fsmap learned FSMap version 28 2024-09-12T03:01:16.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.716+0000 7f5f504a8740 1 -- 172.21.15.5:0/1683577242 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55bfe4c63bc0 con 0x55bfe4c465d0 2024-09-12T03:01:16.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.717+0000 7f5f26ffd640 1 -- 172.21.15.5:0/1683577242 <== mon.1 v2:172.21.15.148:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f3c0a6040 con 0x55bfe4c465d0 2024-09-12T03:01:16.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.717+0000 7f5f26ffd640 1 -- 172.21.15.5:0/1683577242 <== mon.1 v2:172.21.15.148:3300/0 5 ==== fsmap(e 28) ==== 8374+0+0 (secure 0 0 0) 0x7f5f38003c70 con 0x55bfe4c465d0 2024-09-12T03:01:16.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.717+0000 7f5f504a8740 10 client.25606 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T03:01:16.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.717+0000 7f5f504a8740 10 client.25606 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.b 2024-09-12T03:01:16.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.717+0000 7f5f504a8740 20 client.25606 populate_metadata read hostname 'smithi005' 2024-09-12T03:01:16.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.717+0000 7f5f504a8740 1 --2- 172.21.15.5:0/1683577242 >> [v2:172.21.15.146:6838/3036767228,v1:172.21.15.146:6839/3036767228] conn(0x55bfe4c65080 0x55bfe4c67490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:16.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.717+0000 7f5f504a8740 4 client.25606 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:01:16.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.717+0000 7f5f504a8740 1 -- 172.21.15.5:0/1683577242 --> [v2:172.21.15.146:6838/3036767228,v1:172.21.15.146:6839/3036767228] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bfe4bb9860 con 0x55bfe4c65080 2024-09-12T03:01:16.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.717+0000 7f5f46ffd640 1 --2- 172.21.15.5:0/1683577242 >> [v2:172.21.15.146:6838/3036767228,v1:172.21.15.146:6839/3036767228] conn(0x55bfe4c65080 0x55bfe4c67490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:16.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.718+0000 7f5f26ffd640 10 client.25606 ms_handle_connect on v2:172.21.15.146:6838/3036767228 2024-09-12T03:01:16.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.718+0000 7f5f46ffd640 1 --2- 172.21.15.5:0/1683577242 >> [v2:172.21.15.146:6838/3036767228,v1:172.21.15.146:6839/3036767228] conn(0x55bfe4c65080 0x55bfe4c67490 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=631fbf9de63ca9f2 server_cookie=15eb2df4ae7ffff5 in_seq=0 out_seq=0 2024-09-12T03:01:16.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.718+0000 7f5f26ffd640 1 -- 172.21.15.5:0/1683577242 <== mds.3 v2:172.21.15.146:6838/3036767228 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bfe4bb9860 con 0x55bfe4c65080 2024-09-12T03:01:16.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.718+0000 7f5f26ffd640 10 client.25606 handle_command_reply: tid=0 2024-09-12T03:01:16.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.718+0000 7f5f504a8740 1 client.25606 shutdown 2024-09-12T03:01:16.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.718+0000 7f5f504a8740 1 -- 172.21.15.5:0/1683577242 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bfe4c465d0 msgr2=0x55bfe4c55170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:16.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.718+0000 7f5f504a8740 1 --2- 172.21.15.5:0/1683577242 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bfe4c465d0 0x55bfe4c55170 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5f38009620 tx=0x7f5f38012bf0 comp rx=0 tx=0).stop 2024-09-12T03:01:16.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.718+0000 7f5f504a8740 1 -- 172.21.15.5:0/1683577242 shutdown_connections 2024-09-12T03:01:16.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.718+0000 7f5f504a8740 1 --2- 172.21.15.5:0/1683577242 >> [v2:172.21.15.146:6838/3036767228,v1:172.21.15.146:6839/3036767228] conn(0x55bfe4c65080 0x55bfe4c67490 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:16.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.719+0000 7f5f504a8740 1 --2- 172.21.15.5:0/1683577242 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bfe4c55780 0x55bfe4c52a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:16.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.719+0000 7f5f504a8740 1 --2- 172.21.15.5:0/1683577242 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bfe4c47b40 0x55bfe4c52420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:16.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.719+0000 7f5f504a8740 1 --2- 172.21.15.5:0/1683577242 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bfe4c465d0 0x55bfe4c55170 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:16.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.719+0000 7f5f26ffd640 0 client.25606 ms_handle_reset on v2:172.21.15.146:6838/3036767228 2024-09-12T03:01:16.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.719+0000 7f5f504a8740 1 -- 172.21.15.5:0/1683577242 >> 172.21.15.5:0/1683577242 conn(0x55bfe4c0f310 msgr2=0x55bfe4c48930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:16.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.719+0000 7f5f504a8740 1 -- 172.21.15.5:0/1683577242 shutdown_connections 2024-09-12T03:01:16.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.719+0000 7f5f504a8740 1 -- 172.21.15.5:0/1683577242 wait complete. 2024-09-12T03:01:16.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.719+0000 7f5f504a8740 20 client.25606 trim_cache size 0 max 16384 2024-09-12T03:01:16.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.733+0000 7f5f504a8740 1 Processor -- start 2024-09-12T03:01:16.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.733+0000 7f5f504a8740 1 -- start start 2024-09-12T03:01:16.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.733+0000 7f5f504a8740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bfe4c465d0 0x55bfe4c61360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:16.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.733+0000 7f5f504a8740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bfe4c47b40 0x55bfe4c51ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:16.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.733+0000 7f5f504a8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bfe4c55780 0x55bfe4c51400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:16.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.733+0000 7f5f504a8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5f481ab330 con 0x55bfe4c55780 2024-09-12T03:01:16.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.733+0000 7f5f504a8740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f5f481acf00 con 0x55bfe4c465d0 2024-09-12T03:01:16.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.733+0000 7f5f504a8740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5f481aba00 con 0x55bfe4c47b40 2024-09-12T03:01:16.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.733+0000 7f5f47fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bfe4c55780 0x55bfe4c51400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:16.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.733+0000 7f5f477fe640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bfe4c465d0 0x55bfe4c61360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:16.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.733+0000 7f5f46ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bfe4c47b40 0x55bfe4c51ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:16.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.733+0000 7f5f477fe640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bfe4c465d0 0x55bfe4c61360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.5:47058/0 (socket says 172.21.15.5:47058) 2024-09-12T03:01:16.734 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.733+0000 7f5f477fe640 1 -- 172.21.15.5:0/2297757860 learned_addr learned my addr 172.21.15.5:0/2297757860 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:01:16.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.734+0000 7f5f46ffd640 1 -- 172.21.15.5:0/2297757860 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bfe4c465d0 msgr2=0x55bfe4c61360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:01:16.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.734+0000 7f5f46ffd640 1 --2- 172.21.15.5:0/2297757860 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bfe4c465d0 0x55bfe4c61360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:16.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.734+0000 7f5f46ffd640 1 -- 172.21.15.5:0/2297757860 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bfe4c55780 msgr2=0x55bfe4c51400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:01:16.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.734+0000 7f5f46ffd640 1 --2- 172.21.15.5:0/2297757860 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bfe4c55780 0x55bfe4c51400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:16.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.734+0000 7f5f46ffd640 1 -- 172.21.15.5:0/2297757860 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bfe4c43960 con 0x55bfe4c47b40 2024-09-12T03:01:16.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.734+0000 7f5f477fe640 1 --2- 172.21.15.5:0/2297757860 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bfe4c465d0 0x55bfe4c61360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:01:16.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.734+0000 7f5f46ffd640 1 --2- 172.21.15.5:0/2297757860 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bfe4c47b40 0x55bfe4c51ce0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c0ef730 tx=0x7f5f3c03b5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24a77408c8c88cda server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:16.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.734+0000 7f5f2d7fa640 1 -- 172.21.15.5:0/2297757860 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f3c049040 con 0x55bfe4c47b40 2024-09-12T03:01:16.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.734+0000 7f5f2d7fa640 1 -- 172.21.15.5:0/2297757860 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f3c0a6040 con 0x55bfe4c47b40 2024-09-12T03:01:16.738 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.734+0000 7f5f2d7fa640 1 -- 172.21.15.5:0/2297757860 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f3c0bc0b0 con 0x55bfe4c47b40 2024-09-12T03:01:16.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.735+0000 7f5f504a8740 1 -- 172.21.15.5:0/2297757860 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bfe4c47b40 msgr2=0x55bfe4c51ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:16.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.735+0000 7f5f504a8740 1 --2- 172.21.15.5:0/2297757860 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bfe4c47b40 0x55bfe4c51ce0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c0ef730 tx=0x7f5f3c03b5d0 comp rx=0 tx=0).stop 2024-09-12T03:01:16.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.735+0000 7f5f504a8740 1 -- 172.21.15.5:0/2297757860 shutdown_connections 2024-09-12T03:01:16.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.735+0000 7f5f504a8740 1 --2- 172.21.15.5:0/2297757860 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bfe4c55780 0x55bfe4c51400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:16.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.735+0000 7f5f504a8740 1 --2- 172.21.15.5:0/2297757860 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bfe4c47b40 0x55bfe4c51ce0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:16.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.735+0000 7f5f504a8740 1 --2- 172.21.15.5:0/2297757860 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bfe4c465d0 0x55bfe4c61360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:16.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.735+0000 7f5f504a8740 1 -- 172.21.15.5:0/2297757860 >> 172.21.15.5:0/2297757860 conn(0x55bfe4c0f310 msgr2=0x55bfe4c50cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:16.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.735+0000 7f5f504a8740 1 -- 172.21.15.5:0/2297757860 shutdown_connections 2024-09-12T03:01:16.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.735+0000 7f5f504a8740 1 -- 172.21.15.5:0/2297757860 wait complete. 2024-09-12T03:01:16.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.735+0000 7f5f504a8740 1 -- start start 2024-09-12T03:01:16.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.736+0000 7f5f504a8740 1 Processor -- start 2024-09-12T03:01:16.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.736+0000 7f5f504a8740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bfe4c465d0 0x55bfe4c5d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:16.739 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.736+0000 7f5f504a8740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bfe4c47b40 0x55bfe4c4eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:16.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.736+0000 7f5f504a8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bfe4c55780 0x55bfe4c4f420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:16.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.736+0000 7f5f504a8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5f481ac7a0 con 0x55bfe4c55780 2024-09-12T03:01:16.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.736+0000 7f5f504a8740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f5f481ac0d0 con 0x55bfe4c47b40 2024-09-12T03:01:16.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.736+0000 7f5f504a8740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5f481a9310 con 0x55bfe4c465d0 2024-09-12T03:01:16.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.736+0000 7f5f504a8740 10 client.0 _ll_register_callbacks cb 0x7f5f380099c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:01:16.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.736+0000 7f5f477fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bfe4c465d0 0x55bfe4c5d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:16.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.736+0000 7f5f477fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bfe4c465d0 0x55bfe4c5d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.5:50334/0 (socket says 172.21.15.5:50334) 2024-09-12T03:01:16.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.736+0000 7f5f477fe640 1 -- 172.21.15.5:0/3479480590 learned_addr learned my addr 172.21.15.5:0/3479480590 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:01:16.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.736+0000 7f5f46ffd640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bfe4c47b40 0x55bfe4c4eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:16.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.736+0000 7f5f47fff640 1 --2- 172.21.15.5:0/3479480590 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bfe4c55780 0x55bfe4c4f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:16.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.736+0000 7f5f477fe640 1 -- 172.21.15.5:0/3479480590 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bfe4c47b40 msgr2=0x55bfe4c4eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:01:16.740 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.736+0000 7f5f477fe640 1 --2- 172.21.15.5:0/3479480590 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bfe4c47b40 0x55bfe4c4eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:16.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.736+0000 7f5f477fe640 1 -- 172.21.15.5:0/3479480590 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bfe4c55780 msgr2=0x55bfe4c4f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:01:16.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.737+0000 7f5f477fe640 1 --2- 172.21.15.5:0/3479480590 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bfe4c55780 0x55bfe4c4f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:16.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.737+0000 7f5f477fe640 1 -- 172.21.15.5:0/3479480590 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bfe4c4f9e0 con 0x55bfe4c465d0 2024-09-12T03:01:16.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.737+0000 7f5f477fe640 1 --2- 172.21.15.5:0/3479480590 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bfe4c465d0 0x55bfe4c5d9a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5f38001eb0 tx=0x7f5f38009f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=716ed45c00b63331 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:16.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.737+0000 7f5f26ffd640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-09-12T03:01:16.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.737+0000 7f5f26ffd640 1 -- 172.21.15.5:0/3479480590 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f38000e60 con 0x55bfe4c465d0 2024-09-12T03:01:16.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.737+0000 7f5f26ffd640 1 -- 172.21.15.5:0/3479480590 <== mon.1 v2:172.21.15.148:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f5f38010cd0 con 0x55bfe4c465d0 2024-09-12T03:01:16.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.737+0000 7f5f504a8740 1 -- 172.21.15.5:0/3479480590 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f3c0a6040 con 0x55bfe4c465d0 2024-09-12T03:01:16.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.737+0000 7f5f26ffd640 1 -- 172.21.15.5:0/3479480590 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5f38038050 con 0x55bfe4c465d0 2024-09-12T03:01:16.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.737+0000 7f5f504a8740 10 client.25612 fetch_fsmap learned FSMap version 28 2024-09-12T03:01:16.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.737+0000 7f5f504a8740 1 -- 172.21.15.5:0/3479480590 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55bfe4ccfaa0 con 0x55bfe4c465d0 2024-09-12T03:01:16.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.737+0000 7f5f26ffd640 1 -- 172.21.15.5:0/3479480590 <== mon.1 v2:172.21.15.148:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f3c0a6040 con 0x55bfe4c465d0 2024-09-12T03:01:16.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.738+0000 7f5f26ffd640 1 -- 172.21.15.5:0/3479480590 <== mon.1 v2:172.21.15.148:3300/0 5 ==== fsmap(e 28) ==== 8374+0+0 (secure 0 0 0) 0x7f5f3802f070 con 0x55bfe4c465d0 2024-09-12T03:01:16.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.738+0000 7f5f504a8740 10 client.25612 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T03:01:16.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.738+0000 7f5f504a8740 10 client.25612 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.b 2024-09-12T03:01:16.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.738+0000 7f5f504a8740 20 client.25612 populate_metadata read hostname 'smithi005' 2024-09-12T03:01:16.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.738+0000 7f5f504a8740 1 --2- 172.21.15.5:0/3479480590 >> [v2:172.21.15.146:6838/3036767228,v1:172.21.15.146:6839/3036767228] conn(0x55bfe4cd0250 0x55bfe4cd2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:16.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.738+0000 7f5f504a8740 4 client.25612 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:01:16.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.738+0000 7f5f504a8740 1 -- 172.21.15.5:0/3479480590 --> [v2:172.21.15.146:6838/3036767228,v1:172.21.15.146:6839/3036767228] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bfe4bb3d70 con 0x55bfe4cd0250 2024-09-12T03:01:16.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.738+0000 7f5f46ffd640 1 --2- 172.21.15.5:0/3479480590 >> [v2:172.21.15.146:6838/3036767228,v1:172.21.15.146:6839/3036767228] conn(0x55bfe4cd0250 0x55bfe4cd2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:16.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.738+0000 7f5f46ffd640 1 --2- 172.21.15.5:0/3479480590 >> [v2:172.21.15.146:6838/3036767228,v1:172.21.15.146:6839/3036767228] conn(0x55bfe4cd0250 0x55bfe4cd2670 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=85e256498596e29f server_cookie=d2493d5fc39745f1 in_seq=0 out_seq=0 2024-09-12T03:01:16.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.738+0000 7f5f26ffd640 10 client.25612 ms_handle_connect on v2:172.21.15.146:6838/3036767228 2024-09-12T03:01:16.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.738+0000 7f5f26ffd640 1 -- 172.21.15.5:0/3479480590 <== mds.3 v2:172.21.15.146:6838/3036767228 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bfe4bb3d70 con 0x55bfe4cd0250 2024-09-12T03:01:16.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.738+0000 7f5f26ffd640 10 client.25612 handle_command_reply: tid=0 2024-09-12T03:01:16.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.738+0000 7f5f504a8740 1 client.25612 shutdown 2024-09-12T03:01:16.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.739+0000 7f5f504a8740 1 -- 172.21.15.5:0/3479480590 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bfe4c465d0 msgr2=0x55bfe4c5d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:16.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.739+0000 7f5f504a8740 1 --2- 172.21.15.5:0/3479480590 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bfe4c465d0 0x55bfe4c5d9a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5f38001eb0 tx=0x7f5f38009f50 comp rx=0 tx=0).stop 2024-09-12T03:01:16.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.739+0000 7f5f504a8740 1 -- 172.21.15.5:0/3479480590 shutdown_connections 2024-09-12T03:01:16.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.739+0000 7f5f504a8740 1 --2- 172.21.15.5:0/3479480590 >> [v2:172.21.15.146:6838/3036767228,v1:172.21.15.146:6839/3036767228] conn(0x55bfe4cd0250 0x55bfe4cd2670 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:16.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.739+0000 7f5f504a8740 1 --2- 172.21.15.5:0/3479480590 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bfe4c55780 0x55bfe4c4f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:16.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.739+0000 7f5f504a8740 1 --2- 172.21.15.5:0/3479480590 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55bfe4c47b40 0x55bfe4c4eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:16.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.739+0000 7f5f26ffd640 0 client.25612 ms_handle_reset on v2:172.21.15.146:6838/3036767228 2024-09-12T03:01:16.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.739+0000 7f5f504a8740 1 --2- 172.21.15.5:0/3479480590 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55bfe4c465d0 0x55bfe4c5d9a0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:16.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.739+0000 7f5f504a8740 1 -- 172.21.15.5:0/3479480590 >> 172.21.15.5:0/3479480590 conn(0x55bfe4c0f310 msgr2=0x55bfe4c41d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:16.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.739+0000 7f5f504a8740 1 -- 172.21.15.5:0/3479480590 shutdown_connections 2024-09-12T03:01:16.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.739+0000 7f5f504a8740 1 -- 172.21.15.5:0/3479480590 wait complete. 2024-09-12T03:01:16.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.739+0000 7f5f504a8740 20 client.25612 trim_cache size 0 max 16384 2024-09-12T03:01:16.745 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-12T03:01:16.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.739+0000 7f5f4e0cf640 1 -- 172.21.15.5:0/2852753962 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f5f300a2b80 msgr2=0x7f5f300a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:16.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.739+0000 7f5f4e0cf640 1 --2- 172.21.15.5:0/2852753962 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f5f300a2b80 0x7f5f300a5040 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f5f40009270 tx=0x7f5f400091e0 comp rx=0 tx=0).stop 2024-09-12T03:01:16.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.739+0000 7f5f4e0cf640 1 -- 172.21.15.5:0/2852753962 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5f48102790 msgr2=0x7f5f481989d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:16.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.739+0000 7f5f4e0cf640 1 --2- 172.21.15.5:0/2852753962 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5f48102790 0x7f5f481989d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c005940 tx=0x7f5f3c0040c0 comp rx=0 tx=0).stop 2024-09-12T03:01:16.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.740+0000 7f5f4e0cf640 1 -- 172.21.15.5:0/2852753962 shutdown_connections 2024-09-12T03:01:16.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.740+0000 7f5f4e0cf640 1 --2- 172.21.15.5:0/2852753962 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f48106f10 0x7f5f4819cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:16.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.740+0000 7f5f4e0cf640 1 --2- 172.21.15.5:0/2852753962 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5f48102790 0x7f5f481989d0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:16.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.740+0000 7f5f4e0cf640 1 --2- 172.21.15.5:0/2852753962 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f5f300a2b80 0x7f5f300a5040 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:16.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.740+0000 7f5f4e0cf640 1 --2- 172.21.15.5:0/2852753962 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f48101de0 0x7f5f48198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:16.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.740+0000 7f5f4e0cf640 1 -- 172.21.15.5:0/2852753962 >> 172.21.15.5:0/2852753962 conn(0x7f5f480fd8c0 msgr2=0x7f5f480fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:16.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.740+0000 7f5f4e0cf640 1 -- 172.21.15.5:0/2852753962 shutdown_connections 2024-09-12T03:01:16.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:16.740+0000 7f5f4e0cf640 1 -- 172.21.15.5:0/2852753962 wait complete. 2024-09-12T03:01:17.521 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T03:01:17.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:01:17 smithi146 ceph-mon[35011]: pgmap v426: 97 pgs: 97 active+clean; 84 MiB data, 718 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 210 op/s 2024-09-12T03:01:17.796 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:01:17.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:01:17 smithi005 ceph-mon[31197]: pgmap v426: 97 pgs: 97 active+clean; 84 MiB data, 718 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 210 op/s 2024-09-12T03:01:17.915 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:01:17 smithi148 ceph-mon[35090]: pgmap v426: 97 pgs: 97 active+clean; 84 MiB data, 718 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 210 op/s 2024-09-12T03:01:18.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.280+0000 7eff6d81a640 1 -- 172.21.15.5:0/1458019723 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7eff68102ef0 msgr2=0x7eff6810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:18.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.280+0000 7eff6d81a640 1 --2- 172.21.15.5:0/1458019723 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7eff68102ef0 0x7eff6810da40 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7eff60009ca0 tx=0x7eff6002a950 comp rx=0 tx=0).stop 2024-09-12T03:01:18.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.280+0000 7eff6d81a640 1 -- 172.21.15.5:0/1458019723 shutdown_connections 2024-09-12T03:01:18.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.280+0000 7eff6d81a640 1 --2- 172.21.15.5:0/1458019723 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7eff68102ef0 0x7eff6810da40 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:18.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.280+0000 7eff6d81a640 1 --2- 172.21.15.5:0/1458019723 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff68102570 0x7eff681029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:18.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.280+0000 7eff6d81a640 1 --2- 172.21.15.5:0/1458019723 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7eff68100e20 0x7eff68101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:18.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.280+0000 7eff6d81a640 1 -- 172.21.15.5:0/1458019723 >> 172.21.15.5:0/1458019723 conn(0x7eff680fc7d0 msgr2=0x7eff680febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:18.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.280+0000 7eff6d81a640 1 -- 172.21.15.5:0/1458019723 shutdown_connections 2024-09-12T03:01:18.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.280+0000 7eff6d81a640 1 -- 172.21.15.5:0/1458019723 wait complete. 2024-09-12T03:01:18.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.281+0000 7eff6d81a640 1 Processor -- start 2024-09-12T03:01:18.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.281+0000 7eff6d81a640 1 -- start start 2024-09-12T03:01:18.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.281+0000 7eff6d81a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff68100e20 0x7eff68198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:18.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.281+0000 7eff6d81a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7eff68102570 0x7eff68198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:18.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.281+0000 7eff6d81a640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7eff68102ef0 0x7eff6819cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:18.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.281+0000 7eff6d81a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7eff68110f80 con 0x7eff68102570 2024-09-12T03:01:18.283 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.281+0000 7eff6d81a640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7eff68110e00 con 0x7eff68102ef0 2024-09-12T03:01:18.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.281+0000 7eff6d81a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7eff68111100 con 0x7eff68100e20 2024-09-12T03:01:18.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.281+0000 7eff667fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7eff68102570 0x7eff68198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:18.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.281+0000 7eff66ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff68100e20 0x7eff68198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:18.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.281+0000 7eff667fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7eff68102570 0x7eff68198a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41090/0 (socket says 172.21.15.5:41090) 2024-09-12T03:01:18.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.281+0000 7eff667fc640 1 -- 172.21.15.5:0/1639990752 learned_addr learned my addr 172.21.15.5:0/1639990752 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:01:18.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.282+0000 7eff677fe640 1 --2- 172.21.15.5:0/1639990752 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7eff68102ef0 0x7eff6819cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:18.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.282+0000 7eff66ffd640 1 -- 172.21.15.5:0/1639990752 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7eff68102ef0 msgr2=0x7eff6819cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:18.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.282+0000 7eff66ffd640 1 --2- 172.21.15.5:0/1639990752 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7eff68102ef0 0x7eff6819cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:18.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.282+0000 7eff66ffd640 1 -- 172.21.15.5:0/1639990752 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7eff68102570 msgr2=0x7eff68198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:18.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.282+0000 7eff66ffd640 1 --2- 172.21.15.5:0/1639990752 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7eff68102570 0x7eff68198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:18.284 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.282+0000 7eff66ffd640 1 -- 172.21.15.5:0/1639990752 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff6819d4b0 con 0x7eff68100e20 2024-09-12T03:01:18.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.282+0000 7eff667fc640 1 --2- 172.21.15.5:0/1639990752 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7eff68102570 0x7eff68198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:01:18.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.282+0000 7eff66ffd640 1 --2- 172.21.15.5:0/1639990752 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff68100e20 0x7eff68198500 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7eff5800e9f0 tx=0x7eff5800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:18.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.282+0000 7eff6c818640 1 -- 172.21.15.5:0/1639990752 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff5800ccc0 con 0x7eff68100e20 2024-09-12T03:01:18.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.282+0000 7eff6d81a640 1 -- 172.21.15.5:0/1639990752 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff6819d7a0 con 0x7eff68100e20 2024-09-12T03:01:18.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.282+0000 7eff6d81a640 1 -- 172.21.15.5:0/1639990752 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff681a5080 con 0x7eff68100e20 2024-09-12T03:01:18.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.282+0000 7eff6c818640 1 -- 172.21.15.5:0/1639990752 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff58010910 con 0x7eff68100e20 2024-09-12T03:01:18.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.282+0000 7eff6c818640 1 -- 172.21.15.5:0/1639990752 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff58018960 con 0x7eff68100e20 2024-09-12T03:01:18.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.285+0000 7eff6c818640 1 -- 172.21.15.5:0/1639990752 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7eff58010ac0 con 0x7eff68100e20 2024-09-12T03:01:18.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.285+0000 7eff6fbf3740 1 Processor -- start 2024-09-12T03:01:18.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.285+0000 7eff6fbf3740 1 -- start start 2024-09-12T03:01:18.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.286+0000 7eff6fbf3740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dfd7e04b40 0x55dfd7e03090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:18.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.286+0000 7eff6fbf3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfd7e035d0 0x55dfd7e03a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:18.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.286+0000 7eff6fbf3740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dfd7e0e370 0x55dfd7e10760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:18.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.286+0000 7eff6fbf3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55dfd7d692f0 con 0x55dfd7e035d0 2024-09-12T03:01:18.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.286+0000 7eff6fbf3740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55dfd7d10cf0 con 0x55dfd7e0e370 2024-09-12T03:01:18.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.286+0000 7eff6fbf3740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55dfd7d652b0 con 0x55dfd7e04b40 2024-09-12T03:01:18.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.286+0000 7eff6c818640 1 --2- 172.21.15.5:0/1639990752 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7eff500a2b80 0x7eff500a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:18.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.286+0000 7eff66ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfd7e035d0 0x55dfd7e03a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:18.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.286+0000 7eff667fc640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dfd7e0e370 0x55dfd7e10760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:18.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.286+0000 7eff66ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfd7e035d0 0x55dfd7e03a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41092/0 (socket says 172.21.15.5:41092) 2024-09-12T03:01:18.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.286+0000 7eff66ffd640 1 -- 172.21.15.5:0/2893518095 learned_addr learned my addr 172.21.15.5:0/2893518095 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:01:18.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.286+0000 7eff677fe640 1 --2- 172.21.15.5:0/2893518095 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dfd7e04b40 0x55dfd7e03090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:18.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.286+0000 7eff677fe640 1 --2- 172.21.15.5:0/1639990752 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7eff500a2b80 0x7eff500a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:18.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.286+0000 7eff667fc640 1 -- 172.21.15.5:0/2893518095 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dfd7e04b40 msgr2=0x55dfd7e03090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:01:18.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.286+0000 7eff667fc640 1 --2- 172.21.15.5:0/2893518095 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dfd7e04b40 0x55dfd7e03090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:18.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.286+0000 7eff6c818640 1 -- 172.21.15.5:0/1639990752 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7eff580cf2b0 con 0x7eff68100e20 2024-09-12T03:01:18.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.286+0000 7eff667fc640 1 -- 172.21.15.5:0/2893518095 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfd7e035d0 msgr2=0x55dfd7e03a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:01:18.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.286+0000 7eff667fc640 1 --2- 172.21.15.5:0/2893518095 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfd7e035d0 0x55dfd7e03a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:18.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.286+0000 7eff667fc640 1 -- 172.21.15.5:0/2893518095 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dfd7cf53a0 con 0x55dfd7e0e370 2024-09-12T03:01:18.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.287+0000 7eff667fc640 1 --2- 172.21.15.5:0/2893518095 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dfd7e0e370 0x55dfd7e10760 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7eff68105210 tx=0x7eff5c0119a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ace016dda47cb30b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:18.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.287+0000 7eff677fe640 1 --2- 172.21.15.5:0/1639990752 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7eff500a2b80 0x7eff500a5040 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7eff60009430 tx=0x7eff60005c50 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:18.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.287+0000 7eff4dffb640 1 -- 172.21.15.5:0/2893518095 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff5c01e070 con 0x55dfd7e0e370 2024-09-12T03:01:18.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.287+0000 7eff4dffb640 1 -- 172.21.15.5:0/2893518095 <== mon.2 v2:172.21.15.146:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff5c00faa0 con 0x55dfd7e0e370 2024-09-12T03:01:18.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.287+0000 7eff4dffb640 1 -- 172.21.15.5:0/2893518095 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff5c019c30 con 0x55dfd7e0e370 2024-09-12T03:01:18.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.288+0000 7eff6fbf3740 1 -- 172.21.15.5:0/2893518095 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dfd7e0e370 msgr2=0x55dfd7e10760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:18.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.288+0000 7eff6fbf3740 1 --2- 172.21.15.5:0/2893518095 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dfd7e0e370 0x55dfd7e10760 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7eff68105210 tx=0x7eff5c0119a0 comp rx=0 tx=0).stop 2024-09-12T03:01:18.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.288+0000 7eff6fbf3740 1 -- 172.21.15.5:0/2893518095 shutdown_connections 2024-09-12T03:01:18.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.288+0000 7eff6fbf3740 1 --2- 172.21.15.5:0/2893518095 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfd7e035d0 0x55dfd7e03a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:18.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.288+0000 7eff6fbf3740 1 --2- 172.21.15.5:0/2893518095 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dfd7e0e370 0x55dfd7e10760 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:18.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.288+0000 7eff6fbf3740 1 --2- 172.21.15.5:0/2893518095 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dfd7e04b40 0x55dfd7e03090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:18.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.288+0000 7eff6fbf3740 1 -- 172.21.15.5:0/2893518095 >> 172.21.15.5:0/2893518095 conn(0x55dfd7dcc310 msgr2=0x55dfd7dffc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:18.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.288+0000 7eff6fbf3740 1 -- 172.21.15.5:0/2893518095 shutdown_connections 2024-09-12T03:01:18.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.288+0000 7eff6fbf3740 1 -- 172.21.15.5:0/2893518095 wait complete. 2024-09-12T03:01:18.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.288+0000 7eff6fbf3740 1 -- start start 2024-09-12T03:01:18.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.288+0000 7eff6fbf3740 1 Processor -- start 2024-09-12T03:01:18.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.289+0000 7eff6fbf3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfd7e035d0 0x55dfd7e072f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:18.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.289+0000 7eff6fbf3740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dfd7e04b40 0x55dfd7e07830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:18.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.289+0000 7eff6fbf3740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dfd7e0e370 0x55dfd7e07d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:18.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.289+0000 7eff6fbf3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55dfd7e10fb0 con 0x55dfd7e035d0 2024-09-12T03:01:18.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.289+0000 7eff6fbf3740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55dfd7d24060 con 0x55dfd7e0e370 2024-09-12T03:01:18.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.289+0000 7eff6fbf3740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55dfd7e11130 con 0x55dfd7e04b40 2024-09-12T03:01:18.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.289+0000 7eff6fbf3740 10 client.0 _ll_register_callbacks cb 0x55dfd7a778c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:01:18.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.289+0000 7eff66ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfd7e035d0 0x55dfd7e072f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:18.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.289+0000 7eff667fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dfd7e04b40 0x55dfd7e07830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:18.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.289+0000 7eff66ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfd7e035d0 0x55dfd7e072f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41098/0 (socket says 172.21.15.5:41098) 2024-09-12T03:01:18.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.289+0000 7eff667fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dfd7e04b40 0x55dfd7e07830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.5:50390/0 (socket says 172.21.15.5:50390) 2024-09-12T03:01:18.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.289+0000 7eff66ffd640 1 -- 172.21.15.5:0/2015248313 learned_addr learned my addr 172.21.15.5:0/2015248313 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:01:18.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.289+0000 7eff677fe640 1 --2- 172.21.15.5:0/2015248313 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dfd7e0e370 0x55dfd7e07d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:18.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.289+0000 7eff667fc640 1 -- 172.21.15.5:0/2015248313 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dfd7e0e370 msgr2=0x55dfd7e07d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:01:18.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.289+0000 7eff667fc640 1 --2- 172.21.15.5:0/2015248313 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dfd7e0e370 0x55dfd7e07d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:18.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.289+0000 7eff667fc640 1 -- 172.21.15.5:0/2015248313 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfd7e035d0 msgr2=0x55dfd7e072f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:01:18.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.289+0000 7eff667fc640 1 --2- 172.21.15.5:0/2015248313 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfd7e035d0 0x55dfd7e072f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:18.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.289+0000 7eff667fc640 1 -- 172.21.15.5:0/2015248313 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dfd7ada3c0 con 0x55dfd7e04b40 2024-09-12T03:01:18.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.289+0000 7eff667fc640 1 --2- 172.21.15.5:0/2015248313 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dfd7e04b40 0x55dfd7e07830 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7eff5c01c040 tx=0x7eff5c00f7f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bbb12e24ebd71369 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:18.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.290+0000 7eff46ffd640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-09-12T03:01:18.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.290+0000 7eff46ffd640 1 -- 172.21.15.5:0/2015248313 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff5c01e040 con 0x55dfd7e04b40 2024-09-12T03:01:18.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.290+0000 7eff46ffd640 1 -- 172.21.15.5:0/2015248313 <== mon.1 v2:172.21.15.148:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7eff5c019970 con 0x55dfd7e04b40 2024-09-12T03:01:18.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.290+0000 7eff6fbf3740 1 -- 172.21.15.5:0/2015248313 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff5c00faa0 con 0x55dfd7e04b40 2024-09-12T03:01:18.295 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.290+0000 7eff46ffd640 1 -- 172.21.15.5:0/2015248313 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7eff5c009cf0 con 0x55dfd7e04b40 2024-09-12T03:01:18.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.290+0000 7eff6fbf3740 10 client.25624 fetch_fsmap learned FSMap version 28 2024-09-12T03:01:18.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.290+0000 7eff6fbf3740 1 -- 172.21.15.5:0/2015248313 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55dfd7e1cb00 con 0x55dfd7e04b40 2024-09-12T03:01:18.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.290+0000 7eff46ffd640 1 -- 172.21.15.5:0/2015248313 <== mon.1 v2:172.21.15.148:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff5c00faa0 con 0x55dfd7e04b40 2024-09-12T03:01:18.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.290+0000 7eff46ffd640 1 -- 172.21.15.5:0/2015248313 <== mon.1 v2:172.21.15.148:3300/0 5 ==== fsmap(e 28) ==== 8374+0+0 (secure 0 0 0) 0x7eff5c0046c0 con 0x55dfd7e04b40 2024-09-12T03:01:18.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.290+0000 7eff6fbf3740 10 client.25624 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T03:01:18.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.290+0000 7eff6fbf3740 10 client.25624 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-09-12T03:01:18.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.290+0000 7eff6fbf3740 20 client.25624 populate_metadata read hostname 'smithi005' 2024-09-12T03:01:18.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.290+0000 7eff6fbf3740 1 --2- 172.21.15.5:0/2015248313 >> [v2:172.21.15.148:6834/4195086427,v1:172.21.15.148:6835/4195086427] conn(0x55dfd7e1df00 0x55dfd7e202f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:18.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.290+0000 7eff6fbf3740 4 client.25624 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:01:18.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.290+0000 7eff6fbf3740 1 -- 172.21.15.5:0/2015248313 --> [v2:172.21.15.148:6834/4195086427,v1:172.21.15.148:6835/4195086427] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dfd7d76860 con 0x55dfd7e1df00 2024-09-12T03:01:18.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.291+0000 7eff66ffd640 1 --2- 172.21.15.5:0/2015248313 >> [v2:172.21.15.148:6834/4195086427,v1:172.21.15.148:6835/4195086427] conn(0x55dfd7e1df00 0x55dfd7e202f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:18.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.291+0000 7eff66ffd640 1 --2- 172.21.15.5:0/2015248313 >> [v2:172.21.15.148:6834/4195086427,v1:172.21.15.148:6835/4195086427] conn(0x55dfd7e1df00 0x55dfd7e202f0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c70f4d135ee05d2a server_cookie=2dc593595584169f in_seq=0 out_seq=0 2024-09-12T03:01:18.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.291+0000 7eff46ffd640 10 client.25624 ms_handle_connect on v2:172.21.15.148:6834/4195086427 2024-09-12T03:01:18.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.292+0000 7eff46ffd640 1 -- 172.21.15.5:0/2015248313 <== mds.2 v2:172.21.15.148:6834/4195086427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55dfd7d76860 con 0x55dfd7e1df00 2024-09-12T03:01:18.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.292+0000 7eff46ffd640 10 client.25624 handle_command_reply: tid=0 2024-09-12T03:01:18.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.292+0000 7eff6fbf3740 1 client.25624 shutdown 2024-09-12T03:01:18.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.292+0000 7eff6fbf3740 1 -- 172.21.15.5:0/2015248313 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dfd7e04b40 msgr2=0x55dfd7e07830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:18.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.292+0000 7eff6fbf3740 1 --2- 172.21.15.5:0/2015248313 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dfd7e04b40 0x55dfd7e07830 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7eff5c01c040 tx=0x7eff5c00f7f0 comp rx=0 tx=0).stop 2024-09-12T03:01:18.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.292+0000 7eff6fbf3740 1 -- 172.21.15.5:0/2015248313 shutdown_connections 2024-09-12T03:01:18.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.292+0000 7eff6fbf3740 1 --2- 172.21.15.5:0/2015248313 >> [v2:172.21.15.148:6834/4195086427,v1:172.21.15.148:6835/4195086427] conn(0x55dfd7e1df00 0x55dfd7e202f0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:18.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.292+0000 7eff6fbf3740 1 --2- 172.21.15.5:0/2015248313 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dfd7e0e370 0x55dfd7e07d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:18.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.292+0000 7eff6fbf3740 1 --2- 172.21.15.5:0/2015248313 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dfd7e04b40 0x55dfd7e07830 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:18.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.292+0000 7eff6fbf3740 1 --2- 172.21.15.5:0/2015248313 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfd7e035d0 0x55dfd7e072f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:18.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.292+0000 7eff46ffd640 0 client.25624 ms_handle_reset on v2:172.21.15.148:6834/4195086427 2024-09-12T03:01:18.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.292+0000 7eff6fbf3740 1 -- 172.21.15.5:0/2015248313 >> 172.21.15.5:0/2015248313 conn(0x55dfd7dcc310 msgr2=0x55dfd7e05930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:18.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.292+0000 7eff6fbf3740 1 -- 172.21.15.5:0/2015248313 shutdown_connections 2024-09-12T03:01:18.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.292+0000 7eff6fbf3740 1 -- 172.21.15.5:0/2015248313 wait complete. 2024-09-12T03:01:18.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.292+0000 7eff6fbf3740 20 client.25624 trim_cache size 0 max 16384 2024-09-12T03:01:18.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.306+0000 7eff6fbf3740 1 Processor -- start 2024-09-12T03:01:18.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.306+0000 7eff6fbf3740 1 -- start start 2024-09-12T03:01:18.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.306+0000 7eff6fbf3740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dfd7e035d0 0x55dfd7e07bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:18.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.306+0000 7eff6fbf3740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dfd7e04b40 0x55dfd7e0b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:18.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.306+0000 7eff6fbf3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfd7e0e370 0x55dfd7e25b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:18.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.306+0000 7eff6fbf3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7eff5400e400 con 0x55dfd7e0e370 2024-09-12T03:01:18.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.306+0000 7eff6fbf3740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7eff5400ffd0 con 0x55dfd7e035d0 2024-09-12T03:01:18.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.306+0000 7eff6fbf3740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7eff5400ead0 con 0x55dfd7e04b40 2024-09-12T03:01:18.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.306+0000 7eff66ffd640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dfd7e035d0 0x55dfd7e07bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:18.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.306+0000 7eff667fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dfd7e04b40 0x55dfd7e0b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:18.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.306+0000 7eff66ffd640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dfd7e035d0 0x55dfd7e07bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.5:47108/0 (socket says 172.21.15.5:47108) 2024-09-12T03:01:18.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.306+0000 7eff667fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dfd7e04b40 0x55dfd7e0b220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.5:50400/0 (socket says 172.21.15.5:50400) 2024-09-12T03:01:18.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.306+0000 7eff677fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfd7e0e370 0x55dfd7e25b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:18.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.306+0000 7eff66ffd640 1 -- 172.21.15.5:0/577431858 learned_addr learned my addr 172.21.15.5:0/577431858 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:01:18.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.307+0000 7eff66ffd640 1 -- 172.21.15.5:0/577431858 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dfd7e04b40 msgr2=0x55dfd7e0b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:01:18.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.307+0000 7eff66ffd640 1 --2- 172.21.15.5:0/577431858 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dfd7e04b40 0x55dfd7e0b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:18.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.307+0000 7eff66ffd640 1 -- 172.21.15.5:0/577431858 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfd7e0e370 msgr2=0x55dfd7e25b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:01:18.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.307+0000 7eff66ffd640 1 --2- 172.21.15.5:0/577431858 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfd7e0e370 0x55dfd7e25b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:18.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.307+0000 7eff66ffd640 1 -- 172.21.15.5:0/577431858 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dfd7e14660 con 0x55dfd7e035d0 2024-09-12T03:01:18.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.307+0000 7eff667fc640 1 --2- 172.21.15.5:0/577431858 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dfd7e04b40 0x55dfd7e0b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:01:18.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.307+0000 7eff66ffd640 1 --2- 172.21.15.5:0/577431858 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dfd7e035d0 0x55dfd7e07bc0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7eff580cce10 tx=0x7eff58097eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=65c0ac8f0200bef4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:18.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.307+0000 7eff4d7fa640 1 -- 172.21.15.5:0/577431858 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff5809de60 con 0x55dfd7e035d0 2024-09-12T03:01:18.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.307+0000 7eff4d7fa640 1 -- 172.21.15.5:0/577431858 <== mon.2 v2:172.21.15.146:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff58010db0 con 0x55dfd7e035d0 2024-09-12T03:01:18.309 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.307+0000 7eff4d7fa640 1 -- 172.21.15.5:0/577431858 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff580a4790 con 0x55dfd7e035d0 2024-09-12T03:01:18.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.308+0000 7eff6fbf3740 1 -- 172.21.15.5:0/577431858 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dfd7e035d0 msgr2=0x55dfd7e07bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:18.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.308+0000 7eff6fbf3740 1 --2- 172.21.15.5:0/577431858 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dfd7e035d0 0x55dfd7e07bc0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7eff580cce10 tx=0x7eff58097eb0 comp rx=0 tx=0).stop 2024-09-12T03:01:18.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.308+0000 7eff6fbf3740 1 -- 172.21.15.5:0/577431858 shutdown_connections 2024-09-12T03:01:18.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.308+0000 7eff6fbf3740 1 --2- 172.21.15.5:0/577431858 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfd7e0e370 0x55dfd7e25b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:18.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.308+0000 7eff6fbf3740 1 --2- 172.21.15.5:0/577431858 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dfd7e04b40 0x55dfd7e0b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:18.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.308+0000 7eff6fbf3740 1 --2- 172.21.15.5:0/577431858 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dfd7e035d0 0x55dfd7e07bc0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:18.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.308+0000 7eff6fbf3740 1 -- 172.21.15.5:0/577431858 >> 172.21.15.5:0/577431858 conn(0x55dfd7dcc310 msgr2=0x55dfd7e0ab10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:18.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.308+0000 7eff6fbf3740 1 -- 172.21.15.5:0/577431858 shutdown_connections 2024-09-12T03:01:18.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.308+0000 7eff6fbf3740 1 -- 172.21.15.5:0/577431858 wait complete. 2024-09-12T03:01:18.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.308+0000 7eff6fbf3740 1 -- start start 2024-09-12T03:01:18.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.309+0000 7eff6fbf3740 1 Processor -- start 2024-09-12T03:01:18.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.309+0000 7eff6fbf3740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dfd7e035d0 0x55dfd7e1a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:18.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.309+0000 7eff6fbf3740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dfd7e04b40 0x55dfd7e1aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:18.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.309+0000 7eff6fbf3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfd7e0e370 0x55dfd7e12960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:18.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.309+0000 7eff6fbf3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7eff5400f870 con 0x55dfd7e0e370 2024-09-12T03:01:18.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.309+0000 7eff6fbf3740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7eff5400f1a0 con 0x55dfd7e04b40 2024-09-12T03:01:18.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.309+0000 7eff6fbf3740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7eff5400c3e0 con 0x55dfd7e035d0 2024-09-12T03:01:18.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.309+0000 7eff6fbf3740 10 client.0 _ll_register_callbacks cb 0x7eff5c01c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:01:18.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.309+0000 7eff667fc640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dfd7e04b40 0x55dfd7e1aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:18.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.309+0000 7eff677fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfd7e0e370 0x55dfd7e12960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:18.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.309+0000 7eff66ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dfd7e035d0 0x55dfd7e1a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:18.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.309+0000 7eff677fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfd7e0e370 0x55dfd7e12960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41120/0 (socket says 172.21.15.5:41120) 2024-09-12T03:01:18.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.309+0000 7eff677fe640 1 -- 172.21.15.5:0/1713924170 learned_addr learned my addr 172.21.15.5:0/1713924170 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:01:18.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.310+0000 7eff677fe640 1 -- 172.21.15.5:0/1713924170 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dfd7e035d0 msgr2=0x55dfd7e1a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:01:18.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.310+0000 7eff677fe640 1 --2- 172.21.15.5:0/1713924170 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dfd7e035d0 0x55dfd7e1a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:18.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.310+0000 7eff677fe640 1 -- 172.21.15.5:0/1713924170 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dfd7e04b40 msgr2=0x55dfd7e1aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:01:18.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.310+0000 7eff677fe640 1 --2- 172.21.15.5:0/1713924170 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dfd7e04b40 0x55dfd7e1aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:18.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.310+0000 7eff677fe640 1 -- 172.21.15.5:0/1713924170 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dfd7e13050 con 0x55dfd7e0e370 2024-09-12T03:01:18.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.310+0000 7eff66ffd640 1 --2- 172.21.15.5:0/1713924170 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dfd7e035d0 0x55dfd7e1a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:01:18.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.310+0000 7eff667fc640 1 --2- 172.21.15.5:0/1713924170 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dfd7e04b40 0x55dfd7e1aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:01:18.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.310+0000 7eff677fe640 1 --2- 172.21.15.5:0/1713924170 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfd7e0e370 0x55dfd7e12960 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7eff60006010 tx=0x7eff600095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40e1ad2e3cdbe28 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:18.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.310+0000 7eff46ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T03:01:18.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.310+0000 7eff46ffd640 1 -- 172.21.15.5:0/1713924170 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff60062060 con 0x55dfd7e0e370 2024-09-12T03:01:18.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.310+0000 7eff46ffd640 1 -- 172.21.15.5:0/1713924170 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7eff600043d0 con 0x55dfd7e0e370 2024-09-12T03:01:18.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.310+0000 7eff6fbf3740 1 -- 172.21.15.5:0/1713924170 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff58010db0 con 0x55dfd7e0e370 2024-09-12T03:01:18.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.311+0000 7eff46ffd640 1 -- 172.21.15.5:0/1713924170 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7eff60034040 con 0x55dfd7e0e370 2024-09-12T03:01:18.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.311+0000 7eff6fbf3740 10 client.16113 fetch_fsmap learned FSMap version 28 2024-09-12T03:01:18.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.311+0000 7eff6fbf3740 1 -- 172.21.15.5:0/1713924170 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55dfd7e88460 con 0x55dfd7e0e370 2024-09-12T03:01:18.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.311+0000 7eff46ffd640 1 -- 172.21.15.5:0/1713924170 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff58010db0 con 0x55dfd7e0e370 2024-09-12T03:01:18.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.311+0000 7eff46ffd640 1 -- 172.21.15.5:0/1713924170 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8374+0+0 (secure 0 0 0) 0x7eff60030440 con 0x55dfd7e0e370 2024-09-12T03:01:18.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.311+0000 7eff6fbf3740 10 client.16113 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T03:01:18.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.311+0000 7eff6fbf3740 10 client.16113 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-09-12T03:01:18.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.311+0000 7eff6fbf3740 20 client.16113 populate_metadata read hostname 'smithi005' 2024-09-12T03:01:18.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.311+0000 7eff6fbf3740 1 --2- 172.21.15.5:0/1713924170 >> [v2:172.21.15.148:6834/4195086427,v1:172.21.15.148:6835/4195086427] conn(0x55dfd7e88bb0 0x55dfd7e8afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:18.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.311+0000 7eff6fbf3740 4 client.16113 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:01:18.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.311+0000 7eff6fbf3740 1 -- 172.21.15.5:0/1713924170 --> [v2:172.21.15.148:6834/4195086427,v1:172.21.15.148:6835/4195086427] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dfd7d70d70 con 0x55dfd7e88bb0 2024-09-12T03:01:18.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.311+0000 7eff66ffd640 1 --2- 172.21.15.5:0/1713924170 >> [v2:172.21.15.148:6834/4195086427,v1:172.21.15.148:6835/4195086427] conn(0x55dfd7e88bb0 0x55dfd7e8afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:18.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.312+0000 7eff66ffd640 1 --2- 172.21.15.5:0/1713924170 >> [v2:172.21.15.148:6834/4195086427,v1:172.21.15.148:6835/4195086427] conn(0x55dfd7e88bb0 0x55dfd7e8afd0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1d2b515e19294064 server_cookie=115fc2fe0676aae5 in_seq=0 out_seq=0 2024-09-12T03:01:18.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.312+0000 7eff46ffd640 10 client.16113 ms_handle_connect on v2:172.21.15.148:6834/4195086427 2024-09-12T03:01:18.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.312+0000 7eff46ffd640 1 -- 172.21.15.5:0/1713924170 <== mds.2 v2:172.21.15.148:6834/4195086427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dfd7d70d70 con 0x55dfd7e88bb0 2024-09-12T03:01:18.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.312+0000 7eff46ffd640 10 client.16113 handle_command_reply: tid=0 2024-09-12T03:01:18.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.312+0000 7eff6fbf3740 1 client.16113 shutdown 2024-09-12T03:01:18.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.312+0000 7eff6fbf3740 1 -- 172.21.15.5:0/1713924170 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfd7e0e370 msgr2=0x55dfd7e12960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:18.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.312+0000 7eff6fbf3740 1 --2- 172.21.15.5:0/1713924170 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfd7e0e370 0x55dfd7e12960 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7eff60006010 tx=0x7eff600095c0 comp rx=0 tx=0).stop 2024-09-12T03:01:18.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.312+0000 7eff6fbf3740 1 -- 172.21.15.5:0/1713924170 shutdown_connections 2024-09-12T03:01:18.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.312+0000 7eff6fbf3740 1 --2- 172.21.15.5:0/1713924170 >> [v2:172.21.15.148:6834/4195086427,v1:172.21.15.148:6835/4195086427] conn(0x55dfd7e88bb0 0x55dfd7e8afd0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:18.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.312+0000 7eff6fbf3740 1 --2- 172.21.15.5:0/1713924170 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dfd7e0e370 0x55dfd7e12960 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:18.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.312+0000 7eff6fbf3740 1 --2- 172.21.15.5:0/1713924170 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55dfd7e04b40 0x55dfd7e1aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:18.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.312+0000 7eff6fbf3740 1 --2- 172.21.15.5:0/1713924170 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55dfd7e035d0 0x55dfd7e1a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:18.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.312+0000 7eff6fbf3740 1 -- 172.21.15.5:0/1713924170 >> 172.21.15.5:0/1713924170 conn(0x55dfd7dcc310 msgr2=0x55dfd7e05930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:18.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.312+0000 7eff46ffd640 0 client.16113 ms_handle_reset on v2:172.21.15.148:6834/4195086427 2024-09-12T03:01:18.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.312+0000 7eff6fbf3740 1 -- 172.21.15.5:0/1713924170 shutdown_connections 2024-09-12T03:01:18.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.312+0000 7eff6fbf3740 1 -- 172.21.15.5:0/1713924170 wait complete. 2024-09-12T03:01:18.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.313+0000 7eff6fbf3740 20 client.16113 trim_cache size 0 max 16384 2024-09-12T03:01:18.319 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-12T03:01:18.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.313+0000 7eff6d81a640 1 -- 172.21.15.5:0/1639990752 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7eff500a2b80 msgr2=0x7eff500a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:18.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.313+0000 7eff6d81a640 1 --2- 172.21.15.5:0/1639990752 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7eff500a2b80 0x7eff500a5040 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7eff60009430 tx=0x7eff60005c50 comp rx=0 tx=0).stop 2024-09-12T03:01:18.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.313+0000 7eff6d81a640 1 -- 172.21.15.5:0/1639990752 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff68100e20 msgr2=0x7eff68198500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:18.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.313+0000 7eff6d81a640 1 --2- 172.21.15.5:0/1639990752 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff68100e20 0x7eff68198500 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7eff5800e9f0 tx=0x7eff5800eeb0 comp rx=0 tx=0).stop 2024-09-12T03:01:18.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.313+0000 7eff6d81a640 1 -- 172.21.15.5:0/1639990752 shutdown_connections 2024-09-12T03:01:18.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.313+0000 7eff6d81a640 1 --2- 172.21.15.5:0/1639990752 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7eff500a2b80 0x7eff500a5040 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:18.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.313+0000 7eff6d81a640 1 --2- 172.21.15.5:0/1639990752 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7eff68102570 0x7eff68198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:18.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.313+0000 7eff6d81a640 1 --2- 172.21.15.5:0/1639990752 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7eff68102ef0 0x7eff6819cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:18.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.313+0000 7eff6d81a640 1 --2- 172.21.15.5:0/1639990752 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7eff68100e20 0x7eff68198500 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:18.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.313+0000 7eff6d81a640 1 -- 172.21.15.5:0/1639990752 >> 172.21.15.5:0/1639990752 conn(0x7eff680fc7d0 msgr2=0x7eff6806a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:18.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.313+0000 7eff6d81a640 1 -- 172.21.15.5:0/1639990752 shutdown_connections 2024-09-12T03:01:18.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:18.313+0000 7eff6d81a640 1 -- 172.21.15.5:0/1639990752 wait complete. 2024-09-12T03:01:19.115 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T03:01:19.389 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:01:19.695 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:01:19 smithi005 ceph-mon[31197]: pgmap v427: 97 pgs: 97 active+clean; 87 MiB data, 754 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 310 op/s 2024-09-12T03:01:19.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:01:19 smithi146 ceph-mon[35011]: pgmap v427: 97 pgs: 97 active+clean; 87 MiB data, 754 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 310 op/s 2024-09-12T03:01:19.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.895+0000 7f47762dd640 1 -- 172.21.15.5:0/3534638184 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f477010e470 msgr2=0x7f47701108a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:19.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.895+0000 7f47762dd640 1 --2- 172.21.15.5:0/3534638184 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f477010e470 0x7f47701108a0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f4768009ca0 tx=0x7f476802a8c0 comp rx=0 tx=0).stop 2024-09-12T03:01:19.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.895+0000 7f47762dd640 1 -- 172.21.15.5:0/3534638184 shutdown_connections 2024-09-12T03:01:19.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.895+0000 7f47762dd640 1 --2- 172.21.15.5:0/3534638184 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f477010e470 0x7f47701108a0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:19.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.895+0000 7f47762dd640 1 --2- 172.21.15.5:0/3534638184 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4770074f50 0x7f477010df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:19.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.895+0000 7f47762dd640 1 --2- 172.21.15.5:0/3534638184 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f47700745a0 0x7f4770074980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:19.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.895+0000 7f47762dd640 1 -- 172.21.15.5:0/3534638184 >> 172.21.15.5:0/3534638184 conn(0x7f47700fd8d0 msgr2=0x7f47700ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:19.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.895+0000 7f47762dd640 1 -- 172.21.15.5:0/3534638184 shutdown_connections 2024-09-12T03:01:19.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.895+0000 7f47762dd640 1 -- 172.21.15.5:0/3534638184 wait complete. 2024-09-12T03:01:19.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.896+0000 7f47762dd640 1 Processor -- start 2024-09-12T03:01:19.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.896+0000 7f47762dd640 1 -- start start 2024-09-12T03:01:19.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.896+0000 7f47762dd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47700745a0 0x7f477019a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:19.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.896+0000 7f47762dd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4770074f50 0x7f477019ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:19.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.896+0000 7f47762dd640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f477010e470 0x7f477019ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:19.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.896+0000 7f47762dd640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f47701141e0 con 0x7f47700745a0 2024-09-12T03:01:19.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.896+0000 7f47762dd640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f4770114060 con 0x7f477010e470 2024-09-12T03:01:19.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.896+0000 7f47762dd640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4770114360 con 0x7f4770074f50 2024-09-12T03:01:19.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.897+0000 7f476ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47700745a0 0x7f477019a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:19.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.897+0000 7f476ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47700745a0 0x7f477019a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41136/0 (socket says 172.21.15.5:41136) 2024-09-12T03:01:19.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.897+0000 7f476f7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4770074f50 0x7f477019ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:19.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.897+0000 7f476ffff640 1 -- 172.21.15.5:0/3285880422 learned_addr learned my addr 172.21.15.5:0/3285880422 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:01:19.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.897+0000 7f477487a640 1 --2- 172.21.15.5:0/3285880422 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f477010e470 0x7f477019ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:19.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.897+0000 7f476ffff640 1 -- 172.21.15.5:0/3285880422 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4770074f50 msgr2=0x7f477019ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:19.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.897+0000 7f476ffff640 1 --2- 172.21.15.5:0/3285880422 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4770074f50 0x7f477019ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:19.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.897+0000 7f476ffff640 1 -- 172.21.15.5:0/3285880422 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f477010e470 msgr2=0x7f477019ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:19.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.897+0000 7f476ffff640 1 --2- 172.21.15.5:0/3285880422 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f477010e470 0x7f477019ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:19.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.897+0000 7f476ffff640 1 -- 172.21.15.5:0/3285880422 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f477019f530 con 0x7f47700745a0 2024-09-12T03:01:19.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.897+0000 7f476ffff640 1 --2- 172.21.15.5:0/3285880422 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47700745a0 0x7f477019a620 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f476000c920 tx=0x7f476000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:19.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.897+0000 7f476d7fa640 1 -- 172.21.15.5:0/3285880422 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4760007c70 con 0x7f47700745a0 2024-09-12T03:01:19.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.897+0000 7f476d7fa640 1 -- 172.21.15.5:0/3285880422 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4760007e10 con 0x7f47700745a0 2024-09-12T03:01:19.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.897+0000 7f476d7fa640 1 -- 172.21.15.5:0/3285880422 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4760018890 con 0x7f47700745a0 2024-09-12T03:01:19.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.897+0000 7f47762dd640 1 -- 172.21.15.5:0/3285880422 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f477019f820 con 0x7f47700745a0 2024-09-12T03:01:19.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.897+0000 7f47762dd640 1 -- 172.21.15.5:0/3285880422 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47701a7100 con 0x7f47700745a0 2024-09-12T03:01:19.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.900+0000 7f476d7fa640 1 -- 172.21.15.5:0/3285880422 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f4760018a70 con 0x7f47700745a0 2024-09-12T03:01:19.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.900+0000 7f47786b6740 1 Processor -- start 2024-09-12T03:01:19.906 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.900+0000 7f47786b6740 1 -- start start 2024-09-12T03:01:19.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.900+0000 7f47786b6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562da55aab40 0x562da55a9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:19.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.900+0000 7f47786b6740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562da55a95d0 0x562da55a9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:19.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.900+0000 7f47786b6740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562da55b4370 0x562da55b6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:19.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.901+0000 7f47786b6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562da550f2f0 con 0x562da55aab40 2024-09-12T03:01:19.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.901+0000 7f47786b6740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x562da54b6cf0 con 0x562da55b4370 2024-09-12T03:01:19.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.901+0000 7f47786b6740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x562da550b2b0 con 0x562da55a95d0 2024-09-12T03:01:19.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.901+0000 7f476ffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562da55a95d0 0x562da55a9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:19.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.901+0000 7f477487a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562da55aab40 0x562da55a9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:19.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.901+0000 7f477487a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562da55aab40 0x562da55a9090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41142/0 (socket says 172.21.15.5:41142) 2024-09-12T03:01:19.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.901+0000 7f477487a640 1 -- 172.21.15.5:0/1725105903 learned_addr learned my addr 172.21.15.5:0/1725105903 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:01:19.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.901+0000 7f476f7fe640 1 --2- 172.21.15.5:0/1725105903 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562da55b4370 0x562da55b6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:19.907 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.901+0000 7f477487a640 1 -- 172.21.15.5:0/1725105903 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562da55a95d0 msgr2=0x562da55a9a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:01:19.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.901+0000 7f477487a640 1 --2- 172.21.15.5:0/1725105903 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562da55a95d0 0x562da55a9a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:19.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.901+0000 7f477487a640 1 -- 172.21.15.5:0/1725105903 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562da55b4370 msgr2=0x562da55b6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:01:19.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.901+0000 7f477487a640 1 --2- 172.21.15.5:0/1725105903 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562da55b4370 0x562da55b6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:19.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.901+0000 7f477487a640 1 -- 172.21.15.5:0/1725105903 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562da549b3a0 con 0x562da55aab40 2024-09-12T03:01:19.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.901+0000 7f476d7fa640 1 --2- 172.21.15.5:0/3285880422 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f47580a2b80 0x7f47580a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:19.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.901+0000 7f476ffff640 1 --2- 172.21.15.5:0/3285880422 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f47580a2b80 0x7f47580a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:19.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.901+0000 7f476d7fa640 1 -- 172.21.15.5:0/3285880422 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f47600cf0d0 con 0x7f47700745a0 2024-09-12T03:01:19.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.902+0000 7f477487a640 1 --2- 172.21.15.5:0/1725105903 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562da55aab40 0x562da55a9090 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f4768009ca0 tx=0x7f47680098b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9514bd0034b8481c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:19.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.902+0000 7f47567fc640 1 -- 172.21.15.5:0/1725105903 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4768003970 con 0x562da55aab40 2024-09-12T03:01:19.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.902+0000 7f47567fc640 1 -- 172.21.15.5:0/1725105903 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4768036040 con 0x562da55aab40 2024-09-12T03:01:19.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.902+0000 7f47567fc640 1 -- 172.21.15.5:0/1725105903 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47680043e0 con 0x562da55aab40 2024-09-12T03:01:19.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.903+0000 7f47786b6740 1 -- 172.21.15.5:0/1725105903 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562da55aab40 msgr2=0x562da55a9090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:19.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.903+0000 7f476ffff640 1 --2- 172.21.15.5:0/3285880422 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f47580a2b80 0x7f47580a5040 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f47600ccf50 tx=0x7f47600d2bb0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:19.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.903+0000 7f47786b6740 1 --2- 172.21.15.5:0/1725105903 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562da55aab40 0x562da55a9090 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f4768009ca0 tx=0x7f47680098b0 comp rx=0 tx=0).stop 2024-09-12T03:01:19.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.903+0000 7f47786b6740 1 -- 172.21.15.5:0/1725105903 shutdown_connections 2024-09-12T03:01:19.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.903+0000 7f47786b6740 1 --2- 172.21.15.5:0/1725105903 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562da55b4370 0x562da55b6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:19.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.903+0000 7f47786b6740 1 --2- 172.21.15.5:0/1725105903 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562da55a95d0 0x562da55a9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:19.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.903+0000 7f47786b6740 1 --2- 172.21.15.5:0/1725105903 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562da55aab40 0x562da55a9090 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:19.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.903+0000 7f47786b6740 1 -- 172.21.15.5:0/1725105903 >> 172.21.15.5:0/1725105903 conn(0x562da5572310 msgr2=0x562da55a5c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:19.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.903+0000 7f47786b6740 1 -- 172.21.15.5:0/1725105903 shutdown_connections 2024-09-12T03:01:19.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.903+0000 7f47786b6740 1 -- 172.21.15.5:0/1725105903 wait complete. 2024-09-12T03:01:19.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.903+0000 7f47786b6740 1 -- start start 2024-09-12T03:01:19.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.903+0000 7f47786b6740 1 Processor -- start 2024-09-12T03:01:19.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.904+0000 7f47786b6740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562da55a95d0 0x562da55b3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:19.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.904+0000 7f47786b6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562da55aab40 0x562da55b0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:19.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.904+0000 7f47786b6740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562da55b4370 0x562da55b1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:19.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.904+0000 7f47786b6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562da55b6e90 con 0x562da55aab40 2024-09-12T03:01:19.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.904+0000 7f47786b6740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x562da54ca060 con 0x562da55b4370 2024-09-12T03:01:19.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.904+0000 7f47786b6740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x562da55b7010 con 0x562da55a95d0 2024-09-12T03:01:19.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.904+0000 7f47786b6740 10 client.0 _ll_register_callbacks cb 0x562da521d8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:01:19.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.904+0000 7f477487a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562da55a95d0 0x562da55b3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:19.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.904+0000 7f477487a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562da55a95d0 0x562da55b3cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.5:50444/0 (socket says 172.21.15.5:50444) 2024-09-12T03:01:19.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.904+0000 7f476f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562da55aab40 0x562da55b0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:19.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.904+0000 7f477487a640 1 -- 172.21.15.5:0/2460282026 learned_addr learned my addr 172.21.15.5:0/2460282026 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:01:19.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.904+0000 7f477487a640 1 --2- 172.21.15.5:0/2460282026 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562da55b4370 0x562da55b1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:19.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.904+0000 7f477487a640 1 -- 172.21.15.5:0/2460282026 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562da55b4370 msgr2=0x562da55b1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:01:19.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.904+0000 7f477487a640 1 --2- 172.21.15.5:0/2460282026 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562da55b4370 0x562da55b1520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:19.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.904+0000 7f477487a640 1 -- 172.21.15.5:0/2460282026 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562da55aab40 msgr2=0x562da55b0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:01:19.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.904+0000 7f477487a640 1 --2- 172.21.15.5:0/2460282026 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562da55aab40 0x562da55b0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:19.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.904+0000 7f477487a640 1 -- 172.21.15.5:0/2460282026 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562da52803c0 con 0x562da55a95d0 2024-09-12T03:01:19.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.904+0000 7f476f7fe640 1 --2- 172.21.15.5:0/2460282026 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562da55aab40 0x562da55b0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:01:19.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.905+0000 7f477487a640 1 --2- 172.21.15.5:0/2460282026 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562da55a95d0 0x562da55b3cf0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4768031990 tx=0x7f4768004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d00edbffbee2bc42 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:19.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.905+0000 7f474f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-09-12T03:01:19.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.905+0000 7f474f7fe640 1 -- 172.21.15.5:0/2460282026 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4768058060 con 0x562da55a95d0 2024-09-12T03:01:19.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.905+0000 7f47786b6740 1 -- 172.21.15.5:0/2460282026 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4768036040 con 0x562da55a95d0 2024-09-12T03:01:19.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.905+0000 7f474f7fe640 1 -- 172.21.15.5:0/2460282026 <== mon.1 v2:172.21.15.148:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f47680041a0 con 0x562da55a95d0 2024-09-12T03:01:19.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.905+0000 7f474f7fe640 1 -- 172.21.15.5:0/2460282026 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4768004810 con 0x562da55a95d0 2024-09-12T03:01:19.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.905+0000 7f47786b6740 10 client.25642 fetch_fsmap learned FSMap version 28 2024-09-12T03:01:19.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.905+0000 7f47786b6740 1 -- 172.21.15.5:0/2460282026 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562da55b6e90 con 0x562da55a95d0 2024-09-12T03:01:19.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.905+0000 7f474f7fe640 1 -- 172.21.15.5:0/2460282026 <== mon.1 v2:172.21.15.148:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4768036040 con 0x562da55a95d0 2024-09-12T03:01:19.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.906+0000 7f474f7fe640 1 -- 172.21.15.5:0/2460282026 <== mon.1 v2:172.21.15.148:3300/0 5 ==== fsmap(e 28) ==== 8374+0+0 (secure 0 0 0) 0x7f476804d030 con 0x562da55a95d0 2024-09-12T03:01:19.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.906+0000 7f47786b6740 10 client.25642 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T03:01:19.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.906+0000 7f47786b6740 10 client.25642 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-12T03:01:19.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.906+0000 7f47786b6740 20 client.25642 populate_metadata read hostname 'smithi005' 2024-09-12T03:01:19.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.906+0000 7f47786b6740 1 --2- 172.21.15.5:0/2460282026 >> [v2:172.21.15.148:6836/2606873922,v1:172.21.15.148:6837/2606873922] conn(0x562da55c3a40 0x562da55c5e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:19.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.906+0000 7f47786b6740 4 client.25642 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:01:19.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.906+0000 7f47786b6740 1 -- 172.21.15.5:0/2460282026 --> [v2:172.21.15.148:6836/2606873922,v1:172.21.15.148:6837/2606873922] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562da551c860 con 0x562da55c3a40 2024-09-12T03:01:19.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.906+0000 7f476f7fe640 1 --2- 172.21.15.5:0/2460282026 >> [v2:172.21.15.148:6836/2606873922,v1:172.21.15.148:6837/2606873922] conn(0x562da55c3a40 0x562da55c5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:19.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.906+0000 7f476f7fe640 1 --2- 172.21.15.5:0/2460282026 >> [v2:172.21.15.148:6836/2606873922,v1:172.21.15.148:6837/2606873922] conn(0x562da55c3a40 0x562da55c5e30 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=31d8243e0306ea59 server_cookie=35d7c489f9f2286b in_seq=0 out_seq=0 2024-09-12T03:01:19.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.906+0000 7f474f7fe640 10 client.25642 ms_handle_connect on v2:172.21.15.148:6836/2606873922 2024-09-12T03:01:19.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:01:19 smithi148 ceph-mon[35090]: pgmap v427: 97 pgs: 97 active+clean; 87 MiB data, 754 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 310 op/s 2024-09-12T03:01:19.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.907+0000 7f474f7fe640 1 -- 172.21.15.5:0/2460282026 <== mds.1 v2:172.21.15.148:6836/2606873922 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562da551c860 con 0x562da55c3a40 2024-09-12T03:01:19.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.907+0000 7f474f7fe640 10 client.25642 handle_command_reply: tid=0 2024-09-12T03:01:19.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.907+0000 7f47786b6740 1 client.25642 shutdown 2024-09-12T03:01:19.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.907+0000 7f47786b6740 1 -- 172.21.15.5:0/2460282026 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562da55a95d0 msgr2=0x562da55b3cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:19.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.907+0000 7f47786b6740 1 --2- 172.21.15.5:0/2460282026 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562da55a95d0 0x562da55b3cf0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4768031990 tx=0x7f4768004040 comp rx=0 tx=0).stop 2024-09-12T03:01:19.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.907+0000 7f47786b6740 1 -- 172.21.15.5:0/2460282026 shutdown_connections 2024-09-12T03:01:19.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.907+0000 7f47786b6740 1 --2- 172.21.15.5:0/2460282026 >> [v2:172.21.15.148:6836/2606873922,v1:172.21.15.148:6837/2606873922] conn(0x562da55c3a40 0x562da55c5e30 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:19.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.907+0000 7f47786b6740 1 --2- 172.21.15.5:0/2460282026 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562da55aab40 0x562da55b0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:19.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.907+0000 7f47786b6740 1 --2- 172.21.15.5:0/2460282026 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562da55b4370 0x562da55b1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:19.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.907+0000 7f47786b6740 1 --2- 172.21.15.5:0/2460282026 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562da55a95d0 0x562da55b3cf0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:19.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.907+0000 7f474f7fe640 0 client.25642 ms_handle_reset on v2:172.21.15.148:6836/2606873922 2024-09-12T03:01:19.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.907+0000 7f47786b6740 1 -- 172.21.15.5:0/2460282026 >> 172.21.15.5:0/2460282026 conn(0x562da5572310 msgr2=0x562da55ab960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:19.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.907+0000 7f47786b6740 1 -- 172.21.15.5:0/2460282026 shutdown_connections 2024-09-12T03:01:19.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.907+0000 7f47786b6740 1 -- 172.21.15.5:0/2460282026 wait complete. 2024-09-12T03:01:19.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.908+0000 7f47786b6740 20 client.25642 trim_cache size 0 max 16384 2024-09-12T03:01:19.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.921+0000 7f47786b6740 1 Processor -- start 2024-09-12T03:01:19.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.921+0000 7f47786b6740 1 -- start start 2024-09-12T03:01:19.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.922+0000 7f47786b6740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562da55a95d0 0x562da55c0130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:19.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.922+0000 7f47786b6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562da55aab40 0x562da55aff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:19.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.922+0000 7f47786b6740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562da55b4370 0x562da55cb640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:19.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.922+0000 7f47786b6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f47701af5e0 con 0x562da55aab40 2024-09-12T03:01:19.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.922+0000 7f47786b6740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f47701b11b0 con 0x562da55a95d0 2024-09-12T03:01:19.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.922+0000 7f47786b6740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f47701afcb0 con 0x562da55b4370 2024-09-12T03:01:19.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.922+0000 7f477487a640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562da55a95d0 0x562da55c0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:19.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.922+0000 7f476f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562da55aab40 0x562da55aff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:19.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.922+0000 7f477487a640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562da55a95d0 0x562da55c0130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.5:47158/0 (socket says 172.21.15.5:47158) 2024-09-12T03:01:19.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.922+0000 7f477487a640 1 -- 172.21.15.5:0/2255354469 learned_addr learned my addr 172.21.15.5:0/2255354469 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:01:19.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.922+0000 7f477487a640 1 --2- 172.21.15.5:0/2255354469 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562da55b4370 0x562da55cb640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:19.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.922+0000 7f477487a640 1 -- 172.21.15.5:0/2255354469 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562da55b4370 msgr2=0x562da55cb640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:01:19.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.922+0000 7f477487a640 1 --2- 172.21.15.5:0/2255354469 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562da55b4370 0x562da55cb640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:19.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.922+0000 7f477487a640 1 -- 172.21.15.5:0/2255354469 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562da55aab40 msgr2=0x562da55aff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:01:19.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.922+0000 7f477487a640 1 --2- 172.21.15.5:0/2255354469 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562da55aab40 0x562da55aff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:19.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.922+0000 7f477487a640 1 -- 172.21.15.5:0/2255354469 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562da55acb30 con 0x562da55a95d0 2024-09-12T03:01:19.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.922+0000 7f476f7fe640 1 --2- 172.21.15.5:0/2255354469 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562da55aab40 0x562da55aff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:01:19.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.922+0000 7f477487a640 1 --2- 172.21.15.5:0/2255354469 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562da55a95d0 0x562da55c0130 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f47680319c0 tx=0x7f476802a8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56e32d86ddf89dbf server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:19.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.923+0000 7f4755ffb640 1 -- 172.21.15.5:0/2255354469 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4768058040 con 0x562da55a95d0 2024-09-12T03:01:19.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.923+0000 7f4755ffb640 1 -- 172.21.15.5:0/2255354469 <== mon.2 v2:172.21.15.146:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4768036030 con 0x562da55a95d0 2024-09-12T03:01:19.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.923+0000 7f4755ffb640 1 -- 172.21.15.5:0/2255354469 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f476802f5d0 con 0x562da55a95d0 2024-09-12T03:01:19.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.923+0000 7f47786b6740 1 -- 172.21.15.5:0/2255354469 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562da55a95d0 msgr2=0x562da55c0130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:19.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.923+0000 7f47786b6740 1 --2- 172.21.15.5:0/2255354469 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562da55a95d0 0x562da55c0130 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f47680319c0 tx=0x7f476802a8c0 comp rx=0 tx=0).stop 2024-09-12T03:01:19.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.923+0000 7f47786b6740 1 -- 172.21.15.5:0/2255354469 shutdown_connections 2024-09-12T03:01:19.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.923+0000 7f47786b6740 1 --2- 172.21.15.5:0/2255354469 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562da55aab40 0x562da55aff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:19.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.923+0000 7f47786b6740 1 --2- 172.21.15.5:0/2255354469 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562da55b4370 0x562da55cb640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:19.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.923+0000 7f47786b6740 1 --2- 172.21.15.5:0/2255354469 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562da55a95d0 0x562da55c0130 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:19.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.923+0000 7f47786b6740 1 -- 172.21.15.5:0/2255354469 >> 172.21.15.5:0/2255354469 conn(0x562da5572310 msgr2=0x562da55af870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:19.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.923+0000 7f47786b6740 1 -- 172.21.15.5:0/2255354469 shutdown_connections 2024-09-12T03:01:19.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.923+0000 7f47786b6740 1 -- 172.21.15.5:0/2255354469 wait complete. 2024-09-12T03:01:19.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.924+0000 7f47786b6740 1 -- start start 2024-09-12T03:01:19.928 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.924+0000 7f47786b6740 1 Processor -- start 2024-09-12T03:01:19.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.924+0000 7f47786b6740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562da55a95d0 0x562da55bc9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:19.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.924+0000 7f47786b6740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562da55aab40 0x562da55b7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:19.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.924+0000 7f47786b6740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562da55b4370 0x562da55b8230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:19.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.924+0000 7f47786b6740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f47701b0a50 con 0x562da55aab40 2024-09-12T03:01:19.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.924+0000 7f47786b6740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f47701b0380 con 0x562da55b4370 2024-09-12T03:01:19.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.924+0000 7f47786b6740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f47701ad5c0 con 0x562da55a95d0 2024-09-12T03:01:19.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.924+0000 7f47786b6740 10 client.0 _ll_register_callbacks cb 0x7f47640097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:01:19.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.924+0000 7f476f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562da55aab40 0x562da55b7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:19.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.924+0000 7f476f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562da55aab40 0x562da55b7cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41164/0 (socket says 172.21.15.5:41164) 2024-09-12T03:01:19.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.925+0000 7f476f7fe640 1 -- 172.21.15.5:0/1983235135 learned_addr learned my addr 172.21.15.5:0/1983235135 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:01:19.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.924+0000 7f477487a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562da55a95d0 0x562da55bc9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:19.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.925+0000 7f477487a640 1 --2- 172.21.15.5:0/1983235135 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562da55b4370 0x562da55b8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:19.929 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.925+0000 7f477487a640 1 -- 172.21.15.5:0/1983235135 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562da55b4370 msgr2=0x562da55b8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:01:19.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.925+0000 7f477487a640 1 --2- 172.21.15.5:0/1983235135 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562da55b4370 0x562da55b8230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:19.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.925+0000 7f477487a640 1 -- 172.21.15.5:0/1983235135 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562da55aab40 msgr2=0x562da55b7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:01:19.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.925+0000 7f477487a640 1 --2- 172.21.15.5:0/1983235135 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562da55aab40 0x562da55b7cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:19.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.925+0000 7f477487a640 1 -- 172.21.15.5:0/1983235135 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562da55b8a30 con 0x562da55a95d0 2024-09-12T03:01:19.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.925+0000 7f476f7fe640 1 --2- 172.21.15.5:0/1983235135 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562da55aab40 0x562da55b7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:01:19.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.925+0000 7f477487a640 1 --2- 172.21.15.5:0/1983235135 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562da55a95d0 0x562da55bc9c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f476802f470 tx=0x7f4768004a80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ece1c7c98283a1dd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:19.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.925+0000 7f474f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-09-12T03:01:19.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.925+0000 7f474f7fe640 1 -- 172.21.15.5:0/1983235135 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4768058040 con 0x562da55a95d0 2024-09-12T03:01:19.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.925+0000 7f47786b6740 1 -- 172.21.15.5:0/1983235135 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4768036030 con 0x562da55a95d0 2024-09-12T03:01:19.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.925+0000 7f474f7fe640 1 -- 172.21.15.5:0/1983235135 <== mon.1 v2:172.21.15.148:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f476802fba0 con 0x562da55a95d0 2024-09-12T03:01:19.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.926+0000 7f474f7fe640 1 -- 172.21.15.5:0/1983235135 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f4768004450 con 0x562da55a95d0 2024-09-12T03:01:19.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.926+0000 7f47786b6740 10 client.25648 fetch_fsmap learned FSMap version 28 2024-09-12T03:01:19.930 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.926+0000 7f47786b6740 1 -- 172.21.15.5:0/1983235135 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562da562e5e0 con 0x562da55a95d0 2024-09-12T03:01:19.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.926+0000 7f474f7fe640 1 -- 172.21.15.5:0/1983235135 <== mon.1 v2:172.21.15.148:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4768036030 con 0x562da55a95d0 2024-09-12T03:01:19.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.926+0000 7f474f7fe640 1 -- 172.21.15.5:0/1983235135 <== mon.1 v2:172.21.15.148:3300/0 5 ==== fsmap(e 28) ==== 8374+0+0 (secure 0 0 0) 0x7f476804d050 con 0x562da55a95d0 2024-09-12T03:01:19.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.926+0000 7f47786b6740 10 client.25648 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T03:01:19.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.926+0000 7f47786b6740 10 client.25648 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-12T03:01:19.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.926+0000 7f47786b6740 20 client.25648 populate_metadata read hostname 'smithi005' 2024-09-12T03:01:19.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.926+0000 7f47786b6740 1 --2- 172.21.15.5:0/1983235135 >> [v2:172.21.15.148:6836/2606873922,v1:172.21.15.148:6837/2606873922] conn(0x562da562ec30 0x562da5631020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:19.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.926+0000 7f47786b6740 4 client.25648 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:01:19.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.926+0000 7f47786b6740 1 -- 172.21.15.5:0/1983235135 --> [v2:172.21.15.148:6836/2606873922,v1:172.21.15.148:6837/2606873922] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562da5516d70 con 0x562da562ec30 2024-09-12T03:01:19.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.926+0000 7f476f7fe640 1 --2- 172.21.15.5:0/1983235135 >> [v2:172.21.15.148:6836/2606873922,v1:172.21.15.148:6837/2606873922] conn(0x562da562ec30 0x562da5631020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:19.931 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.927+0000 7f476f7fe640 1 --2- 172.21.15.5:0/1983235135 >> [v2:172.21.15.148:6836/2606873922,v1:172.21.15.148:6837/2606873922] conn(0x562da562ec30 0x562da5631020 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2de1f89265858436 server_cookie=63ff1468ef2f0e85 in_seq=0 out_seq=0 2024-09-12T03:01:19.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.927+0000 7f474f7fe640 10 client.25648 ms_handle_connect on v2:172.21.15.148:6836/2606873922 2024-09-12T03:01:19.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.927+0000 7f474f7fe640 1 -- 172.21.15.5:0/1983235135 <== mds.1 v2:172.21.15.148:6836/2606873922 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562da5516d70 con 0x562da562ec30 2024-09-12T03:01:19.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.927+0000 7f474f7fe640 10 client.25648 handle_command_reply: tid=0 2024-09-12T03:01:19.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.927+0000 7f47786b6740 1 client.25648 shutdown 2024-09-12T03:01:19.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.927+0000 7f47786b6740 1 -- 172.21.15.5:0/1983235135 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562da55a95d0 msgr2=0x562da55bc9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:19.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.927+0000 7f47786b6740 1 --2- 172.21.15.5:0/1983235135 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562da55a95d0 0x562da55bc9c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f476802f470 tx=0x7f4768004a80 comp rx=0 tx=0).stop 2024-09-12T03:01:19.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.927+0000 7f47786b6740 1 -- 172.21.15.5:0/1983235135 shutdown_connections 2024-09-12T03:01:19.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.927+0000 7f47786b6740 1 --2- 172.21.15.5:0/1983235135 >> [v2:172.21.15.148:6836/2606873922,v1:172.21.15.148:6837/2606873922] conn(0x562da562ec30 0x562da5631020 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:19.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.927+0000 7f47786b6740 1 --2- 172.21.15.5:0/1983235135 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562da55aab40 0x562da55b7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:19.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.927+0000 7f47786b6740 1 --2- 172.21.15.5:0/1983235135 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562da55b4370 0x562da55b8230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:19.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.927+0000 7f47786b6740 1 --2- 172.21.15.5:0/1983235135 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562da55a95d0 0x562da55bc9c0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:19.932 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.927+0000 7f47786b6740 1 -- 172.21.15.5:0/1983235135 >> 172.21.15.5:0/1983235135 conn(0x562da5572310 msgr2=0x562da55a44e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:19.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.927+0000 7f474f7fe640 0 client.25648 ms_handle_reset on v2:172.21.15.148:6836/2606873922 2024-09-12T03:01:19.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.927+0000 7f47786b6740 1 -- 172.21.15.5:0/1983235135 shutdown_connections 2024-09-12T03:01:19.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.927+0000 7f47786b6740 1 -- 172.21.15.5:0/1983235135 wait complete. 2024-09-12T03:01:19.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.928+0000 7f47786b6740 20 client.25648 trim_cache size 0 max 16384 2024-09-12T03:01:19.933 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-12T03:01:19.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.928+0000 7f47762dd640 1 -- 172.21.15.5:0/3285880422 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f47580a2b80 msgr2=0x7f47580a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:19.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.928+0000 7f47762dd640 1 --2- 172.21.15.5:0/3285880422 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f47580a2b80 0x7f47580a5040 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f47600ccf50 tx=0x7f47600d2bb0 comp rx=0 tx=0).stop 2024-09-12T03:01:19.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.928+0000 7f47762dd640 1 -- 172.21.15.5:0/3285880422 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47700745a0 msgr2=0x7f477019a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:19.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.928+0000 7f47762dd640 1 --2- 172.21.15.5:0/3285880422 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47700745a0 0x7f477019a620 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f476000c920 tx=0x7f476000cde0 comp rx=0 tx=0).stop 2024-09-12T03:01:19.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.928+0000 7f47762dd640 1 -- 172.21.15.5:0/3285880422 shutdown_connections 2024-09-12T03:01:19.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.928+0000 7f47762dd640 1 --2- 172.21.15.5:0/3285880422 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f477010e470 0x7f477019ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:19.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.928+0000 7f47762dd640 1 --2- 172.21.15.5:0/3285880422 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4770074f50 0x7f477019ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:19.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.928+0000 7f47762dd640 1 --2- 172.21.15.5:0/3285880422 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f47580a2b80 0x7f47580a5040 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:19.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.928+0000 7f47762dd640 1 --2- 172.21.15.5:0/3285880422 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47700745a0 0x7f477019a620 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:19.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.928+0000 7f47762dd640 1 -- 172.21.15.5:0/3285880422 >> 172.21.15.5:0/3285880422 conn(0x7f47700fd8d0 msgr2=0x7f47700fe4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:19.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.928+0000 7f47762dd640 1 -- 172.21.15.5:0/3285880422 shutdown_connections 2024-09-12T03:01:19.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:19.928+0000 7f47762dd640 1 -- 172.21.15.5:0/3285880422 wait complete. 2024-09-12T03:01:20.672 INFO:tasks.workunit.client.0.smithi005.stdout:30000 trunc from 0x7bb2f to 0xcdf83 2024-09-12T03:01:20.877 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T03:01:21.005 INFO:tasks.workunit.client.0.smithi005.stdout:copying to largest ever: 0xffcdd 2024-09-12T03:01:21.154 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:01:21.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:01:21 smithi005 ceph-mon[31197]: pgmap v428: 97 pgs: 97 active+clean; 86 MiB data, 763 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.3 MiB/s wr, 501 op/s 2024-09-12T03:01:21.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.672+0000 7fe6d0039640 1 -- 172.21.15.5:0/1314461487 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6c8100a80 msgr2=0x7fe6c8100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:21.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.672+0000 7fe6d0039640 1 --2- 172.21.15.5:0/1314461487 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6c8100a80 0x7fe6c8100ee0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc009930 tx=0x7fe6bc02a420 comp rx=0 tx=0).stop 2024-09-12T03:01:21.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.672+0000 7fe6d0039640 1 -- 172.21.15.5:0/1314461487 shutdown_connections 2024-09-12T03:01:21.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.672+0000 7fe6d0039640 1 --2- 172.21.15.5:0/1314461487 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6c8100a80 0x7fe6c8100ee0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:21.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.672+0000 7fe6d0039640 1 --2- 172.21.15.5:0/1314461487 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe6c8101420 0x7fe6c810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:21.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.672+0000 7fe6d0039640 1 --2- 172.21.15.5:0/1314461487 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe6c8106a80 0x7fe6c8106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:21.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.672+0000 7fe6d0039640 1 -- 172.21.15.5:0/1314461487 >> 172.21.15.5:0/1314461487 conn(0x7fe6c80fc7a0 msgr2=0x7fe6c80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:21.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.672+0000 7fe6d0039640 1 -- 172.21.15.5:0/1314461487 shutdown_connections 2024-09-12T03:01:21.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.672+0000 7fe6d0039640 1 -- 172.21.15.5:0/1314461487 wait complete. 2024-09-12T03:01:21.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.673+0000 7fe6d0039640 1 Processor -- start 2024-09-12T03:01:21.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.673+0000 7fe6d0039640 1 -- start start 2024-09-12T03:01:21.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.673+0000 7fe6d0039640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe6c8100a80 0x7fe6c81992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:21.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.673+0000 7fe6d0039640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6c8101420 0x7fe6c81997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:21.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.673+0000 7fe6d0039640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe6c8106a80 0x7fe6c81a4320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:21.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.673+0000 7fe6d0039640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c8110e80 con 0x7fe6c8101420 2024-09-12T03:01:21.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.673+0000 7fe6d0039640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c8110d00 con 0x7fe6c8106a80 2024-09-12T03:01:21.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.673+0000 7fe6d0039640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c8111000 con 0x7fe6c8100a80 2024-09-12T03:01:21.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.673+0000 7fe6cd5d4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6c8101420 0x7fe6c81997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:21.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.673+0000 7fe6cd5d4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6c8101420 0x7fe6c81997e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41182/0 (socket says 172.21.15.5:41182) 2024-09-12T03:01:21.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.673+0000 7fe6cd5d4640 1 -- 172.21.15.5:0/184552460 learned_addr learned my addr 172.21.15.5:0/184552460 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:01:21.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.673+0000 7fe6cddd5640 1 --2- 172.21.15.5:0/184552460 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe6c8100a80 0x7fe6c81992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:21.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.673+0000 7fe6ce5d6640 1 --2- 172.21.15.5:0/184552460 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe6c8106a80 0x7fe6c81a4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:21.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.674+0000 7fe6cd5d4640 1 -- 172.21.15.5:0/184552460 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe6c8100a80 msgr2=0x7fe6c81992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:21.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.674+0000 7fe6cd5d4640 1 --2- 172.21.15.5:0/184552460 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe6c8100a80 0x7fe6c81992a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:21.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.674+0000 7fe6cd5d4640 1 -- 172.21.15.5:0/184552460 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe6c8106a80 msgr2=0x7fe6c81a4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:21.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.674+0000 7fe6cd5d4640 1 --2- 172.21.15.5:0/184552460 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe6c8106a80 0x7fe6c81a4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:21.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.674+0000 7fe6cd5d4640 1 -- 172.21.15.5:0/184552460 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6c81a4860 con 0x7fe6c8101420 2024-09-12T03:01:21.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.674+0000 7fe6cddd5640 1 --2- 172.21.15.5:0/184552460 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe6c8100a80 0x7fe6c81992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:01:21.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.674+0000 7fe6cd5d4640 1 --2- 172.21.15.5:0/184552460 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6c8101420 0x7fe6c81997e0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc009230 tx=0x7fe6bc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:21.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.674+0000 7fe6c6ffd640 1 -- 172.21.15.5:0/184552460 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6bc03f030 con 0x7fe6c8101420 2024-09-12T03:01:21.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.674+0000 7fe6c6ffd640 1 -- 172.21.15.5:0/184552460 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6bc003ad0 con 0x7fe6c8101420 2024-09-12T03:01:21.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.674+0000 7fe6d0039640 1 -- 172.21.15.5:0/184552460 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6c81a4a30 con 0x7fe6c8101420 2024-09-12T03:01:21.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.674+0000 7fe6c6ffd640 1 -- 172.21.15.5:0/184552460 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6bc035070 con 0x7fe6c8101420 2024-09-12T03:01:21.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.674+0000 7fe6d0039640 1 -- 172.21.15.5:0/184552460 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6c81a4ee0 con 0x7fe6c8101420 2024-09-12T03:01:21.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.677+0000 7fe6c6ffd640 1 -- 172.21.15.5:0/184552460 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fe6bc042030 con 0x7fe6c8101420 2024-09-12T03:01:21.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.677+0000 7fe6d2412740 1 Processor -- start 2024-09-12T03:01:21.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.677+0000 7fe6d2412740 1 -- start start 2024-09-12T03:01:21.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.678+0000 7fe6d2412740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5566dc03fb40 0x5566dc03e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:21.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.678+0000 7fe6d2412740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5566dc03e5d0 0x5566dc03ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:21.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.678+0000 7fe6d2412740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566dc04d780 0x5566dc04fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:21.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.678+0000 7fe6d2412740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5566dbfa42f0 con 0x5566dc04d780 2024-09-12T03:01:21.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.678+0000 7fe6d2412740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5566dbf4bcf0 con 0x5566dc03fb40 2024-09-12T03:01:21.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.678+0000 7fe6d2412740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5566dbfa02b0 con 0x5566dc03e5d0 2024-09-12T03:01:21.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.678+0000 7fe6c6ffd640 1 --2- 172.21.15.5:0/184552460 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fe6b00a2c00 0x7fe6b00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:21.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.678+0000 7fe6cddd5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5566dc03e5d0 0x5566dc03ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:21.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.678+0000 7fe6ce5d6640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5566dc03fb40 0x5566dc03e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:21.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.678+0000 7fe6cddd5640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5566dc03e5d0 0x5566dc03ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.5:50490/0 (socket says 172.21.15.5:50490) 2024-09-12T03:01:21.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.678+0000 7fe6ce5d6640 1 --2- 172.21.15.5:0/184552460 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fe6b00a2c00 0x7fe6b00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:21.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.678+0000 7fe6cddd5640 1 -- 172.21.15.5:0/1470008488 learned_addr learned my addr 172.21.15.5:0/1470008488 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:01:21.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.678+0000 7fe6cd5d4640 1 --2- 172.21.15.5:0/1470008488 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566dc04d780 0x5566dc04fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:21.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.678+0000 7fe6cddd5640 1 -- 172.21.15.5:0/1470008488 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5566dc03fb40 msgr2=0x5566dc03e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:01:21.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.678+0000 7fe6cddd5640 1 --2- 172.21.15.5:0/1470008488 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5566dc03fb40 0x5566dc03e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:21.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.678+0000 7fe6cddd5640 1 -- 172.21.15.5:0/1470008488 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566dc04d780 msgr2=0x5566dc04fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:01:21.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.678+0000 7fe6cddd5640 1 --2- 172.21.15.5:0/1470008488 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566dc04d780 0x5566dc04fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:21.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.678+0000 7fe6cddd5640 1 -- 172.21.15.5:0/1470008488 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566dbf303a0 con 0x5566dc03e5d0 2024-09-12T03:01:21.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.678+0000 7fe6c6ffd640 1 -- 172.21.15.5:0/184552460 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fe6bc0ef560 con 0x7fe6c8101420 2024-09-12T03:01:21.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.678+0000 7fe6ce5d6640 1 --2- 172.21.15.5:0/1470008488 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5566dc03fb40 0x5566dc03e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:01:21.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.678+0000 7fe6cddd5640 1 --2- 172.21.15.5:0/1470008488 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5566dc03e5d0 0x5566dc03ea10 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe6c00107e0 tx=0x7fe6c0010ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=457c001489e3d14c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:21.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.678+0000 7fe6ce5d6640 1 --2- 172.21.15.5:0/184552460 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fe6b00a2c00 0x7fe6b00a50c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fe6b8010980 tx=0x7fe6b8005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:21.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.679+0000 7fe6affff640 1 -- 172.21.15.5:0/1470008488 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c0004d20 con 0x5566dc03e5d0 2024-09-12T03:01:21.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.679+0000 7fe6affff640 1 -- 172.21.15.5:0/1470008488 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6c000ee10 con 0x5566dc03e5d0 2024-09-12T03:01:21.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.679+0000 7fe6affff640 1 -- 172.21.15.5:0/1470008488 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c00058e0 con 0x5566dc03e5d0 2024-09-12T03:01:21.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.679+0000 7fe6d2412740 1 -- 172.21.15.5:0/1470008488 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5566dc03e5d0 msgr2=0x5566dc03ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:21.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.679+0000 7fe6d2412740 1 --2- 172.21.15.5:0/1470008488 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5566dc03e5d0 0x5566dc03ea10 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe6c00107e0 tx=0x7fe6c0010ca0 comp rx=0 tx=0).stop 2024-09-12T03:01:21.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.680+0000 7fe6d2412740 1 -- 172.21.15.5:0/1470008488 shutdown_connections 2024-09-12T03:01:21.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.680+0000 7fe6d2412740 1 --2- 172.21.15.5:0/1470008488 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566dc04d780 0x5566dc04fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:21.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.680+0000 7fe6d2412740 1 --2- 172.21.15.5:0/1470008488 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5566dc03e5d0 0x5566dc03ea10 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:21.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.680+0000 7fe6d2412740 1 --2- 172.21.15.5:0/1470008488 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5566dc03fb40 0x5566dc03e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:21.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.680+0000 7fe6d2412740 1 -- 172.21.15.5:0/1470008488 >> 172.21.15.5:0/1470008488 conn(0x5566dc007310 msgr2=0x5566dc03ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:21.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.680+0000 7fe6d2412740 1 -- 172.21.15.5:0/1470008488 shutdown_connections 2024-09-12T03:01:21.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.680+0000 7fe6d2412740 1 -- 172.21.15.5:0/1470008488 wait complete. 2024-09-12T03:01:21.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.680+0000 7fe6d2412740 1 -- start start 2024-09-12T03:01:21.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.680+0000 7fe6d2412740 1 Processor -- start 2024-09-12T03:01:21.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.681+0000 7fe6d2412740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5566dc03e5d0 0x5566dc04d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:21.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.681+0000 7fe6d2412740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5566dc03fb40 0x5566dc04a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:21.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.681+0000 7fe6d2412740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566dc04d780 0x5566dc04a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:21.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.681+0000 7fe6d2412740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5566dc050480 con 0x5566dc04d780 2024-09-12T03:01:21.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.681+0000 7fe6d2412740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5566dbf5f060 con 0x5566dc03e5d0 2024-09-12T03:01:21.690 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.681+0000 7fe6d2412740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5566dc050600 con 0x5566dc03fb40 2024-09-12T03:01:21.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.681+0000 7fe6cddd5640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5566dc03e5d0 0x5566dc04d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:21.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.681+0000 7fe6d2412740 10 client.0 _ll_register_callbacks cb 0x5566dbcb28c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:01:21.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.681+0000 7fe6cddd5640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5566dc03e5d0 0x5566dc04d110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.5:47218/0 (socket says 172.21.15.5:47218) 2024-09-12T03:01:21.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.681+0000 7fe6cddd5640 1 -- 172.21.15.5:0/3588534740 learned_addr learned my addr 172.21.15.5:0/3588534740 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:01:21.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.681+0000 7fe6cd5d4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5566dc03fb40 0x5566dc04a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:21.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.681+0000 7fe6ce5d6640 1 --2- 172.21.15.5:0/3588534740 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566dc04d780 0x5566dc04a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:21.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.681+0000 7fe6cddd5640 1 -- 172.21.15.5:0/3588534740 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5566dc03fb40 msgr2=0x5566dc04a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:01:21.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.681+0000 7fe6cddd5640 1 --2- 172.21.15.5:0/3588534740 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5566dc03fb40 0x5566dc04a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:21.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.681+0000 7fe6cddd5640 1 -- 172.21.15.5:0/3588534740 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566dc04d780 msgr2=0x5566dc04a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:01:21.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.681+0000 7fe6cddd5640 1 --2- 172.21.15.5:0/3588534740 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566dc04d780 0x5566dc04a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:21.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.681+0000 7fe6cddd5640 1 -- 172.21.15.5:0/3588534740 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566dbd153c0 con 0x5566dc03e5d0 2024-09-12T03:01:21.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.681+0000 7fe6cd5d4640 1 --2- 172.21.15.5:0/3588534740 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5566dc03fb40 0x5566dc04a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:01:21.691 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.681+0000 7fe6cddd5640 1 --2- 172.21.15.5:0/3588534740 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5566dc03e5d0 0x5566dc04d110 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe6c0006010 tx=0x7fe6c0005280 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66b11329cbff7813 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:21.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.681+0000 7fe6ad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2024-09-12T03:01:21.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.682+0000 7fe6ad7fa640 1 -- 172.21.15.5:0/3588534740 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c002a060 con 0x5566dc03e5d0 2024-09-12T03:01:21.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.682+0000 7fe6d2412740 1 -- 172.21.15.5:0/3588534740 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6c000ee10 con 0x5566dc03e5d0 2024-09-12T03:01:21.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.682+0000 7fe6ad7fa640 1 -- 172.21.15.5:0/3588534740 <== mon.2 v2:172.21.15.146:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fe6c0005660 con 0x5566dc03e5d0 2024-09-12T03:01:21.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.682+0000 7fe6ad7fa640 1 -- 172.21.15.5:0/3588534740 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe6c0005860 con 0x5566dc03e5d0 2024-09-12T03:01:21.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.682+0000 7fe6d2412740 10 client.25562 fetch_fsmap learned FSMap version 28 2024-09-12T03:01:21.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.682+0000 7fe6d2412740 1 -- 172.21.15.5:0/3588534740 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5566dc05bee0 con 0x5566dc03e5d0 2024-09-12T03:01:21.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.682+0000 7fe6ad7fa640 1 -- 172.21.15.5:0/3588534740 <== mon.2 v2:172.21.15.146:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6c000ee10 con 0x5566dc03e5d0 2024-09-12T03:01:21.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.682+0000 7fe6ad7fa640 1 -- 172.21.15.5:0/3588534740 <== mon.2 v2:172.21.15.146:3300/0 5 ==== fsmap(e 28) ==== 8374+0+0 (secure 0 0 0) 0x7fe6c001a7f0 con 0x5566dc03e5d0 2024-09-12T03:01:21.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.682+0000 7fe6d2412740 10 client.25562 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T03:01:21.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.682+0000 7fe6d2412740 10 client.25562 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:01:21.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.682+0000 7fe6d2412740 20 client.25562 populate_metadata read hostname 'smithi005' 2024-09-12T03:01:21.692 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.682+0000 7fe6d2412740 1 --2- 172.21.15.5:0/3588534740 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x5566dc05d340 0x5566dc05f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:21.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.682+0000 7fe6d2412740 4 client.25562 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:01:21.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.682+0000 7fe6d2412740 1 -- 172.21.15.5:0/3588534740 --> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5566dbfb1860 con 0x5566dc05d340 2024-09-12T03:01:21.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.683+0000 7fe6cd5d4640 1 --2- 172.21.15.5:0/3588534740 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x5566dc05d340 0x5566dc05f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:21.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.683+0000 7fe6ad7fa640 10 client.25562 ms_handle_connect on v2:172.21.15.148:6838/2317600000 2024-09-12T03:01:21.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.683+0000 7fe6cd5d4640 1 --2- 172.21.15.5:0/3588534740 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x5566dc05d340 0x5566dc05f730 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=342f1f9432c8d741 server_cookie=19c56d43f9fbbdbe in_seq=0 out_seq=0 2024-09-12T03:01:21.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.684+0000 7fe6ad7fa640 1 -- 172.21.15.5:0/3588534740 <== mds.0 v2:172.21.15.148:6838/2317600000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5566dbfb1860 con 0x5566dc05d340 2024-09-12T03:01:21.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.684+0000 7fe6ad7fa640 10 client.25562 handle_command_reply: tid=0 2024-09-12T03:01:21.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.684+0000 7fe6d2412740 1 client.25562 shutdown 2024-09-12T03:01:21.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.684+0000 7fe6d2412740 1 -- 172.21.15.5:0/3588534740 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5566dc03e5d0 msgr2=0x5566dc04d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:21.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.684+0000 7fe6d2412740 1 --2- 172.21.15.5:0/3588534740 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5566dc03e5d0 0x5566dc04d110 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe6c0006010 tx=0x7fe6c0005280 comp rx=0 tx=0).stop 2024-09-12T03:01:21.693 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.684+0000 7fe6d2412740 1 -- 172.21.15.5:0/3588534740 shutdown_connections 2024-09-12T03:01:21.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.684+0000 7fe6d2412740 1 --2- 172.21.15.5:0/3588534740 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x5566dc05d340 0x5566dc05f730 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:21.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.684+0000 7fe6d2412740 1 --2- 172.21.15.5:0/3588534740 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566dc04d780 0x5566dc04a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:21.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.684+0000 7fe6ad7fa640 0 client.25562 ms_handle_reset on v2:172.21.15.148:6838/2317600000 2024-09-12T03:01:21.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.684+0000 7fe6d2412740 1 --2- 172.21.15.5:0/3588534740 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5566dc03fb40 0x5566dc04a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:21.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.684+0000 7fe6d2412740 1 --2- 172.21.15.5:0/3588534740 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5566dc03e5d0 0x5566dc04d110 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:21.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.684+0000 7fe6d2412740 1 -- 172.21.15.5:0/3588534740 >> 172.21.15.5:0/3588534740 conn(0x5566dc007310 msgr2=0x5566dc040960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:21.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.684+0000 7fe6d2412740 1 -- 172.21.15.5:0/3588534740 shutdown_connections 2024-09-12T03:01:21.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.684+0000 7fe6d2412740 1 -- 172.21.15.5:0/3588534740 wait complete. 2024-09-12T03:01:21.694 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.684+0000 7fe6d2412740 20 client.25562 trim_cache size 0 max 16384 2024-09-12T03:01:21.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.698+0000 7fe6d2412740 1 Processor -- start 2024-09-12T03:01:21.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.698+0000 7fe6d2412740 1 -- start start 2024-09-12T03:01:21.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.698+0000 7fe6d2412740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5566dc03e5d0 0x5566dc059750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:21.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.698+0000 7fe6d2412740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566dc03fb40 0x5566dc040820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:21.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.698+0000 7fe6d2412740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5566dc04d780 0x5566dc064ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:21.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.698+0000 7fe6d2412740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c81ab2f0 con 0x5566dc03fb40 2024-09-12T03:01:21.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.698+0000 7fe6d2412740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c81acec0 con 0x5566dc04d780 2024-09-12T03:01:21.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.698+0000 7fe6d2412740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c81ab9c0 con 0x5566dc03e5d0 2024-09-12T03:01:21.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.698+0000 7fe6cd5d4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566dc03fb40 0x5566dc040820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:21.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.698+0000 7fe6cd5d4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566dc03fb40 0x5566dc040820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41204/0 (socket says 172.21.15.5:41204) 2024-09-12T03:01:21.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.698+0000 7fe6cd5d4640 1 -- 172.21.15.5:0/245052043 learned_addr learned my addr 172.21.15.5:0/245052043 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:01:21.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.698+0000 7fe6cddd5640 1 --2- 172.21.15.5:0/245052043 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5566dc03e5d0 0x5566dc059750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:21.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.698+0000 7fe6ce5d6640 1 --2- 172.21.15.5:0/245052043 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5566dc04d780 0x5566dc064ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:21.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.698+0000 7fe6cd5d4640 1 -- 172.21.15.5:0/245052043 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5566dc03e5d0 msgr2=0x5566dc059750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:01:21.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.698+0000 7fe6cd5d4640 1 --2- 172.21.15.5:0/245052043 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5566dc03e5d0 0x5566dc059750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:21.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.698+0000 7fe6cd5d4640 1 -- 172.21.15.5:0/245052043 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5566dc04d780 msgr2=0x5566dc064ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:01:21.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.698+0000 7fe6cd5d4640 1 --2- 172.21.15.5:0/245052043 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5566dc04d780 0x5566dc064ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:21.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.699+0000 7fe6cd5d4640 1 -- 172.21.15.5:0/245052043 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566dc03bdd0 con 0x5566dc03fb40 2024-09-12T03:01:21.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.699+0000 7fe6cd5d4640 1 --2- 172.21.15.5:0/245052043 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566dc03fb40 0x5566dc040820 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc0096c0 tx=0x7fe6bc0ecf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48a4161976bec0fb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:21.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.699+0000 7fe6af7fe640 1 -- 172.21.15.5:0/245052043 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6bc049040 con 0x5566dc03fb40 2024-09-12T03:01:21.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.699+0000 7fe6af7fe640 1 -- 172.21.15.5:0/245052043 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6bc03b460 con 0x5566dc03fb40 2024-09-12T03:01:21.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.699+0000 7fe6af7fe640 1 -- 172.21.15.5:0/245052043 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6bc0b84f0 con 0x5566dc03fb40 2024-09-12T03:01:21.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.700+0000 7fe6d2412740 1 -- 172.21.15.5:0/245052043 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566dc03fb40 msgr2=0x5566dc040820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:21.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.700+0000 7fe6d2412740 1 --2- 172.21.15.5:0/245052043 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566dc03fb40 0x5566dc040820 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc0096c0 tx=0x7fe6bc0ecf40 comp rx=0 tx=0).stop 2024-09-12T03:01:21.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.700+0000 7fe6d2412740 1 -- 172.21.15.5:0/245052043 shutdown_connections 2024-09-12T03:01:21.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.700+0000 7fe6d2412740 1 --2- 172.21.15.5:0/245052043 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566dc03fb40 0x5566dc040820 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:21.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.700+0000 7fe6d2412740 1 --2- 172.21.15.5:0/245052043 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5566dc04d780 0x5566dc064ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:21.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.700+0000 7fe6d2412740 1 --2- 172.21.15.5:0/245052043 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5566dc03e5d0 0x5566dc059750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:21.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.700+0000 7fe6d2412740 1 -- 172.21.15.5:0/245052043 >> 172.21.15.5:0/245052043 conn(0x5566dc007310 msgr2=0x5566dc040110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:21.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.700+0000 7fe6d2412740 1 -- 172.21.15.5:0/245052043 shutdown_connections 2024-09-12T03:01:21.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.700+0000 7fe6d2412740 1 -- 172.21.15.5:0/245052043 wait complete. 2024-09-12T03:01:21.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.700+0000 7fe6d2412740 1 -- start start 2024-09-12T03:01:21.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.700+0000 7fe6d2412740 1 Processor -- start 2024-09-12T03:01:21.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.701+0000 7fe6d2412740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566dc03e5d0 0x5566dc047540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:21.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.701+0000 7fe6d2412740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5566dc03fb40 0x5566dc047a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:21.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.701+0000 7fe6d2412740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5566dc04d780 0x5566dc047fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:21.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.701+0000 7fe6d2412740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c81ac760 con 0x5566dc03e5d0 2024-09-12T03:01:21.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.701+0000 7fe6d2412740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c81ac090 con 0x5566dc04d780 2024-09-12T03:01:21.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.701+0000 7fe6d2412740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fe6c81a92d0 con 0x5566dc03fb40 2024-09-12T03:01:21.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.701+0000 7fe6d2412740 10 client.0 _ll_register_callbacks cb 0x7fe6bc0b8120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:01:21.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.701+0000 7fe6cddd5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566dc03e5d0 0x5566dc047540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:21.706 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.701+0000 7fe6cddd5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566dc03e5d0 0x5566dc047540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41214/0 (socket says 172.21.15.5:41214) 2024-09-12T03:01:21.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.701+0000 7fe6cddd5640 1 -- 172.21.15.5:0/2946206279 learned_addr learned my addr 172.21.15.5:0/2946206279 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:01:21.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.701+0000 7fe6cd5d4640 1 --2- 172.21.15.5:0/2946206279 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5566dc03fb40 0x5566dc047a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:21.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.701+0000 7fe6ce5d6640 1 --2- 172.21.15.5:0/2946206279 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5566dc04d780 0x5566dc047fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:21.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.701+0000 7fe6cddd5640 1 -- 172.21.15.5:0/2946206279 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5566dc03fb40 msgr2=0x5566dc047a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:01:21.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.701+0000 7fe6cddd5640 1 --2- 172.21.15.5:0/2946206279 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5566dc03fb40 0x5566dc047a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:21.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.701+0000 7fe6cddd5640 1 -- 172.21.15.5:0/2946206279 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5566dc04d780 msgr2=0x5566dc047fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:01:21.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.701+0000 7fe6cddd5640 1 --2- 172.21.15.5:0/2946206279 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5566dc04d780 0x5566dc047fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:21.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.701+0000 7fe6cddd5640 1 -- 172.21.15.5:0/2946206279 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566dc0c6f70 con 0x5566dc03e5d0 2024-09-12T03:01:21.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.701+0000 7fe6cddd5640 1 --2- 172.21.15.5:0/2946206279 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566dc03e5d0 0x5566dc047540 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fe6c0000f80 tx=0x7fe6c0005a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63632b77f9eaa6e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:21.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.701+0000 7fe6ad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T03:01:21.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.701+0000 7fe6ad7fa640 1 -- 172.21.15.5:0/2946206279 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c0035030 con 0x5566dc03e5d0 2024-09-12T03:01:21.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.702+0000 7fe6ad7fa640 1 -- 172.21.15.5:0/2946206279 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fe6c000ed50 con 0x5566dc03e5d0 2024-09-12T03:01:21.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.702+0000 7fe6d2412740 1 -- 172.21.15.5:0/2946206279 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6bc03b460 con 0x5566dc03e5d0 2024-09-12T03:01:21.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.702+0000 7fe6ad7fa640 1 -- 172.21.15.5:0/2946206279 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe6c002a040 con 0x5566dc03e5d0 2024-09-12T03:01:21.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.702+0000 7fe6d2412740 10 client.16149 fetch_fsmap learned FSMap version 28 2024-09-12T03:01:21.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.702+0000 7fe6d2412740 1 -- 172.21.15.5:0/2946206279 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5566dc0c8250 con 0x5566dc03e5d0 2024-09-12T03:01:21.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.702+0000 7fe6ad7fa640 1 -- 172.21.15.5:0/2946206279 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6bc03b460 con 0x5566dc03e5d0 2024-09-12T03:01:21.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.702+0000 7fe6ad7fa640 1 -- 172.21.15.5:0/2946206279 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8374+0+0 (secure 0 0 0) 0x7fe6c001a480 con 0x5566dc03e5d0 2024-09-12T03:01:21.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.702+0000 7fe6d2412740 10 client.16149 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T03:01:21.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.702+0000 7fe6d2412740 10 client.16149 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:01:21.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.702+0000 7fe6d2412740 20 client.16149 populate_metadata read hostname 'smithi005' 2024-09-12T03:01:21.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.702+0000 7fe6d2412740 1 --2- 172.21.15.5:0/2946206279 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x5566dc0c89f0 0x5566dc0cae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:21.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.702+0000 7fe6d2412740 4 client.16149 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:01:21.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.702+0000 7fe6d2412740 1 -- 172.21.15.5:0/2946206279 --> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5566dbfabd70 con 0x5566dc0c89f0 2024-09-12T03:01:21.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.702+0000 7fe6cd5d4640 1 --2- 172.21.15.5:0/2946206279 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x5566dc0c89f0 0x5566dc0cae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:21.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.703+0000 7fe6cd5d4640 1 --2- 172.21.15.5:0/2946206279 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x5566dc0c89f0 0x5566dc0cae10 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91917927850e9cc7 server_cookie=9d34320e26b6507d in_seq=0 out_seq=0 2024-09-12T03:01:21.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.703+0000 7fe6ad7fa640 10 client.16149 ms_handle_connect on v2:172.21.15.148:6838/2317600000 2024-09-12T03:01:21.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.704+0000 7fe6ad7fa640 1 -- 172.21.15.5:0/2946206279 <== mds.0 v2:172.21.15.148:6838/2317600000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5566dbfabd70 con 0x5566dc0c89f0 2024-09-12T03:01:21.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.704+0000 7fe6ad7fa640 10 client.16149 handle_command_reply: tid=0 2024-09-12T03:01:21.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.704+0000 7fe6d2412740 1 client.16149 shutdown 2024-09-12T03:01:21.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.704+0000 7fe6d2412740 1 -- 172.21.15.5:0/2946206279 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566dc03e5d0 msgr2=0x5566dc047540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:21.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.704+0000 7fe6d2412740 1 --2- 172.21.15.5:0/2946206279 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566dc03e5d0 0x5566dc047540 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fe6c0000f80 tx=0x7fe6c0005a40 comp rx=0 tx=0).stop 2024-09-12T03:01:21.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.704+0000 7fe6d2412740 1 -- 172.21.15.5:0/2946206279 shutdown_connections 2024-09-12T03:01:21.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.704+0000 7fe6d2412740 1 --2- 172.21.15.5:0/2946206279 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5566dc04d780 0x5566dc047fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:21.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.704+0000 7fe6d2412740 1 --2- 172.21.15.5:0/2946206279 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5566dc03fb40 0x5566dc047a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:21.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.704+0000 7fe6d2412740 1 --2- 172.21.15.5:0/2946206279 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x5566dc0c89f0 0x5566dc0cae10 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:21.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.704+0000 7fe6d2412740 1 --2- 172.21.15.5:0/2946206279 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566dc03e5d0 0x5566dc047540 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:21.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.704+0000 7fe6d2412740 1 -- 172.21.15.5:0/2946206279 >> 172.21.15.5:0/2946206279 conn(0x5566dc007310 msgr2=0x5566dc040930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:21.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.704+0000 7fe6ad7fa640 0 client.16149 ms_handle_reset on v2:172.21.15.148:6838/2317600000 2024-09-12T03:01:21.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.704+0000 7fe6d2412740 1 -- 172.21.15.5:0/2946206279 shutdown_connections 2024-09-12T03:01:21.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.704+0000 7fe6d2412740 1 -- 172.21.15.5:0/2946206279 wait complete. 2024-09-12T03:01:21.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.704+0000 7fe6d2412740 20 client.16149 trim_cache size 0 max 16384 2024-09-12T03:01:21.711 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-12T03:01:21.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.705+0000 7fe6d0039640 1 -- 172.21.15.5:0/184552460 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fe6b00a2c00 msgr2=0x7fe6b00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:21.711 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.705+0000 7fe6d0039640 1 --2- 172.21.15.5:0/184552460 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fe6b00a2c00 0x7fe6b00a50c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fe6b8010980 tx=0x7fe6b8005f90 comp rx=0 tx=0).stop 2024-09-12T03:01:21.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.705+0000 7fe6d0039640 1 -- 172.21.15.5:0/184552460 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6c8101420 msgr2=0x7fe6c81997e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:21.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.705+0000 7fe6d0039640 1 --2- 172.21.15.5:0/184552460 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6c8101420 0x7fe6c81997e0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc009230 tx=0x7fe6bc004270 comp rx=0 tx=0).stop 2024-09-12T03:01:21.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.705+0000 7fe6d0039640 1 -- 172.21.15.5:0/184552460 shutdown_connections 2024-09-12T03:01:21.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.705+0000 7fe6d0039640 1 --2- 172.21.15.5:0/184552460 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fe6b00a2c00 0x7fe6b00a50c0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:21.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.705+0000 7fe6d0039640 1 --2- 172.21.15.5:0/184552460 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe6c8101420 0x7fe6c81997e0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:21.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.705+0000 7fe6d0039640 1 --2- 172.21.15.5:0/184552460 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fe6c8106a80 0x7fe6c81a4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:21.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.705+0000 7fe6d0039640 1 --2- 172.21.15.5:0/184552460 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fe6c8100a80 0x7fe6c81992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:21.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.705+0000 7fe6d0039640 1 -- 172.21.15.5:0/184552460 >> 172.21.15.5:0/184552460 conn(0x7fe6c80fc7a0 msgr2=0x7fe6c80fcfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:21.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.705+0000 7fe6d0039640 1 -- 172.21.15.5:0/184552460 shutdown_connections 2024-09-12T03:01:21.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:21.705+0000 7fe6d0039640 1 -- 172.21.15.5:0/184552460 wait complete. 2024-09-12T03:01:21.713 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:01:21 smithi146 ceph-mon[35011]: pgmap v428: 97 pgs: 97 active+clean; 86 MiB data, 763 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.3 MiB/s wr, 501 op/s 2024-09-12T03:01:21.915 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:01:21 smithi148 ceph-mon[35090]: pgmap v428: 97 pgs: 97 active+clean; 86 MiB data, 763 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.3 MiB/s wr, 501 op/s 2024-09-12T03:01:22.581 INFO:tasks.workunit.client.0.smithi005.stdout:copying to largest ever: 0xffd97 2024-09-12T03:01:23.537 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T03:01:23.537 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T03:01:23.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:01:23 smithi146 ceph-mon[35011]: pgmap v429: 97 pgs: 97 active+clean; 86 MiB data, 763 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.3 MiB/s wr, 500 op/s 2024-09-12T03:01:23.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:01:23 smithi005 ceph-mon[31197]: pgmap v429: 97 pgs: 97 active+clean; 86 MiB data, 763 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.3 MiB/s wr, 500 op/s 2024-09-12T03:01:23.817 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:01:23.915 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:01:23 smithi148 ceph-mon[35090]: pgmap v429: 97 pgs: 97 active+clean; 86 MiB data, 763 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.3 MiB/s wr, 500 op/s 2024-09-12T03:01:24.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.348+0000 7f53a559a640 1 -- 172.21.15.5:0/2193268414 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5398031940 con 0x7f53a8100e20 2024-09-12T03:01:24.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.349+0000 7f53acf80640 1 -- 172.21.15.5:0/2193268414 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53a8100e20 msgr2=0x7f53a8101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:24.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.349+0000 7f53acf80640 1 --2- 172.21.15.5:0/2193268414 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53a8100e20 0x7f53a8101200 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f5398009900 tx=0x7f539802a480 comp rx=0 tx=0).stop 2024-09-12T03:01:24.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.349+0000 7f53acf80640 1 -- 172.21.15.5:0/2193268414 shutdown_connections 2024-09-12T03:01:24.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.349+0000 7f53acf80640 1 --2- 172.21.15.5:0/2193268414 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f53a8102ef0 0x7f53a810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:24.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.349+0000 7f53acf80640 1 --2- 172.21.15.5:0/2193268414 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f53a8102570 0x7f53a81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:24.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.349+0000 7f53acf80640 1 --2- 172.21.15.5:0/2193268414 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53a8100e20 0x7f53a8101200 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:24.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.349+0000 7f53acf80640 1 -- 172.21.15.5:0/2193268414 >> 172.21.15.5:0/2193268414 conn(0x7f53a80fc7d0 msgr2=0x7f53a80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:24.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.349+0000 7f53acf80640 1 -- 172.21.15.5:0/2193268414 shutdown_connections 2024-09-12T03:01:24.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.349+0000 7f53acf80640 1 -- 172.21.15.5:0/2193268414 wait complete. 2024-09-12T03:01:24.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.350+0000 7f53acf80640 1 Processor -- start 2024-09-12T03:01:24.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.350+0000 7f53acf80640 1 -- start start 2024-09-12T03:01:24.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.351+0000 7f53acf80640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53a8100e20 0x7f53a81984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:24.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.351+0000 7f53acf80640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f53a8102570 0x7f53a8198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:24.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.351+0000 7f53acf80640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f53a8102ef0 0x7f53a819cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:24.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.351+0000 7f53acf80640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f53a8110f80 con 0x7f53a8100e20 2024-09-12T03:01:24.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.351+0000 7f53acf80640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f53a8110e00 con 0x7f53a8102570 2024-09-12T03:01:24.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.351+0000 7f53acf80640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f53a8111100 con 0x7f53a8102ef0 2024-09-12T03:01:24.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.351+0000 7f53a659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53a8100e20 0x7f53a81984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:24.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.351+0000 7f53a659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53a8100e20 0x7f53a81984e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41162/0 (socket says 172.21.15.5:41162) 2024-09-12T03:01:24.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.351+0000 7f53a659c640 1 -- 172.21.15.5:0/2837982343 learned_addr learned my addr 172.21.15.5:0/2837982343 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:01:24.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.351+0000 7f53a6d9d640 1 --2- 172.21.15.5:0/2837982343 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f53a8102ef0 0x7f53a819cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:24.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.351+0000 7f53a5d9b640 1 --2- 172.21.15.5:0/2837982343 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f53a8102570 0x7f53a8198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:24.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.351+0000 7f53a659c640 1 -- 172.21.15.5:0/2837982343 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f53a8102ef0 msgr2=0x7f53a819cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:24.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.351+0000 7f53a659c640 1 --2- 172.21.15.5:0/2837982343 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f53a8102ef0 0x7f53a819cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:24.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.351+0000 7f53a659c640 1 -- 172.21.15.5:0/2837982343 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f53a8102570 msgr2=0x7f53a8198a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:24.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.351+0000 7f53a659c640 1 --2- 172.21.15.5:0/2837982343 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f53a8102570 0x7f53a8198a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:24.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.351+0000 7f53a659c640 1 -- 172.21.15.5:0/2837982343 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53a819d490 con 0x7f53a8100e20 2024-09-12T03:01:24.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.351+0000 7f53a659c640 1 --2- 172.21.15.5:0/2837982343 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53a8100e20 0x7f53a81984e0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f5398006010 tx=0x7f539803cd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:24.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.352+0000 7f53977fe640 1 -- 172.21.15.5:0/2837982343 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5398035030 con 0x7f53a8100e20 2024-09-12T03:01:24.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.352+0000 7f53acf80640 1 -- 172.21.15.5:0/2837982343 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53a819d720 con 0x7f53a8100e20 2024-09-12T03:01:24.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.352+0000 7f53977fe640 1 -- 172.21.15.5:0/2837982343 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5398041070 con 0x7f53a8100e20 2024-09-12T03:01:24.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.352+0000 7f53977fe640 1 -- 172.21.15.5:0/2837982343 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5398031870 con 0x7f53a8100e20 2024-09-12T03:01:24.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.352+0000 7f53acf80640 1 -- 172.21.15.5:0/2837982343 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53a8103680 con 0x7f53a8100e20 2024-09-12T03:01:24.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.355+0000 7f53af359740 1 Processor -- start 2024-09-12T03:01:24.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.355+0000 7f53af359740 1 -- start start 2024-09-12T03:01:24.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.355+0000 7f53977fe640 1 -- 172.21.15.5:0/2837982343 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f53980035f0 con 0x7f53a8100e20 2024-09-12T03:01:24.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.355+0000 7f53af359740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b370b19b60 0x55b370b19f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:24.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.355+0000 7f53af359740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b370b1a480 0x55b370b24800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:24.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.356+0000 7f53af359740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b370b24d40 0x55b370b27130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:24.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.356+0000 7f53af359740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b370a7f280 con 0x55b370b19b60 2024-09-12T03:01:24.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.356+0000 7f53af359740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55b370a26b10 con 0x55b370b24d40 2024-09-12T03:01:24.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.356+0000 7f53af359740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55b370a7b240 con 0x55b370b1a480 2024-09-12T03:01:24.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.356+0000 7f53a6d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b370b19b60 0x55b370b19f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:24.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.356+0000 7f53a6d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b370b19b60 0x55b370b19f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41176/0 (socket says 172.21.15.5:41176) 2024-09-12T03:01:24.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.356+0000 7f53a6d9d640 1 -- 172.21.15.5:0/128555869 learned_addr learned my addr 172.21.15.5:0/128555869 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:01:24.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.356+0000 7f53a5d9b640 1 --2- 172.21.15.5:0/128555869 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b370b24d40 0x55b370b27130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:24.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.356+0000 7f53a5d9b640 1 -- 172.21.15.5:0/128555869 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b370b1a480 msgr2=0x55b370b24800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:01:24.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.356+0000 7f53a659c640 1 --2- 172.21.15.5:0/128555869 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b370b1a480 0x55b370b24800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:24.358 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.356+0000 7f53a5d9b640 1 --2- 172.21.15.5:0/128555869 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b370b1a480 0x55b370b24800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:24.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.356+0000 7f53a5d9b640 1 -- 172.21.15.5:0/128555869 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b370b19b60 msgr2=0x55b370b19f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:01:24.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.356+0000 7f53a5d9b640 1 --2- 172.21.15.5:0/128555869 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b370b19b60 0x55b370b19f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:24.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.356+0000 7f53a5d9b640 1 -- 172.21.15.5:0/128555869 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b370a331f0 con 0x55b370b24d40 2024-09-12T03:01:24.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.356+0000 7f53a6d9d640 1 --2- 172.21.15.5:0/128555869 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b370b19b60 0x55b370b19f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:01:24.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.356+0000 7f53a659c640 1 --2- 172.21.15.5:0/128555869 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b370b1a480 0x55b370b24800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:01:24.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.357+0000 7f53a5d9b640 1 --2- 172.21.15.5:0/128555869 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b370b24d40 0x55b370b27130 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f539c0096c0 tx=0x7f539c005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7eb00302b06d3210 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:24.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.357+0000 7f53977fe640 1 --2- 172.21.15.5:0/2837982343 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f53900a2c00 0x7f53900a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:24.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.357+0000 7f5394ff9640 1 -- 172.21.15.5:0/128555869 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f539c023070 con 0x55b370b24d40 2024-09-12T03:01:24.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.357+0000 7f5394ff9640 1 -- 172.21.15.5:0/128555869 <== mon.2 v2:172.21.15.146:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f539c00b8a0 con 0x55b370b24d40 2024-09-12T03:01:24.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.357+0000 7f5394ff9640 1 -- 172.21.15.5:0/128555869 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f539c003a80 con 0x55b370b24d40 2024-09-12T03:01:24.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.357+0000 7f53a6d9d640 1 --2- 172.21.15.5:0/2837982343 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f53900a2c00 0x7f53900a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:24.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.357+0000 7f53977fe640 1 -- 172.21.15.5:0/2837982343 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f53980ef2a0 con 0x7f53a8100e20 2024-09-12T03:01:24.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.358+0000 7f53a6d9d640 1 --2- 172.21.15.5:0/2837982343 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f53900a2c00 0x7f53900a50c0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f53a0011890 tx=0x7f53a0009500 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:24.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.358+0000 7f53af359740 1 -- 172.21.15.5:0/128555869 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b370b24d40 msgr2=0x55b370b27130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:24.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.358+0000 7f53af359740 1 --2- 172.21.15.5:0/128555869 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b370b24d40 0x55b370b27130 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f539c0096c0 tx=0x7f539c005f90 comp rx=0 tx=0).stop 2024-09-12T03:01:24.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.358+0000 7f53af359740 1 -- 172.21.15.5:0/128555869 shutdown_connections 2024-09-12T03:01:24.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.358+0000 7f53af359740 1 --2- 172.21.15.5:0/128555869 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b370b24d40 0x55b370b27130 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:24.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.358+0000 7f53af359740 1 --2- 172.21.15.5:0/128555869 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b370b1a480 0x55b370b24800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:24.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.358+0000 7f53af359740 1 --2- 172.21.15.5:0/128555869 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b370b19b60 0x55b370b19f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:24.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.358+0000 7f53af359740 1 -- 172.21.15.5:0/128555869 >> 172.21.15.5:0/128555869 conn(0x55b370b160f0 msgr2=0x55b370b164d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:24.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.358+0000 7f53af359740 1 -- 172.21.15.5:0/128555869 shutdown_connections 2024-09-12T03:01:24.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.358+0000 7f53af359740 1 -- 172.21.15.5:0/128555869 wait complete. 2024-09-12T03:01:24.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.358+0000 7f53af359740 1 -- start start 2024-09-12T03:01:24.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.359+0000 7f53af359740 1 Processor -- start 2024-09-12T03:01:24.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.359+0000 7f53af359740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b370b19b60 0x55b370b21c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:24.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.359+0000 7f53af359740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b370b1a480 0x55b370b22140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:24.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.359+0000 7f53af359740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b370b24d40 0x55b370b22680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:24.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.359+0000 7f53af359740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55b370b27910 con 0x55b370b24d40 2024-09-12T03:01:24.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.359+0000 7f53af359740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55b370a39f10 con 0x55b370b1a480 2024-09-12T03:01:24.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.359+0000 7f53af359740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55b370b27a90 con 0x55b370b19b60 2024-09-12T03:01:24.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.359+0000 7f53af359740 10 client.0 _ll_register_callbacks cb 0x55b370a02a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:01:24.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.359+0000 7f53a6d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b370b24d40 0x55b370b22680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:24.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.359+0000 7f53a659c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b370b19b60 0x55b370b21c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:24.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.359+0000 7f53a6d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b370b24d40 0x55b370b22680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41178/0 (socket says 172.21.15.5:41178) 2024-09-12T03:01:24.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.359+0000 7f53a659c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b370b19b60 0x55b370b21c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.5:59070/0 (socket says 172.21.15.5:59070) 2024-09-12T03:01:24.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.359+0000 7f53a6d9d640 1 -- 172.21.15.5:0/557818032 learned_addr learned my addr 172.21.15.5:0/557818032 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:01:24.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.359+0000 7f53a5d9b640 1 --2- 172.21.15.5:0/557818032 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b370b1a480 0x55b370b22140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:24.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.360+0000 7f53a6d9d640 1 -- 172.21.15.5:0/557818032 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b370b19b60 msgr2=0x55b370b21c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:01:24.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.360+0000 7f53a6d9d640 1 --2- 172.21.15.5:0/557818032 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b370b19b60 0x55b370b21c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:24.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.360+0000 7f53a6d9d640 1 -- 172.21.15.5:0/557818032 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b370b1a480 msgr2=0x55b370b22140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:01:24.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.360+0000 7f53a6d9d640 1 --2- 172.21.15.5:0/557818032 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b370b1a480 0x55b370b22140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:24.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.360+0000 7f53a6d9d640 1 -- 172.21.15.5:0/557818032 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3707f06a0 con 0x55b370b24d40 2024-09-12T03:01:24.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.360+0000 7f53a6d9d640 1 --2- 172.21.15.5:0/557818032 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b370b24d40 0x55b370b22680 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f53a000fbe0 tx=0x7f53a0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de2d5034e927c2b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:24.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.360+0000 7f538dffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T03:01:24.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.360+0000 7f538dffb640 1 -- 172.21.15.5:0/557818032 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53a0004b20 con 0x55b370b24d40 2024-09-12T03:01:24.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.360+0000 7f538dffb640 1 -- 172.21.15.5:0/557818032 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f53a0017030 con 0x55b370b24d40 2024-09-12T03:01:24.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.360+0000 7f53af359740 1 -- 172.21.15.5:0/557818032 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f539c00b8a0 con 0x55b370b24d40 2024-09-12T03:01:24.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.360+0000 7f538dffb640 1 -- 172.21.15.5:0/557818032 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f53a0004cc0 con 0x55b370b24d40 2024-09-12T03:01:24.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.360+0000 7f53af359740 10 client.16161 fetch_fsmap learned FSMap version 28 2024-09-12T03:01:24.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.361+0000 7f53af359740 1 -- 172.21.15.5:0/557818032 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b370b334c0 con 0x55b370b24d40 2024-09-12T03:01:24.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.361+0000 7f538dffb640 1 -- 172.21.15.5:0/557818032 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f539c00b8a0 con 0x55b370b24d40 2024-09-12T03:01:24.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.361+0000 7f538dffb640 1 -- 172.21.15.5:0/557818032 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) ==== 8374+0+0 (secure 0 0 0) 0x7f53a0004030 con 0x55b370b24d40 2024-09-12T03:01:24.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.361+0000 7f53af359740 10 client.16161 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T03:01:24.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.361+0000 7f53af359740 10 client.16161 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:01:24.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.361+0000 7f53af359740 20 client.16161 populate_metadata read hostname 'smithi005' 2024-09-12T03:01:24.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.361+0000 7f53af359740 1 --2- 172.21.15.5:0/557818032 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55b370b34d90 0x55b370b37230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:24.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.361+0000 7f53af359740 4 client.16161 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:01:24.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.361+0000 7f53af359740 1 -- 172.21.15.5:0/557818032 --> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b370a8c7f0 con 0x55b370b34d90 2024-09-12T03:01:24.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.361+0000 7f53a659c640 1 --2- 172.21.15.5:0/557818032 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55b370b34d90 0x55b370b37230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:24.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.362+0000 7f53a659c640 1 --2- 172.21.15.5:0/557818032 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55b370b34d90 0x55b370b37230 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77bff2a456a81421 server_cookie=948943b42d5511ef in_seq=0 out_seq=0 2024-09-12T03:01:24.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.362+0000 7f538dffb640 10 client.16161 ms_handle_connect on v2:172.21.15.148:6838/2317600000 2024-09-12T03:01:24.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.363+0000 7f538dffb640 1 -- 172.21.15.5:0/557818032 <== mds.0 v2:172.21.15.148:6838/2317600000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b370a8c7f0 con 0x55b370b34d90 2024-09-12T03:01:24.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.363+0000 7f538dffb640 10 client.16161 handle_command_reply: tid=0 2024-09-12T03:01:24.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.363+0000 7f53af359740 1 client.16161 shutdown 2024-09-12T03:01:24.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.363+0000 7f53af359740 1 -- 172.21.15.5:0/557818032 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b370b24d40 msgr2=0x55b370b22680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:24.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.363+0000 7f53af359740 1 --2- 172.21.15.5:0/557818032 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b370b24d40 0x55b370b22680 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f53a000fbe0 tx=0x7f53a0004ab0 comp rx=0 tx=0).stop 2024-09-12T03:01:24.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.363+0000 7f53af359740 1 -- 172.21.15.5:0/557818032 shutdown_connections 2024-09-12T03:01:24.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.363+0000 7f53af359740 1 --2- 172.21.15.5:0/557818032 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55b370b34d90 0x55b370b37230 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:24.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.363+0000 7f53af359740 1 --2- 172.21.15.5:0/557818032 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b370b24d40 0x55b370b22680 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:24.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.363+0000 7f53af359740 1 --2- 172.21.15.5:0/557818032 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b370b1a480 0x55b370b22140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:24.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.363+0000 7f53af359740 1 --2- 172.21.15.5:0/557818032 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b370b19b60 0x55b370b21c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:24.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.363+0000 7f538dffb640 0 client.16161 ms_handle_reset on v2:172.21.15.148:6838/2317600000 2024-09-12T03:01:24.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.363+0000 7f53af359740 1 -- 172.21.15.5:0/557818032 >> 172.21.15.5:0/557818032 conn(0x55b370b160f0 msgr2=0x55b370b1be20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:24.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.363+0000 7f53af359740 1 -- 172.21.15.5:0/557818032 shutdown_connections 2024-09-12T03:01:24.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.363+0000 7f53af359740 1 -- 172.21.15.5:0/557818032 wait complete. 2024-09-12T03:01:24.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.363+0000 7f53af359740 20 client.16161 trim_cache size 0 max 16384 2024-09-12T03:01:24.385 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.385+0000 7f53af359740 1 Processor -- start 2024-09-12T03:01:24.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.385+0000 7f53af359740 1 -- start start 2024-09-12T03:01:24.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.385+0000 7f53af359740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b370b19b60 0x55b370b1c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:24.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.385+0000 7f53af359740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b370b1a480 0x55b370b23a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:24.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.385+0000 7f53af359740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b370b24d40 0x55b370b3cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:24.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.385+0000 7f53af359740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f53a81ab280 con 0x55b370b1a480 2024-09-12T03:01:24.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.385+0000 7f53af359740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f53a81ace50 con 0x55b370b24d40 2024-09-12T03:01:24.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.385+0000 7f53af359740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f53a81ab950 con 0x55b370b19b60 2024-09-12T03:01:24.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.385+0000 7f53a5d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b370b1a480 0x55b370b23a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:24.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.385+0000 7f53a5d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b370b1a480 0x55b370b23a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41180/0 (socket says 172.21.15.5:41180) 2024-09-12T03:01:24.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.385+0000 7f53a5d9b640 1 -- 172.21.15.5:0/2758117855 learned_addr learned my addr 172.21.15.5:0/2758117855 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:01:24.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.385+0000 7f53a6d9d640 1 --2- 172.21.15.5:0/2758117855 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b370b24d40 0x55b370b3cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:24.386 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.386+0000 7f53a659c640 1 --2- 172.21.15.5:0/2758117855 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b370b19b60 0x55b370b1c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:24.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.386+0000 7f53a5d9b640 1 -- 172.21.15.5:0/2758117855 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b370b19b60 msgr2=0x55b370b1c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:01:24.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.386+0000 7f53a5d9b640 1 --2- 172.21.15.5:0/2758117855 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b370b19b60 0x55b370b1c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:24.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.386+0000 7f53a5d9b640 1 -- 172.21.15.5:0/2758117855 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b370b24d40 msgr2=0x55b370b3cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:01:24.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.386+0000 7f53a5d9b640 1 --2- 172.21.15.5:0/2758117855 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b370b24d40 0x55b370b3cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:24.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.386+0000 7f53a5d9b640 1 -- 172.21.15.5:0/2758117855 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b370b27a20 con 0x55b370b1a480 2024-09-12T03:01:24.388 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.386+0000 7f53a6d9d640 1 --2- 172.21.15.5:0/2758117855 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b370b24d40 0x55b370b3cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:01:24.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.386+0000 7f53a5d9b640 1 --2- 172.21.15.5:0/2758117855 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b370b1a480 0x55b370b23a60 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f539c0096c0 tx=0x7f539c009f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f70926011761d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:24.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.386+0000 7f538ffff640 1 -- 172.21.15.5:0/2758117855 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f539c02d020 con 0x55b370b1a480 2024-09-12T03:01:24.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.386+0000 7f538ffff640 1 -- 172.21.15.5:0/2758117855 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f539c024040 con 0x55b370b1a480 2024-09-12T03:01:24.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.386+0000 7f538ffff640 1 -- 172.21.15.5:0/2758117855 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f539c023070 con 0x55b370b1a480 2024-09-12T03:01:24.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.387+0000 7f53af359740 1 -- 172.21.15.5:0/2758117855 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b370b1a480 msgr2=0x55b370b23a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:24.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.387+0000 7f53af359740 1 --2- 172.21.15.5:0/2758117855 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b370b1a480 0x55b370b23a60 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f539c0096c0 tx=0x7f539c009f50 comp rx=0 tx=0).stop 2024-09-12T03:01:24.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.387+0000 7f53af359740 1 -- 172.21.15.5:0/2758117855 shutdown_connections 2024-09-12T03:01:24.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.387+0000 7f53af359740 1 --2- 172.21.15.5:0/2758117855 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b370b1a480 0x55b370b23a60 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:24.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.387+0000 7f53af359740 1 --2- 172.21.15.5:0/2758117855 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b370b24d40 0x55b370b3cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:24.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.387+0000 7f53af359740 1 --2- 172.21.15.5:0/2758117855 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b370b19b60 0x55b370b1c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:24.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.387+0000 7f53af359740 1 -- 172.21.15.5:0/2758117855 >> 172.21.15.5:0/2758117855 conn(0x55b370b160f0 msgr2=0x55b370b23350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:24.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.387+0000 7f53af359740 1 -- 172.21.15.5:0/2758117855 shutdown_connections 2024-09-12T03:01:24.389 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.388+0000 7f53af359740 1 -- 172.21.15.5:0/2758117855 wait complete. 2024-09-12T03:01:24.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.388+0000 7f53af359740 1 -- start start 2024-09-12T03:01:24.390 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.388+0000 7f53af359740 1 Processor -- start 2024-09-12T03:01:24.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.388+0000 7f53af359740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b370b19b60 0x55b370b2d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:24.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.388+0000 7f53af359740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b370b1a480 0x55b370b2d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:24.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.388+0000 7f53af359740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b370b24d40 0x55b370b1e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:24.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.389+0000 7f53a659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b370b19b60 0x55b370b2d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:24.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.389+0000 7f53af359740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f53a81ac6f0 con 0x55b370b19b60 2024-09-12T03:01:24.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.389+0000 7f53af359740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f53a81ac020 con 0x55b370b1a480 2024-09-12T03:01:24.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.389+0000 7f53af359740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f53a81a9280 con 0x55b370b24d40 2024-09-12T03:01:24.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.389+0000 7f53a659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b370b19b60 0x55b370b2d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41190/0 (socket says 172.21.15.5:41190) 2024-09-12T03:01:24.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.389+0000 7f53a659c640 1 -- 172.21.15.5:0/2078631667 learned_addr learned my addr 172.21.15.5:0/2078631667 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:01:24.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.389+0000 7f53a6d9d640 1 --2- 172.21.15.5:0/2078631667 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b370b24d40 0x55b370b1e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:24.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.389+0000 7f53a5d9b640 1 --2- 172.21.15.5:0/2078631667 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b370b1a480 0x55b370b2d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:24.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.389+0000 7f53af359740 10 client.0 _ll_register_callbacks cb 0x7f53a0011e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:01:24.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.389+0000 7f53a659c640 1 -- 172.21.15.5:0/2078631667 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b370b24d40 msgr2=0x55b370b1e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:01:24.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.389+0000 7f53a659c640 1 --2- 172.21.15.5:0/2078631667 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b370b24d40 0x55b370b1e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:24.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.389+0000 7f53a659c640 1 -- 172.21.15.5:0/2078631667 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b370b1a480 msgr2=0x55b370b2d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:01:24.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.389+0000 7f53a659c640 1 --2- 172.21.15.5:0/2078631667 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b370b1a480 0x55b370b2d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:24.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.389+0000 7f53a659c640 1 -- 172.21.15.5:0/2078631667 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b370b1efd0 con 0x55b370b19b60 2024-09-12T03:01:24.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.389+0000 7f53af359740 1 -- 172.21.15.5:0/2078631667 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f539c024040 con 0x55b370b19b60 2024-09-12T03:01:24.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.389+0000 7f53a5d9b640 1 --2- 172.21.15.5:0/2078631667 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b370b1a480 0x55b370b2d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:01:24.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.390+0000 7f53a659c640 1 --2- 172.21.15.5:0/2078631667 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b370b19b60 0x55b370b2d050 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f53980b7f70 tx=0x7f53980bba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50ba2f379bfc65f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:24.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.390+0000 7f538dffb640 10 client.16167 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T03:01:24.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.390+0000 7f538dffb640 1 -- 172.21.15.5:0/2078631667 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53980d3030 con 0x55b370b19b60 2024-09-12T03:01:24.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.390+0000 7f538dffb640 1 -- 172.21.15.5:0/2078631667 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f53980b9e10 con 0x55b370b19b60 2024-09-12T03:01:24.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.390+0000 7f538dffb640 1 -- 172.21.15.5:0/2078631667 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f53980bbb10 con 0x55b370b19b60 2024-09-12T03:01:24.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.390+0000 7f53af359740 10 client.16167 fetch_fsmap learned FSMap version 28 2024-09-12T03:01:24.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.390+0000 7f53af359740 1 -- 172.21.15.5:0/2078631667 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({fsmap=28}) -- 0x55b370b9f710 con 0x55b370b19b60 2024-09-12T03:01:24.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.390+0000 7f538dffb640 1 -- 172.21.15.5:0/2078631667 <== mon.0 v2:172.21.15.5:3300/0 4 ==== fsmap(e 28) ==== 8374+0+0 (secure 0 0 0) 0x7f53980b8180 con 0x55b370b19b60 2024-09-12T03:01:24.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.390+0000 7f53af359740 10 client.16167 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T03:01:24.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.390+0000 7f53af359740 10 client.16167 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:01:24.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.390+0000 7f53af359740 20 client.16167 populate_metadata read hostname 'smithi005' 2024-09-12T03:01:24.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.390+0000 7f53af359740 1 --2- 172.21.15.5:0/2078631667 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55b370b9fdb0 0x55b370ba21d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:24.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.390+0000 7f53af359740 4 client.16167 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T03:01:24.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.390+0000 7f53af359740 1 -- 172.21.15.5:0/2078631667 --> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55b370a86d00 con 0x55b370b9fdb0 2024-09-12T03:01:24.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.391+0000 7f53a5d9b640 1 --2- 172.21.15.5:0/2078631667 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55b370b9fdb0 0x55b370ba21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:24.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.391+0000 7f538dffb640 10 client.16167 ms_handle_connect on v2:172.21.15.148:6838/2317600000 2024-09-12T03:01:24.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.391+0000 7f53a5d9b640 1 --2- 172.21.15.5:0/2078631667 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55b370b9fdb0 0x55b370ba21d0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8f3f37acc20240e server_cookie=4a4248266215e166 in_seq=0 out_seq=0 2024-09-12T03:01:24.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.392+0000 7f538dffb640 1 -- 172.21.15.5:0/2078631667 <== mds.0 v2:172.21.15.148:6838/2317600000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55b370a86d00 con 0x55b370b9fdb0 2024-09-12T03:01:24.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.392+0000 7f538dffb640 10 client.16167 handle_command_reply: tid=0 2024-09-12T03:01:24.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.392+0000 7f53af359740 1 client.16167 shutdown 2024-09-12T03:01:24.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.392+0000 7f53af359740 1 -- 172.21.15.5:0/2078631667 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b370b19b60 msgr2=0x55b370b2d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:24.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.392+0000 7f53af359740 1 --2- 172.21.15.5:0/2078631667 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b370b19b60 0x55b370b2d050 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f53980b7f70 tx=0x7f53980bba10 comp rx=0 tx=0).stop 2024-09-12T03:01:24.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.392+0000 7f53af359740 1 -- 172.21.15.5:0/2078631667 shutdown_connections 2024-09-12T03:01:24.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.392+0000 7f53af359740 1 --2- 172.21.15.5:0/2078631667 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55b370b24d40 0x55b370b1e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:24.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.392+0000 7f53af359740 1 --2- 172.21.15.5:0/2078631667 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55b370b1a480 0x55b370b2d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:24.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.392+0000 7f53af359740 1 --2- 172.21.15.5:0/2078631667 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55b370b9fdb0 0x55b370ba21d0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:24.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.392+0000 7f53af359740 1 --2- 172.21.15.5:0/2078631667 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b370b19b60 0x55b370b2d050 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:24.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.392+0000 7f538dffb640 0 client.16167 ms_handle_reset on v2:172.21.15.148:6838/2317600000 2024-09-12T03:01:24.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.393+0000 7f53af359740 1 -- 172.21.15.5:0/2078631667 >> 172.21.15.5:0/2078631667 conn(0x55b370b160f0 msgr2=0x55b370b15270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:24.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.393+0000 7f53af359740 1 -- 172.21.15.5:0/2078631667 shutdown_connections 2024-09-12T03:01:24.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.393+0000 7f53af359740 1 -- 172.21.15.5:0/2078631667 wait complete. 2024-09-12T03:01:24.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.393+0000 7f53af359740 20 client.16167 trim_cache size 0 max 16384 2024-09-12T03:01:24.397 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-12T03:01:24.397 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-09-12T03:01:24.397 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "af9de749-3eeb-4965-a2b0-fb23758c2c85", 2024-09-12T03:01:24.397 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-09-12T03:01:24.397 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-12T03:01:24.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.393+0000 7f53acf80640 1 -- 172.21.15.5:0/2837982343 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f53900a2c00 msgr2=0x7f53900a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:24.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.393+0000 7f53acf80640 1 --2- 172.21.15.5:0/2837982343 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f53900a2c00 0x7f53900a50c0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f53a0011890 tx=0x7f53a0009500 comp rx=0 tx=0).stop 2024-09-12T03:01:24.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.393+0000 7f53acf80640 1 -- 172.21.15.5:0/2837982343 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53a8100e20 msgr2=0x7f53a81984e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:24.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.393+0000 7f53acf80640 1 --2- 172.21.15.5:0/2837982343 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53a8100e20 0x7f53a81984e0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f5398006010 tx=0x7f539803cd10 comp rx=0 tx=0).stop 2024-09-12T03:01:24.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.394+0000 7f53acf80640 1 -- 172.21.15.5:0/2837982343 shutdown_connections 2024-09-12T03:01:24.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.394+0000 7f53acf80640 1 --2- 172.21.15.5:0/2837982343 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f53a8102ef0 0x7f53a819cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:24.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.394+0000 7f53acf80640 1 --2- 172.21.15.5:0/2837982343 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f53a8102570 0x7f53a8198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:24.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.394+0000 7f53acf80640 1 --2- 172.21.15.5:0/2837982343 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f53900a2c00 0x7f53900a50c0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:24.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.394+0000 7f53acf80640 1 --2- 172.21.15.5:0/2837982343 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f53a8100e20 0x7f53a81984e0 secure :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f5398006010 tx=0x7f539803cd10 comp rx=0 tx=0).stop 2024-09-12T03:01:24.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.394+0000 7f53acf80640 1 -- 172.21.15.5:0/2837982343 >> 172.21.15.5:0/2837982343 conn(0x7f53a80fc7d0 msgr2=0x7f53a806a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:24.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.394+0000 7f53acf80640 1 -- 172.21.15.5:0/2837982343 shutdown_connections 2024-09-12T03:01:24.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:24.394+0000 7f53acf80640 1 -- 172.21.15.5:0/2837982343 wait complete. 2024-09-12T03:01:24.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:01:24 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:01:24.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:01:24 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:01:24.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:01:24 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:01:24.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:01:24 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T03:01:24.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:01:24 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:01:24.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:01:24 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:01:24.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:01:24 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:01:24.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:01:24 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T03:01:24.915 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:01:24 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:01:24.915 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:01:24 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:01:24.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:01:24 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:01:24.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:01:24 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T03:01:25.190 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T03:01:25.473 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:01:25.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:01:25 smithi005 ceph-mon[31197]: pgmap v430: 97 pgs: 97 active+clean; 89 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.5 MiB/s wr, 595 op/s 2024-09-12T03:01:25.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:01:25 smithi005 ceph-mon[31197]: scrub queued for path: / 2024-09-12T03:01:25.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:01:25 smithi005 ceph-mon[31197]: scrub summary: idle+waiting paths [/] 2024-09-12T03:01:25.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:01:25 smithi005 ceph-mon[31197]: scrub summary: active paths [/] 2024-09-12T03:01:25.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:01:25 smithi005 ceph-mon[31197]: scrub summary: active paths [/] 2024-09-12T03:01:25.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:01:25 smithi005 ceph-mon[31197]: scrub summary: idle+waiting paths [/] 2024-09-12T03:01:25.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:01:25 smithi005 ceph-mon[31197]: scrub summary: active paths [/] 2024-09-12T03:01:25.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:01:25 smithi005 ceph-mon[31197]: scrub summary: idle+waiting paths [/] 2024-09-12T03:01:25.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:01:25 smithi005 ceph-mon[31197]: scrub summary: active paths [/] 2024-09-12T03:01:25.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:01:25 smithi005 ceph-mon[31197]: scrub summary: idle+waiting paths [/] 2024-09-12T03:01:25.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:01:25 smithi146 ceph-mon[35011]: pgmap v430: 97 pgs: 97 active+clean; 89 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.5 MiB/s wr, 595 op/s 2024-09-12T03:01:25.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:01:25 smithi146 ceph-mon[35011]: scrub queued for path: / 2024-09-12T03:01:25.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:01:25 smithi146 ceph-mon[35011]: scrub summary: idle+waiting paths [/] 2024-09-12T03:01:25.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:01:25 smithi146 ceph-mon[35011]: scrub summary: active paths [/] 2024-09-12T03:01:25.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:01:25 smithi146 ceph-mon[35011]: scrub summary: active paths [/] 2024-09-12T03:01:25.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:01:25 smithi146 ceph-mon[35011]: scrub summary: idle+waiting paths [/] 2024-09-12T03:01:25.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:01:25 smithi146 ceph-mon[35011]: scrub summary: active paths [/] 2024-09-12T03:01:25.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:01:25 smithi146 ceph-mon[35011]: scrub summary: idle+waiting paths [/] 2024-09-12T03:01:25.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:01:25 smithi146 ceph-mon[35011]: scrub summary: active paths [/] 2024-09-12T03:01:25.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:01:25 smithi146 ceph-mon[35011]: scrub summary: idle+waiting paths [/] 2024-09-12T03:01:25.915 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:01:25 smithi148 ceph-mon[35090]: pgmap v430: 97 pgs: 97 active+clean; 89 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.5 MiB/s wr, 595 op/s 2024-09-12T03:01:25.915 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:01:25 smithi148 ceph-mon[35090]: scrub queued for path: / 2024-09-12T03:01:25.915 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:01:25 smithi148 ceph-mon[35090]: scrub summary: idle+waiting paths [/] 2024-09-12T03:01:25.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:01:25 smithi148 ceph-mon[35090]: scrub summary: active paths [/] 2024-09-12T03:01:25.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:01:25 smithi148 ceph-mon[35090]: scrub summary: active paths [/] 2024-09-12T03:01:25.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:01:25 smithi148 ceph-mon[35090]: scrub summary: idle+waiting paths [/] 2024-09-12T03:01:25.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:01:25 smithi148 ceph-mon[35090]: scrub summary: active paths [/] 2024-09-12T03:01:25.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:01:25 smithi148 ceph-mon[35090]: scrub summary: idle+waiting paths [/] 2024-09-12T03:01:25.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:01:25 smithi148 ceph-mon[35090]: scrub summary: active paths [/] 2024-09-12T03:01:25.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:01:25 smithi148 ceph-mon[35090]: scrub summary: idle+waiting paths [/] 2024-09-12T03:01:26.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.035+0000 7fccc034d640 1 -- 172.21.15.5:0/3151975770 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fccb8109120 msgr2=0x7fccb8110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:26.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.035+0000 7fccc034d640 1 --2- 172.21.15.5:0/3151975770 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fccb8109120 0x7fccb8110ca0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fcca8009ca0 tx=0x7fcca802a8c0 comp rx=0 tx=0).stop 2024-09-12T03:01:26.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.035+0000 7fccc034d640 1 -- 172.21.15.5:0/3151975770 shutdown_connections 2024-09-12T03:01:26.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.035+0000 7fccc034d640 1 --2- 172.21.15.5:0/3151975770 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fccb8109120 0x7fccb8110ca0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:26.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.035+0000 7fccc034d640 1 --2- 172.21.15.5:0/3151975770 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fccb81049a0 0x7fccb81089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:26.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.035+0000 7fccc034d640 1 --2- 172.21.15.5:0/3151975770 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fccb8103ff0 0x7fccb81043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:26.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.035+0000 7fccc034d640 1 -- 172.21.15.5:0/3151975770 >> 172.21.15.5:0/3151975770 conn(0x7fccb80ffab0 msgr2=0x7fccb8101ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:26.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.035+0000 7fccc034d640 1 -- 172.21.15.5:0/3151975770 shutdown_connections 2024-09-12T03:01:26.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.035+0000 7fccc034d640 1 -- 172.21.15.5:0/3151975770 wait complete. 2024-09-12T03:01:26.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.036+0000 7fccc034d640 1 Processor -- start 2024-09-12T03:01:26.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.036+0000 7fccc034d640 1 -- start start 2024-09-12T03:01:26.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.036+0000 7fccc034d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fccb8103ff0 0x7fccb819a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:26.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.036+0000 7fccc034d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fccb81049a0 0x7fccb819ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:26.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.036+0000 7fccc034d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fccb8109120 0x7fccb819eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:26.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.036+0000 7fccc034d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fccb81141a0 con 0x7fccb81049a0 2024-09-12T03:01:26.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.036+0000 7fccc034d640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fccb8114020 con 0x7fccb8103ff0 2024-09-12T03:01:26.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.036+0000 7fccc034d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fccb8114320 con 0x7fccb8109120 2024-09-12T03:01:26.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.036+0000 7fccbd8e8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fccb81049a0 0x7fccb819ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:26.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.037+0000 7fccbe0e9640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fccb8103ff0 0x7fccb819a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:26.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.037+0000 7fccbd8e8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fccb81049a0 0x7fccb819ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41210/0 (socket says 172.21.15.5:41210) 2024-09-12T03:01:26.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.037+0000 7fccbd8e8640 1 -- 172.21.15.5:0/3571938413 learned_addr learned my addr 172.21.15.5:0/3571938413 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:01:26.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.037+0000 7fccbe8ea640 1 --2- 172.21.15.5:0/3571938413 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fccb8109120 0x7fccb819eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:26.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.037+0000 7fccbd8e8640 1 -- 172.21.15.5:0/3571938413 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fccb8109120 msgr2=0x7fccb819eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:26.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.037+0000 7fccbd8e8640 1 --2- 172.21.15.5:0/3571938413 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fccb8109120 0x7fccb819eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:26.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.037+0000 7fccbd8e8640 1 -- 172.21.15.5:0/3571938413 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fccb8103ff0 msgr2=0x7fccb819a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:26.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.037+0000 7fccbd8e8640 1 --2- 172.21.15.5:0/3571938413 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fccb8103ff0 0x7fccb819a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:26.167 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.037+0000 7fccbd8e8640 1 -- 172.21.15.5:0/3571938413 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccb819f590 con 0x7fccb81049a0 2024-09-12T03:01:26.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.037+0000 7fccbe0e9640 1 --2- 172.21.15.5:0/3571938413 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fccb8103ff0 0x7fccb819a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:01:26.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.037+0000 7fccbd8e8640 1 --2- 172.21.15.5:0/3571938413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fccb81049a0 0x7fccb819ab20 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fccac00e9f0 tx=0x7fccac00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:26.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.037+0000 7fccb77fe640 1 -- 172.21.15.5:0/3571938413 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccac00cd80 con 0x7fccb81049a0 2024-09-12T03:01:26.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.037+0000 7fccb77fe640 1 -- 172.21.15.5:0/3571938413 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fccac010910 con 0x7fccb81049a0 2024-09-12T03:01:26.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.037+0000 7fccb77fe640 1 -- 172.21.15.5:0/3571938413 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccac018990 con 0x7fccb81049a0 2024-09-12T03:01:26.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.037+0000 7fccc034d640 1 -- 172.21.15.5:0/3571938413 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccb819f880 con 0x7fccb81049a0 2024-09-12T03:01:26.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.037+0000 7fccc034d640 1 -- 172.21.15.5:0/3571938413 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccb81a7160 con 0x7fccb81049a0 2024-09-12T03:01:26.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.040+0000 7fccb77fe640 1 -- 172.21.15.5:0/3571938413 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fccac010ac0 con 0x7fccb81049a0 2024-09-12T03:01:26.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.040+0000 7fccc2726740 1 Processor -- start 2024-09-12T03:01:26.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.041+0000 7fccc2726740 1 -- start start 2024-09-12T03:01:26.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.041+0000 7fccb77fe640 1 --2- 172.21.15.5:0/3571938413 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fcca00a2c00 0x7fcca00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:26.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.041+0000 7fccc2726740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55be67d814a0 0x55be67d81880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:26.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.041+0000 7fccc2726740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55be67d81dc0 0x55be67d8c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:26.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.041+0000 7fccb77fe640 1 -- 172.21.15.5:0/3571938413 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fccac0cf210 con 0x7fccb81049a0 2024-09-12T03:01:26.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.041+0000 7fccc2726740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55be67d8c680 0x55be67d8ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:26.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.041+0000 7fccc2726740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55be67ce72e0 con 0x55be67d8c680 2024-09-12T03:01:26.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.041+0000 7fccc2726740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55be67c8ece0 con 0x55be67d81dc0 2024-09-12T03:01:26.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.041+0000 7fccc2726740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55be67ce32a0 con 0x55be67d814a0 2024-09-12T03:01:26.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.041+0000 7fccbe8ea640 1 --2- 172.21.15.5:0/3571938413 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fcca00a2c00 0x7fcca00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:26.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.041+0000 7fccbe8ea640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55be67d8c680 0x55be67d8ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:26.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.041+0000 7fccbe0e9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55be67d814a0 0x55be67d81880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:26.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.041+0000 7fccbd8e8640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55be67d81dc0 0x55be67d8c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:26.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.041+0000 7fccbe8ea640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55be67d8c680 0x55be67d8ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41218/0 (socket says 172.21.15.5:41218) 2024-09-12T03:01:26.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.041+0000 7fccbe8ea640 1 -- 172.21.15.5:0/1710252472 learned_addr learned my addr 172.21.15.5:0/1710252472 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:01:26.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.041+0000 7fccbd8e8640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55be67d81dc0 0x55be67d8c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.5:53284/0 (socket says 172.21.15.5:53284) 2024-09-12T03:01:26.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.042+0000 7fccbd8e8640 1 -- 172.21.15.5:0/1710252472 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55be67d814a0 msgr2=0x55be67d81880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:01:26.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.042+0000 7fccbd8e8640 1 --2- 172.21.15.5:0/1710252472 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55be67d814a0 0x55be67d81880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:26.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.042+0000 7fccbd8e8640 1 -- 172.21.15.5:0/1710252472 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55be67d8c680 msgr2=0x55be67d8ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:01:26.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.042+0000 7fccbd8e8640 1 --2- 172.21.15.5:0/1710252472 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55be67d8c680 0x55be67d8ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:26.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.042+0000 7fccbd8e8640 1 -- 172.21.15.5:0/1710252472 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be67c73440 con 0x55be67d81dc0 2024-09-12T03:01:26.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.042+0000 7fccbe8ea640 1 --2- 172.21.15.5:0/3571938413 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fcca00a2c00 0x7fcca00a50c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fcca8009a60 tx=0x7fcca8031c90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:26.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.042+0000 7fccbd8e8640 1 --2- 172.21.15.5:0/1710252472 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55be67d81dc0 0x55be67d8c140 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fccac0ccd20 tx=0x7fccac09b580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f027febfe2f8226 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:26.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.042+0000 7fccb4dea640 1 -- 172.21.15.5:0/1710252472 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccac099bd0 con 0x55be67d81dc0 2024-09-12T03:01:26.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.042+0000 7fccb4dea640 1 -- 172.21.15.5:0/1710252472 <== mon.2 v2:172.21.15.146:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fccac099e30 con 0x55be67d81dc0 2024-09-12T03:01:26.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.042+0000 7fccb4dea640 1 -- 172.21.15.5:0/1710252472 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccac09b1a0 con 0x55be67d81dc0 2024-09-12T03:01:26.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.043+0000 7fccc2726740 1 -- 172.21.15.5:0/1710252472 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55be67d81dc0 msgr2=0x55be67d8c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:26.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.043+0000 7fccc2726740 1 --2- 172.21.15.5:0/1710252472 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55be67d81dc0 0x55be67d8c140 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fccac0ccd20 tx=0x7fccac09b580 comp rx=0 tx=0).stop 2024-09-12T03:01:26.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.043+0000 7fccc2726740 1 -- 172.21.15.5:0/1710252472 shutdown_connections 2024-09-12T03:01:26.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.043+0000 7fccc2726740 1 --2- 172.21.15.5:0/1710252472 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55be67d8c680 0x55be67d8ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:26.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.043+0000 7fccc2726740 1 --2- 172.21.15.5:0/1710252472 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55be67d81dc0 0x55be67d8c140 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:26.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.043+0000 7fccc2726740 1 --2- 172.21.15.5:0/1710252472 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55be67d814a0 0x55be67d81880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:26.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.043+0000 7fccc2726740 1 -- 172.21.15.5:0/1710252472 >> 172.21.15.5:0/1710252472 conn(0x55be67d7da30 msgr2=0x55be67d7de10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:26.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.043+0000 7fccc2726740 1 -- 172.21.15.5:0/1710252472 shutdown_connections 2024-09-12T03:01:26.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.043+0000 7fccc2726740 1 -- 172.21.15.5:0/1710252472 wait complete. 2024-09-12T03:01:26.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.043+0000 7fccc2726740 1 -- start start 2024-09-12T03:01:26.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.043+0000 7fccc2726740 1 Processor -- start 2024-09-12T03:01:26.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.044+0000 7fccc2726740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55be67d814a0 0x55be67d8c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:26.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.044+0000 7fccc2726740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55be67d81dc0 0x55be67d892b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:26.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.044+0000 7fccc2726740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55be67d8c680 0x55be67d89830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:26.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.044+0000 7fccc2726740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55be67d8f1f0 con 0x55be67d814a0 2024-09-12T03:01:26.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.044+0000 7fccc2726740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55be67ca2050 con 0x55be67d8c680 2024-09-12T03:01:26.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.044+0000 7fccc2726740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55be67d8f370 con 0x55be67d81dc0 2024-09-12T03:01:26.171 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.044+0000 7fccbe0e9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55be67d814a0 0x55be67d8c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:26.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.044+0000 7fccc2726740 10 client.0 _ll_register_callbacks cb 0x55be67ce2160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:01:26.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.044+0000 7fccbe0e9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55be67d814a0 0x55be67d8c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41224/0 (socket says 172.21.15.5:41224) 2024-09-12T03:01:26.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.044+0000 7fccbe0e9640 1 -- 172.21.15.5:0/4023516946 learned_addr learned my addr 172.21.15.5:0/4023516946 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:01:26.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.044+0000 7fccbd8e8640 1 --2- 172.21.15.5:0/4023516946 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55be67d81dc0 0x55be67d892b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:26.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.044+0000 7fccbe8ea640 1 --2- 172.21.15.5:0/4023516946 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55be67d8c680 0x55be67d89830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:26.172 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.044+0000 7fccbd8e8640 1 -- 172.21.15.5:0/4023516946 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55be67d8c680 msgr2=0x55be67d89830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:01:26.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.044+0000 7fccbd8e8640 1 --2- 172.21.15.5:0/4023516946 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55be67d8c680 0x55be67d89830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:26.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.044+0000 7fccbd8e8640 1 -- 172.21.15.5:0/4023516946 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55be67d814a0 msgr2=0x55be67d8c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:01:26.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.044+0000 7fccbd8e8640 1 --2- 172.21.15.5:0/4023516946 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55be67d814a0 0x55be67d8c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:26.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.044+0000 7fccbd8e8640 1 -- 172.21.15.5:0/4023516946 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be67a583c0 con 0x55be67d81dc0 2024-09-12T03:01:26.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.044+0000 7fccbe8ea640 1 --2- 172.21.15.5:0/4023516946 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55be67d8c680 0x55be67d89830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:01:26.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.044+0000 7fccbe0e9640 1 --2- 172.21.15.5:0/4023516946 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55be67d814a0 0x55be67d8c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:01:26.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.044+0000 7fccbd8e8640 1 --2- 172.21.15.5:0/4023516946 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55be67d81dc0 0x55be67d892b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fccac0984b0 tx=0x7fccac09bcb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=943daeefea5bc4fa server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:26.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.044+0000 7fcc9dffb640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-09-12T03:01:26.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.044+0000 7fcc9dffb640 1 -- 172.21.15.5:0/4023516946 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccac09af80 con 0x55be67d81dc0 2024-09-12T03:01:26.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.045+0000 7fcc9dffb640 1 -- 172.21.15.5:0/4023516946 <== mon.1 v2:172.21.15.148:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fccac09b120 con 0x55be67d81dc0 2024-09-12T03:01:26.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.045+0000 7fccc2726740 1 -- 172.21.15.5:0/4023516946 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fccac099e30 con 0x55be67d81dc0 2024-09-12T03:01:26.173 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.045+0000 7fcc9dffb640 1 -- 172.21.15.5:0/4023516946 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fccac010430 con 0x55be67d81dc0 2024-09-12T03:01:26.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.045+0000 7fccc2726740 10 client.25693 fetch_fsmap learned FSMap version 28 2024-09-12T03:01:26.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.045+0000 7fccc2726740 1 -- 172.21.15.5:0/4023516946 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55be67d8f1f0 con 0x55be67d81dc0 2024-09-12T03:01:26.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.045+0000 7fcc9dffb640 1 -- 172.21.15.5:0/4023516946 <== mon.1 v2:172.21.15.148:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fccac099e30 con 0x55be67d81dc0 2024-09-12T03:01:26.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.045+0000 7fcc9dffb640 1 -- 172.21.15.5:0/4023516946 <== mon.1 v2:172.21.15.148:3300/0 5 ==== fsmap(e 28) ==== 8374+0+0 (secure 0 0 0) 0x7fccac0a0fc0 con 0x55be67d81dc0 2024-09-12T03:01:26.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.045+0000 7fccc2726740 10 client.25693 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T03:01:26.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.045+0000 7fccc2726740 10 client.25693 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:01:26.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.045+0000 7fccc2726740 20 client.25693 populate_metadata read hostname 'smithi005' 2024-09-12T03:01:26.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.045+0000 7fccc2726740 1 --2- 172.21.15.5:0/4023516946 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55be67d9bd60 0x55be67d9e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:26.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.045+0000 7fccc2726740 4 client.25693 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:01:26.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.045+0000 7fccc2726740 1 -- 172.21.15.5:0/4023516946 --> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55be67cf4850 con 0x55be67d9bd60 2024-09-12T03:01:26.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.045+0000 7fccbe0e9640 1 --2- 172.21.15.5:0/4023516946 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55be67d9bd60 0x55be67d9e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:26.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.046+0000 7fcc9dffb640 10 client.25693 ms_handle_connect on v2:172.21.15.148:6838/2317600000 2024-09-12T03:01:26.174 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.046+0000 7fccbe0e9640 1 --2- 172.21.15.5:0/4023516946 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55be67d9bd60 0x55be67d9e150 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ffde65f2c48df2b server_cookie=77ba2579dcfe0652 in_seq=0 out_seq=0 2024-09-12T03:01:26.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.046+0000 7fcc9dffb640 1 -- 172.21.15.5:0/4023516946 <== mds.0 v2:172.21.15.148:6838/2317600000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55be67cf4850 con 0x55be67d9bd60 2024-09-12T03:01:26.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.046+0000 7fcc9dffb640 10 client.25693 handle_command_reply: tid=0 2024-09-12T03:01:26.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.046+0000 7fccc2726740 1 client.25693 shutdown 2024-09-12T03:01:26.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.047+0000 7fccc2726740 1 -- 172.21.15.5:0/4023516946 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55be67d81dc0 msgr2=0x55be67d892b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:26.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.047+0000 7fccc2726740 1 --2- 172.21.15.5:0/4023516946 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55be67d81dc0 0x55be67d892b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fccac0984b0 tx=0x7fccac09bcb0 comp rx=0 tx=0).stop 2024-09-12T03:01:26.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.047+0000 7fccc2726740 1 -- 172.21.15.5:0/4023516946 shutdown_connections 2024-09-12T03:01:26.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.047+0000 7fccc2726740 1 --2- 172.21.15.5:0/4023516946 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55be67d8c680 0x55be67d89830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:26.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.047+0000 7fccc2726740 1 --2- 172.21.15.5:0/4023516946 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55be67d81dc0 0x55be67d892b0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:26.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.047+0000 7fccc2726740 1 --2- 172.21.15.5:0/4023516946 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55be67d9bd60 0x55be67d9e150 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:26.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.047+0000 7fccc2726740 1 --2- 172.21.15.5:0/4023516946 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55be67d814a0 0x55be67d8c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:26.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.047+0000 7fccc2726740 1 -- 172.21.15.5:0/4023516946 >> 172.21.15.5:0/4023516946 conn(0x55be67d7da30 msgr2=0x55be67d83710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:26.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.047+0000 7fcc9dffb640 0 client.25693 ms_handle_reset on v2:172.21.15.148:6838/2317600000 2024-09-12T03:01:26.175 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.047+0000 7fccc2726740 1 -- 172.21.15.5:0/4023516946 shutdown_connections 2024-09-12T03:01:26.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.047+0000 7fccc2726740 1 -- 172.21.15.5:0/4023516946 wait complete. 2024-09-12T03:01:26.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.047+0000 7fccc2726740 20 client.25693 trim_cache size 0 max 16384 2024-09-12T03:01:26.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.066+0000 7fccc2726740 1 Processor -- start 2024-09-12T03:01:26.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.066+0000 7fccc2726740 1 -- start start 2024-09-12T03:01:26.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.066+0000 7fccc2726740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55be67d814a0 0x55be67d98450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:26.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.066+0000 7fccc2726740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55be67d81dc0 0x55be67da3910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:26.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.066+0000 7fccc2726740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55be67d8c680 0x55be67da3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:26.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.066+0000 7fccc2726740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fccb81ab420 con 0x55be67d81dc0 2024-09-12T03:01:26.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.066+0000 7fccc2726740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fccb81acff0 con 0x55be67d8c680 2024-09-12T03:01:26.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.066+0000 7fccc2726740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fccb81abaf0 con 0x55be67d814a0 2024-09-12T03:01:26.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.066+0000 7fccbd8e8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55be67d81dc0 0x55be67da3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:26.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.066+0000 7fccbd8e8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55be67d81dc0 0x55be67da3910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41226/0 (socket says 172.21.15.5:41226) 2024-09-12T03:01:26.176 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.066+0000 7fccbd8e8640 1 -- 172.21.15.5:0/302856270 learned_addr learned my addr 172.21.15.5:0/302856270 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:01:26.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.066+0000 7fccbe0e9640 1 --2- 172.21.15.5:0/302856270 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55be67d814a0 0x55be67d98450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:26.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.066+0000 7fccbe8ea640 1 --2- 172.21.15.5:0/302856270 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55be67d8c680 0x55be67da3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:26.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.067+0000 7fccbd8e8640 1 -- 172.21.15.5:0/302856270 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55be67d814a0 msgr2=0x55be67d98450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:01:26.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.067+0000 7fccbd8e8640 1 --2- 172.21.15.5:0/302856270 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55be67d814a0 0x55be67d98450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:26.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.067+0000 7fccbd8e8640 1 -- 172.21.15.5:0/302856270 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55be67d8c680 msgr2=0x55be67da3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:01:26.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.067+0000 7fccbd8e8640 1 --2- 172.21.15.5:0/302856270 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55be67d8c680 0x55be67da3e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:26.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.067+0000 7fccbd8e8640 1 -- 172.21.15.5:0/302856270 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be67d89ba0 con 0x55be67d81dc0 2024-09-12T03:01:26.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.067+0000 7fccbe0e9640 1 --2- 172.21.15.5:0/302856270 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55be67d814a0 0x55be67d98450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:01:26.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.067+0000 7fccbd8e8640 1 --2- 172.21.15.5:0/302856270 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55be67d81dc0 0x55be67da3910 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fccac098500 tx=0x7fccac0ccd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=735677c3164925eb server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:26.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.067+0000 7fcc9ffff640 1 -- 172.21.15.5:0/302856270 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccac018c70 con 0x55be67d81dc0 2024-09-12T03:01:26.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.067+0000 7fcc9ffff640 1 -- 172.21.15.5:0/302856270 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fccac0aa050 con 0x55be67d81dc0 2024-09-12T03:01:26.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.067+0000 7fcc9ffff640 1 -- 172.21.15.5:0/302856270 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccac09b490 con 0x55be67d81dc0 2024-09-12T03:01:26.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.068+0000 7fccc2726740 1 -- 172.21.15.5:0/302856270 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55be67d81dc0 msgr2=0x55be67da3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:26.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.068+0000 7fccc2726740 1 --2- 172.21.15.5:0/302856270 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55be67d81dc0 0x55be67da3910 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fccac098500 tx=0x7fccac0ccd70 comp rx=0 tx=0).stop 2024-09-12T03:01:26.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.068+0000 7fccc2726740 1 -- 172.21.15.5:0/302856270 shutdown_connections 2024-09-12T03:01:26.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.068+0000 7fccc2726740 1 --2- 172.21.15.5:0/302856270 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55be67d81dc0 0x55be67da3910 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:26.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.068+0000 7fccc2726740 1 --2- 172.21.15.5:0/302856270 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55be67d8c680 0x55be67da3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:26.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.068+0000 7fccc2726740 1 --2- 172.21.15.5:0/302856270 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55be67d814a0 0x55be67d98450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:26.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.068+0000 7fccc2726740 1 -- 172.21.15.5:0/302856270 >> 172.21.15.5:0/302856270 conn(0x55be67d7da30 msgr2=0x55be67d8b330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:26.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.068+0000 7fccc2726740 1 -- 172.21.15.5:0/302856270 shutdown_connections 2024-09-12T03:01:26.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.068+0000 7fccc2726740 1 -- 172.21.15.5:0/302856270 wait complete. 2024-09-12T03:01:26.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.068+0000 7fccc2726740 1 -- start start 2024-09-12T03:01:26.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.068+0000 7fccc2726740 1 Processor -- start 2024-09-12T03:01:26.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.069+0000 7fccc2726740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55be67d814a0 0x55be67d91a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:26.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.069+0000 7fccc2726740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55be67d81dc0 0x55be67d91fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:26.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.069+0000 7fccc2726740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55be67d8c680 0x55be67e048e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:26.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.069+0000 7fccc2726740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fccb81ac890 con 0x55be67d814a0 2024-09-12T03:01:26.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.069+0000 7fccc2726740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fccb81ac1c0 con 0x55be67d8c680 2024-09-12T03:01:26.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.069+0000 7fccc2726740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fccb81a9400 con 0x55be67d81dc0 2024-09-12T03:01:26.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.069+0000 7fccc2726740 10 client.0 _ll_register_callbacks cb 0x7fccb0005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:01:26.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.069+0000 7fccbd8e8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55be67d81dc0 0x55be67d91fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:26.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.069+0000 7fccbe0e9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55be67d814a0 0x55be67d91a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:26.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.069+0000 7fccbd8e8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55be67d81dc0 0x55be67d91fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.5:59134/0 (socket says 172.21.15.5:59134) 2024-09-12T03:01:26.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.069+0000 7fccbd8e8640 1 -- 172.21.15.5:0/4066607037 learned_addr learned my addr 172.21.15.5:0/4066607037 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:01:26.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.069+0000 7fccbe8ea640 1 --2- 172.21.15.5:0/4066607037 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55be67d8c680 0x55be67e048e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:26.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.069+0000 7fccbd8e8640 1 -- 172.21.15.5:0/4066607037 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55be67d8c680 msgr2=0x55be67e048e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:01:26.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.069+0000 7fccbd8e8640 1 --2- 172.21.15.5:0/4066607037 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55be67d8c680 0x55be67e048e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:26.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.069+0000 7fccbd8e8640 1 -- 172.21.15.5:0/4066607037 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55be67d814a0 msgr2=0x55be67d91a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:01:26.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.069+0000 7fccbd8e8640 1 --2- 172.21.15.5:0/4066607037 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55be67d814a0 0x55be67d91a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:26.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.069+0000 7fccbd8e8640 1 -- 172.21.15.5:0/4066607037 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be67e04e20 con 0x55be67d81dc0 2024-09-12T03:01:26.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.069+0000 7fccbd8e8640 1 --2- 172.21.15.5:0/4066607037 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55be67d81dc0 0x55be67d91fc0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fccac0984b0 tx=0x7fccac09bcb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f6dd8ae8ba2f3f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:26.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.069+0000 7fcc9dffb640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-09-12T03:01:26.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.069+0000 7fcc9dffb640 1 -- 172.21.15.5:0/4066607037 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccac09b490 con 0x55be67d81dc0 2024-09-12T03:01:26.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.070+0000 7fccc2726740 1 -- 172.21.15.5:0/4066607037 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fccac0aa050 con 0x55be67d81dc0 2024-09-12T03:01:26.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.070+0000 7fcc9dffb640 1 -- 172.21.15.5:0/4066607037 <== mon.1 v2:172.21.15.148:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fccac086130 con 0x55be67d81dc0 2024-09-12T03:01:26.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.070+0000 7fcc9dffb640 1 -- 172.21.15.5:0/4066607037 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fccac018c70 con 0x55be67d81dc0 2024-09-12T03:01:26.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.070+0000 7fccc2726740 10 client.25699 fetch_fsmap learned FSMap version 28 2024-09-12T03:01:26.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.070+0000 7fccc2726740 1 -- 172.21.15.5:0/4066607037 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55be67e04e20 con 0x55be67d81dc0 2024-09-12T03:01:26.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.070+0000 7fcc9dffb640 1 -- 172.21.15.5:0/4066607037 <== mon.1 v2:172.21.15.148:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fccac0aa050 con 0x55be67d81dc0 2024-09-12T03:01:26.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.070+0000 7fcc9dffb640 1 -- 172.21.15.5:0/4066607037 <== mon.1 v2:172.21.15.148:3300/0 5 ==== fsmap(e 28) ==== 8374+0+0 (secure 0 0 0) 0x7fccac09a9f0 con 0x55be67d81dc0 2024-09-12T03:01:26.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.070+0000 7fccc2726740 10 client.25699 fetch_fsmap finished waiting for FSMap version 28 2024-09-12T03:01:26.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.070+0000 7fccc2726740 10 client.25699 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:01:26.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.070+0000 7fccc2726740 20 client.25699 populate_metadata read hostname 'smithi005' 2024-09-12T03:01:26.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.070+0000 7fccc2726740 1 --2- 172.21.15.5:0/4066607037 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55be67e06460 0x55be67e08880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:26.181 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-12T03:01:26.181 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-12T03:01:26.181 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-09-12T03:01:26.181 INFO:teuthology.orchestra.run.smithi005.stdout: "af9de749-3eeb-4965-a2b0-fb23758c2c85": { 2024-09-12T03:01:26.181 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-09-12T03:01:26.181 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "af9de749-3eeb-4965-a2b0-fb23758c2c85", 2024-09-12T03:01:26.181 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-09-12T03:01:26.181 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:01:26.182 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:01:26.182 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-12T03:01:26.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.070+0000 7fccc2726740 4 client.25699 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T03:01:26.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.070+0000 7fccc2726740 1 -- 172.21.15.5:0/4066607037 --> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55be67ceed60 con 0x55be67e06460 2024-09-12T03:01:26.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.070+0000 7fccbe0e9640 1 --2- 172.21.15.5:0/4066607037 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55be67e06460 0x55be67e08880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:26.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.071+0000 7fccbe0e9640 1 --2- 172.21.15.5:0/4066607037 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55be67e06460 0x55be67e08880 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d80a841fd4b1b0d server_cookie=6470567204b70239 in_seq=0 out_seq=0 2024-09-12T03:01:26.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.071+0000 7fcc9dffb640 10 client.25699 ms_handle_connect on v2:172.21.15.148:6838/2317600000 2024-09-12T03:01:26.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.071+0000 7fcc9dffb640 1 -- 172.21.15.5:0/4066607037 <== mds.0 v2:172.21.15.148:6838/2317600000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55be67ceed60 con 0x55be67e06460 2024-09-12T03:01:26.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.071+0000 7fcc9dffb640 10 client.25699 handle_command_reply: tid=0 2024-09-12T03:01:26.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.071+0000 7fccc2726740 1 client.25699 shutdown 2024-09-12T03:01:26.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.071+0000 7fccc2726740 1 -- 172.21.15.5:0/4066607037 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55be67d81dc0 msgr2=0x55be67d91fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:26.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.071+0000 7fccc2726740 1 --2- 172.21.15.5:0/4066607037 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55be67d81dc0 0x55be67d91fc0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fccac0984b0 tx=0x7fccac09bcb0 comp rx=0 tx=0).stop 2024-09-12T03:01:26.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.071+0000 7fccc2726740 1 -- 172.21.15.5:0/4066607037 shutdown_connections 2024-09-12T03:01:26.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.071+0000 7fccc2726740 1 --2- 172.21.15.5:0/4066607037 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55be67d8c680 0x55be67e048e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:26.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.071+0000 7fccc2726740 1 --2- 172.21.15.5:0/4066607037 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55be67d81dc0 0x55be67d91fc0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:26.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.071+0000 7fccc2726740 1 --2- 172.21.15.5:0/4066607037 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55be67e06460 0x55be67e08880 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:26.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.071+0000 7fccc2726740 1 --2- 172.21.15.5:0/4066607037 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55be67d814a0 0x55be67d91a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:26.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.071+0000 7fcc9dffb640 0 client.25699 ms_handle_reset on v2:172.21.15.148:6838/2317600000 2024-09-12T03:01:26.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.071+0000 7fccc2726740 1 -- 172.21.15.5:0/4066607037 >> 172.21.15.5:0/4066607037 conn(0x55be67d7da30 msgr2=0x55be67d7c350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:26.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.071+0000 7fccc2726740 1 -- 172.21.15.5:0/4066607037 shutdown_connections 2024-09-12T03:01:26.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.071+0000 7fccc2726740 1 -- 172.21.15.5:0/4066607037 wait complete. 2024-09-12T03:01:26.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.071+0000 7fccc2726740 20 client.25699 trim_cache size 0 max 16384 2024-09-12T03:01:26.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.072+0000 7fccc034d640 1 -- 172.21.15.5:0/3571938413 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fcca00a2c00 msgr2=0x7fcca00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:26.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.072+0000 7fccc034d640 1 --2- 172.21.15.5:0/3571938413 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fcca00a2c00 0x7fcca00a50c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fcca8009a60 tx=0x7fcca8031c90 comp rx=0 tx=0).stop 2024-09-12T03:01:26.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.072+0000 7fccc034d640 1 -- 172.21.15.5:0/3571938413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fccb81049a0 msgr2=0x7fccb819ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:26.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.072+0000 7fccc034d640 1 --2- 172.21.15.5:0/3571938413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fccb81049a0 0x7fccb819ab20 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fccac00e9f0 tx=0x7fccac00eeb0 comp rx=0 tx=0).stop 2024-09-12T03:01:26.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.072+0000 7fccc034d640 1 -- 172.21.15.5:0/3571938413 shutdown_connections 2024-09-12T03:01:26.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.072+0000 7fccc034d640 1 --2- 172.21.15.5:0/3571938413 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fcca00a2c00 0x7fcca00a50c0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:26.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.072+0000 7fccc034d640 1 --2- 172.21.15.5:0/3571938413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fccb81049a0 0x7fccb819ab20 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:26.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.072+0000 7fccc034d640 1 --2- 172.21.15.5:0/3571938413 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fccb8109120 0x7fccb819eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:26.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.072+0000 7fccc034d640 1 --2- 172.21.15.5:0/3571938413 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fccb8103ff0 0x7fccb819a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:26.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.072+0000 7fccc034d640 1 -- 172.21.15.5:0/3571938413 >> 172.21.15.5:0/3571938413 conn(0x7fccb80ffab0 msgr2=0x7fccb8100d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:26.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.072+0000 7fccc034d640 1 -- 172.21.15.5:0/3571938413 shutdown_connections 2024-09-12T03:01:26.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:26.072+0000 7fccc034d640 1 -- 172.21.15.5:0/3571938413 wait complete. 2024-09-12T03:01:26.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:01:26 smithi146 ceph-mon[35011]: scrub summary: active paths [/] 2024-09-12T03:01:26.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:01:26 smithi146 ceph-mon[35011]: scrub summary: idle+waiting paths [/] 2024-09-12T03:01:26.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:01:26 smithi146 ceph-mon[35011]: scrub summary: idle+waiting paths [/] 2024-09-12T03:01:26.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:01:26 smithi005 ceph-mon[31197]: scrub summary: active paths [/] 2024-09-12T03:01:26.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:01:26 smithi005 ceph-mon[31197]: scrub summary: idle+waiting paths [/] 2024-09-12T03:01:26.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:01:26 smithi005 ceph-mon[31197]: scrub summary: idle+waiting paths [/] 2024-09-12T03:01:26.915 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:01:26 smithi148 ceph-mon[35090]: scrub summary: active paths [/] 2024-09-12T03:01:26.915 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:01:26 smithi148 ceph-mon[35090]: scrub summary: idle+waiting paths [/] 2024-09-12T03:01:26.915 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:01:26 smithi148 ceph-mon[35090]: scrub summary: idle+waiting paths [/] 2024-09-12T03:01:26.970 INFO:tasks.cephfs.filesystem:scrub status for tag:af9de749-3eeb-4965-a2b0-fb23758c2c85 - {'path': '/', 'tag': 'af9de749-3eeb-4965-a2b0-fb23758c2c85', 'options': 'recursive,force'} 2024-09-12T03:01:27.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:01:27 smithi146 ceph-mon[35011]: pgmap v431: 97 pgs: 97 active+clean; 89 MiB data, 816 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.8 MiB/s wr, 719 op/s 2024-09-12T03:01:27.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:01:27 smithi005 ceph-mon[31197]: pgmap v431: 97 pgs: 97 active+clean; 89 MiB data, 816 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.8 MiB/s wr, 719 op/s 2024-09-12T03:01:27.915 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:01:27 smithi148 ceph-mon[35090]: pgmap v431: 97 pgs: 97 active+clean; 89 MiB data, 816 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.8 MiB/s wr, 719 op/s 2024-09-12T03:01:29.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:01:29 smithi146 ceph-mon[35011]: pgmap v432: 97 pgs: 97 active+clean; 91 MiB data, 822 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.8 MiB/s wr, 649 op/s 2024-09-12T03:01:29.803 INFO:tasks.workunit.client.0.smithi005.stdout:copying to largest ever: 0xffdb8 2024-09-12T03:01:29.809 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:01:29 smithi005 ceph-mon[31197]: pgmap v432: 97 pgs: 97 active+clean; 91 MiB data, 822 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.8 MiB/s wr, 649 op/s 2024-09-12T03:01:29.915 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:01:29 smithi148 ceph-mon[35090]: pgmap v432: 97 pgs: 97 active+clean; 91 MiB data, 822 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.8 MiB/s wr, 649 op/s 2024-09-12T03:01:31.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:01:31 smithi146 ceph-mon[35011]: pgmap v433: 97 pgs: 97 active+clean; 91 MiB data, 814 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.8 MiB/s wr, 713 op/s 2024-09-12T03:01:31.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:01:31 smithi005 ceph-mon[31197]: pgmap v433: 97 pgs: 97 active+clean; 91 MiB data, 814 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.8 MiB/s wr, 713 op/s 2024-09-12T03:01:31.915 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:01:31 smithi148 ceph-mon[35090]: pgmap v433: 97 pgs: 97 active+clean; 91 MiB data, 814 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.8 MiB/s wr, 713 op/s 2024-09-12T03:01:33.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:01:33 smithi146 ceph-mon[35011]: pgmap v434: 97 pgs: 97 active+clean; 91 MiB data, 820 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.3 MiB/s wr, 521 op/s 2024-09-12T03:01:33.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:01:33 smithi146 ceph-mon[35011]: scrub summary: idle 2024-09-12T03:01:33.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:01:33 smithi146 ceph-mon[35011]: scrub summary: idle 2024-09-12T03:01:33.707 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:01:33 smithi146 ceph-mon[35011]: scrub summary: idle 2024-09-12T03:01:33.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:01:33 smithi005 ceph-mon[31197]: pgmap v434: 97 pgs: 97 active+clean; 91 MiB data, 820 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.3 MiB/s wr, 521 op/s 2024-09-12T03:01:33.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:01:33 smithi005 ceph-mon[31197]: scrub summary: idle 2024-09-12T03:01:33.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:01:33 smithi005 ceph-mon[31197]: scrub summary: idle 2024-09-12T03:01:33.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:01:33 smithi005 ceph-mon[31197]: scrub summary: idle 2024-09-12T03:01:33.915 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:01:33 smithi148 ceph-mon[35090]: pgmap v434: 97 pgs: 97 active+clean; 91 MiB data, 820 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.3 MiB/s wr, 521 op/s 2024-09-12T03:01:33.915 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:01:33 smithi148 ceph-mon[35090]: scrub summary: idle 2024-09-12T03:01:33.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:01:33 smithi148 ceph-mon[35090]: scrub summary: idle 2024-09-12T03:01:33.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:01:33 smithi148 ceph-mon[35090]: scrub summary: idle 2024-09-12T03:01:34.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:01:34 smithi146 ceph-mon[35011]: scrub summary: idle 2024-09-12T03:01:34.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:01:34 smithi005 ceph-mon[31197]: scrub summary: idle 2024-09-12T03:01:34.915 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:01:34 smithi148 ceph-mon[35090]: scrub summary: idle 2024-09-12T03:01:35.706 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:01:35 smithi146 ceph-mon[35011]: pgmap v435: 97 pgs: 97 active+clean; 93 MiB data, 872 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.5 MiB/s wr, 608 op/s 2024-09-12T03:01:35.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:01:35 smithi005 ceph-mon[31197]: pgmap v435: 97 pgs: 97 active+clean; 93 MiB data, 872 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.5 MiB/s wr, 608 op/s 2024-09-12T03:01:35.915 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:01:35 smithi148 ceph-mon[35090]: pgmap v435: 97 pgs: 97 active+clean; 93 MiB data, 872 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.5 MiB/s wr, 608 op/s 2024-09-12T03:01:36.875 INFO:tasks.workunit.client.0.smithi005.stdout:50000 copy from 0x898b8 to 0x92015, (0x875d bytes) at 0xb142c 2024-09-12T03:01:36.875 INFO:tasks.workunit.client.0.smithi005.stdout:All 50000 operations completed A-OK! 2024-09-12T03:01:36.875 INFO:tasks.workunit.client.0.smithi005.stderr:+ ./fsx -z 10MB -N 50000 -p 10000 -l 10485760 2024-09-12T03:01:36.878 INFO:tasks.workunit.client.0.smithi005.stdout:Seed set to 1 2024-09-12T03:01:37.006 INFO:tasks.workunit.client.0.smithi005.stderr:main: filesystem does not support fallocate mode 0, disabling! 2024-09-12T03:01:37.006 INFO:tasks.workunit.client.0.smithi005.stderr:main: filesystem does not support fallocate mode FALLOC_FL_KEEP_SIZE, disabling! 2024-09-12T03:01:37.008 INFO:tasks.workunit.client.0.smithi005.stderr:main: filesystem does not support fallocate mode FALLOC_FL_COLLAPSE_RANGE, disabling! 2024-09-12T03:01:37.008 INFO:tasks.workunit.client.0.smithi005.stderr:main: filesystem does not support fallocate mode FALLOC_FL_INSERT_RANGE, disabling! 2024-09-12T03:01:37.008 INFO:tasks.workunit.client.0.smithi005.stderr:main: filesystem does not support clone range, disabling! 2024-09-12T03:01:37.011 INFO:tasks.workunit.client.0.smithi005.stderr:main: filesystem does not support dedupe range, disabling! 2024-09-12T03:01:37.012 INFO:tasks.workunit.client.0.smithi005.stdout:skipping zero length punch hole 2024-09-12T03:01:37.013 INFO:tasks.workunit.client.0.smithi005.stderr:main: filesystem does not support exchange range, disabling! 2024-09-12T03:01:37.013 INFO:tasks.workunit.client.0.smithi005.stdout:skipping zero length punch hole 2024-09-12T03:01:37.013 INFO:tasks.workunit.client.0.smithi005.stdout:skipping zero length punch hole 2024-09-12T03:01:37.013 INFO:tasks.workunit.client.0.smithi005.stdout:skipping zero size read 2024-09-12T03:01:37.013 INFO:tasks.workunit.client.0.smithi005.stdout:skipping zero length punch hole 2024-09-12T03:01:37.013 INFO:tasks.workunit.client.0.smithi005.stdout:skipping zero size read 2024-09-12T03:01:37.013 INFO:tasks.workunit.client.0.smithi005.stdout:skipping zero length punch hole 2024-09-12T03:01:37.014 INFO:tasks.workunit.client.0.smithi005.stdout:truncating to largest ever: 0x4310e9 2024-09-12T03:01:37.018 INFO:tasks.workunit.client.0.smithi005.stdout:copying to largest ever: 0x954a45 2024-09-12T03:01:37.046 INFO:tasks.workunit.client.0.smithi005.stdout:truncating to largest ever: 0x9bc582 2024-09-12T03:01:37.069 INFO:tasks.workunit.client.0.smithi005.stdout:truncating to largest ever: 0x9eecbc 2024-09-12T03:01:37.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:01:37 smithi005 ceph-mon[31197]: pgmap v436: 97 pgs: 97 active+clean; 93 MiB data, 868 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.4 MiB/s wr, 666 op/s 2024-09-12T03:01:37.915 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:01:37 smithi148 ceph-mon[35090]: pgmap v436: 97 pgs: 97 active+clean; 93 MiB data, 868 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.4 MiB/s wr, 666 op/s 2024-09-12T03:01:37.956 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:01:37 smithi146 ceph-mon[35011]: pgmap v436: 97 pgs: 97 active+clean; 93 MiB data, 868 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.4 MiB/s wr, 666 op/s 2024-09-12T03:01:38.778 INFO:tasks.workunit.client.0.smithi005.stdout:copying to largest ever: 0x9fdd84 2024-09-12T03:01:39.758 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:01:39 smithi146 ceph-mon[35011]: pgmap v437: 97 pgs: 97 active+clean; 98 MiB data, 926 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.2 MiB/s wr, 589 op/s 2024-09-12T03:01:39.758 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:01:39 smithi146 ceph-mon[35011]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:01:39.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:01:39 smithi005 ceph-mon[31197]: pgmap v437: 97 pgs: 97 active+clean; 98 MiB data, 926 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.2 MiB/s wr, 589 op/s 2024-09-12T03:01:39.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:01:39 smithi005 ceph-mon[31197]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:01:39.915 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:01:39 smithi148 ceph-mon[35090]: pgmap v437: 97 pgs: 97 active+clean; 98 MiB data, 926 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.2 MiB/s wr, 589 op/s 2024-09-12T03:01:39.915 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:01:39 smithi148 ceph-mon[35090]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:01:41.717 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:01:41 smithi148 ceph-mon[35090]: pgmap v438: 97 pgs: 97 active+clean; 99 MiB data, 975 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.9 MiB/s wr, 632 op/s 2024-09-12T03:01:41.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:01:41 smithi005 ceph-mon[31197]: pgmap v438: 97 pgs: 97 active+clean; 99 MiB data, 975 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.9 MiB/s wr, 632 op/s 2024-09-12T03:01:41.956 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:01:41 smithi146 ceph-mon[35011]: pgmap v438: 97 pgs: 97 active+clean; 99 MiB data, 975 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.9 MiB/s wr, 632 op/s 2024-09-12T03:01:43.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:01:43 smithi005 ceph-mon[31197]: pgmap v439: 97 pgs: 97 active+clean; 99 MiB data, 983 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.8 MiB/s wr, 507 op/s 2024-09-12T03:01:43.915 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:01:43 smithi148 ceph-mon[35090]: pgmap v439: 97 pgs: 97 active+clean; 99 MiB data, 983 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.8 MiB/s wr, 507 op/s 2024-09-12T03:01:43.956 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:01:43 smithi146 ceph-mon[35011]: pgmap v439: 97 pgs: 97 active+clean; 99 MiB data, 983 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.8 MiB/s wr, 507 op/s 2024-09-12T03:01:45.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:01:45 smithi005 ceph-mon[31197]: pgmap v440: 97 pgs: 97 active+clean; 100 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 3.6 MiB/s wr, 707 op/s 2024-09-12T03:01:45.915 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:01:45 smithi148 ceph-mon[35090]: pgmap v440: 97 pgs: 97 active+clean; 100 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 3.6 MiB/s wr, 707 op/s 2024-09-12T03:01:45.956 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:01:45 smithi146 ceph-mon[35011]: pgmap v440: 97 pgs: 97 active+clean; 100 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 3.6 MiB/s wr, 707 op/s 2024-09-12T03:01:47.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:01:47 smithi005 ceph-mon[31197]: pgmap v441: 97 pgs: 97 active+clean; 104 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 4.8 MiB/s wr, 859 op/s 2024-09-12T03:01:47.915 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:01:47 smithi148 ceph-mon[35090]: pgmap v441: 97 pgs: 97 active+clean; 104 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 4.8 MiB/s wr, 859 op/s 2024-09-12T03:01:47.956 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:01:47 smithi146 ceph-mon[35011]: pgmap v441: 97 pgs: 97 active+clean; 104 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 4.8 MiB/s wr, 859 op/s 2024-09-12T03:01:48.608 INFO:tasks.workunit.client.0.smithi005.stdout:copying to largest ever: 0x9ff773 2024-09-12T03:01:48.968 INFO:tasks.workunit.client.0.smithi005.stdout:20000 punch from 0x34ddcb to 0x35324f, (0x5484 bytes) 2024-09-12T03:01:49.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:01:49 smithi005 ceph-mon[31197]: pgmap v442: 97 pgs: 97 active+clean; 104 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 4.3 MiB/s wr, 804 op/s 2024-09-12T03:01:49.915 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:01:49 smithi148 ceph-mon[35090]: pgmap v442: 97 pgs: 97 active+clean; 104 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 4.3 MiB/s wr, 804 op/s 2024-09-12T03:01:49.956 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:01:49 smithi146 ceph-mon[35011]: pgmap v442: 97 pgs: 97 active+clean; 104 MiB data, 1005 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 4.3 MiB/s wr, 804 op/s 2024-09-12T03:01:51.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:01:51 smithi005 ceph-mon[31197]: pgmap v443: 97 pgs: 97 active+clean; 105 MiB data, 991 MiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 5.0 MiB/s wr, 899 op/s 2024-09-12T03:01:51.915 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:01:51 smithi148 ceph-mon[35090]: pgmap v443: 97 pgs: 97 active+clean; 105 MiB data, 991 MiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 5.0 MiB/s wr, 899 op/s 2024-09-12T03:01:51.956 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:01:51 smithi146 ceph-mon[35011]: pgmap v443: 97 pgs: 97 active+clean; 105 MiB data, 991 MiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 5.0 MiB/s wr, 899 op/s 2024-09-12T03:01:53.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:01:53 smithi005 ceph-mon[31197]: pgmap v444: 97 pgs: 97 active+clean; 104 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.3 MiB/s wr, 813 op/s 2024-09-12T03:01:53.915 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:01:53 smithi148 ceph-mon[35090]: pgmap v444: 97 pgs: 97 active+clean; 104 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.3 MiB/s wr, 813 op/s 2024-09-12T03:01:53.956 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:01:53 smithi146 ceph-mon[35011]: pgmap v444: 97 pgs: 97 active+clean; 104 MiB data, 1010 MiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.3 MiB/s wr, 813 op/s 2024-09-12T03:01:55.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:01:55 smithi005 ceph-mon[31197]: pgmap v445: 97 pgs: 97 active+clean; 106 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.9 MiB/s wr, 927 op/s 2024-09-12T03:01:55.915 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:01:55 smithi148 ceph-mon[35090]: pgmap v445: 97 pgs: 97 active+clean; 106 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.9 MiB/s wr, 927 op/s 2024-09-12T03:01:55.956 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:01:55 smithi146 ceph-mon[35011]: pgmap v445: 97 pgs: 97 active+clean; 106 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.9 MiB/s wr, 927 op/s 2024-09-12T03:01:56.071 INFO:tasks.workunit.client.0.smithi005.stdout:30000 trunc from 0x7a5444 to 0x143a5 2024-09-12T03:01:56.971 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T03:01:57.256 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:01:57.691 INFO:tasks.quiescer.fs.[cephfs]:Total duration (40.3) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-12T03:01:57.691 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 20.15 2024-09-12T03:01:57.692 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:01:57.702 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:01:57 smithi005 ceph-mon[31197]: pgmap v446: 97 pgs: 97 active+clean; 110 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.9 MiB/s wr, 862 op/s 2024-09-12T03:01:57.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.813+0000 7f87171e0640 1 -- 172.21.15.5:0/3388671083 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f87101082a0 msgr2=0x7f871010ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:57.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.813+0000 7f87171e0640 1 --2- 172.21.15.5:0/3388671083 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f87101082a0 0x7f871010ec40 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8708009ca0 tx=0x7f870802a8c0 comp rx=0 tx=0).stop 2024-09-12T03:01:57.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.813+0000 7f87171e0640 1 -- 172.21.15.5:0/3388671083 shutdown_connections 2024-09-12T03:01:57.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.813+0000 7f87171e0640 1 --2- 172.21.15.5:0/3388671083 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f87101036e0 0x7f8710107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:57.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.813+0000 7f87171e0640 1 --2- 172.21.15.5:0/3388671083 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f87101082a0 0x7f871010ec40 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:57.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.813+0000 7f87171e0640 1 --2- 172.21.15.5:0/3388671083 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8710102d30 0x7f8710103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:57.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.813+0000 7f87171e0640 1 -- 172.21.15.5:0/3388671083 >> 172.21.15.5:0/3388671083 conn(0x7f87100fc7d0 msgr2=0x7f87100febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:57.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.813+0000 7f87171e0640 1 -- 172.21.15.5:0/3388671083 shutdown_connections 2024-09-12T03:01:57.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.813+0000 7f87171e0640 1 -- 172.21.15.5:0/3388671083 wait complete. 2024-09-12T03:01:57.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.814+0000 7f87171e0640 1 Processor -- start 2024-09-12T03:01:57.837 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.814+0000 7f87171e0640 1 -- start start 2024-09-12T03:01:57.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.814+0000 7f87171e0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8710102d30 0x7f87101984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:57.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.814+0000 7f87171e0640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f87101036e0 0x7f8710198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:57.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.814+0000 7f87171e0640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f87101082a0 0x7f871019cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:57.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.814+0000 7f87171e0640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f87101120a0 con 0x7f8710102d30 2024-09-12T03:01:57.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.814+0000 7f87171e0640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f8710111f20 con 0x7f87101082a0 2024-09-12T03:01:57.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.814+0000 7f87171e0640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8710112220 con 0x7f87101036e0 2024-09-12T03:01:57.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.815+0000 7f870ffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f87101036e0 0x7f8710198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:57.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.815+0000 7f8714f7c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8710102d30 0x7f87101984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:57.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.815+0000 7f8714f7c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8710102d30 0x7f87101984e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51398/0 (socket says 172.21.15.5:51398) 2024-09-12T03:01:57.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.815+0000 7f8714f7c640 1 -- 172.21.15.5:0/1696752340 learned_addr learned my addr 172.21.15.5:0/1696752340 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:01:57.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.815+0000 7f871577d640 1 --2- 172.21.15.5:0/1696752340 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f87101082a0 0x7f871019cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:57.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.815+0000 7f870ffff640 1 -- 172.21.15.5:0/1696752340 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f87101082a0 msgr2=0x7f871019cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:57.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.815+0000 7f870ffff640 1 --2- 172.21.15.5:0/1696752340 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f87101082a0 0x7f871019cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:57.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.815+0000 7f870ffff640 1 -- 172.21.15.5:0/1696752340 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8710102d30 msgr2=0x7f87101984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:57.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.815+0000 7f870ffff640 1 --2- 172.21.15.5:0/1696752340 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8710102d30 0x7f87101984e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:57.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.815+0000 7f870ffff640 1 -- 172.21.15.5:0/1696752340 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f871019d490 con 0x7f87101036e0 2024-09-12T03:01:57.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.815+0000 7f8714f7c640 1 --2- 172.21.15.5:0/1696752340 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8710102d30 0x7f87101984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:01:57.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.815+0000 7f870ffff640 1 --2- 172.21.15.5:0/1696752340 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f87101036e0 0x7f8710198a20 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f87100697d0 tx=0x7f86fc010050 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:57.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.815+0000 7f870dffb640 1 -- 172.21.15.5:0/1696752340 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86fc00e2e0 con 0x7f87101036e0 2024-09-12T03:01:57.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.816+0000 7f870dffb640 1 -- 172.21.15.5:0/1696752340 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86fc00e480 con 0x7f87101036e0 2024-09-12T03:01:57.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.816+0000 7f870dffb640 1 -- 172.21.15.5:0/1696752340 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86fc0173f0 con 0x7f87101036e0 2024-09-12T03:01:57.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.816+0000 7f87171e0640 1 -- 172.21.15.5:0/1696752340 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f871019d780 con 0x7f87101036e0 2024-09-12T03:01:57.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.816+0000 7f87171e0640 1 -- 172.21.15.5:0/1696752340 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87101008c0 con 0x7f87101036e0 2024-09-12T03:01:57.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.820+0000 7f87195b9740 1 Processor -- start 2024-09-12T03:01:57.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.820+0000 7f87195b9740 1 -- start start 2024-09-12T03:01:57.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.820+0000 7f87195b9740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562410be44a0 0x562410be4880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:57.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.820+0000 7f87195b9740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562410be4dc0 0x562410bef140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:57.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.820+0000 7f870dffb640 1 -- 172.21.15.5:0/1696752340 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f86fc010820 con 0x7f87101036e0 2024-09-12T03:01:57.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.820+0000 7f87195b9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562410bef680 0x562410bf1a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:57.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.820+0000 7f87195b9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562410b4a2e0 con 0x562410bef680 2024-09-12T03:01:57.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.820+0000 7f87195b9740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x562410af1ce0 con 0x562410be44a0 2024-09-12T03:01:57.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.820+0000 7f87195b9740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x562410b462a0 con 0x562410be4dc0 2024-09-12T03:01:57.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.820+0000 7f8714f7c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562410be4dc0 0x562410bef140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:57.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.820+0000 7f8714f7c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562410be4dc0 0x562410bef140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.5:42584/0 (socket says 172.21.15.5:42584) 2024-09-12T03:01:57.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.820+0000 7f8714f7c640 1 -- 172.21.15.5:0/645902017 learned_addr learned my addr 172.21.15.5:0/645902017 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:01:57.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.820+0000 7f870ffff640 1 --2- 172.21.15.5:0/645902017 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562410bef680 0x562410bf1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:57.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.820+0000 7f871577d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562410be44a0 0x562410be4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:57.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.821+0000 7f8714f7c640 1 -- 172.21.15.5:0/645902017 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562410be44a0 msgr2=0x562410be4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:01:57.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.821+0000 7f8714f7c640 1 --2- 172.21.15.5:0/645902017 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562410be44a0 0x562410be4880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:57.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.821+0000 7f8714f7c640 1 -- 172.21.15.5:0/645902017 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562410bef680 msgr2=0x562410bf1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:01:57.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.821+0000 7f8714f7c640 1 --2- 172.21.15.5:0/645902017 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562410bef680 0x562410bf1a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:57.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.821+0000 7f8714f7c640 1 -- 172.21.15.5:0/645902017 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562410ad6440 con 0x562410be4dc0 2024-09-12T03:01:57.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.821+0000 7f870ffff640 1 --2- 172.21.15.5:0/645902017 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562410bef680 0x562410bf1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:01:57.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.821+0000 7f8714f7c640 1 --2- 172.21.15.5:0/645902017 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562410be4dc0 0x562410bef140 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f8704011650 tx=0x7f8704009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=71aebcb9471736fa server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:57.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.821+0000 7f86faffd640 1 -- 172.21.15.5:0/645902017 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f870401e070 con 0x562410be4dc0 2024-09-12T03:01:57.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.821+0000 7f86faffd640 1 -- 172.21.15.5:0/645902017 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f870400fba0 con 0x562410be4dc0 2024-09-12T03:01:57.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.821+0000 7f86faffd640 1 -- 172.21.15.5:0/645902017 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8704019cd0 con 0x562410be4dc0 2024-09-12T03:01:57.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.822+0000 7f870dffb640 1 --2- 172.21.15.5:0/1696752340 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f87000a2b80 0x7f87000a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:57.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.822+0000 7f870dffb640 1 -- 172.21.15.5:0/1696752340 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f86fc0d27a0 con 0x7f87101036e0 2024-09-12T03:01:57.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.822+0000 7f871577d640 1 --2- 172.21.15.5:0/1696752340 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f87000a2b80 0x7f87000a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:57.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.822+0000 7f87195b9740 1 -- 172.21.15.5:0/645902017 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562410be4dc0 msgr2=0x562410bef140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:57.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.822+0000 7f87195b9740 1 --2- 172.21.15.5:0/645902017 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562410be4dc0 0x562410bef140 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f8704011650 tx=0x7f8704009f90 comp rx=0 tx=0).stop 2024-09-12T03:01:57.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.822+0000 7f87195b9740 1 -- 172.21.15.5:0/645902017 shutdown_connections 2024-09-12T03:01:57.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.822+0000 7f87195b9740 1 --2- 172.21.15.5:0/645902017 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562410bef680 0x562410bf1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:57.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.822+0000 7f87195b9740 1 --2- 172.21.15.5:0/645902017 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562410be4dc0 0x562410bef140 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:57.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.822+0000 7f87195b9740 1 --2- 172.21.15.5:0/645902017 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562410be44a0 0x562410be4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:57.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.822+0000 7f87195b9740 1 -- 172.21.15.5:0/645902017 >> 172.21.15.5:0/645902017 conn(0x562410be0a30 msgr2=0x562410be0e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:57.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.822+0000 7f87195b9740 1 -- 172.21.15.5:0/645902017 shutdown_connections 2024-09-12T03:01:57.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.822+0000 7f87195b9740 1 -- 172.21.15.5:0/645902017 wait complete. 2024-09-12T03:01:57.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.823+0000 7f87195b9740 1 -- start start 2024-09-12T03:01:57.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.823+0000 7f87195b9740 1 Processor -- start 2024-09-12T03:01:57.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.823+0000 7f871577d640 1 --2- 172.21.15.5:0/1696752340 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f87000a2b80 0x7f87000a5040 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f8708009430 tx=0x7f8708005c30 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:57.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.823+0000 7f87195b9740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562410be44a0 0x562410bef010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:57.842 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.823+0000 7f87195b9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562410be4dc0 0x562410bec2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:57.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.823+0000 7f87195b9740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562410bef680 0x562410bec840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:57.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.823+0000 7f87195b9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x562410bf2250 con 0x562410be4dc0 2024-09-12T03:01:57.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.823+0000 7f87195b9740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x562410b05050 con 0x562410bef680 2024-09-12T03:01:57.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.823+0000 7f87195b9740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x562410bf23d0 con 0x562410be44a0 2024-09-12T03:01:57.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.823+0000 7f87195b9740 10 client.0 _ll_register_callbacks cb 0x562410b45160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:01:57.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.823+0000 7f8714f7c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562410be44a0 0x562410bef010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:57.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.824+0000 7f870ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562410be4dc0 0x562410bec2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:57.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.824+0000 7f8714f7c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562410be44a0 0x562410bef010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.5:42588/0 (socket says 172.21.15.5:42588) 2024-09-12T03:01:57.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.824+0000 7f8714f7c640 1 -- 172.21.15.5:0/3997023129 learned_addr learned my addr 172.21.15.5:0/3997023129 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:01:57.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.824+0000 7f871577d640 1 --2- 172.21.15.5:0/3997023129 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562410bef680 0x562410bec840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:57.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.824+0000 7f870ffff640 1 -- 172.21.15.5:0/3997023129 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562410be44a0 msgr2=0x562410bef010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:01:57.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.824+0000 7f870ffff640 1 --2- 172.21.15.5:0/3997023129 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562410be44a0 0x562410bef010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:57.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.824+0000 7f870ffff640 1 -- 172.21.15.5:0/3997023129 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562410bef680 msgr2=0x562410bec840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:01:57.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.824+0000 7f870ffff640 1 --2- 172.21.15.5:0/3997023129 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562410bef680 0x562410bec840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:57.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.824+0000 7f870ffff640 1 -- 172.21.15.5:0/3997023129 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624108bb3c0 con 0x562410be4dc0 2024-09-12T03:01:57.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.824+0000 7f871577d640 1 --2- 172.21.15.5:0/3997023129 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562410bef680 0x562410bec840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:01:57.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.824+0000 7f870ffff640 1 --2- 172.21.15.5:0/3997023129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562410be4dc0 0x562410bec2c0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f86fc0d0300 tx=0x7f86fc09ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74715eb7ac3f073a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:57.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.824+0000 7f86effff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T03:01:57.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.825+0000 7f86effff640 1 -- 172.21.15.5:0/3997023129 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86fc0bf0e0 con 0x562410be4dc0 2024-09-12T03:01:57.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.825+0000 7f86effff640 1 -- 172.21.15.5:0/3997023129 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f86fc0a77f0 con 0x562410be4dc0 2024-09-12T03:01:57.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.825+0000 7f87195b9740 1 -- 172.21.15.5:0/3997023129 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f870400fba0 con 0x562410be4dc0 2024-09-12T03:01:57.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.825+0000 7f86effff640 1 -- 172.21.15.5:0/3997023129 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f86fc09f7a0 con 0x562410be4dc0 2024-09-12T03:01:57.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.825+0000 7f87195b9740 10 client.16194 fetch_fsmap learned FSMap version 29 2024-09-12T03:01:57.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.825+0000 7f87195b9740 1 -- 172.21.15.5:0/3997023129 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562410bfdc40 con 0x562410be4dc0 2024-09-12T03:01:57.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.825+0000 7f86effff640 1 -- 172.21.15.5:0/3997023129 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f870400fba0 con 0x562410be4dc0 2024-09-12T03:01:57.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.825+0000 7f86effff640 1 -- 172.21.15.5:0/3997023129 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f86fc016070 con 0x562410be4dc0 2024-09-12T03:01:57.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.825+0000 7f87195b9740 10 client.16194 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T03:01:57.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.825+0000 7f87195b9740 10 client.16194 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:01:57.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.825+0000 7f87195b9740 20 client.16194 populate_metadata read hostname 'smithi005' 2024-09-12T03:01:57.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.825+0000 7f87195b9740 1 --2- 172.21.15.5:0/3997023129 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x562410bff050 0x562410c01460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:57.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.825+0000 7f87195b9740 4 client.16194 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:01:57.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.825+0000 7f87195b9740 1 -- 172.21.15.5:0/3997023129 --> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562410b57850 con 0x562410bff050 2024-09-12T03:01:57.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.826+0000 7f8714f7c640 1 --2- 172.21.15.5:0/3997023129 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x562410bff050 0x562410c01460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:57.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.826+0000 7f86effff640 10 client.16194 ms_handle_connect on v2:172.21.15.148:6838/2317600000 2024-09-12T03:01:57.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.826+0000 7f8714f7c640 1 --2- 172.21.15.5:0/3997023129 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x562410bff050 0x562410c01460 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f287daa22029e3bd server_cookie=a67b6fd049e87a15 in_seq=0 out_seq=0 2024-09-12T03:01:57.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.827+0000 7f86effff640 1 -- 172.21.15.5:0/3997023129 <== mds.0 v2:172.21.15.148:6838/2317600000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562410b57850 con 0x562410bff050 2024-09-12T03:01:57.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.827+0000 7f86effff640 10 client.16194 handle_command_reply: tid=0 2024-09-12T03:01:57.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.827+0000 7f87195b9740 1 client.16194 shutdown 2024-09-12T03:01:57.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.827+0000 7f87195b9740 1 -- 172.21.15.5:0/3997023129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562410be4dc0 msgr2=0x562410bec2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:57.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.827+0000 7f87195b9740 1 --2- 172.21.15.5:0/3997023129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562410be4dc0 0x562410bec2c0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f86fc0d0300 tx=0x7f86fc09ef70 comp rx=0 tx=0).stop 2024-09-12T03:01:57.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.827+0000 7f87195b9740 1 -- 172.21.15.5:0/3997023129 shutdown_connections 2024-09-12T03:01:57.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.827+0000 7f87195b9740 1 --2- 172.21.15.5:0/3997023129 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x562410bff050 0x562410c01460 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:57.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.827+0000 7f87195b9740 1 --2- 172.21.15.5:0/3997023129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562410be4dc0 0x562410bec2c0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:57.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.827+0000 7f86effff640 0 client.16194 ms_handle_reset on v2:172.21.15.148:6838/2317600000 2024-09-12T03:01:57.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.827+0000 7f87195b9740 1 --2- 172.21.15.5:0/3997023129 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562410bef680 0x562410bec840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:57.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.827+0000 7f87195b9740 1 --2- 172.21.15.5:0/3997023129 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562410be44a0 0x562410bef010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:57.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.828+0000 7f87195b9740 1 -- 172.21.15.5:0/3997023129 >> 172.21.15.5:0/3997023129 conn(0x562410be0a30 msgr2=0x562410be6710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:57.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.828+0000 7f87195b9740 1 -- 172.21.15.5:0/3997023129 shutdown_connections 2024-09-12T03:01:57.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.828+0000 7f87195b9740 1 -- 172.21.15.5:0/3997023129 wait complete. 2024-09-12T03:01:57.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.828+0000 7f87195b9740 20 client.16194 trim_cache size 0 max 16384 2024-09-12T03:01:57.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.867+0000 7f87195b9740 1 Processor -- start 2024-09-12T03:01:57.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.867+0000 7f87195b9740 1 -- start start 2024-09-12T03:01:57.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.867+0000 7f87195b9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562410be44a0 0x562410bfb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:57.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.867+0000 7f87195b9740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562410be4dc0 0x562410bebb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:57.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.867+0000 7f87195b9740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562410bef680 0x562410be77f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:57.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.867+0000 7f87195b9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f87101ab450 con 0x562410be44a0 2024-09-12T03:01:57.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.867+0000 7f87195b9740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f87101acff0 con 0x562410be4dc0 2024-09-12T03:01:57.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.867+0000 7f87195b9740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f87101abb20 con 0x562410bef680 2024-09-12T03:01:57.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.867+0000 7f870ffff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562410be4dc0 0x562410bebb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:57.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.867+0000 7f871577d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562410bef680 0x562410be77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:57.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.867+0000 7f870ffff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562410be4dc0 0x562410bebb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.5:34342/0 (socket says 172.21.15.5:34342) 2024-09-12T03:01:57.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.867+0000 7f870ffff640 1 -- 172.21.15.5:0/256323012 learned_addr learned my addr 172.21.15.5:0/256323012 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:01:57.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.868+0000 7f8714f7c640 1 --2- 172.21.15.5:0/256323012 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562410be44a0 0x562410bfb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:57.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.868+0000 7f870ffff640 1 -- 172.21.15.5:0/256323012 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562410bef680 msgr2=0x562410be77f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:01:57.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.868+0000 7f870ffff640 1 --2- 172.21.15.5:0/256323012 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562410bef680 0x562410be77f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:57.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.868+0000 7f870ffff640 1 -- 172.21.15.5:0/256323012 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562410be44a0 msgr2=0x562410bfb4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:01:57.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.868+0000 7f870ffff640 1 --2- 172.21.15.5:0/256323012 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562410be44a0 0x562410bfb4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:57.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.868+0000 7f870ffff640 1 -- 172.21.15.5:0/256323012 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562410bea810 con 0x562410be4dc0 2024-09-12T03:01:57.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.868+0000 7f870ffff640 1 --2- 172.21.15.5:0/256323012 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562410be4dc0 0x562410bebb80 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f86fc00e630 tx=0x7f86fc09d5b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c3f0e17ce7274b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:57.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.868+0000 7f86fa7fc640 1 -- 172.21.15.5:0/256323012 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86fc0d2aa0 con 0x562410be4dc0 2024-09-12T03:01:57.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.868+0000 7f86fa7fc640 1 -- 172.21.15.5:0/256323012 <== mon.2 v2:172.21.15.146:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86fc09f0a0 con 0x562410be4dc0 2024-09-12T03:01:57.870 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.868+0000 7f86fa7fc640 1 -- 172.21.15.5:0/256323012 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86fc0a7a30 con 0x562410be4dc0 2024-09-12T03:01:57.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.869+0000 7f87195b9740 1 -- 172.21.15.5:0/256323012 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562410be4dc0 msgr2=0x562410bebb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:57.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.869+0000 7f87195b9740 1 --2- 172.21.15.5:0/256323012 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562410be4dc0 0x562410bebb80 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f86fc00e630 tx=0x7f86fc09d5b0 comp rx=0 tx=0).stop 2024-09-12T03:01:57.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.869+0000 7f87195b9740 1 -- 172.21.15.5:0/256323012 shutdown_connections 2024-09-12T03:01:57.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.869+0000 7f87195b9740 1 --2- 172.21.15.5:0/256323012 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562410bef680 0x562410be77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:57.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.869+0000 7f87195b9740 1 --2- 172.21.15.5:0/256323012 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562410be4dc0 0x562410bebb80 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:57.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.869+0000 7f87195b9740 1 --2- 172.21.15.5:0/256323012 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562410be44a0 0x562410bfb4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:57.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.869+0000 7f87195b9740 1 -- 172.21.15.5:0/256323012 >> 172.21.15.5:0/256323012 conn(0x562410be0a30 msgr2=0x562410be70e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:57.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.870+0000 7f87195b9740 1 -- 172.21.15.5:0/256323012 shutdown_connections 2024-09-12T03:01:57.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.870+0000 7f87195b9740 1 -- 172.21.15.5:0/256323012 wait complete. 2024-09-12T03:01:57.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.870+0000 7f87195b9740 1 -- start start 2024-09-12T03:01:57.871 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.870+0000 7f87195b9740 1 Processor -- start 2024-09-12T03:01:57.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.871+0000 7f87195b9740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562410be44a0 0x562410bf7d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:57.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.871+0000 7f87195b9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562410be4dc0 0x562410bf3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:57.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.871+0000 7f87195b9740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562410bef680 0x562410bf35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:57.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.871+0000 7f870ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562410be4dc0 0x562410bf3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:57.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.871+0000 7f870ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562410be4dc0 0x562410bf3080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51428/0 (socket says 172.21.15.5:51428) 2024-09-12T03:01:57.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.872+0000 7f870ffff640 1 -- 172.21.15.5:0/1196257323 learned_addr learned my addr 172.21.15.5:0/1196257323 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:01:57.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.872+0000 7f87195b9740 1 -- 172.21.15.5:0/1196257323 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f87101ac8c0 con 0x562410be4dc0 2024-09-12T03:01:57.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.872+0000 7f871577d640 1 --2- 172.21.15.5:0/1196257323 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562410bef680 0x562410bf35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:57.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.872+0000 7f87195b9740 1 -- 172.21.15.5:0/1196257323 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f87101ac1f0 con 0x562410be44a0 2024-09-12T03:01:57.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.872+0000 7f87195b9740 1 -- 172.21.15.5:0/1196257323 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f87101a9430 con 0x562410bef680 2024-09-12T03:01:57.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.872+0000 7f8714f7c640 1 --2- 172.21.15.5:0/1196257323 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562410be44a0 0x562410bf7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:57.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.872+0000 7f871577d640 1 -- 172.21.15.5:0/1196257323 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562410be44a0 msgr2=0x562410bf7d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:01:57.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.872+0000 7f871577d640 1 --2- 172.21.15.5:0/1196257323 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562410be44a0 0x562410bf7d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:57.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.872+0000 7f871577d640 1 -- 172.21.15.5:0/1196257323 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562410be4dc0 msgr2=0x562410bf3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:01:57.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.872+0000 7f871577d640 1 --2- 172.21.15.5:0/1196257323 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562410be4dc0 0x562410bf3080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:57.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.872+0000 7f871577d640 1 -- 172.21.15.5:0/1196257323 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562410bf3df0 con 0x562410bef680 2024-09-12T03:01:57.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.873+0000 7f870ffff640 1 --2- 172.21.15.5:0/1196257323 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562410be4dc0 0x562410bf3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:01:57.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.873+0000 7f87195b9740 10 client.0 _ll_register_callbacks cb 0x7f86fc09b340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:01:57.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.873+0000 7f8714f7c640 1 --2- 172.21.15.5:0/1196257323 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562410be44a0 0x562410bf7d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:01:57.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.873+0000 7f87195b9740 1 -- 172.21.15.5:0/1196257323 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86fc09f0a0 con 0x562410bef680 2024-09-12T03:01:57.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.873+0000 7f871577d640 1 --2- 172.21.15.5:0/1196257323 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562410bef680 0x562410bf35c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8708031040 tx=0x7f870802a8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=91bc198ae173e436 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:57.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.873+0000 7f86effff640 10 client.25717 ms_handle_connect on v2:172.21.15.148:3300/0 2024-09-12T03:01:57.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.873+0000 7f86effff640 1 -- 172.21.15.5:0/1196257323 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8708030890 con 0x562410bef680 2024-09-12T03:01:57.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.873+0000 7f86effff640 1 -- 172.21.15.5:0/1196257323 <== mon.1 v2:172.21.15.148:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f8708030a30 con 0x562410bef680 2024-09-12T03:01:57.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.873+0000 7f86effff640 1 -- 172.21.15.5:0/1196257323 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8708030c50 con 0x562410bef680 2024-09-12T03:01:57.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.873+0000 7f87195b9740 10 client.25717 fetch_fsmap learned FSMap version 29 2024-09-12T03:01:57.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.873+0000 7f87195b9740 1 -- 172.21.15.5:0/1196257323 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({fsmap=29}) -- 0x562410c69bf0 con 0x562410bef680 2024-09-12T03:01:57.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.874+0000 7f86effff640 1 -- 172.21.15.5:0/1196257323 <== mon.1 v2:172.21.15.148:3300/0 4 ==== fsmap(e 29) ==== 8370+0+0 (secure 0 0 0) 0x7f870802f610 con 0x562410bef680 2024-09-12T03:01:57.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.874+0000 7f87195b9740 10 client.25717 fetch_fsmap finished waiting for FSMap version 29 2024-09-12T03:01:57.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.874+0000 7f87195b9740 10 client.25717 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:01:57.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.874+0000 7f87195b9740 20 client.25717 populate_metadata read hostname 'smithi005' 2024-09-12T03:01:57.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.874+0000 7f87195b9740 1 --2- 172.21.15.5:0/1196257323 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x562410bff050 0x562410c6c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:57.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.874+0000 7f87195b9740 4 client.25717 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T03:01:57.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.874+0000 7f87195b9740 1 -- 172.21.15.5:0/1196257323 --> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562410b51d60 con 0x562410bff050 2024-09-12T03:01:57.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.874+0000 7f8714f7c640 1 --2- 172.21.15.5:0/1196257323 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x562410bff050 0x562410c6c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:57.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.874+0000 7f86effff640 10 client.25717 ms_handle_connect on v2:172.21.15.148:6838/2317600000 2024-09-12T03:01:57.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.874+0000 7f8714f7c640 1 --2- 172.21.15.5:0/1196257323 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x562410bff050 0x562410c6c2e0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f90a56a2477bf81 server_cookie=3c1bb3dc8d0c9431 in_seq=0 out_seq=0 2024-09-12T03:01:57.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.875+0000 7f86effff640 1 -- 172.21.15.5:0/1196257323 <== mds.0 v2:172.21.15.148:6838/2317600000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x562410b51d60 con 0x562410bff050 2024-09-12T03:01:57.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.875+0000 7f86effff640 10 client.25717 handle_command_reply: tid=0 2024-09-12T03:01:57.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.875+0000 7f87195b9740 1 client.25717 shutdown 2024-09-12T03:01:57.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.875+0000 7f87195b9740 1 -- 172.21.15.5:0/1196257323 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562410bef680 msgr2=0x562410bf35c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:57.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.875+0000 7f87195b9740 1 --2- 172.21.15.5:0/1196257323 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562410bef680 0x562410bf35c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8708031040 tx=0x7f870802a8c0 comp rx=0 tx=0).stop 2024-09-12T03:01:57.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.875+0000 7f87195b9740 1 -- 172.21.15.5:0/1196257323 shutdown_connections 2024-09-12T03:01:57.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.875+0000 7f87195b9740 1 --2- 172.21.15.5:0/1196257323 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x562410bff050 0x562410c6c2e0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:57.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.875+0000 7f87195b9740 1 --2- 172.21.15.5:0/1196257323 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562410be4dc0 0x562410bf3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:57.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.875+0000 7f86effff640 0 client.25717 ms_handle_reset on v2:172.21.15.148:6838/2317600000 2024-09-12T03:01:57.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.875+0000 7f87195b9740 1 --2- 172.21.15.5:0/1196257323 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x562410bef680 0x562410bf35c0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:57.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.875+0000 7f87195b9740 1 --2- 172.21.15.5:0/1196257323 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x562410be44a0 0x562410bf7d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:57.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.875+0000 7f87195b9740 1 -- 172.21.15.5:0/1196257323 >> 172.21.15.5:0/1196257323 conn(0x562410be0a30 msgr2=0x562410bedb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:57.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.876+0000 7f87195b9740 1 -- 172.21.15.5:0/1196257323 shutdown_connections 2024-09-12T03:01:57.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.876+0000 7f87195b9740 1 -- 172.21.15.5:0/1196257323 wait complete. 2024-09-12T03:01:57.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.876+0000 7f87195b9740 20 client.25717 trim_cache size 0 max 16384 2024-09-12T03:01:57.880 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-12T03:01:57.880 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-09-12T03:01:57.880 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-09-12T03:01:57.880 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-12T03:01:57.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.877+0000 7f87171e0640 1 -- 172.21.15.5:0/1696752340 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f87000a2b80 msgr2=0x7f87000a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:57.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.877+0000 7f87171e0640 1 --2- 172.21.15.5:0/1696752340 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f87000a2b80 0x7f87000a5040 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f8708009430 tx=0x7f8708005c30 comp rx=0 tx=0).stop 2024-09-12T03:01:57.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.877+0000 7f87171e0640 1 -- 172.21.15.5:0/1696752340 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f87101036e0 msgr2=0x7f8710198a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:57.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.877+0000 7f87171e0640 1 --2- 172.21.15.5:0/1696752340 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f87101036e0 0x7f8710198a20 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f87100697d0 tx=0x7f86fc010050 comp rx=0 tx=0).stop 2024-09-12T03:01:57.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.877+0000 7f87171e0640 1 -- 172.21.15.5:0/1696752340 shutdown_connections 2024-09-12T03:01:57.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.877+0000 7f87171e0640 1 --2- 172.21.15.5:0/1696752340 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f87101082a0 0x7f871019cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:57.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.877+0000 7f87171e0640 1 --2- 172.21.15.5:0/1696752340 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f87101036e0 0x7f8710198a20 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:57.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.877+0000 7f87171e0640 1 --2- 172.21.15.5:0/1696752340 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f87000a2b80 0x7f87000a5040 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:57.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.877+0000 7f87171e0640 1 --2- 172.21.15.5:0/1696752340 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8710102d30 0x7f87101984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:57.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.877+0000 7f87171e0640 1 -- 172.21.15.5:0/1696752340 >> 172.21.15.5:0/1696752340 conn(0x7f87100fc7d0 msgr2=0x7f87100fe390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:57.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.877+0000 7f87171e0640 1 -- 172.21.15.5:0/1696752340 shutdown_connections 2024-09-12T03:01:57.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:57.877+0000 7f87171e0640 1 -- 172.21.15.5:0/1696752340 wait complete. 2024-09-12T03:01:57.915 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:01:57 smithi148 ceph-mon[35090]: pgmap v446: 97 pgs: 97 active+clean; 110 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.9 MiB/s wr, 862 op/s 2024-09-12T03:01:57.956 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:01:57 smithi146 ceph-mon[35011]: pgmap v446: 97 pgs: 97 active+clean; 110 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.9 MiB/s wr, 862 op/s 2024-09-12T03:01:58.615 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T03:01:58.616 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:01:58.695 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:01:59.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:59.224+0000 7f3bcad5f640 1 -- 172.21.15.5:0/300620290 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3bc4102ef0 msgr2=0x7f3bc410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:59.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:59.224+0000 7f3bcad5f640 1 --2- 172.21.15.5:0/300620290 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3bc4102ef0 0x7f3bc410da40 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3bbc009c10 tx=0x7f3bbc02a770 comp rx=0 tx=0).stop 2024-09-12T03:01:59.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:59.225+0000 7f3bcad5f640 1 -- 172.21.15.5:0/300620290 shutdown_connections 2024-09-12T03:01:59.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:59.225+0000 7f3bcad5f640 1 --2- 172.21.15.5:0/300620290 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3bc4102ef0 0x7f3bc410da40 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:59.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:59.225+0000 7f3bcad5f640 1 --2- 172.21.15.5:0/300620290 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3bc4102570 0x7f3bc41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:59.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:59.225+0000 7f3bcad5f640 1 --2- 172.21.15.5:0/300620290 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3bc4100e20 0x7f3bc4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:59.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:59.225+0000 7f3bcad5f640 1 -- 172.21.15.5:0/300620290 >> 172.21.15.5:0/300620290 conn(0x7f3bc40fc7d0 msgr2=0x7f3bc40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:59.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:59.225+0000 7f3bcad5f640 1 -- 172.21.15.5:0/300620290 shutdown_connections 2024-09-12T03:01:59.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:59.225+0000 7f3bcad5f640 1 -- 172.21.15.5:0/300620290 wait complete. 2024-09-12T03:01:59.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:59.226+0000 7f3bcad5f640 1 Processor -- start 2024-09-12T03:01:59.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:59.226+0000 7f3bcad5f640 1 -- start start 2024-09-12T03:01:59.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:59.227+0000 7f3bcad5f640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3bc4100e20 0x7f3bc4199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:59.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:59.227+0000 7f3bcad5f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3bc4102570 0x7f3bc41998a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:59.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:59.227+0000 7f3bcad5f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3bc4102ef0 0x7f3bc41a43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:59.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:59.227+0000 7f3bcad5f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3bc4110f30 con 0x7f3bc4102570 2024-09-12T03:01:59.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:59.227+0000 7f3bcad5f640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f3bc4110db0 con 0x7f3bc4100e20 2024-09-12T03:01:59.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:59.227+0000 7f3bcad5f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3bc41110b0 con 0x7f3bc4102ef0 2024-09-12T03:01:59.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:59.227+0000 7f3bc3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3bc4102570 0x7f3bc41998a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:59.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:59.227+0000 7f3bc3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3bc4102570 0x7f3bc41998a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51450/0 (socket says 172.21.15.5:51450) 2024-09-12T03:01:59.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:59.227+0000 7f3bc3fff640 1 -- 172.21.15.5:0/3377140700 learned_addr learned my addr 172.21.15.5:0/3377140700 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:01:59.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:59.227+0000 7f3bc8afb640 1 --2- 172.21.15.5:0/3377140700 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3bc4100e20 0x7f3bc4199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:59.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:59.227+0000 7f3bc92fc640 1 --2- 172.21.15.5:0/3377140700 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3bc4102ef0 0x7f3bc41a43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:59.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:59.227+0000 7f3bc3fff640 1 -- 172.21.15.5:0/3377140700 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3bc4102ef0 msgr2=0x7f3bc41a43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:59.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:59.227+0000 7f3bc3fff640 1 --2- 172.21.15.5:0/3377140700 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3bc4102ef0 0x7f3bc41a43e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:59.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:59.228+0000 7f3bc3fff640 1 -- 172.21.15.5:0/3377140700 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3bc4100e20 msgr2=0x7f3bc4199360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:59.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:59.228+0000 7f3bc3fff640 1 --2- 172.21.15.5:0/3377140700 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3bc4100e20 0x7f3bc4199360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:59.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:59.228+0000 7f3bc3fff640 1 -- 172.21.15.5:0/3377140700 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bc41a4920 con 0x7f3bc4102570 2024-09-12T03:01:59.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:59.228+0000 7f3bc8afb640 1 --2- 172.21.15.5:0/3377140700 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3bc4100e20 0x7f3bc4199360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:01:59.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:59.228+0000 7f3bc92fc640 1 --2- 172.21.15.5:0/3377140700 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3bc4102ef0 0x7f3bc41a43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:01:59.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:59.228+0000 7f3bc3fff640 1 --2- 172.21.15.5:0/3377140700 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3bc4102570 0x7f3bc41998a0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f3bb000d950 tx=0x7f3bb000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:59.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:59.228+0000 7f3bc1ffb640 1 -- 172.21.15.5:0/3377140700 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bb0017070 con 0x7f3bc4102570 2024-09-12T03:01:59.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:59.228+0000 7f3bcad5f640 1 -- 172.21.15.5:0/3377140700 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bc41a4af0 con 0x7f3bc4102570 2024-09-12T03:01:59.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:59.229+0000 7f3bc1ffb640 1 -- 172.21.15.5:0/3377140700 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3bb0009c30 con 0x7f3bc4102570 2024-09-12T03:01:59.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:59.229+0000 7f3bc1ffb640 1 -- 172.21.15.5:0/3377140700 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bb000f6b0 con 0x7f3bc4102570 2024-09-12T03:01:59.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:59.229+0000 7f3bcad5f640 1 -- 172.21.15.5:0/3377140700 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bc41a5080 con 0x7f3bc4102570 2024-09-12T03:01:59.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:59.230+0000 7f3bcad5f640 1 -- 172.21.15.5:0/3377140700 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bc4101200 con 0x7f3bc4102570 2024-09-12T03:01:59.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:59.232+0000 7f3bc1ffb640 1 -- 172.21.15.5:0/3377140700 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f3bb00109d0 con 0x7f3bc4102570 2024-09-12T03:01:59.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:59.232+0000 7f3bc1ffb640 1 --2- 172.21.15.5:0/3377140700 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f3bb40a2b80 0x7f3bb40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:01:59.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:59.232+0000 7f3bc1ffb640 1 -- 172.21.15.5:0/3377140700 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f3bb00ceb10 con 0x7f3bc4102570 2024-09-12T03:01:59.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:59.235+0000 7f3bc8afb640 1 --2- 172.21.15.5:0/3377140700 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f3bb40a2b80 0x7f3bb40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:01:59.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:59.237+0000 7f3bc8afb640 1 --2- 172.21.15.5:0/3377140700 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f3bb40a2b80 0x7f3bb40a5040 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f3bb80096c0 tx=0x7f3bb8009260 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:01:59.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:59.240+0000 7f3bc1ffb640 1 -- 172.21.15.5:0/3377140700 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3bb0009de0 con 0x7f3bc4102570 2024-09-12T03:01:59.272 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:01:59.519 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:59.518+0000 7f3bcad5f640 1 -- 172.21.15.5:0/3377140700 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3bc4109090 con 0x7f3bc4102570 2024-09-12T03:01:59.520 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:59.519+0000 7f3bc1ffb640 1 -- 172.21.15.5:0/3377140700 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11967 (secure 0 0 0) 0x7f3bb009b430 con 0x7f3bc4102570 2024-09-12T03:01:59.520 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-12T03:01:59.521 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":30,"btime":"2024-09-12T03:01:58:481122+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/4135232521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":4135232521},{"type":"v1","addr":"172.21.15.5:6835","nonce":4135232521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/4290221940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":4290221940},{"type":"v1","addr":"172.21.15.5:6837","nonce":4290221940}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T02:56:16.595562+0000","modified":"2024-09-12T03:01:57.686450+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24502,"mds_1":24487,"mds_2":24481,"mds_3":24479,"mds_4":24458},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/26447341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":26447341},{"type":"v1","addr":"172.21.15.5:6839","nonce":26447341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2393443295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2393443295},{"type":"v1","addr":"172.21.15.5:6841","nonce":2393443295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6833/3686970137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3686970137},{"type":"v1","addr":"172.21.15.146:6833","nonce":3686970137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6833/568512209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":568512209},{"type":"v1","addr":"172.21.15.148:6833","nonce":568512209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6835/2540740420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2540740420},{"type":"v1","addr":"172.21.15.146:6835","nonce":2540740420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.146:6837/1673370937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":1673370937},{"type":"v1","addr":"172.21.15.146:6837","nonce":1673370937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":3,"incarnation":22,"state":"up:active","state_seq":6,"addr":"172.21.15.146:6839/3036767228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":3036767228},{"type":"v1","addr":"172.21.15.146:6839","nonce":3036767228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":2,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.148:6835/4195086427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":4195086427},{"type":"v1","addr":"172.21.15.148:6835","nonce":4195086427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.148:6837/2606873922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2606873922},{"type":"v1","addr":"172.21.15.148:6837","nonce":2606873922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6839/2317600000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":2317600000},{"type":"v1","addr":"172.21.15.148:6839","nonce":2317600000}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24502,"qdb_cluster":[24502,24487,24481,24479,24458]},"id":1}]} 2024-09-12T03:01:59.521 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 30 2024-09-12T03:01:59.523 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:59.522+0000 7f3bcad5f640 1 -- 172.21.15.5:0/3377140700 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f3bb40a2b80 msgr2=0x7f3bb40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:59.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:59.522+0000 7f3bcad5f640 1 --2- 172.21.15.5:0/3377140700 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f3bb40a2b80 0x7f3bb40a5040 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f3bb80096c0 tx=0x7f3bb8009260 comp rx=0 tx=0).stop 2024-09-12T03:01:59.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:59.522+0000 7f3bcad5f640 1 -- 172.21.15.5:0/3377140700 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3bc4102570 msgr2=0x7f3bc41998a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:01:59.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:59.522+0000 7f3bcad5f640 1 --2- 172.21.15.5:0/3377140700 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3bc4102570 0x7f3bc41998a0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f3bb000d950 tx=0x7f3bb000de10 comp rx=0 tx=0).stop 2024-09-12T03:01:59.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:59.522+0000 7f3bcad5f640 1 -- 172.21.15.5:0/3377140700 shutdown_connections 2024-09-12T03:01:59.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:59.522+0000 7f3bcad5f640 1 --2- 172.21.15.5:0/3377140700 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f3bb40a2b80 0x7f3bb40a5040 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:59.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:59.522+0000 7f3bcad5f640 1 --2- 172.21.15.5:0/3377140700 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3bc4102570 0x7f3bc41998a0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:59.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:59.522+0000 7f3bcad5f640 1 --2- 172.21.15.5:0/3377140700 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3bc4102ef0 0x7f3bc41a43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:59.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:59.522+0000 7f3bcad5f640 1 --2- 172.21.15.5:0/3377140700 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3bc4100e20 0x7f3bc4199360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:01:59.524 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:59.522+0000 7f3bcad5f640 1 -- 172.21.15.5:0/3377140700 >> 172.21.15.5:0/3377140700 conn(0x7f3bc40fc7d0 msgr2=0x7f3bc41048b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:01:59.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:59.523+0000 7f3bcad5f640 1 -- 172.21.15.5:0/3377140700 shutdown_connections 2024-09-12T03:01:59.525 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:01:59.523+0000 7f3bcad5f640 1 -- 172.21.15.5:0/3377140700 wait complete. 2024-09-12T03:01:59.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:01:59 smithi005 ceph-mon[31197]: pgmap v447: 97 pgs: 97 active+clean; 110 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.3 MiB/s wr, 795 op/s 2024-09-12T03:01:59.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:01:59 smithi005 ceph-mon[31197]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:01:59.915 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:01:59 smithi148 ceph-mon[35090]: pgmap v447: 97 pgs: 97 active+clean; 110 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.3 MiB/s wr, 795 op/s 2024-09-12T03:01:59.915 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:01:59 smithi148 ceph-mon[35090]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:01:59.956 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:01:59 smithi146 ceph-mon[35011]: pgmap v447: 97 pgs: 97 active+clean; 110 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.3 MiB/s wr, 795 op/s 2024-09-12T03:01:59.956 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:01:59 smithi146 ceph-mon[35011]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:02:00.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:00.010+0000 7f2f57447640 1 -- 172.21.15.5:0/1815895736 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f5010e500 msgr2=0x7f2f50110910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:00.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:00.010+0000 7f2f57447640 1 --2- 172.21.15.5:0/1815895736 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f5010e500 0x7f2f50110910 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f2f48009ca0 tx=0x7f2f4802a8c0 comp rx=0 tx=0).stop 2024-09-12T03:02:00.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:00.010+0000 7f2f57447640 1 -- 172.21.15.5:0/1815895736 shutdown_connections 2024-09-12T03:02:00.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:00.010+0000 7f2f57447640 1 --2- 172.21.15.5:0/1815895736 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f5010e500 0x7f2f50110910 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:00.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:00.010+0000 7f2f57447640 1 --2- 172.21.15.5:0/1815895736 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2f50101440 0x7f2f5010df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:00.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:00.010+0000 7f2f57447640 1 --2- 172.21.15.5:0/1815895736 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2f50100b20 0x7f2f50100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:00.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:00.010+0000 7f2f57447640 1 -- 172.21.15.5:0/1815895736 >> 172.21.15.5:0/1815895736 conn(0x7f2f500fc7d0 msgr2=0x7f2f500febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:00.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:00.011+0000 7f2f57447640 1 -- 172.21.15.5:0/1815895736 shutdown_connections 2024-09-12T03:02:00.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:00.011+0000 7f2f57447640 1 -- 172.21.15.5:0/1815895736 wait complete. 2024-09-12T03:02:00.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:00.011+0000 7f2f57447640 1 Processor -- start 2024-09-12T03:02:00.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:00.012+0000 7f2f57447640 1 -- start start 2024-09-12T03:02:00.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:00.012+0000 7f2f57447640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2f50100b20 0x7f2f5019a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:00.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:00.012+0000 7f2f57447640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f50101440 0x7f2f5019ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:00.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:00.012+0000 7f2f57447640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2f5010e500 0x7f2f5019efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:00.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:00.012+0000 7f2f551e3640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2f50100b20 0x7f2f5019a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:00.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:00.012+0000 7f2f551e3640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2f50100b20 0x7f2f5019a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.5:42634/0 (socket says 172.21.15.5:42634) 2024-09-12T03:02:00.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:00.012+0000 7f2f551e3640 1 -- 172.21.15.5:0/2446488674 learned_addr learned my addr 172.21.15.5:0/2446488674 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:02:00.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:00.012+0000 7f2f559e4640 1 --2- 172.21.15.5:0/2446488674 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2f5010e500 0x7f2f5019efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:00.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:00.012+0000 7f2f57447640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2f50114280 con 0x7f2f50101440 2024-09-12T03:02:00.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:00.012+0000 7f2f57447640 1 -- 172.21.15.5:0/2446488674 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f2f50114100 con 0x7f2f5010e500 2024-09-12T03:02:00.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:00.012+0000 7f2f57447640 1 -- 172.21.15.5:0/2446488674 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f2f50114400 con 0x7f2f50100b20 2024-09-12T03:02:00.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:00.013+0000 7f2f551e3640 1 -- 172.21.15.5:0/2446488674 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2f5010e500 msgr2=0x7f2f5019efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:00.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:00.013+0000 7f2f551e3640 1 --2- 172.21.15.5:0/2446488674 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2f5010e500 0x7f2f5019efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:00.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:00.013+0000 7f2f551e3640 1 -- 172.21.15.5:0/2446488674 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f50101440 msgr2=0x7f2f5019ac40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-12T03:02:00.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:00.013+0000 7f2f551e3640 1 --2- 172.21.15.5:0/2446488674 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f50101440 0x7f2f5019ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:00.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:00.013+0000 7f2f551e3640 1 -- 172.21.15.5:0/2446488674 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f5019f6b0 con 0x7f2f50100b20 2024-09-12T03:02:00.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:00.013+0000 7f2f551e3640 1 --2- 172.21.15.5:0/2446488674 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2f50100b20 0x7f2f5019a700 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2f4400e9f0 tx=0x7f2f4400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:00.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:00.013+0000 7f2f427fc640 1 -- 172.21.15.5:0/2446488674 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f4400cd80 con 0x7f2f50100b20 2024-09-12T03:02:00.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:00.013+0000 7f2f57447640 1 -- 172.21.15.5:0/2446488674 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f5019f9a0 con 0x7f2f50100b20 2024-09-12T03:02:00.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:00.013+0000 7f2f427fc640 1 -- 172.21.15.5:0/2446488674 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f44010910 con 0x7f2f50100b20 2024-09-12T03:02:00.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:00.013+0000 7f2f427fc640 1 -- 172.21.15.5:0/2446488674 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f440188f0 con 0x7f2f50100b20 2024-09-12T03:02:00.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:00.013+0000 7f2f57447640 1 -- 172.21.15.5:0/2446488674 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f50102ad0 con 0x7f2f50100b20 2024-09-12T03:02:00.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:00.014+0000 7f2f57447640 1 -- 172.21.15.5:0/2446488674 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f50100f00 con 0x7f2f50100b20 2024-09-12T03:02:00.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:00.016+0000 7f2f427fc640 1 -- 172.21.15.5:0/2446488674 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f2f44010ac0 con 0x7f2f50100b20 2024-09-12T03:02:00.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:00.017+0000 7f2f427fc640 1 --2- 172.21.15.5:0/2446488674 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f2f3c0a2b80 0x7f2f3c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:00.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:00.017+0000 7f2f549e2640 1 --2- 172.21.15.5:0/2446488674 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f2f3c0a2b80 0x7f2f3c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:00.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:00.017+0000 7f2f427fc640 1 -- 172.21.15.5:0/2446488674 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f2f440cf9b0 con 0x7f2f50100b20 2024-09-12T03:02:00.018 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:00.017+0000 7f2f549e2640 1 --2- 172.21.15.5:0/2446488674 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f2f3c0a2b80 0x7f2f3c0a5040 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f2f4c0058c0 tx=0x7f2f4c005850 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:00.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:00.020+0000 7f2f427fc640 1 -- 172.21.15.5:0/2446488674 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2f4409c920 con 0x7f2f50100b20 2024-09-12T03:02:00.113 INFO:tasks.workunit.client.0.smithi005.stdout:truncating to largest ever: 0x9ffb01 2024-09-12T03:02:00.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:00.285+0000 7f2f57447640 1 -- 172.21.15.5:0/2446488674 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2f50109600 con 0x7f2f50100b20 2024-09-12T03:02:00.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:00.286+0000 7f2f427fc640 1 -- 172.21.15.5:0/2446488674 <== mon.1 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11967 (secure 0 0 0) 0x7f2f4409c220 con 0x7f2f50100b20 2024-09-12T03:02:00.289 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-12T03:02:00.289 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":30,"btime":"2024-09-12T03:01:58:481122+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/4135232521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":4135232521},{"type":"v1","addr":"172.21.15.5:6835","nonce":4135232521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/4290221940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":4290221940},{"type":"v1","addr":"172.21.15.5:6837","nonce":4290221940}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T02:56:16.595562+0000","modified":"2024-09-12T03:01:57.686450+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24502,"mds_1":24487,"mds_2":24481,"mds_3":24479,"mds_4":24458},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/26447341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":26447341},{"type":"v1","addr":"172.21.15.5:6839","nonce":26447341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2393443295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2393443295},{"type":"v1","addr":"172.21.15.5:6841","nonce":2393443295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6833/3686970137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3686970137},{"type":"v1","addr":"172.21.15.146:6833","nonce":3686970137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6833/568512209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":568512209},{"type":"v1","addr":"172.21.15.148:6833","nonce":568512209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6835/2540740420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2540740420},{"type":"v1","addr":"172.21.15.146:6835","nonce":2540740420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.146:6837/1673370937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":1673370937},{"type":"v1","addr":"172.21.15.146:6837","nonce":1673370937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":3,"incarnation":22,"state":"up:active","state_seq":6,"addr":"172.21.15.146:6839/3036767228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":3036767228},{"type":"v1","addr":"172.21.15.146:6839","nonce":3036767228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":2,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.148:6835/4195086427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":4195086427},{"type":"v1","addr":"172.21.15.148:6835","nonce":4195086427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.148:6837/2606873922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2606873922},{"type":"v1","addr":"172.21.15.148:6837","nonce":2606873922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6839/2317600000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":2317600000},{"type":"v1","addr":"172.21.15.148:6839","nonce":2317600000}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24502,"qdb_cluster":[24502,24487,24481,24479,24458]},"id":1}]} 2024-09-12T03:02:00.290 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 30 2024-09-12T03:02:00.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:00.289+0000 7f2f57447640 1 -- 172.21.15.5:0/2446488674 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f2f3c0a2b80 msgr2=0x7f2f3c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:00.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:00.289+0000 7f2f57447640 1 --2- 172.21.15.5:0/2446488674 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f2f3c0a2b80 0x7f2f3c0a5040 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f2f4c0058c0 tx=0x7f2f4c005850 comp rx=0 tx=0).stop 2024-09-12T03:02:00.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:00.289+0000 7f2f57447640 1 -- 172.21.15.5:0/2446488674 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2f50100b20 msgr2=0x7f2f5019a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:00.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:00.289+0000 7f2f57447640 1 --2- 172.21.15.5:0/2446488674 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2f50100b20 0x7f2f5019a700 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2f4400e9f0 tx=0x7f2f4400eeb0 comp rx=0 tx=0).stop 2024-09-12T03:02:00.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:00.289+0000 7f2f57447640 1 -- 172.21.15.5:0/2446488674 shutdown_connections 2024-09-12T03:02:00.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:00.289+0000 7f2f57447640 1 --2- 172.21.15.5:0/2446488674 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f2f3c0a2b80 0x7f2f3c0a5040 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:00.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:00.289+0000 7f2f57447640 1 --2- 172.21.15.5:0/2446488674 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f50101440 0x7f2f5019ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:00.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:00.289+0000 7f2f57447640 1 --2- 172.21.15.5:0/2446488674 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2f5010e500 0x7f2f5019efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:00.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:00.289+0000 7f2f57447640 1 --2- 172.21.15.5:0/2446488674 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2f50100b20 0x7f2f5019a700 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:00.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:00.289+0000 7f2f57447640 1 -- 172.21.15.5:0/2446488674 >> 172.21.15.5:0/2446488674 conn(0x7f2f500fc7d0 msgr2=0x7f2f500febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:00.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:00.290+0000 7f2f57447640 1 -- 172.21.15.5:0/2446488674 shutdown_connections 2024-09-12T03:02:00.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:00.290+0000 7f2f57447640 1 -- 172.21.15.5:0/2446488674 wait complete. 2024-09-12T03:02:00.551 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:02:00 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/3377140700' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:02:00.551 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:02:00 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/2446488674' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:02:00.585 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24502 quiesce db / --timeout 90 --expiration 140.15 --await-for 45.0' 2024-09-12T03:02:00.585 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24502 quiesce db / --timeout 90 --expiration 140.15 --await-for 45.0 2024-09-12T03:02:00.915 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:02:00 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/3377140700' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:02:00.915 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:02:00 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/2446488674' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:02:00.956 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:02:00 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/3377140700' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:02:00.956 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:02:00 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/2446488674' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:02:01.367 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-12T03:02:01.467 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:02:01.596 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:02:01 smithi005 ceph-mon[31197]: pgmap v448: 97 pgs: 97 active+clean; 107 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.3 MiB/s wr, 782 op/s 2024-09-12T03:02:01.915 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:02:01 smithi148 ceph-mon[35090]: pgmap v448: 97 pgs: 97 active+clean; 107 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.3 MiB/s wr, 782 op/s 2024-09-12T03:02:01.956 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:02:01 smithi146 ceph-mon[35011]: pgmap v448: 97 pgs: 97 active+clean; 107 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.3 MiB/s wr, 782 op/s 2024-09-12T03:02:02.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.031+0000 7fb846336640 1 -- 172.21.15.5:0/1108456737 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb840100e20 msgr2=0x7fb840101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:02.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.031+0000 7fb846336640 1 --2- 172.21.15.5:0/1108456737 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb840100e20 0x7fb840101200 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb830009930 tx=0x7fb83002a4d0 comp rx=0 tx=0).stop 2024-09-12T03:02:02.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.032+0000 7fb846336640 1 -- 172.21.15.5:0/1108456737 shutdown_connections 2024-09-12T03:02:02.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.032+0000 7fb846336640 1 --2- 172.21.15.5:0/1108456737 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb840102ef0 0x7fb84010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:02.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.032+0000 7fb846336640 1 --2- 172.21.15.5:0/1108456737 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb840102570 0x7fb8401029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:02.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.032+0000 7fb846336640 1 --2- 172.21.15.5:0/1108456737 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb840100e20 0x7fb840101200 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:02.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.032+0000 7fb846336640 1 -- 172.21.15.5:0/1108456737 >> 172.21.15.5:0/1108456737 conn(0x7fb8400fc7d0 msgr2=0x7fb8400febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:02.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.032+0000 7fb846336640 1 -- 172.21.15.5:0/1108456737 shutdown_connections 2024-09-12T03:02:02.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.032+0000 7fb846336640 1 -- 172.21.15.5:0/1108456737 wait complete. 2024-09-12T03:02:02.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.033+0000 7fb846336640 1 Processor -- start 2024-09-12T03:02:02.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.033+0000 7fb846336640 1 -- start start 2024-09-12T03:02:02.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.034+0000 7fb846336640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb840100e20 0x7fb84019a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:02.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.034+0000 7fb846336640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb840102570 0x7fb84019ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:02.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.034+0000 7fb846336640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb840102ef0 0x7fb84019efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:02.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.034+0000 7fb846336640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb84010fe80 con 0x7fb840102570 2024-09-12T03:02:02.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.034+0000 7fb846336640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fb84010fd00 con 0x7fb840100e20 2024-09-12T03:02:02.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.034+0000 7fb846336640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb840110000 con 0x7fb840102ef0 2024-09-12T03:02:02.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.034+0000 7fb83ffff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb840100e20 0x7fb84019a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:02.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.034+0000 7fb83ffff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb840100e20 0x7fb84019a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.5:34392/0 (socket says 172.21.15.5:34392) 2024-09-12T03:02:02.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.034+0000 7fb83ffff640 1 -- 172.21.15.5:0/310386319 learned_addr learned my addr 172.21.15.5:0/310386319 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:02:02.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.034+0000 7fb8448d3640 1 --2- 172.21.15.5:0/310386319 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb840102ef0 0x7fb84019efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:02.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.034+0000 7fb837fff640 1 --2- 172.21.15.5:0/310386319 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb840102570 0x7fb84019ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:02.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.034+0000 7fb83ffff640 1 -- 172.21.15.5:0/310386319 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb840102ef0 msgr2=0x7fb84019efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:02.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.034+0000 7fb83ffff640 1 --2- 172.21.15.5:0/310386319 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb840102ef0 0x7fb84019efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:02.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.034+0000 7fb83ffff640 1 -- 172.21.15.5:0/310386319 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb840102570 msgr2=0x7fb84019ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:02.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.034+0000 7fb83ffff640 1 --2- 172.21.15.5:0/310386319 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb840102570 0x7fb84019ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:02.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.034+0000 7fb83ffff640 1 -- 172.21.15.5:0/310386319 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb84019f6a0 con 0x7fb840100e20 2024-09-12T03:02:02.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.035+0000 7fb8448d3640 1 --2- 172.21.15.5:0/310386319 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb840102ef0 0x7fb84019efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:02:02.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.035+0000 7fb83ffff640 1 --2- 172.21.15.5:0/310386319 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb840100e20 0x7fb84019a6f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb830009230 tx=0x7fb830004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:02.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.035+0000 7fb83dffb640 1 -- 172.21.15.5:0/310386319 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb83003f030 con 0x7fb840100e20 2024-09-12T03:02:02.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.035+0000 7fb83dffb640 1 -- 172.21.15.5:0/310386319 <== mon.2 v2:172.21.15.146:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb830004440 con 0x7fb840100e20 2024-09-12T03:02:02.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.035+0000 7fb83dffb640 1 -- 172.21.15.5:0/310386319 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb830035070 con 0x7fb840100e20 2024-09-12T03:02:02.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.035+0000 7fb846336640 1 -- 172.21.15.5:0/310386319 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb84019f930 con 0x7fb840100e20 2024-09-12T03:02:02.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.035+0000 7fb846336640 1 -- 172.21.15.5:0/310386319 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb84019fdc0 con 0x7fb840100e20 2024-09-12T03:02:02.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.038+0000 7fb83dffb640 1 -- 172.21.15.5:0/310386319 <== mon.2 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fb830042030 con 0x7fb840100e20 2024-09-12T03:02:02.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.039+0000 7fb84870f740 1 Processor -- start 2024-09-12T03:02:02.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.039+0000 7fb84870f740 1 -- start start 2024-09-12T03:02:02.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.039+0000 7fb84870f740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e3d3625850 0x55e3d3625c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:02.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.039+0000 7fb84870f740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e3d3626170 0x55e3d36304f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:02.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.039+0000 7fb84870f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3d3630a30 0x55e3d3632e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:02.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.039+0000 7fb84870f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e3d358a160 con 0x55e3d3630a30 2024-09-12T03:02:02.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.039+0000 7fb84870f740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55e3d3531720 con 0x55e3d3625850 2024-09-12T03:02:02.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.039+0000 7fb84870f740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55e3d3586120 con 0x55e3d3626170 2024-09-12T03:02:02.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.040+0000 7fb8448d3640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e3d3625850 0x55e3d3625c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:02.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.040+0000 7fb8448d3640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e3d3625850 0x55e3d3625c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.5:34394/0 (socket says 172.21.15.5:34394) 2024-09-12T03:02:02.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.040+0000 7fb8448d3640 1 -- 172.21.15.5:0/1251573644 learned_addr learned my addr 172.21.15.5:0/1251573644 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:02:02.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.040+0000 7fb83ffff640 1 --2- 172.21.15.5:0/1251573644 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e3d3626170 0x55e3d36304f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:02.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.040+0000 7fb83dffb640 1 --2- 172.21.15.5:0/310386319 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fb82c0a2bf0 0x7fb82c0a50b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:02.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.040+0000 7fb8448d3640 1 -- 172.21.15.5:0/1251573644 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e3d3626170 msgr2=0x55e3d36304f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:02.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.040+0000 7fb8448d3640 1 --2- 172.21.15.5:0/1251573644 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e3d3626170 0x55e3d36304f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:02.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.040+0000 7fb837fff640 1 --2- 172.21.15.5:0/1251573644 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3d3630a30 0x55e3d3632e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:02.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.040+0000 7fb8448d3640 1 -- 172.21.15.5:0/1251573644 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3d3630a30 msgr2=0x55e3d3632e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:02.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.040+0000 7fb8448d3640 1 --2- 172.21.15.5:0/1251573644 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3d3630a30 0x55e3d3632e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:02.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.040+0000 7fb8448d3640 1 -- 172.21.15.5:0/1251573644 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3d32fbba0 con 0x55e3d3625850 2024-09-12T03:02:02.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.040+0000 7fb83dffb640 1 -- 172.21.15.5:0/310386319 <== mon.2 v2:172.21.15.146:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fb8300ef2f0 con 0x7fb840100e20 2024-09-12T03:02:02.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.040+0000 7fb837fff640 1 --2- 172.21.15.5:0/1251573644 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3d3630a30 0x55e3d3632e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:02:02.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.040+0000 7fb8448d3640 1 --2- 172.21.15.5:0/310386319 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fb82c0a2bf0 0x7fb82c0a50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:02.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.040+0000 7fb8448d3640 1 --2- 172.21.15.5:0/1251573644 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e3d3625850 0x55e3d3625c30 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb838009fd0 tx=0x7fb8380108e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=700020dc70dc105e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:02.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.041+0000 7fb8367fc640 1 -- 172.21.15.5:0/1251573644 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb838022070 con 0x55e3d3625850 2024-09-12T03:02:02.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.041+0000 7fb8367fc640 1 -- 172.21.15.5:0/1251573644 <== mon.2 v2:172.21.15.146:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb838011c20 con 0x55e3d3625850 2024-09-12T03:02:02.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.041+0000 7fb8367fc640 1 -- 172.21.15.5:0/1251573644 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb83801b740 con 0x55e3d3625850 2024-09-12T03:02:02.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.042+0000 7fb8448d3640 1 --2- 172.21.15.5:0/310386319 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fb82c0a2bf0 0x7fb82c0a50b0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fb83801f000 tx=0x7fb83801bdf0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:02.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.042+0000 7fb84870f740 1 -- 172.21.15.5:0/1251573644 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e3d3625850 msgr2=0x55e3d3625c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:02.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.042+0000 7fb84870f740 1 --2- 172.21.15.5:0/1251573644 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e3d3625850 0x55e3d3625c30 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb838009fd0 tx=0x7fb8380108e0 comp rx=0 tx=0).stop 2024-09-12T03:02:02.045 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.042+0000 7fb84870f740 1 -- 172.21.15.5:0/1251573644 shutdown_connections 2024-09-12T03:02:02.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.042+0000 7fb84870f740 1 --2- 172.21.15.5:0/1251573644 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3d3630a30 0x55e3d3632e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:02.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.042+0000 7fb84870f740 1 --2- 172.21.15.5:0/1251573644 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e3d3626170 0x55e3d36304f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:02.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.042+0000 7fb84870f740 1 --2- 172.21.15.5:0/1251573644 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e3d3625850 0x55e3d3625c30 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:02.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.042+0000 7fb84870f740 1 -- 172.21.15.5:0/1251573644 >> 172.21.15.5:0/1251573644 conn(0x55e3d3621de0 msgr2=0x55e3d36221c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:02.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.042+0000 7fb84870f740 1 -- 172.21.15.5:0/1251573644 shutdown_connections 2024-09-12T03:02:02.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.042+0000 7fb84870f740 1 -- 172.21.15.5:0/1251573644 wait complete. 2024-09-12T03:02:02.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.042+0000 7fb84870f740 1 -- start start 2024-09-12T03:02:02.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.043+0000 7fb84870f740 1 Processor -- start 2024-09-12T03:02:02.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.043+0000 7fb84870f740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e3d3625850 0x55e3d362d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:02.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.043+0000 7fb83ffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e3d3625850 0x55e3d362d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:02.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.043+0000 7fb83ffff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e3d3625850 0x55e3d362d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.5:42670/0 (socket says 172.21.15.5:42670) 2024-09-12T03:02:02.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.043+0000 7fb84870f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3d3626170 0x55e3d362de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:02.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.043+0000 7fb83ffff640 1 -- 172.21.15.5:0/4159367935 learned_addr learned my addr 172.21.15.5:0/4159367935 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:02:02.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.043+0000 7fb84870f740 1 --2- 172.21.15.5:0/4159367935 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e3d3630a30 0x55e3d362e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:02.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.044+0000 7fb84870f740 1 -- 172.21.15.5:0/4159367935 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55e3d3633600 con 0x55e3d3626170 2024-09-12T03:02:02.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.044+0000 7fb84870f740 1 -- 172.21.15.5:0/4159367935 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55e3d3512900 con 0x55e3d3630a30 2024-09-12T03:02:02.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.044+0000 7fb84870f740 1 -- 172.21.15.5:0/4159367935 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55e3d3633780 con 0x55e3d3625850 2024-09-12T03:02:02.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.044+0000 7fb8448d3640 1 --2- 172.21.15.5:0/4159367935 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e3d3630a30 0x55e3d362e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:02.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.044+0000 7fb83ffff640 1 -- 172.21.15.5:0/4159367935 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e3d3630a30 msgr2=0x55e3d362e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:02.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.044+0000 7fb83ffff640 1 --2- 172.21.15.5:0/4159367935 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e3d3630a30 0x55e3d362e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:02.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.044+0000 7fb83ffff640 1 -- 172.21.15.5:0/4159367935 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3d3626170 msgr2=0x55e3d362de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:02.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.044+0000 7fb83ffff640 1 --2- 172.21.15.5:0/4159367935 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3d3626170 0x55e3d362de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:02.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.044+0000 7fb83ffff640 1 -- 172.21.15.5:0/4159367935 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3d3552c50 con 0x55e3d3625850 2024-09-12T03:02:02.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.044+0000 7fb84870f740 10 client.0 _ll_register_callbacks cb 0x55e3d3510780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:02:02.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.044+0000 7fb84870f740 1 -- 172.21.15.5:0/4159367935 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb838011c20 con 0x55e3d3625850 2024-09-12T03:02:02.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.044+0000 7fb83ffff640 1 --2- 172.21.15.5:0/4159367935 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e3d3625850 0x55e3d362d8f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb83003b670 tx=0x7fb830005c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d6ea70c2177b4b04 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:02.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.044+0000 7fb81f7fe640 10 client.25741 ms_handle_connect on v2:172.21.15.148:3300/0 2024-09-12T03:02:02.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.044+0000 7fb81f7fe640 1 -- 172.21.15.5:0/4159367935 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb830049070 con 0x55e3d3625850 2024-09-12T03:02:02.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.044+0000 7fb81f7fe640 1 -- 172.21.15.5:0/4159367935 <== mon.1 v2:172.21.15.148:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fb830031bd0 con 0x55e3d3625850 2024-09-12T03:02:02.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.045+0000 7fb81f7fe640 1 -- 172.21.15.5:0/4159367935 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb8300bb640 con 0x55e3d3625850 2024-09-12T03:02:02.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.045+0000 7fb84870f740 10 client.25741 fetch_fsmap learned FSMap version 30 2024-09-12T03:02:02.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.045+0000 7fb84870f740 1 -- 172.21.15.5:0/4159367935 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({fsmap=30}) -- 0x55e3d36336c0 con 0x55e3d3625850 2024-09-12T03:02:02.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.045+0000 7fb81f7fe640 1 -- 172.21.15.5:0/4159367935 <== mon.1 v2:172.21.15.148:3300/0 4 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7fb8300c42e0 con 0x55e3d3625850 2024-09-12T03:02:02.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.045+0000 7fb84870f740 10 client.25741 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:02:02.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.045+0000 7fb84870f740 10 client.25741 resolve_mds: validated gid 24502 aka daemon mds.i 2024-09-12T03:02:02.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.045+0000 7fb84870f740 20 client.25741 populate_metadata read hostname 'smithi005' 2024-09-12T03:02:02.050 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.045+0000 7fb84870f740 1 --2- 172.21.15.5:0/4159367935 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55e3d3640830 0x55e3d3642cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:02.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.045+0000 7fb84870f740 4 client.25741 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:02:02.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.045+0000 7fb84870f740 1 -- 172.21.15.5:0/4159367935 --> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e3d3597890 con 0x55e3d3640830 2024-09-12T03:02:02.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.046+0000 7fb837fff640 1 --2- 172.21.15.5:0/4159367935 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55e3d3640830 0x55e3d3642cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:02.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.046+0000 7fb837fff640 1 --2- 172.21.15.5:0/4159367935 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55e3d3640830 0x55e3d3642cf0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=529a9e257fa042ab server_cookie=6115d06b39f6abc2 in_seq=0 out_seq=0 2024-09-12T03:02:02.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.046+0000 7fb81f7fe640 10 client.25741 ms_handle_connect on v2:172.21.15.148:6838/2317600000 2024-09-12T03:02:02.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.047+0000 7fb81f7fe640 1 -- 172.21.15.5:0/4159367935 <== mds.0 v2:172.21.15.148:6838/2317600000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e3d3597890 con 0x55e3d3640830 2024-09-12T03:02:02.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.047+0000 7fb81f7fe640 10 client.25741 handle_command_reply: tid=0 2024-09-12T03:02:02.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.047+0000 7fb84870f740 1 client.25741 shutdown 2024-09-12T03:02:02.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.048+0000 7fb84870f740 1 -- 172.21.15.5:0/4159367935 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e3d3625850 msgr2=0x55e3d362d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:02.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.048+0000 7fb84870f740 1 --2- 172.21.15.5:0/4159367935 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e3d3625850 0x55e3d362d8f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb83003b670 tx=0x7fb830005c00 comp rx=0 tx=0).stop 2024-09-12T03:02:02.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.048+0000 7fb84870f740 1 -- 172.21.15.5:0/4159367935 shutdown_connections 2024-09-12T03:02:02.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.048+0000 7fb84870f740 1 --2- 172.21.15.5:0/4159367935 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55e3d3640830 0x55e3d3642cf0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:02.051 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.048+0000 7fb84870f740 1 --2- 172.21.15.5:0/4159367935 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3d3626170 0x55e3d362de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:02.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.048+0000 7fb84870f740 1 --2- 172.21.15.5:0/4159367935 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e3d3630a30 0x55e3d362e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:02.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.048+0000 7fb84870f740 1 --2- 172.21.15.5:0/4159367935 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e3d3625850 0x55e3d362d8f0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:02.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.048+0000 7fb84870f740 1 -- 172.21.15.5:0/4159367935 >> 172.21.15.5:0/4159367935 conn(0x55e3d3621de0 msgr2=0x55e3d3627b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:02.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.048+0000 7fb81f7fe640 0 client.25741 ms_handle_reset on v2:172.21.15.148:6838/2317600000 2024-09-12T03:02:02.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.048+0000 7fb84870f740 1 -- 172.21.15.5:0/4159367935 shutdown_connections 2024-09-12T03:02:02.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.048+0000 7fb84870f740 1 -- 172.21.15.5:0/4159367935 wait complete. 2024-09-12T03:02:02.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.049+0000 7fb84870f740 20 client.25741 trim_cache size 0 max 16384 2024-09-12T03:02:02.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.078+0000 7fb84870f740 1 Processor -- start 2024-09-12T03:02:02.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.079+0000 7fb84870f740 1 -- start start 2024-09-12T03:02:02.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.079+0000 7fb84870f740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e3d3625850 0x55e3d3648600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:02.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.079+0000 7fb84870f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3d3626170 0x55e3d3648b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:02.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.079+0000 7fb84870f740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e3d3630a30 0x55e3d362ab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:02.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.079+0000 7fb84870f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb82000e3f0 con 0x55e3d3626170 2024-09-12T03:02:02.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.079+0000 7fb84870f740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fb82000ff60 con 0x55e3d3630a30 2024-09-12T03:02:02.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.079+0000 7fb84870f740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb82000eac0 con 0x55e3d3625850 2024-09-12T03:02:02.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.079+0000 7fb8448d3640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e3d3630a30 0x55e3d362ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:02.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.079+0000 7fb8448d3640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e3d3630a30 0x55e3d362ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.5:34426/0 (socket says 172.21.15.5:34426) 2024-09-12T03:02:02.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.079+0000 7fb8448d3640 1 -- 172.21.15.5:0/781958430 learned_addr learned my addr 172.21.15.5:0/781958430 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:02:02.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.079+0000 7fb837fff640 1 --2- 172.21.15.5:0/781958430 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3d3626170 0x55e3d3648b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:02.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.079+0000 7fb83ffff640 1 --2- 172.21.15.5:0/781958430 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e3d3625850 0x55e3d3648600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:02.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.080+0000 7fb837fff640 1 -- 172.21.15.5:0/781958430 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e3d3625850 msgr2=0x55e3d3648600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:02.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.080+0000 7fb837fff640 1 --2- 172.21.15.5:0/781958430 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e3d3625850 0x55e3d3648600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:02.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.080+0000 7fb837fff640 1 -- 172.21.15.5:0/781958430 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e3d3630a30 msgr2=0x55e3d362ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:02.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.080+0000 7fb837fff640 1 --2- 172.21.15.5:0/781958430 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e3d3630a30 0x55e3d362ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:02.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.080+0000 7fb837fff640 1 -- 172.21.15.5:0/781958430 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3d3625fa0 con 0x55e3d3626170 2024-09-12T03:02:02.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.080+0000 7fb8448d3640 1 --2- 172.21.15.5:0/781958430 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e3d3630a30 0x55e3d362ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:02:02.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.080+0000 7fb83ffff640 1 --2- 172.21.15.5:0/781958430 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e3d3625850 0x55e3d3648600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:02:02.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.080+0000 7fb837fff640 1 --2- 172.21.15.5:0/781958430 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3d3626170 0x55e3d3648b40 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x55e3d36435a0 tx=0x7fb828009850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc7c70de49bd9c69 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:02.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.080+0000 7fb835ffb640 1 -- 172.21.15.5:0/781958430 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb82801f020 con 0x55e3d3626170 2024-09-12T03:02:02.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.080+0000 7fb835ffb640 1 -- 172.21.15.5:0/781958430 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb828003ca0 con 0x55e3d3626170 2024-09-12T03:02:02.082 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.080+0000 7fb835ffb640 1 -- 172.21.15.5:0/781958430 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb82801b520 con 0x55e3d3626170 2024-09-12T03:02:02.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.081+0000 7fb84870f740 1 -- 172.21.15.5:0/781958430 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3d3626170 msgr2=0x55e3d3648b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:02.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.081+0000 7fb84870f740 1 --2- 172.21.15.5:0/781958430 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3d3626170 0x55e3d3648b40 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x55e3d36435a0 tx=0x7fb828009850 comp rx=0 tx=0).stop 2024-09-12T03:02:02.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.081+0000 7fb84870f740 1 -- 172.21.15.5:0/781958430 shutdown_connections 2024-09-12T03:02:02.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.081+0000 7fb84870f740 1 --2- 172.21.15.5:0/781958430 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3d3626170 0x55e3d3648b40 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:02.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.081+0000 7fb84870f740 1 --2- 172.21.15.5:0/781958430 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e3d3630a30 0x55e3d362ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:02.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.081+0000 7fb84870f740 1 --2- 172.21.15.5:0/781958430 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e3d3625850 0x55e3d3648600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:02.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.081+0000 7fb84870f740 1 -- 172.21.15.5:0/781958430 >> 172.21.15.5:0/781958430 conn(0x55e3d3621de0 msgr2=0x55e3d362e0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:02.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.081+0000 7fb84870f740 1 -- 172.21.15.5:0/781958430 shutdown_connections 2024-09-12T03:02:02.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.081+0000 7fb84870f740 1 -- 172.21.15.5:0/781958430 wait complete. 2024-09-12T03:02:02.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.082+0000 7fb84870f740 1 -- start start 2024-09-12T03:02:02.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.082+0000 7fb84870f740 1 Processor -- start 2024-09-12T03:02:02.084 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.082+0000 7fb84870f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3d3625850 0x55e3d36346d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:02.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.083+0000 7fb84870f740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e3d3626170 0x55e3d3634c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:02.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.083+0000 7fb84870f740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e3d3630a30 0x55e3d3635150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:02.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.083+0000 7fb84870f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb82000f860 con 0x55e3d3625850 2024-09-12T03:02:02.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.083+0000 7fb84870f740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fb82000f190 con 0x55e3d3630a30 2024-09-12T03:02:02.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.083+0000 7fb84870f740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb82000c3d0 con 0x55e3d3626170 2024-09-12T03:02:02.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.083+0000 7fb84870f740 10 client.0 _ll_register_callbacks cb 0x7fb8300f2a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:02:02.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.083+0000 7fb8448d3640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e3d3630a30 0x55e3d3635150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:02.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.083+0000 7fb837fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e3d3626170 0x55e3d3634c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:02.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.083+0000 7fb8448d3640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e3d3630a30 0x55e3d3635150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.5:34442/0 (socket says 172.21.15.5:34442) 2024-09-12T03:02:02.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.083+0000 7fb8448d3640 1 -- 172.21.15.5:0/330787819 learned_addr learned my addr 172.21.15.5:0/330787819 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:02:02.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.083+0000 7fb83ffff640 1 --2- 172.21.15.5:0/330787819 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3d3625850 0x55e3d36346d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:02.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.083+0000 7fb8448d3640 1 -- 172.21.15.5:0/330787819 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e3d3626170 msgr2=0x55e3d3634c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:02.085 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.083+0000 7fb8448d3640 1 --2- 172.21.15.5:0/330787819 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e3d3626170 0x55e3d3634c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:02.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.083+0000 7fb8448d3640 1 -- 172.21.15.5:0/330787819 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3d3625850 msgr2=0x55e3d36346d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:02.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.083+0000 7fb8448d3640 1 --2- 172.21.15.5:0/330787819 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3d3625850 0x55e3d36346d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:02.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.083+0000 7fb8448d3640 1 -- 172.21.15.5:0/330787819 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3d36aa5d0 con 0x55e3d3630a30 2024-09-12T03:02:02.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.084+0000 7fb8448d3640 1 --2- 172.21.15.5:0/330787819 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e3d3630a30 0x55e3d3635150 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb83800e790 tx=0x7fb838017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=869583c6e9a9421e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:02.086 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.084+0000 7fb81f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2024-09-12T03:02:02.087 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.084+0000 7fb81f7fe640 1 -- 172.21.15.5:0/330787819 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb838022070 con 0x55e3d3630a30 2024-09-12T03:02:02.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.084+0000 7fb81f7fe640 1 -- 172.21.15.5:0/330787819 <== mon.2 v2:172.21.15.146:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fb838004c00 con 0x55e3d3630a30 2024-09-12T03:02:02.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.084+0000 7fb84870f740 1 -- 172.21.15.5:0/330787819 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb828003ca0 con 0x55e3d3630a30 2024-09-12T03:02:02.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.084+0000 7fb81f7fe640 1 -- 172.21.15.5:0/330787819 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb838004e00 con 0x55e3d3630a30 2024-09-12T03:02:02.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.084+0000 7fb84870f740 10 client.25649 fetch_fsmap learned FSMap version 30 2024-09-12T03:02:02.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.084+0000 7fb84870f740 1 -- 172.21.15.5:0/330787819 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e3d36ab7b0 con 0x55e3d3630a30 2024-09-12T03:02:02.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.084+0000 7fb81f7fe640 1 -- 172.21.15.5:0/330787819 <== mon.2 v2:172.21.15.146:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb838039030 con 0x55e3d3630a30 2024-09-12T03:02:02.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.085+0000 7fb81f7fe640 1 -- 172.21.15.5:0/330787819 <== mon.2 v2:172.21.15.146:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7fb838031070 con 0x55e3d3630a30 2024-09-12T03:02:02.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.085+0000 7fb84870f740 10 client.25649 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:02:02.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.085+0000 7fb84870f740 10 client.25649 resolve_mds: validated gid 24502 aka daemon mds.i 2024-09-12T03:02:02.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.085+0000 7fb84870f740 20 client.25649 populate_metadata read hostname 'smithi005' 2024-09-12T03:02:02.088 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.085+0000 7fb84870f740 1 --2- 172.21.15.5:0/330787819 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55e3d36abe60 0x55e3d36ae280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:02.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.085+0000 7fb84870f740 4 client.25649 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.15, "await_for": 45.0}] 2024-09-12T03:02:02.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.085+0000 7fb84870f740 1 -- 172.21.15.5:0/330787819 --> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.15, "await_for": 45.0}) -- 0x7fb828003ca0 con 0x55e3d36abe60 2024-09-12T03:02:02.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.085+0000 7fb83ffff640 1 --2- 172.21.15.5:0/330787819 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55e3d36abe60 0x55e3d36ae280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:02.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.085+0000 7fb81f7fe640 10 client.25649 ms_handle_connect on v2:172.21.15.148:6838/2317600000 2024-09-12T03:02:02.089 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.086+0000 7fb83ffff640 1 --2- 172.21.15.5:0/330787819 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55e3d36abe60 0x55e3d36ae280 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2ea5c730510468b server_cookie=13c0ea5e6db270bb in_seq=0 out_seq=0 2024-09-12T03:02:02.089 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:02:02.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.574+0000 7f9320df4640 1 -- 172.21.15.5:0/1385720701 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f931c100e20 msgr2=0x7f931c101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:02.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.574+0000 7f9320df4640 1 --2- 172.21.15.5:0/1385720701 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f931c100e20 0x7f931c101200 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f930c009900 tx=0x7f930c02a480 comp rx=0 tx=0).stop 2024-09-12T03:02:02.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.575+0000 7f9320df4640 1 -- 172.21.15.5:0/1385720701 shutdown_connections 2024-09-12T03:02:02.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.575+0000 7f9320df4640 1 --2- 172.21.15.5:0/1385720701 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f931c102ef0 0x7f931c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:02.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.575+0000 7f9320df4640 1 --2- 172.21.15.5:0/1385720701 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f931c102570 0x7f931c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:02.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.575+0000 7f9320df4640 1 --2- 172.21.15.5:0/1385720701 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f931c100e20 0x7f931c101200 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:02.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.575+0000 7f9320df4640 1 -- 172.21.15.5:0/1385720701 >> 172.21.15.5:0/1385720701 conn(0x7f931c0fc7d0 msgr2=0x7f931c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:02.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.575+0000 7f9320df4640 1 -- 172.21.15.5:0/1385720701 shutdown_connections 2024-09-12T03:02:02.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.575+0000 7f9320df4640 1 -- 172.21.15.5:0/1385720701 wait complete. 2024-09-12T03:02:02.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.575+0000 7f9320df4640 1 Processor -- start 2024-09-12T03:02:02.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.576+0000 7f9320df4640 1 -- start start 2024-09-12T03:02:02.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.576+0000 7f9320df4640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f931c100e20 0x7f931c199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:02.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.576+0000 7f9320df4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f931c102570 0x7f931c199870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:02.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.576+0000 7f9320df4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f931c102ef0 0x7f931c1a43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:02.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.576+0000 7f9320df4640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f931c110f80 con 0x7f931c102570 2024-09-12T03:02:02.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.576+0000 7f9320df4640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f931c110e00 con 0x7f931c100e20 2024-09-12T03:02:02.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.576+0000 7f9320df4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f931c111100 con 0x7f931c102ef0 2024-09-12T03:02:02.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.576+0000 7f9311d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f931c102570 0x7f931c199870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:02.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.576+0000 7f9311d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f931c102570 0x7f931c199870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51550/0 (socket says 172.21.15.5:51550) 2024-09-12T03:02:02.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.576+0000 7f9311d9b640 1 -- 172.21.15.5:0/992556316 learned_addr learned my addr 172.21.15.5:0/992556316 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:02:02.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.576+0000 7f931ad9d640 1 --2- 172.21.15.5:0/992556316 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f931c102ef0 0x7f931c1a43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:02.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.576+0000 7f931a59c640 1 --2- 172.21.15.5:0/992556316 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f931c100e20 0x7f931c199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:02.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.576+0000 7f9311d9b640 1 -- 172.21.15.5:0/992556316 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f931c102ef0 msgr2=0x7f931c1a43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:02.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.576+0000 7f9311d9b640 1 --2- 172.21.15.5:0/992556316 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f931c102ef0 0x7f931c1a43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:02.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.576+0000 7f9311d9b640 1 -- 172.21.15.5:0/992556316 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f931c100e20 msgr2=0x7f931c199330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:02.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.576+0000 7f9311d9b640 1 --2- 172.21.15.5:0/992556316 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f931c100e20 0x7f931c199330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:02.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.576+0000 7f9311d9b640 1 -- 172.21.15.5:0/992556316 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f931c1a48f0 con 0x7f931c102570 2024-09-12T03:02:02.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.576+0000 7f9311d9b640 1 --2- 172.21.15.5:0/992556316 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f931c102570 0x7f931c199870 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f930400d950 tx=0x7f930400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:02.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.577+0000 7f9313fff640 1 -- 172.21.15.5:0/992556316 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9304017070 con 0x7f931c102570 2024-09-12T03:02:02.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.577+0000 7f9320df4640 1 -- 172.21.15.5:0/992556316 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f931c1a4ac0 con 0x7f931c102570 2024-09-12T03:02:02.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.577+0000 7f9313fff640 1 -- 172.21.15.5:0/992556316 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9304009c30 con 0x7f931c102570 2024-09-12T03:02:02.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.577+0000 7f9313fff640 1 -- 172.21.15.5:0/992556316 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f930400f6b0 con 0x7f931c102570 2024-09-12T03:02:02.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.577+0000 7f9320df4640 1 -- 172.21.15.5:0/992556316 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f931c1a5000 con 0x7f931c102570 2024-09-12T03:02:02.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.579+0000 7f9313fff640 1 -- 172.21.15.5:0/992556316 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f93040109d0 con 0x7f931c102570 2024-09-12T03:02:02.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.579+0000 7f93231cd740 1 Processor -- start 2024-09-12T03:02:02.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.579+0000 7f93231cd740 1 -- start start 2024-09-12T03:02:02.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.580+0000 7f93231cd740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a82ac18b40 0x55a82ac17090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:02.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.580+0000 7f93231cd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a82ac175d0 0x55a82ac17a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:02.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.580+0000 7f93231cd740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a82ac26780 0x55a82ac28b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:02.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.580+0000 7f93231cd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a82ab7d2f0 con 0x55a82ac175d0 2024-09-12T03:02:02.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.580+0000 7f93231cd740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55a82ab24cf0 con 0x55a82ac18b40 2024-09-12T03:02:02.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.580+0000 7f93231cd740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55a82ab792b0 con 0x55a82ac26780 2024-09-12T03:02:02.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.580+0000 7f931a59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a82ac175d0 0x55a82ac17a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:02.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.580+0000 7f931a59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a82ac175d0 0x55a82ac17a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51554/0 (socket says 172.21.15.5:51554) 2024-09-12T03:02:02.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.580+0000 7f931a59c640 1 -- 172.21.15.5:0/2884460110 learned_addr learned my addr 172.21.15.5:0/2884460110 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:02:02.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.580+0000 7f9311d9b640 1 --2- 172.21.15.5:0/2884460110 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a82ac26780 0x55a82ac28b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:02.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.580+0000 7f931ad9d640 1 --2- 172.21.15.5:0/2884460110 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a82ac18b40 0x55a82ac17090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:02.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.580+0000 7f9313fff640 1 --2- 172.21.15.5:0/992556316 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f93080a2c00 0x7f93080a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:02.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.580+0000 7f931a59c640 1 -- 172.21.15.5:0/2884460110 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a82ac26780 msgr2=0x55a82ac28b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:02.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.580+0000 7f931a59c640 1 --2- 172.21.15.5:0/2884460110 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a82ac26780 0x55a82ac28b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:02.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.580+0000 7f931a59c640 1 -- 172.21.15.5:0/2884460110 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a82ac18b40 msgr2=0x55a82ac17090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:02.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.580+0000 7f931a59c640 1 --2- 172.21.15.5:0/2884460110 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a82ac18b40 0x55a82ac17090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:02.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.580+0000 7f931a59c640 1 -- 172.21.15.5:0/2884460110 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a82ab093a0 con 0x55a82ac175d0 2024-09-12T03:02:02.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.580+0000 7f931ad9d640 1 --2- 172.21.15.5:0/2884460110 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a82ac18b40 0x55a82ac17090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:02:02.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.580+0000 7f931ad9d640 1 --2- 172.21.15.5:0/992556316 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f93080a2c00 0x7f93080a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:02.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.580+0000 7f9313fff640 1 -- 172.21.15.5:0/992556316 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f93040ceb30 con 0x7f931c102570 2024-09-12T03:02:02.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.580+0000 7f931a59c640 1 --2- 172.21.15.5:0/2884460110 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a82ac175d0 0x55a82ac17a10 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f930c02a9a0 tx=0x7f930c0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fead268238d1092 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:02.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.581+0000 7f9310d99640 1 -- 172.21.15.5:0/2884460110 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f930c035030 con 0x55a82ac175d0 2024-09-12T03:02:02.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.581+0000 7f9310d99640 1 -- 172.21.15.5:0/2884460110 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f930c040070 con 0x55a82ac175d0 2024-09-12T03:02:02.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.581+0000 7f9310d99640 1 -- 172.21.15.5:0/2884460110 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f930c031a70 con 0x55a82ac175d0 2024-09-12T03:02:02.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.581+0000 7f93231cd740 1 -- 172.21.15.5:0/2884460110 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a82ac175d0 msgr2=0x55a82ac17a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:02.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.581+0000 7f93231cd740 1 --2- 172.21.15.5:0/2884460110 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a82ac175d0 0x55a82ac17a10 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f930c02a9a0 tx=0x7f930c0040c0 comp rx=0 tx=0).stop 2024-09-12T03:02:02.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.581+0000 7f931ad9d640 1 --2- 172.21.15.5:0/992556316 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f93080a2c00 0x7f93080a50c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f9314005fd0 tx=0x7f9314004a40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:02.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.581+0000 7f93231cd740 1 -- 172.21.15.5:0/2884460110 shutdown_connections 2024-09-12T03:02:02.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.581+0000 7f93231cd740 1 --2- 172.21.15.5:0/2884460110 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a82ac175d0 0x55a82ac17a10 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:02.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.581+0000 7f93231cd740 1 --2- 172.21.15.5:0/2884460110 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a82ac26780 0x55a82ac28b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:02.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.581+0000 7f93231cd740 1 --2- 172.21.15.5:0/2884460110 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a82ac18b40 0x55a82ac17090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:02.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.581+0000 7f93231cd740 1 -- 172.21.15.5:0/2884460110 >> 172.21.15.5:0/2884460110 conn(0x55a82abe0310 msgr2=0x55a82ac13c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:02.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.582+0000 7f93231cd740 1 -- 172.21.15.5:0/2884460110 shutdown_connections 2024-09-12T03:02:02.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.582+0000 7f93231cd740 1 -- 172.21.15.5:0/2884460110 wait complete. 2024-09-12T03:02:02.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.582+0000 7f93231cd740 1 -- start start 2024-09-12T03:02:02.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.582+0000 7f93231cd740 1 Processor -- start 2024-09-12T03:02:02.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.582+0000 7f93231cd740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a82ac175d0 0x55a82ac26150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:02.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.582+0000 7f93231cd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a82ac18b40 0x55a82ac23400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:02.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.582+0000 7f93231cd740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a82ac26780 0x55a82ac239f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:02.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.582+0000 7f93231cd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a82ac29420 con 0x55a82ac18b40 2024-09-12T03:02:02.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.582+0000 7f93231cd740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55a82ab38060 con 0x55a82ac26780 2024-09-12T03:02:02.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.582+0000 7f93231cd740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55a82ac295a0 con 0x55a82ac175d0 2024-09-12T03:02:02.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.582+0000 7f93231cd740 10 client.0 _ll_register_callbacks cb 0x55a82a88b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:02:02.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.582+0000 7f9311d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a82ac18b40 0x55a82ac23400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:02.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.582+0000 7f9311d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a82ac18b40 0x55a82ac23400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51566/0 (socket says 172.21.15.5:51566) 2024-09-12T03:02:02.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.582+0000 7f9311d9b640 1 -- 172.21.15.5:0/1520433717 learned_addr learned my addr 172.21.15.5:0/1520433717 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:02:02.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.582+0000 7f931a59c640 1 --2- 172.21.15.5:0/1520433717 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a82ac175d0 0x55a82ac26150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:02.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.582+0000 7f931ad9d640 1 --2- 172.21.15.5:0/1520433717 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a82ac26780 0x55a82ac239f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:02.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.583+0000 7f9311d9b640 1 -- 172.21.15.5:0/1520433717 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a82ac175d0 msgr2=0x55a82ac26150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:02.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.583+0000 7f9311d9b640 1 --2- 172.21.15.5:0/1520433717 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a82ac175d0 0x55a82ac26150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:02.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.583+0000 7f9311d9b640 1 -- 172.21.15.5:0/1520433717 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a82ac26780 msgr2=0x55a82ac239f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:02.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.583+0000 7f9311d9b640 1 --2- 172.21.15.5:0/1520433717 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a82ac26780 0x55a82ac239f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:02.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.583+0000 7f9311d9b640 1 -- 172.21.15.5:0/1520433717 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a82a8ee3c0 con 0x55a82ac18b40 2024-09-12T03:02:02.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.583+0000 7f9311d9b640 1 --2- 172.21.15.5:0/1520433717 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a82ac18b40 0x55a82ac23400 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f930409b150 tx=0x7f93040d2200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86be3ada4b1e2f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:02.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.583+0000 7f9301ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T03:02:02.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.583+0000 7f9301ffb640 1 -- 172.21.15.5:0/1520433717 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9304020070 con 0x55a82ac18b40 2024-09-12T03:02:02.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.583+0000 7f9301ffb640 1 -- 172.21.15.5:0/1520433717 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f930409a8a0 con 0x55a82ac18b40 2024-09-12T03:02:02.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.583+0000 7f93231cd740 1 -- 172.21.15.5:0/1520433717 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f930c040070 con 0x55a82ac18b40 2024-09-12T03:02:02.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.583+0000 7f9301ffb640 1 -- 172.21.15.5:0/1520433717 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f930409adb0 con 0x55a82ac18b40 2024-09-12T03:02:02.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.583+0000 7f93231cd740 10 client.16233 fetch_fsmap learned FSMap version 30 2024-09-12T03:02:02.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.583+0000 7f93231cd740 1 -- 172.21.15.5:0/1520433717 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a82ac34ec0 con 0x55a82ac18b40 2024-09-12T03:02:02.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.584+0000 7f9301ffb640 1 -- 172.21.15.5:0/1520433717 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f930c040070 con 0x55a82ac18b40 2024-09-12T03:02:02.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.584+0000 7f9301ffb640 1 -- 172.21.15.5:0/1520433717 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f93040971a0 con 0x55a82ac18b40 2024-09-12T03:02:02.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.584+0000 7f93231cd740 10 client.16233 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:02:02.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.584+0000 7f93231cd740 10 client.16233 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-12T03:02:02.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.584+0000 7f93231cd740 20 client.16233 populate_metadata read hostname 'smithi005' 2024-09-12T03:02:02.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.584+0000 7f93231cd740 1 --2- 172.21.15.5:0/1520433717 >> [v2:172.21.15.146:6836/1673370937,v1:172.21.15.146:6837/1673370937] conn(0x55a82ac36210 0x55a82ac38600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:02.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.584+0000 7f93231cd740 4 client.16233 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:02:02.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.584+0000 7f93231cd740 1 -- 172.21.15.5:0/1520433717 --> [v2:172.21.15.146:6836/1673370937,v1:172.21.15.146:6837/1673370937] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a82ab8a860 con 0x55a82ac36210 2024-09-12T03:02:02.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.585+0000 7f931a59c640 1 --2- 172.21.15.5:0/1520433717 >> [v2:172.21.15.146:6836/1673370937,v1:172.21.15.146:6837/1673370937] conn(0x55a82ac36210 0x55a82ac38600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:02.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.585+0000 7f931a59c640 1 --2- 172.21.15.5:0/1520433717 >> [v2:172.21.15.146:6836/1673370937,v1:172.21.15.146:6837/1673370937] conn(0x55a82ac36210 0x55a82ac38600 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1261ea5d6b3cf2b server_cookie=1334752ce5203b5c in_seq=0 out_seq=0 2024-09-12T03:02:02.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.585+0000 7f9301ffb640 10 client.16233 ms_handle_connect on v2:172.21.15.146:6836/1673370937 2024-09-12T03:02:02.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.586+0000 7f9301ffb640 1 -- 172.21.15.5:0/1520433717 <== mds.4 v2:172.21.15.146:6836/1673370937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a82ab8a860 con 0x55a82ac36210 2024-09-12T03:02:02.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.586+0000 7f9301ffb640 10 client.16233 handle_command_reply: tid=0 2024-09-12T03:02:02.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.586+0000 7f93231cd740 1 client.16233 shutdown 2024-09-12T03:02:02.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.586+0000 7f93231cd740 1 -- 172.21.15.5:0/1520433717 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a82ac18b40 msgr2=0x55a82ac23400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:02.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.586+0000 7f93231cd740 1 --2- 172.21.15.5:0/1520433717 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a82ac18b40 0x55a82ac23400 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f930409b150 tx=0x7f93040d2200 comp rx=0 tx=0).stop 2024-09-12T03:02:02.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.586+0000 7f93231cd740 1 -- 172.21.15.5:0/1520433717 shutdown_connections 2024-09-12T03:02:02.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.586+0000 7f93231cd740 1 --2- 172.21.15.5:0/1520433717 >> [v2:172.21.15.146:6836/1673370937,v1:172.21.15.146:6837/1673370937] conn(0x55a82ac36210 0x55a82ac38600 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:02.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.586+0000 7f93231cd740 1 --2- 172.21.15.5:0/1520433717 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a82ac18b40 0x55a82ac23400 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:02.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.586+0000 7f93231cd740 1 --2- 172.21.15.5:0/1520433717 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a82ac26780 0x55a82ac239f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:02.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.586+0000 7f9301ffb640 0 client.16233 ms_handle_reset on v2:172.21.15.146:6836/1673370937 2024-09-12T03:02:02.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.586+0000 7f93231cd740 1 --2- 172.21.15.5:0/1520433717 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a82ac175d0 0x55a82ac26150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:02.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.586+0000 7f93231cd740 1 -- 172.21.15.5:0/1520433717 >> 172.21.15.5:0/1520433717 conn(0x55a82abe0310 msgr2=0x55a82ac19930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:02.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.586+0000 7f93231cd740 1 -- 172.21.15.5:0/1520433717 shutdown_connections 2024-09-12T03:02:02.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.586+0000 7f93231cd740 1 -- 172.21.15.5:0/1520433717 wait complete. 2024-09-12T03:02:02.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.586+0000 7f93231cd740 20 client.16233 trim_cache size 0 max 16384 2024-09-12T03:02:02.600 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.599+0000 7f93231cd740 1 Processor -- start 2024-09-12T03:02:02.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.599+0000 7f93231cd740 1 -- start start 2024-09-12T03:02:02.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.599+0000 7f93231cd740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a82ac175d0 0x55a82ac13e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:02.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.599+0000 7f93231cd740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a82ac18b40 0x55a82ac326f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:02.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.599+0000 7f93231cd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a82ac26780 0x55a82ac22cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:02.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.599+0000 7f93231cd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f931c1ab360 con 0x55a82ac26780 2024-09-12T03:02:02.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.599+0000 7f93231cd740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f931c1aced0 con 0x55a82ac18b40 2024-09-12T03:02:02.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.599+0000 7f93231cd740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f931c1aba30 con 0x55a82ac175d0 2024-09-12T03:02:02.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.600+0000 7f931ad9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a82ac26780 0x55a82ac22cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:02.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.600+0000 7f931a59c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a82ac175d0 0x55a82ac13e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:02.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.600+0000 7f931a59c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a82ac175d0 0x55a82ac13e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.5:42726/0 (socket says 172.21.15.5:42726) 2024-09-12T03:02:02.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.600+0000 7f931a59c640 1 -- 172.21.15.5:0/115800377 learned_addr learned my addr 172.21.15.5:0/115800377 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:02:02.601 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.600+0000 7f9311d9b640 1 --2- 172.21.15.5:0/115800377 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a82ac18b40 0x55a82ac326f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:02.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.600+0000 7f931a59c640 1 -- 172.21.15.5:0/115800377 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a82ac18b40 msgr2=0x55a82ac326f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:02.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.600+0000 7f931a59c640 1 --2- 172.21.15.5:0/115800377 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a82ac18b40 0x55a82ac326f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:02.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.600+0000 7f931a59c640 1 -- 172.21.15.5:0/115800377 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a82ac26780 msgr2=0x55a82ac22cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:02.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.600+0000 7f931a59c640 1 --2- 172.21.15.5:0/115800377 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a82ac26780 0x55a82ac22cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:02.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.600+0000 7f931a59c640 1 -- 172.21.15.5:0/115800377 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a82ac18760 con 0x55a82ac175d0 2024-09-12T03:02:02.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.600+0000 7f931ad9d640 1 --2- 172.21.15.5:0/115800377 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a82ac26780 0x55a82ac22cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:02:02.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.600+0000 7f931a59c640 1 --2- 172.21.15.5:0/115800377 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a82ac175d0 0x55a82ac13e10 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x55a82ac14ac0 tx=0x7f930c0045b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa2be9902938ef5e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:02.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.600+0000 7f9303fff640 1 -- 172.21.15.5:0/115800377 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f930c035040 con 0x55a82ac175d0 2024-09-12T03:02:02.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.601+0000 7f9303fff640 1 -- 172.21.15.5:0/115800377 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f930c040030 con 0x55a82ac175d0 2024-09-12T03:02:02.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.601+0000 7f9303fff640 1 -- 172.21.15.5:0/115800377 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f930c0035f0 con 0x55a82ac175d0 2024-09-12T03:02:02.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.601+0000 7f93231cd740 1 -- 172.21.15.5:0/115800377 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a82ac175d0 msgr2=0x55a82ac13e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:02.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.601+0000 7f93231cd740 1 --2- 172.21.15.5:0/115800377 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a82ac175d0 0x55a82ac13e10 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x55a82ac14ac0 tx=0x7f930c0045b0 comp rx=0 tx=0).stop 2024-09-12T03:02:02.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.601+0000 7f93231cd740 1 -- 172.21.15.5:0/115800377 shutdown_connections 2024-09-12T03:02:02.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.601+0000 7f93231cd740 1 --2- 172.21.15.5:0/115800377 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a82ac26780 0x55a82ac22cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:02.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.601+0000 7f93231cd740 1 --2- 172.21.15.5:0/115800377 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a82ac18b40 0x55a82ac326f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:02.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.601+0000 7f93231cd740 1 --2- 172.21.15.5:0/115800377 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a82ac175d0 0x55a82ac13e10 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:02.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.601+0000 7f93231cd740 1 -- 172.21.15.5:0/115800377 >> 172.21.15.5:0/115800377 conn(0x55a82abe0310 msgr2=0x55a82ac13700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:02.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.601+0000 7f93231cd740 1 -- 172.21.15.5:0/115800377 shutdown_connections 2024-09-12T03:02:02.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.601+0000 7f93231cd740 1 -- 172.21.15.5:0/115800377 wait complete. 2024-09-12T03:02:02.606 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.601+0000 7f93231cd740 1 -- start start 2024-09-12T03:02:02.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.602+0000 7f93231cd740 1 Processor -- start 2024-09-12T03:02:02.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.602+0000 7f93231cd740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a82ac175d0 0x55a82ac203b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:02.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.602+0000 7f93231cd740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a82ac18b40 0x55a82ac208f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:02.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.602+0000 7f93231cd740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a82ac26780 0x55a82ac20e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:02.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.602+0000 7f93231cd740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f931c1ac7d0 con 0x55a82ac175d0 2024-09-12T03:02:02.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.602+0000 7f93231cd740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f931c1ac100 con 0x55a82ac26780 2024-09-12T03:02:02.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.602+0000 7f93231cd740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f931c1a9340 con 0x55a82ac18b40 2024-09-12T03:02:02.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.602+0000 7f93231cd740 10 client.0 _ll_register_callbacks cb 0x7f930400df40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:02:02.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.602+0000 7f931a59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a82ac175d0 0x55a82ac203b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:02.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.602+0000 7f931a59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a82ac175d0 0x55a82ac203b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51572/0 (socket says 172.21.15.5:51572) 2024-09-12T03:02:02.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.602+0000 7f9311d9b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a82ac18b40 0x55a82ac208f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:02.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.602+0000 7f931a59c640 1 -- 172.21.15.5:0/752740545 learned_addr learned my addr 172.21.15.5:0/752740545 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:02:02.607 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.602+0000 7f931ad9d640 1 --2- 172.21.15.5:0/752740545 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a82ac26780 0x55a82ac20e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:02.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.602+0000 7f931a59c640 1 -- 172.21.15.5:0/752740545 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a82ac18b40 msgr2=0x55a82ac208f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:02.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.602+0000 7f931a59c640 1 --2- 172.21.15.5:0/752740545 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a82ac18b40 0x55a82ac208f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:02.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.602+0000 7f931a59c640 1 -- 172.21.15.5:0/752740545 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a82ac26780 msgr2=0x55a82ac20e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:02.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.602+0000 7f931a59c640 1 --2- 172.21.15.5:0/752740545 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a82ac26780 0x55a82ac20e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:02.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.602+0000 7f931a59c640 1 -- 172.21.15.5:0/752740545 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a82ac9fea0 con 0x55a82ac175d0 2024-09-12T03:02:02.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.602+0000 7f931ad9d640 1 --2- 172.21.15.5:0/752740545 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a82ac26780 0x55a82ac20e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:02:02.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.603+0000 7f931a59c640 1 --2- 172.21.15.5:0/752740545 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a82ac175d0 0x55a82ac203b0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f930c001eb0 tx=0x7f930c005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f670d536f9c488f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:02.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.603+0000 7f9301ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T03:02:02.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.603+0000 7f9301ffb640 1 -- 172.21.15.5:0/752740545 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f930c05e060 con 0x55a82ac175d0 2024-09-12T03:02:02.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.603+0000 7f9301ffb640 1 -- 172.21.15.5:0/752740545 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f930c004f70 con 0x55a82ac175d0 2024-09-12T03:02:02.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.603+0000 7f93231cd740 1 -- 172.21.15.5:0/752740545 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f930c040030 con 0x55a82ac175d0 2024-09-12T03:02:02.608 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.603+0000 7f9301ffb640 1 -- 172.21.15.5:0/752740545 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f930c035050 con 0x55a82ac175d0 2024-09-12T03:02:02.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.603+0000 7f93231cd740 10 client.16239 fetch_fsmap learned FSMap version 30 2024-09-12T03:02:02.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.603+0000 7f93231cd740 1 -- 172.21.15.5:0/752740545 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a82aca10e0 con 0x55a82ac175d0 2024-09-12T03:02:02.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.603+0000 7f9301ffb640 1 -- 172.21.15.5:0/752740545 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f930c040030 con 0x55a82ac175d0 2024-09-12T03:02:02.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.603+0000 7f9301ffb640 1 -- 172.21.15.5:0/752740545 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f930c031bf0 con 0x55a82ac175d0 2024-09-12T03:02:02.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.603+0000 7f93231cd740 10 client.16239 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:02:02.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.603+0000 7f93231cd740 10 client.16239 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-12T03:02:02.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.603+0000 7f93231cd740 20 client.16239 populate_metadata read hostname 'smithi005' 2024-09-12T03:02:02.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.603+0000 7f93231cd740 1 --2- 172.21.15.5:0/752740545 >> [v2:172.21.15.146:6836/1673370937,v1:172.21.15.146:6837/1673370937] conn(0x55a82aca1310 0x55a82aca3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:02.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.603+0000 7f93231cd740 4 client.16239 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:02:02.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.603+0000 7f93231cd740 1 -- 172.21.15.5:0/752740545 --> [v2:172.21.15.146:6836/1673370937,v1:172.21.15.146:6837/1673370937] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a82ab84d70 con 0x55a82aca1310 2024-09-12T03:02:02.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.604+0000 7f9311d9b640 1 --2- 172.21.15.5:0/752740545 >> [v2:172.21.15.146:6836/1673370937,v1:172.21.15.146:6837/1673370937] conn(0x55a82aca1310 0x55a82aca3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:02.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.604+0000 7f9311d9b640 1 --2- 172.21.15.5:0/752740545 >> [v2:172.21.15.146:6836/1673370937,v1:172.21.15.146:6837/1673370937] conn(0x55a82aca1310 0x55a82aca3700 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7918c8caafecf87 server_cookie=283b4c60f93a639b in_seq=0 out_seq=0 2024-09-12T03:02:02.609 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.604+0000 7f9301ffb640 10 client.16239 ms_handle_connect on v2:172.21.15.146:6836/1673370937 2024-09-12T03:02:02.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.604+0000 7f9301ffb640 1 -- 172.21.15.5:0/752740545 <== mds.4 v2:172.21.15.146:6836/1673370937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a82ab84d70 con 0x55a82aca1310 2024-09-12T03:02:02.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.604+0000 7f9301ffb640 10 client.16239 handle_command_reply: tid=0 2024-09-12T03:02:02.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.604+0000 7f93231cd740 1 client.16239 shutdown 2024-09-12T03:02:02.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.604+0000 7f93231cd740 1 -- 172.21.15.5:0/752740545 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a82ac175d0 msgr2=0x55a82ac203b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:02.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.604+0000 7f93231cd740 1 --2- 172.21.15.5:0/752740545 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a82ac175d0 0x55a82ac203b0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f930c001eb0 tx=0x7f930c005b50 comp rx=0 tx=0).stop 2024-09-12T03:02:02.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.604+0000 7f93231cd740 1 -- 172.21.15.5:0/752740545 shutdown_connections 2024-09-12T03:02:02.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.604+0000 7f93231cd740 1 --2- 172.21.15.5:0/752740545 >> [v2:172.21.15.146:6836/1673370937,v1:172.21.15.146:6837/1673370937] conn(0x55a82aca1310 0x55a82aca3700 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:02.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.604+0000 7f93231cd740 1 --2- 172.21.15.5:0/752740545 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a82ac26780 0x55a82ac20e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:02.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.604+0000 7f93231cd740 1 --2- 172.21.15.5:0/752740545 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a82ac18b40 0x55a82ac208f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:02.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.604+0000 7f93231cd740 1 --2- 172.21.15.5:0/752740545 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a82ac175d0 0x55a82ac203b0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:02.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.604+0000 7f9301ffb640 0 client.16239 ms_handle_reset on v2:172.21.15.146:6836/1673370937 2024-09-12T03:02:02.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.604+0000 7f93231cd740 1 -- 172.21.15.5:0/752740545 >> 172.21.15.5:0/752740545 conn(0x55a82abe0310 msgr2=0x55a82ac24c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:02.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.604+0000 7f93231cd740 1 -- 172.21.15.5:0/752740545 shutdown_connections 2024-09-12T03:02:02.610 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.604+0000 7f93231cd740 1 -- 172.21.15.5:0/752740545 wait complete. 2024-09-12T03:02:02.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.605+0000 7f93231cd740 20 client.16239 trim_cache size 0 max 16384 2024-09-12T03:02:02.611 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-12T03:02:02.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.605+0000 7f9320df4640 1 -- 172.21.15.5:0/992556316 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f93080a2c00 msgr2=0x7f93080a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:02.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.605+0000 7f9320df4640 1 --2- 172.21.15.5:0/992556316 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f93080a2c00 0x7f93080a50c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f9314005fd0 tx=0x7f9314004a40 comp rx=0 tx=0).stop 2024-09-12T03:02:02.611 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.605+0000 7f9320df4640 1 -- 172.21.15.5:0/992556316 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f931c102570 msgr2=0x7f931c199870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:02.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.605+0000 7f9320df4640 1 --2- 172.21.15.5:0/992556316 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f931c102570 0x7f931c199870 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f930400d950 tx=0x7f930400de10 comp rx=0 tx=0).stop 2024-09-12T03:02:02.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.605+0000 7f9320df4640 1 -- 172.21.15.5:0/992556316 shutdown_connections 2024-09-12T03:02:02.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.605+0000 7f9320df4640 1 --2- 172.21.15.5:0/992556316 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f93080a2c00 0x7f93080a50c0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:02.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.605+0000 7f9320df4640 1 --2- 172.21.15.5:0/992556316 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f931c102570 0x7f931c199870 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:02.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.605+0000 7f9320df4640 1 --2- 172.21.15.5:0/992556316 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f931c102ef0 0x7f931c1a43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:02.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.605+0000 7f9320df4640 1 --2- 172.21.15.5:0/992556316 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f931c100e20 0x7f931c199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:02.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.605+0000 7f9320df4640 1 -- 172.21.15.5:0/992556316 >> 172.21.15.5:0/992556316 conn(0x7f931c0fc7d0 msgr2=0x7f931c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:02.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.605+0000 7f9320df4640 1 -- 172.21.15.5:0/992556316 shutdown_connections 2024-09-12T03:02:02.612 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.605+0000 7f9320df4640 1 -- 172.21.15.5:0/992556316 wait complete. 2024-09-12T03:02:02.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.981+0000 7fb81f7fe640 1 -- 172.21.15.5:0/330787819 <== mds.0 v2:172.21.15.148:6838/2317600000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x7fb828003ca0 con 0x55e3d36abe60 2024-09-12T03:02:02.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.981+0000 7fb81f7fe640 10 client.25649 handle_command_reply: tid=0 2024-09-12T03:02:02.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.981+0000 7fb84870f740 1 client.25649 shutdown 2024-09-12T03:02:02.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.981+0000 7fb84870f740 1 -- 172.21.15.5:0/330787819 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e3d3630a30 msgr2=0x55e3d3635150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:02.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.981+0000 7fb84870f740 1 --2- 172.21.15.5:0/330787819 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e3d3630a30 0x55e3d3635150 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb83800e790 tx=0x7fb838017000 comp rx=0 tx=0).stop 2024-09-12T03:02:02.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.981+0000 7fb84870f740 1 -- 172.21.15.5:0/330787819 shutdown_connections 2024-09-12T03:02:02.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.981+0000 7fb84870f740 1 --2- 172.21.15.5:0/330787819 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55e3d3630a30 0x55e3d3635150 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:02.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.981+0000 7fb84870f740 1 --2- 172.21.15.5:0/330787819 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55e3d3626170 0x55e3d3634c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:02.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.981+0000 7fb84870f740 1 --2- 172.21.15.5:0/330787819 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55e3d36abe60 0x55e3d36ae280 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:02.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.981+0000 7fb84870f740 1 --2- 172.21.15.5:0/330787819 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e3d3625850 0x55e3d36346d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:02.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.981+0000 7fb81f7fe640 0 client.25649 ms_handle_reset on v2:172.21.15.148:6838/2317600000 2024-09-12T03:02:02.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.981+0000 7fb84870f740 1 -- 172.21.15.5:0/330787819 >> 172.21.15.5:0/330787819 conn(0x55e3d3621de0 msgr2=0x55e3d363b8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:02.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.982+0000 7fb84870f740 1 -- 172.21.15.5:0/330787819 shutdown_connections 2024-09-12T03:02:02.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.982+0000 7fb84870f740 1 -- 172.21.15.5:0/330787819 wait complete. 2024-09-12T03:02:02.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.982+0000 7fb84870f740 20 client.25649 trim_cache size 0 max 16384 2024-09-12T03:02:02.984 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-12T03:02:02.984 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 30, 2024-09-12T03:02:02.984 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24502, 2024-09-12T03:02:02.984 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 14, 2024-09-12T03:02:02.984 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-09-12T03:02:02.984 INFO:teuthology.orchestra.run.smithi005.stdout: "42c0cc77": { 2024-09-12T03:02:02.984 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 14, 2024-09-12T03:02:02.985 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-09-12T03:02:02.985 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-12T03:02:02.985 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-09-12T03:02:02.985 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-12T03:02:02.985 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:02:02.985 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 90.0, 2024-09-12T03:02:02.985 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 140.2, 2024-09-12T03:02:02.985 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-09-12T03:02:02.985 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-09-12T03:02:02.985 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-09-12T03:02:02.985 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-12T03:02:02.985 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-09-12T03:02:02.985 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-12T03:02:02.986 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:02:02.986 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:02:02.986 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:02:02.986 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:02:02.986 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:02:02.986 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-12T03:02:02.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.982+0000 7fb846336640 1 -- 172.21.15.5:0/310386319 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fb82c0a2bf0 msgr2=0x7fb82c0a50b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:02.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.982+0000 7fb846336640 1 --2- 172.21.15.5:0/310386319 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fb82c0a2bf0 0x7fb82c0a50b0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fb83801f000 tx=0x7fb83801bdf0 comp rx=0 tx=0).stop 2024-09-12T03:02:02.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.982+0000 7fb846336640 1 -- 172.21.15.5:0/310386319 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb840100e20 msgr2=0x7fb84019a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:02.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.982+0000 7fb846336640 1 --2- 172.21.15.5:0/310386319 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb840100e20 0x7fb84019a6f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb830009230 tx=0x7fb830004040 comp rx=0 tx=0).stop 2024-09-12T03:02:02.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.982+0000 7fb846336640 1 -- 172.21.15.5:0/310386319 shutdown_connections 2024-09-12T03:02:02.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.982+0000 7fb846336640 1 --2- 172.21.15.5:0/310386319 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fb82c0a2bf0 0x7fb82c0a50b0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:02.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.982+0000 7fb846336640 1 --2- 172.21.15.5:0/310386319 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb840102570 0x7fb84019ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:02.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.982+0000 7fb846336640 1 --2- 172.21.15.5:0/310386319 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb840102ef0 0x7fb84019efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:02.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.982+0000 7fb846336640 1 --2- 172.21.15.5:0/310386319 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb840100e20 0x7fb84019a6f0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:02.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.982+0000 7fb846336640 1 -- 172.21.15.5:0/310386319 >> 172.21.15.5:0/310386319 conn(0x7fb8400fc7d0 msgr2=0x7fb84010bdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:02.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.982+0000 7fb846336640 1 -- 172.21.15.5:0/310386319 shutdown_connections 2024-09-12T03:02:02.987 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:02.982+0000 7fb846336640 1 -- 172.21.15.5:0/310386319 wait complete. 2024-09-12T03:02:03.576 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-12T03:02:03.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:02:03 smithi005 ceph-mon[31197]: pgmap v449: 97 pgs: 97 active+clean; 108 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 3.8 MiB/s wr, 691 op/s 2024-09-12T03:02:03.910 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '42c0cc77', quiesce took 6.2 seconds. Will release after: 13.95 2024-09-12T03:02:03.915 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:02:03 smithi148 ceph-mon[35090]: pgmap v449: 97 pgs: 97 active+clean; 108 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 3.8 MiB/s wr, 691 op/s 2024-09-12T03:02:03.946 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:02:03.956 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:02:03 smithi146 ceph-mon[35011]: pgmap v449: 97 pgs: 97 active+clean; 108 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 3.8 MiB/s wr, 691 op/s 2024-09-12T03:02:04.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.422+0000 7f56c3d12640 1 -- 172.21.15.5:0/1822157073 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56bc10c170 msgr2=0x7f56bc10e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:04.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.422+0000 7f56c3d12640 1 --2- 172.21.15.5:0/1822157073 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56bc10c170 0x7f56bc10e610 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f56b4009ca0 tx=0x7f56b402a950 comp rx=0 tx=0).stop 2024-09-12T03:02:04.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.422+0000 7f56c3d12640 1 -- 172.21.15.5:0/1822157073 shutdown_connections 2024-09-12T03:02:04.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.422+0000 7f56c3d12640 1 --2- 172.21.15.5:0/1822157073 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56bc10c170 0x7f56bc10e610 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:04.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.422+0000 7f56c3d12640 1 --2- 172.21.15.5:0/1822157073 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f56bc103660 0x7f56bc10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:04.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.422+0000 7f56c3d12640 1 --2- 172.21.15.5:0/1822157073 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56bc102d40 0x7f56bc103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:04.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.423+0000 7f56c3d12640 1 -- 172.21.15.5:0/1822157073 >> 172.21.15.5:0/1822157073 conn(0x7f56bc0fc7a0 msgr2=0x7f56bc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:04.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.423+0000 7f56c3d12640 1 -- 172.21.15.5:0/1822157073 shutdown_connections 2024-09-12T03:02:04.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.423+0000 7f56c3d12640 1 -- 172.21.15.5:0/1822157073 wait complete. 2024-09-12T03:02:04.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.423+0000 7f56c3d12640 1 Processor -- start 2024-09-12T03:02:04.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.423+0000 7f56c3d12640 1 -- start start 2024-09-12T03:02:04.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.423+0000 7f56c3d12640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f56bc102d40 0x7f56bc1009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:04.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.423+0000 7f56c3d12640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56bc103660 0x7f56bc100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:04.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.423+0000 7f56c3d12640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56bc10c170 0x7f56bc101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:04.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.424+0000 7f56c3d12640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f56bc111f70 con 0x7f56bc103660 2024-09-12T03:02:04.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.424+0000 7f56c3d12640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f56bc111df0 con 0x7f56bc102d40 2024-09-12T03:02:04.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.424+0000 7f56c3d12640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f56bc1120f0 con 0x7f56bc10c170 2024-09-12T03:02:04.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.424+0000 7f56c12ad640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56bc103660 0x7f56bc100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:04.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.424+0000 7f56c1aae640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f56bc102d40 0x7f56bc1009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:04.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.424+0000 7f56c22af640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56bc10c170 0x7f56bc101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:04.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.424+0000 7f56c12ad640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56bc103660 0x7f56bc100ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38248/0 (socket says 172.21.15.5:38248) 2024-09-12T03:02:04.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.424+0000 7f56c12ad640 1 -- 172.21.15.5:0/3555479532 learned_addr learned my addr 172.21.15.5:0/3555479532 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:02:04.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.424+0000 7f56c12ad640 1 -- 172.21.15.5:0/3555479532 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56bc10c170 msgr2=0x7f56bc101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:04.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.424+0000 7f56c12ad640 1 --2- 172.21.15.5:0/3555479532 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56bc10c170 0x7f56bc101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:04.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.424+0000 7f56c12ad640 1 -- 172.21.15.5:0/3555479532 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f56bc102d40 msgr2=0x7f56bc1009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:04.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.424+0000 7f56c12ad640 1 --2- 172.21.15.5:0/3555479532 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f56bc102d40 0x7f56bc1009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:04.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.424+0000 7f56c12ad640 1 -- 172.21.15.5:0/3555479532 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56bc1aad80 con 0x7f56bc103660 2024-09-12T03:02:04.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.424+0000 7f56c12ad640 1 --2- 172.21.15.5:0/3555479532 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56bc103660 0x7f56bc100ee0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f56b000c940 tx=0x7f56b000ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:04.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.424+0000 7f56aeffd640 1 -- 172.21.15.5:0/3555479532 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b0004220 con 0x7f56bc103660 2024-09-12T03:02:04.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.424+0000 7f56aeffd640 1 -- 172.21.15.5:0/3555479532 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56b00043c0 con 0x7f56bc103660 2024-09-12T03:02:04.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.424+0000 7f56c3d12640 1 -- 172.21.15.5:0/3555479532 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56bc1aafb0 con 0x7f56bc103660 2024-09-12T03:02:04.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.425+0000 7f56aeffd640 1 -- 172.21.15.5:0/3555479532 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b0004a00 con 0x7f56bc103660 2024-09-12T03:02:04.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.425+0000 7f56c3d12640 1 -- 172.21.15.5:0/3555479532 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56bc1ab460 con 0x7f56bc103660 2024-09-12T03:02:04.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.427+0000 7f56aeffd640 1 -- 172.21.15.5:0/3555479532 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f56b00079b0 con 0x7f56bc103660 2024-09-12T03:02:04.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.427+0000 7f56c60eb740 1 Processor -- start 2024-09-12T03:02:04.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.427+0000 7f56c60eb740 1 -- start start 2024-09-12T03:02:04.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.427+0000 7f56c60eb740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5565bacdfb40 0x5565bacde090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:04.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.427+0000 7f56c60eb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565bacde5d0 0x5565bacdea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:04.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.427+0000 7f56c60eb740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5565bace9370 0x5565baceb760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:04.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.427+0000 7f56c60eb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5565bac442f0 con 0x5565bacde5d0 2024-09-12T03:02:04.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.427+0000 7f56c60eb740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5565babebcf0 con 0x5565bacdfb40 2024-09-12T03:02:04.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.427+0000 7f56c60eb740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5565bac402b0 con 0x5565bace9370 2024-09-12T03:02:04.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.428+0000 7f56c1aae640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565bacde5d0 0x5565bacdea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:04.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.428+0000 7f56c1aae640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565bacde5d0 0x5565bacdea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38260/0 (socket says 172.21.15.5:38260) 2024-09-12T03:02:04.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.428+0000 7f56c1aae640 1 -- 172.21.15.5:0/1435130685 learned_addr learned my addr 172.21.15.5:0/1435130685 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:02:04.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.428+0000 7f56c12ad640 1 --2- 172.21.15.5:0/1435130685 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5565bace9370 0x5565baceb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:04.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.428+0000 7f56c22af640 1 --2- 172.21.15.5:0/1435130685 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5565bacdfb40 0x5565bacde090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:04.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.428+0000 7f56aeffd640 1 --2- 172.21.15.5:0/3555479532 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f56a80a2c00 0x7f56a80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:04.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.428+0000 7f56c1aae640 1 -- 172.21.15.5:0/1435130685 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5565bace9370 msgr2=0x5565baceb760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:04.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.428+0000 7f56c1aae640 1 --2- 172.21.15.5:0/1435130685 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5565bace9370 0x5565baceb760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:04.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.428+0000 7f56c1aae640 1 -- 172.21.15.5:0/1435130685 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5565bacdfb40 msgr2=0x5565bacde090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:04.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.428+0000 7f56c1aae640 1 --2- 172.21.15.5:0/1435130685 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5565bacdfb40 0x5565bacde090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:04.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.428+0000 7f56c1aae640 1 -- 172.21.15.5:0/1435130685 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565babd03a0 con 0x5565bacde5d0 2024-09-12T03:02:04.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.428+0000 7f56c22af640 1 --2- 172.21.15.5:0/3555479532 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f56a80a2c00 0x7f56a80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:04.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.428+0000 7f56aeffd640 1 -- 172.21.15.5:0/3555479532 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f56b00cee50 con 0x7f56bc103660 2024-09-12T03:02:04.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.428+0000 7f56c1aae640 1 --2- 172.21.15.5:0/1435130685 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565bacde5d0 0x5565bacdea10 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f56b800bd10 tx=0x7f56b80107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82718f45d125c8b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:04.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.428+0000 7f56a7fff640 1 -- 172.21.15.5:0/1435130685 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b8004030 con 0x5565bacde5d0 2024-09-12T03:02:04.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.428+0000 7f56a7fff640 1 -- 172.21.15.5:0/1435130685 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56b80041d0 con 0x5565bacde5d0 2024-09-12T03:02:04.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.428+0000 7f56a7fff640 1 -- 172.21.15.5:0/1435130685 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b8011820 con 0x5565bacde5d0 2024-09-12T03:02:04.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.429+0000 7f56c22af640 1 --2- 172.21.15.5:0/3555479532 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f56a80a2c00 0x7f56a80a50c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f56b4005e30 tx=0x7f56b403c040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:04.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.429+0000 7f56c60eb740 1 -- 172.21.15.5:0/1435130685 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565bacde5d0 msgr2=0x5565bacdea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:04.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.429+0000 7f56c60eb740 1 --2- 172.21.15.5:0/1435130685 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565bacde5d0 0x5565bacdea10 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f56b800bd10 tx=0x7f56b80107a0 comp rx=0 tx=0).stop 2024-09-12T03:02:04.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.429+0000 7f56c60eb740 1 -- 172.21.15.5:0/1435130685 shutdown_connections 2024-09-12T03:02:04.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.429+0000 7f56c60eb740 1 --2- 172.21.15.5:0/1435130685 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565bacde5d0 0x5565bacdea10 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:04.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.429+0000 7f56c60eb740 1 --2- 172.21.15.5:0/1435130685 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5565bace9370 0x5565baceb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:04.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.429+0000 7f56c60eb740 1 --2- 172.21.15.5:0/1435130685 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5565bacdfb40 0x5565bacde090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:04.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.429+0000 7f56c60eb740 1 -- 172.21.15.5:0/1435130685 >> 172.21.15.5:0/1435130685 conn(0x5565baca7310 msgr2=0x5565bacdac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:04.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.429+0000 7f56c60eb740 1 -- 172.21.15.5:0/1435130685 shutdown_connections 2024-09-12T03:02:04.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.429+0000 7f56c60eb740 1 -- 172.21.15.5:0/1435130685 wait complete. 2024-09-12T03:02:04.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.430+0000 7f56c60eb740 1 -- start start 2024-09-12T03:02:04.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.430+0000 7f56c60eb740 1 Processor -- start 2024-09-12T03:02:04.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.430+0000 7f56c60eb740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5565bacde5d0 0x5565bace8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:04.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.430+0000 7f56c60eb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565bacdfb40 0x5565bace5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:04.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.430+0000 7f56c60eb740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5565bace9370 0x5565bace6520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:04.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.430+0000 7f56c60eb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5565bacec070 con 0x5565bacdfb40 2024-09-12T03:02:04.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.430+0000 7f56c60eb740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5565babff060 con 0x5565bacde5d0 2024-09-12T03:02:04.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.430+0000 7f56c60eb740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5565bacec1f0 con 0x5565bace9370 2024-09-12T03:02:04.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.430+0000 7f56c60eb740 10 client.0 _ll_register_callbacks cb 0x5565ba9528c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:02:04.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.430+0000 7f56c12ad640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565bacdfb40 0x5565bace5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:04.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.430+0000 7f56c12ad640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565bacdfb40 0x5565bace5fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38272/0 (socket says 172.21.15.5:38272) 2024-09-12T03:02:04.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.430+0000 7f56c12ad640 1 -- 172.21.15.5:0/2309297656 learned_addr learned my addr 172.21.15.5:0/2309297656 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:02:04.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.431+0000 7f56c22af640 1 --2- 172.21.15.5:0/2309297656 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5565bace9370 0x5565bace6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:04.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.431+0000 7f56c1aae640 1 --2- 172.21.15.5:0/2309297656 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5565bacde5d0 0x5565bace8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:04.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.431+0000 7f56c12ad640 1 -- 172.21.15.5:0/2309297656 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5565bace9370 msgr2=0x5565bace6520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:04.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.431+0000 7f56c12ad640 1 --2- 172.21.15.5:0/2309297656 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5565bace9370 0x5565bace6520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:04.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.431+0000 7f56c12ad640 1 -- 172.21.15.5:0/2309297656 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5565bacde5d0 msgr2=0x5565bace8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:04.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.431+0000 7f56c12ad640 1 --2- 172.21.15.5:0/2309297656 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5565bacde5d0 0x5565bace8cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:04.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.431+0000 7f56c12ad640 1 -- 172.21.15.5:0/2309297656 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565ba9b53c0 con 0x5565bacdfb40 2024-09-12T03:02:04.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.431+0000 7f56c1aae640 1 --2- 172.21.15.5:0/2309297656 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5565bacde5d0 0x5565bace8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:02:04.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.431+0000 7f56c12ad640 1 --2- 172.21.15.5:0/2309297656 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565bacdfb40 0x5565bace5fa0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f56b009b470 tx=0x7f56b0097a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65cdb22c2fa3b563 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:04.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.431+0000 7f56a57fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T03:02:04.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.431+0000 7f56a57fa640 1 -- 172.21.15.5:0/2309297656 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b001d040 con 0x5565bacdfb40 2024-09-12T03:02:04.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.431+0000 7f56a57fa640 1 -- 172.21.15.5:0/2309297656 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f56b0007ca0 con 0x5565bacdfb40 2024-09-12T03:02:04.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.431+0000 7f56c60eb740 1 -- 172.21.15.5:0/2309297656 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56b80041d0 con 0x5565bacdfb40 2024-09-12T03:02:04.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.431+0000 7f56a57fa640 1 -- 172.21.15.5:0/2309297656 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f56b009b2b0 con 0x5565bacdfb40 2024-09-12T03:02:04.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.431+0000 7f56c60eb740 10 client.16251 fetch_fsmap learned FSMap version 30 2024-09-12T03:02:04.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.431+0000 7f56c60eb740 1 -- 172.21.15.5:0/2309297656 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5565bacf7ad0 con 0x5565bacdfb40 2024-09-12T03:02:04.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.431+0000 7f56a57fa640 1 -- 172.21.15.5:0/2309297656 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56b80041d0 con 0x5565bacdfb40 2024-09-12T03:02:04.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.432+0000 7f56a57fa640 1 -- 172.21.15.5:0/2309297656 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f56b0097de0 con 0x5565bacdfb40 2024-09-12T03:02:04.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.432+0000 7f56c60eb740 10 client.16251 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:02:04.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.432+0000 7f56c60eb740 10 client.16251 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.b 2024-09-12T03:02:04.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.432+0000 7f56c60eb740 20 client.16251 populate_metadata read hostname 'smithi005' 2024-09-12T03:02:04.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.432+0000 7f56c60eb740 1 --2- 172.21.15.5:0/2309297656 >> [v2:172.21.15.146:6838/3036767228,v1:172.21.15.146:6839/3036767228] conn(0x5565bacf8f90 0x5565bacfb3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:04.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.432+0000 7f56c60eb740 4 client.16251 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:02:04.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.432+0000 7f56c60eb740 1 -- 172.21.15.5:0/2309297656 --> [v2:172.21.15.146:6838/3036767228,v1:172.21.15.146:6839/3036767228] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5565bac51860 con 0x5565bacf8f90 2024-09-12T03:02:04.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.432+0000 7f56c1aae640 1 --2- 172.21.15.5:0/2309297656 >> [v2:172.21.15.146:6838/3036767228,v1:172.21.15.146:6839/3036767228] conn(0x5565bacf8f90 0x5565bacfb3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:04.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.432+0000 7f56c1aae640 1 --2- 172.21.15.5:0/2309297656 >> [v2:172.21.15.146:6838/3036767228,v1:172.21.15.146:6839/3036767228] conn(0x5565bacf8f90 0x5565bacfb3c0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=fad8f2d886fecf86 server_cookie=576c9553d5ce868e in_seq=0 out_seq=0 2024-09-12T03:02:04.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.432+0000 7f56a57fa640 10 client.16251 ms_handle_connect on v2:172.21.15.146:6838/3036767228 2024-09-12T03:02:04.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.433+0000 7f56a57fa640 1 -- 172.21.15.5:0/2309297656 <== mds.3 v2:172.21.15.146:6838/3036767228 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5565bac51860 con 0x5565bacf8f90 2024-09-12T03:02:04.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.433+0000 7f56a57fa640 10 client.16251 handle_command_reply: tid=0 2024-09-12T03:02:04.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.433+0000 7f56c60eb740 1 client.16251 shutdown 2024-09-12T03:02:04.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.433+0000 7f56c60eb740 1 -- 172.21.15.5:0/2309297656 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565bacdfb40 msgr2=0x5565bace5fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:04.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.433+0000 7f56c60eb740 1 --2- 172.21.15.5:0/2309297656 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565bacdfb40 0x5565bace5fa0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f56b009b470 tx=0x7f56b0097a90 comp rx=0 tx=0).stop 2024-09-12T03:02:04.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.433+0000 7f56c60eb740 1 -- 172.21.15.5:0/2309297656 shutdown_connections 2024-09-12T03:02:04.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.433+0000 7f56c60eb740 1 --2- 172.21.15.5:0/2309297656 >> [v2:172.21.15.146:6838/3036767228,v1:172.21.15.146:6839/3036767228] conn(0x5565bacf8f90 0x5565bacfb3c0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:04.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.433+0000 7f56c60eb740 1 --2- 172.21.15.5:0/2309297656 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565bacdfb40 0x5565bace5fa0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:04.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.433+0000 7f56c60eb740 1 --2- 172.21.15.5:0/2309297656 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5565bace9370 0x5565bace6520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:04.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.433+0000 7f56a57fa640 0 client.16251 ms_handle_reset on v2:172.21.15.146:6838/3036767228 2024-09-12T03:02:04.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.433+0000 7f56c60eb740 1 --2- 172.21.15.5:0/2309297656 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5565bacde5d0 0x5565bace8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:04.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.433+0000 7f56c60eb740 1 -- 172.21.15.5:0/2309297656 >> 172.21.15.5:0/2309297656 conn(0x5565baca7310 msgr2=0x5565bace0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:04.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.434+0000 7f56c60eb740 1 -- 172.21.15.5:0/2309297656 shutdown_connections 2024-09-12T03:02:04.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.434+0000 7f56c60eb740 1 -- 172.21.15.5:0/2309297656 wait complete. 2024-09-12T03:02:04.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.434+0000 7f56c60eb740 20 client.16251 trim_cache size 0 max 16384 2024-09-12T03:02:04.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.447+0000 7f56c60eb740 1 Processor -- start 2024-09-12T03:02:04.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.447+0000 7f56c60eb740 1 -- start start 2024-09-12T03:02:04.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.447+0000 7f56c60eb740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5565bacde5d0 0x5565bacf5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:04.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.447+0000 7f56c60eb740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5565bacdfb40 0x5565bace7f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:04.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.447+0000 7f56c60eb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565bace9370 0x5565bad00c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:04.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.447+0000 7f56c60eb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f56bc1b3c20 con 0x5565bace9370 2024-09-12T03:02:04.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.447+0000 7f56c60eb740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f56bc1b5790 con 0x5565bacdfb40 2024-09-12T03:02:04.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.447+0000 7f56c60eb740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f56bc1b42f0 con 0x5565bacde5d0 2024-09-12T03:02:04.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.447+0000 7f56c22af640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565bace9370 0x5565bad00c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:04.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.447+0000 7f56c22af640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565bace9370 0x5565bad00c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38288/0 (socket says 172.21.15.5:38288) 2024-09-12T03:02:04.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.447+0000 7f56c22af640 1 -- 172.21.15.5:0/1873643139 learned_addr learned my addr 172.21.15.5:0/1873643139 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:02:04.650 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-12T03:02:04.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.447+0000 7f56c12ad640 1 --2- 172.21.15.5:0/1873643139 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5565bacdfb40 0x5565bace7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:04.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.447+0000 7f56c1aae640 1 --2- 172.21.15.5:0/1873643139 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5565bacde5d0 0x5565bacf5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:04.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.447+0000 7f56c22af640 1 -- 172.21.15.5:0/1873643139 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5565bacde5d0 msgr2=0x5565bacf5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:04.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.447+0000 7f56c22af640 1 --2- 172.21.15.5:0/1873643139 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5565bacde5d0 0x5565bacf5340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:04.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.447+0000 7f56c22af640 1 -- 172.21.15.5:0/1873643139 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5565bacdfb40 msgr2=0x5565bace7f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:04.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.447+0000 7f56c22af640 1 --2- 172.21.15.5:0/1873643139 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5565bacdfb40 0x5565bace7f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:04.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.447+0000 7f56c22af640 1 -- 172.21.15.5:0/1873643139 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565bace6890 con 0x5565bace9370 2024-09-12T03:02:04.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.447+0000 7f56c1aae640 1 --2- 172.21.15.5:0/1873643139 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5565bacde5d0 0x5565bacf5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:02:04.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.448+0000 7f56c22af640 1 --2- 172.21.15.5:0/1873643139 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565bace9370 0x5565bad00c90 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f56b4035000 tx=0x7f56b4040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22f5ac69523e3882 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:04.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.448+0000 7f56a77fe640 1 -- 172.21.15.5:0/1873643139 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b4054070 con 0x5565bace9370 2024-09-12T03:02:04.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.448+0000 7f56a77fe640 1 -- 172.21.15.5:0/1873643139 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56b4003c30 con 0x5565bace9370 2024-09-12T03:02:04.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.448+0000 7f56a77fe640 1 -- 172.21.15.5:0/1873643139 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b4034040 con 0x5565bace9370 2024-09-12T03:02:04.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.448+0000 7f56c60eb740 1 -- 172.21.15.5:0/1873643139 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565bace9370 msgr2=0x5565bad00c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:04.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.448+0000 7f56c60eb740 1 --2- 172.21.15.5:0/1873643139 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565bace9370 0x5565bad00c90 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f56b4035000 tx=0x7f56b4040000 comp rx=0 tx=0).stop 2024-09-12T03:02:04.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.448+0000 7f56c60eb740 1 -- 172.21.15.5:0/1873643139 shutdown_connections 2024-09-12T03:02:04.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.448+0000 7f56c60eb740 1 --2- 172.21.15.5:0/1873643139 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565bace9370 0x5565bad00c90 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:04.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.448+0000 7f56c60eb740 1 --2- 172.21.15.5:0/1873643139 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5565bacdfb40 0x5565bace7f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:04.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.448+0000 7f56c60eb740 1 --2- 172.21.15.5:0/1873643139 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5565bacde5d0 0x5565bacf5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:04.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.448+0000 7f56c60eb740 1 -- 172.21.15.5:0/1873643139 >> 172.21.15.5:0/1873643139 conn(0x5565baca7310 msgr2=0x5565bace7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:04.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.449+0000 7f56c60eb740 1 -- 172.21.15.5:0/1873643139 shutdown_connections 2024-09-12T03:02:04.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.449+0000 7f56c60eb740 1 -- 172.21.15.5:0/1873643139 wait complete. 2024-09-12T03:02:04.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.449+0000 7f56c60eb740 1 -- start start 2024-09-12T03:02:04.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.449+0000 7f56c60eb740 1 Processor -- start 2024-09-12T03:02:04.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.449+0000 7f56c60eb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565bacde5d0 0x5565bacf14b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:04.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.449+0000 7f56c60eb740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5565bacdfb40 0x5565bacf19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:04.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.449+0000 7f56c60eb740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5565bace9370 0x5565bacee480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:04.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.449+0000 7f56c60eb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f56bc1b5090 con 0x5565bacde5d0 2024-09-12T03:02:04.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.449+0000 7f56c60eb740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f56bc1b49c0 con 0x5565bacdfb40 2024-09-12T03:02:04.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.449+0000 7f56c60eb740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f56bc1b1c00 con 0x5565bace9370 2024-09-12T03:02:04.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.449+0000 7f56c60eb740 10 client.0 _ll_register_callbacks cb 0x7f56b00058c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:02:04.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.449+0000 7f56c1aae640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565bacde5d0 0x5565bacf14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:04.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.449+0000 7f56c1aae640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565bacde5d0 0x5565bacf14b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38300/0 (socket says 172.21.15.5:38300) 2024-09-12T03:02:04.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.449+0000 7f56c1aae640 1 -- 172.21.15.5:0/930657509 learned_addr learned my addr 172.21.15.5:0/930657509 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:02:04.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.450+0000 7f56c22af640 1 --2- 172.21.15.5:0/930657509 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5565bace9370 0x5565bacee480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:04.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.450+0000 7f56c12ad640 1 --2- 172.21.15.5:0/930657509 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5565bacdfb40 0x5565bacf19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:04.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.450+0000 7f56c1aae640 1 -- 172.21.15.5:0/930657509 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5565bace9370 msgr2=0x5565bacee480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:04.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.450+0000 7f56c1aae640 1 --2- 172.21.15.5:0/930657509 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5565bace9370 0x5565bacee480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:04.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.450+0000 7f56c1aae640 1 -- 172.21.15.5:0/930657509 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5565bacdfb40 msgr2=0x5565bacf19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:04.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.450+0000 7f56c1aae640 1 --2- 172.21.15.5:0/930657509 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5565bacdfb40 0x5565bacf19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:04.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.450+0000 7f56c1aae640 1 -- 172.21.15.5:0/930657509 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565bacee9c0 con 0x5565bacde5d0 2024-09-12T03:02:04.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.450+0000 7f56c12ad640 1 --2- 172.21.15.5:0/930657509 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5565bacdfb40 0x5565bacf19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:02:04.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.450+0000 7f56c1aae640 1 --2- 172.21.15.5:0/930657509 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565bacde5d0 0x5565bacf14b0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f56b800bd10 tx=0x7f56b80058e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e86fad76d60f884f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:04.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.450+0000 7f56a57fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T03:02:04.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.450+0000 7f56a57fa640 1 -- 172.21.15.5:0/930657509 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b8005410 con 0x5565bacde5d0 2024-09-12T03:02:04.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.450+0000 7f56a57fa640 1 -- 172.21.15.5:0/930657509 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f56b80055b0 con 0x5565bacde5d0 2024-09-12T03:02:04.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.450+0000 7f56c60eb740 1 -- 172.21.15.5:0/930657509 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56b4003c30 con 0x5565bacde5d0 2024-09-12T03:02:04.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.450+0000 7f56a57fa640 1 -- 172.21.15.5:0/930657509 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f56b80035f0 con 0x5565bacde5d0 2024-09-12T03:02:04.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.450+0000 7f56c60eb740 10 client.16257 fetch_fsmap learned FSMap version 30 2024-09-12T03:02:04.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.450+0000 7f56c60eb740 1 -- 172.21.15.5:0/930657509 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5565bad64330 con 0x5565bacde5d0 2024-09-12T03:02:04.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.450+0000 7f56a57fa640 1 -- 172.21.15.5:0/930657509 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56b4003c30 con 0x5565bacde5d0 2024-09-12T03:02:04.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.451+0000 7f56a57fa640 1 -- 172.21.15.5:0/930657509 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f56b802f030 con 0x5565bacde5d0 2024-09-12T03:02:04.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.451+0000 7f56c60eb740 10 client.16257 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:02:04.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.451+0000 7f56c60eb740 10 client.16257 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.b 2024-09-12T03:02:04.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.451+0000 7f56c60eb740 20 client.16257 populate_metadata read hostname 'smithi005' 2024-09-12T03:02:04.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.451+0000 7f56c60eb740 1 --2- 172.21.15.5:0/930657509 >> [v2:172.21.15.146:6838/3036767228,v1:172.21.15.146:6839/3036767228] conn(0x5565bacf8f90 0x5565bad667e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:04.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.451+0000 7f56c60eb740 4 client.16257 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:02:04.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.451+0000 7f56c60eb740 1 -- 172.21.15.5:0/930657509 --> [v2:172.21.15.146:6838/3036767228,v1:172.21.15.146:6839/3036767228] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5565bac4bd70 con 0x5565bacf8f90 2024-09-12T03:02:04.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.451+0000 7f56c12ad640 1 --2- 172.21.15.5:0/930657509 >> [v2:172.21.15.146:6838/3036767228,v1:172.21.15.146:6839/3036767228] conn(0x5565bacf8f90 0x5565bad667e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:04.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.451+0000 7f56a57fa640 10 client.16257 ms_handle_connect on v2:172.21.15.146:6838/3036767228 2024-09-12T03:02:04.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.451+0000 7f56c12ad640 1 --2- 172.21.15.5:0/930657509 >> [v2:172.21.15.146:6838/3036767228,v1:172.21.15.146:6839/3036767228] conn(0x5565bacf8f90 0x5565bad667e0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5b3d2298a861af6a server_cookie=d1e9412c09c9c286 in_seq=0 out_seq=0 2024-09-12T03:02:04.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.452+0000 7f56a57fa640 1 -- 172.21.15.5:0/930657509 <== mds.3 v2:172.21.15.146:6838/3036767228 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5565bac4bd70 con 0x5565bacf8f90 2024-09-12T03:02:04.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.452+0000 7f56a57fa640 10 client.16257 handle_command_reply: tid=0 2024-09-12T03:02:04.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.452+0000 7f56c60eb740 1 client.16257 shutdown 2024-09-12T03:02:04.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.452+0000 7f56c60eb740 1 -- 172.21.15.5:0/930657509 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565bacde5d0 msgr2=0x5565bacf14b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:04.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.452+0000 7f56c60eb740 1 --2- 172.21.15.5:0/930657509 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565bacde5d0 0x5565bacf14b0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f56b800bd10 tx=0x7f56b80058e0 comp rx=0 tx=0).stop 2024-09-12T03:02:04.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.452+0000 7f56c60eb740 1 -- 172.21.15.5:0/930657509 shutdown_connections 2024-09-12T03:02:04.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.452+0000 7f56c60eb740 1 --2- 172.21.15.5:0/930657509 >> [v2:172.21.15.146:6838/3036767228,v1:172.21.15.146:6839/3036767228] conn(0x5565bacf8f90 0x5565bad667e0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:04.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.452+0000 7f56c60eb740 1 --2- 172.21.15.5:0/930657509 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5565bace9370 0x5565bacee480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:04.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.452+0000 7f56c60eb740 1 --2- 172.21.15.5:0/930657509 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5565bacdfb40 0x5565bacf19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:04.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.452+0000 7f56c60eb740 1 --2- 172.21.15.5:0/930657509 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565bacde5d0 0x5565bacf14b0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:04.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.452+0000 7f56a57fa640 0 client.16257 ms_handle_reset on v2:172.21.15.146:6838/3036767228 2024-09-12T03:02:04.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.452+0000 7f56c60eb740 1 -- 172.21.15.5:0/930657509 >> 172.21.15.5:0/930657509 conn(0x5565baca7310 msgr2=0x5565bace0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:04.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.452+0000 7f56c60eb740 1 -- 172.21.15.5:0/930657509 shutdown_connections 2024-09-12T03:02:04.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.452+0000 7f56c60eb740 1 -- 172.21.15.5:0/930657509 wait complete. 2024-09-12T03:02:04.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.452+0000 7f56c60eb740 20 client.16257 trim_cache size 0 max 16384 2024-09-12T03:02:04.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.452+0000 7f56c3d12640 1 -- 172.21.15.5:0/3555479532 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f56a80a2c00 msgr2=0x7f56a80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:04.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.452+0000 7f56c3d12640 1 --2- 172.21.15.5:0/3555479532 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f56a80a2c00 0x7f56a80a50c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f56b4005e30 tx=0x7f56b403c040 comp rx=0 tx=0).stop 2024-09-12T03:02:04.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.452+0000 7f56c3d12640 1 -- 172.21.15.5:0/3555479532 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56bc103660 msgr2=0x7f56bc100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:04.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.452+0000 7f56c3d12640 1 --2- 172.21.15.5:0/3555479532 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56bc103660 0x7f56bc100ee0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f56b000c940 tx=0x7f56b000ce00 comp rx=0 tx=0).stop 2024-09-12T03:02:04.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.452+0000 7f56c3d12640 1 -- 172.21.15.5:0/3555479532 shutdown_connections 2024-09-12T03:02:04.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.452+0000 7f56c3d12640 1 --2- 172.21.15.5:0/3555479532 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f56a80a2c00 0x7f56a80a50c0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:04.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.452+0000 7f56c3d12640 1 --2- 172.21.15.5:0/3555479532 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56bc103660 0x7f56bc100ee0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:04.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.452+0000 7f56c3d12640 1 --2- 172.21.15.5:0/3555479532 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f56bc10c170 0x7f56bc101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:04.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.452+0000 7f56c3d12640 1 --2- 172.21.15.5:0/3555479532 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f56bc102d40 0x7f56bc1009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:04.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.452+0000 7f56c3d12640 1 -- 172.21.15.5:0/3555479532 >> 172.21.15.5:0/3555479532 conn(0x7f56bc0fc7a0 msgr2=0x7f56bc0fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:04.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.453+0000 7f56c3d12640 1 -- 172.21.15.5:0/3555479532 shutdown_connections 2024-09-12T03:02:04.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:04.453+0000 7f56c3d12640 1 -- 172.21.15.5:0/3555479532 wait complete. 2024-09-12T03:02:05.478 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T03:02:05.779 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:02:05.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:02:05 smithi005 ceph-mon[31197]: pgmap v450: 97 pgs: 97 active+clean; 112 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.0 MiB/s wr, 726 op/s 2024-09-12T03:02:05.915 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:02:05 smithi148 ceph-mon[35090]: pgmap v450: 97 pgs: 97 active+clean; 112 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.0 MiB/s wr, 726 op/s 2024-09-12T03:02:05.956 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:02:05 smithi146 ceph-mon[35011]: pgmap v450: 97 pgs: 97 active+clean; 112 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.0 MiB/s wr, 726 op/s 2024-09-12T03:02:06.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.284+0000 7f496a24c640 1 -- 172.21.15.5:0/1669322477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4964101410 msgr2=0x7f496410d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:06.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.284+0000 7f496a24c640 1 --2- 172.21.15.5:0/1669322477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4964101410 0x7f496410d940 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f495c009cd0 tx=0x7f495c02a8e0 comp rx=0 tx=0).stop 2024-09-12T03:02:06.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.284+0000 7f496a24c640 1 -- 172.21.15.5:0/1669322477 shutdown_connections 2024-09-12T03:02:06.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.284+0000 7f496a24c640 1 --2- 172.21.15.5:0/1669322477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4964101410 0x7f496410d940 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:06.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.284+0000 7f496a24c640 1 --2- 172.21.15.5:0/1669322477 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4964100a70 0x7f4964100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:06.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.284+0000 7f496a24c640 1 --2- 172.21.15.5:0/1669322477 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4964106a70 0x7f4964106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:06.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.284+0000 7f496a24c640 1 -- 172.21.15.5:0/1669322477 >> 172.21.15.5:0/1669322477 conn(0x7f49640fc790 msgr2=0x7f49640febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:06.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.284+0000 7f496a24c640 1 -- 172.21.15.5:0/1669322477 shutdown_connections 2024-09-12T03:02:06.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.284+0000 7f496a24c640 1 -- 172.21.15.5:0/1669322477 wait complete. 2024-09-12T03:02:06.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.285+0000 7f496a24c640 1 Processor -- start 2024-09-12T03:02:06.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.285+0000 7f496a24c640 1 -- start start 2024-09-12T03:02:06.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.285+0000 7f496a24c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4964100a70 0x7f49641992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:06.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.285+0000 7f496a24c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4964101410 0x7f49641997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:06.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.285+0000 7f496a24c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4964106a70 0x7f49641a4330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:06.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.285+0000 7f496a24c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4964110e70 con 0x7f4964100a70 2024-09-12T03:02:06.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.285+0000 7f496a24c640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f4964110cf0 con 0x7f4964101410 2024-09-12T03:02:06.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.285+0000 7f496a24c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4964110ff0 con 0x7f4964106a70 2024-09-12T03:02:06.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.285+0000 7f49637fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4964100a70 0x7f49641992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:06.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.285+0000 7f49637fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4964100a70 0x7f49641992b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38312/0 (socket says 172.21.15.5:38312) 2024-09-12T03:02:06.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.285+0000 7f49637fe640 1 -- 172.21.15.5:0/1714194857 learned_addr learned my addr 172.21.15.5:0/1714194857 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:02:06.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.285+0000 7f4963fff640 1 --2- 172.21.15.5:0/1714194857 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4964106a70 0x7f49641a4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:06.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.286+0000 7f4962ffd640 1 --2- 172.21.15.5:0/1714194857 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4964101410 0x7f49641997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:06.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.286+0000 7f49637fe640 1 -- 172.21.15.5:0/1714194857 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4964106a70 msgr2=0x7f49641a4330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:06.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.286+0000 7f49637fe640 1 --2- 172.21.15.5:0/1714194857 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4964106a70 0x7f49641a4330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:06.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.286+0000 7f49637fe640 1 -- 172.21.15.5:0/1714194857 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4964101410 msgr2=0x7f49641997f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:06.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.286+0000 7f49637fe640 1 --2- 172.21.15.5:0/1714194857 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4964101410 0x7f49641997f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:06.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.286+0000 7f49637fe640 1 -- 172.21.15.5:0/1714194857 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49641a4870 con 0x7f4964100a70 2024-09-12T03:02:06.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.286+0000 7f49637fe640 1 --2- 172.21.15.5:0/1714194857 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4964100a70 0x7f49641992b0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f495400d950 tx=0x7f495400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:06.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.286+0000 7f4960ff9640 1 -- 172.21.15.5:0/1714194857 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4954017070 con 0x7f4964100a70 2024-09-12T03:02:06.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.286+0000 7f4960ff9640 1 -- 172.21.15.5:0/1714194857 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4954009c30 con 0x7f4964100a70 2024-09-12T03:02:06.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.286+0000 7f4960ff9640 1 -- 172.21.15.5:0/1714194857 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f495400f6b0 con 0x7f4964100a70 2024-09-12T03:02:06.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.286+0000 7f496a24c640 1 -- 172.21.15.5:0/1714194857 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49641a4a40 con 0x7f4964100a70 2024-09-12T03:02:06.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.286+0000 7f496a24c640 1 -- 172.21.15.5:0/1714194857 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49641a4f00 con 0x7f4964100a70 2024-09-12T03:02:06.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.289+0000 7f4960ff9640 1 -- 172.21.15.5:0/1714194857 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f49540109d0 con 0x7f4964100a70 2024-09-12T03:02:06.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.289+0000 7f496c625740 1 Processor -- start 2024-09-12T03:02:06.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.289+0000 7f496c625740 1 -- start start 2024-09-12T03:02:06.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.289+0000 7f496c625740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d99baa0b40 0x55d99ba9f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:06.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.289+0000 7f496c625740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d99ba9f5d0 0x55d99ba9fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:06.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.289+0000 7f496c625740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d99baaa370 0x55d99baac760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:06.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.289+0000 7f496c625740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d99ba052f0 con 0x55d99baa0b40 2024-09-12T03:02:06.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.290+0000 7f496c625740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55d99b9accf0 con 0x55d99ba9f5d0 2024-09-12T03:02:06.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.290+0000 7f496c625740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55d99ba012b0 con 0x55d99baaa370 2024-09-12T03:02:06.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.290+0000 7f4963fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d99baa0b40 0x55d99ba9f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:06.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.290+0000 7f4963fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d99baa0b40 0x55d99ba9f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38314/0 (socket says 172.21.15.5:38314) 2024-09-12T03:02:06.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.290+0000 7f4963fff640 1 -- 172.21.15.5:0/454370531 learned_addr learned my addr 172.21.15.5:0/454370531 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:02:06.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.290+0000 7f49637fe640 1 --2- 172.21.15.5:0/454370531 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d99ba9f5d0 0x55d99ba9fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:06.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.290+0000 7f4962ffd640 1 --2- 172.21.15.5:0/454370531 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d99baaa370 0x55d99baac760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:06.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.290+0000 7f4963fff640 1 -- 172.21.15.5:0/454370531 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d99baaa370 msgr2=0x55d99baac760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:06.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.290+0000 7f4963fff640 1 --2- 172.21.15.5:0/454370531 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d99baaa370 0x55d99baac760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:06.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.290+0000 7f4963fff640 1 -- 172.21.15.5:0/454370531 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d99ba9f5d0 msgr2=0x55d99ba9fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:06.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.290+0000 7f4963fff640 1 --2- 172.21.15.5:0/454370531 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d99ba9f5d0 0x55d99ba9fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:06.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.290+0000 7f4963fff640 1 -- 172.21.15.5:0/454370531 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d99b9913a0 con 0x55d99baa0b40 2024-09-12T03:02:06.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.290+0000 7f49637fe640 1 --2- 172.21.15.5:0/454370531 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d99ba9f5d0 0x55d99ba9fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:02:06.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.290+0000 7f4960ff9640 1 --2- 172.21.15.5:0/1714194857 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f494c0a2bc0 0x7f494c0a5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:06.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.290+0000 7f4963fff640 1 --2- 172.21.15.5:0/454370531 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d99baa0b40 0x55d99ba9f090 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f495c02ae50 tx=0x7f495c009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aea413b67186d695 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:06.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.290+0000 7f49637fe640 1 --2- 172.21.15.5:0/1714194857 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f494c0a2bc0 0x7f494c0a5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:06.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.290+0000 7f4960ff9640 1 -- 172.21.15.5:0/1714194857 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f49540cec70 con 0x7f4964100a70 2024-09-12T03:02:06.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.290+0000 7f4949ffb640 1 -- 172.21.15.5:0/454370531 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f495c031630 con 0x55d99baa0b40 2024-09-12T03:02:06.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.291+0000 7f4949ffb640 1 -- 172.21.15.5:0/454370531 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f495c036040 con 0x55d99baa0b40 2024-09-12T03:02:06.293 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.291+0000 7f4949ffb640 1 -- 172.21.15.5:0/454370531 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f495c0043c0 con 0x55d99baa0b40 2024-09-12T03:02:06.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.291+0000 7f496c625740 1 -- 172.21.15.5:0/454370531 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d99baa0b40 msgr2=0x55d99ba9f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:06.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.291+0000 7f49637fe640 1 --2- 172.21.15.5:0/1714194857 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f494c0a2bc0 0x7f494c0a5080 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f49540ccaf0 tx=0x7f4954086040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:06.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.291+0000 7f496c625740 1 --2- 172.21.15.5:0/454370531 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d99baa0b40 0x55d99ba9f090 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f495c02ae50 tx=0x7f495c009a20 comp rx=0 tx=0).stop 2024-09-12T03:02:06.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.291+0000 7f496c625740 1 -- 172.21.15.5:0/454370531 shutdown_connections 2024-09-12T03:02:06.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.291+0000 7f496c625740 1 --2- 172.21.15.5:0/454370531 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d99baaa370 0x55d99baac760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:06.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.291+0000 7f496c625740 1 --2- 172.21.15.5:0/454370531 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d99ba9f5d0 0x55d99ba9fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:06.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.291+0000 7f496c625740 1 --2- 172.21.15.5:0/454370531 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d99baa0b40 0x55d99ba9f090 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:06.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.291+0000 7f496c625740 1 -- 172.21.15.5:0/454370531 >> 172.21.15.5:0/454370531 conn(0x55d99ba68310 msgr2=0x55d99ba9bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:06.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.292+0000 7f496c625740 1 -- 172.21.15.5:0/454370531 shutdown_connections 2024-09-12T03:02:06.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.292+0000 7f496c625740 1 -- 172.21.15.5:0/454370531 wait complete. 2024-09-12T03:02:06.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.292+0000 7f496c625740 1 -- start start 2024-09-12T03:02:06.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.292+0000 7f496c625740 1 Processor -- start 2024-09-12T03:02:06.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.292+0000 7f496c625740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d99ba9f5d0 0x55d99baa9d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:06.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.292+0000 7f496c625740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d99baa0b40 0x55d99baa6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:06.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.292+0000 7f496c625740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d99baaa370 0x55d99baa75e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:06.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.292+0000 7f496c625740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d99baad010 con 0x55d99baaa370 2024-09-12T03:02:06.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.292+0000 7f496c625740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55d99b9c0060 con 0x55d99ba9f5d0 2024-09-12T03:02:06.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.292+0000 7f496c625740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55d99baad190 con 0x55d99baa0b40 2024-09-12T03:02:06.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.292+0000 7f496c625740 10 client.0 _ll_register_callbacks cb 0x55d99b7138c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:02:06.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.293+0000 7f4963fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d99baaa370 0x55d99baa75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:06.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.293+0000 7f4963fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d99baaa370 0x55d99baa75e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38326/0 (socket says 172.21.15.5:38326) 2024-09-12T03:02:06.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.293+0000 7f4963fff640 1 -- 172.21.15.5:0/1967374434 learned_addr learned my addr 172.21.15.5:0/1967374434 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:02:06.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.293+0000 7f4962ffd640 1 --2- 172.21.15.5:0/1967374434 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d99baa0b40 0x55d99baa6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:06.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.293+0000 7f4963fff640 1 --2- 172.21.15.5:0/1967374434 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d99ba9f5d0 0x55d99baa9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:06.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.293+0000 7f4962ffd640 1 -- 172.21.15.5:0/1967374434 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d99ba9f5d0 msgr2=0x55d99baa9d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:06.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.293+0000 7f4962ffd640 1 --2- 172.21.15.5:0/1967374434 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d99ba9f5d0 0x55d99baa9d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:06.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.293+0000 7f4962ffd640 1 -- 172.21.15.5:0/1967374434 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d99baaa370 msgr2=0x55d99baa75e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:06.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.293+0000 7f4962ffd640 1 --2- 172.21.15.5:0/1967374434 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d99baaa370 0x55d99baa75e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:06.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.293+0000 7f4962ffd640 1 -- 172.21.15.5:0/1967374434 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d99b7763c0 con 0x55d99baa0b40 2024-09-12T03:02:06.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.293+0000 7f4963fff640 1 --2- 172.21.15.5:0/1967374434 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d99baaa370 0x55d99baa75e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:02:06.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.293+0000 7f4962ffd640 1 --2- 172.21.15.5:0/1967374434 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d99baa0b40 0x55d99baa6ff0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f496406b2e0 tx=0x7f4958012b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc6d02aeffe5c62d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:06.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.293+0000 7f4942ffd640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-09-12T03:02:06.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.293+0000 7f4942ffd640 1 -- 172.21.15.5:0/1967374434 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4958004320 con 0x55d99baa0b40 2024-09-12T03:02:06.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.293+0000 7f4942ffd640 1 -- 172.21.15.5:0/1967374434 <== mon.1 v2:172.21.15.148:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f49580119a0 con 0x55d99baa0b40 2024-09-12T03:02:06.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.293+0000 7f496c625740 1 -- 172.21.15.5:0/1967374434 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f495c036040 con 0x55d99baa0b40 2024-09-12T03:02:06.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.294+0000 7f4942ffd640 1 -- 172.21.15.5:0/1967374434 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f49580044c0 con 0x55d99baa0b40 2024-09-12T03:02:06.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.294+0000 7f496c625740 10 client.25792 fetch_fsmap learned FSMap version 30 2024-09-12T03:02:06.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.294+0000 7f496c625740 1 -- 172.21.15.5:0/1967374434 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d99bab8ae0 con 0x55d99baa0b40 2024-09-12T03:02:06.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.294+0000 7f4942ffd640 1 -- 172.21.15.5:0/1967374434 <== mon.1 v2:172.21.15.148:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f495c036040 con 0x55d99baa0b40 2024-09-12T03:02:06.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.294+0000 7f4942ffd640 1 -- 172.21.15.5:0/1967374434 <== mon.1 v2:172.21.15.148:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f495800f440 con 0x55d99baa0b40 2024-09-12T03:02:06.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.294+0000 7f496c625740 10 client.25792 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:02:06.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.294+0000 7f496c625740 10 client.25792 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-09-12T03:02:06.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.294+0000 7f496c625740 20 client.25792 populate_metadata read hostname 'smithi005' 2024-09-12T03:02:06.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.294+0000 7f496c625740 1 --2- 172.21.15.5:0/1967374434 >> [v2:172.21.15.148:6834/4195086427,v1:172.21.15.148:6835/4195086427] conn(0x55d99bab9ff0 0x55d99babc410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:06.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.294+0000 7f496c625740 4 client.25792 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:02:06.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.294+0000 7f496c625740 1 -- 172.21.15.5:0/1967374434 --> [v2:172.21.15.148:6834/4195086427,v1:172.21.15.148:6835/4195086427] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d99ba12860 con 0x55d99bab9ff0 2024-09-12T03:02:06.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.294+0000 7f4963fff640 1 --2- 172.21.15.5:0/1967374434 >> [v2:172.21.15.148:6834/4195086427,v1:172.21.15.148:6835/4195086427] conn(0x55d99bab9ff0 0x55d99babc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:06.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.295+0000 7f4963fff640 1 --2- 172.21.15.5:0/1967374434 >> [v2:172.21.15.148:6834/4195086427,v1:172.21.15.148:6835/4195086427] conn(0x55d99bab9ff0 0x55d99babc410 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6c39f0876ed694ed server_cookie=bcfd11dfa14f99cc in_seq=0 out_seq=0 2024-09-12T03:02:06.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.295+0000 7f4942ffd640 10 client.25792 ms_handle_connect on v2:172.21.15.148:6834/4195086427 2024-09-12T03:02:06.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.296+0000 7f4942ffd640 1 -- 172.21.15.5:0/1967374434 <== mds.2 v2:172.21.15.148:6834/4195086427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d99ba12860 con 0x55d99bab9ff0 2024-09-12T03:02:06.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.296+0000 7f4942ffd640 10 client.25792 handle_command_reply: tid=0 2024-09-12T03:02:06.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.296+0000 7f496c625740 1 client.25792 shutdown 2024-09-12T03:02:06.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.296+0000 7f496c625740 1 -- 172.21.15.5:0/1967374434 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d99baa0b40 msgr2=0x55d99baa6ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:06.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.296+0000 7f496c625740 1 --2- 172.21.15.5:0/1967374434 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d99baa0b40 0x55d99baa6ff0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f496406b2e0 tx=0x7f4958012b60 comp rx=0 tx=0).stop 2024-09-12T03:02:06.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.296+0000 7f496c625740 1 -- 172.21.15.5:0/1967374434 shutdown_connections 2024-09-12T03:02:06.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.296+0000 7f496c625740 1 --2- 172.21.15.5:0/1967374434 >> [v2:172.21.15.148:6834/4195086427,v1:172.21.15.148:6835/4195086427] conn(0x55d99bab9ff0 0x55d99babc410 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:06.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.296+0000 7f496c625740 1 --2- 172.21.15.5:0/1967374434 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d99baaa370 0x55d99baa75e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:06.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.296+0000 7f496c625740 1 --2- 172.21.15.5:0/1967374434 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d99baa0b40 0x55d99baa6ff0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:06.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.296+0000 7f496c625740 1 --2- 172.21.15.5:0/1967374434 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d99ba9f5d0 0x55d99baa9d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:06.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.296+0000 7f4942ffd640 0 client.25792 ms_handle_reset on v2:172.21.15.148:6834/4195086427 2024-09-12T03:02:06.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.296+0000 7f496c625740 1 -- 172.21.15.5:0/1967374434 >> 172.21.15.5:0/1967374434 conn(0x55d99ba68310 msgr2=0x55d99baa1930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:06.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.296+0000 7f496c625740 1 -- 172.21.15.5:0/1967374434 shutdown_connections 2024-09-12T03:02:06.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.296+0000 7f496c625740 1 -- 172.21.15.5:0/1967374434 wait complete. 2024-09-12T03:02:06.302 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.296+0000 7f496c625740 20 client.25792 trim_cache size 0 max 16384 2024-09-12T03:02:06.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.309+0000 7f496c625740 1 Processor -- start 2024-09-12T03:02:06.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.309+0000 7f496c625740 1 -- start start 2024-09-12T03:02:06.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.309+0000 7f496c625740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d99ba9f5d0 0x55d99ba9bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:06.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.309+0000 7f496c625740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d99baa0b40 0x55d99bab62b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:06.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.309+0000 7f496c625740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d99baaa370 0x55d99baa5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:06.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.309+0000 7f496c625740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f49641ad5b0 con 0x55d99baa0b40 2024-09-12T03:02:06.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.310+0000 7f496c625740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f49641af120 con 0x55d99baaa370 2024-09-12T03:02:06.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.310+0000 7f496c625740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f49641adc80 con 0x55d99ba9f5d0 2024-09-12T03:02:06.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.310+0000 7f4962ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d99baa0b40 0x55d99bab62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:06.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.310+0000 7f4962ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d99baa0b40 0x55d99bab62b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38342/0 (socket says 172.21.15.5:38342) 2024-09-12T03:02:06.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.310+0000 7f4962ffd640 1 -- 172.21.15.5:0/1223272166 learned_addr learned my addr 172.21.15.5:0/1223272166 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:02:06.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.310+0000 7f4963fff640 1 --2- 172.21.15.5:0/1223272166 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d99baaa370 0x55d99baa5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:06.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.310+0000 7f4963fff640 1 --2- 172.21.15.5:0/1223272166 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d99ba9f5d0 0x55d99ba9bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:06.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.310+0000 7f4962ffd640 1 -- 172.21.15.5:0/1223272166 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d99ba9f5d0 msgr2=0x55d99ba9bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:06.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.310+0000 7f4962ffd640 1 --2- 172.21.15.5:0/1223272166 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d99ba9f5d0 0x55d99ba9bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:06.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.310+0000 7f4962ffd640 1 -- 172.21.15.5:0/1223272166 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d99baaa370 msgr2=0x55d99baa5fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:06.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.310+0000 7f4962ffd640 1 --2- 172.21.15.5:0/1223272166 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d99baaa370 0x55d99baa5fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:06.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.310+0000 7f4962ffd640 1 -- 172.21.15.5:0/1223272166 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d99ba9b4f0 con 0x55d99baa0b40 2024-09-12T03:02:06.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.310+0000 7f4963fff640 1 --2- 172.21.15.5:0/1223272166 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d99baaa370 0x55d99baa5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:02:06.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.310+0000 7f4962ffd640 1 --2- 172.21.15.5:0/1223272166 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d99baa0b40 0x55d99bab62b0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f4958009ff0 tx=0x7f4958009240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9f30b101eed7033 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:06.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.310+0000 7f49497fa640 1 -- 172.21.15.5:0/1223272166 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f495800f440 con 0x55d99baa0b40 2024-09-12T03:02:06.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.310+0000 7f49497fa640 1 -- 172.21.15.5:0/1223272166 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f495800f5e0 con 0x55d99baa0b40 2024-09-12T03:02:06.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.310+0000 7f49497fa640 1 -- 172.21.15.5:0/1223272166 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4958005040 con 0x55d99baa0b40 2024-09-12T03:02:06.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.311+0000 7f496c625740 1 -- 172.21.15.5:0/1223272166 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d99baa0b40 msgr2=0x55d99bab62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:06.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.311+0000 7f496c625740 1 --2- 172.21.15.5:0/1223272166 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d99baa0b40 0x55d99bab62b0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f4958009ff0 tx=0x7f4958009240 comp rx=0 tx=0).stop 2024-09-12T03:02:06.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.311+0000 7f496c625740 1 -- 172.21.15.5:0/1223272166 shutdown_connections 2024-09-12T03:02:06.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.311+0000 7f496c625740 1 --2- 172.21.15.5:0/1223272166 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d99baa0b40 0x55d99bab62b0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:06.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.311+0000 7f496c625740 1 --2- 172.21.15.5:0/1223272166 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d99baaa370 0x55d99baa5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:06.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.311+0000 7f496c625740 1 --2- 172.21.15.5:0/1223272166 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d99ba9f5d0 0x55d99ba9bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:06.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.311+0000 7f496c625740 1 -- 172.21.15.5:0/1223272166 >> 172.21.15.5:0/1223272166 conn(0x55d99ba68310 msgr2=0x55d99baa58c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:06.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.311+0000 7f496c625740 1 -- 172.21.15.5:0/1223272166 shutdown_connections 2024-09-12T03:02:06.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.311+0000 7f496c625740 1 -- 172.21.15.5:0/1223272166 wait complete. 2024-09-12T03:02:06.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.311+0000 7f496c625740 1 -- start start 2024-09-12T03:02:06.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.312+0000 7f496c625740 1 Processor -- start 2024-09-12T03:02:06.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.312+0000 7f496c625740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d99ba9f5d0 0x55d99bab2420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:06.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.312+0000 7f496c625740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d99baa0b40 0x55d99bab2960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:06.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.312+0000 7f496c625740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d99baaa370 0x55d99baa3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:06.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.312+0000 7f496c625740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f49641aea20 con 0x55d99baaa370 2024-09-12T03:02:06.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.312+0000 7f496c625740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f49641ae350 con 0x55d99baa0b40 2024-09-12T03:02:06.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.312+0000 7f496c625740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f49641ab590 con 0x55d99ba9f5d0 2024-09-12T03:02:06.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.312+0000 7f496c625740 10 client.0 _ll_register_callbacks cb 0x7f49580097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:02:06.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.312+0000 7f4963fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d99baaa370 0x55d99baa3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:06.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.312+0000 7f4963fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d99baaa370 0x55d99baa3260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38344/0 (socket says 172.21.15.5:38344) 2024-09-12T03:02:06.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.312+0000 7f4963fff640 1 -- 172.21.15.5:0/1789732030 learned_addr learned my addr 172.21.15.5:0/1789732030 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:02:06.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.312+0000 7f4963fff640 1 --2- 172.21.15.5:0/1789732030 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d99ba9f5d0 0x55d99bab2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:06.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.312+0000 7f4963fff640 1 -- 172.21.15.5:0/1789732030 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d99ba9f5d0 msgr2=0x55d99bab2420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:06.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.312+0000 7f4963fff640 1 --2- 172.21.15.5:0/1789732030 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d99ba9f5d0 0x55d99bab2420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:06.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.312+0000 7f4963fff640 1 -- 172.21.15.5:0/1789732030 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d99baa0b40 msgr2=0x55d99bab2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:06.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.312+0000 7f4963fff640 1 --2- 172.21.15.5:0/1789732030 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d99baa0b40 0x55d99bab2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:06.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.312+0000 7f4963fff640 1 -- 172.21.15.5:0/1789732030 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d99baa4e10 con 0x55d99baaa370 2024-09-12T03:02:06.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.313+0000 7f4963fff640 1 --2- 172.21.15.5:0/1789732030 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d99baaa370 0x55d99baa3260 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f495c005e30 tx=0x7f4958009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3707e81ff41234ff server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:06.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.313+0000 7f4942ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T03:02:06.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.313+0000 7f4942ffd640 1 -- 172.21.15.5:0/1789732030 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f495c0316f0 con 0x55d99baaa370 2024-09-12T03:02:06.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.313+0000 7f4942ffd640 1 -- 172.21.15.5:0/1789732030 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f495c004b40 con 0x55d99baaa370 2024-09-12T03:02:06.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.313+0000 7f496c625740 1 -- 172.21.15.5:0/1789732030 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f495800f5e0 con 0x55d99baaa370 2024-09-12T03:02:06.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.313+0000 7f4942ffd640 1 -- 172.21.15.5:0/1789732030 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f495c004d40 con 0x55d99baaa370 2024-09-12T03:02:06.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.313+0000 7f496c625740 10 client.16275 fetch_fsmap learned FSMap version 30 2024-09-12T03:02:06.318 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.313+0000 7f496c625740 1 -- 172.21.15.5:0/1789732030 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d99bb28480 con 0x55d99baaa370 2024-09-12T03:02:06.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.313+0000 7f4942ffd640 1 -- 172.21.15.5:0/1789732030 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f495800f5e0 con 0x55d99baaa370 2024-09-12T03:02:06.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.313+0000 7f4942ffd640 1 -- 172.21.15.5:0/1789732030 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f495c030070 con 0x55d99baaa370 2024-09-12T03:02:06.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.313+0000 7f496c625740 10 client.16275 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:02:06.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.313+0000 7f496c625740 10 client.16275 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-09-12T03:02:06.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.313+0000 7f496c625740 20 client.16275 populate_metadata read hostname 'smithi005' 2024-09-12T03:02:06.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.313+0000 7f496c625740 1 --2- 172.21.15.5:0/1789732030 >> [v2:172.21.15.148:6834/4195086427,v1:172.21.15.148:6835/4195086427] conn(0x55d99bb28810 0x55d99bb2ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:06.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.313+0000 7f496c625740 4 client.16275 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:02:06.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.313+0000 7f496c625740 1 -- 172.21.15.5:0/1789732030 --> [v2:172.21.15.148:6834/4195086427,v1:172.21.15.148:6835/4195086427] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d99ba0cd70 con 0x55d99bb28810 2024-09-12T03:02:06.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.314+0000 7f4962ffd640 1 --2- 172.21.15.5:0/1789732030 >> [v2:172.21.15.148:6834/4195086427,v1:172.21.15.148:6835/4195086427] conn(0x55d99bb28810 0x55d99bb2ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:06.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.314+0000 7f4962ffd640 1 --2- 172.21.15.5:0/1789732030 >> [v2:172.21.15.148:6834/4195086427,v1:172.21.15.148:6835/4195086427] conn(0x55d99bb28810 0x55d99bb2ac30 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=129ec74421ddba68 server_cookie=535258b13931d3e0 in_seq=0 out_seq=0 2024-09-12T03:02:06.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.314+0000 7f4942ffd640 10 client.16275 ms_handle_connect on v2:172.21.15.148:6834/4195086427 2024-09-12T03:02:06.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.314+0000 7f4942ffd640 1 -- 172.21.15.5:0/1789732030 <== mds.2 v2:172.21.15.148:6834/4195086427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d99ba0cd70 con 0x55d99bb28810 2024-09-12T03:02:06.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.314+0000 7f4942ffd640 10 client.16275 handle_command_reply: tid=0 2024-09-12T03:02:06.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.314+0000 7f496c625740 1 client.16275 shutdown 2024-09-12T03:02:06.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.315+0000 7f496c625740 1 -- 172.21.15.5:0/1789732030 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d99baaa370 msgr2=0x55d99baa3260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:06.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.315+0000 7f496c625740 1 --2- 172.21.15.5:0/1789732030 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d99baaa370 0x55d99baa3260 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f495c005e30 tx=0x7f4958009ca0 comp rx=0 tx=0).stop 2024-09-12T03:02:06.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.315+0000 7f496c625740 1 -- 172.21.15.5:0/1789732030 shutdown_connections 2024-09-12T03:02:06.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.315+0000 7f496c625740 1 --2- 172.21.15.5:0/1789732030 >> [v2:172.21.15.148:6834/4195086427,v1:172.21.15.148:6835/4195086427] conn(0x55d99bb28810 0x55d99bb2ac30 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:06.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.315+0000 7f496c625740 1 --2- 172.21.15.5:0/1789732030 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d99baaa370 0x55d99baa3260 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:06.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.315+0000 7f496c625740 1 --2- 172.21.15.5:0/1789732030 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d99baa0b40 0x55d99bab2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:06.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.315+0000 7f496c625740 1 --2- 172.21.15.5:0/1789732030 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d99ba9f5d0 0x55d99bab2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:06.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.315+0000 7f4942ffd640 0 client.16275 ms_handle_reset on v2:172.21.15.148:6834/4195086427 2024-09-12T03:02:06.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.315+0000 7f496c625740 1 -- 172.21.15.5:0/1789732030 >> 172.21.15.5:0/1789732030 conn(0x55d99ba68310 msgr2=0x55d99baa8850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:06.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.315+0000 7f496c625740 1 -- 172.21.15.5:0/1789732030 shutdown_connections 2024-09-12T03:02:06.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.315+0000 7f496c625740 1 -- 172.21.15.5:0/1789732030 wait complete. 2024-09-12T03:02:06.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.315+0000 7f496c625740 20 client.16275 trim_cache size 0 max 16384 2024-09-12T03:02:06.321 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-12T03:02:06.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.315+0000 7f496a24c640 1 -- 172.21.15.5:0/1714194857 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f494c0a2bc0 msgr2=0x7f494c0a5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:06.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.315+0000 7f496a24c640 1 --2- 172.21.15.5:0/1714194857 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f494c0a2bc0 0x7f494c0a5080 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f49540ccaf0 tx=0x7f4954086040 comp rx=0 tx=0).stop 2024-09-12T03:02:06.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.315+0000 7f496a24c640 1 -- 172.21.15.5:0/1714194857 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4964100a70 msgr2=0x7f49641992b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:06.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.315+0000 7f496a24c640 1 --2- 172.21.15.5:0/1714194857 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4964100a70 0x7f49641992b0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f495400d950 tx=0x7f495400de10 comp rx=0 tx=0).stop 2024-09-12T03:02:06.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.315+0000 7f496a24c640 1 -- 172.21.15.5:0/1714194857 shutdown_connections 2024-09-12T03:02:06.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.315+0000 7f496a24c640 1 --2- 172.21.15.5:0/1714194857 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4964106a70 0x7f49641a4330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:06.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.315+0000 7f496a24c640 1 --2- 172.21.15.5:0/1714194857 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4964101410 0x7f49641997f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:06.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.315+0000 7f496a24c640 1 --2- 172.21.15.5:0/1714194857 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f494c0a2bc0 0x7f494c0a5080 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:06.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.315+0000 7f496a24c640 1 --2- 172.21.15.5:0/1714194857 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4964100a70 0x7f49641992b0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:06.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.315+0000 7f496a24c640 1 -- 172.21.15.5:0/1714194857 >> 172.21.15.5:0/1714194857 conn(0x7f49640fc790 msgr2=0x7f49640fcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:06.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.316+0000 7f496a24c640 1 -- 172.21.15.5:0/1714194857 shutdown_connections 2024-09-12T03:02:06.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:06.316+0000 7f496a24c640 1 -- 172.21.15.5:0/1714194857 wait complete. 2024-09-12T03:02:06.966 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T03:02:07.223 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:02:07.616 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:02:07 smithi005 ceph-mon[31197]: pgmap v451: 97 pgs: 97 active+clean; 114 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.7 MiB/s wr, 660 op/s 2024-09-12T03:02:07.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.699+0000 7fcd26475640 1 -- 172.21.15.5:0/2741836240 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcd20106f10 msgr2=0x7fcd2010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:07.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.699+0000 7fcd26475640 1 --2- 172.21.15.5:0/2741836240 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcd20106f10 0x7fcd2010ea90 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fcd18009ca0 tx=0x7fcd1802a8c0 comp rx=0 tx=0).stop 2024-09-12T03:02:07.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.699+0000 7fcd26475640 1 -- 172.21.15.5:0/2741836240 shutdown_connections 2024-09-12T03:02:07.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.699+0000 7fcd26475640 1 --2- 172.21.15.5:0/2741836240 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcd20106f10 0x7fcd2010ea90 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:07.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.699+0000 7fcd26475640 1 --2- 172.21.15.5:0/2741836240 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd20102790 0x7fcd201067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:07.716 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.699+0000 7fcd26475640 1 --2- 172.21.15.5:0/2741836240 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcd20101de0 0x7fcd201021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:07.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.699+0000 7fcd26475640 1 -- 172.21.15.5:0/2741836240 >> 172.21.15.5:0/2741836240 conn(0x7fcd200fd8c0 msgr2=0x7fcd200ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:07.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.699+0000 7fcd26475640 1 -- 172.21.15.5:0/2741836240 shutdown_connections 2024-09-12T03:02:07.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.699+0000 7fcd26475640 1 -- 172.21.15.5:0/2741836240 wait complete. 2024-09-12T03:02:07.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.700+0000 7fcd26475640 1 Processor -- start 2024-09-12T03:02:07.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.700+0000 7fcd26475640 1 -- start start 2024-09-12T03:02:07.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.700+0000 7fcd26475640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd20101de0 0x7fcd20198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:07.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.700+0000 7fcd26475640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcd20102790 0x7fcd20198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:07.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.700+0000 7fcd26475640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcd20106f10 0x7fcd2019cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:07.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.700+0000 7fcd26475640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcd20111f90 con 0x7fcd20102790 2024-09-12T03:02:07.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.700+0000 7fcd26475640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fcd20111e10 con 0x7fcd20106f10 2024-09-12T03:02:07.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.700+0000 7fcd26475640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fcd20112110 con 0x7fcd20101de0 2024-09-12T03:02:07.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.700+0000 7fcd1f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcd20102790 0x7fcd20198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:07.717 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.701+0000 7fcd1f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcd20102790 0x7fcd20198950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38362/0 (socket says 172.21.15.5:38362) 2024-09-12T03:02:07.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.701+0000 7fcd1f7fe640 1 -- 172.21.15.5:0/2190450098 learned_addr learned my addr 172.21.15.5:0/2190450098 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:02:07.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.701+0000 7fcd24a12640 1 --2- 172.21.15.5:0/2190450098 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcd20106f10 0x7fcd2019cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:07.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.701+0000 7fcd1ffff640 1 --2- 172.21.15.5:0/2190450098 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd20101de0 0x7fcd20198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:07.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.701+0000 7fcd1f7fe640 1 -- 172.21.15.5:0/2190450098 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd20101de0 msgr2=0x7fcd20198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:07.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.701+0000 7fcd1f7fe640 1 --2- 172.21.15.5:0/2190450098 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd20101de0 0x7fcd20198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:07.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.701+0000 7fcd1f7fe640 1 -- 172.21.15.5:0/2190450098 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcd20106f10 msgr2=0x7fcd2019cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:07.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.701+0000 7fcd1f7fe640 1 --2- 172.21.15.5:0/2190450098 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcd20106f10 0x7fcd2019cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:07.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.701+0000 7fcd1f7fe640 1 -- 172.21.15.5:0/2190450098 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd2019d3c0 con 0x7fcd20102790 2024-09-12T03:02:07.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.701+0000 7fcd24a12640 1 --2- 172.21.15.5:0/2190450098 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcd20106f10 0x7fcd2019cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:02:07.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.701+0000 7fcd1f7fe640 1 --2- 172.21.15.5:0/2190450098 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcd20102790 0x7fcd20198950 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fcd1400b4a0 tx=0x7fcd1400b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:07.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.701+0000 7fcd1d7fa640 1 -- 172.21.15.5:0/2190450098 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd14015020 con 0x7fcd20102790 2024-09-12T03:02:07.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.701+0000 7fcd1d7fa640 1 -- 172.21.15.5:0/2190450098 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd14011d10 con 0x7fcd20102790 2024-09-12T03:02:07.718 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.701+0000 7fcd26475640 1 -- 172.21.15.5:0/2190450098 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd2019d6b0 con 0x7fcd20102790 2024-09-12T03:02:07.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.701+0000 7fcd1d7fa640 1 -- 172.21.15.5:0/2190450098 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd1400f2e0 con 0x7fcd20102790 2024-09-12T03:02:07.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.701+0000 7fcd26475640 1 -- 172.21.15.5:0/2190450098 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd201a4f90 con 0x7fcd20102790 2024-09-12T03:02:07.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.703+0000 7fcd1d7fa640 1 -- 172.21.15.5:0/2190450098 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fcd14018020 con 0x7fcd20102790 2024-09-12T03:02:07.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.704+0000 7fcd2884e740 1 Processor -- start 2024-09-12T03:02:07.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.704+0000 7fcd2884e740 1 -- start start 2024-09-12T03:02:07.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.704+0000 7fcd2884e740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563255516b40 0x563255515090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:07.719 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.704+0000 7fcd2884e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632555155d0 0x563255515a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:07.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.704+0000 7fcd2884e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563255524780 0x563255526b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:07.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.704+0000 7fcd2884e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56325547b2f0 con 0x5632555155d0 2024-09-12T03:02:07.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.704+0000 7fcd2884e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x563255422cf0 con 0x563255524780 2024-09-12T03:02:07.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.704+0000 7fcd2884e740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5632554772b0 con 0x563255516b40 2024-09-12T03:02:07.720 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.704+0000 7fcd1d7fa640 1 --2- 172.21.15.5:0/2190450098 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fcd080a2b80 0x7fcd080a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:07.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.704+0000 7fcd1ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632555155d0 0x563255515a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:07.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.704+0000 7fcd1ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632555155d0 0x563255515a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38366/0 (socket says 172.21.15.5:38366) 2024-09-12T03:02:07.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.704+0000 7fcd1ffff640 1 -- 172.21.15.5:0/677880125 learned_addr learned my addr 172.21.15.5:0/677880125 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:02:07.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.704+0000 7fcd24a12640 1 --2- 172.21.15.5:0/2190450098 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fcd080a2b80 0x7fcd080a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:07.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.704+0000 7fcd1d7fa640 1 -- 172.21.15.5:0/2190450098 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fcd140ce920 con 0x7fcd20102790 2024-09-12T03:02:07.721 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.704+0000 7fcd1f7fe640 1 --2- 172.21.15.5:0/677880125 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563255524780 0x563255526b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:07.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.704+0000 7fcd24a12640 1 --2- 172.21.15.5:0/677880125 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563255516b40 0x563255515090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:07.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.705+0000 7fcd1ffff640 1 -- 172.21.15.5:0/677880125 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563255516b40 msgr2=0x563255515090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:07.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.705+0000 7fcd1ffff640 1 --2- 172.21.15.5:0/677880125 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563255516b40 0x563255515090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:07.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.705+0000 7fcd1ffff640 1 -- 172.21.15.5:0/677880125 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563255524780 msgr2=0x563255526b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:07.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.705+0000 7fcd1ffff640 1 --2- 172.21.15.5:0/677880125 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563255524780 0x563255526b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:07.722 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.705+0000 7fcd1ffff640 1 -- 172.21.15.5:0/677880125 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632554073a0 con 0x5632555155d0 2024-09-12T03:02:07.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.705+0000 7fcd1f7fe640 1 --2- 172.21.15.5:0/677880125 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563255524780 0x563255526b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:02:07.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.705+0000 7fcd1ffff640 1 --2- 172.21.15.5:0/677880125 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632555155d0 0x563255515a10 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fcd1000ad00 tx=0x7fcd1000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f005a932c73bca15 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:07.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.705+0000 7fcd24a12640 1 --2- 172.21.15.5:0/2190450098 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fcd080a2b80 0x7fcd080a5040 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fcd1802f540 tx=0x7fcd18005c30 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:07.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.705+0000 7fcd067fc640 1 -- 172.21.15.5:0/677880125 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd1001d070 con 0x5632555155d0 2024-09-12T03:02:07.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.705+0000 7fcd067fc640 1 -- 172.21.15.5:0/677880125 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd10018b20 con 0x5632555155d0 2024-09-12T03:02:07.723 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.705+0000 7fcd067fc640 1 -- 172.21.15.5:0/677880125 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd10010b30 con 0x5632555155d0 2024-09-12T03:02:07.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.706+0000 7fcd2884e740 1 -- 172.21.15.5:0/677880125 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632555155d0 msgr2=0x563255515a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:07.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.706+0000 7fcd2884e740 1 --2- 172.21.15.5:0/677880125 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632555155d0 0x563255515a10 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fcd1000ad00 tx=0x7fcd1000f790 comp rx=0 tx=0).stop 2024-09-12T03:02:07.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.706+0000 7fcd2884e740 1 -- 172.21.15.5:0/677880125 shutdown_connections 2024-09-12T03:02:07.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.706+0000 7fcd2884e740 1 --2- 172.21.15.5:0/677880125 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632555155d0 0x563255515a10 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:07.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.706+0000 7fcd2884e740 1 --2- 172.21.15.5:0/677880125 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563255524780 0x563255526b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:07.724 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.706+0000 7fcd2884e740 1 --2- 172.21.15.5:0/677880125 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563255516b40 0x563255515090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:07.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.706+0000 7fcd2884e740 1 -- 172.21.15.5:0/677880125 >> 172.21.15.5:0/677880125 conn(0x5632554de310 msgr2=0x563255511c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:07.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.706+0000 7fcd2884e740 1 -- 172.21.15.5:0/677880125 shutdown_connections 2024-09-12T03:02:07.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.706+0000 7fcd2884e740 1 -- 172.21.15.5:0/677880125 wait complete. 2024-09-12T03:02:07.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.706+0000 7fcd2884e740 1 -- start start 2024-09-12T03:02:07.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.706+0000 7fcd2884e740 1 Processor -- start 2024-09-12T03:02:07.725 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.707+0000 7fcd2884e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5632555155d0 0x563255524130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:07.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.707+0000 7fcd2884e740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563255516b40 0x5632555213e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:07.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.707+0000 7fcd2884e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563255524780 0x563255521980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:07.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.707+0000 7fcd2884e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x563255527480 con 0x563255524780 2024-09-12T03:02:07.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.707+0000 7fcd2884e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x563255436060 con 0x5632555155d0 2024-09-12T03:02:07.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.707+0000 7fcd2884e740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x563255527600 con 0x563255516b40 2024-09-12T03:02:07.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.707+0000 7fcd2884e740 10 client.0 _ll_register_callbacks cb 0x5632551898c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:02:07.726 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.707+0000 7fcd24a12640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563255524780 0x563255521980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:07.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.707+0000 7fcd24a12640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563255524780 0x563255521980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38376/0 (socket says 172.21.15.5:38376) 2024-09-12T03:02:07.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.707+0000 7fcd24a12640 1 -- 172.21.15.5:0/4280833499 learned_addr learned my addr 172.21.15.5:0/4280833499 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:02:07.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.707+0000 7fcd1f7fe640 1 --2- 172.21.15.5:0/4280833499 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563255516b40 0x5632555213e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:07.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.707+0000 7fcd1ffff640 1 --2- 172.21.15.5:0/4280833499 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5632555155d0 0x563255524130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:07.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.707+0000 7fcd24a12640 1 -- 172.21.15.5:0/4280833499 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563255516b40 msgr2=0x5632555213e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:07.727 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.707+0000 7fcd24a12640 1 --2- 172.21.15.5:0/4280833499 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563255516b40 0x5632555213e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:07.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.707+0000 7fcd24a12640 1 -- 172.21.15.5:0/4280833499 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5632555155d0 msgr2=0x563255524130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:07.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.707+0000 7fcd24a12640 1 --2- 172.21.15.5:0/4280833499 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5632555155d0 0x563255524130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:07.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.707+0000 7fcd24a12640 1 -- 172.21.15.5:0/4280833499 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632551ec3c0 con 0x563255524780 2024-09-12T03:02:07.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.707+0000 7fcd24a12640 1 --2- 172.21.15.5:0/4280833499 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563255524780 0x563255521980 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fcd18009ca0 tx=0x7fcd1802a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=118a46ba542a350e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:07.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.707+0000 7fccff7fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T03:02:07.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.707+0000 7fccff7fe640 1 -- 172.21.15.5:0/4280833499 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd1804f070 con 0x563255524780 2024-09-12T03:02:07.728 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.708+0000 7fccff7fe640 1 -- 172.21.15.5:0/4280833499 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fcd1802fba0 con 0x563255524780 2024-09-12T03:02:07.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.708+0000 7fcd2884e740 1 -- 172.21.15.5:0/4280833499 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd10018b20 con 0x563255524780 2024-09-12T03:02:07.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.708+0000 7fccff7fe640 1 -- 172.21.15.5:0/4280833499 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fcd1805a050 con 0x563255524780 2024-09-12T03:02:07.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.708+0000 7fcd2884e740 10 client.16287 fetch_fsmap learned FSMap version 30 2024-09-12T03:02:07.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.708+0000 7fcd2884e740 1 -- 172.21.15.5:0/4280833499 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563255532ed0 con 0x563255524780 2024-09-12T03:02:07.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.708+0000 7fccff7fe640 1 -- 172.21.15.5:0/4280833499 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd10018b20 con 0x563255524780 2024-09-12T03:02:07.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.708+0000 7fccff7fe640 1 -- 172.21.15.5:0/4280833499 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7fcd18049040 con 0x563255524780 2024-09-12T03:02:07.729 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.708+0000 7fcd2884e740 10 client.16287 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:02:07.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.708+0000 7fcd2884e740 10 client.16287 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-12T03:02:07.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.708+0000 7fcd2884e740 20 client.16287 populate_metadata read hostname 'smithi005' 2024-09-12T03:02:07.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.708+0000 7fcd2884e740 1 --2- 172.21.15.5:0/4280833499 >> [v2:172.21.15.148:6836/2606873922,v1:172.21.15.148:6837/2606873922] conn(0x5632555342e0 0x563255536770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:07.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.708+0000 7fcd2884e740 4 client.16287 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:02:07.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.708+0000 7fcd2884e740 1 -- 172.21.15.5:0/4280833499 --> [v2:172.21.15.148:6836/2606873922,v1:172.21.15.148:6837/2606873922] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563255488860 con 0x5632555342e0 2024-09-12T03:02:07.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.708+0000 7fcd1ffff640 1 --2- 172.21.15.5:0/4280833499 >> [v2:172.21.15.148:6836/2606873922,v1:172.21.15.148:6837/2606873922] conn(0x5632555342e0 0x563255536770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:07.730 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.709+0000 7fccff7fe640 10 client.16287 ms_handle_connect on v2:172.21.15.148:6836/2606873922 2024-09-12T03:02:07.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.709+0000 7fcd1ffff640 1 --2- 172.21.15.5:0/4280833499 >> [v2:172.21.15.148:6836/2606873922,v1:172.21.15.148:6837/2606873922] conn(0x5632555342e0 0x563255536770 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9f3e92018ccfa978 server_cookie=8957e0c72b6f26f2 in_seq=0 out_seq=0 2024-09-12T03:02:07.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.710+0000 7fccff7fe640 1 -- 172.21.15.5:0/4280833499 <== mds.1 v2:172.21.15.148:6836/2606873922 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563255488860 con 0x5632555342e0 2024-09-12T03:02:07.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.710+0000 7fccff7fe640 10 client.16287 handle_command_reply: tid=0 2024-09-12T03:02:07.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.710+0000 7fcd2884e740 1 client.16287 shutdown 2024-09-12T03:02:07.731 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.710+0000 7fcd2884e740 1 -- 172.21.15.5:0/4280833499 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563255524780 msgr2=0x563255521980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:07.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.710+0000 7fcd2884e740 1 --2- 172.21.15.5:0/4280833499 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563255524780 0x563255521980 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fcd18009ca0 tx=0x7fcd1802a8c0 comp rx=0 tx=0).stop 2024-09-12T03:02:07.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.710+0000 7fcd2884e740 1 -- 172.21.15.5:0/4280833499 shutdown_connections 2024-09-12T03:02:07.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.710+0000 7fcd2884e740 1 --2- 172.21.15.5:0/4280833499 >> [v2:172.21.15.148:6836/2606873922,v1:172.21.15.148:6837/2606873922] conn(0x5632555342e0 0x563255536770 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:07.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.710+0000 7fcd2884e740 1 --2- 172.21.15.5:0/4280833499 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563255524780 0x563255521980 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:07.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.710+0000 7fcd2884e740 1 --2- 172.21.15.5:0/4280833499 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563255516b40 0x5632555213e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:07.732 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.710+0000 7fcd2884e740 1 --2- 172.21.15.5:0/4280833499 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5632555155d0 0x563255524130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:07.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.710+0000 7fccff7fe640 0 client.16287 ms_handle_reset on v2:172.21.15.148:6836/2606873922 2024-09-12T03:02:07.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.710+0000 7fcd2884e740 1 -- 172.21.15.5:0/4280833499 >> 172.21.15.5:0/4280833499 conn(0x5632554de310 msgr2=0x563255517960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:07.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.710+0000 7fcd2884e740 1 -- 172.21.15.5:0/4280833499 shutdown_connections 2024-09-12T03:02:07.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.710+0000 7fcd2884e740 1 -- 172.21.15.5:0/4280833499 wait complete. 2024-09-12T03:02:07.733 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.710+0000 7fcd2884e740 20 client.16287 trim_cache size 0 max 16384 2024-09-12T03:02:07.741 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.723+0000 7fcd2884e740 1 Processor -- start 2024-09-12T03:02:07.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.723+0000 7fcd2884e740 1 -- start start 2024-09-12T03:02:07.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.723+0000 7fcd2884e740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5632555155d0 0x563255530720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:07.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.723+0000 7fcd2884e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563255516b40 0x5632555213a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:07.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.723+0000 7fcd2884e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563255524780 0x5632555218e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:07.742 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.723+0000 7fcd2884e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcd201ab2b0 con 0x563255516b40 2024-09-12T03:02:07.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.723+0000 7fcd2884e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fcd201ace20 con 0x563255524780 2024-09-12T03:02:07.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.723+0000 7fcd2884e740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fcd201ab980 con 0x5632555155d0 2024-09-12T03:02:07.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.723+0000 7fcd1f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563255516b40 0x5632555213a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:07.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.723+0000 7fcd1f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563255516b40 0x5632555213a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38386/0 (socket says 172.21.15.5:38386) 2024-09-12T03:02:07.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.723+0000 7fcd1f7fe640 1 -- 172.21.15.5:0/4168494720 learned_addr learned my addr 172.21.15.5:0/4168494720 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:02:07.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.724+0000 7fcd1ffff640 1 --2- 172.21.15.5:0/4168494720 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5632555155d0 0x563255530720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:07.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.724+0000 7fcd24a12640 1 --2- 172.21.15.5:0/4168494720 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563255524780 0x5632555218e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:07.743 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.724+0000 7fcd1f7fe640 1 -- 172.21.15.5:0/4168494720 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5632555155d0 msgr2=0x563255530720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:07.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.724+0000 7fcd1f7fe640 1 --2- 172.21.15.5:0/4168494720 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5632555155d0 0x563255530720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:07.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.724+0000 7fcd1f7fe640 1 -- 172.21.15.5:0/4168494720 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563255524780 msgr2=0x5632555218e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:07.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.724+0000 7fcd1f7fe640 1 --2- 172.21.15.5:0/4168494720 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563255524780 0x5632555218e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:07.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.724+0000 7fcd1f7fe640 1 -- 172.21.15.5:0/4168494720 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563255526fb0 con 0x563255516b40 2024-09-12T03:02:07.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.724+0000 7fcd1ffff640 1 --2- 172.21.15.5:0/4168494720 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5632555155d0 0x563255530720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:02:07.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.724+0000 7fcd1f7fe640 1 --2- 172.21.15.5:0/4168494720 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563255516b40 0x5632555213a0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fcd140cc480 tx=0x7fcd140bb040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d76807687fe00b26 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:07.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.724+0000 7fcd05ffb640 1 -- 172.21.15.5:0/4168494720 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd1409f540 con 0x563255516b40 2024-09-12T03:02:07.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.724+0000 7fcd05ffb640 1 -- 172.21.15.5:0/4168494720 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd140a3e00 con 0x563255516b40 2024-09-12T03:02:07.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.724+0000 7fcd05ffb640 1 -- 172.21.15.5:0/4168494720 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd14097800 con 0x563255516b40 2024-09-12T03:02:07.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.725+0000 7fcd2884e740 1 -- 172.21.15.5:0/4168494720 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563255516b40 msgr2=0x5632555213a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:07.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.725+0000 7fcd2884e740 1 --2- 172.21.15.5:0/4168494720 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563255516b40 0x5632555213a0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fcd140cc480 tx=0x7fcd140bb040 comp rx=0 tx=0).stop 2024-09-12T03:02:07.744 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.725+0000 7fcd2884e740 1 -- 172.21.15.5:0/4168494720 shutdown_connections 2024-09-12T03:02:07.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.725+0000 7fcd2884e740 1 --2- 172.21.15.5:0/4168494720 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563255516b40 0x5632555213a0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:07.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.725+0000 7fcd2884e740 1 --2- 172.21.15.5:0/4168494720 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563255524780 0x5632555218e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:07.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.725+0000 7fcd2884e740 1 --2- 172.21.15.5:0/4168494720 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5632555155d0 0x563255530720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:07.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.725+0000 7fcd2884e740 1 -- 172.21.15.5:0/4168494720 >> 172.21.15.5:0/4168494720 conn(0x5632554de310 msgr2=0x56325551c720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:07.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.725+0000 7fcd2884e740 1 -- 172.21.15.5:0/4168494720 shutdown_connections 2024-09-12T03:02:07.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.725+0000 7fcd2884e740 1 -- 172.21.15.5:0/4168494720 wait complete. 2024-09-12T03:02:07.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.725+0000 7fcd2884e740 1 -- start start 2024-09-12T03:02:07.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.725+0000 7fcd2884e740 1 Processor -- start 2024-09-12T03:02:07.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.726+0000 7fcd2884e740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5632555155d0 0x56325552cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:07.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.726+0000 7fcd2884e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563255524780 0x56325553b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:07.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.726+0000 7fcd2884e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56325553bd20 0x56325553c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:07.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.726+0000 7fcd2884e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcd201ac720 con 0x56325553bd20 2024-09-12T03:02:07.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.726+0000 7fcd2884e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fcd201ac050 con 0x563255524780 2024-09-12T03:02:07.745 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.726+0000 7fcd2884e740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fcd201a9290 con 0x5632555155d0 2024-09-12T03:02:07.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.726+0000 7fcd2884e740 10 client.0 _ll_register_callbacks cb 0x7fcd18009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:02:07.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.726+0000 7fcd24a12640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56325553bd20 0x56325553c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:07.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.726+0000 7fcd24a12640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56325553bd20 0x56325553c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38394/0 (socket says 172.21.15.5:38394) 2024-09-12T03:02:07.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.726+0000 7fcd24a12640 1 -- 172.21.15.5:0/3141090346 learned_addr learned my addr 172.21.15.5:0/3141090346 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:02:07.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.726+0000 7fcd1ffff640 1 --2- 172.21.15.5:0/3141090346 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5632555155d0 0x56325552cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:07.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.726+0000 7fcd1f7fe640 1 --2- 172.21.15.5:0/3141090346 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563255524780 0x56325553b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:07.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.726+0000 7fcd24a12640 1 -- 172.21.15.5:0/3141090346 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5632555155d0 msgr2=0x56325552cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:07.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.726+0000 7fcd24a12640 1 --2- 172.21.15.5:0/3141090346 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5632555155d0 0x56325552cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:07.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.726+0000 7fcd24a12640 1 -- 172.21.15.5:0/3141090346 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563255524780 msgr2=0x56325553b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:07.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.726+0000 7fcd24a12640 1 --2- 172.21.15.5:0/3141090346 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563255524780 0x56325553b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:07.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.726+0000 7fcd24a12640 1 -- 172.21.15.5:0/3141090346 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56325552aa30 con 0x56325553bd20 2024-09-12T03:02:07.746 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.726+0000 7fcd1f7fe640 1 --2- 172.21.15.5:0/3141090346 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563255524780 0x56325553b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:02:07.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.726+0000 7fcd24a12640 1 --2- 172.21.15.5:0/3141090346 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56325553bd20 0x56325553c1a0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fcd18009c70 tx=0x7fcd1804bb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26de0acfdbb7f9a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:07.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.726+0000 7fccff7fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T03:02:07.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.727+0000 7fccff7fe640 1 -- 172.21.15.5:0/3141090346 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd18063060 con 0x56325553bd20 2024-09-12T03:02:07.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.727+0000 7fccff7fe640 1 -- 172.21.15.5:0/3141090346 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fcd180305c0 con 0x56325553bd20 2024-09-12T03:02:07.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.727+0000 7fcd2884e740 1 -- 172.21.15.5:0/3141090346 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd140a3e00 con 0x56325553bd20 2024-09-12T03:02:07.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.727+0000 7fccff7fe640 1 -- 172.21.15.5:0/3141090346 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fcd18057070 con 0x56325553bd20 2024-09-12T03:02:07.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.727+0000 7fcd2884e740 10 client.16293 fetch_fsmap learned FSMap version 30 2024-09-12T03:02:07.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.727+0000 7fcd2884e740 1 -- 172.21.15.5:0/3141090346 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56325559f300 con 0x56325553bd20 2024-09-12T03:02:07.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.727+0000 7fccff7fe640 1 -- 172.21.15.5:0/3141090346 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd140a3e00 con 0x56325553bd20 2024-09-12T03:02:07.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.727+0000 7fccff7fe640 1 -- 172.21.15.5:0/3141090346 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7fcd1803c7e0 con 0x56325553bd20 2024-09-12T03:02:07.747 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.727+0000 7fcd2884e740 10 client.16293 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:02:07.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.727+0000 7fcd2884e740 10 client.16293 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-12T03:02:07.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.727+0000 7fcd2884e740 20 client.16293 populate_metadata read hostname 'smithi005' 2024-09-12T03:02:07.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.727+0000 7fcd2884e740 1 --2- 172.21.15.5:0/3141090346 >> [v2:172.21.15.148:6836/2606873922,v1:172.21.15.148:6837/2606873922] conn(0x56325559fa10 0x5632555a1e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:07.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.727+0000 7fcd2884e740 4 client.16293 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:02:07.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.727+0000 7fcd2884e740 1 -- 172.21.15.5:0/3141090346 --> [v2:172.21.15.148:6836/2606873922,v1:172.21.15.148:6837/2606873922] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563255482d70 con 0x56325559fa10 2024-09-12T03:02:07.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.728+0000 7fcd1ffff640 1 --2- 172.21.15.5:0/3141090346 >> [v2:172.21.15.148:6836/2606873922,v1:172.21.15.148:6837/2606873922] conn(0x56325559fa10 0x5632555a1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:07.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.728+0000 7fcd1ffff640 1 --2- 172.21.15.5:0/3141090346 >> [v2:172.21.15.148:6836/2606873922,v1:172.21.15.148:6837/2606873922] conn(0x56325559fa10 0x5632555a1e30 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=706768a9e874faaf server_cookie=7f0e1755f0058b6e in_seq=0 out_seq=0 2024-09-12T03:02:07.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.728+0000 7fccff7fe640 10 client.16293 ms_handle_connect on v2:172.21.15.148:6836/2606873922 2024-09-12T03:02:07.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.728+0000 7fccff7fe640 1 -- 172.21.15.5:0/3141090346 <== mds.1 v2:172.21.15.148:6836/2606873922 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563255482d70 con 0x56325559fa10 2024-09-12T03:02:07.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.728+0000 7fccff7fe640 10 client.16293 handle_command_reply: tid=0 2024-09-12T03:02:07.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.728+0000 7fcd2884e740 1 client.16293 shutdown 2024-09-12T03:02:07.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.728+0000 7fcd2884e740 1 -- 172.21.15.5:0/3141090346 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56325553bd20 msgr2=0x56325553c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:07.748 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.728+0000 7fcd2884e740 1 --2- 172.21.15.5:0/3141090346 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56325553bd20 0x56325553c1a0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fcd18009c70 tx=0x7fcd1804bb00 comp rx=0 tx=0).stop 2024-09-12T03:02:07.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.728+0000 7fcd2884e740 1 -- 172.21.15.5:0/3141090346 shutdown_connections 2024-09-12T03:02:07.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.728+0000 7fcd2884e740 1 --2- 172.21.15.5:0/3141090346 >> [v2:172.21.15.148:6836/2606873922,v1:172.21.15.148:6837/2606873922] conn(0x56325559fa10 0x5632555a1e30 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:07.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.728+0000 7fcd2884e740 1 --2- 172.21.15.5:0/3141090346 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56325553bd20 0x56325553c1a0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:07.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.728+0000 7fcd2884e740 1 --2- 172.21.15.5:0/3141090346 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563255524780 0x56325553b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:07.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.728+0000 7fccff7fe640 0 client.16293 ms_handle_reset on v2:172.21.15.148:6836/2606873922 2024-09-12T03:02:07.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.728+0000 7fcd2884e740 1 --2- 172.21.15.5:0/3141090346 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5632555155d0 0x56325552cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:07.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.728+0000 7fcd2884e740 1 -- 172.21.15.5:0/3141090346 >> 172.21.15.5:0/3141090346 conn(0x5632554de310 msgr2=0x5632555213a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:07.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.728+0000 7fcd2884e740 1 -- 172.21.15.5:0/3141090346 shutdown_connections 2024-09-12T03:02:07.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.729+0000 7fcd2884e740 1 -- 172.21.15.5:0/3141090346 wait complete. 2024-09-12T03:02:07.749 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.729+0000 7fcd2884e740 20 client.16293 trim_cache size 0 max 16384 2024-09-12T03:02:07.750 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-12T03:02:07.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.729+0000 7fcd26475640 1 -- 172.21.15.5:0/2190450098 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fcd080a2b80 msgr2=0x7fcd080a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:07.750 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.729+0000 7fcd26475640 1 --2- 172.21.15.5:0/2190450098 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fcd080a2b80 0x7fcd080a5040 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fcd1802f540 tx=0x7fcd18005c30 comp rx=0 tx=0).stop 2024-09-12T03:02:07.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.729+0000 7fcd26475640 1 -- 172.21.15.5:0/2190450098 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcd20102790 msgr2=0x7fcd20198950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:07.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.729+0000 7fcd26475640 1 --2- 172.21.15.5:0/2190450098 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcd20102790 0x7fcd20198950 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fcd1400b4a0 tx=0x7fcd1400b8a0 comp rx=0 tx=0).stop 2024-09-12T03:02:07.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.729+0000 7fcd26475640 1 -- 172.21.15.5:0/2190450098 shutdown_connections 2024-09-12T03:02:07.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.729+0000 7fcd26475640 1 --2- 172.21.15.5:0/2190450098 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fcd080a2b80 0x7fcd080a5040 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:07.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.729+0000 7fcd26475640 1 --2- 172.21.15.5:0/2190450098 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcd20102790 0x7fcd20198950 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:07.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.729+0000 7fcd26475640 1 --2- 172.21.15.5:0/2190450098 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcd20106f10 0x7fcd2019cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:07.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.729+0000 7fcd26475640 1 --2- 172.21.15.5:0/2190450098 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcd20101de0 0x7fcd20198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:07.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.729+0000 7fcd26475640 1 -- 172.21.15.5:0/2190450098 >> 172.21.15.5:0/2190450098 conn(0x7fcd200fd8c0 msgr2=0x7fcd200ffcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:07.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.729+0000 7fcd26475640 1 -- 172.21.15.5:0/2190450098 shutdown_connections 2024-09-12T03:02:07.751 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:07.729+0000 7fcd26475640 1 -- 172.21.15.5:0/2190450098 wait complete. 2024-09-12T03:02:07.915 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:02:07 smithi148 ceph-mon[35090]: pgmap v451: 97 pgs: 97 active+clean; 114 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.7 MiB/s wr, 660 op/s 2024-09-12T03:02:07.956 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:02:07 smithi146 ceph-mon[35011]: pgmap v451: 97 pgs: 97 active+clean; 114 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.7 MiB/s wr, 660 op/s 2024-09-12T03:02:08.637 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T03:02:08.896 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:02:09.328 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.328+0000 7fb9cb545640 1 -- 172.21.15.5:0/607528452 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9c4074eb0 msgr2=0x7fb9c4112650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:09.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.328+0000 7fb9cb545640 1 --2- 172.21.15.5:0/607528452 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9c4074eb0 0x7fb9c4112650 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fb9b4009900 tx=0x7fb9b402a480 comp rx=0 tx=0).stop 2024-09-12T03:02:09.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.328+0000 7fb9cb545640 1 -- 172.21.15.5:0/607528452 shutdown_connections 2024-09-12T03:02:09.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.328+0000 7fb9cb545640 1 --2- 172.21.15.5:0/607528452 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9c4074eb0 0x7fb9c4112650 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:09.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.328+0000 7fb9cb545640 1 --2- 172.21.15.5:0/607528452 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb9c4112b90 0x7fb9c4114f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:09.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.328+0000 7fb9cb545640 1 --2- 172.21.15.5:0/607528452 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb9c4074590 0x7fb9c4074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:09.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.328+0000 7fb9cb545640 1 -- 172.21.15.5:0/607528452 >> 172.21.15.5:0/607528452 conn(0x7fb9c40fd8c0 msgr2=0x7fb9c40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:09.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.328+0000 7fb9cb545640 1 -- 172.21.15.5:0/607528452 shutdown_connections 2024-09-12T03:02:09.329 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.328+0000 7fb9cb545640 1 -- 172.21.15.5:0/607528452 wait complete. 2024-09-12T03:02:09.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.329+0000 7fb9cb545640 1 Processor -- start 2024-09-12T03:02:09.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.329+0000 7fb9cb545640 1 -- start start 2024-09-12T03:02:09.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.329+0000 7fb9cb545640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb9c4074590 0x7fb9c41a4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:09.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.329+0000 7fb9cb545640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9c4074eb0 0x7fb9c41a4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:09.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.329+0000 7fb9cb545640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb9c4112b90 0x7fb9c419e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:09.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.329+0000 7fb9cb545640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb9c4118560 con 0x7fb9c4074eb0 2024-09-12T03:02:09.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.329+0000 7fb9cb545640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fb9c41183e0 con 0x7fb9c4074590 2024-09-12T03:02:09.331 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.329+0000 7fb9cb545640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb9c41186e0 con 0x7fb9c4112b90 2024-09-12T03:02:09.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.329+0000 7fb9c8ae0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9c4074eb0 0x7fb9c41a4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:09.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.329+0000 7fb9c8ae0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9c4074eb0 0x7fb9c41a4c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38404/0 (socket says 172.21.15.5:38404) 2024-09-12T03:02:09.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.329+0000 7fb9c8ae0640 1 -- 172.21.15.5:0/440017542 learned_addr learned my addr 172.21.15.5:0/440017542 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:02:09.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.329+0000 7fb9c92e1640 1 --2- 172.21.15.5:0/440017542 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb9c4074590 0x7fb9c41a4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:09.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.329+0000 7fb9c9ae2640 1 --2- 172.21.15.5:0/440017542 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb9c4112b90 0x7fb9c419e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:09.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.330+0000 7fb9c8ae0640 1 -- 172.21.15.5:0/440017542 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb9c4112b90 msgr2=0x7fb9c419e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:09.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.330+0000 7fb9c8ae0640 1 --2- 172.21.15.5:0/440017542 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb9c4112b90 0x7fb9c419e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:09.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.330+0000 7fb9c8ae0640 1 -- 172.21.15.5:0/440017542 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb9c4074590 msgr2=0x7fb9c41a4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:09.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.330+0000 7fb9c8ae0640 1 --2- 172.21.15.5:0/440017542 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb9c4074590 0x7fb9c41a4730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:09.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.330+0000 7fb9c8ae0640 1 -- 172.21.15.5:0/440017542 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9c419f020 con 0x7fb9c4074eb0 2024-09-12T03:02:09.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.330+0000 7fb9c92e1640 1 --2- 172.21.15.5:0/440017542 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb9c4074590 0x7fb9c41a4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:02:09.332 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.330+0000 7fb9c8ae0640 1 --2- 172.21.15.5:0/440017542 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9c4074eb0 0x7fb9c41a4c70 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fb9b4006010 tx=0x7fb9b4004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:09.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.330+0000 7fb9ba7fc640 1 -- 172.21.15.5:0/440017542 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9b403f030 con 0x7fb9c4074eb0 2024-09-12T03:02:09.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.330+0000 7fb9cb545640 1 -- 172.21.15.5:0/440017542 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9c419f2b0 con 0x7fb9c4074eb0 2024-09-12T03:02:09.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.330+0000 7fb9ba7fc640 1 -- 172.21.15.5:0/440017542 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9b4004440 con 0x7fb9c4074eb0 2024-09-12T03:02:09.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.330+0000 7fb9ba7fc640 1 -- 172.21.15.5:0/440017542 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9b4035070 con 0x7fb9c4074eb0 2024-09-12T03:02:09.333 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.330+0000 7fb9cb545640 1 -- 172.21.15.5:0/440017542 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9c41ab600 con 0x7fb9c4074eb0 2024-09-12T03:02:09.334 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.333+0000 7fb9ba7fc640 1 -- 172.21.15.5:0/440017542 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fb9b4042030 con 0x7fb9c4074eb0 2024-09-12T03:02:09.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.333+0000 7fb9cd91e740 1 Processor -- start 2024-09-12T03:02:09.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.333+0000 7fb9cd91e740 1 -- start start 2024-09-12T03:02:09.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.333+0000 7fb9cd91e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556d355e7b40 0x556d355e6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:09.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.333+0000 7fb9cd91e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556d355e65d0 0x556d355e6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:09.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.333+0000 7fb9cd91e740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556d355f1370 0x556d355f3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:09.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.333+0000 7fb9cd91e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556d3554c2f0 con 0x556d355e65d0 2024-09-12T03:02:09.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.333+0000 7fb9cd91e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x556d354f3cf0 con 0x556d355e7b40 2024-09-12T03:02:09.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.333+0000 7fb9cd91e740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x556d355482b0 con 0x556d355f1370 2024-09-12T03:02:09.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.333+0000 7fb9c92e1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556d355e65d0 0x556d355e6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:09.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.334+0000 7fb9ba7fc640 1 --2- 172.21.15.5:0/440017542 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fb9b00a2c00 0x7fb9b00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:09.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.334+0000 7fb9c92e1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556d355e65d0 0x556d355e6a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38420/0 (socket says 172.21.15.5:38420) 2024-09-12T03:02:09.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.334+0000 7fb9c92e1640 1 -- 172.21.15.5:0/2322051181 learned_addr learned my addr 172.21.15.5:0/2322051181 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:02:09.335 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.334+0000 7fb9c9ae2640 1 --2- 172.21.15.5:0/2322051181 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556d355e7b40 0x556d355e6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:09.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.334+0000 7fb9c8ae0640 1 --2- 172.21.15.5:0/2322051181 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556d355f1370 0x556d355f3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:09.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.334+0000 7fb9c9ae2640 1 --2- 172.21.15.5:0/440017542 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fb9b00a2c00 0x7fb9b00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:09.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.334+0000 7fb9ba7fc640 1 -- 172.21.15.5:0/440017542 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fb9b40eebf0 con 0x7fb9c4074eb0 2024-09-12T03:02:09.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.334+0000 7fb9c92e1640 1 -- 172.21.15.5:0/2322051181 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556d355f1370 msgr2=0x556d355f3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:09.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.334+0000 7fb9c92e1640 1 --2- 172.21.15.5:0/2322051181 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556d355f1370 0x556d355f3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:09.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.334+0000 7fb9c92e1640 1 -- 172.21.15.5:0/2322051181 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556d355e7b40 msgr2=0x556d355e6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:09.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.334+0000 7fb9c92e1640 1 --2- 172.21.15.5:0/2322051181 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556d355e7b40 0x556d355e6090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:09.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.334+0000 7fb9c92e1640 1 -- 172.21.15.5:0/2322051181 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d354d83a0 con 0x556d355e65d0 2024-09-12T03:02:09.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.334+0000 7fb9c8ae0640 1 --2- 172.21.15.5:0/2322051181 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556d355f1370 0x556d355f3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:02:09.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.334+0000 7fb9c92e1640 1 --2- 172.21.15.5:0/2322051181 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556d355e65d0 0x556d355e6a10 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fb9c00114f0 tx=0x7fb9c00119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=776d05994f62ed9e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:09.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.334+0000 7fb9c9ae2640 1 --2- 172.21.15.5:0/440017542 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fb9b00a2c00 0x7fb9b00a50c0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fb9c4076600 tx=0x7fb9bc009500 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:09.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.334+0000 7fb9af7fe640 1 -- 172.21.15.5:0/2322051181 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9c0004020 con 0x556d355e65d0 2024-09-12T03:02:09.336 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.334+0000 7fb9af7fe640 1 -- 172.21.15.5:0/2322051181 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9c00041c0 con 0x556d355e65d0 2024-09-12T03:02:09.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.334+0000 7fb9af7fe640 1 -- 172.21.15.5:0/2322051181 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9c000f940 con 0x556d355e65d0 2024-09-12T03:02:09.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.335+0000 7fb9cd91e740 1 -- 172.21.15.5:0/2322051181 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556d355e65d0 msgr2=0x556d355e6a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:09.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.335+0000 7fb9cd91e740 1 --2- 172.21.15.5:0/2322051181 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556d355e65d0 0x556d355e6a10 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fb9c00114f0 tx=0x7fb9c00119b0 comp rx=0 tx=0).stop 2024-09-12T03:02:09.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.335+0000 7fb9cd91e740 1 -- 172.21.15.5:0/2322051181 shutdown_connections 2024-09-12T03:02:09.340 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.335+0000 7fb9cd91e740 1 --2- 172.21.15.5:0/2322051181 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556d355e65d0 0x556d355e6a10 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:09.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.335+0000 7fb9cd91e740 1 --2- 172.21.15.5:0/2322051181 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556d355f1370 0x556d355f3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:09.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.335+0000 7fb9cd91e740 1 --2- 172.21.15.5:0/2322051181 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556d355e7b40 0x556d355e6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:09.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.335+0000 7fb9cd91e740 1 -- 172.21.15.5:0/2322051181 >> 172.21.15.5:0/2322051181 conn(0x556d355af310 msgr2=0x556d355e2c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:09.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.335+0000 7fb9cd91e740 1 -- 172.21.15.5:0/2322051181 shutdown_connections 2024-09-12T03:02:09.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.335+0000 7fb9cd91e740 1 -- 172.21.15.5:0/2322051181 wait complete. 2024-09-12T03:02:09.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.336+0000 7fb9cd91e740 1 -- start start 2024-09-12T03:02:09.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.336+0000 7fb9cd91e740 1 Processor -- start 2024-09-12T03:02:09.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.336+0000 7fb9cd91e740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556d355e65d0 0x556d355f0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:09.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.336+0000 7fb9cd91e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556d355e7b40 0x556d355edfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:09.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.336+0000 7fb9cd91e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556d355f1370 0x556d355ee520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:09.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.336+0000 7fb9cd91e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x556d355f4070 con 0x556d355f1370 2024-09-12T03:02:09.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.336+0000 7fb9cd91e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x556d35507060 con 0x556d355e7b40 2024-09-12T03:02:09.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.336+0000 7fb9cd91e740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x556d355f41f0 con 0x556d355e65d0 2024-09-12T03:02:09.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.336+0000 7fb9cd91e740 10 client.0 _ll_register_callbacks cb 0x556d3525a8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:02:09.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.336+0000 7fb9c9ae2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556d355f1370 0x556d355ee520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:09.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.336+0000 7fb9c9ae2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556d355f1370 0x556d355ee520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38430/0 (socket says 172.21.15.5:38430) 2024-09-12T03:02:09.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.336+0000 7fb9c9ae2640 1 -- 172.21.15.5:0/3137911434 learned_addr learned my addr 172.21.15.5:0/3137911434 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:02:09.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.336+0000 7fb9c92e1640 1 --2- 172.21.15.5:0/3137911434 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556d355e65d0 0x556d355f0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:09.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.336+0000 7fb9c8ae0640 1 --2- 172.21.15.5:0/3137911434 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556d355e7b40 0x556d355edfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:09.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.337+0000 7fb9c9ae2640 1 -- 172.21.15.5:0/3137911434 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556d355e65d0 msgr2=0x556d355f0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:09.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.337+0000 7fb9c9ae2640 1 --2- 172.21.15.5:0/3137911434 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556d355e65d0 0x556d355f0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:09.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.337+0000 7fb9c9ae2640 1 -- 172.21.15.5:0/3137911434 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556d355e7b40 msgr2=0x556d355edfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:09.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.337+0000 7fb9c9ae2640 1 --2- 172.21.15.5:0/3137911434 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556d355e7b40 0x556d355edfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:09.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.337+0000 7fb9c9ae2640 1 -- 172.21.15.5:0/3137911434 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d352bd3c0 con 0x556d355f1370 2024-09-12T03:02:09.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.337+0000 7fb9c9ae2640 1 --2- 172.21.15.5:0/3137911434 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556d355f1370 0x556d355ee520 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc00fef0 tx=0x7fb9bc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc3ac7ee7d4fa4b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:09.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.337+0000 7fb9acff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T03:02:09.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.337+0000 7fb9acff9640 1 -- 172.21.15.5:0/3137911434 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9bc02e060 con 0x556d355f1370 2024-09-12T03:02:09.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.337+0000 7fb9cd91e740 1 -- 172.21.15.5:0/3137911434 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9c00041c0 con 0x556d355f1370 2024-09-12T03:02:09.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.337+0000 7fb9acff9640 1 -- 172.21.15.5:0/3137911434 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fb9bc021c00 con 0x556d355f1370 2024-09-12T03:02:09.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.337+0000 7fb9acff9640 1 -- 172.21.15.5:0/3137911434 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb9bc021e00 con 0x556d355f1370 2024-09-12T03:02:09.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.337+0000 7fb9cd91e740 10 client.16305 fetch_fsmap learned FSMap version 30 2024-09-12T03:02:09.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.337+0000 7fb9cd91e740 1 -- 172.21.15.5:0/3137911434 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556d355ffb00 con 0x556d355f1370 2024-09-12T03:02:09.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.338+0000 7fb9acff9640 1 -- 172.21.15.5:0/3137911434 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9c00041c0 con 0x556d355f1370 2024-09-12T03:02:09.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.338+0000 7fb9acff9640 1 -- 172.21.15.5:0/3137911434 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7fb9bc01d070 con 0x556d355f1370 2024-09-12T03:02:09.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.338+0000 7fb9cd91e740 10 client.16305 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:02:09.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.338+0000 7fb9cd91e740 10 client.16305 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:02:09.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.338+0000 7fb9cd91e740 20 client.16305 populate_metadata read hostname 'smithi005' 2024-09-12T03:02:09.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.338+0000 7fb9cd91e740 1 --2- 172.21.15.5:0/3137911434 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x556d35600fc0 0x556d35603430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:09.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.338+0000 7fb9cd91e740 4 client.16305 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:02:09.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.338+0000 7fb9cd91e740 1 -- 172.21.15.5:0/3137911434 --> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556d35559860 con 0x556d35600fc0 2024-09-12T03:02:09.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.338+0000 7fb9c92e1640 1 --2- 172.21.15.5:0/3137911434 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x556d35600fc0 0x556d35603430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:09.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.338+0000 7fb9acff9640 10 client.16305 ms_handle_connect on v2:172.21.15.148:6838/2317600000 2024-09-12T03:02:09.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.338+0000 7fb9c92e1640 1 --2- 172.21.15.5:0/3137911434 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x556d35600fc0 0x556d35603430 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6fc178534e813d84 server_cookie=d4d486c3466869d0 in_seq=0 out_seq=0 2024-09-12T03:02:09.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.339+0000 7fb9acff9640 1 -- 172.21.15.5:0/3137911434 <== mds.0 v2:172.21.15.148:6838/2317600000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556d35559860 con 0x556d35600fc0 2024-09-12T03:02:09.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.339+0000 7fb9acff9640 10 client.16305 handle_command_reply: tid=0 2024-09-12T03:02:09.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.339+0000 7fb9cd91e740 1 client.16305 shutdown 2024-09-12T03:02:09.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.339+0000 7fb9cd91e740 1 -- 172.21.15.5:0/3137911434 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556d355f1370 msgr2=0x556d355ee520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:09.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.339+0000 7fb9cd91e740 1 --2- 172.21.15.5:0/3137911434 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556d355f1370 0x556d355ee520 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc00fef0 tx=0x7fb9bc004a40 comp rx=0 tx=0).stop 2024-09-12T03:02:09.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.339+0000 7fb9cd91e740 1 -- 172.21.15.5:0/3137911434 shutdown_connections 2024-09-12T03:02:09.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.339+0000 7fb9cd91e740 1 --2- 172.21.15.5:0/3137911434 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x556d35600fc0 0x556d35603430 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:09.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.339+0000 7fb9cd91e740 1 --2- 172.21.15.5:0/3137911434 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556d355f1370 0x556d355ee520 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:09.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.339+0000 7fb9cd91e740 1 --2- 172.21.15.5:0/3137911434 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556d355e7b40 0x556d355edfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:09.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.339+0000 7fb9acff9640 0 client.16305 ms_handle_reset on v2:172.21.15.148:6838/2317600000 2024-09-12T03:02:09.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.339+0000 7fb9cd91e740 1 --2- 172.21.15.5:0/3137911434 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556d355e65d0 0x556d355f0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:09.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.339+0000 7fb9cd91e740 1 -- 172.21.15.5:0/3137911434 >> 172.21.15.5:0/3137911434 conn(0x556d355af310 msgr2=0x556d355e8960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:09.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.340+0000 7fb9cd91e740 1 -- 172.21.15.5:0/3137911434 shutdown_connections 2024-09-12T03:02:09.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.340+0000 7fb9cd91e740 1 -- 172.21.15.5:0/3137911434 wait complete. 2024-09-12T03:02:09.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.340+0000 7fb9cd91e740 20 client.16305 trim_cache size 0 max 16384 2024-09-12T03:02:09.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.353+0000 7fb9cd91e740 1 Processor -- start 2024-09-12T03:02:09.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.353+0000 7fb9cd91e740 1 -- start start 2024-09-12T03:02:09.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.353+0000 7fb9cd91e740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556d355e65d0 0x556d355ed860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:09.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.353+0000 7fb9cd91e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556d355e7b40 0x556d355eff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:09.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.353+0000 7fb9cd91e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556d355f1370 0x556d355e9120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:09.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.353+0000 7fb9cd91e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb9c41b3b70 con 0x556d355f1370 2024-09-12T03:02:09.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.353+0000 7fb9cd91e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fb9c41b56e0 con 0x556d355e7b40 2024-09-12T03:02:09.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.353+0000 7fb9cd91e740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb9c41b4240 con 0x556d355e65d0 2024-09-12T03:02:09.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.353+0000 7fb9c9ae2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556d355f1370 0x556d355e9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:09.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.353+0000 7fb9c9ae2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556d355f1370 0x556d355e9120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38436/0 (socket says 172.21.15.5:38436) 2024-09-12T03:02:09.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.353+0000 7fb9c9ae2640 1 -- 172.21.15.5:0/1708087453 learned_addr learned my addr 172.21.15.5:0/1708087453 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:02:09.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.353+0000 7fb9c8ae0640 1 --2- 172.21.15.5:0/1708087453 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556d355e7b40 0x556d355eff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:09.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.353+0000 7fb9c92e1640 1 --2- 172.21.15.5:0/1708087453 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556d355e65d0 0x556d355ed860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:09.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.353+0000 7fb9c9ae2640 1 -- 172.21.15.5:0/1708087453 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556d355e65d0 msgr2=0x556d355ed860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:09.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.353+0000 7fb9c9ae2640 1 --2- 172.21.15.5:0/1708087453 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556d355e65d0 0x556d355ed860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:09.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.353+0000 7fb9c9ae2640 1 -- 172.21.15.5:0/1708087453 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556d355e7b40 msgr2=0x556d355eff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:09.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.353+0000 7fb9c9ae2640 1 --2- 172.21.15.5:0/1708087453 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556d355e7b40 0x556d355eff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:09.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.353+0000 7fb9c9ae2640 1 -- 172.21.15.5:0/1708087453 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d355ecf70 con 0x556d355f1370 2024-09-12T03:02:09.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.353+0000 7fb9c8ae0640 1 --2- 172.21.15.5:0/1708087453 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556d355e7b40 0x556d355eff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:02:09.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.353+0000 7fb9c9ae2640 1 --2- 172.21.15.5:0/1708087453 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556d355f1370 0x556d355e9120 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc029560 tx=0x7fb9bc021880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb16a6a52bd6b008 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:09.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.354+0000 7fb9aeffd640 1 -- 172.21.15.5:0/1708087453 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9bc039020 con 0x556d355f1370 2024-09-12T03:02:09.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.354+0000 7fb9aeffd640 1 -- 172.21.15.5:0/1708087453 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9bc009ce0 con 0x556d355f1370 2024-09-12T03:02:09.355 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.354+0000 7fb9aeffd640 1 -- 172.21.15.5:0/1708087453 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9bc005420 con 0x556d355f1370 2024-09-12T03:02:09.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.354+0000 7fb9cd91e740 1 -- 172.21.15.5:0/1708087453 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556d355f1370 msgr2=0x556d355e9120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:09.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.354+0000 7fb9cd91e740 1 --2- 172.21.15.5:0/1708087453 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556d355f1370 0x556d355e9120 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc029560 tx=0x7fb9bc021880 comp rx=0 tx=0).stop 2024-09-12T03:02:09.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.354+0000 7fb9cd91e740 1 -- 172.21.15.5:0/1708087453 shutdown_connections 2024-09-12T03:02:09.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.354+0000 7fb9cd91e740 1 --2- 172.21.15.5:0/1708087453 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556d355f1370 0x556d355e9120 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:09.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.354+0000 7fb9cd91e740 1 --2- 172.21.15.5:0/1708087453 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556d355e7b40 0x556d355eff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:09.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.354+0000 7fb9cd91e740 1 --2- 172.21.15.5:0/1708087453 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556d355e65d0 0x556d355ed860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:09.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.354+0000 7fb9cd91e740 1 -- 172.21.15.5:0/1708087453 >> 172.21.15.5:0/1708087453 conn(0x556d355af310 msgr2=0x556d355ef800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:09.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.354+0000 7fb9cd91e740 1 -- 172.21.15.5:0/1708087453 shutdown_connections 2024-09-12T03:02:09.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.354+0000 7fb9cd91e740 1 -- 172.21.15.5:0/1708087453 wait complete. 2024-09-12T03:02:09.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.355+0000 7fb9cd91e740 1 -- start start 2024-09-12T03:02:09.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.355+0000 7fb9cd91e740 1 Processor -- start 2024-09-12T03:02:09.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.355+0000 7fb9cd91e740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556d355e65d0 0x556d355eaf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:09.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.355+0000 7fb9cd91e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556d355e7b40 0x556d355eb490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:09.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.355+0000 7fb9cd91e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556d355f1370 0x556d355eb9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:09.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.355+0000 7fb9cd91e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb9c41b4fe0 con 0x556d355f1370 2024-09-12T03:02:09.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.355+0000 7fb9cd91e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fb9c41b4910 con 0x556d355e7b40 2024-09-12T03:02:09.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.355+0000 7fb9cd91e740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb9c41b1b50 con 0x556d355e65d0 2024-09-12T03:02:09.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.355+0000 7fb9cd91e740 10 client.0 _ll_register_callbacks cb 0x7fb9bc00fd80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:02:09.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.355+0000 7fb9c9ae2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556d355f1370 0x556d355eb9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:09.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.355+0000 7fb9c9ae2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556d355f1370 0x556d355eb9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38450/0 (socket says 172.21.15.5:38450) 2024-09-12T03:02:09.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.355+0000 7fb9c9ae2640 1 -- 172.21.15.5:0/3721071249 learned_addr learned my addr 172.21.15.5:0/3721071249 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:02:09.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.355+0000 7fb9c92e1640 1 --2- 172.21.15.5:0/3721071249 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556d355e65d0 0x556d355eaf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:09.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.356+0000 7fb9c9ae2640 1 -- 172.21.15.5:0/3721071249 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556d355e65d0 msgr2=0x556d355eaf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:09.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.356+0000 7fb9c8ae0640 1 --2- 172.21.15.5:0/3721071249 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556d355e7b40 0x556d355eb490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:09.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.356+0000 7fb9c9ae2640 1 --2- 172.21.15.5:0/3721071249 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556d355e65d0 0x556d355eaf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:09.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.356+0000 7fb9c9ae2640 1 -- 172.21.15.5:0/3721071249 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556d355e7b40 msgr2=0x556d355eb490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:09.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.356+0000 7fb9c9ae2640 1 --2- 172.21.15.5:0/3721071249 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556d355e7b40 0x556d355eb490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:09.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.356+0000 7fb9c9ae2640 1 -- 172.21.15.5:0/3721071249 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d3566acd0 con 0x556d355f1370 2024-09-12T03:02:09.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.356+0000 7fb9c92e1640 1 --2- 172.21.15.5:0/3721071249 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556d355e65d0 0x556d355eaf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:02:09.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.356+0000 7fb9c8ae0640 1 --2- 172.21.15.5:0/3721071249 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556d355e7b40 0x556d355eb490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:02:09.361 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.356+0000 7fb9c9ae2640 1 --2- 172.21.15.5:0/3721071249 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556d355f1370 0x556d355eb9d0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc00fc90 tx=0x7fb9bc0050c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69c3a485afbf93e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:09.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.356+0000 7fb9acff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T03:02:09.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.356+0000 7fb9acff9640 1 -- 172.21.15.5:0/3721071249 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9bc039070 con 0x556d355f1370 2024-09-12T03:02:09.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.356+0000 7fb9acff9640 1 -- 172.21.15.5:0/3721071249 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fb9bc021c00 con 0x556d355f1370 2024-09-12T03:02:09.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.356+0000 7fb9cd91e740 1 -- 172.21.15.5:0/3721071249 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9bc009ce0 con 0x556d355f1370 2024-09-12T03:02:09.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.356+0000 7fb9acff9640 1 -- 172.21.15.5:0/3721071249 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb9bc003b80 con 0x556d355f1370 2024-09-12T03:02:09.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.356+0000 7fb9cd91e740 10 client.16311 fetch_fsmap learned FSMap version 30 2024-09-12T03:02:09.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.356+0000 7fb9cd91e740 1 -- 172.21.15.5:0/3721071249 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556d3566bf10 con 0x556d355f1370 2024-09-12T03:02:09.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.356+0000 7fb9acff9640 1 -- 172.21.15.5:0/3721071249 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9bc009ce0 con 0x556d355f1370 2024-09-12T03:02:09.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.357+0000 7fb9acff9640 1 -- 172.21.15.5:0/3721071249 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7fb9bc045030 con 0x556d355f1370 2024-09-12T03:02:09.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.357+0000 7fb9cd91e740 10 client.16311 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:02:09.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.357+0000 7fb9cd91e740 10 client.16311 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:02:09.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.357+0000 7fb9cd91e740 20 client.16311 populate_metadata read hostname 'smithi005' 2024-09-12T03:02:09.362 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.357+0000 7fb9cd91e740 1 --2- 172.21.15.5:0/3721071249 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x556d3566c5c0 0x556d3566e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:09.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.357+0000 7fb9cd91e740 4 client.16311 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:02:09.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.357+0000 7fb9cd91e740 1 -- 172.21.15.5:0/3721071249 --> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556d35553d70 con 0x556d3566c5c0 2024-09-12T03:02:09.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.357+0000 7fb9c92e1640 1 --2- 172.21.15.5:0/3721071249 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x556d3566c5c0 0x556d3566e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:09.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.357+0000 7fb9c92e1640 1 --2- 172.21.15.5:0/3721071249 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x556d3566c5c0 0x556d3566e9e0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51c440a80fd20308 server_cookie=5cd9424883b9a84b in_seq=0 out_seq=0 2024-09-12T03:02:09.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.357+0000 7fb9acff9640 10 client.16311 ms_handle_connect on v2:172.21.15.148:6838/2317600000 2024-09-12T03:02:09.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.358+0000 7fb9acff9640 1 -- 172.21.15.5:0/3721071249 <== mds.0 v2:172.21.15.148:6838/2317600000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556d35553d70 con 0x556d3566c5c0 2024-09-12T03:02:09.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.358+0000 7fb9acff9640 10 client.16311 handle_command_reply: tid=0 2024-09-12T03:02:09.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.358+0000 7fb9cd91e740 1 client.16311 shutdown 2024-09-12T03:02:09.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.358+0000 7fb9cd91e740 1 -- 172.21.15.5:0/3721071249 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556d355f1370 msgr2=0x556d355eb9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:09.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.358+0000 7fb9cd91e740 1 --2- 172.21.15.5:0/3721071249 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556d355f1370 0x556d355eb9d0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc00fc90 tx=0x7fb9bc0050c0 comp rx=0 tx=0).stop 2024-09-12T03:02:09.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.358+0000 7fb9cd91e740 1 -- 172.21.15.5:0/3721071249 shutdown_connections 2024-09-12T03:02:09.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.358+0000 7fb9cd91e740 1 --2- 172.21.15.5:0/3721071249 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x556d3566c5c0 0x556d3566e9e0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:09.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.358+0000 7fb9cd91e740 1 --2- 172.21.15.5:0/3721071249 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556d355f1370 0x556d355eb9d0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:09.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.358+0000 7fb9cd91e740 1 --2- 172.21.15.5:0/3721071249 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x556d355e7b40 0x556d355eb490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:09.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.358+0000 7fb9cd91e740 1 --2- 172.21.15.5:0/3721071249 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x556d355e65d0 0x556d355eaf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:09.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.358+0000 7fb9cd91e740 1 -- 172.21.15.5:0/3721071249 >> 172.21.15.5:0/3721071249 conn(0x556d355af310 msgr2=0x556d355fc300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:09.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.358+0000 7fb9acff9640 0 client.16311 ms_handle_reset on v2:172.21.15.148:6838/2317600000 2024-09-12T03:02:09.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.358+0000 7fb9cd91e740 1 -- 172.21.15.5:0/3721071249 shutdown_connections 2024-09-12T03:02:09.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.358+0000 7fb9cd91e740 1 -- 172.21.15.5:0/3721071249 wait complete. 2024-09-12T03:02:09.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.358+0000 7fb9cd91e740 20 client.16311 trim_cache size 0 max 16384 2024-09-12T03:02:09.364 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-12T03:02:09.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.359+0000 7fb9cb545640 1 -- 172.21.15.5:0/440017542 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fb9b00a2c00 msgr2=0x7fb9b00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:09.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.359+0000 7fb9cb545640 1 --2- 172.21.15.5:0/440017542 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fb9b00a2c00 0x7fb9b00a50c0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fb9c4076600 tx=0x7fb9bc009500 comp rx=0 tx=0).stop 2024-09-12T03:02:09.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.359+0000 7fb9cb545640 1 -- 172.21.15.5:0/440017542 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9c4074eb0 msgr2=0x7fb9c41a4c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:09.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.359+0000 7fb9cb545640 1 --2- 172.21.15.5:0/440017542 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9c4074eb0 0x7fb9c41a4c70 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fb9b4006010 tx=0x7fb9b4004040 comp rx=0 tx=0).stop 2024-09-12T03:02:09.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.359+0000 7fb9cb545640 1 -- 172.21.15.5:0/440017542 shutdown_connections 2024-09-12T03:02:09.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.359+0000 7fb9cb545640 1 --2- 172.21.15.5:0/440017542 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fb9b00a2c00 0x7fb9b00a50c0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:09.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.359+0000 7fb9cb545640 1 --2- 172.21.15.5:0/440017542 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9c4074eb0 0x7fb9c41a4c70 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:09.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.359+0000 7fb9cb545640 1 --2- 172.21.15.5:0/440017542 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb9c4112b90 0x7fb9c419e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:09.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.359+0000 7fb9cb545640 1 --2- 172.21.15.5:0/440017542 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb9c4074590 0x7fb9c41a4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:09.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.359+0000 7fb9cb545640 1 -- 172.21.15.5:0/440017542 >> 172.21.15.5:0/440017542 conn(0x7fb9c40fd8c0 msgr2=0x7fb9c40ff2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:09.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.359+0000 7fb9cb545640 1 -- 172.21.15.5:0/440017542 shutdown_connections 2024-09-12T03:02:09.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:09.359+0000 7fb9cb545640 1 -- 172.21.15.5:0/440017542 wait complete. 2024-09-12T03:02:09.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:02:09 smithi005 ceph-mon[31197]: pgmap v452: 97 pgs: 97 active+clean; 114 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 2.8 MiB/s wr, 526 op/s 2024-09-12T03:02:09.758 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:02:09 smithi146 ceph-mon[35011]: pgmap v452: 97 pgs: 97 active+clean; 114 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 2.8 MiB/s wr, 526 op/s 2024-09-12T03:02:09.915 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:02:09 smithi148 ceph-mon[35090]: pgmap v452: 97 pgs: 97 active+clean; 114 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 2.8 MiB/s wr, 526 op/s 2024-09-12T03:02:11.282 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T03:02:11.283 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T03:02:11.557 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:02:11.678 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:02:11 smithi005 ceph-mon[31197]: pgmap v453: 97 pgs: 97 active+clean; 114 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 2.0 MiB/s wr, 354 op/s 2024-09-12T03:02:11.915 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:02:11 smithi148 ceph-mon[35090]: pgmap v453: 97 pgs: 97 active+clean; 114 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 2.0 MiB/s wr, 354 op/s 2024-09-12T03:02:11.956 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:02:11 smithi146 ceph-mon[35011]: pgmap v453: 97 pgs: 97 active+clean; 114 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 2.0 MiB/s wr, 354 op/s 2024-09-12T03:02:12.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.038+0000 7f2d37f2a640 1 -- 172.21.15.5:0/1370548217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d30074620 msgr2=0x7f2d30074a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:12.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.038+0000 7f2d37f2a640 1 --2- 172.21.15.5:0/1370548217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d30074620 0x7f2d30074a00 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f2d2c009900 tx=0x7f2d2c02a480 comp rx=0 tx=0).stop 2024-09-12T03:02:12.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.038+0000 7f2d37f2a640 1 -- 172.21.15.5:0/1370548217 shutdown_connections 2024-09-12T03:02:12.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.038+0000 7f2d37f2a640 1 --2- 172.21.15.5:0/1370548217 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2d3010e3c0 0x7f2d30110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:12.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.038+0000 7f2d37f2a640 1 --2- 172.21.15.5:0/1370548217 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2d30074f40 0x7f2d3010de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:12.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.038+0000 7f2d37f2a640 1 --2- 172.21.15.5:0/1370548217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d30074620 0x7f2d30074a00 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:12.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.038+0000 7f2d37f2a640 1 -- 172.21.15.5:0/1370548217 >> 172.21.15.5:0/1370548217 conn(0x7f2d300fd8c0 msgr2=0x7f2d300ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:12.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.038+0000 7f2d37f2a640 1 -- 172.21.15.5:0/1370548217 shutdown_connections 2024-09-12T03:02:12.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.039+0000 7f2d37f2a640 1 -- 172.21.15.5:0/1370548217 wait complete. 2024-09-12T03:02:12.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.039+0000 7f2d37f2a640 1 Processor -- start 2024-09-12T03:02:12.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.039+0000 7f2d37f2a640 1 -- start start 2024-09-12T03:02:12.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.039+0000 7f2d37f2a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2d30074620 0x7f2d3019a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:12.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.039+0000 7f2d37f2a640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2d30074f40 0x7f2d3019ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:12.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.039+0000 7f2d37f2a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d3010e3c0 0x7f2d3019ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:12.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.039+0000 7f2d37f2a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2d301141c0 con 0x7f2d3010e3c0 2024-09-12T03:02:12.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.039+0000 7f2d37f2a640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f2d30114040 con 0x7f2d30074f40 2024-09-12T03:02:12.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.039+0000 7f2d37f2a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f2d30114340 con 0x7f2d30074620 2024-09-12T03:02:12.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.040+0000 7f2d364c7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d3010e3c0 0x7f2d3019ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:12.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.040+0000 7f2d364c7640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d3010e3c0 0x7f2d3019ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38464/0 (socket says 172.21.15.5:38464) 2024-09-12T03:02:12.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.040+0000 7f2d364c7640 1 -- 172.21.15.5:0/573902394 learned_addr learned my addr 172.21.15.5:0/573902394 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:02:12.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.040+0000 7f2d354c5640 1 --2- 172.21.15.5:0/573902394 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2d30074f40 0x7f2d3019ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:12.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.040+0000 7f2d35cc6640 1 --2- 172.21.15.5:0/573902394 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2d30074620 0x7f2d3019a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:12.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.040+0000 7f2d364c7640 1 -- 172.21.15.5:0/573902394 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2d30074620 msgr2=0x7f2d3019a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:12.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.040+0000 7f2d364c7640 1 --2- 172.21.15.5:0/573902394 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2d30074620 0x7f2d3019a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:12.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.040+0000 7f2d364c7640 1 -- 172.21.15.5:0/573902394 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2d30074f40 msgr2=0x7f2d3019ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:12.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.040+0000 7f2d364c7640 1 --2- 172.21.15.5:0/573902394 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2d30074f40 0x7f2d3019ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:12.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.040+0000 7f2d364c7640 1 -- 172.21.15.5:0/573902394 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d3019f670 con 0x7f2d3010e3c0 2024-09-12T03:02:12.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.040+0000 7f2d35cc6640 1 --2- 172.21.15.5:0/573902394 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2d30074620 0x7f2d3019a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:02:12.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.040+0000 7f2d364c7640 1 --2- 172.21.15.5:0/573902394 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d3010e3c0 0x7f2d3019ef90 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f2d2800b970 tx=0x7f2d2800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:12.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.040+0000 7f2d26ffd640 1 -- 172.21.15.5:0/573902394 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d28019070 con 0x7f2d3010e3c0 2024-09-12T03:02:12.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.040+0000 7f2d26ffd640 1 -- 172.21.15.5:0/573902394 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d2800cb40 con 0x7f2d3010e3c0 2024-09-12T03:02:12.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.040+0000 7f2d37f2a640 1 -- 172.21.15.5:0/573902394 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d3019f960 con 0x7f2d3010e3c0 2024-09-12T03:02:12.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.040+0000 7f2d26ffd640 1 -- 172.21.15.5:0/573902394 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d28007650 con 0x7f2d3010e3c0 2024-09-12T03:02:12.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.040+0000 7f2d37f2a640 1 -- 172.21.15.5:0/573902394 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d301a7240 con 0x7f2d3010e3c0 2024-09-12T03:02:12.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.043+0000 7f2d26ffd640 1 -- 172.21.15.5:0/573902394 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f2d2801c070 con 0x7f2d3010e3c0 2024-09-12T03:02:12.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.043+0000 7f2d3a303740 1 Processor -- start 2024-09-12T03:02:12.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.043+0000 7f2d3a303740 1 -- start start 2024-09-12T03:02:12.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.043+0000 7f2d3a303740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558196deeb60 0x558196deef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:12.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.043+0000 7f2d3a303740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558196def480 0x558196df9800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:12.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.043+0000 7f2d3a303740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558196df9d40 0x558196dfc130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:12.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.043+0000 7f2d3a303740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558196d54280 con 0x558196def480 2024-09-12T03:02:12.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.043+0000 7f2d3a303740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x558196cfbb10 con 0x558196deeb60 2024-09-12T03:02:12.046 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.043+0000 7f2d3a303740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x558196d50240 con 0x558196df9d40 2024-09-12T03:02:12.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.044+0000 7f2d35cc6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558196def480 0x558196df9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:12.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.044+0000 7f2d35cc6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558196def480 0x558196df9800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38472/0 (socket says 172.21.15.5:38472) 2024-09-12T03:02:12.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.044+0000 7f2d35cc6640 1 -- 172.21.15.5:0/2342811989 learned_addr learned my addr 172.21.15.5:0/2342811989 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:02:12.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.044+0000 7f2d354c5640 1 --2- 172.21.15.5:0/2342811989 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558196deeb60 0x558196deef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:12.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.044+0000 7f2d354c5640 1 --2- 172.21.15.5:0/2342811989 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558196df9d40 0x558196dfc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:12.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.044+0000 7f2d26ffd640 1 --2- 172.21.15.5:0/573902394 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f2d1c0a2b80 0x7f2d1c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:12.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.044+0000 7f2d35cc6640 1 -- 172.21.15.5:0/2342811989 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558196df9d40 msgr2=0x558196dfc130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:12.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.044+0000 7f2d35cc6640 1 --2- 172.21.15.5:0/2342811989 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558196df9d40 0x558196dfc130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:12.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.044+0000 7f2d35cc6640 1 -- 172.21.15.5:0/2342811989 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558196deeb60 msgr2=0x558196deef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:12.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.044+0000 7f2d35cc6640 1 --2- 172.21.15.5:0/2342811989 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558196deeb60 0x558196deef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:12.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.044+0000 7f2d35cc6640 1 -- 172.21.15.5:0/2342811989 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558196d081f0 con 0x558196def480 2024-09-12T03:02:12.047 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.044+0000 7f2d354c5640 1 --2- 172.21.15.5:0/2342811989 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558196df9d40 0x558196dfc130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:02:12.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.044+0000 7f2d26ffd640 1 -- 172.21.15.5:0/573902394 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f2d280ce630 con 0x7f2d3010e3c0 2024-09-12T03:02:12.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.044+0000 7f2d364c7640 1 --2- 172.21.15.5:0/573902394 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f2d1c0a2b80 0x7f2d1c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:12.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.044+0000 7f2d35cc6640 1 --2- 172.21.15.5:0/2342811989 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558196def480 0x558196df9800 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f2d2c033000 tx=0x7f2d2c032d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=705a69c0be41cea2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:12.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.044+0000 7f2d1bfff640 1 -- 172.21.15.5:0/2342811989 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d2c03f070 con 0x558196def480 2024-09-12T03:02:12.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.045+0000 7f2d1bfff640 1 -- 172.21.15.5:0/2342811989 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d2c0317b0 con 0x558196def480 2024-09-12T03:02:12.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.045+0000 7f2d364c7640 1 --2- 172.21.15.5:0/573902394 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f2d1c0a2b80 0x7f2d1c0a5040 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f2d280973b0 tx=0x7f2d280af040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:12.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.045+0000 7f2d1bfff640 1 -- 172.21.15.5:0/2342811989 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d2c035070 con 0x558196def480 2024-09-12T03:02:12.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.045+0000 7f2d3a303740 1 -- 172.21.15.5:0/2342811989 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558196def480 msgr2=0x558196df9800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:12.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.045+0000 7f2d3a303740 1 --2- 172.21.15.5:0/2342811989 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558196def480 0x558196df9800 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f2d2c033000 tx=0x7f2d2c032d60 comp rx=0 tx=0).stop 2024-09-12T03:02:12.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.045+0000 7f2d3a303740 1 -- 172.21.15.5:0/2342811989 shutdown_connections 2024-09-12T03:02:12.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.045+0000 7f2d3a303740 1 --2- 172.21.15.5:0/2342811989 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558196def480 0x558196df9800 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:12.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.045+0000 7f2d3a303740 1 --2- 172.21.15.5:0/2342811989 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558196df9d40 0x558196dfc130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:12.048 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.045+0000 7f2d3a303740 1 --2- 172.21.15.5:0/2342811989 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558196deeb60 0x558196deef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:12.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.045+0000 7f2d3a303740 1 -- 172.21.15.5:0/2342811989 >> 172.21.15.5:0/2342811989 conn(0x558196deb0f0 msgr2=0x558196deb4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:12.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.046+0000 7f2d3a303740 1 -- 172.21.15.5:0/2342811989 shutdown_connections 2024-09-12T03:02:12.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.046+0000 7f2d3a303740 1 -- 172.21.15.5:0/2342811989 wait complete. 2024-09-12T03:02:12.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.046+0000 7f2d3a303740 1 -- start start 2024-09-12T03:02:12.049 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.046+0000 7f2d3a303740 1 Processor -- start 2024-09-12T03:02:12.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.046+0000 7f2d3a303740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558196deeb60 0x558196df6c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:12.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.046+0000 7f2d3a303740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558196def480 0x558196df7140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:12.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.046+0000 7f2d3a303740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558196df9d40 0x558196df7680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:12.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.046+0000 7f2d3a303740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558196dfc910 con 0x558196def480 2024-09-12T03:02:12.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.046+0000 7f2d3a303740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x558196d0ef10 con 0x558196deeb60 2024-09-12T03:02:12.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.046+0000 7f2d3a303740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x558196dfca90 con 0x558196df9d40 2024-09-12T03:02:12.052 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.046+0000 7f2d3a303740 10 client.0 _ll_register_callbacks cb 0x558196cd7a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:02:12.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.047+0000 7f2d35cc6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558196def480 0x558196df7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:12.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.047+0000 7f2d35cc6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558196def480 0x558196df7140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38486/0 (socket says 172.21.15.5:38486) 2024-09-12T03:02:12.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.047+0000 7f2d35cc6640 1 -- 172.21.15.5:0/1028801651 learned_addr learned my addr 172.21.15.5:0/1028801651 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:02:12.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.047+0000 7f2d354c5640 1 --2- 172.21.15.5:0/1028801651 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558196deeb60 0x558196df6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:12.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.047+0000 7f2d354c5640 1 --2- 172.21.15.5:0/1028801651 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558196df9d40 0x558196df7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:12.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.047+0000 7f2d35cc6640 1 -- 172.21.15.5:0/1028801651 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558196df9d40 msgr2=0x558196df7680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:12.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.047+0000 7f2d35cc6640 1 --2- 172.21.15.5:0/1028801651 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558196df9d40 0x558196df7680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:12.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.047+0000 7f2d35cc6640 1 -- 172.21.15.5:0/1028801651 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558196deeb60 msgr2=0x558196df6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:12.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.047+0000 7f2d35cc6640 1 --2- 172.21.15.5:0/1028801651 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558196deeb60 0x558196df6c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:12.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.047+0000 7f2d35cc6640 1 -- 172.21.15.5:0/1028801651 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558196ac56a0 con 0x558196def480 2024-09-12T03:02:12.053 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.047+0000 7f2d354c5640 1 --2- 172.21.15.5:0/1028801651 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558196df9d40 0x558196df7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:02:12.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.047+0000 7f2d35cc6640 1 --2- 172.21.15.5:0/1028801651 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558196def480 0x558196df7140 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f2d2c009900 tx=0x7f2d2c0052b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9121940ba2790e3c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:12.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.047+0000 7f2d197fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T03:02:12.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.047+0000 7f2d197fa640 1 -- 172.21.15.5:0/1028801651 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d2c04a020 con 0x558196def480 2024-09-12T03:02:12.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.047+0000 7f2d197fa640 1 -- 172.21.15.5:0/1028801651 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f2d2c0037f0 con 0x558196def480 2024-09-12T03:02:12.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.047+0000 7f2d3a303740 1 -- 172.21.15.5:0/1028801651 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d2c0317b0 con 0x558196def480 2024-09-12T03:02:12.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.047+0000 7f2d197fa640 1 -- 172.21.15.5:0/1028801651 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2d2c03f070 con 0x558196def480 2024-09-12T03:02:12.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.048+0000 7f2d3a303740 10 client.16323 fetch_fsmap learned FSMap version 30 2024-09-12T03:02:12.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.048+0000 7f2d3a303740 1 -- 172.21.15.5:0/1028801651 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558196e084f0 con 0x558196def480 2024-09-12T03:02:12.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.048+0000 7f2d197fa640 1 -- 172.21.15.5:0/1028801651 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d2c0317b0 con 0x558196def480 2024-09-12T03:02:12.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.048+0000 7f2d197fa640 1 -- 172.21.15.5:0/1028801651 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f2d2c004da0 con 0x558196def480 2024-09-12T03:02:12.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.048+0000 7f2d3a303740 10 client.16323 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:02:12.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.048+0000 7f2d3a303740 10 client.16323 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:02:12.054 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.048+0000 7f2d3a303740 20 client.16323 populate_metadata read hostname 'smithi005' 2024-09-12T03:02:12.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.048+0000 7f2d3a303740 1 --2- 172.21.15.5:0/1028801651 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x558196e09840 0x558196e0bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:12.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.048+0000 7f2d3a303740 4 client.16323 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:02:12.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.048+0000 7f2d3a303740 1 -- 172.21.15.5:0/1028801651 --> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558196d617f0 con 0x558196e09840 2024-09-12T03:02:12.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.048+0000 7f2d354c5640 1 --2- 172.21.15.5:0/1028801651 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x558196e09840 0x558196e0bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:12.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.049+0000 7f2d197fa640 10 client.16323 ms_handle_connect on v2:172.21.15.148:6838/2317600000 2024-09-12T03:02:12.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.049+0000 7f2d354c5640 1 --2- 172.21.15.5:0/1028801651 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x558196e09840 0x558196e0bc30 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7b2a138622643a2 server_cookie=61cecdc91919529 in_seq=0 out_seq=0 2024-09-12T03:02:12.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.049+0000 7f2d197fa640 1 -- 172.21.15.5:0/1028801651 <== mds.0 v2:172.21.15.148:6838/2317600000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558196d617f0 con 0x558196e09840 2024-09-12T03:02:12.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.049+0000 7f2d197fa640 10 client.16323 handle_command_reply: tid=0 2024-09-12T03:02:12.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.049+0000 7f2d3a303740 1 client.16323 shutdown 2024-09-12T03:02:12.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.050+0000 7f2d3a303740 1 -- 172.21.15.5:0/1028801651 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558196def480 msgr2=0x558196df7140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:12.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.050+0000 7f2d3a303740 1 --2- 172.21.15.5:0/1028801651 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558196def480 0x558196df7140 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f2d2c009900 tx=0x7f2d2c0052b0 comp rx=0 tx=0).stop 2024-09-12T03:02:12.055 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.050+0000 7f2d3a303740 1 -- 172.21.15.5:0/1028801651 shutdown_connections 2024-09-12T03:02:12.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.050+0000 7f2d3a303740 1 --2- 172.21.15.5:0/1028801651 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x558196e09840 0x558196e0bc30 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:12.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.050+0000 7f2d3a303740 1 --2- 172.21.15.5:0/1028801651 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558196def480 0x558196df7140 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:12.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.050+0000 7f2d3a303740 1 --2- 172.21.15.5:0/1028801651 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558196df9d40 0x558196df7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:12.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.050+0000 7f2d3a303740 1 --2- 172.21.15.5:0/1028801651 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558196deeb60 0x558196df6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:12.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.050+0000 7f2d197fa640 0 client.16323 ms_handle_reset on v2:172.21.15.148:6838/2317600000 2024-09-12T03:02:12.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.050+0000 7f2d3a303740 1 -- 172.21.15.5:0/1028801651 >> 172.21.15.5:0/1028801651 conn(0x558196deb0f0 msgr2=0x558196dea270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:12.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.050+0000 7f2d3a303740 1 -- 172.21.15.5:0/1028801651 shutdown_connections 2024-09-12T03:02:12.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.050+0000 7f2d3a303740 1 -- 172.21.15.5:0/1028801651 wait complete. 2024-09-12T03:02:12.056 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.050+0000 7f2d3a303740 20 client.16323 trim_cache size 0 max 16384 2024-09-12T03:02:12.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.067+0000 7f2d3a303740 1 Processor -- start 2024-09-12T03:02:12.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.067+0000 7f2d3a303740 1 -- start start 2024-09-12T03:02:12.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.068+0000 7f2d3a303740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558196deeb60 0x558196df8a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:12.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.068+0000 7f2d3a303740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558196def480 0x558196e11260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:12.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.068+0000 7f2d3a303740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558196df9d40 0x558196e117a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:12.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.068+0000 7f2d3a303740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2d301af770 con 0x558196df9d40 2024-09-12T03:02:12.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.068+0000 7f2d3a303740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f2d301b12e0 con 0x558196def480 2024-09-12T03:02:12.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.068+0000 7f2d3a303740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f2d301afe40 con 0x558196deeb60 2024-09-12T03:02:12.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.068+0000 7f2d354c5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558196df9d40 0x558196e117a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:12.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.068+0000 7f2d354c5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558196df9d40 0x558196e117a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38492/0 (socket says 172.21.15.5:38492) 2024-09-12T03:02:12.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.068+0000 7f2d354c5640 1 -- 172.21.15.5:0/259150360 learned_addr learned my addr 172.21.15.5:0/259150360 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:02:12.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.068+0000 7f2d35cc6640 1 --2- 172.21.15.5:0/259150360 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558196def480 0x558196e11260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:12.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.068+0000 7f2d354c5640 1 --2- 172.21.15.5:0/259150360 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558196deeb60 0x558196df8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:12.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.068+0000 7f2d354c5640 1 -- 172.21.15.5:0/259150360 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558196deeb60 msgr2=0x558196df8a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:12.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.068+0000 7f2d354c5640 1 --2- 172.21.15.5:0/259150360 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558196deeb60 0x558196df8a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:12.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.068+0000 7f2d354c5640 1 -- 172.21.15.5:0/259150360 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558196def480 msgr2=0x558196e11260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:12.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.068+0000 7f2d354c5640 1 --2- 172.21.15.5:0/259150360 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558196def480 0x558196e11260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:12.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.068+0000 7f2d354c5640 1 -- 172.21.15.5:0/259150360 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558196df94d0 con 0x558196df9d40 2024-09-12T03:02:12.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.068+0000 7f2d35cc6640 1 --2- 172.21.15.5:0/259150360 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558196def480 0x558196e11260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:02:12.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.068+0000 7f2d354c5640 1 --2- 172.21.15.5:0/259150360 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558196df9d40 0x558196e117a0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f2d20005d00 tx=0x7f2d2000b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e30c4014e658be74 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:12.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.068+0000 7f2d1b7fe640 1 -- 172.21.15.5:0/259150360 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d20025040 con 0x558196df9d40 2024-09-12T03:02:12.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.068+0000 7f2d1b7fe640 1 -- 172.21.15.5:0/259150360 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d200044b0 con 0x558196df9d40 2024-09-12T03:02:12.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.068+0000 7f2d1b7fe640 1 -- 172.21.15.5:0/259150360 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d20003850 con 0x558196df9d40 2024-09-12T03:02:12.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.069+0000 7f2d3a303740 1 -- 172.21.15.5:0/259150360 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558196df9d40 msgr2=0x558196e117a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:12.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.069+0000 7f2d3a303740 1 --2- 172.21.15.5:0/259150360 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558196df9d40 0x558196e117a0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f2d20005d00 tx=0x7f2d2000b390 comp rx=0 tx=0).stop 2024-09-12T03:02:12.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.069+0000 7f2d3a303740 1 -- 172.21.15.5:0/259150360 shutdown_connections 2024-09-12T03:02:12.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.069+0000 7f2d3a303740 1 --2- 172.21.15.5:0/259150360 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558196df9d40 0x558196e117a0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:12.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.069+0000 7f2d3a303740 1 --2- 172.21.15.5:0/259150360 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558196def480 0x558196e11260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:12.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.069+0000 7f2d3a303740 1 --2- 172.21.15.5:0/259150360 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558196deeb60 0x558196df8a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:12.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.069+0000 7f2d3a303740 1 -- 172.21.15.5:0/259150360 >> 172.21.15.5:0/259150360 conn(0x558196deb0f0 msgr2=0x558196df8350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:12.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.069+0000 7f2d3a303740 1 -- 172.21.15.5:0/259150360 shutdown_connections 2024-09-12T03:02:12.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.069+0000 7f2d3a303740 1 -- 172.21.15.5:0/259150360 wait complete. 2024-09-12T03:02:12.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.069+0000 7f2d3a303740 1 -- start start 2024-09-12T03:02:12.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.070+0000 7f2d3a303740 1 Processor -- start 2024-09-12T03:02:12.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.070+0000 7f2d3a303740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558196deeb60 0x558196dfe0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:12.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.070+0000 7f2d3a303740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558196def480 0x558196dfe5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:12.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.070+0000 7f2d3a303740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558196e71220 0x558196e73610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:12.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.070+0000 7f2d3a303740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2d301b0be0 con 0x558196deeb60 2024-09-12T03:02:12.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.070+0000 7f2d3a303740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f2d301b0510 con 0x558196def480 2024-09-12T03:02:12.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.070+0000 7f2d3a303740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f2d301ad750 con 0x558196e71220 2024-09-12T03:02:12.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.070+0000 7f2d3a303740 10 client.0 _ll_register_callbacks cb 0x7f2d2000b7c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:02:12.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.070+0000 7f2d354c5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558196deeb60 0x558196dfe0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:12.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.070+0000 7f2d354c5640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558196deeb60 0x558196dfe0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38496/0 (socket says 172.21.15.5:38496) 2024-09-12T03:02:12.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.070+0000 7f2d354c5640 1 -- 172.21.15.5:0/375735980 learned_addr learned my addr 172.21.15.5:0/375735980 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:02:12.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.070+0000 7f2d35cc6640 1 --2- 172.21.15.5:0/375735980 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558196def480 0x558196dfe5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:12.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.070+0000 7f2d354c5640 1 -- 172.21.15.5:0/375735980 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558196e71220 msgr2=0x558196e73610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:12.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.070+0000 7f2d354c5640 1 --2- 172.21.15.5:0/375735980 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558196e71220 0x558196e73610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:12.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.070+0000 7f2d354c5640 1 -- 172.21.15.5:0/375735980 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558196def480 msgr2=0x558196dfe5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:12.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.070+0000 7f2d354c5640 1 --2- 172.21.15.5:0/375735980 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558196def480 0x558196dfe5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:12.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.070+0000 7f2d354c5640 1 -- 172.21.15.5:0/375735980 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558196e73b50 con 0x558196deeb60 2024-09-12T03:02:12.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.070+0000 7f2d35cc6640 1 --2- 172.21.15.5:0/375735980 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558196def480 0x558196dfe5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:02:12.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.071+0000 7f2d354c5640 1 --2- 172.21.15.5:0/375735980 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558196deeb60 0x558196dfe0b0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f2d20004380 tx=0x7f2d20004ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21cd0481ae9237e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:12.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.071+0000 7f2d197fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T03:02:12.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.071+0000 7f2d197fa640 1 -- 172.21.15.5:0/375735980 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d20025030 con 0x558196deeb60 2024-09-12T03:02:12.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.071+0000 7f2d197fa640 1 -- 172.21.15.5:0/375735980 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f2d20004fd0 con 0x558196deeb60 2024-09-12T03:02:12.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.071+0000 7f2d3a303740 1 -- 172.21.15.5:0/375735980 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d200044b0 con 0x558196deeb60 2024-09-12T03:02:12.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.071+0000 7f2d197fa640 1 -- 172.21.15.5:0/375735980 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2d2003e050 con 0x558196deeb60 2024-09-12T03:02:12.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.071+0000 7f2d3a303740 10 client.16329 fetch_fsmap learned FSMap version 30 2024-09-12T03:02:12.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.071+0000 7f2d3a303740 1 -- 172.21.15.5:0/375735980 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558196e74e60 con 0x558196deeb60 2024-09-12T03:02:12.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.071+0000 7f2d197fa640 1 -- 172.21.15.5:0/375735980 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d200044b0 con 0x558196deeb60 2024-09-12T03:02:12.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.071+0000 7f2d197fa640 1 -- 172.21.15.5:0/375735980 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f2d2002b460 con 0x558196deeb60 2024-09-12T03:02:12.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.071+0000 7f2d3a303740 10 client.16329 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:02:12.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.071+0000 7f2d3a303740 10 client.16329 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:02:12.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.071+0000 7f2d3a303740 20 client.16329 populate_metadata read hostname 'smithi005' 2024-09-12T03:02:12.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.071+0000 7f2d3a303740 1 --2- 172.21.15.5:0/375735980 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x558196e753b0 0x558196e777d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:12.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.072+0000 7f2d3a303740 4 client.16329 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T03:02:12.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.072+0000 7f2d3a303740 1 -- 172.21.15.5:0/375735980 --> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x558196d5bd00 con 0x558196e753b0 2024-09-12T03:02:12.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.072+0000 7f2d35cc6640 1 --2- 172.21.15.5:0/375735980 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x558196e753b0 0x558196e777d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:12.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.072+0000 7f2d35cc6640 1 --2- 172.21.15.5:0/375735980 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x558196e753b0 0x558196e777d0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=395305be8df0e2aa server_cookie=9442ba9200c507d3 in_seq=0 out_seq=0 2024-09-12T03:02:12.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.072+0000 7f2d197fa640 10 client.16329 ms_handle_connect on v2:172.21.15.148:6838/2317600000 2024-09-12T03:02:12.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.073+0000 7f2d197fa640 1 -- 172.21.15.5:0/375735980 <== mds.0 v2:172.21.15.148:6838/2317600000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x558196d5bd00 con 0x558196e753b0 2024-09-12T03:02:12.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.073+0000 7f2d197fa640 10 client.16329 handle_command_reply: tid=0 2024-09-12T03:02:12.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.073+0000 7f2d3a303740 1 client.16329 shutdown 2024-09-12T03:02:12.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.073+0000 7f2d3a303740 1 -- 172.21.15.5:0/375735980 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558196deeb60 msgr2=0x558196dfe0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:12.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.073+0000 7f2d3a303740 1 --2- 172.21.15.5:0/375735980 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558196deeb60 0x558196dfe0b0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f2d20004380 tx=0x7f2d20004ee0 comp rx=0 tx=0).stop 2024-09-12T03:02:12.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.074+0000 7f2d3a303740 1 -- 172.21.15.5:0/375735980 shutdown_connections 2024-09-12T03:02:12.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.074+0000 7f2d3a303740 1 --2- 172.21.15.5:0/375735980 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558196e71220 0x558196e73610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:12.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.074+0000 7f2d3a303740 1 --2- 172.21.15.5:0/375735980 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558196def480 0x558196dfe5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:12.078 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.074+0000 7f2d3a303740 1 --2- 172.21.15.5:0/375735980 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x558196e753b0 0x558196e777d0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:12.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.074+0000 7f2d3a303740 1 --2- 172.21.15.5:0/375735980 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558196deeb60 0x558196dfe0b0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:12.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.074+0000 7f2d3a303740 1 -- 172.21.15.5:0/375735980 >> 172.21.15.5:0/375735980 conn(0x558196deb0f0 msgr2=0x558196dea270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:12.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.074+0000 7f2d197fa640 0 client.16329 ms_handle_reset on v2:172.21.15.148:6838/2317600000 2024-09-12T03:02:12.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.074+0000 7f2d3a303740 1 -- 172.21.15.5:0/375735980 shutdown_connections 2024-09-12T03:02:12.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.074+0000 7f2d3a303740 1 -- 172.21.15.5:0/375735980 wait complete. 2024-09-12T03:02:12.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.074+0000 7f2d3a303740 20 client.16329 trim_cache size 0 max 16384 2024-09-12T03:02:12.079 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-12T03:02:12.079 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-09-12T03:02:12.079 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "02646886-4faa-43bf-b97f-66bfd7952f7b", 2024-09-12T03:02:12.079 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-09-12T03:02:12.079 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-12T03:02:12.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.074+0000 7f2d37f2a640 1 -- 172.21.15.5:0/573902394 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f2d1c0a2b80 msgr2=0x7f2d1c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:12.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.074+0000 7f2d37f2a640 1 --2- 172.21.15.5:0/573902394 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f2d1c0a2b80 0x7f2d1c0a5040 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f2d280973b0 tx=0x7f2d280af040 comp rx=0 tx=0).stop 2024-09-12T03:02:12.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.074+0000 7f2d37f2a640 1 -- 172.21.15.5:0/573902394 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d3010e3c0 msgr2=0x7f2d3019ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:12.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.074+0000 7f2d37f2a640 1 --2- 172.21.15.5:0/573902394 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d3010e3c0 0x7f2d3019ef90 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f2d2800b970 tx=0x7f2d2800be30 comp rx=0 tx=0).stop 2024-09-12T03:02:12.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.074+0000 7f2d37f2a640 1 -- 172.21.15.5:0/573902394 shutdown_connections 2024-09-12T03:02:12.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.074+0000 7f2d37f2a640 1 --2- 172.21.15.5:0/573902394 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f2d1c0a2b80 0x7f2d1c0a5040 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:12.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.074+0000 7f2d37f2a640 1 --2- 172.21.15.5:0/573902394 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d3010e3c0 0x7f2d3019ef90 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:12.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.074+0000 7f2d37f2a640 1 --2- 172.21.15.5:0/573902394 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2d30074f40 0x7f2d3019ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:12.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.074+0000 7f2d37f2a640 1 --2- 172.21.15.5:0/573902394 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2d30074620 0x7f2d3019a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:12.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.074+0000 7f2d37f2a640 1 -- 172.21.15.5:0/573902394 >> 172.21.15.5:0/573902394 conn(0x7f2d300fd8c0 msgr2=0x7f2d300fe520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:12.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.075+0000 7f2d37f2a640 1 -- 172.21.15.5:0/573902394 shutdown_connections 2024-09-12T03:02:12.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:12.075+0000 7f2d37f2a640 1 -- 172.21.15.5:0/573902394 wait complete. 2024-09-12T03:02:12.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:02:12 smithi005 ceph-mon[31197]: scrub summary: active paths [/] 2024-09-12T03:02:12.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:02:12 smithi005 ceph-mon[31197]: scrub summary: idle+waiting paths [/] 2024-09-12T03:02:12.869 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T03:02:12.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:02:12 smithi148 ceph-mon[35090]: scrub summary: active paths [/] 2024-09-12T03:02:12.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:02:12 smithi148 ceph-mon[35090]: scrub summary: idle+waiting paths [/] 2024-09-12T03:02:12.956 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:02:12 smithi146 ceph-mon[35011]: scrub summary: active paths [/] 2024-09-12T03:02:12.956 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:02:12 smithi146 ceph-mon[35011]: scrub summary: idle+waiting paths [/] 2024-09-12T03:02:13.124 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:02:13.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.646+0000 7f814c9a8640 1 -- 172.21.15.5:0/2515264089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8148106a80 msgr2=0x7f8148106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:13.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.646+0000 7f814c9a8640 1 --2- 172.21.15.5:0/2515264089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8148106a80 0x7f8148106e60 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f8138009930 tx=0x7f813802a420 comp rx=0 tx=0).stop 2024-09-12T03:02:13.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.646+0000 7f814c9a8640 1 -- 172.21.15.5:0/2515264089 shutdown_connections 2024-09-12T03:02:13.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.646+0000 7f814c9a8640 1 --2- 172.21.15.5:0/2515264089 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8148101420 0x7f814810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:13.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.646+0000 7f814c9a8640 1 --2- 172.21.15.5:0/2515264089 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8148100a80 0x7f8148100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:13.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.646+0000 7f814c9a8640 1 --2- 172.21.15.5:0/2515264089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8148106a80 0x7f8148106e60 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:13.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.646+0000 7f814c9a8640 1 -- 172.21.15.5:0/2515264089 >> 172.21.15.5:0/2515264089 conn(0x7f81480fc7a0 msgr2=0x7f81480febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:13.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.646+0000 7f814c9a8640 1 -- 172.21.15.5:0/2515264089 shutdown_connections 2024-09-12T03:02:13.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.646+0000 7f814c9a8640 1 -- 172.21.15.5:0/2515264089 wait complete. 2024-09-12T03:02:13.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.648+0000 7f814c9a8640 1 Processor -- start 2024-09-12T03:02:13.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.648+0000 7f814c9a8640 1 -- start start 2024-09-12T03:02:13.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.648+0000 7f814c9a8640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8148100a80 0x7f8148102a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:13.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.648+0000 7f814c9a8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8148101420 0x7f8148102f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:13.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.648+0000 7f814c9a8640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8148106a80 0x7f81481034b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:13.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.648+0000 7f814c9a8640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8148110e80 con 0x7f8148106a80 2024-09-12T03:02:13.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.648+0000 7f814c9a8640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f8148110d00 con 0x7f8148100a80 2024-09-12T03:02:13.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.648+0000 7f814c9a8640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8148111000 con 0x7f8148101420 2024-09-12T03:02:13.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.648+0000 7f8146d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8148106a80 0x7f81481034b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:13.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.649+0000 7f8146d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8148106a80 0x7f81481034b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39716/0 (socket says 172.21.15.5:39716) 2024-09-12T03:02:13.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.649+0000 7f8145d9b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8148101420 0x7f8148102f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:13.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.649+0000 7f8146d9d640 1 -- 172.21.15.5:0/182975230 learned_addr learned my addr 172.21.15.5:0/182975230 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:02:13.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.649+0000 7f814659c640 1 --2- 172.21.15.5:0/182975230 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8148100a80 0x7f8148102a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:13.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.649+0000 7f8146d9d640 1 -- 172.21.15.5:0/182975230 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8148101420 msgr2=0x7f8148102f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:13.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.649+0000 7f8146d9d640 1 --2- 172.21.15.5:0/182975230 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8148101420 0x7f8148102f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:13.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.649+0000 7f8146d9d640 1 -- 172.21.15.5:0/182975230 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8148100a80 msgr2=0x7f8148102a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:13.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.649+0000 7f8146d9d640 1 --2- 172.21.15.5:0/182975230 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8148100a80 0x7f8148102a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:13.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.649+0000 7f8146d9d640 1 -- 172.21.15.5:0/182975230 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81481a68b0 con 0x7f8148106a80 2024-09-12T03:02:13.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.649+0000 7f814659c640 1 --2- 172.21.15.5:0/182975230 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8148100a80 0x7f8148102a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:02:13.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.649+0000 7f8146d9d640 1 --2- 172.21.15.5:0/182975230 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8148106a80 0x7f81481034b0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f814000cce0 tx=0x7f814000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:13.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.649+0000 7f81377fe640 1 -- 172.21.15.5:0/182975230 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f814000f450 con 0x7f8148106a80 2024-09-12T03:02:13.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.649+0000 7f81377fe640 1 -- 172.21.15.5:0/182975230 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f814000fad0 con 0x7f8148106a80 2024-09-12T03:02:13.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.649+0000 7f814c9a8640 1 -- 172.21.15.5:0/182975230 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81481a6ba0 con 0x7f8148106a80 2024-09-12T03:02:13.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.649+0000 7f81377fe640 1 -- 172.21.15.5:0/182975230 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8140018c10 con 0x7f8148106a80 2024-09-12T03:02:13.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.649+0000 7f814c9a8640 1 -- 172.21.15.5:0/182975230 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81481a7090 con 0x7f8148106a80 2024-09-12T03:02:13.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.652+0000 7f81377fe640 1 -- 172.21.15.5:0/182975230 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f814000f5f0 con 0x7f8148106a80 2024-09-12T03:02:13.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.652+0000 7f814ed81740 1 Processor -- start 2024-09-12T03:02:13.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.652+0000 7f814ed81740 1 -- start start 2024-09-12T03:02:13.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.652+0000 7f814ed81740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560af254b4a0 0x560af254b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:13.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.652+0000 7f814ed81740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560af254bdc0 0x560af2556140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:13.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.652+0000 7f814ed81740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560af2556680 0x560af2558a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:13.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.652+0000 7f814ed81740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560af24b12e0 con 0x560af254b4a0 2024-09-12T03:02:13.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.652+0000 7f814ed81740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x560af2458ce0 con 0x560af2556680 2024-09-12T03:02:13.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.652+0000 7f814ed81740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x560af24ad2a0 con 0x560af254bdc0 2024-09-12T03:02:13.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.652+0000 7f81377fe640 1 --2- 172.21.15.5:0/182975230 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f81300a2b80 0x7f81300a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:13.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.653+0000 7f8145d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560af254b4a0 0x560af254b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:13.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.653+0000 7f8145d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560af254b4a0 0x560af254b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39724/0 (socket says 172.21.15.5:39724) 2024-09-12T03:02:13.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.653+0000 7f8145d9b640 1 -- 172.21.15.5:0/2105679706 learned_addr learned my addr 172.21.15.5:0/2105679706 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:02:13.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.653+0000 7f814659c640 1 --2- 172.21.15.5:0/2105679706 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560af254bdc0 0x560af2556140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:13.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.653+0000 7f8146d9d640 1 --2- 172.21.15.5:0/182975230 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f81300a2b80 0x7f81300a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:13.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.653+0000 7f81377fe640 1 -- 172.21.15.5:0/182975230 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f81400cf6b0 con 0x7f8148106a80 2024-09-12T03:02:13.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.653+0000 7f8145d9b640 1 -- 172.21.15.5:0/2105679706 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560af254bdc0 msgr2=0x560af2556140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:13.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.653+0000 7f8145d9b640 1 --2- 172.21.15.5:0/2105679706 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560af254bdc0 0x560af2556140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:13.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.653+0000 7f8145d9b640 1 -- 172.21.15.5:0/2105679706 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560af2556680 msgr2=0x560af2558a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:13.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.653+0000 7f8145d9b640 1 --2- 172.21.15.5:0/2105679706 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560af2556680 0x560af2558a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:13.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.653+0000 7f8145d9b640 1 -- 172.21.15.5:0/2105679706 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560af243d440 con 0x560af254b4a0 2024-09-12T03:02:13.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.653+0000 7f8145d9b640 1 --2- 172.21.15.5:0/2105679706 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560af254b4a0 0x560af254b880 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f813c00ac70 tx=0x7f813c0117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22c61eccef083394 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:13.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.653+0000 7f8146d9d640 1 --2- 172.21.15.5:0/182975230 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f81300a2b80 0x7f81300a5040 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f814001f000 tx=0x7f8140066af0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:13.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.653+0000 7f8134ff9640 1 -- 172.21.15.5:0/2105679706 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f813c005020 con 0x560af254b4a0 2024-09-12T03:02:13.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.653+0000 7f8134ff9640 1 -- 172.21.15.5:0/2105679706 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f813c0047a0 con 0x560af254b4a0 2024-09-12T03:02:13.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.653+0000 7f8134ff9640 1 -- 172.21.15.5:0/2105679706 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f813c0126b0 con 0x560af254b4a0 2024-09-12T03:02:13.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.654+0000 7f814ed81740 1 -- 172.21.15.5:0/2105679706 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560af254b4a0 msgr2=0x560af254b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:13.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.654+0000 7f814ed81740 1 --2- 172.21.15.5:0/2105679706 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560af254b4a0 0x560af254b880 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f813c00ac70 tx=0x7f813c0117b0 comp rx=0 tx=0).stop 2024-09-12T03:02:13.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.654+0000 7f814ed81740 1 -- 172.21.15.5:0/2105679706 shutdown_connections 2024-09-12T03:02:13.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.654+0000 7f814ed81740 1 --2- 172.21.15.5:0/2105679706 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560af2556680 0x560af2558a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:13.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.654+0000 7f814ed81740 1 --2- 172.21.15.5:0/2105679706 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560af254bdc0 0x560af2556140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:13.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.654+0000 7f814ed81740 1 --2- 172.21.15.5:0/2105679706 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560af254b4a0 0x560af254b880 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:13.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.654+0000 7f814ed81740 1 -- 172.21.15.5:0/2105679706 >> 172.21.15.5:0/2105679706 conn(0x560af2547a30 msgr2=0x560af2547e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:13.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.654+0000 7f814ed81740 1 -- 172.21.15.5:0/2105679706 shutdown_connections 2024-09-12T03:02:13.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.654+0000 7f814ed81740 1 -- 172.21.15.5:0/2105679706 wait complete. 2024-09-12T03:02:13.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.655+0000 7f814ed81740 1 -- start start 2024-09-12T03:02:13.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.655+0000 7f814ed81740 1 Processor -- start 2024-09-12T03:02:13.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.655+0000 7f814ed81740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560af254b4a0 0x560af2555fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:13.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.655+0000 7f814ed81740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560af254bdc0 0x560af2553270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:13.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.655+0000 7f814ed81740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560af2556680 0x560af25537b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:13.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.655+0000 7f814ed81740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560af2559250 con 0x560af254bdc0 2024-09-12T03:02:13.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.655+0000 7f814ed81740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x560af246c050 con 0x560af2556680 2024-09-12T03:02:13.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.655+0000 7f814ed81740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x560af25593d0 con 0x560af254b4a0 2024-09-12T03:02:13.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.655+0000 7f814ed81740 10 client.0 _ll_register_callbacks cb 0x560af24ac160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:02:13.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.655+0000 7f814659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560af254bdc0 0x560af2553270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:13.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.655+0000 7f814659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560af254bdc0 0x560af2553270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39734/0 (socket says 172.21.15.5:39734) 2024-09-12T03:02:13.660 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.655+0000 7f814659c640 1 -- 172.21.15.5:0/291240926 learned_addr learned my addr 172.21.15.5:0/291240926 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:02:13.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.655+0000 7f8145d9b640 1 --2- 172.21.15.5:0/291240926 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560af254b4a0 0x560af2555fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:13.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.655+0000 7f814659c640 1 -- 172.21.15.5:0/291240926 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560af254b4a0 msgr2=0x560af2555fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:13.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.655+0000 7f814659c640 1 --2- 172.21.15.5:0/291240926 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560af254b4a0 0x560af2555fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:13.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.656+0000 7f814659c640 1 -- 172.21.15.5:0/291240926 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560af2556680 msgr2=0x560af25537b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:13.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.656+0000 7f814659c640 1 --2- 172.21.15.5:0/291240926 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560af2556680 0x560af25537b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:13.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.656+0000 7f814659c640 1 -- 172.21.15.5:0/291240926 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560af22223c0 con 0x560af254bdc0 2024-09-12T03:02:13.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.656+0000 7f8145d9b640 1 --2- 172.21.15.5:0/291240926 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560af254b4a0 0x560af2555fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:02:13.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.656+0000 7f814659c640 1 --2- 172.21.15.5:0/291240926 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560af254bdc0 0x560af2553270 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f8138031550 tx=0x7f8138004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a32f91d88364a6a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:13.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.656+0000 7f812dffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T03:02:13.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.656+0000 7f812dffb640 1 -- 172.21.15.5:0/291240926 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8138035040 con 0x560af254bdc0 2024-09-12T03:02:13.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.656+0000 7f812dffb640 1 -- 172.21.15.5:0/291240926 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f8138005020 con 0x560af254bdc0 2024-09-12T03:02:13.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.656+0000 7f814ed81740 1 -- 172.21.15.5:0/291240926 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f813c0047a0 con 0x560af254bdc0 2024-09-12T03:02:13.661 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.656+0000 7f812dffb640 1 -- 172.21.15.5:0/291240926 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8138005220 con 0x560af254bdc0 2024-09-12T03:02:13.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.656+0000 7f814ed81740 10 client.16341 fetch_fsmap learned FSMap version 30 2024-09-12T03:02:13.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.656+0000 7f814ed81740 1 -- 172.21.15.5:0/291240926 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560af2564c00 con 0x560af254bdc0 2024-09-12T03:02:13.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.656+0000 7f812dffb640 1 -- 172.21.15.5:0/291240926 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f813c0047a0 con 0x560af254bdc0 2024-09-12T03:02:13.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.657+0000 7f812dffb640 1 -- 172.21.15.5:0/291240926 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f8138003790 con 0x560af254bdc0 2024-09-12T03:02:13.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.657+0000 7f814ed81740 10 client.16341 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:02:13.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.657+0000 7f814ed81740 10 client.16341 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:02:13.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.657+0000 7f814ed81740 20 client.16341 populate_metadata read hostname 'smithi005' 2024-09-12T03:02:13.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.657+0000 7f814ed81740 1 --2- 172.21.15.5:0/291240926 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x560af2566110 0x560af2568530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:13.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.657+0000 7f814ed81740 4 client.16341 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:02:13.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.657+0000 7f814ed81740 1 -- 172.21.15.5:0/291240926 --> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560af24be850 con 0x560af2566110 2024-09-12T03:02:13.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.657+0000 7f8145d9b640 1 --2- 172.21.15.5:0/291240926 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x560af2566110 0x560af2568530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:13.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.657+0000 7f8145d9b640 1 --2- 172.21.15.5:0/291240926 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x560af2566110 0x560af2568530 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=967ccce5dcef5264 server_cookie=62d1e642f98285fa in_seq=0 out_seq=0 2024-09-12T03:02:13.662 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.657+0000 7f812dffb640 10 client.16341 ms_handle_connect on v2:172.21.15.148:6838/2317600000 2024-09-12T03:02:13.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.658+0000 7f812dffb640 1 -- 172.21.15.5:0/291240926 <== mds.0 v2:172.21.15.148:6838/2317600000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560af24be850 con 0x560af2566110 2024-09-12T03:02:13.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.658+0000 7f812dffb640 10 client.16341 handle_command_reply: tid=0 2024-09-12T03:02:13.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.658+0000 7f814ed81740 1 client.16341 shutdown 2024-09-12T03:02:13.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.658+0000 7f814ed81740 1 -- 172.21.15.5:0/291240926 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560af254bdc0 msgr2=0x560af2553270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:13.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.658+0000 7f814ed81740 1 --2- 172.21.15.5:0/291240926 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560af254bdc0 0x560af2553270 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f8138031550 tx=0x7f8138004270 comp rx=0 tx=0).stop 2024-09-12T03:02:13.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.658+0000 7f814ed81740 1 -- 172.21.15.5:0/291240926 shutdown_connections 2024-09-12T03:02:13.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.658+0000 7f814ed81740 1 --2- 172.21.15.5:0/291240926 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x560af2566110 0x560af2568530 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:13.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.658+0000 7f814ed81740 1 --2- 172.21.15.5:0/291240926 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560af254bdc0 0x560af2553270 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:13.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.658+0000 7f814ed81740 1 --2- 172.21.15.5:0/291240926 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560af2556680 0x560af25537b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:13.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.658+0000 7f814ed81740 1 --2- 172.21.15.5:0/291240926 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560af254b4a0 0x560af2555fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:13.663 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.658+0000 7f814ed81740 1 -- 172.21.15.5:0/291240926 >> 172.21.15.5:0/291240926 conn(0x560af2547a30 msgr2=0x560af254d710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:13.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.658+0000 7f812dffb640 0 client.16341 ms_handle_reset on v2:172.21.15.148:6838/2317600000 2024-09-12T03:02:13.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.658+0000 7f814ed81740 1 -- 172.21.15.5:0/291240926 shutdown_connections 2024-09-12T03:02:13.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.658+0000 7f814ed81740 1 -- 172.21.15.5:0/291240926 wait complete. 2024-09-12T03:02:13.664 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.659+0000 7f814ed81740 20 client.16341 trim_cache size 0 max 16384 2024-09-12T03:02:13.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.676+0000 7f814ed81740 1 Processor -- start 2024-09-12T03:02:13.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.676+0000 7f814ed81740 1 -- start start 2024-09-12T03:02:13.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.676+0000 7f814ed81740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560af254b4a0 0x560af25624b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:13.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.676+0000 7f814ed81740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560af2556680 0x560af25551e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:13.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.676+0000 7f814ed81740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560af256dde0 0x560af256e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:13.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.676+0000 7f814ed81740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f81481af6c0 con 0x560af2556680 2024-09-12T03:02:13.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.676+0000 7f814ed81740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f81481b1230 con 0x560af254b4a0 2024-09-12T03:02:13.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.676+0000 7f814ed81740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f81481afd90 con 0x560af256dde0 2024-09-12T03:02:13.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.676+0000 7f814659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560af2556680 0x560af25551e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:13.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.676+0000 7f814659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560af2556680 0x560af25551e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39750/0 (socket says 172.21.15.5:39750) 2024-09-12T03:02:13.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.676+0000 7f814659c640 1 -- 172.21.15.5:0/4088251902 learned_addr learned my addr 172.21.15.5:0/4088251902 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:02:13.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.676+0000 7f8145d9b640 1 --2- 172.21.15.5:0/4088251902 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560af254b4a0 0x560af25624b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:13.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.676+0000 7f814659c640 1 -- 172.21.15.5:0/4088251902 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560af256dde0 msgr2=0x560af256e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:13.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.676+0000 7f814659c640 1 --2- 172.21.15.5:0/4088251902 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560af256dde0 0x560af256e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:13.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.676+0000 7f814659c640 1 -- 172.21.15.5:0/4088251902 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560af254b4a0 msgr2=0x560af25624b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:13.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.676+0000 7f814659c640 1 --2- 172.21.15.5:0/4088251902 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560af254b4a0 0x560af25624b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:13.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.676+0000 7f814659c640 1 -- 172.21.15.5:0/4088251902 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560af2552220 con 0x560af2556680 2024-09-12T03:02:13.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.676+0000 7f814659c640 1 --2- 172.21.15.5:0/4088251902 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560af2556680 0x560af25551e0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f8138006010 tx=0x7f8138009290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9159ef1b895c9116 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:13.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.677+0000 7f812ffff640 1 -- 172.21.15.5:0/4088251902 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8138035070 con 0x560af2556680 2024-09-12T03:02:13.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.677+0000 7f812ffff640 1 -- 172.21.15.5:0/4088251902 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81380042e0 con 0x560af2556680 2024-09-12T03:02:13.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.677+0000 7f812ffff640 1 -- 172.21.15.5:0/4088251902 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8138004c70 con 0x560af2556680 2024-09-12T03:02:13.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.677+0000 7f814ed81740 1 -- 172.21.15.5:0/4088251902 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560af2556680 msgr2=0x560af25551e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:13.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.677+0000 7f814ed81740 1 --2- 172.21.15.5:0/4088251902 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560af2556680 0x560af25551e0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f8138006010 tx=0x7f8138009290 comp rx=0 tx=0).stop 2024-09-12T03:02:13.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.677+0000 7f814ed81740 1 -- 172.21.15.5:0/4088251902 shutdown_connections 2024-09-12T03:02:13.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.677+0000 7f814ed81740 1 --2- 172.21.15.5:0/4088251902 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560af2556680 0x560af25551e0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:13.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.677+0000 7f814ed81740 1 --2- 172.21.15.5:0/4088251902 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560af256dde0 0x560af256e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:13.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.677+0000 7f814ed81740 1 --2- 172.21.15.5:0/4088251902 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560af254b4a0 0x560af25624b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:13.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.677+0000 7f814ed81740 1 -- 172.21.15.5:0/4088251902 >> 172.21.15.5:0/4088251902 conn(0x560af2547a30 msgr2=0x560af2554ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:13.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.677+0000 7f814ed81740 1 -- 172.21.15.5:0/4088251902 shutdown_connections 2024-09-12T03:02:13.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.677+0000 7f814ed81740 1 -- 172.21.15.5:0/4088251902 wait complete. 2024-09-12T03:02:13.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.678+0000 7f814ed81740 1 -- start start 2024-09-12T03:02:13.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.678+0000 7f814ed81740 1 Processor -- start 2024-09-12T03:02:13.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.678+0000 7f814ed81740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560af254b4a0 0x560af255a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:13.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.678+0000 7f814ed81740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560af2556680 0x560af255acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:13.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.678+0000 7f814ed81740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560af256dde0 0x560af25cfdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:13.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.678+0000 7f814ed81740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f81481b0b30 con 0x560af256dde0 2024-09-12T03:02:13.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.678+0000 7f814ed81740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f81481b0460 con 0x560af254b4a0 2024-09-12T03:02:13.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.678+0000 7f814ed81740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f81481ad6a0 con 0x560af2556680 2024-09-12T03:02:13.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.678+0000 7f814ed81740 10 client.0 _ll_register_callbacks cb 0x7f8138031a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:02:13.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.678+0000 7f8145d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560af256dde0 0x560af25cfdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:13.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.678+0000 7f8145d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560af256dde0 0x560af25cfdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39758/0 (socket says 172.21.15.5:39758) 2024-09-12T03:02:13.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.678+0000 7f8145d9b640 1 -- 172.21.15.5:0/721404520 learned_addr learned my addr 172.21.15.5:0/721404520 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:02:13.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.678+0000 7f8145d9b640 1 --2- 172.21.15.5:0/721404520 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560af254b4a0 0x560af255a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:13.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.678+0000 7f814659c640 1 --2- 172.21.15.5:0/721404520 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560af2556680 0x560af255acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:13.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.679+0000 7f8145d9b640 1 -- 172.21.15.5:0/721404520 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560af2556680 msgr2=0x560af255acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:13.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.679+0000 7f8145d9b640 1 --2- 172.21.15.5:0/721404520 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560af2556680 0x560af255acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:13.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.679+0000 7f8145d9b640 1 -- 172.21.15.5:0/721404520 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560af254b4a0 msgr2=0x560af255a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:13.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.679+0000 7f8145d9b640 1 --2- 172.21.15.5:0/721404520 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560af254b4a0 0x560af255a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:13.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.679+0000 7f8145d9b640 1 -- 172.21.15.5:0/721404520 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560af255b5b0 con 0x560af256dde0 2024-09-12T03:02:13.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.679+0000 7f8145d9b640 1 --2- 172.21.15.5:0/721404520 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560af256dde0 0x560af25cfdb0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f813c00aab0 tx=0x7f813c0117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d06b494a258773aa server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:13.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.679+0000 7f812dffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T03:02:13.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.679+0000 7f812dffb640 1 -- 172.21.15.5:0/721404520 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f813c027740 con 0x560af256dde0 2024-09-12T03:02:13.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.679+0000 7f812dffb640 1 -- 172.21.15.5:0/721404520 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f813c0278e0 con 0x560af256dde0 2024-09-12T03:02:13.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.679+0000 7f814ed81740 1 -- 172.21.15.5:0/721404520 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81380042e0 con 0x560af256dde0 2024-09-12T03:02:13.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.679+0000 7f812dffb640 1 -- 172.21.15.5:0/721404520 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f813c011210 con 0x560af256dde0 2024-09-12T03:02:13.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.679+0000 7f814ed81740 10 client.16347 fetch_fsmap learned FSMap version 30 2024-09-12T03:02:13.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.679+0000 7f814ed81740 1 -- 172.21.15.5:0/721404520 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560af25d1410 con 0x560af256dde0 2024-09-12T03:02:13.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.679+0000 7f812dffb640 1 -- 172.21.15.5:0/721404520 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81380042e0 con 0x560af256dde0 2024-09-12T03:02:13.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.679+0000 7f812dffb640 1 -- 172.21.15.5:0/721404520 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f813c032420 con 0x560af256dde0 2024-09-12T03:02:13.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.680+0000 7f814ed81740 10 client.16347 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:02:13.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.680+0000 7f814ed81740 10 client.16347 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:02:13.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.680+0000 7f814ed81740 20 client.16347 populate_metadata read hostname 'smithi005' 2024-09-12T03:02:13.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.680+0000 7f814ed81740 1 --2- 172.21.15.5:0/721404520 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x560af25d18b0 0x560af25d3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:13.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.680+0000 7f814ed81740 4 client.16347 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T03:02:13.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.680+0000 7f814ed81740 1 -- 172.21.15.5:0/721404520 --> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560af24b8d60 con 0x560af25d18b0 2024-09-12T03:02:13.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.680+0000 7f814659c640 1 --2- 172.21.15.5:0/721404520 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x560af25d18b0 0x560af25d3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:13.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.680+0000 7f814659c640 1 --2- 172.21.15.5:0/721404520 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x560af25d18b0 0x560af25d3ca0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ee18b755791defb server_cookie=382602252e168c3c in_seq=0 out_seq=0 2024-09-12T03:02:13.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.680+0000 7f812dffb640 10 client.16347 ms_handle_connect on v2:172.21.15.148:6838/2317600000 2024-09-12T03:02:13.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.680+0000 7f812dffb640 1 -- 172.21.15.5:0/721404520 <== mds.0 v2:172.21.15.148:6838/2317600000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x560af24b8d60 con 0x560af25d18b0 2024-09-12T03:02:13.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.681+0000 7f812dffb640 10 client.16347 handle_command_reply: tid=0 2024-09-12T03:02:13.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.681+0000 7f814ed81740 1 client.16347 shutdown 2024-09-12T03:02:13.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.681+0000 7f814ed81740 1 -- 172.21.15.5:0/721404520 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560af256dde0 msgr2=0x560af25cfdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:13.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.681+0000 7f814ed81740 1 --2- 172.21.15.5:0/721404520 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560af256dde0 0x560af25cfdb0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f813c00aab0 tx=0x7f813c0117b0 comp rx=0 tx=0).stop 2024-09-12T03:02:13.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.681+0000 7f814ed81740 1 -- 172.21.15.5:0/721404520 shutdown_connections 2024-09-12T03:02:13.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.681+0000 7f814ed81740 1 --2- 172.21.15.5:0/721404520 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x560af25d18b0 0x560af25d3ca0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:13.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.681+0000 7f814ed81740 1 --2- 172.21.15.5:0/721404520 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560af256dde0 0x560af25cfdb0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:13.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.681+0000 7f814ed81740 1 --2- 172.21.15.5:0/721404520 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560af2556680 0x560af255acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:13.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.681+0000 7f814ed81740 1 --2- 172.21.15.5:0/721404520 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560af254b4a0 0x560af255a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:13.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.681+0000 7f814ed81740 1 -- 172.21.15.5:0/721404520 >> 172.21.15.5:0/721404520 conn(0x560af2547a30 msgr2=0x560af2546350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:13.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.681+0000 7f812dffb640 0 client.16347 ms_handle_reset on v2:172.21.15.148:6838/2317600000 2024-09-12T03:02:13.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.681+0000 7f814ed81740 1 -- 172.21.15.5:0/721404520 shutdown_connections 2024-09-12T03:02:13.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.681+0000 7f814ed81740 1 -- 172.21.15.5:0/721404520 wait complete. 2024-09-12T03:02:13.687 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.681+0000 7f814ed81740 20 client.16347 trim_cache size 0 max 16384 2024-09-12T03:02:13.687 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-12T03:02:13.687 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-12T03:02:13.687 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-09-12T03:02:13.687 INFO:teuthology.orchestra.run.smithi005.stdout: "02646886-4faa-43bf-b97f-66bfd7952f7b": { 2024-09-12T03:02:13.687 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-09-12T03:02:13.687 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "02646886-4faa-43bf-b97f-66bfd7952f7b", 2024-09-12T03:02:13.687 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-09-12T03:02:13.687 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:02:13.688 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:02:13.688 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-12T03:02:13.688 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.682+0000 7f814c9a8640 1 -- 172.21.15.5:0/182975230 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f81300a2b80 msgr2=0x7f81300a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:13.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.682+0000 7f814c9a8640 1 --2- 172.21.15.5:0/182975230 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f81300a2b80 0x7f81300a5040 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f814001f000 tx=0x7f8140066af0 comp rx=0 tx=0).stop 2024-09-12T03:02:13.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.682+0000 7f814c9a8640 1 -- 172.21.15.5:0/182975230 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8148106a80 msgr2=0x7f81481034b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:13.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.682+0000 7f814c9a8640 1 --2- 172.21.15.5:0/182975230 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8148106a80 0x7f81481034b0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f814000cce0 tx=0x7f814000a620 comp rx=0 tx=0).stop 2024-09-12T03:02:13.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.682+0000 7f814c9a8640 1 -- 172.21.15.5:0/182975230 shutdown_connections 2024-09-12T03:02:13.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.682+0000 7f814c9a8640 1 --2- 172.21.15.5:0/182975230 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f81300a2b80 0x7f81300a5040 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:13.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.682+0000 7f814c9a8640 1 --2- 172.21.15.5:0/182975230 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8148106a80 0x7f81481034b0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:13.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.682+0000 7f814c9a8640 1 --2- 172.21.15.5:0/182975230 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8148101420 0x7f8148102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:13.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.682+0000 7f814c9a8640 1 --2- 172.21.15.5:0/182975230 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8148100a80 0x7f8148102a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:13.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.682+0000 7f814c9a8640 1 -- 172.21.15.5:0/182975230 >> 172.21.15.5:0/182975230 conn(0x7f81480fc7a0 msgr2=0x7f8148104c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:13.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.682+0000 7f814c9a8640 1 -- 172.21.15.5:0/182975230 shutdown_connections 2024-09-12T03:02:13.689 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:13.682+0000 7f814c9a8640 1 -- 172.21.15.5:0/182975230 wait complete. 2024-09-12T03:02:13.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:02:13 smithi005 ceph-mon[31197]: pgmap v454: 97 pgs: 97 active+clean; 114 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.4 MiB/s wr, 270 op/s 2024-09-12T03:02:13.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:02:13 smithi005 ceph-mon[31197]: scrub queued for path: / 2024-09-12T03:02:13.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:02:13 smithi005 ceph-mon[31197]: scrub summary: idle+waiting paths [/] 2024-09-12T03:02:13.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:02:13 smithi005 ceph-mon[31197]: scrub summary: active paths [/] 2024-09-12T03:02:13.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:02:13 smithi005 ceph-mon[31197]: scrub summary: active paths [/] 2024-09-12T03:02:13.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:02:13 smithi005 ceph-mon[31197]: scrub summary: idle+waiting paths [/] 2024-09-12T03:02:13.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:02:13 smithi005 ceph-mon[31197]: scrub summary: active paths [/] 2024-09-12T03:02:13.811 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:02:13 smithi005 ceph-mon[31197]: scrub summary: idle+waiting paths [/] 2024-09-12T03:02:13.811 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:02:13 smithi005 ceph-mon[31197]: scrub summary: active paths [/] 2024-09-12T03:02:13.811 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:02:13 smithi005 ceph-mon[31197]: scrub summary: idle+waiting paths [/] 2024-09-12T03:02:13.811 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:02:13 smithi005 ceph-mon[31197]: scrub summary: idle+waiting paths [/] 2024-09-12T03:02:13.915 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:02:13 smithi148 ceph-mon[35090]: pgmap v454: 97 pgs: 97 active+clean; 114 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.4 MiB/s wr, 270 op/s 2024-09-12T03:02:13.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:02:13 smithi148 ceph-mon[35090]: scrub queued for path: / 2024-09-12T03:02:13.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:02:13 smithi148 ceph-mon[35090]: scrub summary: idle+waiting paths [/] 2024-09-12T03:02:13.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:02:13 smithi148 ceph-mon[35090]: scrub summary: active paths [/] 2024-09-12T03:02:13.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:02:13 smithi148 ceph-mon[35090]: scrub summary: active paths [/] 2024-09-12T03:02:13.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:02:13 smithi148 ceph-mon[35090]: scrub summary: idle+waiting paths [/] 2024-09-12T03:02:13.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:02:13 smithi148 ceph-mon[35090]: scrub summary: active paths [/] 2024-09-12T03:02:13.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:02:13 smithi148 ceph-mon[35090]: scrub summary: idle+waiting paths [/] 2024-09-12T03:02:13.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:02:13 smithi148 ceph-mon[35090]: scrub summary: active paths [/] 2024-09-12T03:02:13.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:02:13 smithi148 ceph-mon[35090]: scrub summary: idle+waiting paths [/] 2024-09-12T03:02:13.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:02:13 smithi148 ceph-mon[35090]: scrub summary: idle+waiting paths [/] 2024-09-12T03:02:13.956 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:02:13 smithi146 ceph-mon[35011]: pgmap v454: 97 pgs: 97 active+clean; 114 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 1.4 MiB/s wr, 270 op/s 2024-09-12T03:02:13.956 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:02:13 smithi146 ceph-mon[35011]: scrub queued for path: / 2024-09-12T03:02:13.956 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:02:13 smithi146 ceph-mon[35011]: scrub summary: idle+waiting paths [/] 2024-09-12T03:02:13.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:02:13 smithi146 ceph-mon[35011]: scrub summary: active paths [/] 2024-09-12T03:02:13.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:02:13 smithi146 ceph-mon[35011]: scrub summary: active paths [/] 2024-09-12T03:02:13.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:02:13 smithi146 ceph-mon[35011]: scrub summary: idle+waiting paths [/] 2024-09-12T03:02:13.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:02:13 smithi146 ceph-mon[35011]: scrub summary: active paths [/] 2024-09-12T03:02:13.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:02:13 smithi146 ceph-mon[35011]: scrub summary: idle+waiting paths [/] 2024-09-12T03:02:13.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:02:13 smithi146 ceph-mon[35011]: scrub summary: active paths [/] 2024-09-12T03:02:13.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:02:13 smithi146 ceph-mon[35011]: scrub summary: idle+waiting paths [/] 2024-09-12T03:02:13.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:02:13 smithi146 ceph-mon[35011]: scrub summary: idle+waiting paths [/] 2024-09-12T03:02:14.565 INFO:tasks.cephfs.filesystem:scrub status for tag:02646886-4faa-43bf-b97f-66bfd7952f7b - {'path': '/', 'tag': '02646886-4faa-43bf-b97f-66bfd7952f7b', 'options': 'recursive,force'} 2024-09-12T03:02:15.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:02:15 smithi005 ceph-mon[31197]: pgmap v455: 97 pgs: 97 active+clean; 114 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.0 MiB/s wr, 179 op/s 2024-09-12T03:02:15.915 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:02:15 smithi148 ceph-mon[35090]: pgmap v455: 97 pgs: 97 active+clean; 114 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.0 MiB/s wr, 179 op/s 2024-09-12T03:02:15.956 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:02:15 smithi146 ceph-mon[35011]: pgmap v455: 97 pgs: 97 active+clean; 114 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 1.0 MiB/s wr, 179 op/s 2024-09-12T03:02:17.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:02:17 smithi005 ceph-mon[31197]: pgmap v456: 97 pgs: 97 active+clean; 114 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 482 KiB/s wr, 98 op/s 2024-09-12T03:02:17.861 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:02:17.915 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:02:17 smithi148 ceph-mon[35090]: pgmap v456: 97 pgs: 97 active+clean; 114 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 482 KiB/s wr, 98 op/s 2024-09-12T03:02:17.956 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:02:17 smithi146 ceph-mon[35011]: pgmap v456: 97 pgs: 97 active+clean; 114 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 482 KiB/s wr, 98 op/s 2024-09-12T03:02:18.133 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:02:18.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:18.614+0000 7f343e23e640 1 -- 172.21.15.5:0/4029374630 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3438101420 msgr2=0x7f343810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:18.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:18.614+0000 7f343e23e640 1 --2- 172.21.15.5:0/4029374630 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3438101420 0x7f343810d950 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f3430009930 tx=0x7f343002a420 comp rx=0 tx=0).stop 2024-09-12T03:02:18.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:18.614+0000 7f343e23e640 1 -- 172.21.15.5:0/4029374630 shutdown_connections 2024-09-12T03:02:18.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:18.614+0000 7f343e23e640 1 --2- 172.21.15.5:0/4029374630 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3438101420 0x7f343810d950 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:18.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:18.614+0000 7f343e23e640 1 --2- 172.21.15.5:0/4029374630 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3438100a80 0x7f3438100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:18.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:18.614+0000 7f343e23e640 1 --2- 172.21.15.5:0/4029374630 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3438106a80 0x7f3438106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:18.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:18.614+0000 7f343e23e640 1 -- 172.21.15.5:0/4029374630 >> 172.21.15.5:0/4029374630 conn(0x7f34380fc7a0 msgr2=0x7f34380febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:18.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:18.614+0000 7f343e23e640 1 -- 172.21.15.5:0/4029374630 shutdown_connections 2024-09-12T03:02:18.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:18.614+0000 7f343e23e640 1 -- 172.21.15.5:0/4029374630 wait complete. 2024-09-12T03:02:18.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:18.614+0000 7f343e23e640 1 Processor -- start 2024-09-12T03:02:18.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:18.615+0000 7f343e23e640 1 -- start start 2024-09-12T03:02:18.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:18.615+0000 7f343e23e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3438100a80 0x7f3438198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:18.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:18.615+0000 7f343e23e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3438101420 0x7f3438198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:18.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:18.615+0000 7f343e23e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3438106a80 0x7f343819ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:18.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:18.615+0000 7f343e23e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3438110e80 con 0x7f3438106a80 2024-09-12T03:02:18.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:18.615+0000 7f343e23e640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f3438110d00 con 0x7f3438101420 2024-09-12T03:02:18.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:18.615+0000 7f343e23e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3438111000 con 0x7f3438100a80 2024-09-12T03:02:18.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:18.615+0000 7f3437fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3438106a80 0x7f343819ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:18.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:18.615+0000 7f3437fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3438106a80 0x7f343819ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39778/0 (socket says 172.21.15.5:39778) 2024-09-12T03:02:18.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:18.615+0000 7f3437fff640 1 -- 172.21.15.5:0/73942492 learned_addr learned my addr 172.21.15.5:0/73942492 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:02:18.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:18.615+0000 7f34377fe640 1 --2- 172.21.15.5:0/73942492 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3438100a80 0x7f3438198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:18.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:18.615+0000 7f3436ffd640 1 --2- 172.21.15.5:0/73942492 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3438101420 0x7f3438198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:18.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:18.615+0000 7f3437fff640 1 -- 172.21.15.5:0/73942492 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3438100a80 msgr2=0x7f3438198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:18.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:18.615+0000 7f3437fff640 1 --2- 172.21.15.5:0/73942492 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3438100a80 0x7f3438198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:18.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:18.615+0000 7f3437fff640 1 -- 172.21.15.5:0/73942492 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3438101420 msgr2=0x7f3438198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:18.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:18.615+0000 7f3437fff640 1 --2- 172.21.15.5:0/73942492 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3438101420 0x7f3438198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:18.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:18.615+0000 7f3437fff640 1 -- 172.21.15.5:0/73942492 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f343819d3b0 con 0x7f3438106a80 2024-09-12T03:02:18.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:18.615+0000 7f3437fff640 1 --2- 172.21.15.5:0/73942492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3438106a80 0x7f343819ccd0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f3430005ab0 tx=0x7f34300042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:18.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:18.616+0000 7f3434ff9640 1 -- 172.21.15.5:0/73942492 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f343003f030 con 0x7f3438106a80 2024-09-12T03:02:18.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:18.616+0000 7f343e23e640 1 -- 172.21.15.5:0/73942492 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f343819d640 con 0x7f3438106a80 2024-09-12T03:02:18.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:18.616+0000 7f3434ff9640 1 -- 172.21.15.5:0/73942492 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3430003ce0 con 0x7f3438106a80 2024-09-12T03:02:18.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:18.616+0000 7f3434ff9640 1 -- 172.21.15.5:0/73942492 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3430035070 con 0x7f3438106a80 2024-09-12T03:02:18.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:18.616+0000 7f343e23e640 1 -- 172.21.15.5:0/73942492 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34381a4f80 con 0x7f3438106a80 2024-09-12T03:02:18.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:18.617+0000 7f343e23e640 1 -- 172.21.15.5:0/73942492 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34381021c0 con 0x7f3438106a80 2024-09-12T03:02:18.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:18.618+0000 7f3434ff9640 1 -- 172.21.15.5:0/73942492 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f3430042030 con 0x7f3438106a80 2024-09-12T03:02:18.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:18.619+0000 7f3434ff9640 1 --2- 172.21.15.5:0/73942492 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f34200a2b80 0x7f34200a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:18.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:18.619+0000 7f34377fe640 1 --2- 172.21.15.5:0/73942492 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f34200a2b80 0x7f34200a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:18.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:18.619+0000 7f3434ff9640 1 -- 172.21.15.5:0/73942492 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f34300ef870 con 0x7f3438106a80 2024-09-12T03:02:18.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:18.619+0000 7f34377fe640 1 --2- 172.21.15.5:0/73942492 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f34200a2b80 0x7f34200a5040 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f3428005fd0 tx=0x7f3428009680 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:18.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:18.622+0000 7f3434ff9640 1 -- 172.21.15.5:0/73942492 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f34300bc130 con 0x7f3438106a80 2024-09-12T03:02:18.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:02:18 smithi005 ceph-mon[31197]: scrub summary: idle 2024-09-12T03:02:18.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:02:18 smithi005 ceph-mon[31197]: scrub summary: idle 2024-09-12T03:02:18.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:18.836+0000 7f343e23e640 1 -- 172.21.15.5:0/73942492 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f343810a000 con 0x7f3438106a80 2024-09-12T03:02:18.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:18.836+0000 7f3434ff9640 1 -- 172.21.15.5:0/73942492 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11967 (secure 0 0 0) 0x7f34300c0430 con 0x7f3438106a80 2024-09-12T03:02:18.837 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-12T03:02:18.837 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":30,"btime":"2024-09-12T03:01:58:481122+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/4135232521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":4135232521},{"type":"v1","addr":"172.21.15.5:6835","nonce":4135232521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/4290221940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":4290221940},{"type":"v1","addr":"172.21.15.5:6837","nonce":4290221940}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T02:56:16.595562+0000","modified":"2024-09-12T03:01:57.686450+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24502,"mds_1":24487,"mds_2":24481,"mds_3":24479,"mds_4":24458},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/26447341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":26447341},{"type":"v1","addr":"172.21.15.5:6839","nonce":26447341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2393443295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2393443295},{"type":"v1","addr":"172.21.15.5:6841","nonce":2393443295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6833/3686970137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3686970137},{"type":"v1","addr":"172.21.15.146:6833","nonce":3686970137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6833/568512209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":568512209},{"type":"v1","addr":"172.21.15.148:6833","nonce":568512209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6835/2540740420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2540740420},{"type":"v1","addr":"172.21.15.146:6835","nonce":2540740420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.146:6837/1673370937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":1673370937},{"type":"v1","addr":"172.21.15.146:6837","nonce":1673370937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":3,"incarnation":22,"state":"up:active","state_seq":6,"addr":"172.21.15.146:6839/3036767228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":3036767228},{"type":"v1","addr":"172.21.15.146:6839","nonce":3036767228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":2,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.148:6835/4195086427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":4195086427},{"type":"v1","addr":"172.21.15.148:6835","nonce":4195086427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.148:6837/2606873922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2606873922},{"type":"v1","addr":"172.21.15.148:6837","nonce":2606873922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6839/2317600000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":2317600000},{"type":"v1","addr":"172.21.15.148:6839","nonce":2317600000}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24502,"qdb_cluster":[24502,24487,24481,24479,24458]},"id":1}]} 2024-09-12T03:02:18.837 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 30 2024-09-12T03:02:18.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:18.838+0000 7f343e23e640 1 -- 172.21.15.5:0/73942492 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f34200a2b80 msgr2=0x7f34200a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:18.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:18.838+0000 7f343e23e640 1 --2- 172.21.15.5:0/73942492 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f34200a2b80 0x7f34200a5040 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f3428005fd0 tx=0x7f3428009680 comp rx=0 tx=0).stop 2024-09-12T03:02:18.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:18.838+0000 7f343e23e640 1 -- 172.21.15.5:0/73942492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3438106a80 msgr2=0x7f343819ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:18.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:18.838+0000 7f343e23e640 1 --2- 172.21.15.5:0/73942492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3438106a80 0x7f343819ccd0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f3430005ab0 tx=0x7f34300042f0 comp rx=0 tx=0).stop 2024-09-12T03:02:18.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:18.838+0000 7f343e23e640 1 -- 172.21.15.5:0/73942492 shutdown_connections 2024-09-12T03:02:18.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:18.838+0000 7f343e23e640 1 --2- 172.21.15.5:0/73942492 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f34200a2b80 0x7f34200a5040 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:18.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:18.838+0000 7f343e23e640 1 --2- 172.21.15.5:0/73942492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3438106a80 0x7f343819ccd0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:18.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:18.838+0000 7f343e23e640 1 --2- 172.21.15.5:0/73942492 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3438101420 0x7f3438198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:18.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:18.839+0000 7f343e23e640 1 --2- 172.21.15.5:0/73942492 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3438100a80 0x7f3438198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:18.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:18.839+0000 7f343e23e640 1 -- 172.21.15.5:0/73942492 >> 172.21.15.5:0/73942492 conn(0x7f34380fc7a0 msgr2=0x7f343806a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:18.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:18.839+0000 7f343e23e640 1 -- 172.21.15.5:0/73942492 shutdown_connections 2024-09-12T03:02:18.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:18.839+0000 7f343e23e640 1 -- 172.21.15.5:0/73942492 wait complete. 2024-09-12T03:02:18.915 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:02:18 smithi148 ceph-mon[35090]: scrub summary: idle 2024-09-12T03:02:18.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:02:18 smithi148 ceph-mon[35090]: scrub summary: idle 2024-09-12T03:02:18.956 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:02:18 smithi146 ceph-mon[35011]: scrub summary: idle 2024-09-12T03:02:18.957 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:02:18 smithi146 ceph-mon[35011]: scrub summary: idle 2024-09-12T03:02:19.580 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24502 quiesce db --set-id 42c0cc77 --release --await' 2024-09-12T03:02:19.580 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24502 quiesce db --set-id 42c0cc77 --release --await 2024-09-12T03:02:19.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:02:19 smithi005 ceph-mon[31197]: pgmap v457: 97 pgs: 97 active+clean; 114 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 16 KiB/s wr, 10 op/s 2024-09-12T03:02:19.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:02:19 smithi005 ceph-mon[31197]: scrub summary: idle 2024-09-12T03:02:19.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:02:19 smithi005 ceph-mon[31197]: scrub summary: idle 2024-09-12T03:02:19.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:02:19 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/73942492' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:02:19.845 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:02:19.915 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:02:19 smithi148 ceph-mon[35090]: pgmap v457: 97 pgs: 97 active+clean; 114 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 16 KiB/s wr, 10 op/s 2024-09-12T03:02:19.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:02:19 smithi148 ceph-mon[35090]: scrub summary: idle 2024-09-12T03:02:19.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:02:19 smithi148 ceph-mon[35090]: scrub summary: idle 2024-09-12T03:02:19.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:02:19 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/73942492' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:02:19.956 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:02:19 smithi146 ceph-mon[35011]: pgmap v457: 97 pgs: 97 active+clean; 114 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 16 KiB/s wr, 10 op/s 2024-09-12T03:02:19.956 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:02:19 smithi146 ceph-mon[35011]: scrub summary: idle 2024-09-12T03:02:19.956 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:02:19 smithi146 ceph-mon[35011]: scrub summary: idle 2024-09-12T03:02:19.956 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:02:19 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/73942492' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:02:20.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.329+0000 7ff40e226640 1 -- 172.21.15.5:0/568393146 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff408102790 msgr2=0x7ff4081067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:20.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.329+0000 7ff40e226640 1 --2- 172.21.15.5:0/568393146 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff408102790 0x7ff4081067e0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc009900 tx=0x7ff3fc02a410 comp rx=0 tx=0).stop 2024-09-12T03:02:20.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.329+0000 7ff40e226640 1 -- 172.21.15.5:0/568393146 shutdown_connections 2024-09-12T03:02:20.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.329+0000 7ff40e226640 1 --2- 172.21.15.5:0/568393146 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff408102790 0x7ff4081067e0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:20.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.329+0000 7ff40e226640 1 --2- 172.21.15.5:0/568393146 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff408106f10 0x7ff40810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:20.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.329+0000 7ff40e226640 1 --2- 172.21.15.5:0/568393146 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff408101de0 0x7ff4081021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:20.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.329+0000 7ff40e226640 1 -- 172.21.15.5:0/568393146 >> 172.21.15.5:0/568393146 conn(0x7ff4080fd8c0 msgr2=0x7ff4080ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:20.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.329+0000 7ff40e226640 1 -- 172.21.15.5:0/568393146 shutdown_connections 2024-09-12T03:02:20.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.329+0000 7ff40e226640 1 -- 172.21.15.5:0/568393146 wait complete. 2024-09-12T03:02:20.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.329+0000 7ff40e226640 1 Processor -- start 2024-09-12T03:02:20.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.330+0000 7ff40e226640 1 -- start start 2024-09-12T03:02:20.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.330+0000 7ff40e226640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff408101de0 0x7ff4081984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:20.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.330+0000 7ff40e226640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff408102790 0x7ff408198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:20.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.330+0000 7ff40e226640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff408106f10 0x7ff40819cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:20.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.330+0000 7ff40e226640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff408111f70 con 0x7ff408101de0 2024-09-12T03:02:20.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.330+0000 7ff40e226640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ff408111df0 con 0x7ff408106f10 2024-09-12T03:02:20.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.330+0000 7ff40e226640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff4081120f0 con 0x7ff408102790 2024-09-12T03:02:20.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.330+0000 7ff4077fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff408101de0 0x7ff4081984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:20.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.330+0000 7ff4077fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff408101de0 0x7ff4081984f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39798/0 (socket says 172.21.15.5:39798) 2024-09-12T03:02:20.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.330+0000 7ff4077fe640 1 -- 172.21.15.5:0/1943730451 learned_addr learned my addr 172.21.15.5:0/1943730451 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:02:20.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.330+0000 7ff407fff640 1 --2- 172.21.15.5:0/1943730451 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff408106f10 0x7ff40819cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:20.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.330+0000 7ff4077fe640 1 -- 172.21.15.5:0/1943730451 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff408102790 msgr2=0x7ff408198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:20.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.330+0000 7ff406ffd640 1 --2- 172.21.15.5:0/1943730451 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff408102790 0x7ff408198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:20.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.330+0000 7ff4077fe640 1 --2- 172.21.15.5:0/1943730451 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff408102790 0x7ff408198a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:20.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.330+0000 7ff4077fe640 1 -- 172.21.15.5:0/1943730451 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff408106f10 msgr2=0x7ff40819cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:20.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.330+0000 7ff4077fe640 1 --2- 172.21.15.5:0/1943730451 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff408106f10 0x7ff40819cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:20.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.330+0000 7ff4077fe640 1 -- 172.21.15.5:0/1943730451 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff40819d4a0 con 0x7ff408101de0 2024-09-12T03:02:20.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.330+0000 7ff407fff640 1 --2- 172.21.15.5:0/1943730451 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff408106f10 0x7ff40819cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:02:20.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.331+0000 7ff4077fe640 1 --2- 172.21.15.5:0/1943730451 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff408101de0 0x7ff4081984f0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7ff3f800c970 tx=0x7ff3f800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:20.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.331+0000 7ff404ff9640 1 -- 172.21.15.5:0/1943730451 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3f8007bb0 con 0x7ff408101de0 2024-09-12T03:02:20.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.331+0000 7ff404ff9640 1 -- 172.21.15.5:0/1943730451 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3f8007d50 con 0x7ff408101de0 2024-09-12T03:02:20.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.331+0000 7ff40e226640 1 -- 172.21.15.5:0/1943730451 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff40819d790 con 0x7ff408101de0 2024-09-12T03:02:20.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.331+0000 7ff404ff9640 1 -- 172.21.15.5:0/1943730451 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3f800f450 con 0x7ff408101de0 2024-09-12T03:02:20.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.331+0000 7ff40e226640 1 -- 172.21.15.5:0/1943730451 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4081a5070 con 0x7ff408101de0 2024-09-12T03:02:20.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.334+0000 7ff4105ff740 1 Processor -- start 2024-09-12T03:02:20.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.334+0000 7ff4105ff740 1 -- start start 2024-09-12T03:02:20.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.334+0000 7ff404ff9640 1 -- 172.21.15.5:0/1943730451 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7ff3f8005020 con 0x7ff408101de0 2024-09-12T03:02:20.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.334+0000 7ff4105ff740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558784579f60 0x55878457a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:20.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.334+0000 7ff4105ff740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55878457a880 0x558784589110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:20.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.334+0000 7ff4105ff740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558784589650 0x55878458ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:20.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.334+0000 7ff4105ff740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5587844df620 con 0x558784579f60 2024-09-12T03:02:20.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.334+0000 7ff4105ff740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x558784486e50 con 0x558784589650 2024-09-12T03:02:20.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.334+0000 7ff4105ff740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5587844db5e0 con 0x55878457a880 2024-09-12T03:02:20.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.334+0000 7ff407fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558784579f60 0x55878457a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:20.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.334+0000 7ff407fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558784579f60 0x55878457a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39810/0 (socket says 172.21.15.5:39810) 2024-09-12T03:02:20.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.334+0000 7ff407fff640 1 -- 172.21.15.5:0/1053923242 learned_addr learned my addr 172.21.15.5:0/1053923242 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:02:20.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.334+0000 7ff4077fe640 1 --2- 172.21.15.5:0/1053923242 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55878457a880 0x558784589110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:20.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.334+0000 7ff406ffd640 1 --2- 172.21.15.5:0/1053923242 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558784589650 0x55878458ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:20.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.335+0000 7ff407fff640 1 -- 172.21.15.5:0/1053923242 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55878457a880 msgr2=0x558784589110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:20.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.335+0000 7ff407fff640 1 --2- 172.21.15.5:0/1053923242 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55878457a880 0x558784589110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:20.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.335+0000 7ff407fff640 1 -- 172.21.15.5:0/1053923242 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558784589650 msgr2=0x55878458ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:20.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.335+0000 7ff407fff640 1 --2- 172.21.15.5:0/1053923242 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558784589650 0x55878458ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:20.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.335+0000 7ff407fff640 1 -- 172.21.15.5:0/1053923242 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558784477d40 con 0x558784579f60 2024-09-12T03:02:20.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.335+0000 7ff407fff640 1 --2- 172.21.15.5:0/1053923242 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558784579f60 0x55878457a340 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7ff400009fd0 tx=0x7ff40000f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=916e6c838b75093b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:20.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.335+0000 7ff404ff9640 1 --2- 172.21.15.5:0/1943730451 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7ff3f00a2c00 0x7ff3f00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:20.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.335+0000 7ff3edffb640 1 -- 172.21.15.5:0/1053923242 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff400015070 con 0x558784579f60 2024-09-12T03:02:20.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.335+0000 7ff3edffb640 1 -- 172.21.15.5:0/1053923242 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff400011be0 con 0x558784579f60 2024-09-12T03:02:20.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.335+0000 7ff3edffb640 1 -- 172.21.15.5:0/1053923242 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff400010c30 con 0x558784579f60 2024-09-12T03:02:20.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.335+0000 7ff404ff9640 1 -- 172.21.15.5:0/1943730451 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7ff3f80cee40 con 0x7ff408101de0 2024-09-12T03:02:20.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.335+0000 7ff4077fe640 1 --2- 172.21.15.5:0/1943730451 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7ff3f00a2c00 0x7ff3f00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:20.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.336+0000 7ff4105ff740 1 -- 172.21.15.5:0/1053923242 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558784579f60 msgr2=0x55878457a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:20.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.336+0000 7ff4105ff740 1 --2- 172.21.15.5:0/1053923242 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558784579f60 0x55878457a340 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7ff400009fd0 tx=0x7ff40000f7f0 comp rx=0 tx=0).stop 2024-09-12T03:02:20.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.336+0000 7ff4077fe640 1 --2- 172.21.15.5:0/1943730451 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7ff3f00a2c00 0x7ff3f00a50c0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7ff3f80cccc0 tx=0x7ff3f80d2920 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:20.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.336+0000 7ff4105ff740 1 -- 172.21.15.5:0/1053923242 shutdown_connections 2024-09-12T03:02:20.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.336+0000 7ff4105ff740 1 --2- 172.21.15.5:0/1053923242 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558784589650 0x55878458ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:20.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.336+0000 7ff4105ff740 1 --2- 172.21.15.5:0/1053923242 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55878457a880 0x558784589110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:20.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.336+0000 7ff4105ff740 1 --2- 172.21.15.5:0/1053923242 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558784579f60 0x55878457a340 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:20.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.336+0000 7ff4105ff740 1 -- 172.21.15.5:0/1053923242 >> 172.21.15.5:0/1053923242 conn(0x5587845765f0 msgr2=0x5587845769d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:20.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.336+0000 7ff4105ff740 1 -- 172.21.15.5:0/1053923242 shutdown_connections 2024-09-12T03:02:20.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.336+0000 7ff4105ff740 1 -- 172.21.15.5:0/1053923242 wait complete. 2024-09-12T03:02:20.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.336+0000 7ff4105ff740 1 -- start start 2024-09-12T03:02:20.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.337+0000 7ff4105ff740 1 Processor -- start 2024-09-12T03:02:20.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.337+0000 7ff4105ff740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558784579f60 0x558784586640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:20.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.337+0000 7ff4105ff740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55878457a880 0x558784586b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:20.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.337+0000 7ff4105ff740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558784589650 0x5587845870c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:20.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.337+0000 7ff4105ff740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55878458c280 con 0x558784589650 2024-09-12T03:02:20.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.337+0000 7ff4105ff740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x558784467f10 con 0x558784579f60 2024-09-12T03:02:20.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.337+0000 7ff4105ff740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55878458c400 con 0x55878457a880 2024-09-12T03:02:20.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.337+0000 7ff4105ff740 10 client.0 _ll_register_callbacks cb 0x55878448bdb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:02:20.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.337+0000 7ff407fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558784589650 0x5587845870c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:20.347 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.337+0000 7ff407fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558784589650 0x5587845870c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39820/0 (socket says 172.21.15.5:39820) 2024-09-12T03:02:20.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.337+0000 7ff407fff640 1 -- 172.21.15.5:0/200378509 learned_addr learned my addr 172.21.15.5:0/200378509 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:02:20.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.337+0000 7ff407fff640 1 --2- 172.21.15.5:0/200378509 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558784579f60 0x558784586640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:20.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.337+0000 7ff406ffd640 1 --2- 172.21.15.5:0/200378509 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55878457a880 0x558784586b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:20.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.337+0000 7ff407fff640 1 -- 172.21.15.5:0/200378509 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55878457a880 msgr2=0x558784586b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:20.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.337+0000 7ff407fff640 1 --2- 172.21.15.5:0/200378509 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55878457a880 0x558784586b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:20.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.337+0000 7ff407fff640 1 -- 172.21.15.5:0/200378509 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558784579f60 msgr2=0x558784586640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:20.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.337+0000 7ff407fff640 1 --2- 172.21.15.5:0/200378509 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558784579f60 0x558784586640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:20.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.337+0000 7ff407fff640 1 -- 172.21.15.5:0/200378509 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587841ee580 con 0x558784589650 2024-09-12T03:02:20.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.338+0000 7ff407fff640 1 --2- 172.21.15.5:0/200378509 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558784589650 0x5587845870c0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7ff400010910 tx=0x7ff400010810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c608a0dfa8d6065 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:20.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.338+0000 7ff3e6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T03:02:20.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.338+0000 7ff3e6ffd640 1 -- 172.21.15.5:0/200378509 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff400015040 con 0x558784589650 2024-09-12T03:02:20.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.338+0000 7ff3e6ffd640 1 -- 172.21.15.5:0/200378509 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7ff400010a70 con 0x558784589650 2024-09-12T03:02:20.348 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.338+0000 7ff4105ff740 1 -- 172.21.15.5:0/200378509 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff400011be0 con 0x558784589650 2024-09-12T03:02:20.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.338+0000 7ff3e6ffd640 1 -- 172.21.15.5:0/200378509 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff400010c70 con 0x558784589650 2024-09-12T03:02:20.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.338+0000 7ff4105ff740 10 client.16365 fetch_fsmap learned FSMap version 30 2024-09-12T03:02:20.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.338+0000 7ff4105ff740 1 -- 172.21.15.5:0/200378509 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5587842b8960 con 0x558784589650 2024-09-12T03:02:20.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.338+0000 7ff3e6ffd640 1 -- 172.21.15.5:0/200378509 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff400011be0 con 0x558784589650 2024-09-12T03:02:20.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.338+0000 7ff3e6ffd640 1 -- 172.21.15.5:0/200378509 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7ff400028030 con 0x558784589650 2024-09-12T03:02:20.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.338+0000 7ff4105ff740 10 client.16365 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:02:20.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.338+0000 7ff4105ff740 10 client.16365 resolve_mds: validated gid 24502 aka daemon mds.i 2024-09-12T03:02:20.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.338+0000 7ff4105ff740 20 client.16365 populate_metadata read hostname 'smithi005' 2024-09-12T03:02:20.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.338+0000 7ff4105ff740 1 --2- 172.21.15.5:0/200378509 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x558784599300 0x55878459b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:20.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.338+0000 7ff4105ff740 4 client.16365 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:02:20.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.338+0000 7ff4105ff740 1 -- 172.21.15.5:0/200378509 --> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5587844ecb90 con 0x558784599300 2024-09-12T03:02:20.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.339+0000 7ff406ffd640 1 --2- 172.21.15.5:0/200378509 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x558784599300 0x55878459b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:20.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.339+0000 7ff3e6ffd640 10 client.16365 ms_handle_connect on v2:172.21.15.148:6838/2317600000 2024-09-12T03:02:20.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.339+0000 7ff406ffd640 1 --2- 172.21.15.5:0/200378509 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x558784599300 0x55878459b6f0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=688df545dd8d2a14 server_cookie=701f1d3a54f51daa in_seq=0 out_seq=0 2024-09-12T03:02:20.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.340+0000 7ff3e6ffd640 1 -- 172.21.15.5:0/200378509 <== mds.0 v2:172.21.15.148:6838/2317600000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5587844ecb90 con 0x558784599300 2024-09-12T03:02:20.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.340+0000 7ff3e6ffd640 10 client.16365 handle_command_reply: tid=0 2024-09-12T03:02:20.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.340+0000 7ff4105ff740 1 client.16365 shutdown 2024-09-12T03:02:20.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.340+0000 7ff4105ff740 1 -- 172.21.15.5:0/200378509 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558784589650 msgr2=0x5587845870c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:20.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.340+0000 7ff4105ff740 1 --2- 172.21.15.5:0/200378509 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558784589650 0x5587845870c0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7ff400010910 tx=0x7ff400010810 comp rx=0 tx=0).stop 2024-09-12T03:02:20.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.340+0000 7ff4105ff740 1 -- 172.21.15.5:0/200378509 shutdown_connections 2024-09-12T03:02:20.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.340+0000 7ff4105ff740 1 --2- 172.21.15.5:0/200378509 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x558784599300 0x55878459b6f0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:20.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.340+0000 7ff4105ff740 1 --2- 172.21.15.5:0/200378509 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558784589650 0x5587845870c0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:20.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.340+0000 7ff4105ff740 1 --2- 172.21.15.5:0/200378509 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55878457a880 0x558784586b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:20.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.340+0000 7ff3e6ffd640 0 client.16365 ms_handle_reset on v2:172.21.15.148:6838/2317600000 2024-09-12T03:02:20.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.340+0000 7ff4105ff740 1 --2- 172.21.15.5:0/200378509 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558784579f60 0x558784586640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:20.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.340+0000 7ff4105ff740 1 -- 172.21.15.5:0/200378509 >> 172.21.15.5:0/200378509 conn(0x5587845765f0 msgr2=0x55878457c320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:20.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.340+0000 7ff4105ff740 1 -- 172.21.15.5:0/200378509 shutdown_connections 2024-09-12T03:02:20.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.340+0000 7ff4105ff740 1 -- 172.21.15.5:0/200378509 wait complete. 2024-09-12T03:02:20.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.340+0000 7ff4105ff740 20 client.16365 trim_cache size 0 max 16384 2024-09-12T03:02:20.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.358+0000 7ff4105ff740 1 Processor -- start 2024-09-12T03:02:20.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.358+0000 7ff4105ff740 1 -- start start 2024-09-12T03:02:20.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.358+0000 7ff4105ff740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558784579f60 0x5587845a0da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:20.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.358+0000 7ff4105ff740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55878457a880 0x5587845a12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:20.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.358+0000 7ff4105ff740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558784589650 0x558784591680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:20.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.358+0000 7ff4105ff740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff4081ab3b0 con 0x558784589650 2024-09-12T03:02:20.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.358+0000 7ff4105ff740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ff4081acf20 con 0x55878457a880 2024-09-12T03:02:20.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.358+0000 7ff4105ff740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff4081aba80 con 0x558784579f60 2024-09-12T03:02:20.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.358+0000 7ff407fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558784589650 0x558784591680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:20.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.358+0000 7ff407fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558784589650 0x558784591680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39830/0 (socket says 172.21.15.5:39830) 2024-09-12T03:02:20.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.358+0000 7ff407fff640 1 -- 172.21.15.5:0/2456575806 learned_addr learned my addr 172.21.15.5:0/2456575806 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:02:20.359 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.358+0000 7ff407fff640 1 --2- 172.21.15.5:0/2456575806 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558784579f60 0x5587845a0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:20.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.358+0000 7ff406ffd640 1 --2- 172.21.15.5:0/2456575806 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55878457a880 0x5587845a12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:20.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.358+0000 7ff407fff640 1 -- 172.21.15.5:0/2456575806 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558784579f60 msgr2=0x5587845a0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:20.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.358+0000 7ff407fff640 1 --2- 172.21.15.5:0/2456575806 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558784579f60 0x5587845a0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:20.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.358+0000 7ff407fff640 1 -- 172.21.15.5:0/2456575806 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55878457a880 msgr2=0x5587845a12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:20.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.358+0000 7ff407fff640 1 --2- 172.21.15.5:0/2456575806 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55878457a880 0x5587845a12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:20.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.358+0000 7ff407fff640 1 -- 172.21.15.5:0/2456575806 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587845855f0 con 0x558784589650 2024-09-12T03:02:20.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.358+0000 7ff407fff640 1 --2- 172.21.15.5:0/2456575806 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558784589650 0x558784591680 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7ff400005d70 tx=0x7ff40000f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69c8abe243f80a14 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:20.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.359+0000 7ff3ed7fa640 1 -- 172.21.15.5:0/2456575806 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff400015030 con 0x558784589650 2024-09-12T03:02:20.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.359+0000 7ff3ed7fa640 1 -- 172.21.15.5:0/2456575806 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff40000fcf0 con 0x558784589650 2024-09-12T03:02:20.360 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.359+0000 7ff3ed7fa640 1 -- 172.21.15.5:0/2456575806 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff40001e070 con 0x558784589650 2024-09-12T03:02:20.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.359+0000 7ff4105ff740 1 -- 172.21.15.5:0/2456575806 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558784589650 msgr2=0x558784591680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:20.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.359+0000 7ff4105ff740 1 --2- 172.21.15.5:0/2456575806 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558784589650 0x558784591680 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7ff400005d70 tx=0x7ff40000f7f0 comp rx=0 tx=0).stop 2024-09-12T03:02:20.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.359+0000 7ff4105ff740 1 -- 172.21.15.5:0/2456575806 shutdown_connections 2024-09-12T03:02:20.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.359+0000 7ff4105ff740 1 --2- 172.21.15.5:0/2456575806 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558784589650 0x558784591680 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:20.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.359+0000 7ff4105ff740 1 --2- 172.21.15.5:0/2456575806 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55878457a880 0x5587845a12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:20.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.359+0000 7ff4105ff740 1 --2- 172.21.15.5:0/2456575806 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x558784579f60 0x5587845a0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:20.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.359+0000 7ff4105ff740 1 -- 172.21.15.5:0/2456575806 >> 172.21.15.5:0/2456575806 conn(0x5587845765f0 msgr2=0x558784586600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:20.363 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.359+0000 7ff4105ff740 1 -- 172.21.15.5:0/2456575806 shutdown_connections 2024-09-12T03:02:20.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.359+0000 7ff4105ff740 1 -- 172.21.15.5:0/2456575806 wait complete. 2024-09-12T03:02:20.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.360+0000 7ff4105ff740 1 -- start start 2024-09-12T03:02:20.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.360+0000 7ff4105ff740 1 Processor -- start 2024-09-12T03:02:20.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.360+0000 7ff4105ff740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558784579f60 0x55878458dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:20.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.360+0000 7ff4105ff740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55878457a880 0x55878458e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:20.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.360+0000 7ff4105ff740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558784589650 0x558784602d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:20.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.360+0000 7ff4105ff740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff4081ac820 con 0x558784589650 2024-09-12T03:02:20.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.360+0000 7ff4105ff740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ff4081ac150 con 0x558784579f60 2024-09-12T03:02:20.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.360+0000 7ff4105ff740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff4081a9390 con 0x55878457a880 2024-09-12T03:02:20.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.360+0000 7ff4105ff740 10 client.0 _ll_register_callbacks cb 0x7ff400010460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:02:20.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.360+0000 7ff407fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558784589650 0x558784602d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:20.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.360+0000 7ff407fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558784589650 0x558784602d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39842/0 (socket says 172.21.15.5:39842) 2024-09-12T03:02:20.364 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.360+0000 7ff407fff640 1 -- 172.21.15.5:0/214568384 learned_addr learned my addr 172.21.15.5:0/214568384 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:02:20.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.360+0000 7ff406ffd640 1 --2- 172.21.15.5:0/214568384 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55878457a880 0x55878458e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:20.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.361+0000 7ff407fff640 1 --2- 172.21.15.5:0/214568384 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558784579f60 0x55878458dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:20.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.361+0000 7ff406ffd640 1 -- 172.21.15.5:0/214568384 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558784579f60 msgr2=0x55878458dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:20.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.361+0000 7ff406ffd640 1 --2- 172.21.15.5:0/214568384 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558784579f60 0x55878458dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:20.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.361+0000 7ff406ffd640 1 -- 172.21.15.5:0/214568384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558784589650 msgr2=0x558784602d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:20.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.361+0000 7ff406ffd640 1 --2- 172.21.15.5:0/214568384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558784589650 0x558784602d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:20.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.361+0000 7ff406ffd640 1 -- 172.21.15.5:0/214568384 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558784603330 con 0x55878457a880 2024-09-12T03:02:20.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.361+0000 7ff407fff640 1 --2- 172.21.15.5:0/214568384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558784589650 0x558784602d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:02:20.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.361+0000 7ff406ffd640 1 --2- 172.21.15.5:0/214568384 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55878457a880 0x55878458e160 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc006010 tx=0x7ff3fc0042e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=885b974048e1cfe9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:20.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.361+0000 7ff3e6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-09-12T03:02:20.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.361+0000 7ff3e6ffd640 1 -- 172.21.15.5:0/214568384 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3fc035040 con 0x55878457a880 2024-09-12T03:02:20.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.361+0000 7ff3e6ffd640 1 -- 172.21.15.5:0/214568384 <== mon.1 v2:172.21.15.148:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7ff3fc005450 con 0x55878457a880 2024-09-12T03:02:20.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.361+0000 7ff4105ff740 1 -- 172.21.15.5:0/214568384 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff40000fcf0 con 0x55878457a880 2024-09-12T03:02:20.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.361+0000 7ff3e6ffd640 1 -- 172.21.15.5:0/214568384 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff3fc005650 con 0x55878457a880 2024-09-12T03:02:20.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.361+0000 7ff4105ff740 10 client.25867 fetch_fsmap learned FSMap version 30 2024-09-12T03:02:20.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.361+0000 7ff4105ff740 1 -- 172.21.15.5:0/214568384 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558784604620 con 0x55878457a880 2024-09-12T03:02:20.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.362+0000 7ff3e6ffd640 1 -- 172.21.15.5:0/214568384 <== mon.1 v2:172.21.15.148:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff40000fcf0 con 0x55878457a880 2024-09-12T03:02:20.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.362+0000 7ff3e6ffd640 1 -- 172.21.15.5:0/214568384 <== mon.1 v2:172.21.15.148:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7ff3fc0037b0 con 0x55878457a880 2024-09-12T03:02:20.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.362+0000 7ff4105ff740 10 client.25867 fetch_fsmap finished waiting for FSMap version 30 2024-09-12T03:02:20.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.362+0000 7ff4105ff740 10 client.25867 resolve_mds: validated gid 24502 aka daemon mds.i 2024-09-12T03:02:20.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.362+0000 7ff4105ff740 20 client.25867 populate_metadata read hostname 'smithi005' 2024-09-12T03:02:20.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.362+0000 7ff4105ff740 1 --2- 172.21.15.5:0/214568384 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x558784599300 0x558784606dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:20.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.362+0000 7ff4105ff740 4 client.25867 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "42c0cc77", "release": true, "await": true}] 2024-09-12T03:02:20.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.362+0000 7ff4105ff740 1 -- 172.21.15.5:0/214568384 --> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] -- command(tid 0: {"prefix": "quiesce db", "set_id": "42c0cc77", "release": true, "await": true}) -- 0x5587844e70a0 con 0x558784599300 2024-09-12T03:02:20.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.362+0000 7ff407fff640 1 --2- 172.21.15.5:0/214568384 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x558784599300 0x558784606dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:20.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.363+0000 7ff407fff640 1 --2- 172.21.15.5:0/214568384 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x558784599300 0x558784606dc0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=115c31fa3239b3b9 server_cookie=cd57fe8062aeb5c4 in_seq=0 out_seq=0 2024-09-12T03:02:20.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.363+0000 7ff3e6ffd640 10 client.25867 ms_handle_connect on v2:172.21.15.148:6838/2317600000 2024-09-12T03:02:20.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.552+0000 7ff3e6ffd640 1 -- 172.21.15.5:0/214568384 <== mds.0 v2:172.21.15.148:6838/2317600000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5587844e70a0 con 0x558784599300 2024-09-12T03:02:20.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.552+0000 7ff3e6ffd640 10 client.25867 handle_command_reply: tid=0 2024-09-12T03:02:20.553 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.552+0000 7ff4105ff740 1 client.25867 shutdown 2024-09-12T03:02:20.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.552+0000 7ff4105ff740 1 -- 172.21.15.5:0/214568384 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55878457a880 msgr2=0x55878458e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:20.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.552+0000 7ff4105ff740 1 --2- 172.21.15.5:0/214568384 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55878457a880 0x55878458e160 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc006010 tx=0x7ff3fc0042e0 comp rx=0 tx=0).stop 2024-09-12T03:02:20.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.553+0000 7ff4105ff740 1 -- 172.21.15.5:0/214568384 shutdown_connections 2024-09-12T03:02:20.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.553+0000 7ff4105ff740 1 --2- 172.21.15.5:0/214568384 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x558784599300 0x558784606dc0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:20.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.553+0000 7ff4105ff740 1 --2- 172.21.15.5:0/214568384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558784589650 0x558784602d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:20.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.553+0000 7ff4105ff740 1 --2- 172.21.15.5:0/214568384 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55878457a880 0x55878458e160 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:20.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.553+0000 7ff4105ff740 1 --2- 172.21.15.5:0/214568384 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558784579f60 0x55878458dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:20.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.553+0000 7ff3e6ffd640 0 client.25867 ms_handle_reset on v2:172.21.15.148:6838/2317600000 2024-09-12T03:02:20.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.553+0000 7ff4105ff740 1 -- 172.21.15.5:0/214568384 >> 172.21.15.5:0/214568384 conn(0x5587845765f0 msgr2=0x55878457ba90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:20.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.553+0000 7ff4105ff740 1 -- 172.21.15.5:0/214568384 shutdown_connections 2024-09-12T03:02:20.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.553+0000 7ff4105ff740 1 -- 172.21.15.5:0/214568384 wait complete. 2024-09-12T03:02:20.554 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.553+0000 7ff4105ff740 20 client.25867 trim_cache size 0 max 16384 2024-09-12T03:02:20.555 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-12T03:02:20.555 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 30, 2024-09-12T03:02:20.555 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24502, 2024-09-12T03:02:20.555 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 16, 2024-09-12T03:02:20.555 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-09-12T03:02:20.555 INFO:teuthology.orchestra.run.smithi005.stdout: "42c0cc77": { 2024-09-12T03:02:20.555 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 16, 2024-09-12T03:02:20.555 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-09-12T03:02:20.555 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-12T03:02:20.555 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-09-12T03:02:20.555 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-12T03:02:20.556 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:02:20.556 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 90.0, 2024-09-12T03:02:20.556 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 140.2, 2024-09-12T03:02:20.556 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-09-12T03:02:20.556 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-09-12T03:02:20.556 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-09-12T03:02:20.556 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-12T03:02:20.556 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-09-12T03:02:20.556 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-12T03:02:20.556 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:02:20.556 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:02:20.556 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:02:20.556 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:02:20.556 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:02:20.557 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-12T03:02:20.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.553+0000 7ff40e226640 1 -- 172.21.15.5:0/1943730451 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7ff3f00a2c00 msgr2=0x7ff3f00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:20.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.553+0000 7ff40e226640 1 --2- 172.21.15.5:0/1943730451 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7ff3f00a2c00 0x7ff3f00a50c0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7ff3f80cccc0 tx=0x7ff3f80d2920 comp rx=0 tx=0).stop 2024-09-12T03:02:20.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.553+0000 7ff40e226640 1 -- 172.21.15.5:0/1943730451 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff408101de0 msgr2=0x7ff4081984f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:20.557 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.553+0000 7ff40e226640 1 --2- 172.21.15.5:0/1943730451 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff408101de0 0x7ff4081984f0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7ff3f800c970 tx=0x7ff3f800ce30 comp rx=0 tx=0).stop 2024-09-12T03:02:20.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.554+0000 7ff40e226640 1 -- 172.21.15.5:0/1943730451 shutdown_connections 2024-09-12T03:02:20.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.554+0000 7ff40e226640 1 --2- 172.21.15.5:0/1943730451 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff408106f10 0x7ff40819cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:20.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.554+0000 7ff40e226640 1 --2- 172.21.15.5:0/1943730451 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff408102790 0x7ff408198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:20.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.554+0000 7ff40e226640 1 --2- 172.21.15.5:0/1943730451 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7ff3f00a2c00 0x7ff3f00a50c0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:20.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.554+0000 7ff40e226640 1 --2- 172.21.15.5:0/1943730451 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff408101de0 0x7ff4081984f0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:20.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.554+0000 7ff40e226640 1 -- 172.21.15.5:0/1943730451 >> 172.21.15.5:0/1943730451 conn(0x7ff4080fd8c0 msgr2=0x7ff4080fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:20.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.554+0000 7ff40e226640 1 -- 172.21.15.5:0/1943730451 shutdown_connections 2024-09-12T03:02:20.558 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:20.554+0000 7ff40e226640 1 -- 172.21.15.5:0/1943730451 wait complete. 2024-09-12T03:02:21.450 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '42c0cc77', total seconds elapsed: 23.8 2024-09-12T03:02:21.450 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 20.15 2024-09-12T03:02:21.450 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:02:21.753 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:02:21.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:02:21 smithi005 ceph-mon[31197]: pgmap v458: 97 pgs: 97 active+clean; 114 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 16 KiB/s wr, 11 op/s 2024-09-12T03:02:21.915 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:02:21 smithi148 ceph-mon[35090]: pgmap v458: 97 pgs: 97 active+clean; 114 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 16 KiB/s wr, 11 op/s 2024-09-12T03:02:21.956 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:02:21 smithi146 ceph-mon[35011]: pgmap v458: 97 pgs: 97 active+clean; 114 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 16 KiB/s wr, 11 op/s 2024-09-12T03:02:22.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:22.294+0000 7f2edab1c640 1 -- 172.21.15.5:0/195136160 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ed4101430 msgr2=0x7f2ed410d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:22.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:22.294+0000 7f2edab1c640 1 --2- 172.21.15.5:0/195136160 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ed4101430 0x7f2ed410d960 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc009be0 tx=0x7f2ecc02a6c0 comp rx=0 tx=0).stop 2024-09-12T03:02:22.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:22.294+0000 7f2edab1c640 1 -- 172.21.15.5:0/195136160 shutdown_connections 2024-09-12T03:02:22.296 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:22.294+0000 7f2edab1c640 1 --2- 172.21.15.5:0/195136160 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ed4100a90 0x7f2ed4100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:22.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:22.294+0000 7f2edab1c640 1 --2- 172.21.15.5:0/195136160 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ed4101430 0x7f2ed410d960 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:22.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:22.294+0000 7f2edab1c640 1 --2- 172.21.15.5:0/195136160 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ed4106a90 0x7f2ed4106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:22.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:22.294+0000 7f2edab1c640 1 -- 172.21.15.5:0/195136160 >> 172.21.15.5:0/195136160 conn(0x7f2ed40fc7d0 msgr2=0x7f2ed40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:22.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:22.294+0000 7f2edab1c640 1 -- 172.21.15.5:0/195136160 shutdown_connections 2024-09-12T03:02:22.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:22.294+0000 7f2edab1c640 1 -- 172.21.15.5:0/195136160 wait complete. 2024-09-12T03:02:22.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:22.295+0000 7f2edab1c640 1 Processor -- start 2024-09-12T03:02:22.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:22.295+0000 7f2edab1c640 1 -- start start 2024-09-12T03:02:22.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:22.295+0000 7f2edab1c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ed4100a90 0x7f2ed4198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:22.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:22.295+0000 7f2edab1c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ed4101430 0x7f2ed4198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:22.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:22.295+0000 7f2edab1c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ed4106a90 0x7f2ed419cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:22.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:22.295+0000 7f2edab1c640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f2ed4111fb0 con 0x7f2ed4101430 2024-09-12T03:02:22.297 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:22.295+0000 7f2edab1c640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f2ed4111e30 con 0x7f2ed4106a90 2024-09-12T03:02:22.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:22.295+0000 7f2edab1c640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f2ed4112130 con 0x7f2ed4100a90 2024-09-12T03:02:22.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:22.295+0000 7f2ed3fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ed4101430 0x7f2ed4198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:22.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:22.295+0000 7f2ed88b8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ed4100a90 0x7f2ed4198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:22.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:22.295+0000 7f2ed88b8640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ed4100a90 0x7f2ed4198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.5:38042/0 (socket says 172.21.15.5:38042) 2024-09-12T03:02:22.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:22.295+0000 7f2ed88b8640 1 -- 172.21.15.5:0/1009804899 learned_addr learned my addr 172.21.15.5:0/1009804899 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:02:22.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:22.295+0000 7f2ed90b9640 1 --2- 172.21.15.5:0/1009804899 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ed4106a90 0x7f2ed419cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:22.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:22.296+0000 7f2ed88b8640 1 -- 172.21.15.5:0/1009804899 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ed4106a90 msgr2=0x7f2ed419cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:22.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:22.296+0000 7f2ed88b8640 1 --2- 172.21.15.5:0/1009804899 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ed4106a90 0x7f2ed419cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:22.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:22.296+0000 7f2ed88b8640 1 -- 172.21.15.5:0/1009804899 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ed4101430 msgr2=0x7f2ed4198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:22.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:22.296+0000 7f2ed88b8640 1 --2- 172.21.15.5:0/1009804899 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ed4101430 0x7f2ed4198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:22.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:22.296+0000 7f2ed88b8640 1 -- 172.21.15.5:0/1009804899 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ed419d3e0 con 0x7f2ed4100a90 2024-09-12T03:02:22.298 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:22.296+0000 7f2ed88b8640 1 --2- 172.21.15.5:0/1009804899 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ed4100a90 0x7f2ed4198430 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2ec800e9c0 tx=0x7f2ec800edc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:22.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:22.296+0000 7f2ed1ffb640 1 -- 172.21.15.5:0/1009804899 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec800cd80 con 0x7f2ed4100a90 2024-09-12T03:02:22.299 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:22.296+0000 7f2ed1ffb640 1 -- 172.21.15.5:0/1009804899 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ec80035f0 con 0x7f2ed4100a90 2024-09-12T03:02:22.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:22.296+0000 7f2edab1c640 1 -- 172.21.15.5:0/1009804899 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ed419d6d0 con 0x7f2ed4100a90 2024-09-12T03:02:22.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:22.297+0000 7f2ed1ffb640 1 -- 172.21.15.5:0/1009804899 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec8010670 con 0x7f2ed4100a90 2024-09-12T03:02:22.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:22.297+0000 7f2edab1c640 1 -- 172.21.15.5:0/1009804899 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ed41a4fb0 con 0x7f2ed4100a90 2024-09-12T03:02:22.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:22.298+0000 7f2edab1c640 1 -- 172.21.15.5:0/1009804899 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ed4100ef0 con 0x7f2ed4100a90 2024-09-12T03:02:22.300 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:22.299+0000 7f2ed1ffb640 1 -- 172.21.15.5:0/1009804899 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f2ec80037a0 con 0x7f2ed4100a90 2024-09-12T03:02:22.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:22.300+0000 7f2ed1ffb640 1 --2- 172.21.15.5:0/1009804899 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f2ec40a2c80 0x7f2ec40a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:22.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:22.300+0000 7f2ed1ffb640 1 -- 172.21.15.5:0/1009804899 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f2ec80cf320 con 0x7f2ed4100a90 2024-09-12T03:02:22.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:22.300+0000 7f2ed3fff640 1 --2- 172.21.15.5:0/1009804899 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f2ec40a2c80 0x7f2ec40a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:22.301 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:22.301+0000 7f2ed3fff640 1 --2- 172.21.15.5:0/1009804899 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f2ec40a2c80 0x7f2ec40a5140 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f2ed40fed00 tx=0x7f2ec0009110 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:22.305 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:22.305+0000 7f2ed1ffb640 1 -- 172.21.15.5:0/1009804899 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2ec809c230 con 0x7f2ed4100a90 2024-09-12T03:02:22.344 INFO:tasks.workunit.client.0.smithi005.stdout:40000 mapwrite 0x1becad thru 0x1c6a92 (0x7de6 bytes) 2024-09-12T03:02:22.581 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-12T03:02:22.582 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":30,"btime":"2024-09-12T03:01:58:481122+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/4135232521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":4135232521},{"type":"v1","addr":"172.21.15.5:6835","nonce":4135232521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/4290221940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":4290221940},{"type":"v1","addr":"172.21.15.5:6837","nonce":4290221940}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T02:56:16.595562+0000","modified":"2024-09-12T03:01:57.686450+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24502,"mds_1":24487,"mds_2":24481,"mds_3":24479,"mds_4":24458},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/26447341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":26447341},{"type":"v1","addr":"172.21.15.5:6839","nonce":26447341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2393443295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2393443295},{"type":"v1","addr":"172.21.15.5:6841","nonce":2393443295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6833/3686970137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3686970137},{"type":"v1","addr":"172.21.15.146:6833","nonce":3686970137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6833/568512209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":568512209},{"type":"v1","addr":"172.21.15.148:6833","nonce":568512209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6835/2540740420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2540740420},{"type":"v1","addr":"172.21.15.146:6835","nonce":2540740420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.146:6837/1673370937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":1673370937},{"type":"v1","addr":"172.21.15.146:6837","nonce":1673370937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":3,"incarnation":22,"state":"up:active","state_seq":6,"addr":"172.21.15.146:6839/3036767228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":3036767228},{"type":"v1","addr":"172.21.15.146:6839","nonce":3036767228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":2,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.148:6835/4195086427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":4195086427},{"type":"v1","addr":"172.21.15.148:6835","nonce":4195086427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.148:6837/2606873922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2606873922},{"type":"v1","addr":"172.21.15.148:6837","nonce":2606873922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6839/2317600000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":2317600000},{"type":"v1","addr":"172.21.15.148:6839","nonce":2317600000}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24502,"qdb_cluster":[24502,24487,24481,24479,24458]},"id":1}]} 2024-09-12T03:02:22.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:22.580+0000 7f2edab1c640 1 -- 172.21.15.5:0/1009804899 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2ed41a5290 con 0x7f2ed4100a90 2024-09-12T03:02:22.582 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:22.581+0000 7f2ed1ffb640 1 -- 172.21.15.5:0/1009804899 <== mon.1 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11967 (secure 0 0 0) 0x7f2ec809bb30 con 0x7f2ed4100a90 2024-09-12T03:02:22.582 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 30 2024-09-12T03:02:22.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:22.583+0000 7f2edab1c640 1 -- 172.21.15.5:0/1009804899 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f2ec40a2c80 msgr2=0x7f2ec40a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:22.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:22.583+0000 7f2edab1c640 1 --2- 172.21.15.5:0/1009804899 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f2ec40a2c80 0x7f2ec40a5140 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f2ed40fed00 tx=0x7f2ec0009110 comp rx=0 tx=0).stop 2024-09-12T03:02:22.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:22.583+0000 7f2edab1c640 1 -- 172.21.15.5:0/1009804899 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ed4100a90 msgr2=0x7f2ed4198430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:22.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:22.583+0000 7f2edab1c640 1 --2- 172.21.15.5:0/1009804899 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ed4100a90 0x7f2ed4198430 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2ec800e9c0 tx=0x7f2ec800edc0 comp rx=0 tx=0).stop 2024-09-12T03:02:22.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:22.583+0000 7f2edab1c640 1 -- 172.21.15.5:0/1009804899 shutdown_connections 2024-09-12T03:02:22.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:22.583+0000 7f2edab1c640 1 --2- 172.21.15.5:0/1009804899 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f2ec40a2c80 0x7f2ec40a5140 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:22.584 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:22.583+0000 7f2edab1c640 1 --2- 172.21.15.5:0/1009804899 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2ed4101430 0x7f2ed4198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:22.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:22.583+0000 7f2edab1c640 1 --2- 172.21.15.5:0/1009804899 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2ed4106a90 0x7f2ed419cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:22.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:22.583+0000 7f2edab1c640 1 --2- 172.21.15.5:0/1009804899 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2ed4100a90 0x7f2ed4198430 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:22.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:22.583+0000 7f2edab1c640 1 -- 172.21.15.5:0/1009804899 >> 172.21.15.5:0/1009804899 conn(0x7f2ed40fc7d0 msgr2=0x7f2ed40fce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:22.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:22.584+0000 7f2edab1c640 1 -- 172.21.15.5:0/1009804899 shutdown_connections 2024-09-12T03:02:22.585 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:22.584+0000 7f2edab1c640 1 -- 172.21.15.5:0/1009804899 wait complete. 2024-09-12T03:02:23.388 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24502 quiesce db / --timeout 90 --expiration 140.15 --await-for 45.0' 2024-09-12T03:02:23.388 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24502 quiesce db / --timeout 90 --expiration 140.15 --await-for 45.0 2024-09-12T03:02:23.594 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:02:23 smithi005 ceph-mon[31197]: pgmap v459: 97 pgs: 97 active+clean; 114 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 16 KiB/s wr, 10 op/s 2024-09-12T03:02:23.594 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:02:23 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/1009804899' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:02:23.707 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:02:23.915 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:02:23 smithi148 ceph-mon[35090]: pgmap v459: 97 pgs: 97 active+clean; 114 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 16 KiB/s wr, 10 op/s 2024-09-12T03:02:23.915 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:02:23 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/1009804899' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:02:23.956 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:02:23 smithi146 ceph-mon[35011]: pgmap v459: 97 pgs: 97 active+clean; 114 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 16 KiB/s wr, 10 op/s 2024-09-12T03:02:23.956 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:02:23 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/1009804899' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:02:24.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.305+0000 7feb04ebc640 1 -- 172.21.15.5:0/1597710088 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7feb00100100 msgr2=0x7feb00100520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:24.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.305+0000 7feb04ebc640 1 --2- 172.21.15.5:0/1597710088 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7feb00100100 0x7feb00100520 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7feae8009860 tx=0x7feae802a340 comp rx=0 tx=0).stop 2024-09-12T03:02:24.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.305+0000 7feb04ebc640 1 -- 172.21.15.5:0/1597710088 shutdown_connections 2024-09-12T03:02:24.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.305+0000 7feb04ebc640 1 --2- 172.21.15.5:0/1597710088 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb00100c80 0x7feb00110890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:24.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.305+0000 7feb04ebc640 1 --2- 172.21.15.5:0/1597710088 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7feb00100100 0x7feb00100520 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:24.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.305+0000 7feb04ebc640 1 --2- 172.21.15.5:0/1597710088 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb00102a50 0x7feb00102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:24.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.305+0000 7feb04ebc640 1 -- 172.21.15.5:0/1597710088 >> 172.21.15.5:0/1597710088 conn(0x7feb00077230 msgr2=0x7feb000fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:24.307 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.305+0000 7feb04ebc640 1 -- 172.21.15.5:0/1597710088 shutdown_connections 2024-09-12T03:02:24.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.305+0000 7feb04ebc640 1 -- 172.21.15.5:0/1597710088 wait complete. 2024-09-12T03:02:24.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.306+0000 7feb04ebc640 1 Processor -- start 2024-09-12T03:02:24.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.306+0000 7feb04ebc640 1 -- start start 2024-09-12T03:02:24.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.307+0000 7feb04ebc640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7feb00100100 0x7feb0019a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:24.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.307+0000 7feb04ebc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb00100c80 0x7feb0019abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:24.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.307+0000 7feb04ebc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb00102a50 0x7feb0019ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:24.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.307+0000 7feb04ebc640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7feb00114100 con 0x7feb00102a50 2024-09-12T03:02:24.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.307+0000 7feb04ebc640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7feb00113f80 con 0x7feb00100100 2024-09-12T03:02:24.308 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.307+0000 7feb04ebc640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7feb00114280 con 0x7feb00100c80 2024-09-12T03:02:24.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.307+0000 7feafed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb00102a50 0x7feb0019ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:24.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.307+0000 7feafe59c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7feb00100100 0x7feb0019a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:24.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.307+0000 7feaf7fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb00100c80 0x7feb0019abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:24.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.307+0000 7feafe59c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7feb00100100 0x7feb0019a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.5:53328/0 (socket says 172.21.15.5:53328) 2024-09-12T03:02:24.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.307+0000 7feafed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb00102a50 0x7feb0019ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35908/0 (socket says 172.21.15.5:35908) 2024-09-12T03:02:24.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.307+0000 7feaf7fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb00100c80 0x7feb0019abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.5:33314/0 (socket says 172.21.15.5:33314) 2024-09-12T03:02:24.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.307+0000 7feafe59c640 1 -- 172.21.15.5:0/1086275297 learned_addr learned my addr 172.21.15.5:0/1086275297 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:02:24.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.308+0000 7feafe59c640 1 -- 172.21.15.5:0/1086275297 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb00100c80 msgr2=0x7feb0019abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:24.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.308+0000 7feafe59c640 1 --2- 172.21.15.5:0/1086275297 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb00100c80 0x7feb0019abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:24.310 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.308+0000 7feafe59c640 1 -- 172.21.15.5:0/1086275297 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb00102a50 msgr2=0x7feb0019ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:24.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.308+0000 7feafe59c640 1 --2- 172.21.15.5:0/1086275297 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb00102a50 0x7feb0019ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:24.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.308+0000 7feafe59c640 1 -- 172.21.15.5:0/1086275297 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb0019f650 con 0x7feb00100100 2024-09-12T03:02:24.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.308+0000 7feaf7fff640 1 --2- 172.21.15.5:0/1086275297 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb00100c80 0x7feb0019abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:02:24.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.308+0000 7feafed9d640 1 --2- 172.21.15.5:0/1086275297 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb00102a50 0x7feb0019ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:02:24.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.308+0000 7feafe59c640 1 --2- 172.21.15.5:0/1086275297 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7feb00100100 0x7feb0019a6a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7feaf000e9f0 tx=0x7feaf000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:24.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.308+0000 7feaf77fe640 1 -- 172.21.15.5:0/1086275297 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf000cd80 con 0x7feb00100100 2024-09-12T03:02:24.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.308+0000 7feaf77fe640 1 -- 172.21.15.5:0/1086275297 <== mon.2 v2:172.21.15.146:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feaf0010910 con 0x7feb00100100 2024-09-12T03:02:24.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.308+0000 7feb04ebc640 1 -- 172.21.15.5:0/1086275297 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb0019f940 con 0x7feb00100100 2024-09-12T03:02:24.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.308+0000 7feb04ebc640 1 -- 172.21.15.5:0/1086275297 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb001a7220 con 0x7feb00100100 2024-09-12T03:02:24.311 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.308+0000 7feaf77fe640 1 -- 172.21.15.5:0/1086275297 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf00188f0 con 0x7feb00100100 2024-09-12T03:02:24.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.311+0000 7feaf77fe640 1 -- 172.21.15.5:0/1086275297 <== mon.2 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7feaf0010ac0 con 0x7feb00100100 2024-09-12T03:02:24.312 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.311+0000 7feb07295740 1 Processor -- start 2024-09-12T03:02:24.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.311+0000 7feb07295740 1 -- start start 2024-09-12T03:02:24.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.312+0000 7feb07295740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5614db526850 0x5614db526c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:24.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.312+0000 7feb07295740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5614db527170 0x5614db5314f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:24.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.312+0000 7feb07295740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614db531a30 0x5614db533e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:24.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.312+0000 7feb07295740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5614db48b160 con 0x5614db531a30 2024-09-12T03:02:24.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.312+0000 7feb07295740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5614db432720 con 0x5614db527170 2024-09-12T03:02:24.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.312+0000 7feb07295740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5614db487120 con 0x5614db526850 2024-09-12T03:02:24.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.312+0000 7feaf77fe640 1 --2- 172.21.15.5:0/1086275297 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7feaec0a2b80 0x7feaec0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:24.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.312+0000 7feaf7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614db531a30 0x5614db533e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:24.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.312+0000 7feaf7fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614db531a30 0x5614db533e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35912/0 (socket says 172.21.15.5:35912) 2024-09-12T03:02:24.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.312+0000 7feaf7fff640 1 -- 172.21.15.5:0/3076926111 learned_addr learned my addr 172.21.15.5:0/3076926111 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:02:24.313 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.312+0000 7feafe59c640 1 --2- 172.21.15.5:0/3076926111 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5614db527170 0x5614db5314f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:24.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.312+0000 7feaf77fe640 1 -- 172.21.15.5:0/1086275297 <== mon.2 v2:172.21.15.146:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7feaf00cf300 con 0x7feb00100100 2024-09-12T03:02:24.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.312+0000 7feaf7fff640 1 -- 172.21.15.5:0/3076926111 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5614db526850 msgr2=0x5614db526c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:24.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.312+0000 7feafed9d640 1 --2- 172.21.15.5:0/3076926111 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5614db526850 0x5614db526c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:24.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.312+0000 7feaf7fff640 1 --2- 172.21.15.5:0/3076926111 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5614db526850 0x5614db526c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:24.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.312+0000 7feaf7fff640 1 -- 172.21.15.5:0/3076926111 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5614db527170 msgr2=0x5614db5314f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:24.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.312+0000 7feaf7fff640 1 --2- 172.21.15.5:0/3076926111 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5614db527170 0x5614db5314f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:24.314 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.312+0000 7feaf7fff640 1 -- 172.21.15.5:0/3076926111 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614db1fcba0 con 0x5614db531a30 2024-09-12T03:02:24.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.313+0000 7feaf7fff640 1 --2- 172.21.15.5:0/3076926111 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614db531a30 0x5614db533e20 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7feae802a860 tx=0x7feae8009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28d40cd0cfdb12f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:24.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.313+0000 7feafed9d640 1 --2- 172.21.15.5:0/1086275297 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7feaec0a2b80 0x7feaec0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:24.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.313+0000 7feaf4ff9640 1 -- 172.21.15.5:0/3076926111 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feae8037040 con 0x5614db531a30 2024-09-12T03:02:24.315 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.313+0000 7feaf4ff9640 1 -- 172.21.15.5:0/3076926111 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feae8004230 con 0x5614db531a30 2024-09-12T03:02:24.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.314+0000 7feaf4ff9640 1 -- 172.21.15.5:0/3076926111 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feae8003c80 con 0x5614db531a30 2024-09-12T03:02:24.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.314+0000 7feb07295740 1 -- 172.21.15.5:0/3076926111 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614db531a30 msgr2=0x5614db533e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:24.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.314+0000 7feb07295740 1 --2- 172.21.15.5:0/3076926111 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614db531a30 0x5614db533e20 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7feae802a860 tx=0x7feae8009510 comp rx=0 tx=0).stop 2024-09-12T03:02:24.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.314+0000 7feafed9d640 1 --2- 172.21.15.5:0/1086275297 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7feaec0a2b80 0x7feaec0a5040 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7feaf8005fd0 tx=0x7feaf8004a40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:24.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.314+0000 7feb07295740 1 -- 172.21.15.5:0/3076926111 shutdown_connections 2024-09-12T03:02:24.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.314+0000 7feb07295740 1 --2- 172.21.15.5:0/3076926111 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614db531a30 0x5614db533e20 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:24.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.314+0000 7feb07295740 1 --2- 172.21.15.5:0/3076926111 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5614db527170 0x5614db5314f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:24.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.314+0000 7feb07295740 1 --2- 172.21.15.5:0/3076926111 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5614db526850 0x5614db526c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:24.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.314+0000 7feb07295740 1 -- 172.21.15.5:0/3076926111 >> 172.21.15.5:0/3076926111 conn(0x5614db522de0 msgr2=0x5614db5231c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:24.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.315+0000 7feb07295740 1 -- 172.21.15.5:0/3076926111 shutdown_connections 2024-09-12T03:02:24.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.315+0000 7feb07295740 1 -- 172.21.15.5:0/3076926111 wait complete. 2024-09-12T03:02:24.316 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.315+0000 7feb07295740 1 -- start start 2024-09-12T03:02:24.317 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.315+0000 7feb07295740 1 Processor -- start 2024-09-12T03:02:24.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.315+0000 7feb07295740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5614db526850 0x5614db52e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:24.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.315+0000 7feb07295740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5614db527170 0x5614db52eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:24.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.315+0000 7feb07295740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614db531a30 0x5614db52f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:24.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.315+0000 7feb07295740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5614db534650 con 0x5614db531a30 2024-09-12T03:02:24.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.315+0000 7feb07295740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5614db413900 con 0x5614db526850 2024-09-12T03:02:24.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.315+0000 7feb07295740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5614db5347d0 con 0x5614db527170 2024-09-12T03:02:24.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.315+0000 7feb07295740 10 client.0 _ll_register_callbacks cb 0x5614db411780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:02:24.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.316+0000 7feafe59c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5614db526850 0x5614db52e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:24.319 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.316+0000 7feafed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614db531a30 0x5614db52f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:24.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.316+0000 7feafe59c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5614db526850 0x5614db52e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.5:53334/0 (socket says 172.21.15.5:53334) 2024-09-12T03:02:24.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.316+0000 7feafe59c640 1 -- 172.21.15.5:0/4211762899 learned_addr learned my addr 172.21.15.5:0/4211762899 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:02:24.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.316+0000 7feaf7fff640 1 --2- 172.21.15.5:0/4211762899 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5614db527170 0x5614db52eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:24.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.316+0000 7feafe59c640 1 -- 172.21.15.5:0/4211762899 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5614db527170 msgr2=0x5614db52eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:24.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.316+0000 7feafe59c640 1 --2- 172.21.15.5:0/4211762899 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5614db527170 0x5614db52eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:24.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.316+0000 7feafe59c640 1 -- 172.21.15.5:0/4211762899 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614db531a30 msgr2=0x5614db52f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:24.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.316+0000 7feafe59c640 1 --2- 172.21.15.5:0/4211762899 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614db531a30 0x5614db52f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:24.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.316+0000 7feafe59c640 1 -- 172.21.15.5:0/4211762899 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614db453c50 con 0x5614db526850 2024-09-12T03:02:24.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.316+0000 7feafe59c640 1 --2- 172.21.15.5:0/4211762899 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5614db526850 0x5614db52e860 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7feaf009b920 tx=0x7feaf001f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2583fdd5e30c49fe server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:24.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.316+0000 7feae5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2024-09-12T03:02:24.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.317+0000 7feae5ffb640 1 -- 172.21.15.5:0/4211762899 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf0092060 con 0x5614db526850 2024-09-12T03:02:24.320 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.317+0000 7feae5ffb640 1 -- 172.21.15.5:0/4211762899 <== mon.2 v2:172.21.15.146:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7feaf009b070 con 0x5614db526850 2024-09-12T03:02:24.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.317+0000 7feb07295740 1 -- 172.21.15.5:0/4211762899 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feae8004230 con 0x5614db526850 2024-09-12T03:02:24.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.317+0000 7feae5ffb640 1 -- 172.21.15.5:0/4211762899 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7feaf0014ed0 con 0x5614db526850 2024-09-12T03:02:24.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.317+0000 7feb07295740 10 client.25781 fetch_fsmap learned FSMap version 31 2024-09-12T03:02:24.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.317+0000 7feb07295740 1 -- 172.21.15.5:0/4211762899 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5614db5402e0 con 0x5614db526850 2024-09-12T03:02:24.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.317+0000 7feae5ffb640 1 -- 172.21.15.5:0/4211762899 <== mon.2 v2:172.21.15.146:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feae8004230 con 0x5614db526850 2024-09-12T03:02:24.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.317+0000 7feae5ffb640 1 -- 172.21.15.5:0/4211762899 <== mon.2 v2:172.21.15.146:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7feaf00ba2a0 con 0x5614db526850 2024-09-12T03:02:24.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.318+0000 7feb07295740 10 client.25781 fetch_fsmap finished waiting for FSMap version 31 2024-09-12T03:02:24.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.318+0000 7feb07295740 10 client.25781 resolve_mds: validated gid 24502 aka daemon mds.i 2024-09-12T03:02:24.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.318+0000 7feb07295740 20 client.25781 populate_metadata read hostname 'smithi005' 2024-09-12T03:02:24.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.318+0000 7feb07295740 1 --2- 172.21.15.5:0/4211762899 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x5614db5417a0 0x5614db543bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:24.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.318+0000 7feb07295740 4 client.25781 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:02:24.321 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.318+0000 7feb07295740 1 -- 172.21.15.5:0/4211762899 --> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5614db498890 con 0x5614db5417a0 2024-09-12T03:02:24.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.318+0000 7feaf7fff640 1 --2- 172.21.15.5:0/4211762899 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x5614db5417a0 0x5614db543bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:24.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.319+0000 7feae5ffb640 10 client.25781 ms_handle_connect on v2:172.21.15.148:6838/2317600000 2024-09-12T03:02:24.322 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.319+0000 7feaf7fff640 1 --2- 172.21.15.5:0/4211762899 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x5614db5417a0 0x5614db543bf0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=386dfd32185c83a2 server_cookie=8894af9210d4cdd8 in_seq=0 out_seq=0 2024-09-12T03:02:24.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.319+0000 7feae5ffb640 1 -- 172.21.15.5:0/4211762899 <== mds.0 v2:172.21.15.148:6838/2317600000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5614db498890 con 0x5614db5417a0 2024-09-12T03:02:24.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.319+0000 7feae5ffb640 10 client.25781 handle_command_reply: tid=0 2024-09-12T03:02:24.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.319+0000 7feb07295740 1 client.25781 shutdown 2024-09-12T03:02:24.323 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.319+0000 7feb07295740 1 -- 172.21.15.5:0/4211762899 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5614db526850 msgr2=0x5614db52e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:24.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.319+0000 7feb07295740 1 --2- 172.21.15.5:0/4211762899 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5614db526850 0x5614db52e860 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7feaf009b920 tx=0x7feaf001f000 comp rx=0 tx=0).stop 2024-09-12T03:02:24.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.320+0000 7feb07295740 1 -- 172.21.15.5:0/4211762899 shutdown_connections 2024-09-12T03:02:24.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.320+0000 7feb07295740 1 --2- 172.21.15.5:0/4211762899 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x5614db5417a0 0x5614db543bf0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:24.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.320+0000 7feae5ffb640 0 client.25781 ms_handle_reset on v2:172.21.15.148:6838/2317600000 2024-09-12T03:02:24.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.320+0000 7feb07295740 1 --2- 172.21.15.5:0/4211762899 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614db531a30 0x5614db52f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:24.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.320+0000 7feb07295740 1 --2- 172.21.15.5:0/4211762899 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5614db527170 0x5614db52eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:24.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.320+0000 7feb07295740 1 --2- 172.21.15.5:0/4211762899 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5614db526850 0x5614db52e860 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:24.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.320+0000 7feb07295740 1 -- 172.21.15.5:0/4211762899 >> 172.21.15.5:0/4211762899 conn(0x5614db522de0 msgr2=0x5614db528ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:24.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.320+0000 7feb07295740 1 -- 172.21.15.5:0/4211762899 shutdown_connections 2024-09-12T03:02:24.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.320+0000 7feb07295740 1 -- 172.21.15.5:0/4211762899 wait complete. 2024-09-12T03:02:24.324 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.320+0000 7feb07295740 20 client.25781 trim_cache size 0 max 16384 2024-09-12T03:02:24.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.343+0000 7feb07295740 1 Processor -- start 2024-09-12T03:02:24.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.343+0000 7feb07295740 1 -- start start 2024-09-12T03:02:24.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.343+0000 7feb07295740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5614db526850 0x5614db5292b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:24.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.343+0000 7feb07295740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5614db527170 0x5614db52ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:24.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.343+0000 7feb07295740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614db531a30 0x5614db5494e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:24.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.343+0000 7feb07295740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7feb001ab330 con 0x5614db531a30 2024-09-12T03:02:24.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.343+0000 7feb07295740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7feb001ace70 con 0x5614db527170 2024-09-12T03:02:24.344 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.343+0000 7feb07295740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7feb001aba00 con 0x5614db526850 2024-09-12T03:02:24.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.343+0000 7feafed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614db531a30 0x5614db5494e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:24.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.343+0000 7feaf7fff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5614db527170 0x5614db52ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:24.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.343+0000 7feafed9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614db531a30 0x5614db5494e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35928/0 (socket says 172.21.15.5:35928) 2024-09-12T03:02:24.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.343+0000 7feaf7fff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5614db527170 0x5614db52ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.5:53342/0 (socket says 172.21.15.5:53342) 2024-09-12T03:02:24.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.343+0000 7feafed9d640 1 -- 172.21.15.5:0/468789314 learned_addr learned my addr 172.21.15.5:0/468789314 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:02:24.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.343+0000 7feafe59c640 1 --2- 172.21.15.5:0/468789314 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5614db526850 0x5614db5292b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:24.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.343+0000 7feaf7fff640 1 -- 172.21.15.5:0/468789314 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5614db526850 msgr2=0x5614db5292b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:24.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.343+0000 7feaf7fff640 1 --2- 172.21.15.5:0/468789314 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5614db526850 0x5614db5292b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:24.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.343+0000 7feaf7fff640 1 -- 172.21.15.5:0/468789314 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614db531a30 msgr2=0x5614db5494e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:24.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.343+0000 7feaf7fff640 1 --2- 172.21.15.5:0/468789314 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614db531a30 0x5614db5494e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:24.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.343+0000 7feaf7fff640 1 -- 172.21.15.5:0/468789314 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614db526fa0 con 0x5614db527170 2024-09-12T03:02:24.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.343+0000 7feafed9d640 1 --2- 172.21.15.5:0/468789314 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614db531a30 0x5614db5494e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:02:24.345 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.344+0000 7feaf7fff640 1 --2- 172.21.15.5:0/468789314 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5614db527170 0x5614db52ef30 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7feae8009830 tx=0x7feae80058d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf1852cfcf6bc50b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:24.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.344+0000 7feae7fff640 1 -- 172.21.15.5:0/468789314 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feae8037040 con 0x5614db527170 2024-09-12T03:02:24.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.344+0000 7feae7fff640 1 -- 172.21.15.5:0/468789314 <== mon.2 v2:172.21.15.146:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feae8030750 con 0x5614db527170 2024-09-12T03:02:24.346 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.344+0000 7feae7fff640 1 -- 172.21.15.5:0/468789314 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feae803d440 con 0x5614db527170 2024-09-12T03:02:24.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.345+0000 7feb07295740 1 -- 172.21.15.5:0/468789314 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5614db527170 msgr2=0x5614db52ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:24.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.345+0000 7feb07295740 1 --2- 172.21.15.5:0/468789314 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5614db527170 0x5614db52ef30 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7feae8009830 tx=0x7feae80058d0 comp rx=0 tx=0).stop 2024-09-12T03:02:24.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.345+0000 7feb07295740 1 -- 172.21.15.5:0/468789314 shutdown_connections 2024-09-12T03:02:24.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.345+0000 7feb07295740 1 --2- 172.21.15.5:0/468789314 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614db531a30 0x5614db5494e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:24.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.345+0000 7feb07295740 1 --2- 172.21.15.5:0/468789314 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5614db527170 0x5614db52ef30 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:24.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.345+0000 7feb07295740 1 --2- 172.21.15.5:0/468789314 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5614db526850 0x5614db5292b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:24.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.345+0000 7feb07295740 1 -- 172.21.15.5:0/468789314 >> 172.21.15.5:0/468789314 conn(0x5614db522de0 msgr2=0x5614db52e820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:24.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.345+0000 7feb07295740 1 -- 172.21.15.5:0/468789314 shutdown_connections 2024-09-12T03:02:24.349 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.345+0000 7feb07295740 1 -- 172.21.15.5:0/468789314 wait complete. 2024-09-12T03:02:24.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.345+0000 7feb07295740 1 -- start start 2024-09-12T03:02:24.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.345+0000 7feb07295740 1 Processor -- start 2024-09-12T03:02:24.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.345+0000 7feb07295740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614db526850 0x5614db535920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:24.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.345+0000 7feb07295740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5614db527170 0x5614db535e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:24.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.345+0000 7feb07295740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5614db531a30 0x5614db5363a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:24.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.345+0000 7feb07295740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7feb001ac7a0 con 0x5614db526850 2024-09-12T03:02:24.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.345+0000 7feb07295740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7feb001ac0d0 con 0x5614db531a30 2024-09-12T03:02:24.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.345+0000 7feb07295740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7feb001a9330 con 0x5614db527170 2024-09-12T03:02:24.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.345+0000 7feb07295740 10 client.0 _ll_register_callbacks cb 0x7feaf000e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:02:24.350 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.346+0000 7feafe59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614db526850 0x5614db535920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:24.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.346+0000 7feaf7fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5614db527170 0x5614db535e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:24.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.346+0000 7feafe59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614db526850 0x5614db535920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35944/0 (socket says 172.21.15.5:35944) 2024-09-12T03:02:24.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.346+0000 7feafed9d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5614db531a30 0x5614db5363a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:24.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.346+0000 7feafe59c640 1 -- 172.21.15.5:0/273501699 learned_addr learned my addr 172.21.15.5:0/273501699 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:02:24.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.346+0000 7feafe59c640 1 -- 172.21.15.5:0/273501699 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5614db527170 msgr2=0x5614db535e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:24.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.346+0000 7feafe59c640 1 --2- 172.21.15.5:0/273501699 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5614db527170 0x5614db535e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:24.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.346+0000 7feafe59c640 1 -- 172.21.15.5:0/273501699 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5614db531a30 msgr2=0x5614db5363a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:24.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.346+0000 7feafe59c640 1 --2- 172.21.15.5:0/273501699 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5614db531a30 0x5614db5363a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:24.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.346+0000 7feafe59c640 1 -- 172.21.15.5:0/273501699 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614db5ab650 con 0x5614db526850 2024-09-12T03:02:24.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.346+0000 7feaf7fff640 1 --2- 172.21.15.5:0/273501699 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5614db527170 0x5614db535e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:02:24.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.346+0000 7feafe59c640 1 --2- 172.21.15.5:0/273501699 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614db526850 0x5614db535920 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7feaf009bbb0 tx=0x7feaf0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=683d459a7c55e154 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:24.351 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.346+0000 7feae5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T03:02:24.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.346+0000 7feae5ffb640 1 -- 172.21.15.5:0/273501699 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf007f060 con 0x5614db526850 2024-09-12T03:02:24.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.346+0000 7feae5ffb640 1 -- 172.21.15.5:0/273501699 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7feaf0010430 con 0x5614db526850 2024-09-12T03:02:24.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.346+0000 7feb07295740 1 -- 172.21.15.5:0/273501699 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feae8030750 con 0x5614db526850 2024-09-12T03:02:24.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.347+0000 7feae5ffb640 1 -- 172.21.15.5:0/273501699 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7feaf00a47e0 con 0x5614db526850 2024-09-12T03:02:24.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.347+0000 7feb07295740 10 client.16392 fetch_fsmap learned FSMap version 31 2024-09-12T03:02:24.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.347+0000 7feb07295740 1 -- 172.21.15.5:0/273501699 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5614db5ac8d0 con 0x5614db526850 2024-09-12T03:02:24.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.347+0000 7feae5ffb640 1 -- 172.21.15.5:0/273501699 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feae8030750 con 0x5614db526850 2024-09-12T03:02:24.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.347+0000 7feae5ffb640 1 -- 172.21.15.5:0/273501699 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7feaf0018a90 con 0x5614db526850 2024-09-12T03:02:24.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.347+0000 7feb07295740 10 client.16392 fetch_fsmap finished waiting for FSMap version 31 2024-09-12T03:02:24.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.347+0000 7feb07295740 10 client.16392 resolve_mds: validated gid 24502 aka daemon mds.i 2024-09-12T03:02:24.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.347+0000 7feb07295740 20 client.16392 populate_metadata read hostname 'smithi005' 2024-09-12T03:02:24.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.347+0000 7feb07295740 1 --2- 172.21.15.5:0/273501699 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x5614db5ace80 0x5614db5af270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:24.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.347+0000 7feb07295740 4 client.16392 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.15, "await_for": 45.0}] 2024-09-12T03:02:24.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.347+0000 7feb07295740 1 -- 172.21.15.5:0/273501699 --> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.15, "await_for": 45.0}) -- 0x5614db492be0 con 0x5614db5ace80 2024-09-12T03:02:24.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.347+0000 7feaf7fff640 1 --2- 172.21.15.5:0/273501699 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x5614db5ace80 0x5614db5af270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:24.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.348+0000 7feae5ffb640 10 client.16392 ms_handle_connect on v2:172.21.15.148:6838/2317600000 2024-09-12T03:02:24.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:24.348+0000 7feaf7fff640 1 --2- 172.21.15.5:0/273501699 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x5614db5ace80 0x5614db5af270 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ebe0819c913c50d3 server_cookie=bf669267d0945b95 in_seq=0 out_seq=0 2024-09-12T03:02:24.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:02:24 smithi005 ceph-mon[31197]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:02:24.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:02:24 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:02:24.915 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:02:24 smithi148 ceph-mon[35090]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:02:24.915 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:02:24 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:02:24.956 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:02:24 smithi146 ceph-mon[35011]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:02:24.956 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:02:24 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:02:25.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:25.281+0000 7feae5ffb640 1 -- 172.21.15.5:0/273501699 <== mds.0 v2:172.21.15.148:6838/2317600000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5614db492be0 con 0x5614db5ace80 2024-09-12T03:02:25.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:25.281+0000 7feae5ffb640 10 client.16392 handle_command_reply: tid=0 2024-09-12T03:02:25.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:25.281+0000 7feb07295740 1 client.16392 shutdown 2024-09-12T03:02:25.281 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:25.281+0000 7feb07295740 1 -- 172.21.15.5:0/273501699 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614db526850 msgr2=0x5614db535920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:25.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:25.281+0000 7feb07295740 1 --2- 172.21.15.5:0/273501699 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614db526850 0x5614db535920 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7feaf009bbb0 tx=0x7feaf0000bc0 comp rx=0 tx=0).stop 2024-09-12T03:02:25.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:25.281+0000 7feb07295740 1 -- 172.21.15.5:0/273501699 shutdown_connections 2024-09-12T03:02:25.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:25.281+0000 7feb07295740 1 --2- 172.21.15.5:0/273501699 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5614db531a30 0x5614db5363a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:25.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:25.281+0000 7feb07295740 1 --2- 172.21.15.5:0/273501699 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5614db527170 0x5614db535e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:25.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:25.281+0000 7feb07295740 1 --2- 172.21.15.5:0/273501699 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x5614db5ace80 0x5614db5af270 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:25.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:25.281+0000 7feb07295740 1 --2- 172.21.15.5:0/273501699 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614db526850 0x5614db535920 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:25.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:25.281+0000 7feae5ffb640 0 client.16392 ms_handle_reset on v2:172.21.15.148:6838/2317600000 2024-09-12T03:02:25.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:25.281+0000 7feb07295740 1 -- 172.21.15.5:0/273501699 >> 172.21.15.5:0/273501699 conn(0x5614db522de0 msgr2=0x5614db53d0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:25.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:25.281+0000 7feb07295740 1 -- 172.21.15.5:0/273501699 shutdown_connections 2024-09-12T03:02:25.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:25.281+0000 7feb07295740 1 -- 172.21.15.5:0/273501699 wait complete. 2024-09-12T03:02:25.282 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:25.281+0000 7feb07295740 20 client.16392 trim_cache size 0 max 16384 2024-09-12T03:02:25.283 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-12T03:02:25.283 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 31, 2024-09-12T03:02:25.283 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24502, 2024-09-12T03:02:25.283 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 18, 2024-09-12T03:02:25.283 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-09-12T03:02:25.283 INFO:teuthology.orchestra.run.smithi005.stdout: "24601adb": { 2024-09-12T03:02:25.283 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 18, 2024-09-12T03:02:25.283 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-09-12T03:02:25.283 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-12T03:02:25.283 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-09-12T03:02:25.283 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-12T03:02:25.284 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:02:25.284 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 90.0, 2024-09-12T03:02:25.284 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 140.2, 2024-09-12T03:02:25.284 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-09-12T03:02:25.284 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-09-12T03:02:25.284 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-09-12T03:02:25.284 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-12T03:02:25.284 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "QUIESCED", 2024-09-12T03:02:25.284 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-12T03:02:25.284 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:02:25.284 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:02:25.284 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:02:25.284 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:02:25.284 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:02:25.285 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-12T03:02:25.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:25.281+0000 7feb04ebc640 1 -- 172.21.15.5:0/1086275297 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7feaec0a2b80 msgr2=0x7feaec0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:25.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:25.281+0000 7feb04ebc640 1 --2- 172.21.15.5:0/1086275297 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7feaec0a2b80 0x7feaec0a5040 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7feaf8005fd0 tx=0x7feaf8004a40 comp rx=0 tx=0).stop 2024-09-12T03:02:25.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:25.281+0000 7feb04ebc640 1 -- 172.21.15.5:0/1086275297 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7feb00100100 msgr2=0x7feb0019a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:25.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:25.281+0000 7feb04ebc640 1 --2- 172.21.15.5:0/1086275297 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7feb00100100 0x7feb0019a6a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7feaf000e9f0 tx=0x7feaf000eeb0 comp rx=0 tx=0).stop 2024-09-12T03:02:25.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:25.281+0000 7feb04ebc640 1 -- 172.21.15.5:0/1086275297 shutdown_connections 2024-09-12T03:02:25.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:25.281+0000 7feb04ebc640 1 --2- 172.21.15.5:0/1086275297 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7feaec0a2b80 0x7feaec0a5040 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:25.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:25.281+0000 7feb04ebc640 1 --2- 172.21.15.5:0/1086275297 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb00102a50 0x7feb0019ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:25.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:25.281+0000 7feb04ebc640 1 --2- 172.21.15.5:0/1086275297 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7feb00100c80 0x7feb0019abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:25.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:25.281+0000 7feb04ebc640 1 --2- 172.21.15.5:0/1086275297 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7feb00100100 0x7feb0019a6a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:25.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:25.281+0000 7feb04ebc640 1 -- 172.21.15.5:0/1086275297 >> 172.21.15.5:0/1086275297 conn(0x7feb00077230 msgr2=0x7feb000fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:25.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:25.282+0000 7feb04ebc640 1 -- 172.21.15.5:0/1086275297 shutdown_connections 2024-09-12T03:02:25.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:25.282+0000 7feb04ebc640 1 -- 172.21.15.5:0/1086275297 wait complete. 2024-09-12T03:02:25.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:02:25 smithi005 ceph-mon[31197]: pgmap v460: 97 pgs: 97 active+clean; 114 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 202 KiB/s wr, 37 op/s 2024-09-12T03:02:25.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:02:25 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:02:25.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:02:25 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:02:25.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:02:25 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T03:02:25.915 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:02:25 smithi148 ceph-mon[35090]: pgmap v460: 97 pgs: 97 active+clean; 114 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 202 KiB/s wr, 37 op/s 2024-09-12T03:02:25.915 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:02:25 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:02:25.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:02:25 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:02:25.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:02:25 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T03:02:25.956 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:02:25 smithi146 ceph-mon[35011]: pgmap v460: 97 pgs: 97 active+clean; 114 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 344 KiB/s rd, 202 KiB/s wr, 37 op/s 2024-09-12T03:02:25.956 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:02:25 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:02:25.956 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:02:25 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:02:25.956 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:02:25 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T03:02:26.037 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '24601adb', quiesce took 4.6 seconds. Will release after: 15.549999999999999 2024-09-12T03:02:27.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:02:27 smithi005 ceph-mon[31197]: pgmap v461: 97 pgs: 97 active+clean; 113 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 939 KiB/s wr, 186 op/s 2024-09-12T03:02:27.915 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:02:27 smithi148 ceph-mon[35090]: pgmap v461: 97 pgs: 97 active+clean; 113 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 939 KiB/s wr, 186 op/s 2024-09-12T03:02:27.956 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:02:27 smithi146 ceph-mon[35011]: pgmap v461: 97 pgs: 97 active+clean; 113 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 939 KiB/s wr, 186 op/s 2024-09-12T03:02:29.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:02:29 smithi005 ceph-mon[31197]: pgmap v462: 97 pgs: 97 active+clean; 113 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.1 MiB/s wr, 216 op/s 2024-09-12T03:02:29.915 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:02:29 smithi148 ceph-mon[35090]: pgmap v462: 97 pgs: 97 active+clean; 113 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.1 MiB/s wr, 216 op/s 2024-09-12T03:02:29.956 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:02:29 smithi146 ceph-mon[35011]: pgmap v462: 97 pgs: 97 active+clean; 113 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.1 MiB/s wr, 216 op/s 2024-09-12T03:02:31.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:02:31 smithi005 ceph-mon[31197]: pgmap v463: 97 pgs: 97 active+clean; 113 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.1 MiB/s wr, 217 op/s 2024-09-12T03:02:31.915 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:02:31 smithi148 ceph-mon[35090]: pgmap v463: 97 pgs: 97 active+clean; 113 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.1 MiB/s wr, 217 op/s 2024-09-12T03:02:31.956 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:02:31 smithi146 ceph-mon[35011]: pgmap v463: 97 pgs: 97 active+clean; 113 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.1 MiB/s wr, 217 op/s 2024-09-12T03:02:33.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:02:33 smithi005 ceph-mon[31197]: pgmap v464: 97 pgs: 97 active+clean; 113 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.1 MiB/s wr, 216 op/s 2024-09-12T03:02:33.915 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:02:33 smithi148 ceph-mon[35090]: pgmap v464: 97 pgs: 97 active+clean; 113 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.1 MiB/s wr, 216 op/s 2024-09-12T03:02:33.956 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:02:33 smithi146 ceph-mon[35011]: pgmap v464: 97 pgs: 97 active+clean; 113 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.1 MiB/s wr, 216 op/s 2024-09-12T03:02:35.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:02:35 smithi005 ceph-mon[31197]: pgmap v465: 97 pgs: 97 active+clean; 113 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.1 MiB/s wr, 217 op/s 2024-09-12T03:02:35.915 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:02:35 smithi148 ceph-mon[35090]: pgmap v465: 97 pgs: 97 active+clean; 113 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.1 MiB/s wr, 217 op/s 2024-09-12T03:02:35.956 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:02:35 smithi146 ceph-mon[35011]: pgmap v465: 97 pgs: 97 active+clean; 113 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.1 MiB/s wr, 217 op/s 2024-09-12T03:02:36.915 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:02:36 smithi148 ceph-mon[35090]: pgmap v466: 97 pgs: 97 active+clean; 113 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 968 KiB/s wr, 191 op/s 2024-09-12T03:02:36.956 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:02:36 smithi146 ceph-mon[35011]: pgmap v466: 97 pgs: 97 active+clean; 113 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 968 KiB/s wr, 191 op/s 2024-09-12T03:02:37.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:02:36 smithi005 ceph-mon[31197]: pgmap v466: 97 pgs: 97 active+clean; 113 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 968 KiB/s wr, 191 op/s 2024-09-12T03:02:38.956 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:02:38 smithi146 ceph-mon[35011]: pgmap v467: 97 pgs: 97 active+clean; 113 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 640 KiB/s rd, 219 KiB/s wr, 41 op/s 2024-09-12T03:02:39.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:02:38 smithi005 ceph-mon[31197]: pgmap v467: 97 pgs: 97 active+clean; 113 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 640 KiB/s rd, 219 KiB/s wr, 41 op/s 2024-09-12T03:02:39.165 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:02:38 smithi148 ceph-mon[35090]: pgmap v467: 97 pgs: 97 active+clean; 113 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 640 KiB/s rd, 219 KiB/s wr, 41 op/s 2024-09-12T03:02:40.956 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:02:40 smithi146 ceph-mon[35011]: pgmap v468: 97 pgs: 97 active+clean; 113 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-12T03:02:41.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:02:40 smithi005 ceph-mon[31197]: pgmap v468: 97 pgs: 97 active+clean; 113 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-12T03:02:41.165 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:02:40 smithi148 ceph-mon[35090]: pgmap v468: 97 pgs: 97 active+clean; 113 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-12T03:02:41.588 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:02:41.861 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:02:42.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:42.338+0000 7f84a01a9640 1 -- 172.21.15.5:0/400244198 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8498106a80 msgr2=0x7f8498106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:42.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:42.338+0000 7f84a01a9640 1 --2- 172.21.15.5:0/400244198 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8498106a80 0x7f8498106e60 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f848c009930 tx=0x7f848c02a420 comp rx=0 tx=0).stop 2024-09-12T03:02:42.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:42.338+0000 7f84a01a9640 1 -- 172.21.15.5:0/400244198 shutdown_connections 2024-09-12T03:02:42.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:42.338+0000 7f84a01a9640 1 --2- 172.21.15.5:0/400244198 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8498101420 0x7f849810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:42.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:42.338+0000 7f84a01a9640 1 --2- 172.21.15.5:0/400244198 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8498100a80 0x7f8498100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:42.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:42.338+0000 7f84a01a9640 1 --2- 172.21.15.5:0/400244198 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8498106a80 0x7f8498106e60 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:42.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:42.338+0000 7f84a01a9640 1 -- 172.21.15.5:0/400244198 >> 172.21.15.5:0/400244198 conn(0x7f84980fc7a0 msgr2=0x7f84980febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:42.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:42.338+0000 7f84a01a9640 1 -- 172.21.15.5:0/400244198 shutdown_connections 2024-09-12T03:02:42.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:42.338+0000 7f84a01a9640 1 -- 172.21.15.5:0/400244198 wait complete. 2024-09-12T03:02:42.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:42.339+0000 7f84a01a9640 1 Processor -- start 2024-09-12T03:02:42.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:42.339+0000 7f84a01a9640 1 -- start start 2024-09-12T03:02:42.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:42.339+0000 7f84a01a9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8498100a80 0x7f84981984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:42.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:42.339+0000 7f84a01a9640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8498101420 0x7f84981989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:42.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:42.339+0000 7f84a01a9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8498106a80 0x7f849819cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:42.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:42.339+0000 7f84a01a9640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8498110e80 con 0x7f8498100a80 2024-09-12T03:02:42.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:42.339+0000 7f84a01a9640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f8498110d00 con 0x7f8498101420 2024-09-12T03:02:42.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:42.339+0000 7f84a01a9640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8498111000 con 0x7f8498106a80 2024-09-12T03:02:42.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:42.339+0000 7f849df45640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8498100a80 0x7f84981984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:42.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:42.339+0000 7f849df45640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8498100a80 0x7f84981984b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56986/0 (socket says 172.21.15.5:56986) 2024-09-12T03:02:42.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:42.340+0000 7f849df45640 1 -- 172.21.15.5:0/270458455 learned_addr learned my addr 172.21.15.5:0/270458455 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:02:42.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:42.340+0000 7f849d744640 1 --2- 172.21.15.5:0/270458455 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8498101420 0x7f84981989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:42.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:42.340+0000 7f849e746640 1 --2- 172.21.15.5:0/270458455 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8498106a80 0x7f849819cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:42.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:42.340+0000 7f849df45640 1 -- 172.21.15.5:0/270458455 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8498106a80 msgr2=0x7f849819cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:42.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:42.340+0000 7f849df45640 1 --2- 172.21.15.5:0/270458455 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8498106a80 0x7f849819cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:42.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:42.340+0000 7f849df45640 1 -- 172.21.15.5:0/270458455 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8498101420 msgr2=0x7f84981989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:42.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:42.340+0000 7f849df45640 1 --2- 172.21.15.5:0/270458455 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8498101420 0x7f84981989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:42.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:42.340+0000 7f849df45640 1 -- 172.21.15.5:0/270458455 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f849819d460 con 0x7f8498100a80 2024-09-12T03:02:42.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:42.340+0000 7f849d744640 1 --2- 172.21.15.5:0/270458455 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8498101420 0x7f84981989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:02:42.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:42.340+0000 7f849df45640 1 --2- 172.21.15.5:0/270458455 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8498100a80 0x7f84981984b0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f848c009230 tx=0x7f848c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:42.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:42.340+0000 7f848affd640 1 -- 172.21.15.5:0/270458455 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f848c03f030 con 0x7f8498100a80 2024-09-12T03:02:42.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:42.340+0000 7f848affd640 1 -- 172.21.15.5:0/270458455 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f848c003ad0 con 0x7f8498100a80 2024-09-12T03:02:42.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:42.340+0000 7f84a01a9640 1 -- 172.21.15.5:0/270458455 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f849819d6f0 con 0x7f8498100a80 2024-09-12T03:02:42.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:42.340+0000 7f848affd640 1 -- 172.21.15.5:0/270458455 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f848c035070 con 0x7f8498100a80 2024-09-12T03:02:42.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:42.340+0000 7f84a01a9640 1 -- 172.21.15.5:0/270458455 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84981a5030 con 0x7f8498100a80 2024-09-12T03:02:42.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:42.341+0000 7f84a01a9640 1 -- 172.21.15.5:0/270458455 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84981021c0 con 0x7f8498100a80 2024-09-12T03:02:42.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:42.342+0000 7f848affd640 1 -- 172.21.15.5:0/270458455 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f848c042030 con 0x7f8498100a80 2024-09-12T03:02:42.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:42.343+0000 7f848affd640 1 --2- 172.21.15.5:0/270458455 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f84840a2b80 0x7f84840a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:42.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:42.343+0000 7f849d744640 1 --2- 172.21.15.5:0/270458455 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f84840a2b80 0x7f84840a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:42.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:42.343+0000 7f848affd640 1 -- 172.21.15.5:0/270458455 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f848c0ef5d0 con 0x7f8498100a80 2024-09-12T03:02:42.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:42.344+0000 7f849d744640 1 --2- 172.21.15.5:0/270458455 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f84840a2b80 0x7f84840a5040 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f84940058c0 tx=0x7f8494005850 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:42.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:42.347+0000 7f848affd640 1 -- 172.21.15.5:0/270458455 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f848c0bc540 con 0x7f8498100a80 2024-09-12T03:02:42.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:42.559+0000 7f84a01a9640 1 -- 172.21.15.5:0/270458455 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8498100ee0 con 0x7f8498100a80 2024-09-12T03:02:42.560 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:42.559+0000 7f848affd640 1 -- 172.21.15.5:0/270458455 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11966 (secure 0 0 0) 0x7f848c0bbe40 con 0x7f8498100a80 2024-09-12T03:02:42.561 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-12T03:02:42.561 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":31,"btime":"2024-09-12T03:02:23:528668+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/4135232521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":4135232521},{"type":"v1","addr":"172.21.15.5:6835","nonce":4135232521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/4290221940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":4290221940},{"type":"v1","addr":"172.21.15.5:6837","nonce":4290221940}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":31,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T02:56:16.595562+0000","modified":"2024-09-12T03:02:22.771568+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24502,"mds_1":24487,"mds_2":24481,"mds_3":24479,"mds_4":24458},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/26447341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":26447341},{"type":"v1","addr":"172.21.15.5:6839","nonce":26447341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2393443295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2393443295},{"type":"v1","addr":"172.21.15.5:6841","nonce":2393443295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6833/3686970137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3686970137},{"type":"v1","addr":"172.21.15.146:6833","nonce":3686970137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6833/568512209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":568512209},{"type":"v1","addr":"172.21.15.148:6833","nonce":568512209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6835/2540740420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2540740420},{"type":"v1","addr":"172.21.15.146:6835","nonce":2540740420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.146:6837/1673370937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":1673370937},{"type":"v1","addr":"172.21.15.146:6837","nonce":1673370937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":3,"incarnation":22,"state":"up:active","state_seq":6,"addr":"172.21.15.146:6839/3036767228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":3036767228},{"type":"v1","addr":"172.21.15.146:6839","nonce":3036767228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":2,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.148:6835/4195086427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":4195086427},{"type":"v1","addr":"172.21.15.148:6835","nonce":4195086427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.148:6837/2606873922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2606873922},{"type":"v1","addr":"172.21.15.148:6837","nonce":2606873922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6839/2317600000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":2317600000},{"type":"v1","addr":"172.21.15.148:6839","nonce":2317600000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24502,"qdb_cluster":[24458,24479,24481,24487,24502]},"id":1}]} 2024-09-12T03:02:42.561 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 31 2024-09-12T03:02:42.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:42.561+0000 7f84a01a9640 1 -- 172.21.15.5:0/270458455 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f84840a2b80 msgr2=0x7f84840a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:42.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:42.562+0000 7f84a01a9640 1 --2- 172.21.15.5:0/270458455 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f84840a2b80 0x7f84840a5040 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f84940058c0 tx=0x7f8494005850 comp rx=0 tx=0).stop 2024-09-12T03:02:42.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:42.562+0000 7f84a01a9640 1 -- 172.21.15.5:0/270458455 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8498100a80 msgr2=0x7f84981984b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:42.563 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:42.562+0000 7f84a01a9640 1 --2- 172.21.15.5:0/270458455 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8498100a80 0x7f84981984b0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f848c009230 tx=0x7f848c004270 comp rx=0 tx=0).stop 2024-09-12T03:02:42.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:42.562+0000 7f84a01a9640 1 -- 172.21.15.5:0/270458455 shutdown_connections 2024-09-12T03:02:42.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:42.562+0000 7f84a01a9640 1 --2- 172.21.15.5:0/270458455 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8498106a80 0x7f849819cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:42.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:42.562+0000 7f84a01a9640 1 --2- 172.21.15.5:0/270458455 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8498101420 0x7f84981989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:42.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:42.562+0000 7f84a01a9640 1 --2- 172.21.15.5:0/270458455 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f84840a2b80 0x7f84840a5040 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:42.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:42.562+0000 7f84a01a9640 1 --2- 172.21.15.5:0/270458455 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8498100a80 0x7f84981984b0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:42.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:42.562+0000 7f84a01a9640 1 -- 172.21.15.5:0/270458455 >> 172.21.15.5:0/270458455 conn(0x7f84980fc7a0 msgr2=0x7f8498104930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:42.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:42.562+0000 7f84a01a9640 1 -- 172.21.15.5:0/270458455 shutdown_connections 2024-09-12T03:02:42.564 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:42.562+0000 7f84a01a9640 1 -- 172.21.15.5:0/270458455 wait complete. 2024-09-12T03:02:42.865 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:02:42 smithi005 ceph-mon[31197]: pgmap v469: 97 pgs: 97 active+clean; 113 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2024-09-12T03:02:42.865 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:02:42 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/270458455' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:02:42.956 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:02:42 smithi146 ceph-mon[35011]: pgmap v469: 97 pgs: 97 active+clean; 113 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2024-09-12T03:02:42.956 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:02:42 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/270458455' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:02:42.967 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:02:42 smithi148 ceph-mon[35090]: pgmap v469: 97 pgs: 97 active+clean; 113 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2024-09-12T03:02:42.967 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:02:42 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/270458455' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:02:43.309 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24502 quiesce db --set-id 24601adb --release --await' 2024-09-12T03:02:43.355 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24502 quiesce db --set-id 24601adb --release --await 2024-09-12T03:02:43.619 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:02:44.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.115+0000 7f92d53ee640 1 -- 172.21.15.5:0/1103731821 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f92d0100e20 msgr2=0x7f92d0101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:44.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.115+0000 7f92d53ee640 1 --2- 172.21.15.5:0/1103731821 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f92d0100e20 0x7f92d0101200 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f92c0009ca0 tx=0x7f92c002a950 comp rx=0 tx=0).stop 2024-09-12T03:02:44.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.115+0000 7f92d53ee640 1 -- 172.21.15.5:0/1103731821 shutdown_connections 2024-09-12T03:02:44.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.115+0000 7f92d53ee640 1 --2- 172.21.15.5:0/1103731821 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92d0102ef0 0x7f92d010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:44.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.115+0000 7f92d53ee640 1 --2- 172.21.15.5:0/1103731821 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f92d0102570 0x7f92d01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:44.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.115+0000 7f92d53ee640 1 --2- 172.21.15.5:0/1103731821 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f92d0100e20 0x7f92d0101200 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:44.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.115+0000 7f92d53ee640 1 -- 172.21.15.5:0/1103731821 >> 172.21.15.5:0/1103731821 conn(0x7f92d00fc7d0 msgr2=0x7f92d00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:44.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.115+0000 7f92d53ee640 1 -- 172.21.15.5:0/1103731821 shutdown_connections 2024-09-12T03:02:44.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.115+0000 7f92d53ee640 1 -- 172.21.15.5:0/1103731821 wait complete. 2024-09-12T03:02:44.117 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.115+0000 7f92d53ee640 1 Processor -- start 2024-09-12T03:02:44.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.116+0000 7f92d53ee640 1 -- start start 2024-09-12T03:02:44.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.116+0000 7f92d53ee640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f92d0100e20 0x7f92d0197460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:44.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.116+0000 7f92d53ee640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92d0102570 0x7f92d01979a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:44.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.116+0000 7f92d53ee640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f92d0102ef0 0x7f92d019bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:44.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.116+0000 7f92d53ee640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f92d0110f80 con 0x7f92d0100e20 2024-09-12T03:02:44.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.116+0000 7f92d53ee640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f92d0110e00 con 0x7f92d0102ef0 2024-09-12T03:02:44.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.116+0000 7f92d53ee640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f92d0111100 con 0x7f92d0102570 2024-09-12T03:02:44.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.116+0000 7f92ceffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f92d0100e20 0x7f92d0197460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:44.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.116+0000 7f92ceffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f92d0100e20 0x7f92d0197460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56804/0 (socket says 172.21.15.5:56804) 2024-09-12T03:02:44.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.116+0000 7f92ceffd640 1 -- 172.21.15.5:0/3629212602 learned_addr learned my addr 172.21.15.5:0/3629212602 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:02:44.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.116+0000 7f92ce7fc640 1 --2- 172.21.15.5:0/3629212602 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92d0102570 0x7f92d01979a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:44.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.116+0000 7f92cf7fe640 1 --2- 172.21.15.5:0/3629212602 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f92d0102ef0 0x7f92d019bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:44.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.117+0000 7f92ceffd640 1 -- 172.21.15.5:0/3629212602 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92d0102570 msgr2=0x7f92d01979a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:44.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.117+0000 7f92ceffd640 1 --2- 172.21.15.5:0/3629212602 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92d0102570 0x7f92d01979a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:44.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.117+0000 7f92ceffd640 1 -- 172.21.15.5:0/3629212602 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f92d0102ef0 msgr2=0x7f92d019bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:44.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.117+0000 7f92ceffd640 1 --2- 172.21.15.5:0/3629212602 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f92d0102ef0 0x7f92d019bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:44.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.117+0000 7f92ceffd640 1 -- 172.21.15.5:0/3629212602 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92d019c410 con 0x7f92d0100e20 2024-09-12T03:02:44.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.117+0000 7f92ceffd640 1 --2- 172.21.15.5:0/3629212602 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f92d0100e20 0x7f92d0197460 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f92c0005cc0 tx=0x7f92c002f650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:44.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.117+0000 7f92b7fff640 1 -- 172.21.15.5:0/3629212602 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92c0034040 con 0x7f92d0100e20 2024-09-12T03:02:44.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.117+0000 7f92d53ee640 1 -- 172.21.15.5:0/3629212602 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92d019c6a0 con 0x7f92d0100e20 2024-09-12T03:02:44.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.117+0000 7f92b7fff640 1 -- 172.21.15.5:0/3629212602 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92c0030920 con 0x7f92d0100e20 2024-09-12T03:02:44.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.117+0000 7f92b7fff640 1 -- 172.21.15.5:0/3629212602 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92c0004830 con 0x7f92d0100e20 2024-09-12T03:02:44.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.117+0000 7f92d53ee640 1 -- 172.21.15.5:0/3629212602 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92d01a3fe0 con 0x7f92d0100e20 2024-09-12T03:02:44.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.119+0000 7f92b7fff640 1 -- 172.21.15.5:0/3629212602 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f92c0037020 con 0x7f92d0100e20 2024-09-12T03:02:44.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.120+0000 7f92d77c7740 1 Processor -- start 2024-09-12T03:02:44.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.120+0000 7f92d77c7740 1 -- start start 2024-09-12T03:02:44.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.120+0000 7f92d77c7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651932df270 0x5651932df650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:44.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.120+0000 7f92d77c7740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5651932dfb90 0x5651932e9f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:44.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.120+0000 7f92d77c7740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5651932ea450 0x5651932ec840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:44.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.120+0000 7f92d77c7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x565193242620 con 0x5651932df270 2024-09-12T03:02:44.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.120+0000 7f92d77c7740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5651931e9e50 con 0x5651932ea450 2024-09-12T03:02:44.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.120+0000 7f92d77c7740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x56519323e5e0 con 0x5651932dfb90 2024-09-12T03:02:44.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.120+0000 7f92b7fff640 1 --2- 172.21.15.5:0/3629212602 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f92b80a2b80 0x7f92b80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:44.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.120+0000 7f92cf7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651932df270 0x5651932df650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:44.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.120+0000 7f92cf7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651932df270 0x5651932df650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56816/0 (socket says 172.21.15.5:56816) 2024-09-12T03:02:44.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.120+0000 7f92cf7fe640 1 -- 172.21.15.5:0/3894674350 learned_addr learned my addr 172.21.15.5:0/3894674350 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:02:44.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.120+0000 7f92cf7fe640 1 --2- 172.21.15.5:0/3629212602 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f92b80a2b80 0x7f92b80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:44.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.121+0000 7f92b7fff640 1 -- 172.21.15.5:0/3629212602 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f92c00effd0 con 0x7f92d0100e20 2024-09-12T03:02:44.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.121+0000 7f92ceffd640 1 --2- 172.21.15.5:0/3894674350 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5651932dfb90 0x5651932e9f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:44.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.121+0000 7f92ce7fc640 1 --2- 172.21.15.5:0/3894674350 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5651932ea450 0x5651932ec840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:44.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.121+0000 7f92cf7fe640 1 -- 172.21.15.5:0/3894674350 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5651932dfb90 msgr2=0x5651932e9f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:44.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.121+0000 7f92cf7fe640 1 --2- 172.21.15.5:0/3894674350 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5651932dfb90 0x5651932e9f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:44.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.121+0000 7f92cf7fe640 1 -- 172.21.15.5:0/3894674350 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5651932ea450 msgr2=0x5651932ec840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:44.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.121+0000 7f92cf7fe640 1 --2- 172.21.15.5:0/3894674350 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5651932ea450 0x5651932ec840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:44.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.121+0000 7f92cf7fe640 1 -- 172.21.15.5:0/3894674350 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651931dad40 con 0x5651932df270 2024-09-12T03:02:44.126 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.121+0000 7f92cf7fe640 1 --2- 172.21.15.5:0/3894674350 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651932df270 0x5651932df650 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f92c80129f0 tx=0x7f92c8012eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d344c0f340b512f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:44.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.121+0000 7f92cf7fe640 1 --2- 172.21.15.5:0/3629212602 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f92b80a2b80 0x7f92b80a5040 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f92c8010740 tx=0x7f92c8017000 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:44.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.121+0000 7f92b57fa640 1 -- 172.21.15.5:0/3894674350 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92c8024070 con 0x5651932df270 2024-09-12T03:02:44.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.121+0000 7f92b57fa640 1 -- 172.21.15.5:0/3894674350 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92c800b9c0 con 0x5651932df270 2024-09-12T03:02:44.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.121+0000 7f92b57fa640 1 -- 172.21.15.5:0/3894674350 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92c8004c70 con 0x5651932df270 2024-09-12T03:02:44.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.122+0000 7f92d77c7740 1 -- 172.21.15.5:0/3894674350 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651932df270 msgr2=0x5651932df650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:44.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.122+0000 7f92d77c7740 1 --2- 172.21.15.5:0/3894674350 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651932df270 0x5651932df650 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f92c80129f0 tx=0x7f92c8012eb0 comp rx=0 tx=0).stop 2024-09-12T03:02:44.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.122+0000 7f92d77c7740 1 -- 172.21.15.5:0/3894674350 shutdown_connections 2024-09-12T03:02:44.127 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.122+0000 7f92d77c7740 1 --2- 172.21.15.5:0/3894674350 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5651932ea450 0x5651932ec840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:44.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.122+0000 7f92d77c7740 1 --2- 172.21.15.5:0/3894674350 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5651932dfb90 0x5651932e9f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:44.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.122+0000 7f92d77c7740 1 --2- 172.21.15.5:0/3894674350 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651932df270 0x5651932df650 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:44.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.122+0000 7f92d77c7740 1 -- 172.21.15.5:0/3894674350 >> 172.21.15.5:0/3894674350 conn(0x5651932db800 msgr2=0x5651932dbbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:44.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.122+0000 7f92d77c7740 1 -- 172.21.15.5:0/3894674350 shutdown_connections 2024-09-12T03:02:44.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.122+0000 7f92d77c7740 1 -- 172.21.15.5:0/3894674350 wait complete. 2024-09-12T03:02:44.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.122+0000 7f92d77c7740 1 -- start start 2024-09-12T03:02:44.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.122+0000 7f92d77c7740 1 Processor -- start 2024-09-12T03:02:44.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.123+0000 7f92d77c7740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5651932df270 0x5651932e73e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:44.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.123+0000 7f92d77c7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651932dfb90 0x5651932e7920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:44.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.123+0000 7f92d77c7740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5651932ea450 0x5651932e7e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:44.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.123+0000 7f92d77c7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5651932ed020 con 0x5651932dfb90 2024-09-12T03:02:44.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.123+0000 7f92d77c7740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5651931caf10 con 0x5651932df270 2024-09-12T03:02:44.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.123+0000 7f92d77c7740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5651932ed1a0 con 0x5651932ea450 2024-09-12T03:02:44.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.123+0000 7f92d77c7740 10 client.0 _ll_register_callbacks cb 0x5651931eedb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:02:44.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.123+0000 7f92ce7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651932dfb90 0x5651932e7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:44.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.123+0000 7f92ce7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651932dfb90 0x5651932e7920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56818/0 (socket says 172.21.15.5:56818) 2024-09-12T03:02:44.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.123+0000 7f92ce7fc640 1 -- 172.21.15.5:0/2596514387 learned_addr learned my addr 172.21.15.5:0/2596514387 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:02:44.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.123+0000 7f92cf7fe640 1 --2- 172.21.15.5:0/2596514387 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5651932ea450 0x5651932e7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:44.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.123+0000 7f92ceffd640 1 --2- 172.21.15.5:0/2596514387 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5651932df270 0x5651932e73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:44.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.123+0000 7f92ce7fc640 1 -- 172.21.15.5:0/2596514387 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5651932ea450 msgr2=0x5651932e7e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:44.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.123+0000 7f92ce7fc640 1 --2- 172.21.15.5:0/2596514387 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5651932ea450 0x5651932e7e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:44.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.123+0000 7f92ce7fc640 1 -- 172.21.15.5:0/2596514387 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5651932df270 msgr2=0x5651932e73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:44.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.123+0000 7f92ce7fc640 1 --2- 172.21.15.5:0/2596514387 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5651932df270 0x5651932e73e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:44.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.123+0000 7f92ce7fc640 1 -- 172.21.15.5:0/2596514387 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565192f51580 con 0x5651932dfb90 2024-09-12T03:02:44.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.123+0000 7f92cf7fe640 1 --2- 172.21.15.5:0/2596514387 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5651932ea450 0x5651932e7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:02:44.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.123+0000 7f92ceffd640 1 --2- 172.21.15.5:0/2596514387 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5651932df270 0x5651932e73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:02:44.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.124+0000 7f92ce7fc640 1 --2- 172.21.15.5:0/2596514387 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651932dfb90 0x5651932e7920 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f92c400bad0 tx=0x7f92c400bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58bb567e836b1dae server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:44.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.124+0000 7f92b2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T03:02:44.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.124+0000 7f92b2ff5640 1 -- 172.21.15.5:0/2596514387 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92c4003e20 con 0x5651932dfb90 2024-09-12T03:02:44.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.124+0000 7f92b2ff5640 1 -- 172.21.15.5:0/2596514387 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f92c4011440 con 0x5651932dfb90 2024-09-12T03:02:44.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.124+0000 7f92d77c7740 1 -- 172.21.15.5:0/2596514387 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92c800b9c0 con 0x5651932dfb90 2024-09-12T03:02:44.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.124+0000 7f92b2ff5640 1 -- 172.21.15.5:0/2596514387 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f92c4005020 con 0x5651932dfb90 2024-09-12T03:02:44.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.124+0000 7f92d77c7740 10 client.16410 fetch_fsmap learned FSMap version 31 2024-09-12T03:02:44.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.124+0000 7f92d77c7740 1 -- 172.21.15.5:0/2596514387 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56519301b960 con 0x5651932dfb90 2024-09-12T03:02:44.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.124+0000 7f92b2ff5640 1 -- 172.21.15.5:0/2596514387 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92c800b9c0 con 0x5651932dfb90 2024-09-12T03:02:44.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.124+0000 7f92b2ff5640 1 -- 172.21.15.5:0/2596514387 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f92c4024070 con 0x5651932dfb90 2024-09-12T03:02:44.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.124+0000 7f92d77c7740 10 client.16410 fetch_fsmap finished waiting for FSMap version 31 2024-09-12T03:02:44.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.124+0000 7f92d77c7740 10 client.16410 resolve_mds: validated gid 24502 aka daemon mds.i 2024-09-12T03:02:44.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.124+0000 7f92d77c7740 20 client.16410 populate_metadata read hostname 'smithi005' 2024-09-12T03:02:44.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.124+0000 7f92d77c7740 1 --2- 172.21.15.5:0/2596514387 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x5651932fa1d0 0x5651932fc610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:44.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.124+0000 7f92d77c7740 4 client.16410 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:02:44.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.124+0000 7f92d77c7740 1 -- 172.21.15.5:0/2596514387 --> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56519324fb90 con 0x5651932fa1d0 2024-09-12T03:02:44.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.125+0000 7f92ceffd640 1 --2- 172.21.15.5:0/2596514387 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x5651932fa1d0 0x5651932fc610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:44.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.125+0000 7f92b2ff5640 10 client.16410 ms_handle_connect on v2:172.21.15.148:6838/2317600000 2024-09-12T03:02:44.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.125+0000 7f92ceffd640 1 --2- 172.21.15.5:0/2596514387 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x5651932fa1d0 0x5651932fc610 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a714f847071ed219 server_cookie=e4264272871388c2 in_seq=0 out_seq=0 2024-09-12T03:02:44.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.126+0000 7f92b2ff5640 1 -- 172.21.15.5:0/2596514387 <== mds.0 v2:172.21.15.148:6838/2317600000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56519324fb90 con 0x5651932fa1d0 2024-09-12T03:02:44.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.126+0000 7f92b2ff5640 10 client.16410 handle_command_reply: tid=0 2024-09-12T03:02:44.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.126+0000 7f92d77c7740 1 client.16410 shutdown 2024-09-12T03:02:44.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.126+0000 7f92d77c7740 1 -- 172.21.15.5:0/2596514387 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651932dfb90 msgr2=0x5651932e7920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:44.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.126+0000 7f92d77c7740 1 --2- 172.21.15.5:0/2596514387 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651932dfb90 0x5651932e7920 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f92c400bad0 tx=0x7f92c400bf90 comp rx=0 tx=0).stop 2024-09-12T03:02:44.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.126+0000 7f92d77c7740 1 -- 172.21.15.5:0/2596514387 shutdown_connections 2024-09-12T03:02:44.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.126+0000 7f92d77c7740 1 --2- 172.21.15.5:0/2596514387 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x5651932fa1d0 0x5651932fc610 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:44.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.126+0000 7f92d77c7740 1 --2- 172.21.15.5:0/2596514387 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651932dfb90 0x5651932e7920 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:44.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.126+0000 7f92d77c7740 1 --2- 172.21.15.5:0/2596514387 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5651932ea450 0x5651932e7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:44.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.126+0000 7f92d77c7740 1 --2- 172.21.15.5:0/2596514387 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5651932df270 0x5651932e73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:44.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.126+0000 7f92d77c7740 1 -- 172.21.15.5:0/2596514387 >> 172.21.15.5:0/2596514387 conn(0x5651932db800 msgr2=0x5651932d8750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:44.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.126+0000 7f92b2ff5640 0 client.16410 ms_handle_reset on v2:172.21.15.148:6838/2317600000 2024-09-12T03:02:44.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.126+0000 7f92d77c7740 1 -- 172.21.15.5:0/2596514387 shutdown_connections 2024-09-12T03:02:44.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.126+0000 7f92d77c7740 1 -- 172.21.15.5:0/2596514387 wait complete. 2024-09-12T03:02:44.134 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.126+0000 7f92d77c7740 20 client.16410 trim_cache size 0 max 16384 2024-09-12T03:02:44.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.143+0000 7f92d77c7740 1 Processor -- start 2024-09-12T03:02:44.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.143+0000 7f92d77c7740 1 -- start start 2024-09-12T03:02:44.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.143+0000 7f92d77c7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651932df270 0x565193301ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:44.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.143+0000 7f92d77c7740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5651932dfb90 0x565193302420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:44.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.143+0000 7f92d77c7740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5651932ea450 0x5651932e4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:44.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.143+0000 7f92d77c7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f92d01ab2c0 con 0x5651932df270 2024-09-12T03:02:44.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.143+0000 7f92d77c7740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f92d01ace00 con 0x5651932dfb90 2024-09-12T03:02:44.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.143+0000 7f92d77c7740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f92d01ab990 con 0x5651932ea450 2024-09-12T03:02:44.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.144+0000 7f92ceffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651932df270 0x565193301ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:44.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.144+0000 7f92ceffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651932df270 0x565193301ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56826/0 (socket says 172.21.15.5:56826) 2024-09-12T03:02:44.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.144+0000 7f92ceffd640 1 -- 172.21.15.5:0/3091393471 learned_addr learned my addr 172.21.15.5:0/3091393471 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:02:44.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.144+0000 7f92ce7fc640 1 --2- 172.21.15.5:0/3091393471 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5651932dfb90 0x565193302420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:44.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.144+0000 7f92cf7fe640 1 --2- 172.21.15.5:0/3091393471 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5651932ea450 0x5651932e4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:44.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.144+0000 7f92ceffd640 1 -- 172.21.15.5:0/3091393471 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5651932ea450 msgr2=0x5651932e4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:44.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.144+0000 7f92ceffd640 1 --2- 172.21.15.5:0/3091393471 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5651932ea450 0x5651932e4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:44.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.144+0000 7f92ceffd640 1 -- 172.21.15.5:0/3091393471 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5651932dfb90 msgr2=0x565193302420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:44.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.144+0000 7f92ceffd640 1 --2- 172.21.15.5:0/3091393471 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5651932dfb90 0x565193302420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:44.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.144+0000 7f92ceffd640 1 -- 172.21.15.5:0/3091393471 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651932f8cb0 con 0x5651932df270 2024-09-12T03:02:44.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.144+0000 7f92ceffd640 1 --2- 172.21.15.5:0/3091393471 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651932df270 0x565193301ee0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f92c00eda80 tx=0x7f92c003c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e296174c4d7268a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:44.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.144+0000 7f92b4ff9640 1 -- 172.21.15.5:0/3091393471 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92c00c0b90 con 0x5651932df270 2024-09-12T03:02:44.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.144+0000 7f92b4ff9640 1 -- 172.21.15.5:0/3091393471 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92c00c5450 con 0x5651932df270 2024-09-12T03:02:44.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.144+0000 7f92b4ff9640 1 -- 172.21.15.5:0/3091393471 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92c00a70b0 con 0x5651932df270 2024-09-12T03:02:44.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.145+0000 7f92d77c7740 1 -- 172.21.15.5:0/3091393471 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651932df270 msgr2=0x565193301ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:44.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.145+0000 7f92d77c7740 1 --2- 172.21.15.5:0/3091393471 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651932df270 0x565193301ee0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f92c00eda80 tx=0x7f92c003c040 comp rx=0 tx=0).stop 2024-09-12T03:02:44.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.145+0000 7f92d77c7740 1 -- 172.21.15.5:0/3091393471 shutdown_connections 2024-09-12T03:02:44.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.145+0000 7f92d77c7740 1 --2- 172.21.15.5:0/3091393471 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5651932ea450 0x5651932e4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:44.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.145+0000 7f92d77c7740 1 --2- 172.21.15.5:0/3091393471 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5651932dfb90 0x565193302420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:44.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.145+0000 7f92d77c7740 1 --2- 172.21.15.5:0/3091393471 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651932df270 0x565193301ee0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:44.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.145+0000 7f92d77c7740 1 -- 172.21.15.5:0/3091393471 >> 172.21.15.5:0/3091393471 conn(0x5651932db800 msgr2=0x5651932e8c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:44.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.145+0000 7f92d77c7740 1 -- 172.21.15.5:0/3091393471 shutdown_connections 2024-09-12T03:02:44.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.145+0000 7f92d77c7740 1 -- 172.21.15.5:0/3091393471 wait complete. 2024-09-12T03:02:44.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.145+0000 7f92d77c7740 1 -- start start 2024-09-12T03:02:44.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.145+0000 7f92d77c7740 1 Processor -- start 2024-09-12T03:02:44.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.146+0000 7f92d77c7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651932df270 0x5651932f23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:44.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.146+0000 7f92d77c7740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5651932dfb90 0x5651932f2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:44.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.146+0000 7f92d77c7740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5651932ea450 0x5651932ef3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:44.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.146+0000 7f92d77c7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f92d01ac730 con 0x5651932df270 2024-09-12T03:02:44.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.146+0000 7f92d77c7740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f92d01ac060 con 0x5651932dfb90 2024-09-12T03:02:44.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.146+0000 7f92d77c7740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f92d01a92c0 con 0x5651932ea450 2024-09-12T03:02:44.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.146+0000 7f92d77c7740 10 client.0 _ll_register_callbacks cb 0x7f92c40043c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:02:44.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.146+0000 7f92ceffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651932df270 0x5651932f23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:44.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.146+0000 7f92ceffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651932df270 0x5651932f23f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56828/0 (socket says 172.21.15.5:56828) 2024-09-12T03:02:44.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.146+0000 7f92ceffd640 1 -- 172.21.15.5:0/2475930913 learned_addr learned my addr 172.21.15.5:0/2475930913 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:02:44.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.146+0000 7f92ce7fc640 1 --2- 172.21.15.5:0/2475930913 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5651932dfb90 0x5651932f2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:44.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.146+0000 7f92cf7fe640 1 --2- 172.21.15.5:0/2475930913 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5651932ea450 0x5651932ef3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:44.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.146+0000 7f92ceffd640 1 -- 172.21.15.5:0/2475930913 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5651932ea450 msgr2=0x5651932ef3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:44.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.146+0000 7f92ceffd640 1 --2- 172.21.15.5:0/2475930913 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5651932ea450 0x5651932ef3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:44.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.146+0000 7f92ceffd640 1 -- 172.21.15.5:0/2475930913 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5651932dfb90 msgr2=0x5651932f2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:44.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.146+0000 7f92ceffd640 1 --2- 172.21.15.5:0/2475930913 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5651932dfb90 0x5651932f2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:44.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.146+0000 7f92ceffd640 1 -- 172.21.15.5:0/2475930913 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651932ef900 con 0x5651932df270 2024-09-12T03:02:44.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.146+0000 7f92cf7fe640 1 --2- 172.21.15.5:0/2475930913 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5651932ea450 0x5651932ef3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:02:44.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.146+0000 7f92ceffd640 1 --2- 172.21.15.5:0/2475930913 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651932df270 0x5651932f23f0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f92c00049d0 tx=0x7f92c0003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5544fffcf4f1d432 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:44.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.146+0000 7f92b2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T03:02:44.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.146+0000 7f92b2ff5640 1 -- 172.21.15.5:0/2475930913 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92c00d9070 con 0x5651932df270 2024-09-12T03:02:44.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.146+0000 7f92b2ff5640 1 -- 172.21.15.5:0/2475930913 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f92c00edb90 con 0x5651932df270 2024-09-12T03:02:44.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.147+0000 7f92d77c7740 1 -- 172.21.15.5:0/2475930913 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92c00c5450 con 0x5651932df270 2024-09-12T03:02:44.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.147+0000 7f92b2ff5640 1 -- 172.21.15.5:0/2475930913 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f92c00d4030 con 0x5651932df270 2024-09-12T03:02:44.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.147+0000 7f92d77c7740 10 client.16416 fetch_fsmap learned FSMap version 31 2024-09-12T03:02:44.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.147+0000 7f92d77c7740 1 -- 172.21.15.5:0/2475930913 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5651933656b0 con 0x5651932df270 2024-09-12T03:02:44.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.147+0000 7f92b2ff5640 1 -- 172.21.15.5:0/2475930913 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92c00c5450 con 0x5651932df270 2024-09-12T03:02:44.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.147+0000 7f92b2ff5640 1 -- 172.21.15.5:0/2475930913 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f92c00a4020 con 0x5651932df270 2024-09-12T03:02:44.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.147+0000 7f92d77c7740 10 client.16416 fetch_fsmap finished waiting for FSMap version 31 2024-09-12T03:02:44.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.147+0000 7f92d77c7740 10 client.16416 resolve_mds: validated gid 24502 aka daemon mds.i 2024-09-12T03:02:44.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.147+0000 7f92d77c7740 20 client.16416 populate_metadata read hostname 'smithi005' 2024-09-12T03:02:44.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.147+0000 7f92d77c7740 1 --2- 172.21.15.5:0/2475930913 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x565193365da0 0x5651933681c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:44.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.147+0000 7f92d77c7740 4 client.16416 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "24601adb", "release": true, "await": true}] 2024-09-12T03:02:44.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.147+0000 7f92d77c7740 1 -- 172.21.15.5:0/2475930913 --> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] -- command(tid 0: {"prefix": "quiesce db", "set_id": "24601adb", "release": true, "await": true}) -- 0x56519324a0a0 con 0x565193365da0 2024-09-12T03:02:44.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.147+0000 7f92ce7fc640 1 --2- 172.21.15.5:0/2475930913 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x565193365da0 0x5651933681c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:44.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.148+0000 7f92ce7fc640 1 --2- 172.21.15.5:0/2475930913 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x565193365da0 0x5651933681c0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8275bcbdf59fd8a3 server_cookie=89430c57fcb4f1c7 in_seq=0 out_seq=0 2024-09-12T03:02:44.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.148+0000 7f92b2ff5640 10 client.16416 ms_handle_connect on v2:172.21.15.148:6838/2317600000 2024-09-12T03:02:44.352 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.351+0000 7f92b2ff5640 1 -- 172.21.15.5:0/2475930913 <== mds.0 v2:172.21.15.148:6838/2317600000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x56519324a0a0 con 0x565193365da0 2024-09-12T03:02:44.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.351+0000 7f92b2ff5640 10 client.16416 handle_command_reply: tid=0 2024-09-12T03:02:44.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.351+0000 7f92d77c7740 1 client.16416 shutdown 2024-09-12T03:02:44.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.351+0000 7f92d77c7740 1 -- 172.21.15.5:0/2475930913 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651932df270 msgr2=0x5651932f23f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:44.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.351+0000 7f92d77c7740 1 --2- 172.21.15.5:0/2475930913 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651932df270 0x5651932f23f0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f92c00049d0 tx=0x7f92c0003f10 comp rx=0 tx=0).stop 2024-09-12T03:02:44.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.351+0000 7f92d77c7740 1 -- 172.21.15.5:0/2475930913 shutdown_connections 2024-09-12T03:02:44.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.351+0000 7f92d77c7740 1 --2- 172.21.15.5:0/2475930913 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5651932ea450 0x5651932ef3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:44.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.351+0000 7f92d77c7740 1 --2- 172.21.15.5:0/2475930913 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5651932dfb90 0x5651932f2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:44.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.351+0000 7f92d77c7740 1 --2- 172.21.15.5:0/2475930913 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x565193365da0 0x5651933681c0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:44.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.351+0000 7f92d77c7740 1 --2- 172.21.15.5:0/2475930913 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651932df270 0x5651932f23f0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:44.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.351+0000 7f92b2ff5640 0 client.16416 ms_handle_reset on v2:172.21.15.148:6838/2317600000 2024-09-12T03:02:44.353 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.351+0000 7f92d77c7740 1 -- 172.21.15.5:0/2475930913 >> 172.21.15.5:0/2475930913 conn(0x5651932db800 msgr2=0x5651932e5b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:44.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.351+0000 7f92d77c7740 1 -- 172.21.15.5:0/2475930913 shutdown_connections 2024-09-12T03:02:44.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.351+0000 7f92d77c7740 1 -- 172.21.15.5:0/2475930913 wait complete. 2024-09-12T03:02:44.354 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.352+0000 7f92d77c7740 20 client.16416 trim_cache size 0 max 16384 2024-09-12T03:02:44.354 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-12T03:02:44.354 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 31, 2024-09-12T03:02:44.354 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24502, 2024-09-12T03:02:44.354 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 20, 2024-09-12T03:02:44.354 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": { 2024-09-12T03:02:44.354 INFO:teuthology.orchestra.run.smithi005.stdout: "24601adb": { 2024-09-12T03:02:44.354 INFO:teuthology.orchestra.run.smithi005.stdout: "version": 20, 2024-09-12T03:02:44.355 INFO:teuthology.orchestra.run.smithi005.stdout: "age_ref": 0.0, 2024-09-12T03:02:44.355 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-12T03:02:44.355 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-09-12T03:02:44.355 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-12T03:02:44.355 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:02:44.355 INFO:teuthology.orchestra.run.smithi005.stdout: "timeout": 90.0, 2024-09-12T03:02:44.355 INFO:teuthology.orchestra.run.smithi005.stdout: "expiration": 140.2, 2024-09-12T03:02:44.355 INFO:teuthology.orchestra.run.smithi005.stdout: "members": { 2024-09-12T03:02:44.355 INFO:teuthology.orchestra.run.smithi005.stdout: "file:/": { 2024-09-12T03:02:44.355 INFO:teuthology.orchestra.run.smithi005.stdout: "excluded": false, 2024-09-12T03:02:44.355 INFO:teuthology.orchestra.run.smithi005.stdout: "state": { 2024-09-12T03:02:44.355 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RELEASED", 2024-09-12T03:02:44.355 INFO:teuthology.orchestra.run.smithi005.stdout: "age": 0.0 2024-09-12T03:02:44.355 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:02:44.356 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:02:44.356 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:02:44.356 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:02:44.356 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:02:44.356 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-12T03:02:44.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.352+0000 7f92d53ee640 1 -- 172.21.15.5:0/3629212602 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f92b80a2b80 msgr2=0x7f92b80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:44.356 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.352+0000 7f92d53ee640 1 --2- 172.21.15.5:0/3629212602 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f92b80a2b80 0x7f92b80a5040 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f92c8010740 tx=0x7f92c8017000 comp rx=0 tx=0).stop 2024-09-12T03:02:44.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.352+0000 7f92d53ee640 1 -- 172.21.15.5:0/3629212602 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f92d0100e20 msgr2=0x7f92d0197460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:44.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.352+0000 7f92d53ee640 1 --2- 172.21.15.5:0/3629212602 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f92d0100e20 0x7f92d0197460 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f92c0005cc0 tx=0x7f92c002f650 comp rx=0 tx=0).stop 2024-09-12T03:02:44.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.352+0000 7f92d53ee640 1 -- 172.21.15.5:0/3629212602 shutdown_connections 2024-09-12T03:02:44.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.352+0000 7f92d53ee640 1 --2- 172.21.15.5:0/3629212602 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f92d0102ef0 0x7f92d019bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:44.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.352+0000 7f92d53ee640 1 --2- 172.21.15.5:0/3629212602 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f92d0102570 0x7f92d01979a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:44.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.352+0000 7f92d53ee640 1 --2- 172.21.15.5:0/3629212602 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f92b80a2b80 0x7f92b80a5040 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:44.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.352+0000 7f92d53ee640 1 --2- 172.21.15.5:0/3629212602 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f92d0100e20 0x7f92d0197460 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:44.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.352+0000 7f92d53ee640 1 -- 172.21.15.5:0/3629212602 >> 172.21.15.5:0/3629212602 conn(0x7f92d00fc7d0 msgr2=0x7f92d00fe200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:44.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.353+0000 7f92d53ee640 1 -- 172.21.15.5:0/3629212602 shutdown_connections 2024-09-12T03:02:44.357 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:44.353+0000 7f92d53ee640 1 -- 172.21.15.5:0/3629212602 wait complete. 2024-09-12T03:02:44.566 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T03:02:44.956 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:02:44 smithi146 ceph-mon[35011]: pgmap v470: 97 pgs: 97 active+clean; 113 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1023 B/s wr, 10 op/s 2024-09-12T03:02:45.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:02:44 smithi005 ceph-mon[31197]: pgmap v470: 97 pgs: 97 active+clean; 113 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1023 B/s wr, 10 op/s 2024-09-12T03:02:45.165 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '24601adb', total seconds elapsed: 23.7 2024-09-12T03:02:45.165 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:02:44 smithi148 ceph-mon[35090]: pgmap v470: 97 pgs: 97 active+clean; 113 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1023 B/s wr, 10 op/s 2024-09-12T03:02:45.300 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:02:45.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.874+0000 7fb400abf640 1 -- 172.21.15.5:0/723107413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3fc102d30 msgr2=0x7fb3fc103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:45.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.874+0000 7fb400abf640 1 --2- 172.21.15.5:0/723107413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3fc102d30 0x7fb3fc103110 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fb3ec009930 tx=0x7fb3ec02a420 comp rx=0 tx=0).stop 2024-09-12T03:02:45.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.874+0000 7fb400abf640 1 -- 172.21.15.5:0/723107413 shutdown_connections 2024-09-12T03:02:45.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.874+0000 7fb400abf640 1 --2- 172.21.15.5:0/723107413 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb3fc1082a0 0x7fb3fc10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:45.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.874+0000 7fb400abf640 1 --2- 172.21.15.5:0/723107413 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3fc1036e0 0x7fb3fc107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:45.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.874+0000 7fb400abf640 1 --2- 172.21.15.5:0/723107413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3fc102d30 0x7fb3fc103110 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:45.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.874+0000 7fb400abf640 1 -- 172.21.15.5:0/723107413 >> 172.21.15.5:0/723107413 conn(0x7fb3fc0fc7d0 msgr2=0x7fb3fc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:45.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.874+0000 7fb400abf640 1 -- 172.21.15.5:0/723107413 shutdown_connections 2024-09-12T03:02:45.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.874+0000 7fb400abf640 1 -- 172.21.15.5:0/723107413 wait complete. 2024-09-12T03:02:45.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.875+0000 7fb400abf640 1 Processor -- start 2024-09-12T03:02:45.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.875+0000 7fb400abf640 1 -- start start 2024-09-12T03:02:45.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.875+0000 7fb400abf640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3fc102d30 0x7fb3fc1985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:45.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.875+0000 7fb400abf640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3fc1036e0 0x7fb3fc198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:45.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.875+0000 7fb400abf640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb3fc1082a0 0x7fb3fc19ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:45.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.875+0000 7fb400abf640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb3fc1120a0 con 0x7fb3fc102d30 2024-09-12T03:02:45.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.875+0000 7fb400abf640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fb3fc111f20 con 0x7fb3fc1082a0 2024-09-12T03:02:45.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.875+0000 7fb400abf640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb3fc112220 con 0x7fb3fc1036e0 2024-09-12T03:02:45.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.876+0000 7fb3fa59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3fc102d30 0x7fb3fc1985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:45.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.876+0000 7fb3fad9d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb3fc1082a0 0x7fb3fc19ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:45.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.876+0000 7fb3f9d9b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3fc1036e0 0x7fb3fc198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:45.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.876+0000 7fb3fad9d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb3fc1082a0 0x7fb3fc19ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.5:60494/0 (socket says 172.21.15.5:60494) 2024-09-12T03:02:45.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.876+0000 7fb3fa59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3fc102d30 0x7fb3fc1985e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56860/0 (socket says 172.21.15.5:56860) 2024-09-12T03:02:45.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.876+0000 7fb3f9d9b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3fc1036e0 0x7fb3fc198b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.5:33184/0 (socket says 172.21.15.5:33184) 2024-09-12T03:02:45.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.876+0000 7fb3fa59c640 1 -- 172.21.15.5:0/532253882 learned_addr learned my addr 172.21.15.5:0/532253882 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:02:45.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.876+0000 7fb3fa59c640 1 -- 172.21.15.5:0/532253882 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3fc1036e0 msgr2=0x7fb3fc198b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:45.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.876+0000 7fb3fa59c640 1 --2- 172.21.15.5:0/532253882 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3fc1036e0 0x7fb3fc198b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:45.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.876+0000 7fb3fa59c640 1 -- 172.21.15.5:0/532253882 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb3fc1082a0 msgr2=0x7fb3fc19ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:45.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.876+0000 7fb3fa59c640 1 --2- 172.21.15.5:0/532253882 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb3fc1082a0 0x7fb3fc19ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:45.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.876+0000 7fb3fa59c640 1 -- 172.21.15.5:0/532253882 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3fc19d590 con 0x7fb3fc102d30 2024-09-12T03:02:45.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.876+0000 7fb3fad9d640 1 --2- 172.21.15.5:0/532253882 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb3fc1082a0 0x7fb3fc19ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:02:45.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.876+0000 7fb3fa59c640 1 --2- 172.21.15.5:0/532253882 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3fc102d30 0x7fb3fc1985e0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fb3ec009230 tx=0x7fb3ec004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:45.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.876+0000 7fb3eb7fe640 1 -- 172.21.15.5:0/532253882 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3ec03f030 con 0x7fb3fc102d30 2024-09-12T03:02:45.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.876+0000 7fb3eb7fe640 1 -- 172.21.15.5:0/532253882 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3ec004440 con 0x7fb3fc102d30 2024-09-12T03:02:45.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.876+0000 7fb400abf640 1 -- 172.21.15.5:0/532253882 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3fc19d820 con 0x7fb3fc102d30 2024-09-12T03:02:45.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.876+0000 7fb3eb7fe640 1 -- 172.21.15.5:0/532253882 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3ec035070 con 0x7fb3fc102d30 2024-09-12T03:02:45.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.877+0000 7fb400abf640 1 -- 172.21.15.5:0/532253882 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3fc1008c0 con 0x7fb3fc102d30 2024-09-12T03:02:45.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.879+0000 7fb3eb7fe640 1 -- 172.21.15.5:0/532253882 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fb3ec042030 con 0x7fb3fc102d30 2024-09-12T03:02:45.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.880+0000 7fb402e98740 1 Processor -- start 2024-09-12T03:02:45.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.880+0000 7fb402e98740 1 -- start start 2024-09-12T03:02:45.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.881+0000 7fb402e98740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5578dc6df4a0 0x5578dc6df880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:45.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.881+0000 7fb402e98740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5578dc6dfdc0 0x5578dc6ea140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:45.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.881+0000 7fb402e98740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5578dc6ea680 0x5578dc6eca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:45.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.881+0000 7fb402e98740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5578dc6452e0 con 0x5578dc6dfdc0 2024-09-12T03:02:45.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.881+0000 7fb402e98740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5578dc5ecce0 con 0x5578dc6df4a0 2024-09-12T03:02:45.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.881+0000 7fb402e98740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5578dc6412a0 con 0x5578dc6ea680 2024-09-12T03:02:45.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.881+0000 7fb3eb7fe640 1 --2- 172.21.15.5:0/532253882 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fb3e40a2b80 0x7fb3e40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:45.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.881+0000 7fb3fa59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5578dc6dfdc0 0x5578dc6ea140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:45.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.881+0000 7fb3fa59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5578dc6dfdc0 0x5578dc6ea140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56864/0 (socket says 172.21.15.5:56864) 2024-09-12T03:02:45.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.881+0000 7fb3fa59c640 1 -- 172.21.15.5:0/806139073 learned_addr learned my addr 172.21.15.5:0/806139073 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:02:45.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.881+0000 7fb3f9d9b640 1 --2- 172.21.15.5:0/806139073 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5578dc6ea680 0x5578dc6eca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:45.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.881+0000 7fb3fad9d640 1 --2- 172.21.15.5:0/532253882 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fb3e40a2b80 0x7fb3e40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:45.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.881+0000 7fb3fad9d640 1 --2- 172.21.15.5:0/806139073 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5578dc6df4a0 0x5578dc6df880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:45.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.881+0000 7fb3eb7fe640 1 -- 172.21.15.5:0/532253882 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fb3ec0ef350 con 0x7fb3fc102d30 2024-09-12T03:02:45.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.881+0000 7fb3fa59c640 1 -- 172.21.15.5:0/806139073 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5578dc6ea680 msgr2=0x5578dc6eca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:45.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.882+0000 7fb3fa59c640 1 --2- 172.21.15.5:0/806139073 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5578dc6ea680 0x5578dc6eca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:45.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.882+0000 7fb3fa59c640 1 -- 172.21.15.5:0/806139073 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5578dc6df4a0 msgr2=0x5578dc6df880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:45.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.882+0000 7fb3fa59c640 1 --2- 172.21.15.5:0/806139073 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5578dc6df4a0 0x5578dc6df880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:45.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.882+0000 7fb3fa59c640 1 -- 172.21.15.5:0/806139073 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578dc5d1440 con 0x5578dc6dfdc0 2024-09-12T03:02:45.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.882+0000 7fb3fad9d640 1 --2- 172.21.15.5:0/806139073 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5578dc6df4a0 0x5578dc6df880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:02:45.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.882+0000 7fb3f9d9b640 1 --2- 172.21.15.5:0/806139073 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5578dc6ea680 0x5578dc6eca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:02:45.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.882+0000 7fb3fa59c640 1 --2- 172.21.15.5:0/806139073 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5578dc6dfdc0 0x5578dc6ea140 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fb3ec0bbe30 tx=0x7fb3ec009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3fc5b2ac99c734e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:45.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.882+0000 7fb3e8ff9640 1 -- 172.21.15.5:0/806139073 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3ec049070 con 0x5578dc6dfdc0 2024-09-12T03:02:45.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.882+0000 7fb3e8ff9640 1 -- 172.21.15.5:0/806139073 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3ec0bbc70 con 0x5578dc6dfdc0 2024-09-12T03:02:45.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.882+0000 7fb3e8ff9640 1 -- 172.21.15.5:0/806139073 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3ec0c43a0 con 0x5578dc6dfdc0 2024-09-12T03:02:45.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.882+0000 7fb3fad9d640 1 --2- 172.21.15.5:0/532253882 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fb3e40a2b80 0x7fb3e40a5040 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fb3f4009fd0 tx=0x7fb3f4013040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:45.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.883+0000 7fb402e98740 1 -- 172.21.15.5:0/806139073 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5578dc6dfdc0 msgr2=0x5578dc6ea140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:45.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.883+0000 7fb402e98740 1 --2- 172.21.15.5:0/806139073 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5578dc6dfdc0 0x5578dc6ea140 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fb3ec0bbe30 tx=0x7fb3ec009260 comp rx=0 tx=0).stop 2024-09-12T03:02:45.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.883+0000 7fb402e98740 1 -- 172.21.15.5:0/806139073 shutdown_connections 2024-09-12T03:02:45.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.883+0000 7fb402e98740 1 --2- 172.21.15.5:0/806139073 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5578dc6dfdc0 0x5578dc6ea140 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:45.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.883+0000 7fb402e98740 1 --2- 172.21.15.5:0/806139073 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5578dc6ea680 0x5578dc6eca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:45.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.883+0000 7fb402e98740 1 --2- 172.21.15.5:0/806139073 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5578dc6df4a0 0x5578dc6df880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:45.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.883+0000 7fb402e98740 1 -- 172.21.15.5:0/806139073 >> 172.21.15.5:0/806139073 conn(0x5578dc6dba30 msgr2=0x5578dc6dbe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:45.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.883+0000 7fb402e98740 1 -- 172.21.15.5:0/806139073 shutdown_connections 2024-09-12T03:02:45.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.883+0000 7fb402e98740 1 -- 172.21.15.5:0/806139073 wait complete. 2024-09-12T03:02:45.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.884+0000 7fb402e98740 1 -- start start 2024-09-12T03:02:45.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.884+0000 7fb402e98740 1 Processor -- start 2024-09-12T03:02:45.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.884+0000 7fb402e98740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5578dc6df4a0 0x5578dc6e9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:45.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.884+0000 7fb402e98740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5578dc6dfdc0 0x5578dc6e7280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:45.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.884+0000 7fb402e98740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5578dc6ea680 0x5578dc6e77e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:45.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.884+0000 7fb402e98740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5578dc6ed010 con 0x5578dc6dfdc0 2024-09-12T03:02:45.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.884+0000 7fb402e98740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5578dc600050 con 0x5578dc6ea680 2024-09-12T03:02:45.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.884+0000 7fb402e98740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5578dc6ed190 con 0x5578dc6df4a0 2024-09-12T03:02:45.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.884+0000 7fb402e98740 10 client.0 _ll_register_callbacks cb 0x5578dc640160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:02:45.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.884+0000 7fb3f9d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5578dc6dfdc0 0x5578dc6e7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:45.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.884+0000 7fb3f9d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5578dc6dfdc0 0x5578dc6e7280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56880/0 (socket says 172.21.15.5:56880) 2024-09-12T03:02:45.896 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.884+0000 7fb3f9d9b640 1 -- 172.21.15.5:0/3600281821 learned_addr learned my addr 172.21.15.5:0/3600281821 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:02:45.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.884+0000 7fb3fa59c640 1 --2- 172.21.15.5:0/3600281821 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5578dc6df4a0 0x5578dc6e9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:45.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.884+0000 7fb3fad9d640 1 --2- 172.21.15.5:0/3600281821 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5578dc6ea680 0x5578dc6e77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:45.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.885+0000 7fb3f9d9b640 1 -- 172.21.15.5:0/3600281821 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5578dc6df4a0 msgr2=0x5578dc6e9fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:45.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.885+0000 7fb3f9d9b640 1 --2- 172.21.15.5:0/3600281821 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5578dc6df4a0 0x5578dc6e9fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:45.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.885+0000 7fb3f9d9b640 1 -- 172.21.15.5:0/3600281821 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5578dc6ea680 msgr2=0x5578dc6e77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:45.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.885+0000 7fb3f9d9b640 1 --2- 172.21.15.5:0/3600281821 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5578dc6ea680 0x5578dc6e77e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:45.897 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.885+0000 7fb3f9d9b640 1 -- 172.21.15.5:0/3600281821 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578dc3b63c0 con 0x5578dc6dfdc0 2024-09-12T03:02:45.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.885+0000 7fb3f9d9b640 1 --2- 172.21.15.5:0/3600281821 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5578dc6dfdc0 0x5578dc6e7280 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fb3f00126e0 tx=0x7fb3f0012ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32319e2566459e20 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:45.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.885+0000 7fb3e1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T03:02:45.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.885+0000 7fb3e1ffb640 1 -- 172.21.15.5:0/3600281821 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f0004050 con 0x5578dc6dfdc0 2024-09-12T03:02:45.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.885+0000 7fb3e1ffb640 1 -- 172.21.15.5:0/3600281821 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fb3f00041f0 con 0x5578dc6dfdc0 2024-09-12T03:02:45.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.885+0000 7fb402e98740 1 -- 172.21.15.5:0/3600281821 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5578dc652850 con 0x5578dc6dfdc0 2024-09-12T03:02:45.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.885+0000 7fb3e1ffb640 1 -- 172.21.15.5:0/3600281821 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fb3f00043f0 con 0x5578dc6dfdc0 2024-09-12T03:02:45.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.885+0000 7fb402e98740 10 client.16428 fetch_fsmap learned FSMap version 31 2024-09-12T03:02:45.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.885+0000 7fb402e98740 1 -- 172.21.15.5:0/3600281821 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5578dc6f8920 con 0x5578dc6dfdc0 2024-09-12T03:02:45.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.885+0000 7fb3e1ffb640 1 -- 172.21.15.5:0/3600281821 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5578dc652850 con 0x5578dc6dfdc0 2024-09-12T03:02:45.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.886+0000 7fb3e1ffb640 1 -- 172.21.15.5:0/3600281821 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fb3f0003ca0 con 0x5578dc6dfdc0 2024-09-12T03:02:45.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.886+0000 7fb402e98740 10 client.16428 fetch_fsmap finished waiting for FSMap version 31 2024-09-12T03:02:45.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.886+0000 7fb402e98740 10 client.16428 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:02:45.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.886+0000 7fb402e98740 20 client.16428 populate_metadata read hostname 'smithi005' 2024-09-12T03:02:45.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.886+0000 7fb402e98740 1 --2- 172.21.15.5:0/3600281821 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x5578dc6f9e30 0x5578dc6fc250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:45.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.886+0000 7fb402e98740 4 client.16428 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:02:45.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.886+0000 7fb402e98740 1 -- 172.21.15.5:0/3600281821 --> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5578dc64cd60 con 0x5578dc6f9e30 2024-09-12T03:02:45.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.886+0000 7fb3fa59c640 1 --2- 172.21.15.5:0/3600281821 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x5578dc6f9e30 0x5578dc6fc250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:45.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.886+0000 7fb3e1ffb640 10 client.16428 ms_handle_connect on v2:172.21.15.148:6838/2317600000 2024-09-12T03:02:45.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.886+0000 7fb3fa59c640 1 --2- 172.21.15.5:0/3600281821 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x5578dc6f9e30 0x5578dc6fc250 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f20e2acd3806e97 server_cookie=3342b409f369e0c5 in_seq=0 out_seq=0 2024-09-12T03:02:45.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.887+0000 7fb3e1ffb640 1 -- 172.21.15.5:0/3600281821 <== mds.0 v2:172.21.15.148:6838/2317600000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5578dc64cd60 con 0x5578dc6f9e30 2024-09-12T03:02:45.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.887+0000 7fb3e1ffb640 10 client.16428 handle_command_reply: tid=0 2024-09-12T03:02:45.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.887+0000 7fb402e98740 1 client.16428 shutdown 2024-09-12T03:02:45.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.887+0000 7fb402e98740 1 -- 172.21.15.5:0/3600281821 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5578dc6dfdc0 msgr2=0x5578dc6e7280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:45.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.887+0000 7fb402e98740 1 --2- 172.21.15.5:0/3600281821 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5578dc6dfdc0 0x5578dc6e7280 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fb3f00126e0 tx=0x7fb3f0012ba0 comp rx=0 tx=0).stop 2024-09-12T03:02:45.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.887+0000 7fb402e98740 1 -- 172.21.15.5:0/3600281821 shutdown_connections 2024-09-12T03:02:45.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.887+0000 7fb402e98740 1 --2- 172.21.15.5:0/3600281821 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x5578dc6f9e30 0x5578dc6fc250 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:45.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.887+0000 7fb402e98740 1 --2- 172.21.15.5:0/3600281821 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5578dc6dfdc0 0x5578dc6e7280 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:45.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.887+0000 7fb402e98740 1 --2- 172.21.15.5:0/3600281821 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5578dc6ea680 0x5578dc6e77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:45.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.887+0000 7fb402e98740 1 --2- 172.21.15.5:0/3600281821 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5578dc6df4a0 0x5578dc6e9fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:45.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.887+0000 7fb402e98740 1 -- 172.21.15.5:0/3600281821 >> 172.21.15.5:0/3600281821 conn(0x5578dc6dba30 msgr2=0x5578dc6e1760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:45.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.887+0000 7fb3e1ffb640 0 client.16428 ms_handle_reset on v2:172.21.15.148:6838/2317600000 2024-09-12T03:02:45.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.888+0000 7fb402e98740 1 -- 172.21.15.5:0/3600281821 shutdown_connections 2024-09-12T03:02:45.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.888+0000 7fb402e98740 1 -- 172.21.15.5:0/3600281821 wait complete. 2024-09-12T03:02:45.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.888+0000 7fb402e98740 20 client.16428 trim_cache size 0 max 16384 2024-09-12T03:02:45.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.906+0000 7fb402e98740 1 Processor -- start 2024-09-12T03:02:45.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.906+0000 7fb402e98740 1 -- start start 2024-09-12T03:02:45.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.906+0000 7fb402e98740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5578dc6df4a0 0x5578dc6fbe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:45.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.906+0000 7fb402e98740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5578dc6dfdc0 0x5578dc6f60b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:45.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.906+0000 7fb402e98740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5578dc6ea680 0x5578dc6e27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:45.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.906+0000 7fb402e98740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb3fc1af820 con 0x5578dc6df4a0 2024-09-12T03:02:45.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.906+0000 7fb402e98740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fb3fc1b1360 con 0x5578dc6ea680 2024-09-12T03:02:45.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.906+0000 7fb402e98740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb3fc1afef0 con 0x5578dc6dfdc0 2024-09-12T03:02:45.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.906+0000 7fb3fa59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5578dc6df4a0 0x5578dc6fbe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:45.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.906+0000 7fb3fa59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5578dc6df4a0 0x5578dc6fbe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56892/0 (socket says 172.21.15.5:56892) 2024-09-12T03:02:45.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.906+0000 7fb3f9d9b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5578dc6dfdc0 0x5578dc6f60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:45.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.906+0000 7fb3fa59c640 1 -- 172.21.15.5:0/2325316727 learned_addr learned my addr 172.21.15.5:0/2325316727 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:02:45.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.906+0000 7fb3fad9d640 1 --2- 172.21.15.5:0/2325316727 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5578dc6ea680 0x5578dc6e27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:45.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.907+0000 7fb3f9d9b640 1 -- 172.21.15.5:0/2325316727 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5578dc6ea680 msgr2=0x5578dc6e27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:45.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.907+0000 7fb3f9d9b640 1 --2- 172.21.15.5:0/2325316727 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5578dc6ea680 0x5578dc6e27f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:45.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.907+0000 7fb3f9d9b640 1 -- 172.21.15.5:0/2325316727 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5578dc6df4a0 msgr2=0x5578dc6fbe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:45.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.907+0000 7fb3f9d9b640 1 --2- 172.21.15.5:0/2325316727 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5578dc6df4a0 0x5578dc6fbe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:45.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.907+0000 7fb3f9d9b640 1 -- 172.21.15.5:0/2325316727 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578dc6db860 con 0x5578dc6dfdc0 2024-09-12T03:02:45.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.907+0000 7fb3f9d9b640 1 --2- 172.21.15.5:0/2325316727 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5578dc6dfdc0 0x5578dc6f60b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb3f0010670 tx=0x7fb3f00096c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d54e1c78a14128f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:45.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.907+0000 7fb3e3fff640 1 -- 172.21.15.5:0/2325316727 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f0010830 con 0x5578dc6dfdc0 2024-09-12T03:02:45.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.907+0000 7fb3e3fff640 1 -- 172.21.15.5:0/2325316727 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3f000f920 con 0x5578dc6dfdc0 2024-09-12T03:02:45.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.907+0000 7fb3e3fff640 1 -- 172.21.15.5:0/2325316727 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f0005430 con 0x5578dc6dfdc0 2024-09-12T03:02:45.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.908+0000 7fb402e98740 1 -- 172.21.15.5:0/2325316727 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5578dc6dfdc0 msgr2=0x5578dc6f60b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:45.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.908+0000 7fb402e98740 1 --2- 172.21.15.5:0/2325316727 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5578dc6dfdc0 0x5578dc6f60b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb3f0010670 tx=0x7fb3f00096c0 comp rx=0 tx=0).stop 2024-09-12T03:02:45.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.908+0000 7fb402e98740 1 -- 172.21.15.5:0/2325316727 shutdown_connections 2024-09-12T03:02:45.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.908+0000 7fb402e98740 1 --2- 172.21.15.5:0/2325316727 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5578dc6ea680 0x5578dc6e27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:45.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.908+0000 7fb402e98740 1 --2- 172.21.15.5:0/2325316727 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5578dc6dfdc0 0x5578dc6f60b0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:45.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.908+0000 7fb402e98740 1 --2- 172.21.15.5:0/2325316727 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5578dc6df4a0 0x5578dc6fbe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:45.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.908+0000 7fb402e98740 1 -- 172.21.15.5:0/2325316727 >> 172.21.15.5:0/2325316727 conn(0x5578dc6dba30 msgr2=0x5578dc6e20e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:45.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.908+0000 7fb402e98740 1 -- 172.21.15.5:0/2325316727 shutdown_connections 2024-09-12T03:02:45.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.908+0000 7fb402e98740 1 -- 172.21.15.5:0/2325316727 wait complete. 2024-09-12T03:02:45.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.908+0000 7fb402e98740 1 -- start start 2024-09-12T03:02:45.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.909+0000 7fb402e98740 1 Processor -- start 2024-09-12T03:02:45.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.909+0000 7fb402e98740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5578dc6df4a0 0x5578dc6f25a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:45.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.909+0000 7fb402e98740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5578dc6dfdc0 0x5578dc6f2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:45.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.909+0000 7fb402e98740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5578dc6ea680 0x5578dc6edb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:45.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.909+0000 7fb402e98740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fb3fc1b0c90 con 0x5578dc6ea680 2024-09-12T03:02:45.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.909+0000 7fb402e98740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fb3fc1b05c0 con 0x5578dc6df4a0 2024-09-12T03:02:45.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.909+0000 7fb402e98740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fb3fc1ad800 con 0x5578dc6dfdc0 2024-09-12T03:02:45.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.909+0000 7fb402e98740 10 client.0 _ll_register_callbacks cb 0x7fb3f0009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:02:45.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.909+0000 7fb3fad9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5578dc6ea680 0x5578dc6edb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:45.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.909+0000 7fb3fad9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5578dc6ea680 0x5578dc6edb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56902/0 (socket says 172.21.15.5:56902) 2024-09-12T03:02:45.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.909+0000 7fb3fad9d640 1 -- 172.21.15.5:0/48805054 learned_addr learned my addr 172.21.15.5:0/48805054 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:02:45.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.909+0000 7fb3fa59c640 1 --2- 172.21.15.5:0/48805054 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5578dc6df4a0 0x5578dc6f25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:45.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.909+0000 7fb3f9d9b640 1 --2- 172.21.15.5:0/48805054 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5578dc6dfdc0 0x5578dc6f2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:45.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.910+0000 7fb3fad9d640 1 -- 172.21.15.5:0/48805054 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5578dc6dfdc0 msgr2=0x5578dc6f2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:45.919 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.910+0000 7fb3fad9d640 1 --2- 172.21.15.5:0/48805054 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5578dc6dfdc0 0x5578dc6f2ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:45.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.910+0000 7fb3fad9d640 1 -- 172.21.15.5:0/48805054 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5578dc6df4a0 msgr2=0x5578dc6f25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:45.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.910+0000 7fb3fad9d640 1 --2- 172.21.15.5:0/48805054 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5578dc6df4a0 0x5578dc6f25a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:45.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.910+0000 7fb3fad9d640 1 -- 172.21.15.5:0/48805054 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578dc6ee3a0 con 0x5578dc6ea680 2024-09-12T03:02:45.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.910+0000 7fb3fa59c640 1 --2- 172.21.15.5:0/48805054 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5578dc6df4a0 0x5578dc6f25a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:02:45.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.910+0000 7fb3fad9d640 1 --2- 172.21.15.5:0/48805054 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5578dc6ea680 0x5578dc6edb70 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fb3f4011280 tx=0x7fb3f4009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7202f98efa920d87 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:45.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.910+0000 7fb3e1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T03:02:45.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.910+0000 7fb3e1ffb640 1 -- 172.21.15.5:0/48805054 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f4004b20 con 0x5578dc6ea680 2024-09-12T03:02:45.920 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.910+0000 7fb3e1ffb640 1 -- 172.21.15.5:0/48805054 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fb3f4017030 con 0x5578dc6ea680 2024-09-12T03:02:45.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.910+0000 7fb402e98740 1 -- 172.21.15.5:0/48805054 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3f000f920 con 0x5578dc6ea680 2024-09-12T03:02:45.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.910+0000 7fb3e1ffb640 1 -- 172.21.15.5:0/48805054 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fb3f4004cc0 con 0x5578dc6ea680 2024-09-12T03:02:45.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.910+0000 7fb402e98740 10 client.16434 fetch_fsmap learned FSMap version 31 2024-09-12T03:02:45.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.910+0000 7fb402e98740 1 -- 172.21.15.5:0/48805054 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5578dc764760 con 0x5578dc6ea680 2024-09-12T03:02:45.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.911+0000 7fb3e1ffb640 1 -- 172.21.15.5:0/48805054 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3f000f920 con 0x5578dc6ea680 2024-09-12T03:02:45.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.911+0000 7fb3e1ffb640 1 -- 172.21.15.5:0/48805054 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fb3f4026530 con 0x5578dc6ea680 2024-09-12T03:02:45.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.911+0000 7fb402e98740 10 client.16434 fetch_fsmap finished waiting for FSMap version 31 2024-09-12T03:02:45.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.911+0000 7fb402e98740 10 client.16434 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:02:45.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.911+0000 7fb402e98740 20 client.16434 populate_metadata read hostname 'smithi005' 2024-09-12T03:02:45.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.911+0000 7fb402e98740 1 --2- 172.21.15.5:0/48805054 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x5578dc764f30 0x5578dc767350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:45.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.911+0000 7fb402e98740 4 client.16434 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T03:02:45.921 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.911+0000 7fb402e98740 1 -- 172.21.15.5:0/48805054 --> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5578dc5c9210 con 0x5578dc764f30 2024-09-12T03:02:45.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.911+0000 7fb3fa59c640 1 --2- 172.21.15.5:0/48805054 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x5578dc764f30 0x5578dc767350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:45.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.911+0000 7fb3e1ffb640 10 client.16434 ms_handle_connect on v2:172.21.15.148:6838/2317600000 2024-09-12T03:02:45.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.911+0000 7fb3fa59c640 1 --2- 172.21.15.5:0/48805054 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x5578dc764f30 0x5578dc767350 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f80eea1a561f0bbe server_cookie=7f1a96edd8ed356b in_seq=0 out_seq=0 2024-09-12T03:02:45.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.912+0000 7fb3e1ffb640 1 -- 172.21.15.5:0/48805054 <== mds.0 v2:172.21.15.148:6838/2317600000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5578dc5c9210 con 0x5578dc764f30 2024-09-12T03:02:45.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.912+0000 7fb3e1ffb640 10 client.16434 handle_command_reply: tid=0 2024-09-12T03:02:45.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.912+0000 7fb402e98740 1 client.16434 shutdown 2024-09-12T03:02:45.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.912+0000 7fb402e98740 1 -- 172.21.15.5:0/48805054 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5578dc6ea680 msgr2=0x5578dc6edb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:45.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.912+0000 7fb402e98740 1 --2- 172.21.15.5:0/48805054 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5578dc6ea680 0x5578dc6edb70 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fb3f4011280 tx=0x7fb3f4009f60 comp rx=0 tx=0).stop 2024-09-12T03:02:45.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.912+0000 7fb402e98740 1 -- 172.21.15.5:0/48805054 shutdown_connections 2024-09-12T03:02:45.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.912+0000 7fb402e98740 1 --2- 172.21.15.5:0/48805054 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x5578dc764f30 0x5578dc767350 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:45.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.912+0000 7fb402e98740 1 --2- 172.21.15.5:0/48805054 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5578dc6ea680 0x5578dc6edb70 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:45.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.912+0000 7fb402e98740 1 --2- 172.21.15.5:0/48805054 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5578dc6dfdc0 0x5578dc6f2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:45.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.912+0000 7fb3e1ffb640 0 client.16434 ms_handle_reset on v2:172.21.15.148:6838/2317600000 2024-09-12T03:02:45.922 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.912+0000 7fb402e98740 1 --2- 172.21.15.5:0/48805054 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5578dc6df4a0 0x5578dc6f25a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:45.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.912+0000 7fb402e98740 1 -- 172.21.15.5:0/48805054 >> 172.21.15.5:0/48805054 conn(0x5578dc6dba30 msgr2=0x5578dc6e28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:45.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.913+0000 7fb402e98740 1 -- 172.21.15.5:0/48805054 shutdown_connections 2024-09-12T03:02:45.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.913+0000 7fb402e98740 1 -- 172.21.15.5:0/48805054 wait complete. 2024-09-12T03:02:45.923 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.913+0000 7fb402e98740 20 client.16434 trim_cache size 0 max 16384 2024-09-12T03:02:45.923 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-12T03:02:45.923 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-09-12T03:02:45.923 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-09-12T03:02:45.923 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-12T03:02:45.924 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.913+0000 7fb400abf640 1 -- 172.21.15.5:0/532253882 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fb3e40a2b80 msgr2=0x7fb3e40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:45.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.913+0000 7fb400abf640 1 --2- 172.21.15.5:0/532253882 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fb3e40a2b80 0x7fb3e40a5040 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fb3f4009fd0 tx=0x7fb3f4013040 comp rx=0 tx=0).stop 2024-09-12T03:02:45.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.913+0000 7fb400abf640 1 -- 172.21.15.5:0/532253882 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3fc102d30 msgr2=0x7fb3fc1985e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:45.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.913+0000 7fb400abf640 1 --2- 172.21.15.5:0/532253882 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3fc102d30 0x7fb3fc1985e0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fb3ec009230 tx=0x7fb3ec004040 comp rx=0 tx=0).stop 2024-09-12T03:02:45.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.913+0000 7fb400abf640 1 -- 172.21.15.5:0/532253882 shutdown_connections 2024-09-12T03:02:45.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.913+0000 7fb400abf640 1 --2- 172.21.15.5:0/532253882 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fb3fc1082a0 0x7fb3fc19ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:45.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.913+0000 7fb400abf640 1 --2- 172.21.15.5:0/532253882 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fb3fc1036e0 0x7fb3fc198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:45.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.913+0000 7fb400abf640 1 --2- 172.21.15.5:0/532253882 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fb3e40a2b80 0x7fb3e40a5040 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:45.925 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.913+0000 7fb400abf640 1 --2- 172.21.15.5:0/532253882 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3fc102d30 0x7fb3fc1985e0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:45.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.913+0000 7fb400abf640 1 -- 172.21.15.5:0/532253882 >> 172.21.15.5:0/532253882 conn(0x7fb3fc0fc7d0 msgr2=0x7fb3fc105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:45.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.914+0000 7fb400abf640 1 -- 172.21.15.5:0/532253882 shutdown_connections 2024-09-12T03:02:45.926 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:45.914+0000 7fb400abf640 1 -- 172.21.15.5:0/532253882 wait complete. 2024-09-12T03:02:46.693 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T03:02:46.694 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:02:46.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:02:46 smithi005 ceph-mon[31197]: pgmap v471: 97 pgs: 97 active+clean; 113 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 105 KiB/s wr, 33 op/s 2024-09-12T03:02:46.981 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:02:47.165 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:02:46 smithi148 ceph-mon[35090]: pgmap v471: 97 pgs: 97 active+clean; 113 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 105 KiB/s wr, 33 op/s 2024-09-12T03:02:47.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:02:46 smithi146 ceph-mon[35011]: pgmap v471: 97 pgs: 97 active+clean; 113 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 105 KiB/s wr, 33 op/s 2024-09-12T03:02:47.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:47.585+0000 7fdd7e208640 1 -- 172.21.15.5:0/819317976 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdd78100e20 msgr2=0x7fdd78101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:47.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:47.585+0000 7fdd7e208640 1 --2- 172.21.15.5:0/819317976 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdd78100e20 0x7fdd78101200 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fdd68009890 tx=0x7fdd6802a410 comp rx=0 tx=0).stop 2024-09-12T03:02:47.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:47.585+0000 7fdd7e208640 1 -- 172.21.15.5:0/819317976 shutdown_connections 2024-09-12T03:02:47.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:47.585+0000 7fdd7e208640 1 --2- 172.21.15.5:0/819317976 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd78102ef0 0x7fdd7810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:47.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:47.585+0000 7fdd7e208640 1 --2- 172.21.15.5:0/819317976 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdd78102570 0x7fdd781029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:47.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:47.585+0000 7fdd7e208640 1 --2- 172.21.15.5:0/819317976 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdd78100e20 0x7fdd78101200 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:47.586 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:47.585+0000 7fdd7e208640 1 -- 172.21.15.5:0/819317976 >> 172.21.15.5:0/819317976 conn(0x7fdd780fc7d0 msgr2=0x7fdd780febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:47.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:47.585+0000 7fdd7e208640 1 -- 172.21.15.5:0/819317976 shutdown_connections 2024-09-12T03:02:47.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:47.585+0000 7fdd7e208640 1 -- 172.21.15.5:0/819317976 wait complete. 2024-09-12T03:02:47.587 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:47.586+0000 7fdd7e208640 1 Processor -- start 2024-09-12T03:02:47.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:47.586+0000 7fdd7e208640 1 -- start start 2024-09-12T03:02:47.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:47.586+0000 7fdd7e208640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdd78100e20 0x7fdd781985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:47.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:47.586+0000 7fdd7e208640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdd78102570 0x7fdd78198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:47.588 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:47.586+0000 7fdd7e208640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd78102ef0 0x7fdd7819ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:47.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:47.586+0000 7fdd7e208640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fdd78112090 con 0x7fdd78102ef0 2024-09-12T03:02:47.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:47.586+0000 7fdd7e208640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fdd78111f10 con 0x7fdd78100e20 2024-09-12T03:02:47.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:47.586+0000 7fdd7e208640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fdd78112210 con 0x7fdd78102570 2024-09-12T03:02:47.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:47.587+0000 7fdd76ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdd78102570 0x7fdd78198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:47.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:47.587+0000 7fdd77fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd78102ef0 0x7fdd7819ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:47.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:47.587+0000 7fdd76ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdd78102570 0x7fdd78198af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.5:33224/0 (socket says 172.21.15.5:33224) 2024-09-12T03:02:47.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:47.587+0000 7fdd76ffd640 1 -- 172.21.15.5:0/982765595 learned_addr learned my addr 172.21.15.5:0/982765595 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:02:47.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:47.587+0000 7fdd777fe640 1 --2- 172.21.15.5:0/982765595 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdd78100e20 0x7fdd781985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:47.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:47.587+0000 7fdd77fff640 1 -- 172.21.15.5:0/982765595 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdd78102570 msgr2=0x7fdd78198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:47.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:47.587+0000 7fdd77fff640 1 --2- 172.21.15.5:0/982765595 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdd78102570 0x7fdd78198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:47.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:47.587+0000 7fdd77fff640 1 -- 172.21.15.5:0/982765595 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdd78100e20 msgr2=0x7fdd781985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:47.589 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:47.587+0000 7fdd77fff640 1 --2- 172.21.15.5:0/982765595 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdd78100e20 0x7fdd781985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:47.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:47.587+0000 7fdd77fff640 1 -- 172.21.15.5:0/982765595 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd7819d560 con 0x7fdd78102ef0 2024-09-12T03:02:47.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:47.587+0000 7fdd76ffd640 1 --2- 172.21.15.5:0/982765595 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdd78102570 0x7fdd78198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:02:47.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:47.587+0000 7fdd77fff640 1 --2- 172.21.15.5:0/982765595 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd78102ef0 0x7fdd7819ce80 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fdd7000ec50 tx=0x7fdd7000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:47.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:47.587+0000 7fdd74ff9640 1 -- 172.21.15.5:0/982765595 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd70003b70 con 0x7fdd78102ef0 2024-09-12T03:02:47.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:47.588+0000 7fdd74ff9640 1 -- 172.21.15.5:0/982765595 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd70003d10 con 0x7fdd78102ef0 2024-09-12T03:02:47.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:47.588+0000 7fdd7e208640 1 -- 172.21.15.5:0/982765595 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd7819d850 con 0x7fdd78102ef0 2024-09-12T03:02:47.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:47.588+0000 7fdd74ff9640 1 -- 172.21.15.5:0/982765595 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd70010890 con 0x7fdd78102ef0 2024-09-12T03:02:47.590 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:47.588+0000 7fdd7e208640 1 -- 172.21.15.5:0/982765595 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd781a5130 con 0x7fdd78102ef0 2024-09-12T03:02:47.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:47.589+0000 7fdd7e208640 1 -- 172.21.15.5:0/982765595 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd78101200 con 0x7fdd78102ef0 2024-09-12T03:02:47.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:47.591+0000 7fdd74ff9640 1 -- 172.21.15.5:0/982765595 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fdd70004b40 con 0x7fdd78102ef0 2024-09-12T03:02:47.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:47.591+0000 7fdd74ff9640 1 --2- 172.21.15.5:0/982765595 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fdd600a2c00 0x7fdd600a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:47.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:47.591+0000 7fdd74ff9640 1 -- 172.21.15.5:0/982765595 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fdd700cf5a0 con 0x7fdd78102ef0 2024-09-12T03:02:47.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:47.591+0000 7fdd777fe640 1 --2- 172.21.15.5:0/982765595 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fdd600a2c00 0x7fdd600a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:47.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:47.592+0000 7fdd777fe640 1 --2- 172.21.15.5:0/982765595 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fdd600a2c00 0x7fdd600a50c0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fdd680095f0 tx=0x7fdd68005dc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:47.596 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:47.596+0000 7fdd74ff9640 1 -- 172.21.15.5:0/982765595 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdd7009c5a0 con 0x7fdd78102ef0 2024-09-12T03:02:47.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:47.834+0000 7fdd7e208640 1 -- 172.21.15.5:0/982765595 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdd78199910 con 0x7fdd78102ef0 2024-09-12T03:02:47.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:47.835+0000 7fdd74ff9640 1 -- 172.21.15.5:0/982765595 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11966 (secure 0 0 0) 0x7fdd7009bea0 con 0x7fdd78102ef0 2024-09-12T03:02:47.836 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-12T03:02:47.836 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":31,"btime":"2024-09-12T03:02:23:528668+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/4135232521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":4135232521},{"type":"v1","addr":"172.21.15.5:6835","nonce":4135232521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/4290221940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":4290221940},{"type":"v1","addr":"172.21.15.5:6837","nonce":4290221940}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":31,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T02:56:16.595562+0000","modified":"2024-09-12T03:02:22.771568+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24502,"mds_1":24487,"mds_2":24481,"mds_3":24479,"mds_4":24458},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/26447341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":26447341},{"type":"v1","addr":"172.21.15.5:6839","nonce":26447341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2393443295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2393443295},{"type":"v1","addr":"172.21.15.5:6841","nonce":2393443295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6833/3686970137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3686970137},{"type":"v1","addr":"172.21.15.146:6833","nonce":3686970137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6833/568512209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":568512209},{"type":"v1","addr":"172.21.15.148:6833","nonce":568512209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6835/2540740420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2540740420},{"type":"v1","addr":"172.21.15.146:6835","nonce":2540740420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.146:6837/1673370937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":1673370937},{"type":"v1","addr":"172.21.15.146:6837","nonce":1673370937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":3,"incarnation":22,"state":"up:active","state_seq":6,"addr":"172.21.15.146:6839/3036767228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":3036767228},{"type":"v1","addr":"172.21.15.146:6839","nonce":3036767228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":2,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.148:6835/4195086427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":4195086427},{"type":"v1","addr":"172.21.15.148:6835","nonce":4195086427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.148:6837/2606873922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2606873922},{"type":"v1","addr":"172.21.15.148:6837","nonce":2606873922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6839/2317600000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":2317600000},{"type":"v1","addr":"172.21.15.148:6839","nonce":2317600000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24502,"qdb_cluster":[24458,24479,24481,24487,24502]},"id":1}]} 2024-09-12T03:02:47.836 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 31 2024-09-12T03:02:47.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:47.838+0000 7fdd7e208640 1 -- 172.21.15.5:0/982765595 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fdd600a2c00 msgr2=0x7fdd600a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:47.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:47.838+0000 7fdd7e208640 1 --2- 172.21.15.5:0/982765595 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fdd600a2c00 0x7fdd600a50c0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fdd680095f0 tx=0x7fdd68005dc0 comp rx=0 tx=0).stop 2024-09-12T03:02:47.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:47.838+0000 7fdd7e208640 1 -- 172.21.15.5:0/982765595 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd78102ef0 msgr2=0x7fdd7819ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:47.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:47.838+0000 7fdd7e208640 1 --2- 172.21.15.5:0/982765595 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd78102ef0 0x7fdd7819ce80 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fdd7000ec50 tx=0x7fdd7000c6a0 comp rx=0 tx=0).stop 2024-09-12T03:02:47.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:47.838+0000 7fdd7e208640 1 -- 172.21.15.5:0/982765595 shutdown_connections 2024-09-12T03:02:47.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:47.838+0000 7fdd7e208640 1 --2- 172.21.15.5:0/982765595 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fdd600a2c00 0x7fdd600a50c0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:47.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:47.838+0000 7fdd7e208640 1 --2- 172.21.15.5:0/982765595 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd78102ef0 0x7fdd7819ce80 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:47.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:47.838+0000 7fdd7e208640 1 --2- 172.21.15.5:0/982765595 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fdd78102570 0x7fdd78198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:47.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:47.838+0000 7fdd7e208640 1 --2- 172.21.15.5:0/982765595 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fdd78100e20 0x7fdd781985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:47.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:47.838+0000 7fdd7e208640 1 -- 172.21.15.5:0/982765595 >> 172.21.15.5:0/982765595 conn(0x7fdd780fc7d0 msgr2=0x7fdd780fd0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:47.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:47.839+0000 7fdd7e208640 1 -- 172.21.15.5:0/982765595 shutdown_connections 2024-09-12T03:02:47.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:47.839+0000 7fdd7e208640 1 -- 172.21.15.5:0/982765595 wait complete. 2024-09-12T03:02:48.649 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-12T03:02:48.933 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:02:49.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:02:48 smithi005 ceph-mon[31197]: pgmap v472: 97 pgs: 97 active+clean; 113 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 981 KiB/s rd, 396 KiB/s wr, 77 op/s 2024-09-12T03:02:49.056 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:02:48 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/982765595' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:02:49.165 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:02:48 smithi148 ceph-mon[35090]: pgmap v472: 97 pgs: 97 active+clean; 113 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 981 KiB/s rd, 396 KiB/s wr, 77 op/s 2024-09-12T03:02:49.165 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:02:48 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/982765595' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:02:49.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:02:48 smithi146 ceph-mon[35011]: pgmap v472: 97 pgs: 97 active+clean; 113 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 981 KiB/s rd, 396 KiB/s wr, 77 op/s 2024-09-12T03:02:49.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:02:48 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/982765595' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:02:49.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.461+0000 7f353af0e640 1 -- 172.21.15.5:0/16102106 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3534102ef0 msgr2=0x7f353410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:49.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.461+0000 7f353af0e640 1 --2- 172.21.15.5:0/16102106 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3534102ef0 0x7f353410da40 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f352c00fb10 tx=0x7f352c02e600 comp rx=0 tx=0).stop 2024-09-12T03:02:49.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.461+0000 7f353af0e640 1 -- 172.21.15.5:0/16102106 shutdown_connections 2024-09-12T03:02:49.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.461+0000 7f353af0e640 1 --2- 172.21.15.5:0/16102106 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3534102ef0 0x7f353410da40 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:49.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.461+0000 7f353af0e640 1 --2- 172.21.15.5:0/16102106 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3534102570 0x7f35341029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:49.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.461+0000 7f353af0e640 1 --2- 172.21.15.5:0/16102106 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3534100e20 0x7f3534101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:49.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.461+0000 7f353af0e640 1 -- 172.21.15.5:0/16102106 >> 172.21.15.5:0/16102106 conn(0x7f35340fc7d0 msgr2=0x7f35340febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:49.463 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.461+0000 7f353af0e640 1 -- 172.21.15.5:0/16102106 shutdown_connections 2024-09-12T03:02:49.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.461+0000 7f353af0e640 1 -- 172.21.15.5:0/16102106 wait complete. 2024-09-12T03:02:49.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.462+0000 7f353af0e640 1 Processor -- start 2024-09-12T03:02:49.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.462+0000 7f353af0e640 1 -- start start 2024-09-12T03:02:49.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.462+0000 7f353af0e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3534100e20 0x7f3534196260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:49.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.462+0000 7f353af0e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3534102570 0x7f35341967a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:49.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.462+0000 7f353af0e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3534102ef0 0x7f353419ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:49.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.462+0000 7f353af0e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f353410fec0 con 0x7f3534102570 2024-09-12T03:02:49.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.462+0000 7f353af0e640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f353410fd40 con 0x7f3534100e20 2024-09-12T03:02:49.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.462+0000 7f353af0e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3534110040 con 0x7f3534102ef0 2024-09-12T03:02:49.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.462+0000 7f3533fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3534102570 0x7f35341967a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:49.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.462+0000 7f3538caa640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3534100e20 0x7f3534196260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:49.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.462+0000 7f3533fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3534102570 0x7f35341967a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56948/0 (socket says 172.21.15.5:56948) 2024-09-12T03:02:49.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.462+0000 7f3533fff640 1 -- 172.21.15.5:0/4289320873 learned_addr learned my addr 172.21.15.5:0/4289320873 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:02:49.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.462+0000 7f35394ab640 1 --2- 172.21.15.5:0/4289320873 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3534102ef0 0x7f353419ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:49.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.463+0000 7f3538caa640 1 -- 172.21.15.5:0/4289320873 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3534102ef0 msgr2=0x7f353419ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:49.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.463+0000 7f3538caa640 1 --2- 172.21.15.5:0/4289320873 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3534102ef0 0x7f353419ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:49.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.463+0000 7f3538caa640 1 -- 172.21.15.5:0/4289320873 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3534102570 msgr2=0x7f35341967a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:49.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.463+0000 7f3538caa640 1 --2- 172.21.15.5:0/4289320873 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3534102570 0x7f35341967a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:49.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.463+0000 7f3538caa640 1 -- 172.21.15.5:0/4289320873 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f353419b210 con 0x7f3534100e20 2024-09-12T03:02:49.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.463+0000 7f3538caa640 1 --2- 172.21.15.5:0/4289320873 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3534100e20 0x7f3534196260 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3524009fd0 tx=0x7f352400ef90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:49.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.463+0000 7f3531ffb640 1 -- 172.21.15.5:0/4289320873 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3524004b20 con 0x7f3534100e20 2024-09-12T03:02:49.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.463+0000 7f3531ffb640 1 -- 172.21.15.5:0/4289320873 <== mon.2 v2:172.21.15.146:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3524004cc0 con 0x7f3534100e20 2024-09-12T03:02:49.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.463+0000 7f353af0e640 1 -- 172.21.15.5:0/4289320873 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f353419b500 con 0x7f3534100e20 2024-09-12T03:02:49.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.463+0000 7f3531ffb640 1 -- 172.21.15.5:0/4289320873 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35240107b0 con 0x7f3534100e20 2024-09-12T03:02:49.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.463+0000 7f353af0e640 1 -- 172.21.15.5:0/4289320873 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35341a2de0 con 0x7f3534100e20 2024-09-12T03:02:49.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.466+0000 7f3531ffb640 1 -- 172.21.15.5:0/4289320873 <== mon.2 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f3524005020 con 0x7f3534100e20 2024-09-12T03:02:49.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.466+0000 7f353d2e7740 1 Processor -- start 2024-09-12T03:02:49.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.466+0000 7f353d2e7740 1 -- start start 2024-09-12T03:02:49.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.466+0000 7f353d2e7740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fb29a70b40 0x55fb29a6f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:49.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.467+0000 7f353d2e7740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fb29a6f5d0 0x55fb29a6fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:49.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.467+0000 7f353d2e7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fb29a7a370 0x55fb29a7c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:49.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.467+0000 7f353d2e7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55fb299d52f0 con 0x55fb29a7a370 2024-09-12T03:02:49.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.467+0000 7f353d2e7740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55fb2997ccf0 con 0x55fb29a6f5d0 2024-09-12T03:02:49.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.467+0000 7f353d2e7740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55fb299d12b0 con 0x55fb29a70b40 2024-09-12T03:02:49.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.467+0000 7f35394ab640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fb29a70b40 0x55fb29a6f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:49.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.467+0000 7f3538caa640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fb29a6f5d0 0x55fb29a6fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:49.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.467+0000 7f3533fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fb29a7a370 0x55fb29a7c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:49.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.467+0000 7f35394ab640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fb29a70b40 0x55fb29a6f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.5:33248/0 (socket says 172.21.15.5:33248) 2024-09-12T03:02:49.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.467+0000 7f35394ab640 1 -- 172.21.15.5:0/2570138407 learned_addr learned my addr 172.21.15.5:0/2570138407 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:02:49.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.467+0000 7f3531ffb640 1 --2- 172.21.15.5:0/4289320873 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f35200a2c00 0x7f35200a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:49.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.467+0000 7f35394ab640 1 -- 172.21.15.5:0/2570138407 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fb29a6f5d0 msgr2=0x55fb29a6fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:49.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.467+0000 7f35394ab640 1 --2- 172.21.15.5:0/2570138407 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fb29a6f5d0 0x55fb29a6fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:49.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.467+0000 7f35394ab640 1 -- 172.21.15.5:0/2570138407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fb29a7a370 msgr2=0x55fb29a7c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:49.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.467+0000 7f35394ab640 1 --2- 172.21.15.5:0/2570138407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fb29a7a370 0x55fb29a7c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:49.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.467+0000 7f35394ab640 1 -- 172.21.15.5:0/2570138407 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb299613a0 con 0x55fb29a70b40 2024-09-12T03:02:49.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.467+0000 7f3533fff640 1 --2- 172.21.15.5:0/2570138407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fb29a7a370 0x55fb29a7c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:02:49.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.467+0000 7f35394ab640 1 --2- 172.21.15.5:0/4289320873 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f35200a2c00 0x7f35200a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:49.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.467+0000 7f3531ffb640 1 -- 172.21.15.5:0/4289320873 <== mon.2 v2:172.21.15.146:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f35240cf730 con 0x7f3534100e20 2024-09-12T03:02:49.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.467+0000 7f35394ab640 1 --2- 172.21.15.5:0/2570138407 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fb29a70b40 0x55fb29a6f090 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f352c009230 tx=0x7f352c00f1e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=37b0cc01118037b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:49.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.468+0000 7f352affd640 1 -- 172.21.15.5:0/2570138407 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f352c054070 con 0x55fb29a70b40 2024-09-12T03:02:49.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.468+0000 7f352affd640 1 -- 172.21.15.5:0/2570138407 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f352c00eb40 con 0x55fb29a70b40 2024-09-12T03:02:49.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.468+0000 7f352affd640 1 -- 172.21.15.5:0/2570138407 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f352c003a50 con 0x55fb29a70b40 2024-09-12T03:02:49.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.468+0000 7f35394ab640 1 --2- 172.21.15.5:0/4289320873 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f35200a2c00 0x7f35200a50c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f352c051000 tx=0x7f352c04e000 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:49.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.469+0000 7f353d2e7740 1 -- 172.21.15.5:0/2570138407 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fb29a70b40 msgr2=0x55fb29a6f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:49.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.469+0000 7f353d2e7740 1 --2- 172.21.15.5:0/2570138407 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fb29a70b40 0x55fb29a6f090 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f352c009230 tx=0x7f352c00f1e0 comp rx=0 tx=0).stop 2024-09-12T03:02:49.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.469+0000 7f353d2e7740 1 -- 172.21.15.5:0/2570138407 shutdown_connections 2024-09-12T03:02:49.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.469+0000 7f353d2e7740 1 --2- 172.21.15.5:0/2570138407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fb29a7a370 0x55fb29a7c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:49.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.469+0000 7f353d2e7740 1 --2- 172.21.15.5:0/2570138407 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fb29a6f5d0 0x55fb29a6fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:49.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.469+0000 7f353d2e7740 1 --2- 172.21.15.5:0/2570138407 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fb29a70b40 0x55fb29a6f090 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:49.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.469+0000 7f353d2e7740 1 -- 172.21.15.5:0/2570138407 >> 172.21.15.5:0/2570138407 conn(0x55fb29a38310 msgr2=0x55fb29a6bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:49.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.469+0000 7f353d2e7740 1 -- 172.21.15.5:0/2570138407 shutdown_connections 2024-09-12T03:02:49.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.469+0000 7f353d2e7740 1 -- 172.21.15.5:0/2570138407 wait complete. 2024-09-12T03:02:49.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.470+0000 7f353d2e7740 1 -- start start 2024-09-12T03:02:49.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.470+0000 7f353d2e7740 1 Processor -- start 2024-09-12T03:02:49.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.470+0000 7f353d2e7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fb29a6f5d0 0x55fb29a79d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:49.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.470+0000 7f353d2e7740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fb29a70b40 0x55fb29a77040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:49.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.470+0000 7f353d2e7740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fb29a7a370 0x55fb29a77650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:49.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.471+0000 7f353d2e7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55fb29a7d070 con 0x55fb29a6f5d0 2024-09-12T03:02:49.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.471+0000 7f3538caa640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fb29a6f5d0 0x55fb29a79d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:49.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.471+0000 7f35394ab640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fb29a7a370 0x55fb29a77650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:49.477 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.471+0000 7f3533fff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fb29a70b40 0x55fb29a77040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:49.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.471+0000 7f35394ab640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fb29a7a370 0x55fb29a77650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.5:33256/0 (socket says 172.21.15.5:33256) 2024-09-12T03:02:49.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.471+0000 7f3533fff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fb29a70b40 0x55fb29a77040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.5:60554/0 (socket says 172.21.15.5:60554) 2024-09-12T03:02:49.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.471+0000 7f35394ab640 1 -- 172.21.15.5:0/3807455470 learned_addr learned my addr 172.21.15.5:0/3807455470 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:02:49.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.471+0000 7f353d2e7740 1 -- 172.21.15.5:0/3807455470 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55fb29990060 con 0x55fb29a70b40 2024-09-12T03:02:49.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.471+0000 7f353d2e7740 1 -- 172.21.15.5:0/3807455470 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55fb29a7d1f0 con 0x55fb29a7a370 2024-09-12T03:02:49.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.471+0000 7f353d2e7740 10 client.0 _ll_register_callbacks cb 0x55fb296e38c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:02:49.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.471+0000 7f3538caa640 1 -- 172.21.15.5:0/3807455470 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fb29a7a370 msgr2=0x55fb29a77650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:49.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.471+0000 7f3538caa640 1 --2- 172.21.15.5:0/3807455470 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fb29a7a370 0x55fb29a77650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:49.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.471+0000 7f3538caa640 1 -- 172.21.15.5:0/3807455470 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fb29a70b40 msgr2=0x55fb29a77040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:49.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.471+0000 7f3538caa640 1 --2- 172.21.15.5:0/3807455470 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fb29a70b40 0x55fb29a77040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:49.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.471+0000 7f3538caa640 1 -- 172.21.15.5:0/3807455470 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb297463c0 con 0x55fb29a6f5d0 2024-09-12T03:02:49.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.471+0000 7f3538caa640 1 --2- 172.21.15.5:0/3807455470 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fb29a6f5d0 0x55fb29a79d90 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f35240cd290 tx=0x7f352400cea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aabdbfec74123c31 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:49.478 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.471+0000 7f3513fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T03:02:49.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.472+0000 7f3513fff640 1 -- 172.21.15.5:0/3807455470 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f352409b900 con 0x55fb29a6f5d0 2024-09-12T03:02:49.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.472+0000 7f3513fff640 1 -- 172.21.15.5:0/3807455470 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f3524005590 con 0x55fb29a6f5d0 2024-09-12T03:02:49.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.472+0000 7f353d2e7740 1 -- 172.21.15.5:0/3807455470 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f352c00eb40 con 0x55fb29a6f5d0 2024-09-12T03:02:49.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.472+0000 7f3513fff640 1 -- 172.21.15.5:0/3807455470 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f352409c6a0 con 0x55fb29a6f5d0 2024-09-12T03:02:49.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.472+0000 7f353d2e7740 10 client.16452 fetch_fsmap learned FSMap version 31 2024-09-12T03:02:49.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.472+0000 7f353d2e7740 1 -- 172.21.15.5:0/3807455470 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55fb29a88bd0 con 0x55fb29a6f5d0 2024-09-12T03:02:49.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.472+0000 7f3513fff640 1 -- 172.21.15.5:0/3807455470 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f352c00eb40 con 0x55fb29a6f5d0 2024-09-12T03:02:49.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.472+0000 7f3513fff640 1 -- 172.21.15.5:0/3807455470 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f35240cfc70 con 0x55fb29a6f5d0 2024-09-12T03:02:49.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.472+0000 7f353d2e7740 10 client.16452 fetch_fsmap finished waiting for FSMap version 31 2024-09-12T03:02:49.479 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.472+0000 7f353d2e7740 10 client.16452 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-12T03:02:49.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.472+0000 7f353d2e7740 20 client.16452 populate_metadata read hostname 'smithi005' 2024-09-12T03:02:49.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.472+0000 7f353d2e7740 1 --2- 172.21.15.5:0/3807455470 >> [v2:172.21.15.146:6836/1673370937,v1:172.21.15.146:6837/1673370937] conn(0x55fb29a8a090 0x55fb29a8c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:49.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.472+0000 7f353d2e7740 4 client.16452 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:02:49.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.473+0000 7f353d2e7740 1 -- 172.21.15.5:0/3807455470 --> [v2:172.21.15.146:6836/1673370937,v1:172.21.15.146:6837/1673370937] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fb299e2860 con 0x55fb29a8a090 2024-09-12T03:02:49.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.473+0000 7f3533fff640 1 --2- 172.21.15.5:0/3807455470 >> [v2:172.21.15.146:6836/1673370937,v1:172.21.15.146:6837/1673370937] conn(0x55fb29a8a090 0x55fb29a8c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:49.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.473+0000 7f3533fff640 1 --2- 172.21.15.5:0/3807455470 >> [v2:172.21.15.146:6836/1673370937,v1:172.21.15.146:6837/1673370937] conn(0x55fb29a8a090 0x55fb29a8c4c0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=48a5d8f95df9b5bd server_cookie=6a09cb73a9f29ee6 in_seq=0 out_seq=0 2024-09-12T03:02:49.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.473+0000 7f3513fff640 10 client.16452 ms_handle_connect on v2:172.21.15.146:6836/1673370937 2024-09-12T03:02:49.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.474+0000 7f3513fff640 1 -- 172.21.15.5:0/3807455470 <== mds.4 v2:172.21.15.146:6836/1673370937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fb299e2860 con 0x55fb29a8a090 2024-09-12T03:02:49.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.474+0000 7f3513fff640 10 client.16452 handle_command_reply: tid=0 2024-09-12T03:02:49.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.474+0000 7f353d2e7740 1 client.16452 shutdown 2024-09-12T03:02:49.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.474+0000 7f353d2e7740 1 -- 172.21.15.5:0/3807455470 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fb29a6f5d0 msgr2=0x55fb29a79d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:49.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.474+0000 7f353d2e7740 1 --2- 172.21.15.5:0/3807455470 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fb29a6f5d0 0x55fb29a79d90 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f35240cd290 tx=0x7f352400cea0 comp rx=0 tx=0).stop 2024-09-12T03:02:49.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.474+0000 7f353d2e7740 1 -- 172.21.15.5:0/3807455470 shutdown_connections 2024-09-12T03:02:49.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.474+0000 7f353d2e7740 1 --2- 172.21.15.5:0/3807455470 >> [v2:172.21.15.146:6836/1673370937,v1:172.21.15.146:6837/1673370937] conn(0x55fb29a8a090 0x55fb29a8c4c0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:49.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.474+0000 7f353d2e7740 1 --2- 172.21.15.5:0/3807455470 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fb29a7a370 0x55fb29a77650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:49.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.474+0000 7f353d2e7740 1 --2- 172.21.15.5:0/3807455470 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fb29a70b40 0x55fb29a77040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:49.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.474+0000 7f353d2e7740 1 --2- 172.21.15.5:0/3807455470 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fb29a6f5d0 0x55fb29a79d90 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:49.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.474+0000 7f3513fff640 0 client.16452 ms_handle_reset on v2:172.21.15.146:6836/1673370937 2024-09-12T03:02:49.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.474+0000 7f353d2e7740 1 -- 172.21.15.5:0/3807455470 >> 172.21.15.5:0/3807455470 conn(0x55fb29a38310 msgr2=0x55fb29a71960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:49.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.474+0000 7f353d2e7740 1 -- 172.21.15.5:0/3807455470 shutdown_connections 2024-09-12T03:02:49.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.474+0000 7f353d2e7740 1 -- 172.21.15.5:0/3807455470 wait complete. 2024-09-12T03:02:49.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.474+0000 7f353d2e7740 20 client.16452 trim_cache size 0 max 16384 2024-09-12T03:02:49.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.489+0000 7f353d2e7740 1 Processor -- start 2024-09-12T03:02:49.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.489+0000 7f353d2e7740 1 -- start start 2024-09-12T03:02:49.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.490+0000 7f353d2e7740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fb29a6f5d0 0x55fb29a72120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:49.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.490+0000 7f353d2e7740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fb29a70b40 0x55fb29a72660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:49.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.490+0000 7f353d2e7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fb29a7a370 0x55fb29a91d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:49.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.490+0000 7f353d2e7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f35341ab150 con 0x55fb29a7a370 2024-09-12T03:02:49.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.490+0000 7f3533fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fb29a70b40 0x55fb29a72660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:49.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.490+0000 7f3538caa640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fb29a6f5d0 0x55fb29a72120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:49.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.490+0000 7f3533fff640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fb29a70b40 0x55fb29a72660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.5:33266/0 (socket says 172.21.15.5:33266) 2024-09-12T03:02:49.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.490+0000 7f3538caa640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fb29a6f5d0 0x55fb29a72120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.5:60562/0 (socket says 172.21.15.5:60562) 2024-09-12T03:02:49.491 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.490+0000 7f3533fff640 1 -- 172.21.15.5:0/135090708 learned_addr learned my addr 172.21.15.5:0/135090708 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:02:49.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.490+0000 7f353d2e7740 1 -- 172.21.15.5:0/135090708 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f35341acc90 con 0x55fb29a6f5d0 2024-09-12T03:02:49.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.490+0000 7f353d2e7740 1 -- 172.21.15.5:0/135090708 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f35341ab820 con 0x55fb29a70b40 2024-09-12T03:02:49.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.490+0000 7f35394ab640 1 --2- 172.21.15.5:0/135090708 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fb29a7a370 0x55fb29a91d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:49.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.490+0000 7f3538caa640 1 -- 172.21.15.5:0/135090708 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fb29a70b40 msgr2=0x55fb29a72660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:49.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.490+0000 7f3538caa640 1 --2- 172.21.15.5:0/135090708 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fb29a70b40 0x55fb29a72660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:49.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.490+0000 7f3538caa640 1 -- 172.21.15.5:0/135090708 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fb29a7a370 msgr2=0x55fb29a91d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:49.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.490+0000 7f3538caa640 1 --2- 172.21.15.5:0/135090708 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fb29a7a370 0x55fb29a91d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:49.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.490+0000 7f3538caa640 1 -- 172.21.15.5:0/135090708 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb29a7a0c0 con 0x55fb29a6f5d0 2024-09-12T03:02:49.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.490+0000 7f35394ab640 1 --2- 172.21.15.5:0/135090708 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fb29a7a370 0x55fb29a91d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:02:49.492 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.491+0000 7f3538caa640 1 --2- 172.21.15.5:0/135090708 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fb29a6f5d0 0x55fb29a72120 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f35240cd5b0 tx=0x7f3524005990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6cb66798c3beb356 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:49.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.491+0000 7f352a7fc640 1 -- 172.21.15.5:0/135090708 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f352409b900 con 0x55fb29a6f5d0 2024-09-12T03:02:49.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.491+0000 7f352a7fc640 1 -- 172.21.15.5:0/135090708 <== mon.2 v2:172.21.15.146:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f352409c450 con 0x55fb29a6f5d0 2024-09-12T03:02:49.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.491+0000 7f352a7fc640 1 -- 172.21.15.5:0/135090708 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3524005590 con 0x55fb29a6f5d0 2024-09-12T03:02:49.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.492+0000 7f353d2e7740 1 -- 172.21.15.5:0/135090708 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fb29a6f5d0 msgr2=0x55fb29a72120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:49.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.492+0000 7f353d2e7740 1 --2- 172.21.15.5:0/135090708 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fb29a6f5d0 0x55fb29a72120 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f35240cd5b0 tx=0x7f3524005990 comp rx=0 tx=0).stop 2024-09-12T03:02:49.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.492+0000 7f353d2e7740 1 -- 172.21.15.5:0/135090708 shutdown_connections 2024-09-12T03:02:49.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.492+0000 7f353d2e7740 1 --2- 172.21.15.5:0/135090708 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fb29a7a370 0x55fb29a91d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:49.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.492+0000 7f353d2e7740 1 --2- 172.21.15.5:0/135090708 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fb29a70b40 0x55fb29a72660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:49.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.492+0000 7f353d2e7740 1 --2- 172.21.15.5:0/135090708 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fb29a6f5d0 0x55fb29a72120 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:49.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.492+0000 7f353d2e7740 1 -- 172.21.15.5:0/135090708 >> 172.21.15.5:0/135090708 conn(0x55fb29a38310 msgr2=0x55fb29a6ad20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:49.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.492+0000 7f353d2e7740 1 -- 172.21.15.5:0/135090708 shutdown_connections 2024-09-12T03:02:49.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.492+0000 7f353d2e7740 1 -- 172.21.15.5:0/135090708 wait complete. 2024-09-12T03:02:49.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.492+0000 7f353d2e7740 1 -- start start 2024-09-12T03:02:49.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.492+0000 7f353d2e7740 1 Processor -- start 2024-09-12T03:02:49.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.493+0000 7f353d2e7740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fb29a6f5d0 0x55fb29a7e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:49.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.493+0000 7f353d2e7740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fb29a70b40 0x55fb29a7e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:49.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.493+0000 7f353d2e7740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fb29a7a370 0x55fb29a7ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:49.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.493+0000 7f353d2e7740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f35341ac5c0 con 0x55fb29a7a370 2024-09-12T03:02:49.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.493+0000 7f353d2e7740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f35341abef0 con 0x55fb29a70b40 2024-09-12T03:02:49.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.493+0000 7f353d2e7740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f35341a9130 con 0x55fb29a6f5d0 2024-09-12T03:02:49.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.493+0000 7f353d2e7740 10 client.0 _ll_register_callbacks cb 0x7f352409be60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:02:49.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.493+0000 7f35394ab640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fb29a7a370 0x55fb29a7ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:49.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.493+0000 7f3533fff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fb29a70b40 0x55fb29a7e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:49.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.493+0000 7f3538caa640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fb29a6f5d0 0x55fb29a7e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:49.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.493+0000 7f3533fff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fb29a70b40 0x55fb29a7e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.5:60574/0 (socket says 172.21.15.5:60574) 2024-09-12T03:02:49.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.493+0000 7f3533fff640 1 -- 172.21.15.5:0/2551832981 learned_addr learned my addr 172.21.15.5:0/2551832981 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:02:49.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.493+0000 7f3533fff640 1 -- 172.21.15.5:0/2551832981 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fb29a6f5d0 msgr2=0x55fb29a7e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:49.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.493+0000 7f3533fff640 1 --2- 172.21.15.5:0/2551832981 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fb29a6f5d0 0x55fb29a7e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:49.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.493+0000 7f3533fff640 1 -- 172.21.15.5:0/2551832981 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fb29a7a370 msgr2=0x55fb29a7ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:49.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.493+0000 7f3533fff640 1 --2- 172.21.15.5:0/2551832981 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fb29a7a370 0x55fb29a7ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:49.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.493+0000 7f3533fff640 1 -- 172.21.15.5:0/2551832981 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb29af3f00 con 0x55fb29a70b40 2024-09-12T03:02:49.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.493+0000 7f3533fff640 1 --2- 172.21.15.5:0/2551832981 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fb29a70b40 0x55fb29a7e8a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f35340fec10 tx=0x7f351c0039c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db51e9c1d04f4989 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:49.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.493+0000 7f3513fff640 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2024-09-12T03:02:49.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.493+0000 7f3513fff640 1 -- 172.21.15.5:0/2551832981 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f351c005220 con 0x55fb29a70b40 2024-09-12T03:02:49.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.493+0000 7f353d2e7740 1 -- 172.21.15.5:0/2551832981 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f352409c450 con 0x55fb29a70b40 2024-09-12T03:02:49.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.494+0000 7f3513fff640 1 -- 172.21.15.5:0/2551832981 <== mon.2 v2:172.21.15.146:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f351c0053c0 con 0x55fb29a70b40 2024-09-12T03:02:49.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.494+0000 7f3513fff640 1 -- 172.21.15.5:0/2551832981 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f351c0055c0 con 0x55fb29a70b40 2024-09-12T03:02:49.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.494+0000 7f353d2e7740 10 client.25850 fetch_fsmap learned FSMap version 31 2024-09-12T03:02:49.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.494+0000 7f353d2e7740 1 -- 172.21.15.5:0/2551832981 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55fb29af5250 con 0x55fb29a70b40 2024-09-12T03:02:49.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.494+0000 7f3513fff640 1 -- 172.21.15.5:0/2551832981 <== mon.2 v2:172.21.15.146:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f352409c450 con 0x55fb29a70b40 2024-09-12T03:02:49.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.494+0000 7f3513fff640 1 -- 172.21.15.5:0/2551832981 <== mon.2 v2:172.21.15.146:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f351c0218e0 con 0x55fb29a70b40 2024-09-12T03:02:49.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.494+0000 7f353d2e7740 10 client.25850 fetch_fsmap finished waiting for FSMap version 31 2024-09-12T03:02:49.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.494+0000 7f353d2e7740 10 client.25850 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-12T03:02:49.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.494+0000 7f353d2e7740 20 client.25850 populate_metadata read hostname 'smithi005' 2024-09-12T03:02:49.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.494+0000 7f353d2e7740 1 --2- 172.21.15.5:0/2551832981 >> [v2:172.21.15.146:6836/1673370937,v1:172.21.15.146:6837/1673370937] conn(0x55fb29af5a00 0x55fb29af7e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:49.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.494+0000 7f353d2e7740 4 client.25850 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:02:49.499 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.494+0000 7f353d2e7740 1 -- 172.21.15.5:0/2551832981 --> [v2:172.21.15.146:6836/1673370937,v1:172.21.15.146:6837/1673370937] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fb299dcd70 con 0x55fb29af5a00 2024-09-12T03:02:49.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.495+0000 7f3538caa640 1 --2- 172.21.15.5:0/2551832981 >> [v2:172.21.15.146:6836/1673370937,v1:172.21.15.146:6837/1673370937] conn(0x55fb29af5a00 0x55fb29af7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:49.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.495+0000 7f3538caa640 1 --2- 172.21.15.5:0/2551832981 >> [v2:172.21.15.146:6836/1673370937,v1:172.21.15.146:6837/1673370937] conn(0x55fb29af5a00 0x55fb29af7e20 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7c2aa81bb85cb3b4 server_cookie=787a40cd6281a7ba in_seq=0 out_seq=0 2024-09-12T03:02:49.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.496+0000 7f3513fff640 10 client.25850 ms_handle_connect on v2:172.21.15.146:6836/1673370937 2024-09-12T03:02:49.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.496+0000 7f3513fff640 1 -- 172.21.15.5:0/2551832981 <== mds.4 v2:172.21.15.146:6836/1673370937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fb299dcd70 con 0x55fb29af5a00 2024-09-12T03:02:49.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.496+0000 7f3513fff640 10 client.25850 handle_command_reply: tid=0 2024-09-12T03:02:49.500 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.496+0000 7f353d2e7740 1 client.25850 shutdown 2024-09-12T03:02:49.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.496+0000 7f353d2e7740 1 -- 172.21.15.5:0/2551832981 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fb29a70b40 msgr2=0x55fb29a7e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:49.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.496+0000 7f353d2e7740 1 --2- 172.21.15.5:0/2551832981 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fb29a70b40 0x55fb29a7e8a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f35340fec10 tx=0x7f351c0039c0 comp rx=0 tx=0).stop 2024-09-12T03:02:49.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.496+0000 7f353d2e7740 1 -- 172.21.15.5:0/2551832981 shutdown_connections 2024-09-12T03:02:49.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.496+0000 7f353d2e7740 1 --2- 172.21.15.5:0/2551832981 >> [v2:172.21.15.146:6836/1673370937,v1:172.21.15.146:6837/1673370937] conn(0x55fb29af5a00 0x55fb29af7e20 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:49.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.496+0000 7f3513fff640 0 client.25850 ms_handle_reset on v2:172.21.15.146:6836/1673370937 2024-09-12T03:02:49.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.496+0000 7f353d2e7740 1 --2- 172.21.15.5:0/2551832981 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fb29a7a370 0x55fb29a7ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:49.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.496+0000 7f353d2e7740 1 --2- 172.21.15.5:0/2551832981 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55fb29a70b40 0x55fb29a7e8a0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:49.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.496+0000 7f353d2e7740 1 --2- 172.21.15.5:0/2551832981 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55fb29a6f5d0 0x55fb29a7e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:49.501 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.496+0000 7f353d2e7740 1 -- 172.21.15.5:0/2551832981 >> 172.21.15.5:0/2551832981 conn(0x55fb29a38310 msgr2=0x55fb29a6a500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:49.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.496+0000 7f353d2e7740 1 -- 172.21.15.5:0/2551832981 shutdown_connections 2024-09-12T03:02:49.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.496+0000 7f353d2e7740 1 -- 172.21.15.5:0/2551832981 wait complete. 2024-09-12T03:02:49.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.496+0000 7f353d2e7740 20 client.25850 trim_cache size 0 max 16384 2024-09-12T03:02:49.502 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-12T03:02:49.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.497+0000 7f353af0e640 1 -- 172.21.15.5:0/4289320873 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f35200a2c00 msgr2=0x7f35200a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:49.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.497+0000 7f353af0e640 1 --2- 172.21.15.5:0/4289320873 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f35200a2c00 0x7f35200a50c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f352c051000 tx=0x7f352c04e000 comp rx=0 tx=0).stop 2024-09-12T03:02:49.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.497+0000 7f353af0e640 1 -- 172.21.15.5:0/4289320873 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3534100e20 msgr2=0x7f3534196260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:49.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.497+0000 7f353af0e640 1 --2- 172.21.15.5:0/4289320873 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3534100e20 0x7f3534196260 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3524009fd0 tx=0x7f352400ef90 comp rx=0 tx=0).stop 2024-09-12T03:02:49.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.497+0000 7f353af0e640 1 -- 172.21.15.5:0/4289320873 shutdown_connections 2024-09-12T03:02:49.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.497+0000 7f353af0e640 1 --2- 172.21.15.5:0/4289320873 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f35200a2c00 0x7f35200a50c0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:49.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.497+0000 7f353af0e640 1 --2- 172.21.15.5:0/4289320873 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3534102570 0x7f35341967a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:49.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.497+0000 7f353af0e640 1 --2- 172.21.15.5:0/4289320873 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3534102ef0 0x7f353419ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:49.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.497+0000 7f353af0e640 1 --2- 172.21.15.5:0/4289320873 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3534100e20 0x7f3534196260 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:49.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.497+0000 7f353af0e640 1 -- 172.21.15.5:0/4289320873 >> 172.21.15.5:0/4289320873 conn(0x7f35340fc7d0 msgr2=0x7f3534105ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:49.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.497+0000 7f353af0e640 1 -- 172.21.15.5:0/4289320873 shutdown_connections 2024-09-12T03:02:49.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:49.497+0000 7f353af0e640 1 -- 172.21.15.5:0/4289320873 wait complete. 2024-09-12T03:02:50.342 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-12T03:02:50.625 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:02:50.759 INFO:tasks.workunit.client.0.smithi005.stdout:50000 write 0x5304f4 thru 0x53fa11 (0xf51e bytes) 2024-09-12T03:02:50.760 INFO:tasks.workunit.client.0.smithi005.stdout:All 50000 operations completed A-OK! 2024-09-12T03:02:50.763 INFO:tasks.workunit.client.0.smithi005.stderr:+ ./fsx -z 100MB -N 50000 -p 10000 -l 104857600 2024-09-12T03:02:50.765 INFO:tasks.workunit.client.0.smithi005.stdout:Seed set to 1 2024-09-12T03:02:51.030 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:02:50 smithi005 ceph-mon[31197]: pgmap v473: 97 pgs: 97 active+clean; 115 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.1 MiB/s wr, 222 op/s 2024-09-12T03:02:51.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.115+0000 7f4a99a93640 1 -- 172.21.15.5:0/1100057247 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a941027a0 msgr2=0x7f4a941067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:51.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.115+0000 7f4a99a93640 1 --2- 172.21.15.5:0/1100057247 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a941027a0 0x7f4a941067f0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c009900 tx=0x7f4a7c02a410 comp rx=0 tx=0).stop 2024-09-12T03:02:51.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.115+0000 7f4a99a93640 1 -- 172.21.15.5:0/1100057247 shutdown_connections 2024-09-12T03:02:51.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.115+0000 7f4a99a93640 1 --2- 172.21.15.5:0/1100057247 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a941027a0 0x7f4a941067f0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:51.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.115+0000 7f4a99a93640 1 --2- 172.21.15.5:0/1100057247 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4a94106f20 0x7f4a9410eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:51.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.115+0000 7f4a99a93640 1 --2- 172.21.15.5:0/1100057247 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a94101df0 0x7f4a941021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:51.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.115+0000 7f4a99a93640 1 -- 172.21.15.5:0/1100057247 >> 172.21.15.5:0/1100057247 conn(0x7f4a940fd8f0 msgr2=0x7f4a940ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:51.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.115+0000 7f4a99a93640 1 -- 172.21.15.5:0/1100057247 shutdown_connections 2024-09-12T03:02:51.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.115+0000 7f4a99a93640 1 -- 172.21.15.5:0/1100057247 wait complete. 2024-09-12T03:02:51.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.115+0000 7f4a99a93640 1 Processor -- start 2024-09-12T03:02:51.118 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.116+0000 7f4a99a93640 1 -- start start 2024-09-12T03:02:51.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.116+0000 7f4a99a93640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a94101df0 0x7f4a940744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:51.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.116+0000 7f4a99a93640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4a941027a0 0x7f4a94074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:51.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.116+0000 7f4a99a93640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a94106f20 0x7f4a94074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:51.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.116+0000 7f4a99a93640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4a94111f40 con 0x7f4a94106f20 2024-09-12T03:02:51.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.116+0000 7f4a99a93640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f4a94111dc0 con 0x7f4a941027a0 2024-09-12T03:02:51.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.116+0000 7f4a99a93640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4a941120c0 con 0x7f4a94101df0 2024-09-12T03:02:51.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.116+0000 7f4a93fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a94106f20 0x7f4a94074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:51.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.116+0000 7f4a93fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a94106f20 0x7f4a94074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56998/0 (socket says 172.21.15.5:56998) 2024-09-12T03:02:51.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.116+0000 7f4a93fff640 1 -- 172.21.15.5:0/2878604223 learned_addr learned my addr 172.21.15.5:0/2878604223 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:02:51.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.116+0000 7f4a937fe640 1 --2- 172.21.15.5:0/2878604223 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a94101df0 0x7f4a940744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:51.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.116+0000 7f4a92ffd640 1 --2- 172.21.15.5:0/2878604223 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4a941027a0 0x7f4a94074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:51.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.116+0000 7f4a93fff640 1 -- 172.21.15.5:0/2878604223 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a94101df0 msgr2=0x7f4a940744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:51.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.116+0000 7f4a93fff640 1 --2- 172.21.15.5:0/2878604223 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a94101df0 0x7f4a940744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:51.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.116+0000 7f4a93fff640 1 -- 172.21.15.5:0/2878604223 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4a941027a0 msgr2=0x7f4a94074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:51.119 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.116+0000 7f4a93fff640 1 --2- 172.21.15.5:0/2878604223 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4a941027a0 0x7f4a94074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:51.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.116+0000 7f4a93fff640 1 -- 172.21.15.5:0/2878604223 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a941a6900 con 0x7f4a94106f20 2024-09-12T03:02:51.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.117+0000 7f4a93fff640 1 --2- 172.21.15.5:0/2878604223 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a94106f20 0x7f4a94074f70 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c00eb10 tx=0x7f4a8c00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:51.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.117+0000 7f4a90ff9640 1 -- 172.21.15.5:0/2878604223 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a8c00ce20 con 0x7f4a94106f20 2024-09-12T03:02:51.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.117+0000 7f4a99a93640 1 -- 172.21.15.5:0/2878604223 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a941a6bf0 con 0x7f4a94106f20 2024-09-12T03:02:51.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.117+0000 7f4a90ff9640 1 -- 172.21.15.5:0/2878604223 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a8c005020 con 0x7f4a94106f20 2024-09-12T03:02:51.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.117+0000 7f4a90ff9640 1 -- 172.21.15.5:0/2878604223 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a8c010690 con 0x7f4a94106f20 2024-09-12T03:02:51.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.117+0000 7f4a99a93640 1 -- 172.21.15.5:0/2878604223 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a941a7100 con 0x7f4a94106f20 2024-09-12T03:02:51.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.119+0000 7f4a90ff9640 1 -- 172.21.15.5:0/2878604223 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f4a8c004a40 con 0x7f4a94106f20 2024-09-12T03:02:51.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.120+0000 7f4a90ff9640 1 --2- 172.21.15.5:0/2878604223 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f4a800a2b80 0x7f4a800a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:51.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.120+0000 7f4a937fe640 1 --2- 172.21.15.5:0/2878604223 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f4a800a2b80 0x7f4a800a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:51.122 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.120+0000 7f4a90ff9640 1 -- 172.21.15.5:0/2878604223 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f4a8c0cf480 con 0x7f4a94106f20 2024-09-12T03:02:51.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.120+0000 7f4a9be6c740 1 Processor -- start 2024-09-12T03:02:51.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.120+0000 7f4a9be6c740 1 -- start start 2024-09-12T03:02:51.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.120+0000 7f4a9be6c740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560ca128cb40 0x560ca128b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:51.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.120+0000 7f4a9be6c740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560ca128b5d0 0x560ca128ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:51.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.120+0000 7f4a9be6c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ca1296370 0x560ca1298760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:51.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.120+0000 7f4a9be6c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560ca11f12f0 con 0x560ca1296370 2024-09-12T03:02:51.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.120+0000 7f4a9be6c740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x560ca1198cf0 con 0x560ca128cb40 2024-09-12T03:02:51.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.120+0000 7f4a9be6c740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x560ca11ed2b0 con 0x560ca128b5d0 2024-09-12T03:02:51.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.121+0000 7f4a937fe640 1 --2- 172.21.15.5:0/2878604223 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f4a800a2b80 0x7f4a800a5040 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f4a8400abe0 tx=0x7f4a8400a2d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:51.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.121+0000 7f4a93fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ca1296370 0x560ca1298760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:51.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.121+0000 7f4a93fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ca1296370 0x560ca1298760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57000/0 (socket says 172.21.15.5:57000) 2024-09-12T03:02:51.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.121+0000 7f4a93fff640 1 -- 172.21.15.5:0/685743803 learned_addr learned my addr 172.21.15.5:0/685743803 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:02:51.123 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.121+0000 7f4a937fe640 1 --2- 172.21.15.5:0/685743803 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560ca128cb40 0x560ca128b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:51.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.121+0000 7f4a92ffd640 1 --2- 172.21.15.5:0/685743803 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560ca128b5d0 0x560ca128ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:51.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.121+0000 7f4a93fff640 1 -- 172.21.15.5:0/685743803 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560ca128b5d0 msgr2=0x560ca128ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:51.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.121+0000 7f4a93fff640 1 --2- 172.21.15.5:0/685743803 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560ca128b5d0 0x560ca128ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:51.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.121+0000 7f4a93fff640 1 -- 172.21.15.5:0/685743803 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560ca128cb40 msgr2=0x560ca128b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:51.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.121+0000 7f4a93fff640 1 --2- 172.21.15.5:0/685743803 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560ca128cb40 0x560ca128b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:51.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.121+0000 7f4a93fff640 1 -- 172.21.15.5:0/685743803 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ca117d3a0 con 0x560ca1296370 2024-09-12T03:02:51.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.121+0000 7f4a937fe640 1 --2- 172.21.15.5:0/685743803 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560ca128cb40 0x560ca128b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:02:51.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.121+0000 7f4a93fff640 1 --2- 172.21.15.5:0/685743803 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ca1296370 0x560ca1298760 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c09b9f0 tx=0x7f4a8c09bcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b350e495ba5d9a7c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:51.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.121+0000 7f4a89ffb640 1 -- 172.21.15.5:0/685743803 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a8c099ff0 con 0x560ca1296370 2024-09-12T03:02:51.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.121+0000 7f4a89ffb640 1 -- 172.21.15.5:0/685743803 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a8c004d30 con 0x560ca1296370 2024-09-12T03:02:51.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.121+0000 7f4a89ffb640 1 -- 172.21.15.5:0/685743803 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a8c0cf2e0 con 0x560ca1296370 2024-09-12T03:02:51.124 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.122+0000 7f4a9be6c740 1 -- 172.21.15.5:0/685743803 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ca1296370 msgr2=0x560ca1298760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:51.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.122+0000 7f4a9be6c740 1 --2- 172.21.15.5:0/685743803 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ca1296370 0x560ca1298760 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c09b9f0 tx=0x7f4a8c09bcf0 comp rx=0 tx=0).stop 2024-09-12T03:02:51.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.122+0000 7f4a9be6c740 1 -- 172.21.15.5:0/685743803 shutdown_connections 2024-09-12T03:02:51.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.122+0000 7f4a9be6c740 1 --2- 172.21.15.5:0/685743803 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ca1296370 0x560ca1298760 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:51.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.122+0000 7f4a9be6c740 1 --2- 172.21.15.5:0/685743803 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560ca128b5d0 0x560ca128ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:51.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.122+0000 7f4a9be6c740 1 --2- 172.21.15.5:0/685743803 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560ca128cb40 0x560ca128b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:51.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.122+0000 7f4a9be6c740 1 -- 172.21.15.5:0/685743803 >> 172.21.15.5:0/685743803 conn(0x560ca1254310 msgr2=0x560ca1287c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:51.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.122+0000 7f4a9be6c740 1 -- 172.21.15.5:0/685743803 shutdown_connections 2024-09-12T03:02:51.125 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.122+0000 7f4a9be6c740 1 -- 172.21.15.5:0/685743803 wait complete. 2024-09-12T03:02:51.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.123+0000 7f4a9be6c740 1 -- start start 2024-09-12T03:02:51.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.123+0000 7f4a9be6c740 1 Processor -- start 2024-09-12T03:02:51.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.123+0000 7f4a9be6c740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560ca128b5d0 0x560ca1295d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:51.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.123+0000 7f4a9be6c740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560ca128cb40 0x560ca1293010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:51.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.123+0000 7f4a9be6c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ca1296370 0x560ca1293600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:51.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.123+0000 7f4a9be6c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560ca1299070 con 0x560ca1296370 2024-09-12T03:02:51.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.123+0000 7f4a9be6c740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x560ca11ac060 con 0x560ca128cb40 2024-09-12T03:02:51.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.123+0000 7f4a9be6c740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x560ca12991f0 con 0x560ca128b5d0 2024-09-12T03:02:51.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.123+0000 7f4a9be6c740 10 client.0 _ll_register_callbacks cb 0x560ca0eff8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:02:51.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.123+0000 7f4a93fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ca1296370 0x560ca1293600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:51.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.123+0000 7f4a93fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ca1296370 0x560ca1293600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57012/0 (socket says 172.21.15.5:57012) 2024-09-12T03:02:51.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.123+0000 7f4a93fff640 1 -- 172.21.15.5:0/1498625227 learned_addr learned my addr 172.21.15.5:0/1498625227 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:02:51.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.123+0000 7f4a937fe640 1 --2- 172.21.15.5:0/1498625227 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560ca128b5d0 0x560ca1295d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:51.129 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.123+0000 7f4a92ffd640 1 --2- 172.21.15.5:0/1498625227 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560ca128cb40 0x560ca1293010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:51.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.124+0000 7f4a937fe640 1 -- 172.21.15.5:0/1498625227 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560ca128cb40 msgr2=0x560ca1293010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:51.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.124+0000 7f4a937fe640 1 --2- 172.21.15.5:0/1498625227 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560ca128cb40 0x560ca1293010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:51.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.124+0000 7f4a937fe640 1 -- 172.21.15.5:0/1498625227 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ca1296370 msgr2=0x560ca1293600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:51.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.124+0000 7f4a937fe640 1 --2- 172.21.15.5:0/1498625227 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ca1296370 0x560ca1293600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:51.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.124+0000 7f4a937fe640 1 -- 172.21.15.5:0/1498625227 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ca0f623c0 con 0x560ca128b5d0 2024-09-12T03:02:51.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.124+0000 7f4a93fff640 1 --2- 172.21.15.5:0/1498625227 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ca1296370 0x560ca1293600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:02:51.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.124+0000 7f4a937fe640 1 --2- 172.21.15.5:0/1498625227 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560ca128b5d0 0x560ca1295d60 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4a8400f850 tx=0x7f4a84004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b9d758c26aadfb8a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:51.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.124+0000 7f4a72ffd640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-09-12T03:02:51.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.124+0000 7f4a72ffd640 1 -- 172.21.15.5:0/1498625227 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a8402a060 con 0x560ca128b5d0 2024-09-12T03:02:51.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.124+0000 7f4a72ffd640 1 -- 172.21.15.5:0/1498625227 <== mon.1 v2:172.21.15.148:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f4a8401b650 con 0x560ca128b5d0 2024-09-12T03:02:51.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.124+0000 7f4a9be6c740 1 -- 172.21.15.5:0/1498625227 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a8c004d30 con 0x560ca128b5d0 2024-09-12T03:02:51.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.124+0000 7f4a72ffd640 1 -- 172.21.15.5:0/1498625227 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f4a84016070 con 0x560ca128b5d0 2024-09-12T03:02:51.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.124+0000 7f4a9be6c740 10 client.25954 fetch_fsmap learned FSMap version 31 2024-09-12T03:02:51.130 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.124+0000 7f4a9be6c740 1 -- 172.21.15.5:0/1498625227 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x560ca12a4ba0 con 0x560ca128b5d0 2024-09-12T03:02:51.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.125+0000 7f4a72ffd640 1 -- 172.21.15.5:0/1498625227 <== mon.1 v2:172.21.15.148:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a8c004d30 con 0x560ca128b5d0 2024-09-12T03:02:51.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.125+0000 7f4a72ffd640 1 -- 172.21.15.5:0/1498625227 <== mon.1 v2:172.21.15.148:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f4a84024030 con 0x560ca128b5d0 2024-09-12T03:02:51.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.125+0000 7f4a9be6c740 10 client.25954 fetch_fsmap finished waiting for FSMap version 31 2024-09-12T03:02:51.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.125+0000 7f4a9be6c740 10 client.25954 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.b 2024-09-12T03:02:51.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.125+0000 7f4a9be6c740 20 client.25954 populate_metadata read hostname 'smithi005' 2024-09-12T03:02:51.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.125+0000 7f4a9be6c740 1 --2- 172.21.15.5:0/1498625227 >> [v2:172.21.15.146:6838/3036767228,v1:172.21.15.146:6839/3036767228] conn(0x560ca12a6060 0x560ca12a8450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:51.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.125+0000 7f4a9be6c740 4 client.25954 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:02:51.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.125+0000 7f4a9be6c740 1 -- 172.21.15.5:0/1498625227 --> [v2:172.21.15.146:6838/3036767228,v1:172.21.15.146:6839/3036767228] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560ca11fe860 con 0x560ca12a6060 2024-09-12T03:02:51.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.125+0000 7f4a92ffd640 1 --2- 172.21.15.5:0/1498625227 >> [v2:172.21.15.146:6838/3036767228,v1:172.21.15.146:6839/3036767228] conn(0x560ca12a6060 0x560ca12a8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:51.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.126+0000 7f4a92ffd640 1 --2- 172.21.15.5:0/1498625227 >> [v2:172.21.15.146:6838/3036767228,v1:172.21.15.146:6839/3036767228] conn(0x560ca12a6060 0x560ca12a8450 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=10e8b69ff8b8fd3f server_cookie=bf285659e5dbd549 in_seq=0 out_seq=0 2024-09-12T03:02:51.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.126+0000 7f4a72ffd640 10 client.25954 ms_handle_connect on v2:172.21.15.146:6838/3036767228 2024-09-12T03:02:51.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.126+0000 7f4a72ffd640 1 -- 172.21.15.5:0/1498625227 <== mds.3 v2:172.21.15.146:6838/3036767228 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560ca11fe860 con 0x560ca12a6060 2024-09-12T03:02:51.131 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.126+0000 7f4a72ffd640 10 client.25954 handle_command_reply: tid=0 2024-09-12T03:02:51.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.126+0000 7f4a9be6c740 1 client.25954 shutdown 2024-09-12T03:02:51.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.126+0000 7f4a9be6c740 1 -- 172.21.15.5:0/1498625227 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560ca128b5d0 msgr2=0x560ca1295d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:51.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.126+0000 7f4a9be6c740 1 --2- 172.21.15.5:0/1498625227 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560ca128b5d0 0x560ca1295d60 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4a8400f850 tx=0x7f4a84004730 comp rx=0 tx=0).stop 2024-09-12T03:02:51.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.126+0000 7f4a9be6c740 1 -- 172.21.15.5:0/1498625227 shutdown_connections 2024-09-12T03:02:51.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.126+0000 7f4a9be6c740 1 --2- 172.21.15.5:0/1498625227 >> [v2:172.21.15.146:6838/3036767228,v1:172.21.15.146:6839/3036767228] conn(0x560ca12a6060 0x560ca12a8450 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:51.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.126+0000 7f4a9be6c740 1 --2- 172.21.15.5:0/1498625227 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ca1296370 0x560ca1293600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:51.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.126+0000 7f4a9be6c740 1 --2- 172.21.15.5:0/1498625227 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560ca128cb40 0x560ca1293010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:51.132 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.126+0000 7f4a9be6c740 1 --2- 172.21.15.5:0/1498625227 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560ca128b5d0 0x560ca1295d60 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:51.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.126+0000 7f4a72ffd640 0 client.25954 ms_handle_reset on v2:172.21.15.146:6838/3036767228 2024-09-12T03:02:51.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.126+0000 7f4a9be6c740 1 -- 172.21.15.5:0/1498625227 >> 172.21.15.5:0/1498625227 conn(0x560ca1254310 msgr2=0x560ca128d990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:51.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.127+0000 7f4a9be6c740 1 -- 172.21.15.5:0/1498625227 shutdown_connections 2024-09-12T03:02:51.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.127+0000 7f4a9be6c740 1 -- 172.21.15.5:0/1498625227 wait complete. 2024-09-12T03:02:51.133 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.127+0000 7f4a9be6c740 20 client.25954 trim_cache size 0 max 16384 2024-09-12T03:02:51.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.140+0000 7f4a9be6c740 1 Processor -- start 2024-09-12T03:02:51.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.140+0000 7f4a9be6c740 1 -- start start 2024-09-12T03:02:51.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.140+0000 7f4a9be6c740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560ca128b5d0 0x560ca1286bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:51.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.140+0000 7f4a9be6c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ca128cb40 0x560ca128e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:51.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.140+0000 7f4a9be6c740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560ca1296370 0x560ca128e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:51.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.140+0000 7f4a9be6c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4a7400e400 con 0x560ca128cb40 2024-09-12T03:02:51.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.140+0000 7f4a9be6c740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f4a7400ff40 con 0x560ca1296370 2024-09-12T03:02:51.141 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.140+0000 7f4a9be6c740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4a7400ead0 con 0x560ca128b5d0 2024-09-12T03:02:51.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.140+0000 7f4a92ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ca128cb40 0x560ca128e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:51.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.140+0000 7f4a92ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ca128cb40 0x560ca128e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57016/0 (socket says 172.21.15.5:57016) 2024-09-12T03:02:51.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.140+0000 7f4a92ffd640 1 -- 172.21.15.5:0/3643188639 learned_addr learned my addr 172.21.15.5:0/3643188639 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:02:51.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.140+0000 7f4a937fe640 1 --2- 172.21.15.5:0/3643188639 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560ca128b5d0 0x560ca1286bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:51.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.140+0000 7f4a93fff640 1 --2- 172.21.15.5:0/3643188639 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560ca1296370 0x560ca128e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:51.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.141+0000 7f4a92ffd640 1 -- 172.21.15.5:0/3643188639 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560ca128b5d0 msgr2=0x560ca1286bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:51.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.141+0000 7f4a92ffd640 1 --2- 172.21.15.5:0/3643188639 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560ca128b5d0 0x560ca1286bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:51.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.141+0000 7f4a92ffd640 1 -- 172.21.15.5:0/3643188639 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560ca1296370 msgr2=0x560ca128e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:51.142 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.141+0000 7f4a92ffd640 1 --2- 172.21.15.5:0/3643188639 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560ca1296370 0x560ca128e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:51.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.141+0000 7f4a92ffd640 1 -- 172.21.15.5:0/3643188639 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ca12a8ad0 con 0x560ca128cb40 2024-09-12T03:02:51.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.141+0000 7f4a93fff640 1 --2- 172.21.15.5:0/3643188639 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560ca1296370 0x560ca128e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:02:51.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.141+0000 7f4a92ffd640 1 --2- 172.21.15.5:0/3643188639 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ca128cb40 0x560ca128e120 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c02a930 tx=0x7f4a7c031b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7aca2a22f201dd50 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:51.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.141+0000 7f4a897fa640 1 -- 172.21.15.5:0/3643188639 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a7c04d030 con 0x560ca128cb40 2024-09-12T03:02:51.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.141+0000 7f4a897fa640 1 -- 172.21.15.5:0/3643188639 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a7c005020 con 0x560ca128cb40 2024-09-12T03:02:51.143 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.141+0000 7f4a897fa640 1 -- 172.21.15.5:0/3643188639 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a7c035040 con 0x560ca128cb40 2024-09-12T03:02:51.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.142+0000 7f4a9be6c740 1 -- 172.21.15.5:0/3643188639 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ca128cb40 msgr2=0x560ca128e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:51.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.142+0000 7f4a9be6c740 1 --2- 172.21.15.5:0/3643188639 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ca128cb40 0x560ca128e120 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c02a930 tx=0x7f4a7c031b00 comp rx=0 tx=0).stop 2024-09-12T03:02:51.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.142+0000 7f4a9be6c740 1 -- 172.21.15.5:0/3643188639 shutdown_connections 2024-09-12T03:02:51.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.142+0000 7f4a9be6c740 1 --2- 172.21.15.5:0/3643188639 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ca128cb40 0x560ca128e120 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:51.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.142+0000 7f4a9be6c740 1 --2- 172.21.15.5:0/3643188639 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560ca1296370 0x560ca128e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:51.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.142+0000 7f4a9be6c740 1 --2- 172.21.15.5:0/3643188639 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560ca128b5d0 0x560ca1286bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:51.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.142+0000 7f4a9be6c740 1 -- 172.21.15.5:0/3643188639 >> 172.21.15.5:0/3643188639 conn(0x560ca1254310 msgr2=0x560ca12864e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:51.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.142+0000 7f4a9be6c740 1 -- 172.21.15.5:0/3643188639 shutdown_connections 2024-09-12T03:02:51.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.142+0000 7f4a9be6c740 1 -- 172.21.15.5:0/3643188639 wait complete. 2024-09-12T03:02:51.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.142+0000 7f4a9be6c740 1 -- start start 2024-09-12T03:02:51.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.142+0000 7f4a9be6c740 1 Processor -- start 2024-09-12T03:02:51.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.143+0000 7f4a9be6c740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560ca128b5d0 0x560ca129edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:51.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.143+0000 7f4a9be6c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ca128cb40 0x560ca128ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:51.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.143+0000 7f4a9be6c740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560ca1296370 0x560ca1290690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:51.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.143+0000 7f4a9be6c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4a7400f870 con 0x560ca128cb40 2024-09-12T03:02:51.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.143+0000 7f4a9be6c740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f4a7400f1a0 con 0x560ca1296370 2024-09-12T03:02:51.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.143+0000 7f4a9be6c740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4a7400c3e0 con 0x560ca128b5d0 2024-09-12T03:02:51.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.143+0000 7f4a9be6c740 10 client.0 _ll_register_callbacks cb 0x7f4a8400fc20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:02:51.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.143+0000 7f4a92ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ca128cb40 0x560ca128ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:51.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.143+0000 7f4a92ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ca128cb40 0x560ca128ff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57026/0 (socket says 172.21.15.5:57026) 2024-09-12T03:02:51.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.143+0000 7f4a92ffd640 1 -- 172.21.15.5:0/2886297533 learned_addr learned my addr 172.21.15.5:0/2886297533 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:02:51.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.143+0000 7f4a93fff640 1 --2- 172.21.15.5:0/2886297533 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560ca1296370 0x560ca1290690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:51.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.143+0000 7f4a937fe640 1 --2- 172.21.15.5:0/2886297533 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560ca128b5d0 0x560ca129edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:51.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.143+0000 7f4a93fff640 1 -- 172.21.15.5:0/2886297533 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560ca128b5d0 msgr2=0x560ca129edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:51.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.143+0000 7f4a93fff640 1 --2- 172.21.15.5:0/2886297533 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560ca128b5d0 0x560ca129edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:51.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.143+0000 7f4a93fff640 1 -- 172.21.15.5:0/2886297533 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ca128cb40 msgr2=0x560ca128ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:51.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.143+0000 7f4a93fff640 1 --2- 172.21.15.5:0/2886297533 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ca128cb40 0x560ca128ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:51.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.143+0000 7f4a93fff640 1 -- 172.21.15.5:0/2886297533 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ca130fcf0 con 0x560ca1296370 2024-09-12T03:02:51.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.143+0000 7f4a92ffd640 1 --2- 172.21.15.5:0/2886297533 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ca128cb40 0x560ca128ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:02:51.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.143+0000 7f4a93fff640 1 --2- 172.21.15.5:0/2886297533 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560ca1296370 0x560ca1290690 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c0049d0 tx=0x7f4a8c09bbe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3c3ca9a87485555 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:51.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.143+0000 7f4a72ffd640 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2024-09-12T03:02:51.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.144+0000 7f4a72ffd640 1 -- 172.21.15.5:0/2886297533 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a8c0cf0b0 con 0x560ca1296370 2024-09-12T03:02:51.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.144+0000 7f4a72ffd640 1 -- 172.21.15.5:0/2886297533 <== mon.2 v2:172.21.15.146:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f4a8c0cf250 con 0x560ca1296370 2024-09-12T03:02:51.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.144+0000 7f4a9be6c740 1 -- 172.21.15.5:0/2886297533 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a7c005020 con 0x560ca1296370 2024-09-12T03:02:51.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.144+0000 7f4a72ffd640 1 -- 172.21.15.5:0/2886297533 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f4a8c07ab60 con 0x560ca1296370 2024-09-12T03:02:51.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.144+0000 7f4a9be6c740 10 client.25868 fetch_fsmap learned FSMap version 31 2024-09-12T03:02:51.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.144+0000 7f4a9be6c740 1 -- 172.21.15.5:0/2886297533 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x560ca1310ed0 con 0x560ca1296370 2024-09-12T03:02:51.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.144+0000 7f4a72ffd640 1 -- 172.21.15.5:0/2886297533 <== mon.2 v2:172.21.15.146:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a7c005020 con 0x560ca1296370 2024-09-12T03:02:51.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.144+0000 7f4a72ffd640 1 -- 172.21.15.5:0/2886297533 <== mon.2 v2:172.21.15.146:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f4a8c09acd0 con 0x560ca1296370 2024-09-12T03:02:51.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.144+0000 7f4a9be6c740 10 client.25868 fetch_fsmap finished waiting for FSMap version 31 2024-09-12T03:02:51.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.144+0000 7f4a9be6c740 10 client.25868 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.b 2024-09-12T03:02:51.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.144+0000 7f4a9be6c740 20 client.25868 populate_metadata read hostname 'smithi005' 2024-09-12T03:02:51.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.144+0000 7f4a9be6c740 1 --2- 172.21.15.5:0/2886297533 >> [v2:172.21.15.146:6838/3036767228,v1:172.21.15.146:6839/3036767228] conn(0x560ca1311220 0x560ca1313610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:51.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.144+0000 7f4a9be6c740 4 client.25868 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:02:51.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.144+0000 7f4a9be6c740 1 -- 172.21.15.5:0/2886297533 --> [v2:172.21.15.146:6838/3036767228,v1:172.21.15.146:6839/3036767228] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560ca11f8d70 con 0x560ca1311220 2024-09-12T03:02:51.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.144+0000 7f4a937fe640 1 --2- 172.21.15.5:0/2886297533 >> [v2:172.21.15.146:6838/3036767228,v1:172.21.15.146:6839/3036767228] conn(0x560ca1311220 0x560ca1313610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:51.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.145+0000 7f4a72ffd640 10 client.25868 ms_handle_connect on v2:172.21.15.146:6838/3036767228 2024-09-12T03:02:51.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.145+0000 7f4a937fe640 1 --2- 172.21.15.5:0/2886297533 >> [v2:172.21.15.146:6838/3036767228,v1:172.21.15.146:6839/3036767228] conn(0x560ca1311220 0x560ca1313610 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5b66d065885d9231 server_cookie=30daec42262ec67d in_seq=0 out_seq=0 2024-09-12T03:02:51.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.145+0000 7f4a72ffd640 1 -- 172.21.15.5:0/2886297533 <== mds.3 v2:172.21.15.146:6838/3036767228 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560ca11f8d70 con 0x560ca1311220 2024-09-12T03:02:51.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.145+0000 7f4a72ffd640 10 client.25868 handle_command_reply: tid=0 2024-09-12T03:02:51.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.145+0000 7f4a9be6c740 1 client.25868 shutdown 2024-09-12T03:02:51.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.145+0000 7f4a9be6c740 1 -- 172.21.15.5:0/2886297533 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560ca1296370 msgr2=0x560ca1290690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:51.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.145+0000 7f4a9be6c740 1 --2- 172.21.15.5:0/2886297533 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560ca1296370 0x560ca1290690 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c0049d0 tx=0x7f4a8c09bbe0 comp rx=0 tx=0).stop 2024-09-12T03:02:51.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.145+0000 7f4a9be6c740 1 -- 172.21.15.5:0/2886297533 shutdown_connections 2024-09-12T03:02:51.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.145+0000 7f4a9be6c740 1 --2- 172.21.15.5:0/2886297533 >> [v2:172.21.15.146:6838/3036767228,v1:172.21.15.146:6839/3036767228] conn(0x560ca1311220 0x560ca1313610 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:51.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.145+0000 7f4a9be6c740 1 --2- 172.21.15.5:0/2886297533 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ca128cb40 0x560ca128ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:51.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.145+0000 7f4a9be6c740 1 --2- 172.21.15.5:0/2886297533 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560ca1296370 0x560ca1290690 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:51.150 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.145+0000 7f4a72ffd640 0 client.25868 ms_handle_reset on v2:172.21.15.146:6838/3036767228 2024-09-12T03:02:51.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.145+0000 7f4a9be6c740 1 --2- 172.21.15.5:0/2886297533 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560ca128b5d0 0x560ca129edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:51.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.145+0000 7f4a9be6c740 1 -- 172.21.15.5:0/2886297533 >> 172.21.15.5:0/2886297533 conn(0x560ca1254310 msgr2=0x560ca1286d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:51.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.145+0000 7f4a9be6c740 1 -- 172.21.15.5:0/2886297533 shutdown_connections 2024-09-12T03:02:51.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.145+0000 7f4a9be6c740 1 -- 172.21.15.5:0/2886297533 wait complete. 2024-09-12T03:02:51.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.145+0000 7f4a9be6c740 20 client.25868 trim_cache size 0 max 16384 2024-09-12T03:02:51.151 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-12T03:02:51.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.146+0000 7f4a99a93640 1 -- 172.21.15.5:0/2878604223 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f4a800a2b80 msgr2=0x7f4a800a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:51.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.146+0000 7f4a99a93640 1 --2- 172.21.15.5:0/2878604223 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f4a800a2b80 0x7f4a800a5040 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f4a8400abe0 tx=0x7f4a8400a2d0 comp rx=0 tx=0).stop 2024-09-12T03:02:51.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.146+0000 7f4a99a93640 1 -- 172.21.15.5:0/2878604223 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a94106f20 msgr2=0x7f4a94074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:51.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.146+0000 7f4a99a93640 1 --2- 172.21.15.5:0/2878604223 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a94106f20 0x7f4a94074f70 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c00eb10 tx=0x7f4a8c00ef10 comp rx=0 tx=0).stop 2024-09-12T03:02:51.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.146+0000 7f4a99a93640 1 -- 172.21.15.5:0/2878604223 shutdown_connections 2024-09-12T03:02:51.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.146+0000 7f4a99a93640 1 --2- 172.21.15.5:0/2878604223 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f4a800a2b80 0x7f4a800a5040 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:51.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.146+0000 7f4a99a93640 1 --2- 172.21.15.5:0/2878604223 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a94106f20 0x7f4a94074f70 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:51.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.146+0000 7f4a99a93640 1 --2- 172.21.15.5:0/2878604223 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4a941027a0 0x7f4a94074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:51.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.146+0000 7f4a99a93640 1 --2- 172.21.15.5:0/2878604223 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4a94101df0 0x7f4a940744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:51.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.146+0000 7f4a99a93640 1 -- 172.21.15.5:0/2878604223 >> 172.21.15.5:0/2878604223 conn(0x7f4a940fd8f0 msgr2=0x7f4a940fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:51.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.146+0000 7f4a99a93640 1 -- 172.21.15.5:0/2878604223 shutdown_connections 2024-09-12T03:02:51.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:51.146+0000 7f4a99a93640 1 -- 172.21.15.5:0/2878604223 wait complete. 2024-09-12T03:02:51.165 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:02:50 smithi148 ceph-mon[35090]: pgmap v473: 97 pgs: 97 active+clean; 115 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.1 MiB/s wr, 222 op/s 2024-09-12T03:02:51.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:02:50 smithi146 ceph-mon[35011]: pgmap v473: 97 pgs: 97 active+clean; 115 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.1 MiB/s wr, 222 op/s 2024-09-12T03:02:51.838 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T03:02:51.967 INFO:tasks.workunit.client.0.smithi005.stderr:main: filesystem does not support fallocate mode 0, disabling! 2024-09-12T03:02:51.968 INFO:tasks.workunit.client.0.smithi005.stderr:main: filesystem does not support fallocate mode FALLOC_FL_KEEP_SIZE, disabling! 2024-09-12T03:02:51.970 INFO:tasks.workunit.client.0.smithi005.stderr:main: filesystem does not support fallocate mode FALLOC_FL_COLLAPSE_RANGE, disabling! 2024-09-12T03:02:51.970 INFO:tasks.workunit.client.0.smithi005.stderr:main: filesystem does not support fallocate mode FALLOC_FL_INSERT_RANGE, disabling! 2024-09-12T03:02:51.970 INFO:tasks.workunit.client.0.smithi005.stderr:main: filesystem does not support clone range, disabling! 2024-09-12T03:02:51.974 INFO:tasks.workunit.client.0.smithi005.stderr:main: filesystem does not support dedupe range, disabling! 2024-09-12T03:02:51.976 INFO:tasks.workunit.client.0.smithi005.stderr:main: filesystem does not support exchange range, disabling! 2024-09-12T03:02:51.976 INFO:tasks.workunit.client.0.smithi005.stdout:skipping zero size read 2024-09-12T03:02:51.987 INFO:tasks.workunit.client.0.smithi005.stdout:truncating to largest ever: 0x1c37b90 2024-09-12T03:02:51.995 INFO:tasks.workunit.client.0.smithi005.stdout:truncating to largest ever: 0x5bd330d 2024-09-12T03:02:52.008 INFO:tasks.workunit.client.0.smithi005.stdout:copying to largest ever: 0x637bf99 2024-09-12T03:02:52.119 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:02:52.180 INFO:tasks.workunit.client.0.smithi005.stdout:copying to largest ever: 0x63d9b9c 2024-09-12T03:02:52.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.640+0000 7f2221b59640 1 -- 172.21.15.5:0/1928763308 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f221c102780 msgr2=0x7f221c1067d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:52.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.640+0000 7f2221b59640 1 --2- 172.21.15.5:0/1928763308 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f221c102780 0x7f221c1067d0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f2204009900 tx=0x7f220402a410 comp rx=0 tx=0).stop 2024-09-12T03:02:52.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.640+0000 7f2221b59640 1 -- 172.21.15.5:0/1928763308 shutdown_connections 2024-09-12T03:02:52.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.640+0000 7f2221b59640 1 --2- 172.21.15.5:0/1928763308 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f221c102780 0x7f221c1067d0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:52.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.640+0000 7f2221b59640 1 --2- 172.21.15.5:0/1928763308 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f221c106f00 0x7f221c10ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:52.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.640+0000 7f2221b59640 1 --2- 172.21.15.5:0/1928763308 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f221c101dd0 0x7f221c1021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:52.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.640+0000 7f2221b59640 1 -- 172.21.15.5:0/1928763308 >> 172.21.15.5:0/1928763308 conn(0x7f221c0fd890 msgr2=0x7f221c0ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:52.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.640+0000 7f2221b59640 1 -- 172.21.15.5:0/1928763308 shutdown_connections 2024-09-12T03:02:52.642 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.640+0000 7f2221b59640 1 -- 172.21.15.5:0/1928763308 wait complete. 2024-09-12T03:02:52.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.641+0000 7f2221b59640 1 Processor -- start 2024-09-12T03:02:52.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.642+0000 7f2221b59640 1 -- start start 2024-09-12T03:02:52.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.642+0000 7f2221b59640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f221c101dd0 0x7f221c1984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:52.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.642+0000 7f2221b59640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f221c102780 0x7f221c1989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:52.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.642+0000 7f2221b59640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f221c106f00 0x7f221c19cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:52.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.642+0000 7f2221b59640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f221c111f80 con 0x7f221c102780 2024-09-12T03:02:52.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.642+0000 7f2221b59640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f221c111e00 con 0x7f221c101dd0 2024-09-12T03:02:52.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.642+0000 7f2221b59640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f221c112100 con 0x7f221c106f00 2024-09-12T03:02:52.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.642+0000 7f2212ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f221c102780 0x7f221c1989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:52.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.642+0000 7f2212ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f221c102780 0x7f221c1989e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57038/0 (socket says 172.21.15.5:57038) 2024-09-12T03:02:52.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.642+0000 7f2212ffd640 1 -- 172.21.15.5:0/479414756 learned_addr learned my addr 172.21.15.5:0/479414756 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:02:52.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.642+0000 7f221bfff640 1 --2- 172.21.15.5:0/479414756 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f221c106f00 0x7f221c19cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:52.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.642+0000 7f221b7fe640 1 --2- 172.21.15.5:0/479414756 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f221c101dd0 0x7f221c1984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:52.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.643+0000 7f2212ffd640 1 -- 172.21.15.5:0/479414756 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f221c106f00 msgr2=0x7f221c19cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:52.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.643+0000 7f2212ffd640 1 --2- 172.21.15.5:0/479414756 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f221c106f00 0x7f221c19cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:52.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.643+0000 7f2212ffd640 1 -- 172.21.15.5:0/479414756 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f221c101dd0 msgr2=0x7f221c1984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:52.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.643+0000 7f2212ffd640 1 --2- 172.21.15.5:0/479414756 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f221c101dd0 0x7f221c1984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:52.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.643+0000 7f2212ffd640 1 -- 172.21.15.5:0/479414756 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f221c19d450 con 0x7f221c102780 2024-09-12T03:02:52.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.643+0000 7f2212ffd640 1 --2- 172.21.15.5:0/479414756 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f221c102780 0x7f221c1989e0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f2204006010 tx=0x7f22040042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:52.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.643+0000 7f22197fa640 1 -- 172.21.15.5:0/479414756 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f220403f030 con 0x7f221c102780 2024-09-12T03:02:52.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.643+0000 7f22197fa640 1 -- 172.21.15.5:0/479414756 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2204003ce0 con 0x7f221c102780 2024-09-12T03:02:52.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.643+0000 7f2221b59640 1 -- 172.21.15.5:0/479414756 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f221c19d6e0 con 0x7f221c102780 2024-09-12T03:02:52.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.643+0000 7f22197fa640 1 -- 172.21.15.5:0/479414756 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2204035070 con 0x7f221c102780 2024-09-12T03:02:52.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.644+0000 7f2221b59640 1 -- 172.21.15.5:0/479414756 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f221c1a5020 con 0x7f221c102780 2024-09-12T03:02:52.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.646+0000 7f22197fa640 1 -- 172.21.15.5:0/479414756 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f2204042030 con 0x7f221c102780 2024-09-12T03:02:52.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.646+0000 7f2223f32740 1 Processor -- start 2024-09-12T03:02:52.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.647+0000 7f2223f32740 1 -- start start 2024-09-12T03:02:52.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.647+0000 7f2223f32740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560a1cdf5b40 0x560a1cdf4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:52.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.647+0000 7f2223f32740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560a1cdf45d0 0x560a1cdf4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:52.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.647+0000 7f2223f32740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a1cdff370 0x560a1ce01760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:52.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.647+0000 7f2223f32740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560a1cd5a2f0 con 0x560a1cdff370 2024-09-12T03:02:52.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.647+0000 7f2223f32740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x560a1cd01cf0 con 0x560a1cdf5b40 2024-09-12T03:02:52.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.647+0000 7f2223f32740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x560a1cd562b0 con 0x560a1cdf45d0 2024-09-12T03:02:52.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.647+0000 7f22197fa640 1 --2- 172.21.15.5:0/479414756 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f22080a2c00 0x7f22080a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:52.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.647+0000 7f2212ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a1cdff370 0x560a1ce01760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:52.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.647+0000 7f221b7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560a1cdf45d0 0x560a1cdf4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:52.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.647+0000 7f221b7fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560a1cdf45d0 0x560a1cdf4a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.5:33360/0 (socket says 172.21.15.5:33360) 2024-09-12T03:02:52.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.647+0000 7f221b7fe640 1 -- 172.21.15.5:0/3161242481 learned_addr learned my addr 172.21.15.5:0/3161242481 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:02:52.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.647+0000 7f221bfff640 1 --2- 172.21.15.5:0/3161242481 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560a1cdf5b40 0x560a1cdf4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:52.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.647+0000 7f22197fa640 1 -- 172.21.15.5:0/479414756 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f22040ef5c0 con 0x7f221c102780 2024-09-12T03:02:52.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.647+0000 7f2212ffd640 1 -- 172.21.15.5:0/3161242481 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560a1cdf45d0 msgr2=0x560a1cdf4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:52.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.647+0000 7f2212ffd640 1 --2- 172.21.15.5:0/3161242481 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560a1cdf45d0 0x560a1cdf4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:52.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.647+0000 7f2212ffd640 1 -- 172.21.15.5:0/3161242481 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560a1cdf5b40 msgr2=0x560a1cdf4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:52.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.647+0000 7f2212ffd640 1 --2- 172.21.15.5:0/3161242481 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560a1cdf5b40 0x560a1cdf4090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:52.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.647+0000 7f2212ffd640 1 -- 172.21.15.5:0/3161242481 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a1cce63a0 con 0x560a1cdff370 2024-09-12T03:02:52.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.647+0000 7f221bfff640 1 --2- 172.21.15.5:0/479414756 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f22080a2c00 0x7f22080a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:52.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.648+0000 7f2212ffd640 1 --2- 172.21.15.5:0/3161242481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a1cdff370 0x560a1ce01760 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f22040b88c0 tx=0x7f22040b8190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2c1d6e07d0056a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:52.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.648+0000 7f2211ffb640 1 -- 172.21.15.5:0/3161242481 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2204049030 con 0x560a1cdff370 2024-09-12T03:02:52.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.648+0000 7f2211ffb640 1 -- 172.21.15.5:0/3161242481 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f220403b580 con 0x560a1cdff370 2024-09-12T03:02:52.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.648+0000 7f2211ffb640 1 -- 172.21.15.5:0/3161242481 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22040c02a0 con 0x560a1cdff370 2024-09-12T03:02:52.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.649+0000 7f2223f32740 1 -- 172.21.15.5:0/3161242481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a1cdff370 msgr2=0x560a1ce01760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:52.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.649+0000 7f2223f32740 1 --2- 172.21.15.5:0/3161242481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a1cdff370 0x560a1ce01760 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f22040b88c0 tx=0x7f22040b8190 comp rx=0 tx=0).stop 2024-09-12T03:02:52.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.649+0000 7f221bfff640 1 --2- 172.21.15.5:0/479414756 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f22080a2c00 0x7f22080a50c0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f221400a4e0 tx=0x7f2214009650 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:52.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.649+0000 7f2223f32740 1 -- 172.21.15.5:0/3161242481 shutdown_connections 2024-09-12T03:02:52.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.649+0000 7f2223f32740 1 --2- 172.21.15.5:0/3161242481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a1cdff370 0x560a1ce01760 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:52.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.649+0000 7f2223f32740 1 --2- 172.21.15.5:0/3161242481 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560a1cdf45d0 0x560a1cdf4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:52.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.649+0000 7f2223f32740 1 --2- 172.21.15.5:0/3161242481 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560a1cdf5b40 0x560a1cdf4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:52.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.649+0000 7f2223f32740 1 -- 172.21.15.5:0/3161242481 >> 172.21.15.5:0/3161242481 conn(0x560a1cdbd310 msgr2=0x560a1cdf0c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:52.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.649+0000 7f2223f32740 1 -- 172.21.15.5:0/3161242481 shutdown_connections 2024-09-12T03:02:52.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.649+0000 7f2223f32740 1 -- 172.21.15.5:0/3161242481 wait complete. 2024-09-12T03:02:52.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.649+0000 7f2223f32740 1 -- start start 2024-09-12T03:02:52.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.649+0000 7f2223f32740 1 Processor -- start 2024-09-12T03:02:52.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.650+0000 7f2223f32740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a1cdf45d0 0x560a1cdfedc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:52.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.650+0000 7f2223f32740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560a1cdf5b40 0x560a1cdfc070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:52.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.650+0000 7f2223f32740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560a1cdff370 0x560a1cdfc800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:52.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.650+0000 7f2223f32740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x560a1ce01e30 con 0x560a1cdf45d0 2024-09-12T03:02:52.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.650+0000 7f2223f32740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x560a1cd15060 con 0x560a1cdff370 2024-09-12T03:02:52.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.650+0000 7f2223f32740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x560a1ce01fb0 con 0x560a1cdf5b40 2024-09-12T03:02:52.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.650+0000 7f2223f32740 10 client.0 _ll_register_callbacks cb 0x560a1ca688c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:02:52.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.650+0000 7f2212ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560a1cdf5b40 0x560a1cdfc070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:52.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.650+0000 7f221b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a1cdf45d0 0x560a1cdfedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:52.653 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.650+0000 7f221bfff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560a1cdff370 0x560a1cdfc800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:52.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.650+0000 7f2212ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560a1cdf5b40 0x560a1cdfc070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.5:33376/0 (socket says 172.21.15.5:33376) 2024-09-12T03:02:52.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.650+0000 7f221bfff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560a1cdff370 0x560a1cdfc800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.5:60664/0 (socket says 172.21.15.5:60664) 2024-09-12T03:02:52.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.650+0000 7f2212ffd640 1 -- 172.21.15.5:0/270186181 learned_addr learned my addr 172.21.15.5:0/270186181 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:02:52.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.650+0000 7f221b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a1cdf45d0 0x560a1cdfedc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57054/0 (socket says 172.21.15.5:57054) 2024-09-12T03:02:52.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.650+0000 7f2212ffd640 1 -- 172.21.15.5:0/270186181 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560a1cdff370 msgr2=0x560a1cdfc800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:52.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.650+0000 7f2212ffd640 1 --2- 172.21.15.5:0/270186181 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560a1cdff370 0x560a1cdfc800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:52.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.650+0000 7f2212ffd640 1 -- 172.21.15.5:0/270186181 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a1cdf45d0 msgr2=0x560a1cdfedc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:52.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.650+0000 7f2212ffd640 1 --2- 172.21.15.5:0/270186181 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a1cdf45d0 0x560a1cdfedc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:52.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.650+0000 7f2212ffd640 1 -- 172.21.15.5:0/270186181 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a1cacb3c0 con 0x560a1cdf5b40 2024-09-12T03:02:52.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.650+0000 7f221b7fe640 1 --2- 172.21.15.5:0/270186181 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a1cdf45d0 0x560a1cdfedc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:02:52.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.650+0000 7f2212ffd640 1 --2- 172.21.15.5:0/270186181 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560a1cdf5b40 0x560a1cdfc070 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f22040c4630 tx=0x7f22040bb850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb38fdb1a84eed39 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:52.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.651+0000 7f21faffd640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-09-12T03:02:52.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.651+0000 7f21faffd640 1 -- 172.21.15.5:0/270186181 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2204049050 con 0x560a1cdf5b40 2024-09-12T03:02:52.654 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.651+0000 7f21faffd640 1 -- 172.21.15.5:0/270186181 <== mon.1 v2:172.21.15.148:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f22040b8560 con 0x560a1cdf5b40 2024-09-12T03:02:52.655 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.652+0000 7f2223f32740 1 -- 172.21.15.5:0/270186181 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560a1cd67860 con 0x560a1cdf5b40 2024-09-12T03:02:52.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.652+0000 7f21faffd640 1 -- 172.21.15.5:0/270186181 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f22040bc5f0 con 0x560a1cdf5b40 2024-09-12T03:02:52.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.652+0000 7f2223f32740 10 client.25969 fetch_fsmap learned FSMap version 31 2024-09-12T03:02:52.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.652+0000 7f2223f32740 1 -- 172.21.15.5:0/270186181 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x560a1ce01e30 con 0x560a1cdf5b40 2024-09-12T03:02:52.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.654+0000 7f21faffd640 1 -- 172.21.15.5:0/270186181 <== mon.1 v2:172.21.15.148:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x560a1cd67860 con 0x560a1cdf5b40 2024-09-12T03:02:52.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.654+0000 7f21faffd640 1 -- 172.21.15.5:0/270186181 <== mon.1 v2:172.21.15.148:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f22040c46d0 con 0x560a1cdf5b40 2024-09-12T03:02:52.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.654+0000 7f2223f32740 10 client.25969 fetch_fsmap finished waiting for FSMap version 31 2024-09-12T03:02:52.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.654+0000 7f2223f32740 10 client.25969 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-09-12T03:02:52.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.654+0000 7f2223f32740 20 client.25969 populate_metadata read hostname 'smithi005' 2024-09-12T03:02:52.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.654+0000 7f2223f32740 1 --2- 172.21.15.5:0/270186181 >> [v2:172.21.15.148:6834/4195086427,v1:172.21.15.148:6835/4195086427] conn(0x560a1ce0ea00 0x560a1ce10df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:52.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.654+0000 7f2223f32740 4 client.25969 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:02:52.656 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.654+0000 7f2223f32740 1 -- 172.21.15.5:0/270186181 --> [v2:172.21.15.148:6834/4195086427,v1:172.21.15.148:6835/4195086427] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560a1cd61d70 con 0x560a1ce0ea00 2024-09-12T03:02:52.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.654+0000 7f221b7fe640 1 --2- 172.21.15.5:0/270186181 >> [v2:172.21.15.148:6834/4195086427,v1:172.21.15.148:6835/4195086427] conn(0x560a1ce0ea00 0x560a1ce10df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:52.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.655+0000 7f221b7fe640 1 --2- 172.21.15.5:0/270186181 >> [v2:172.21.15.148:6834/4195086427,v1:172.21.15.148:6835/4195086427] conn(0x560a1ce0ea00 0x560a1ce10df0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1666876f70f18d5c server_cookie=2bab9cebf0e8d050 in_seq=0 out_seq=0 2024-09-12T03:02:52.657 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.655+0000 7f21faffd640 10 client.25969 ms_handle_connect on v2:172.21.15.148:6834/4195086427 2024-09-12T03:02:52.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.656+0000 7f21faffd640 1 -- 172.21.15.5:0/270186181 <== mds.2 v2:172.21.15.148:6834/4195086427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560a1cd61d70 con 0x560a1ce0ea00 2024-09-12T03:02:52.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.656+0000 7f21faffd640 10 client.25969 handle_command_reply: tid=0 2024-09-12T03:02:52.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.656+0000 7f2223f32740 1 client.25969 shutdown 2024-09-12T03:02:52.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.656+0000 7f2223f32740 1 -- 172.21.15.5:0/270186181 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560a1cdf5b40 msgr2=0x560a1cdfc070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:52.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.656+0000 7f2223f32740 1 --2- 172.21.15.5:0/270186181 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560a1cdf5b40 0x560a1cdfc070 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f22040c4630 tx=0x7f22040bb850 comp rx=0 tx=0).stop 2024-09-12T03:02:52.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.656+0000 7f2223f32740 1 -- 172.21.15.5:0/270186181 shutdown_connections 2024-09-12T03:02:52.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.656+0000 7f2223f32740 1 --2- 172.21.15.5:0/270186181 >> [v2:172.21.15.148:6834/4195086427,v1:172.21.15.148:6835/4195086427] conn(0x560a1ce0ea00 0x560a1ce10df0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:52.658 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.656+0000 7f2223f32740 1 --2- 172.21.15.5:0/270186181 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560a1cdff370 0x560a1cdfc800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:52.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.656+0000 7f2223f32740 1 --2- 172.21.15.5:0/270186181 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560a1cdf5b40 0x560a1cdfc070 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:52.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.656+0000 7f2223f32740 1 --2- 172.21.15.5:0/270186181 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a1cdf45d0 0x560a1cdfedc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:52.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.656+0000 7f21faffd640 0 client.25969 ms_handle_reset on v2:172.21.15.148:6834/4195086427 2024-09-12T03:02:52.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.656+0000 7f2223f32740 1 -- 172.21.15.5:0/270186181 >> 172.21.15.5:0/270186181 conn(0x560a1cdbd310 msgr2=0x560a1cdf6990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:52.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.656+0000 7f2223f32740 1 -- 172.21.15.5:0/270186181 shutdown_connections 2024-09-12T03:02:52.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.656+0000 7f2223f32740 1 -- 172.21.15.5:0/270186181 wait complete. 2024-09-12T03:02:52.659 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.657+0000 7f2223f32740 20 client.25969 trim_cache size 0 max 16384 2024-09-12T03:02:52.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.672+0000 7f2223f32740 1 Processor -- start 2024-09-12T03:02:52.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.672+0000 7f2223f32740 1 -- start start 2024-09-12T03:02:52.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.672+0000 7f2223f32740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560a1cdf45d0 0x560a1cdfb930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:52.672 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.672+0000 7f2223f32740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560a1cdf5b40 0x560a1ce16650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:52.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.672+0000 7f2223f32740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a1cdff370 0x560a1ce16b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:52.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.672+0000 7f2223f32740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f221c1ab370 con 0x560a1cdff370 2024-09-12T03:02:52.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.672+0000 7f2223f32740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f221c1aceb0 con 0x560a1cdf45d0 2024-09-12T03:02:52.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.672+0000 7f2223f32740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f221c1aba40 con 0x560a1cdf5b40 2024-09-12T03:02:52.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.672+0000 7f2212ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560a1cdf5b40 0x560a1ce16650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:52.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.672+0000 7f2212ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560a1cdf5b40 0x560a1ce16650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.5:33378/0 (socket says 172.21.15.5:33378) 2024-09-12T03:02:52.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.673+0000 7f2212ffd640 1 -- 172.21.15.5:0/3798202915 learned_addr learned my addr 172.21.15.5:0/3798202915 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:02:52.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.673+0000 7f221bfff640 1 --2- 172.21.15.5:0/3798202915 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a1cdff370 0x560a1ce16b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:52.674 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.673+0000 7f2212ffd640 1 -- 172.21.15.5:0/3798202915 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560a1cdf45d0 msgr2=0x560a1cdfb930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:52.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.673+0000 7f2212ffd640 1 --2- 172.21.15.5:0/3798202915 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560a1cdf45d0 0x560a1cdfb930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:52.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.673+0000 7f2212ffd640 1 -- 172.21.15.5:0/3798202915 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a1cdff370 msgr2=0x560a1ce16b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:52.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.673+0000 7f2212ffd640 1 --2- 172.21.15.5:0/3798202915 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a1cdff370 0x560a1ce16b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:52.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.673+0000 7f2212ffd640 1 -- 172.21.15.5:0/3798202915 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a1cdfcb30 con 0x560a1cdf5b40 2024-09-12T03:02:52.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.673+0000 7f2212ffd640 1 --2- 172.21.15.5:0/3798202915 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560a1cdf5b40 0x560a1ce16650 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f22040bbe80 tx=0x7f22040b82c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b65f1d2b636e7955 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:52.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.673+0000 7f22117fa640 1 -- 172.21.15.5:0/3798202915 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2204049070 con 0x560a1cdf5b40 2024-09-12T03:02:52.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.673+0000 7f22117fa640 1 -- 172.21.15.5:0/3798202915 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f22040a80d0 con 0x560a1cdf5b40 2024-09-12T03:02:52.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.674+0000 7f22117fa640 1 -- 172.21.15.5:0/3798202915 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22040c46d0 con 0x560a1cdf5b40 2024-09-12T03:02:52.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.674+0000 7f2223f32740 1 -- 172.21.15.5:0/3798202915 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560a1cdf5b40 msgr2=0x560a1ce16650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:52.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.674+0000 7f2223f32740 1 --2- 172.21.15.5:0/3798202915 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560a1cdf5b40 0x560a1ce16650 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f22040bbe80 tx=0x7f22040b82c0 comp rx=0 tx=0).stop 2024-09-12T03:02:52.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.674+0000 7f2223f32740 1 -- 172.21.15.5:0/3798202915 shutdown_connections 2024-09-12T03:02:52.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.674+0000 7f2223f32740 1 --2- 172.21.15.5:0/3798202915 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a1cdff370 0x560a1ce16b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:52.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.674+0000 7f2223f32740 1 --2- 172.21.15.5:0/3798202915 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560a1cdf5b40 0x560a1ce16650 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:52.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.674+0000 7f2223f32740 1 --2- 172.21.15.5:0/3798202915 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560a1cdf45d0 0x560a1cdfb930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:52.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.674+0000 7f2223f32740 1 -- 172.21.15.5:0/3798202915 >> 172.21.15.5:0/3798202915 conn(0x560a1cdbd310 msgr2=0x560a1cdfb160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:52.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.675+0000 7f2223f32740 1 -- 172.21.15.5:0/3798202915 shutdown_connections 2024-09-12T03:02:52.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.675+0000 7f2223f32740 1 -- 172.21.15.5:0/3798202915 wait complete. 2024-09-12T03:02:52.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.675+0000 7f2223f32740 1 -- start start 2024-09-12T03:02:52.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.675+0000 7f2223f32740 1 Processor -- start 2024-09-12T03:02:52.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.675+0000 7f2223f32740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560a1cdf45d0 0x560a1ce02360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:52.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.675+0000 7f2223f32740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a1cdf5b40 0x560a1ce028a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:52.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.675+0000 7f2223f32740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560a1cdff370 0x560a1ce77610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:52.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.676+0000 7f2223f32740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f221c1ac7e0 con 0x560a1cdf5b40 2024-09-12T03:02:52.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.676+0000 7f2223f32740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f221c1ac110 con 0x560a1cdf45d0 2024-09-12T03:02:52.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.676+0000 7f2223f32740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f221c1a9350 con 0x560a1cdff370 2024-09-12T03:02:52.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.676+0000 7f2223f32740 10 client.0 _ll_register_callbacks cb 0x7f220c00b440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:02:52.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.676+0000 7f221b7fe640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560a1cdf45d0 0x560a1ce02360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:52.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.676+0000 7f221b7fe640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560a1cdf45d0 0x560a1ce02360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.5:60686/0 (socket says 172.21.15.5:60686) 2024-09-12T03:02:52.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.676+0000 7f2212ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a1cdf5b40 0x560a1ce028a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:52.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.676+0000 7f221b7fe640 1 -- 172.21.15.5:0/608651716 learned_addr learned my addr 172.21.15.5:0/608651716 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:02:52.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.676+0000 7f221bfff640 1 --2- 172.21.15.5:0/608651716 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560a1cdff370 0x560a1ce77610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:52.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.676+0000 7f221b7fe640 1 -- 172.21.15.5:0/608651716 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560a1cdff370 msgr2=0x560a1ce77610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:52.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.676+0000 7f221b7fe640 1 --2- 172.21.15.5:0/608651716 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560a1cdff370 0x560a1ce77610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:52.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.676+0000 7f221b7fe640 1 -- 172.21.15.5:0/608651716 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a1cdf5b40 msgr2=0x560a1ce028a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:52.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.676+0000 7f221b7fe640 1 --2- 172.21.15.5:0/608651716 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a1cdf5b40 0x560a1ce028a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:52.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.676+0000 7f221b7fe640 1 -- 172.21.15.5:0/608651716 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a1ce77b50 con 0x560a1cdf45d0 2024-09-12T03:02:52.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.676+0000 7f221b7fe640 1 --2- 172.21.15.5:0/608651716 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560a1cdf45d0 0x560a1ce02360 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x560a1ce115c0 tx=0x7f220c0138f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80f6d03413e51adc server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:52.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.676+0000 7f21faffd640 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2024-09-12T03:02:52.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.677+0000 7f21faffd640 1 -- 172.21.15.5:0/608651716 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f220c010690 con 0x560a1cdf45d0 2024-09-12T03:02:52.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.677+0000 7f21faffd640 1 -- 172.21.15.5:0/608651716 <== mon.2 v2:172.21.15.146:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f220c010830 con 0x560a1cdf45d0 2024-09-12T03:02:52.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.677+0000 7f2223f32740 1 -- 172.21.15.5:0/608651716 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22040a80d0 con 0x560a1cdf45d0 2024-09-12T03:02:52.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.677+0000 7f21faffd640 1 -- 172.21.15.5:0/608651716 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f220c010a30 con 0x560a1cdf45d0 2024-09-12T03:02:52.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.677+0000 7f2223f32740 10 client.25883 fetch_fsmap learned FSMap version 31 2024-09-12T03:02:52.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.677+0000 7f2223f32740 1 -- 172.21.15.5:0/608651716 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x560a1ce78d50 con 0x560a1cdf45d0 2024-09-12T03:02:52.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.677+0000 7f21faffd640 1 -- 172.21.15.5:0/608651716 <== mon.2 v2:172.21.15.146:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f220c010bd0 con 0x560a1cdf45d0 2024-09-12T03:02:52.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.677+0000 7f21faffd640 1 -- 172.21.15.5:0/608651716 <== mon.2 v2:172.21.15.146:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f220c011440 con 0x560a1cdf45d0 2024-09-12T03:02:52.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.677+0000 7f2223f32740 10 client.25883 fetch_fsmap finished waiting for FSMap version 31 2024-09-12T03:02:52.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.677+0000 7f2223f32740 10 client.25883 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-09-12T03:02:52.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.677+0000 7f2223f32740 20 client.25883 populate_metadata read hostname 'smithi005' 2024-09-12T03:02:52.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.677+0000 7f2223f32740 1 --2- 172.21.15.5:0/608651716 >> [v2:172.21.15.148:6834/4195086427,v1:172.21.15.148:6835/4195086427] conn(0x560a1ce794a0 0x560a1ce7b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:52.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.677+0000 7f2223f32740 4 client.25883 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:02:52.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.677+0000 7f2223f32740 1 -- 172.21.15.5:0/608651716 --> [v2:172.21.15.148:6834/4195086427,v1:172.21.15.148:6835/4195086427] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f22040a80d0 con 0x560a1ce794a0 2024-09-12T03:02:52.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.678+0000 7f2212ffd640 1 --2- 172.21.15.5:0/608651716 >> [v2:172.21.15.148:6834/4195086427,v1:172.21.15.148:6835/4195086427] conn(0x560a1ce794a0 0x560a1ce7b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:52.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.678+0000 7f2212ffd640 1 --2- 172.21.15.5:0/608651716 >> [v2:172.21.15.148:6834/4195086427,v1:172.21.15.148:6835/4195086427] conn(0x560a1ce794a0 0x560a1ce7b8c0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c234d2a4ab1268bf server_cookie=51a0e9dcf041f9 in_seq=0 out_seq=0 2024-09-12T03:02:52.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.678+0000 7f21faffd640 10 client.25883 ms_handle_connect on v2:172.21.15.148:6834/4195086427 2024-09-12T03:02:52.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.678+0000 7f21faffd640 1 -- 172.21.15.5:0/608651716 <== mds.2 v2:172.21.15.148:6834/4195086427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f22040a80d0 con 0x560a1ce794a0 2024-09-12T03:02:52.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.678+0000 7f21faffd640 10 client.25883 handle_command_reply: tid=0 2024-09-12T03:02:52.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.678+0000 7f2223f32740 1 client.25883 shutdown 2024-09-12T03:02:52.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.678+0000 7f2223f32740 1 -- 172.21.15.5:0/608651716 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560a1cdf45d0 msgr2=0x560a1ce02360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:52.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.678+0000 7f2223f32740 1 --2- 172.21.15.5:0/608651716 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560a1cdf45d0 0x560a1ce02360 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x560a1ce115c0 tx=0x7f220c0138f0 comp rx=0 tx=0).stop 2024-09-12T03:02:52.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.679+0000 7f2223f32740 1 -- 172.21.15.5:0/608651716 shutdown_connections 2024-09-12T03:02:52.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.679+0000 7f2223f32740 1 --2- 172.21.15.5:0/608651716 >> [v2:172.21.15.148:6834/4195086427,v1:172.21.15.148:6835/4195086427] conn(0x560a1ce794a0 0x560a1ce7b8c0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:52.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.679+0000 7f21faffd640 0 client.25883 ms_handle_reset on v2:172.21.15.148:6834/4195086427 2024-09-12T03:02:52.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.679+0000 7f2223f32740 1 --2- 172.21.15.5:0/608651716 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a1cdf5b40 0x560a1ce028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:52.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.679+0000 7f2223f32740 1 --2- 172.21.15.5:0/608651716 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x560a1cdff370 0x560a1ce77610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:52.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.679+0000 7f2223f32740 1 --2- 172.21.15.5:0/608651716 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x560a1cdf45d0 0x560a1ce02360 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:52.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.679+0000 7f2223f32740 1 -- 172.21.15.5:0/608651716 >> 172.21.15.5:0/608651716 conn(0x560a1cdbd310 msgr2=0x560a1cdef4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:52.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.679+0000 7f2223f32740 1 -- 172.21.15.5:0/608651716 shutdown_connections 2024-09-12T03:02:52.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.679+0000 7f2223f32740 1 -- 172.21.15.5:0/608651716 wait complete. 2024-09-12T03:02:52.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.679+0000 7f2223f32740 20 client.25883 trim_cache size 0 max 16384 2024-09-12T03:02:52.684 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-12T03:02:52.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.680+0000 7f2221b59640 1 -- 172.21.15.5:0/479414756 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f22080a2c00 msgr2=0x7f22080a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:52.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.680+0000 7f2221b59640 1 --2- 172.21.15.5:0/479414756 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f22080a2c00 0x7f22080a50c0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f221400a4e0 tx=0x7f2214009650 comp rx=0 tx=0).stop 2024-09-12T03:02:52.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.680+0000 7f2221b59640 1 -- 172.21.15.5:0/479414756 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f221c102780 msgr2=0x7f221c1989e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:52.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.680+0000 7f2221b59640 1 --2- 172.21.15.5:0/479414756 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f221c102780 0x7f221c1989e0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f2204006010 tx=0x7f22040042f0 comp rx=0 tx=0).stop 2024-09-12T03:02:52.685 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.680+0000 7f2221b59640 1 -- 172.21.15.5:0/479414756 shutdown_connections 2024-09-12T03:02:52.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.680+0000 7f2221b59640 1 --2- 172.21.15.5:0/479414756 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f22080a2c00 0x7f22080a50c0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:52.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.680+0000 7f2221b59640 1 --2- 172.21.15.5:0/479414756 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f221c102780 0x7f221c1989e0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:52.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.680+0000 7f2221b59640 1 --2- 172.21.15.5:0/479414756 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f221c106f00 0x7f221c19cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:52.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.680+0000 7f2221b59640 1 --2- 172.21.15.5:0/479414756 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f221c101dd0 0x7f221c1984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:52.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.680+0000 7f2221b59640 1 -- 172.21.15.5:0/479414756 >> 172.21.15.5:0/479414756 conn(0x7f221c0fd890 msgr2=0x7f221c0fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:52.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.680+0000 7f2221b59640 1 -- 172.21.15.5:0/479414756 shutdown_connections 2024-09-12T03:02:52.686 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:52.680+0000 7f2221b59640 1 -- 172.21.15.5:0/479414756 wait complete. 2024-09-12T03:02:52.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:02:52 smithi005 ceph-mon[31197]: pgmap v474: 97 pgs: 97 active+clean; 115 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.4 MiB/s wr, 258 op/s 2024-09-12T03:02:53.165 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:02:52 smithi148 ceph-mon[35090]: pgmap v474: 97 pgs: 97 active+clean; 115 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.4 MiB/s wr, 258 op/s 2024-09-12T03:02:53.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:02:52 smithi146 ceph-mon[35011]: pgmap v474: 97 pgs: 97 active+clean; 115 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 1.4 MiB/s wr, 258 op/s 2024-09-12T03:02:53.604 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T03:02:53.921 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:02:54.464 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.464+0000 7f1d0cef0640 1 -- 172.21.15.5:0/813336933 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d08100a70 msgr2=0x7f1d08100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:54.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.464+0000 7f1d0cef0640 1 --2- 172.21.15.5:0/813336933 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d08100a70 0x7f1d08100ed0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f1cf0009930 tx=0x7f1cf002a420 comp rx=0 tx=0).stop 2024-09-12T03:02:54.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.464+0000 7f1d0cef0640 1 -- 172.21.15.5:0/813336933 shutdown_connections 2024-09-12T03:02:54.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.464+0000 7f1d0cef0640 1 --2- 172.21.15.5:0/813336933 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d08100a70 0x7f1d08100ed0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:54.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.464+0000 7f1d0cef0640 1 --2- 172.21.15.5:0/813336933 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d08101410 0x7f1d0810d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:54.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.464+0000 7f1d0cef0640 1 --2- 172.21.15.5:0/813336933 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1d08106a70 0x7f1d08106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:54.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.464+0000 7f1d0cef0640 1 -- 172.21.15.5:0/813336933 >> 172.21.15.5:0/813336933 conn(0x7f1d080fc790 msgr2=0x7f1d080febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:54.465 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.464+0000 7f1d0cef0640 1 -- 172.21.15.5:0/813336933 shutdown_connections 2024-09-12T03:02:54.466 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.464+0000 7f1d0cef0640 1 -- 172.21.15.5:0/813336933 wait complete. 2024-09-12T03:02:54.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.465+0000 7f1d0cef0640 1 Processor -- start 2024-09-12T03:02:54.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.465+0000 7f1d0cef0640 1 -- start start 2024-09-12T03:02:54.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.465+0000 7f1d0cef0640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d08100a70 0x7f1d081983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:54.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.465+0000 7f1d0cef0640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1d08101410 0x7f1d08198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:54.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.465+0000 7f1d0cef0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d08106a70 0x7f1d0819ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:54.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.465+0000 7f1d0cef0640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1d08110e70 con 0x7f1d08106a70 2024-09-12T03:02:54.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.465+0000 7f1d0cef0640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f1d08110cf0 con 0x7f1d08101410 2024-09-12T03:02:54.467 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.465+0000 7f1d0cef0640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1d08110ff0 con 0x7f1d08100a70 2024-09-12T03:02:54.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.465+0000 7f1d06d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d08106a70 0x7f1d0819ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:54.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.465+0000 7f1d0659c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d08100a70 0x7f1d081983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:54.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.465+0000 7f1d05d9b640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1d08101410 0x7f1d08198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:54.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.465+0000 7f1d0659c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d08100a70 0x7f1d081983f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.5:47568/0 (socket says 172.21.15.5:47568) 2024-09-12T03:02:54.468 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.465+0000 7f1d0659c640 1 -- 172.21.15.5:0/1576490601 learned_addr learned my addr 172.21.15.5:0/1576490601 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:02:54.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.465+0000 7f1d06d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d08106a70 0x7f1d0819ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37076/0 (socket says 172.21.15.5:37076) 2024-09-12T03:02:54.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.466+0000 7f1d05d9b640 1 -- 172.21.15.5:0/1576490601 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d08100a70 msgr2=0x7f1d081983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:54.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.466+0000 7f1d05d9b640 1 --2- 172.21.15.5:0/1576490601 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d08100a70 0x7f1d081983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:54.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.466+0000 7f1d05d9b640 1 -- 172.21.15.5:0/1576490601 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d08106a70 msgr2=0x7f1d0819ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:54.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.467+0000 7f1d05d9b640 1 --2- 172.21.15.5:0/1576490601 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d08106a70 0x7f1d0819ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:54.470 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.467+0000 7f1d05d9b640 1 -- 172.21.15.5:0/1576490601 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d0819d3a0 con 0x7f1d08101410 2024-09-12T03:02:54.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.467+0000 7f1d06d9d640 1 --2- 172.21.15.5:0/1576490601 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d08106a70 0x7f1d0819ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:02:54.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.468+0000 7f1d05d9b640 1 --2- 172.21.15.5:0/1576490601 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1d08101410 0x7f1d08198930 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1cf0005940 tx=0x7f1cf0031b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:54.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.468+0000 7f1cff7fe640 1 -- 172.21.15.5:0/1576490601 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cf003f070 con 0x7f1d08101410 2024-09-12T03:02:54.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.468+0000 7f1cff7fe640 1 -- 172.21.15.5:0/1576490601 <== mon.2 v2:172.21.15.146:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cf0003ad0 con 0x7f1d08101410 2024-09-12T03:02:54.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.468+0000 7f1cff7fe640 1 -- 172.21.15.5:0/1576490601 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cf0035070 con 0x7f1d08101410 2024-09-12T03:02:54.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.468+0000 7f1d0cef0640 1 -- 172.21.15.5:0/1576490601 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d0819d630 con 0x7f1d08101410 2024-09-12T03:02:54.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.469+0000 7f1d0cef0640 1 -- 172.21.15.5:0/1576490601 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d0819dac0 con 0x7f1d08101410 2024-09-12T03:02:54.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.471+0000 7f1cff7fe640 1 -- 172.21.15.5:0/1576490601 <== mon.2 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f1cf003b460 con 0x7f1d08101410 2024-09-12T03:02:54.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.472+0000 7f1d0f2c9740 1 Processor -- start 2024-09-12T03:02:54.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.472+0000 7f1d0f2c9740 1 -- start start 2024-09-12T03:02:54.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.472+0000 7f1d0f2c9740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5559686e3b40 0x5559686e2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:54.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.472+0000 7f1d0f2c9740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5559686e25d0 0x5559686e2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:54.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.472+0000 7f1d0f2c9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559686ed370 0x5559686ef760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:54.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.472+0000 7f1d0f2c9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5559686482f0 con 0x5559686ed370 2024-09-12T03:02:54.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.472+0000 7f1d0f2c9740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5559685efcf0 con 0x5559686e25d0 2024-09-12T03:02:54.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.472+0000 7f1d0f2c9740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5559686442b0 con 0x5559686e3b40 2024-09-12T03:02:54.474 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.472+0000 7f1d06d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5559686e3b40 0x5559686e2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:54.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.472+0000 7f1cff7fe640 1 --2- 172.21.15.5:0/1576490601 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f1cf40a2c00 0x7f1cf40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:54.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.472+0000 7f1d06d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5559686e3b40 0x5559686e2090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.5:47584/0 (socket says 172.21.15.5:47584) 2024-09-12T03:02:54.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.472+0000 7f1d06d9d640 1 -- 172.21.15.5:0/2275606415 learned_addr learned my addr 172.21.15.5:0/2275606415 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:02:54.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.472+0000 7f1d05d9b640 1 --2- 172.21.15.5:0/2275606415 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559686ed370 0x5559686ef760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:54.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.473+0000 7f1d0659c640 1 --2- 172.21.15.5:0/2275606415 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5559686e25d0 0x5559686e2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:54.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.473+0000 7f1d06d9d640 1 --2- 172.21.15.5:0/1576490601 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f1cf40a2c00 0x7f1cf40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:54.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.473+0000 7f1d06d9d640 1 -- 172.21.15.5:0/2275606415 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5559686e25d0 msgr2=0x5559686e2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:54.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.473+0000 7f1d06d9d640 1 --2- 172.21.15.5:0/2275606415 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5559686e25d0 0x5559686e2a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:54.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.473+0000 7f1d06d9d640 1 -- 172.21.15.5:0/2275606415 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559686ed370 msgr2=0x5559686ef760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:54.475 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.473+0000 7f1cff7fe640 1 -- 172.21.15.5:0/1576490601 <== mon.2 v2:172.21.15.146:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f1cf00ef370 con 0x7f1d08101410 2024-09-12T03:02:54.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.473+0000 7f1d06d9d640 1 --2- 172.21.15.5:0/2275606415 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559686ed370 0x5559686ef760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:54.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.473+0000 7f1d06d9d640 1 -- 172.21.15.5:0/2275606415 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559685d43a0 con 0x5559686e3b40 2024-09-12T03:02:54.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.473+0000 7f1d0659c640 1 --2- 172.21.15.5:0/2275606415 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5559686e25d0 0x5559686e2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:02:54.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.473+0000 7f1d05d9b640 1 --2- 172.21.15.5:0/2275606415 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559686ed370 0x5559686ef760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:02:54.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.473+0000 7f1d06d9d640 1 --2- 172.21.15.5:0/2275606415 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5559686e3b40 0x5559686e2090 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1d00004650 tx=0x7f1d0000f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d405542c03a042b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:54.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.473+0000 7f1cfcff9640 1 -- 172.21.15.5:0/2275606415 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d00023020 con 0x5559686e3b40 2024-09-12T03:02:54.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.474+0000 7f1d06d9d640 1 --2- 172.21.15.5:0/1576490601 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f1cf40a2c00 0x7f1cf40a50c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f1d00024000 tx=0x7f1d00022000 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:54.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.474+0000 7f1cfcff9640 1 -- 172.21.15.5:0/2275606415 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d0001e8c0 con 0x5559686e3b40 2024-09-12T03:02:54.476 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.474+0000 7f1cfcff9640 1 -- 172.21.15.5:0/2275606415 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d00027610 con 0x5559686e3b40 2024-09-12T03:02:54.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.474+0000 7f1d0f2c9740 1 -- 172.21.15.5:0/2275606415 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5559686e3b40 msgr2=0x5559686e2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:54.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.474+0000 7f1d0f2c9740 1 --2- 172.21.15.5:0/2275606415 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5559686e3b40 0x5559686e2090 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1d00004650 tx=0x7f1d0000f790 comp rx=0 tx=0).stop 2024-09-12T03:02:54.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.475+0000 7f1d0f2c9740 1 -- 172.21.15.5:0/2275606415 shutdown_connections 2024-09-12T03:02:54.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.475+0000 7f1d0f2c9740 1 --2- 172.21.15.5:0/2275606415 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559686ed370 0x5559686ef760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:54.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.475+0000 7f1d0f2c9740 1 --2- 172.21.15.5:0/2275606415 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5559686e25d0 0x5559686e2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:54.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.475+0000 7f1d0f2c9740 1 --2- 172.21.15.5:0/2275606415 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5559686e3b40 0x5559686e2090 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:54.480 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.475+0000 7f1d0f2c9740 1 -- 172.21.15.5:0/2275606415 >> 172.21.15.5:0/2275606415 conn(0x5559686ab310 msgr2=0x5559686dec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:54.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.475+0000 7f1d0f2c9740 1 -- 172.21.15.5:0/2275606415 shutdown_connections 2024-09-12T03:02:54.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.475+0000 7f1d0f2c9740 1 -- 172.21.15.5:0/2275606415 wait complete. 2024-09-12T03:02:54.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.475+0000 7f1d0f2c9740 1 -- start start 2024-09-12T03:02:54.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.475+0000 7f1d0f2c9740 1 Processor -- start 2024-09-12T03:02:54.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.476+0000 7f1d0f2c9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559686e25d0 0x5559686ecd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:54.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.476+0000 7f1d0f2c9740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5559686e3b40 0x5559686ea000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:54.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.476+0000 7f1d0f2c9740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5559686ed370 0x5559686ea5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:54.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.476+0000 7f1d0f2c9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5559686f0070 con 0x5559686e25d0 2024-09-12T03:02:54.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.476+0000 7f1d0f2c9740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x555968603060 con 0x5559686ed370 2024-09-12T03:02:54.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.476+0000 7f1d0f2c9740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5559686f01f0 con 0x5559686e3b40 2024-09-12T03:02:54.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.476+0000 7f1d0f2c9740 10 client.0 _ll_register_callbacks cb 0x5559683568c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:02:54.481 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.476+0000 7f1d06d9d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5559686ed370 0x5559686ea5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:54.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.476+0000 7f1d05d9b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5559686e3b40 0x5559686ea000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:54.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.476+0000 7f1d0659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559686e25d0 0x5559686ecd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:54.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.476+0000 7f1d06d9d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5559686ed370 0x5559686ea5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.5:51222/0 (socket says 172.21.15.5:51222) 2024-09-12T03:02:54.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.476+0000 7f1d06d9d640 1 -- 172.21.15.5:0/3901301303 learned_addr learned my addr 172.21.15.5:0/3901301303 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:02:54.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.476+0000 7f1d05d9b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5559686e3b40 0x5559686ea000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.5:47594/0 (socket says 172.21.15.5:47594) 2024-09-12T03:02:54.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.476+0000 7f1d05d9b640 1 -- 172.21.15.5:0/3901301303 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5559686ed370 msgr2=0x5559686ea5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:54.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.476+0000 7f1d05d9b640 1 --2- 172.21.15.5:0/3901301303 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5559686ed370 0x5559686ea5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:54.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.476+0000 7f1d05d9b640 1 -- 172.21.15.5:0/3901301303 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559686e25d0 msgr2=0x5559686ecd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:54.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.476+0000 7f1d05d9b640 1 --2- 172.21.15.5:0/3901301303 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559686e25d0 0x5559686ecd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:54.482 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.476+0000 7f1d05d9b640 1 -- 172.21.15.5:0/3901301303 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559683b93c0 con 0x5559686e3b40 2024-09-12T03:02:54.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.477+0000 7f1d06d9d640 1 --2- 172.21.15.5:0/3901301303 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5559686ed370 0x5559686ea5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:02:54.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.477+0000 7f1d05d9b640 1 --2- 172.21.15.5:0/3901301303 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5559686e3b40 0x5559686ea000 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1cf002a440 tx=0x7f1cf0003f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=728f9f981c30a7b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:54.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.477+0000 7f1cedffb640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-09-12T03:02:54.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.477+0000 7f1cedffb640 1 -- 172.21.15.5:0/3901301303 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cf0048050 con 0x5559686e3b40 2024-09-12T03:02:54.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.477+0000 7f1cedffb640 1 -- 172.21.15.5:0/3901301303 <== mon.1 v2:172.21.15.148:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f1cf00b82a0 con 0x5559686e3b40 2024-09-12T03:02:54.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.477+0000 7f1d0f2c9740 1 -- 172.21.15.5:0/3901301303 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d0001e8c0 con 0x5559686e3b40 2024-09-12T03:02:54.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.477+0000 7f1cedffb640 1 -- 172.21.15.5:0/3901301303 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f1cf00bff30 con 0x5559686e3b40 2024-09-12T03:02:54.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.477+0000 7f1d0f2c9740 10 client.25987 fetch_fsmap learned FSMap version 31 2024-09-12T03:02:54.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.477+0000 7f1d0f2c9740 1 -- 172.21.15.5:0/3901301303 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5559686f0120 con 0x5559686e3b40 2024-09-12T03:02:54.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.477+0000 7f1cedffb640 1 -- 172.21.15.5:0/3901301303 <== mon.1 v2:172.21.15.148:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cf00c47f0 con 0x5559686e3b40 2024-09-12T03:02:54.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.478+0000 7f1cedffb640 1 -- 172.21.15.5:0/3901301303 <== mon.1 v2:172.21.15.148:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f1cf004b060 con 0x5559686e3b40 2024-09-12T03:02:54.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.478+0000 7f1d0f2c9740 10 client.25987 fetch_fsmap finished waiting for FSMap version 31 2024-09-12T03:02:54.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.478+0000 7f1d0f2c9740 10 client.25987 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-12T03:02:54.483 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.478+0000 7f1d0f2c9740 20 client.25987 populate_metadata read hostname 'smithi005' 2024-09-12T03:02:54.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.478+0000 7f1d0f2c9740 1 --2- 172.21.15.5:0/3901301303 >> [v2:172.21.15.148:6836/2606873922,v1:172.21.15.148:6837/2606873922] conn(0x5559686fd0a0 0x5559686ff490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:54.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.478+0000 7f1d0f2c9740 4 client.25987 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:02:54.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.478+0000 7f1d0f2c9740 1 -- 172.21.15.5:0/3901301303 --> [v2:172.21.15.148:6836/2606873922,v1:172.21.15.148:6837/2606873922] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f1d0001e8c0 con 0x5559686fd0a0 2024-09-12T03:02:54.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.478+0000 7f1d0659c640 1 --2- 172.21.15.5:0/3901301303 >> [v2:172.21.15.148:6836/2606873922,v1:172.21.15.148:6837/2606873922] conn(0x5559686fd0a0 0x5559686ff490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:54.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.478+0000 7f1d0659c640 1 --2- 172.21.15.5:0/3901301303 >> [v2:172.21.15.148:6836/2606873922,v1:172.21.15.148:6837/2606873922] conn(0x5559686fd0a0 0x5559686ff490 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b0e9de7ed078105f server_cookie=eae5a337e8ea9945 in_seq=0 out_seq=0 2024-09-12T03:02:54.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.478+0000 7f1cedffb640 10 client.25987 ms_handle_connect on v2:172.21.15.148:6836/2606873922 2024-09-12T03:02:54.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.479+0000 7f1cedffb640 1 -- 172.21.15.5:0/3901301303 <== mds.1 v2:172.21.15.148:6836/2606873922 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f1d0001e8c0 con 0x5559686fd0a0 2024-09-12T03:02:54.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.479+0000 7f1cedffb640 10 client.25987 handle_command_reply: tid=0 2024-09-12T03:02:54.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.479+0000 7f1d0f2c9740 1 client.25987 shutdown 2024-09-12T03:02:54.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.479+0000 7f1d0f2c9740 1 -- 172.21.15.5:0/3901301303 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5559686e3b40 msgr2=0x5559686ea000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:54.484 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.479+0000 7f1d0f2c9740 1 --2- 172.21.15.5:0/3901301303 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5559686e3b40 0x5559686ea000 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1cf002a440 tx=0x7f1cf0003f60 comp rx=0 tx=0).stop 2024-09-12T03:02:54.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.479+0000 7f1d0f2c9740 1 -- 172.21.15.5:0/3901301303 shutdown_connections 2024-09-12T03:02:54.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.479+0000 7f1d0f2c9740 1 --2- 172.21.15.5:0/3901301303 >> [v2:172.21.15.148:6836/2606873922,v1:172.21.15.148:6837/2606873922] conn(0x5559686fd0a0 0x5559686ff490 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:54.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.479+0000 7f1d0f2c9740 1 --2- 172.21.15.5:0/3901301303 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5559686ed370 0x5559686ea5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:54.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.479+0000 7f1d0f2c9740 1 --2- 172.21.15.5:0/3901301303 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5559686e3b40 0x5559686ea000 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:54.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.479+0000 7f1d0f2c9740 1 --2- 172.21.15.5:0/3901301303 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559686e25d0 0x5559686ecd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:54.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.479+0000 7f1cedffb640 0 client.25987 ms_handle_reset on v2:172.21.15.148:6836/2606873922 2024-09-12T03:02:54.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.479+0000 7f1d0f2c9740 1 -- 172.21.15.5:0/3901301303 >> 172.21.15.5:0/3901301303 conn(0x5559686ab310 msgr2=0x5559686e4960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:54.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.479+0000 7f1d0f2c9740 1 -- 172.21.15.5:0/3901301303 shutdown_connections 2024-09-12T03:02:54.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.480+0000 7f1d0f2c9740 1 -- 172.21.15.5:0/3901301303 wait complete. 2024-09-12T03:02:54.485 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.480+0000 7f1d0f2c9740 20 client.25987 trim_cache size 0 max 16384 2024-09-12T03:02:54.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.495+0000 7f1d0f2c9740 1 Processor -- start 2024-09-12T03:02:54.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.495+0000 7f1d0f2c9740 1 -- start start 2024-09-12T03:02:54.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.495+0000 7f1d0f2c9740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5559686e25d0 0x5559686f9390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:54.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.495+0000 7f1d0f2c9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559686e3b40 0x555968704d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:54.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.495+0000 7f1d0f2c9740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5559686ed370 0x555968705240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:54.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.495+0000 7f1d0f2c9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1d081af6d0 con 0x5559686e3b40 2024-09-12T03:02:54.496 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.495+0000 7f1d0f2c9740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f1d081b1210 con 0x5559686e25d0 2024-09-12T03:02:54.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.495+0000 7f1d0f2c9740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1d081afda0 con 0x5559686ed370 2024-09-12T03:02:54.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.495+0000 7f1d05d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559686e3b40 0x555968704d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:54.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.495+0000 7f1d0659c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5559686e25d0 0x5559686f9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:54.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.495+0000 7f1d0659c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5559686e25d0 0x5559686f9390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.5:51226/0 (socket says 172.21.15.5:51226) 2024-09-12T03:02:54.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.495+0000 7f1d0659c640 1 -- 172.21.15.5:0/4221854259 learned_addr learned my addr 172.21.15.5:0/4221854259 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:02:54.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.495+0000 7f1d06d9d640 1 --2- 172.21.15.5:0/4221854259 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5559686ed370 0x555968705240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:54.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.495+0000 7f1d0659c640 1 -- 172.21.15.5:0/4221854259 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5559686ed370 msgr2=0x555968705240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:54.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.495+0000 7f1d0659c640 1 --2- 172.21.15.5:0/4221854259 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5559686ed370 0x555968705240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:54.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.495+0000 7f1d0659c640 1 -- 172.21.15.5:0/4221854259 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559686e3b40 msgr2=0x555968704d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:54.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.495+0000 7f1d0659c640 1 --2- 172.21.15.5:0/4221854259 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559686e3b40 0x555968704d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:54.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.495+0000 7f1d0659c640 1 -- 172.21.15.5:0/4221854259 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559686ea940 con 0x5559686e25d0 2024-09-12T03:02:54.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.496+0000 7f1d0659c640 1 --2- 172.21.15.5:0/4221854259 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5559686e25d0 0x5559686f9390 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1cf8000f80 tx=0x7f1cf8009500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1000065da3440197 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:54.497 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.496+0000 7f1ceffff640 1 -- 172.21.15.5:0/4221854259 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cf8004d20 con 0x5559686e25d0 2024-09-12T03:02:54.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.496+0000 7f1ceffff640 1 -- 172.21.15.5:0/4221854259 <== mon.2 v2:172.21.15.146:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cf8003e50 con 0x5559686e25d0 2024-09-12T03:02:54.498 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.496+0000 7f1ceffff640 1 -- 172.21.15.5:0/4221854259 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cf8004280 con 0x5559686e25d0 2024-09-12T03:02:54.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.497+0000 7f1d0f2c9740 1 -- 172.21.15.5:0/4221854259 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5559686e25d0 msgr2=0x5559686f9390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:54.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.497+0000 7f1d0f2c9740 1 --2- 172.21.15.5:0/4221854259 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5559686e25d0 0x5559686f9390 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1cf8000f80 tx=0x7f1cf8009500 comp rx=0 tx=0).stop 2024-09-12T03:02:54.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.497+0000 7f1d0f2c9740 1 -- 172.21.15.5:0/4221854259 shutdown_connections 2024-09-12T03:02:54.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.497+0000 7f1d0f2c9740 1 --2- 172.21.15.5:0/4221854259 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559686e3b40 0x555968704d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:54.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.497+0000 7f1d0f2c9740 1 --2- 172.21.15.5:0/4221854259 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5559686ed370 0x555968705240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:54.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.497+0000 7f1d0f2c9740 1 --2- 172.21.15.5:0/4221854259 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5559686e25d0 0x5559686f9390 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:54.502 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.497+0000 7f1d0f2c9740 1 -- 172.21.15.5:0/4221854259 >> 172.21.15.5:0/4221854259 conn(0x5559686ab310 msgr2=0x5559686e5260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:54.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.497+0000 7f1d0f2c9740 1 -- 172.21.15.5:0/4221854259 shutdown_connections 2024-09-12T03:02:54.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.497+0000 7f1d0f2c9740 1 -- 172.21.15.5:0/4221854259 wait complete. 2024-09-12T03:02:54.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.497+0000 7f1d0f2c9740 1 -- start start 2024-09-12T03:02:54.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.497+0000 7f1d0f2c9740 1 Processor -- start 2024-09-12T03:02:54.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.498+0000 7f1d0f2c9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559686e25d0 0x5559686f1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:54.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.498+0000 7f1d0f2c9740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5559686e3b40 0x5559686f1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:54.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.498+0000 7f1d0f2c9740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5559686ed370 0x555968766cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:54.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.498+0000 7f1d0659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559686e25d0 0x5559686f1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:54.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.498+0000 7f1d0659c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559686e25d0 0x5559686f1640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37106/0 (socket says 172.21.15.5:37106) 2024-09-12T03:02:54.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.498+0000 7f1d05d9b640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5559686e3b40 0x5559686f1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:54.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.498+0000 7f1d05d9b640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5559686e3b40 0x5559686f1b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.5:51232/0 (socket says 172.21.15.5:51232) 2024-09-12T03:02:54.503 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.498+0000 7f1d0659c640 1 -- 172.21.15.5:0/2787623670 learned_addr learned my addr 172.21.15.5:0/2787623670 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:02:54.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.498+0000 7f1d0f2c9740 1 -- 172.21.15.5:0/2787623670 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1d081b0b40 con 0x5559686e25d0 2024-09-12T03:02:54.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.498+0000 7f1d0f2c9740 1 -- 172.21.15.5:0/2787623670 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f1d081b0470 con 0x5559686e3b40 2024-09-12T03:02:54.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.498+0000 7f1d0f2c9740 1 -- 172.21.15.5:0/2787623670 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1d081ad6b0 con 0x5559686ed370 2024-09-12T03:02:54.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.498+0000 7f1d0f2c9740 10 client.0 _ll_register_callbacks cb 0x7f1cf00bba90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:02:54.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.498+0000 7f1d06d9d640 1 --2- 172.21.15.5:0/2787623670 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5559686ed370 0x555968766cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:54.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.498+0000 7f1d0659c640 1 -- 172.21.15.5:0/2787623670 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5559686ed370 msgr2=0x555968766cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:54.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.498+0000 7f1d0659c640 1 --2- 172.21.15.5:0/2787623670 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5559686ed370 0x555968766cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:54.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.498+0000 7f1d0659c640 1 -- 172.21.15.5:0/2787623670 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5559686e3b40 msgr2=0x5559686f1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:54.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.498+0000 7f1d0659c640 1 --2- 172.21.15.5:0/2787623670 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5559686e3b40 0x5559686f1b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:54.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.498+0000 7f1d0659c640 1 -- 172.21.15.5:0/2787623670 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555968767210 con 0x5559686e25d0 2024-09-12T03:02:54.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.499+0000 7f1d0659c640 1 --2- 172.21.15.5:0/2787623670 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559686e25d0 0x5559686f1640 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f1cf8009fa0 tx=0x7f1cf8011790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b725e22853b0e243 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:54.504 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.499+0000 7f1cedffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T03:02:54.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.499+0000 7f1cedffb640 1 -- 172.21.15.5:0/2787623670 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cf800fa70 con 0x5559686e25d0 2024-09-12T03:02:54.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.499+0000 7f1cedffb640 1 -- 172.21.15.5:0/2787623670 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f1cf800fc10 con 0x5559686e25d0 2024-09-12T03:02:54.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.499+0000 7f1d0f2c9740 1 -- 172.21.15.5:0/2787623670 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1cf8003e50 con 0x5559686e25d0 2024-09-12T03:02:54.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.499+0000 7f1cedffb640 1 -- 172.21.15.5:0/2787623670 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f1cf8004000 con 0x5559686e25d0 2024-09-12T03:02:54.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.499+0000 7f1d0f2c9740 10 client.16512 fetch_fsmap learned FSMap version 31 2024-09-12T03:02:54.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.499+0000 7f1d0f2c9740 1 -- 172.21.15.5:0/2787623670 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x555968768380 con 0x5559686e25d0 2024-09-12T03:02:54.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.500+0000 7f1cedffb640 1 -- 172.21.15.5:0/2787623670 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1cf8003e50 con 0x5559686e25d0 2024-09-12T03:02:54.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.500+0000 7f1cedffb640 1 -- 172.21.15.5:0/2787623670 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f1cf802f060 con 0x5559686e25d0 2024-09-12T03:02:54.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.500+0000 7f1d0f2c9740 10 client.16512 fetch_fsmap finished waiting for FSMap version 31 2024-09-12T03:02:54.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.500+0000 7f1d0f2c9740 10 client.16512 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-12T03:02:54.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.500+0000 7f1d0f2c9740 20 client.16512 populate_metadata read hostname 'smithi005' 2024-09-12T03:02:54.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.500+0000 7f1d0f2c9740 1 --2- 172.21.15.5:0/2787623670 >> [v2:172.21.15.148:6836/2606873922,v1:172.21.15.148:6837/2606873922] conn(0x555968768990 0x55596876adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:54.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.500+0000 7f1d0f2c9740 4 client.16512 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:02:54.505 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.500+0000 7f1d0f2c9740 1 -- 172.21.15.5:0/2787623670 --> [v2:172.21.15.148:6836/2606873922,v1:172.21.15.148:6837/2606873922] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555968655860 con 0x555968768990 2024-09-12T03:02:54.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.500+0000 7f1d05d9b640 1 --2- 172.21.15.5:0/2787623670 >> [v2:172.21.15.148:6836/2606873922,v1:172.21.15.148:6837/2606873922] conn(0x555968768990 0x55596876adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:54.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.500+0000 7f1d05d9b640 1 --2- 172.21.15.5:0/2787623670 >> [v2:172.21.15.148:6836/2606873922,v1:172.21.15.148:6837/2606873922] conn(0x555968768990 0x55596876adb0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ce5dc91b250856f9 server_cookie=58c87d1041f4f9f0 in_seq=0 out_seq=0 2024-09-12T03:02:54.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.501+0000 7f1cedffb640 10 client.16512 ms_handle_connect on v2:172.21.15.148:6836/2606873922 2024-09-12T03:02:54.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.501+0000 7f1cedffb640 1 -- 172.21.15.5:0/2787623670 <== mds.1 v2:172.21.15.148:6836/2606873922 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555968655860 con 0x555968768990 2024-09-12T03:02:54.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.501+0000 7f1cedffb640 10 client.16512 handle_command_reply: tid=0 2024-09-12T03:02:54.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.501+0000 7f1d0f2c9740 1 client.16512 shutdown 2024-09-12T03:02:54.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.501+0000 7f1d0f2c9740 1 -- 172.21.15.5:0/2787623670 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559686e25d0 msgr2=0x5559686f1640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:54.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.501+0000 7f1d0f2c9740 1 --2- 172.21.15.5:0/2787623670 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559686e25d0 0x5559686f1640 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f1cf8009fa0 tx=0x7f1cf8011790 comp rx=0 tx=0).stop 2024-09-12T03:02:54.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.501+0000 7f1d0f2c9740 1 -- 172.21.15.5:0/2787623670 shutdown_connections 2024-09-12T03:02:54.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.501+0000 7f1d0f2c9740 1 --2- 172.21.15.5:0/2787623670 >> [v2:172.21.15.148:6836/2606873922,v1:172.21.15.148:6837/2606873922] conn(0x555968768990 0x55596876adb0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:54.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.501+0000 7f1d0f2c9740 1 --2- 172.21.15.5:0/2787623670 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5559686ed370 0x555968766cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:54.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.501+0000 7f1d0f2c9740 1 --2- 172.21.15.5:0/2787623670 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5559686e3b40 0x5559686f1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:54.506 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.501+0000 7f1d0f2c9740 1 --2- 172.21.15.5:0/2787623670 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5559686e25d0 0x5559686f1640 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:54.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.501+0000 7f1d0f2c9740 1 -- 172.21.15.5:0/2787623670 >> 172.21.15.5:0/2787623670 conn(0x5559686ab310 msgr2=0x5559686e4110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:54.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.501+0000 7f1cedffb640 0 client.16512 ms_handle_reset on v2:172.21.15.148:6836/2606873922 2024-09-12T03:02:54.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.501+0000 7f1d0f2c9740 1 -- 172.21.15.5:0/2787623670 shutdown_connections 2024-09-12T03:02:54.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.501+0000 7f1d0f2c9740 1 -- 172.21.15.5:0/2787623670 wait complete. 2024-09-12T03:02:54.507 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.501+0000 7f1d0f2c9740 20 client.16512 trim_cache size 0 max 16384 2024-09-12T03:02:54.507 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-12T03:02:54.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.502+0000 7f1d0cef0640 1 -- 172.21.15.5:0/1576490601 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f1cf40a2c00 msgr2=0x7f1cf40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:54.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.502+0000 7f1d0cef0640 1 --2- 172.21.15.5:0/1576490601 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f1cf40a2c00 0x7f1cf40a50c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f1d00024000 tx=0x7f1d00022000 comp rx=0 tx=0).stop 2024-09-12T03:02:54.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.502+0000 7f1d0cef0640 1 -- 172.21.15.5:0/1576490601 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1d08101410 msgr2=0x7f1d08198930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:54.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.502+0000 7f1d0cef0640 1 --2- 172.21.15.5:0/1576490601 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1d08101410 0x7f1d08198930 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1cf0005940 tx=0x7f1cf0031b50 comp rx=0 tx=0).stop 2024-09-12T03:02:54.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.502+0000 7f1d0cef0640 1 -- 172.21.15.5:0/1576490601 shutdown_connections 2024-09-12T03:02:54.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.502+0000 7f1d0cef0640 1 --2- 172.21.15.5:0/1576490601 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f1cf40a2c00 0x7f1cf40a50c0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:54.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.502+0000 7f1d0cef0640 1 --2- 172.21.15.5:0/1576490601 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d08106a70 0x7f1d0819ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:54.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.502+0000 7f1d0cef0640 1 --2- 172.21.15.5:0/1576490601 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1d08101410 0x7f1d08198930 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:54.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.502+0000 7f1d0cef0640 1 --2- 172.21.15.5:0/1576490601 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1d08100a70 0x7f1d081983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:54.508 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.502+0000 7f1d0cef0640 1 -- 172.21.15.5:0/1576490601 >> 172.21.15.5:0/1576490601 conn(0x7f1d080fc790 msgr2=0x7f1d0806a970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:54.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.502+0000 7f1d0cef0640 1 -- 172.21.15.5:0/1576490601 shutdown_connections 2024-09-12T03:02:54.509 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:54.502+0000 7f1d0cef0640 1 -- 172.21.15.5:0/1576490601 wait complete. 2024-09-12T03:02:55.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:02:54 smithi005 ceph-mon[31197]: pgmap v475: 97 pgs: 97 active+clean; 118 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 1.9 MiB/s wr, 326 op/s 2024-09-12T03:02:55.165 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:02:54 smithi148 ceph-mon[35090]: pgmap v475: 97 pgs: 97 active+clean; 118 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 1.9 MiB/s wr, 326 op/s 2024-09-12T03:02:55.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:02:54 smithi146 ceph-mon[35011]: pgmap v475: 97 pgs: 97 active+clean; 118 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 1.9 MiB/s wr, 326 op/s 2024-09-12T03:02:55.287 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T03:02:55.601 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:02:56.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.170+0000 7f47dcc1a640 1 -- 172.21.15.5:0/417931341 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47d8106f20 msgr2=0x7f47d810eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:56.177 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.170+0000 7f47dcc1a640 1 --2- 172.21.15.5:0/417931341 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47d8106f20 0x7f47d810eaa0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f47d0009ca0 tx=0x7f47d002a8c0 comp rx=0 tx=0).stop 2024-09-12T03:02:56.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.170+0000 7f47dcc1a640 1 -- 172.21.15.5:0/417931341 shutdown_connections 2024-09-12T03:02:56.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.170+0000 7f47dcc1a640 1 --2- 172.21.15.5:0/417931341 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47d8106f20 0x7f47d810eaa0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:56.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.170+0000 7f47dcc1a640 1 --2- 172.21.15.5:0/417931341 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f47d81027a0 0x7f47d81067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:56.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.170+0000 7f47dcc1a640 1 --2- 172.21.15.5:0/417931341 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47d8101df0 0x7f47d81021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:56.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.170+0000 7f47dcc1a640 1 -- 172.21.15.5:0/417931341 >> 172.21.15.5:0/417931341 conn(0x7f47d80fd8f0 msgr2=0x7f47d80ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:56.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.170+0000 7f47dcc1a640 1 -- 172.21.15.5:0/417931341 shutdown_connections 2024-09-12T03:02:56.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.170+0000 7f47dcc1a640 1 -- 172.21.15.5:0/417931341 wait complete. 2024-09-12T03:02:56.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.171+0000 7f47dcc1a640 1 Processor -- start 2024-09-12T03:02:56.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.173+0000 7f47dcc1a640 1 -- start start 2024-09-12T03:02:56.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.174+0000 7f47dcc1a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47d8101df0 0x7f47d80744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:56.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.174+0000 7f47dcc1a640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f47d81027a0 0x7f47d8074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:56.178 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.174+0000 7f47dcc1a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47d8106f20 0x7f47d8074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:56.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.174+0000 7f47dcc1a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f47d8111f80 con 0x7f47d8106f20 2024-09-12T03:02:56.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.174+0000 7f47dcc1a640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f47d8111e00 con 0x7f47d81027a0 2024-09-12T03:02:56.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.174+0000 7f47dcc1a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f47d8112100 con 0x7f47d8101df0 2024-09-12T03:02:56.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.174+0000 7f47d659c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47d8101df0 0x7f47d80744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:56.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.174+0000 7f47d659c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47d8101df0 0x7f47d80744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.5:47618/0 (socket says 172.21.15.5:47618) 2024-09-12T03:02:56.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.174+0000 7f47d659c640 1 -- 172.21.15.5:0/3803750600 learned_addr learned my addr 172.21.15.5:0/3803750600 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:02:56.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.175+0000 7f47d6d9d640 1 --2- 172.21.15.5:0/3803750600 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47d8106f20 0x7f47d8074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:56.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.175+0000 7f47d659c640 1 -- 172.21.15.5:0/3803750600 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f47d81027a0 msgr2=0x7f47d8074a30 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-12T03:02:56.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.175+0000 7f47d659c640 1 --2- 172.21.15.5:0/3803750600 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f47d81027a0 0x7f47d8074a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:56.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.175+0000 7f47d659c640 1 -- 172.21.15.5:0/3803750600 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47d8106f20 msgr2=0x7f47d8074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:56.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.175+0000 7f47d659c640 1 --2- 172.21.15.5:0/3803750600 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47d8106f20 0x7f47d8074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:56.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.175+0000 7f47d659c640 1 -- 172.21.15.5:0/3803750600 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47d81a6880 con 0x7f47d8101df0 2024-09-12T03:02:56.179 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.175+0000 7f47d6d9d640 1 --2- 172.21.15.5:0/3803750600 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47d8106f20 0x7f47d8074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:02:56.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.177+0000 7f47d659c640 1 --2- 172.21.15.5:0/3803750600 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47d8101df0 0x7f47d80744f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f47c800e9f0 tx=0x7f47c800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:56.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.177+0000 7f47c77fe640 1 -- 172.21.15.5:0/3803750600 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47c800cd80 con 0x7f47d8101df0 2024-09-12T03:02:56.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.177+0000 7f47c77fe640 1 -- 172.21.15.5:0/3803750600 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47c8010910 con 0x7f47d8101df0 2024-09-12T03:02:56.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.177+0000 7f47c77fe640 1 -- 172.21.15.5:0/3803750600 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47c8018990 con 0x7f47d8101df0 2024-09-12T03:02:56.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.177+0000 7f47dcc1a640 1 -- 172.21.15.5:0/3803750600 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47d81a6b70 con 0x7f47d8101df0 2024-09-12T03:02:56.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.177+0000 7f47dcc1a640 1 -- 172.21.15.5:0/3803750600 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47d81a7060 con 0x7f47d8101df0 2024-09-12T03:02:56.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.180+0000 7f47c77fe640 1 -- 172.21.15.5:0/3803750600 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f47c8018b30 con 0x7f47d8101df0 2024-09-12T03:02:56.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.180+0000 7f47deff3740 1 Processor -- start 2024-09-12T03:02:56.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.180+0000 7f47deff3740 1 -- start start 2024-09-12T03:02:56.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.181+0000 7f47deff3740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563630db0b40 0x563630daf090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:56.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.181+0000 7f47deff3740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563630daf5d0 0x563630dafa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:56.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.181+0000 7f47deff3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563630dba370 0x563630dbc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:56.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.181+0000 7f47deff3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x563630d152f0 con 0x563630dba370 2024-09-12T03:02:56.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.181+0000 7f47deff3740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x563630cbccf0 con 0x563630daf5d0 2024-09-12T03:02:56.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.181+0000 7f47deff3740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x563630d112b0 con 0x563630db0b40 2024-09-12T03:02:56.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.181+0000 7f47d6d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563630db0b40 0x563630daf090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:56.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.181+0000 7f47d6d9d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563630db0b40 0x563630daf090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.5:47620/0 (socket says 172.21.15.5:47620) 2024-09-12T03:02:56.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.181+0000 7f47d6d9d640 1 -- 172.21.15.5:0/364827470 learned_addr learned my addr 172.21.15.5:0/364827470 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:02:56.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.181+0000 7f47d659c640 1 --2- 172.21.15.5:0/364827470 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563630daf5d0 0x563630dafa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:56.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.181+0000 7f47d5d9b640 1 --2- 172.21.15.5:0/364827470 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563630dba370 0x563630dbc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:56.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.181+0000 7f47c77fe640 1 --2- 172.21.15.5:0/3803750600 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f47c00a2c00 0x7f47c00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:56.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.181+0000 7f47d6d9d640 1 -- 172.21.15.5:0/364827470 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563630daf5d0 msgr2=0x563630dafa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:56.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.181+0000 7f47d6d9d640 1 --2- 172.21.15.5:0/364827470 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563630daf5d0 0x563630dafa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:56.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.181+0000 7f47d6d9d640 1 -- 172.21.15.5:0/364827470 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563630dba370 msgr2=0x563630dbc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:56.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.181+0000 7f47d6d9d640 1 --2- 172.21.15.5:0/364827470 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563630dba370 0x563630dbc760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:56.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.181+0000 7f47d6d9d640 1 -- 172.21.15.5:0/364827470 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563630ca13a0 con 0x563630db0b40 2024-09-12T03:02:56.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.181+0000 7f47d5d9b640 1 --2- 172.21.15.5:0/364827470 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563630dba370 0x563630dbc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:02:56.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.182+0000 7f47c77fe640 1 -- 172.21.15.5:0/3803750600 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f47c80cf400 con 0x7f47d8101df0 2024-09-12T03:02:56.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.182+0000 7f47d6d9d640 1 --2- 172.21.15.5:0/364827470 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563630db0b40 0x563630daf090 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f47d002f510 tx=0x7f47d0009800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b522fe21c42ba2d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:56.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.182+0000 7f47d6d9d640 1 --2- 172.21.15.5:0/3803750600 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f47c00a2c00 0x7f47c00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:56.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.182+0000 7f47c4ff9640 1 -- 172.21.15.5:0/364827470 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47d0049040 con 0x563630db0b40 2024-09-12T03:02:56.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.182+0000 7f47c4ff9640 1 -- 172.21.15.5:0/364827470 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47d0036070 con 0x563630db0b40 2024-09-12T03:02:56.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.182+0000 7f47c4ff9640 1 -- 172.21.15.5:0/364827470 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47d00044e0 con 0x563630db0b40 2024-09-12T03:02:56.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.183+0000 7f47deff3740 1 -- 172.21.15.5:0/364827470 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563630db0b40 msgr2=0x563630daf090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:56.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.183+0000 7f47deff3740 1 --2- 172.21.15.5:0/364827470 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563630db0b40 0x563630daf090 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f47d002f510 tx=0x7f47d0009800 comp rx=0 tx=0).stop 2024-09-12T03:02:56.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.183+0000 7f47d6d9d640 1 --2- 172.21.15.5:0/3803750600 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f47c00a2c00 0x7f47c00a50c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f47d00049d0 tx=0x7f47d0004760 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:56.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.183+0000 7f47deff3740 1 -- 172.21.15.5:0/364827470 shutdown_connections 2024-09-12T03:02:56.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.183+0000 7f47deff3740 1 --2- 172.21.15.5:0/364827470 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563630dba370 0x563630dbc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:56.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.183+0000 7f47deff3740 1 --2- 172.21.15.5:0/364827470 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563630daf5d0 0x563630dafa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:56.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.183+0000 7f47deff3740 1 --2- 172.21.15.5:0/364827470 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563630db0b40 0x563630daf090 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:56.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.183+0000 7f47deff3740 1 -- 172.21.15.5:0/364827470 >> 172.21.15.5:0/364827470 conn(0x563630d78310 msgr2=0x563630dabc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:56.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.183+0000 7f47deff3740 1 -- 172.21.15.5:0/364827470 shutdown_connections 2024-09-12T03:02:56.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.183+0000 7f47deff3740 1 -- 172.21.15.5:0/364827470 wait complete. 2024-09-12T03:02:56.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.183+0000 7f47deff3740 1 -- start start 2024-09-12T03:02:56.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.183+0000 7f47deff3740 1 Processor -- start 2024-09-12T03:02:56.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.184+0000 7f47deff3740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563630daf5d0 0x563630db9d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:56.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.184+0000 7f47deff3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563630db0b40 0x563630db6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:56.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.184+0000 7f47deff3740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563630dba370 0x563630db75d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:56.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.184+0000 7f47deff3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x563630dbd010 con 0x563630db0b40 2024-09-12T03:02:56.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.184+0000 7f47deff3740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x563630cd0060 con 0x563630dba370 2024-09-12T03:02:56.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.184+0000 7f47deff3740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x563630dbd190 con 0x563630daf5d0 2024-09-12T03:02:56.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.184+0000 7f47d5d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563630db0b40 0x563630db6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:56.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.184+0000 7f47deff3740 10 client.0 _ll_register_callbacks cb 0x563630a238c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:02:56.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.184+0000 7f47d5d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563630db0b40 0x563630db6fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37142/0 (socket says 172.21.15.5:37142) 2024-09-12T03:02:56.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.184+0000 7f47d5d9b640 1 -- 172.21.15.5:0/2139535112 learned_addr learned my addr 172.21.15.5:0/2139535112 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:02:56.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.184+0000 7f47d659c640 1 --2- 172.21.15.5:0/2139535112 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563630daf5d0 0x563630db9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:56.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.184+0000 7f47d6d9d640 1 --2- 172.21.15.5:0/2139535112 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563630dba370 0x563630db75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:56.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.185+0000 7f47d6d9d640 1 -- 172.21.15.5:0/2139535112 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563630daf5d0 msgr2=0x563630db9d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:56.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.185+0000 7f47d6d9d640 1 --2- 172.21.15.5:0/2139535112 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563630daf5d0 0x563630db9d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:56.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.185+0000 7f47d6d9d640 1 -- 172.21.15.5:0/2139535112 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563630db0b40 msgr2=0x563630db6fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:56.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.185+0000 7f47d6d9d640 1 --2- 172.21.15.5:0/2139535112 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563630db0b40 0x563630db6fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:56.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.185+0000 7f47d6d9d640 1 -- 172.21.15.5:0/2139535112 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563630a863c0 con 0x563630dba370 2024-09-12T03:02:56.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.185+0000 7f47d659c640 1 --2- 172.21.15.5:0/2139535112 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563630daf5d0 0x563630db9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:02:56.189 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.185+0000 7f47d6d9d640 1 --2- 172.21.15.5:0/2139535112 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563630dba370 0x563630db75d0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f47d0009ca0 tx=0x7f47d0042000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=945044827244f8e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:56.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.185+0000 7f47bdffb640 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2024-09-12T03:02:56.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.185+0000 7f47bdffb640 1 -- 172.21.15.5:0/2139535112 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47d005f060 con 0x563630dba370 2024-09-12T03:02:56.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.185+0000 7f47bdffb640 1 -- 172.21.15.5:0/2139535112 <== mon.2 v2:172.21.15.146:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f47d00041e0 con 0x563630dba370 2024-09-12T03:02:56.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.185+0000 7f47deff3740 1 -- 172.21.15.5:0/2139535112 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47d0036070 con 0x563630dba370 2024-09-12T03:02:56.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.186+0000 7f47bdffb640 1 -- 172.21.15.5:0/2139535112 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f47d0056040 con 0x563630dba370 2024-09-12T03:02:56.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.186+0000 7f47deff3740 10 client.25910 fetch_fsmap learned FSMap version 31 2024-09-12T03:02:56.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.186+0000 7f47deff3740 1 -- 172.21.15.5:0/2139535112 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x563630dbd010 con 0x563630dba370 2024-09-12T03:02:56.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.186+0000 7f47bdffb640 1 -- 172.21.15.5:0/2139535112 <== mon.2 v2:172.21.15.146:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47d0036070 con 0x563630dba370 2024-09-12T03:02:56.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.186+0000 7f47bdffb640 1 -- 172.21.15.5:0/2139535112 <== mon.2 v2:172.21.15.146:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f47d0031450 con 0x563630dba370 2024-09-12T03:02:56.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.186+0000 7f47deff3740 10 client.25910 fetch_fsmap finished waiting for FSMap version 31 2024-09-12T03:02:56.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.186+0000 7f47deff3740 10 client.25910 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:02:56.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.186+0000 7f47deff3740 20 client.25910 populate_metadata read hostname 'smithi005' 2024-09-12T03:02:56.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.186+0000 7f47deff3740 1 --2- 172.21.15.5:0/2139535112 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x563630dc9bf0 0x563630dcbfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:56.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.186+0000 7f47deff3740 4 client.25910 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:02:56.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.186+0000 7f47deff3740 1 -- 172.21.15.5:0/2139535112 --> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563630d22860 con 0x563630dc9bf0 2024-09-12T03:02:56.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.186+0000 7f47d659c640 1 --2- 172.21.15.5:0/2139535112 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x563630dc9bf0 0x563630dcbfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:56.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.187+0000 7f47d659c640 1 --2- 172.21.15.5:0/2139535112 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x563630dc9bf0 0x563630dcbfe0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e1e2de0863581b7 server_cookie=172690376cf26ff1 in_seq=0 out_seq=0 2024-09-12T03:02:56.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.187+0000 7f47bdffb640 10 client.25910 ms_handle_connect on v2:172.21.15.148:6838/2317600000 2024-09-12T03:02:56.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.187+0000 7f47bdffb640 1 -- 172.21.15.5:0/2139535112 <== mds.0 v2:172.21.15.148:6838/2317600000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563630d22860 con 0x563630dc9bf0 2024-09-12T03:02:56.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.187+0000 7f47bdffb640 10 client.25910 handle_command_reply: tid=0 2024-09-12T03:02:56.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.187+0000 7f47deff3740 1 client.25910 shutdown 2024-09-12T03:02:56.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.188+0000 7f47deff3740 1 -- 172.21.15.5:0/2139535112 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563630dba370 msgr2=0x563630db75d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:56.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.188+0000 7f47deff3740 1 --2- 172.21.15.5:0/2139535112 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563630dba370 0x563630db75d0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f47d0009ca0 tx=0x7f47d0042000 comp rx=0 tx=0).stop 2024-09-12T03:02:56.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.188+0000 7f47deff3740 1 -- 172.21.15.5:0/2139535112 shutdown_connections 2024-09-12T03:02:56.192 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.188+0000 7f47deff3740 1 --2- 172.21.15.5:0/2139535112 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x563630dc9bf0 0x563630dcbfe0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:56.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.188+0000 7f47deff3740 1 --2- 172.21.15.5:0/2139535112 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563630db0b40 0x563630db6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:56.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.188+0000 7f47deff3740 1 --2- 172.21.15.5:0/2139535112 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563630dba370 0x563630db75d0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:56.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.188+0000 7f47deff3740 1 --2- 172.21.15.5:0/2139535112 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563630daf5d0 0x563630db9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:56.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.188+0000 7f47bdffb640 0 client.25910 ms_handle_reset on v2:172.21.15.148:6838/2317600000 2024-09-12T03:02:56.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.188+0000 7f47deff3740 1 -- 172.21.15.5:0/2139535112 >> 172.21.15.5:0/2139535112 conn(0x563630d78310 msgr2=0x563630db1960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:56.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.188+0000 7f47deff3740 1 -- 172.21.15.5:0/2139535112 shutdown_connections 2024-09-12T03:02:56.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.188+0000 7f47deff3740 1 -- 172.21.15.5:0/2139535112 wait complete. 2024-09-12T03:02:56.193 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.188+0000 7f47deff3740 20 client.25910 trim_cache size 0 max 16384 2024-09-12T03:02:56.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.204+0000 7f47deff3740 1 Processor -- start 2024-09-12T03:02:56.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.204+0000 7f47deff3740 1 -- start start 2024-09-12T03:02:56.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.205+0000 7f47deff3740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563630daf5d0 0x563630dc62b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:56.205 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.205+0000 7f47deff3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563630db0b40 0x563630db5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:56.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.205+0000 7f47deff3740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563630dba370 0x563630db2120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:56.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.205+0000 7f47deff3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f47d81af760 con 0x563630db0b40 2024-09-12T03:02:56.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.205+0000 7f47deff3740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f47d81b12a0 con 0x563630dba370 2024-09-12T03:02:56.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.205+0000 7f47deff3740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f47d81afe30 con 0x563630daf5d0 2024-09-12T03:02:56.206 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.206+0000 7f47d659c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563630daf5d0 0x563630dc62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:56.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.206+0000 7f47d659c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563630daf5d0 0x563630dc62b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.5:47632/0 (socket says 172.21.15.5:47632) 2024-09-12T03:02:56.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.206+0000 7f47d659c640 1 -- 172.21.15.5:0/2874810499 learned_addr learned my addr 172.21.15.5:0/2874810499 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:02:56.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.206+0000 7f47d5d9b640 1 --2- 172.21.15.5:0/2874810499 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563630db0b40 0x563630db5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:56.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.206+0000 7f47d6d9d640 1 --2- 172.21.15.5:0/2874810499 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563630dba370 0x563630db2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:56.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.206+0000 7f47d5d9b640 1 -- 172.21.15.5:0/2874810499 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563630daf5d0 msgr2=0x563630dc62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:56.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.206+0000 7f47d5d9b640 1 --2- 172.21.15.5:0/2874810499 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563630daf5d0 0x563630dc62b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:56.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.206+0000 7f47d5d9b640 1 -- 172.21.15.5:0/2874810499 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563630dba370 msgr2=0x563630db2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:56.207 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.206+0000 7f47d5d9b640 1 --2- 172.21.15.5:0/2874810499 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563630dba370 0x563630db2120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:56.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.206+0000 7f47d5d9b640 1 -- 172.21.15.5:0/2874810499 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563630db77c0 con 0x563630db0b40 2024-09-12T03:02:56.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.207+0000 7f47d5d9b640 1 --2- 172.21.15.5:0/2874810499 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563630db0b40 0x563630db5fc0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x563630db8590 tx=0x7f47cc00da50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=688f0e12ed8f7dfd server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:56.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.208+0000 7f47bffff640 1 -- 172.21.15.5:0/2874810499 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47cc003970 con 0x563630db0b40 2024-09-12T03:02:56.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.208+0000 7f47bffff640 1 -- 172.21.15.5:0/2874810499 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47cc011440 con 0x563630db0b40 2024-09-12T03:02:56.208 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.208+0000 7f47bffff640 1 -- 172.21.15.5:0/2874810499 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47cc00f610 con 0x563630db0b40 2024-09-12T03:02:56.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.209+0000 7f47deff3740 1 -- 172.21.15.5:0/2874810499 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563630db0b40 msgr2=0x563630db5fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:56.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.209+0000 7f47deff3740 1 --2- 172.21.15.5:0/2874810499 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563630db0b40 0x563630db5fc0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x563630db8590 tx=0x7f47cc00da50 comp rx=0 tx=0).stop 2024-09-12T03:02:56.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.209+0000 7f47deff3740 1 -- 172.21.15.5:0/2874810499 shutdown_connections 2024-09-12T03:02:56.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.209+0000 7f47deff3740 1 --2- 172.21.15.5:0/2874810499 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563630db0b40 0x563630db5fc0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:56.209 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.209+0000 7f47deff3740 1 --2- 172.21.15.5:0/2874810499 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563630dba370 0x563630db2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:56.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.209+0000 7f47deff3740 1 --2- 172.21.15.5:0/2874810499 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563630daf5d0 0x563630dc62b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:56.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.209+0000 7f47deff3740 1 -- 172.21.15.5:0/2874810499 >> 172.21.15.5:0/2874810499 conn(0x563630d78310 msgr2=0x563630db58b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:56.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.209+0000 7f47deff3740 1 -- 172.21.15.5:0/2874810499 shutdown_connections 2024-09-12T03:02:56.210 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.209+0000 7f47deff3740 1 -- 172.21.15.5:0/2874810499 wait complete. 2024-09-12T03:02:56.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.209+0000 7f47deff3740 1 -- start start 2024-09-12T03:02:56.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.210+0000 7f47deff3740 1 Processor -- start 2024-09-12T03:02:56.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.210+0000 7f47deff3740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563630daf5d0 0x563630dc2b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:56.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.210+0000 7f47deff3740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563630db0b40 0x563630dbde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:56.212 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.210+0000 7f47deff3740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563630dba370 0x563630dbe3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:56.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.210+0000 7f47deff3740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f47d81b0bd0 con 0x563630db0b40 2024-09-12T03:02:56.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.210+0000 7f47deff3740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f47d81b0500 con 0x563630dba370 2024-09-12T03:02:56.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.210+0000 7f47deff3740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f47d81ad740 con 0x563630daf5d0 2024-09-12T03:02:56.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.210+0000 7f47deff3740 10 client.0 _ll_register_callbacks cb 0x7f47d0053040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:02:56.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.210+0000 7f47d5d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563630db0b40 0x563630dbde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:56.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.210+0000 7f47d659c640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563630daf5d0 0x563630dc2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:56.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.210+0000 7f47d5d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563630db0b40 0x563630dbde80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37154/0 (socket says 172.21.15.5:37154) 2024-09-12T03:02:56.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.210+0000 7f47d5d9b640 1 -- 172.21.15.5:0/3371968507 learned_addr learned my addr 172.21.15.5:0/3371968507 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:02:56.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.210+0000 7f47d6d9d640 1 --2- 172.21.15.5:0/3371968507 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563630dba370 0x563630dbe3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:56.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.210+0000 7f47d5d9b640 1 -- 172.21.15.5:0/3371968507 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563630daf5d0 msgr2=0x563630dc2b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:56.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.210+0000 7f47d5d9b640 1 --2- 172.21.15.5:0/3371968507 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563630daf5d0 0x563630dc2b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:56.213 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.210+0000 7f47d5d9b640 1 -- 172.21.15.5:0/3371968507 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563630dba370 msgr2=0x563630dbe3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:56.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.210+0000 7f47d5d9b640 1 --2- 172.21.15.5:0/3371968507 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563630dba370 0x563630dbe3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:56.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.211+0000 7f47d5d9b640 1 -- 172.21.15.5:0/3371968507 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563630dbebf0 con 0x563630db0b40 2024-09-12T03:02:56.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.211+0000 7f47d659c640 1 --2- 172.21.15.5:0/3371968507 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563630daf5d0 0x563630dc2b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:02:56.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.211+0000 7f47d6d9d640 1 --2- 172.21.15.5:0/3371968507 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563630dba370 0x563630dbe3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:02:56.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.211+0000 7f47d5d9b640 1 --2- 172.21.15.5:0/3371968507 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563630db0b40 0x563630dbde80 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x563630db8590 tx=0x7f47cc005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ea5706004560a06 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:56.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.211+0000 7f47bdffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T03:02:56.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.211+0000 7f47bdffb640 1 -- 172.21.15.5:0/3371968507 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47cc02b060 con 0x563630db0b40 2024-09-12T03:02:56.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.211+0000 7f47bdffb640 1 -- 172.21.15.5:0/3371968507 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f47cc0115f0 con 0x563630db0b40 2024-09-12T03:02:56.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.211+0000 7f47deff3740 1 -- 172.21.15.5:0/3371968507 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47cc011440 con 0x563630db0b40 2024-09-12T03:02:56.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.211+0000 7f47bdffb640 1 -- 172.21.15.5:0/3371968507 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f47cc0117f0 con 0x563630db0b40 2024-09-12T03:02:56.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.211+0000 7f47deff3740 10 client.16527 fetch_fsmap learned FSMap version 31 2024-09-12T03:02:56.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.211+0000 7f47deff3740 1 -- 172.21.15.5:0/3371968507 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x563630e3cfb0 con 0x563630db0b40 2024-09-12T03:02:56.214 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.211+0000 7f47bdffb640 1 -- 172.21.15.5:0/3371968507 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47cc011440 con 0x563630db0b40 2024-09-12T03:02:56.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.212+0000 7f47bdffb640 1 -- 172.21.15.5:0/3371968507 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f47cc019020 con 0x563630db0b40 2024-09-12T03:02:56.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.212+0000 7f47deff3740 10 client.16527 fetch_fsmap finished waiting for FSMap version 31 2024-09-12T03:02:56.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.212+0000 7f47deff3740 10 client.16527 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:02:56.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.212+0000 7f47deff3740 20 client.16527 populate_metadata read hostname 'smithi005' 2024-09-12T03:02:56.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.212+0000 7f47deff3740 1 --2- 172.21.15.5:0/3371968507 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x563630e3d660 0x563630e3fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:56.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.212+0000 7f47deff3740 4 client.16527 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:02:56.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.212+0000 7f47deff3740 1 -- 172.21.15.5:0/3371968507 --> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563630d1cd70 con 0x563630e3d660 2024-09-12T03:02:56.215 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.212+0000 7f47d659c640 1 --2- 172.21.15.5:0/3371968507 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x563630e3d660 0x563630e3fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:56.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.213+0000 7f47bdffb640 10 client.16527 ms_handle_connect on v2:172.21.15.148:6838/2317600000 2024-09-12T03:02:56.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.213+0000 7f47d659c640 1 --2- 172.21.15.5:0/3371968507 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x563630e3d660 0x563630e3fa80 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d1e3878e1a4aaaf server_cookie=b21c0401d228f277 in_seq=0 out_seq=0 2024-09-12T03:02:56.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.213+0000 7f47bdffb640 1 -- 172.21.15.5:0/3371968507 <== mds.0 v2:172.21.15.148:6838/2317600000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563630d1cd70 con 0x563630e3d660 2024-09-12T03:02:56.217 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.213+0000 7f47bdffb640 10 client.16527 handle_command_reply: tid=0 2024-09-12T03:02:56.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.213+0000 7f47deff3740 1 client.16527 shutdown 2024-09-12T03:02:56.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.213+0000 7f47deff3740 1 -- 172.21.15.5:0/3371968507 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563630db0b40 msgr2=0x563630dbde80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:56.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.213+0000 7f47deff3740 1 --2- 172.21.15.5:0/3371968507 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563630db0b40 0x563630dbde80 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x563630db8590 tx=0x7f47cc005270 comp rx=0 tx=0).stop 2024-09-12T03:02:56.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.213+0000 7f47deff3740 1 -- 172.21.15.5:0/3371968507 shutdown_connections 2024-09-12T03:02:56.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.213+0000 7f47deff3740 1 --2- 172.21.15.5:0/3371968507 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x563630e3d660 0x563630e3fa80 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:56.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.213+0000 7f47deff3740 1 --2- 172.21.15.5:0/3371968507 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563630db0b40 0x563630dbde80 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:56.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.213+0000 7f47deff3740 1 --2- 172.21.15.5:0/3371968507 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563630dba370 0x563630dbe3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:56.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.213+0000 7f47deff3740 1 --2- 172.21.15.5:0/3371968507 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563630daf5d0 0x563630dc2b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:56.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.213+0000 7f47deff3740 1 -- 172.21.15.5:0/3371968507 >> 172.21.15.5:0/3371968507 conn(0x563630d78310 msgr2=0x563630db60d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:56.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.213+0000 7f47bdffb640 0 client.16527 ms_handle_reset on v2:172.21.15.148:6838/2317600000 2024-09-12T03:02:56.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.213+0000 7f47deff3740 1 -- 172.21.15.5:0/3371968507 shutdown_connections 2024-09-12T03:02:56.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.213+0000 7f47deff3740 1 -- 172.21.15.5:0/3371968507 wait complete. 2024-09-12T03:02:56.218 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.213+0000 7f47deff3740 20 client.16527 trim_cache size 0 max 16384 2024-09-12T03:02:56.219 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-12T03:02:56.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.214+0000 7f47dcc1a640 1 -- 172.21.15.5:0/3803750600 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f47c00a2c00 msgr2=0x7f47c00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:56.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.214+0000 7f47dcc1a640 1 --2- 172.21.15.5:0/3803750600 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f47c00a2c00 0x7f47c00a50c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f47d00049d0 tx=0x7f47d0004760 comp rx=0 tx=0).stop 2024-09-12T03:02:56.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.214+0000 7f47dcc1a640 1 -- 172.21.15.5:0/3803750600 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47d8101df0 msgr2=0x7f47d80744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:56.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.214+0000 7f47dcc1a640 1 --2- 172.21.15.5:0/3803750600 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47d8101df0 0x7f47d80744f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f47c800e9f0 tx=0x7f47c800eeb0 comp rx=0 tx=0).stop 2024-09-12T03:02:56.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.214+0000 7f47dcc1a640 1 -- 172.21.15.5:0/3803750600 shutdown_connections 2024-09-12T03:02:56.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.214+0000 7f47dcc1a640 1 --2- 172.21.15.5:0/3803750600 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f47c00a2c00 0x7f47c00a50c0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:56.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.214+0000 7f47dcc1a640 1 --2- 172.21.15.5:0/3803750600 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f47d8106f20 0x7f47d8074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:56.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.214+0000 7f47dcc1a640 1 --2- 172.21.15.5:0/3803750600 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f47d81027a0 0x7f47d8074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:56.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.214+0000 7f47dcc1a640 1 --2- 172.21.15.5:0/3803750600 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f47d8101df0 0x7f47d80744f0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:56.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.214+0000 7f47dcc1a640 1 -- 172.21.15.5:0/3803750600 >> 172.21.15.5:0/3803750600 conn(0x7f47d80fd8f0 msgr2=0x7f47d80fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:56.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.214+0000 7f47dcc1a640 1 -- 172.21.15.5:0/3803750600 shutdown_connections 2024-09-12T03:02:56.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:56.214+0000 7f47dcc1a640 1 -- 172.21.15.5:0/3803750600 wait complete. 2024-09-12T03:02:56.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:02:56 smithi005 ceph-mon[31197]: pgmap v476: 97 pgs: 97 active+clean; 135 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 2.4 MiB/s wr, 492 op/s 2024-09-12T03:02:57.165 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:02:56 smithi148 ceph-mon[35090]: pgmap v476: 97 pgs: 97 active+clean; 135 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 2.4 MiB/s wr, 492 op/s 2024-09-12T03:02:57.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:02:56 smithi146 ceph-mon[35011]: pgmap v476: 97 pgs: 97 active+clean; 135 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 2.4 MiB/s wr, 492 op/s 2024-09-12T03:02:58.045 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T03:02:58.046 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T03:02:58.368 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:02:58.787 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:02:58 smithi005 ceph-mon[31197]: pgmap v477: 97 pgs: 97 active+clean; 133 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 2.7 MiB/s wr, 623 op/s 2024-09-12T03:02:58.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.896+0000 7ff677495640 1 -- 172.21.15.5:0/1828013167 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff670109450 msgr2=0x7ff670109830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:58.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.896+0000 7ff677495640 1 --2- 172.21.15.5:0/1828013167 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff670109450 0x7ff670109830 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7ff664009900 tx=0x7ff66402a410 comp rx=0 tx=0).stop 2024-09-12T03:02:58.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.896+0000 7ff677495640 1 -- 172.21.15.5:0/1828013167 shutdown_connections 2024-09-12T03:02:58.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.896+0000 7ff677495640 1 --2- 172.21.15.5:0/1828013167 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff67010ddc0 0x7ff670115940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:58.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.896+0000 7ff677495640 1 --2- 172.21.15.5:0/1828013167 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff670109e00 0x7ff67010d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:58.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.896+0000 7ff677495640 1 --2- 172.21.15.5:0/1828013167 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff670109450 0x7ff670109830 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:58.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.896+0000 7ff677495640 1 -- 172.21.15.5:0/1828013167 >> 172.21.15.5:0/1828013167 conn(0x7ff67006e140 msgr2=0x7ff67006e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:58.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.896+0000 7ff677495640 1 -- 172.21.15.5:0/1828013167 shutdown_connections 2024-09-12T03:02:58.898 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.896+0000 7ff677495640 1 -- 172.21.15.5:0/1828013167 wait complete. 2024-09-12T03:02:58.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.897+0000 7ff677495640 1 Processor -- start 2024-09-12T03:02:58.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.897+0000 7ff677495640 1 -- start start 2024-09-12T03:02:58.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.897+0000 7ff677495640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff670109450 0x7ff6701a68b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:58.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.897+0000 7ff677495640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff670109e00 0x7ff6701a6df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:58.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.897+0000 7ff677495640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff67010ddc0 0x7ff6701a0a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:58.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.897+0000 7ff677495640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff6701185a0 con 0x7ff670109e00 2024-09-12T03:02:58.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.897+0000 7ff677495640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ff670118420 con 0x7ff670109450 2024-09-12T03:02:58.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.897+0000 7ff677495640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff670118720 con 0x7ff67010ddc0 2024-09-12T03:02:58.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.897+0000 7ff674a30640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff670109e00 0x7ff6701a6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:58.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.897+0000 7ff674a30640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff670109e00 0x7ff6701a6df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37176/0 (socket says 172.21.15.5:37176) 2024-09-12T03:02:58.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.897+0000 7ff674a30640 1 -- 172.21.15.5:0/3112398669 learned_addr learned my addr 172.21.15.5:0/3112398669 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:02:58.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.897+0000 7ff675231640 1 --2- 172.21.15.5:0/3112398669 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff670109450 0x7ff6701a68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:58.899 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.898+0000 7ff675a32640 1 --2- 172.21.15.5:0/3112398669 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff67010ddc0 0x7ff6701a0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:58.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.898+0000 7ff675231640 1 -- 172.21.15.5:0/3112398669 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff67010ddc0 msgr2=0x7ff6701a0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:58.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.898+0000 7ff675231640 1 --2- 172.21.15.5:0/3112398669 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff67010ddc0 0x7ff6701a0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:58.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.898+0000 7ff675231640 1 -- 172.21.15.5:0/3112398669 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff670109e00 msgr2=0x7ff6701a6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:58.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.898+0000 7ff675231640 1 --2- 172.21.15.5:0/3112398669 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff670109e00 0x7ff6701a6df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:58.900 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.898+0000 7ff675231640 1 -- 172.21.15.5:0/3112398669 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6701a11a0 con 0x7ff670109450 2024-09-12T03:02:58.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.898+0000 7ff674a30640 1 --2- 172.21.15.5:0/3112398669 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff670109e00 0x7ff6701a6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:02:58.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.898+0000 7ff675231640 1 --2- 172.21.15.5:0/3112398669 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff670109450 0x7ff6701a68b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff664005940 tx=0x7ff6640040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:58.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.898+0000 7ff6627fc640 1 -- 172.21.15.5:0/3112398669 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff66403f030 con 0x7ff670109450 2024-09-12T03:02:58.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.898+0000 7ff6627fc640 1 -- 172.21.15.5:0/3112398669 <== mon.2 v2:172.21.15.146:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6640044c0 con 0x7ff670109450 2024-09-12T03:02:58.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.898+0000 7ff677495640 1 -- 172.21.15.5:0/3112398669 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6701a1430 con 0x7ff670109450 2024-09-12T03:02:58.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.898+0000 7ff6627fc640 1 -- 172.21.15.5:0/3112398669 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff664035070 con 0x7ff670109450 2024-09-12T03:02:58.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.898+0000 7ff677495640 1 -- 172.21.15.5:0/3112398669 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff67019e750 con 0x7ff670109450 2024-09-12T03:02:58.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.901+0000 7ff6627fc640 1 -- 172.21.15.5:0/3112398669 <== mon.2 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7ff664042030 con 0x7ff670109450 2024-09-12T03:02:58.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.901+0000 7ff67986e740 1 Processor -- start 2024-09-12T03:02:58.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.902+0000 7ff67986e740 1 -- start start 2024-09-12T03:02:58.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.902+0000 7ff6627fc640 1 --2- 172.21.15.5:0/3112398669 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7ff65c0a6f90 0x7ff65c0a9450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:58.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.902+0000 7ff6627fc640 1 -- 172.21.15.5:0/3112398669 <== mon.2 v2:172.21.15.146:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7ff6640ef320 con 0x7ff670109450 2024-09-12T03:02:58.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.902+0000 7ff675a32640 1 --2- 172.21.15.5:0/3112398669 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7ff65c0a6f90 0x7ff65c0a9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:58.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.902+0000 7ff67986e740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55916672cb60 0x55916672cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:58.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.902+0000 7ff67986e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55916672d480 0x559166737800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:58.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.902+0000 7ff67986e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559166737d40 0x55916673a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:58.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.902+0000 7ff67986e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559166692280 con 0x55916672d480 2024-09-12T03:02:58.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.902+0000 7ff67986e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x559166639b10 con 0x559166737d40 2024-09-12T03:02:58.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.902+0000 7ff67986e740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55916668e240 con 0x55916672cb60 2024-09-12T03:02:58.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.902+0000 7ff674a30640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55916672d480 0x559166737800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:58.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.902+0000 7ff675231640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55916672cb60 0x55916672cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:58.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.902+0000 7ff675231640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55916672cb60 0x55916672cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.5:47656/0 (socket says 172.21.15.5:47656) 2024-09-12T03:02:58.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.902+0000 7ff674a30640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55916672d480 0x559166737800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37190/0 (socket says 172.21.15.5:37190) 2024-09-12T03:02:58.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.902+0000 7ff675231640 1 -- 172.21.15.5:0/1607836402 learned_addr learned my addr 172.21.15.5:0/1607836402 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:02:58.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.902+0000 7ff675a32640 1 --2- 172.21.15.5:0/1607836402 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559166737d40 0x55916673a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:58.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.902+0000 7ff675231640 1 -- 172.21.15.5:0/1607836402 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559166737d40 msgr2=0x55916673a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:58.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.902+0000 7ff675231640 1 --2- 172.21.15.5:0/1607836402 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559166737d40 0x55916673a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:58.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.902+0000 7ff675231640 1 -- 172.21.15.5:0/1607836402 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55916672d480 msgr2=0x559166737800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:58.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.902+0000 7ff675231640 1 --2- 172.21.15.5:0/1607836402 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55916672d480 0x559166737800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:58.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.902+0000 7ff675231640 1 -- 172.21.15.5:0/1607836402 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591666461f0 con 0x55916672cb60 2024-09-12T03:02:58.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.903+0000 7ff674a30640 1 --2- 172.21.15.5:0/1607836402 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55916672d480 0x559166737800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:02:58.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.903+0000 7ff675a32640 1 --2- 172.21.15.5:0/3112398669 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7ff65c0a6f90 0x7ff65c0a9450 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7ff668005fd0 tx=0x7ff668011b40 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:58.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.903+0000 7ff675231640 1 --2- 172.21.15.5:0/1607836402 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55916672cb60 0x55916672cf40 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff6640b80a0 tx=0x7ff66402a410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=169e10a16c36dfe server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:58.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.903+0000 7ff65b7fe640 1 -- 172.21.15.5:0/1607836402 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff664049040 con 0x55916672cb60 2024-09-12T03:02:58.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.903+0000 7ff65b7fe640 1 -- 172.21.15.5:0/1607836402 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6640bff40 con 0x55916672cb60 2024-09-12T03:02:58.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.903+0000 7ff65b7fe640 1 -- 172.21.15.5:0/1607836402 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6640bbea0 con 0x55916672cb60 2024-09-12T03:02:58.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.904+0000 7ff67986e740 1 -- 172.21.15.5:0/1607836402 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55916672cb60 msgr2=0x55916672cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:58.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.904+0000 7ff67986e740 1 --2- 172.21.15.5:0/1607836402 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55916672cb60 0x55916672cf40 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff6640b80a0 tx=0x7ff66402a410 comp rx=0 tx=0).stop 2024-09-12T03:02:58.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.904+0000 7ff67986e740 1 -- 172.21.15.5:0/1607836402 shutdown_connections 2024-09-12T03:02:58.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.904+0000 7ff67986e740 1 --2- 172.21.15.5:0/1607836402 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55916672d480 0x559166737800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:58.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.904+0000 7ff67986e740 1 --2- 172.21.15.5:0/1607836402 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559166737d40 0x55916673a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:58.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.904+0000 7ff67986e740 1 --2- 172.21.15.5:0/1607836402 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55916672cb60 0x55916672cf40 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:58.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.904+0000 7ff67986e740 1 -- 172.21.15.5:0/1607836402 >> 172.21.15.5:0/1607836402 conn(0x5591667290f0 msgr2=0x5591667294d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:58.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.904+0000 7ff67986e740 1 -- 172.21.15.5:0/1607836402 shutdown_connections 2024-09-12T03:02:58.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.904+0000 7ff67986e740 1 -- 172.21.15.5:0/1607836402 wait complete. 2024-09-12T03:02:58.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.904+0000 7ff67986e740 1 -- start start 2024-09-12T03:02:58.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.904+0000 7ff67986e740 1 Processor -- start 2024-09-12T03:02:58.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.905+0000 7ff67986e740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55916672cb60 0x559166734c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:58.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.905+0000 7ff67986e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55916672d480 0x559166735160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:58.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.905+0000 7ff67986e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559166737d40 0x5591667356a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:58.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.905+0000 7ff67986e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55916673a6d0 con 0x55916672d480 2024-09-12T03:02:58.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.905+0000 7ff67986e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55916664cf10 con 0x559166737d40 2024-09-12T03:02:58.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.905+0000 7ff67986e740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55916673a850 con 0x55916672cb60 2024-09-12T03:02:58.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.905+0000 7ff67986e740 10 client.0 _ll_register_callbacks cb 0x559166615a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:02:58.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.905+0000 7ff674a30640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55916672d480 0x559166735160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:58.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.905+0000 7ff674a30640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55916672d480 0x559166735160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37196/0 (socket says 172.21.15.5:37196) 2024-09-12T03:02:58.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.905+0000 7ff674a30640 1 -- 172.21.15.5:0/126491783 learned_addr learned my addr 172.21.15.5:0/126491783 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:02:58.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.906+0000 7ff675a32640 1 --2- 172.21.15.5:0/126491783 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559166737d40 0x5591667356a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:58.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.907+0000 7ff674a30640 1 -- 172.21.15.5:0/126491783 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55916672cb60 msgr2=0x559166734c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:58.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.907+0000 7ff674a30640 1 --2- 172.21.15.5:0/126491783 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55916672cb60 0x559166734c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:58.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.907+0000 7ff674a30640 1 -- 172.21.15.5:0/126491783 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559166737d40 msgr2=0x5591667356a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:58.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.907+0000 7ff674a30640 1 --2- 172.21.15.5:0/126491783 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559166737d40 0x5591667356a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:58.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.907+0000 7ff674a30640 1 -- 172.21.15.5:0/126491783 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591664036a0 con 0x55916672d480 2024-09-12T03:02:58.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.908+0000 7ff674a30640 1 --2- 172.21.15.5:0/126491783 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55916672d480 0x559166735160 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7ff66c00ec90 tx=0x7ff66c00d7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35d3f8b1078930ad server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:58.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.908+0000 7ff658ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T03:02:58.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.908+0000 7ff658ff9640 1 -- 172.21.15.5:0/126491783 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff66c005020 con 0x55916672d480 2024-09-12T03:02:58.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.908+0000 7ff658ff9640 1 -- 172.21.15.5:0/126491783 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7ff66c009040 con 0x55916672d480 2024-09-12T03:02:58.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.908+0000 7ff67986e740 1 -- 172.21.15.5:0/126491783 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6640bff40 con 0x55916672d480 2024-09-12T03:02:58.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.908+0000 7ff658ff9640 1 -- 172.21.15.5:0/126491783 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ff66c0051c0 con 0x55916672d480 2024-09-12T03:02:58.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.908+0000 7ff67986e740 10 client.16539 fetch_fsmap learned FSMap version 31 2024-09-12T03:02:58.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.908+0000 7ff67986e740 1 -- 172.21.15.5:0/126491783 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5591667462a0 con 0x55916672d480 2024-09-12T03:02:58.916 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.909+0000 7ff658ff9640 1 -- 172.21.15.5:0/126491783 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6640bff40 con 0x55916672d480 2024-09-12T03:02:58.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.909+0000 7ff658ff9640 1 -- 172.21.15.5:0/126491783 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7ff66c00b040 con 0x55916672d480 2024-09-12T03:02:58.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.909+0000 7ff67986e740 10 client.16539 fetch_fsmap finished waiting for FSMap version 31 2024-09-12T03:02:58.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.909+0000 7ff67986e740 10 client.16539 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:02:58.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.909+0000 7ff67986e740 20 client.16539 populate_metadata read hostname 'smithi005' 2024-09-12T03:02:58.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.909+0000 7ff67986e740 1 --2- 172.21.15.5:0/126491783 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x559166747800 0x559166749c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:58.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.909+0000 7ff67986e740 4 client.16539 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:02:58.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.909+0000 7ff67986e740 1 -- 172.21.15.5:0/126491783 --> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55916669f7f0 con 0x559166747800 2024-09-12T03:02:58.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.909+0000 7ff675231640 1 --2- 172.21.15.5:0/126491783 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x559166747800 0x559166749c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:58.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.910+0000 7ff658ff9640 10 client.16539 ms_handle_connect on v2:172.21.15.148:6838/2317600000 2024-09-12T03:02:58.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.910+0000 7ff675231640 1 --2- 172.21.15.5:0/126491783 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x559166747800 0x559166749c80 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6585bff3c9e4c744 server_cookie=b35208a75026fc0c in_seq=0 out_seq=0 2024-09-12T03:02:58.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.911+0000 7ff658ff9640 1 -- 172.21.15.5:0/126491783 <== mds.0 v2:172.21.15.148:6838/2317600000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55916669f7f0 con 0x559166747800 2024-09-12T03:02:58.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.911+0000 7ff658ff9640 10 client.16539 handle_command_reply: tid=0 2024-09-12T03:02:58.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.911+0000 7ff67986e740 1 client.16539 shutdown 2024-09-12T03:02:58.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.911+0000 7ff67986e740 1 -- 172.21.15.5:0/126491783 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55916672d480 msgr2=0x559166735160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:58.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.911+0000 7ff67986e740 1 --2- 172.21.15.5:0/126491783 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55916672d480 0x559166735160 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7ff66c00ec90 tx=0x7ff66c00d7b0 comp rx=0 tx=0).stop 2024-09-12T03:02:58.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.911+0000 7ff67986e740 1 -- 172.21.15.5:0/126491783 shutdown_connections 2024-09-12T03:02:58.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.911+0000 7ff67986e740 1 --2- 172.21.15.5:0/126491783 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x559166747800 0x559166749c80 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:58.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.911+0000 7ff67986e740 1 --2- 172.21.15.5:0/126491783 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55916672d480 0x559166735160 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:58.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.911+0000 7ff67986e740 1 --2- 172.21.15.5:0/126491783 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559166737d40 0x5591667356a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:58.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.911+0000 7ff658ff9640 0 client.16539 ms_handle_reset on v2:172.21.15.148:6838/2317600000 2024-09-12T03:02:58.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.911+0000 7ff67986e740 1 --2- 172.21.15.5:0/126491783 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55916672cb60 0x559166734c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:58.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.911+0000 7ff67986e740 1 -- 172.21.15.5:0/126491783 >> 172.21.15.5:0/126491783 conn(0x5591667290f0 msgr2=0x55916672ee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:58.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.912+0000 7ff67986e740 1 -- 172.21.15.5:0/126491783 shutdown_connections 2024-09-12T03:02:58.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.912+0000 7ff67986e740 1 -- 172.21.15.5:0/126491783 wait complete. 2024-09-12T03:02:58.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.912+0000 7ff67986e740 20 client.16539 trim_cache size 0 max 16384 2024-09-12T03:02:58.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.932+0000 7ff67986e740 1 Processor -- start 2024-09-12T03:02:58.933 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.932+0000 7ff67986e740 1 -- start start 2024-09-12T03:02:58.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.933+0000 7ff67986e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55916672cb60 0x559166736e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:58.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.933+0000 7ff67986e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55916672d480 0x55916672f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:58.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.933+0000 7ff67986e740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559166737d40 0x55916674f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:58.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.933+0000 7ff67986e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff6701b3b40 con 0x55916672d480 2024-09-12T03:02:58.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.933+0000 7ff67986e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ff6701b5680 con 0x55916672cb60 2024-09-12T03:02:58.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.933+0000 7ff67986e740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff6701b4210 con 0x559166737d40 2024-09-12T03:02:58.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.933+0000 7ff674a30640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55916672d480 0x55916672f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:58.934 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.933+0000 7ff674a30640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55916672d480 0x55916672f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37198/0 (socket says 172.21.15.5:37198) 2024-09-12T03:02:58.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.933+0000 7ff674a30640 1 -- 172.21.15.5:0/1917028604 learned_addr learned my addr 172.21.15.5:0/1917028604 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:02:58.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.933+0000 7ff675231640 1 --2- 172.21.15.5:0/1917028604 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55916672cb60 0x559166736e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:58.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.933+0000 7ff675a32640 1 --2- 172.21.15.5:0/1917028604 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559166737d40 0x55916674f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:58.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.933+0000 7ff675231640 1 -- 172.21.15.5:0/1917028604 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559166737d40 msgr2=0x55916674f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:58.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.933+0000 7ff675231640 1 --2- 172.21.15.5:0/1917028604 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559166737d40 0x55916674f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:58.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.933+0000 7ff675231640 1 -- 172.21.15.5:0/1917028604 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55916672d480 msgr2=0x55916672f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:58.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.933+0000 7ff675231640 1 --2- 172.21.15.5:0/1917028604 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55916672d480 0x55916672f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:58.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.933+0000 7ff675231640 1 -- 172.21.15.5:0/1917028604 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55916672d2b0 con 0x55916672cb60 2024-09-12T03:02:58.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.934+0000 7ff675231640 1 --2- 172.21.15.5:0/1917028604 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55916672cb60 0x559166736e40 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff6640bbbd0 tx=0x7ff664000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e6da23fc5b9c27e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:58.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.934+0000 7ff65affd640 1 -- 172.21.15.5:0/1917028604 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6640d8070 con 0x55916672cb60 2024-09-12T03:02:58.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.934+0000 7ff65affd640 1 -- 172.21.15.5:0/1917028604 <== mon.2 v2:172.21.15.146:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff6640b9ce0 con 0x55916672cb60 2024-09-12T03:02:58.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.934+0000 7ff65affd640 1 -- 172.21.15.5:0/1917028604 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff664049040 con 0x55916672cb60 2024-09-12T03:02:58.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.935+0000 7ff67986e740 1 -- 172.21.15.5:0/1917028604 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55916672cb60 msgr2=0x559166736e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:58.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.935+0000 7ff67986e740 1 --2- 172.21.15.5:0/1917028604 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55916672cb60 0x559166736e40 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff6640bbbd0 tx=0x7ff664000bc0 comp rx=0 tx=0).stop 2024-09-12T03:02:58.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.935+0000 7ff67986e740 1 -- 172.21.15.5:0/1917028604 shutdown_connections 2024-09-12T03:02:58.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.935+0000 7ff67986e740 1 --2- 172.21.15.5:0/1917028604 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55916672d480 0x55916672f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:58.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.935+0000 7ff67986e740 1 --2- 172.21.15.5:0/1917028604 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559166737d40 0x55916674f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:58.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.935+0000 7ff67986e740 1 --2- 172.21.15.5:0/1917028604 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55916672cb60 0x559166736e40 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:58.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.935+0000 7ff67986e740 1 -- 172.21.15.5:0/1917028604 >> 172.21.15.5:0/1917028604 conn(0x5591667290f0 msgr2=0x559166733be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:58.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.935+0000 7ff67986e740 1 -- 172.21.15.5:0/1917028604 shutdown_connections 2024-09-12T03:02:58.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.935+0000 7ff67986e740 1 -- 172.21.15.5:0/1917028604 wait complete. 2024-09-12T03:02:58.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.935+0000 7ff67986e740 1 -- start start 2024-09-12T03:02:58.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.935+0000 7ff67986e740 1 Processor -- start 2024-09-12T03:02:58.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.936+0000 7ff67986e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55916672cb60 0x55916673b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:58.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.936+0000 7ff67986e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55916672d480 0x55916673bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:58.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.936+0000 7ff67986e740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559166737d40 0x55916673c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:58.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.936+0000 7ff674a30640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55916672d480 0x55916673bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:58.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.936+0000 7ff675231640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55916672cb60 0x55916673b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:58.940 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.936+0000 7ff675231640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55916672cb60 0x55916673b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.5:51350/0 (socket says 172.21.15.5:51350) 2024-09-12T03:02:58.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.936+0000 7ff675231640 1 -- 172.21.15.5:0/3636285769 learned_addr learned my addr 172.21.15.5:0/3636285769 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:02:58.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.936+0000 7ff67986e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff6701b4fb0 con 0x55916672d480 2024-09-12T03:02:58.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.936+0000 7ff67986e740 1 -- 172.21.15.5:0/3636285769 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ff6701b48e0 con 0x55916672cb60 2024-09-12T03:02:58.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.936+0000 7ff67986e740 1 -- 172.21.15.5:0/3636285769 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff6701b1b20 con 0x559166737d40 2024-09-12T03:02:58.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.936+0000 7ff675a32640 1 --2- 172.21.15.5:0/3636285769 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559166737d40 0x55916673c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:58.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.936+0000 7ff67986e740 10 client.0 _ll_register_callbacks cb 0x7ff66c00ae10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:02:58.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.936+0000 7ff675231640 1 -- 172.21.15.5:0/3636285769 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559166737d40 msgr2=0x55916673c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:58.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.936+0000 7ff675231640 1 --2- 172.21.15.5:0/3636285769 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559166737d40 0x55916673c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:58.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.936+0000 7ff675231640 1 -- 172.21.15.5:0/3636285769 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55916672d480 msgr2=0x55916673bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:02:58.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.936+0000 7ff675231640 1 --2- 172.21.15.5:0/3636285769 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55916672d480 0x55916673bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:58.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.936+0000 7ff675231640 1 -- 172.21.15.5:0/3636285769 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591667b1680 con 0x55916672cb60 2024-09-12T03:02:58.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.936+0000 7ff674a30640 1 --2- 172.21.15.5:0/3636285769 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55916672d480 0x55916673bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:02:58.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.937+0000 7ff675231640 1 --2- 172.21.15.5:0/3636285769 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55916672cb60 0x55916673b810 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff6640bb940 tx=0x7ff6640b78d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62956e6876b31113 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:02:58.941 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.937+0000 7ff658ff9640 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2024-09-12T03:02:58.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.937+0000 7ff658ff9640 1 -- 172.21.15.5:0/3636285769 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6640a0060 con 0x55916672cb60 2024-09-12T03:02:58.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.937+0000 7ff658ff9640 1 -- 172.21.15.5:0/3636285769 <== mon.2 v2:172.21.15.146:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7ff6640b82b0 con 0x55916672cb60 2024-09-12T03:02:58.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.937+0000 7ff67986e740 1 -- 172.21.15.5:0/3636285769 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559166699d00 con 0x55916672cb60 2024-09-12T03:02:58.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.937+0000 7ff658ff9640 1 -- 172.21.15.5:0/3636285769 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ff6640d8070 con 0x55916672cb60 2024-09-12T03:02:58.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.938+0000 7ff67986e740 10 client.25934 fetch_fsmap learned FSMap version 31 2024-09-12T03:02:58.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.938+0000 7ff67986e740 1 -- 172.21.15.5:0/3636285769 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5591667b2900 con 0x55916672cb60 2024-09-12T03:02:58.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.938+0000 7ff658ff9640 1 -- 172.21.15.5:0/3636285769 <== mon.2 v2:172.21.15.146:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x559166699d00 con 0x55916672cb60 2024-09-12T03:02:58.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.938+0000 7ff658ff9640 1 -- 172.21.15.5:0/3636285769 <== mon.2 v2:172.21.15.146:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7ff6640ee040 con 0x55916672cb60 2024-09-12T03:02:58.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.938+0000 7ff67986e740 10 client.25934 fetch_fsmap finished waiting for FSMap version 31 2024-09-12T03:02:58.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.938+0000 7ff67986e740 10 client.25934 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:02:58.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.938+0000 7ff67986e740 20 client.25934 populate_metadata read hostname 'smithi005' 2024-09-12T03:02:58.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.938+0000 7ff67986e740 1 --2- 172.21.15.5:0/3636285769 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x5591667b2bf0 0x5591667b4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:02:58.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.938+0000 7ff67986e740 4 client.25934 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T03:02:58.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.938+0000 7ff67986e740 1 -- 172.21.15.5:0/3636285769 --> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x559166616110 con 0x5591667b2bf0 2024-09-12T03:02:58.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.938+0000 7ff674a30640 1 --2- 172.21.15.5:0/3636285769 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x5591667b2bf0 0x5591667b4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:02:58.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.939+0000 7ff658ff9640 10 client.25934 ms_handle_connect on v2:172.21.15.148:6838/2317600000 2024-09-12T03:02:58.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.939+0000 7ff674a30640 1 --2- 172.21.15.5:0/3636285769 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x5591667b2bf0 0x5591667b4fe0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d3bf21f603eb68 server_cookie=4a37baacf8a3cfbc in_seq=0 out_seq=0 2024-09-12T03:02:58.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.940+0000 7ff658ff9640 1 -- 172.21.15.5:0/3636285769 <== mds.0 v2:172.21.15.148:6838/2317600000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x559166616110 con 0x5591667b2bf0 2024-09-12T03:02:58.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.940+0000 7ff658ff9640 10 client.25934 handle_command_reply: tid=0 2024-09-12T03:02:58.944 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.940+0000 7ff67986e740 1 client.25934 shutdown 2024-09-12T03:02:58.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.940+0000 7ff67986e740 1 -- 172.21.15.5:0/3636285769 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55916672cb60 msgr2=0x55916673b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:58.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.940+0000 7ff67986e740 1 --2- 172.21.15.5:0/3636285769 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55916672cb60 0x55916673b810 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff6640bb940 tx=0x7ff6640b78d0 comp rx=0 tx=0).stop 2024-09-12T03:02:58.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.940+0000 7ff67986e740 1 -- 172.21.15.5:0/3636285769 shutdown_connections 2024-09-12T03:02:58.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.940+0000 7ff67986e740 1 --2- 172.21.15.5:0/3636285769 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x5591667b2bf0 0x5591667b4fe0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:58.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.940+0000 7ff67986e740 1 --2- 172.21.15.5:0/3636285769 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55916672d480 0x55916673bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:58.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.940+0000 7ff67986e740 1 --2- 172.21.15.5:0/3636285769 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559166737d40 0x55916673c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:58.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.940+0000 7ff658ff9640 0 client.25934 ms_handle_reset on v2:172.21.15.148:6838/2317600000 2024-09-12T03:02:58.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.940+0000 7ff67986e740 1 --2- 172.21.15.5:0/3636285769 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55916672cb60 0x55916673b810 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:58.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.940+0000 7ff67986e740 1 -- 172.21.15.5:0/3636285769 >> 172.21.15.5:0/3636285769 conn(0x5591667290f0 msgr2=0x5591667333c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:58.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.940+0000 7ff67986e740 1 -- 172.21.15.5:0/3636285769 shutdown_connections 2024-09-12T03:02:58.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.940+0000 7ff67986e740 1 -- 172.21.15.5:0/3636285769 wait complete. 2024-09-12T03:02:58.945 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.941+0000 7ff67986e740 20 client.25934 trim_cache size 0 max 16384 2024-09-12T03:02:58.946 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-12T03:02:58.946 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-09-12T03:02:58.946 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "0af9ebc1-25c8-44f5-a071-2cb5e056e2ff", 2024-09-12T03:02:58.946 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-09-12T03:02:58.946 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-12T03:02:58.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.941+0000 7ff677495640 1 -- 172.21.15.5:0/3112398669 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7ff65c0a6f90 msgr2=0x7ff65c0a9450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:58.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.941+0000 7ff677495640 1 --2- 172.21.15.5:0/3112398669 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7ff65c0a6f90 0x7ff65c0a9450 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7ff668005fd0 tx=0x7ff668011b40 comp rx=0 tx=0).stop 2024-09-12T03:02:58.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.941+0000 7ff677495640 1 -- 172.21.15.5:0/3112398669 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff670109450 msgr2=0x7ff6701a68b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:02:58.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.941+0000 7ff677495640 1 --2- 172.21.15.5:0/3112398669 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff670109450 0x7ff6701a68b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff664005940 tx=0x7ff6640040c0 comp rx=0 tx=0).stop 2024-09-12T03:02:58.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.941+0000 7ff677495640 1 -- 172.21.15.5:0/3112398669 shutdown_connections 2024-09-12T03:02:58.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.941+0000 7ff677495640 1 --2- 172.21.15.5:0/3112398669 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7ff65c0a6f90 0x7ff65c0a9450 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:58.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.941+0000 7ff677495640 1 --2- 172.21.15.5:0/3112398669 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff670109e00 0x7ff6701a6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:58.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.941+0000 7ff677495640 1 --2- 172.21.15.5:0/3112398669 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff67010ddc0 0x7ff6701a0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:58.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.941+0000 7ff677495640 1 --2- 172.21.15.5:0/3112398669 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff670109450 0x7ff6701a68b0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:02:58.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.941+0000 7ff677495640 1 -- 172.21.15.5:0/3112398669 >> 172.21.15.5:0/3112398669 conn(0x7ff67006e140 msgr2=0x7ff670071bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:02:58.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.941+0000 7ff677495640 1 -- 172.21.15.5:0/3112398669 shutdown_connections 2024-09-12T03:02:58.948 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:02:58.941+0000 7ff677495640 1 -- 172.21.15.5:0/3112398669 wait complete. 2024-09-12T03:02:59.165 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:02:58 smithi148 ceph-mon[35090]: pgmap v477: 97 pgs: 97 active+clean; 133 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 2.7 MiB/s wr, 623 op/s 2024-09-12T03:02:59.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:02:58 smithi146 ceph-mon[35011]: pgmap v477: 97 pgs: 97 active+clean; 133 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 2.7 MiB/s wr, 623 op/s 2024-09-12T03:02:59.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:02:59 smithi005 ceph-mon[31197]: scrub queued for path: / 2024-09-12T03:02:59.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:02:59 smithi005 ceph-mon[31197]: scrub summary: idle+waiting paths [/] 2024-09-12T03:02:59.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:02:59 smithi005 ceph-mon[31197]: scrub summary: active paths [/] 2024-09-12T03:02:59.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:02:59 smithi005 ceph-mon[31197]: scrub summary: active paths [/] 2024-09-12T03:02:59.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:02:59 smithi005 ceph-mon[31197]: scrub summary: idle+waiting paths [/] 2024-09-12T03:02:59.811 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:02:59 smithi005 ceph-mon[31197]: scrub summary: active paths [/] 2024-09-12T03:02:59.811 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:02:59 smithi005 ceph-mon[31197]: scrub summary: idle+waiting paths [/] 2024-09-12T03:02:59.811 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:02:59 smithi005 ceph-mon[31197]: scrub summary: active paths [/] 2024-09-12T03:02:59.811 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:02:59 smithi005 ceph-mon[31197]: scrub summary: idle+waiting paths [/] 2024-09-12T03:02:59.811 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:02:59 smithi005 ceph-mon[31197]: scrub summary: idle+waiting paths [/] 2024-09-12T03:02:59.926 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T03:03:00.165 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:02:59 smithi148 ceph-mon[35090]: scrub queued for path: / 2024-09-12T03:03:00.165 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:02:59 smithi148 ceph-mon[35090]: scrub summary: idle+waiting paths [/] 2024-09-12T03:03:00.165 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:02:59 smithi148 ceph-mon[35090]: scrub summary: active paths [/] 2024-09-12T03:03:00.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:02:59 smithi148 ceph-mon[35090]: scrub summary: active paths [/] 2024-09-12T03:03:00.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:02:59 smithi148 ceph-mon[35090]: scrub summary: idle+waiting paths [/] 2024-09-12T03:03:00.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:02:59 smithi148 ceph-mon[35090]: scrub summary: active paths [/] 2024-09-12T03:03:00.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:02:59 smithi148 ceph-mon[35090]: scrub summary: idle+waiting paths [/] 2024-09-12T03:03:00.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:02:59 smithi148 ceph-mon[35090]: scrub summary: active paths [/] 2024-09-12T03:03:00.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:02:59 smithi148 ceph-mon[35090]: scrub summary: idle+waiting paths [/] 2024-09-12T03:03:00.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:02:59 smithi148 ceph-mon[35090]: scrub summary: idle+waiting paths [/] 2024-09-12T03:03:00.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:02:59 smithi146 ceph-mon[35011]: scrub queued for path: / 2024-09-12T03:03:00.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:02:59 smithi146 ceph-mon[35011]: scrub summary: idle+waiting paths [/] 2024-09-12T03:03:00.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:02:59 smithi146 ceph-mon[35011]: scrub summary: active paths [/] 2024-09-12T03:03:00.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:02:59 smithi146 ceph-mon[35011]: scrub summary: active paths [/] 2024-09-12T03:03:00.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:02:59 smithi146 ceph-mon[35011]: scrub summary: idle+waiting paths [/] 2024-09-12T03:03:00.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:02:59 smithi146 ceph-mon[35011]: scrub summary: active paths [/] 2024-09-12T03:03:00.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:02:59 smithi146 ceph-mon[35011]: scrub summary: idle+waiting paths [/] 2024-09-12T03:03:00.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:02:59 smithi146 ceph-mon[35011]: scrub summary: active paths [/] 2024-09-12T03:03:00.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:02:59 smithi146 ceph-mon[35011]: scrub summary: idle+waiting paths [/] 2024-09-12T03:03:00.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:02:59 smithi146 ceph-mon[35011]: scrub summary: idle+waiting paths [/] 2024-09-12T03:03:00.243 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:03:00.769 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.752+0000 7ff8aba13640 1 -- 172.21.15.5:0/794736348 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8a4106f10 msgr2=0x7ff8a410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:00.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.752+0000 7ff8aba13640 1 --2- 172.21.15.5:0/794736348 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8a4106f10 0x7ff8a410ea90 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7ff89c009ca0 tx=0x7ff89c02a8c0 comp rx=0 tx=0).stop 2024-09-12T03:03:00.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.753+0000 7ff8aba13640 1 -- 172.21.15.5:0/794736348 shutdown_connections 2024-09-12T03:03:00.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.753+0000 7ff8aba13640 1 --2- 172.21.15.5:0/794736348 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8a4106f10 0x7ff8a410ea90 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:00.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.753+0000 7ff8aba13640 1 --2- 172.21.15.5:0/794736348 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff8a4102790 0x7ff8a41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:00.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.753+0000 7ff8aba13640 1 --2- 172.21.15.5:0/794736348 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff8a4101de0 0x7ff8a41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:00.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.753+0000 7ff8aba13640 1 -- 172.21.15.5:0/794736348 >> 172.21.15.5:0/794736348 conn(0x7ff8a40fd8c0 msgr2=0x7ff8a40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:00.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.753+0000 7ff8aba13640 1 -- 172.21.15.5:0/794736348 shutdown_connections 2024-09-12T03:03:00.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.753+0000 7ff8aba13640 1 -- 172.21.15.5:0/794736348 wait complete. 2024-09-12T03:03:00.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.753+0000 7ff8aba13640 1 Processor -- start 2024-09-12T03:03:00.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.753+0000 7ff8aba13640 1 -- start start 2024-09-12T03:03:00.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.754+0000 7ff8aba13640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8a4101de0 0x7ff8a4198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:00.770 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.754+0000 7ff8aba13640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff8a4102790 0x7ff8a4198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:00.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.754+0000 7ff8aba13640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff8a4106f10 0x7ff8a419cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:00.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.754+0000 7ff8aba13640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff8a4111fc0 con 0x7ff8a4101de0 2024-09-12T03:03:00.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.754+0000 7ff8aba13640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ff8a4111e40 con 0x7ff8a4106f10 2024-09-12T03:03:00.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.754+0000 7ff8aba13640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff8a4112140 con 0x7ff8a4102790 2024-09-12T03:03:00.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.754+0000 7ff8a8fae640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff8a4102790 0x7ff8a4198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:00.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.754+0000 7ff8a97af640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8a4101de0 0x7ff8a4198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:00.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.754+0000 7ff8a8fae640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff8a4102790 0x7ff8a4198880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.5:47704/0 (socket says 172.21.15.5:47704) 2024-09-12T03:03:00.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.754+0000 7ff8a8fae640 1 -- 172.21.15.5:0/3821304148 learned_addr learned my addr 172.21.15.5:0/3821304148 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:03:00.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.754+0000 7ff8a9fb0640 1 --2- 172.21.15.5:0/3821304148 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff8a4106f10 0x7ff8a419cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:00.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.754+0000 7ff8a8fae640 1 -- 172.21.15.5:0/3821304148 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff8a4106f10 msgr2=0x7ff8a419cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:00.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.754+0000 7ff8a8fae640 1 --2- 172.21.15.5:0/3821304148 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff8a4106f10 0x7ff8a419cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:00.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.754+0000 7ff8a8fae640 1 -- 172.21.15.5:0/3821304148 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8a4101de0 msgr2=0x7ff8a4198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:00.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.754+0000 7ff8a8fae640 1 --2- 172.21.15.5:0/3821304148 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8a4101de0 0x7ff8a4198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:00.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.754+0000 7ff8a8fae640 1 -- 172.21.15.5:0/3821304148 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8a419d2f0 con 0x7ff8a4102790 2024-09-12T03:03:00.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.754+0000 7ff8a9fb0640 1 --2- 172.21.15.5:0/3821304148 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff8a4106f10 0x7ff8a419cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:03:00.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.754+0000 7ff8a97af640 1 --2- 172.21.15.5:0/3821304148 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8a4101de0 0x7ff8a4198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:03:00.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.755+0000 7ff8a8fae640 1 --2- 172.21.15.5:0/3821304148 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff8a4102790 0x7ff8a4198880 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff89800c920 tx=0x7ff89800cde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:00.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.755+0000 7ff8967fc640 1 -- 172.21.15.5:0/3821304148 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff898007c70 con 0x7ff8a4102790 2024-09-12T03:03:00.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.755+0000 7ff8967fc640 1 -- 172.21.15.5:0/3821304148 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff898007e10 con 0x7ff8a4102790 2024-09-12T03:03:00.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.755+0000 7ff8aba13640 1 -- 172.21.15.5:0/3821304148 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8a419d5e0 con 0x7ff8a4102790 2024-09-12T03:03:00.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.755+0000 7ff8967fc640 1 -- 172.21.15.5:0/3821304148 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff898018890 con 0x7ff8a4102790 2024-09-12T03:03:00.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.755+0000 7ff8aba13640 1 -- 172.21.15.5:0/3821304148 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8a41a4ec0 con 0x7ff8a4102790 2024-09-12T03:03:00.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.757+0000 7ff8967fc640 1 -- 172.21.15.5:0/3821304148 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7ff898018a70 con 0x7ff8a4102790 2024-09-12T03:03:00.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.758+0000 7ff8addec740 1 Processor -- start 2024-09-12T03:03:00.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.758+0000 7ff8addec740 1 -- start start 2024-09-12T03:03:00.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.759+0000 7ff8967fc640 1 --2- 172.21.15.5:0/3821304148 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7ff8900a2b80 0x7ff8900a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:00.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.759+0000 7ff8addec740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563c268e33a0 0x563c268e3780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:00.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.759+0000 7ff8addec740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c268e3cc0 0x563c268f2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:00.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.759+0000 7ff8addec740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563c268f2a90 0x563c268f4e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:00.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.759+0000 7ff8addec740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x563c268492e0 con 0x563c268e3cc0 2024-09-12T03:03:00.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.759+0000 7ff8addec740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x563c267f0ce0 con 0x563c268f2a90 2024-09-12T03:03:00.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.759+0000 7ff8addec740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x563c268452a0 con 0x563c268e33a0 2024-09-12T03:03:00.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.759+0000 7ff8a9fb0640 1 --2- 172.21.15.5:0/3821304148 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7ff8900a2b80 0x7ff8900a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:00.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.759+0000 7ff8a8fae640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c268e3cc0 0x563c268f2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:00.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.759+0000 7ff8a97af640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563c268e33a0 0x563c268e3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:00.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.759+0000 7ff8a8fae640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c268e3cc0 0x563c268f2550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37244/0 (socket says 172.21.15.5:37244) 2024-09-12T03:03:00.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.759+0000 7ff8a97af640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563c268e33a0 0x563c268e3780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.5:47716/0 (socket says 172.21.15.5:47716) 2024-09-12T03:03:00.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.759+0000 7ff8a8fae640 1 -- 172.21.15.5:0/4055675915 learned_addr learned my addr 172.21.15.5:0/4055675915 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:03:00.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.759+0000 7ff8967fc640 1 -- 172.21.15.5:0/3821304148 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7ff8980cefe0 con 0x7ff8a4102790 2024-09-12T03:03:00.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.759+0000 7ff8a9fb0640 1 --2- 172.21.15.5:0/4055675915 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563c268f2a90 0x563c268f4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:00.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.759+0000 7ff8a8fae640 1 -- 172.21.15.5:0/4055675915 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563c268e33a0 msgr2=0x563c268e3780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:00.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.759+0000 7ff8a8fae640 1 --2- 172.21.15.5:0/4055675915 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563c268e33a0 0x563c268e3780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:00.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.759+0000 7ff8a8fae640 1 -- 172.21.15.5:0/4055675915 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563c268f2a90 msgr2=0x563c268f4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:00.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.759+0000 7ff8a8fae640 1 --2- 172.21.15.5:0/4055675915 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563c268f2a90 0x563c268f4e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:00.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.759+0000 7ff8a8fae640 1 -- 172.21.15.5:0/4055675915 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c267d5440 con 0x563c268e3cc0 2024-09-12T03:03:00.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.759+0000 7ff8a9fb0640 1 --2- 172.21.15.5:0/3821304148 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7ff8900a2b80 0x7ff8900a5040 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7ff89c006010 tx=0x7ff89c02f500 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:00.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.759+0000 7ff8a8fae640 1 --2- 172.21.15.5:0/4055675915 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c268e3cc0 0x563c268f2550 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7ff89809b880 tx=0x7ff8980148a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa0d4ffd77ca9366 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:00.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.760+0000 7ff88f7fe640 1 -- 172.21.15.5:0/4055675915 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff89809b250 con 0x563c268e3cc0 2024-09-12T03:03:00.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.760+0000 7ff88f7fe640 1 -- 172.21.15.5:0/4055675915 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff89809bff0 con 0x563c268e3cc0 2024-09-12T03:03:00.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.760+0000 7ff88f7fe640 1 -- 172.21.15.5:0/4055675915 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff898015d70 con 0x563c268e3cc0 2024-09-12T03:03:00.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.760+0000 7ff8addec740 1 -- 172.21.15.5:0/4055675915 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c268e3cc0 msgr2=0x563c268f2550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:00.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.760+0000 7ff8addec740 1 --2- 172.21.15.5:0/4055675915 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c268e3cc0 0x563c268f2550 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7ff89809b880 tx=0x7ff8980148a0 comp rx=0 tx=0).stop 2024-09-12T03:03:00.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.761+0000 7ff8addec740 1 -- 172.21.15.5:0/4055675915 shutdown_connections 2024-09-12T03:03:00.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.761+0000 7ff8addec740 1 --2- 172.21.15.5:0/4055675915 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c268e3cc0 0x563c268f2550 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:00.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.761+0000 7ff8addec740 1 --2- 172.21.15.5:0/4055675915 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563c268f2a90 0x563c268f4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:00.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.761+0000 7ff8addec740 1 --2- 172.21.15.5:0/4055675915 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563c268e33a0 0x563c268e3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:00.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.761+0000 7ff8addec740 1 -- 172.21.15.5:0/4055675915 >> 172.21.15.5:0/4055675915 conn(0x563c268dfa30 msgr2=0x563c268dfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:00.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.761+0000 7ff8addec740 1 -- 172.21.15.5:0/4055675915 shutdown_connections 2024-09-12T03:03:00.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.761+0000 7ff8addec740 1 -- 172.21.15.5:0/4055675915 wait complete. 2024-09-12T03:03:00.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.761+0000 7ff8addec740 1 -- start start 2024-09-12T03:03:00.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.761+0000 7ff8addec740 1 Processor -- start 2024-09-12T03:03:00.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.762+0000 7ff8addec740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c268e33a0 0x563c268f2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:00.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.762+0000 7ff8addec740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563c268e3cc0 0x563c268ef5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:00.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.762+0000 7ff8addec740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563c268f2a90 0x563c268efb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:00.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.762+0000 7ff8addec740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x563c268f56c0 con 0x563c268e33a0 2024-09-12T03:03:00.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.762+0000 7ff8addec740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x563c26804050 con 0x563c268f2a90 2024-09-12T03:03:00.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.762+0000 7ff8addec740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x563c268f5840 con 0x563c268e3cc0 2024-09-12T03:03:00.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.762+0000 7ff8addec740 10 client.0 _ll_register_callbacks cb 0x563c26844160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:03:00.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.762+0000 7ff8a97af640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c268e33a0 0x563c268f2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:00.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.762+0000 7ff8a97af640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c268e33a0 0x563c268f2330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37260/0 (socket says 172.21.15.5:37260) 2024-09-12T03:03:00.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.762+0000 7ff8a97af640 1 -- 172.21.15.5:0/965916475 learned_addr learned my addr 172.21.15.5:0/965916475 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:03:00.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.762+0000 7ff8a9fb0640 1 --2- 172.21.15.5:0/965916475 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563c268f2a90 0x563c268efb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:00.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.762+0000 7ff8a8fae640 1 --2- 172.21.15.5:0/965916475 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563c268e3cc0 0x563c268ef5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:00.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.762+0000 7ff8a9fb0640 1 -- 172.21.15.5:0/965916475 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563c268e3cc0 msgr2=0x563c268ef5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:00.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.762+0000 7ff8a9fb0640 1 --2- 172.21.15.5:0/965916475 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563c268e3cc0 0x563c268ef5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:00.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.762+0000 7ff8a9fb0640 1 -- 172.21.15.5:0/965916475 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c268e33a0 msgr2=0x563c268f2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:00.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.762+0000 7ff8a9fb0640 1 --2- 172.21.15.5:0/965916475 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c268e33a0 0x563c268f2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:00.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.762+0000 7ff8a9fb0640 1 -- 172.21.15.5:0/965916475 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c265ba3c0 con 0x563c268f2a90 2024-09-12T03:03:00.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.762+0000 7ff8a97af640 1 --2- 172.21.15.5:0/965916475 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c268e33a0 0x563c268f2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:03:00.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.762+0000 7ff8a9fb0640 1 --2- 172.21.15.5:0/965916475 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563c268f2a90 0x563c268efb20 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff89c005e30 tx=0x7ff89c02a8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c435ebaccf32bf1d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:00.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.762+0000 7ff88cff9640 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2024-09-12T03:03:00.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.763+0000 7ff88cff9640 1 -- 172.21.15.5:0/965916475 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff89c02fdf0 con 0x563c268f2a90 2024-09-12T03:03:00.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.763+0000 7ff8addec740 1 -- 172.21.15.5:0/965916475 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff89809bff0 con 0x563c268f2a90 2024-09-12T03:03:00.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.763+0000 7ff88cff9640 1 -- 172.21.15.5:0/965916475 <== mon.2 v2:172.21.15.146:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7ff89c0305c0 con 0x563c268f2a90 2024-09-12T03:03:00.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.763+0000 7ff88cff9640 1 -- 172.21.15.5:0/965916475 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ff89c031990 con 0x563c268f2a90 2024-09-12T03:03:00.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.763+0000 7ff8addec740 10 client.25946 fetch_fsmap learned FSMap version 31 2024-09-12T03:03:00.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.763+0000 7ff8addec740 1 -- 172.21.15.5:0/965916475 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x563c26901050 con 0x563c268f2a90 2024-09-12T03:03:00.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.763+0000 7ff88cff9640 1 -- 172.21.15.5:0/965916475 <== mon.2 v2:172.21.15.146:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff89809bff0 con 0x563c268f2a90 2024-09-12T03:03:00.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.763+0000 7ff88cff9640 1 -- 172.21.15.5:0/965916475 <== mon.2 v2:172.21.15.146:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7ff89c04c850 con 0x563c268f2a90 2024-09-12T03:03:00.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.763+0000 7ff8addec740 10 client.25946 fetch_fsmap finished waiting for FSMap version 31 2024-09-12T03:03:00.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.763+0000 7ff8addec740 10 client.25946 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:03:00.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.763+0000 7ff8addec740 20 client.25946 populate_metadata read hostname 'smithi005' 2024-09-12T03:03:00.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.763+0000 7ff8addec740 1 --2- 172.21.15.5:0/965916475 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x563c26902440 0x563c26904850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:00.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.763+0000 7ff8addec740 4 client.25946 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:03:00.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.763+0000 7ff8addec740 1 -- 172.21.15.5:0/965916475 --> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563c26856850 con 0x563c26902440 2024-09-12T03:03:00.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.764+0000 7ff8a97af640 1 --2- 172.21.15.5:0/965916475 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x563c26902440 0x563c26904850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:00.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.764+0000 7ff88cff9640 10 client.25946 ms_handle_connect on v2:172.21.15.148:6838/2317600000 2024-09-12T03:03:00.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.764+0000 7ff8a97af640 1 --2- 172.21.15.5:0/965916475 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x563c26902440 0x563c26904850 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e42781e32ae670c8 server_cookie=25d81e3f07f3a554 in_seq=0 out_seq=0 2024-09-12T03:03:00.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.765+0000 7ff88cff9640 1 -- 172.21.15.5:0/965916475 <== mds.0 v2:172.21.15.148:6838/2317600000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563c26856850 con 0x563c26902440 2024-09-12T03:03:00.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.765+0000 7ff88cff9640 10 client.25946 handle_command_reply: tid=0 2024-09-12T03:03:00.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.765+0000 7ff8addec740 1 client.25946 shutdown 2024-09-12T03:03:00.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.766+0000 7ff8addec740 1 -- 172.21.15.5:0/965916475 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563c268f2a90 msgr2=0x563c268efb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:00.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.766+0000 7ff8addec740 1 --2- 172.21.15.5:0/965916475 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563c268f2a90 0x563c268efb20 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff89c005e30 tx=0x7ff89c02a8c0 comp rx=0 tx=0).stop 2024-09-12T03:03:00.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.766+0000 7ff8addec740 1 -- 172.21.15.5:0/965916475 shutdown_connections 2024-09-12T03:03:00.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.766+0000 7ff8addec740 1 --2- 172.21.15.5:0/965916475 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563c268f2a90 0x563c268efb20 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:00.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.766+0000 7ff8addec740 1 --2- 172.21.15.5:0/965916475 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563c268e3cc0 0x563c268ef5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:00.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.766+0000 7ff8addec740 1 --2- 172.21.15.5:0/965916475 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x563c26902440 0x563c26904850 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:00.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.766+0000 7ff8addec740 1 --2- 172.21.15.5:0/965916475 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c268e33a0 0x563c268f2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:00.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.766+0000 7ff88cff9640 0 client.25946 ms_handle_reset on v2:172.21.15.148:6838/2317600000 2024-09-12T03:03:00.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.766+0000 7ff8addec740 1 -- 172.21.15.5:0/965916475 >> 172.21.15.5:0/965916475 conn(0x563c268dfa30 msgr2=0x563c268e5790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:00.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.766+0000 7ff8addec740 1 -- 172.21.15.5:0/965916475 shutdown_connections 2024-09-12T03:03:00.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.766+0000 7ff8addec740 1 -- 172.21.15.5:0/965916475 wait complete. 2024-09-12T03:03:00.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.766+0000 7ff8addec740 20 client.25946 trim_cache size 0 max 16384 2024-09-12T03:03:00.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.788+0000 7ff8addec740 1 Processor -- start 2024-09-12T03:03:00.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.788+0000 7ff8addec740 1 -- start start 2024-09-12T03:03:00.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.789+0000 7ff8addec740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563c268e33a0 0x563c268fe990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:00.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.789+0000 7ff8addec740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c268e3cc0 0x563c2690a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:00.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.789+0000 7ff8addec740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563c268f2a90 0x563c2690a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:00.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.789+0000 7ff8addec740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff8a41ab280 con 0x563c268e3cc0 2024-09-12T03:03:00.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.789+0000 7ff8addec740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ff8a41acdc0 con 0x563c268e33a0 2024-09-12T03:03:00.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.789+0000 7ff8addec740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff8a41ab950 con 0x563c268f2a90 2024-09-12T03:03:00.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.789+0000 7ff8a8fae640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c268e3cc0 0x563c2690a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:00.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.789+0000 7ff8a8fae640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c268e3cc0 0x563c2690a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37266/0 (socket says 172.21.15.5:37266) 2024-09-12T03:03:00.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.789+0000 7ff8a8fae640 1 -- 172.21.15.5:0/1037227247 learned_addr learned my addr 172.21.15.5:0/1037227247 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:03:00.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.789+0000 7ff8a9fb0640 1 --2- 172.21.15.5:0/1037227247 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563c268f2a90 0x563c2690a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:00.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.789+0000 7ff8a97af640 1 --2- 172.21.15.5:0/1037227247 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563c268e33a0 0x563c268fe990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:00.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.789+0000 7ff8a8fae640 1 -- 172.21.15.5:0/1037227247 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563c268f2a90 msgr2=0x563c2690a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:00.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.789+0000 7ff8a8fae640 1 --2- 172.21.15.5:0/1037227247 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563c268f2a90 0x563c2690a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:00.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.789+0000 7ff8a8fae640 1 -- 172.21.15.5:0/1037227247 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563c268e33a0 msgr2=0x563c268fe990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:00.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.789+0000 7ff8a8fae640 1 --2- 172.21.15.5:0/1037227247 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563c268e33a0 0x563c268fe990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:00.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.789+0000 7ff8a8fae640 1 -- 172.21.15.5:0/1037227247 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c268f26a0 con 0x563c268e3cc0 2024-09-12T03:03:00.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.789+0000 7ff8a9fb0640 1 --2- 172.21.15.5:0/1037227247 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563c268f2a90 0x563c2690a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:03:00.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.789+0000 7ff8a97af640 1 --2- 172.21.15.5:0/1037227247 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563c268e33a0 0x563c268fe990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:03:00.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.790+0000 7ff8a8fae640 1 --2- 172.21.15.5:0/1037227247 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c268e3cc0 0x563c2690a130 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7ff898007590 tx=0x7ff898099dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca408281f9719f87 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:00.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.790+0000 7ff88effd640 1 -- 172.21.15.5:0/1037227247 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8980b4070 con 0x563c268e3cc0 2024-09-12T03:03:00.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.790+0000 7ff88effd640 1 -- 172.21.15.5:0/1037227247 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff898099bf0 con 0x563c268e3cc0 2024-09-12T03:03:00.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.790+0000 7ff88effd640 1 -- 172.21.15.5:0/1037227247 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff898060ef0 con 0x563c268e3cc0 2024-09-12T03:03:00.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.791+0000 7ff8addec740 1 -- 172.21.15.5:0/1037227247 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c268e3cc0 msgr2=0x563c2690a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:00.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.791+0000 7ff8addec740 1 --2- 172.21.15.5:0/1037227247 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c268e3cc0 0x563c2690a130 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7ff898007590 tx=0x7ff898099dd0 comp rx=0 tx=0).stop 2024-09-12T03:03:00.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.791+0000 7ff8addec740 1 -- 172.21.15.5:0/1037227247 shutdown_connections 2024-09-12T03:03:00.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.791+0000 7ff8addec740 1 --2- 172.21.15.5:0/1037227247 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c268e3cc0 0x563c2690a130 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:00.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.791+0000 7ff8addec740 1 --2- 172.21.15.5:0/1037227247 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563c268f2a90 0x563c2690a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:00.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.791+0000 7ff8addec740 1 --2- 172.21.15.5:0/1037227247 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563c268e33a0 0x563c268fe990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:00.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.791+0000 7ff8addec740 1 -- 172.21.15.5:0/1037227247 >> 172.21.15.5:0/1037227247 conn(0x563c268dfa30 msgr2=0x563c268edeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:00.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.791+0000 7ff8addec740 1 -- 172.21.15.5:0/1037227247 shutdown_connections 2024-09-12T03:03:00.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.791+0000 7ff8addec740 1 -- 172.21.15.5:0/1037227247 wait complete. 2024-09-12T03:03:00.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.791+0000 7ff8addec740 1 -- start start 2024-09-12T03:03:00.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.791+0000 7ff8addec740 1 Processor -- start 2024-09-12T03:03:00.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.792+0000 7ff8addec740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563c268e33a0 0x563c268fb0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:00.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.792+0000 7ff8addec740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563c268e3cc0 0x563c268f6410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:00.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.792+0000 7ff8addec740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c268f2a90 0x563c268f6950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:00.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.792+0000 7ff8a97af640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563c268e33a0 0x563c268fb0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:00.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.792+0000 7ff8a97af640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563c268e33a0 0x563c268fb0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.5:47744/0 (socket says 172.21.15.5:47744) 2024-09-12T03:03:00.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.792+0000 7ff8a8fae640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563c268e3cc0 0x563c268f6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:00.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.792+0000 7ff8a8fae640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563c268e3cc0 0x563c268f6410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.5:51398/0 (socket says 172.21.15.5:51398) 2024-09-12T03:03:00.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.792+0000 7ff8a8fae640 1 -- 172.21.15.5:0/3208508851 learned_addr learned my addr 172.21.15.5:0/3208508851 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:03:00.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.792+0000 7ff8addec740 1 -- 172.21.15.5:0/3208508851 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7ff8a41ac6f0 con 0x563c268f2a90 2024-09-12T03:03:00.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.792+0000 7ff8addec740 1 -- 172.21.15.5:0/3208508851 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7ff8a41ac020 con 0x563c268e3cc0 2024-09-12T03:03:00.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.792+0000 7ff8addec740 1 -- 172.21.15.5:0/3208508851 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7ff8a41a9260 con 0x563c268e33a0 2024-09-12T03:03:00.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.792+0000 7ff8a9fb0640 1 --2- 172.21.15.5:0/3208508851 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c268f2a90 0x563c268f6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:00.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.792+0000 7ff8addec740 10 client.0 _ll_register_callbacks cb 0x7ff89c031450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:03:00.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.793+0000 7ff8a97af640 1 -- 172.21.15.5:0/3208508851 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563c268e3cc0 msgr2=0x563c268f6410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:00.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.793+0000 7ff8a97af640 1 --2- 172.21.15.5:0/3208508851 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563c268e3cc0 0x563c268f6410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:00.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.793+0000 7ff8a97af640 1 -- 172.21.15.5:0/3208508851 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c268f2a90 msgr2=0x563c268f6950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:00.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.793+0000 7ff8a97af640 1 --2- 172.21.15.5:0/3208508851 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c268f2a90 0x563c268f6950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:00.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.793+0000 7ff8a97af640 1 -- 172.21.15.5:0/3208508851 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c268f6ff0 con 0x563c268e33a0 2024-09-12T03:03:00.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.793+0000 7ff8a9fb0640 1 --2- 172.21.15.5:0/3208508851 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c268f2a90 0x563c268f6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:03:00.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.793+0000 7ff8a97af640 1 --2- 172.21.15.5:0/3208508851 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563c268e33a0 0x563c268fb0e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x563c268f01c0 tx=0x7ff8a0013ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d8aa6f1e5824290d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:00.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.793+0000 7ff88cff9640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-09-12T03:03:00.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.793+0000 7ff88cff9640 1 -- 172.21.15.5:0/3208508851 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8a0003b20 con 0x563c268e33a0 2024-09-12T03:03:00.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.793+0000 7ff88cff9640 1 -- 172.21.15.5:0/3208508851 <== mon.1 v2:172.21.15.148:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7ff8a0003cc0 con 0x563c268e33a0 2024-09-12T03:03:00.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.793+0000 7ff8addec740 1 -- 172.21.15.5:0/3208508851 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff898099bf0 con 0x563c268e33a0 2024-09-12T03:03:00.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.793+0000 7ff88cff9640 1 -- 172.21.15.5:0/3208508851 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ff8a0005020 con 0x563c268e33a0 2024-09-12T03:03:00.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.793+0000 7ff8addec740 10 client.26041 fetch_fsmap learned FSMap version 31 2024-09-12T03:03:00.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.793+0000 7ff8addec740 1 -- 172.21.15.5:0/3208508851 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x563c2696cfe0 con 0x563c268e33a0 2024-09-12T03:03:00.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.794+0000 7ff88cff9640 1 -- 172.21.15.5:0/3208508851 <== mon.1 v2:172.21.15.148:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff898099bf0 con 0x563c268e33a0 2024-09-12T03:03:00.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.794+0000 7ff88cff9640 1 -- 172.21.15.5:0/3208508851 <== mon.1 v2:172.21.15.148:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7ff8a0004c50 con 0x563c268e33a0 2024-09-12T03:03:00.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.794+0000 7ff8addec740 10 client.26041 fetch_fsmap finished waiting for FSMap version 31 2024-09-12T03:03:00.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.794+0000 7ff8addec740 10 client.26041 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:03:00.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.794+0000 7ff8addec740 20 client.26041 populate_metadata read hostname 'smithi005' 2024-09-12T03:03:00.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.794+0000 7ff8addec740 1 --2- 172.21.15.5:0/3208508851 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x563c2696d700 0x563c2696fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:00.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.794+0000 7ff8addec740 4 client.26041 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T03:03:00.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.794+0000 7ff8addec740 1 -- 172.21.15.5:0/3208508851 --> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563c26850d60 con 0x563c2696d700 2024-09-12T03:03:00.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.794+0000 7ff8a8fae640 1 --2- 172.21.15.5:0/3208508851 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x563c2696d700 0x563c2696fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:00.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.795+0000 7ff8a8fae640 1 --2- 172.21.15.5:0/3208508851 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x563c2696d700 0x563c2696fb20 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70ffe0ba99f82190 server_cookie=84dd09018382b2c3 in_seq=0 out_seq=0 2024-09-12T03:03:00.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.795+0000 7ff88cff9640 10 client.26041 ms_handle_connect on v2:172.21.15.148:6838/2317600000 2024-09-12T03:03:00.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.795+0000 7ff88cff9640 1 -- 172.21.15.5:0/3208508851 <== mds.0 v2:172.21.15.148:6838/2317600000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x563c26850d60 con 0x563c2696d700 2024-09-12T03:03:00.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.795+0000 7ff88cff9640 10 client.26041 handle_command_reply: tid=0 2024-09-12T03:03:00.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.795+0000 7ff8addec740 1 client.26041 shutdown 2024-09-12T03:03:00.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.795+0000 7ff8addec740 1 -- 172.21.15.5:0/3208508851 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563c268e33a0 msgr2=0x563c268fb0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:00.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.795+0000 7ff8addec740 1 --2- 172.21.15.5:0/3208508851 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563c268e33a0 0x563c268fb0e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x563c268f01c0 tx=0x7ff8a0013ca0 comp rx=0 tx=0).stop 2024-09-12T03:03:00.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.795+0000 7ff8addec740 1 -- 172.21.15.5:0/3208508851 shutdown_connections 2024-09-12T03:03:00.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.795+0000 7ff8addec740 1 --2- 172.21.15.5:0/3208508851 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x563c2696d700 0x563c2696fb20 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:00.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.795+0000 7ff8addec740 1 --2- 172.21.15.5:0/3208508851 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563c268f2a90 0x563c268f6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:00.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.795+0000 7ff8addec740 1 --2- 172.21.15.5:0/3208508851 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x563c268e3cc0 0x563c268f6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:00.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.795+0000 7ff8addec740 1 --2- 172.21.15.5:0/3208508851 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x563c268e33a0 0x563c268fb0e0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:00.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.795+0000 7ff8addec740 1 -- 172.21.15.5:0/3208508851 >> 172.21.15.5:0/3208508851 conn(0x563c268dfa30 msgr2=0x563c268e4f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:00.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.795+0000 7ff88cff9640 0 client.26041 ms_handle_reset on v2:172.21.15.148:6838/2317600000 2024-09-12T03:03:00.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.795+0000 7ff8addec740 1 -- 172.21.15.5:0/3208508851 shutdown_connections 2024-09-12T03:03:00.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.795+0000 7ff8addec740 1 -- 172.21.15.5:0/3208508851 wait complete. 2024-09-12T03:03:00.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.796+0000 7ff8addec740 20 client.26041 trim_cache size 0 max 16384 2024-09-12T03:03:00.801 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-12T03:03:00.801 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-12T03:03:00.801 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-09-12T03:03:00.801 INFO:teuthology.orchestra.run.smithi005.stdout: "0af9ebc1-25c8-44f5-a071-2cb5e056e2ff": { 2024-09-12T03:03:00.801 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-09-12T03:03:00.801 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "0af9ebc1-25c8-44f5-a071-2cb5e056e2ff", 2024-09-12T03:03:00.801 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-09-12T03:03:00.801 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:03:00.801 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:03:00.802 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-12T03:03:00.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.796+0000 7ff8aba13640 1 -- 172.21.15.5:0/3821304148 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7ff8900a2b80 msgr2=0x7ff8900a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:00.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.796+0000 7ff8aba13640 1 --2- 172.21.15.5:0/3821304148 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7ff8900a2b80 0x7ff8900a5040 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7ff89c006010 tx=0x7ff89c02f500 comp rx=0 tx=0).stop 2024-09-12T03:03:00.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.796+0000 7ff8aba13640 1 -- 172.21.15.5:0/3821304148 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff8a4102790 msgr2=0x7ff8a4198880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:00.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.796+0000 7ff8aba13640 1 --2- 172.21.15.5:0/3821304148 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff8a4102790 0x7ff8a4198880 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff89800c920 tx=0x7ff89800cde0 comp rx=0 tx=0).stop 2024-09-12T03:03:00.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.796+0000 7ff8aba13640 1 -- 172.21.15.5:0/3821304148 shutdown_connections 2024-09-12T03:03:00.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.796+0000 7ff8aba13640 1 --2- 172.21.15.5:0/3821304148 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7ff8a4106f10 0x7ff8a419cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:00.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.796+0000 7ff8aba13640 1 --2- 172.21.15.5:0/3821304148 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7ff8a4102790 0x7ff8a4198880 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:00.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.796+0000 7ff8aba13640 1 --2- 172.21.15.5:0/3821304148 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7ff8900a2b80 0x7ff8900a5040 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:00.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.796+0000 7ff8aba13640 1 --2- 172.21.15.5:0/3821304148 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff8a4101de0 0x7ff8a4198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:00.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.796+0000 7ff8aba13640 1 -- 172.21.15.5:0/3821304148 >> 172.21.15.5:0/3821304148 conn(0x7ff8a40fd8c0 msgr2=0x7ff8a40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:00.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.797+0000 7ff8aba13640 1 -- 172.21.15.5:0/3821304148 shutdown_connections 2024-09-12T03:03:00.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:00.797+0000 7ff8aba13640 1 -- 172.21.15.5:0/3821304148 wait complete. 2024-09-12T03:03:01.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:03:00 smithi005 ceph-mon[31197]: scrub summary: active paths [/] 2024-09-12T03:03:01.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:03:00 smithi005 ceph-mon[31197]: scrub summary: idle+waiting paths [/] 2024-09-12T03:03:01.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:03:00 smithi005 ceph-mon[31197]: pgmap v478: 97 pgs: 97 active+clean; 131 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 2.8 MiB/s wr, 779 op/s 2024-09-12T03:03:01.165 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:03:00 smithi148 ceph-mon[35090]: scrub summary: active paths [/] 2024-09-12T03:03:01.165 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:03:00 smithi148 ceph-mon[35090]: scrub summary: idle+waiting paths [/] 2024-09-12T03:03:01.165 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:03:00 smithi148 ceph-mon[35090]: pgmap v478: 97 pgs: 97 active+clean; 131 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 2.8 MiB/s wr, 779 op/s 2024-09-12T03:03:01.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:03:00 smithi146 ceph-mon[35011]: scrub summary: active paths [/] 2024-09-12T03:03:01.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:03:00 smithi146 ceph-mon[35011]: scrub summary: idle+waiting paths [/] 2024-09-12T03:03:01.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:03:00 smithi146 ceph-mon[35011]: pgmap v478: 97 pgs: 97 active+clean; 131 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 2.8 MiB/s wr, 779 op/s 2024-09-12T03:03:01.508 INFO:tasks.cephfs.filesystem:scrub status for tag:0af9ebc1-25c8-44f5-a071-2cb5e056e2ff - {'path': '/', 'tag': '0af9ebc1-25c8-44f5-a071-2cb5e056e2ff', 'options': 'recursive,force'} 2024-09-12T03:03:02.130 INFO:tasks.workunit.client.0.smithi005.stdout:10000 mapread 0x33cc4e3 thru 0x33cec16 (0x2734 bytes) 2024-09-12T03:03:03.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:03:02 smithi005 ceph-mon[31197]: pgmap v479: 97 pgs: 97 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 2.5 MiB/s wr, 771 op/s 2024-09-12T03:03:03.165 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:03:02 smithi148 ceph-mon[35090]: pgmap v479: 97 pgs: 97 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 2.5 MiB/s wr, 771 op/s 2024-09-12T03:03:03.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:03:02 smithi146 ceph-mon[35011]: pgmap v479: 97 pgs: 97 active+clean; 127 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 2.5 MiB/s wr, 771 op/s 2024-09-12T03:03:05.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:03:04 smithi005 ceph-mon[31197]: pgmap v480: 97 pgs: 97 active+clean; 142 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 2.8 MiB/s wr, 937 op/s 2024-09-12T03:03:05.165 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:03:04 smithi148 ceph-mon[35090]: pgmap v480: 97 pgs: 97 active+clean; 142 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 2.8 MiB/s wr, 937 op/s 2024-09-12T03:03:05.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:03:04 smithi146 ceph-mon[35011]: pgmap v480: 97 pgs: 97 active+clean; 142 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 2.8 MiB/s wr, 937 op/s 2024-09-12T03:03:07.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:03:06 smithi005 ceph-mon[31197]: pgmap v481: 97 pgs: 97 active+clean; 146 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 22 MiB/s rd, 2.9 MiB/s wr, 1.11k op/s 2024-09-12T03:03:07.165 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:03:06 smithi148 ceph-mon[35090]: pgmap v481: 97 pgs: 97 active+clean; 146 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 22 MiB/s rd, 2.9 MiB/s wr, 1.11k op/s 2024-09-12T03:03:07.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:03:06 smithi146 ceph-mon[35011]: pgmap v481: 97 pgs: 97 active+clean; 146 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 22 MiB/s rd, 2.9 MiB/s wr, 1.11k op/s 2024-09-12T03:03:08.037 INFO:tasks.workunit.client.0.smithi005.stdout:truncating to largest ever: 0x63e5d2a 2024-09-12T03:03:09.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:03:08 smithi005 ceph-mon[31197]: pgmap v482: 97 pgs: 97 active+clean; 143 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 24 MiB/s rd, 2.8 MiB/s wr, 1.11k op/s 2024-09-12T03:03:09.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:03:08 smithi005 ceph-mon[31197]: scrub summary: idle 2024-09-12T03:03:09.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:03:08 smithi005 ceph-mon[31197]: scrub summary: idle 2024-09-12T03:03:09.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:03:08 smithi005 ceph-mon[31197]: scrub summary: idle 2024-09-12T03:03:09.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:03:08 smithi005 ceph-mon[31197]: scrub summary: idle 2024-09-12T03:03:09.165 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:03:08 smithi148 ceph-mon[35090]: pgmap v482: 97 pgs: 97 active+clean; 143 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 24 MiB/s rd, 2.8 MiB/s wr, 1.11k op/s 2024-09-12T03:03:09.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:03:08 smithi148 ceph-mon[35090]: scrub summary: idle 2024-09-12T03:03:09.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:03:08 smithi148 ceph-mon[35090]: scrub summary: idle 2024-09-12T03:03:09.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:03:08 smithi148 ceph-mon[35090]: scrub summary: idle 2024-09-12T03:03:09.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:03:08 smithi148 ceph-mon[35090]: scrub summary: idle 2024-09-12T03:03:09.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:03:08 smithi146 ceph-mon[35011]: pgmap v482: 97 pgs: 97 active+clean; 143 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 24 MiB/s rd, 2.8 MiB/s wr, 1.11k op/s 2024-09-12T03:03:09.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:03:08 smithi146 ceph-mon[35011]: scrub summary: idle 2024-09-12T03:03:09.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:03:08 smithi146 ceph-mon[35011]: scrub summary: idle 2024-09-12T03:03:09.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:03:08 smithi146 ceph-mon[35011]: scrub summary: idle 2024-09-12T03:03:09.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:03:08 smithi146 ceph-mon[35011]: scrub summary: idle 2024-09-12T03:03:10.367 INFO:tasks.workunit.client.0.smithi005.stdout:truncating to largest ever: 0x63e9b52 2024-09-12T03:03:11.008 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:03:10 smithi146 ceph-mon[35011]: pgmap v483: 97 pgs: 97 active+clean; 145 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 24 MiB/s rd, 3.0 MiB/s wr, 1.15k op/s 2024-09-12T03:03:11.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:03:10 smithi005 ceph-mon[31197]: pgmap v483: 97 pgs: 97 active+clean; 145 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 24 MiB/s rd, 3.0 MiB/s wr, 1.15k op/s 2024-09-12T03:03:11.165 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:03:10 smithi148 ceph-mon[35090]: pgmap v483: 97 pgs: 97 active+clean; 145 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 24 MiB/s rd, 3.0 MiB/s wr, 1.15k op/s 2024-09-12T03:03:12.112 INFO:tasks.workunit.client.0.smithi005.stdout:20000 read 0xa6cbf1 thru 0xa6e358 (0x1768 bytes) 2024-09-12T03:03:13.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:03:12 smithi005 ceph-mon[31197]: pgmap v484: 97 pgs: 97 active+clean; 147 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 26 MiB/s rd, 2.9 MiB/s wr, 1.08k op/s 2024-09-12T03:03:13.165 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:03:12 smithi148 ceph-mon[35090]: pgmap v484: 97 pgs: 97 active+clean; 147 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 26 MiB/s rd, 2.9 MiB/s wr, 1.08k op/s 2024-09-12T03:03:13.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:03:12 smithi146 ceph-mon[35011]: pgmap v484: 97 pgs: 97 active+clean; 147 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 26 MiB/s rd, 2.9 MiB/s wr, 1.08k op/s 2024-09-12T03:03:13.767 INFO:tasks.workunit.client.0.smithi005.stdout:copying to largest ever: 0x63ed6b9 2024-09-12T03:03:15.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:03:14 smithi005 ceph-mon[31197]: pgmap v485: 97 pgs: 97 active+clean; 143 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 27 MiB/s rd, 2.9 MiB/s wr, 1.15k op/s 2024-09-12T03:03:15.165 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:03:14 smithi148 ceph-mon[35090]: pgmap v485: 97 pgs: 97 active+clean; 143 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 27 MiB/s rd, 2.9 MiB/s wr, 1.15k op/s 2024-09-12T03:03:15.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:03:14 smithi146 ceph-mon[35011]: pgmap v485: 97 pgs: 97 active+clean; 143 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 27 MiB/s rd, 2.9 MiB/s wr, 1.15k op/s 2024-09-12T03:03:17.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:03:16 smithi005 ceph-mon[31197]: pgmap v486: 97 pgs: 97 active+clean; 144 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 24 MiB/s rd, 3.1 MiB/s wr, 1.18k op/s 2024-09-12T03:03:17.165 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:03:16 smithi148 ceph-mon[35090]: pgmap v486: 97 pgs: 97 active+clean; 144 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 24 MiB/s rd, 3.1 MiB/s wr, 1.18k op/s 2024-09-12T03:03:17.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:03:16 smithi146 ceph-mon[35011]: pgmap v486: 97 pgs: 97 active+clean; 144 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 24 MiB/s rd, 3.1 MiB/s wr, 1.18k op/s 2024-09-12T03:03:19.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:03:18 smithi005 ceph-mon[31197]: pgmap v487: 97 pgs: 97 active+clean; 137 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 23 MiB/s rd, 2.9 MiB/s wr, 1.10k op/s 2024-09-12T03:03:19.165 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:03:18 smithi148 ceph-mon[35090]: pgmap v487: 97 pgs: 97 active+clean; 137 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 23 MiB/s rd, 2.9 MiB/s wr, 1.10k op/s 2024-09-12T03:03:19.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:03:18 smithi146 ceph-mon[35011]: pgmap v487: 97 pgs: 97 active+clean; 137 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 23 MiB/s rd, 2.9 MiB/s wr, 1.10k op/s 2024-09-12T03:03:21.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:03:20 smithi005 ceph-mon[31197]: pgmap v488: 97 pgs: 97 active+clean; 132 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 21 MiB/s rd, 3.0 MiB/s wr, 1.12k op/s 2024-09-12T03:03:21.165 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:03:20 smithi148 ceph-mon[35090]: pgmap v488: 97 pgs: 97 active+clean; 132 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 21 MiB/s rd, 3.0 MiB/s wr, 1.12k op/s 2024-09-12T03:03:21.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:03:20 smithi146 ceph-mon[35011]: pgmap v488: 97 pgs: 97 active+clean; 132 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 21 MiB/s rd, 3.0 MiB/s wr, 1.12k op/s 2024-09-12T03:03:23.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:03:22 smithi005 ceph-mon[31197]: pgmap v489: 97 pgs: 97 active+clean; 131 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 2.8 MiB/s wr, 1.06k op/s 2024-09-12T03:03:23.165 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:03:22 smithi148 ceph-mon[35090]: pgmap v489: 97 pgs: 97 active+clean; 131 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 2.8 MiB/s wr, 1.06k op/s 2024-09-12T03:03:23.198 INFO:tasks.workunit.client.0.smithi005.stdout:copying to largest ever: 0x63ef9ee 2024-09-12T03:03:23.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:03:22 smithi146 ceph-mon[35011]: pgmap v489: 97 pgs: 97 active+clean; 131 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 2.8 MiB/s wr, 1.06k op/s 2024-09-12T03:03:25.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:03:24 smithi005 ceph-mon[31197]: pgmap v490: 97 pgs: 97 active+clean; 139 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 2.9 MiB/s wr, 1.11k op/s 2024-09-12T03:03:25.165 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:03:24 smithi148 ceph-mon[35090]: pgmap v490: 97 pgs: 97 active+clean; 139 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 2.9 MiB/s wr, 1.11k op/s 2024-09-12T03:03:25.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:03:24 smithi146 ceph-mon[35011]: pgmap v490: 97 pgs: 97 active+clean; 139 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 2.9 MiB/s wr, 1.11k op/s 2024-09-12T03:03:25.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:03:25 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:03:25.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:03:25 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:03:25.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:03:25 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:03:25.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:03:25 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T03:03:26.165 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:03:25 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:03:26.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:03:25 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:03:26.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:03:25 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:03:26.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:03:25 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T03:03:26.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:03:25 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:03:26.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:03:25 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:03:26.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:03:25 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:03:26.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:03:25 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T03:03:27.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:03:26 smithi005 ceph-mon[31197]: pgmap v491: 97 pgs: 97 active+clean; 148 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 3.0 MiB/s wr, 1.12k op/s 2024-09-12T03:03:27.165 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:03:26 smithi148 ceph-mon[35090]: pgmap v491: 97 pgs: 97 active+clean; 148 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 3.0 MiB/s wr, 1.12k op/s 2024-09-12T03:03:27.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:03:26 smithi146 ceph-mon[35011]: pgmap v491: 97 pgs: 97 active+clean; 148 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 3.0 MiB/s wr, 1.12k op/s 2024-09-12T03:03:29.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:03:28 smithi005 ceph-mon[31197]: pgmap v492: 97 pgs: 97 active+clean; 146 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 2.7 MiB/s wr, 1.04k op/s 2024-09-12T03:03:29.165 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:03:28 smithi148 ceph-mon[35090]: pgmap v492: 97 pgs: 97 active+clean; 146 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 2.7 MiB/s wr, 1.04k op/s 2024-09-12T03:03:29.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:03:28 smithi146 ceph-mon[35011]: pgmap v492: 97 pgs: 97 active+clean; 146 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 2.7 MiB/s wr, 1.04k op/s 2024-09-12T03:03:31.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:03:30 smithi005 ceph-mon[31197]: pgmap v493: 97 pgs: 97 active+clean; 142 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 2.7 MiB/s wr, 1.06k op/s 2024-09-12T03:03:31.165 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:03:30 smithi148 ceph-mon[35090]: pgmap v493: 97 pgs: 97 active+clean; 142 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 2.7 MiB/s wr, 1.06k op/s 2024-09-12T03:03:31.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:03:30 smithi146 ceph-mon[35011]: pgmap v493: 97 pgs: 97 active+clean; 142 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 2.7 MiB/s wr, 1.06k op/s 2024-09-12T03:03:31.509 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T03:03:31.847 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:03:32.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.358+0000 7fa49b020640 1 -- 172.21.15.5:0/727681233 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa494106f10 msgr2=0x7fa49410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:32.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.358+0000 7fa49b020640 1 --2- 172.21.15.5:0/727681233 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa494106f10 0x7fa49410ea90 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fa484009ca0 tx=0x7fa48402a8c0 comp rx=0 tx=0).stop 2024-09-12T03:03:32.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.358+0000 7fa49b020640 1 -- 172.21.15.5:0/727681233 shutdown_connections 2024-09-12T03:03:32.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.358+0000 7fa49b020640 1 --2- 172.21.15.5:0/727681233 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa494106f10 0x7fa49410ea90 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:32.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.358+0000 7fa49b020640 1 --2- 172.21.15.5:0/727681233 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa494102790 0x7fa4941067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:32.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.358+0000 7fa49b020640 1 --2- 172.21.15.5:0/727681233 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa494101de0 0x7fa4941021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:32.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.358+0000 7fa49b020640 1 -- 172.21.15.5:0/727681233 >> 172.21.15.5:0/727681233 conn(0x7fa4940fd8c0 msgr2=0x7fa4940ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:32.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.358+0000 7fa49b020640 1 -- 172.21.15.5:0/727681233 shutdown_connections 2024-09-12T03:03:32.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.358+0000 7fa49b020640 1 -- 172.21.15.5:0/727681233 wait complete. 2024-09-12T03:03:32.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.359+0000 7fa49b020640 1 Processor -- start 2024-09-12T03:03:32.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.359+0000 7fa49b020640 1 -- start start 2024-09-12T03:03:32.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.359+0000 7fa49b020640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa494101de0 0x7fa49419a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:32.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.359+0000 7fa49b020640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa494102790 0x7fa49419ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:32.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.359+0000 7fa49b020640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa494106f10 0x7fa49419eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:32.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.359+0000 7fa49b020640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa494114170 con 0x7fa494102790 2024-09-12T03:03:32.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.359+0000 7fa49b020640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fa494113ff0 con 0x7fa494106f10 2024-09-12T03:03:32.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.359+0000 7fa49b020640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa4941142f0 con 0x7fa494101de0 2024-09-12T03:03:32.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.359+0000 7fa493fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa494102790 0x7fa49419ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:32.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.359+0000 7fa493fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa494102790 0x7fa49419ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33832/0 (socket says 172.21.15.5:33832) 2024-09-12T03:03:32.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.359+0000 7fa4995bd640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa494106f10 0x7fa49419eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:32.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.359+0000 7fa493fff640 1 -- 172.21.15.5:0/2013844754 learned_addr learned my addr 172.21.15.5:0/2013844754 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:03:32.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.359+0000 7fa498dbc640 1 --2- 172.21.15.5:0/2013844754 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa494101de0 0x7fa49419a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:32.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.360+0000 7fa493fff640 1 -- 172.21.15.5:0/2013844754 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa494101de0 msgr2=0x7fa49419a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:32.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.360+0000 7fa493fff640 1 --2- 172.21.15.5:0/2013844754 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa494101de0 0x7fa49419a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:32.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.360+0000 7fa493fff640 1 -- 172.21.15.5:0/2013844754 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa494106f10 msgr2=0x7fa49419eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:32.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.360+0000 7fa493fff640 1 --2- 172.21.15.5:0/2013844754 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa494106f10 0x7fa49419eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:32.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.360+0000 7fa493fff640 1 -- 172.21.15.5:0/2013844754 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa49419f5a0 con 0x7fa494102790 2024-09-12T03:03:32.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.360+0000 7fa493fff640 1 --2- 172.21.15.5:0/2013844754 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa494102790 0x7fa49419ab30 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fa48800ea60 tx=0x7fa48800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:32.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.360+0000 7fa491ffb640 1 -- 172.21.15.5:0/2013844754 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa48800cde0 con 0x7fa494102790 2024-09-12T03:03:32.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.360+0000 7fa491ffb640 1 -- 172.21.15.5:0/2013844754 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4880035f0 con 0x7fa494102790 2024-09-12T03:03:32.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.360+0000 7fa49b020640 1 -- 172.21.15.5:0/2013844754 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa49419f890 con 0x7fa494102790 2024-09-12T03:03:32.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.360+0000 7fa491ffb640 1 -- 172.21.15.5:0/2013844754 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa488010670 con 0x7fa494102790 2024-09-12T03:03:32.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.360+0000 7fa49b020640 1 -- 172.21.15.5:0/2013844754 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4941a7170 con 0x7fa494102790 2024-09-12T03:03:32.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.363+0000 7fa49d3f9740 1 Processor -- start 2024-09-12T03:03:32.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.363+0000 7fa491ffb640 1 -- 172.21.15.5:0/2013844754 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fa4880037a0 con 0x7fa494102790 2024-09-12T03:03:32.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.363+0000 7fa49d3f9740 1 -- start start 2024-09-12T03:03:32.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.364+0000 7fa49d3f9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2817e34a0 0x55a2817e3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:32.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.364+0000 7fa49d3f9740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a2817e3dc0 0x55a2817ee140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:32.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.364+0000 7fa49d3f9740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a2817ee680 0x55a2817f0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:32.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.364+0000 7fa49d3f9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a2817492e0 con 0x55a2817e34a0 2024-09-12T03:03:32.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.364+0000 7fa49d3f9740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55a2816f0ce0 con 0x55a2817e3dc0 2024-09-12T03:03:32.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.364+0000 7fa49d3f9740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55a2817452a0 con 0x55a2817ee680 2024-09-12T03:03:32.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.364+0000 7fa4995bd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2817e34a0 0x55a2817e3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:32.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.364+0000 7fa4995bd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2817e34a0 0x55a2817e3880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33834/0 (socket says 172.21.15.5:33834) 2024-09-12T03:03:32.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.364+0000 7fa4995bd640 1 -- 172.21.15.5:0/4128650824 learned_addr learned my addr 172.21.15.5:0/4128650824 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:03:32.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.364+0000 7fa498dbc640 1 --2- 172.21.15.5:0/4128650824 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a2817e3dc0 0x55a2817ee140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:32.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.364+0000 7fa491ffb640 1 --2- 172.21.15.5:0/2013844754 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fa47c0a2c00 0x7fa47c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:32.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.365+0000 7fa498dbc640 1 -- 172.21.15.5:0/4128650824 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a2817ee680 msgr2=0x55a2817f0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:32.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.365+0000 7fa493fff640 1 --2- 172.21.15.5:0/4128650824 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a2817ee680 0x55a2817f0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:32.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.365+0000 7fa498dbc640 1 --2- 172.21.15.5:0/4128650824 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a2817ee680 0x55a2817f0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:32.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.365+0000 7fa491ffb640 1 -- 172.21.15.5:0/2013844754 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fa4880cf260 con 0x7fa494102790 2024-09-12T03:03:32.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.365+0000 7fa498dbc640 1 -- 172.21.15.5:0/4128650824 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2817e34a0 msgr2=0x55a2817e3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:32.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.365+0000 7fa498dbc640 1 --2- 172.21.15.5:0/4128650824 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2817e34a0 0x55a2817e3880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:32.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.365+0000 7fa498dbc640 1 -- 172.21.15.5:0/4128650824 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2816d5440 con 0x55a2817e3dc0 2024-09-12T03:03:32.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.365+0000 7fa4995bd640 1 --2- 172.21.15.5:0/4128650824 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2817e34a0 0x55a2817e3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:03:32.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.365+0000 7fa493fff640 1 --2- 172.21.15.5:0/4128650824 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a2817ee680 0x55a2817f0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:03:32.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.365+0000 7fa4995bd640 1 --2- 172.21.15.5:0/2013844754 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fa47c0a2c00 0x7fa47c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:32.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.365+0000 7fa498dbc640 1 --2- 172.21.15.5:0/4128650824 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a2817e3dc0 0x55a2817ee140 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa48c009c20 tx=0x7fa48c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a060e508f1383fa7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:32.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.365+0000 7fa4995bd640 1 --2- 172.21.15.5:0/2013844754 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fa47c0a2c00 0x7fa47c0a50c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fa484005fe0 tx=0x7fa48402a8c0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:32.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.366+0000 7fa47affd640 1 -- 172.21.15.5:0/4128650824 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa48c01e070 con 0x55a2817e3dc0 2024-09-12T03:03:32.371 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.366+0000 7fa47affd640 1 -- 172.21.15.5:0/4128650824 <== mon.2 v2:172.21.15.146:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa48c009dd0 con 0x55a2817e3dc0 2024-09-12T03:03:32.372 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.366+0000 7fa47affd640 1 -- 172.21.15.5:0/4128650824 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa48c018870 con 0x55a2817e3dc0 2024-09-12T03:03:32.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.366+0000 7fa49d3f9740 1 -- 172.21.15.5:0/4128650824 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a2817e3dc0 msgr2=0x55a2817ee140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:32.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.366+0000 7fa49d3f9740 1 --2- 172.21.15.5:0/4128650824 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a2817e3dc0 0x55a2817ee140 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa48c009c20 tx=0x7fa48c011810 comp rx=0 tx=0).stop 2024-09-12T03:03:32.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.367+0000 7fa49d3f9740 1 -- 172.21.15.5:0/4128650824 shutdown_connections 2024-09-12T03:03:32.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.367+0000 7fa49d3f9740 1 --2- 172.21.15.5:0/4128650824 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a2817ee680 0x55a2817f0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:32.377 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.367+0000 7fa49d3f9740 1 --2- 172.21.15.5:0/4128650824 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a2817e3dc0 0x55a2817ee140 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:32.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.367+0000 7fa49d3f9740 1 --2- 172.21.15.5:0/4128650824 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2817e34a0 0x55a2817e3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:32.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.367+0000 7fa49d3f9740 1 -- 172.21.15.5:0/4128650824 >> 172.21.15.5:0/4128650824 conn(0x55a2817dfa30 msgr2=0x55a2817dfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:32.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.367+0000 7fa49d3f9740 1 -- 172.21.15.5:0/4128650824 shutdown_connections 2024-09-12T03:03:32.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.367+0000 7fa49d3f9740 1 -- 172.21.15.5:0/4128650824 wait complete. 2024-09-12T03:03:32.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.367+0000 7fa49d3f9740 1 -- start start 2024-09-12T03:03:32.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.367+0000 7fa49d3f9740 1 Processor -- start 2024-09-12T03:03:32.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.368+0000 7fa49d3f9740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a2817e34a0 0x55a2817edf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:32.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.368+0000 7fa49d3f9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2817e3dc0 0x55a2817eb1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:32.378 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.368+0000 7fa49d3f9740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a2817ee680 0x55a2817eb720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:32.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.368+0000 7fa49d3f9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55a2817f1250 con 0x55a2817e3dc0 2024-09-12T03:03:32.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.368+0000 7fa49d3f9740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55a281704050 con 0x55a2817ee680 2024-09-12T03:03:32.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.368+0000 7fa49d3f9740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55a2817f13d0 con 0x55a2817e34a0 2024-09-12T03:03:32.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.368+0000 7fa49d3f9740 10 client.0 _ll_register_callbacks cb 0x55a281744160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:03:32.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.368+0000 7fa498dbc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a2817e34a0 0x55a2817edf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:32.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.368+0000 7fa498dbc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a2817e34a0 0x55a2817edf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.5:43708/0 (socket says 172.21.15.5:43708) 2024-09-12T03:03:32.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.368+0000 7fa498dbc640 1 -- 172.21.15.5:0/3790801684 learned_addr learned my addr 172.21.15.5:0/3790801684 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:03:32.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.368+0000 7fa493fff640 1 --2- 172.21.15.5:0/3790801684 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2817e3dc0 0x55a2817eb1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:32.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.368+0000 7fa4995bd640 1 --2- 172.21.15.5:0/3790801684 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a2817ee680 0x55a2817eb720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:32.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.368+0000 7fa498dbc640 1 -- 172.21.15.5:0/3790801684 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a2817ee680 msgr2=0x55a2817eb720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:32.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.368+0000 7fa498dbc640 1 --2- 172.21.15.5:0/3790801684 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a2817ee680 0x55a2817eb720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:32.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.368+0000 7fa498dbc640 1 -- 172.21.15.5:0/3790801684 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2817e3dc0 msgr2=0x55a2817eb1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:32.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.368+0000 7fa498dbc640 1 --2- 172.21.15.5:0/3790801684 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2817e3dc0 0x55a2817eb1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:32.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.368+0000 7fa498dbc640 1 -- 172.21.15.5:0/3790801684 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2814ba3c0 con 0x55a2817e34a0 2024-09-12T03:03:32.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.368+0000 7fa473fff640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-09-12T03:03:32.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.368+0000 7fa498dbc640 1 --2- 172.21.15.5:0/3790801684 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a2817e34a0 0x55a2817edf30 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa48c011d10 tx=0x7fa48c019e10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c3f46a66b73fb08 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:32.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.369+0000 7fa473fff640 1 -- 172.21.15.5:0/3790801684 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa48c028020 con 0x55a2817e34a0 2024-09-12T03:03:32.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.369+0000 7fa473fff640 1 -- 172.21.15.5:0/3790801684 <== mon.1 v2:172.21.15.148:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fa48c005050 con 0x55a2817e34a0 2024-09-12T03:03:32.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.369+0000 7fa49d3f9740 1 -- 172.21.15.5:0/3790801684 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa48c009dd0 con 0x55a2817e34a0 2024-09-12T03:03:32.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.369+0000 7fa473fff640 1 -- 172.21.15.5:0/3790801684 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fa48c01e040 con 0x55a2817e34a0 2024-09-12T03:03:32.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.369+0000 7fa49d3f9740 10 client.26050 fetch_fsmap learned FSMap version 31 2024-09-12T03:03:32.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.369+0000 7fa49d3f9740 1 -- 172.21.15.5:0/3790801684 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55a2817fcae0 con 0x55a2817e34a0 2024-09-12T03:03:32.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.370+0000 7fa473fff640 1 -- 172.21.15.5:0/3790801684 <== mon.1 v2:172.21.15.148:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa48c009dd0 con 0x55a2817e34a0 2024-09-12T03:03:32.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.370+0000 7fa473fff640 1 -- 172.21.15.5:0/3790801684 <== mon.1 v2:172.21.15.148:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fa48c0194a0 con 0x55a2817e34a0 2024-09-12T03:03:32.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.370+0000 7fa49d3f9740 10 client.26050 fetch_fsmap finished waiting for FSMap version 31 2024-09-12T03:03:32.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.370+0000 7fa49d3f9740 10 client.26050 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:03:32.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.370+0000 7fa49d3f9740 20 client.26050 populate_metadata read hostname 'smithi005' 2024-09-12T03:03:32.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.370+0000 7fa49d3f9740 1 --2- 172.21.15.5:0/3790801684 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55a2817fdfa0 0x55a281800390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:32.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.370+0000 7fa49d3f9740 4 client.26050 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:03:32.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.370+0000 7fa49d3f9740 1 -- 172.21.15.5:0/3790801684 --> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a281756850 con 0x55a2817fdfa0 2024-09-12T03:03:32.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.370+0000 7fa493fff640 1 --2- 172.21.15.5:0/3790801684 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55a2817fdfa0 0x55a281800390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:32.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.371+0000 7fa493fff640 1 --2- 172.21.15.5:0/3790801684 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55a2817fdfa0 0x55a281800390 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7658bf2f9fa968d server_cookie=58636f6beec3da7b in_seq=0 out_seq=0 2024-09-12T03:03:32.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.371+0000 7fa473fff640 10 client.26050 ms_handle_connect on v2:172.21.15.148:6838/2317600000 2024-09-12T03:03:32.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.371+0000 7fa473fff640 1 -- 172.21.15.5:0/3790801684 <== mds.0 v2:172.21.15.148:6838/2317600000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a281756850 con 0x55a2817fdfa0 2024-09-12T03:03:32.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.371+0000 7fa473fff640 10 client.26050 handle_command_reply: tid=0 2024-09-12T03:03:32.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.371+0000 7fa49d3f9740 1 client.26050 shutdown 2024-09-12T03:03:32.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.371+0000 7fa49d3f9740 1 -- 172.21.15.5:0/3790801684 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a2817e34a0 msgr2=0x55a2817edf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:32.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.371+0000 7fa49d3f9740 1 --2- 172.21.15.5:0/3790801684 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a2817e34a0 0x55a2817edf30 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fa48c011d10 tx=0x7fa48c019e10 comp rx=0 tx=0).stop 2024-09-12T03:03:32.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.372+0000 7fa49d3f9740 1 -- 172.21.15.5:0/3790801684 shutdown_connections 2024-09-12T03:03:32.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.372+0000 7fa49d3f9740 1 --2- 172.21.15.5:0/3790801684 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55a2817fdfa0 0x55a281800390 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:32.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.372+0000 7fa473fff640 0 client.26050 ms_handle_reset on v2:172.21.15.148:6838/2317600000 2024-09-12T03:03:32.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.372+0000 7fa49d3f9740 1 --2- 172.21.15.5:0/3790801684 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2817e3dc0 0x55a2817eb1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:32.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.372+0000 7fa49d3f9740 1 --2- 172.21.15.5:0/3790801684 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a2817ee680 0x55a2817eb720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:32.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.372+0000 7fa49d3f9740 1 --2- 172.21.15.5:0/3790801684 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a2817e34a0 0x55a2817edf30 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:32.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.372+0000 7fa49d3f9740 1 -- 172.21.15.5:0/3790801684 >> 172.21.15.5:0/3790801684 conn(0x55a2817dfa30 msgr2=0x55a2817e5760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:32.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.372+0000 7fa49d3f9740 1 -- 172.21.15.5:0/3790801684 shutdown_connections 2024-09-12T03:03:32.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.372+0000 7fa49d3f9740 1 -- 172.21.15.5:0/3790801684 wait complete. 2024-09-12T03:03:32.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.372+0000 7fa49d3f9740 20 client.26050 trim_cache size 0 max 16384 2024-09-12T03:03:32.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.393+0000 7fa49d3f9740 1 Processor -- start 2024-09-12T03:03:32.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.394+0000 7fa49d3f9740 1 -- start start 2024-09-12T03:03:32.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.394+0000 7fa49d3f9740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a2817e34a0 0x55a2817fa4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:32.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.394+0000 7fa49d3f9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2817e3dc0 0x55a281805ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:32.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.394+0000 7fa49d3f9740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a2817ee680 0x55a281806010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:32.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.394+0000 7fa49d3f9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa4941af800 con 0x55a2817e3dc0 2024-09-12T03:03:32.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.394+0000 7fa49d3f9740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fa4941b1340 con 0x55a2817e34a0 2024-09-12T03:03:32.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.394+0000 7fa49d3f9740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa4941afed0 con 0x55a2817ee680 2024-09-12T03:03:32.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.394+0000 7fa493fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2817e3dc0 0x55a281805ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:32.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.394+0000 7fa498dbc640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a2817e34a0 0x55a2817fa4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:32.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.394+0000 7fa493fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2817e3dc0 0x55a281805ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33854/0 (socket says 172.21.15.5:33854) 2024-09-12T03:03:32.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.394+0000 7fa493fff640 1 -- 172.21.15.5:0/1656680467 learned_addr learned my addr 172.21.15.5:0/1656680467 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:03:32.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.394+0000 7fa498dbc640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a2817e34a0 0x55a2817fa4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.5:59036/0 (socket says 172.21.15.5:59036) 2024-09-12T03:03:32.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.394+0000 7fa4995bd640 1 --2- 172.21.15.5:0/1656680467 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a2817ee680 0x55a281806010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:32.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.394+0000 7fa493fff640 1 -- 172.21.15.5:0/1656680467 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a2817ee680 msgr2=0x55a281806010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:32.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.394+0000 7fa493fff640 1 --2- 172.21.15.5:0/1656680467 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a2817ee680 0x55a281806010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:32.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.394+0000 7fa493fff640 1 -- 172.21.15.5:0/1656680467 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a2817e34a0 msgr2=0x55a2817fa4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:32.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.394+0000 7fa493fff640 1 --2- 172.21.15.5:0/1656680467 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a2817e34a0 0x55a2817fa4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:32.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.394+0000 7fa493fff640 1 -- 172.21.15.5:0/1656680467 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2817df860 con 0x55a2817e3dc0 2024-09-12T03:03:32.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.394+0000 7fa498dbc640 1 --2- 172.21.15.5:0/1656680467 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a2817e34a0 0x55a2817fa4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:03:32.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.395+0000 7fa493fff640 1 --2- 172.21.15.5:0/1656680467 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2817e3dc0 0x55a281805ad0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fa48800ea30 tx=0x7fa488014d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3477b7ceedaac7f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:32.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.395+0000 7fa47a7fc640 1 -- 172.21.15.5:0/1656680467 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4880b4050 con 0x55a2817e3dc0 2024-09-12T03:03:32.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.395+0000 7fa47a7fc640 1 -- 172.21.15.5:0/1656680467 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa48809fe80 con 0x55a2817e3dc0 2024-09-12T03:03:32.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.395+0000 7fa47a7fc640 1 -- 172.21.15.5:0/1656680467 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa488098140 con 0x55a2817e3dc0 2024-09-12T03:03:32.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.396+0000 7fa49d3f9740 1 -- 172.21.15.5:0/1656680467 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2817e3dc0 msgr2=0x55a281805ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:32.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.396+0000 7fa49d3f9740 1 --2- 172.21.15.5:0/1656680467 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2817e3dc0 0x55a281805ad0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fa48800ea30 tx=0x7fa488014d50 comp rx=0 tx=0).stop 2024-09-12T03:03:32.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.396+0000 7fa49d3f9740 1 -- 172.21.15.5:0/1656680467 shutdown_connections 2024-09-12T03:03:32.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.396+0000 7fa49d3f9740 1 --2- 172.21.15.5:0/1656680467 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2817e3dc0 0x55a281805ad0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:32.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.396+0000 7fa49d3f9740 1 --2- 172.21.15.5:0/1656680467 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a2817ee680 0x55a281806010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:32.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.396+0000 7fa49d3f9740 1 --2- 172.21.15.5:0/1656680467 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a2817e34a0 0x55a2817fa4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:32.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.396+0000 7fa49d3f9740 1 -- 172.21.15.5:0/1656680467 >> 172.21.15.5:0/1656680467 conn(0x55a2817dfa30 msgr2=0x55a2817deb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:32.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.396+0000 7fa49d3f9740 1 -- 172.21.15.5:0/1656680467 shutdown_connections 2024-09-12T03:03:32.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.396+0000 7fa49d3f9740 1 -- 172.21.15.5:0/1656680467 wait complete. 2024-09-12T03:03:32.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.396+0000 7fa49d3f9740 1 -- start start 2024-09-12T03:03:32.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.396+0000 7fa49d3f9740 1 Processor -- start 2024-09-12T03:03:32.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.397+0000 7fa49d3f9740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a2817e34a0 0x55a2817f6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:32.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.397+0000 7fa49d3f9740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2817e3dc0 0x55a2817f2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:32.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.397+0000 7fa49d3f9740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a2817ee680 0x55a2817f29a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:32.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.397+0000 7fa49d3f9740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa4941b0c70 con 0x55a2817e3dc0 2024-09-12T03:03:32.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.397+0000 7fa49d3f9740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fa4941b05a0 con 0x55a2817e34a0 2024-09-12T03:03:32.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.397+0000 7fa49d3f9740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa4941ad7e0 con 0x55a2817ee680 2024-09-12T03:03:32.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.397+0000 7fa49d3f9740 10 client.0 _ll_register_callbacks cb 0x7fa48c01c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:03:32.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.397+0000 7fa493fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2817e3dc0 0x55a2817f2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:32.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.397+0000 7fa493fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2817e3dc0 0x55a2817f2210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33860/0 (socket says 172.21.15.5:33860) 2024-09-12T03:03:32.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.397+0000 7fa493fff640 1 -- 172.21.15.5:0/1034419152 learned_addr learned my addr 172.21.15.5:0/1034419152 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:03:32.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.397+0000 7fa4995bd640 1 --2- 172.21.15.5:0/1034419152 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a2817ee680 0x55a2817f29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:32.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.397+0000 7fa498dbc640 1 --2- 172.21.15.5:0/1034419152 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a2817e34a0 0x55a2817f6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:32.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.397+0000 7fa493fff640 1 -- 172.21.15.5:0/1034419152 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a2817ee680 msgr2=0x55a2817f29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:32.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.397+0000 7fa493fff640 1 --2- 172.21.15.5:0/1034419152 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a2817ee680 0x55a2817f29a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:32.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.397+0000 7fa493fff640 1 -- 172.21.15.5:0/1034419152 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a2817e34a0 msgr2=0x55a2817f6ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:32.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.397+0000 7fa493fff640 1 --2- 172.21.15.5:0/1034419152 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a2817e34a0 0x55a2817f6ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:32.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.397+0000 7fa493fff640 1 -- 172.21.15.5:0/1034419152 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a281867aa0 con 0x55a2817e3dc0 2024-09-12T03:03:32.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.397+0000 7fa4995bd640 1 --2- 172.21.15.5:0/1034419152 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a2817ee680 0x55a2817f29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:03:32.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.397+0000 7fa493fff640 1 --2- 172.21.15.5:0/1034419152 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2817e3dc0 0x55a2817f2210 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fa48809b7d0 tx=0x7fa488099dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6bb9ffa351ceb2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:32.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.397+0000 7fa473fff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T03:03:32.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.398+0000 7fa473fff640 1 -- 172.21.15.5:0/1034419152 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa48807e030 con 0x55a2817e3dc0 2024-09-12T03:03:32.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.398+0000 7fa473fff640 1 -- 172.21.15.5:0/1034419152 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fa488003b30 con 0x55a2817e3dc0 2024-09-12T03:03:32.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.398+0000 7fa49d3f9740 1 -- 172.21.15.5:0/1034419152 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa48809fe80 con 0x55a2817e3dc0 2024-09-12T03:03:32.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.398+0000 7fa473fff640 1 -- 172.21.15.5:0/1034419152 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fa4880b4020 con 0x55a2817e3dc0 2024-09-12T03:03:32.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.398+0000 7fa49d3f9740 10 client.16581 fetch_fsmap learned FSMap version 31 2024-09-12T03:03:32.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.398+0000 7fa49d3f9740 1 -- 172.21.15.5:0/1034419152 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55a281868c30 con 0x55a2817e3dc0 2024-09-12T03:03:32.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.398+0000 7fa473fff640 1 -- 172.21.15.5:0/1034419152 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4880a4740 con 0x55a2817e3dc0 2024-09-12T03:03:32.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.398+0000 7fa473fff640 1 -- 172.21.15.5:0/1034419152 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fa4880b6ce0 con 0x55a2817e3dc0 2024-09-12T03:03:32.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.398+0000 7fa49d3f9740 10 client.16581 fetch_fsmap finished waiting for FSMap version 31 2024-09-12T03:03:32.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.398+0000 7fa49d3f9740 10 client.16581 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:03:32.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.398+0000 7fa49d3f9740 20 client.16581 populate_metadata read hostname 'smithi005' 2024-09-12T03:03:32.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.398+0000 7fa49d3f9740 1 --2- 172.21.15.5:0/1034419152 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55a2817fdfa0 0x55a28186b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:32.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.398+0000 7fa49d3f9740 4 client.16581 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T03:03:32.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.398+0000 7fa49d3f9740 1 -- 172.21.15.5:0/1034419152 --> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7fa48809fe80 con 0x55a2817fdfa0 2024-09-12T03:03:32.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.398+0000 7fa498dbc640 1 --2- 172.21.15.5:0/1034419152 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55a2817fdfa0 0x55a28186b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:32.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.399+0000 7fa473fff640 10 client.16581 ms_handle_connect on v2:172.21.15.148:6838/2317600000 2024-09-12T03:03:32.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.399+0000 7fa498dbc640 1 --2- 172.21.15.5:0/1034419152 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55a2817fdfa0 0x55a28186b310 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=612babdeef9435bb server_cookie=6592c5527767f1c8 in_seq=0 out_seq=0 2024-09-12T03:03:32.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.399+0000 7fa473fff640 1 -- 172.21.15.5:0/1034419152 <== mds.0 v2:172.21.15.148:6838/2317600000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7fa48809fe80 con 0x55a2817fdfa0 2024-09-12T03:03:32.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.399+0000 7fa473fff640 10 client.16581 handle_command_reply: tid=0 2024-09-12T03:03:32.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.399+0000 7fa49d3f9740 1 client.16581 shutdown 2024-09-12T03:03:32.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.399+0000 7fa49d3f9740 1 -- 172.21.15.5:0/1034419152 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2817e3dc0 msgr2=0x55a2817f2210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:32.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.399+0000 7fa49d3f9740 1 --2- 172.21.15.5:0/1034419152 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2817e3dc0 0x55a2817f2210 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fa48809b7d0 tx=0x7fa488099dd0 comp rx=0 tx=0).stop 2024-09-12T03:03:32.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.400+0000 7fa49d3f9740 1 -- 172.21.15.5:0/1034419152 shutdown_connections 2024-09-12T03:03:32.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.400+0000 7fa49d3f9740 1 --2- 172.21.15.5:0/1034419152 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55a2817fdfa0 0x55a28186b310 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:32.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.400+0000 7fa49d3f9740 1 --2- 172.21.15.5:0/1034419152 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2817e3dc0 0x55a2817f2210 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:32.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.400+0000 7fa49d3f9740 1 --2- 172.21.15.5:0/1034419152 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55a2817ee680 0x55a2817f29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:32.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.400+0000 7fa49d3f9740 1 --2- 172.21.15.5:0/1034419152 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55a2817e34a0 0x55a2817f6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:32.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.400+0000 7fa49d3f9740 1 -- 172.21.15.5:0/1034419152 >> 172.21.15.5:0/1034419152 conn(0x55a2817dfa30 msgr2=0x55a2817e4ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:32.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.400+0000 7fa473fff640 0 client.16581 ms_handle_reset on v2:172.21.15.148:6838/2317600000 2024-09-12T03:03:32.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.400+0000 7fa49d3f9740 1 -- 172.21.15.5:0/1034419152 shutdown_connections 2024-09-12T03:03:32.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.400+0000 7fa49d3f9740 1 -- 172.21.15.5:0/1034419152 wait complete. 2024-09-12T03:03:32.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.400+0000 7fa49d3f9740 20 client.16581 trim_cache size 0 max 16384 2024-09-12T03:03:32.407 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-12T03:03:32.407 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-09-12T03:03:32.407 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-09-12T03:03:32.407 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-12T03:03:32.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.401+0000 7fa49b020640 1 -- 172.21.15.5:0/2013844754 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fa47c0a2c00 msgr2=0x7fa47c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:32.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.402+0000 7fa49b020640 1 --2- 172.21.15.5:0/2013844754 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fa47c0a2c00 0x7fa47c0a50c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fa484005fe0 tx=0x7fa48402a8c0 comp rx=0 tx=0).stop 2024-09-12T03:03:32.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.402+0000 7fa49b020640 1 -- 172.21.15.5:0/2013844754 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa494102790 msgr2=0x7fa49419ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:32.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.402+0000 7fa49b020640 1 --2- 172.21.15.5:0/2013844754 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa494102790 0x7fa49419ab30 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fa48800ea60 tx=0x7fa48800ef20 comp rx=0 tx=0).stop 2024-09-12T03:03:32.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.404+0000 7fa49b020640 1 -- 172.21.15.5:0/2013844754 shutdown_connections 2024-09-12T03:03:32.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.404+0000 7fa49b020640 1 --2- 172.21.15.5:0/2013844754 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fa47c0a2c00 0x7fa47c0a50c0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:32.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.404+0000 7fa49b020640 1 --2- 172.21.15.5:0/2013844754 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa494102790 0x7fa49419ab30 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:32.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.404+0000 7fa49b020640 1 --2- 172.21.15.5:0/2013844754 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa494106f10 0x7fa49419eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:32.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.404+0000 7fa49b020640 1 --2- 172.21.15.5:0/2013844754 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa494101de0 0x7fa49419a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:32.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.404+0000 7fa49b020640 1 -- 172.21.15.5:0/2013844754 >> 172.21.15.5:0/2013844754 conn(0x7fa4940fd8c0 msgr2=0x7fa4940fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:32.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.406+0000 7fa49b020640 1 -- 172.21.15.5:0/2013844754 shutdown_connections 2024-09-12T03:03:32.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:32.406+0000 7fa49b020640 1 -- 172.21.15.5:0/2013844754 wait complete. 2024-09-12T03:03:33.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:03:32 smithi005 ceph-mon[31197]: pgmap v494: 97 pgs: 97 active+clean; 141 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 2.5 MiB/s wr, 991 op/s 2024-09-12T03:03:33.165 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:03:32 smithi148 ceph-mon[35090]: pgmap v494: 97 pgs: 97 active+clean; 141 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 2.5 MiB/s wr, 991 op/s 2024-09-12T03:03:33.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:03:32 smithi146 ceph-mon[35011]: pgmap v494: 97 pgs: 97 active+clean; 141 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 2.5 MiB/s wr, 991 op/s 2024-09-12T03:03:33.396 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T03:03:33.397 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:03:33.713 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:03:34.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:34.230+0000 7f4207e2a640 1 -- 172.21.15.5:0/2262903263 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4200102d90 msgr2=0x7f4200103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:34.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:34.230+0000 7f4207e2a640 1 --2- 172.21.15.5:0/2262903263 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4200102d90 0x7f4200103170 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f41fc009900 tx=0x7f41fc02a480 comp rx=0 tx=0).stop 2024-09-12T03:03:34.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:34.230+0000 7f4207e2a640 1 -- 172.21.15.5:0/2262903263 shutdown_connections 2024-09-12T03:03:34.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:34.230+0000 7f4207e2a640 1 --2- 172.21.15.5:0/2262903263 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f420010c170 0x7f420010e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:34.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:34.230+0000 7f4207e2a640 1 --2- 172.21.15.5:0/2262903263 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f42001036b0 0x7f420010bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:34.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:34.230+0000 7f4207e2a640 1 --2- 172.21.15.5:0/2262903263 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4200102d90 0x7f4200103170 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:34.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:34.230+0000 7f4207e2a640 1 -- 172.21.15.5:0/2262903263 >> 172.21.15.5:0/2262903263 conn(0x7f42000fc790 msgr2=0x7f42000febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:34.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:34.230+0000 7f4207e2a640 1 -- 172.21.15.5:0/2262903263 shutdown_connections 2024-09-12T03:03:34.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:34.230+0000 7f4207e2a640 1 -- 172.21.15.5:0/2262903263 wait complete. 2024-09-12T03:03:34.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:34.231+0000 7f4207e2a640 1 Processor -- start 2024-09-12T03:03:34.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:34.231+0000 7f4207e2a640 1 -- start start 2024-09-12T03:03:34.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:34.231+0000 7f4207e2a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4200102d90 0x7f42001009f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:34.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:34.231+0000 7f4207e2a640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f42001036b0 0x7f4200100f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:34.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:34.231+0000 7f4207e2a640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f420010c170 0x7f4200101470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:34.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:34.231+0000 7f4207e2a640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4200111f50 con 0x7f4200102d90 2024-09-12T03:03:34.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:34.231+0000 7f4207e2a640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f4200111dd0 con 0x7f42001036b0 2024-09-12T03:03:34.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:34.231+0000 7f4207e2a640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f42001120d0 con 0x7f420010c170 2024-09-12T03:03:34.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:34.231+0000 7f4205bc6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4200102d90 0x7f42001009f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:34.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:34.231+0000 7f4205bc6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4200102d90 0x7f42001009f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49440/0 (socket says 172.21.15.5:49440) 2024-09-12T03:03:34.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:34.231+0000 7f4205bc6640 1 -- 172.21.15.5:0/1865070321 learned_addr learned my addr 172.21.15.5:0/1865070321 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:03:34.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:34.231+0000 7f42053c5640 1 --2- 172.21.15.5:0/1865070321 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f42001036b0 0x7f4200100f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:34.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:34.232+0000 7f42063c7640 1 --2- 172.21.15.5:0/1865070321 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f420010c170 0x7f4200101470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:34.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:34.232+0000 7f4205bc6640 1 -- 172.21.15.5:0/1865070321 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f420010c170 msgr2=0x7f4200101470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:34.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:34.232+0000 7f4205bc6640 1 --2- 172.21.15.5:0/1865070321 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f420010c170 0x7f4200101470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:34.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:34.232+0000 7f4205bc6640 1 -- 172.21.15.5:0/1865070321 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f42001036b0 msgr2=0x7f4200100f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:34.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:34.232+0000 7f4205bc6640 1 --2- 172.21.15.5:0/1865070321 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f42001036b0 0x7f4200100f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:34.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:34.232+0000 7f4205bc6640 1 -- 172.21.15.5:0/1865070321 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42001a6930 con 0x7f4200102d90 2024-09-12T03:03:34.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:34.232+0000 7f42053c5640 1 --2- 172.21.15.5:0/1865070321 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f42001036b0 0x7f4200100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:03:34.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:34.232+0000 7f4205bc6640 1 --2- 172.21.15.5:0/1865070321 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4200102d90 0x7f42001009f0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f41fc006010 tx=0x7f41fc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:34.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:34.232+0000 7f41f6ffd640 1 -- 172.21.15.5:0/1865070321 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41fc03f030 con 0x7f4200102d90 2024-09-12T03:03:34.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:34.232+0000 7f4207e2a640 1 -- 172.21.15.5:0/1865070321 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42001a6bc0 con 0x7f4200102d90 2024-09-12T03:03:34.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:34.232+0000 7f41f6ffd640 1 -- 172.21.15.5:0/1865070321 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41fc003ce0 con 0x7f4200102d90 2024-09-12T03:03:34.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:34.232+0000 7f41f6ffd640 1 -- 172.21.15.5:0/1865070321 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41fc035070 con 0x7f4200102d90 2024-09-12T03:03:34.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:34.232+0000 7f4207e2a640 1 -- 172.21.15.5:0/1865070321 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42001a70a0 con 0x7f4200102d90 2024-09-12T03:03:34.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:34.233+0000 7f4207e2a640 1 -- 172.21.15.5:0/1865070321 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4200103170 con 0x7f4200102d90 2024-09-12T03:03:34.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:34.235+0000 7f41f6ffd640 1 -- 172.21.15.5:0/1865070321 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f41fc042030 con 0x7f4200102d90 2024-09-12T03:03:34.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:34.236+0000 7f41f6ffd640 1 --2- 172.21.15.5:0/1865070321 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f41ec0a2b80 0x7f41ec0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:34.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:34.236+0000 7f42053c5640 1 --2- 172.21.15.5:0/1865070321 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f41ec0a2b80 0x7f41ec0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:34.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:34.236+0000 7f41f6ffd640 1 -- 172.21.15.5:0/1865070321 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f41fc0ef720 con 0x7f4200102d90 2024-09-12T03:03:34.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:34.237+0000 7f42053c5640 1 --2- 172.21.15.5:0/1865070321 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f41ec0a2b80 0x7f41ec0a5040 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f4200102480 tx=0x7f41f0005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:34.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:34.240+0000 7f41f6ffd640 1 -- 172.21.15.5:0/1865070321 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f41fc0bc100 con 0x7f4200102d90 2024-09-12T03:03:34.517 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:34.517+0000 7f4207e2a640 1 -- 172.21.15.5:0/1865070321 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4200107210 con 0x7f4200102d90 2024-09-12T03:03:34.518 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:34.518+0000 7f41f6ffd640 1 -- 172.21.15.5:0/1865070321 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11966 (secure 0 0 0) 0x7f41fc0c0400 con 0x7f4200102d90 2024-09-12T03:03:34.519 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-12T03:03:34.519 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":31,"btime":"2024-09-12T03:02:23:528668+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/4135232521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":4135232521},{"type":"v1","addr":"172.21.15.5:6835","nonce":4135232521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/4290221940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":4290221940},{"type":"v1","addr":"172.21.15.5:6837","nonce":4290221940}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":31,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T02:56:16.595562+0000","modified":"2024-09-12T03:02:22.771568+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24502,"mds_1":24487,"mds_2":24481,"mds_3":24479,"mds_4":24458},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/26447341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":26447341},{"type":"v1","addr":"172.21.15.5:6839","nonce":26447341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2393443295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2393443295},{"type":"v1","addr":"172.21.15.5:6841","nonce":2393443295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6833/3686970137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3686970137},{"type":"v1","addr":"172.21.15.146:6833","nonce":3686970137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6833/568512209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":568512209},{"type":"v1","addr":"172.21.15.148:6833","nonce":568512209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6835/2540740420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2540740420},{"type":"v1","addr":"172.21.15.146:6835","nonce":2540740420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.146:6837/1673370937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":1673370937},{"type":"v1","addr":"172.21.15.146:6837","nonce":1673370937}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":3,"incarnation":22,"state":"up:active","state_seq":6,"addr":"172.21.15.146:6839/3036767228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":3036767228},{"type":"v1","addr":"172.21.15.146:6839","nonce":3036767228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":2,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.148:6835/4195086427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":4195086427},{"type":"v1","addr":"172.21.15.148:6835","nonce":4195086427}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.148:6837/2606873922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2606873922},{"type":"v1","addr":"172.21.15.148:6837","nonce":2606873922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6839/2317600000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":2317600000},{"type":"v1","addr":"172.21.15.148:6839","nonce":2317600000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24502,"qdb_cluster":[24458,24479,24481,24487,24502]},"id":1}]} 2024-09-12T03:03:34.519 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 31 2024-09-12T03:03:34.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:34.521+0000 7f4207e2a640 1 -- 172.21.15.5:0/1865070321 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f41ec0a2b80 msgr2=0x7f41ec0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:34.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:34.521+0000 7f4207e2a640 1 --2- 172.21.15.5:0/1865070321 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f41ec0a2b80 0x7f41ec0a5040 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f4200102480 tx=0x7f41f0005f90 comp rx=0 tx=0).stop 2024-09-12T03:03:34.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:34.521+0000 7f4207e2a640 1 -- 172.21.15.5:0/1865070321 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4200102d90 msgr2=0x7f42001009f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:34.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:34.521+0000 7f4207e2a640 1 --2- 172.21.15.5:0/1865070321 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4200102d90 0x7f42001009f0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f41fc006010 tx=0x7f41fc0042f0 comp rx=0 tx=0).stop 2024-09-12T03:03:34.521 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:34.521+0000 7f4207e2a640 1 -- 172.21.15.5:0/1865070321 shutdown_connections 2024-09-12T03:03:34.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:34.521+0000 7f4207e2a640 1 --2- 172.21.15.5:0/1865070321 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f420010c170 0x7f4200101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:34.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:34.521+0000 7f4207e2a640 1 --2- 172.21.15.5:0/1865070321 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f42001036b0 0x7f4200100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:34.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:34.521+0000 7f4207e2a640 1 --2- 172.21.15.5:0/1865070321 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f41ec0a2b80 0x7f41ec0a5040 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:34.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:34.521+0000 7f4207e2a640 1 --2- 172.21.15.5:0/1865070321 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4200102d90 0x7f42001009f0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:34.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:34.521+0000 7f4207e2a640 1 -- 172.21.15.5:0/1865070321 >> 172.21.15.5:0/1865070321 conn(0x7f42000fc790 msgr2=0x7f42000fdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:34.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:34.521+0000 7f4207e2a640 1 -- 172.21.15.5:0/1865070321 shutdown_connections 2024-09-12T03:03:34.522 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:34.521+0000 7f4207e2a640 1 -- 172.21.15.5:0/1865070321 wait complete. 2024-09-12T03:03:34.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:03:34 smithi005 ceph-mon[31197]: pgmap v495: 97 pgs: 97 active+clean; 141 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 23 MiB/s rd, 2.9 MiB/s wr, 1.11k op/s 2024-09-12T03:03:34.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:03:34 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/1865070321' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:03:35.165 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:03:34 smithi148 ceph-mon[35090]: pgmap v495: 97 pgs: 97 active+clean; 141 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 23 MiB/s rd, 2.9 MiB/s wr, 1.11k op/s 2024-09-12T03:03:35.165 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:03:34 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/1865070321' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:03:35.197 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-12T03:03:35.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:03:34 smithi146 ceph-mon[35011]: pgmap v495: 97 pgs: 97 active+clean; 141 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 23 MiB/s rd, 2.9 MiB/s wr, 1.11k op/s 2024-09-12T03:03:35.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:03:34 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/1865070321' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:03:35.523 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:03:36.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.591+0000 7f7d83235640 1 -- 172.21.15.5:0/3283186550 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d7c10c2f0 msgr2=0x7f7d7c10e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:36.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.591+0000 7f7d83235640 1 --2- 172.21.15.5:0/3283186550 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d7c10c2f0 0x7f7d7c10e790 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f7d74009ca0 tx=0x7f7d7402a8c0 comp rx=0 tx=0).stop 2024-09-12T03:03:36.615 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.591+0000 7f7d83235640 1 -- 172.21.15.5:0/3283186550 shutdown_connections 2024-09-12T03:03:36.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.591+0000 7f7d83235640 1 --2- 172.21.15.5:0/3283186550 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d7c10c2f0 0x7f7d7c10e790 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:36.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.591+0000 7f7d83235640 1 --2- 172.21.15.5:0/3283186550 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7d7c1037a0 0x7f7d7c10bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:36.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.591+0000 7f7d83235640 1 --2- 172.21.15.5:0/3283186550 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d7c102df0 0x7f7d7c1031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:36.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.591+0000 7f7d83235640 1 -- 172.21.15.5:0/3283186550 >> 172.21.15.5:0/3283186550 conn(0x7f7d7c0fc7d0 msgr2=0x7f7d7c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:36.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.592+0000 7f7d83235640 1 -- 172.21.15.5:0/3283186550 shutdown_connections 2024-09-12T03:03:36.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.592+0000 7f7d83235640 1 -- 172.21.15.5:0/3283186550 wait complete. 2024-09-12T03:03:36.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.592+0000 7f7d83235640 1 Processor -- start 2024-09-12T03:03:36.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.592+0000 7f7d83235640 1 -- start start 2024-09-12T03:03:36.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.593+0000 7f7d83235640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7d7c102df0 0x7f7d7c100a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:36.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.593+0000 7f7d83235640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d7c1037a0 0x7f7d7c100f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:36.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.593+0000 7f7d83235640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d7c10c2f0 0x7f7d7c1014d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:36.616 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.593+0000 7f7d83235640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7d7c112110 con 0x7f7d7c1037a0 2024-09-12T03:03:36.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.593+0000 7f7d83235640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f7d7c111f90 con 0x7f7d7c102df0 2024-09-12T03:03:36.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.593+0000 7f7d83235640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f7d7c112290 con 0x7f7d7c10c2f0 2024-09-12T03:03:36.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.593+0000 7f7d7bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d7c1037a0 0x7f7d7c100f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:36.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.593+0000 7f7d80fd1640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7d7c102df0 0x7f7d7c100a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:36.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.593+0000 7f7d7bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d7c1037a0 0x7f7d7c100f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49452/0 (socket says 172.21.15.5:49452) 2024-09-12T03:03:36.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.593+0000 7f7d7bfff640 1 -- 172.21.15.5:0/3971694590 learned_addr learned my addr 172.21.15.5:0/3971694590 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:03:36.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.593+0000 7f7d817d2640 1 --2- 172.21.15.5:0/3971694590 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d7c10c2f0 0x7f7d7c1014d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:36.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.593+0000 7f7d80fd1640 1 -- 172.21.15.5:0/3971694590 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d7c10c2f0 msgr2=0x7f7d7c1014d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:36.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.593+0000 7f7d80fd1640 1 --2- 172.21.15.5:0/3971694590 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d7c10c2f0 0x7f7d7c1014d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:36.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.593+0000 7f7d80fd1640 1 -- 172.21.15.5:0/3971694590 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d7c1037a0 msgr2=0x7f7d7c100f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:36.617 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.593+0000 7f7d80fd1640 1 --2- 172.21.15.5:0/3971694590 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d7c1037a0 0x7f7d7c100f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:36.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.593+0000 7f7d80fd1640 1 -- 172.21.15.5:0/3971694590 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d7c1a6a70 con 0x7f7d7c102df0 2024-09-12T03:03:36.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.593+0000 7f7d817d2640 1 --2- 172.21.15.5:0/3971694590 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d7c10c2f0 0x7f7d7c1014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:03:36.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.594+0000 7f7d80fd1640 1 --2- 172.21.15.5:0/3971694590 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7d7c102df0 0x7f7d7c100a50 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7d7000e9f0 tx=0x7f7d7000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:36.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.594+0000 7f7d79ffb640 1 -- 172.21.15.5:0/3971694590 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d7000ccc0 con 0x7f7d7c102df0 2024-09-12T03:03:36.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.594+0000 7f7d79ffb640 1 -- 172.21.15.5:0/3971694590 <== mon.2 v2:172.21.15.146:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d70005020 con 0x7f7d7c102df0 2024-09-12T03:03:36.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.594+0000 7f7d79ffb640 1 -- 172.21.15.5:0/3971694590 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d700049e0 con 0x7f7d7c102df0 2024-09-12T03:03:36.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.594+0000 7f7d83235640 1 -- 172.21.15.5:0/3971694590 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d7c1a6d60 con 0x7f7d7c102df0 2024-09-12T03:03:36.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.595+0000 7f7d83235640 1 -- 172.21.15.5:0/3971694590 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d7c1a71f0 con 0x7f7d7c102df0 2024-09-12T03:03:36.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.598+0000 7f7d79ffb640 1 -- 172.21.15.5:0/3971694590 <== mon.2 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f7d70005560 con 0x7f7d7c102df0 2024-09-12T03:03:36.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.598+0000 7f7d8560e740 1 Processor -- start 2024-09-12T03:03:36.618 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.598+0000 7f7d8560e740 1 -- start start 2024-09-12T03:03:36.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.599+0000 7f7d8560e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557425f2fb40 0x557425f2e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:36.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.599+0000 7f7d8560e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557425f2e5d0 0x557425f2ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:36.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.599+0000 7f7d8560e740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557425f39370 0x557425f3b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:36.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.599+0000 7f7d8560e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557425e942f0 con 0x557425f2fb40 2024-09-12T03:03:36.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.599+0000 7f7d8560e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x557425e3bcf0 con 0x557425f2e5d0 2024-09-12T03:03:36.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.599+0000 7f7d8560e740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x557425e902b0 con 0x557425f39370 2024-09-12T03:03:36.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.599+0000 7f7d80fd1640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557425f2e5d0 0x557425f2ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:36.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.599+0000 7f7d817d2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557425f2fb40 0x557425f2e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:36.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.599+0000 7f7d80fd1640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557425f2e5d0 0x557425f2ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.5:50032/0 (socket says 172.21.15.5:50032) 2024-09-12T03:03:36.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.599+0000 7f7d80fd1640 1 -- 172.21.15.5:0/2871601356 learned_addr learned my addr 172.21.15.5:0/2871601356 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:03:36.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.599+0000 7f7d7bfff640 1 --2- 172.21.15.5:0/2871601356 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557425f39370 0x557425f3b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:36.619 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.599+0000 7f7d79ffb640 1 --2- 172.21.15.5:0/3971694590 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f7d6c0a2b80 0x7f7d6c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:36.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.599+0000 7f7d80fd1640 1 -- 172.21.15.5:0/2871601356 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557425f39370 msgr2=0x557425f3b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:36.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.599+0000 7f7d80fd1640 1 --2- 172.21.15.5:0/2871601356 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557425f39370 0x557425f3b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:36.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.599+0000 7f7d80fd1640 1 -- 172.21.15.5:0/2871601356 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557425f2fb40 msgr2=0x557425f2e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:36.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.599+0000 7f7d80fd1640 1 --2- 172.21.15.5:0/2871601356 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557425f2fb40 0x557425f2e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:36.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.599+0000 7f7d80fd1640 1 -- 172.21.15.5:0/2871601356 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557425e203a0 con 0x557425f2e5d0 2024-09-12T03:03:36.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.599+0000 7f7d7bfff640 1 --2- 172.21.15.5:0/2871601356 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557425f39370 0x557425f3b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:03:36.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.599+0000 7f7d817d2640 1 --2- 172.21.15.5:0/3971694590 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f7d6c0a2b80 0x7f7d6c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:36.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.599+0000 7f7d79ffb640 1 -- 172.21.15.5:0/3971694590 <== mon.2 v2:172.21.15.146:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f7d700cef90 con 0x7f7d7c102df0 2024-09-12T03:03:36.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.600+0000 7f7d80fd1640 1 --2- 172.21.15.5:0/2871601356 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557425f2e5d0 0x557425f2ea10 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7d700c3600 tx=0x7f7d700c6430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe143e9acde74090 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:36.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.600+0000 7f7d66ffd640 1 -- 172.21.15.5:0/2871601356 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d7001c070 con 0x557425f2e5d0 2024-09-12T03:03:36.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.600+0000 7f7d817d2640 1 --2- 172.21.15.5:0/3971694590 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f7d6c0a2b80 0x7f7d6c0a5040 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f7d74004af0 tx=0x7f7d7402a8c0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:36.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.600+0000 7f7d66ffd640 1 -- 172.21.15.5:0/2871601356 <== mon.2 v2:172.21.15.146:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d700c3630 con 0x557425f2e5d0 2024-09-12T03:03:36.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.600+0000 7f7d66ffd640 1 -- 172.21.15.5:0/2871601356 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d700d2bf0 con 0x557425f2e5d0 2024-09-12T03:03:36.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.601+0000 7f7d8560e740 1 -- 172.21.15.5:0/2871601356 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557425f2e5d0 msgr2=0x557425f2ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:36.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.601+0000 7f7d8560e740 1 --2- 172.21.15.5:0/2871601356 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557425f2e5d0 0x557425f2ea10 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7d700c3600 tx=0x7f7d700c6430 comp rx=0 tx=0).stop 2024-09-12T03:03:36.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.601+0000 7f7d8560e740 1 -- 172.21.15.5:0/2871601356 shutdown_connections 2024-09-12T03:03:36.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.601+0000 7f7d8560e740 1 --2- 172.21.15.5:0/2871601356 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557425f39370 0x557425f3b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:36.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.601+0000 7f7d8560e740 1 --2- 172.21.15.5:0/2871601356 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557425f2e5d0 0x557425f2ea10 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:36.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.601+0000 7f7d8560e740 1 --2- 172.21.15.5:0/2871601356 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557425f2fb40 0x557425f2e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:36.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.601+0000 7f7d8560e740 1 -- 172.21.15.5:0/2871601356 >> 172.21.15.5:0/2871601356 conn(0x557425ef7310 msgr2=0x557425f2ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:36.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.601+0000 7f7d8560e740 1 -- 172.21.15.5:0/2871601356 shutdown_connections 2024-09-12T03:03:36.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.601+0000 7f7d8560e740 1 -- 172.21.15.5:0/2871601356 wait complete. 2024-09-12T03:03:36.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.601+0000 7f7d8560e740 1 -- start start 2024-09-12T03:03:36.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.601+0000 7f7d8560e740 1 Processor -- start 2024-09-12T03:03:36.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.602+0000 7f7d8560e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557425f2e5d0 0x557425f38db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:36.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.602+0000 7f7d8560e740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557425f2fb40 0x557425f36060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:36.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.602+0000 7f7d8560e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557425f39370 0x557425f367f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:36.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.602+0000 7f7d8560e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x557425f3bfb0 con 0x557425f2e5d0 2024-09-12T03:03:36.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.602+0000 7f7d8560e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x557425e4f060 con 0x557425f39370 2024-09-12T03:03:36.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.602+0000 7f7d8560e740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x557425f3c130 con 0x557425f2fb40 2024-09-12T03:03:36.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.602+0000 7f7d8560e740 10 client.0 _ll_register_callbacks cb 0x557425ba28c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:03:36.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.602+0000 7f7d80fd1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557425f2e5d0 0x557425f38db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:36.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.602+0000 7f7d80fd1640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557425f2e5d0 0x557425f38db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49484/0 (socket says 172.21.15.5:49484) 2024-09-12T03:03:36.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.602+0000 7f7d80fd1640 1 -- 172.21.15.5:0/2705874688 learned_addr learned my addr 172.21.15.5:0/2705874688 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:03:36.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.602+0000 7f7d7bfff640 1 --2- 172.21.15.5:0/2705874688 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557425f2fb40 0x557425f36060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:36.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.602+0000 7f7d817d2640 1 --2- 172.21.15.5:0/2705874688 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557425f39370 0x557425f367f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:36.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.602+0000 7f7d80fd1640 1 -- 172.21.15.5:0/2705874688 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557425f2fb40 msgr2=0x557425f36060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:36.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.602+0000 7f7d80fd1640 1 --2- 172.21.15.5:0/2705874688 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557425f2fb40 0x557425f36060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:36.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.602+0000 7f7d80fd1640 1 -- 172.21.15.5:0/2705874688 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557425f39370 msgr2=0x557425f367f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:36.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.602+0000 7f7d80fd1640 1 --2- 172.21.15.5:0/2705874688 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557425f39370 0x557425f367f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:36.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.602+0000 7f7d80fd1640 1 -- 172.21.15.5:0/2705874688 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557425c053c0 con 0x557425f2e5d0 2024-09-12T03:03:36.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.603+0000 7f7d80fd1640 1 --2- 172.21.15.5:0/2705874688 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557425f2e5d0 0x557425f38db0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f7d70098270 tx=0x7f7d700d2690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba8aa34c5da62626 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:36.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.603+0000 7f7d5bfff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T03:03:36.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.603+0000 7f7d5bfff640 1 -- 172.21.15.5:0/2705874688 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d7001c070 con 0x557425f2e5d0 2024-09-12T03:03:36.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.603+0000 7f7d5bfff640 1 -- 172.21.15.5:0/2705874688 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f7d70097f10 con 0x557425f2e5d0 2024-09-12T03:03:36.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.603+0000 7f7d8560e740 1 -- 172.21.15.5:0/2705874688 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x557425ea1860 con 0x557425f2e5d0 2024-09-12T03:03:36.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.603+0000 7f7d5bfff640 1 -- 172.21.15.5:0/2705874688 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f7d7009b3e0 con 0x557425f2e5d0 2024-09-12T03:03:36.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.603+0000 7f7d8560e740 10 client.16599 fetch_fsmap learned FSMap version 31 2024-09-12T03:03:36.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.603+0000 7f7d8560e740 1 -- 172.21.15.5:0/2705874688 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x557425f47ad0 con 0x557425f2e5d0 2024-09-12T03:03:36.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.603+0000 7f7d5bfff640 1 -- 172.21.15.5:0/2705874688 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x557425ea1860 con 0x557425f2e5d0 2024-09-12T03:03:36.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.603+0000 7f7d5bfff640 1 -- 172.21.15.5:0/2705874688 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f7d700975f0 con 0x557425f2e5d0 2024-09-12T03:03:36.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.604+0000 7f7d8560e740 10 client.16599 fetch_fsmap finished waiting for FSMap version 31 2024-09-12T03:03:36.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.604+0000 7f7d8560e740 10 client.16599 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-12T03:03:36.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.604+0000 7f7d8560e740 20 client.16599 populate_metadata read hostname 'smithi005' 2024-09-12T03:03:36.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.604+0000 7f7d8560e740 1 --2- 172.21.15.5:0/2705874688 >> [v2:172.21.15.146:6836/1673370937,v1:172.21.15.146:6837/1673370937] conn(0x557425f48e20 0x557425f4b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:36.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.604+0000 7f7d8560e740 4 client.16599 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:03:36.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.604+0000 7f7d8560e740 1 -- 172.21.15.5:0/2705874688 --> [v2:172.21.15.146:6836/1673370937,v1:172.21.15.146:6837/1673370937] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557425e9bd70 con 0x557425f48e20 2024-09-12T03:03:36.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.604+0000 7f7d7bfff640 1 --2- 172.21.15.5:0/2705874688 >> [v2:172.21.15.146:6836/1673370937,v1:172.21.15.146:6837/1673370937] conn(0x557425f48e20 0x557425f4b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:36.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.604+0000 7f7d5bfff640 10 client.16599 ms_handle_connect on v2:172.21.15.146:6836/1673370937 2024-09-12T03:03:36.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.604+0000 7f7d7bfff640 1 --2- 172.21.15.5:0/2705874688 >> [v2:172.21.15.146:6836/1673370937,v1:172.21.15.146:6837/1673370937] conn(0x557425f48e20 0x557425f4b210 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e764098113cb81c6 server_cookie=b2e54260873f5616 in_seq=0 out_seq=0 2024-09-12T03:03:36.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.605+0000 7f7d5bfff640 1 -- 172.21.15.5:0/2705874688 <== mds.4 v2:172.21.15.146:6836/1673370937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557425e9bd70 con 0x557425f48e20 2024-09-12T03:03:36.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.605+0000 7f7d5bfff640 10 client.16599 handle_command_reply: tid=0 2024-09-12T03:03:36.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.605+0000 7f7d8560e740 1 client.16599 shutdown 2024-09-12T03:03:36.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.605+0000 7f7d8560e740 1 -- 172.21.15.5:0/2705874688 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557425f2e5d0 msgr2=0x557425f38db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:36.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.605+0000 7f7d8560e740 1 --2- 172.21.15.5:0/2705874688 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557425f2e5d0 0x557425f38db0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f7d70098270 tx=0x7f7d700d2690 comp rx=0 tx=0).stop 2024-09-12T03:03:36.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.605+0000 7f7d8560e740 1 -- 172.21.15.5:0/2705874688 shutdown_connections 2024-09-12T03:03:36.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.605+0000 7f7d8560e740 1 --2- 172.21.15.5:0/2705874688 >> [v2:172.21.15.146:6836/1673370937,v1:172.21.15.146:6837/1673370937] conn(0x557425f48e20 0x557425f4b210 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:36.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.605+0000 7f7d8560e740 1 --2- 172.21.15.5:0/2705874688 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557425f39370 0x557425f367f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:36.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.605+0000 7f7d8560e740 1 --2- 172.21.15.5:0/2705874688 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557425f2fb40 0x557425f36060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:36.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.605+0000 7f7d5bfff640 0 client.16599 ms_handle_reset on v2:172.21.15.146:6836/1673370937 2024-09-12T03:03:36.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.605+0000 7f7d8560e740 1 --2- 172.21.15.5:0/2705874688 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557425f2e5d0 0x557425f38db0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:36.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.605+0000 7f7d8560e740 1 -- 172.21.15.5:0/2705874688 >> 172.21.15.5:0/2705874688 conn(0x557425ef7310 msgr2=0x557425f309c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:36.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.606+0000 7f7d8560e740 1 -- 172.21.15.5:0/2705874688 shutdown_connections 2024-09-12T03:03:36.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.606+0000 7f7d8560e740 1 -- 172.21.15.5:0/2705874688 wait complete. 2024-09-12T03:03:36.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.606+0000 7f7d8560e740 20 client.16599 trim_cache size 0 max 16384 2024-09-12T03:03:36.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.622+0000 7f7d8560e740 1 Processor -- start 2024-09-12T03:03:36.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.622+0000 7f7d8560e740 1 -- start start 2024-09-12T03:03:36.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.623+0000 7f7d8560e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557425f2e5d0 0x557425f35920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:36.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.623+0000 7f7d8560e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557425f2fb40 0x557425f31120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:36.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.623+0000 7f7d8560e740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557425f39370 0x557425f31660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:36.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.623+0000 7f7d8560e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7d7c1af7d0 con 0x557425f2fb40 2024-09-12T03:03:36.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.623+0000 7f7d8560e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f7d7c1b1310 con 0x557425f2e5d0 2024-09-12T03:03:36.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.623+0000 7f7d8560e740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f7d7c1afea0 con 0x557425f39370 2024-09-12T03:03:36.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.623+0000 7f7d817d2640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557425f39370 0x557425f31660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:36.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.623+0000 7f7d7bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557425f2fb40 0x557425f31120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:36.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.623+0000 7f7d80fd1640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557425f2e5d0 0x557425f35920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:36.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.623+0000 7f7d80fd1640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557425f2e5d0 0x557425f35920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.5:50046/0 (socket says 172.21.15.5:50046) 2024-09-12T03:03:36.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.623+0000 7f7d817d2640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557425f39370 0x557425f31660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.5:51636/0 (socket says 172.21.15.5:51636) 2024-09-12T03:03:36.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.623+0000 7f7d7bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557425f2fb40 0x557425f31120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49498/0 (socket says 172.21.15.5:49498) 2024-09-12T03:03:36.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.623+0000 7f7d80fd1640 1 -- 172.21.15.5:0/1526804197 learned_addr learned my addr 172.21.15.5:0/1526804197 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:03:36.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.623+0000 7f7d80fd1640 1 -- 172.21.15.5:0/1526804197 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557425f39370 msgr2=0x557425f31660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:36.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.623+0000 7f7d80fd1640 1 --2- 172.21.15.5:0/1526804197 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557425f39370 0x557425f31660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:36.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.623+0000 7f7d80fd1640 1 -- 172.21.15.5:0/1526804197 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557425f2fb40 msgr2=0x557425f31120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:36.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.623+0000 7f7d80fd1640 1 --2- 172.21.15.5:0/1526804197 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557425f2fb40 0x557425f31120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:36.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.623+0000 7f7d80fd1640 1 -- 172.21.15.5:0/1526804197 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557425f2b960 con 0x557425f2e5d0 2024-09-12T03:03:36.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.624+0000 7f7d80fd1640 1 --2- 172.21.15.5:0/1526804197 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557425f2e5d0 0x557425f35920 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7d7009b830 tx=0x7f7d70099db0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=961d74f0775d448b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:36.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.624+0000 7f7d667fc640 1 -- 172.21.15.5:0/1526804197 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d7001c070 con 0x557425f2e5d0 2024-09-12T03:03:36.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.624+0000 7f7d667fc640 1 -- 172.21.15.5:0/1526804197 <== mon.2 v2:172.21.15.146:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d7009bfa0 con 0x557425f2e5d0 2024-09-12T03:03:36.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.624+0000 7f7d667fc640 1 -- 172.21.15.5:0/1526804197 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d700975f0 con 0x557425f2e5d0 2024-09-12T03:03:36.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.625+0000 7f7d8560e740 1 -- 172.21.15.5:0/1526804197 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557425f2e5d0 msgr2=0x557425f35920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:36.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.625+0000 7f7d8560e740 1 --2- 172.21.15.5:0/1526804197 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557425f2e5d0 0x557425f35920 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7d7009b830 tx=0x7f7d70099db0 comp rx=0 tx=0).stop 2024-09-12T03:03:36.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.625+0000 7f7d8560e740 1 -- 172.21.15.5:0/1526804197 shutdown_connections 2024-09-12T03:03:36.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.625+0000 7f7d8560e740 1 --2- 172.21.15.5:0/1526804197 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557425f2fb40 0x557425f31120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:36.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.625+0000 7f7d8560e740 1 --2- 172.21.15.5:0/1526804197 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557425f39370 0x557425f31660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:36.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.625+0000 7f7d8560e740 1 --2- 172.21.15.5:0/1526804197 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557425f2e5d0 0x557425f35920 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:36.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.625+0000 7f7d8560e740 1 -- 172.21.15.5:0/1526804197 >> 172.21.15.5:0/1526804197 conn(0x557425ef7310 msgr2=0x557425f35150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:36.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.625+0000 7f7d8560e740 1 -- 172.21.15.5:0/1526804197 shutdown_connections 2024-09-12T03:03:36.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.625+0000 7f7d8560e740 1 -- 172.21.15.5:0/1526804197 wait complete. 2024-09-12T03:03:36.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.625+0000 7f7d8560e740 1 -- start start 2024-09-12T03:03:36.632 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.625+0000 7f7d8560e740 1 Processor -- start 2024-09-12T03:03:36.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.626+0000 7f7d8560e740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557425f2e5d0 0x557425f4b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:36.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.626+0000 7f7d8560e740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557425f2fb40 0x557425f4bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:36.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.626+0000 7f7d8560e740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557425f39370 0x557425fab720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:36.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.626+0000 7f7d8560e740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7d7c1b0c40 con 0x557425f2e5d0 2024-09-12T03:03:36.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.626+0000 7f7d8560e740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f7d7c1b0570 con 0x557425f2fb40 2024-09-12T03:03:36.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.626+0000 7f7d8560e740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f7d7c1ad7b0 con 0x557425f39370 2024-09-12T03:03:36.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.626+0000 7f7d8560e740 10 client.0 _ll_register_callbacks cb 0x7f7d68005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:03:36.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.626+0000 7f7d817d2640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557425f39370 0x557425fab720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:36.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.626+0000 7f7d817d2640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557425f39370 0x557425fab720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.5:51652/0 (socket says 172.21.15.5:51652) 2024-09-12T03:03:36.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.626+0000 7f7d817d2640 1 -- 172.21.15.5:0/1962105835 learned_addr learned my addr 172.21.15.5:0/1962105835 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:03:36.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.626+0000 7f7d7bfff640 1 --2- 172.21.15.5:0/1962105835 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557425f2fb40 0x557425f4bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:36.633 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.626+0000 7f7d80fd1640 1 --2- 172.21.15.5:0/1962105835 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557425f2e5d0 0x557425f4b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:36.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.626+0000 7f7d7bfff640 1 -- 172.21.15.5:0/1962105835 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557425f39370 msgr2=0x557425fab720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:36.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.626+0000 7f7d7bfff640 1 --2- 172.21.15.5:0/1962105835 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557425f39370 0x557425fab720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:36.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.627+0000 7f7d7bfff640 1 -- 172.21.15.5:0/1962105835 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557425f2e5d0 msgr2=0x557425f4b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:36.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.627+0000 7f7d7bfff640 1 --2- 172.21.15.5:0/1962105835 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557425f2e5d0 0x557425f4b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:36.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.627+0000 7f7d7bfff640 1 -- 172.21.15.5:0/1962105835 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557425fabd10 con 0x557425f2fb40 2024-09-12T03:03:36.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.627+0000 7f7d80fd1640 1 --2- 172.21.15.5:0/1962105835 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557425f2e5d0 0x557425f4b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:03:36.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.628+0000 7f7d7bfff640 1 --2- 172.21.15.5:0/1962105835 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557425f2fb40 0x557425f4bd30 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x557425f376a0 tx=0x7f7d6801fb30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1d9a887ca2b12ef server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:36.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.628+0000 7f7d5bfff640 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2024-09-12T03:03:36.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.628+0000 7f7d5bfff640 1 -- 172.21.15.5:0/1962105835 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d68028cf0 con 0x557425f2fb40 2024-09-12T03:03:36.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.628+0000 7f7d5bfff640 1 -- 172.21.15.5:0/1962105835 <== mon.2 v2:172.21.15.146:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f7d68005020 con 0x557425f2fb40 2024-09-12T03:03:36.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.628+0000 7f7d8560e740 1 -- 172.21.15.5:0/1962105835 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d7009bfa0 con 0x557425f2fb40 2024-09-12T03:03:36.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.629+0000 7f7d5bfff640 1 -- 172.21.15.5:0/1962105835 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f7d68005220 con 0x557425f2fb40 2024-09-12T03:03:36.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.629+0000 7f7d8560e740 10 client.25991 fetch_fsmap learned FSMap version 31 2024-09-12T03:03:36.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.629+0000 7f7d8560e740 1 -- 172.21.15.5:0/1962105835 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x557425fb30f0 con 0x557425f2fb40 2024-09-12T03:03:36.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.630+0000 7f7d5bfff640 1 -- 172.21.15.5:0/1962105835 <== mon.2 v2:172.21.15.146:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d7009bfa0 con 0x557425f2fb40 2024-09-12T03:03:36.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.630+0000 7f7d5bfff640 1 -- 172.21.15.5:0/1962105835 <== mon.2 v2:172.21.15.146:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f7d680049a0 con 0x557425f2fb40 2024-09-12T03:03:36.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.631+0000 7f7d8560e740 10 client.25991 fetch_fsmap finished waiting for FSMap version 31 2024-09-12T03:03:36.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.631+0000 7f7d8560e740 10 client.25991 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-12T03:03:36.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.631+0000 7f7d8560e740 20 client.25991 populate_metadata read hostname 'smithi005' 2024-09-12T03:03:36.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.631+0000 7f7d8560e740 1 --2- 172.21.15.5:0/1962105835 >> [v2:172.21.15.146:6836/1673370937,v1:172.21.15.146:6837/1673370937] conn(0x557425fb37f0 0x557425fb5c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:36.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.631+0000 7f7d8560e740 4 client.25991 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:03:36.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.631+0000 7f7d8560e740 1 -- 172.21.15.5:0/1962105835 --> [v2:172.21.15.146:6836/1673370937,v1:172.21.15.146:6837/1673370937] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557425e18170 con 0x557425fb37f0 2024-09-12T03:03:36.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.631+0000 7f7d80fd1640 1 --2- 172.21.15.5:0/1962105835 >> [v2:172.21.15.146:6836/1673370937,v1:172.21.15.146:6837/1673370937] conn(0x557425fb37f0 0x557425fb5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:36.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.632+0000 7f7d5bfff640 10 client.25991 ms_handle_connect on v2:172.21.15.146:6836/1673370937 2024-09-12T03:03:36.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.632+0000 7f7d80fd1640 1 --2- 172.21.15.5:0/1962105835 >> [v2:172.21.15.146:6836/1673370937,v1:172.21.15.146:6837/1673370937] conn(0x557425fb37f0 0x557425fb5c10 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2b31a22a62554f64 server_cookie=816d5607dca304f8 in_seq=0 out_seq=0 2024-09-12T03:03:36.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.632+0000 7f7d5bfff640 1 -- 172.21.15.5:0/1962105835 <== mds.4 v2:172.21.15.146:6836/1673370937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557425e18170 con 0x557425fb37f0 2024-09-12T03:03:36.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.632+0000 7f7d5bfff640 10 client.25991 handle_command_reply: tid=0 2024-09-12T03:03:36.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.632+0000 7f7d8560e740 1 client.25991 shutdown 2024-09-12T03:03:36.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.632+0000 7f7d8560e740 1 -- 172.21.15.5:0/1962105835 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557425f2fb40 msgr2=0x557425f4bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:36.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.632+0000 7f7d8560e740 1 --2- 172.21.15.5:0/1962105835 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557425f2fb40 0x557425f4bd30 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x557425f376a0 tx=0x7f7d6801fb30 comp rx=0 tx=0).stop 2024-09-12T03:03:36.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.632+0000 7f7d8560e740 1 -- 172.21.15.5:0/1962105835 shutdown_connections 2024-09-12T03:03:36.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.632+0000 7f7d8560e740 1 --2- 172.21.15.5:0/1962105835 >> [v2:172.21.15.146:6836/1673370937,v1:172.21.15.146:6837/1673370937] conn(0x557425fb37f0 0x557425fb5c10 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:36.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.632+0000 7f7d8560e740 1 --2- 172.21.15.5:0/1962105835 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x557425f39370 0x557425fab720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:36.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.632+0000 7f7d5bfff640 0 client.25991 ms_handle_reset on v2:172.21.15.146:6836/1673370937 2024-09-12T03:03:36.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.632+0000 7f7d8560e740 1 --2- 172.21.15.5:0/1962105835 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x557425f2fb40 0x557425f4bd30 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:36.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.632+0000 7f7d8560e740 1 --2- 172.21.15.5:0/1962105835 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557425f2e5d0 0x557425f4b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:36.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.633+0000 7f7d8560e740 1 -- 172.21.15.5:0/1962105835 >> 172.21.15.5:0/1962105835 conn(0x557425ef7310 msgr2=0x557425f294e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:36.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.633+0000 7f7d8560e740 1 -- 172.21.15.5:0/1962105835 shutdown_connections 2024-09-12T03:03:36.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.633+0000 7f7d8560e740 1 -- 172.21.15.5:0/1962105835 wait complete. 2024-09-12T03:03:36.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.633+0000 7f7d8560e740 20 client.25991 trim_cache size 0 max 16384 2024-09-12T03:03:36.639 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-12T03:03:36.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.634+0000 7f7d83235640 1 -- 172.21.15.5:0/3971694590 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f7d6c0a2b80 msgr2=0x7f7d6c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:36.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.634+0000 7f7d83235640 1 --2- 172.21.15.5:0/3971694590 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f7d6c0a2b80 0x7f7d6c0a5040 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f7d74004af0 tx=0x7f7d7402a8c0 comp rx=0 tx=0).stop 2024-09-12T03:03:36.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.634+0000 7f7d83235640 1 -- 172.21.15.5:0/3971694590 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7d7c102df0 msgr2=0x7f7d7c100a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:36.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.634+0000 7f7d83235640 1 --2- 172.21.15.5:0/3971694590 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7d7c102df0 0x7f7d7c100a50 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7d7000e9f0 tx=0x7f7d7000eeb0 comp rx=0 tx=0).stop 2024-09-12T03:03:36.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.634+0000 7f7d83235640 1 -- 172.21.15.5:0/3971694590 shutdown_connections 2024-09-12T03:03:36.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.634+0000 7f7d83235640 1 --2- 172.21.15.5:0/3971694590 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f7d6c0a2b80 0x7f7d6c0a5040 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:36.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.634+0000 7f7d83235640 1 --2- 172.21.15.5:0/3971694590 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d7c1037a0 0x7f7d7c100f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:36.640 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.634+0000 7f7d83235640 1 --2- 172.21.15.5:0/3971694590 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7d7c10c2f0 0x7f7d7c1014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:36.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.634+0000 7f7d83235640 1 --2- 172.21.15.5:0/3971694590 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7d7c102df0 0x7f7d7c100a50 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:36.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.634+0000 7f7d83235640 1 -- 172.21.15.5:0/3971694590 >> 172.21.15.5:0/3971694590 conn(0x7f7d7c0fc7d0 msgr2=0x7f7d7c0fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:36.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.634+0000 7f7d83235640 1 -- 172.21.15.5:0/3971694590 shutdown_connections 2024-09-12T03:03:36.641 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:36.634+0000 7f7d83235640 1 -- 172.21.15.5:0/3971694590 wait complete. 2024-09-12T03:03:37.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:03:36 smithi005 ceph-mon[31197]: pgmap v496: 97 pgs: 97 active+clean; 137 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 MiB/s rd, 2.8 MiB/s wr, 1.07k op/s 2024-09-12T03:03:37.165 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:03:36 smithi148 ceph-mon[35090]: pgmap v496: 97 pgs: 97 active+clean; 137 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 MiB/s rd, 2.8 MiB/s wr, 1.07k op/s 2024-09-12T03:03:37.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:03:36 smithi146 ceph-mon[35011]: pgmap v496: 97 pgs: 97 active+clean; 137 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 21 MiB/s rd, 2.8 MiB/s wr, 1.07k op/s 2024-09-12T03:03:37.363 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-12T03:03:37.684 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:03:38.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.207+0000 7f3a69532640 1 -- 172.21.15.5:0/267469014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a64102ef0 msgr2=0x7f3a6410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:38.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.207+0000 7f3a69532640 1 --2- 172.21.15.5:0/267469014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a64102ef0 0x7f3a6410da40 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c009ca0 tx=0x7f3a5c02a950 comp rx=0 tx=0).stop 2024-09-12T03:03:38.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.207+0000 7f3a69532640 1 -- 172.21.15.5:0/267469014 shutdown_connections 2024-09-12T03:03:38.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.207+0000 7f3a69532640 1 --2- 172.21.15.5:0/267469014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a64102ef0 0x7f3a6410da40 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:38.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.207+0000 7f3a69532640 1 --2- 172.21.15.5:0/267469014 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3a64102570 0x7f3a641029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:38.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.207+0000 7f3a69532640 1 --2- 172.21.15.5:0/267469014 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3a64100e20 0x7f3a64101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:38.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.207+0000 7f3a69532640 1 -- 172.21.15.5:0/267469014 >> 172.21.15.5:0/267469014 conn(0x7f3a640fc7d0 msgr2=0x7f3a640febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:38.220 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.207+0000 7f3a69532640 1 -- 172.21.15.5:0/267469014 shutdown_connections 2024-09-12T03:03:38.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.207+0000 7f3a69532640 1 -- 172.21.15.5:0/267469014 wait complete. 2024-09-12T03:03:38.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.208+0000 7f3a69532640 1 Processor -- start 2024-09-12T03:03:38.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.208+0000 7f3a69532640 1 -- start start 2024-09-12T03:03:38.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.208+0000 7f3a69532640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a64100e20 0x7f3a6419a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:38.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.208+0000 7f3a69532640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3a64102570 0x7f3a6419abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:38.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.208+0000 7f3a69532640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3a64102ef0 0x7f3a6419ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:38.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.208+0000 7f3a69532640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3a64110f80 con 0x7f3a64100e20 2024-09-12T03:03:38.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.208+0000 7f3a69532640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f3a64110e00 con 0x7f3a64102570 2024-09-12T03:03:38.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.208+0000 7f3a69532640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3a64111100 con 0x7f3a64102ef0 2024-09-12T03:03:38.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.208+0000 7f3a62ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a64100e20 0x7f3a6419a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:38.221 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.208+0000 7f3a62ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a64100e20 0x7f3a6419a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49526/0 (socket says 172.21.15.5:49526) 2024-09-12T03:03:38.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.208+0000 7f3a637fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3a64102ef0 0x7f3a6419ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:38.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.208+0000 7f3a62ffd640 1 -- 172.21.15.5:0/2124821715 learned_addr learned my addr 172.21.15.5:0/2124821715 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:03:38.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.208+0000 7f3a5bfff640 1 --2- 172.21.15.5:0/2124821715 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3a64102570 0x7f3a6419abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:38.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.209+0000 7f3a62ffd640 1 -- 172.21.15.5:0/2124821715 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3a64102ef0 msgr2=0x7f3a6419ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:38.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.209+0000 7f3a62ffd640 1 --2- 172.21.15.5:0/2124821715 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3a64102ef0 0x7f3a6419ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:38.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.209+0000 7f3a62ffd640 1 -- 172.21.15.5:0/2124821715 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3a64102570 msgr2=0x7f3a6419abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:38.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.209+0000 7f3a62ffd640 1 --2- 172.21.15.5:0/2124821715 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3a64102570 0x7f3a6419abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:38.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.209+0000 7f3a62ffd640 1 -- 172.21.15.5:0/2124821715 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a6419f640 con 0x7f3a64100e20 2024-09-12T03:03:38.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.209+0000 7f3a637fe640 1 --2- 172.21.15.5:0/2124821715 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3a64102ef0 0x7f3a6419ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:03:38.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.209+0000 7f3a5bfff640 1 --2- 172.21.15.5:0/2124821715 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3a64102570 0x7f3a6419abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:03:38.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.209+0000 7f3a62ffd640 1 --2- 172.21.15.5:0/2124821715 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a64100e20 0x7f3a6419a690 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f3a5400d950 tx=0x7f3a5400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:38.222 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.209+0000 7f3a60ff9640 1 -- 172.21.15.5:0/2124821715 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a54017030 con 0x7f3a64100e20 2024-09-12T03:03:38.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.209+0000 7f3a60ff9640 1 -- 172.21.15.5:0/2124821715 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a54004020 con 0x7f3a64100e20 2024-09-12T03:03:38.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.209+0000 7f3a69532640 1 -- 172.21.15.5:0/2124821715 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a6419f930 con 0x7f3a64100e20 2024-09-12T03:03:38.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.209+0000 7f3a60ff9640 1 -- 172.21.15.5:0/2124821715 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a540047a0 con 0x7f3a64100e20 2024-09-12T03:03:38.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.209+0000 7f3a69532640 1 -- 172.21.15.5:0/2124821715 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a641a7210 con 0x7f3a64100e20 2024-09-12T03:03:38.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.212+0000 7f3a60ff9640 1 -- 172.21.15.5:0/2124821715 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f3a540041d0 con 0x7f3a64100e20 2024-09-12T03:03:38.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.212+0000 7f3a6b90b740 1 Processor -- start 2024-09-12T03:03:38.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.212+0000 7f3a6b90b740 1 -- start start 2024-09-12T03:03:38.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.213+0000 7f3a6b90b740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56085b4c8b40 0x56085b4c7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:38.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.213+0000 7f3a6b90b740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56085b4c75d0 0x56085b4c7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:38.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.213+0000 7f3a6b90b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56085b4d2370 0x56085b4d4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:38.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.213+0000 7f3a60ff9640 1 --2- 172.21.15.5:0/2124821715 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f3a500a2c00 0x7f3a500a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:38.223 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.213+0000 7f3a637fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56085b4c8b40 0x56085b4c7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:38.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.213+0000 7f3a637fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56085b4c8b40 0x56085b4c7090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.5:51682/0 (socket says 172.21.15.5:51682) 2024-09-12T03:03:38.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.213+0000 7f3a62ffd640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56085b4c75d0 0x56085b4c7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:38.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.213+0000 7f3a637fe640 1 -- 172.21.15.5:0/679716111 learned_addr learned my addr 172.21.15.5:0/679716111 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:03:38.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.213+0000 7f3a6b90b740 1 -- 172.21.15.5:0/679716111 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56085b42d2f0 con 0x56085b4d2370 2024-09-12T03:03:38.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.213+0000 7f3a6b90b740 1 -- 172.21.15.5:0/679716111 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x56085b3d4cf0 con 0x56085b4c75d0 2024-09-12T03:03:38.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.213+0000 7f3a6b90b740 1 -- 172.21.15.5:0/679716111 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x56085b4292b0 con 0x56085b4c8b40 2024-09-12T03:03:38.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.214+0000 7f3a637fe640 1 --2- 172.21.15.5:0/2124821715 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f3a500a2c00 0x7f3a500a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:38.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.214+0000 7f3a60ff9640 1 -- 172.21.15.5:0/2124821715 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f3a540ce910 con 0x7f3a64100e20 2024-09-12T03:03:38.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.214+0000 7f3a62ffd640 1 -- 172.21.15.5:0/679716111 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56085b4c8b40 msgr2=0x56085b4c7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:38.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.214+0000 7f3a62ffd640 1 --2- 172.21.15.5:0/679716111 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56085b4c8b40 0x56085b4c7090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:38.224 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.214+0000 7f3a62ffd640 1 -- 172.21.15.5:0/679716111 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56085b4d2370 msgr2=0x56085b4d4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:38.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.214+0000 7f3a62ffd640 1 --2- 172.21.15.5:0/679716111 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56085b4d2370 0x56085b4d4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:38.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.214+0000 7f3a62ffd640 1 -- 172.21.15.5:0/679716111 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56085b3b93a0 con 0x56085b4c75d0 2024-09-12T03:03:38.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.214+0000 7f3a62ffd640 1 --2- 172.21.15.5:0/679716111 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56085b4c75d0 0x56085b4c7a10 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3a5409b0f0 tx=0x7f3a5406e220 comp rx=0 tx=0).ready entity=mon.2 client_cookie=797b3986ac107ae1 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:38.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.214+0000 7f3a637fe640 1 --2- 172.21.15.5:0/2124821715 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f3a500a2c00 0x7f3a500a50c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c02fd80 tx=0x7f3a5c009380 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:38.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.214+0000 7f3a597fa640 1 -- 172.21.15.5:0/679716111 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a5409b160 con 0x56085b4c75d0 2024-09-12T03:03:38.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.214+0000 7f3a597fa640 1 -- 172.21.15.5:0/679716111 <== mon.2 v2:172.21.15.146:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a5409f510 con 0x56085b4c75d0 2024-09-12T03:03:38.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.214+0000 7f3a597fa640 1 -- 172.21.15.5:0/679716111 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a540a3940 con 0x56085b4c75d0 2024-09-12T03:03:38.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.215+0000 7f3a6b90b740 1 -- 172.21.15.5:0/679716111 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56085b4c75d0 msgr2=0x56085b4c7a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:38.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.215+0000 7f3a6b90b740 1 --2- 172.21.15.5:0/679716111 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56085b4c75d0 0x56085b4c7a10 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3a5409b0f0 tx=0x7f3a5406e220 comp rx=0 tx=0).stop 2024-09-12T03:03:38.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.216+0000 7f3a6b90b740 1 -- 172.21.15.5:0/679716111 shutdown_connections 2024-09-12T03:03:38.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.216+0000 7f3a6b90b740 1 --2- 172.21.15.5:0/679716111 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56085b4d2370 0x56085b4d4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:38.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.216+0000 7f3a6b90b740 1 --2- 172.21.15.5:0/679716111 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56085b4c75d0 0x56085b4c7a10 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:38.225 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.216+0000 7f3a6b90b740 1 --2- 172.21.15.5:0/679716111 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56085b4c8b40 0x56085b4c7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:38.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.216+0000 7f3a6b90b740 1 -- 172.21.15.5:0/679716111 >> 172.21.15.5:0/679716111 conn(0x56085b490310 msgr2=0x56085b4c3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:38.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.216+0000 7f3a6b90b740 1 -- 172.21.15.5:0/679716111 shutdown_connections 2024-09-12T03:03:38.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.216+0000 7f3a6b90b740 1 -- 172.21.15.5:0/679716111 wait complete. 2024-09-12T03:03:38.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.216+0000 7f3a6b90b740 1 -- start start 2024-09-12T03:03:38.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.216+0000 7f3a6b90b740 1 Processor -- start 2024-09-12T03:03:38.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.217+0000 7f3a6b90b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56085b4c75d0 0x56085b4cf0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:38.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.217+0000 7f3a6b90b740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56085b4c8b40 0x56085b4cf630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:38.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.217+0000 7f3a6b90b740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56085b4d2370 0x56085b4cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:38.226 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.217+0000 7f3a6b90b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x56085b4d5010 con 0x56085b4c75d0 2024-09-12T03:03:38.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.217+0000 7f3a6b90b740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x56085b3e8060 con 0x56085b4d2370 2024-09-12T03:03:38.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.217+0000 7f3a6b90b740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x56085b4d5190 con 0x56085b4c8b40 2024-09-12T03:03:38.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.217+0000 7f3a637fe640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56085b4d2370 0x56085b4cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:38.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.217+0000 7f3a637fe640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56085b4d2370 0x56085b4cfb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.5:50100/0 (socket says 172.21.15.5:50100) 2024-09-12T03:03:38.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.217+0000 7f3a637fe640 1 -- 172.21.15.5:0/3311633092 learned_addr learned my addr 172.21.15.5:0/3311633092 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:03:38.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.217+0000 7f3a5bfff640 1 --2- 172.21.15.5:0/3311633092 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56085b4c8b40 0x56085b4cf630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:38.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.217+0000 7f3a6b90b740 10 client.0 _ll_register_callbacks cb 0x56085b13b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:03:38.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.217+0000 7f3a637fe640 1 -- 172.21.15.5:0/3311633092 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56085b4c8b40 msgr2=0x56085b4cf630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:38.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.217+0000 7f3a637fe640 1 --2- 172.21.15.5:0/3311633092 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56085b4c8b40 0x56085b4cf630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:38.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.217+0000 7f3a637fe640 1 -- 172.21.15.5:0/3311633092 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56085b4c75d0 msgr2=0x56085b4cf0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:38.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.217+0000 7f3a637fe640 1 --2- 172.21.15.5:0/3311633092 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56085b4c75d0 0x56085b4cf0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:38.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.217+0000 7f3a637fe640 1 -- 172.21.15.5:0/3311633092 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56085b19e3c0 con 0x56085b4d2370 2024-09-12T03:03:38.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.218+0000 7f3a637fe640 1 --2- 172.21.15.5:0/3311633092 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56085b4d2370 0x56085b4cfb70 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c030910 tx=0x7f3a5c004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ddefd245e0092ff server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:38.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.218+0000 7f3a4a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2024-09-12T03:03:38.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.218+0000 7f3a4a7fc640 1 -- 172.21.15.5:0/3311633092 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a5c04f070 con 0x56085b4d2370 2024-09-12T03:03:38.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.218+0000 7f3a4a7fc640 1 -- 172.21.15.5:0/3311633092 <== mon.2 v2:172.21.15.146:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f3a5c04b860 con 0x56085b4d2370 2024-09-12T03:03:38.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.218+0000 7f3a6b90b740 1 -- 172.21.15.5:0/3311633092 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a5409f510 con 0x56085b4d2370 2024-09-12T03:03:38.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.218+0000 7f3a4a7fc640 1 -- 172.21.15.5:0/3311633092 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f3a5c034040 con 0x56085b4d2370 2024-09-12T03:03:38.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.218+0000 7f3a6b90b740 10 client.26003 fetch_fsmap learned FSMap version 31 2024-09-12T03:03:38.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.218+0000 7f3a6b90b740 1 -- 172.21.15.5:0/3311633092 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56085b4e0b80 con 0x56085b4d2370 2024-09-12T03:03:38.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.218+0000 7f3a4a7fc640 1 -- 172.21.15.5:0/3311633092 <== mon.2 v2:172.21.15.146:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a5409f510 con 0x56085b4d2370 2024-09-12T03:03:38.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.219+0000 7f3a4a7fc640 1 -- 172.21.15.5:0/3311633092 <== mon.2 v2:172.21.15.146:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f3a5c0548f0 con 0x56085b4d2370 2024-09-12T03:03:38.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.219+0000 7f3a6b90b740 10 client.26003 fetch_fsmap finished waiting for FSMap version 31 2024-09-12T03:03:38.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.219+0000 7f3a6b90b740 10 client.26003 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.b 2024-09-12T03:03:38.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.219+0000 7f3a6b90b740 20 client.26003 populate_metadata read hostname 'smithi005' 2024-09-12T03:03:38.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.219+0000 7f3a6b90b740 1 --2- 172.21.15.5:0/3311633092 >> [v2:172.21.15.146:6838/3036767228,v1:172.21.15.146:6839/3036767228] conn(0x56085b4e2040 0x56085b4e44d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:38.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.219+0000 7f3a6b90b740 4 client.26003 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:03:38.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.219+0000 7f3a6b90b740 1 -- 172.21.15.5:0/3311633092 --> [v2:172.21.15.146:6838/3036767228,v1:172.21.15.146:6839/3036767228] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56085b43a860 con 0x56085b4e2040 2024-09-12T03:03:38.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.219+0000 7f3a62ffd640 1 --2- 172.21.15.5:0/3311633092 >> [v2:172.21.15.146:6838/3036767228,v1:172.21.15.146:6839/3036767228] conn(0x56085b4e2040 0x56085b4e44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:38.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.220+0000 7f3a4a7fc640 10 client.26003 ms_handle_connect on v2:172.21.15.146:6838/3036767228 2024-09-12T03:03:38.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.220+0000 7f3a62ffd640 1 --2- 172.21.15.5:0/3311633092 >> [v2:172.21.15.146:6838/3036767228,v1:172.21.15.146:6839/3036767228] conn(0x56085b4e2040 0x56085b4e44d0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=113164ec3eec8527 server_cookie=691e94f1091014bc in_seq=0 out_seq=0 2024-09-12T03:03:38.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.220+0000 7f3a4a7fc640 1 -- 172.21.15.5:0/3311633092 <== mds.3 v2:172.21.15.146:6838/3036767228 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56085b43a860 con 0x56085b4e2040 2024-09-12T03:03:38.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.220+0000 7f3a4a7fc640 10 client.26003 handle_command_reply: tid=0 2024-09-12T03:03:38.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.220+0000 7f3a6b90b740 1 client.26003 shutdown 2024-09-12T03:03:38.230 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.220+0000 7f3a6b90b740 1 -- 172.21.15.5:0/3311633092 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56085b4d2370 msgr2=0x56085b4cfb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:38.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.220+0000 7f3a6b90b740 1 --2- 172.21.15.5:0/3311633092 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56085b4d2370 0x56085b4cfb70 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c030910 tx=0x7f3a5c004a40 comp rx=0 tx=0).stop 2024-09-12T03:03:38.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.220+0000 7f3a6b90b740 1 -- 172.21.15.5:0/3311633092 shutdown_connections 2024-09-12T03:03:38.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.220+0000 7f3a6b90b740 1 --2- 172.21.15.5:0/3311633092 >> [v2:172.21.15.146:6838/3036767228,v1:172.21.15.146:6839/3036767228] conn(0x56085b4e2040 0x56085b4e44d0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:38.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.220+0000 7f3a4a7fc640 0 client.26003 ms_handle_reset on v2:172.21.15.146:6838/3036767228 2024-09-12T03:03:38.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.220+0000 7f3a6b90b740 1 --2- 172.21.15.5:0/3311633092 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56085b4d2370 0x56085b4cfb70 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:38.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.220+0000 7f3a6b90b740 1 --2- 172.21.15.5:0/3311633092 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56085b4c8b40 0x56085b4cf630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:38.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.220+0000 7f3a6b90b740 1 --2- 172.21.15.5:0/3311633092 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56085b4c75d0 0x56085b4cf0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:38.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.220+0000 7f3a6b90b740 1 -- 172.21.15.5:0/3311633092 >> 172.21.15.5:0/3311633092 conn(0x56085b490310 msgr2=0x56085b4c99c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:38.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.221+0000 7f3a6b90b740 1 -- 172.21.15.5:0/3311633092 shutdown_connections 2024-09-12T03:03:38.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.221+0000 7f3a6b90b740 1 -- 172.21.15.5:0/3311633092 wait complete. 2024-09-12T03:03:38.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.221+0000 7f3a6b90b740 20 client.26003 trim_cache size 0 max 16384 2024-09-12T03:03:38.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.238+0000 7f3a6b90b740 1 Processor -- start 2024-09-12T03:03:38.238 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.238+0000 7f3a6b90b740 1 -- start start 2024-09-12T03:03:38.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.239+0000 7f3a6b90b740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56085b4c75d0 0x56085b4ca120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:38.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.239+0000 7f3a6b90b740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56085b4c8b40 0x56085b4ca660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:38.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.239+0000 7f3a6b90b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56085b4d2370 0x56085b4e9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:38.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.239+0000 7f3a6b90b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3a641af830 con 0x56085b4d2370 2024-09-12T03:03:38.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.239+0000 7f3a6b90b740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f3a641b1370 con 0x56085b4c8b40 2024-09-12T03:03:38.239 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.239+0000 7f3a6b90b740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3a641aff00 con 0x56085b4c75d0 2024-09-12T03:03:38.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.239+0000 7f3a637fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56085b4d2370 0x56085b4e9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:38.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.239+0000 7f3a637fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56085b4d2370 0x56085b4e9da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49548/0 (socket says 172.21.15.5:49548) 2024-09-12T03:03:38.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.239+0000 7f3a637fe640 1 -- 172.21.15.5:0/1905076607 learned_addr learned my addr 172.21.15.5:0/1905076607 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:03:38.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.239+0000 7f3a62ffd640 1 --2- 172.21.15.5:0/1905076607 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56085b4c75d0 0x56085b4ca120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:38.240 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.239+0000 7f3a5bfff640 1 --2- 172.21.15.5:0/1905076607 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56085b4c8b40 0x56085b4ca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:38.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.239+0000 7f3a637fe640 1 -- 172.21.15.5:0/1905076607 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56085b4c75d0 msgr2=0x56085b4ca120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:38.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.239+0000 7f3a637fe640 1 --2- 172.21.15.5:0/1905076607 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56085b4c75d0 0x56085b4ca120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:38.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.239+0000 7f3a637fe640 1 -- 172.21.15.5:0/1905076607 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56085b4c8b40 msgr2=0x56085b4ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:38.243 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.239+0000 7f3a637fe640 1 --2- 172.21.15.5:0/1905076607 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56085b4c8b40 0x56085b4ca660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:38.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.239+0000 7f3a637fe640 1 -- 172.21.15.5:0/1905076607 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56085b4c4dd0 con 0x56085b4d2370 2024-09-12T03:03:38.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.239+0000 7f3a5bfff640 1 --2- 172.21.15.5:0/1905076607 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56085b4c8b40 0x56085b4ca660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:03:38.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.240+0000 7f3a62ffd640 1 --2- 172.21.15.5:0/1905076607 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56085b4c75d0 0x56085b4ca120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:03:38.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.240+0000 7f3a637fe640 1 --2- 172.21.15.5:0/1905076607 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56085b4d2370 0x56085b4e9da0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c02f7f0 tx=0x7f3a5c005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=329ef74bfa801731 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:38.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.240+0000 7f3a58ff9640 1 -- 172.21.15.5:0/1905076607 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a5c04f070 con 0x56085b4d2370 2024-09-12T03:03:38.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.240+0000 7f3a58ff9640 1 -- 172.21.15.5:0/1905076607 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a5c003e50 con 0x56085b4d2370 2024-09-12T03:03:38.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.240+0000 7f3a58ff9640 1 -- 172.21.15.5:0/1905076607 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a5c0546c0 con 0x56085b4d2370 2024-09-12T03:03:38.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.241+0000 7f3a6b90b740 1 -- 172.21.15.5:0/1905076607 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56085b4d2370 msgr2=0x56085b4e9da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:38.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.241+0000 7f3a6b90b740 1 --2- 172.21.15.5:0/1905076607 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56085b4d2370 0x56085b4e9da0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c02f7f0 tx=0x7f3a5c005ef0 comp rx=0 tx=0).stop 2024-09-12T03:03:38.244 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.241+0000 7f3a6b90b740 1 -- 172.21.15.5:0/1905076607 shutdown_connections 2024-09-12T03:03:38.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.241+0000 7f3a6b90b740 1 --2- 172.21.15.5:0/1905076607 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56085b4d2370 0x56085b4e9da0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:38.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.241+0000 7f3a6b90b740 1 --2- 172.21.15.5:0/1905076607 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56085b4c8b40 0x56085b4ca660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:38.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.241+0000 7f3a6b90b740 1 --2- 172.21.15.5:0/1905076607 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56085b4c75d0 0x56085b4ca120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:38.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.241+0000 7f3a6b90b740 1 -- 172.21.15.5:0/1905076607 >> 172.21.15.5:0/1905076607 conn(0x56085b490310 msgr2=0x56085b4d1170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:38.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.241+0000 7f3a6b90b740 1 -- 172.21.15.5:0/1905076607 shutdown_connections 2024-09-12T03:03:38.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.241+0000 7f3a6b90b740 1 -- 172.21.15.5:0/1905076607 wait complete. 2024-09-12T03:03:38.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.241+0000 7f3a6b90b740 1 -- start start 2024-09-12T03:03:38.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.241+0000 7f3a6b90b740 1 Processor -- start 2024-09-12T03:03:38.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.242+0000 7f3a6b90b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56085b4c75d0 0x56085b4da8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:38.245 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.242+0000 7f3a6b90b740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56085b4c8b40 0x56085b4cbb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:38.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.242+0000 7f3a6b90b740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56085b4d2370 0x56085b4cc090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:38.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.242+0000 7f3a6b90b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f3a641b0ca0 con 0x56085b4c75d0 2024-09-12T03:03:38.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.242+0000 7f3a6b90b740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f3a641b05d0 con 0x56085b4c8b40 2024-09-12T03:03:38.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.242+0000 7f3a6b90b740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f3a641ad810 con 0x56085b4d2370 2024-09-12T03:03:38.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.242+0000 7f3a6b90b740 10 client.0 _ll_register_callbacks cb 0x7f3a5c005f90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:03:38.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.242+0000 7f3a62ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56085b4c75d0 0x56085b4da8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:38.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.242+0000 7f3a62ffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56085b4c75d0 0x56085b4da8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49550/0 (socket says 172.21.15.5:49550) 2024-09-12T03:03:38.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.242+0000 7f3a62ffd640 1 -- 172.21.15.5:0/1943094230 learned_addr learned my addr 172.21.15.5:0/1943094230 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:03:38.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.242+0000 7f3a637fe640 1 --2- 172.21.15.5:0/1943094230 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56085b4d2370 0x56085b4cc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:38.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.242+0000 7f3a62ffd640 1 -- 172.21.15.5:0/1943094230 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56085b4d2370 msgr2=0x56085b4cc090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:38.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.242+0000 7f3a62ffd640 1 --2- 172.21.15.5:0/1943094230 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56085b4d2370 0x56085b4cc090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:38.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.242+0000 7f3a62ffd640 1 -- 172.21.15.5:0/1943094230 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56085b4c8b40 msgr2=0x56085b4cbb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:38.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.242+0000 7f3a62ffd640 1 --2- 172.21.15.5:0/1943094230 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56085b4c8b40 0x56085b4cbb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:38.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.242+0000 7f3a62ffd640 1 -- 172.21.15.5:0/1943094230 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56085b4cc650 con 0x56085b4c75d0 2024-09-12T03:03:38.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.242+0000 7f3a62ffd640 1 --2- 172.21.15.5:0/1943094230 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56085b4c75d0 0x56085b4da8a0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f3a540975c0 tx=0x7f3a54005130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5936965438b61409 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:38.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.242+0000 7f3a4a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T03:03:38.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.242+0000 7f3a4a7fc640 1 -- 172.21.15.5:0/1943094230 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a5409b160 con 0x56085b4c75d0 2024-09-12T03:03:38.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.242+0000 7f3a4a7fc640 1 -- 172.21.15.5:0/1943094230 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f3a540977d0 con 0x56085b4c75d0 2024-09-12T03:03:38.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.242+0000 7f3a6b90b740 1 -- 172.21.15.5:0/1943094230 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a5c003e50 con 0x56085b4c75d0 2024-09-12T03:03:38.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.243+0000 7f3a4a7fc640 1 -- 172.21.15.5:0/1943094230 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f3a540a3dd0 con 0x56085b4c75d0 2024-09-12T03:03:38.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.243+0000 7f3a6b90b740 10 client.16617 fetch_fsmap learned FSMap version 31 2024-09-12T03:03:38.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.243+0000 7f3a6b90b740 1 -- 172.21.15.5:0/1943094230 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56085b54cb20 con 0x56085b4c75d0 2024-09-12T03:03:38.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.243+0000 7f3a4a7fc640 1 -- 172.21.15.5:0/1943094230 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a5c003e50 con 0x56085b4c75d0 2024-09-12T03:03:38.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.243+0000 7f3a4a7fc640 1 -- 172.21.15.5:0/1943094230 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f3a540a3940 con 0x56085b4c75d0 2024-09-12T03:03:38.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.243+0000 7f3a6b90b740 10 client.16617 fetch_fsmap finished waiting for FSMap version 31 2024-09-12T03:03:38.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.243+0000 7f3a6b90b740 10 client.16617 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.b 2024-09-12T03:03:38.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.243+0000 7f3a6b90b740 20 client.16617 populate_metadata read hostname 'smithi005' 2024-09-12T03:03:38.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.243+0000 7f3a6b90b740 1 --2- 172.21.15.5:0/1943094230 >> [v2:172.21.15.146:6838/3036767228,v1:172.21.15.146:6839/3036767228] conn(0x56085b54ce10 0x56085b54f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:38.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.243+0000 7f3a6b90b740 4 client.16617 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:03:38.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.243+0000 7f3a6b90b740 1 -- 172.21.15.5:0/1943094230 --> [v2:172.21.15.146:6838/3036767228,v1:172.21.15.146:6839/3036767228] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56085b434d70 con 0x56085b54ce10 2024-09-12T03:03:38.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.243+0000 7f3a5bfff640 1 --2- 172.21.15.5:0/1943094230 >> [v2:172.21.15.146:6838/3036767228,v1:172.21.15.146:6839/3036767228] conn(0x56085b54ce10 0x56085b54f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:38.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.244+0000 7f3a5bfff640 1 --2- 172.21.15.5:0/1943094230 >> [v2:172.21.15.146:6838/3036767228,v1:172.21.15.146:6839/3036767228] conn(0x56085b54ce10 0x56085b54f200 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=38be6a0ed75112d7 server_cookie=c07e23c4b0c73a3d in_seq=0 out_seq=0 2024-09-12T03:03:38.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.244+0000 7f3a4a7fc640 10 client.16617 ms_handle_connect on v2:172.21.15.146:6838/3036767228 2024-09-12T03:03:38.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.244+0000 7f3a4a7fc640 1 -- 172.21.15.5:0/1943094230 <== mds.3 v2:172.21.15.146:6838/3036767228 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56085b434d70 con 0x56085b54ce10 2024-09-12T03:03:38.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.244+0000 7f3a4a7fc640 10 client.16617 handle_command_reply: tid=0 2024-09-12T03:03:38.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.244+0000 7f3a6b90b740 1 client.16617 shutdown 2024-09-12T03:03:38.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.244+0000 7f3a6b90b740 1 -- 172.21.15.5:0/1943094230 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56085b4c75d0 msgr2=0x56085b4da8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:38.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.244+0000 7f3a6b90b740 1 --2- 172.21.15.5:0/1943094230 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56085b4c75d0 0x56085b4da8a0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f3a540975c0 tx=0x7f3a54005130 comp rx=0 tx=0).stop 2024-09-12T03:03:38.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.244+0000 7f3a6b90b740 1 -- 172.21.15.5:0/1943094230 shutdown_connections 2024-09-12T03:03:38.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.244+0000 7f3a6b90b740 1 --2- 172.21.15.5:0/1943094230 >> [v2:172.21.15.146:6838/3036767228,v1:172.21.15.146:6839/3036767228] conn(0x56085b54ce10 0x56085b54f200 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:38.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.244+0000 7f3a6b90b740 1 --2- 172.21.15.5:0/1943094230 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x56085b4d2370 0x56085b4cc090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:38.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.244+0000 7f3a6b90b740 1 --2- 172.21.15.5:0/1943094230 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x56085b4c8b40 0x56085b4cbb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:38.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.244+0000 7f3a6b90b740 1 --2- 172.21.15.5:0/1943094230 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56085b4c75d0 0x56085b4da8a0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:38.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.244+0000 7f3a6b90b740 1 -- 172.21.15.5:0/1943094230 >> 172.21.15.5:0/1943094230 conn(0x56085b490310 msgr2=0x56085b4c24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:38.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.244+0000 7f3a4a7fc640 0 client.16617 ms_handle_reset on v2:172.21.15.146:6838/3036767228 2024-09-12T03:03:38.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.244+0000 7f3a6b90b740 1 -- 172.21.15.5:0/1943094230 shutdown_connections 2024-09-12T03:03:38.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.244+0000 7f3a6b90b740 1 -- 172.21.15.5:0/1943094230 wait complete. 2024-09-12T03:03:38.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.244+0000 7f3a6b90b740 20 client.16617 trim_cache size 0 max 16384 2024-09-12T03:03:38.251 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-12T03:03:38.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.245+0000 7f3a69532640 1 -- 172.21.15.5:0/2124821715 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f3a500a2c00 msgr2=0x7f3a500a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:38.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.245+0000 7f3a69532640 1 --2- 172.21.15.5:0/2124821715 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f3a500a2c00 0x7f3a500a50c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c02fd80 tx=0x7f3a5c009380 comp rx=0 tx=0).stop 2024-09-12T03:03:38.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.245+0000 7f3a69532640 1 -- 172.21.15.5:0/2124821715 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a64100e20 msgr2=0x7f3a6419a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:38.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.245+0000 7f3a69532640 1 --2- 172.21.15.5:0/2124821715 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a64100e20 0x7f3a6419a690 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f3a5400d950 tx=0x7f3a5400de10 comp rx=0 tx=0).stop 2024-09-12T03:03:38.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.245+0000 7f3a69532640 1 -- 172.21.15.5:0/2124821715 shutdown_connections 2024-09-12T03:03:38.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.245+0000 7f3a69532640 1 --2- 172.21.15.5:0/2124821715 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f3a64102ef0 0x7f3a6419ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:38.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.245+0000 7f3a69532640 1 --2- 172.21.15.5:0/2124821715 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f3a64102570 0x7f3a6419abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:38.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.245+0000 7f3a69532640 1 --2- 172.21.15.5:0/2124821715 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f3a500a2c00 0x7f3a500a50c0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:38.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.245+0000 7f3a69532640 1 --2- 172.21.15.5:0/2124821715 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a64100e20 0x7f3a6419a690 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:38.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.245+0000 7f3a69532640 1 -- 172.21.15.5:0/2124821715 >> 172.21.15.5:0/2124821715 conn(0x7f3a640fc7d0 msgr2=0x7f3a6406a4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:38.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.245+0000 7f3a69532640 1 -- 172.21.15.5:0/2124821715 shutdown_connections 2024-09-12T03:03:38.253 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:38.245+0000 7f3a69532640 1 -- 172.21.15.5:0/2124821715 wait complete. 2024-09-12T03:03:38.975 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:03:38 smithi005 ceph-mon[31197]: pgmap v497: 97 pgs: 97 active+clean; 136 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 21 MiB/s rd, 2.5 MiB/s wr, 995 op/s 2024-09-12T03:03:39.013 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T03:03:39.165 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:03:38 smithi148 ceph-mon[35090]: pgmap v497: 97 pgs: 97 active+clean; 136 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 21 MiB/s rd, 2.5 MiB/s wr, 995 op/s 2024-09-12T03:03:39.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:03:38 smithi146 ceph-mon[35011]: pgmap v497: 97 pgs: 97 active+clean; 136 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 21 MiB/s rd, 2.5 MiB/s wr, 995 op/s 2024-09-12T03:03:39.327 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:03:39.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.867+0000 7f6541e71640 1 -- 172.21.15.5:0/1508161230 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f653c102d30 msgr2=0x7f653c103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:39.872 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.867+0000 7f6541e71640 1 --2- 172.21.15.5:0/1508161230 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f653c102d30 0x7f653c103110 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f652c009930 tx=0x7f652c02a420 comp rx=0 tx=0).stop 2024-09-12T03:03:39.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.867+0000 7f6541e71640 1 -- 172.21.15.5:0/1508161230 shutdown_connections 2024-09-12T03:03:39.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.867+0000 7f6541e71640 1 --2- 172.21.15.5:0/1508161230 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f653c1082a0 0x7f653c10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:39.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.867+0000 7f6541e71640 1 --2- 172.21.15.5:0/1508161230 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f653c1036e0 0x7f653c107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:39.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.867+0000 7f6541e71640 1 --2- 172.21.15.5:0/1508161230 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f653c102d30 0x7f653c103110 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:39.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.867+0000 7f6541e71640 1 -- 172.21.15.5:0/1508161230 >> 172.21.15.5:0/1508161230 conn(0x7f653c0fc7d0 msgr2=0x7f653c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:39.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.867+0000 7f6541e71640 1 -- 172.21.15.5:0/1508161230 shutdown_connections 2024-09-12T03:03:39.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.867+0000 7f6541e71640 1 -- 172.21.15.5:0/1508161230 wait complete. 2024-09-12T03:03:39.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.867+0000 7f6541e71640 1 Processor -- start 2024-09-12T03:03:39.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.868+0000 7f6541e71640 1 -- start start 2024-09-12T03:03:39.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.868+0000 7f6541e71640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f653c102d30 0x7f653c198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:39.873 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.868+0000 7f6541e71640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f653c1036e0 0x7f653c198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:39.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.868+0000 7f6541e71640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f653c1082a0 0x7f653c19ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:39.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.868+0000 7f6541e71640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f653c112090 con 0x7f653c102d30 2024-09-12T03:03:39.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.868+0000 7f6541e71640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f653c111f10 con 0x7f653c1082a0 2024-09-12T03:03:39.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.868+0000 7f6541e71640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f653c112210 con 0x7f653c1036e0 2024-09-12T03:03:39.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.868+0000 7f653b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f653c102d30 0x7f653c198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:39.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.868+0000 7f653b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f653c102d30 0x7f653c198590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49580/0 (socket says 172.21.15.5:49580) 2024-09-12T03:03:39.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.868+0000 7f653b7fe640 1 -- 172.21.15.5:0/629939473 learned_addr learned my addr 172.21.15.5:0/629939473 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:03:39.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.868+0000 7f653affd640 1 --2- 172.21.15.5:0/629939473 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f653c1036e0 0x7f653c198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:39.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.868+0000 7f653bfff640 1 --2- 172.21.15.5:0/629939473 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f653c1082a0 0x7f653c19ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:39.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.869+0000 7f653b7fe640 1 -- 172.21.15.5:0/629939473 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f653c1036e0 msgr2=0x7f653c198ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:39.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.869+0000 7f653b7fe640 1 --2- 172.21.15.5:0/629939473 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f653c1036e0 0x7f653c198ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:39.874 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.869+0000 7f653b7fe640 1 -- 172.21.15.5:0/629939473 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f653c1082a0 msgr2=0x7f653c19ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:39.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.869+0000 7f653b7fe640 1 --2- 172.21.15.5:0/629939473 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f653c1082a0 0x7f653c19ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:39.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.869+0000 7f653b7fe640 1 -- 172.21.15.5:0/629939473 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f653c19d540 con 0x7f653c102d30 2024-09-12T03:03:39.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.869+0000 7f653b7fe640 1 --2- 172.21.15.5:0/629939473 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f653c102d30 0x7f653c198590 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f652c009230 tx=0x7f652c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:39.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.869+0000 7f6538ff9640 1 -- 172.21.15.5:0/629939473 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f652c03f030 con 0x7f653c102d30 2024-09-12T03:03:39.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.869+0000 7f6538ff9640 1 -- 172.21.15.5:0/629939473 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f652c004440 con 0x7f653c102d30 2024-09-12T03:03:39.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.869+0000 7f6541e71640 1 -- 172.21.15.5:0/629939473 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f653c19d7d0 con 0x7f653c102d30 2024-09-12T03:03:39.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.869+0000 7f6538ff9640 1 -- 172.21.15.5:0/629939473 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f652c035070 con 0x7f653c102d30 2024-09-12T03:03:39.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.869+0000 7f6541e71640 1 -- 172.21.15.5:0/629939473 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f653c1a5110 con 0x7f653c102d30 2024-09-12T03:03:39.875 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.872+0000 7f6538ff9640 1 -- 172.21.15.5:0/629939473 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f652c042030 con 0x7f653c102d30 2024-09-12T03:03:39.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.873+0000 7f6538ff9640 1 --2- 172.21.15.5:0/629939473 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f65240a2c00 0x7f65240a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:39.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.873+0000 7f6538ff9640 1 -- 172.21.15.5:0/629939473 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f652c0ef2b0 con 0x7f653c102d30 2024-09-12T03:03:39.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.873+0000 7f653affd640 1 --2- 172.21.15.5:0/629939473 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f65240a2c00 0x7f65240a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:39.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.874+0000 7f654424a740 1 Processor -- start 2024-09-12T03:03:39.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.874+0000 7f654424a740 1 -- start start 2024-09-12T03:03:39.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.874+0000 7f653affd640 1 --2- 172.21.15.5:0/629939473 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f65240a2c00 0x7f65240a50c0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f653c0fee10 tx=0x7f6530005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:39.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.874+0000 7f654424a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d676144b40 0x55d676143090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:39.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.874+0000 7f654424a740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d6761435d0 0x55d676143a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:39.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.874+0000 7f654424a740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d67614e370 0x55d676150760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:39.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.874+0000 7f654424a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d6760a92f0 con 0x55d676144b40 2024-09-12T03:03:39.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.874+0000 7f653affd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d6761435d0 0x55d676143a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:39.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.874+0000 7f653affd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d6761435d0 0x55d676143a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.5:51728/0 (socket says 172.21.15.5:51728) 2024-09-12T03:03:39.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.874+0000 7f653affd640 1 -- 172.21.15.5:0/1490801430 learned_addr learned my addr 172.21.15.5:0/1490801430 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:03:39.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.874+0000 7f653b7fe640 1 --2- 172.21.15.5:0/1490801430 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d676144b40 0x55d676143090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:39.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.874+0000 7f653bfff640 1 --2- 172.21.15.5:0/1490801430 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d67614e370 0x55d676150760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:39.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.874+0000 7f654424a740 1 -- 172.21.15.5:0/1490801430 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55d676050cf0 con 0x55d67614e370 2024-09-12T03:03:39.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.874+0000 7f654424a740 1 -- 172.21.15.5:0/1490801430 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55d6760a52b0 con 0x55d6761435d0 2024-09-12T03:03:39.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.875+0000 7f653affd640 1 -- 172.21.15.5:0/1490801430 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d67614e370 msgr2=0x55d676150760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:39.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.875+0000 7f653affd640 1 --2- 172.21.15.5:0/1490801430 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d67614e370 0x55d676150760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:39.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.875+0000 7f653affd640 1 -- 172.21.15.5:0/1490801430 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d676144b40 msgr2=0x55d676143090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:39.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.875+0000 7f653affd640 1 --2- 172.21.15.5:0/1490801430 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d676144b40 0x55d676143090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:39.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.875+0000 7f653affd640 1 -- 172.21.15.5:0/1490801430 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6760353a0 con 0x55d6761435d0 2024-09-12T03:03:39.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.875+0000 7f653b7fe640 1 --2- 172.21.15.5:0/1490801430 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d676144b40 0x55d676143090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:03:39.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.875+0000 7f653affd640 1 --2- 172.21.15.5:0/1490801430 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d6761435d0 0x55d676143a10 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f653000e600 tx=0x7f6530014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d84417895bbe5d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:39.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.875+0000 7f6521ffb640 1 -- 172.21.15.5:0/1490801430 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6530004020 con 0x55d6761435d0 2024-09-12T03:03:39.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.875+0000 7f6521ffb640 1 -- 172.21.15.5:0/1490801430 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65300041c0 con 0x55d6761435d0 2024-09-12T03:03:39.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.875+0000 7f6521ffb640 1 -- 172.21.15.5:0/1490801430 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f653001c700 con 0x55d6761435d0 2024-09-12T03:03:39.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.876+0000 7f654424a740 1 -- 172.21.15.5:0/1490801430 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d6761435d0 msgr2=0x55d676143a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:39.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.876+0000 7f654424a740 1 --2- 172.21.15.5:0/1490801430 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d6761435d0 0x55d676143a10 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f653000e600 tx=0x7f6530014000 comp rx=0 tx=0).stop 2024-09-12T03:03:39.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.876+0000 7f654424a740 1 -- 172.21.15.5:0/1490801430 shutdown_connections 2024-09-12T03:03:39.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.876+0000 7f654424a740 1 --2- 172.21.15.5:0/1490801430 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d67614e370 0x55d676150760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:39.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.876+0000 7f654424a740 1 --2- 172.21.15.5:0/1490801430 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d6761435d0 0x55d676143a10 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:39.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.876+0000 7f654424a740 1 --2- 172.21.15.5:0/1490801430 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d676144b40 0x55d676143090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:39.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.876+0000 7f654424a740 1 -- 172.21.15.5:0/1490801430 >> 172.21.15.5:0/1490801430 conn(0x55d67610c310 msgr2=0x55d67613fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:39.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.876+0000 7f654424a740 1 -- 172.21.15.5:0/1490801430 shutdown_connections 2024-09-12T03:03:39.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.876+0000 7f654424a740 1 -- 172.21.15.5:0/1490801430 wait complete. 2024-09-12T03:03:39.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.877+0000 7f654424a740 1 -- start start 2024-09-12T03:03:39.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.877+0000 7f654424a740 1 Processor -- start 2024-09-12T03:03:39.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.877+0000 7f654424a740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d6761435d0 0x55d67614dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:39.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.877+0000 7f654424a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d676144b40 0x55d67614b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:39.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.877+0000 7f654424a740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d67614e370 0x55d67614b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:39.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.877+0000 7f654424a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55d676151070 con 0x55d676144b40 2024-09-12T03:03:39.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.877+0000 7f654424a740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55d676064060 con 0x55d6761435d0 2024-09-12T03:03:39.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.877+0000 7f654424a740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55d6761511f0 con 0x55d67614e370 2024-09-12T03:03:39.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.877+0000 7f654424a740 10 client.0 _ll_register_callbacks cb 0x55d675db78c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:03:39.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.878+0000 7f653affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d676144b40 0x55d67614b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:39.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.878+0000 7f653affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d676144b40 0x55d67614b030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49594/0 (socket says 172.21.15.5:49594) 2024-09-12T03:03:39.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.878+0000 7f653affd640 1 -- 172.21.15.5:0/2846869477 learned_addr learned my addr 172.21.15.5:0/2846869477 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:03:39.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.878+0000 7f653bfff640 1 --2- 172.21.15.5:0/2846869477 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d67614e370 0x55d67614b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:39.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.878+0000 7f653b7fe640 1 --2- 172.21.15.5:0/2846869477 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d6761435d0 0x55d67614dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:39.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.878+0000 7f653b7fe640 1 -- 172.21.15.5:0/2846869477 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d67614e370 msgr2=0x55d67614b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:39.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.878+0000 7f653b7fe640 1 --2- 172.21.15.5:0/2846869477 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d67614e370 0x55d67614b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:39.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.878+0000 7f653b7fe640 1 -- 172.21.15.5:0/2846869477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d676144b40 msgr2=0x55d67614b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:39.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.878+0000 7f653b7fe640 1 --2- 172.21.15.5:0/2846869477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d676144b40 0x55d67614b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:39.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.878+0000 7f653b7fe640 1 -- 172.21.15.5:0/2846869477 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d675e1a3c0 con 0x55d6761435d0 2024-09-12T03:03:39.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.878+0000 7f653affd640 1 --2- 172.21.15.5:0/2846869477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d676144b40 0x55d67614b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:03:39.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.878+0000 7f653bfff640 1 --2- 172.21.15.5:0/2846869477 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d67614e370 0x55d67614b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:03:39.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.878+0000 7f653b7fe640 1 --2- 172.21.15.5:0/2846869477 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d6761435d0 0x55d67614dd80 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f652c009a50 tx=0x7f652c0bb5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ab66f33f3a51444 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:39.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.878+0000 7f651affd640 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2024-09-12T03:03:39.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.878+0000 7f651affd640 1 -- 172.21.15.5:0/2846869477 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f652c049040 con 0x55d6761435d0 2024-09-12T03:03:39.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.878+0000 7f651affd640 1 -- 172.21.15.5:0/2846869477 <== mon.2 v2:172.21.15.146:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f652c0b8240 con 0x55d6761435d0 2024-09-12T03:03:39.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.879+0000 7f654424a740 1 -- 172.21.15.5:0/2846869477 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65300041c0 con 0x55d6761435d0 2024-09-12T03:03:39.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.879+0000 7f651affd640 1 -- 172.21.15.5:0/2846869477 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f652c031bd0 con 0x55d6761435d0 2024-09-12T03:03:39.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.879+0000 7f654424a740 10 client.26018 fetch_fsmap learned FSMap version 31 2024-09-12T03:03:39.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.879+0000 7f654424a740 1 -- 172.21.15.5:0/2846869477 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55d67615cb40 con 0x55d6761435d0 2024-09-12T03:03:39.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.879+0000 7f651affd640 1 -- 172.21.15.5:0/2846869477 <== mon.2 v2:172.21.15.146:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65300041c0 con 0x55d6761435d0 2024-09-12T03:03:39.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.879+0000 7f651affd640 1 -- 172.21.15.5:0/2846869477 <== mon.2 v2:172.21.15.146:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f652c0bb020 con 0x55d6761435d0 2024-09-12T03:03:39.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.879+0000 7f654424a740 10 client.26018 fetch_fsmap finished waiting for FSMap version 31 2024-09-12T03:03:39.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.879+0000 7f654424a740 10 client.26018 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-09-12T03:03:39.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.879+0000 7f654424a740 20 client.26018 populate_metadata read hostname 'smithi005' 2024-09-12T03:03:39.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.879+0000 7f654424a740 1 --2- 172.21.15.5:0/2846869477 >> [v2:172.21.15.148:6834/4195086427,v1:172.21.15.148:6835/4195086427] conn(0x55d67615dff0 0x55d676160400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:39.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.879+0000 7f654424a740 4 client.26018 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:03:39.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.879+0000 7f654424a740 1 -- 172.21.15.5:0/2846869477 --> [v2:172.21.15.148:6834/4195086427,v1:172.21.15.148:6835/4195086427] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d6760b6860 con 0x55d67615dff0 2024-09-12T03:03:39.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.879+0000 7f653affd640 1 --2- 172.21.15.5:0/2846869477 >> [v2:172.21.15.148:6834/4195086427,v1:172.21.15.148:6835/4195086427] conn(0x55d67615dff0 0x55d676160400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:39.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.880+0000 7f653affd640 1 --2- 172.21.15.5:0/2846869477 >> [v2:172.21.15.148:6834/4195086427,v1:172.21.15.148:6835/4195086427] conn(0x55d67615dff0 0x55d676160400 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8ff042b20d96bca4 server_cookie=e4d24c24c1c46ce6 in_seq=0 out_seq=0 2024-09-12T03:03:39.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.880+0000 7f651affd640 10 client.26018 ms_handle_connect on v2:172.21.15.148:6834/4195086427 2024-09-12T03:03:39.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.880+0000 7f651affd640 1 -- 172.21.15.5:0/2846869477 <== mds.2 v2:172.21.15.148:6834/4195086427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d6760b6860 con 0x55d67615dff0 2024-09-12T03:03:39.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.880+0000 7f651affd640 10 client.26018 handle_command_reply: tid=0 2024-09-12T03:03:39.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.881+0000 7f654424a740 1 client.26018 shutdown 2024-09-12T03:03:39.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.881+0000 7f654424a740 1 -- 172.21.15.5:0/2846869477 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d6761435d0 msgr2=0x55d67614dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:39.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.881+0000 7f654424a740 1 --2- 172.21.15.5:0/2846869477 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d6761435d0 0x55d67614dd80 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f652c009a50 tx=0x7f652c0bb5f0 comp rx=0 tx=0).stop 2024-09-12T03:03:39.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.881+0000 7f654424a740 1 -- 172.21.15.5:0/2846869477 shutdown_connections 2024-09-12T03:03:39.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.881+0000 7f654424a740 1 --2- 172.21.15.5:0/2846869477 >> [v2:172.21.15.148:6834/4195086427,v1:172.21.15.148:6835/4195086427] conn(0x55d67615dff0 0x55d676160400 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:39.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.881+0000 7f654424a740 1 --2- 172.21.15.5:0/2846869477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d676144b40 0x55d67614b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:39.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.881+0000 7f654424a740 1 --2- 172.21.15.5:0/2846869477 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d67614e370 0x55d67614b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:39.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.881+0000 7f654424a740 1 --2- 172.21.15.5:0/2846869477 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d6761435d0 0x55d67614dd80 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:39.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.881+0000 7f654424a740 1 -- 172.21.15.5:0/2846869477 >> 172.21.15.5:0/2846869477 conn(0x55d67610c310 msgr2=0x55d676145960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:39.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.881+0000 7f651affd640 0 client.26018 ms_handle_reset on v2:172.21.15.148:6834/4195086427 2024-09-12T03:03:39.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.881+0000 7f654424a740 1 -- 172.21.15.5:0/2846869477 shutdown_connections 2024-09-12T03:03:39.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.881+0000 7f654424a740 1 -- 172.21.15.5:0/2846869477 wait complete. 2024-09-12T03:03:39.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.881+0000 7f654424a740 20 client.26018 trim_cache size 0 max 16384 2024-09-12T03:03:39.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.900+0000 7f654424a740 1 Processor -- start 2024-09-12T03:03:39.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.900+0000 7f654424a740 1 -- start start 2024-09-12T03:03:39.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.901+0000 7f654424a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d6761435d0 0x55d67613fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:39.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.901+0000 7f654424a740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d676144b40 0x55d676144300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:39.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.901+0000 7f654424a740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d67614e370 0x55d67615a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:39.901 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.901+0000 7f654424a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f653c1ad570 con 0x55d6761435d0 2024-09-12T03:03:39.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.901+0000 7f654424a740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f653c1af0b0 con 0x55d676144b40 2024-09-12T03:03:39.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.901+0000 7f654424a740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f653c1adc40 con 0x55d67614e370 2024-09-12T03:03:39.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.901+0000 7f653affd640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d676144b40 0x55d676144300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:39.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.901+0000 7f653affd640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d676144b40 0x55d676144300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.5:50168/0 (socket says 172.21.15.5:50168) 2024-09-12T03:03:39.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.901+0000 7f653affd640 1 -- 172.21.15.5:0/700306402 learned_addr learned my addr 172.21.15.5:0/700306402 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:03:39.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.901+0000 7f653b7fe640 1 --2- 172.21.15.5:0/700306402 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d6761435d0 0x55d67613fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:39.902 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.901+0000 7f653bfff640 1 --2- 172.21.15.5:0/700306402 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d67614e370 0x55d67615a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:39.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.901+0000 7f653affd640 1 -- 172.21.15.5:0/700306402 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d67614e370 msgr2=0x55d67615a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:39.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.901+0000 7f653affd640 1 --2- 172.21.15.5:0/700306402 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d67614e370 0x55d67615a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:39.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.901+0000 7f653affd640 1 -- 172.21.15.5:0/700306402 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d6761435d0 msgr2=0x55d67613fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:39.903 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.901+0000 7f653affd640 1 --2- 172.21.15.5:0/700306402 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d6761435d0 0x55d67613fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:39.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.901+0000 7f653affd640 1 -- 172.21.15.5:0/700306402 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d67614e0f0 con 0x55d676144b40 2024-09-12T03:03:39.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.902+0000 7f653affd640 1 --2- 172.21.15.5:0/700306402 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d676144b40 0x55d676144300 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f6530009730 tx=0x7f6530004000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9647f0a840f42fe7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:39.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.902+0000 7f65217fa640 1 -- 172.21.15.5:0/700306402 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65300052f0 con 0x55d676144b40 2024-09-12T03:03:39.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.902+0000 7f65217fa640 1 -- 172.21.15.5:0/700306402 <== mon.2 v2:172.21.15.146:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6530005490 con 0x55d676144b40 2024-09-12T03:03:39.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.902+0000 7f65217fa640 1 -- 172.21.15.5:0/700306402 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6530026a80 con 0x55d676144b40 2024-09-12T03:03:39.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.902+0000 7f654424a740 1 -- 172.21.15.5:0/700306402 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d676144b40 msgr2=0x55d676144300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:39.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.902+0000 7f654424a740 1 --2- 172.21.15.5:0/700306402 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d676144b40 0x55d676144300 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f6530009730 tx=0x7f6530004000 comp rx=0 tx=0).stop 2024-09-12T03:03:39.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.903+0000 7f654424a740 1 -- 172.21.15.5:0/700306402 shutdown_connections 2024-09-12T03:03:39.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.903+0000 7f654424a740 1 --2- 172.21.15.5:0/700306402 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d67614e370 0x55d67615a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:39.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.903+0000 7f654424a740 1 --2- 172.21.15.5:0/700306402 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d676144b40 0x55d676144300 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:39.904 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.903+0000 7f654424a740 1 --2- 172.21.15.5:0/700306402 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d6761435d0 0x55d67613fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:39.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.903+0000 7f654424a740 1 -- 172.21.15.5:0/700306402 >> 172.21.15.5:0/700306402 conn(0x55d67610c310 msgr2=0x55d67613f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:39.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.903+0000 7f654424a740 1 -- 172.21.15.5:0/700306402 shutdown_connections 2024-09-12T03:03:39.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.903+0000 7f654424a740 1 -- 172.21.15.5:0/700306402 wait complete. 2024-09-12T03:03:39.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.903+0000 7f654424a740 1 -- start start 2024-09-12T03:03:39.905 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.903+0000 7f654424a740 1 Processor -- start 2024-09-12T03:03:39.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.904+0000 7f654424a740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d6761435d0 0x55d676156660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:39.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.904+0000 7f654424a740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d676144b40 0x55d676156ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:39.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.904+0000 7f654424a740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d67614e370 0x55d676151a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:39.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.904+0000 7f654424a740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f653c1ae9e0 con 0x55d676144b40 2024-09-12T03:03:39.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.904+0000 7f654424a740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f653c1ae310 con 0x55d6761435d0 2024-09-12T03:03:39.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.904+0000 7f654424a740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f653c1ab550 con 0x55d67614e370 2024-09-12T03:03:39.908 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.904+0000 7f654424a740 10 client.0 _ll_register_callbacks cb 0x7f652c0bb9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:03:39.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.904+0000 7f653b7fe640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d6761435d0 0x55d676156660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:39.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.904+0000 7f653affd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d676144b40 0x55d676156ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:39.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.904+0000 7f653b7fe640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d6761435d0 0x55d676156660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.5:50176/0 (socket says 172.21.15.5:50176) 2024-09-12T03:03:39.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.904+0000 7f653b7fe640 1 -- 172.21.15.5:0/1294564955 learned_addr learned my addr 172.21.15.5:0/1294564955 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:03:39.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.904+0000 7f653bfff640 1 --2- 172.21.15.5:0/1294564955 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d67614e370 0x55d676151a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:39.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.904+0000 7f653b7fe640 1 -- 172.21.15.5:0/1294564955 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d67614e370 msgr2=0x55d676151a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:39.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.904+0000 7f653b7fe640 1 --2- 172.21.15.5:0/1294564955 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d67614e370 0x55d676151a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:39.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.904+0000 7f653b7fe640 1 -- 172.21.15.5:0/1294564955 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d676144b40 msgr2=0x55d676156ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:39.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.904+0000 7f653b7fe640 1 --2- 172.21.15.5:0/1294564955 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d676144b40 0x55d676156ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:39.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.904+0000 7f653b7fe640 1 -- 172.21.15.5:0/1294564955 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d676152250 con 0x55d6761435d0 2024-09-12T03:03:39.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.904+0000 7f653b7fe640 1 --2- 172.21.15.5:0/1294564955 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d6761435d0 0x55d676156660 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f652c0b8030 tx=0x7f652c0b7920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b32c89c295611eb3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:39.909 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.904+0000 7f651affd640 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2024-09-12T03:03:39.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.905+0000 7f651affd640 1 -- 172.21.15.5:0/1294564955 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f652c049040 con 0x55d6761435d0 2024-09-12T03:03:39.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.905+0000 7f651affd640 1 -- 172.21.15.5:0/1294564955 <== mon.2 v2:172.21.15.146:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f652c0c4300 con 0x55d6761435d0 2024-09-12T03:03:39.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.905+0000 7f654424a740 1 -- 172.21.15.5:0/1294564955 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6530005490 con 0x55d6761435d0 2024-09-12T03:03:39.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.905+0000 7f651affd640 1 -- 172.21.15.5:0/1294564955 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f652c0bb710 con 0x55d6761435d0 2024-09-12T03:03:39.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.905+0000 7f654424a740 10 client.26024 fetch_fsmap learned FSMap version 31 2024-09-12T03:03:39.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.905+0000 7f654424a740 1 -- 172.21.15.5:0/1294564955 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55d6761ccc20 con 0x55d6761435d0 2024-09-12T03:03:39.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.905+0000 7f651affd640 1 -- 172.21.15.5:0/1294564955 <== mon.2 v2:172.21.15.146:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f652c0bce50 con 0x55d6761435d0 2024-09-12T03:03:39.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.905+0000 7f651affd640 1 -- 172.21.15.5:0/1294564955 <== mon.2 v2:172.21.15.146:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f652c03bc30 con 0x55d6761435d0 2024-09-12T03:03:39.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.905+0000 7f654424a740 10 client.26024 fetch_fsmap finished waiting for FSMap version 31 2024-09-12T03:03:39.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.905+0000 7f654424a740 10 client.26024 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-09-12T03:03:39.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.906+0000 7f654424a740 20 client.26024 populate_metadata read hostname 'smithi005' 2024-09-12T03:03:39.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.906+0000 7f654424a740 1 --2- 172.21.15.5:0/1294564955 >> [v2:172.21.15.148:6834/4195086427,v1:172.21.15.148:6835/4195086427] conn(0x55d6761cd1b0 0x55d6761cf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:39.910 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.906+0000 7f654424a740 4 client.26024 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:03:39.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.906+0000 7f654424a740 1 -- 172.21.15.5:0/1294564955 --> [v2:172.21.15.148:6834/4195086427,v1:172.21.15.148:6835/4195086427] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f6530005490 con 0x55d6761cd1b0 2024-09-12T03:03:39.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.906+0000 7f653affd640 1 --2- 172.21.15.5:0/1294564955 >> [v2:172.21.15.148:6834/4195086427,v1:172.21.15.148:6835/4195086427] conn(0x55d6761cd1b0 0x55d6761cf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:39.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.907+0000 7f651affd640 10 client.26024 ms_handle_connect on v2:172.21.15.148:6834/4195086427 2024-09-12T03:03:39.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.907+0000 7f653affd640 1 --2- 172.21.15.5:0/1294564955 >> [v2:172.21.15.148:6834/4195086427,v1:172.21.15.148:6835/4195086427] conn(0x55d6761cd1b0 0x55d6761cf5a0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=401647406ce7a0d5 server_cookie=fd2f4c51e3bddb8b in_seq=0 out_seq=0 2024-09-12T03:03:39.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.907+0000 7f651affd640 1 -- 172.21.15.5:0/1294564955 <== mds.2 v2:172.21.15.148:6834/4195086427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f6530005490 con 0x55d6761cd1b0 2024-09-12T03:03:39.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.907+0000 7f651affd640 10 client.26024 handle_command_reply: tid=0 2024-09-12T03:03:39.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.907+0000 7f654424a740 1 client.26024 shutdown 2024-09-12T03:03:39.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.907+0000 7f654424a740 1 -- 172.21.15.5:0/1294564955 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d6761435d0 msgr2=0x55d676156660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:39.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.907+0000 7f654424a740 1 --2- 172.21.15.5:0/1294564955 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d6761435d0 0x55d676156660 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f652c0b8030 tx=0x7f652c0b7920 comp rx=0 tx=0).stop 2024-09-12T03:03:39.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.907+0000 7f654424a740 1 -- 172.21.15.5:0/1294564955 shutdown_connections 2024-09-12T03:03:39.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.907+0000 7f654424a740 1 --2- 172.21.15.5:0/1294564955 >> [v2:172.21.15.148:6834/4195086427,v1:172.21.15.148:6835/4195086427] conn(0x55d6761cd1b0 0x55d6761cf5a0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:39.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.908+0000 7f654424a740 1 --2- 172.21.15.5:0/1294564955 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d676144b40 0x55d676156ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:39.911 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.908+0000 7f651affd640 0 client.26024 ms_handle_reset on v2:172.21.15.148:6834/4195086427 2024-09-12T03:03:39.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.908+0000 7f654424a740 1 --2- 172.21.15.5:0/1294564955 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55d67614e370 0x55d676151a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:39.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.908+0000 7f654424a740 1 --2- 172.21.15.5:0/1294564955 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55d6761435d0 0x55d676156660 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:39.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.908+0000 7f654424a740 1 -- 172.21.15.5:0/1294564955 >> 172.21.15.5:0/1294564955 conn(0x55d67610c310 msgr2=0x55d6761494d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:39.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.908+0000 7f654424a740 1 -- 172.21.15.5:0/1294564955 shutdown_connections 2024-09-12T03:03:39.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.908+0000 7f654424a740 1 -- 172.21.15.5:0/1294564955 wait complete. 2024-09-12T03:03:39.912 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.908+0000 7f654424a740 20 client.26024 trim_cache size 0 max 16384 2024-09-12T03:03:39.912 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-12T03:03:39.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.909+0000 7f6541e71640 1 -- 172.21.15.5:0/629939473 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f65240a2c00 msgr2=0x7f65240a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:39.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.909+0000 7f6541e71640 1 --2- 172.21.15.5:0/629939473 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f65240a2c00 0x7f65240a50c0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f653c0fee10 tx=0x7f6530005f90 comp rx=0 tx=0).stop 2024-09-12T03:03:39.913 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.909+0000 7f6541e71640 1 -- 172.21.15.5:0/629939473 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f653c102d30 msgr2=0x7f653c198590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:39.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.909+0000 7f6541e71640 1 --2- 172.21.15.5:0/629939473 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f653c102d30 0x7f653c198590 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f652c009230 tx=0x7f652c004040 comp rx=0 tx=0).stop 2024-09-12T03:03:39.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.909+0000 7f6541e71640 1 -- 172.21.15.5:0/629939473 shutdown_connections 2024-09-12T03:03:39.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.909+0000 7f6541e71640 1 --2- 172.21.15.5:0/629939473 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f653c1082a0 0x7f653c19ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:39.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.909+0000 7f6541e71640 1 --2- 172.21.15.5:0/629939473 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f653c1036e0 0x7f653c198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:39.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.909+0000 7f6541e71640 1 --2- 172.21.15.5:0/629939473 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f65240a2c00 0x7f65240a50c0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:39.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.909+0000 7f6541e71640 1 --2- 172.21.15.5:0/629939473 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f653c102d30 0x7f653c198590 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:39.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.909+0000 7f6541e71640 1 -- 172.21.15.5:0/629939473 >> 172.21.15.5:0/629939473 conn(0x7f653c0fc7d0 msgr2=0x7f653c105d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:39.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.910+0000 7f6541e71640 1 -- 172.21.15.5:0/629939473 shutdown_connections 2024-09-12T03:03:39.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:39.910+0000 7f6541e71640 1 -- 172.21.15.5:0/629939473 wait complete. 2024-09-12T03:03:40.584 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T03:03:40.903 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:03:41.029 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:03:40 smithi005 ceph-mon[31197]: pgmap v498: 97 pgs: 97 active+clean; 142 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 2.5 MiB/s wr, 1.01k op/s 2024-09-12T03:03:41.165 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:03:40 smithi148 ceph-mon[35090]: pgmap v498: 97 pgs: 97 active+clean; 142 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 2.5 MiB/s wr, 1.01k op/s 2024-09-12T03:03:41.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:03:40 smithi146 ceph-mon[35011]: pgmap v498: 97 pgs: 97 active+clean; 142 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 2.5 MiB/s wr, 1.01k op/s 2024-09-12T03:03:41.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.381+0000 7f8bcdf34640 1 -- 172.21.15.5:0/4190422209 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bc8102570 msgr2=0x7f8bc81029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:41.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.381+0000 7f8bcdf34640 1 --2- 172.21.15.5:0/4190422209 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bc8102570 0x7f8bc81029b0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc009900 tx=0x7f8bbc02a480 comp rx=0 tx=0).stop 2024-09-12T03:03:41.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.381+0000 7f8bcdf34640 1 -- 172.21.15.5:0/4190422209 shutdown_connections 2024-09-12T03:03:41.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.381+0000 7f8bcdf34640 1 --2- 172.21.15.5:0/4190422209 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bc8102570 0x7f8bc81029b0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:41.392 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.381+0000 7f8bcdf34640 1 --2- 172.21.15.5:0/4190422209 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8bc8102ef0 0x7f8bc810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:41.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.381+0000 7f8bcdf34640 1 --2- 172.21.15.5:0/4190422209 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8bc8100e20 0x7f8bc8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:41.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.381+0000 7f8bcdf34640 1 -- 172.21.15.5:0/4190422209 >> 172.21.15.5:0/4190422209 conn(0x7f8bc80fc7d0 msgr2=0x7f8bc80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:41.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.381+0000 7f8bcdf34640 1 -- 172.21.15.5:0/4190422209 shutdown_connections 2024-09-12T03:03:41.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.381+0000 7f8bcdf34640 1 -- 172.21.15.5:0/4190422209 wait complete. 2024-09-12T03:03:41.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.382+0000 7f8bcdf34640 1 Processor -- start 2024-09-12T03:03:41.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.382+0000 7f8bcdf34640 1 -- start start 2024-09-12T03:03:41.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.383+0000 7f8bcdf34640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bc8100e20 0x7f8bc81993a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:41.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.383+0000 7f8bcdf34640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8bc8102570 0x7f8bc81998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:41.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.383+0000 7f8bcdf34640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8bc8102ef0 0x7f8bc81a4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:41.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.383+0000 7f8bcdf34640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc8110fd0 con 0x7f8bc8100e20 2024-09-12T03:03:41.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.383+0000 7f8bcdf34640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc8110e50 con 0x7f8bc8102ef0 2024-09-12T03:03:41.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.383+0000 7f8bcdf34640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc8111150 con 0x7f8bc8102570 2024-09-12T03:03:41.393 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.383+0000 7f8bc77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bc8100e20 0x7f8bc81993a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:41.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.383+0000 7f8bc77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bc8100e20 0x7f8bc81993a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49634/0 (socket says 172.21.15.5:49634) 2024-09-12T03:03:41.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.383+0000 7f8bc77fe640 1 -- 172.21.15.5:0/2309469653 learned_addr learned my addr 172.21.15.5:0/2309469653 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:03:41.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.383+0000 7f8bc7fff640 1 --2- 172.21.15.5:0/2309469653 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8bc8102ef0 0x7f8bc81a4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:41.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.383+0000 7f8bc6ffd640 1 --2- 172.21.15.5:0/2309469653 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8bc8102570 0x7f8bc81998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:41.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.383+0000 7f8bc77fe640 1 -- 172.21.15.5:0/2309469653 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8bc8102570 msgr2=0x7f8bc81998e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:41.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.383+0000 7f8bc77fe640 1 --2- 172.21.15.5:0/2309469653 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8bc8102570 0x7f8bc81998e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:41.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.383+0000 7f8bc77fe640 1 -- 172.21.15.5:0/2309469653 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8bc8102ef0 msgr2=0x7f8bc81a4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:41.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.383+0000 7f8bc77fe640 1 --2- 172.21.15.5:0/2309469653 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8bc8102ef0 0x7f8bc81a4420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:41.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.383+0000 7f8bc77fe640 1 -- 172.21.15.5:0/2309469653 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bc81a4960 con 0x7f8bc8100e20 2024-09-12T03:03:41.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.384+0000 7f8bc77fe640 1 --2- 172.21.15.5:0/2309469653 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bc8100e20 0x7f8bc81993a0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f8bb800d950 tx=0x7f8bb800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:41.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.384+0000 7f8bc4ff9640 1 -- 172.21.15.5:0/2309469653 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb8017070 con 0x7f8bc8100e20 2024-09-12T03:03:41.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.384+0000 7f8bc4ff9640 1 -- 172.21.15.5:0/2309469653 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8bb8009c30 con 0x7f8bc8100e20 2024-09-12T03:03:41.394 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.384+0000 7f8bc4ff9640 1 -- 172.21.15.5:0/2309469653 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb800f6b0 con 0x7f8bc8100e20 2024-09-12T03:03:41.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.384+0000 7f8bcdf34640 1 -- 172.21.15.5:0/2309469653 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bc81a4b30 con 0x7f8bc8100e20 2024-09-12T03:03:41.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.384+0000 7f8bcdf34640 1 -- 172.21.15.5:0/2309469653 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bc81a4ff0 con 0x7f8bc8100e20 2024-09-12T03:03:41.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.387+0000 7f8bc4ff9640 1 -- 172.21.15.5:0/2309469653 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f8bb80109d0 con 0x7f8bc8100e20 2024-09-12T03:03:41.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.387+0000 7f8bd030d740 1 Processor -- start 2024-09-12T03:03:41.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.387+0000 7f8bc4ff9640 1 --2- 172.21.15.5:0/2309469653 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f8bb00a2b80 0x7f8bb00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:41.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.387+0000 7f8bd030d740 1 -- start start 2024-09-12T03:03:41.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.387+0000 7f8bc4ff9640 1 -- 172.21.15.5:0/2309469653 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f8bb80cebd0 con 0x7f8bc8100e20 2024-09-12T03:03:41.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.387+0000 7f8bc7fff640 1 --2- 172.21.15.5:0/2309469653 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f8bb00a2b80 0x7f8bb00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:41.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.388+0000 7f8bd030d740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564dc0dd9b40 0x564dc0dd8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:41.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.388+0000 7f8bd030d740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564dc0dd85d0 0x564dc0dd8a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:41.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.388+0000 7f8bd030d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564dc0de3370 0x564dc0de5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:41.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.388+0000 7f8bd030d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564dc0d3e2f0 con 0x564dc0de3370 2024-09-12T03:03:41.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.388+0000 7f8bd030d740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x564dc0ce5cf0 con 0x564dc0dd85d0 2024-09-12T03:03:41.395 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.388+0000 7f8bd030d740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x564dc0d3a2b0 con 0x564dc0dd9b40 2024-09-12T03:03:41.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.388+0000 7f8bc77fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564dc0dd9b40 0x564dc0dd8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:41.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.388+0000 7f8bc77fe640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564dc0dd9b40 0x564dc0dd8090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.5:51784/0 (socket says 172.21.15.5:51784) 2024-09-12T03:03:41.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.388+0000 7f8bc77fe640 1 -- 172.21.15.5:0/1717421752 learned_addr learned my addr 172.21.15.5:0/1717421752 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:03:41.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.388+0000 7f8bc6ffd640 1 --2- 172.21.15.5:0/1717421752 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564dc0dd85d0 0x564dc0dd8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:41.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.388+0000 7f8bc7fff640 1 --2- 172.21.15.5:0/1717421752 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564dc0de3370 0x564dc0de5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:41.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.388+0000 7f8bc77fe640 1 -- 172.21.15.5:0/1717421752 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564dc0dd85d0 msgr2=0x564dc0dd8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:41.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.388+0000 7f8bc77fe640 1 --2- 172.21.15.5:0/1717421752 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564dc0dd85d0 0x564dc0dd8a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:41.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.388+0000 7f8bc77fe640 1 -- 172.21.15.5:0/1717421752 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564dc0de3370 msgr2=0x564dc0de5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:41.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.388+0000 7f8bc77fe640 1 --2- 172.21.15.5:0/1717421752 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564dc0de3370 0x564dc0de5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:41.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.388+0000 7f8bc77fe640 1 -- 172.21.15.5:0/1717421752 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564dc0cca3a0 con 0x564dc0dd9b40 2024-09-12T03:03:41.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.388+0000 7f8bc7fff640 1 --2- 172.21.15.5:0/2309469653 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f8bb00a2b80 0x7f8bb00a5040 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f8bc0005fd0 tx=0x7f8bc0011810 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:41.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.388+0000 7f8bc77fe640 1 --2- 172.21.15.5:0/1717421752 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564dc0dd9b40 0x564dc0dd8090 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f8bb8000c00 tx=0x7f8bb8005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bbb6dd9156404e4c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:41.396 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.389+0000 7f8badffb640 1 -- 172.21.15.5:0/1717421752 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb8020040 con 0x564dc0dd9b40 2024-09-12T03:03:41.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.389+0000 7f8badffb640 1 -- 172.21.15.5:0/1717421752 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8bb809f7f0 con 0x564dc0dd9b40 2024-09-12T03:03:41.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.389+0000 7f8badffb640 1 -- 172.21.15.5:0/1717421752 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb809a940 con 0x564dc0dd9b40 2024-09-12T03:03:41.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.389+0000 7f8bd030d740 1 -- 172.21.15.5:0/1717421752 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564dc0dd9b40 msgr2=0x564dc0dd8090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:41.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.389+0000 7f8bd030d740 1 --2- 172.21.15.5:0/1717421752 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564dc0dd9b40 0x564dc0dd8090 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f8bb8000c00 tx=0x7f8bb8005880 comp rx=0 tx=0).stop 2024-09-12T03:03:41.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.390+0000 7f8bd030d740 1 -- 172.21.15.5:0/1717421752 shutdown_connections 2024-09-12T03:03:41.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.390+0000 7f8bd030d740 1 --2- 172.21.15.5:0/1717421752 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564dc0de3370 0x564dc0de5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:41.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.390+0000 7f8bd030d740 1 --2- 172.21.15.5:0/1717421752 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564dc0dd85d0 0x564dc0dd8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:41.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.390+0000 7f8bd030d740 1 --2- 172.21.15.5:0/1717421752 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564dc0dd9b40 0x564dc0dd8090 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:41.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.390+0000 7f8bd030d740 1 -- 172.21.15.5:0/1717421752 >> 172.21.15.5:0/1717421752 conn(0x564dc0da1310 msgr2=0x564dc0dd4c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:41.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.390+0000 7f8bd030d740 1 -- 172.21.15.5:0/1717421752 shutdown_connections 2024-09-12T03:03:41.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.390+0000 7f8bd030d740 1 -- 172.21.15.5:0/1717421752 wait complete. 2024-09-12T03:03:41.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.390+0000 7f8bd030d740 1 -- start start 2024-09-12T03:03:41.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.390+0000 7f8bd030d740 1 Processor -- start 2024-09-12T03:03:41.397 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.390+0000 7f8bd030d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564dc0dd85d0 0x564dc0de2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:41.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.390+0000 7f8bd030d740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564dc0dd9b40 0x564dc0de00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:41.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.390+0000 7f8bd030d740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564dc0de3370 0x564dc0de0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:41.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.390+0000 7f8bd030d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564dc0de5fb0 con 0x564dc0dd85d0 2024-09-12T03:03:41.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.390+0000 7f8bd030d740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x564dc0cf9060 con 0x564dc0de3370 2024-09-12T03:03:41.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.390+0000 7f8bd030d740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x564dc0de6130 con 0x564dc0dd9b40 2024-09-12T03:03:41.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.390+0000 7f8bd030d740 10 client.0 _ll_register_callbacks cb 0x564dc0a4c8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:03:41.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.391+0000 7f8bc77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564dc0dd85d0 0x564dc0de2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:41.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.391+0000 7f8bc6ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564dc0dd9b40 0x564dc0de00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:41.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.391+0000 7f8bc77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564dc0dd85d0 0x564dc0de2e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49650/0 (socket says 172.21.15.5:49650) 2024-09-12T03:03:41.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.391+0000 7f8bc77fe640 1 -- 172.21.15.5:0/2094829930 learned_addr learned my addr 172.21.15.5:0/2094829930 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:03:41.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.391+0000 7f8bc7fff640 1 --2- 172.21.15.5:0/2094829930 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564dc0de3370 0x564dc0de0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:41.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.391+0000 7f8bc6ffd640 1 -- 172.21.15.5:0/2094829930 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564dc0de3370 msgr2=0x564dc0de0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:41.398 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.391+0000 7f8bc6ffd640 1 --2- 172.21.15.5:0/2094829930 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564dc0de3370 0x564dc0de0850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:41.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.391+0000 7f8bc6ffd640 1 -- 172.21.15.5:0/2094829930 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564dc0dd85d0 msgr2=0x564dc0de2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:41.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.391+0000 7f8bc6ffd640 1 --2- 172.21.15.5:0/2094829930 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564dc0dd85d0 0x564dc0de2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:41.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.391+0000 7f8bc6ffd640 1 -- 172.21.15.5:0/2094829930 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564dc0aaf3c0 con 0x564dc0dd9b40 2024-09-12T03:03:41.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.391+0000 7f8bc77fe640 1 --2- 172.21.15.5:0/2094829930 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564dc0dd85d0 0x564dc0de2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:03:41.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.391+0000 7f8bc6ffd640 1 --2- 172.21.15.5:0/2094829930 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564dc0dd9b40 0x564dc0de00c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc02aa70 tx=0x7f8bbc0044f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=822639bf2e9bbf41 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:41.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.391+0000 7f8ba6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-09-12T03:03:41.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.391+0000 7f8ba6ffd640 1 -- 172.21.15.5:0/2094829930 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bbc035040 con 0x564dc0dd9b40 2024-09-12T03:03:41.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.392+0000 7f8ba6ffd640 1 -- 172.21.15.5:0/2094829930 <== mon.1 v2:172.21.15.148:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f8bbc005510 con 0x564dc0dd9b40 2024-09-12T03:03:41.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.392+0000 7f8bd030d740 1 -- 172.21.15.5:0/2094829930 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8bb809f7f0 con 0x564dc0dd9b40 2024-09-12T03:03:41.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.392+0000 7f8ba6ffd640 1 -- 172.21.15.5:0/2094829930 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f8bbc0046c0 con 0x564dc0dd9b40 2024-09-12T03:03:41.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.392+0000 7f8bd030d740 10 client.26128 fetch_fsmap learned FSMap version 31 2024-09-12T03:03:41.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.392+0000 7f8bd030d740 1 -- 172.21.15.5:0/2094829930 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x564dc0df1b00 con 0x564dc0dd9b40 2024-09-12T03:03:41.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.392+0000 7f8ba6ffd640 1 -- 172.21.15.5:0/2094829930 <== mon.1 v2:172.21.15.148:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8bb809f7f0 con 0x564dc0dd9b40 2024-09-12T03:03:41.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.392+0000 7f8ba6ffd640 1 -- 172.21.15.5:0/2094829930 <== mon.1 v2:172.21.15.148:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f8bbc003bc0 con 0x564dc0dd9b40 2024-09-12T03:03:41.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.392+0000 7f8bd030d740 10 client.26128 fetch_fsmap finished waiting for FSMap version 31 2024-09-12T03:03:41.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.392+0000 7f8bd030d740 10 client.26128 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-12T03:03:41.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.392+0000 7f8bd030d740 20 client.26128 populate_metadata read hostname 'smithi005' 2024-09-12T03:03:41.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.392+0000 7f8bd030d740 1 --2- 172.21.15.5:0/2094829930 >> [v2:172.21.15.148:6836/2606873922,v1:172.21.15.148:6837/2606873922] conn(0x564dc0df2fc0 0x564dc0df53b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:41.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.392+0000 7f8bd030d740 4 client.26128 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:03:41.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.392+0000 7f8bd030d740 1 -- 172.21.15.5:0/2094829930 --> [v2:172.21.15.148:6836/2606873922,v1:172.21.15.148:6837/2606873922] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564dc0d4b860 con 0x564dc0df2fc0 2024-09-12T03:03:41.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.393+0000 7f8bc77fe640 1 --2- 172.21.15.5:0/2094829930 >> [v2:172.21.15.148:6836/2606873922,v1:172.21.15.148:6837/2606873922] conn(0x564dc0df2fc0 0x564dc0df53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:41.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.393+0000 7f8ba6ffd640 10 client.26128 ms_handle_connect on v2:172.21.15.148:6836/2606873922 2024-09-12T03:03:41.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.393+0000 7f8bc77fe640 1 --2- 172.21.15.5:0/2094829930 >> [v2:172.21.15.148:6836/2606873922,v1:172.21.15.148:6837/2606873922] conn(0x564dc0df2fc0 0x564dc0df53b0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=542979ad8d425194 server_cookie=1149b70337f5c340 in_seq=0 out_seq=0 2024-09-12T03:03:41.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.394+0000 7f8ba6ffd640 1 -- 172.21.15.5:0/2094829930 <== mds.1 v2:172.21.15.148:6836/2606873922 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564dc0d4b860 con 0x564dc0df2fc0 2024-09-12T03:03:41.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.394+0000 7f8ba6ffd640 10 client.26128 handle_command_reply: tid=0 2024-09-12T03:03:41.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.394+0000 7f8bd030d740 1 client.26128 shutdown 2024-09-12T03:03:41.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.395+0000 7f8bd030d740 1 -- 172.21.15.5:0/2094829930 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564dc0dd9b40 msgr2=0x564dc0de00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:41.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.395+0000 7f8bd030d740 1 --2- 172.21.15.5:0/2094829930 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564dc0dd9b40 0x564dc0de00c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc02aa70 tx=0x7f8bbc0044f0 comp rx=0 tx=0).stop 2024-09-12T03:03:41.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.395+0000 7f8bd030d740 1 -- 172.21.15.5:0/2094829930 shutdown_connections 2024-09-12T03:03:41.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.395+0000 7f8bd030d740 1 --2- 172.21.15.5:0/2094829930 >> [v2:172.21.15.148:6836/2606873922,v1:172.21.15.148:6837/2606873922] conn(0x564dc0df2fc0 0x564dc0df53b0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:41.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.395+0000 7f8bd030d740 1 --2- 172.21.15.5:0/2094829930 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564dc0de3370 0x564dc0de0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:41.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.395+0000 7f8bd030d740 1 --2- 172.21.15.5:0/2094829930 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564dc0dd9b40 0x564dc0de00c0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:41.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.395+0000 7f8bd030d740 1 --2- 172.21.15.5:0/2094829930 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564dc0dd85d0 0x564dc0de2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:41.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.395+0000 7f8bd030d740 1 -- 172.21.15.5:0/2094829930 >> 172.21.15.5:0/2094829930 conn(0x564dc0da1310 msgr2=0x564dc0dda960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:41.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.395+0000 7f8ba6ffd640 0 client.26128 ms_handle_reset on v2:172.21.15.148:6836/2606873922 2024-09-12T03:03:41.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.395+0000 7f8bd030d740 1 -- 172.21.15.5:0/2094829930 shutdown_connections 2024-09-12T03:03:41.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.395+0000 7f8bd030d740 1 -- 172.21.15.5:0/2094829930 wait complete. 2024-09-12T03:03:41.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.395+0000 7f8bd030d740 20 client.26128 trim_cache size 0 max 16384 2024-09-12T03:03:41.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.411+0000 7f8bd030d740 1 Processor -- start 2024-09-12T03:03:41.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.411+0000 7f8bd030d740 1 -- start start 2024-09-12T03:03:41.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.411+0000 7f8bd030d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564dc0dd85d0 0x564dc0dfac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:41.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.412+0000 7f8bd030d740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564dc0dd9b40 0x564dc0dfb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:41.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.412+0000 7f8bd030d740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564dc0de3370 0x564dc0ddd320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:41.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.412+0000 7f8bd030d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc81ad570 con 0x564dc0dd85d0 2024-09-12T03:03:41.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.412+0000 7f8bd030d740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc81af0b0 con 0x564dc0dd9b40 2024-09-12T03:03:41.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.412+0000 7f8bd030d740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc81adc40 con 0x564dc0de3370 2024-09-12T03:03:41.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.412+0000 7f8bc77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564dc0dd85d0 0x564dc0dfac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:41.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.412+0000 7f8bc77fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564dc0dd85d0 0x564dc0dfac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49656/0 (socket says 172.21.15.5:49656) 2024-09-12T03:03:41.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.412+0000 7f8bc77fe640 1 -- 172.21.15.5:0/1862660906 learned_addr learned my addr 172.21.15.5:0/1862660906 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:03:41.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.413+0000 7f8bc77fe640 1 -- 172.21.15.5:0/1862660906 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564dc0de3370 msgr2=0x564dc0ddd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:41.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.413+0000 7f8bc77fe640 1 --2- 172.21.15.5:0/1862660906 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564dc0de3370 0x564dc0ddd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:41.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.413+0000 7f8bc77fe640 1 -- 172.21.15.5:0/1862660906 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564dc0dd9b40 msgr2=0x564dc0dfb1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:41.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.413+0000 7f8bc77fe640 1 --2- 172.21.15.5:0/1862660906 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564dc0dd9b40 0x564dc0dfb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:41.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.413+0000 7f8bc77fe640 1 -- 172.21.15.5:0/1862660906 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564dc0de0bc0 con 0x564dc0dd85d0 2024-09-12T03:03:41.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.413+0000 7f8bc77fe640 1 --2- 172.21.15.5:0/1862660906 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564dc0dd85d0 0x564dc0dfac60 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f8bb80cc730 tx=0x7f8bb80ce040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e8dde272db13162 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:41.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.414+0000 7f8bad7fa640 1 -- 172.21.15.5:0/1862660906 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb8020040 con 0x564dc0dd85d0 2024-09-12T03:03:41.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.414+0000 7f8bad7fa640 1 -- 172.21.15.5:0/1862660906 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8bb8099870 con 0x564dc0dd85d0 2024-09-12T03:03:41.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.414+0000 7f8bad7fa640 1 -- 172.21.15.5:0/1862660906 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb809a940 con 0x564dc0dd85d0 2024-09-12T03:03:41.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.415+0000 7f8bd030d740 1 -- 172.21.15.5:0/1862660906 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564dc0dd85d0 msgr2=0x564dc0dfac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:41.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.415+0000 7f8bd030d740 1 --2- 172.21.15.5:0/1862660906 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564dc0dd85d0 0x564dc0dfac60 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f8bb80cc730 tx=0x7f8bb80ce040 comp rx=0 tx=0).stop 2024-09-12T03:03:41.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.415+0000 7f8bd030d740 1 -- 172.21.15.5:0/1862660906 shutdown_connections 2024-09-12T03:03:41.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.415+0000 7f8bd030d740 1 --2- 172.21.15.5:0/1862660906 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564dc0de3370 0x564dc0ddd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:41.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.415+0000 7f8bd030d740 1 --2- 172.21.15.5:0/1862660906 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564dc0dd9b40 0x564dc0dfb1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:41.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.415+0000 7f8bd030d740 1 --2- 172.21.15.5:0/1862660906 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564dc0dd85d0 0x564dc0dfac60 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:41.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.415+0000 7f8bd030d740 1 -- 172.21.15.5:0/1862660906 >> 172.21.15.5:0/1862660906 conn(0x564dc0da1310 msgr2=0x564dc0ddb2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:41.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.415+0000 7f8bd030d740 1 -- 172.21.15.5:0/1862660906 shutdown_connections 2024-09-12T03:03:41.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.415+0000 7f8bd030d740 1 -- 172.21.15.5:0/1862660906 wait complete. 2024-09-12T03:03:41.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.416+0000 7f8bd030d740 1 -- start start 2024-09-12T03:03:41.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.416+0000 7f8bd030d740 1 Processor -- start 2024-09-12T03:03:41.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.417+0000 7f8bd030d740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564dc0dd85d0 0x564dc0deb780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:41.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.417+0000 7f8bd030d740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564dc0dd9b40 0x564dc0debcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:41.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.417+0000 7f8bd030d740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564dc0de3370 0x564dc0de6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:41.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.417+0000 7f8bd030d740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc81ae9e0 con 0x564dc0dd85d0 2024-09-12T03:03:41.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.417+0000 7f8bd030d740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc81ae310 con 0x564dc0de3370 2024-09-12T03:03:41.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.417+0000 7f8bd030d740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc81ab550 con 0x564dc0dd9b40 2024-09-12T03:03:41.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.417+0000 7f8bd030d740 10 client.0 _ll_register_callbacks cb 0x7f8bbc009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:03:41.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.417+0000 7f8bc6ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564dc0dd9b40 0x564dc0debcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:41.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.417+0000 7f8bc7fff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564dc0de3370 0x564dc0de6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:41.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.417+0000 7f8bc6ffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564dc0dd9b40 0x564dc0debcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.5:51806/0 (socket says 172.21.15.5:51806) 2024-09-12T03:03:41.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.417+0000 7f8bc7fff640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564dc0de3370 0x564dc0de6d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.5:50242/0 (socket says 172.21.15.5:50242) 2024-09-12T03:03:41.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.417+0000 7f8bc6ffd640 1 -- 172.21.15.5:0/1620980312 learned_addr learned my addr 172.21.15.5:0/1620980312 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:03:41.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.417+0000 7f8bc77fe640 1 --2- 172.21.15.5:0/1620980312 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564dc0dd85d0 0x564dc0deb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:41.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.417+0000 7f8bc6ffd640 1 -- 172.21.15.5:0/1620980312 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564dc0de3370 msgr2=0x564dc0de6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:41.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.417+0000 7f8bc6ffd640 1 --2- 172.21.15.5:0/1620980312 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564dc0de3370 0x564dc0de6d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:41.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.417+0000 7f8bc6ffd640 1 -- 172.21.15.5:0/1620980312 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564dc0dd85d0 msgr2=0x564dc0deb780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:41.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.417+0000 7f8bc6ffd640 1 --2- 172.21.15.5:0/1620980312 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564dc0dd85d0 0x564dc0deb780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:41.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.417+0000 7f8bc6ffd640 1 -- 172.21.15.5:0/1620980312 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564dc0de7500 con 0x564dc0dd9b40 2024-09-12T03:03:41.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.417+0000 7f8bc7fff640 1 --2- 172.21.15.5:0/1620980312 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564dc0de3370 0x564dc0de6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:03:41.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.417+0000 7f8bc77fe640 1 --2- 172.21.15.5:0/1620980312 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564dc0dd85d0 0x564dc0deb780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:03:41.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.418+0000 7f8bc6ffd640 1 --2- 172.21.15.5:0/1620980312 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564dc0dd9b40 0x564dc0debcc0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc02aa40 tx=0x7f8bbc031db0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f81b15ce9400c9d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:41.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.418+0000 7f8ba6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-09-12T03:03:41.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.418+0000 7f8ba6ffd640 1 -- 172.21.15.5:0/1620980312 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bbc035070 con 0x564dc0dd9b40 2024-09-12T03:03:41.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.418+0000 7f8bd030d740 1 -- 172.21.15.5:0/1620980312 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8bb8099870 con 0x564dc0dd9b40 2024-09-12T03:03:41.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.418+0000 7f8ba6ffd640 1 -- 172.21.15.5:0/1620980312 <== mon.1 v2:172.21.15.148:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f8bbc0317e0 con 0x564dc0dd9b40 2024-09-12T03:03:41.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.418+0000 7f8ba6ffd640 1 -- 172.21.15.5:0/1620980312 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f8bbc060050 con 0x564dc0dd9b40 2024-09-12T03:03:41.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.418+0000 7f8bd030d740 10 client.26131 fetch_fsmap learned FSMap version 31 2024-09-12T03:03:41.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.418+0000 7f8bd030d740 1 -- 172.21.15.5:0/1620980312 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x564dc0e5d8e0 con 0x564dc0dd9b40 2024-09-12T03:03:41.420 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.418+0000 7f8ba6ffd640 1 -- 172.21.15.5:0/1620980312 <== mon.1 v2:172.21.15.148:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8bb8099870 con 0x564dc0dd9b40 2024-09-12T03:03:41.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.418+0000 7f8ba6ffd640 1 -- 172.21.15.5:0/1620980312 <== mon.1 v2:172.21.15.148:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f8bbc003ad0 con 0x564dc0dd9b40 2024-09-12T03:03:41.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.418+0000 7f8bd030d740 10 client.26131 fetch_fsmap finished waiting for FSMap version 31 2024-09-12T03:03:41.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.418+0000 7f8bd030d740 10 client.26131 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-12T03:03:41.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.418+0000 7f8bd030d740 20 client.26131 populate_metadata read hostname 'smithi005' 2024-09-12T03:03:41.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.418+0000 7f8bd030d740 1 --2- 172.21.15.5:0/1620980312 >> [v2:172.21.15.148:6836/2606873922,v1:172.21.15.148:6837/2606873922] conn(0x564dc0e6f410 0x564dc0e71800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:41.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.418+0000 7f8bd030d740 4 client.26131 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:03:41.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.418+0000 7f8bd030d740 1 -- 172.21.15.5:0/1620980312 --> [v2:172.21.15.148:6836/2606873922,v1:172.21.15.148:6837/2606873922] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564dc0d45d70 con 0x564dc0e6f410 2024-09-12T03:03:41.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.419+0000 7f8bc77fe640 1 --2- 172.21.15.5:0/1620980312 >> [v2:172.21.15.148:6836/2606873922,v1:172.21.15.148:6837/2606873922] conn(0x564dc0e6f410 0x564dc0e71800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:41.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.419+0000 7f8ba6ffd640 10 client.26131 ms_handle_connect on v2:172.21.15.148:6836/2606873922 2024-09-12T03:03:41.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.419+0000 7f8bc77fe640 1 --2- 172.21.15.5:0/1620980312 >> [v2:172.21.15.148:6836/2606873922,v1:172.21.15.148:6837/2606873922] conn(0x564dc0e6f410 0x564dc0e71800 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5ea96ecb90add2ab server_cookie=df0313ef0a1ac6fa in_seq=0 out_seq=0 2024-09-12T03:03:41.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.419+0000 7f8ba6ffd640 1 -- 172.21.15.5:0/1620980312 <== mds.1 v2:172.21.15.148:6836/2606873922 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564dc0d45d70 con 0x564dc0e6f410 2024-09-12T03:03:41.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.419+0000 7f8ba6ffd640 10 client.26131 handle_command_reply: tid=0 2024-09-12T03:03:41.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.419+0000 7f8bd030d740 1 client.26131 shutdown 2024-09-12T03:03:41.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.420+0000 7f8bd030d740 1 -- 172.21.15.5:0/1620980312 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564dc0dd9b40 msgr2=0x564dc0debcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:41.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.420+0000 7f8bd030d740 1 --2- 172.21.15.5:0/1620980312 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564dc0dd9b40 0x564dc0debcc0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc02aa40 tx=0x7f8bbc031db0 comp rx=0 tx=0).stop 2024-09-12T03:03:41.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.420+0000 7f8bd030d740 1 -- 172.21.15.5:0/1620980312 shutdown_connections 2024-09-12T03:03:41.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.420+0000 7f8bd030d740 1 --2- 172.21.15.5:0/1620980312 >> [v2:172.21.15.148:6836/2606873922,v1:172.21.15.148:6837/2606873922] conn(0x564dc0e6f410 0x564dc0e71800 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:41.423 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.420+0000 7f8bd030d740 1 --2- 172.21.15.5:0/1620980312 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564dc0de3370 0x564dc0de6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:41.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.420+0000 7f8bd030d740 1 --2- 172.21.15.5:0/1620980312 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564dc0dd9b40 0x564dc0debcc0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:41.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.420+0000 7f8ba6ffd640 0 client.26131 ms_handle_reset on v2:172.21.15.148:6836/2606873922 2024-09-12T03:03:41.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.420+0000 7f8bd030d740 1 --2- 172.21.15.5:0/1620980312 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564dc0dd85d0 0x564dc0deb780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:41.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.420+0000 7f8bd030d740 1 -- 172.21.15.5:0/1620980312 >> 172.21.15.5:0/1620980312 conn(0x564dc0da1310 msgr2=0x564dc0dde990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:41.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.420+0000 7f8bd030d740 1 -- 172.21.15.5:0/1620980312 shutdown_connections 2024-09-12T03:03:41.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.420+0000 7f8bd030d740 1 -- 172.21.15.5:0/1620980312 wait complete. 2024-09-12T03:03:41.424 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.420+0000 7f8bd030d740 20 client.26131 trim_cache size 0 max 16384 2024-09-12T03:03:41.424 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-12T03:03:41.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.420+0000 7f8bcdf34640 1 -- 172.21.15.5:0/2309469653 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f8bb00a2b80 msgr2=0x7f8bb00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:41.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.420+0000 7f8bcdf34640 1 --2- 172.21.15.5:0/2309469653 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f8bb00a2b80 0x7f8bb00a5040 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f8bc0005fd0 tx=0x7f8bc0011810 comp rx=0 tx=0).stop 2024-09-12T03:03:41.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.420+0000 7f8bcdf34640 1 -- 172.21.15.5:0/2309469653 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bc8100e20 msgr2=0x7f8bc81993a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:41.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.420+0000 7f8bcdf34640 1 --2- 172.21.15.5:0/2309469653 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bc8100e20 0x7f8bc81993a0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f8bb800d950 tx=0x7f8bb800de10 comp rx=0 tx=0).stop 2024-09-12T03:03:41.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.421+0000 7f8bcdf34640 1 -- 172.21.15.5:0/2309469653 shutdown_connections 2024-09-12T03:03:41.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.421+0000 7f8bcdf34640 1 --2- 172.21.15.5:0/2309469653 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8bc8102ef0 0x7f8bc81a4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:41.425 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.421+0000 7f8bcdf34640 1 --2- 172.21.15.5:0/2309469653 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8bc8102570 0x7f8bc81998e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:41.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.421+0000 7f8bcdf34640 1 --2- 172.21.15.5:0/2309469653 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f8bb00a2b80 0x7f8bb00a5040 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:41.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.421+0000 7f8bcdf34640 1 --2- 172.21.15.5:0/2309469653 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bc8100e20 0x7f8bc81993a0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:41.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.421+0000 7f8bcdf34640 1 -- 172.21.15.5:0/2309469653 >> 172.21.15.5:0/2309469653 conn(0x7f8bc80fc7d0 msgr2=0x7f8bc80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:41.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.421+0000 7f8bcdf34640 1 -- 172.21.15.5:0/2309469653 shutdown_connections 2024-09-12T03:03:41.426 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:41.421+0000 7f8bcdf34640 1 -- 172.21.15.5:0/2309469653 wait complete. 2024-09-12T03:03:42.139 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T03:03:42.487 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:03:42.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:03:42 smithi005 ceph-mon[31197]: pgmap v499: 97 pgs: 97 active+clean; 153 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 2.4 MiB/s wr, 937 op/s 2024-09-12T03:03:43.020 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.018+0000 7f68952cb640 1 -- 172.21.15.5:0/3379065052 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6890100e20 msgr2=0x7f6890101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:43.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.018+0000 7f68952cb640 1 --2- 172.21.15.5:0/3379065052 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6890100e20 0x7f6890101200 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f6880009900 tx=0x7f688002a480 comp rx=0 tx=0).stop 2024-09-12T03:03:43.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.018+0000 7f68952cb640 1 -- 172.21.15.5:0/3379065052 shutdown_connections 2024-09-12T03:03:43.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.018+0000 7f68952cb640 1 --2- 172.21.15.5:0/3379065052 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6890102ef0 0x7f689010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:43.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.018+0000 7f68952cb640 1 --2- 172.21.15.5:0/3379065052 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6890102570 0x7f68901029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:43.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.018+0000 7f68952cb640 1 --2- 172.21.15.5:0/3379065052 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6890100e20 0x7f6890101200 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:43.021 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.018+0000 7f68952cb640 1 -- 172.21.15.5:0/3379065052 >> 172.21.15.5:0/3379065052 conn(0x7f68900fc7d0 msgr2=0x7f68900febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:43.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.023+0000 7f68952cb640 1 -- 172.21.15.5:0/3379065052 shutdown_connections 2024-09-12T03:03:43.023 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.023+0000 7f68952cb640 1 -- 172.21.15.5:0/3379065052 wait complete. 2024-09-12T03:03:43.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.024+0000 7f68952cb640 1 Processor -- start 2024-09-12T03:03:43.025 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.025+0000 7f68952cb640 1 -- start start 2024-09-12T03:03:43.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.025+0000 7f68952cb640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6890100e20 0x7f68901974b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:43.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.025+0000 7f68952cb640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6890102570 0x7f68901979f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:43.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.025+0000 7f68952cb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6890102ef0 0x7f689019bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:43.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.025+0000 7f68952cb640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f6890110fd0 con 0x7f6890102ef0 2024-09-12T03:03:43.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.025+0000 7f68952cb640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f6890110e50 con 0x7f6890100e20 2024-09-12T03:03:43.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.025+0000 7f68952cb640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f6890111150 con 0x7f6890102570 2024-09-12T03:03:43.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.025+0000 7f688f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6890102ef0 0x7f689019bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:43.027 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.025+0000 7f688e7fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6890102570 0x7f68901979f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:43.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.025+0000 7f688e7fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6890102570 0x7f68901979f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.5:51814/0 (socket says 172.21.15.5:51814) 2024-09-12T03:03:43.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.025+0000 7f688e7fc640 1 -- 172.21.15.5:0/766388 learned_addr learned my addr 172.21.15.5:0/766388 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:03:43.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.026+0000 7f688effd640 1 --2- 172.21.15.5:0/766388 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6890100e20 0x7f68901974b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:43.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.026+0000 7f688e7fc640 1 -- 172.21.15.5:0/766388 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6890100e20 msgr2=0x7f68901974b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:43.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.026+0000 7f688e7fc640 1 --2- 172.21.15.5:0/766388 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6890100e20 0x7f68901974b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:43.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.026+0000 7f688e7fc640 1 -- 172.21.15.5:0/766388 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6890102ef0 msgr2=0x7f689019bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:43.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.026+0000 7f688e7fc640 1 --2- 172.21.15.5:0/766388 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6890102ef0 0x7f689019bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:43.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.026+0000 7f688e7fc640 1 -- 172.21.15.5:0/766388 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f689019c460 con 0x7f6890102570 2024-09-12T03:03:43.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.026+0000 7f688effd640 1 --2- 172.21.15.5:0/766388 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6890100e20 0x7f68901974b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:03:43.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.026+0000 7f688f7fe640 1 --2- 172.21.15.5:0/766388 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6890102ef0 0x7f689019bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:03:43.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.026+0000 7f688e7fc640 1 --2- 172.21.15.5:0/766388 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6890102570 0x7f68901979f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f688400e9f0 tx=0x7f688400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:43.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.027+0000 7f6877fff640 1 -- 172.21.15.5:0/766388 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f688400cd80 con 0x7f6890102570 2024-09-12T03:03:43.028 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.027+0000 7f6877fff640 1 -- 172.21.15.5:0/766388 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6884010910 con 0x7f6890102570 2024-09-12T03:03:43.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.027+0000 7f68952cb640 1 -- 172.21.15.5:0/766388 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f689019c750 con 0x7f6890102570 2024-09-12T03:03:43.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.027+0000 7f6877fff640 1 -- 172.21.15.5:0/766388 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6884018990 con 0x7f6890102570 2024-09-12T03:03:43.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.027+0000 7f68952cb640 1 -- 172.21.15.5:0/766388 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6890069f70 con 0x7f6890102570 2024-09-12T03:03:43.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.029+0000 7f6877fff640 1 -- 172.21.15.5:0/766388 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f6884018b30 con 0x7f6890102570 2024-09-12T03:03:43.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.030+0000 7f68976a4740 1 Processor -- start 2024-09-12T03:03:43.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.030+0000 7f68976a4740 1 -- start start 2024-09-12T03:03:43.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.030+0000 7f6877fff640 1 --2- 172.21.15.5:0/766388 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f68780a2c00 0x7f68780a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:43.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.031+0000 7f688f7fe640 1 --2- 172.21.15.5:0/766388 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f68780a2c00 0x7f68780a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:43.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.031+0000 7f6877fff640 1 -- 172.21.15.5:0/766388 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f68840cf440 con 0x7f6890102570 2024-09-12T03:03:43.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.031+0000 7f68976a4740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559f0952bb40 0x559f0952a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:43.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.031+0000 7f68976a4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f0952a5d0 0x559f0952aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:43.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.031+0000 7f68976a4740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559f09539780 0x559f0953bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:43.032 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.031+0000 7f68976a4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559f094902f0 con 0x559f0952a5d0 2024-09-12T03:03:43.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.031+0000 7f68976a4740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x559f09437cf0 con 0x559f0952bb40 2024-09-12T03:03:43.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.031+0000 7f68976a4740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x559f0948c2b0 con 0x559f09539780 2024-09-12T03:03:43.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.031+0000 7f688effd640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559f0952bb40 0x559f0952a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:43.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.031+0000 7f688e7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f0952a5d0 0x559f0952aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:43.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.031+0000 7f688effd640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559f0952bb40 0x559f0952a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.5:50264/0 (socket says 172.21.15.5:50264) 2024-09-12T03:03:43.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.031+0000 7f688effd640 1 -- 172.21.15.5:0/2880478756 learned_addr learned my addr 172.21.15.5:0/2880478756 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:03:43.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.031+0000 7f688f7fe640 1 --2- 172.21.15.5:0/766388 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f68780a2c00 0x7f68780a50c0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f6888005fd0 tx=0x7f6888009500 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:43.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.031+0000 7f688f7fe640 1 --2- 172.21.15.5:0/2880478756 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559f09539780 0x559f0953bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:43.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.031+0000 7f688effd640 1 -- 172.21.15.5:0/2880478756 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559f09539780 msgr2=0x559f0953bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:43.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.031+0000 7f688effd640 1 --2- 172.21.15.5:0/2880478756 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559f09539780 0x559f0953bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:43.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.031+0000 7f688effd640 1 -- 172.21.15.5:0/2880478756 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f0952a5d0 msgr2=0x559f0952aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:43.033 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.031+0000 7f688effd640 1 --2- 172.21.15.5:0/2880478756 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f0952a5d0 0x559f0952aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:43.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.031+0000 7f688effd640 1 -- 172.21.15.5:0/2880478756 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f0941c3a0 con 0x559f0952bb40 2024-09-12T03:03:43.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.032+0000 7f688effd640 1 --2- 172.21.15.5:0/2880478756 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559f0952bb40 0x559f0952a090 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6880009900 tx=0x7f6880009650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2285aba8617b3d45 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:43.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.032+0000 7f68757fa640 1 -- 172.21.15.5:0/2880478756 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6880035040 con 0x559f0952bb40 2024-09-12T03:03:43.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.032+0000 7f68757fa640 1 -- 172.21.15.5:0/2880478756 <== mon.2 v2:172.21.15.146:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6880004760 con 0x559f0952bb40 2024-09-12T03:03:43.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.032+0000 7f68757fa640 1 -- 172.21.15.5:0/2880478756 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6880003a20 con 0x559f0952bb40 2024-09-12T03:03:43.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.033+0000 7f68976a4740 1 -- 172.21.15.5:0/2880478756 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559f0952bb40 msgr2=0x559f0952a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:43.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.033+0000 7f68976a4740 1 --2- 172.21.15.5:0/2880478756 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559f0952bb40 0x559f0952a090 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6880009900 tx=0x7f6880009650 comp rx=0 tx=0).stop 2024-09-12T03:03:43.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.033+0000 7f68976a4740 1 -- 172.21.15.5:0/2880478756 shutdown_connections 2024-09-12T03:03:43.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.033+0000 7f68976a4740 1 --2- 172.21.15.5:0/2880478756 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f0952a5d0 0x559f0952aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:43.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.033+0000 7f68976a4740 1 --2- 172.21.15.5:0/2880478756 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559f09539780 0x559f0953bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:43.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.033+0000 7f68976a4740 1 --2- 172.21.15.5:0/2880478756 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559f0952bb40 0x559f0952a090 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:43.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.033+0000 7f68976a4740 1 -- 172.21.15.5:0/2880478756 >> 172.21.15.5:0/2880478756 conn(0x559f094f3310 msgr2=0x559f09526c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:43.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.034+0000 7f68976a4740 1 -- 172.21.15.5:0/2880478756 shutdown_connections 2024-09-12T03:03:43.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.034+0000 7f68976a4740 1 -- 172.21.15.5:0/2880478756 wait complete. 2024-09-12T03:03:43.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.034+0000 7f68976a4740 1 -- start start 2024-09-12T03:03:43.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.034+0000 7f68976a4740 1 Processor -- start 2024-09-12T03:03:43.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.035+0000 7f68976a4740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559f0952a5d0 0x559f09539170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:43.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.035+0000 7f68976a4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f0952bb40 0x559f09536420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:43.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.035+0000 7f68976a4740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559f09539780 0x559f09536a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:43.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.035+0000 7f68976a4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x559f0953c420 con 0x559f0952bb40 2024-09-12T03:03:43.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.035+0000 7f68976a4740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x559f0944b060 con 0x559f09539780 2024-09-12T03:03:43.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.035+0000 7f688effd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559f0952a5d0 0x559f09539170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:43.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.035+0000 7f688effd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559f0952a5d0 0x559f09539170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.5:51822/0 (socket says 172.21.15.5:51822) 2024-09-12T03:03:43.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.035+0000 7f688effd640 1 -- 172.21.15.5:0/984591027 learned_addr learned my addr 172.21.15.5:0/984591027 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:03:43.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.035+0000 7f68976a4740 1 -- 172.21.15.5:0/984591027 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x559f0953c5a0 con 0x559f0952a5d0 2024-09-12T03:03:43.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.035+0000 7f688e7fc640 1 --2- 172.21.15.5:0/984591027 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f0952bb40 0x559f09536420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:43.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.035+0000 7f688f7fe640 1 --2- 172.21.15.5:0/984591027 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559f09539780 0x559f09536a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:43.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.036+0000 7f68976a4740 10 client.0 _ll_register_callbacks cb 0x559f0919e8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:03:43.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.036+0000 7f688f7fe640 1 -- 172.21.15.5:0/984591027 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559f0952a5d0 msgr2=0x559f09539170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:43.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.036+0000 7f688f7fe640 1 --2- 172.21.15.5:0/984591027 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559f0952a5d0 0x559f09539170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:43.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.036+0000 7f688f7fe640 1 -- 172.21.15.5:0/984591027 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f0952bb40 msgr2=0x559f09536420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:43.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.036+0000 7f688f7fe640 1 --2- 172.21.15.5:0/984591027 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f0952bb40 0x559f09536420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:43.041 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.036+0000 7f688f7fe640 1 -- 172.21.15.5:0/984591027 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f092013c0 con 0x559f09539780 2024-09-12T03:03:43.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.036+0000 7f688effd640 1 --2- 172.21.15.5:0/984591027 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559f0952a5d0 0x559f09539170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:03:43.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.036+0000 7f688f7fe640 1 --2- 172.21.15.5:0/984591027 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559f09539780 0x559f09536a10 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f6888012fd0 tx=0x7f6888004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0e5ce19fae84848 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:43.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.036+0000 7f6872ff5640 10 client.0 ms_handle_connect on v2:172.21.15.146:3300/0 2024-09-12T03:03:43.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.036+0000 7f6872ff5640 1 -- 172.21.15.5:0/984591027 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68880045e0 con 0x559f09539780 2024-09-12T03:03:43.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.036+0000 7f6872ff5640 1 -- 172.21.15.5:0/984591027 <== mon.2 v2:172.21.15.146:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f6888004780 con 0x559f09539780 2024-09-12T03:03:43.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.036+0000 7f68976a4740 1 -- 172.21.15.5:0/984591027 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6880004760 con 0x559f09539780 2024-09-12T03:03:43.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.037+0000 7f6872ff5640 1 -- 172.21.15.5:0/984591027 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f6888027440 con 0x559f09539780 2024-09-12T03:03:43.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.037+0000 7f68976a4740 10 client.26051 fetch_fsmap learned FSMap version 31 2024-09-12T03:03:43.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.037+0000 7f68976a4740 1 -- 172.21.15.5:0/984591027 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x559f09547f20 con 0x559f09539780 2024-09-12T03:03:43.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.037+0000 7f6872ff5640 1 -- 172.21.15.5:0/984591027 <== mon.2 v2:172.21.15.146:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f688802d030 con 0x559f09539780 2024-09-12T03:03:43.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.037+0000 7f6872ff5640 1 -- 172.21.15.5:0/984591027 <== mon.2 v2:172.21.15.146:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f6888025070 con 0x559f09539780 2024-09-12T03:03:43.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.037+0000 7f68976a4740 10 client.26051 fetch_fsmap finished waiting for FSMap version 31 2024-09-12T03:03:43.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.037+0000 7f68976a4740 10 client.26051 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:03:43.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.037+0000 7f68976a4740 20 client.26051 populate_metadata read hostname 'smithi005' 2024-09-12T03:03:43.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.037+0000 7f68976a4740 1 --2- 172.21.15.5:0/984591027 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x559f09549330 0x559f0954b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:43.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.037+0000 7f68976a4740 4 client.26051 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:03:43.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.037+0000 7f68976a4740 1 -- 172.21.15.5:0/984591027 --> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f6880004760 con 0x559f09549330 2024-09-12T03:03:43.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.037+0000 7f688effd640 1 --2- 172.21.15.5:0/984591027 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x559f09549330 0x559f0954b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:43.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.038+0000 7f688effd640 1 --2- 172.21.15.5:0/984591027 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x559f09549330 0x559f0954b780 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8288d8b5427594c2 server_cookie=f2813605380238fa in_seq=0 out_seq=0 2024-09-12T03:03:43.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.038+0000 7f6872ff5640 10 client.26051 ms_handle_connect on v2:172.21.15.148:6838/2317600000 2024-09-12T03:03:43.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.038+0000 7f6872ff5640 1 -- 172.21.15.5:0/984591027 <== mds.0 v2:172.21.15.148:6838/2317600000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f6880004760 con 0x559f09549330 2024-09-12T03:03:43.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.038+0000 7f6872ff5640 10 client.26051 handle_command_reply: tid=0 2024-09-12T03:03:43.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.038+0000 7f68976a4740 1 client.26051 shutdown 2024-09-12T03:03:43.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.039+0000 7f68976a4740 1 -- 172.21.15.5:0/984591027 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559f09539780 msgr2=0x559f09536a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:43.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.039+0000 7f68976a4740 1 --2- 172.21.15.5:0/984591027 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559f09539780 0x559f09536a10 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f6888012fd0 tx=0x7f6888004ab0 comp rx=0 tx=0).stop 2024-09-12T03:03:43.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.039+0000 7f68976a4740 1 -- 172.21.15.5:0/984591027 shutdown_connections 2024-09-12T03:03:43.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.039+0000 7f68976a4740 1 --2- 172.21.15.5:0/984591027 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x559f09549330 0x559f0954b780 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:43.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.039+0000 7f6872ff5640 0 client.26051 ms_handle_reset on v2:172.21.15.148:6838/2317600000 2024-09-12T03:03:43.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.039+0000 7f68976a4740 1 --2- 172.21.15.5:0/984591027 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f0952bb40 0x559f09536420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:43.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.039+0000 7f68976a4740 1 --2- 172.21.15.5:0/984591027 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559f09539780 0x559f09536a10 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:43.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.039+0000 7f68976a4740 1 --2- 172.21.15.5:0/984591027 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559f0952a5d0 0x559f09539170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:43.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.039+0000 7f68976a4740 1 -- 172.21.15.5:0/984591027 >> 172.21.15.5:0/984591027 conn(0x559f094f3310 msgr2=0x559f0952c960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:43.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.039+0000 7f68976a4740 1 -- 172.21.15.5:0/984591027 shutdown_connections 2024-09-12T03:03:43.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.039+0000 7f68976a4740 1 -- 172.21.15.5:0/984591027 wait complete. 2024-09-12T03:03:43.044 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.039+0000 7f68976a4740 20 client.26051 trim_cache size 0 max 16384 2024-09-12T03:03:43.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.058+0000 7f68976a4740 1 Processor -- start 2024-09-12T03:03:43.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.058+0000 7f68976a4740 1 -- start start 2024-09-12T03:03:43.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.059+0000 7f68976a4740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559f0952a5d0 0x559f095456c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:43.060 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.059+0000 7f68976a4740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559f0952bb40 0x559f09538390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:43.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.059+0000 7f68976a4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f09539780 0x559f095510a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:43.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.059+0000 7f68976a4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f68901ab350 con 0x559f09539780 2024-09-12T03:03:43.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.059+0000 7f68976a4740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f68901ace90 con 0x559f0952bb40 2024-09-12T03:03:43.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.059+0000 7f68976a4740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f68901aba20 con 0x559f0952a5d0 2024-09-12T03:03:43.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.059+0000 7f688f7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f09539780 0x559f095510a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:43.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.059+0000 7f688effd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559f0952a5d0 0x559f095456c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:43.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.059+0000 7f688effd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559f0952a5d0 0x559f095456c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.5:51830/0 (socket says 172.21.15.5:51830) 2024-09-12T03:03:43.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.059+0000 7f688effd640 1 -- 172.21.15.5:0/1927230016 learned_addr learned my addr 172.21.15.5:0/1927230016 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:03:43.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.059+0000 7f688e7fc640 1 --2- 172.21.15.5:0/1927230016 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559f0952bb40 0x559f09538390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:43.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.059+0000 7f688effd640 1 -- 172.21.15.5:0/1927230016 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559f0952bb40 msgr2=0x559f09538390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:43.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.059+0000 7f688effd640 1 --2- 172.21.15.5:0/1927230016 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559f0952bb40 0x559f09538390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:43.061 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.059+0000 7f688effd640 1 -- 172.21.15.5:0/1927230016 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f09539780 msgr2=0x559f095510a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:43.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.059+0000 7f688effd640 1 --2- 172.21.15.5:0/1927230016 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f09539780 0x559f095510a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:43.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.059+0000 7f688effd640 1 -- 172.21.15.5:0/1927230016 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f09526ed0 con 0x559f0952a5d0 2024-09-12T03:03:43.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.059+0000 7f688f7fe640 1 --2- 172.21.15.5:0/1927230016 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f09539780 0x559f095510a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:03:43.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.060+0000 7f688effd640 1 --2- 172.21.15.5:0/1927230016 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559f0952a5d0 0x559f095456c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6880005c40 tx=0x7f6880000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a25e5a0eb75e276e server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:43.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.060+0000 7f6874ff9640 1 -- 172.21.15.5:0/1927230016 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6880035050 con 0x559f0952a5d0 2024-09-12T03:03:43.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.060+0000 7f6874ff9640 1 -- 172.21.15.5:0/1927230016 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6880003ad0 con 0x559f0952a5d0 2024-09-12T03:03:43.062 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.060+0000 7f6874ff9640 1 -- 172.21.15.5:0/1927230016 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6880031b00 con 0x559f0952a5d0 2024-09-12T03:03:43.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.061+0000 7f68976a4740 1 -- 172.21.15.5:0/1927230016 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559f0952a5d0 msgr2=0x559f095456c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:43.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.061+0000 7f68976a4740 1 --2- 172.21.15.5:0/1927230016 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559f0952a5d0 0x559f095456c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6880005c40 tx=0x7f6880000bc0 comp rx=0 tx=0).stop 2024-09-12T03:03:43.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.061+0000 7f68976a4740 1 -- 172.21.15.5:0/1927230016 shutdown_connections 2024-09-12T03:03:43.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.061+0000 7f68976a4740 1 --2- 172.21.15.5:0/1927230016 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f09539780 0x559f095510a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:43.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.061+0000 7f68976a4740 1 --2- 172.21.15.5:0/1927230016 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559f0952bb40 0x559f09538390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:43.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.061+0000 7f68976a4740 1 --2- 172.21.15.5:0/1927230016 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559f0952a5d0 0x559f095456c0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:43.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.061+0000 7f68976a4740 1 -- 172.21.15.5:0/1927230016 >> 172.21.15.5:0/1927230016 conn(0x559f094f3310 msgr2=0x559f09537c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:43.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.061+0000 7f68976a4740 1 -- 172.21.15.5:0/1927230016 shutdown_connections 2024-09-12T03:03:43.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.061+0000 7f68976a4740 1 -- 172.21.15.5:0/1927230016 wait complete. 2024-09-12T03:03:43.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.062+0000 7f68976a4740 1 -- start start 2024-09-12T03:03:43.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.062+0000 7f68976a4740 1 Processor -- start 2024-09-12T03:03:43.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.062+0000 7f68976a4740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f0952a5d0 0x559f09542130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:43.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.062+0000 7f68976a4740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559f0952bb40 0x559f0953d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:43.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.062+0000 7f68976a4740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559f09539780 0x559f0953dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:43.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.062+0000 7f68976a4740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f68901ac7c0 con 0x559f0952a5d0 2024-09-12T03:03:43.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.062+0000 7f68976a4740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f68901ac0f0 con 0x559f09539780 2024-09-12T03:03:43.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.062+0000 7f68976a4740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f68901a9330 con 0x559f0952bb40 2024-09-12T03:03:43.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.062+0000 7f68976a4740 10 client.0 _ll_register_callbacks cb 0x7f6888003970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:03:43.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.063+0000 7f688e7fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559f0952bb40 0x559f0953d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:43.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.063+0000 7f688e7fc640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559f0952bb40 0x559f0953d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.5:51844/0 (socket says 172.21.15.5:51844) 2024-09-12T03:03:43.069 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.063+0000 7f688e7fc640 1 -- 172.21.15.5:0/3447021637 learned_addr learned my addr 172.21.15.5:0/3447021637 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:03:43.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.063+0000 7f688effd640 1 --2- 172.21.15.5:0/3447021637 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f0952a5d0 0x559f09542130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:43.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.063+0000 7f688e7fc640 1 -- 172.21.15.5:0/3447021637 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559f09539780 msgr2=0x559f0953dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:43.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.063+0000 7f688e7fc640 1 --2- 172.21.15.5:0/3447021637 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559f09539780 0x559f0953dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:43.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.063+0000 7f688e7fc640 1 -- 172.21.15.5:0/3447021637 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f0952a5d0 msgr2=0x559f09542130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:43.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.063+0000 7f688e7fc640 1 --2- 172.21.15.5:0/3447021637 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f0952a5d0 0x559f09542130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:43.070 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.063+0000 7f688e7fc640 1 -- 172.21.15.5:0/3447021637 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f095b3070 con 0x559f0952bb40 2024-09-12T03:03:43.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.063+0000 7f688effd640 1 --2- 172.21.15.5:0/3447021637 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f0952a5d0 0x559f09542130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:03:43.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.063+0000 7f688e7fc640 1 --2- 172.21.15.5:0/3447021637 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559f0952bb40 0x559f0953d460 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f68840ccef0 tx=0x7f688409c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e73608317bd3721c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:43.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.063+0000 7f6872ff5640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-09-12T03:03:43.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.064+0000 7f6872ff5640 1 -- 172.21.15.5:0/3447021637 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68840989a0 con 0x559f0952bb40 2024-09-12T03:03:43.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.064+0000 7f6872ff5640 1 -- 172.21.15.5:0/3447021637 <== mon.1 v2:172.21.15.148:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f6884098b40 con 0x559f0952bb40 2024-09-12T03:03:43.071 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.064+0000 7f68976a4740 1 -- 172.21.15.5:0/3447021637 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6880003ad0 con 0x559f0952bb40 2024-09-12T03:03:43.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.064+0000 7f6872ff5640 1 -- 172.21.15.5:0/3447021637 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f688409bb10 con 0x559f0952bb40 2024-09-12T03:03:43.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.064+0000 7f68976a4740 10 client.26149 fetch_fsmap learned FSMap version 31 2024-09-12T03:03:43.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.064+0000 7f68976a4740 1 -- 172.21.15.5:0/3447021637 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x559f095b4230 con 0x559f0952bb40 2024-09-12T03:03:43.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.064+0000 7f6872ff5640 1 -- 172.21.15.5:0/3447021637 <== mon.1 v2:172.21.15.148:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f688409bcb0 con 0x559f0952bb40 2024-09-12T03:03:43.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.064+0000 7f6872ff5640 1 -- 172.21.15.5:0/3447021637 <== mon.1 v2:172.21.15.148:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f68840ba970 con 0x559f0952bb40 2024-09-12T03:03:43.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.065+0000 7f68976a4740 10 client.26149 fetch_fsmap finished waiting for FSMap version 31 2024-09-12T03:03:43.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.065+0000 7f68976a4740 10 client.26149 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:03:43.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.065+0000 7f68976a4740 20 client.26149 populate_metadata read hostname 'smithi005' 2024-09-12T03:03:43.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.065+0000 7f68976a4740 1 --2- 172.21.15.5:0/3447021637 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x559f095b4960 0x559f095b6d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:43.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.065+0000 7f68976a4740 4 client.26149 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:03:43.073 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.065+0000 7f68976a4740 1 -- 172.21.15.5:0/3447021637 --> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f6880003ad0 con 0x559f095b4960 2024-09-12T03:03:43.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.065+0000 7f688effd640 1 --2- 172.21.15.5:0/3447021637 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x559f095b4960 0x559f095b6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:43.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.065+0000 7f688effd640 1 --2- 172.21.15.5:0/3447021637 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x559f095b4960 0x559f095b6d80 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14680963a2e0f7d8 server_cookie=58f0180e11b77fdf in_seq=0 out_seq=0 2024-09-12T03:03:43.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.065+0000 7f6872ff5640 10 client.26149 ms_handle_connect on v2:172.21.15.148:6838/2317600000 2024-09-12T03:03:43.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.066+0000 7f6872ff5640 1 -- 172.21.15.5:0/3447021637 <== mds.0 v2:172.21.15.148:6838/2317600000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f6880003ad0 con 0x559f095b4960 2024-09-12T03:03:43.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.066+0000 7f6872ff5640 10 client.26149 handle_command_reply: tid=0 2024-09-12T03:03:43.074 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.066+0000 7f68976a4740 1 client.26149 shutdown 2024-09-12T03:03:43.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.066+0000 7f68976a4740 1 -- 172.21.15.5:0/3447021637 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559f0952bb40 msgr2=0x559f0953d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:43.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.066+0000 7f68976a4740 1 --2- 172.21.15.5:0/3447021637 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559f0952bb40 0x559f0953d460 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f68840ccef0 tx=0x7f688409c470 comp rx=0 tx=0).stop 2024-09-12T03:03:43.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.066+0000 7f68976a4740 1 -- 172.21.15.5:0/3447021637 shutdown_connections 2024-09-12T03:03:43.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.066+0000 7f68976a4740 1 --2- 172.21.15.5:0/3447021637 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x559f09539780 0x559f0953dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:43.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.066+0000 7f68976a4740 1 --2- 172.21.15.5:0/3447021637 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x559f0952bb40 0x559f0953d460 secure :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f68840ccef0 tx=0x7f688409c470 comp rx=0 tx=0).stop 2024-09-12T03:03:43.075 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.066+0000 7f68976a4740 1 --2- 172.21.15.5:0/3447021637 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x559f095b4960 0x559f095b6d80 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:43.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.066+0000 7f6872ff5640 0 client.26149 ms_handle_reset on v2:172.21.15.148:6838/2317600000 2024-09-12T03:03:43.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.066+0000 7f68976a4740 1 --2- 172.21.15.5:0/3447021637 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f0952a5d0 0x559f09542130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:43.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.066+0000 7f68976a4740 1 -- 172.21.15.5:0/3447021637 >> 172.21.15.5:0/3447021637 conn(0x559f094f3310 msgr2=0x559f095348c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:43.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.066+0000 7f68976a4740 1 -- 172.21.15.5:0/3447021637 shutdown_connections 2024-09-12T03:03:43.076 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.066+0000 7f68976a4740 1 -- 172.21.15.5:0/3447021637 wait complete. 2024-09-12T03:03:43.077 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.066+0000 7f68976a4740 20 client.26149 trim_cache size 0 max 16384 2024-09-12T03:03:43.077 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-12T03:03:43.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.067+0000 7f68952cb640 1 -- 172.21.15.5:0/766388 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f68780a2c00 msgr2=0x7f68780a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:43.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.067+0000 7f68952cb640 1 --2- 172.21.15.5:0/766388 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f68780a2c00 0x7f68780a50c0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f6888005fd0 tx=0x7f6888009500 comp rx=0 tx=0).stop 2024-09-12T03:03:43.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.067+0000 7f68952cb640 1 -- 172.21.15.5:0/766388 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6890102570 msgr2=0x7f68901979f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:43.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.067+0000 7f68952cb640 1 --2- 172.21.15.5:0/766388 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6890102570 0x7f68901979f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f688400e9f0 tx=0x7f688400eeb0 comp rx=0 tx=0).stop 2024-09-12T03:03:43.079 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.067+0000 7f68952cb640 1 -- 172.21.15.5:0/766388 shutdown_connections 2024-09-12T03:03:43.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.067+0000 7f68952cb640 1 --2- 172.21.15.5:0/766388 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f68780a2c00 0x7f68780a50c0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:43.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.067+0000 7f68952cb640 1 --2- 172.21.15.5:0/766388 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6890102ef0 0x7f689019bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:43.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.067+0000 7f68952cb640 1 --2- 172.21.15.5:0/766388 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f6890102570 0x7f68901979f0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:43.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.067+0000 7f68952cb640 1 --2- 172.21.15.5:0/766388 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f6890100e20 0x7f68901974b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:43.080 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.067+0000 7f68952cb640 1 -- 172.21.15.5:0/766388 >> 172.21.15.5:0/766388 conn(0x7f68900fc7d0 msgr2=0x7f68901068d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:43.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.067+0000 7f68952cb640 1 -- 172.21.15.5:0/766388 shutdown_connections 2024-09-12T03:03:43.081 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:43.067+0000 7f68952cb640 1 -- 172.21.15.5:0/766388 wait complete. 2024-09-12T03:03:43.165 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:03:42 smithi148 ceph-mon[35090]: pgmap v499: 97 pgs: 97 active+clean; 153 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 2.4 MiB/s wr, 937 op/s 2024-09-12T03:03:43.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:03:42 smithi146 ceph-mon[35011]: pgmap v499: 97 pgs: 97 active+clean; 153 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 2.4 MiB/s wr, 937 op/s 2024-09-12T03:03:44.768 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-12T03:03:44.769 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-12T03:03:45.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:03:44 smithi005 ceph-mon[31197]: pgmap v500: 97 pgs: 97 active+clean; 153 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 2.7 MiB/s wr, 1.03k op/s 2024-09-12T03:03:45.085 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:03:45.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:03:44 smithi148 ceph-mon[35090]: pgmap v500: 97 pgs: 97 active+clean; 153 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 2.7 MiB/s wr, 1.03k op/s 2024-09-12T03:03:45.778 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:03:44 smithi146 ceph-mon[35011]: pgmap v500: 97 pgs: 97 active+clean; 153 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 2.7 MiB/s wr, 1.03k op/s 2024-09-12T03:03:45.779 INFO:tasks.workunit.client.0.smithi005.stdout:All 50000 operations completed A-OK! 2024-09-12T03:03:45.779 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-12T03:03:45.779 DEBUG:teuthology.orchestra.run.smithi005:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-12T03:03:45.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.709+0000 7fc9cb7cf640 1 -- 172.21.15.5:0/96367034 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9c4100e20 msgr2=0x7fc9c4101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:45.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.709+0000 7fc9cb7cf640 1 --2- 172.21.15.5:0/96367034 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9c4100e20 0x7fc9c4101200 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fc9c0009900 tx=0x7fc9c002a480 comp rx=0 tx=0).stop 2024-09-12T03:03:45.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.709+0000 7fc9cb7cf640 1 -- 172.21.15.5:0/96367034 shutdown_connections 2024-09-12T03:03:45.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.709+0000 7fc9cb7cf640 1 --2- 172.21.15.5:0/96367034 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc9c4102ef0 0x7fc9c410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:45.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.709+0000 7fc9cb7cf640 1 --2- 172.21.15.5:0/96367034 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc9c4102570 0x7fc9c41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:45.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.709+0000 7fc9cb7cf640 1 --2- 172.21.15.5:0/96367034 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9c4100e20 0x7fc9c4101200 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:45.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.709+0000 7fc9cb7cf640 1 -- 172.21.15.5:0/96367034 >> 172.21.15.5:0/96367034 conn(0x7fc9c40fc7d0 msgr2=0x7fc9c40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:45.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.709+0000 7fc9cb7cf640 1 -- 172.21.15.5:0/96367034 shutdown_connections 2024-09-12T03:03:45.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.709+0000 7fc9cb7cf640 1 -- 172.21.15.5:0/96367034 wait complete. 2024-09-12T03:03:45.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.709+0000 7fc9cb7cf640 1 Processor -- start 2024-09-12T03:03:45.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.709+0000 7fc9cb7cf640 1 -- start start 2024-09-12T03:03:45.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.710+0000 7fc9cb7cf640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc9c4100e20 0x7fc9c4103c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:45.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.710+0000 7fc9cb7cf640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc9c4102570 0x7fc9c41041b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:45.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.710+0000 7fc9cb7cf640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9c4102ef0 0x7fc9c41046f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:45.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.710+0000 7fc9cb7cf640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc9c4110f80 con 0x7fc9c4102ef0 2024-09-12T03:03:45.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.710+0000 7fc9cb7cf640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fc9c4110e00 con 0x7fc9c4102570 2024-09-12T03:03:45.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.710+0000 7fc9cb7cf640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc9c4111100 con 0x7fc9c4100e20 2024-09-12T03:03:45.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.710+0000 7fc9c9d6c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9c4102ef0 0x7fc9c41046f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:45.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.710+0000 7fc9c9d6c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9c4102ef0 0x7fc9c41046f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39294/0 (socket says 172.21.15.5:39294) 2024-09-12T03:03:45.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.710+0000 7fc9c9d6c640 1 -- 172.21.15.5:0/1012825692 learned_addr learned my addr 172.21.15.5:0/1012825692 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:03:45.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.710+0000 7fc9c956b640 1 --2- 172.21.15.5:0/1012825692 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc9c4100e20 0x7fc9c4103c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:45.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.710+0000 7fc9c8d6a640 1 --2- 172.21.15.5:0/1012825692 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc9c4102570 0x7fc9c41041b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:45.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.710+0000 7fc9c9d6c640 1 -- 172.21.15.5:0/1012825692 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc9c4100e20 msgr2=0x7fc9c4103c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:45.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.710+0000 7fc9c9d6c640 1 --2- 172.21.15.5:0/1012825692 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc9c4100e20 0x7fc9c4103c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:45.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.710+0000 7fc9c9d6c640 1 -- 172.21.15.5:0/1012825692 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc9c4102570 msgr2=0x7fc9c41041b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:45.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.710+0000 7fc9c9d6c640 1 --2- 172.21.15.5:0/1012825692 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc9c4102570 0x7fc9c41041b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:45.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.710+0000 7fc9c9d6c640 1 -- 172.21.15.5:0/1012825692 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9c41a69a0 con 0x7fc9c4102ef0 2024-09-12T03:03:45.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.710+0000 7fc9c9d6c640 1 --2- 172.21.15.5:0/1012825692 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9c4102ef0 0x7fc9c41046f0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fc9bc009fd0 tx=0x7fc9bc00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:45.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.710+0000 7fc9ba7fc640 1 -- 172.21.15.5:0/1012825692 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9bc017070 con 0x7fc9c4102ef0 2024-09-12T03:03:45.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.710+0000 7fc9ba7fc640 1 -- 172.21.15.5:0/1012825692 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9bc003e50 con 0x7fc9c4102ef0 2024-09-12T03:03:45.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.710+0000 7fc9cb7cf640 1 -- 172.21.15.5:0/1012825692 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9c41a6c90 con 0x7fc9c4102ef0 2024-09-12T03:03:45.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.711+0000 7fc9ba7fc640 1 -- 172.21.15.5:0/1012825692 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9bc0106c0 con 0x7fc9c4102ef0 2024-09-12T03:03:45.790 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.711+0000 7fc9cb7cf640 1 -- 172.21.15.5:0/1012825692 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9c41a7180 con 0x7fc9c4102ef0 2024-09-12T03:03:45.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.713+0000 7fc9ba7fc640 1 -- 172.21.15.5:0/1012825692 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fc9bc003970 con 0x7fc9c4102ef0 2024-09-12T03:03:45.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.713+0000 7fc9cdba8740 1 Processor -- start 2024-09-12T03:03:45.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.713+0000 7fc9cdba8740 1 -- start start 2024-09-12T03:03:45.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.714+0000 7fc9cdba8740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c54e163b60 0x55c54e163f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:45.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.714+0000 7fc9cdba8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c54e164480 0x55c54e16e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:45.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.714+0000 7fc9cdba8740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c54e16ed40 0x55c54e171130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:45.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.714+0000 7fc9cdba8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55c54e0c9280 con 0x55c54e164480 2024-09-12T03:03:45.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.714+0000 7fc9cdba8740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55c54e070b10 con 0x55c54e163b60 2024-09-12T03:03:45.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.714+0000 7fc9cdba8740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55c54e0c5240 con 0x55c54e16ed40 2024-09-12T03:03:45.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.714+0000 7fc9ba7fc640 1 --2- 172.21.15.5:0/1012825692 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fc9b00a2b80 0x7fc9b00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:45.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.714+0000 7fc9c956b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c54e164480 0x55c54e16e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:45.791 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.714+0000 7fc9c956b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c54e164480 0x55c54e16e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39298/0 (socket says 172.21.15.5:39298) 2024-09-12T03:03:45.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.714+0000 7fc9c956b640 1 -- 172.21.15.5:0/561259575 learned_addr learned my addr 172.21.15.5:0/561259575 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:03:45.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.714+0000 7fc9c8d6a640 1 --2- 172.21.15.5:0/561259575 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c54e163b60 0x55c54e163f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:45.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.714+0000 7fc9c9d6c640 1 --2- 172.21.15.5:0/1012825692 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fc9b00a2b80 0x7fc9b00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:45.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.714+0000 7fc9c8d6a640 1 --2- 172.21.15.5:0/561259575 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c54e16ed40 0x55c54e171130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:45.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.714+0000 7fc9ba7fc640 1 -- 172.21.15.5:0/1012825692 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fc9bc0ced10 con 0x7fc9c4102ef0 2024-09-12T03:03:45.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.714+0000 7fc9c956b640 1 -- 172.21.15.5:0/561259575 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c54e16ed40 msgr2=0x55c54e171130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:45.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.714+0000 7fc9c956b640 1 --2- 172.21.15.5:0/561259575 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c54e16ed40 0x55c54e171130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:45.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.714+0000 7fc9c956b640 1 -- 172.21.15.5:0/561259575 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c54e163b60 msgr2=0x55c54e163f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:45.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.714+0000 7fc9c956b640 1 --2- 172.21.15.5:0/561259575 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c54e163b60 0x55c54e163f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:45.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.714+0000 7fc9c956b640 1 -- 172.21.15.5:0/561259575 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c54e07d1f0 con 0x55c54e164480 2024-09-12T03:03:45.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.714+0000 7fc9c8d6a640 1 --2- 172.21.15.5:0/561259575 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c54e163b60 0x55c54e163f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:03:45.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.715+0000 7fc9c956b640 1 --2- 172.21.15.5:0/561259575 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c54e164480 0x55c54e16e800 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fc9c002a9a0 tx=0x7fc9c0005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a5195c9bb3cbb93 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:45.792 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.715+0000 7fc9c9d6c640 1 --2- 172.21.15.5:0/1012825692 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fc9b00a2b80 0x7fc9b00a5040 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fc9bc00dfd0 tx=0x7fc9bc0044b0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:45.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.715+0000 7fc9af7fe640 1 -- 172.21.15.5:0/561259575 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9c0035030 con 0x55c54e164480 2024-09-12T03:03:45.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.715+0000 7fc9af7fe640 1 -- 172.21.15.5:0/561259575 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9c0040070 con 0x55c54e164480 2024-09-12T03:03:45.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.715+0000 7fc9af7fe640 1 -- 172.21.15.5:0/561259575 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9c0031830 con 0x55c54e164480 2024-09-12T03:03:45.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.715+0000 7fc9cdba8740 1 -- 172.21.15.5:0/561259575 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c54e164480 msgr2=0x55c54e16e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:45.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.716+0000 7fc9cdba8740 1 --2- 172.21.15.5:0/561259575 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c54e164480 0x55c54e16e800 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fc9c002a9a0 tx=0x7fc9c0005e80 comp rx=0 tx=0).stop 2024-09-12T03:03:45.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.716+0000 7fc9cdba8740 1 -- 172.21.15.5:0/561259575 shutdown_connections 2024-09-12T03:03:45.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.716+0000 7fc9cdba8740 1 --2- 172.21.15.5:0/561259575 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c54e164480 0x55c54e16e800 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:45.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.716+0000 7fc9cdba8740 1 --2- 172.21.15.5:0/561259575 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c54e16ed40 0x55c54e171130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:45.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.716+0000 7fc9cdba8740 1 --2- 172.21.15.5:0/561259575 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c54e163b60 0x55c54e163f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:45.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.716+0000 7fc9cdba8740 1 -- 172.21.15.5:0/561259575 >> 172.21.15.5:0/561259575 conn(0x55c54e1600f0 msgr2=0x55c54e1604d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:45.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.716+0000 7fc9cdba8740 1 -- 172.21.15.5:0/561259575 shutdown_connections 2024-09-12T03:03:45.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.716+0000 7fc9cdba8740 1 -- 172.21.15.5:0/561259575 wait complete. 2024-09-12T03:03:45.793 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.716+0000 7fc9cdba8740 1 -- start start 2024-09-12T03:03:45.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.716+0000 7fc9cdba8740 1 Processor -- start 2024-09-12T03:03:45.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.717+0000 7fc9cdba8740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c54e163b60 0x55c54e16bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:45.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.717+0000 7fc9cdba8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c54e164480 0x55c54e16c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:45.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.717+0000 7fc9cdba8740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c54e16ed40 0x55c54e16c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:45.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.717+0000 7fc9cdba8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55c54e171910 con 0x55c54e164480 2024-09-12T03:03:45.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.717+0000 7fc9cdba8740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55c54e083f10 con 0x55c54e16ed40 2024-09-12T03:03:45.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.717+0000 7fc9cdba8740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55c54e171a90 con 0x55c54e163b60 2024-09-12T03:03:45.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.717+0000 7fc9cdba8740 10 client.0 _ll_register_callbacks cb 0x55c54e04ca40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:03:45.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.717+0000 7fc9c956b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c54e164480 0x55c54e16c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:45.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.717+0000 7fc9c956b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c54e164480 0x55c54e16c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39308/0 (socket says 172.21.15.5:39308) 2024-09-12T03:03:45.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.717+0000 7fc9c8d6a640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c54e16ed40 0x55c54e16c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:45.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.717+0000 7fc9c956b640 1 -- 172.21.15.5:0/382990240 learned_addr learned my addr 172.21.15.5:0/382990240 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:03:45.794 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.717+0000 7fc9c8d6a640 1 --2- 172.21.15.5:0/382990240 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c54e163b60 0x55c54e16bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:45.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.717+0000 7fc9c956b640 1 -- 172.21.15.5:0/382990240 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c54e163b60 msgr2=0x55c54e16bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:45.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.717+0000 7fc9c956b640 1 --2- 172.21.15.5:0/382990240 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c54e163b60 0x55c54e16bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:45.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.717+0000 7fc9c956b640 1 -- 172.21.15.5:0/382990240 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c54e16ed40 msgr2=0x55c54e16c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:45.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.717+0000 7fc9c956b640 1 --2- 172.21.15.5:0/382990240 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c54e16ed40 0x55c54e16c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:45.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.717+0000 7fc9c956b640 1 -- 172.21.15.5:0/382990240 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c54de3a6a0 con 0x55c54e164480 2024-09-12T03:03:45.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.717+0000 7fc9c8d6a640 1 --2- 172.21.15.5:0/382990240 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c54e16ed40 0x55c54e16c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:03:45.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.717+0000 7fc9c956b640 1 --2- 172.21.15.5:0/382990240 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c54e164480 0x55c54e16c110 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fc9c002a4c0 tx=0x7fc9c002ac30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de572593dd6f0d56 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:45.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.717+0000 7fc9acff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T03:03:45.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.717+0000 7fc9acff9640 1 -- 172.21.15.5:0/382990240 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9c0035070 con 0x55c54e164480 2024-09-12T03:03:45.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.717+0000 7fc9acff9640 1 -- 172.21.15.5:0/382990240 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fc9c0003a00 con 0x55c54e164480 2024-09-12T03:03:45.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.717+0000 7fc9cdba8740 1 -- 172.21.15.5:0/382990240 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9c0040070 con 0x55c54e164480 2024-09-12T03:03:45.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.718+0000 7fc9acff9640 1 -- 172.21.15.5:0/382990240 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fc9c0003c00 con 0x55c54e164480 2024-09-12T03:03:45.795 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.718+0000 7fc9cdba8740 10 client.16680 fetch_fsmap learned FSMap version 31 2024-09-12T03:03:45.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.718+0000 7fc9cdba8740 1 -- 172.21.15.5:0/382990240 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55c54e17d4f0 con 0x55c54e164480 2024-09-12T03:03:45.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.718+0000 7fc9acff9640 1 -- 172.21.15.5:0/382990240 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9c0040070 con 0x55c54e164480 2024-09-12T03:03:45.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.718+0000 7fc9acff9640 1 -- 172.21.15.5:0/382990240 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fc9c0045570 con 0x55c54e164480 2024-09-12T03:03:45.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.718+0000 7fc9cdba8740 10 client.16680 fetch_fsmap finished waiting for FSMap version 31 2024-09-12T03:03:45.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.718+0000 7fc9cdba8740 10 client.16680 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:03:45.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.718+0000 7fc9cdba8740 20 client.16680 populate_metadata read hostname 'smithi005' 2024-09-12T03:03:45.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.718+0000 7fc9cdba8740 1 --2- 172.21.15.5:0/382990240 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55c54e17e840 0x55c54e180c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:45.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.718+0000 7fc9cdba8740 4 client.16680 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:03:45.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.718+0000 7fc9cdba8740 1 -- 172.21.15.5:0/382990240 --> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c54e0d67f0 con 0x55c54e17e840 2024-09-12T03:03:45.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.718+0000 7fc9c8d6a640 1 --2- 172.21.15.5:0/382990240 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55c54e17e840 0x55c54e180c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:45.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.719+0000 7fc9c8d6a640 1 --2- 172.21.15.5:0/382990240 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55c54e17e840 0x55c54e180c30 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed4f1b5313dd248c server_cookie=2a52d6f71ff3371c in_seq=0 out_seq=0 2024-09-12T03:03:45.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.719+0000 7fc9acff9640 10 client.16680 ms_handle_connect on v2:172.21.15.148:6838/2317600000 2024-09-12T03:03:45.796 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.719+0000 7fc9acff9640 1 -- 172.21.15.5:0/382990240 <== mds.0 v2:172.21.15.148:6838/2317600000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c54e0d67f0 con 0x55c54e17e840 2024-09-12T03:03:45.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.719+0000 7fc9acff9640 10 client.16680 handle_command_reply: tid=0 2024-09-12T03:03:45.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.720+0000 7fc9cdba8740 1 client.16680 shutdown 2024-09-12T03:03:45.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.720+0000 7fc9cdba8740 1 -- 172.21.15.5:0/382990240 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c54e164480 msgr2=0x55c54e16c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:45.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.720+0000 7fc9cdba8740 1 --2- 172.21.15.5:0/382990240 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c54e164480 0x55c54e16c110 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fc9c002a4c0 tx=0x7fc9c002ac30 comp rx=0 tx=0).stop 2024-09-12T03:03:45.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.720+0000 7fc9cdba8740 1 -- 172.21.15.5:0/382990240 shutdown_connections 2024-09-12T03:03:45.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.720+0000 7fc9cdba8740 1 --2- 172.21.15.5:0/382990240 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55c54e17e840 0x55c54e180c30 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:45.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.720+0000 7fc9cdba8740 1 --2- 172.21.15.5:0/382990240 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c54e164480 0x55c54e16c110 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:45.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.720+0000 7fc9cdba8740 1 --2- 172.21.15.5:0/382990240 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c54e16ed40 0x55c54e16c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:45.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.720+0000 7fc9cdba8740 1 --2- 172.21.15.5:0/382990240 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c54e163b60 0x55c54e16bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:45.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.720+0000 7fc9cdba8740 1 -- 172.21.15.5:0/382990240 >> 172.21.15.5:0/382990240 conn(0x55c54e1600f0 msgr2=0x55c54e15f270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:45.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.720+0000 7fc9acff9640 0 client.16680 ms_handle_reset on v2:172.21.15.148:6838/2317600000 2024-09-12T03:03:45.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.720+0000 7fc9cdba8740 1 -- 172.21.15.5:0/382990240 shutdown_connections 2024-09-12T03:03:45.797 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.720+0000 7fc9cdba8740 1 -- 172.21.15.5:0/382990240 wait complete. 2024-09-12T03:03:45.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.720+0000 7fc9cdba8740 20 client.16680 trim_cache size 0 max 16384 2024-09-12T03:03:45.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.737+0000 7fc9cdba8740 1 Processor -- start 2024-09-12T03:03:45.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.737+0000 7fc9cdba8740 1 -- start start 2024-09-12T03:03:45.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.737+0000 7fc9cdba8740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c54e163b60 0x55c54e17a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:45.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.737+0000 7fc9cdba8740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c54e164480 0x55c54e168e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:45.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.737+0000 7fc9cdba8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c54e16ed40 0x55c54e169370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:45.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.737+0000 7fc9cdba8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc9c41af6d0 con 0x55c54e16ed40 2024-09-12T03:03:45.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.737+0000 7fc9cdba8740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fc9c41b1210 con 0x55c54e164480 2024-09-12T03:03:45.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.737+0000 7fc9cdba8740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc9c41afda0 con 0x55c54e163b60 2024-09-12T03:03:45.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.738+0000 7fc9c8d6a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c54e16ed40 0x55c54e169370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:45.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.738+0000 7fc9c8d6a640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c54e16ed40 0x55c54e169370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39312/0 (socket says 172.21.15.5:39312) 2024-09-12T03:03:45.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.738+0000 7fc9c8d6a640 1 -- 172.21.15.5:0/2392895064 learned_addr learned my addr 172.21.15.5:0/2392895064 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:03:45.798 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.738+0000 7fc9c8d6a640 1 --2- 172.21.15.5:0/2392895064 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c54e163b60 0x55c54e17a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:45.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.738+0000 7fc9c956b640 1 --2- 172.21.15.5:0/2392895064 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c54e164480 0x55c54e168e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:45.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.738+0000 7fc9c8d6a640 1 -- 172.21.15.5:0/2392895064 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c54e163b60 msgr2=0x55c54e17a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:45.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.738+0000 7fc9c8d6a640 1 --2- 172.21.15.5:0/2392895064 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c54e163b60 0x55c54e17a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:45.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.738+0000 7fc9c8d6a640 1 -- 172.21.15.5:0/2392895064 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c54e164480 msgr2=0x55c54e168e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:45.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.738+0000 7fc9c8d6a640 1 --2- 172.21.15.5:0/2392895064 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c54e164480 0x55c54e168e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:45.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.738+0000 7fc9c8d6a640 1 -- 172.21.15.5:0/2392895064 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c54e1642b0 con 0x55c54e16ed40 2024-09-12T03:03:45.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.738+0000 7fc9c8d6a640 1 --2- 172.21.15.5:0/2392895064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c54e16ed40 0x55c54e169370 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fc9b4004300 tx=0x7fc9b4009ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91fbd71e11e63f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:45.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.738+0000 7fc9aeffd640 1 -- 172.21.15.5:0/2392895064 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9b401a040 con 0x55c54e16ed40 2024-09-12T03:03:45.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.738+0000 7fc9aeffd640 1 -- 172.21.15.5:0/2392895064 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9b4022440 con 0x55c54e16ed40 2024-09-12T03:03:45.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.738+0000 7fc9aeffd640 1 -- 172.21.15.5:0/2392895064 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9b402b680 con 0x55c54e16ed40 2024-09-12T03:03:45.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.739+0000 7fc9cdba8740 1 -- 172.21.15.5:0/2392895064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c54e16ed40 msgr2=0x55c54e169370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:45.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.739+0000 7fc9cdba8740 1 --2- 172.21.15.5:0/2392895064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c54e16ed40 0x55c54e169370 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fc9b4004300 tx=0x7fc9b4009ac0 comp rx=0 tx=0).stop 2024-09-12T03:03:45.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.739+0000 7fc9cdba8740 1 -- 172.21.15.5:0/2392895064 shutdown_connections 2024-09-12T03:03:45.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.739+0000 7fc9cdba8740 1 --2- 172.21.15.5:0/2392895064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c54e16ed40 0x55c54e169370 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:45.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.739+0000 7fc9cdba8740 1 --2- 172.21.15.5:0/2392895064 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c54e164480 0x55c54e168e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:45.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.739+0000 7fc9cdba8740 1 --2- 172.21.15.5:0/2392895064 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c54e163b60 0x55c54e17a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:45.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.739+0000 7fc9cdba8740 1 -- 172.21.15.5:0/2392895064 >> 172.21.15.5:0/2392895064 conn(0x55c54e1600f0 msgr2=0x55c54e179d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:45.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.739+0000 7fc9cdba8740 1 -- 172.21.15.5:0/2392895064 shutdown_connections 2024-09-12T03:03:45.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.739+0000 7fc9cdba8740 1 -- 172.21.15.5:0/2392895064 wait complete. 2024-09-12T03:03:45.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.739+0000 7fc9cdba8740 1 -- start start 2024-09-12T03:03:45.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.740+0000 7fc9cdba8740 1 Processor -- start 2024-09-12T03:03:45.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.740+0000 7fc9cdba8740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c54e163b60 0x55c54e1838a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:45.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.740+0000 7fc9cdba8740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c54e164480 0x55c54e183de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:45.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.740+0000 7fc9cdba8740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c54e16ed40 0x55c54e180e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:45.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.740+0000 7fc9cdba8740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc9c41b0b40 con 0x55c54e164480 2024-09-12T03:03:45.800 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.740+0000 7fc9cdba8740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fc9c41b0470 con 0x55c54e163b60 2024-09-12T03:03:45.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.740+0000 7fc9cdba8740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc9c41ad6b0 con 0x55c54e16ed40 2024-09-12T03:03:45.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.740+0000 7fc9cdba8740 10 client.0 _ll_register_callbacks cb 0x7fc9b4009a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:03:45.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.740+0000 7fc9c956b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c54e164480 0x55c54e183de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:45.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.740+0000 7fc9c956b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c54e164480 0x55c54e183de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39324/0 (socket says 172.21.15.5:39324) 2024-09-12T03:03:45.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.740+0000 7fc9c956b640 1 -- 172.21.15.5:0/1559945026 learned_addr learned my addr 172.21.15.5:0/1559945026 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:03:45.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.740+0000 7fc9c8d6a640 1 --2- 172.21.15.5:0/1559945026 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c54e163b60 0x55c54e1838a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:45.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.740+0000 7fc9c8d6a640 1 --2- 172.21.15.5:0/1559945026 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c54e16ed40 0x55c54e180e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:45.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.740+0000 7fc9c956b640 1 -- 172.21.15.5:0/1559945026 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c54e16ed40 msgr2=0x55c54e180e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:45.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.740+0000 7fc9c956b640 1 --2- 172.21.15.5:0/1559945026 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c54e16ed40 0x55c54e180e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:45.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.740+0000 7fc9c956b640 1 -- 172.21.15.5:0/1559945026 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c54e163b60 msgr2=0x55c54e1838a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:45.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.740+0000 7fc9c956b640 1 --2- 172.21.15.5:0/1559945026 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c54e163b60 0x55c54e1838a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:45.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.740+0000 7fc9c956b640 1 -- 172.21.15.5:0/1559945026 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c54e181500 con 0x55c54e164480 2024-09-12T03:03:45.801 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.740+0000 7fc9c8d6a640 1 --2- 172.21.15.5:0/1559945026 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c54e16ed40 0x55c54e180e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:03:45.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.741+0000 7fc9c956b640 1 --2- 172.21.15.5:0/1559945026 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c54e164480 0x55c54e183de0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fc9c0000c00 tx=0x7fc9c00058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=385ea1ddb830214b server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:45.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.741+0000 7fc9acff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T03:03:45.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.741+0000 7fc9acff9640 1 -- 172.21.15.5:0/1559945026 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9c005f060 con 0x55c54e164480 2024-09-12T03:03:45.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.741+0000 7fc9acff9640 1 -- 172.21.15.5:0/1559945026 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fc9c0045db0 con 0x55c54e164480 2024-09-12T03:03:45.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.741+0000 7fc9cdba8740 1 -- 172.21.15.5:0/1559945026 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9b4022440 con 0x55c54e164480 2024-09-12T03:03:45.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.741+0000 7fc9acff9640 1 -- 172.21.15.5:0/1559945026 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fc9c0035040 con 0x55c54e164480 2024-09-12T03:03:45.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.741+0000 7fc9cdba8740 10 client.16686 fetch_fsmap learned FSMap version 31 2024-09-12T03:03:45.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.741+0000 7fc9cdba8740 1 -- 172.21.15.5:0/1559945026 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55c54e1e8ca0 con 0x55c54e164480 2024-09-12T03:03:45.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.741+0000 7fc9acff9640 1 -- 172.21.15.5:0/1559945026 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9b4022440 con 0x55c54e164480 2024-09-12T03:03:45.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.741+0000 7fc9acff9640 1 -- 172.21.15.5:0/1559945026 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fc9c0031a20 con 0x55c54e164480 2024-09-12T03:03:45.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.742+0000 7fc9cdba8740 10 client.16686 fetch_fsmap finished waiting for FSMap version 31 2024-09-12T03:03:45.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.742+0000 7fc9cdba8740 10 client.16686 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:03:45.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.742+0000 7fc9cdba8740 20 client.16686 populate_metadata read hostname 'smithi005' 2024-09-12T03:03:45.802 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.742+0000 7fc9cdba8740 1 --2- 172.21.15.5:0/1559945026 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55c54e1e9300 0x55c54e1eb720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:45.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.742+0000 7fc9cdba8740 4 client.16686 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-12T03:03:45.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.742+0000 7fc9cdba8740 1 -- 172.21.15.5:0/1559945026 --> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55c54e0d0d00 con 0x55c54e1e9300 2024-09-12T03:03:45.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.742+0000 7fc9c8d6a640 1 --2- 172.21.15.5:0/1559945026 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55c54e1e9300 0x55c54e1eb720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:45.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.742+0000 7fc9c8d6a640 1 --2- 172.21.15.5:0/1559945026 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55c54e1e9300 0x55c54e1eb720 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20cdf0f95dd7b7ec server_cookie=53e9b505d931c268 in_seq=0 out_seq=0 2024-09-12T03:03:45.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.742+0000 7fc9acff9640 10 client.16686 ms_handle_connect on v2:172.21.15.148:6838/2317600000 2024-09-12T03:03:45.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.743+0000 7fc9acff9640 1 -- 172.21.15.5:0/1559945026 <== mds.0 v2:172.21.15.148:6838/2317600000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55c54e0d0d00 con 0x55c54e1e9300 2024-09-12T03:03:45.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.743+0000 7fc9acff9640 10 client.16686 handle_command_reply: tid=0 2024-09-12T03:03:45.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.743+0000 7fc9cdba8740 1 client.16686 shutdown 2024-09-12T03:03:45.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.743+0000 7fc9cdba8740 1 -- 172.21.15.5:0/1559945026 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c54e164480 msgr2=0x55c54e183de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:45.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.743+0000 7fc9cdba8740 1 --2- 172.21.15.5:0/1559945026 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c54e164480 0x55c54e183de0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fc9c0000c00 tx=0x7fc9c00058d0 comp rx=0 tx=0).stop 2024-09-12T03:03:45.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.743+0000 7fc9cdba8740 1 -- 172.21.15.5:0/1559945026 shutdown_connections 2024-09-12T03:03:45.803 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.743+0000 7fc9cdba8740 1 --2- 172.21.15.5:0/1559945026 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55c54e1e9300 0x55c54e1eb720 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:45.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.743+0000 7fc9cdba8740 1 --2- 172.21.15.5:0/1559945026 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c54e164480 0x55c54e183de0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:45.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.743+0000 7fc9cdba8740 1 --2- 172.21.15.5:0/1559945026 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c54e16ed40 0x55c54e180e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:45.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.743+0000 7fc9cdba8740 1 --2- 172.21.15.5:0/1559945026 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c54e163b60 0x55c54e1838a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:45.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.743+0000 7fc9cdba8740 1 -- 172.21.15.5:0/1559945026 >> 172.21.15.5:0/1559945026 conn(0x55c54e1600f0 msgr2=0x55c54e15f270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:45.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.743+0000 7fc9acff9640 0 client.16686 ms_handle_reset on v2:172.21.15.148:6838/2317600000 2024-09-12T03:03:45.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.743+0000 7fc9cdba8740 1 -- 172.21.15.5:0/1559945026 shutdown_connections 2024-09-12T03:03:45.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.743+0000 7fc9cdba8740 1 -- 172.21.15.5:0/1559945026 wait complete. 2024-09-12T03:03:45.804 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.744+0000 7fc9cdba8740 20 client.16686 trim_cache size 0 max 16384 2024-09-12T03:03:45.804 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-12T03:03:45.805 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2024-09-12T03:03:45.805 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "efd99e26-f40b-4fb4-b002-9dbcf78af4c1", 2024-09-12T03:03:45.805 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2024-09-12T03:03:45.805 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-12T03:03:45.805 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.744+0000 7fc9cb7cf640 1 -- 172.21.15.5:0/1012825692 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fc9b00a2b80 msgr2=0x7fc9b00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:45.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.744+0000 7fc9cb7cf640 1 --2- 172.21.15.5:0/1012825692 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fc9b00a2b80 0x7fc9b00a5040 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fc9bc00dfd0 tx=0x7fc9bc0044b0 comp rx=0 tx=0).stop 2024-09-12T03:03:45.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.744+0000 7fc9cb7cf640 1 -- 172.21.15.5:0/1012825692 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9c4102ef0 msgr2=0x7fc9c41046f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:45.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.744+0000 7fc9cb7cf640 1 --2- 172.21.15.5:0/1012825692 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9c4102ef0 0x7fc9c41046f0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fc9bc009fd0 tx=0x7fc9bc00b260 comp rx=0 tx=0).stop 2024-09-12T03:03:45.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.744+0000 7fc9cb7cf640 1 -- 172.21.15.5:0/1012825692 shutdown_connections 2024-09-12T03:03:45.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.744+0000 7fc9cb7cf640 1 --2- 172.21.15.5:0/1012825692 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fc9b00a2b80 0x7fc9b00a5040 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:45.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.744+0000 7fc9cb7cf640 1 --2- 172.21.15.5:0/1012825692 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9c4102ef0 0x7fc9c41046f0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:45.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.744+0000 7fc9cb7cf640 1 --2- 172.21.15.5:0/1012825692 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc9c4102570 0x7fc9c41041b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:45.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.744+0000 7fc9cb7cf640 1 --2- 172.21.15.5:0/1012825692 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc9c4100e20 0x7fc9c4103c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:45.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.744+0000 7fc9cb7cf640 1 -- 172.21.15.5:0/1012825692 >> 172.21.15.5:0/1012825692 conn(0x7fc9c40fc7d0 msgr2=0x7fc9c4105eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:45.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.744+0000 7fc9cb7cf640 1 -- 172.21.15.5:0/1012825692 shutdown_connections 2024-09-12T03:03:45.806 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:45.744+0000 7fc9cb7cf640 1 -- 172.21.15.5:0/1012825692 wait complete. 2024-09-12T03:03:46.975 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T03:03:47.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:03:46 smithi005 ceph-mon[31197]: pgmap v501: 97 pgs: 97 active+clean; 154 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 2.3 MiB/s wr, 919 op/s 2024-09-12T03:03:47.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:03:46 smithi005 ceph-mon[31197]: scrub queued for path: / 2024-09-12T03:03:47.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:03:46 smithi005 ceph-mon[31197]: scrub summary: idle+waiting paths [/] 2024-09-12T03:03:47.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:03:46 smithi005 ceph-mon[31197]: scrub summary: active paths [/] 2024-09-12T03:03:47.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:03:46 smithi005 ceph-mon[31197]: scrub summary: active paths [/] 2024-09-12T03:03:47.061 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:03:46 smithi005 ceph-mon[31197]: scrub summary: idle+waiting paths [/] 2024-09-12T03:03:47.061 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:03:46 smithi005 ceph-mon[31197]: scrub summary: active paths [/] 2024-09-12T03:03:47.061 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:03:46 smithi005 ceph-mon[31197]: scrub summary: idle+waiting paths [/] 2024-09-12T03:03:47.165 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:03:46 smithi148 ceph-mon[35090]: pgmap v501: 97 pgs: 97 active+clean; 154 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 2.3 MiB/s wr, 919 op/s 2024-09-12T03:03:47.165 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:03:46 smithi148 ceph-mon[35090]: scrub queued for path: / 2024-09-12T03:03:47.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:03:46 smithi148 ceph-mon[35090]: scrub summary: idle+waiting paths [/] 2024-09-12T03:03:47.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:03:46 smithi148 ceph-mon[35090]: scrub summary: active paths [/] 2024-09-12T03:03:47.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:03:46 smithi148 ceph-mon[35090]: scrub summary: active paths [/] 2024-09-12T03:03:47.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:03:46 smithi148 ceph-mon[35090]: scrub summary: idle+waiting paths [/] 2024-09-12T03:03:47.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:03:46 smithi148 ceph-mon[35090]: scrub summary: active paths [/] 2024-09-12T03:03:47.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:03:46 smithi148 ceph-mon[35090]: scrub summary: idle+waiting paths [/] 2024-09-12T03:03:47.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:03:46 smithi146 ceph-mon[35011]: pgmap v501: 97 pgs: 97 active+clean; 154 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 2.3 MiB/s wr, 919 op/s 2024-09-12T03:03:47.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:03:46 smithi146 ceph-mon[35011]: scrub queued for path: / 2024-09-12T03:03:47.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:03:46 smithi146 ceph-mon[35011]: scrub summary: idle+waiting paths [/] 2024-09-12T03:03:47.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:03:46 smithi146 ceph-mon[35011]: scrub summary: active paths [/] 2024-09-12T03:03:47.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:03:46 smithi146 ceph-mon[35011]: scrub summary: active paths [/] 2024-09-12T03:03:47.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:03:46 smithi146 ceph-mon[35011]: scrub summary: idle+waiting paths [/] 2024-09-12T03:03:47.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:03:46 smithi146 ceph-mon[35011]: scrub summary: active paths [/] 2024-09-12T03:03:47.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:03:46 smithi146 ceph-mon[35011]: scrub summary: idle+waiting paths [/] 2024-09-12T03:03:47.242 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:03:48.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:47.995+0000 7fa0a80f9640 1 -- 172.21.15.5:0/976226060 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0a0101dd0 msgr2=0x7fa0a01021b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:48.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:47.995+0000 7fa0a80f9640 1 --2- 172.21.15.5:0/976226060 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0a0101dd0 0x7fa0a01021b0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fa09c009900 tx=0x7fa09c02a410 comp rx=0 tx=0).stop 2024-09-12T03:03:48.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:47.995+0000 7fa0a80f9640 1 -- 172.21.15.5:0/976226060 shutdown_connections 2024-09-12T03:03:48.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:47.995+0000 7fa0a80f9640 1 --2- 172.21.15.5:0/976226060 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa0a0106f00 0x7fa0a010ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:48.003 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:47.995+0000 7fa0a80f9640 1 --2- 172.21.15.5:0/976226060 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa0a0102780 0x7fa0a01067d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:48.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:47.995+0000 7fa0a80f9640 1 --2- 172.21.15.5:0/976226060 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0a0101dd0 0x7fa0a01021b0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:48.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:47.995+0000 7fa0a80f9640 1 -- 172.21.15.5:0/976226060 >> 172.21.15.5:0/976226060 conn(0x7fa0a00fd8b0 msgr2=0x7fa0a00ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:48.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:47.995+0000 7fa0a80f9640 1 -- 172.21.15.5:0/976226060 shutdown_connections 2024-09-12T03:03:48.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:47.995+0000 7fa0a80f9640 1 -- 172.21.15.5:0/976226060 wait complete. 2024-09-12T03:03:48.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:47.996+0000 7fa0a80f9640 1 Processor -- start 2024-09-12T03:03:48.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:47.996+0000 7fa0a80f9640 1 -- start start 2024-09-12T03:03:48.004 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:47.996+0000 7fa0a80f9640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa0a0101dd0 0x7fa0a0198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:48.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:47.996+0000 7fa0a80f9640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa0a0102780 0x7fa0a0198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:48.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:47.996+0000 7fa0a80f9640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0a0106f00 0x7fa0a019cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:48.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:47.996+0000 7fa0a80f9640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa0a0111f60 con 0x7fa0a0106f00 2024-09-12T03:03:48.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:47.996+0000 7fa0a80f9640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fa0a0111de0 con 0x7fa0a0102780 2024-09-12T03:03:48.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:47.996+0000 7fa0a80f9640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa0a01120e0 con 0x7fa0a0101dd0 2024-09-12T03:03:48.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:47.996+0000 7fa0a6696640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0a0106f00 0x7fa0a019cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:48.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:47.996+0000 7fa0a6696640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0a0106f00 0x7fa0a019cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39354/0 (socket says 172.21.15.5:39354) 2024-09-12T03:03:48.005 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:47.996+0000 7fa0a6696640 1 -- 172.21.15.5:0/2245053751 learned_addr learned my addr 172.21.15.5:0/2245053751 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:03:48.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:47.996+0000 7fa0a5e95640 1 --2- 172.21.15.5:0/2245053751 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa0a0101dd0 0x7fa0a0198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:48.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:47.996+0000 7fa0a5694640 1 --2- 172.21.15.5:0/2245053751 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa0a0102780 0x7fa0a0198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:48.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:47.996+0000 7fa0a6696640 1 -- 172.21.15.5:0/2245053751 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa0a0101dd0 msgr2=0x7fa0a0198520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:48.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:47.996+0000 7fa0a6696640 1 --2- 172.21.15.5:0/2245053751 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa0a0101dd0 0x7fa0a0198520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:48.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:47.996+0000 7fa0a6696640 1 -- 172.21.15.5:0/2245053751 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa0a0102780 msgr2=0x7fa0a0198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:48.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:47.996+0000 7fa0a6696640 1 --2- 172.21.15.5:0/2245053751 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa0a0102780 0x7fa0a0198a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:48.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:47.996+0000 7fa0a6696640 1 -- 172.21.15.5:0/2245053751 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0a019d4d0 con 0x7fa0a0106f00 2024-09-12T03:03:48.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:47.997+0000 7fa0a6696640 1 --2- 172.21.15.5:0/2245053751 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0a0106f00 0x7fa0a019cdf0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fa09800eb10 tx=0x7fa09800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:48.006 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:47.997+0000 7fa096ffd640 1 -- 172.21.15.5:0/2245053751 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa098004b20 con 0x7fa0a0106f00 2024-09-12T03:03:48.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:47.997+0000 7fa096ffd640 1 -- 172.21.15.5:0/2245053751 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa098004cc0 con 0x7fa0a0106f00 2024-09-12T03:03:48.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:47.997+0000 7fa0a80f9640 1 -- 172.21.15.5:0/2245053751 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0a019d7c0 con 0x7fa0a0106f00 2024-09-12T03:03:48.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:47.997+0000 7fa096ffd640 1 -- 172.21.15.5:0/2245053751 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa098010760 con 0x7fa0a0106f00 2024-09-12T03:03:48.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:47.997+0000 7fa0a80f9640 1 -- 172.21.15.5:0/2245053751 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0a0074420 con 0x7fa0a0106f00 2024-09-12T03:03:48.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:47.999+0000 7fa0aa4d2740 1 Processor -- start 2024-09-12T03:03:48.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.000+0000 7fa096ffd640 1 -- 172.21.15.5:0/2245053751 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fa098005020 con 0x7fa0a0106f00 2024-09-12T03:03:48.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.000+0000 7fa0aa4d2740 1 -- start start 2024-09-12T03:03:48.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.000+0000 7fa0aa4d2740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cbf5e4d4a0 0x55cbf5e4d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:48.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.000+0000 7fa0aa4d2740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cbf5e4ddc0 0x55cbf5e58140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:48.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.000+0000 7fa0aa4d2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbf5e58680 0x55cbf5e5aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:48.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.000+0000 7fa0aa4d2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55cbf5db32e0 con 0x55cbf5e58680 2024-09-12T03:03:48.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.000+0000 7fa0aa4d2740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55cbf5d5ace0 con 0x55cbf5e4d4a0 2024-09-12T03:03:48.007 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.000+0000 7fa0aa4d2740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55cbf5daf2a0 con 0x55cbf5e4ddc0 2024-09-12T03:03:48.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.000+0000 7fa0a5694640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbf5e58680 0x55cbf5e5aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:48.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.000+0000 7fa0a5e95640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cbf5e4ddc0 0x55cbf5e58140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:48.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.000+0000 7fa0a5694640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbf5e58680 0x55cbf5e5aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39362/0 (socket says 172.21.15.5:39362) 2024-09-12T03:03:48.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.000+0000 7fa0a5694640 1 -- 172.21.15.5:0/881783443 learned_addr learned my addr 172.21.15.5:0/881783443 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:03:48.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.000+0000 7fa0a5694640 1 --2- 172.21.15.5:0/881783443 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cbf5e4d4a0 0x55cbf5e4d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:48.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.001+0000 7fa096ffd640 1 --2- 172.21.15.5:0/2245053751 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fa08c0a2b80 0x7fa08c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:48.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.001+0000 7fa0a5e95640 1 -- 172.21.15.5:0/881783443 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cbf5e4d4a0 msgr2=0x55cbf5e4d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:48.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.001+0000 7fa0a5e95640 1 --2- 172.21.15.5:0/881783443 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cbf5e4d4a0 0x55cbf5e4d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:48.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.001+0000 7fa0a5e95640 1 -- 172.21.15.5:0/881783443 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbf5e58680 msgr2=0x55cbf5e5aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:48.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.001+0000 7fa0a5e95640 1 --2- 172.21.15.5:0/881783443 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbf5e58680 0x55cbf5e5aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:48.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.001+0000 7fa0a5e95640 1 -- 172.21.15.5:0/881783443 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cbf5d3f440 con 0x55cbf5e4ddc0 2024-09-12T03:03:48.008 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.001+0000 7fa0a5694640 1 --2- 172.21.15.5:0/881783443 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbf5e58680 0x55cbf5e5aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:03:48.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.001+0000 7fa0a6696640 1 --2- 172.21.15.5:0/2245053751 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fa08c0a2b80 0x7fa08c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:48.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.001+0000 7fa096ffd640 1 -- 172.21.15.5:0/2245053751 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fa0980cf520 con 0x7fa0a0106f00 2024-09-12T03:03:48.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.001+0000 7fa0a5e95640 1 --2- 172.21.15.5:0/881783443 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cbf5e4ddc0 0x55cbf5e58140 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa09c005940 tx=0x7fa09c005be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef03f4d91d739908 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:48.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.001+0000 7fa08bfff640 1 -- 172.21.15.5:0/881783443 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa09c040070 con 0x55cbf5e4ddc0 2024-09-12T03:03:48.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.001+0000 7fa08bfff640 1 -- 172.21.15.5:0/881783443 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa09c031870 con 0x55cbf5e4ddc0 2024-09-12T03:03:48.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.001+0000 7fa08bfff640 1 -- 172.21.15.5:0/881783443 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa09c035040 con 0x55cbf5e4ddc0 2024-09-12T03:03:48.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.002+0000 7fa0aa4d2740 1 -- 172.21.15.5:0/881783443 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cbf5e4ddc0 msgr2=0x55cbf5e58140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:48.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.002+0000 7fa0a6696640 1 --2- 172.21.15.5:0/2245053751 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fa08c0a2b80 0x7fa08c0a5040 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fa09809bd20 tx=0x7fa09806ad10 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:48.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.002+0000 7fa0aa4d2740 1 --2- 172.21.15.5:0/881783443 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cbf5e4ddc0 0x55cbf5e58140 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa09c005940 tx=0x7fa09c005be0 comp rx=0 tx=0).stop 2024-09-12T03:03:48.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.002+0000 7fa0aa4d2740 1 -- 172.21.15.5:0/881783443 shutdown_connections 2024-09-12T03:03:48.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.002+0000 7fa0aa4d2740 1 --2- 172.21.15.5:0/881783443 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbf5e58680 0x55cbf5e5aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:48.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.002+0000 7fa0aa4d2740 1 --2- 172.21.15.5:0/881783443 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cbf5e4ddc0 0x55cbf5e58140 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:48.009 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.002+0000 7fa0aa4d2740 1 --2- 172.21.15.5:0/881783443 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cbf5e4d4a0 0x55cbf5e4d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:48.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.002+0000 7fa0aa4d2740 1 -- 172.21.15.5:0/881783443 >> 172.21.15.5:0/881783443 conn(0x55cbf5e49a30 msgr2=0x55cbf5e49e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:48.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.002+0000 7fa0aa4d2740 1 -- 172.21.15.5:0/881783443 shutdown_connections 2024-09-12T03:03:48.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.002+0000 7fa0aa4d2740 1 -- 172.21.15.5:0/881783443 wait complete. 2024-09-12T03:03:48.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.003+0000 7fa0aa4d2740 1 -- start start 2024-09-12T03:03:48.010 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.003+0000 7fa0aa4d2740 1 Processor -- start 2024-09-12T03:03:48.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.003+0000 7fa0aa4d2740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cbf5e4d4a0 0x55cbf5e57fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:48.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.003+0000 7fa0aa4d2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbf5e4ddc0 0x55cbf5e55250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:48.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.003+0000 7fa0aa4d2740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cbf5e58680 0x55cbf5e55790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:48.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.003+0000 7fa0aa4d2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55cbf5e5b250 con 0x55cbf5e4ddc0 2024-09-12T03:03:48.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.003+0000 7fa0aa4d2740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55cbf5d6e050 con 0x55cbf5e58680 2024-09-12T03:03:48.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.003+0000 7fa0aa4d2740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55cbf5e5b3d0 con 0x55cbf5e4d4a0 2024-09-12T03:03:48.013 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.003+0000 7fa0aa4d2740 10 client.0 _ll_register_callbacks cb 0x55cbf5dae160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:03:48.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.003+0000 7fa0a5694640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cbf5e4d4a0 0x55cbf5e57fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:48.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.003+0000 7fa0a5e95640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbf5e4ddc0 0x55cbf5e55250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:48.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.003+0000 7fa0a5694640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cbf5e4d4a0 0x55cbf5e57fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.5:55050/0 (socket says 172.21.15.5:55050) 2024-09-12T03:03:48.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.003+0000 7fa0a5694640 1 -- 172.21.15.5:0/4291117934 learned_addr learned my addr 172.21.15.5:0/4291117934 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:03:48.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.003+0000 7fa0a5694640 1 --2- 172.21.15.5:0/4291117934 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cbf5e58680 0x55cbf5e55790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:48.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.004+0000 7fa0a5694640 1 -- 172.21.15.5:0/4291117934 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cbf5e58680 msgr2=0x55cbf5e55790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:48.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.004+0000 7fa0a5694640 1 --2- 172.21.15.5:0/4291117934 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cbf5e58680 0x55cbf5e55790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:48.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.004+0000 7fa0a5694640 1 -- 172.21.15.5:0/4291117934 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbf5e4ddc0 msgr2=0x55cbf5e55250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:48.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.004+0000 7fa0a5694640 1 --2- 172.21.15.5:0/4291117934 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbf5e4ddc0 0x55cbf5e55250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:48.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.004+0000 7fa0a5694640 1 -- 172.21.15.5:0/4291117934 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cbf5b243c0 con 0x55cbf5e4d4a0 2024-09-12T03:03:48.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.004+0000 7fa0a5694640 1 --2- 172.21.15.5:0/4291117934 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cbf5e4d4a0 0x55cbf5e57fa0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa090009fc0 tx=0x7fa090004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=38cd5e4102993f5a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:48.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.004+0000 7fa0897fa640 10 client.0 ms_handle_connect on v2:172.21.15.148:3300/0 2024-09-12T03:03:48.014 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.004+0000 7fa0897fa640 1 -- 172.21.15.5:0/4291117934 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa09001fbd0 con 0x55cbf5e4d4a0 2024-09-12T03:03:48.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.004+0000 7fa0aa4d2740 1 -- 172.21.15.5:0/4291117934 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa09c031870 con 0x55cbf5e4d4a0 2024-09-12T03:03:48.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.004+0000 7fa0897fa640 1 -- 172.21.15.5:0/4291117934 <== mon.1 v2:172.21.15.148:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fa09001fd70 con 0x55cbf5e4d4a0 2024-09-12T03:03:48.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.004+0000 7fa0897fa640 1 -- 172.21.15.5:0/4291117934 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fa090004030 con 0x55cbf5e4d4a0 2024-09-12T03:03:48.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.004+0000 7fa0aa4d2740 10 client.26179 fetch_fsmap learned FSMap version 31 2024-09-12T03:03:48.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.004+0000 7fa0aa4d2740 1 -- 172.21.15.5:0/4291117934 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55cbf5e66c80 con 0x55cbf5e4d4a0 2024-09-12T03:03:48.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.005+0000 7fa0897fa640 1 -- 172.21.15.5:0/4291117934 <== mon.1 v2:172.21.15.148:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa09c031870 con 0x55cbf5e4d4a0 2024-09-12T03:03:48.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.005+0000 7fa0897fa640 1 -- 172.21.15.5:0/4291117934 <== mon.1 v2:172.21.15.148:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fa09002a030 con 0x55cbf5e4d4a0 2024-09-12T03:03:48.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.005+0000 7fa0aa4d2740 10 client.26179 fetch_fsmap finished waiting for FSMap version 31 2024-09-12T03:03:48.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.005+0000 7fa0aa4d2740 10 client.26179 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:03:48.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.005+0000 7fa0aa4d2740 20 client.26179 populate_metadata read hostname 'smithi005' 2024-09-12T03:03:48.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.005+0000 7fa0aa4d2740 1 --2- 172.21.15.5:0/4291117934 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55cbf5e68090 0x55cbf5e6a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:48.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.005+0000 7fa0aa4d2740 4 client.26179 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:03:48.015 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.005+0000 7fa0aa4d2740 1 -- 172.21.15.5:0/4291117934 --> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cbf5dc0850 con 0x55cbf5e68090 2024-09-12T03:03:48.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.005+0000 7fa0a5e95640 1 --2- 172.21.15.5:0/4291117934 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55cbf5e68090 0x55cbf5e6a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:48.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.006+0000 7fa0a5e95640 1 --2- 172.21.15.5:0/4291117934 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55cbf5e68090 0x55cbf5e6a480 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1297115cadc69cf server_cookie=4edb84fd20833f9e in_seq=0 out_seq=0 2024-09-12T03:03:48.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.006+0000 7fa0897fa640 10 client.26179 ms_handle_connect on v2:172.21.15.148:6838/2317600000 2024-09-12T03:03:48.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.009+0000 7fa0897fa640 1 -- 172.21.15.5:0/4291117934 <== mds.0 v2:172.21.15.148:6838/2317600000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cbf5dc0850 con 0x55cbf5e68090 2024-09-12T03:03:48.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.009+0000 7fa0897fa640 10 client.26179 handle_command_reply: tid=0 2024-09-12T03:03:48.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.009+0000 7fa0aa4d2740 1 client.26179 shutdown 2024-09-12T03:03:48.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.009+0000 7fa0aa4d2740 1 -- 172.21.15.5:0/4291117934 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cbf5e4d4a0 msgr2=0x55cbf5e57fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:48.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.009+0000 7fa0aa4d2740 1 --2- 172.21.15.5:0/4291117934 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cbf5e4d4a0 0x55cbf5e57fa0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa090009fc0 tx=0x7fa090004730 comp rx=0 tx=0).stop 2024-09-12T03:03:48.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.009+0000 7fa0aa4d2740 1 -- 172.21.15.5:0/4291117934 shutdown_connections 2024-09-12T03:03:48.016 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.009+0000 7fa0aa4d2740 1 --2- 172.21.15.5:0/4291117934 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55cbf5e68090 0x55cbf5e6a480 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:48.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.009+0000 7fa0aa4d2740 1 --2- 172.21.15.5:0/4291117934 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbf5e4ddc0 0x55cbf5e55250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:48.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.009+0000 7fa0aa4d2740 1 --2- 172.21.15.5:0/4291117934 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cbf5e58680 0x55cbf5e55790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:48.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.009+0000 7fa0897fa640 0 client.26179 ms_handle_reset on v2:172.21.15.148:6838/2317600000 2024-09-12T03:03:48.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.009+0000 7fa0aa4d2740 1 --2- 172.21.15.5:0/4291117934 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cbf5e4d4a0 0x55cbf5e57fa0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:48.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.009+0000 7fa0aa4d2740 1 -- 172.21.15.5:0/4291117934 >> 172.21.15.5:0/4291117934 conn(0x55cbf5e49a30 msgr2=0x55cbf5e4f710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:48.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.009+0000 7fa0aa4d2740 1 -- 172.21.15.5:0/4291117934 shutdown_connections 2024-09-12T03:03:48.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.009+0000 7fa0aa4d2740 1 -- 172.21.15.5:0/4291117934 wait complete. 2024-09-12T03:03:48.017 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.009+0000 7fa0aa4d2740 20 client.26179 trim_cache size 0 max 16384 2024-09-12T03:03:48.029 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.028+0000 7fa0aa4d2740 1 Processor -- start 2024-09-12T03:03:48.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.028+0000 7fa0aa4d2740 1 -- start start 2024-09-12T03:03:48.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.028+0000 7fa0aa4d2740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cbf5e4d4a0 0x55cbf5e644b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:48.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.028+0000 7fa0aa4d2740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cbf5e4ddc0 0x55cbf5e6fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:48.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.028+0000 7fa0aa4d2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbf5e58680 0x55cbf5e70240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:48.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.028+0000 7fa0aa4d2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa0a01ab330 con 0x55cbf5e58680 2024-09-12T03:03:48.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.028+0000 7fa0aa4d2740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fa0a01ace70 con 0x55cbf5e4d4a0 2024-09-12T03:03:48.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.028+0000 7fa0aa4d2740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa0a01aba00 con 0x55cbf5e4ddc0 2024-09-12T03:03:48.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.028+0000 7fa0a5694640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbf5e58680 0x55cbf5e70240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:48.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.029+0000 7fa0a5e95640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cbf5e4ddc0 0x55cbf5e6fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:48.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.029+0000 7fa0a5e95640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cbf5e4ddc0 0x55cbf5e6fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.5:55056/0 (socket says 172.21.15.5:55056) 2024-09-12T03:03:48.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.029+0000 7fa0a5694640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbf5e58680 0x55cbf5e70240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39366/0 (socket says 172.21.15.5:39366) 2024-09-12T03:03:48.030 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.029+0000 7fa0a5e95640 1 -- 172.21.15.5:0/15612585 learned_addr learned my addr 172.21.15.5:0/15612585 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:03:48.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.029+0000 7fa0a5694640 1 --2- 172.21.15.5:0/15612585 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cbf5e4d4a0 0x55cbf5e644b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:48.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.029+0000 7fa0a5e95640 1 -- 172.21.15.5:0/15612585 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cbf5e4d4a0 msgr2=0x55cbf5e644b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:48.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.029+0000 7fa0a5e95640 1 --2- 172.21.15.5:0/15612585 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cbf5e4d4a0 0x55cbf5e644b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:48.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.029+0000 7fa0a5e95640 1 -- 172.21.15.5:0/15612585 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbf5e58680 msgr2=0x55cbf5e70240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:48.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.029+0000 7fa0a5e95640 1 --2- 172.21.15.5:0/15612585 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbf5e58680 0x55cbf5e70240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:48.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.029+0000 7fa0a5e95640 1 -- 172.21.15.5:0/15612585 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cbf5e4a6c0 con 0x55cbf5e4ddc0 2024-09-12T03:03:48.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.029+0000 7fa0a5e95640 1 --2- 172.21.15.5:0/15612585 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cbf5e4ddc0 0x55cbf5e6fd00 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa09c031470 tx=0x7fa09c0058d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b419e97bd1fab7c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:48.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.029+0000 7fa08b7fe640 1 -- 172.21.15.5:0/15612585 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa09c040040 con 0x55cbf5e4ddc0 2024-09-12T03:03:48.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.029+0000 7fa08b7fe640 1 -- 172.21.15.5:0/15612585 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa09c0035f0 con 0x55cbf5e4ddc0 2024-09-12T03:03:48.031 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.029+0000 7fa08b7fe640 1 -- 172.21.15.5:0/15612585 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa09c035040 con 0x55cbf5e4ddc0 2024-09-12T03:03:48.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.030+0000 7fa0aa4d2740 1 -- 172.21.15.5:0/15612585 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cbf5e4ddc0 msgr2=0x55cbf5e6fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:48.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.030+0000 7fa0aa4d2740 1 --2- 172.21.15.5:0/15612585 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cbf5e4ddc0 0x55cbf5e6fd00 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa09c031470 tx=0x7fa09c0058d0 comp rx=0 tx=0).stop 2024-09-12T03:03:48.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.030+0000 7fa0aa4d2740 1 -- 172.21.15.5:0/15612585 shutdown_connections 2024-09-12T03:03:48.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.030+0000 7fa0aa4d2740 1 --2- 172.21.15.5:0/15612585 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbf5e58680 0x55cbf5e70240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:48.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.030+0000 7fa0aa4d2740 1 --2- 172.21.15.5:0/15612585 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cbf5e4ddc0 0x55cbf5e6fd00 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:48.034 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.030+0000 7fa0aa4d2740 1 --2- 172.21.15.5:0/15612585 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cbf5e4d4a0 0x55cbf5e644b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:48.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.030+0000 7fa0aa4d2740 1 -- 172.21.15.5:0/15612585 >> 172.21.15.5:0/15612585 conn(0x55cbf5e49a30 msgr2=0x55cbf5e50900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:48.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.030+0000 7fa0aa4d2740 1 -- 172.21.15.5:0/15612585 shutdown_connections 2024-09-12T03:03:48.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.030+0000 7fa0aa4d2740 1 -- 172.21.15.5:0/15612585 wait complete. 2024-09-12T03:03:48.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.030+0000 7fa0aa4d2740 1 -- start start 2024-09-12T03:03:48.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.030+0000 7fa0aa4d2740 1 Processor -- start 2024-09-12T03:03:48.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.031+0000 7fa0aa4d2740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbf5e4d4a0 0x55cbf5e60eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:48.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.031+0000 7fa0aa4d2740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cbf5e4ddc0 0x55cbf5e5c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:48.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.031+0000 7fa0aa4d2740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cbf5e58680 0x55cbf5e5c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:48.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.031+0000 7fa0aa4d2740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fa0a01ac7a0 con 0x55cbf5e4d4a0 2024-09-12T03:03:48.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.031+0000 7fa0aa4d2740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fa0a01ac0d0 con 0x55cbf5e4ddc0 2024-09-12T03:03:48.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.031+0000 7fa0aa4d2740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fa0a01a9310 con 0x55cbf5e58680 2024-09-12T03:03:48.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.031+0000 7fa0aa4d2740 10 client.0 _ll_register_callbacks cb 0x7fa090005890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:03:48.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.031+0000 7fa0a5694640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbf5e4d4a0 0x55cbf5e60eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:48.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.031+0000 7fa0a5694640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbf5e4d4a0 0x55cbf5e60eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39370/0 (socket says 172.21.15.5:39370) 2024-09-12T03:03:48.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.031+0000 7fa0a5694640 1 -- 172.21.15.5:0/1638893212 learned_addr learned my addr 172.21.15.5:0/1638893212 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:03:48.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.031+0000 7fa0a5e95640 1 --2- 172.21.15.5:0/1638893212 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cbf5e4ddc0 0x55cbf5e5c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:48.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.031+0000 7fa0a5694640 1 --2- 172.21.15.5:0/1638893212 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cbf5e58680 0x55cbf5e5c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:48.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.031+0000 7fa0a5694640 1 -- 172.21.15.5:0/1638893212 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cbf5e58680 msgr2=0x55cbf5e5c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:48.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.031+0000 7fa0a5694640 1 --2- 172.21.15.5:0/1638893212 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cbf5e58680 0x55cbf5e5c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:48.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.031+0000 7fa0a5694640 1 -- 172.21.15.5:0/1638893212 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cbf5e4ddc0 msgr2=0x55cbf5e5c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:48.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.031+0000 7fa0a5694640 1 --2- 172.21.15.5:0/1638893212 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cbf5e4ddc0 0x55cbf5e5c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:48.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.031+0000 7fa0a5694640 1 -- 172.21.15.5:0/1638893212 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cbf5e5d020 con 0x55cbf5e4d4a0 2024-09-12T03:03:48.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.031+0000 7fa0a5e95640 1 --2- 172.21.15.5:0/1638893212 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cbf5e4ddc0 0x55cbf5e5c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:03:48.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.032+0000 7fa0a5694640 1 --2- 172.21.15.5:0/1638893212 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbf5e4d4a0 0x55cbf5e60eb0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fa090021620 tx=0x7fa090012040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=533afb53bda9fe41 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:48.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.032+0000 7fa0897fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T03:03:48.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.032+0000 7fa0897fa640 1 -- 172.21.15.5:0/1638893212 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa09001fc90 con 0x55cbf5e4d4a0 2024-09-12T03:03:48.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.032+0000 7fa0897fa640 1 -- 172.21.15.5:0/1638893212 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fa09001fe30 con 0x55cbf5e4d4a0 2024-09-12T03:03:48.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.032+0000 7fa0aa4d2740 1 -- 172.21.15.5:0/1638893212 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa09c0035f0 con 0x55cbf5e4d4a0 2024-09-12T03:03:48.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.032+0000 7fa0897fa640 1 -- 172.21.15.5:0/1638893212 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fa090004ea0 con 0x55cbf5e4d4a0 2024-09-12T03:03:48.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.032+0000 7fa0aa4d2740 10 client.16704 fetch_fsmap learned FSMap version 31 2024-09-12T03:03:48.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.032+0000 7fa0aa4d2740 1 -- 172.21.15.5:0/1638893212 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55cbf5ed2e10 con 0x55cbf5e4d4a0 2024-09-12T03:03:48.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.032+0000 7fa0897fa640 1 -- 172.21.15.5:0/1638893212 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa09c0035f0 con 0x55cbf5e4d4a0 2024-09-12T03:03:48.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.032+0000 7fa0897fa640 1 -- 172.21.15.5:0/1638893212 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7fa09002a070 con 0x55cbf5e4d4a0 2024-09-12T03:03:48.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.032+0000 7fa0aa4d2740 10 client.16704 fetch_fsmap finished waiting for FSMap version 31 2024-09-12T03:03:48.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.032+0000 7fa0aa4d2740 10 client.16704 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:03:48.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.032+0000 7fa0aa4d2740 20 client.16704 populate_metadata read hostname 'smithi005' 2024-09-12T03:03:48.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.032+0000 7fa0aa4d2740 1 --2- 172.21.15.5:0/1638893212 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55cbf5ed3470 0x55cbf5ed5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:48.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.032+0000 7fa0aa4d2740 4 client.16704 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T03:03:48.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.032+0000 7fa0aa4d2740 1 -- 172.21.15.5:0/1638893212 --> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55cbf5e55b20 con 0x55cbf5ed3470 2024-09-12T03:03:48.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.033+0000 7fa0a5e95640 1 --2- 172.21.15.5:0/1638893212 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55cbf5ed3470 0x55cbf5ed5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:48.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.033+0000 7fa0a5e95640 1 --2- 172.21.15.5:0/1638893212 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55cbf5ed3470 0x55cbf5ed5890 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4aeda463440f900 server_cookie=834ddc374c341f20 in_seq=0 out_seq=0 2024-09-12T03:03:48.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.033+0000 7fa0897fa640 10 client.16704 ms_handle_connect on v2:172.21.15.148:6838/2317600000 2024-09-12T03:03:48.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.037+0000 7fa0897fa640 1 -- 172.21.15.5:0/1638893212 <== mds.0 v2:172.21.15.148:6838/2317600000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+268 (crc 0 0 0) 0x55cbf5e55b20 con 0x55cbf5ed3470 2024-09-12T03:03:48.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.037+0000 7fa0897fa640 10 client.16704 handle_command_reply: tid=0 2024-09-12T03:03:48.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.037+0000 7fa0aa4d2740 1 client.16704 shutdown 2024-09-12T03:03:48.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.038+0000 7fa0aa4d2740 1 -- 172.21.15.5:0/1638893212 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbf5e4d4a0 msgr2=0x55cbf5e60eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:48.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.038+0000 7fa0aa4d2740 1 --2- 172.21.15.5:0/1638893212 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbf5e4d4a0 0x55cbf5e60eb0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fa090021620 tx=0x7fa090012040 comp rx=0 tx=0).stop 2024-09-12T03:03:48.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.038+0000 7fa0aa4d2740 1 -- 172.21.15.5:0/1638893212 shutdown_connections 2024-09-12T03:03:48.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.038+0000 7fa0aa4d2740 1 --2- 172.21.15.5:0/1638893212 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55cbf5e58680 0x55cbf5e5c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:48.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.038+0000 7fa0aa4d2740 1 --2- 172.21.15.5:0/1638893212 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55cbf5e4ddc0 0x55cbf5e5c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:48.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.038+0000 7fa0aa4d2740 1 --2- 172.21.15.5:0/1638893212 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x55cbf5ed3470 0x55cbf5ed5890 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:48.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.038+0000 7fa0aa4d2740 1 --2- 172.21.15.5:0/1638893212 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbf5e4d4a0 0x55cbf5e60eb0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:48.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.038+0000 7fa0897fa640 0 client.16704 ms_handle_reset on v2:172.21.15.148:6838/2317600000 2024-09-12T03:03:48.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.038+0000 7fa0aa4d2740 1 -- 172.21.15.5:0/1638893212 >> 172.21.15.5:0/1638893212 conn(0x55cbf5e49a30 msgr2=0x55cbf5e56ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:48.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.038+0000 7fa0aa4d2740 1 -- 172.21.15.5:0/1638893212 shutdown_connections 2024-09-12T03:03:48.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.038+0000 7fa0aa4d2740 1 -- 172.21.15.5:0/1638893212 wait complete. 2024-09-12T03:03:48.040 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.038+0000 7fa0aa4d2740 20 client.16704 trim_cache size 0 max 16384 2024-09-12T03:03:48.040 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-12T03:03:48.041 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "scrub active (1431 inodes in the stack)", 2024-09-12T03:03:48.041 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2024-09-12T03:03:48.041 INFO:teuthology.orchestra.run.smithi005.stdout: "efd99e26-f40b-4fb4-b002-9dbcf78af4c1": { 2024-09-12T03:03:48.041 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2024-09-12T03:03:48.041 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "efd99e26-f40b-4fb4-b002-9dbcf78af4c1", 2024-09-12T03:03:48.041 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2024-09-12T03:03:48.041 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:03:48.041 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:03:48.041 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-12T03:03:48.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.038+0000 7fa0a80f9640 1 -- 172.21.15.5:0/2245053751 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fa08c0a2b80 msgr2=0x7fa08c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:48.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.038+0000 7fa0a80f9640 1 --2- 172.21.15.5:0/2245053751 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fa08c0a2b80 0x7fa08c0a5040 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fa09809bd20 tx=0x7fa09806ad10 comp rx=0 tx=0).stop 2024-09-12T03:03:48.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.038+0000 7fa0a80f9640 1 -- 172.21.15.5:0/2245053751 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0a0106f00 msgr2=0x7fa0a019cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:48.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.038+0000 7fa0a80f9640 1 --2- 172.21.15.5:0/2245053751 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0a0106f00 0x7fa0a019cdf0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fa09800eb10 tx=0x7fa09800c2d0 comp rx=0 tx=0).stop 2024-09-12T03:03:48.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.038+0000 7fa0a80f9640 1 -- 172.21.15.5:0/2245053751 shutdown_connections 2024-09-12T03:03:48.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.038+0000 7fa0a80f9640 1 --2- 172.21.15.5:0/2245053751 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fa08c0a2b80 0x7fa08c0a5040 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:48.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.038+0000 7fa0a80f9640 1 --2- 172.21.15.5:0/2245053751 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0a0106f00 0x7fa0a019cdf0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:48.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.038+0000 7fa0a80f9640 1 --2- 172.21.15.5:0/2245053751 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fa0a0102780 0x7fa0a0198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:48.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.038+0000 7fa0a80f9640 1 --2- 172.21.15.5:0/2245053751 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fa0a0101dd0 0x7fa0a0198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:48.042 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.038+0000 7fa0a80f9640 1 -- 172.21.15.5:0/2245053751 >> 172.21.15.5:0/2245053751 conn(0x7fa0a00fd8b0 msgr2=0x7fa0a00feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:48.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.039+0000 7fa0a80f9640 1 -- 172.21.15.5:0/2245053751 shutdown_connections 2024-09-12T03:03:48.043 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:48.039+0000 7fa0a80f9640 1 -- 172.21.15.5:0/2245053751 wait complete. 2024-09-12T03:03:48.810 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:03:48 smithi005 ceph-mon[31197]: pgmap v502: 97 pgs: 97 active+clean; 145 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 2.3 MiB/s wr, 881 op/s 2024-09-12T03:03:48.862 INFO:tasks.cephfs.filesystem:scrub status for tag:efd99e26-f40b-4fb4-b002-9dbcf78af4c1 - {'path': '/', 'tag': 'efd99e26-f40b-4fb4-b002-9dbcf78af4c1', 'options': 'recursive,force'} 2024-09-12T03:03:49.165 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:03:48 smithi148 ceph-mon[35090]: pgmap v502: 97 pgs: 97 active+clean; 145 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 2.3 MiB/s wr, 881 op/s 2024-09-12T03:03:49.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:03:48 smithi146 ceph-mon[35011]: pgmap v502: 97 pgs: 97 active+clean; 145 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 2.3 MiB/s wr, 881 op/s 2024-09-12T03:03:51.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:03:50 smithi005 ceph-mon[31197]: pgmap v503: 97 pgs: 97 active+clean; 145 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 2.7 MiB/s wr, 840 op/s 2024-09-12T03:03:51.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:03:50 smithi005 ceph-mon[31197]: scrub summary: active paths [/] 2024-09-12T03:03:51.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:03:50 smithi005 ceph-mon[31197]: scrub summary: idle+waiting paths [/] 2024-09-12T03:03:51.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:03:50 smithi005 ceph-mon[31197]: scrub summary: active paths [/] 2024-09-12T03:03:51.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:03:50 smithi005 ceph-mon[31197]: scrub summary: idle+waiting paths [/] 2024-09-12T03:03:51.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:03:50 smithi005 ceph-mon[31197]: scrub summary: idle+waiting paths [/] 2024-09-12T03:03:51.165 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:03:50 smithi148 ceph-mon[35090]: pgmap v503: 97 pgs: 97 active+clean; 145 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 2.7 MiB/s wr, 840 op/s 2024-09-12T03:03:51.165 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:03:50 smithi148 ceph-mon[35090]: scrub summary: active paths [/] 2024-09-12T03:03:51.165 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:03:50 smithi148 ceph-mon[35090]: scrub summary: idle+waiting paths [/] 2024-09-12T03:03:51.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:03:50 smithi148 ceph-mon[35090]: scrub summary: active paths [/] 2024-09-12T03:03:51.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:03:50 smithi148 ceph-mon[35090]: scrub summary: idle+waiting paths [/] 2024-09-12T03:03:51.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:03:50 smithi148 ceph-mon[35090]: scrub summary: idle+waiting paths [/] 2024-09-12T03:03:51.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:03:50 smithi146 ceph-mon[35011]: pgmap v503: 97 pgs: 97 active+clean; 145 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 2.7 MiB/s wr, 840 op/s 2024-09-12T03:03:51.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:03:50 smithi146 ceph-mon[35011]: scrub summary: active paths [/] 2024-09-12T03:03:51.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:03:50 smithi146 ceph-mon[35011]: scrub summary: idle+waiting paths [/] 2024-09-12T03:03:51.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:03:50 smithi146 ceph-mon[35011]: scrub summary: active paths [/] 2024-09-12T03:03:51.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:03:50 smithi146 ceph-mon[35011]: scrub summary: idle+waiting paths [/] 2024-09-12T03:03:51.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:03:50 smithi146 ceph-mon[35011]: scrub summary: idle+waiting paths [/] 2024-09-12T03:03:53.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:03:52 smithi005 ceph-mon[31197]: pgmap v504: 97 pgs: 97 active+clean; 134 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 2.2 MiB/s wr, 712 op/s 2024-09-12T03:03:53.165 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:03:52 smithi148 ceph-mon[35090]: pgmap v504: 97 pgs: 97 active+clean; 134 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 2.2 MiB/s wr, 712 op/s 2024-09-12T03:03:53.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:03:52 smithi146 ceph-mon[35011]: pgmap v504: 97 pgs: 97 active+clean; 134 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 2.2 MiB/s wr, 712 op/s 2024-09-12T03:03:55.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:03:54 smithi005 ceph-mon[31197]: pgmap v505: 97 pgs: 97 active+clean; 135 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 2.2 MiB/s wr, 689 op/s 2024-09-12T03:03:55.165 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:03:54 smithi148 ceph-mon[35090]: pgmap v505: 97 pgs: 97 active+clean; 135 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 2.2 MiB/s wr, 689 op/s 2024-09-12T03:03:55.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:03:54 smithi146 ceph-mon[35011]: pgmap v505: 97 pgs: 97 active+clean; 135 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 2.2 MiB/s wr, 689 op/s 2024-09-12T03:03:55.913 INFO:tasks.workunit:Stopping ['suites/fsx.sh'] on client.0... 2024-09-12T03:03:55.913 DEBUG:teuthology.orchestra.run.smithi005:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-12T03:03:56.237 DEBUG:teuthology.parallel:result is None 2024-09-12T03:03:56.238 DEBUG:teuthology.orchestra.run.smithi005:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-09-12T03:03:56.275 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-09-12T03:03:56.275 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-09-12T03:03:56.286 INFO:tasks.quiescer:joining Quiescers 2024-09-12T03:03:56.286 WARNING:tasks.quiescer:The quiescer is requested to stop, running cancel all 2024-09-12T03:03:56.286 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:03:56.584 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:03:57.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:03:56 smithi005 ceph-mon[31197]: pgmap v506: 97 pgs: 97 active+clean; 144 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 2.1 MiB/s wr, 551 op/s 2024-09-12T03:03:57.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:03:56 smithi005 ceph-mon[31197]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:03:57.165 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:03:56 smithi148 ceph-mon[35090]: pgmap v506: 97 pgs: 97 active+clean; 144 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 2.1 MiB/s wr, 551 op/s 2024-09-12T03:03:57.165 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:03:56 smithi148 ceph-mon[35090]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:03:57.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:03:56 smithi146 ceph-mon[35011]: pgmap v506: 97 pgs: 97 active+clean; 144 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 2.1 MiB/s wr, 551 op/s 2024-09-12T03:03:57.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:03:56 smithi146 ceph-mon[35011]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:03:57.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:57.412+0000 7fc428ddb640 1 -- 172.21.15.5:0/1378164627 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc424106f10 msgr2=0x7fc42410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:57.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:57.412+0000 7fc428ddb640 1 --2- 172.21.15.5:0/1378164627 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc424106f10 0x7fc42410ea90 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fc41c009ca0 tx=0x7fc41c02a8c0 comp rx=0 tx=0).stop 2024-09-12T03:03:57.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:57.412+0000 7fc428ddb640 1 -- 172.21.15.5:0/1378164627 shutdown_connections 2024-09-12T03:03:57.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:57.412+0000 7fc428ddb640 1 --2- 172.21.15.5:0/1378164627 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc424106f10 0x7fc42410ea90 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:57.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:57.412+0000 7fc428ddb640 1 --2- 172.21.15.5:0/1378164627 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc424102790 0x7fc4241067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:57.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:57.412+0000 7fc428ddb640 1 --2- 172.21.15.5:0/1378164627 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc424101de0 0x7fc4241021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:57.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:57.412+0000 7fc428ddb640 1 -- 172.21.15.5:0/1378164627 >> 172.21.15.5:0/1378164627 conn(0x7fc4240fd8c0 msgr2=0x7fc4240ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:57.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:57.412+0000 7fc428ddb640 1 -- 172.21.15.5:0/1378164627 shutdown_connections 2024-09-12T03:03:57.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:57.412+0000 7fc428ddb640 1 -- 172.21.15.5:0/1378164627 wait complete. 2024-09-12T03:03:57.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:57.412+0000 7fc428ddb640 1 Processor -- start 2024-09-12T03:03:57.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:57.413+0000 7fc428ddb640 1 -- start start 2024-09-12T03:03:57.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:57.413+0000 7fc428ddb640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc424101de0 0x7fc4241982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:57.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:57.413+0000 7fc428ddb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc424102790 0x7fc4241987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:57.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:57.413+0000 7fc428ddb640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc424106f10 0x7fc42419cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:57.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:57.413+0000 7fc428ddb640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc424111f70 con 0x7fc424102790 2024-09-12T03:03:57.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:57.413+0000 7fc428ddb640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fc424111df0 con 0x7fc424106f10 2024-09-12T03:03:57.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:57.413+0000 7fc428ddb640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc4241120f0 con 0x7fc424101de0 2024-09-12T03:03:57.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:57.413+0000 7fc41bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc424102790 0x7fc4241987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:57.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:57.413+0000 7fc41bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc424102790 0x7fc4241987f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56040/0 (socket says 172.21.15.5:56040) 2024-09-12T03:03:57.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:57.413+0000 7fc41bfff640 1 -- 172.21.15.5:0/496192221 learned_addr learned my addr 172.21.15.5:0/496192221 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:03:57.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:57.413+0000 7fc422d9d640 1 --2- 172.21.15.5:0/496192221 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc424106f10 0x7fc42419cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:57.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:57.413+0000 7fc42259c640 1 --2- 172.21.15.5:0/496192221 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc424101de0 0x7fc4241982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:57.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:57.413+0000 7fc41bfff640 1 -- 172.21.15.5:0/496192221 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc424101de0 msgr2=0x7fc4241982b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:57.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:57.413+0000 7fc41bfff640 1 --2- 172.21.15.5:0/496192221 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc424101de0 0x7fc4241982b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:57.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:57.413+0000 7fc41bfff640 1 -- 172.21.15.5:0/496192221 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc424106f10 msgr2=0x7fc42419cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:57.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:57.413+0000 7fc41bfff640 1 --2- 172.21.15.5:0/496192221 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc424106f10 0x7fc42419cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:57.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:57.413+0000 7fc41bfff640 1 -- 172.21.15.5:0/496192221 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc42419d260 con 0x7fc424102790 2024-09-12T03:03:57.416 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:57.414+0000 7fc41bfff640 1 --2- 172.21.15.5:0/496192221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc424102790 0x7fc4241987f0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fc40c00c920 tx=0x7fc40c00cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:57.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:57.414+0000 7fc41b7fe640 1 -- 172.21.15.5:0/496192221 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc40c007bb0 con 0x7fc424102790 2024-09-12T03:03:57.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:57.414+0000 7fc428ddb640 1 -- 172.21.15.5:0/496192221 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc42419d550 con 0x7fc424102790 2024-09-12T03:03:57.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:57.414+0000 7fc41b7fe640 1 -- 172.21.15.5:0/496192221 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc40c007d50 con 0x7fc424102790 2024-09-12T03:03:57.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:57.414+0000 7fc41b7fe640 1 -- 172.21.15.5:0/496192221 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc40c018780 con 0x7fc424102790 2024-09-12T03:03:57.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:57.414+0000 7fc428ddb640 1 -- 172.21.15.5:0/496192221 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4241a4e30 con 0x7fc424102790 2024-09-12T03:03:57.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:57.415+0000 7fc428ddb640 1 -- 172.21.15.5:0/496192221 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4241021c0 con 0x7fc424102790 2024-09-12T03:03:57.417 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:57.416+0000 7fc41b7fe640 1 -- 172.21.15.5:0/496192221 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fc40c00f450 con 0x7fc424102790 2024-09-12T03:03:57.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:57.417+0000 7fc41b7fe640 1 --2- 172.21.15.5:0/496192221 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fc4100a2b80 0x7fc4100a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:57.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:57.417+0000 7fc42259c640 1 --2- 172.21.15.5:0/496192221 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fc4100a2b80 0x7fc4100a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:57.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:57.417+0000 7fc41b7fe640 1 -- 172.21.15.5:0/496192221 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fc40c0cefb0 con 0x7fc424102790 2024-09-12T03:03:57.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:57.418+0000 7fc42259c640 1 --2- 172.21.15.5:0/496192221 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fc4100a2b80 0x7fc4100a5040 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fc41400abe0 tx=0x7fc41400a2d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:57.422 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:57.422+0000 7fc41b7fe640 1 -- 172.21.15.5:0/496192221 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc40c014030 con 0x7fc424102790 2024-09-12T03:03:57.634 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:57.634+0000 7fc428ddb640 1 -- 172.21.15.5:0/496192221 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc4241a50f0 con 0x7fc424102790 2024-09-12T03:03:57.635 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:57.635+0000 7fc41b7fe640 1 -- 172.21.15.5:0/496192221 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11968 (secure 0 0 0) 0x7fc40c09bfc0 con 0x7fc424102790 2024-09-12T03:03:57.636 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-12T03:03:57.636 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":32,"btime":"2024-09-12T03:03:55:793984+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/4135232521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":4135232521},{"type":"v1","addr":"172.21.15.5:6835","nonce":4135232521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/4290221940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":4290221940},{"type":"v1","addr":"172.21.15.5:6837","nonce":4290221940}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T02:56:16.595562+0000","modified":"2024-09-12T03:03:55.239382+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24502,"mds_1":24487,"mds_2":24481,"mds_3":24479,"mds_4":24458},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/26447341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":26447341},{"type":"v1","addr":"172.21.15.5:6839","nonce":26447341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2393443295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2393443295},{"type":"v1","addr":"172.21.15.5:6841","nonce":2393443295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6833/3686970137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3686970137},{"type":"v1","addr":"172.21.15.146:6833","nonce":3686970137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6833/568512209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":568512209},{"type":"v1","addr":"172.21.15.148:6833","nonce":568512209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6835/2540740420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2540740420},{"type":"v1","addr":"172.21.15.146:6835","nonce":2540740420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.146:6837/1673370937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":1673370937},{"type":"v1","addr":"172.21.15.146:6837","nonce":1673370937}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":3,"incarnation":22,"state":"up:active","state_seq":6,"addr":"172.21.15.146:6839/3036767228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":3036767228},{"type":"v1","addr":"172.21.15.146:6839","nonce":3036767228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":2,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.148:6835/4195086427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":4195086427},{"type":"v1","addr":"172.21.15.148:6835","nonce":4195086427}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.148:6837/2606873922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2606873922},{"type":"v1","addr":"172.21.15.148:6837","nonce":2606873922}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6839/2317600000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":2317600000},{"type":"v1","addr":"172.21.15.148:6839","nonce":2317600000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24502,"qdb_cluster":[24502,24487,24481,24479,24458]},"id":1}]} 2024-09-12T03:03:57.636 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 32 2024-09-12T03:03:57.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:57.638+0000 7fc428ddb640 1 -- 172.21.15.5:0/496192221 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fc4100a2b80 msgr2=0x7fc4100a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:57.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:57.638+0000 7fc428ddb640 1 --2- 172.21.15.5:0/496192221 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fc4100a2b80 0x7fc4100a5040 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fc41400abe0 tx=0x7fc41400a2d0 comp rx=0 tx=0).stop 2024-09-12T03:03:57.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:57.638+0000 7fc428ddb640 1 -- 172.21.15.5:0/496192221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc424102790 msgr2=0x7fc4241987f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:57.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:57.638+0000 7fc428ddb640 1 --2- 172.21.15.5:0/496192221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc424102790 0x7fc4241987f0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fc40c00c920 tx=0x7fc40c00cd20 comp rx=0 tx=0).stop 2024-09-12T03:03:57.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:57.638+0000 7fc428ddb640 1 -- 172.21.15.5:0/496192221 shutdown_connections 2024-09-12T03:03:57.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:57.638+0000 7fc428ddb640 1 --2- 172.21.15.5:0/496192221 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fc4100a2b80 0x7fc4100a5040 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:57.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:57.638+0000 7fc428ddb640 1 --2- 172.21.15.5:0/496192221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc424102790 0x7fc4241987f0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:57.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:57.638+0000 7fc428ddb640 1 --2- 172.21.15.5:0/496192221 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc424106f10 0x7fc42419cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:57.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:57.638+0000 7fc428ddb640 1 --2- 172.21.15.5:0/496192221 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc424101de0 0x7fc4241982b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:57.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:57.638+0000 7fc428ddb640 1 -- 172.21.15.5:0/496192221 >> 172.21.15.5:0/496192221 conn(0x7fc4240fd8c0 msgr2=0x7fc4240fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:57.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:57.638+0000 7fc428ddb640 1 -- 172.21.15.5:0/496192221 shutdown_connections 2024-09-12T03:03:57.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:57.638+0000 7fc428ddb640 1 -- 172.21.15.5:0/496192221 wait complete. 2024-09-12T03:03:58.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:03:57 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/496192221' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:03:58.165 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:03:57 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/496192221' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:03:58.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:03:57 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/496192221' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:03:58.402 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24502 quiesce db --cancel --all' 2024-09-12T03:03:58.402 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- timeout 105 ceph tell mds.24502 quiesce db --cancel --all 2024-09-12T03:03:58.662 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:03:59.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:03:58 smithi005 ceph-mon[31197]: pgmap v507: 97 pgs: 97 active+clean; 135 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.8 MiB/s wr, 527 op/s 2024-09-12T03:03:59.059 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:03:58 smithi005 ceph-mon[31197]: scrub summary: idle 2024-09-12T03:03:59.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:03:58 smithi005 ceph-mon[31197]: scrub summary: idle 2024-09-12T03:03:59.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:03:58 smithi005 ceph-mon[31197]: scrub summary: idle 2024-09-12T03:03:59.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:03:58 smithi005 ceph-mon[31197]: scrub summary: idle 2024-09-12T03:03:59.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:03:58 smithi005 ceph-mon[31197]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:03:59.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.148+0000 7fece8c52640 1 -- 172.21.15.5:0/673975183 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fece41009d0 msgr2=0x7fece4100db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:59.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.148+0000 7fece8c52640 1 --2- 172.21.15.5:0/673975183 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fece41009d0 0x7fece4100db0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fecd4009ca0 tx=0x7fecd402a8c0 comp rx=0 tx=0).stop 2024-09-12T03:03:59.148 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.148+0000 7fece8c52640 1 -- 172.21.15.5:0/673975183 shutdown_connections 2024-09-12T03:03:59.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.148+0000 7fece8c52640 1 --2- 172.21.15.5:0/673975183 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fece410e4f0 0x7fece4110900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:59.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.148+0000 7fece8c52640 1 --2- 172.21.15.5:0/673975183 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fece4101380 0x7fece410dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:59.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.148+0000 7fece8c52640 1 --2- 172.21.15.5:0/673975183 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fece41009d0 0x7fece4100db0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:59.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.148+0000 7fece8c52640 1 -- 172.21.15.5:0/673975183 >> 172.21.15.5:0/673975183 conn(0x7fece40fc7d0 msgr2=0x7fece40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:59.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.148+0000 7fece8c52640 1 -- 172.21.15.5:0/673975183 shutdown_connections 2024-09-12T03:03:59.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.148+0000 7fece8c52640 1 -- 172.21.15.5:0/673975183 wait complete. 2024-09-12T03:03:59.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.149+0000 7fece8c52640 1 Processor -- start 2024-09-12T03:03:59.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.149+0000 7fece8c52640 1 -- start start 2024-09-12T03:03:59.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.149+0000 7fece8c52640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fece41009d0 0x7fece41a0400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:59.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.149+0000 7fece8c52640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fece4101380 0x7fece41a0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:59.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.149+0000 7fece8c52640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fece410e4f0 0x7fece419a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:59.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.149+0000 7fece8c52640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fece41142a0 con 0x7fece41009d0 2024-09-12T03:03:59.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.149+0000 7fece8c52640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fece4114120 con 0x7fece410e4f0 2024-09-12T03:03:59.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.149+0000 7fece8c52640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fece4114420 con 0x7fece4101380 2024-09-12T03:03:59.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.149+0000 7fece259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fece41009d0 0x7fece41a0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:59.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.149+0000 7fece259c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fece41009d0 0x7fece41a0400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56060/0 (socket says 172.21.15.5:56060) 2024-09-12T03:03:59.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.149+0000 7fece259c640 1 -- 172.21.15.5:0/570665278 learned_addr learned my addr 172.21.15.5:0/570665278 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:03:59.151 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.149+0000 7fece1d9b640 1 --2- 172.21.15.5:0/570665278 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fece4101380 0x7fece41a0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:59.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.149+0000 7fece259c640 1 -- 172.21.15.5:0/570665278 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fece4101380 msgr2=0x7fece41a0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:59.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.149+0000 7fece2d9d640 1 --2- 172.21.15.5:0/570665278 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fece410e4f0 0x7fece419a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:59.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.149+0000 7fece259c640 1 --2- 172.21.15.5:0/570665278 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fece4101380 0x7fece41a0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:59.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.149+0000 7fece259c640 1 -- 172.21.15.5:0/570665278 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fece410e4f0 msgr2=0x7fece419a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:59.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.149+0000 7fece259c640 1 --2- 172.21.15.5:0/570665278 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fece410e4f0 0x7fece419a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:59.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.149+0000 7fece259c640 1 -- 172.21.15.5:0/570665278 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fece419acf0 con 0x7fece41009d0 2024-09-12T03:03:59.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.150+0000 7fece259c640 1 --2- 172.21.15.5:0/570665278 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fece41009d0 0x7fece41a0400 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fecd4005e30 tx=0x7fecd4003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:59.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.150+0000 7fecd37fe640 1 -- 172.21.15.5:0/570665278 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecd40316d0 con 0x7fece41009d0 2024-09-12T03:03:59.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.150+0000 7fecd37fe640 1 -- 172.21.15.5:0/570665278 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fecd4031870 con 0x7fece41009d0 2024-09-12T03:03:59.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.150+0000 7fece8c52640 1 -- 172.21.15.5:0/570665278 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fece419af80 con 0x7fece41009d0 2024-09-12T03:03:59.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.150+0000 7fecd37fe640 1 -- 172.21.15.5:0/570665278 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecd402fb90 con 0x7fece41009d0 2024-09-12T03:03:59.152 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.150+0000 7fece8c52640 1 -- 172.21.15.5:0/570665278 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fece41a72d0 con 0x7fece41009d0 2024-09-12T03:03:59.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.152+0000 7fecd37fe640 1 -- 172.21.15.5:0/570665278 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fecd4004040 con 0x7fece41009d0 2024-09-12T03:03:59.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.153+0000 7feceb02b740 1 Processor -- start 2024-09-12T03:03:59.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.153+0000 7feceb02b740 1 -- start start 2024-09-12T03:03:59.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.153+0000 7feceb02b740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564c66033a00 0x564c66033de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:59.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.153+0000 7feceb02b740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564c66034320 0x564c6603e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:59.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.153+0000 7feceb02b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c6603ebe0 0x564c66040fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:59.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.153+0000 7feceb02b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564c65f98200 con 0x564c6603ebe0 2024-09-12T03:03:59.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.153+0000 7feceb02b740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x564c65f3edc0 con 0x564c66033a00 2024-09-12T03:03:59.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.153+0000 7feceb02b740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x564c65f941c0 con 0x564c66034320 2024-09-12T03:03:59.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.153+0000 7fecd37fe640 1 --2- 172.21.15.5:0/570665278 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7feccc0a2b80 0x7feccc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:59.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.154+0000 7fece1d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c6603ebe0 0x564c66040fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:59.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.154+0000 7fece1d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c6603ebe0 0x564c66040fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56064/0 (socket says 172.21.15.5:56064) 2024-09-12T03:03:59.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.154+0000 7fece1d9b640 1 -- 172.21.15.5:0/964454505 learned_addr learned my addr 172.21.15.5:0/964454505 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:03:59.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.154+0000 7fece259c640 1 --2- 172.21.15.5:0/964454505 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564c66034320 0x564c6603e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:59.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.154+0000 7fece2d9d640 1 --2- 172.21.15.5:0/964454505 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564c66033a00 0x564c66033de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:59.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.154+0000 7fece2d9d640 1 --2- 172.21.15.5:0/570665278 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7feccc0a2b80 0x7feccc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:59.155 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.154+0000 7fece1d9b640 1 -- 172.21.15.5:0/964454505 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564c66034320 msgr2=0x564c6603e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:59.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.154+0000 7fece1d9b640 1 --2- 172.21.15.5:0/964454505 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564c66034320 0x564c6603e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:59.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.154+0000 7fece1d9b640 1 -- 172.21.15.5:0/964454505 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564c66033a00 msgr2=0x564c66033de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:59.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.154+0000 7fece1d9b640 1 --2- 172.21.15.5:0/964454505 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564c66033a00 0x564c66033de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:59.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.154+0000 7fece1d9b640 1 -- 172.21.15.5:0/964454505 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c65d09620 con 0x564c6603ebe0 2024-09-12T03:03:59.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.154+0000 7fecd37fe640 1 -- 172.21.15.5:0/570665278 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fecd40f06e0 con 0x7fece41009d0 2024-09-12T03:03:59.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.154+0000 7fece2d9d640 1 --2- 172.21.15.5:0/570665278 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7feccc0a2b80 0x7feccc0a5040 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fecdc00ffd0 tx=0x7fecdc0058e0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:59.156 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.154+0000 7fece1d9b640 1 --2- 172.21.15.5:0/964454505 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c6603ebe0 0x564c66040fd0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fecd800bc40 tx=0x7fecd800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aec05362ee46840f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:59.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.154+0000 7fecd0ff9640 1 -- 172.21.15.5:0/964454505 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecd8015070 con 0x564c6603ebe0 2024-09-12T03:03:59.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.154+0000 7fecd0ff9640 1 -- 172.21.15.5:0/964454505 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fecd801a940 con 0x564c6603ebe0 2024-09-12T03:03:59.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.154+0000 7fecd0ff9640 1 -- 172.21.15.5:0/964454505 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecd80119f0 con 0x564c6603ebe0 2024-09-12T03:03:59.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.155+0000 7feceb02b740 1 -- 172.21.15.5:0/964454505 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c6603ebe0 msgr2=0x564c66040fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:59.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.155+0000 7feceb02b740 1 --2- 172.21.15.5:0/964454505 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c6603ebe0 0x564c66040fd0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fecd800bc40 tx=0x7fecd800f730 comp rx=0 tx=0).stop 2024-09-12T03:03:59.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.155+0000 7feceb02b740 1 -- 172.21.15.5:0/964454505 shutdown_connections 2024-09-12T03:03:59.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.155+0000 7feceb02b740 1 --2- 172.21.15.5:0/964454505 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c6603ebe0 0x564c66040fd0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:59.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.155+0000 7feceb02b740 1 --2- 172.21.15.5:0/964454505 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564c66034320 0x564c6603e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:59.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.155+0000 7feceb02b740 1 --2- 172.21.15.5:0/964454505 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564c66033a00 0x564c66033de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:59.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.155+0000 7feceb02b740 1 -- 172.21.15.5:0/964454505 >> 172.21.15.5:0/964454505 conn(0x564c6602ff90 msgr2=0x564c66030370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:59.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.155+0000 7feceb02b740 1 -- 172.21.15.5:0/964454505 shutdown_connections 2024-09-12T03:03:59.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.155+0000 7feceb02b740 1 -- 172.21.15.5:0/964454505 wait complete. 2024-09-12T03:03:59.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.156+0000 7feceb02b740 1 -- start start 2024-09-12T03:03:59.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.156+0000 7feceb02b740 1 Processor -- start 2024-09-12T03:03:59.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.156+0000 7feceb02b740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564c66033a00 0x564c6603e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:59.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.156+0000 7feceb02b740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564c66034320 0x564c6603b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:59.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.156+0000 7feceb02b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c6603ebe0 0x564c6603bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:59.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.156+0000 7feceb02b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564c660417b0 con 0x564c6603ebe0 2024-09-12T03:03:59.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.156+0000 7feceb02b740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x564c65f32c50 con 0x564c66034320 2024-09-12T03:03:59.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.156+0000 7feceb02b740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x564c66041930 con 0x564c66033a00 2024-09-12T03:03:59.161 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.156+0000 7feceb02b740 10 client.0 _ll_register_callbacks cb 0x564c65f44ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:03:59.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.156+0000 7fece2d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c6603ebe0 0x564c6603bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:59.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.156+0000 7fece2d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c6603ebe0 0x564c6603bc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56068/0 (socket says 172.21.15.5:56068) 2024-09-12T03:03:59.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.156+0000 7fece2d9d640 1 -- 172.21.15.5:0/1034583848 learned_addr learned my addr 172.21.15.5:0/1034583848 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:03:59.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.156+0000 7fece259c640 1 --2- 172.21.15.5:0/1034583848 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564c66033a00 0x564c6603e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:59.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.157+0000 7fece1d9b640 1 --2- 172.21.15.5:0/1034583848 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564c66034320 0x564c6603b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:59.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.157+0000 7fece2d9d640 1 -- 172.21.15.5:0/1034583848 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564c66033a00 msgr2=0x564c6603e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:59.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.157+0000 7fece2d9d640 1 --2- 172.21.15.5:0/1034583848 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564c66033a00 0x564c6603e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:59.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.157+0000 7fece2d9d640 1 -- 172.21.15.5:0/1034583848 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564c66034320 msgr2=0x564c6603b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:59.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.157+0000 7fece2d9d640 1 --2- 172.21.15.5:0/1034583848 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564c66034320 0x564c6603b700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:59.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.157+0000 7fece2d9d640 1 -- 172.21.15.5:0/1034583848 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c6603c470 con 0x564c6603ebe0 2024-09-12T03:03:59.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.157+0000 7fece2d9d640 1 --2- 172.21.15.5:0/1034583848 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c6603ebe0 0x564c6603bc40 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fecdc0078b0 tx=0x7fecdc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49bafc67ff37804a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:59.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.157+0000 7fecc9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T03:03:59.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.157+0000 7fecc9ffb640 1 -- 172.21.15.5:0/1034583848 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecdc024070 con 0x564c6603ebe0 2024-09-12T03:03:59.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.157+0000 7fecc9ffb640 1 -- 172.21.15.5:0/1034583848 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fecdc00dd50 con 0x564c6603ebe0 2024-09-12T03:03:59.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.157+0000 7feceb02b740 1 -- 172.21.15.5:0/1034583848 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fecd801a940 con 0x564c6603ebe0 2024-09-12T03:03:59.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.157+0000 7fecc9ffb640 1 -- 172.21.15.5:0/1034583848 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fecdc030050 con 0x564c6603ebe0 2024-09-12T03:03:59.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.157+0000 7feceb02b740 10 client.16722 fetch_fsmap learned FSMap version 33 2024-09-12T03:03:59.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.157+0000 7feceb02b740 1 -- 172.21.15.5:0/1034583848 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x564c6604d260 con 0x564c6603ebe0 2024-09-12T03:03:59.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.157+0000 7fecc9ffb640 1 -- 172.21.15.5:0/1034583848 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fecd801a940 con 0x564c6603ebe0 2024-09-12T03:03:59.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.158+0000 7fecc9ffb640 1 -- 172.21.15.5:0/1034583848 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 33) ==== 8374+0+0 (secure 0 0 0) 0x7fecdc01d440 con 0x564c6603ebe0 2024-09-12T03:03:59.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.158+0000 7feceb02b740 10 client.16722 fetch_fsmap finished waiting for FSMap version 33 2024-09-12T03:03:59.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.158+0000 7feceb02b740 10 client.16722 resolve_mds: validated gid 24502 aka daemon mds.i 2024-09-12T03:03:59.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.158+0000 7feceb02b740 20 client.16722 populate_metadata read hostname 'smithi005' 2024-09-12T03:03:59.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.158+0000 7feceb02b740 1 --2- 172.21.15.5:0/1034583848 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x564c6604e6c0 0x564c66050b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:59.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.158+0000 7feceb02b740 4 client.16722 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:03:59.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.158+0000 7feceb02b740 1 -- 172.21.15.5:0/1034583848 --> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564c65fa5770 con 0x564c6604e6c0 2024-09-12T03:03:59.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.158+0000 7fece259c640 1 --2- 172.21.15.5:0/1034583848 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x564c6604e6c0 0x564c66050b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:59.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.158+0000 7fece259c640 1 --2- 172.21.15.5:0/1034583848 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x564c6604e6c0 0x564c66050b80 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2151d19cffea93dc server_cookie=5dff4b604f5f9269 in_seq=0 out_seq=0 2024-09-12T03:03:59.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.158+0000 7fecc9ffb640 10 client.16722 ms_handle_connect on v2:172.21.15.148:6838/2317600000 2024-09-12T03:03:59.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.159+0000 7fecc9ffb640 1 -- 172.21.15.5:0/1034583848 <== mds.0 v2:172.21.15.148:6838/2317600000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564c65fa5770 con 0x564c6604e6c0 2024-09-12T03:03:59.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.159+0000 7fecc9ffb640 10 client.16722 handle_command_reply: tid=0 2024-09-12T03:03:59.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.159+0000 7feceb02b740 1 client.16722 shutdown 2024-09-12T03:03:59.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.159+0000 7feceb02b740 1 -- 172.21.15.5:0/1034583848 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c6603ebe0 msgr2=0x564c6603bc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:59.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.159+0000 7feceb02b740 1 --2- 172.21.15.5:0/1034583848 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c6603ebe0 0x564c6603bc40 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fecdc0078b0 tx=0x7fecdc0046c0 comp rx=0 tx=0).stop 2024-09-12T03:03:59.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.159+0000 7feceb02b740 1 -- 172.21.15.5:0/1034583848 shutdown_connections 2024-09-12T03:03:59.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.159+0000 7feceb02b740 1 --2- 172.21.15.5:0/1034583848 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x564c6604e6c0 0x564c66050b80 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:59.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.159+0000 7feceb02b740 1 --2- 172.21.15.5:0/1034583848 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c6603ebe0 0x564c6603bc40 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:59.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.159+0000 7feceb02b740 1 --2- 172.21.15.5:0/1034583848 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564c66034320 0x564c6603b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:59.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.159+0000 7feceb02b740 1 --2- 172.21.15.5:0/1034583848 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564c66033a00 0x564c6603e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:59.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.159+0000 7fecc9ffb640 0 client.16722 ms_handle_reset on v2:172.21.15.148:6838/2317600000 2024-09-12T03:03:59.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.159+0000 7feceb02b740 1 -- 172.21.15.5:0/1034583848 >> 172.21.15.5:0/1034583848 conn(0x564c6602ff90 msgr2=0x564c6602f1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:59.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.159+0000 7feceb02b740 1 -- 172.21.15.5:0/1034583848 shutdown_connections 2024-09-12T03:03:59.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.159+0000 7feceb02b740 1 -- 172.21.15.5:0/1034583848 wait complete. 2024-09-12T03:03:59.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.160+0000 7feceb02b740 20 client.16722 trim_cache size 0 max 16384 2024-09-12T03:03:59.165 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:03:58 smithi148 ceph-mon[35090]: pgmap v507: 97 pgs: 97 active+clean; 135 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.8 MiB/s wr, 527 op/s 2024-09-12T03:03:59.165 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:03:58 smithi148 ceph-mon[35090]: scrub summary: idle 2024-09-12T03:03:59.165 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:03:58 smithi148 ceph-mon[35090]: scrub summary: idle 2024-09-12T03:03:59.165 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:03:58 smithi148 ceph-mon[35090]: scrub summary: idle 2024-09-12T03:03:59.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:03:58 smithi148 ceph-mon[35090]: scrub summary: idle 2024-09-12T03:03:59.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:03:58 smithi148 ceph-mon[35090]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:03:59.180 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.176+0000 7feceb02b740 1 Processor -- start 2024-09-12T03:03:59.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.176+0000 7feceb02b740 1 -- start start 2024-09-12T03:03:59.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.177+0000 7feceb02b740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564c66033a00 0x564c6604aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:59.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.177+0000 7feceb02b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c66034320 0x564c66036d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:59.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.177+0000 7feceb02b740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564c6603ebe0 0x564c66056450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:59.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.177+0000 7feceb02b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fece41af710 con 0x564c66034320 2024-09-12T03:03:59.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.177+0000 7feceb02b740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fece41b12e0 con 0x564c66033a00 2024-09-12T03:03:59.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.177+0000 7feceb02b740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fece41afe10 con 0x564c6603ebe0 2024-09-12T03:03:59.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.177+0000 7fece1d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c66034320 0x564c66036d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:59.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.177+0000 7fece1d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c66034320 0x564c66036d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56080/0 (socket says 172.21.15.5:56080) 2024-09-12T03:03:59.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.177+0000 7fece1d9b640 1 -- 172.21.15.5:0/2635698542 learned_addr learned my addr 172.21.15.5:0/2635698542 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:03:59.181 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.177+0000 7fece259c640 1 --2- 172.21.15.5:0/2635698542 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564c66033a00 0x564c6604aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:59.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.177+0000 7fece2d9d640 1 --2- 172.21.15.5:0/2635698542 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564c6603ebe0 0x564c66056450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:59.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.177+0000 7fece1d9b640 1 -- 172.21.15.5:0/2635698542 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564c6603ebe0 msgr2=0x564c66056450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:59.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.177+0000 7fece1d9b640 1 --2- 172.21.15.5:0/2635698542 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564c6603ebe0 0x564c66056450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:59.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.177+0000 7fece1d9b640 1 -- 172.21.15.5:0/2635698542 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564c66033a00 msgr2=0x564c6604aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:59.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.177+0000 7fece1d9b640 1 --2- 172.21.15.5:0/2635698542 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564c66033a00 0x564c6604aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:59.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.177+0000 7fece1d9b640 1 -- 172.21.15.5:0/2635698542 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c6602fdc0 con 0x564c66034320 2024-09-12T03:03:59.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.177+0000 7fece2d9d640 1 --2- 172.21.15.5:0/2635698542 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564c6603ebe0 0x564c66056450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:03:59.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.177+0000 7fece1d9b640 1 --2- 172.21.15.5:0/2635698542 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c66034320 0x564c66036d50 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fecd8000c00 tx=0x7fecd800b3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebf44c6437b90512 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:59.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.177+0000 7feccbfff640 1 -- 172.21.15.5:0/2635698542 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecd8015040 con 0x564c66034320 2024-09-12T03:03:59.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.178+0000 7feccbfff640 1 -- 172.21.15.5:0/2635698542 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fecd800bd30 con 0x564c66034320 2024-09-12T03:03:59.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.178+0000 7feccbfff640 1 -- 172.21.15.5:0/2635698542 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecd80048e0 con 0x564c66034320 2024-09-12T03:03:59.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.178+0000 7feceb02b740 1 -- 172.21.15.5:0/2635698542 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c66034320 msgr2=0x564c66036d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:59.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.178+0000 7feceb02b740 1 --2- 172.21.15.5:0/2635698542 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c66034320 0x564c66036d50 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fecd8000c00 tx=0x7fecd800b3a0 comp rx=0 tx=0).stop 2024-09-12T03:03:59.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.178+0000 7feceb02b740 1 -- 172.21.15.5:0/2635698542 shutdown_connections 2024-09-12T03:03:59.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.178+0000 7feceb02b740 1 --2- 172.21.15.5:0/2635698542 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c66034320 0x564c66036d50 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:59.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.178+0000 7feceb02b740 1 --2- 172.21.15.5:0/2635698542 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564c6603ebe0 0x564c66056450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:59.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.178+0000 7feceb02b740 1 --2- 172.21.15.5:0/2635698542 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564c66033a00 0x564c6604aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:59.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.178+0000 7feceb02b740 1 -- 172.21.15.5:0/2635698542 >> 172.21.15.5:0/2635698542 conn(0x564c6602ff90 msgr2=0x564c66036640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:59.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.178+0000 7feceb02b740 1 -- 172.21.15.5:0/2635698542 shutdown_connections 2024-09-12T03:03:59.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.178+0000 7feceb02b740 1 -- 172.21.15.5:0/2635698542 wait complete. 2024-09-12T03:03:59.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.178+0000 7feceb02b740 1 -- start start 2024-09-12T03:03:59.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.179+0000 7feceb02b740 1 Processor -- start 2024-09-12T03:03:59.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.179+0000 7feceb02b740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564c66033a00 0x564c660427f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:59.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.179+0000 7feceb02b740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564c66034320 0x564c66042d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:59.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.179+0000 7feceb02b740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c6603ebe0 0x564c66043270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:59.183 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.179+0000 7feceb02b740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fece41b0c10 con 0x564c6603ebe0 2024-09-12T03:03:59.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.179+0000 7feceb02b740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fece41b0510 con 0x564c66033a00 2024-09-12T03:03:59.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.179+0000 7feceb02b740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fece41ad6f0 con 0x564c66034320 2024-09-12T03:03:59.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.179+0000 7feceb02b740 10 client.0 _ll_register_callbacks cb 0x7fecdc00f820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:03:59.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.179+0000 7fece2d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c6603ebe0 0x564c66043270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:59.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.179+0000 7fece2d9d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c6603ebe0 0x564c66043270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56094/0 (socket says 172.21.15.5:56094) 2024-09-12T03:03:59.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.179+0000 7fece2d9d640 1 -- 172.21.15.5:0/3180071300 learned_addr learned my addr 172.21.15.5:0/3180071300 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:03:59.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.179+0000 7fece259c640 1 --2- 172.21.15.5:0/3180071300 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564c66033a00 0x564c660427f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:59.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.179+0000 7fece1d9b640 1 --2- 172.21.15.5:0/3180071300 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564c66034320 0x564c66042d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:59.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.179+0000 7fece2d9d640 1 -- 172.21.15.5:0/3180071300 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564c66034320 msgr2=0x564c66042d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:59.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.179+0000 7fece2d9d640 1 --2- 172.21.15.5:0/3180071300 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564c66034320 0x564c66042d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:59.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.179+0000 7fece2d9d640 1 -- 172.21.15.5:0/3180071300 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564c66033a00 msgr2=0x564c660427f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:03:59.184 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.179+0000 7fece2d9d640 1 --2- 172.21.15.5:0/3180071300 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564c66033a00 0x564c660427f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:59.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.179+0000 7fece2d9d640 1 -- 172.21.15.5:0/3180071300 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c660b8420 con 0x564c6603ebe0 2024-09-12T03:03:59.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.179+0000 7fece2d9d640 1 --2- 172.21.15.5:0/3180071300 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c6603ebe0 0x564c66043270 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fecdc00fd30 tx=0x7fecdc00d8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9345bb07c0df8aec server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:03:59.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.179+0000 7fecc9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T03:03:59.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.180+0000 7fecc9ffb640 1 -- 172.21.15.5:0/3180071300 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecdc038020 con 0x564c6603ebe0 2024-09-12T03:03:59.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.180+0000 7feceb02b740 1 -- 172.21.15.5:0/3180071300 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fecd800bd30 con 0x564c6603ebe0 2024-09-12T03:03:59.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.180+0000 7fecc9ffb640 1 -- 172.21.15.5:0/3180071300 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fecdc01d5b0 con 0x564c6603ebe0 2024-09-12T03:03:59.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.180+0000 7fecc9ffb640 1 -- 172.21.15.5:0/3180071300 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fecdc024040 con 0x564c6603ebe0 2024-09-12T03:03:59.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.180+0000 7feceb02b740 10 client.16728 fetch_fsmap learned FSMap version 33 2024-09-12T03:03:59.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.180+0000 7feceb02b740 1 -- 172.21.15.5:0/3180071300 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x564c660b9570 con 0x564c6603ebe0 2024-09-12T03:03:59.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.180+0000 7fecc9ffb640 1 -- 172.21.15.5:0/3180071300 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fecd800bd30 con 0x564c6603ebe0 2024-09-12T03:03:59.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.180+0000 7fecc9ffb640 1 -- 172.21.15.5:0/3180071300 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 33) ==== 8374+0+0 (secure 0 0 0) 0x7fecdc004730 con 0x564c6603ebe0 2024-09-12T03:03:59.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.180+0000 7feceb02b740 10 client.16728 fetch_fsmap finished waiting for FSMap version 33 2024-09-12T03:03:59.185 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.180+0000 7feceb02b740 10 client.16728 resolve_mds: validated gid 24502 aka daemon mds.i 2024-09-12T03:03:59.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.180+0000 7feceb02b740 20 client.16728 populate_metadata read hostname 'smithi005' 2024-09-12T03:03:59.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.180+0000 7feceb02b740 1 --2- 172.21.15.5:0/3180071300 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x564c660b9c60 0x564c660bc080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:03:59.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.180+0000 7feceb02b740 4 client.16728 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "quiesce db", "cancel": true, "all": true}] 2024-09-12T03:03:59.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.180+0000 7feceb02b740 1 -- 172.21.15.5:0/3180071300 --> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] -- command(tid 0: {"prefix": "quiesce db", "cancel": true, "all": true}) -- 0x564c65f9fc80 con 0x564c660b9c60 2024-09-12T03:03:59.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.181+0000 7fece259c640 1 --2- 172.21.15.5:0/3180071300 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x564c660b9c60 0x564c660bc080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:03:59.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.181+0000 7fece259c640 1 --2- 172.21.15.5:0/3180071300 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x564c660b9c60 0x564c660bc080 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d6bfe10643b9ac1 server_cookie=4e7ceef29e6480b4 in_seq=0 out_seq=0 2024-09-12T03:03:59.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.181+0000 7fecc9ffb640 10 client.16728 ms_handle_connect on v2:172.21.15.148:6838/2317600000 2024-09-12T03:03:59.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.181+0000 7fecc9ffb640 1 -- 172.21.15.5:0/3180071300 <== mds.0 v2:172.21.15.148:6838/2317600000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+80 (crc 0 0 0) 0x564c65f9fc80 con 0x564c660b9c60 2024-09-12T03:03:59.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.181+0000 7fecc9ffb640 10 client.16728 handle_command_reply: tid=0 2024-09-12T03:03:59.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.181+0000 7feceb02b740 1 client.16728 shutdown 2024-09-12T03:03:59.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.182+0000 7feceb02b740 1 -- 172.21.15.5:0/3180071300 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c6603ebe0 msgr2=0x564c66043270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:59.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.182+0000 7feceb02b740 1 --2- 172.21.15.5:0/3180071300 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c6603ebe0 0x564c66043270 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fecdc00fd30 tx=0x7fecdc00d8d0 comp rx=0 tx=0).stop 2024-09-12T03:03:59.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.182+0000 7feceb02b740 1 -- 172.21.15.5:0/3180071300 shutdown_connections 2024-09-12T03:03:59.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.182+0000 7feceb02b740 1 --2- 172.21.15.5:0/3180071300 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x564c660b9c60 0x564c660bc080 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:59.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.182+0000 7feceb02b740 1 --2- 172.21.15.5:0/3180071300 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c6603ebe0 0x564c66043270 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:59.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.182+0000 7feceb02b740 1 --2- 172.21.15.5:0/3180071300 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564c66034320 0x564c66042d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:59.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.182+0000 7fecc9ffb640 0 client.16728 ms_handle_reset on v2:172.21.15.148:6838/2317600000 2024-09-12T03:03:59.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.182+0000 7feceb02b740 1 --2- 172.21.15.5:0/3180071300 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564c66033a00 0x564c660427f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:59.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.182+0000 7feceb02b740 1 -- 172.21.15.5:0/3180071300 >> 172.21.15.5:0/3180071300 conn(0x564c6602ff90 msgr2=0x564c6603cf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:59.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.182+0000 7feceb02b740 1 -- 172.21.15.5:0/3180071300 shutdown_connections 2024-09-12T03:03:59.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.182+0000 7feceb02b740 1 -- 172.21.15.5:0/3180071300 wait complete. 2024-09-12T03:03:59.188 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.182+0000 7feceb02b740 20 client.16728 trim_cache size 0 max 16384 2024-09-12T03:03:59.189 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-12T03:03:59.189 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 33, 2024-09-12T03:03:59.189 INFO:teuthology.orchestra.run.smithi005.stdout: "leader": 24502, 2024-09-12T03:03:59.189 INFO:teuthology.orchestra.run.smithi005.stdout: "set_version": 20, 2024-09-12T03:03:59.189 INFO:teuthology.orchestra.run.smithi005.stdout: "sets": {} 2024-09-12T03:03:59.189 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-12T03:03:59.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.182+0000 7fece8c52640 1 -- 172.21.15.5:0/570665278 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7feccc0a2b80 msgr2=0x7feccc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:59.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.182+0000 7fece8c52640 1 --2- 172.21.15.5:0/570665278 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7feccc0a2b80 0x7feccc0a5040 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fecdc00ffd0 tx=0x7fecdc0058e0 comp rx=0 tx=0).stop 2024-09-12T03:03:59.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.182+0000 7fece8c52640 1 -- 172.21.15.5:0/570665278 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fece41009d0 msgr2=0x7fece41a0400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:03:59.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.182+0000 7fece8c52640 1 --2- 172.21.15.5:0/570665278 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fece41009d0 0x7fece41a0400 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fecd4005e30 tx=0x7fecd4003f90 comp rx=0 tx=0).stop 2024-09-12T03:03:59.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.183+0000 7fece8c52640 1 -- 172.21.15.5:0/570665278 shutdown_connections 2024-09-12T03:03:59.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.183+0000 7fece8c52640 1 --2- 172.21.15.5:0/570665278 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fece410e4f0 0x7fece419a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:59.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.183+0000 7fece8c52640 1 --2- 172.21.15.5:0/570665278 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fece4101380 0x7fece41a0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:59.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.183+0000 7fece8c52640 1 --2- 172.21.15.5:0/570665278 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7feccc0a2b80 0x7feccc0a5040 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:59.190 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.183+0000 7fece8c52640 1 --2- 172.21.15.5:0/570665278 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fece41009d0 0x7fece41a0400 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:03:59.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.183+0000 7fece8c52640 1 -- 172.21.15.5:0/570665278 >> 172.21.15.5:0/570665278 conn(0x7fece40fc7d0 msgr2=0x7fece40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:03:59.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.183+0000 7fece8c52640 1 -- 172.21.15.5:0/570665278 shutdown_connections 2024-09-12T03:03:59.191 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:03:59.183+0000 7fece8c52640 1 -- 172.21.15.5:0/570665278 wait complete. 2024-09-12T03:03:59.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:03:58 smithi146 ceph-mon[35011]: pgmap v507: 97 pgs: 97 active+clean; 135 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 1.8 MiB/s wr, 527 op/s 2024-09-12T03:03:59.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:03:58 smithi146 ceph-mon[35011]: scrub summary: idle 2024-09-12T03:03:59.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:03:58 smithi146 ceph-mon[35011]: scrub summary: idle 2024-09-12T03:03:59.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:03:58 smithi146 ceph-mon[35011]: scrub summary: idle 2024-09-12T03:03:59.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:03:58 smithi146 ceph-mon[35011]: scrub summary: idle 2024-09-12T03:03:59.207 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:03:58 smithi146 ceph-mon[35011]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-09-12T03:04:00.008 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 20.1 2024-09-12T03:04:00.009 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:04:00.281 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:04:00.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:00.723+0000 7f087baef640 1 -- 172.21.15.5:0/3465730120 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0874102ef0 msgr2=0x7f087410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:00.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:00.723+0000 7f087baef640 1 --2- 172.21.15.5:0/3465730120 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0874102ef0 0x7f087410da40 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f086c009900 tx=0x7f086c02a480 comp rx=0 tx=0).stop 2024-09-12T03:04:00.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:00.723+0000 7f087baef640 1 -- 172.21.15.5:0/3465730120 shutdown_connections 2024-09-12T03:04:00.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:00.723+0000 7f087baef640 1 --2- 172.21.15.5:0/3465730120 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0874102ef0 0x7f087410da40 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:00.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:00.723+0000 7f087baef640 1 --2- 172.21.15.5:0/3465730120 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0874102570 0x7f08741029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:00.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:00.723+0000 7f087baef640 1 --2- 172.21.15.5:0/3465730120 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0874100e20 0x7f0874101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:00.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:00.723+0000 7f087baef640 1 -- 172.21.15.5:0/3465730120 >> 172.21.15.5:0/3465730120 conn(0x7f08740fc7d0 msgr2=0x7f08740febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:00.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:00.723+0000 7f087baef640 1 -- 172.21.15.5:0/3465730120 shutdown_connections 2024-09-12T03:04:00.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:00.723+0000 7f087baef640 1 -- 172.21.15.5:0/3465730120 wait complete. 2024-09-12T03:04:00.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:00.723+0000 7f087baef640 1 Processor -- start 2024-09-12T03:04:00.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:00.724+0000 7f087baef640 1 -- start start 2024-09-12T03:04:00.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:00.724+0000 7f087baef640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0874100e20 0x7f0874198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:00.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:00.724+0000 7f087baef640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0874102570 0x7f08741989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:00.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:00.724+0000 7f087baef640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0874102ef0 0x7f087419cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:00.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:00.724+0000 7f087baef640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f0874110f80 con 0x7f0874100e20 2024-09-12T03:04:00.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:00.724+0000 7f087baef640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f0874110e00 con 0x7f0874102ef0 2024-09-12T03:04:00.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:00.724+0000 7f087baef640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f0874111100 con 0x7f0874102570 2024-09-12T03:04:00.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:00.724+0000 7f087988b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0874100e20 0x7f0874198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:00.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:00.724+0000 7f087988b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0874100e20 0x7f0874198490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56114/0 (socket says 172.21.15.5:56114) 2024-09-12T03:04:00.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:00.724+0000 7f087988b640 1 -- 172.21.15.5:0/3643882908 learned_addr learned my addr 172.21.15.5:0/3643882908 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:04:00.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:00.724+0000 7f087a08c640 1 --2- 172.21.15.5:0/3643882908 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0874102ef0 0x7f087419cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:00.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:00.724+0000 7f087908a640 1 --2- 172.21.15.5:0/3643882908 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0874102570 0x7f08741989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:00.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:00.724+0000 7f087988b640 1 -- 172.21.15.5:0/3643882908 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0874102570 msgr2=0x7f08741989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:00.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:00.724+0000 7f087988b640 1 --2- 172.21.15.5:0/3643882908 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0874102570 0x7f08741989d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:00.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:00.724+0000 7f087988b640 1 -- 172.21.15.5:0/3643882908 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0874102ef0 msgr2=0x7f087419cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:00.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:00.724+0000 7f087988b640 1 --2- 172.21.15.5:0/3643882908 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0874102ef0 0x7f087419cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:00.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:00.724+0000 7f087988b640 1 -- 172.21.15.5:0/3643882908 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f087419d440 con 0x7f0874100e20 2024-09-12T03:04:00.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:00.724+0000 7f087a08c640 1 --2- 172.21.15.5:0/3643882908 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0874102ef0 0x7f087419cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:04:00.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:00.724+0000 7f087908a640 1 --2- 172.21.15.5:0/3643882908 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0874102570 0x7f08741989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:04:00.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:00.725+0000 7f087988b640 1 --2- 172.21.15.5:0/3643882908 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0874100e20 0x7f0874198490 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f087000ea80 tx=0x7f087000ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:00.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:00.725+0000 7f086affd640 1 -- 172.21.15.5:0/3643882908 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f087000ce40 con 0x7f0874100e20 2024-09-12T03:04:00.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:00.725+0000 7f087baef640 1 -- 172.21.15.5:0/3643882908 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f087419d730 con 0x7f0874100e20 2024-09-12T03:04:00.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:00.725+0000 7f086affd640 1 -- 172.21.15.5:0/3643882908 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0870005020 con 0x7f0874100e20 2024-09-12T03:04:00.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:00.725+0000 7f086affd640 1 -- 172.21.15.5:0/3643882908 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0870011760 con 0x7f0874100e20 2024-09-12T03:04:00.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:00.725+0000 7f087baef640 1 -- 172.21.15.5:0/3643882908 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08741a5010 con 0x7f0874100e20 2024-09-12T03:04:00.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:00.726+0000 7f087baef640 1 -- 172.21.15.5:0/3643882908 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0874101200 con 0x7f0874100e20 2024-09-12T03:04:00.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:00.727+0000 7f086affd640 1 -- 172.21.15.5:0/3643882908 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f08700051d0 con 0x7f0874100e20 2024-09-12T03:04:00.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:00.728+0000 7f086affd640 1 --2- 172.21.15.5:0/3643882908 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f08600a2b80 0x7f08600a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:00.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:00.728+0000 7f087908a640 1 --2- 172.21.15.5:0/3643882908 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f08600a2b80 0x7f08600a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:00.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:00.728+0000 7f086affd640 1 -- 172.21.15.5:0/3643882908 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f08700cf7d0 con 0x7f0874100e20 2024-09-12T03:04:00.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:00.728+0000 7f087908a640 1 --2- 172.21.15.5:0/3643882908 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f08600a2b80 0x7f08600a5040 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f08740fef00 tx=0x7f0864005d30 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:00.889 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:00.732+0000 7f086affd640 1 -- 172.21.15.5:0/3643882908 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0870016030 con 0x7f0874100e20 2024-09-12T03:04:00.942 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:00.942+0000 7f087baef640 1 -- 172.21.15.5:0/3643882908 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f08741997f0 con 0x7f0874100e20 2024-09-12T03:04:00.943 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:04:00 smithi005 ceph-mon[31197]: pgmap v508: 97 pgs: 97 active+clean; 134 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.7 MiB/s wr, 481 op/s 2024-09-12T03:04:00.943 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:00.943+0000 7f086affd640 1 -- 172.21.15.5:0/3643882908 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11969 (secure 0 0 0) 0x7f087009c6e0 con 0x7f0874100e20 2024-09-12T03:04:00.943 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-12T03:04:00.944 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":33,"btime":"2024-09-12T03:03:57:799392+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/4135232521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":4135232521},{"type":"v1","addr":"172.21.15.5:6835","nonce":4135232521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/4290221940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":4290221940},{"type":"v1","addr":"172.21.15.5:6837","nonce":4290221940}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T02:56:16.595562+0000","modified":"2024-09-12T03:03:57.585077+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24502,"mds_1":24487,"mds_2":24481,"mds_3":24479,"mds_4":24458},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/26447341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":26447341},{"type":"v1","addr":"172.21.15.5:6839","nonce":26447341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2393443295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2393443295},{"type":"v1","addr":"172.21.15.5:6841","nonce":2393443295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6833/3686970137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3686970137},{"type":"v1","addr":"172.21.15.146:6833","nonce":3686970137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6833/568512209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":568512209},{"type":"v1","addr":"172.21.15.148:6833","nonce":568512209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6835/2540740420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2540740420},{"type":"v1","addr":"172.21.15.146:6835","nonce":2540740420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.146:6837/1673370937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":1673370937},{"type":"v1","addr":"172.21.15.146:6837","nonce":1673370937}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":3,"incarnation":22,"state":"up:active","state_seq":6,"addr":"172.21.15.146:6839/3036767228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":3036767228},{"type":"v1","addr":"172.21.15.146:6839","nonce":3036767228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":2,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.148:6835/4195086427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":4195086427},{"type":"v1","addr":"172.21.15.148:6835","nonce":4195086427}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.148:6837/2606873922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2606873922},{"type":"v1","addr":"172.21.15.148:6837","nonce":2606873922}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6839/2317600000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":2317600000},{"type":"v1","addr":"172.21.15.148:6839","nonce":2317600000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24502,"qdb_cluster":[24458,24479,24481,24487,24502]},"id":1}]} 2024-09-12T03:04:00.944 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 33 2024-09-12T03:04:00.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:00.946+0000 7f087baef640 1 -- 172.21.15.5:0/3643882908 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f08600a2b80 msgr2=0x7f08600a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:00.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:00.946+0000 7f087baef640 1 --2- 172.21.15.5:0/3643882908 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f08600a2b80 0x7f08600a5040 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f08740fef00 tx=0x7f0864005d30 comp rx=0 tx=0).stop 2024-09-12T03:04:00.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:00.946+0000 7f087baef640 1 -- 172.21.15.5:0/3643882908 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0874100e20 msgr2=0x7f0874198490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:00.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:00.946+0000 7f087baef640 1 --2- 172.21.15.5:0/3643882908 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0874100e20 0x7f0874198490 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f087000ea80 tx=0x7f087000ef40 comp rx=0 tx=0).stop 2024-09-12T03:04:00.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:00.946+0000 7f087baef640 1 -- 172.21.15.5:0/3643882908 shutdown_connections 2024-09-12T03:04:00.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:00.946+0000 7f087baef640 1 --2- 172.21.15.5:0/3643882908 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f0874102ef0 0x7f087419cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:00.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:00.946+0000 7f087baef640 1 --2- 172.21.15.5:0/3643882908 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f0874102570 0x7f08741989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:00.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:00.946+0000 7f087baef640 1 --2- 172.21.15.5:0/3643882908 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f08600a2b80 0x7f08600a5040 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:00.946 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:00.946+0000 7f087baef640 1 --2- 172.21.15.5:0/3643882908 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0874100e20 0x7f0874198490 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:00.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:00.946+0000 7f087baef640 1 -- 172.21.15.5:0/3643882908 >> 172.21.15.5:0/3643882908 conn(0x7f08740fc7d0 msgr2=0x7f087406a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:00.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:00.946+0000 7f087baef640 1 -- 172.21.15.5:0/3643882908 shutdown_connections 2024-09-12T03:04:00.947 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:00.946+0000 7f087baef640 1 -- 172.21.15.5:0/3643882908 wait complete. 2024-09-12T03:04:01.165 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:04:00 smithi148 ceph-mon[35090]: pgmap v508: 97 pgs: 97 active+clean; 134 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.7 MiB/s wr, 481 op/s 2024-09-12T03:04:01.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:04:00 smithi146 ceph-mon[35011]: pgmap v508: 97 pgs: 97 active+clean; 134 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.7 MiB/s wr, 481 op/s 2024-09-12T03:04:01.629 INFO:tasks.quiescer:done joining Quiescers 2024-09-12T03:04:01.629 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-09-12T03:04:01.648 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-09-12T03:04:02.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:04:01 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/3643882908' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:04:02.165 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:04:01 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/3643882908' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:04:02.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:04:01 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/3643882908' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:04:03.165 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:04:02 smithi148 ceph-mon[35090]: pgmap v509: 97 pgs: 97 active+clean; 132 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 391 op/s 2024-09-12T03:04:03.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:04:02 smithi146 ceph-mon[35011]: pgmap v509: 97 pgs: 97 active+clean; 132 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 391 op/s 2024-09-12T03:04:03.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:04:02 smithi005 ceph-mon[31197]: pgmap v509: 97 pgs: 97 active+clean; 132 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.0 MiB/s wr, 391 op/s 2024-09-12T03:04:05.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:04:04 smithi005 ceph-mon[31197]: pgmap v510: 97 pgs: 97 active+clean; 119 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1022 KiB/s rd, 1.0 MiB/s wr, 359 op/s 2024-09-12T03:04:05.165 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:04:04 smithi148 ceph-mon[35090]: pgmap v510: 97 pgs: 97 active+clean; 119 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1022 KiB/s rd, 1.0 MiB/s wr, 359 op/s 2024-09-12T03:04:05.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:04:04 smithi146 ceph-mon[35011]: pgmap v510: 97 pgs: 97 active+clean; 119 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1022 KiB/s rd, 1.0 MiB/s wr, 359 op/s 2024-09-12T03:04:07.165 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:04:06 smithi148 ceph-mon[35090]: pgmap v511: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 688 KiB/s rd, 712 KiB/s wr, 283 op/s 2024-09-12T03:04:07.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:04:06 smithi146 ceph-mon[35011]: pgmap v511: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 688 KiB/s rd, 712 KiB/s wr, 283 op/s 2024-09-12T03:04:07.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:04:06 smithi005 ceph-mon[31197]: pgmap v511: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 688 KiB/s rd, 712 KiB/s wr, 283 op/s 2024-09-12T03:04:09.165 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:04:08 smithi148 ceph-mon[35090]: pgmap v512: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 244 KiB/s wr, 205 op/s 2024-09-12T03:04:09.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:04:08 smithi146 ceph-mon[35011]: pgmap v512: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 244 KiB/s wr, 205 op/s 2024-09-12T03:04:09.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:04:08 smithi005 ceph-mon[31197]: pgmap v512: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 244 KiB/s wr, 205 op/s 2024-09-12T03:04:11.165 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:04:10 smithi148 ceph-mon[35090]: pgmap v513: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 223 KiB/s wr, 92 op/s 2024-09-12T03:04:11.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:04:10 smithi146 ceph-mon[35011]: pgmap v513: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 223 KiB/s wr, 92 op/s 2024-09-12T03:04:11.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:04:10 smithi005 ceph-mon[31197]: pgmap v513: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 223 KiB/s wr, 92 op/s 2024-09-12T03:04:13.165 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:04:12 smithi148 ceph-mon[35090]: pgmap v514: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 31 op/s 2024-09-12T03:04:13.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:04:12 smithi146 ceph-mon[35011]: pgmap v514: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 31 op/s 2024-09-12T03:04:13.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:04:12 smithi005 ceph-mon[31197]: pgmap v514: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 31 op/s 2024-09-12T03:04:15.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:04:14 smithi148 ceph-mon[35090]: pgmap v515: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.2 KiB/s wr, 24 op/s 2024-09-12T03:04:15.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:04:14 smithi146 ceph-mon[35011]: pgmap v515: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.2 KiB/s wr, 24 op/s 2024-09-12T03:04:15.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:04:14 smithi005 ceph-mon[31197]: pgmap v515: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.2 KiB/s wr, 24 op/s 2024-09-12T03:04:17.165 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:04:16 smithi148 ceph-mon[35090]: pgmap v516: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.2 KiB/s wr, 18 op/s 2024-09-12T03:04:17.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:04:16 smithi146 ceph-mon[35011]: pgmap v516: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.2 KiB/s wr, 18 op/s 2024-09-12T03:04:17.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:04:16 smithi005 ceph-mon[31197]: pgmap v516: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.2 KiB/s wr, 18 op/s 2024-09-12T03:04:18.863 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-12T03:04:19.136 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:04:19.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:04:18 smithi148 ceph-mon[35090]: pgmap v517: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 938 B/s wr, 12 op/s 2024-09-12T03:04:19.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:04:18 smithi146 ceph-mon[35011]: pgmap v517: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 938 B/s wr, 12 op/s 2024-09-12T03:04:19.259 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:04:18 smithi005 ceph-mon[31197]: pgmap v517: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 938 B/s wr, 12 op/s 2024-09-12T03:04:19.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.605+0000 7f771a2d2640 1 -- 172.21.15.5:0/3334938437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7714102ef0 msgr2=0x7f771410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:19.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.605+0000 7f771a2d2640 1 --2- 172.21.15.5:0/3334938437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7714102ef0 0x7f771410da40 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f770c009ca0 tx=0x7f770c02a950 comp rx=0 tx=0).stop 2024-09-12T03:04:19.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.605+0000 7f771a2d2640 1 -- 172.21.15.5:0/3334938437 shutdown_connections 2024-09-12T03:04:19.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.605+0000 7f771a2d2640 1 --2- 172.21.15.5:0/3334938437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7714102ef0 0x7f771410da40 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:19.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.605+0000 7f771a2d2640 1 --2- 172.21.15.5:0/3334938437 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7714102570 0x7f77141029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:19.620 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.605+0000 7f771a2d2640 1 --2- 172.21.15.5:0/3334938437 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7714100e20 0x7f7714101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:19.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.605+0000 7f771a2d2640 1 -- 172.21.15.5:0/3334938437 >> 172.21.15.5:0/3334938437 conn(0x7f77140fc7d0 msgr2=0x7f77140febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:19.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.605+0000 7f771a2d2640 1 -- 172.21.15.5:0/3334938437 shutdown_connections 2024-09-12T03:04:19.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.605+0000 7f771a2d2640 1 -- 172.21.15.5:0/3334938437 wait complete. 2024-09-12T03:04:19.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.606+0000 7f771a2d2640 1 Processor -- start 2024-09-12T03:04:19.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.606+0000 7f771a2d2640 1 -- start start 2024-09-12T03:04:19.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.606+0000 7f771a2d2640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7714100e20 0x7f7714198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:19.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.606+0000 7f771a2d2640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7714102570 0x7f7714198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:19.621 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.606+0000 7f771a2d2640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7714102ef0 0x7f771419cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:19.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.606+0000 7f771a2d2640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7714110f80 con 0x7f7714100e20 2024-09-12T03:04:19.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.606+0000 7f771a2d2640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f7714110e00 con 0x7f7714102ef0 2024-09-12T03:04:19.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.606+0000 7f771a2d2640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f7714111100 con 0x7f7714102570 2024-09-12T03:04:19.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.606+0000 7f7713fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7714100e20 0x7f7714198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:19.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.607+0000 7f7713fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7714100e20 0x7f7714198450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43868/0 (socket says 172.21.15.5:43868) 2024-09-12T03:04:19.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.607+0000 7f7713fff640 1 -- 172.21.15.5:0/3556992256 learned_addr learned my addr 172.21.15.5:0/3556992256 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:04:19.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.607+0000 7f770bfff640 1 --2- 172.21.15.5:0/3556992256 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7714102570 0x7f7714198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:19.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.607+0000 7f771886f640 1 --2- 172.21.15.5:0/3556992256 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7714102ef0 0x7f771419cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:19.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.607+0000 7f7713fff640 1 -- 172.21.15.5:0/3556992256 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7714102570 msgr2=0x7f7714198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:19.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.607+0000 7f7713fff640 1 --2- 172.21.15.5:0/3556992256 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7714102570 0x7f7714198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:19.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.607+0000 7f7713fff640 1 -- 172.21.15.5:0/3556992256 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7714102ef0 msgr2=0x7f771419cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:19.622 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.607+0000 7f7713fff640 1 --2- 172.21.15.5:0/3556992256 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7714102ef0 0x7f771419cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:19.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.607+0000 7f7713fff640 1 -- 172.21.15.5:0/3556992256 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f771419d400 con 0x7f7714100e20 2024-09-12T03:04:19.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.607+0000 7f771886f640 1 --2- 172.21.15.5:0/3556992256 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7714102ef0 0x7f771419cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:04:19.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.607+0000 7f7713fff640 1 --2- 172.21.15.5:0/3556992256 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7714100e20 0x7f7714198450 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f770400d950 tx=0x7f770400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:19.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.607+0000 7f7711ffb640 1 -- 172.21.15.5:0/3556992256 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7704017070 con 0x7f7714100e20 2024-09-12T03:04:19.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.607+0000 7f7711ffb640 1 -- 172.21.15.5:0/3556992256 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7704009c30 con 0x7f7714100e20 2024-09-12T03:04:19.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.607+0000 7f771a2d2640 1 -- 172.21.15.5:0/3556992256 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f771419d6f0 con 0x7f7714100e20 2024-09-12T03:04:19.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.607+0000 7f7711ffb640 1 -- 172.21.15.5:0/3556992256 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f770400f6b0 con 0x7f7714100e20 2024-09-12T03:04:19.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.607+0000 7f771a2d2640 1 -- 172.21.15.5:0/3556992256 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77141a4fd0 con 0x7f7714100e20 2024-09-12T03:04:19.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.610+0000 7f7711ffb640 1 -- 172.21.15.5:0/3556992256 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f77040109d0 con 0x7f7714100e20 2024-09-12T03:04:19.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.610+0000 7f771c6ab740 1 Processor -- start 2024-09-12T03:04:19.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.610+0000 7f771c6ab740 1 -- start start 2024-09-12T03:04:19.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.610+0000 7f7711ffb640 1 --2- 172.21.15.5:0/3556992256 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f77000a2b80 0x7f77000a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:19.623 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.610+0000 7f771c6ab740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564cbf2d58b0 0x564cbf2d5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:19.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.610+0000 7f771886f640 1 --2- 172.21.15.5:0/3556992256 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f77000a2b80 0x7f77000a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:19.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.610+0000 7f771c6ab740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564cbf2d61d0 0x564cbf2e0550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:19.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.610+0000 7f7711ffb640 1 -- 172.21.15.5:0/3556992256 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f77040cec80 con 0x7f7714100e20 2024-09-12T03:04:19.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.610+0000 7f771c6ab740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564cbf2e0a90 0x564cbf2e2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:19.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.610+0000 7f771c6ab740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564cbf2372e0 con 0x564cbf2d58b0 2024-09-12T03:04:19.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.610+0000 7f771c6ab740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x564cbf1dece0 con 0x564cbf2d61d0 2024-09-12T03:04:19.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.610+0000 7f771c6ab740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x564cbf2332a0 con 0x564cbf2e0a90 2024-09-12T03:04:19.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.611+0000 7f7713fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564cbf2d58b0 0x564cbf2d5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:19.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.611+0000 7f7713fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564cbf2d58b0 0x564cbf2d5c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43880/0 (socket says 172.21.15.5:43880) 2024-09-12T03:04:19.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.611+0000 7f7713fff640 1 -- 172.21.15.5:0/3979479293 learned_addr learned my addr 172.21.15.5:0/3979479293 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:04:19.624 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.611+0000 7f770bfff640 1 --2- 172.21.15.5:0/3979479293 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564cbf2d61d0 0x564cbf2e0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:19.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.611+0000 7f7713fff640 1 -- 172.21.15.5:0/3979479293 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564cbf2e0a90 msgr2=0x564cbf2e2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:19.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.611+0000 7f7713fff640 1 --2- 172.21.15.5:0/3979479293 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564cbf2e0a90 0x564cbf2e2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:19.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.611+0000 7f7713fff640 1 -- 172.21.15.5:0/3979479293 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564cbf2d61d0 msgr2=0x564cbf2e0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:19.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.611+0000 7f7713fff640 1 --2- 172.21.15.5:0/3979479293 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564cbf2d61d0 0x564cbf2e0550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:19.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.611+0000 7f7713fff640 1 -- 172.21.15.5:0/3979479293 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564cbf1c3440 con 0x564cbf2d58b0 2024-09-12T03:04:19.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.611+0000 7f770bfff640 1 --2- 172.21.15.5:0/3979479293 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564cbf2d61d0 0x564cbf2e0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:04:19.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.611+0000 7f771886f640 1 --2- 172.21.15.5:0/3556992256 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f77000a2b80 0x7f77000a5040 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f770c005cc0 tx=0x7f770c009820 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:19.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.611+0000 7f7713fff640 1 --2- 172.21.15.5:0/3979479293 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564cbf2d58b0 0x564cbf2d5c90 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f770409b1f0 tx=0x7f7704005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4779bbe6f458d5b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:19.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.611+0000 7f770a7fc640 1 -- 172.21.15.5:0/3979479293 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7704020040 con 0x564cbf2d58b0 2024-09-12T03:04:19.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.611+0000 7f770a7fc640 1 -- 172.21.15.5:0/3979479293 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f770409b4f0 con 0x564cbf2d58b0 2024-09-12T03:04:19.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.611+0000 7f770a7fc640 1 -- 172.21.15.5:0/3979479293 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f770409ab40 con 0x564cbf2d58b0 2024-09-12T03:04:19.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.612+0000 7f771c6ab740 1 -- 172.21.15.5:0/3979479293 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564cbf2d58b0 msgr2=0x564cbf2d5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:19.625 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.612+0000 7f771c6ab740 1 --2- 172.21.15.5:0/3979479293 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564cbf2d58b0 0x564cbf2d5c90 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f770409b1f0 tx=0x7f7704005f90 comp rx=0 tx=0).stop 2024-09-12T03:04:19.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.612+0000 7f771c6ab740 1 -- 172.21.15.5:0/3979479293 shutdown_connections 2024-09-12T03:04:19.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.612+0000 7f771c6ab740 1 --2- 172.21.15.5:0/3979479293 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564cbf2e0a90 0x564cbf2e2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:19.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.612+0000 7f771c6ab740 1 --2- 172.21.15.5:0/3979479293 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564cbf2d61d0 0x564cbf2e0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:19.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.612+0000 7f771c6ab740 1 --2- 172.21.15.5:0/3979479293 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564cbf2d58b0 0x564cbf2d5c90 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:19.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.612+0000 7f771c6ab740 1 -- 172.21.15.5:0/3979479293 >> 172.21.15.5:0/3979479293 conn(0x564cbf2d1e40 msgr2=0x564cbf2d2220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:19.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.612+0000 7f771c6ab740 1 -- 172.21.15.5:0/3979479293 shutdown_connections 2024-09-12T03:04:19.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.612+0000 7f771c6ab740 1 -- 172.21.15.5:0/3979479293 wait complete. 2024-09-12T03:04:19.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.613+0000 7f771c6ab740 1 -- start start 2024-09-12T03:04:19.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.613+0000 7f771c6ab740 1 Processor -- start 2024-09-12T03:04:19.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.613+0000 7f771c6ab740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564cbf2d58b0 0x564cbf2e03d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:19.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.613+0000 7f771c6ab740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564cbf2d61d0 0x564cbf2dd680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:19.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.613+0000 7f771c6ab740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564cbf2e0a90 0x564cbf2ddbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:19.626 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.613+0000 7f771c6ab740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x564cbf2e3660 con 0x564cbf2d58b0 2024-09-12T03:04:19.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.613+0000 7f771c6ab740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x564cbf1f2050 con 0x564cbf2e0a90 2024-09-12T03:04:19.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.613+0000 7f771c6ab740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x564cbf2e37e0 con 0x564cbf2d61d0 2024-09-12T03:04:19.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.613+0000 7f771c6ab740 10 client.0 _ll_register_callbacks cb 0x564cbf232160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:04:19.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.613+0000 7f7713fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564cbf2d58b0 0x564cbf2e03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:19.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.613+0000 7f7713fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564cbf2d58b0 0x564cbf2e03d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43888/0 (socket says 172.21.15.5:43888) 2024-09-12T03:04:19.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.613+0000 7f7713fff640 1 -- 172.21.15.5:0/2242122169 learned_addr learned my addr 172.21.15.5:0/2242122169 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:04:19.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.613+0000 7f770bfff640 1 --2- 172.21.15.5:0/2242122169 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564cbf2d61d0 0x564cbf2dd680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:19.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.614+0000 7f771886f640 1 --2- 172.21.15.5:0/2242122169 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564cbf2e0a90 0x564cbf2ddbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:19.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.614+0000 7f7713fff640 1 -- 172.21.15.5:0/2242122169 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564cbf2d61d0 msgr2=0x564cbf2dd680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:19.627 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.614+0000 7f7713fff640 1 --2- 172.21.15.5:0/2242122169 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564cbf2d61d0 0x564cbf2dd680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:19.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.614+0000 7f7713fff640 1 -- 172.21.15.5:0/2242122169 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564cbf2e0a90 msgr2=0x564cbf2ddbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:19.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.614+0000 7f7713fff640 1 --2- 172.21.15.5:0/2242122169 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564cbf2e0a90 0x564cbf2ddbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:19.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.614+0000 7f7713fff640 1 -- 172.21.15.5:0/2242122169 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564cbefa83c0 con 0x564cbf2d58b0 2024-09-12T03:04:19.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.614+0000 7f771886f640 1 --2- 172.21.15.5:0/2242122169 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564cbf2e0a90 0x564cbf2ddbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:04:19.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.614+0000 7f7713fff640 1 --2- 172.21.15.5:0/2242122169 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564cbf2d58b0 0x564cbf2e03d0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f7704097ec0 tx=0x7f7704010cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=743bacffa2c87fd3 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:19.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.614+0000 7f76f37fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T03:04:19.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.614+0000 7f76f37fe640 1 -- 172.21.15.5:0/2242122169 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77040b3020 con 0x564cbf2d58b0 2024-09-12T03:04:19.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.614+0000 7f76f37fe640 1 -- 172.21.15.5:0/2242122169 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f770409ab50 con 0x564cbf2d58b0 2024-09-12T03:04:19.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.614+0000 7f771c6ab740 1 -- 172.21.15.5:0/2242122169 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f770409b4f0 con 0x564cbf2d58b0 2024-09-12T03:04:19.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.614+0000 7f76f37fe640 1 -- 172.21.15.5:0/2242122169 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f7704020070 con 0x564cbf2d58b0 2024-09-12T03:04:19.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.614+0000 7f771c6ab740 10 client.16746 fetch_fsmap learned FSMap version 33 2024-09-12T03:04:19.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.614+0000 7f771c6ab740 1 -- 172.21.15.5:0/2242122169 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x564cbf2ef040 con 0x564cbf2d58b0 2024-09-12T03:04:19.628 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.614+0000 7f76f37fe640 1 -- 172.21.15.5:0/2242122169 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f770409b4f0 con 0x564cbf2d58b0 2024-09-12T03:04:19.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.615+0000 7f76f37fe640 1 -- 172.21.15.5:0/2242122169 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 33) ==== 8374+0+0 (secure 0 0 0) 0x7f770409a3b0 con 0x564cbf2d58b0 2024-09-12T03:04:19.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.615+0000 7f771c6ab740 10 client.16746 fetch_fsmap finished waiting for FSMap version 33 2024-09-12T03:04:19.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.615+0000 7f771c6ab740 10 client.16746 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:04:19.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.615+0000 7f771c6ab740 20 client.16746 populate_metadata read hostname 'smithi005' 2024-09-12T03:04:19.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.615+0000 7f771c6ab740 1 --2- 172.21.15.5:0/2242122169 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x564cbf2f0390 0x564cbf2f2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:19.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.615+0000 7f771c6ab740 4 client.16746 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:04:19.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.615+0000 7f771c6ab740 1 -- 172.21.15.5:0/2242122169 --> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564cbf244850 con 0x564cbf2f0390 2024-09-12T03:04:19.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.615+0000 7f770bfff640 1 --2- 172.21.15.5:0/2242122169 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x564cbf2f0390 0x564cbf2f2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:19.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.615+0000 7f770bfff640 1 --2- 172.21.15.5:0/2242122169 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x564cbf2f0390 0x564cbf2f2780 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4bf0e6543980e4c6 server_cookie=3488a7119cfef8be in_seq=0 out_seq=0 2024-09-12T03:04:19.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.615+0000 7f76f37fe640 10 client.16746 ms_handle_connect on v2:172.21.15.148:6838/2317600000 2024-09-12T03:04:19.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.616+0000 7f76f37fe640 1 -- 172.21.15.5:0/2242122169 <== mds.0 v2:172.21.15.148:6838/2317600000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564cbf244850 con 0x564cbf2f0390 2024-09-12T03:04:19.629 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.616+0000 7f76f37fe640 10 client.16746 handle_command_reply: tid=0 2024-09-12T03:04:19.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.616+0000 7f771c6ab740 1 client.16746 shutdown 2024-09-12T03:04:19.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.616+0000 7f771c6ab740 1 -- 172.21.15.5:0/2242122169 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564cbf2d58b0 msgr2=0x564cbf2e03d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:19.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.616+0000 7f771c6ab740 1 --2- 172.21.15.5:0/2242122169 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564cbf2d58b0 0x564cbf2e03d0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f7704097ec0 tx=0x7f7704010cc0 comp rx=0 tx=0).stop 2024-09-12T03:04:19.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.616+0000 7f771c6ab740 1 -- 172.21.15.5:0/2242122169 shutdown_connections 2024-09-12T03:04:19.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.616+0000 7f771c6ab740 1 --2- 172.21.15.5:0/2242122169 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564cbf2e0a90 0x564cbf2ddbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:19.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.616+0000 7f771c6ab740 1 --2- 172.21.15.5:0/2242122169 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564cbf2d61d0 0x564cbf2dd680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:19.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.616+0000 7f771c6ab740 1 --2- 172.21.15.5:0/2242122169 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x564cbf2f0390 0x564cbf2f2780 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:19.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.617+0000 7f771c6ab740 1 --2- 172.21.15.5:0/2242122169 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564cbf2d58b0 0x564cbf2e03d0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:19.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.617+0000 7f76f37fe640 0 client.16746 ms_handle_reset on v2:172.21.15.148:6838/2317600000 2024-09-12T03:04:19.630 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.617+0000 7f771c6ab740 1 -- 172.21.15.5:0/2242122169 >> 172.21.15.5:0/2242122169 conn(0x564cbf2d1e40 msgr2=0x564cbf2ccb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:19.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.617+0000 7f771c6ab740 1 -- 172.21.15.5:0/2242122169 shutdown_connections 2024-09-12T03:04:19.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.617+0000 7f771c6ab740 1 -- 172.21.15.5:0/2242122169 wait complete. 2024-09-12T03:04:19.631 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.617+0000 7f771c6ab740 20 client.16746 trim_cache size 0 max 16384 2024-09-12T03:04:19.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.634+0000 7f771c6ab740 1 Processor -- start 2024-09-12T03:04:19.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.635+0000 7f771c6ab740 1 -- start start 2024-09-12T03:04:19.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.635+0000 7f771c6ab740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564cbf2d58b0 0x564cbf2ec8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:19.636 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.635+0000 7f771c6ab740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564cbf2d61d0 0x564cbf2f7de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:19.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.635+0000 7f771c6ab740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564cbf2e0a90 0x564cbf2f8320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:19.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.635+0000 7f771c6ab740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f77141ab280 con 0x564cbf2d58b0 2024-09-12T03:04:19.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.635+0000 7f771c6ab740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f77141ace50 con 0x564cbf2e0a90 2024-09-12T03:04:19.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.635+0000 7f771c6ab740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f77141ab980 con 0x564cbf2d61d0 2024-09-12T03:04:19.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.635+0000 7f7713fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564cbf2d58b0 0x564cbf2ec8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:19.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.635+0000 7f7713fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564cbf2d58b0 0x564cbf2ec8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43894/0 (socket says 172.21.15.5:43894) 2024-09-12T03:04:19.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.635+0000 7f7713fff640 1 -- 172.21.15.5:0/3427841739 learned_addr learned my addr 172.21.15.5:0/3427841739 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:04:19.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.635+0000 7f770bfff640 1 --2- 172.21.15.5:0/3427841739 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564cbf2d61d0 0x564cbf2f7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:19.637 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.635+0000 7f771886f640 1 --2- 172.21.15.5:0/3427841739 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564cbf2e0a90 0x564cbf2f8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:19.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.635+0000 7f7713fff640 1 -- 172.21.15.5:0/3427841739 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564cbf2d61d0 msgr2=0x564cbf2f7de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:19.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.635+0000 7f7713fff640 1 --2- 172.21.15.5:0/3427841739 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564cbf2d61d0 0x564cbf2f7de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:19.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.635+0000 7f7713fff640 1 -- 172.21.15.5:0/3427841739 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564cbf2e0a90 msgr2=0x564cbf2f8320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:19.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.635+0000 7f7713fff640 1 --2- 172.21.15.5:0/3427841739 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564cbf2e0a90 0x564cbf2f8320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:19.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.635+0000 7f7713fff640 1 -- 172.21.15.5:0/3427841739 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564cbf2d1c70 con 0x564cbf2d58b0 2024-09-12T03:04:19.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.635+0000 7f770bfff640 1 --2- 172.21.15.5:0/3427841739 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564cbf2d61d0 0x564cbf2f7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:04:19.638 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.636+0000 7f7713fff640 1 --2- 172.21.15.5:0/3427841739 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564cbf2d58b0 0x564cbf2ec8f0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f7704010fd0 tx=0x7f7704010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f7224dc3fdd4836 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:19.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.636+0000 7f7709ffb640 1 -- 172.21.15.5:0/3427841739 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77040b3070 con 0x564cbf2d58b0 2024-09-12T03:04:19.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.636+0000 7f7709ffb640 1 -- 172.21.15.5:0/3427841739 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7704097040 con 0x564cbf2d58b0 2024-09-12T03:04:19.639 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.636+0000 7f7709ffb640 1 -- 172.21.15.5:0/3427841739 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f770409a3b0 con 0x564cbf2d58b0 2024-09-12T03:04:19.643 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.637+0000 7f771c6ab740 1 -- 172.21.15.5:0/3427841739 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564cbf2d58b0 msgr2=0x564cbf2ec8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:19.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.637+0000 7f771c6ab740 1 --2- 172.21.15.5:0/3427841739 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564cbf2d58b0 0x564cbf2ec8f0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f7704010fd0 tx=0x7f7704010d70 comp rx=0 tx=0).stop 2024-09-12T03:04:19.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.637+0000 7f771c6ab740 1 -- 172.21.15.5:0/3427841739 shutdown_connections 2024-09-12T03:04:19.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.637+0000 7f771c6ab740 1 --2- 172.21.15.5:0/3427841739 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564cbf2e0a90 0x564cbf2f8320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:19.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.637+0000 7f771c6ab740 1 --2- 172.21.15.5:0/3427841739 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564cbf2d61d0 0x564cbf2f7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:19.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.637+0000 7f771c6ab740 1 --2- 172.21.15.5:0/3427841739 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564cbf2d58b0 0x564cbf2ec8f0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:19.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.637+0000 7f771c6ab740 1 -- 172.21.15.5:0/3427841739 >> 172.21.15.5:0/3427841739 conn(0x564cbf2d1e40 msgr2=0x564cbf2d7b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:19.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.637+0000 7f771c6ab740 1 -- 172.21.15.5:0/3427841739 shutdown_connections 2024-09-12T03:04:19.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.637+0000 7f771c6ab740 1 -- 172.21.15.5:0/3427841739 wait complete. 2024-09-12T03:04:19.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.637+0000 7f771c6ab740 1 -- start start 2024-09-12T03:04:19.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.637+0000 7f771c6ab740 1 Processor -- start 2024-09-12T03:04:19.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.638+0000 7f771c6ab740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564cbf2d58b0 0x564cbf2e9170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:19.644 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.638+0000 7f771c6ab740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564cbf2d61d0 0x564cbf2da370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:19.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.638+0000 7f771c6ab740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564cbf2e0a90 0x564cbf2da8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:19.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.638+0000 7f771c6ab740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f77141ac780 con 0x564cbf2d61d0 2024-09-12T03:04:19.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.638+0000 7f771c6ab740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f77141ac080 con 0x564cbf2e0a90 2024-09-12T03:04:19.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.638+0000 7f771c6ab740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f77141a9260 con 0x564cbf2d58b0 2024-09-12T03:04:19.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.638+0000 7f771c6ab740 10 client.0 _ll_register_callbacks cb 0x7f770409f070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:04:19.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.638+0000 7f770bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564cbf2d61d0 0x564cbf2da370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:19.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.638+0000 7f770bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564cbf2d61d0 0x564cbf2da370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43896/0 (socket says 172.21.15.5:43896) 2024-09-12T03:04:19.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.638+0000 7f770bfff640 1 -- 172.21.15.5:0/2093483658 learned_addr learned my addr 172.21.15.5:0/2093483658 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:04:19.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.638+0000 7f771886f640 1 --2- 172.21.15.5:0/2093483658 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564cbf2e0a90 0x564cbf2da8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:19.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.638+0000 7f7713fff640 1 --2- 172.21.15.5:0/2093483658 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564cbf2d58b0 0x564cbf2e9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:19.645 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.638+0000 7f770bfff640 1 -- 172.21.15.5:0/2093483658 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564cbf2d58b0 msgr2=0x564cbf2e9170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:19.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.638+0000 7f770bfff640 1 --2- 172.21.15.5:0/2093483658 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564cbf2d58b0 0x564cbf2e9170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:19.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.638+0000 7f770bfff640 1 -- 172.21.15.5:0/2093483658 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564cbf2e0a90 msgr2=0x564cbf2da8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:19.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.638+0000 7f770bfff640 1 --2- 172.21.15.5:0/2093483658 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564cbf2e0a90 0x564cbf2da8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:19.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.638+0000 7f770bfff640 1 -- 172.21.15.5:0/2093483658 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564cbf2db0c0 con 0x564cbf2d61d0 2024-09-12T03:04:19.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.639+0000 7f770bfff640 1 --2- 172.21.15.5:0/2093483658 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564cbf2d61d0 0x564cbf2da370 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f76fc0041c0 tx=0x7f76fc00bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe4f73252ae75056 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:19.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.639+0000 7f76f37fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T03:04:19.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.639+0000 7f76f37fe640 1 -- 172.21.15.5:0/2093483658 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76fc005220 con 0x564cbf2d61d0 2024-09-12T03:04:19.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.639+0000 7f76f37fe640 1 -- 172.21.15.5:0/2093483658 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f76fc0053c0 con 0x564cbf2d61d0 2024-09-12T03:04:19.646 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.639+0000 7f771c6ab740 1 -- 172.21.15.5:0/2093483658 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564cbf23ed60 con 0x564cbf2d61d0 2024-09-12T03:04:19.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.639+0000 7f76f37fe640 1 -- 172.21.15.5:0/2093483658 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f76fc003e50 con 0x564cbf2d61d0 2024-09-12T03:04:19.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.639+0000 7f771c6ab740 10 client.16752 fetch_fsmap learned FSMap version 33 2024-09-12T03:04:19.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.639+0000 7f771c6ab740 1 -- 172.21.15.5:0/2093483658 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x564cbf35ae60 con 0x564cbf2d61d0 2024-09-12T03:04:19.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.639+0000 7f76f37fe640 1 -- 172.21.15.5:0/2093483658 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x564cbf23ed60 con 0x564cbf2d61d0 2024-09-12T03:04:19.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.640+0000 7f76f37fe640 1 -- 172.21.15.5:0/2093483658 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 33) ==== 8374+0+0 (secure 0 0 0) 0x7f76fc01d860 con 0x564cbf2d61d0 2024-09-12T03:04:19.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.640+0000 7f771c6ab740 10 client.16752 fetch_fsmap finished waiting for FSMap version 33 2024-09-12T03:04:19.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.640+0000 7f771c6ab740 10 client.16752 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:04:19.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.640+0000 7f771c6ab740 20 client.16752 populate_metadata read hostname 'smithi005' 2024-09-12T03:04:19.647 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.640+0000 7f771c6ab740 1 --2- 172.21.15.5:0/2093483658 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x564cbf35b5d0 0x564cbf35d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:19.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.640+0000 7f771c6ab740 4 client.16752 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-12T03:04:19.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.640+0000 7f771c6ab740 1 -- 172.21.15.5:0/2093483658 --> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564cbf1bb210 con 0x564cbf35b5d0 2024-09-12T03:04:19.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.640+0000 7f7713fff640 1 --2- 172.21.15.5:0/2093483658 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x564cbf35b5d0 0x564cbf35d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:19.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.640+0000 7f7713fff640 1 --2- 172.21.15.5:0/2093483658 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x564cbf35b5d0 0x564cbf35d9f0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1b47f731b67634d server_cookie=1e016c4b66d450c1 in_seq=0 out_seq=0 2024-09-12T03:04:19.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.640+0000 7f76f37fe640 10 client.16752 ms_handle_connect on v2:172.21.15.148:6838/2317600000 2024-09-12T03:04:19.648 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.640+0000 7f76f37fe640 1 -- 172.21.15.5:0/2093483658 <== mds.0 v2:172.21.15.148:6838/2317600000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x564cbf1bb210 con 0x564cbf35b5d0 2024-09-12T03:04:19.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.640+0000 7f76f37fe640 10 client.16752 handle_command_reply: tid=0 2024-09-12T03:04:19.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.641+0000 7f771c6ab740 1 client.16752 shutdown 2024-09-12T03:04:19.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.641+0000 7f771c6ab740 1 -- 172.21.15.5:0/2093483658 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564cbf2d61d0 msgr2=0x564cbf2da370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:19.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.641+0000 7f771c6ab740 1 --2- 172.21.15.5:0/2093483658 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564cbf2d61d0 0x564cbf2da370 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f76fc0041c0 tx=0x7f76fc00bee0 comp rx=0 tx=0).stop 2024-09-12T03:04:19.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.641+0000 7f771c6ab740 1 -- 172.21.15.5:0/2093483658 shutdown_connections 2024-09-12T03:04:19.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.641+0000 7f771c6ab740 1 --2- 172.21.15.5:0/2093483658 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x564cbf35b5d0 0x564cbf35d9f0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:19.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.641+0000 7f771c6ab740 1 --2- 172.21.15.5:0/2093483658 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564cbf2d61d0 0x564cbf2da370 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:19.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.641+0000 7f771c6ab740 1 --2- 172.21.15.5:0/2093483658 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x564cbf2e0a90 0x564cbf2da8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:19.649 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.641+0000 7f771c6ab740 1 --2- 172.21.15.5:0/2093483658 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x564cbf2d58b0 0x564cbf2e9170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:19.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.641+0000 7f76f37fe640 0 client.16752 ms_handle_reset on v2:172.21.15.148:6838/2317600000 2024-09-12T03:04:19.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.641+0000 7f771c6ab740 1 -- 172.21.15.5:0/2093483658 >> 172.21.15.5:0/2093483658 conn(0x564cbf2d1e40 msgr2=0x564cbf2cc270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:19.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.641+0000 7f771c6ab740 1 -- 172.21.15.5:0/2093483658 shutdown_connections 2024-09-12T03:04:19.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.641+0000 7f771c6ab740 1 -- 172.21.15.5:0/2093483658 wait complete. 2024-09-12T03:04:19.650 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.641+0000 7f771c6ab740 20 client.16752 trim_cache size 0 max 16384 2024-09-12T03:04:19.650 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-12T03:04:19.650 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2024-09-12T03:04:19.650 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2024-09-12T03:04:19.650 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-12T03:04:19.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.642+0000 7f771a2d2640 1 -- 172.21.15.5:0/3556992256 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f77000a2b80 msgr2=0x7f77000a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:19.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.642+0000 7f771a2d2640 1 --2- 172.21.15.5:0/3556992256 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f77000a2b80 0x7f77000a5040 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f770c005cc0 tx=0x7f770c009820 comp rx=0 tx=0).stop 2024-09-12T03:04:19.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.642+0000 7f771a2d2640 1 -- 172.21.15.5:0/3556992256 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7714100e20 msgr2=0x7f7714198450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:19.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.642+0000 7f771a2d2640 1 --2- 172.21.15.5:0/3556992256 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7714100e20 0x7f7714198450 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f770400d950 tx=0x7f770400de10 comp rx=0 tx=0).stop 2024-09-12T03:04:19.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.642+0000 7f771a2d2640 1 -- 172.21.15.5:0/3556992256 shutdown_connections 2024-09-12T03:04:19.651 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.642+0000 7f771a2d2640 1 --2- 172.21.15.5:0/3556992256 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7714102ef0 0x7f771419cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:19.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.642+0000 7f771a2d2640 1 --2- 172.21.15.5:0/3556992256 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7714102570 0x7f7714198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:19.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.642+0000 7f771a2d2640 1 --2- 172.21.15.5:0/3556992256 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f77000a2b80 0x7f77000a5040 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:19.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.642+0000 7f771a2d2640 1 --2- 172.21.15.5:0/3556992256 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7714100e20 0x7f7714198450 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:19.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.642+0000 7f771a2d2640 1 -- 172.21.15.5:0/3556992256 >> 172.21.15.5:0/3556992256 conn(0x7f77140fc7d0 msgr2=0x7f771406a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:19.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.642+0000 7f771a2d2640 1 -- 172.21.15.5:0/3556992256 shutdown_connections 2024-09-12T03:04:19.652 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:19.642+0000 7f771a2d2640 1 -- 172.21.15.5:0/3556992256 wait complete. 2024-09-12T03:04:20.403 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-12T03:04:20.404 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:04:20.667 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:04:21.041 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:04:20 smithi005 ceph-mon[31197]: pgmap v518: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 853 B/s wr, 12 op/s 2024-09-12T03:04:21.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:21.119+0000 7f7c07370640 1 -- 172.21.15.5:0/551997521 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c00106f10 msgr2=0x7f7c0010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:21.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:21.119+0000 7f7c07370640 1 --2- 172.21.15.5:0/551997521 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c00106f10 0x7f7c0010ea90 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f7bf8009ca0 tx=0x7f7bf802a8c0 comp rx=0 tx=0).stop 2024-09-12T03:04:21.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:21.119+0000 7f7c07370640 1 -- 172.21.15.5:0/551997521 shutdown_connections 2024-09-12T03:04:21.144 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:21.119+0000 7f7c07370640 1 --2- 172.21.15.5:0/551997521 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c00106f10 0x7f7c0010ea90 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:21.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:21.119+0000 7f7c07370640 1 --2- 172.21.15.5:0/551997521 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7c00102790 0x7f7c001067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:21.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:21.119+0000 7f7c07370640 1 --2- 172.21.15.5:0/551997521 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7c00101de0 0x7f7c001021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:21.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:21.119+0000 7f7c07370640 1 -- 172.21.15.5:0/551997521 >> 172.21.15.5:0/551997521 conn(0x7f7c000fd8c0 msgr2=0x7f7c000ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:21.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:21.119+0000 7f7c07370640 1 -- 172.21.15.5:0/551997521 shutdown_connections 2024-09-12T03:04:21.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:21.119+0000 7f7c07370640 1 -- 172.21.15.5:0/551997521 wait complete. 2024-09-12T03:04:21.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:21.120+0000 7f7c07370640 1 Processor -- start 2024-09-12T03:04:21.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:21.120+0000 7f7c07370640 1 -- start start 2024-09-12T03:04:21.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:21.120+0000 7f7c07370640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7c00101de0 0x7f7c00198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:21.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:21.120+0000 7f7c07370640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c00102790 0x7f7c001988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:21.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:21.120+0000 7f7c07370640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7c00106f10 0x7f7c0019cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:21.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:21.120+0000 7f7c07370640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7c00111f70 con 0x7f7c00102790 2024-09-12T03:04:21.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:21.120+0000 7f7c07370640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f7c00111df0 con 0x7f7c00106f10 2024-09-12T03:04:21.145 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:21.120+0000 7f7c07370640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f7c001120f0 con 0x7f7c00101de0 2024-09-12T03:04:21.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:21.120+0000 7f7c0490b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c00102790 0x7f7c001988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:21.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:21.120+0000 7f7c0490b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c00102790 0x7f7c001988a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43912/0 (socket says 172.21.15.5:43912) 2024-09-12T03:04:21.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:21.120+0000 7f7c0490b640 1 -- 172.21.15.5:0/2421481322 learned_addr learned my addr 172.21.15.5:0/2421481322 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:04:21.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:21.120+0000 7f7c0510c640 1 --2- 172.21.15.5:0/2421481322 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7c00101de0 0x7f7c00198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:21.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:21.120+0000 7f7c0590d640 1 --2- 172.21.15.5:0/2421481322 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7c00106f10 0x7f7c0019cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:21.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:21.121+0000 7f7c0490b640 1 -- 172.21.15.5:0/2421481322 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7c00101de0 msgr2=0x7f7c00198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:21.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:21.121+0000 7f7c0490b640 1 --2- 172.21.15.5:0/2421481322 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7c00101de0 0x7f7c00198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:21.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:21.121+0000 7f7c0490b640 1 -- 172.21.15.5:0/2421481322 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7c00106f10 msgr2=0x7f7c0019cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:21.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:21.121+0000 7f7c0490b640 1 --2- 172.21.15.5:0/2421481322 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7c00106f10 0x7f7c0019cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:21.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:21.121+0000 7f7c0490b640 1 -- 172.21.15.5:0/2421481322 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c0019d270 con 0x7f7c00102790 2024-09-12T03:04:21.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:21.121+0000 7f7c0510c640 1 --2- 172.21.15.5:0/2421481322 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7c00101de0 0x7f7c00198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:04:21.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:21.121+0000 7f7c0590d640 1 --2- 172.21.15.5:0/2421481322 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7c00106f10 0x7f7c0019cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:04:21.146 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:21.121+0000 7f7c0490b640 1 --2- 172.21.15.5:0/2421481322 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c00102790 0x7f7c001988a0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f7bf000c920 tx=0x7f7bf000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:21.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:21.121+0000 7f7bf67fc640 1 -- 172.21.15.5:0/2421481322 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bf0007c70 con 0x7f7c00102790 2024-09-12T03:04:21.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:21.121+0000 7f7bf67fc640 1 -- 172.21.15.5:0/2421481322 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7bf0007e10 con 0x7f7c00102790 2024-09-12T03:04:21.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:21.121+0000 7f7c07370640 1 -- 172.21.15.5:0/2421481322 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c0019d560 con 0x7f7c00102790 2024-09-12T03:04:21.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:21.121+0000 7f7bf67fc640 1 -- 172.21.15.5:0/2421481322 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bf0018890 con 0x7f7c00102790 2024-09-12T03:04:21.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:21.121+0000 7f7c07370640 1 -- 172.21.15.5:0/2421481322 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c001a4e40 con 0x7f7c00102790 2024-09-12T03:04:21.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:21.122+0000 7f7c07370640 1 -- 172.21.15.5:0/2421481322 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c001021c0 con 0x7f7c00102790 2024-09-12T03:04:21.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:21.123+0000 7f7bf67fc640 1 -- 172.21.15.5:0/2421481322 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f7bf0018a50 con 0x7f7c00102790 2024-09-12T03:04:21.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:21.124+0000 7f7bf67fc640 1 --2- 172.21.15.5:0/2421481322 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f7bec0a2c00 0x7f7bec0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:21.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:21.124+0000 7f7c0510c640 1 --2- 172.21.15.5:0/2421481322 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f7bec0a2c00 0x7f7bec0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:21.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:21.124+0000 7f7bf67fc640 1 -- 172.21.15.5:0/2421481322 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f7bf00cf3c0 con 0x7f7c00102790 2024-09-12T03:04:21.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:21.125+0000 7f7c0510c640 1 --2- 172.21.15.5:0/2421481322 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f7bec0a2c00 0x7f7bec0a50c0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc00baa0 tx=0x7f7bfc005d90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:21.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:21.127+0000 7f7bf67fc640 1 -- 172.21.15.5:0/2421481322 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7bf0014030 con 0x7f7c00102790 2024-09-12T03:04:21.165 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:04:20 smithi148 ceph-mon[35090]: pgmap v518: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 853 B/s wr, 12 op/s 2024-09-12T03:04:21.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:04:20 smithi146 ceph-mon[35011]: pgmap v518: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 853 B/s wr, 12 op/s 2024-09-12T03:04:21.337 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:21.337+0000 7f7c07370640 1 -- 172.21.15.5:0/2421481322 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7c001996c0 con 0x7f7c00102790 2024-09-12T03:04:21.338 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:21.338+0000 7f7bf67fc640 1 -- 172.21.15.5:0/2421481322 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11969 (secure 0 0 0) 0x7f7bf001f300 con 0x7f7c00102790 2024-09-12T03:04:21.339 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-12T03:04:21.339 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":33,"btime":"2024-09-12T03:03:57:799392+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/4135232521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":4135232521},{"type":"v1","addr":"172.21.15.5:6835","nonce":4135232521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/4290221940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":4290221940},{"type":"v1","addr":"172.21.15.5:6837","nonce":4290221940}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T02:56:16.595562+0000","modified":"2024-09-12T03:03:57.585077+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24502,"mds_1":24487,"mds_2":24481,"mds_3":24479,"mds_4":24458},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/26447341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":26447341},{"type":"v1","addr":"172.21.15.5:6839","nonce":26447341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2393443295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2393443295},{"type":"v1","addr":"172.21.15.5:6841","nonce":2393443295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6833/3686970137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3686970137},{"type":"v1","addr":"172.21.15.146:6833","nonce":3686970137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6833/568512209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":568512209},{"type":"v1","addr":"172.21.15.148:6833","nonce":568512209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6835/2540740420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2540740420},{"type":"v1","addr":"172.21.15.146:6835","nonce":2540740420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.146:6837/1673370937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":1673370937},{"type":"v1","addr":"172.21.15.146:6837","nonce":1673370937}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":3,"incarnation":22,"state":"up:active","state_seq":6,"addr":"172.21.15.146:6839/3036767228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":3036767228},{"type":"v1","addr":"172.21.15.146:6839","nonce":3036767228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":2,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.148:6835/4195086427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":4195086427},{"type":"v1","addr":"172.21.15.148:6835","nonce":4195086427}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.148:6837/2606873922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2606873922},{"type":"v1","addr":"172.21.15.148:6837","nonce":2606873922}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6839/2317600000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":2317600000},{"type":"v1","addr":"172.21.15.148:6839","nonce":2317600000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24502,"qdb_cluster":[24458,24479,24481,24487,24502]},"id":1}]} 2024-09-12T03:04:21.341 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 33 2024-09-12T03:04:21.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:21.340+0000 7f7c07370640 1 -- 172.21.15.5:0/2421481322 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f7bec0a2c00 msgr2=0x7f7bec0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:21.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:21.340+0000 7f7c07370640 1 --2- 172.21.15.5:0/2421481322 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f7bec0a2c00 0x7f7bec0a50c0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc00baa0 tx=0x7f7bfc005d90 comp rx=0 tx=0).stop 2024-09-12T03:04:21.341 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:21.340+0000 7f7c07370640 1 -- 172.21.15.5:0/2421481322 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c00102790 msgr2=0x7f7c001988a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:21.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:21.340+0000 7f7c07370640 1 --2- 172.21.15.5:0/2421481322 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c00102790 0x7f7c001988a0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f7bf000c920 tx=0x7f7bf000cde0 comp rx=0 tx=0).stop 2024-09-12T03:04:21.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:21.340+0000 7f7c07370640 1 -- 172.21.15.5:0/2421481322 shutdown_connections 2024-09-12T03:04:21.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:21.340+0000 7f7c07370640 1 --2- 172.21.15.5:0/2421481322 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f7bec0a2c00 0x7f7bec0a50c0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:21.342 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:21.340+0000 7f7c07370640 1 --2- 172.21.15.5:0/2421481322 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7c00102790 0x7f7c001988a0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:21.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:21.340+0000 7f7c07370640 1 --2- 172.21.15.5:0/2421481322 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7c00106f10 0x7f7c0019cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:21.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:21.340+0000 7f7c07370640 1 --2- 172.21.15.5:0/2421481322 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7c00101de0 0x7f7c00198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:21.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:21.340+0000 7f7c07370640 1 -- 172.21.15.5:0/2421481322 >> 172.21.15.5:0/2421481322 conn(0x7f7c000fd8c0 msgr2=0x7f7c000fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:21.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:21.340+0000 7f7c07370640 1 -- 172.21.15.5:0/2421481322 shutdown_connections 2024-09-12T03:04:21.343 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:21.340+0000 7f7c07370640 1 -- 172.21.15.5:0/2421481322 wait complete. 2024-09-12T03:04:22.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:04:21 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/2421481322' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:04:22.105 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-09-12T03:04:22.165 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:04:21 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/2421481322' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:04:22.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:04:21 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/2421481322' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:04:22.363 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:04:22.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.830+0000 7fcfc7573640 1 -- 172.21.15.5:0/3203909939 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfc0074e30 msgr2=0x7fcfc010df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:22.843 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.830+0000 7fcfc7573640 1 --2- 172.21.15.5:0/3203909939 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfc0074e30 0x7fcfc010df00 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fcfb4009900 tx=0x7fcfb402a480 comp rx=0 tx=0).stop 2024-09-12T03:04:22.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.830+0000 7fcfc7573640 1 -- 172.21.15.5:0/3203909939 shutdown_connections 2024-09-12T03:04:22.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.830+0000 7fcfc7573640 1 --2- 172.21.15.5:0/3203909939 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfc0074e30 0x7fcfc010df00 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:22.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.830+0000 7fcfc7573640 1 --2- 172.21.15.5:0/3203909939 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcfc010e440 0x7fcfc0110830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:22.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.830+0000 7fcfc7573640 1 --2- 172.21.15.5:0/3203909939 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcfc0074510 0x7fcfc00748f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:22.844 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.830+0000 7fcfc7573640 1 -- 172.21.15.5:0/3203909939 >> 172.21.15.5:0/3203909939 conn(0x7fcfc00fd8c0 msgr2=0x7fcfc00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:22.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.830+0000 7fcfc7573640 1 -- 172.21.15.5:0/3203909939 shutdown_connections 2024-09-12T03:04:22.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.830+0000 7fcfc7573640 1 -- 172.21.15.5:0/3203909939 wait complete. 2024-09-12T03:04:22.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.831+0000 7fcfc7573640 1 Processor -- start 2024-09-12T03:04:22.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.831+0000 7fcfc7573640 1 -- start start 2024-09-12T03:04:22.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.831+0000 7fcfc7573640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfc0074510 0x7fcfc01a0400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:22.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.831+0000 7fcfc7573640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcfc0074e30 0x7fcfc01a0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:22.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.831+0000 7fcfc7573640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcfc010e440 0x7fcfc019a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:22.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.831+0000 7fcfc7573640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc01141d0 con 0x7fcfc0074510 2024-09-12T03:04:22.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.831+0000 7fcfc7573640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc0114050 con 0x7fcfc0074e30 2024-09-12T03:04:22.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.831+0000 7fcfc7573640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc0114350 con 0x7fcfc010e440 2024-09-12T03:04:22.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.831+0000 7fcfc530f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfc0074510 0x7fcfc01a0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:22.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.831+0000 7fcfc530f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfc0074510 0x7fcfc01a0400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43932/0 (socket says 172.21.15.5:43932) 2024-09-12T03:04:22.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.831+0000 7fcfc530f640 1 -- 172.21.15.5:0/4267807340 learned_addr learned my addr 172.21.15.5:0/4267807340 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:04:22.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.831+0000 7fcfc4b0e640 1 --2- 172.21.15.5:0/4267807340 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcfc0074e30 0x7fcfc01a0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:22.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.831+0000 7fcfc5b10640 1 --2- 172.21.15.5:0/4267807340 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcfc010e440 0x7fcfc019a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:22.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.831+0000 7fcfc530f640 1 -- 172.21.15.5:0/4267807340 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcfc010e440 msgr2=0x7fcfc019a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:22.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.831+0000 7fcfc530f640 1 --2- 172.21.15.5:0/4267807340 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcfc010e440 0x7fcfc019a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:22.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.831+0000 7fcfc530f640 1 -- 172.21.15.5:0/4267807340 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcfc0074e30 msgr2=0x7fcfc01a0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:22.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.831+0000 7fcfc530f640 1 --2- 172.21.15.5:0/4267807340 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcfc0074e30 0x7fcfc01a0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:22.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.831+0000 7fcfc530f640 1 -- 172.21.15.5:0/4267807340 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfc019acf0 con 0x7fcfc0074510 2024-09-12T03:04:22.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.832+0000 7fcfc530f640 1 --2- 172.21.15.5:0/4267807340 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfc0074510 0x7fcfc01a0400 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc00cce0 tx=0x7fcfbc00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:22.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.832+0000 7fcfb27fc640 1 -- 172.21.15.5:0/4267807340 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfbc004b20 con 0x7fcfc0074510 2024-09-12T03:04:22.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.832+0000 7fcfb27fc640 1 -- 172.21.15.5:0/4267807340 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfbc004cc0 con 0x7fcfc0074510 2024-09-12T03:04:22.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.832+0000 7fcfc7573640 1 -- 172.21.15.5:0/4267807340 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfc019afe0 con 0x7fcfc0074510 2024-09-12T03:04:22.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.832+0000 7fcfb27fc640 1 -- 172.21.15.5:0/4267807340 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfbc00f7d0 con 0x7fcfc0074510 2024-09-12T03:04:22.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.832+0000 7fcfc7573640 1 -- 172.21.15.5:0/4267807340 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfc01a72d0 con 0x7fcfc0074510 2024-09-12T03:04:22.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.834+0000 7fcfb27fc640 1 -- 172.21.15.5:0/4267807340 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fcfbc00f970 con 0x7fcfc0074510 2024-09-12T03:04:22.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.834+0000 7fcfc994c740 1 Processor -- start 2024-09-12T03:04:22.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.834+0000 7fcfc994c740 1 -- start start 2024-09-12T03:04:22.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.835+0000 7fcfc994c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55928f8f8b40 0x55928f8f7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:22.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.835+0000 7fcfc994c740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55928f8f75d0 0x55928f8f7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:22.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.835+0000 7fcfc994c740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55928f902370 0x55928f904760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:22.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.835+0000 7fcfc994c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55928f85d2f0 con 0x55928f8f8b40 2024-09-12T03:04:22.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.835+0000 7fcfc994c740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55928f804cf0 con 0x55928f8f75d0 2024-09-12T03:04:22.850 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.835+0000 7fcfc994c740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55928f8592b0 con 0x55928f902370 2024-09-12T03:04:22.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.835+0000 7fcfc5b10640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55928f8f8b40 0x55928f8f7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:22.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.835+0000 7fcfc5b10640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55928f8f8b40 0x55928f8f7090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43940/0 (socket says 172.21.15.5:43940) 2024-09-12T03:04:22.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.835+0000 7fcfc5b10640 1 -- 172.21.15.5:0/2711120016 learned_addr learned my addr 172.21.15.5:0/2711120016 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:04:22.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.835+0000 7fcfc4b0e640 1 --2- 172.21.15.5:0/2711120016 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55928f902370 0x55928f904760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:22.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.835+0000 7fcfc530f640 1 --2- 172.21.15.5:0/2711120016 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55928f8f75d0 0x55928f8f7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:22.851 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.835+0000 7fcfc5b10640 1 -- 172.21.15.5:0/2711120016 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55928f902370 msgr2=0x55928f904760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:22.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.835+0000 7fcfc5b10640 1 --2- 172.21.15.5:0/2711120016 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55928f902370 0x55928f904760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:22.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.835+0000 7fcfc5b10640 1 -- 172.21.15.5:0/2711120016 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55928f8f75d0 msgr2=0x55928f8f7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:22.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.835+0000 7fcfc5b10640 1 --2- 172.21.15.5:0/2711120016 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55928f8f75d0 0x55928f8f7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:22.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.835+0000 7fcfc5b10640 1 -- 172.21.15.5:0/2711120016 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55928f7e93a0 con 0x55928f8f8b40 2024-09-12T03:04:22.852 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.835+0000 7fcfc530f640 1 --2- 172.21.15.5:0/2711120016 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55928f8f75d0 0x55928f8f7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:04:22.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.835+0000 7fcfc4b0e640 1 --2- 172.21.15.5:0/2711120016 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55928f902370 0x55928f904760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:04:22.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.835+0000 7fcfc5b10640 1 --2- 172.21.15.5:0/2711120016 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55928f8f8b40 0x55928f8f7090 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fcfb800a860 tx=0x7fcfb800ad20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e67138b10cc0b4ad server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:22.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.836+0000 7fcfb27fc640 1 --2- 172.21.15.5:0/4267807340 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fcfac0a2b80 0x7fcfac0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:22.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.836+0000 7fcfab7fe640 1 -- 172.21.15.5:0/2711120016 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb8015070 con 0x55928f8f8b40 2024-09-12T03:04:22.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.836+0000 7fcfc530f640 1 --2- 172.21.15.5:0/4267807340 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fcfac0a2b80 0x7fcfac0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:22.853 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.836+0000 7fcfab7fe640 1 -- 172.21.15.5:0/2711120016 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfb800cb40 con 0x55928f8f8b40 2024-09-12T03:04:22.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.836+0000 7fcfb27fc640 1 -- 172.21.15.5:0/4267807340 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fcfbc0cf200 con 0x7fcfc0074510 2024-09-12T03:04:22.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.836+0000 7fcfab7fe640 1 -- 172.21.15.5:0/2711120016 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb80185c0 con 0x55928f8f8b40 2024-09-12T03:04:22.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.836+0000 7fcfc994c740 1 -- 172.21.15.5:0/2711120016 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55928f8f8b40 msgr2=0x55928f8f7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:22.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.836+0000 7fcfc994c740 1 --2- 172.21.15.5:0/2711120016 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55928f8f8b40 0x55928f8f7090 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fcfb800a860 tx=0x7fcfb800ad20 comp rx=0 tx=0).stop 2024-09-12T03:04:22.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.836+0000 7fcfc530f640 1 --2- 172.21.15.5:0/4267807340 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fcfac0a2b80 0x7fcfac0a5040 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc00ab20 tx=0x7fcfbc005940 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:22.854 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.837+0000 7fcfc994c740 1 -- 172.21.15.5:0/2711120016 shutdown_connections 2024-09-12T03:04:22.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.837+0000 7fcfc994c740 1 --2- 172.21.15.5:0/2711120016 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55928f902370 0x55928f904760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:22.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.837+0000 7fcfc994c740 1 --2- 172.21.15.5:0/2711120016 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55928f8f75d0 0x55928f8f7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:22.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.837+0000 7fcfc994c740 1 --2- 172.21.15.5:0/2711120016 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55928f8f8b40 0x55928f8f7090 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:22.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.837+0000 7fcfc994c740 1 -- 172.21.15.5:0/2711120016 >> 172.21.15.5:0/2711120016 conn(0x55928f8c0310 msgr2=0x55928f8f3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:22.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.837+0000 7fcfc994c740 1 -- 172.21.15.5:0/2711120016 shutdown_connections 2024-09-12T03:04:22.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.837+0000 7fcfc994c740 1 -- 172.21.15.5:0/2711120016 wait complete. 2024-09-12T03:04:22.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.837+0000 7fcfc994c740 1 -- start start 2024-09-12T03:04:22.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.837+0000 7fcfc994c740 1 Processor -- start 2024-09-12T03:04:22.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.837+0000 7fcfc994c740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55928f8f75d0 0x55928f901d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:22.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.837+0000 7fcfc994c740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55928f8f8b40 0x55928f8ff000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:22.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.838+0000 7fcfc994c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55928f902370 0x55928f8ff5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:22.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.838+0000 7fcfc994c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55928f905070 con 0x55928f902370 2024-09-12T03:04:22.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.838+0000 7fcfc994c740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55928f818060 con 0x55928f8f8b40 2024-09-12T03:04:22.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.838+0000 7fcfc994c740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55928f9051f0 con 0x55928f8f75d0 2024-09-12T03:04:22.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.838+0000 7fcfc994c740 10 client.0 _ll_register_callbacks cb 0x55928f56b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:04:22.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.838+0000 7fcfc5b10640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55928f902370 0x55928f8ff5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:22.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.838+0000 7fcfc5b10640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55928f902370 0x55928f8ff5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43944/0 (socket says 172.21.15.5:43944) 2024-09-12T03:04:22.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.838+0000 7fcfc5b10640 1 -- 172.21.15.5:0/2478713812 learned_addr learned my addr 172.21.15.5:0/2478713812 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:04:22.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.838+0000 7fcfc5b10640 1 --2- 172.21.15.5:0/2478713812 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55928f8f75d0 0x55928f901d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:22.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.838+0000 7fcfc4b0e640 1 --2- 172.21.15.5:0/2478713812 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55928f8f8b40 0x55928f8ff000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:22.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.838+0000 7fcfc5b10640 1 -- 172.21.15.5:0/2478713812 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55928f8f75d0 msgr2=0x55928f901d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:22.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.838+0000 7fcfc5b10640 1 --2- 172.21.15.5:0/2478713812 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55928f8f75d0 0x55928f901d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:22.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.838+0000 7fcfc5b10640 1 -- 172.21.15.5:0/2478713812 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55928f8f8b40 msgr2=0x55928f8ff000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:22.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.838+0000 7fcfc5b10640 1 --2- 172.21.15.5:0/2478713812 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55928f8f8b40 0x55928f8ff000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:22.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.838+0000 7fcfc5b10640 1 -- 172.21.15.5:0/2478713812 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55928f5ce3c0 con 0x55928f902370 2024-09-12T03:04:22.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.838+0000 7fcfc5b10640 1 --2- 172.21.15.5:0/2478713812 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55928f902370 0x55928f8ff5d0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fcfb8006c60 tx=0x7fcfb80047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19d0bb66463bfb43 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:22.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.838+0000 7fcfa8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T03:04:22.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.838+0000 7fcfa8ff9640 1 -- 172.21.15.5:0/2478713812 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb8015040 con 0x55928f902370 2024-09-12T03:04:22.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.838+0000 7fcfa8ff9640 1 -- 172.21.15.5:0/2478713812 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fcfb80113f0 con 0x55928f902370 2024-09-12T03:04:22.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.838+0000 7fcfc994c740 1 -- 172.21.15.5:0/2478713812 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcfb800cb40 con 0x55928f902370 2024-09-12T03:04:22.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.839+0000 7fcfa8ff9640 1 -- 172.21.15.5:0/2478713812 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fcfb80115f0 con 0x55928f902370 2024-09-12T03:04:22.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.839+0000 7fcfc994c740 10 client.16770 fetch_fsmap learned FSMap version 33 2024-09-12T03:04:22.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.839+0000 7fcfc994c740 1 -- 172.21.15.5:0/2478713812 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55928f910a60 con 0x55928f902370 2024-09-12T03:04:22.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.839+0000 7fcfa8ff9640 1 -- 172.21.15.5:0/2478713812 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfb800cb40 con 0x55928f902370 2024-09-12T03:04:22.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.839+0000 7fcfa8ff9640 1 -- 172.21.15.5:0/2478713812 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 33) ==== 8374+0+0 (secure 0 0 0) 0x7fcfb8028020 con 0x55928f902370 2024-09-12T03:04:22.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.839+0000 7fcfc994c740 10 client.16770 fetch_fsmap finished waiting for FSMap version 33 2024-09-12T03:04:22.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.839+0000 7fcfc994c740 10 client.16770 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-12T03:04:22.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.839+0000 7fcfc994c740 20 client.16770 populate_metadata read hostname 'smithi005' 2024-09-12T03:04:22.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.839+0000 7fcfc994c740 1 --2- 172.21.15.5:0/2478713812 >> [v2:172.21.15.146:6836/1673370937,v1:172.21.15.146:6837/1673370937] conn(0x55928f911db0 0x55928f9141a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:22.861 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.839+0000 7fcfc994c740 4 client.16770 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:04:22.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.839+0000 7fcfc994c740 1 -- 172.21.15.5:0/2478713812 --> [v2:172.21.15.146:6836/1673370937,v1:172.21.15.146:6837/1673370937] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55928f86a860 con 0x55928f911db0 2024-09-12T03:04:22.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.839+0000 7fcfc4b0e640 1 --2- 172.21.15.5:0/2478713812 >> [v2:172.21.15.146:6836/1673370937,v1:172.21.15.146:6837/1673370937] conn(0x55928f911db0 0x55928f9141a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:22.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.840+0000 7fcfc4b0e640 1 --2- 172.21.15.5:0/2478713812 >> [v2:172.21.15.146:6836/1673370937,v1:172.21.15.146:6837/1673370937] conn(0x55928f911db0 0x55928f9141a0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=40c5c453eca2c3ac server_cookie=2f260cf219a0479d in_seq=0 out_seq=0 2024-09-12T03:04:22.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.840+0000 7fcfa8ff9640 10 client.16770 ms_handle_connect on v2:172.21.15.146:6836/1673370937 2024-09-12T03:04:22.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.840+0000 7fcfa8ff9640 1 -- 172.21.15.5:0/2478713812 <== mds.4 v2:172.21.15.146:6836/1673370937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55928f86a860 con 0x55928f911db0 2024-09-12T03:04:22.862 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.841+0000 7fcfa8ff9640 10 client.16770 handle_command_reply: tid=0 2024-09-12T03:04:22.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.841+0000 7fcfc994c740 1 client.16770 shutdown 2024-09-12T03:04:22.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.841+0000 7fcfc994c740 1 -- 172.21.15.5:0/2478713812 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55928f902370 msgr2=0x55928f8ff5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:22.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.841+0000 7fcfc994c740 1 --2- 172.21.15.5:0/2478713812 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55928f902370 0x55928f8ff5d0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fcfb8006c60 tx=0x7fcfb80047d0 comp rx=0 tx=0).stop 2024-09-12T03:04:22.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.841+0000 7fcfc994c740 1 -- 172.21.15.5:0/2478713812 shutdown_connections 2024-09-12T03:04:22.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.841+0000 7fcfc994c740 1 --2- 172.21.15.5:0/2478713812 >> [v2:172.21.15.146:6836/1673370937,v1:172.21.15.146:6837/1673370937] conn(0x55928f911db0 0x55928f9141a0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:22.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.841+0000 7fcfc994c740 1 --2- 172.21.15.5:0/2478713812 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55928f902370 0x55928f8ff5d0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:22.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.841+0000 7fcfc994c740 1 --2- 172.21.15.5:0/2478713812 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55928f8f8b40 0x55928f8ff000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:22.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.841+0000 7fcfa8ff9640 0 client.16770 ms_handle_reset on v2:172.21.15.146:6836/1673370937 2024-09-12T03:04:22.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.841+0000 7fcfc994c740 1 --2- 172.21.15.5:0/2478713812 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55928f8f75d0 0x55928f901d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:22.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.841+0000 7fcfc994c740 1 -- 172.21.15.5:0/2478713812 >> 172.21.15.5:0/2478713812 conn(0x55928f8c0310 msgr2=0x55928f8f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:22.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.841+0000 7fcfc994c740 1 -- 172.21.15.5:0/2478713812 shutdown_connections 2024-09-12T03:04:22.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.841+0000 7fcfc994c740 1 -- 172.21.15.5:0/2478713812 wait complete. 2024-09-12T03:04:22.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.841+0000 7fcfc994c740 20 client.16770 trim_cache size 0 max 16384 2024-09-12T03:04:22.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.854+0000 7fcfc994c740 1 Processor -- start 2024-09-12T03:04:22.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.854+0000 7fcfc994c740 1 -- start start 2024-09-12T03:04:22.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.855+0000 7fcfc994c740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55928f8f75d0 0x55928f8f3e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:22.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.855+0000 7fcfc994c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55928f8f8b40 0x55928f90e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:22.876 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.855+0000 7fcfc994c740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55928f902370 0x55928f919800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:22.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.855+0000 7fcfc994c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc01af860 con 0x55928f8f8b40 2024-09-12T03:04:22.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.855+0000 7fcfc994c740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc01b1430 con 0x55928f902370 2024-09-12T03:04:22.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.855+0000 7fcfc994c740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc01aff60 con 0x55928f8f75d0 2024-09-12T03:04:22.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.855+0000 7fcfc4b0e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55928f8f8b40 0x55928f90e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:22.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.855+0000 7fcfc4b0e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55928f8f8b40 0x55928f90e310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43948/0 (socket says 172.21.15.5:43948) 2024-09-12T03:04:22.877 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.855+0000 7fcfc4b0e640 1 -- 172.21.15.5:0/2124904580 learned_addr learned my addr 172.21.15.5:0/2124904580 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:04:22.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.855+0000 7fcfc5b10640 1 --2- 172.21.15.5:0/2124904580 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55928f8f75d0 0x55928f8f3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:22.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.855+0000 7fcfc5b10640 1 --2- 172.21.15.5:0/2124904580 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55928f902370 0x55928f919800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:22.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.855+0000 7fcfc4b0e640 1 -- 172.21.15.5:0/2124904580 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55928f8f75d0 msgr2=0x55928f8f3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:22.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.855+0000 7fcfc4b0e640 1 --2- 172.21.15.5:0/2124904580 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55928f8f75d0 0x55928f8f3e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:22.878 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.855+0000 7fcfc4b0e640 1 -- 172.21.15.5:0/2124904580 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55928f902370 msgr2=0x55928f919800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:22.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.855+0000 7fcfc4b0e640 1 --2- 172.21.15.5:0/2124904580 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55928f902370 0x55928f919800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:22.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.855+0000 7fcfc4b0e640 1 -- 172.21.15.5:0/2124904580 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55928f8fa120 con 0x55928f8f8b40 2024-09-12T03:04:22.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.855+0000 7fcfc4b0e640 1 --2- 172.21.15.5:0/2124904580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55928f8f8b40 0x55928f90e310 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fcfb4005ec0 tx=0x7fcfb40319e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5e1ecb3cf71a1da server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:22.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.855+0000 7fcfaaffd640 1 -- 172.21.15.5:0/2124904580 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb404b070 con 0x55928f8f8b40 2024-09-12T03:04:22.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.856+0000 7fcfaaffd640 1 -- 172.21.15.5:0/2124904580 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfb4005500 con 0x55928f8f8b40 2024-09-12T03:04:22.879 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.856+0000 7fcfaaffd640 1 -- 172.21.15.5:0/2124904580 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb4035040 con 0x55928f8f8b40 2024-09-12T03:04:22.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.856+0000 7fcfc994c740 1 -- 172.21.15.5:0/2124904580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55928f8f8b40 msgr2=0x55928f90e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:22.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.856+0000 7fcfc994c740 1 --2- 172.21.15.5:0/2124904580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55928f8f8b40 0x55928f90e310 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fcfb4005ec0 tx=0x7fcfb40319e0 comp rx=0 tx=0).stop 2024-09-12T03:04:22.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.856+0000 7fcfc994c740 1 -- 172.21.15.5:0/2124904580 shutdown_connections 2024-09-12T03:04:22.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.856+0000 7fcfc994c740 1 --2- 172.21.15.5:0/2124904580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55928f8f8b40 0x55928f90e310 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:22.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.856+0000 7fcfc994c740 1 --2- 172.21.15.5:0/2124904580 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55928f902370 0x55928f919800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:22.880 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.856+0000 7fcfc994c740 1 --2- 172.21.15.5:0/2124904580 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55928f8f75d0 0x55928f8f3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:22.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.856+0000 7fcfc994c740 1 -- 172.21.15.5:0/2124904580 >> 172.21.15.5:0/2124904580 conn(0x55928f8c0310 msgr2=0x55928f8f3700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:22.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.856+0000 7fcfc994c740 1 -- 172.21.15.5:0/2124904580 shutdown_connections 2024-09-12T03:04:22.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.856+0000 7fcfc994c740 1 -- 172.21.15.5:0/2124904580 wait complete. 2024-09-12T03:04:22.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.857+0000 7fcfc994c740 1 -- start start 2024-09-12T03:04:22.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.857+0000 7fcfc994c740 1 Processor -- start 2024-09-12T03:04:22.881 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.857+0000 7fcfc994c740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55928f8f75d0 0x55928f9063f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:22.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.857+0000 7fcfc994c740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55928f8f8b40 0x55928f906930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:22.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.857+0000 7fcfc994c740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55928f902370 0x55928f906e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:22.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.857+0000 7fcfc994c740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc01b0d60 con 0x55928f8f75d0 2024-09-12T03:04:22.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.857+0000 7fcfc994c740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc01b0660 con 0x55928f8f8b40 2024-09-12T03:04:22.882 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.857+0000 7fcfc994c740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc01ad840 con 0x55928f902370 2024-09-12T03:04:22.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.857+0000 7fcfc994c740 10 client.0 _ll_register_callbacks cb 0x7fcfb800a6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:04:22.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.857+0000 7fcfc5b10640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55928f8f75d0 0x55928f9063f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:22.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.857+0000 7fcfc5b10640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55928f8f75d0 0x55928f9063f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43956/0 (socket says 172.21.15.5:43956) 2024-09-12T03:04:22.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.857+0000 7fcfc5b10640 1 -- 172.21.15.5:0/842510245 learned_addr learned my addr 172.21.15.5:0/842510245 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:04:22.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.857+0000 7fcfc5b10640 1 --2- 172.21.15.5:0/842510245 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55928f902370 0x55928f906e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:22.883 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.857+0000 7fcfc4b0e640 1 --2- 172.21.15.5:0/842510245 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55928f8f8b40 0x55928f906930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:22.884 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.858+0000 7fcfc5b10640 1 -- 172.21.15.5:0/842510245 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55928f902370 msgr2=0x55928f906e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:22.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.858+0000 7fcfc5b10640 1 --2- 172.21.15.5:0/842510245 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55928f902370 0x55928f906e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:22.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.858+0000 7fcfc5b10640 1 -- 172.21.15.5:0/842510245 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55928f8f8b40 msgr2=0x55928f906930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:22.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.858+0000 7fcfc5b10640 1 --2- 172.21.15.5:0/842510245 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55928f8f8b40 0x55928f906930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:22.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.858+0000 7fcfc5b10640 1 -- 172.21.15.5:0/842510245 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55928f97b9d0 con 0x55928f8f75d0 2024-09-12T03:04:22.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.858+0000 7fcfc5b10640 1 --2- 172.21.15.5:0/842510245 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55928f8f75d0 0x55928f9063f0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fcfb801a840 tx=0x7fcfb80047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86ef083bd25129a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:22.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.858+0000 7fcfa8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T03:04:22.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.858+0000 7fcfa8ff9640 1 -- 172.21.15.5:0/842510245 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb803a070 con 0x55928f8f75d0 2024-09-12T03:04:22.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.858+0000 7fcfc994c740 1 -- 172.21.15.5:0/842510245 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcfb4005500 con 0x55928f8f75d0 2024-09-12T03:04:22.885 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.858+0000 7fcfa8ff9640 1 -- 172.21.15.5:0/842510245 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fcfb80113f0 con 0x55928f8f75d0 2024-09-12T03:04:22.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.858+0000 7fcfa8ff9640 1 -- 172.21.15.5:0/842510245 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fcfb801e040 con 0x55928f8f75d0 2024-09-12T03:04:22.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.858+0000 7fcfc994c740 10 client.16776 fetch_fsmap learned FSMap version 33 2024-09-12T03:04:22.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.858+0000 7fcfc994c740 1 -- 172.21.15.5:0/842510245 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55928f97ccb0 con 0x55928f8f75d0 2024-09-12T03:04:22.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.858+0000 7fcfa8ff9640 1 -- 172.21.15.5:0/842510245 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfb4005500 con 0x55928f8f75d0 2024-09-12T03:04:22.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.859+0000 7fcfa8ff9640 1 -- 172.21.15.5:0/842510245 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 33) ==== 8374+0+0 (secure 0 0 0) 0x7fcfb8028040 con 0x55928f8f75d0 2024-09-12T03:04:22.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.859+0000 7fcfc994c740 10 client.16776 fetch_fsmap finished waiting for FSMap version 33 2024-09-12T03:04:22.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.859+0000 7fcfc994c740 10 client.16776 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-09-12T03:04:22.886 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.859+0000 7fcfc994c740 20 client.16776 populate_metadata read hostname 'smithi005' 2024-09-12T03:04:22.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.859+0000 7fcfc994c740 1 --2- 172.21.15.5:0/842510245 >> [v2:172.21.15.146:6836/1673370937,v1:172.21.15.146:6837/1673370937] conn(0x55928f97d320 0x55928f97f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:22.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.859+0000 7fcfc994c740 4 client.16776 mds_command: new command op to 24458 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:04:22.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.859+0000 7fcfc994c740 1 -- 172.21.15.5:0/842510245 --> [v2:172.21.15.146:6836/1673370937,v1:172.21.15.146:6837/1673370937] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55928f8fac20 con 0x55928f97d320 2024-09-12T03:04:22.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.859+0000 7fcfc4b0e640 1 --2- 172.21.15.5:0/842510245 >> [v2:172.21.15.146:6836/1673370937,v1:172.21.15.146:6837/1673370937] conn(0x55928f97d320 0x55928f97f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:22.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.859+0000 7fcfc4b0e640 1 --2- 172.21.15.5:0/842510245 >> [v2:172.21.15.146:6836/1673370937,v1:172.21.15.146:6837/1673370937] conn(0x55928f97d320 0x55928f97f760 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=77af7b1ee4bc36f9 server_cookie=d496fc26b6b2501 in_seq=0 out_seq=0 2024-09-12T03:04:22.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.859+0000 7fcfa8ff9640 10 client.16776 ms_handle_connect on v2:172.21.15.146:6836/1673370937 2024-09-12T03:04:22.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.860+0000 7fcfa8ff9640 1 -- 172.21.15.5:0/842510245 <== mds.4 v2:172.21.15.146:6836/1673370937 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55928f8fac20 con 0x55928f97d320 2024-09-12T03:04:22.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.860+0000 7fcfa8ff9640 10 client.16776 handle_command_reply: tid=0 2024-09-12T03:04:22.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.860+0000 7fcfc994c740 1 client.16776 shutdown 2024-09-12T03:04:22.887 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.860+0000 7fcfc994c740 1 -- 172.21.15.5:0/842510245 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55928f8f75d0 msgr2=0x55928f9063f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:22.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.860+0000 7fcfc994c740 1 --2- 172.21.15.5:0/842510245 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55928f8f75d0 0x55928f9063f0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fcfb801a840 tx=0x7fcfb80047d0 comp rx=0 tx=0).stop 2024-09-12T03:04:22.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.860+0000 7fcfc994c740 1 -- 172.21.15.5:0/842510245 shutdown_connections 2024-09-12T03:04:22.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.860+0000 7fcfc994c740 1 --2- 172.21.15.5:0/842510245 >> [v2:172.21.15.146:6836/1673370937,v1:172.21.15.146:6837/1673370937] conn(0x55928f97d320 0x55928f97f760 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:22.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.860+0000 7fcfc994c740 1 --2- 172.21.15.5:0/842510245 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55928f902370 0x55928f906e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:22.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.860+0000 7fcfa8ff9640 0 client.16776 ms_handle_reset on v2:172.21.15.146:6836/1673370937 2024-09-12T03:04:22.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.860+0000 7fcfc994c740 1 --2- 172.21.15.5:0/842510245 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55928f8f8b40 0x55928f906930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:22.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.860+0000 7fcfc994c740 1 --2- 172.21.15.5:0/842510245 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55928f8f75d0 0x55928f9063f0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:22.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.860+0000 7fcfc994c740 1 -- 172.21.15.5:0/842510245 >> 172.21.15.5:0/842510245 conn(0x55928f8c0310 msgr2=0x55928f8f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:22.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.860+0000 7fcfc994c740 1 -- 172.21.15.5:0/842510245 shutdown_connections 2024-09-12T03:04:22.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.860+0000 7fcfc994c740 1 -- 172.21.15.5:0/842510245 wait complete. 2024-09-12T03:04:22.888 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.860+0000 7fcfc994c740 20 client.16776 trim_cache size 0 max 16384 2024-09-12T03:04:22.889 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-12T03:04:22.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.861+0000 7fcfc7573640 1 -- 172.21.15.5:0/4267807340 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fcfac0a2b80 msgr2=0x7fcfac0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:22.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.861+0000 7fcfc7573640 1 --2- 172.21.15.5:0/4267807340 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fcfac0a2b80 0x7fcfac0a5040 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc00ab20 tx=0x7fcfbc005940 comp rx=0 tx=0).stop 2024-09-12T03:04:22.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.861+0000 7fcfc7573640 1 -- 172.21.15.5:0/4267807340 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfc0074510 msgr2=0x7fcfc01a0400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:22.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.861+0000 7fcfc7573640 1 --2- 172.21.15.5:0/4267807340 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfc0074510 0x7fcfc01a0400 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc00cce0 tx=0x7fcfbc00a620 comp rx=0 tx=0).stop 2024-09-12T03:04:22.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.861+0000 7fcfc7573640 1 -- 172.21.15.5:0/4267807340 shutdown_connections 2024-09-12T03:04:22.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.861+0000 7fcfc7573640 1 --2- 172.21.15.5:0/4267807340 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcfc010e440 0x7fcfc019a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:22.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.861+0000 7fcfc7573640 1 --2- 172.21.15.5:0/4267807340 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcfc0074e30 0x7fcfc01a0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:22.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.861+0000 7fcfc7573640 1 --2- 172.21.15.5:0/4267807340 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fcfac0a2b80 0x7fcfac0a5040 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:22.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.861+0000 7fcfc7573640 1 --2- 172.21.15.5:0/4267807340 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfc0074510 0x7fcfc01a0400 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:22.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.861+0000 7fcfc7573640 1 -- 172.21.15.5:0/4267807340 >> 172.21.15.5:0/4267807340 conn(0x7fcfc00fd8c0 msgr2=0x7fcfc00fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:22.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.861+0000 7fcfc7573640 1 -- 172.21.15.5:0/4267807340 shutdown_connections 2024-09-12T03:04:22.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:22.861+0000 7fcfc7573640 1 -- 172.21.15.5:0/4267807340 wait complete. 2024-09-12T03:04:23.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:04:22 smithi005 ceph-mon[31197]: pgmap v519: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-12T03:04:23.165 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:04:22 smithi148 ceph-mon[35090]: pgmap v519: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-12T03:04:23.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:04:22 smithi146 ceph-mon[35011]: pgmap v519: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-12T03:04:23.471 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-09-12T03:04:23.730 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:04:24.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.238+0000 7f4ca2586640 1 -- 172.21.15.5:0/1710421381 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c9c10e3c0 msgr2=0x7f4c9c110810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:24.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.238+0000 7f4ca2586640 1 --2- 172.21.15.5:0/1710421381 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c9c10e3c0 0x7f4c9c110810 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f4c94009ca0 tx=0x7f4c9402a950 comp rx=0 tx=0).stop 2024-09-12T03:04:24.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.239+0000 7f4ca2586640 1 -- 172.21.15.5:0/1710421381 shutdown_connections 2024-09-12T03:04:24.246 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.239+0000 7f4ca2586640 1 --2- 172.21.15.5:0/1710421381 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c9c10e3c0 0x7f4c9c110810 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:24.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.239+0000 7f4ca2586640 1 --2- 172.21.15.5:0/1710421381 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4c9c074f40 0x7f4c9c10de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:24.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.239+0000 7f4ca2586640 1 --2- 172.21.15.5:0/1710421381 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4c9c074620 0x7f4c9c074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:24.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.239+0000 7f4ca2586640 1 -- 172.21.15.5:0/1710421381 >> 172.21.15.5:0/1710421381 conn(0x7f4c9c0fd8c0 msgr2=0x7f4c9c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:24.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.239+0000 7f4ca2586640 1 -- 172.21.15.5:0/1710421381 shutdown_connections 2024-09-12T03:04:24.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.239+0000 7f4ca2586640 1 -- 172.21.15.5:0/1710421381 wait complete. 2024-09-12T03:04:24.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.239+0000 7f4ca2586640 1 Processor -- start 2024-09-12T03:04:24.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.239+0000 7f4ca2586640 1 -- start start 2024-09-12T03:04:24.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.240+0000 7f4ca2586640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c9c074620 0x7f4c9c1a0400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:24.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.240+0000 7f4ca2586640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4c9c074f40 0x7f4c9c1a0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:24.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.240+0000 7f4ca2586640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4c9c10e3c0 0x7f4c9c19a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:24.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.240+0000 7f4ca2586640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4c9c114230 con 0x7f4c9c074620 2024-09-12T03:04:24.247 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.240+0000 7f4ca2586640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f4c9c1140b0 con 0x7f4c9c10e3c0 2024-09-12T03:04:24.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.240+0000 7f4ca2586640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4c9c1143b0 con 0x7f4c9c074f40 2024-09-12T03:04:24.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.240+0000 7f4c9bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c9c074620 0x7f4c9c1a0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:24.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.240+0000 7f4c9bfff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c9c074620 0x7f4c9c1a0400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53304/0 (socket says 172.21.15.5:53304) 2024-09-12T03:04:24.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.240+0000 7f4c9bfff640 1 -- 172.21.15.5:0/361997121 learned_addr learned my addr 172.21.15.5:0/361997121 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:04:24.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.240+0000 7f4c9b7fe640 1 --2- 172.21.15.5:0/361997121 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4c9c074f40 0x7f4c9c1a0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:24.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.240+0000 7f4ca0b23640 1 --2- 172.21.15.5:0/361997121 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4c9c10e3c0 0x7f4c9c19a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:24.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.240+0000 7f4c9bfff640 1 -- 172.21.15.5:0/361997121 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4c9c074f40 msgr2=0x7f4c9c1a0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:24.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.240+0000 7f4c9bfff640 1 --2- 172.21.15.5:0/361997121 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4c9c074f40 0x7f4c9c1a0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:24.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.240+0000 7f4c9bfff640 1 -- 172.21.15.5:0/361997121 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4c9c10e3c0 msgr2=0x7f4c9c19a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:24.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.240+0000 7f4c9bfff640 1 --2- 172.21.15.5:0/361997121 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4c9c10e3c0 0x7f4c9c19a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:24.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.240+0000 7f4c9bfff640 1 -- 172.21.15.5:0/361997121 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c9c19acf0 con 0x7f4c9c074620 2024-09-12T03:04:24.248 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.240+0000 7f4ca0b23640 1 --2- 172.21.15.5:0/361997121 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4c9c10e3c0 0x7f4c9c19a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:04:24.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.240+0000 7f4c9bfff640 1 --2- 172.21.15.5:0/361997121 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c9c074620 0x7f4c9c1a0400 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c00b4a0 tx=0x7f4c8c00b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:24.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.241+0000 7f4c997fa640 1 -- 172.21.15.5:0/361997121 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c8c015020 con 0x7f4c9c074620 2024-09-12T03:04:24.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.241+0000 7f4c997fa640 1 -- 172.21.15.5:0/361997121 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c8c011d10 con 0x7f4c9c074620 2024-09-12T03:04:24.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.241+0000 7f4ca2586640 1 -- 172.21.15.5:0/361997121 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c9c19afe0 con 0x7f4c9c074620 2024-09-12T03:04:24.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.241+0000 7f4c997fa640 1 -- 172.21.15.5:0/361997121 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c8c00f2e0 con 0x7f4c9c074620 2024-09-12T03:04:24.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.241+0000 7f4ca2586640 1 -- 172.21.15.5:0/361997121 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c9c1a72d0 con 0x7f4c9c074620 2024-09-12T03:04:24.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.243+0000 7f4c997fa640 1 -- 172.21.15.5:0/361997121 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f4c8c018020 con 0x7f4c9c074620 2024-09-12T03:04:24.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.243+0000 7f4ca495f740 1 Processor -- start 2024-09-12T03:04:24.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.243+0000 7f4ca495f740 1 -- start start 2024-09-12T03:04:24.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.244+0000 7f4ca495f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55865e872b40 0x55865e871090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:24.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.244+0000 7f4ca495f740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55865e8715d0 0x55865e871a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:24.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.244+0000 7f4ca495f740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55865e87c370 0x55865e87e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:24.249 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.244+0000 7f4ca495f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55865e7d72f0 con 0x55865e872b40 2024-09-12T03:04:24.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.244+0000 7f4ca495f740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55865e77ecf0 con 0x55865e87c370 2024-09-12T03:04:24.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.244+0000 7f4ca495f740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55865e7d32b0 con 0x55865e8715d0 2024-09-12T03:04:24.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.244+0000 7f4ca0b23640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55865e872b40 0x55865e871090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:24.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.244+0000 7f4ca0b23640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55865e872b40 0x55865e871090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53318/0 (socket says 172.21.15.5:53318) 2024-09-12T03:04:24.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.244+0000 7f4ca0b23640 1 -- 172.21.15.5:0/223367743 learned_addr learned my addr 172.21.15.5:0/223367743 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:04:24.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.244+0000 7f4c997fa640 1 --2- 172.21.15.5:0/361997121 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f4c840a2b80 0x7f4c840a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:24.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.244+0000 7f4c9bfff640 1 --2- 172.21.15.5:0/223367743 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55865e8715d0 0x55865e871a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:24.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.244+0000 7f4c9b7fe640 1 --2- 172.21.15.5:0/223367743 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55865e87c370 0x55865e87e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:24.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.244+0000 7f4ca0b23640 1 --2- 172.21.15.5:0/361997121 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f4c840a2b80 0x7f4c840a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:24.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.244+0000 7f4ca0b23640 1 -- 172.21.15.5:0/223367743 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55865e8715d0 msgr2=0x55865e871a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:24.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.244+0000 7f4ca0b23640 1 --2- 172.21.15.5:0/223367743 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55865e8715d0 0x55865e871a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:24.250 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.244+0000 7f4ca0b23640 1 -- 172.21.15.5:0/223367743 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55865e87c370 msgr2=0x55865e87e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:24.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.244+0000 7f4ca0b23640 1 --2- 172.21.15.5:0/223367743 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55865e87c370 0x55865e87e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:24.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.244+0000 7f4ca0b23640 1 -- 172.21.15.5:0/223367743 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55865e7633a0 con 0x55865e872b40 2024-09-12T03:04:24.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.244+0000 7f4c997fa640 1 -- 172.21.15.5:0/361997121 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f4c8c0ce940 con 0x7f4c9c074620 2024-09-12T03:04:24.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.244+0000 7f4c9bfff640 1 --2- 172.21.15.5:0/223367743 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55865e8715d0 0x55865e871a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:04:24.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.245+0000 7f4ca0b23640 1 --2- 172.21.15.5:0/223367743 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55865e872b40 0x55865e871090 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f4c9402fd90 tx=0x7f4c9402f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=371ec837af5b4407 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:24.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.245+0000 7f4c827fc640 1 -- 172.21.15.5:0/223367743 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c94034040 con 0x55865e872b40 2024-09-12T03:04:24.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.245+0000 7f4c827fc640 1 -- 172.21.15.5:0/223367743 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c9404e070 con 0x55865e872b40 2024-09-12T03:04:24.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.245+0000 7f4c827fc640 1 -- 172.21.15.5:0/223367743 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c94049830 con 0x55865e872b40 2024-09-12T03:04:24.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.245+0000 7f4ca495f740 1 -- 172.21.15.5:0/223367743 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55865e872b40 msgr2=0x55865e871090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:24.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.246+0000 7f4ca0b23640 1 --2- 172.21.15.5:0/361997121 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f4c840a2b80 0x7f4c840a5040 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f4c9404d000 tx=0x7f4c9404b000 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:24.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.246+0000 7f4ca495f740 1 --2- 172.21.15.5:0/223367743 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55865e872b40 0x55865e871090 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f4c9402fd90 tx=0x7f4c9402f4a0 comp rx=0 tx=0).stop 2024-09-12T03:04:24.251 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.246+0000 7f4ca495f740 1 -- 172.21.15.5:0/223367743 shutdown_connections 2024-09-12T03:04:24.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.246+0000 7f4ca495f740 1 --2- 172.21.15.5:0/223367743 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55865e87c370 0x55865e87e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:24.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.246+0000 7f4ca495f740 1 --2- 172.21.15.5:0/223367743 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55865e8715d0 0x55865e871a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:24.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.246+0000 7f4ca495f740 1 --2- 172.21.15.5:0/223367743 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55865e872b40 0x55865e871090 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:24.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.246+0000 7f4ca495f740 1 -- 172.21.15.5:0/223367743 >> 172.21.15.5:0/223367743 conn(0x55865e83a310 msgr2=0x55865e86dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:24.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.246+0000 7f4ca495f740 1 -- 172.21.15.5:0/223367743 shutdown_connections 2024-09-12T03:04:24.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.246+0000 7f4ca495f740 1 -- 172.21.15.5:0/223367743 wait complete. 2024-09-12T03:04:24.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.246+0000 7f4ca495f740 1 -- start start 2024-09-12T03:04:24.252 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.246+0000 7f4ca495f740 1 Processor -- start 2024-09-12T03:04:24.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.247+0000 7f4ca495f740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55865e8715d0 0x55865e87bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:24.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.247+0000 7f4ca495f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55865e872b40 0x55865e879020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:24.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.247+0000 7f4ca495f740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55865e87c370 0x55865e879610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:24.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.247+0000 7f4ca495f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55865e87f070 con 0x55865e872b40 2024-09-12T03:04:24.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.247+0000 7f4ca495f740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55865e792060 con 0x55865e8715d0 2024-09-12T03:04:24.255 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.247+0000 7f4ca495f740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55865e87f1f0 con 0x55865e87c370 2024-09-12T03:04:24.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.247+0000 7f4ca495f740 10 client.0 _ll_register_callbacks cb 0x55865e4e58c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:04:24.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.247+0000 7f4c9b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55865e872b40 0x55865e879020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:24.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.247+0000 7f4c9b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55865e872b40 0x55865e879020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53326/0 (socket says 172.21.15.5:53326) 2024-09-12T03:04:24.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.247+0000 7f4c9b7fe640 1 -- 172.21.15.5:0/1801777155 learned_addr learned my addr 172.21.15.5:0/1801777155 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:04:24.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.247+0000 7f4c9bfff640 1 --2- 172.21.15.5:0/1801777155 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55865e8715d0 0x55865e87bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:24.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.247+0000 7f4ca0b23640 1 --2- 172.21.15.5:0/1801777155 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55865e87c370 0x55865e879610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:24.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.247+0000 7f4c9b7fe640 1 -- 172.21.15.5:0/1801777155 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55865e87c370 msgr2=0x55865e879610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:24.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.247+0000 7f4c9b7fe640 1 --2- 172.21.15.5:0/1801777155 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55865e87c370 0x55865e879610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:24.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.247+0000 7f4c9b7fe640 1 -- 172.21.15.5:0/1801777155 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55865e8715d0 msgr2=0x55865e87bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:24.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.247+0000 7f4c9b7fe640 1 --2- 172.21.15.5:0/1801777155 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55865e8715d0 0x55865e87bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:24.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.247+0000 7f4c9b7fe640 1 -- 172.21.15.5:0/1801777155 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55865e5483c0 con 0x55865e872b40 2024-09-12T03:04:24.256 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.247+0000 7f4ca0b23640 1 --2- 172.21.15.5:0/1801777155 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55865e87c370 0x55865e879610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:04:24.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.247+0000 7f4c9bfff640 1 --2- 172.21.15.5:0/1801777155 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55865e8715d0 0x55865e87bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:04:24.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.247+0000 7f4c9b7fe640 1 --2- 172.21.15.5:0/1801777155 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55865e872b40 0x55865e879020 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f4c90011570 tx=0x7f4c90011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65646bd73237e165 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:24.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.247+0000 7f4c7b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T03:04:24.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.247+0000 7f4c7b7fe640 1 -- 172.21.15.5:0/1801777155 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c90013670 con 0x55865e872b40 2024-09-12T03:04:24.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.247+0000 7f4c7b7fe640 1 -- 172.21.15.5:0/1801777155 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f4c90013810 con 0x55865e872b40 2024-09-12T03:04:24.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.248+0000 7f4ca495f740 1 -- 172.21.15.5:0/1801777155 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c9404e070 con 0x55865e872b40 2024-09-12T03:04:24.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.248+0000 7f4c7b7fe640 1 -- 172.21.15.5:0/1801777155 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f4c90013a50 con 0x55865e872b40 2024-09-12T03:04:24.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.248+0000 7f4ca495f740 10 client.16788 fetch_fsmap learned FSMap version 33 2024-09-12T03:04:24.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.248+0000 7f4ca495f740 1 -- 172.21.15.5:0/1801777155 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55865e88ab70 con 0x55865e872b40 2024-09-12T03:04:24.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.248+0000 7f4c7b7fe640 1 -- 172.21.15.5:0/1801777155 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c9404e070 con 0x55865e872b40 2024-09-12T03:04:24.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.248+0000 7f4c7b7fe640 1 -- 172.21.15.5:0/1801777155 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 33) ==== 8374+0+0 (secure 0 0 0) 0x7f4c9001d400 con 0x55865e872b40 2024-09-12T03:04:24.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.248+0000 7f4ca495f740 10 client.16788 fetch_fsmap finished waiting for FSMap version 33 2024-09-12T03:04:24.257 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.248+0000 7f4ca495f740 10 client.16788 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.b 2024-09-12T03:04:24.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.248+0000 7f4ca495f740 20 client.16788 populate_metadata read hostname 'smithi005' 2024-09-12T03:04:24.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.248+0000 7f4ca495f740 1 --2- 172.21.15.5:0/1801777155 >> [v2:172.21.15.146:6838/3036767228,v1:172.21.15.146:6839/3036767228] conn(0x55865e88c080 0x55865e88e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:24.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.248+0000 7f4ca495f740 4 client.16788 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:04:24.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.248+0000 7f4ca495f740 1 -- 172.21.15.5:0/1801777155 --> [v2:172.21.15.146:6838/3036767228,v1:172.21.15.146:6839/3036767228] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55865e7e4860 con 0x55865e88c080 2024-09-12T03:04:24.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.249+0000 7f4c9bfff640 1 --2- 172.21.15.5:0/1801777155 >> [v2:172.21.15.146:6838/3036767228,v1:172.21.15.146:6839/3036767228] conn(0x55865e88c080 0x55865e88e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:24.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.249+0000 7f4c7b7fe640 10 client.16788 ms_handle_connect on v2:172.21.15.146:6838/3036767228 2024-09-12T03:04:24.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.249+0000 7f4c9bfff640 1 --2- 172.21.15.5:0/1801777155 >> [v2:172.21.15.146:6838/3036767228,v1:172.21.15.146:6839/3036767228] conn(0x55865e88c080 0x55865e88e4b0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e2559b593b412d06 server_cookie=e1750ae5906c503d in_seq=0 out_seq=0 2024-09-12T03:04:24.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.250+0000 7f4c7b7fe640 1 -- 172.21.15.5:0/1801777155 <== mds.3 v2:172.21.15.146:6838/3036767228 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55865e7e4860 con 0x55865e88c080 2024-09-12T03:04:24.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.250+0000 7f4c7b7fe640 10 client.16788 handle_command_reply: tid=0 2024-09-12T03:04:24.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.250+0000 7f4ca495f740 1 client.16788 shutdown 2024-09-12T03:04:24.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.250+0000 7f4ca495f740 1 -- 172.21.15.5:0/1801777155 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55865e872b40 msgr2=0x55865e879020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:24.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.250+0000 7f4ca495f740 1 --2- 172.21.15.5:0/1801777155 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55865e872b40 0x55865e879020 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f4c90011570 tx=0x7f4c90011a30 comp rx=0 tx=0).stop 2024-09-12T03:04:24.258 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.250+0000 7f4ca495f740 1 -- 172.21.15.5:0/1801777155 shutdown_connections 2024-09-12T03:04:24.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.250+0000 7f4ca495f740 1 --2- 172.21.15.5:0/1801777155 >> [v2:172.21.15.146:6838/3036767228,v1:172.21.15.146:6839/3036767228] conn(0x55865e88c080 0x55865e88e4b0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:24.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.250+0000 7f4ca495f740 1 --2- 172.21.15.5:0/1801777155 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55865e872b40 0x55865e879020 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:24.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.250+0000 7f4ca495f740 1 --2- 172.21.15.5:0/1801777155 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55865e87c370 0x55865e879610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:24.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.250+0000 7f4ca495f740 1 --2- 172.21.15.5:0/1801777155 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55865e8715d0 0x55865e87bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:24.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.250+0000 7f4c7b7fe640 0 client.16788 ms_handle_reset on v2:172.21.15.146:6838/3036767228 2024-09-12T03:04:24.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.250+0000 7f4ca495f740 1 -- 172.21.15.5:0/1801777155 >> 172.21.15.5:0/1801777155 conn(0x55865e83a310 msgr2=0x55865e873960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:24.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.250+0000 7f4ca495f740 1 -- 172.21.15.5:0/1801777155 shutdown_connections 2024-09-12T03:04:24.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.250+0000 7f4ca495f740 1 -- 172.21.15.5:0/1801777155 wait complete. 2024-09-12T03:04:24.259 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.250+0000 7f4ca495f740 20 client.16788 trim_cache size 0 max 16384 2024-09-12T03:04:24.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.263+0000 7f4ca495f740 1 Processor -- start 2024-09-12T03:04:24.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.263+0000 7f4ca495f740 1 -- start start 2024-09-12T03:04:24.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.263+0000 7f4ca495f740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55865e8715d0 0x55865e8788e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:24.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.263+0000 7f4ca495f740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55865e872b40 0x55865e888310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:24.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.263+0000 7f4ca495f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55865e87c370 0x55865e893dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:24.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.263+0000 7f4ca495f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4c9c1af930 con 0x55865e87c370 2024-09-12T03:04:24.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.264+0000 7f4ca495f740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f4c9c1b1500 con 0x55865e872b40 2024-09-12T03:04:24.264 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.264+0000 7f4ca495f740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4c9c1b0030 con 0x55865e8715d0 2024-09-12T03:04:24.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.264+0000 7f4ca0b23640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55865e87c370 0x55865e893dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:24.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.264+0000 7f4ca0b23640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55865e87c370 0x55865e893dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53338/0 (socket says 172.21.15.5:53338) 2024-09-12T03:04:24.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.264+0000 7f4ca0b23640 1 -- 172.21.15.5:0/225709986 learned_addr learned my addr 172.21.15.5:0/225709986 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:04:24.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.264+0000 7f4c9b7fe640 1 --2- 172.21.15.5:0/225709986 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55865e872b40 0x55865e888310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:24.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.264+0000 7f4c9bfff640 1 --2- 172.21.15.5:0/225709986 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55865e8715d0 0x55865e8788e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:24.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.264+0000 7f4ca0b23640 1 -- 172.21.15.5:0/225709986 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55865e8715d0 msgr2=0x55865e8788e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:24.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.264+0000 7f4ca0b23640 1 --2- 172.21.15.5:0/225709986 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55865e8715d0 0x55865e8788e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:24.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.264+0000 7f4ca0b23640 1 -- 172.21.15.5:0/225709986 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55865e872b40 msgr2=0x55865e888310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:24.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.264+0000 7f4ca0b23640 1 --2- 172.21.15.5:0/225709986 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55865e872b40 0x55865e888310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:24.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.264+0000 7f4ca0b23640 1 -- 172.21.15.5:0/225709986 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55865e888850 con 0x55865e87c370 2024-09-12T03:04:24.265 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.264+0000 7f4c9bfff640 1 --2- 172.21.15.5:0/225709986 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55865e8715d0 0x55865e8788e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:04:24.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.264+0000 7f4ca0b23640 1 --2- 172.21.15.5:0/225709986 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55865e87c370 0x55865e893dc0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f4c94009270 tx=0x7f4c9402f8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ff4a5f8ada83079 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:24.268 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.264+0000 7f4c81ffb640 1 -- 172.21.15.5:0/225709986 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c94059020 con 0x55865e87c370 2024-09-12T03:04:24.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.264+0000 7f4c81ffb640 1 -- 172.21.15.5:0/225709986 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c9404e070 con 0x55865e87c370 2024-09-12T03:04:24.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.264+0000 7f4c81ffb640 1 -- 172.21.15.5:0/225709986 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c94004280 con 0x55865e87c370 2024-09-12T03:04:24.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.265+0000 7f4ca495f740 1 -- 172.21.15.5:0/225709986 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55865e87c370 msgr2=0x55865e893dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:24.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.265+0000 7f4ca495f740 1 --2- 172.21.15.5:0/225709986 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55865e87c370 0x55865e893dc0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f4c94009270 tx=0x7f4c9402f8a0 comp rx=0 tx=0).stop 2024-09-12T03:04:24.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.265+0000 7f4ca495f740 1 -- 172.21.15.5:0/225709986 shutdown_connections 2024-09-12T03:04:24.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.265+0000 7f4ca495f740 1 --2- 172.21.15.5:0/225709986 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55865e87c370 0x55865e893dc0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:24.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.265+0000 7f4ca495f740 1 --2- 172.21.15.5:0/225709986 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55865e872b40 0x55865e888310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:24.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.265+0000 7f4ca495f740 1 --2- 172.21.15.5:0/225709986 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55865e8715d0 0x55865e8788e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:24.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.265+0000 7f4ca495f740 1 -- 172.21.15.5:0/225709986 >> 172.21.15.5:0/225709986 conn(0x55865e83a310 msgr2=0x55865e8778f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:24.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.265+0000 7f4ca495f740 1 -- 172.21.15.5:0/225709986 shutdown_connections 2024-09-12T03:04:24.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.265+0000 7f4ca495f740 1 -- 172.21.15.5:0/225709986 wait complete. 2024-09-12T03:04:24.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.265+0000 7f4ca495f740 1 -- start start 2024-09-12T03:04:24.269 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.266+0000 7f4ca495f740 1 Processor -- start 2024-09-12T03:04:24.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.266+0000 7f4ca495f740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55865e8715d0 0x55865e8801f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:24.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.266+0000 7f4ca495f740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55865e872b40 0x55865e880730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:24.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.266+0000 7f4ca495f740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55865e87c370 0x55865e880c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:24.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.266+0000 7f4ca495f740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4c9c1b0e30 con 0x55865e872b40 2024-09-12T03:04:24.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.266+0000 7f4ca495f740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f4c9c1b0730 con 0x55865e8715d0 2024-09-12T03:04:24.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.266+0000 7f4ca495f740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4c9c1ad910 con 0x55865e87c370 2024-09-12T03:04:24.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.266+0000 7f4ca495f740 10 client.0 _ll_register_callbacks cb 0x7f4c90009510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:04:24.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.266+0000 7f4c9b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55865e872b40 0x55865e880730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:24.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.266+0000 7f4c9b7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55865e872b40 0x55865e880730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53342/0 (socket says 172.21.15.5:53342) 2024-09-12T03:04:24.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.266+0000 7f4c9b7fe640 1 -- 172.21.15.5:0/2395930182 learned_addr learned my addr 172.21.15.5:0/2395930182 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:04:24.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.266+0000 7f4c9bfff640 1 --2- 172.21.15.5:0/2395930182 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55865e8715d0 0x55865e8801f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:24.270 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.266+0000 7f4ca0b23640 1 --2- 172.21.15.5:0/2395930182 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55865e87c370 0x55865e880c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:24.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.266+0000 7f4c9b7fe640 1 -- 172.21.15.5:0/2395930182 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55865e87c370 msgr2=0x55865e880c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:24.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.266+0000 7f4c9b7fe640 1 --2- 172.21.15.5:0/2395930182 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55865e87c370 0x55865e880c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:24.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.266+0000 7f4c9b7fe640 1 -- 172.21.15.5:0/2395930182 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55865e8715d0 msgr2=0x55865e8801f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:24.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.266+0000 7f4c9b7fe640 1 --2- 172.21.15.5:0/2395930182 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55865e8715d0 0x55865e8801f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:24.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.266+0000 7f4c9b7fe640 1 -- 172.21.15.5:0/2395930182 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55865e8f5dc0 con 0x55865e872b40 2024-09-12T03:04:24.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.267+0000 7f4c9b7fe640 1 --2- 172.21.15.5:0/2395930182 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55865e872b40 0x55865e880730 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f4c9001d8a0 tx=0x7f4c90006e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee69c2f0895731b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:24.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.267+0000 7f4c7b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T03:04:24.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.267+0000 7f4c7b7fe640 1 -- 172.21.15.5:0/2395930182 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c90013df0 con 0x55865e872b40 2024-09-12T03:04:24.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.267+0000 7f4c7b7fe640 1 -- 172.21.15.5:0/2395930182 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f4c9001da90 con 0x55865e872b40 2024-09-12T03:04:24.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.267+0000 7f4ca495f740 1 -- 172.21.15.5:0/2395930182 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c9404e070 con 0x55865e872b40 2024-09-12T03:04:24.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.267+0000 7f4c7b7fe640 1 -- 172.21.15.5:0/2395930182 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f4c90038050 con 0x55865e872b40 2024-09-12T03:04:24.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.267+0000 7f4ca495f740 10 client.16794 fetch_fsmap learned FSMap version 33 2024-09-12T03:04:24.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.267+0000 7f4ca495f740 1 -- 172.21.15.5:0/2395930182 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55865e8f70b0 con 0x55865e872b40 2024-09-12T03:04:24.271 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.267+0000 7f4c7b7fe640 1 -- 172.21.15.5:0/2395930182 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c9404e070 con 0x55865e872b40 2024-09-12T03:04:24.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.267+0000 7f4c7b7fe640 1 -- 172.21.15.5:0/2395930182 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 33) ==== 8374+0+0 (secure 0 0 0) 0x7f4c90031b60 con 0x55865e872b40 2024-09-12T03:04:24.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.267+0000 7f4ca495f740 10 client.16794 fetch_fsmap finished waiting for FSMap version 33 2024-09-12T03:04:24.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.267+0000 7f4ca495f740 10 client.16794 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.b 2024-09-12T03:04:24.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.267+0000 7f4ca495f740 20 client.16794 populate_metadata read hostname 'smithi005' 2024-09-12T03:04:24.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.267+0000 7f4ca495f740 1 --2- 172.21.15.5:0/2395930182 >> [v2:172.21.15.146:6838/3036767228,v1:172.21.15.146:6839/3036767228] conn(0x55865e8f7860 0x55865e8f9c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:24.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.267+0000 7f4ca495f740 4 client.16794 mds_command: new command op to 24479 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:04:24.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.267+0000 7f4ca495f740 1 -- 172.21.15.5:0/2395930182 --> [v2:172.21.15.146:6838/3036767228,v1:172.21.15.146:6839/3036767228] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55865e7ded70 con 0x55865e8f7860 2024-09-12T03:04:24.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.268+0000 7f4c9bfff640 1 --2- 172.21.15.5:0/2395930182 >> [v2:172.21.15.146:6838/3036767228,v1:172.21.15.146:6839/3036767228] conn(0x55865e8f7860 0x55865e8f9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:24.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.268+0000 7f4c9bfff640 1 --2- 172.21.15.5:0/2395930182 >> [v2:172.21.15.146:6838/3036767228,v1:172.21.15.146:6839/3036767228] conn(0x55865e8f7860 0x55865e8f9c80 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8a86ada4f5cbfa55 server_cookie=d7e47326cef9ac37 in_seq=0 out_seq=0 2024-09-12T03:04:24.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.268+0000 7f4c7b7fe640 10 client.16794 ms_handle_connect on v2:172.21.15.146:6838/3036767228 2024-09-12T03:04:24.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.268+0000 7f4c7b7fe640 1 -- 172.21.15.5:0/2395930182 <== mds.3 v2:172.21.15.146:6838/3036767228 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55865e7ded70 con 0x55865e8f7860 2024-09-12T03:04:24.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.269+0000 7f4c7b7fe640 10 client.16794 handle_command_reply: tid=0 2024-09-12T03:04:24.272 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.269+0000 7f4ca495f740 1 client.16794 shutdown 2024-09-12T03:04:24.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.269+0000 7f4ca495f740 1 -- 172.21.15.5:0/2395930182 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55865e872b40 msgr2=0x55865e880730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:24.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.269+0000 7f4ca495f740 1 --2- 172.21.15.5:0/2395930182 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55865e872b40 0x55865e880730 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f4c9001d8a0 tx=0x7f4c90006e50 comp rx=0 tx=0).stop 2024-09-12T03:04:24.273 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.269+0000 7f4ca495f740 1 -- 172.21.15.5:0/2395930182 shutdown_connections 2024-09-12T03:04:24.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.269+0000 7f4ca495f740 1 --2- 172.21.15.5:0/2395930182 >> [v2:172.21.15.146:6838/3036767228,v1:172.21.15.146:6839/3036767228] conn(0x55865e8f7860 0x55865e8f9c80 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:24.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.269+0000 7f4ca495f740 1 --2- 172.21.15.5:0/2395930182 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55865e872b40 0x55865e880730 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:24.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.269+0000 7f4ca495f740 1 --2- 172.21.15.5:0/2395930182 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55865e87c370 0x55865e880c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:24.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.269+0000 7f4ca495f740 1 --2- 172.21.15.5:0/2395930182 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55865e8715d0 0x55865e8801f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:24.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.269+0000 7f4ca495f740 1 -- 172.21.15.5:0/2395930182 >> 172.21.15.5:0/2395930182 conn(0x55865e83a310 msgr2=0x55865e87a880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:24.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.269+0000 7f4c7b7fe640 0 client.16794 ms_handle_reset on v2:172.21.15.146:6838/3036767228 2024-09-12T03:04:24.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.269+0000 7f4ca495f740 1 -- 172.21.15.5:0/2395930182 shutdown_connections 2024-09-12T03:04:24.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.269+0000 7f4ca495f740 1 -- 172.21.15.5:0/2395930182 wait complete. 2024-09-12T03:04:24.274 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.269+0000 7f4ca495f740 20 client.16794 trim_cache size 0 max 16384 2024-09-12T03:04:24.274 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-12T03:04:24.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.269+0000 7f4ca2586640 1 -- 172.21.15.5:0/361997121 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f4c840a2b80 msgr2=0x7f4c840a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:24.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.269+0000 7f4ca2586640 1 --2- 172.21.15.5:0/361997121 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f4c840a2b80 0x7f4c840a5040 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f4c9404d000 tx=0x7f4c9404b000 comp rx=0 tx=0).stop 2024-09-12T03:04:24.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.269+0000 7f4ca2586640 1 -- 172.21.15.5:0/361997121 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c9c074620 msgr2=0x7f4c9c1a0400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:24.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.269+0000 7f4ca2586640 1 --2- 172.21.15.5:0/361997121 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c9c074620 0x7f4c9c1a0400 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c00b4a0 tx=0x7f4c8c00b8a0 comp rx=0 tx=0).stop 2024-09-12T03:04:24.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.270+0000 7f4ca2586640 1 -- 172.21.15.5:0/361997121 shutdown_connections 2024-09-12T03:04:24.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.270+0000 7f4ca2586640 1 --2- 172.21.15.5:0/361997121 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4c9c10e3c0 0x7f4c9c19a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:24.275 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.270+0000 7f4ca2586640 1 --2- 172.21.15.5:0/361997121 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4c9c074f40 0x7f4c9c1a0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:24.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.270+0000 7f4ca2586640 1 --2- 172.21.15.5:0/361997121 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f4c840a2b80 0x7f4c840a5040 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:24.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.270+0000 7f4ca2586640 1 --2- 172.21.15.5:0/361997121 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c9c074620 0x7f4c9c1a0400 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:24.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.270+0000 7f4ca2586640 1 -- 172.21.15.5:0/361997121 >> 172.21.15.5:0/361997121 conn(0x7f4c9c0fd8c0 msgr2=0x7f4c9c0ff0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:24.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.270+0000 7f4ca2586640 1 -- 172.21.15.5:0/361997121 shutdown_connections 2024-09-12T03:04:24.276 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:24.270+0000 7f4ca2586640 1 -- 172.21.15.5:0/361997121 wait complete. 2024-09-12T03:04:24.979 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-12T03:04:25.165 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:04:24 smithi148 ceph-mon[35090]: pgmap v520: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-12T03:04:25.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:04:24 smithi146 ceph-mon[35011]: pgmap v520: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-12T03:04:25.238 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:04:25.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:04:24 smithi005 ceph-mon[31197]: pgmap v520: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 767 B/s wr, 11 op/s 2024-09-12T03:04:25.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.731+0000 7fcab5802640 1 -- 172.21.15.5:0/299639761 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcab01068f0 msgr2=0x7fcab0106cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:25.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.731+0000 7fcab5802640 1 --2- 172.21.15.5:0/299639761 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcab01068f0 0x7fcab0106cd0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fcaa0009930 tx=0x7fcaa002a420 comp rx=0 tx=0).stop 2024-09-12T03:04:25.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.731+0000 7fcab5802640 1 -- 172.21.15.5:0/299639761 shutdown_connections 2024-09-12T03:04:25.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.731+0000 7fcab5802640 1 --2- 172.21.15.5:0/299639761 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcab0101290 0x7fcab010d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:25.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.731+0000 7fcab5802640 1 --2- 172.21.15.5:0/299639761 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcab01008f0 0x7fcab0100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:25.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.731+0000 7fcab5802640 1 --2- 172.21.15.5:0/299639761 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcab01068f0 0x7fcab0106cd0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:25.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.731+0000 7fcab5802640 1 -- 172.21.15.5:0/299639761 >> 172.21.15.5:0/299639761 conn(0x7fcab00fc630 msgr2=0x7fcab00fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:25.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.731+0000 7fcab5802640 1 -- 172.21.15.5:0/299639761 shutdown_connections 2024-09-12T03:04:25.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.731+0000 7fcab5802640 1 -- 172.21.15.5:0/299639761 wait complete. 2024-09-12T03:04:25.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.732+0000 7fcab5802640 1 Processor -- start 2024-09-12T03:04:25.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.732+0000 7fcab5802640 1 -- start start 2024-09-12T03:04:25.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.733+0000 7fcab5802640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcab01008f0 0x7fcab0198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:25.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.733+0000 7fcab5802640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcab0101290 0x7fcab0198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:25.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.733+0000 7fcab5802640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcab01068f0 0x7fcab019cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:25.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.733+0000 7fcab5802640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcab0110d40 con 0x7fcab01008f0 2024-09-12T03:04:25.772 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.733+0000 7fcab5802640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fcab0110bc0 con 0x7fcab01068f0 2024-09-12T03:04:25.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.733+0000 7fcab5802640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fcab0110ec0 con 0x7fcab0101290 2024-09-12T03:04:25.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.733+0000 7fcaaeffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcab01008f0 0x7fcab0198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:25.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.733+0000 7fcaaeffd640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcab01008f0 0x7fcab0198320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53370/0 (socket says 172.21.15.5:53370) 2024-09-12T03:04:25.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.733+0000 7fcaaeffd640 1 -- 172.21.15.5:0/708623574 learned_addr learned my addr 172.21.15.5:0/708623574 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:04:25.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.733+0000 7fcaaf7fe640 1 --2- 172.21.15.5:0/708623574 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcab01068f0 0x7fcab019cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:25.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.733+0000 7fcaae7fc640 1 --2- 172.21.15.5:0/708623574 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcab0101290 0x7fcab0198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:25.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.733+0000 7fcaaeffd640 1 -- 172.21.15.5:0/708623574 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcab0101290 msgr2=0x7fcab0198860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:25.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.733+0000 7fcaaeffd640 1 --2- 172.21.15.5:0/708623574 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcab0101290 0x7fcab0198860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:25.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.733+0000 7fcaaeffd640 1 -- 172.21.15.5:0/708623574 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcab01068f0 msgr2=0x7fcab019cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:25.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.733+0000 7fcaaeffd640 1 --2- 172.21.15.5:0/708623574 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcab01068f0 0x7fcab019cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:25.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.733+0000 7fcaaeffd640 1 -- 172.21.15.5:0/708623574 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcab019d2d0 con 0x7fcab01008f0 2024-09-12T03:04:25.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.734+0000 7fcaaeffd640 1 --2- 172.21.15.5:0/708623574 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcab01008f0 0x7fcab0198320 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fcaa002a940 tx=0x7fcaa0003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:25.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.734+0000 7fcab4800640 1 -- 172.21.15.5:0/708623574 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa003f070 con 0x7fcab01008f0 2024-09-12T03:04:25.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.734+0000 7fcab5802640 1 -- 172.21.15.5:0/708623574 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcab019d560 con 0x7fcab01008f0 2024-09-12T03:04:25.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.734+0000 7fcab4800640 1 -- 172.21.15.5:0/708623574 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcaa0031c90 con 0x7fcab01008f0 2024-09-12T03:04:25.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.734+0000 7fcab4800640 1 -- 172.21.15.5:0/708623574 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa0035030 con 0x7fcab01008f0 2024-09-12T03:04:25.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.734+0000 7fcab5802640 1 -- 172.21.15.5:0/708623574 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcab01a4ea0 con 0x7fcab01008f0 2024-09-12T03:04:25.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.737+0000 7fcab7bdb740 1 Processor -- start 2024-09-12T03:04:25.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.737+0000 7fcab7bdb740 1 -- start start 2024-09-12T03:04:25.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.737+0000 7fcab4800640 1 -- 172.21.15.5:0/708623574 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fcaa0042070 con 0x7fcab01008f0 2024-09-12T03:04:25.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.737+0000 7fcab7bdb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5cc4b3d20 0x55c5cc4b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:25.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.737+0000 7fcab7bdb740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c5cc4b27b0 0x55c5cc4b2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:25.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.737+0000 7fcab7bdb740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c5cc4c1960 0x55c5cc4c3d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:25.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.737+0000 7fcab7bdb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55c5cc4182f0 con 0x55c5cc4b3d20 2024-09-12T03:04:25.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.737+0000 7fcab7bdb740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55c5cc3bfcf0 con 0x55c5cc4b27b0 2024-09-12T03:04:25.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.737+0000 7fcab7bdb740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55c5cc4142b0 con 0x55c5cc4c1960 2024-09-12T03:04:25.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.737+0000 7fcaaf7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5cc4b3d20 0x55c5cc4b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:25.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.737+0000 7fcaaf7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5cc4b3d20 0x55c5cc4b2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53382/0 (socket says 172.21.15.5:53382) 2024-09-12T03:04:25.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.737+0000 7fcaaeffd640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c5cc4b27b0 0x55c5cc4b2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:25.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.737+0000 7fcaaf7fe640 1 -- 172.21.15.5:0/3990213728 learned_addr learned my addr 172.21.15.5:0/3990213728 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:04:25.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.737+0000 7fcaae7fc640 1 --2- 172.21.15.5:0/3990213728 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c5cc4c1960 0x55c5cc4c3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:25.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.738+0000 7fcaaf7fe640 1 -- 172.21.15.5:0/3990213728 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c5cc4c1960 msgr2=0x55c5cc4c3d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:25.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.738+0000 7fcaaf7fe640 1 --2- 172.21.15.5:0/3990213728 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c5cc4c1960 0x55c5cc4c3d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:25.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.738+0000 7fcaaf7fe640 1 -- 172.21.15.5:0/3990213728 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c5cc4b27b0 msgr2=0x55c5cc4b2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:25.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.738+0000 7fcaaf7fe640 1 --2- 172.21.15.5:0/3990213728 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c5cc4b27b0 0x55c5cc4b2bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:25.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.738+0000 7fcaaf7fe640 1 -- 172.21.15.5:0/3990213728 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5cc3a43a0 con 0x55c5cc4b3d20 2024-09-12T03:04:25.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.738+0000 7fcaaf7fe640 1 --2- 172.21.15.5:0/3990213728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5cc4b3d20 0x55c5cc4b2270 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fcaa8009f90 tx=0x7fcaa8010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50466f533f554194 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:25.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.738+0000 7fcaa5ffb640 1 -- 172.21.15.5:0/3990213728 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa8011700 con 0x55c5cc4b3d20 2024-09-12T03:04:25.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.738+0000 7fcaa5ffb640 1 -- 172.21.15.5:0/3990213728 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcaa8011d80 con 0x55c5cc4b3d20 2024-09-12T03:04:25.775 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.738+0000 7fcab4800640 1 --2- 172.21.15.5:0/708623574 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fca9c0a2b80 0x7fca9c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:25.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.738+0000 7fcaa5ffb640 1 -- 172.21.15.5:0/3990213728 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa8018380 con 0x55c5cc4b3d20 2024-09-12T03:04:25.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.739+0000 7fcaaeffd640 1 --2- 172.21.15.5:0/708623574 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fca9c0a2b80 0x7fca9c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:25.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.739+0000 7fcab4800640 1 -- 172.21.15.5:0/708623574 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fcaa00ef3a0 con 0x7fcab01008f0 2024-09-12T03:04:25.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.739+0000 7fcab7bdb740 1 -- 172.21.15.5:0/3990213728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5cc4b3d20 msgr2=0x55c5cc4b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:25.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.739+0000 7fcaaeffd640 1 --2- 172.21.15.5:0/708623574 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fca9c0a2b80 0x7fca9c0a5040 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x55c5cc4afca0 tx=0x7fcaa00f2e80 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:25.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.739+0000 7fcab7bdb740 1 --2- 172.21.15.5:0/3990213728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5cc4b3d20 0x55c5cc4b2270 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fcaa8009f90 tx=0x7fcaa8010860 comp rx=0 tx=0).stop 2024-09-12T03:04:25.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.739+0000 7fcab7bdb740 1 -- 172.21.15.5:0/3990213728 shutdown_connections 2024-09-12T03:04:25.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.739+0000 7fcab7bdb740 1 --2- 172.21.15.5:0/3990213728 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c5cc4c1960 0x55c5cc4c3d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:25.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.739+0000 7fcab7bdb740 1 --2- 172.21.15.5:0/3990213728 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c5cc4b27b0 0x55c5cc4b2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:25.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.739+0000 7fcab7bdb740 1 --2- 172.21.15.5:0/3990213728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5cc4b3d20 0x55c5cc4b2270 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:25.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.739+0000 7fcab7bdb740 1 -- 172.21.15.5:0/3990213728 >> 172.21.15.5:0/3990213728 conn(0x55c5cc47b310 msgr2=0x55c5cc4aee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:25.776 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.739+0000 7fcab7bdb740 1 -- 172.21.15.5:0/3990213728 shutdown_connections 2024-09-12T03:04:25.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.739+0000 7fcab7bdb740 1 -- 172.21.15.5:0/3990213728 wait complete. 2024-09-12T03:04:25.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.740+0000 7fcab7bdb740 1 -- start start 2024-09-12T03:04:25.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.740+0000 7fcab7bdb740 1 Processor -- start 2024-09-12T03:04:25.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.741+0000 7fcab7bdb740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c5cc4b27b0 0x55c5cc4c1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:25.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.741+0000 7fcab7bdb740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c5cc4b3d20 0x55c5cc4be5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:25.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.741+0000 7fcab7bdb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5cc4c1960 0x55c5cc4beb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:25.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.741+0000 7fcab7bdb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x55c5cc4c4660 con 0x55c5cc4c1960 2024-09-12T03:04:25.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.741+0000 7fcab7bdb740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x55c5cc3d3060 con 0x55c5cc4b27b0 2024-09-12T03:04:25.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.741+0000 7fcab7bdb740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x55c5cc4c47e0 con 0x55c5cc4b3d20 2024-09-12T03:04:25.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.741+0000 7fcab7bdb740 10 client.0 _ll_register_callbacks cb 0x55c5cc1268c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:04:25.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.741+0000 7fcaaf7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5cc4c1960 0x55c5cc4beb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:25.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.741+0000 7fcaaf7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5cc4c1960 0x55c5cc4beb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53392/0 (socket says 172.21.15.5:53392) 2024-09-12T03:04:25.777 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.741+0000 7fcaaf7fe640 1 -- 172.21.15.5:0/2803902217 learned_addr learned my addr 172.21.15.5:0/2803902217 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:04:25.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.741+0000 7fcaae7fc640 1 --2- 172.21.15.5:0/2803902217 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c5cc4b3d20 0x55c5cc4be5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:25.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.741+0000 7fcaaf7fe640 1 --2- 172.21.15.5:0/2803902217 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c5cc4b27b0 0x55c5cc4c1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:25.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.741+0000 7fcaaf7fe640 1 -- 172.21.15.5:0/2803902217 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c5cc4b3d20 msgr2=0x55c5cc4be5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:25.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.741+0000 7fcaaf7fe640 1 --2- 172.21.15.5:0/2803902217 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c5cc4b3d20 0x55c5cc4be5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:25.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.741+0000 7fcaaf7fe640 1 -- 172.21.15.5:0/2803902217 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c5cc4b27b0 msgr2=0x55c5cc4c1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:25.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.741+0000 7fcaaf7fe640 1 --2- 172.21.15.5:0/2803902217 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c5cc4b27b0 0x55c5cc4c1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:25.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.741+0000 7fcaaf7fe640 1 -- 172.21.15.5:0/2803902217 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5cc1893c0 con 0x55c5cc4c1960 2024-09-12T03:04:25.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.741+0000 7fcaae7fc640 1 --2- 172.21.15.5:0/2803902217 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c5cc4b3d20 0x55c5cc4be5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:04:25.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.741+0000 7fcaaf7fe640 1 --2- 172.21.15.5:0/2803902217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5cc4c1960 0x55c5cc4beb60 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fcaa80198a0 tx=0x7fcaa8013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b973cddbddd3e11 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:25.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.741+0000 7fca8effd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T03:04:25.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.742+0000 7fca8effd640 1 -- 172.21.15.5:0/2803902217 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa8003970 con 0x55c5cc4c1960 2024-09-12T03:04:25.778 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.742+0000 7fcab7bdb740 1 -- 172.21.15.5:0/2803902217 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcaa8011d80 con 0x55c5cc4c1960 2024-09-12T03:04:25.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.742+0000 7fca8effd640 1 -- 172.21.15.5:0/2803902217 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fcaa8003b10 con 0x55c5cc4c1960 2024-09-12T03:04:25.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.742+0000 7fca8effd640 1 -- 172.21.15.5:0/2803902217 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fcaa8003d10 con 0x55c5cc4c1960 2024-09-12T03:04:25.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.742+0000 7fcab7bdb740 10 client.16806 fetch_fsmap learned FSMap version 33 2024-09-12T03:04:25.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.742+0000 7fcab7bdb740 1 -- 172.21.15.5:0/2803902217 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55c5cc4d00b0 con 0x55c5cc4c1960 2024-09-12T03:04:25.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.742+0000 7fca8effd640 1 -- 172.21.15.5:0/2803902217 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcaa8011d80 con 0x55c5cc4c1960 2024-09-12T03:04:25.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.742+0000 7fca8effd640 1 -- 172.21.15.5:0/2803902217 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 33) ==== 8374+0+0 (secure 0 0 0) 0x7fcaa8025070 con 0x55c5cc4c1960 2024-09-12T03:04:25.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.742+0000 7fcab7bdb740 10 client.16806 fetch_fsmap finished waiting for FSMap version 33 2024-09-12T03:04:25.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.742+0000 7fcab7bdb740 10 client.16806 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-09-12T03:04:25.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.742+0000 7fcab7bdb740 20 client.16806 populate_metadata read hostname 'smithi005' 2024-09-12T03:04:25.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.742+0000 7fcab7bdb740 1 --2- 172.21.15.5:0/2803902217 >> [v2:172.21.15.148:6834/4195086427,v1:172.21.15.148:6835/4195086427] conn(0x55c5cc4d1460 0x55c5cc4d3850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:25.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.742+0000 7fcab7bdb740 4 client.16806 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:04:25.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.742+0000 7fcab7bdb740 1 -- 172.21.15.5:0/2803902217 --> [v2:172.21.15.148:6834/4195086427,v1:172.21.15.148:6835/4195086427] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c5cc425860 con 0x55c5cc4d1460 2024-09-12T03:04:25.779 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.743+0000 7fcaae7fc640 1 --2- 172.21.15.5:0/2803902217 >> [v2:172.21.15.148:6834/4195086427,v1:172.21.15.148:6835/4195086427] conn(0x55c5cc4d1460 0x55c5cc4d3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:25.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.743+0000 7fcaae7fc640 1 --2- 172.21.15.5:0/2803902217 >> [v2:172.21.15.148:6834/4195086427,v1:172.21.15.148:6835/4195086427] conn(0x55c5cc4d1460 0x55c5cc4d3850 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2ae9b967016ffc17 server_cookie=6ac09b6f6b3bc65 in_seq=0 out_seq=0 2024-09-12T03:04:25.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.743+0000 7fca8effd640 10 client.16806 ms_handle_connect on v2:172.21.15.148:6834/4195086427 2024-09-12T03:04:25.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.744+0000 7fca8effd640 1 -- 172.21.15.5:0/2803902217 <== mds.2 v2:172.21.15.148:6834/4195086427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c5cc425860 con 0x55c5cc4d1460 2024-09-12T03:04:25.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.744+0000 7fca8effd640 10 client.16806 handle_command_reply: tid=0 2024-09-12T03:04:25.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.744+0000 7fcab7bdb740 1 client.16806 shutdown 2024-09-12T03:04:25.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.744+0000 7fcab7bdb740 1 -- 172.21.15.5:0/2803902217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5cc4c1960 msgr2=0x55c5cc4beb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:25.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.744+0000 7fcab7bdb740 1 --2- 172.21.15.5:0/2803902217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5cc4c1960 0x55c5cc4beb60 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fcaa80198a0 tx=0x7fcaa8013040 comp rx=0 tx=0).stop 2024-09-12T03:04:25.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.744+0000 7fcab7bdb740 1 -- 172.21.15.5:0/2803902217 shutdown_connections 2024-09-12T03:04:25.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.744+0000 7fcab7bdb740 1 --2- 172.21.15.5:0/2803902217 >> [v2:172.21.15.148:6834/4195086427,v1:172.21.15.148:6835/4195086427] conn(0x55c5cc4d1460 0x55c5cc4d3850 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:25.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.744+0000 7fcab7bdb740 1 --2- 172.21.15.5:0/2803902217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5cc4c1960 0x55c5cc4beb60 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:25.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.744+0000 7fcab7bdb740 1 --2- 172.21.15.5:0/2803902217 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c5cc4b3d20 0x55c5cc4be5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:25.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.744+0000 7fcab7bdb740 1 --2- 172.21.15.5:0/2803902217 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c5cc4b27b0 0x55c5cc4c1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:25.780 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.744+0000 7fcab7bdb740 1 -- 172.21.15.5:0/2803902217 >> 172.21.15.5:0/2803902217 conn(0x55c5cc47b310 msgr2=0x55c5cc4b4b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:25.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.744+0000 7fca8effd640 0 client.16806 ms_handle_reset on v2:172.21.15.148:6834/4195086427 2024-09-12T03:04:25.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.744+0000 7fcab7bdb740 1 -- 172.21.15.5:0/2803902217 shutdown_connections 2024-09-12T03:04:25.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.744+0000 7fcab7bdb740 1 -- 172.21.15.5:0/2803902217 wait complete. 2024-09-12T03:04:25.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.744+0000 7fcab7bdb740 20 client.16806 trim_cache size 0 max 16384 2024-09-12T03:04:25.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.757+0000 7fcab7bdb740 1 Processor -- start 2024-09-12T03:04:25.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.757+0000 7fcab7bdb740 1 -- start start 2024-09-12T03:04:25.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.758+0000 7fcab7bdb740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c5cc4b27b0 0x55c5cc4cd900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:25.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.758+0000 7fcab7bdb740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c5cc4b3d20 0x55c5cc4d3470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:25.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.758+0000 7fcab7bdb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5cc4c1960 0x55c5cc4bd5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:25.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.758+0000 7fcab7bdb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcab01ab280 con 0x55c5cc4c1960 2024-09-12T03:04:25.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.758+0000 7fcab7bdb740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fcab01ace50 con 0x55c5cc4b3d20 2024-09-12T03:04:25.781 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.758+0000 7fcab7bdb740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fcab01ab980 con 0x55c5cc4b27b0 2024-09-12T03:04:25.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.758+0000 7fcaaf7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5cc4c1960 0x55c5cc4bd5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:25.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.758+0000 7fcaaf7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5cc4c1960 0x55c5cc4bd5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53408/0 (socket says 172.21.15.5:53408) 2024-09-12T03:04:25.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.758+0000 7fcaae7fc640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c5cc4b3d20 0x55c5cc4d3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:25.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.758+0000 7fcaaf7fe640 1 -- 172.21.15.5:0/3223756204 learned_addr learned my addr 172.21.15.5:0/3223756204 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:04:25.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.758+0000 7fcaaf7fe640 1 --2- 172.21.15.5:0/3223756204 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c5cc4b27b0 0x55c5cc4cd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:25.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.758+0000 7fcaae7fc640 1 -- 172.21.15.5:0/3223756204 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c5cc4b27b0 msgr2=0x55c5cc4cd900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:25.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.758+0000 7fcaae7fc640 1 --2- 172.21.15.5:0/3223756204 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c5cc4b27b0 0x55c5cc4cd900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:25.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.758+0000 7fcaae7fc640 1 -- 172.21.15.5:0/3223756204 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5cc4c1960 msgr2=0x55c5cc4bd5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:25.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.758+0000 7fcaae7fc640 1 --2- 172.21.15.5:0/3223756204 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5cc4c1960 0x55c5cc4bd5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:25.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.758+0000 7fcaae7fc640 1 -- 172.21.15.5:0/3223756204 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5cc4c0f30 con 0x55c5cc4b3d20 2024-09-12T03:04:25.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.758+0000 7fcaaf7fe640 1 --2- 172.21.15.5:0/3223756204 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5cc4c1960 0x55c5cc4bd5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:04:25.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.759+0000 7fcaae7fc640 1 --2- 172.21.15.5:0/3223756204 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c5cc4b3d20 0x55c5cc4d3470 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x55c5cc4bfc00 tx=0x7fca9800b390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5050d61a14ee8a7f server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:25.782 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.759+0000 7fcaa57fa640 1 -- 172.21.15.5:0/3223756204 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca98005020 con 0x55c5cc4b3d20 2024-09-12T03:04:25.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.759+0000 7fcaa57fa640 1 -- 172.21.15.5:0/3223756204 <== mon.2 v2:172.21.15.146:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca980047a0 con 0x55c5cc4b3d20 2024-09-12T03:04:25.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.759+0000 7fcaa57fa640 1 -- 172.21.15.5:0/3223756204 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca980266c0 con 0x55c5cc4b3d20 2024-09-12T03:04:25.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.759+0000 7fcab7bdb740 1 -- 172.21.15.5:0/3223756204 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c5cc4b3d20 msgr2=0x55c5cc4d3470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:25.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.759+0000 7fcab7bdb740 1 --2- 172.21.15.5:0/3223756204 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c5cc4b3d20 0x55c5cc4d3470 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x55c5cc4bfc00 tx=0x7fca9800b390 comp rx=0 tx=0).stop 2024-09-12T03:04:25.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.760+0000 7fcab7bdb740 1 -- 172.21.15.5:0/3223756204 shutdown_connections 2024-09-12T03:04:25.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.760+0000 7fcab7bdb740 1 --2- 172.21.15.5:0/3223756204 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5cc4c1960 0x55c5cc4bd5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:25.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.760+0000 7fcab7bdb740 1 --2- 172.21.15.5:0/3223756204 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c5cc4b3d20 0x55c5cc4d3470 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:25.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.760+0000 7fcab7bdb740 1 --2- 172.21.15.5:0/3223756204 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c5cc4b27b0 0x55c5cc4cd900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:25.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.760+0000 7fcab7bdb740 1 -- 172.21.15.5:0/3223756204 >> 172.21.15.5:0/3223756204 conn(0x55c5cc47b310 msgr2=0x55c5cc4bce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:25.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.760+0000 7fcab7bdb740 1 -- 172.21.15.5:0/3223756204 shutdown_connections 2024-09-12T03:04:25.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.760+0000 7fcab7bdb740 1 -- 172.21.15.5:0/3223756204 wait complete. 2024-09-12T03:04:25.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.760+0000 7fcab7bdb740 1 -- start start 2024-09-12T03:04:25.783 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.760+0000 7fcab7bdb740 1 Processor -- start 2024-09-12T03:04:25.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.760+0000 7fcab7bdb740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c5cc4b27b0 0x55c5cc4c9e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:25.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.760+0000 7fcab7bdb740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5cc4b3d20 0x55c5cc4c51a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:25.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.760+0000 7fcab7bdb740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c5cc4c1960 0x55c5cc4c56e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:25.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.760+0000 7fcab7bdb740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcab01ac780 con 0x55c5cc4b3d20 2024-09-12T03:04:25.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.760+0000 7fcab7bdb740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fcab01ac080 con 0x55c5cc4b27b0 2024-09-12T03:04:25.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.760+0000 7fcab7bdb740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fcab01a9260 con 0x55c5cc4c1960 2024-09-12T03:04:25.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.760+0000 7fcab7bdb740 10 client.0 _ll_register_callbacks cb 0x7fcaa00f2d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:04:25.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.761+0000 7fcaae7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5cc4b3d20 0x55c5cc4c51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:25.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.761+0000 7fcaae7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5cc4b3d20 0x55c5cc4c51a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53412/0 (socket says 172.21.15.5:53412) 2024-09-12T03:04:25.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.761+0000 7fcaae7fc640 1 -- 172.21.15.5:0/4106890699 learned_addr learned my addr 172.21.15.5:0/4106890699 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:04:25.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.761+0000 7fcaaf7fe640 1 --2- 172.21.15.5:0/4106890699 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c5cc4b27b0 0x55c5cc4c9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:25.784 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.761+0000 7fcaaf7fe640 1 --2- 172.21.15.5:0/4106890699 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c5cc4c1960 0x55c5cc4c56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:25.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.761+0000 7fcaae7fc640 1 -- 172.21.15.5:0/4106890699 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c5cc4c1960 msgr2=0x55c5cc4c56e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:25.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.761+0000 7fcaae7fc640 1 --2- 172.21.15.5:0/4106890699 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c5cc4c1960 0x55c5cc4c56e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:25.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.761+0000 7fcaae7fc640 1 -- 172.21.15.5:0/4106890699 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c5cc4b27b0 msgr2=0x55c5cc4c9e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:25.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.761+0000 7fcaae7fc640 1 --2- 172.21.15.5:0/4106890699 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c5cc4b27b0 0x55c5cc4c9e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:25.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.761+0000 7fcaae7fc640 1 -- 172.21.15.5:0/4106890699 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5cc4c5c20 con 0x55c5cc4b3d20 2024-09-12T03:04:25.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.761+0000 7fcaaf7fe640 1 --2- 172.21.15.5:0/4106890699 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c5cc4b27b0 0x55c5cc4c9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:04:25.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.761+0000 7fcaae7fc640 1 --2- 172.21.15.5:0/4106890699 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5cc4b3d20 0x55c5cc4c51a0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fca98000c00 tx=0x7fca9800b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbb919bffbb285d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:25.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.761+0000 7fca8effd640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T03:04:25.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.761+0000 7fca8effd640 1 -- 172.21.15.5:0/4106890699 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca98004d60 con 0x55c5cc4b3d20 2024-09-12T03:04:25.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.761+0000 7fca8effd640 1 -- 172.21.15.5:0/4106890699 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fca98005020 con 0x55c5cc4b3d20 2024-09-12T03:04:25.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.761+0000 7fcab7bdb740 1 -- 172.21.15.5:0/4106890699 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca980047a0 con 0x55c5cc4b3d20 2024-09-12T03:04:25.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.762+0000 7fca8effd640 1 -- 172.21.15.5:0/4106890699 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fca9800be50 con 0x55c5cc4b3d20 2024-09-12T03:04:25.785 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.762+0000 7fcab7bdb740 10 client.16812 fetch_fsmap learned FSMap version 33 2024-09-12T03:04:25.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.762+0000 7fcab7bdb740 1 -- 172.21.15.5:0/4106890699 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55c5cc53bb60 con 0x55c5cc4b3d20 2024-09-12T03:04:25.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.762+0000 7fca8effd640 1 -- 172.21.15.5:0/4106890699 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca980047a0 con 0x55c5cc4b3d20 2024-09-12T03:04:25.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.762+0000 7fca8effd640 1 -- 172.21.15.5:0/4106890699 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 33) ==== 8374+0+0 (secure 0 0 0) 0x7fca98026bd0 con 0x55c5cc4b3d20 2024-09-12T03:04:25.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.762+0000 7fcab7bdb740 10 client.16812 fetch_fsmap finished waiting for FSMap version 33 2024-09-12T03:04:25.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.762+0000 7fcab7bdb740 10 client.16812 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.f 2024-09-12T03:04:25.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.762+0000 7fcab7bdb740 20 client.16812 populate_metadata read hostname 'smithi005' 2024-09-12T03:04:25.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.762+0000 7fcab7bdb740 1 --2- 172.21.15.5:0/4106890699 >> [v2:172.21.15.148:6834/4195086427,v1:172.21.15.148:6835/4195086427] conn(0x55c5cc53c200 0x55c5cc53e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:25.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.762+0000 7fcab7bdb740 4 client.16812 mds_command: new command op to 24481 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:04:25.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.762+0000 7fcab7bdb740 1 -- 172.21.15.5:0/4106890699 --> [v2:172.21.15.148:6834/4195086427,v1:172.21.15.148:6835/4195086427] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c5cc41fd70 con 0x55c5cc53c200 2024-09-12T03:04:25.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.762+0000 7fcaaf7fe640 1 --2- 172.21.15.5:0/4106890699 >> [v2:172.21.15.148:6834/4195086427,v1:172.21.15.148:6835/4195086427] conn(0x55c5cc53c200 0x55c5cc53e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:25.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.763+0000 7fcaaf7fe640 1 --2- 172.21.15.5:0/4106890699 >> [v2:172.21.15.148:6834/4195086427,v1:172.21.15.148:6835/4195086427] conn(0x55c5cc53c200 0x55c5cc53e620 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2c38095df4fe7fd4 server_cookie=8592a0c924aba9a8 in_seq=0 out_seq=0 2024-09-12T03:04:25.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.763+0000 7fca8effd640 10 client.16812 ms_handle_connect on v2:172.21.15.148:6834/4195086427 2024-09-12T03:04:25.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.763+0000 7fca8effd640 1 -- 172.21.15.5:0/4106890699 <== mds.2 v2:172.21.15.148:6834/4195086427 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c5cc41fd70 con 0x55c5cc53c200 2024-09-12T03:04:25.786 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.763+0000 7fca8effd640 10 client.16812 handle_command_reply: tid=0 2024-09-12T03:04:25.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.763+0000 7fcab7bdb740 1 client.16812 shutdown 2024-09-12T03:04:25.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.763+0000 7fcab7bdb740 1 -- 172.21.15.5:0/4106890699 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5cc4b3d20 msgr2=0x55c5cc4c51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:25.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.763+0000 7fcab7bdb740 1 --2- 172.21.15.5:0/4106890699 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5cc4b3d20 0x55c5cc4c51a0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fca98000c00 tx=0x7fca9800b9f0 comp rx=0 tx=0).stop 2024-09-12T03:04:25.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.763+0000 7fcab7bdb740 1 -- 172.21.15.5:0/4106890699 shutdown_connections 2024-09-12T03:04:25.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.763+0000 7fcab7bdb740 1 --2- 172.21.15.5:0/4106890699 >> [v2:172.21.15.148:6834/4195086427,v1:172.21.15.148:6835/4195086427] conn(0x55c5cc53c200 0x55c5cc53e620 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:25.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.763+0000 7fcab7bdb740 1 --2- 172.21.15.5:0/4106890699 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5cc4b3d20 0x55c5cc4c51a0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:25.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.763+0000 7fcab7bdb740 1 --2- 172.21.15.5:0/4106890699 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55c5cc4c1960 0x55c5cc4c56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:25.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.763+0000 7fca8effd640 0 client.16812 ms_handle_reset on v2:172.21.15.148:6834/4195086427 2024-09-12T03:04:25.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.763+0000 7fcab7bdb740 1 --2- 172.21.15.5:0/4106890699 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55c5cc4b27b0 0x55c5cc4c9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:25.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.763+0000 7fcab7bdb740 1 -- 172.21.15.5:0/4106890699 >> 172.21.15.5:0/4106890699 conn(0x55c5cc47b310 msgr2=0x55c5cc4bd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:25.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.763+0000 7fcab7bdb740 1 -- 172.21.15.5:0/4106890699 shutdown_connections 2024-09-12T03:04:25.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.763+0000 7fcab7bdb740 1 -- 172.21.15.5:0/4106890699 wait complete. 2024-09-12T03:04:25.787 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.763+0000 7fcab7bdb740 20 client.16812 trim_cache size 0 max 16384 2024-09-12T03:04:25.788 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-12T03:04:25.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.764+0000 7fcab5802640 1 -- 172.21.15.5:0/708623574 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fca9c0a2b80 msgr2=0x7fca9c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:25.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.764+0000 7fcab5802640 1 --2- 172.21.15.5:0/708623574 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fca9c0a2b80 0x7fca9c0a5040 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x55c5cc4afca0 tx=0x7fcaa00f2e80 comp rx=0 tx=0).stop 2024-09-12T03:04:25.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.764+0000 7fcab5802640 1 -- 172.21.15.5:0/708623574 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcab01008f0 msgr2=0x7fcab0198320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:25.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.764+0000 7fcab5802640 1 --2- 172.21.15.5:0/708623574 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcab01008f0 0x7fcab0198320 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fcaa002a940 tx=0x7fcaa0003b90 comp rx=0 tx=0).stop 2024-09-12T03:04:25.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.764+0000 7fcab5802640 1 -- 172.21.15.5:0/708623574 shutdown_connections 2024-09-12T03:04:25.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.764+0000 7fcab5802640 1 --2- 172.21.15.5:0/708623574 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcab01068f0 0x7fcab019cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:25.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.764+0000 7fcab5802640 1 --2- 172.21.15.5:0/708623574 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcab0101290 0x7fcab0198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:25.788 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.764+0000 7fcab5802640 1 --2- 172.21.15.5:0/708623574 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fca9c0a2b80 0x7fca9c0a5040 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:25.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.764+0000 7fcab5802640 1 --2- 172.21.15.5:0/708623574 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcab01008f0 0x7fcab0198320 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:25.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.764+0000 7fcab5802640 1 -- 172.21.15.5:0/708623574 >> 172.21.15.5:0/708623574 conn(0x7fcab00fc630 msgr2=0x7fcab006a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:25.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.764+0000 7fcab5802640 1 -- 172.21.15.5:0/708623574 shutdown_connections 2024-09-12T03:04:25.789 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:25.764+0000 7fcab5802640 1 -- 172.21.15.5:0/708623574 wait complete. 2024-09-12T03:04:26.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:04:25 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:04:26.165 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:04:25 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:04:26.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:04:25 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-12T03:04:26.613 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-12T03:04:26.872 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:04:27.112 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:04:26 smithi005 ceph-mon[31197]: pgmap v521: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-12T03:04:27.113 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:04:26 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:04:27.113 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:04:26 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:04:27.113 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:04:26 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T03:04:27.165 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:04:26 smithi148 ceph-mon[35090]: pgmap v521: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-12T03:04:27.165 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:04:26 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:04:27.165 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:04:26 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:04:27.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:04:26 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T03:04:27.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:04:26 smithi146 ceph-mon[35011]: pgmap v521: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 767 B/s wr, 10 op/s 2024-09-12T03:04:27.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:04:26 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-12T03:04:27.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:04:26 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-12T03:04:27.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:04:26 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' 2024-09-12T03:04:27.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.364+0000 7f45170aa640 1 -- 172.21.15.5:0/93104129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4510101420 msgr2=0x7f451010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:27.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.364+0000 7f45170aa640 1 --2- 172.21.15.5:0/93104129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4510101420 0x7f451010d950 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f4508009930 tx=0x7f450802a420 comp rx=0 tx=0).stop 2024-09-12T03:04:27.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.364+0000 7f45170aa640 1 -- 172.21.15.5:0/93104129 shutdown_connections 2024-09-12T03:04:27.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.364+0000 7f45170aa640 1 --2- 172.21.15.5:0/93104129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4510101420 0x7f451010d950 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:27.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.364+0000 7f45170aa640 1 --2- 172.21.15.5:0/93104129 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4510100a80 0x7f4510100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:27.365 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.364+0000 7f45170aa640 1 --2- 172.21.15.5:0/93104129 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4510106a80 0x7f4510106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:27.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.364+0000 7f45170aa640 1 -- 172.21.15.5:0/93104129 >> 172.21.15.5:0/93104129 conn(0x7f45100fc7a0 msgr2=0x7f45100febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:27.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.364+0000 7f45170aa640 1 -- 172.21.15.5:0/93104129 shutdown_connections 2024-09-12T03:04:27.366 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.365+0000 7f45170aa640 1 -- 172.21.15.5:0/93104129 wait complete. 2024-09-12T03:04:27.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.365+0000 7f45170aa640 1 Processor -- start 2024-09-12T03:04:27.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.365+0000 7f45170aa640 1 -- start start 2024-09-12T03:04:27.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.366+0000 7f45170aa640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4510100a80 0x7f45101972c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:27.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.366+0000 7f45170aa640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4510101420 0x7f4510197800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:27.367 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.366+0000 7f45170aa640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4510106a80 0x7f451019bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:27.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.366+0000 7f45170aa640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f4510110e80 con 0x7f4510106a80 2024-09-12T03:04:27.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.366+0000 7f45170aa640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f4510110d00 con 0x7f4510100a80 2024-09-12T03:04:27.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.366+0000 7f45170aa640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f4510111000 con 0x7f4510101420 2024-09-12T03:04:27.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.366+0000 7f4515647640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4510106a80 0x7f451019bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:27.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.366+0000 7f4515647640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4510106a80 0x7f451019bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53436/0 (socket says 172.21.15.5:53436) 2024-09-12T03:04:27.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.366+0000 7f4515647640 1 -- 172.21.15.5:0/3366961776 learned_addr learned my addr 172.21.15.5:0/3366961776 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:04:27.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.366+0000 7f4514e46640 1 --2- 172.21.15.5:0/3366961776 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4510100a80 0x7f45101972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:27.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.366+0000 7f450ffff640 1 --2- 172.21.15.5:0/3366961776 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4510101420 0x7f4510197800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:27.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.366+0000 7f4515647640 1 -- 172.21.15.5:0/3366961776 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4510101420 msgr2=0x7f4510197800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:27.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.366+0000 7f4515647640 1 --2- 172.21.15.5:0/3366961776 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4510101420 0x7f4510197800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:27.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.366+0000 7f4515647640 1 -- 172.21.15.5:0/3366961776 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4510100a80 msgr2=0x7f45101972c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:27.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.366+0000 7f4515647640 1 --2- 172.21.15.5:0/3366961776 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4510100a80 0x7f45101972c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:27.368 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.366+0000 7f4515647640 1 -- 172.21.15.5:0/3366961776 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f451019c270 con 0x7f4510106a80 2024-09-12T03:04:27.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.367+0000 7f4515647640 1 --2- 172.21.15.5:0/3366961776 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4510106a80 0x7f451019bb90 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f4508005ab0 tx=0x7f45080042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:27.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.367+0000 7f450dffb640 1 -- 172.21.15.5:0/3366961776 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f450803f030 con 0x7f4510106a80 2024-09-12T03:04:27.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.367+0000 7f450dffb640 1 -- 172.21.15.5:0/3366961776 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4508003ce0 con 0x7f4510106a80 2024-09-12T03:04:27.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.367+0000 7f45170aa640 1 -- 172.21.15.5:0/3366961776 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f451019c500 con 0x7f4510106a80 2024-09-12T03:04:27.369 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.367+0000 7f450dffb640 1 -- 172.21.15.5:0/3366961776 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4508035070 con 0x7f4510106a80 2024-09-12T03:04:27.370 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.367+0000 7f45170aa640 1 -- 172.21.15.5:0/3366961776 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f451006a790 con 0x7f4510106a80 2024-09-12T03:04:27.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.369+0000 7f450dffb640 1 -- 172.21.15.5:0/3366961776 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f4508042030 con 0x7f4510106a80 2024-09-12T03:04:27.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.370+0000 7f4519483740 1 Processor -- start 2024-09-12T03:04:27.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.370+0000 7f4519483740 1 -- start start 2024-09-12T03:04:27.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.370+0000 7f4519483740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55824948ab40 0x558249489090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:27.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.370+0000 7f4519483740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5582494895d0 0x558249489a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:27.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.370+0000 7f4519483740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558249494370 0x558249496760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:27.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.370+0000 7f4519483740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x5582493ef2f0 con 0x55824948ab40 2024-09-12T03:04:27.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.370+0000 7f4519483740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x558249396cf0 con 0x558249494370 2024-09-12T03:04:27.373 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.370+0000 7f4519483740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5582493eb2b0 con 0x5582494895d0 2024-09-12T03:04:27.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.371+0000 7f450dffb640 1 --2- 172.21.15.5:0/3366961776 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f44fc0a2b80 0x7f44fc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:27.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.371+0000 7f450ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55824948ab40 0x558249489090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:27.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.371+0000 7f450ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55824948ab40 0x558249489090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53444/0 (socket says 172.21.15.5:53444) 2024-09-12T03:04:27.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.371+0000 7f450ffff640 1 -- 172.21.15.5:0/1649096066 learned_addr learned my addr 172.21.15.5:0/1649096066 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:04:27.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.371+0000 7f4514e46640 1 --2- 172.21.15.5:0/1649096066 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5582494895d0 0x558249489a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:27.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.371+0000 7f4515647640 1 --2- 172.21.15.5:0/3366961776 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f44fc0a2b80 0x7f44fc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:27.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.371+0000 7f450ffff640 1 --2- 172.21.15.5:0/1649096066 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558249494370 0x558249496760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:27.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.371+0000 7f450dffb640 1 -- 172.21.15.5:0/3366961776 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f45080ef480 con 0x7f4510106a80 2024-09-12T03:04:27.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.371+0000 7f450ffff640 1 -- 172.21.15.5:0/1649096066 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5582494895d0 msgr2=0x558249489a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:27.374 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.371+0000 7f450ffff640 1 --2- 172.21.15.5:0/1649096066 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5582494895d0 0x558249489a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:27.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.371+0000 7f450ffff640 1 -- 172.21.15.5:0/1649096066 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558249494370 msgr2=0x558249496760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:27.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.371+0000 7f450ffff640 1 --2- 172.21.15.5:0/1649096066 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558249494370 0x558249496760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:27.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.371+0000 7f450ffff640 1 -- 172.21.15.5:0/1649096066 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55824937b3a0 con 0x55824948ab40 2024-09-12T03:04:27.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.371+0000 7f450ffff640 1 --2- 172.21.15.5:0/1649096066 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55824948ab40 0x558249489090 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f45000078a0 tx=0x7f4500007b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4ad6bd8d854b2c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:27.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.371+0000 7f4515647640 1 --2- 172.21.15.5:0/3366961776 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f44fc0a2b80 0x7f44fc0a5040 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f45080b81a0 tx=0x7f450808acc0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:27.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.371+0000 7f4506ffd640 1 -- 172.21.15.5:0/1649096066 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45000110d0 con 0x55824948ab40 2024-09-12T03:04:27.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.371+0000 7f4506ffd640 1 -- 172.21.15.5:0/1649096066 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4500013020 con 0x55824948ab40 2024-09-12T03:04:27.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.372+0000 7f4506ffd640 1 -- 172.21.15.5:0/1649096066 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4500005020 con 0x55824948ab40 2024-09-12T03:04:27.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.372+0000 7f4519483740 1 -- 172.21.15.5:0/1649096066 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55824948ab40 msgr2=0x558249489090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:27.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.372+0000 7f4519483740 1 --2- 172.21.15.5:0/1649096066 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55824948ab40 0x558249489090 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f45000078a0 tx=0x7f4500007b00 comp rx=0 tx=0).stop 2024-09-12T03:04:27.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.372+0000 7f4519483740 1 -- 172.21.15.5:0/1649096066 shutdown_connections 2024-09-12T03:04:27.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.372+0000 7f4519483740 1 --2- 172.21.15.5:0/1649096066 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x558249494370 0x558249496760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:27.375 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.372+0000 7f4519483740 1 --2- 172.21.15.5:0/1649096066 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5582494895d0 0x558249489a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:27.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.372+0000 7f4519483740 1 --2- 172.21.15.5:0/1649096066 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55824948ab40 0x558249489090 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:27.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.372+0000 7f4519483740 1 -- 172.21.15.5:0/1649096066 >> 172.21.15.5:0/1649096066 conn(0x558249452310 msgr2=0x558249485c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:27.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.372+0000 7f4519483740 1 -- 172.21.15.5:0/1649096066 shutdown_connections 2024-09-12T03:04:27.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.372+0000 7f4519483740 1 -- 172.21.15.5:0/1649096066 wait complete. 2024-09-12T03:04:27.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.373+0000 7f4519483740 1 -- start start 2024-09-12T03:04:27.376 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.373+0000 7f4519483740 1 Processor -- start 2024-09-12T03:04:27.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.373+0000 7f4519483740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5582494895d0 0x558249493cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:27.379 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.373+0000 7f4519483740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55824948ab40 0x558249490f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:27.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.373+0000 7f4519483740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558249494370 0x5582494914d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:27.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.373+0000 7f4519483740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x558249497070 con 0x558249494370 2024-09-12T03:04:27.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.373+0000 7f4519483740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x5582493aa060 con 0x5582494895d0 2024-09-12T03:04:27.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.373+0000 7f4519483740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x5582494971f0 con 0x55824948ab40 2024-09-12T03:04:27.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.373+0000 7f4519483740 10 client.0 _ll_register_callbacks cb 0x5582490fd8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:04:27.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.373+0000 7f450ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558249494370 0x5582494914d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:27.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.373+0000 7f450ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558249494370 0x5582494914d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53458/0 (socket says 172.21.15.5:53458) 2024-09-12T03:04:27.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.373+0000 7f450ffff640 1 -- 172.21.15.5:0/3274668733 learned_addr learned my addr 172.21.15.5:0/3274668733 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:04:27.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.373+0000 7f4514e46640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55824948ab40 0x558249490f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:27.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.374+0000 7f450ffff640 1 --2- 172.21.15.5:0/3274668733 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5582494895d0 0x558249493cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:27.380 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.374+0000 7f450ffff640 1 -- 172.21.15.5:0/3274668733 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55824948ab40 msgr2=0x558249490f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:27.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.374+0000 7f450ffff640 1 --2- 172.21.15.5:0/3274668733 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55824948ab40 0x558249490f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:27.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.374+0000 7f450ffff640 1 -- 172.21.15.5:0/3274668733 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5582494895d0 msgr2=0x558249493cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:27.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.374+0000 7f450ffff640 1 --2- 172.21.15.5:0/3274668733 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5582494895d0 0x558249493cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:27.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.374+0000 7f450ffff640 1 -- 172.21.15.5:0/3274668733 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582491603c0 con 0x558249494370 2024-09-12T03:04:27.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.374+0000 7f450ffff640 1 --2- 172.21.15.5:0/3274668733 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558249494370 0x5582494914d0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f4500005fd0 tx=0x7f4500011d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fcf29f03458934d server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:27.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.374+0000 7f44effff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T03:04:27.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.374+0000 7f44effff640 1 -- 172.21.15.5:0/3274668733 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4500004c50 con 0x558249494370 2024-09-12T03:04:27.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.374+0000 7f44effff640 1 -- 172.21.15.5:0/3274668733 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f4500004df0 con 0x558249494370 2024-09-12T03:04:27.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.374+0000 7f4519483740 1 -- 172.21.15.5:0/3274668733 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4500013020 con 0x558249494370 2024-09-12T03:04:27.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.374+0000 7f44effff640 1 -- 172.21.15.5:0/3274668733 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f4500005050 con 0x558249494370 2024-09-12T03:04:27.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.374+0000 7f4519483740 10 client.16824 fetch_fsmap learned FSMap version 33 2024-09-12T03:04:27.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.374+0000 7f4519483740 1 -- 172.21.15.5:0/3274668733 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5582494a2a80 con 0x558249494370 2024-09-12T03:04:27.381 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.374+0000 7f44effff640 1 -- 172.21.15.5:0/3274668733 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4500013020 con 0x558249494370 2024-09-12T03:04:27.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.375+0000 7f44effff640 1 -- 172.21.15.5:0/3274668733 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 33) ==== 8374+0+0 (secure 0 0 0) 0x7f45000126b0 con 0x558249494370 2024-09-12T03:04:27.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.375+0000 7f4519483740 10 client.16824 fetch_fsmap finished waiting for FSMap version 33 2024-09-12T03:04:27.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.375+0000 7f4519483740 10 client.16824 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-12T03:04:27.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.375+0000 7f4519483740 20 client.16824 populate_metadata read hostname 'smithi005' 2024-09-12T03:04:27.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.375+0000 7f4519483740 1 --2- 172.21.15.5:0/3274668733 >> [v2:172.21.15.148:6836/2606873922,v1:172.21.15.148:6837/2606873922] conn(0x5582494a3ee0 0x5582494a62d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:27.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.375+0000 7f4519483740 4 client.16824 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:04:27.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.375+0000 7f4519483740 1 -- 172.21.15.5:0/3274668733 --> [v2:172.21.15.148:6836/2606873922,v1:172.21.15.148:6837/2606873922] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5582493fc860 con 0x5582494a3ee0 2024-09-12T03:04:27.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.375+0000 7f4514e46640 1 --2- 172.21.15.5:0/3274668733 >> [v2:172.21.15.148:6836/2606873922,v1:172.21.15.148:6837/2606873922] conn(0x5582494a3ee0 0x5582494a62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:27.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.375+0000 7f44effff640 10 client.16824 ms_handle_connect on v2:172.21.15.148:6836/2606873922 2024-09-12T03:04:27.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.375+0000 7f4514e46640 1 --2- 172.21.15.5:0/3274668733 >> [v2:172.21.15.148:6836/2606873922,v1:172.21.15.148:6837/2606873922] conn(0x5582494a3ee0 0x5582494a62d0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=39368a95f2af8f58 server_cookie=ad70ab254a2826ca in_seq=0 out_seq=0 2024-09-12T03:04:27.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.376+0000 7f44effff640 1 -- 172.21.15.5:0/3274668733 <== mds.1 v2:172.21.15.148:6836/2606873922 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5582493fc860 con 0x5582494a3ee0 2024-09-12T03:04:27.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.376+0000 7f44effff640 10 client.16824 handle_command_reply: tid=0 2024-09-12T03:04:27.382 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.376+0000 7f4519483740 1 client.16824 shutdown 2024-09-12T03:04:27.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.376+0000 7f4519483740 1 -- 172.21.15.5:0/3274668733 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558249494370 msgr2=0x5582494914d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:27.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.376+0000 7f4519483740 1 --2- 172.21.15.5:0/3274668733 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558249494370 0x5582494914d0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f4500005fd0 tx=0x7f4500011d10 comp rx=0 tx=0).stop 2024-09-12T03:04:27.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.376+0000 7f4519483740 1 -- 172.21.15.5:0/3274668733 shutdown_connections 2024-09-12T03:04:27.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.376+0000 7f4519483740 1 --2- 172.21.15.5:0/3274668733 >> [v2:172.21.15.148:6836/2606873922,v1:172.21.15.148:6837/2606873922] conn(0x5582494a3ee0 0x5582494a62d0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:27.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.376+0000 7f4519483740 1 --2- 172.21.15.5:0/3274668733 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558249494370 0x5582494914d0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:27.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.376+0000 7f4519483740 1 --2- 172.21.15.5:0/3274668733 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x55824948ab40 0x558249490f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:27.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.376+0000 7f4519483740 1 --2- 172.21.15.5:0/3274668733 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x5582494895d0 0x558249493cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:27.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.376+0000 7f44effff640 0 client.16824 ms_handle_reset on v2:172.21.15.148:6836/2606873922 2024-09-12T03:04:27.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.376+0000 7f4519483740 1 -- 172.21.15.5:0/3274668733 >> 172.21.15.5:0/3274668733 conn(0x558249452310 msgr2=0x55824948b960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:27.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.376+0000 7f4519483740 1 -- 172.21.15.5:0/3274668733 shutdown_connections 2024-09-12T03:04:27.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.377+0000 7f4519483740 1 -- 172.21.15.5:0/3274668733 wait complete. 2024-09-12T03:04:27.383 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.377+0000 7f4519483740 20 client.16824 trim_cache size 0 max 16384 2024-09-12T03:04:27.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.390+0000 7f4519483740 1 Processor -- start 2024-09-12T03:04:27.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.390+0000 7f4519483740 1 -- start start 2024-09-12T03:04:27.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.390+0000 7f4519483740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5582494895d0 0x5582494a0310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:27.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.390+0000 7f4519483740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55824948ab40 0x55824948c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:27.399 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.390+0000 7f4519483740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558249494370 0x55824948c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:27.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.390+0000 7f4519483740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f45101ab2c0 con 0x558249494370 2024-09-12T03:04:27.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.390+0000 7f4519483740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f45101ace90 con 0x55824948ab40 2024-09-12T03:04:27.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.390+0000 7f4519483740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f45101ab9c0 con 0x5582494895d0 2024-09-12T03:04:27.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.390+0000 7f450ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558249494370 0x55824948c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:27.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.390+0000 7f450ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558249494370 0x55824948c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53472/0 (socket says 172.21.15.5:53472) 2024-09-12T03:04:27.400 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.390+0000 7f450ffff640 1 -- 172.21.15.5:0/3369950730 learned_addr learned my addr 172.21.15.5:0/3369950730 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:04:27.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.390+0000 7f4514e46640 1 --2- 172.21.15.5:0/3369950730 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55824948ab40 0x55824948c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:27.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.390+0000 7f450ffff640 1 --2- 172.21.15.5:0/3369950730 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5582494895d0 0x5582494a0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:27.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.390+0000 7f450ffff640 1 -- 172.21.15.5:0/3369950730 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5582494895d0 msgr2=0x5582494a0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:27.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.390+0000 7f450ffff640 1 --2- 172.21.15.5:0/3369950730 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5582494895d0 0x5582494a0310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:27.401 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.390+0000 7f450ffff640 1 -- 172.21.15.5:0/3369950730 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55824948ab40 msgr2=0x55824948c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:27.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.390+0000 7f450ffff640 1 --2- 172.21.15.5:0/3369950730 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55824948ab40 0x55824948c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:27.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.390+0000 7f450ffff640 1 -- 172.21.15.5:0/3369950730 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55824948f4c0 con 0x558249494370 2024-09-12T03:04:27.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.391+0000 7f450ffff640 1 --2- 172.21.15.5:0/3369950730 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558249494370 0x55824948c660 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f45000078a0 tx=0x7f4500007830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a22d038469d243b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:27.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.391+0000 7f45067fc640 1 -- 172.21.15.5:0/3369950730 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45000079d0 con 0x558249494370 2024-09-12T03:04:27.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.391+0000 7f45067fc640 1 -- 172.21.15.5:0/3369950730 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4500013620 con 0x558249494370 2024-09-12T03:04:27.402 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.391+0000 7f45067fc640 1 -- 172.21.15.5:0/3369950730 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4500012f30 con 0x558249494370 2024-09-12T03:04:27.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.391+0000 7f4519483740 1 -- 172.21.15.5:0/3369950730 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558249494370 msgr2=0x55824948c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:27.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.391+0000 7f4519483740 1 --2- 172.21.15.5:0/3369950730 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558249494370 0x55824948c660 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f45000078a0 tx=0x7f4500007830 comp rx=0 tx=0).stop 2024-09-12T03:04:27.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.391+0000 7f4519483740 1 -- 172.21.15.5:0/3369950730 shutdown_connections 2024-09-12T03:04:27.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.391+0000 7f4519483740 1 --2- 172.21.15.5:0/3369950730 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558249494370 0x55824948c660 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:27.403 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.391+0000 7f4519483740 1 --2- 172.21.15.5:0/3369950730 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55824948ab40 0x55824948c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:27.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.391+0000 7f4519483740 1 --2- 172.21.15.5:0/3369950730 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5582494895d0 0x5582494a0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:27.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.391+0000 7f4519483740 1 -- 172.21.15.5:0/3369950730 >> 172.21.15.5:0/3369950730 conn(0x558249452310 msgr2=0x558249490f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:27.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.391+0000 7f4519483740 1 -- 172.21.15.5:0/3369950730 shutdown_connections 2024-09-12T03:04:27.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.391+0000 7f4519483740 1 -- 172.21.15.5:0/3369950730 wait complete. 2024-09-12T03:04:27.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.392+0000 7f4519483740 1 -- start start 2024-09-12T03:04:27.404 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.392+0000 7f4519483740 1 Processor -- start 2024-09-12T03:04:27.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.392+0000 7f4519483740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5582494895d0 0x55824949c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:27.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.392+0000 7f4519483740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55824948ab40 0x558249497cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:27.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.392+0000 7f4519483740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558249494370 0x558249498230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:27.405 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.392+0000 7f4519483740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f45101ac7c0 con 0x558249494370 2024-09-12T03:04:27.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.392+0000 7f4519483740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f45101ac0c0 con 0x55824948ab40 2024-09-12T03:04:27.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.392+0000 7f4519483740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f45101a92a0 con 0x5582494895d0 2024-09-12T03:04:27.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.392+0000 7f4519483740 10 client.0 _ll_register_callbacks cb 0x7f44f800d8a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:04:27.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.392+0000 7f450ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558249494370 0x558249498230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:27.406 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.392+0000 7f450ffff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558249494370 0x558249498230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53474/0 (socket says 172.21.15.5:53474) 2024-09-12T03:04:27.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.392+0000 7f450ffff640 1 -- 172.21.15.5:0/3938055882 learned_addr learned my addr 172.21.15.5:0/3938055882 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:04:27.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.392+0000 7f450ffff640 1 --2- 172.21.15.5:0/3938055882 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5582494895d0 0x55824949c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:27.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.393+0000 7f4514e46640 1 --2- 172.21.15.5:0/3938055882 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55824948ab40 0x558249497cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:27.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.393+0000 7f450ffff640 1 -- 172.21.15.5:0/3938055882 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5582494895d0 msgr2=0x55824949c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:27.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.393+0000 7f450ffff640 1 --2- 172.21.15.5:0/3938055882 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5582494895d0 0x55824949c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:27.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.393+0000 7f450ffff640 1 -- 172.21.15.5:0/3938055882 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55824948ab40 msgr2=0x558249497cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:27.407 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.393+0000 7f450ffff640 1 --2- 172.21.15.5:0/3938055882 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55824948ab40 0x558249497cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:27.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.393+0000 7f450ffff640 1 -- 172.21.15.5:0/3938055882 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558249498820 con 0x558249494370 2024-09-12T03:04:27.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.393+0000 7f450ffff640 1 --2- 172.21.15.5:0/3938055882 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558249494370 0x558249498230 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f4500012d50 tx=0x7f4500015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2db0404ce5daaf6c server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:27.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.393+0000 7f44effff640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T03:04:27.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.393+0000 7f44effff640 1 -- 172.21.15.5:0/3938055882 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4500004c50 con 0x558249494370 2024-09-12T03:04:27.408 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.393+0000 7f44effff640 1 -- 172.21.15.5:0/3938055882 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f450001a070 con 0x558249494370 2024-09-12T03:04:27.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.393+0000 7f4519483740 1 -- 172.21.15.5:0/3938055882 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4500013620 con 0x558249494370 2024-09-12T03:04:27.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.393+0000 7f44effff640 1 -- 172.21.15.5:0/3938055882 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f4500013110 con 0x558249494370 2024-09-12T03:04:27.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.393+0000 7f4519483740 10 client.16830 fetch_fsmap learned FSMap version 33 2024-09-12T03:04:27.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.393+0000 7f4519483740 1 -- 172.21.15.5:0/3938055882 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55824950e6d0 con 0x558249494370 2024-09-12T03:04:27.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.394+0000 7f44effff640 1 -- 172.21.15.5:0/3938055882 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4500013620 con 0x558249494370 2024-09-12T03:04:27.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.394+0000 7f44effff640 1 -- 172.21.15.5:0/3938055882 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 33) ==== 8374+0+0 (secure 0 0 0) 0x7f4500005580 con 0x558249494370 2024-09-12T03:04:27.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.394+0000 7f4519483740 10 client.16830 fetch_fsmap finished waiting for FSMap version 33 2024-09-12T03:04:27.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.394+0000 7f4519483740 10 client.16830 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-09-12T03:04:27.409 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.394+0000 7f4519483740 20 client.16830 populate_metadata read hostname 'smithi005' 2024-09-12T03:04:27.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.394+0000 7f4519483740 1 --2- 172.21.15.5:0/3938055882 >> [v2:172.21.15.148:6836/2606873922,v1:172.21.15.148:6837/2606873922] conn(0x55824950ecd0 0x5582495110c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:27.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.394+0000 7f4519483740 4 client.16830 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:04:27.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.394+0000 7f4519483740 1 -- 172.21.15.5:0/3938055882 --> [v2:172.21.15.148:6836/2606873922,v1:172.21.15.148:6837/2606873922] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5582493f6d70 con 0x55824950ecd0 2024-09-12T03:04:27.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.394+0000 7f4514e46640 1 --2- 172.21.15.5:0/3938055882 >> [v2:172.21.15.148:6836/2606873922,v1:172.21.15.148:6837/2606873922] conn(0x55824950ecd0 0x5582495110c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:27.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.395+0000 7f4514e46640 1 --2- 172.21.15.5:0/3938055882 >> [v2:172.21.15.148:6836/2606873922,v1:172.21.15.148:6837/2606873922] conn(0x55824950ecd0 0x5582495110c0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=34b1daa03e4a0d3f server_cookie=34dd036960a2b285 in_seq=0 out_seq=0 2024-09-12T03:04:27.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.395+0000 7f44effff640 10 client.16830 ms_handle_connect on v2:172.21.15.148:6836/2606873922 2024-09-12T03:04:27.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.395+0000 7f44effff640 1 -- 172.21.15.5:0/3938055882 <== mds.1 v2:172.21.15.148:6836/2606873922 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5582493f6d70 con 0x55824950ecd0 2024-09-12T03:04:27.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.395+0000 7f44effff640 10 client.16830 handle_command_reply: tid=0 2024-09-12T03:04:27.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.395+0000 7f4519483740 1 client.16830 shutdown 2024-09-12T03:04:27.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.395+0000 7f4519483740 1 -- 172.21.15.5:0/3938055882 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558249494370 msgr2=0x558249498230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:27.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.395+0000 7f4519483740 1 --2- 172.21.15.5:0/3938055882 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558249494370 0x558249498230 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f4500012d50 tx=0x7f4500015040 comp rx=0 tx=0).stop 2024-09-12T03:04:27.410 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.395+0000 7f4519483740 1 -- 172.21.15.5:0/3938055882 shutdown_connections 2024-09-12T03:04:27.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.395+0000 7f4519483740 1 --2- 172.21.15.5:0/3938055882 >> [v2:172.21.15.148:6836/2606873922,v1:172.21.15.148:6837/2606873922] conn(0x55824950ecd0 0x5582495110c0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:27.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.395+0000 7f44effff640 0 client.16830 ms_handle_reset on v2:172.21.15.148:6836/2606873922 2024-09-12T03:04:27.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.395+0000 7f4519483740 1 --2- 172.21.15.5:0/3938055882 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558249494370 0x558249498230 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:27.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.395+0000 7f4519483740 1 --2- 172.21.15.5:0/3938055882 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x55824948ab40 0x558249497cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:27.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.395+0000 7f4519483740 1 --2- 172.21.15.5:0/3938055882 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x5582494895d0 0x55824949c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:27.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.395+0000 7f4519483740 1 -- 172.21.15.5:0/3938055882 >> 172.21.15.5:0/3938055882 conn(0x558249452310 msgr2=0x5582494844e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:27.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.395+0000 7f4519483740 1 -- 172.21.15.5:0/3938055882 shutdown_connections 2024-09-12T03:04:27.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.395+0000 7f4519483740 1 -- 172.21.15.5:0/3938055882 wait complete. 2024-09-12T03:04:27.411 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.395+0000 7f4519483740 20 client.16830 trim_cache size 0 max 16384 2024-09-12T03:04:27.411 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-12T03:04:27.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.396+0000 7f45170aa640 1 -- 172.21.15.5:0/3366961776 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f44fc0a2b80 msgr2=0x7f44fc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:27.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.396+0000 7f45170aa640 1 --2- 172.21.15.5:0/3366961776 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f44fc0a2b80 0x7f44fc0a5040 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f45080b81a0 tx=0x7f450808acc0 comp rx=0 tx=0).stop 2024-09-12T03:04:27.412 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.396+0000 7f45170aa640 1 -- 172.21.15.5:0/3366961776 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4510106a80 msgr2=0x7f451019bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:27.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.396+0000 7f45170aa640 1 --2- 172.21.15.5:0/3366961776 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4510106a80 0x7f451019bb90 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f4508005ab0 tx=0x7f45080042f0 comp rx=0 tx=0).stop 2024-09-12T03:04:27.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.396+0000 7f45170aa640 1 -- 172.21.15.5:0/3366961776 shutdown_connections 2024-09-12T03:04:27.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.396+0000 7f45170aa640 1 --2- 172.21.15.5:0/3366961776 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f44fc0a2b80 0x7f44fc0a5040 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:27.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.396+0000 7f45170aa640 1 --2- 172.21.15.5:0/3366961776 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4510106a80 0x7f451019bb90 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:27.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.396+0000 7f45170aa640 1 --2- 172.21.15.5:0/3366961776 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f4510101420 0x7f4510197800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:27.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.396+0000 7f45170aa640 1 --2- 172.21.15.5:0/3366961776 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f4510100a80 0x7f45101972c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:27.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.396+0000 7f45170aa640 1 -- 172.21.15.5:0/3366961776 >> 172.21.15.5:0/3366961776 conn(0x7f45100fc7a0 msgr2=0x7f45100fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:27.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.396+0000 7f45170aa640 1 -- 172.21.15.5:0/3366961776 shutdown_connections 2024-09-12T03:04:27.413 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:27.396+0000 7f45170aa640 1 -- 172.21.15.5:0/3366961776 wait complete. 2024-09-12T03:04:28.106 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-12T03:04:28.364 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:04:28.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.831+0000 7f51cfd4f640 1 -- 172.21.15.5:0/1930038833 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51c8102ef0 msgr2=0x7f51c810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:28.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.831+0000 7f51cfd4f640 1 --2- 172.21.15.5:0/1930038833 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51c8102ef0 0x7f51c810da40 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f51b8009900 tx=0x7f51b802a480 comp rx=0 tx=0).stop 2024-09-12T03:04:28.833 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.831+0000 7f51cfd4f640 1 -- 172.21.15.5:0/1930038833 shutdown_connections 2024-09-12T03:04:28.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.831+0000 7f51cfd4f640 1 --2- 172.21.15.5:0/1930038833 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51c8102ef0 0x7f51c810da40 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:28.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.831+0000 7f51cfd4f640 1 --2- 172.21.15.5:0/1930038833 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f51c8102570 0x7f51c81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:28.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.831+0000 7f51cfd4f640 1 --2- 172.21.15.5:0/1930038833 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51c8100e20 0x7f51c8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:28.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.831+0000 7f51cfd4f640 1 -- 172.21.15.5:0/1930038833 >> 172.21.15.5:0/1930038833 conn(0x7f51c80fc7d0 msgr2=0x7f51c80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:28.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.831+0000 7f51cfd4f640 1 -- 172.21.15.5:0/1930038833 shutdown_connections 2024-09-12T03:04:28.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.831+0000 7f51cfd4f640 1 -- 172.21.15.5:0/1930038833 wait complete. 2024-09-12T03:04:28.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.832+0000 7f51cfd4f640 1 Processor -- start 2024-09-12T03:04:28.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.832+0000 7f51cfd4f640 1 -- start start 2024-09-12T03:04:28.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.832+0000 7f51cfd4f640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51c8100e20 0x7f51c819a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:28.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.832+0000 7f51cfd4f640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51c8102570 0x7f51c819aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:28.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.832+0000 7f51cfd4f640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f51c8102ef0 0x7f51c819edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:28.834 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.832+0000 7f51cfd4f640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f51c8110f80 con 0x7f51c8100e20 2024-09-12T03:04:28.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.832+0000 7f51cfd4f640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f51c8110e00 con 0x7f51c8102ef0 2024-09-12T03:04:28.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.832+0000 7f51cfd4f640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f51c8111100 con 0x7f51c8102570 2024-09-12T03:04:28.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.832+0000 7f51cdaeb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51c8100e20 0x7f51c819a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:28.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.832+0000 7f51cdaeb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51c8100e20 0x7f51c819a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53492/0 (socket says 172.21.15.5:53492) 2024-09-12T03:04:28.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.832+0000 7f51cdaeb640 1 -- 172.21.15.5:0/2128317287 learned_addr learned my addr 172.21.15.5:0/2128317287 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:04:28.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.832+0000 7f51cd2ea640 1 --2- 172.21.15.5:0/2128317287 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51c8102570 0x7f51c819aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:28.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.832+0000 7f51cdaeb640 1 -- 172.21.15.5:0/2128317287 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51c8102570 msgr2=0x7f51c819aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:28.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.832+0000 7f51cdaeb640 1 --2- 172.21.15.5:0/2128317287 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51c8102570 0x7f51c819aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:28.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.832+0000 7f51ce2ec640 1 --2- 172.21.15.5:0/2128317287 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f51c8102ef0 0x7f51c819edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:28.835 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.832+0000 7f51cdaeb640 1 -- 172.21.15.5:0/2128317287 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f51c8102ef0 msgr2=0x7f51c819edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:28.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.832+0000 7f51cdaeb640 1 --2- 172.21.15.5:0/2128317287 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f51c8102ef0 0x7f51c819edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:28.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.832+0000 7f51cdaeb640 1 -- 172.21.15.5:0/2128317287 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51c819f4d0 con 0x7f51c8100e20 2024-09-12T03:04:28.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.833+0000 7f51cdaeb640 1 --2- 172.21.15.5:0/2128317287 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51c8100e20 0x7f51c819a520 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f51c000cce0 tx=0x7f51c000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:28.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.833+0000 7f51c6ffd640 1 -- 172.21.15.5:0/2128317287 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51c000f650 con 0x7f51c8100e20 2024-09-12T03:04:28.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.833+0000 7f51c6ffd640 1 -- 172.21.15.5:0/2128317287 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51c000fcd0 con 0x7f51c8100e20 2024-09-12T03:04:28.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.833+0000 7f51cfd4f640 1 -- 172.21.15.5:0/2128317287 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51c819f7c0 con 0x7f51c8100e20 2024-09-12T03:04:28.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.833+0000 7f51c6ffd640 1 -- 172.21.15.5:0/2128317287 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51c0018c70 con 0x7f51c8100e20 2024-09-12T03:04:28.836 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.833+0000 7f51cfd4f640 1 -- 172.21.15.5:0/2128317287 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51c81a70a0 con 0x7f51c8100e20 2024-09-12T03:04:28.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.835+0000 7f51c6ffd640 1 -- 172.21.15.5:0/2128317287 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f51c000f7f0 con 0x7f51c8100e20 2024-09-12T03:04:28.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.835+0000 7f51d2128740 1 Processor -- start 2024-09-12T03:04:28.838 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.836+0000 7f51d2128740 1 -- start start 2024-09-12T03:04:28.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.836+0000 7f51d2128740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555db39a3b40 0x555db39a2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:28.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.836+0000 7f51d2128740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555db39a25d0 0x555db39a2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:28.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.836+0000 7f51d2128740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555db39ad370 0x555db39af760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:28.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.836+0000 7f51d2128740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555db39082f0 con 0x555db39ad370 2024-09-12T03:04:28.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.836+0000 7f51d2128740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x555db38afcf0 con 0x555db39a3b40 2024-09-12T03:04:28.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.836+0000 7f51d2128740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x555db39042b0 con 0x555db39a25d0 2024-09-12T03:04:28.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.836+0000 7f51cd2ea640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555db39ad370 0x555db39af760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:28.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.836+0000 7f51cdaeb640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555db39a25d0 0x555db39a2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:28.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.836+0000 7f51cd2ea640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555db39ad370 0x555db39af760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53500/0 (socket says 172.21.15.5:53500) 2024-09-12T03:04:28.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.836+0000 7f51cd2ea640 1 -- 172.21.15.5:0/1068980043 learned_addr learned my addr 172.21.15.5:0/1068980043 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:04:28.839 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.836+0000 7f51c6ffd640 1 --2- 172.21.15.5:0/2128317287 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f51b00a2c00 0x7f51b00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:28.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.836+0000 7f51ce2ec640 1 --2- 172.21.15.5:0/1068980043 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555db39a3b40 0x555db39a2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:28.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.837+0000 7f51cd2ea640 1 -- 172.21.15.5:0/1068980043 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555db39a25d0 msgr2=0x555db39a2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:28.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.837+0000 7f51cd2ea640 1 --2- 172.21.15.5:0/1068980043 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555db39a25d0 0x555db39a2a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:28.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.837+0000 7f51cd2ea640 1 -- 172.21.15.5:0/1068980043 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555db39a3b40 msgr2=0x555db39a2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:28.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.837+0000 7f51ce2ec640 1 --2- 172.21.15.5:0/2128317287 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f51b00a2c00 0x7f51b00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:28.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.837+0000 7f51cd2ea640 1 --2- 172.21.15.5:0/1068980043 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555db39a3b40 0x555db39a2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:28.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.837+0000 7f51cd2ea640 1 -- 172.21.15.5:0/1068980043 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555db38943a0 con 0x555db39ad370 2024-09-12T03:04:28.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.837+0000 7f51c6ffd640 1 -- 172.21.15.5:0/2128317287 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f51c00cf5c0 con 0x7f51c8100e20 2024-09-12T03:04:28.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.837+0000 7f51cd2ea640 1 --2- 172.21.15.5:0/1068980043 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555db39ad370 0x555db39af760 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f51bc00cc40 tx=0x7f51bc00e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fffd2c508cf149a server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:28.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.837+0000 7f51affff640 1 -- 172.21.15.5:0/1068980043 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51bc014070 con 0x555db39ad370 2024-09-12T03:04:28.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.837+0000 7f51ce2ec640 1 --2- 172.21.15.5:0/2128317287 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f51b00a2c00 0x7f51b00a50c0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f51b8009230 tx=0x7f51b802a480 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:28.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.837+0000 7f51affff640 1 -- 172.21.15.5:0/1068980043 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51bc0047a0 con 0x555db39ad370 2024-09-12T03:04:28.840 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.837+0000 7f51affff640 1 -- 172.21.15.5:0/1068980043 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51bc010720 con 0x555db39ad370 2024-09-12T03:04:28.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.838+0000 7f51d2128740 1 -- 172.21.15.5:0/1068980043 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555db39ad370 msgr2=0x555db39af760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:28.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.838+0000 7f51d2128740 1 --2- 172.21.15.5:0/1068980043 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555db39ad370 0x555db39af760 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f51bc00cc40 tx=0x7f51bc00e720 comp rx=0 tx=0).stop 2024-09-12T03:04:28.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.838+0000 7f51d2128740 1 -- 172.21.15.5:0/1068980043 shutdown_connections 2024-09-12T03:04:28.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.838+0000 7f51d2128740 1 --2- 172.21.15.5:0/1068980043 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555db39ad370 0x555db39af760 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:28.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.838+0000 7f51d2128740 1 --2- 172.21.15.5:0/1068980043 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555db39a25d0 0x555db39a2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:28.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.838+0000 7f51d2128740 1 --2- 172.21.15.5:0/1068980043 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555db39a3b40 0x555db39a2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:28.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.838+0000 7f51d2128740 1 -- 172.21.15.5:0/1068980043 >> 172.21.15.5:0/1068980043 conn(0x555db396b310 msgr2=0x555db399ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:28.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.838+0000 7f51d2128740 1 -- 172.21.15.5:0/1068980043 shutdown_connections 2024-09-12T03:04:28.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.838+0000 7f51d2128740 1 -- 172.21.15.5:0/1068980043 wait complete. 2024-09-12T03:04:28.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.838+0000 7f51d2128740 1 -- start start 2024-09-12T03:04:28.841 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.838+0000 7f51d2128740 1 Processor -- start 2024-09-12T03:04:28.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.839+0000 7f51d2128740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555db39a25d0 0x555db39acd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:28.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.839+0000 7f51d2128740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555db39a3b40 0x555db39a9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:28.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.839+0000 7f51d2128740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555db39ad370 0x555db39aa5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:28.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.839+0000 7f51d2128740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x555db39b0070 con 0x555db39a25d0 2024-09-12T03:04:28.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.839+0000 7f51d2128740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x555db38c3060 con 0x555db39a3b40 2024-09-12T03:04:28.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.839+0000 7f51d2128740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x555db39b01f0 con 0x555db39ad370 2024-09-12T03:04:28.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.839+0000 7f51d2128740 10 client.0 _ll_register_callbacks cb 0x555db36168c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:04:28.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.839+0000 7f51cdaeb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555db39a25d0 0x555db39acd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:28.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.839+0000 7f51cdaeb640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555db39a25d0 0x555db39acd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53516/0 (socket says 172.21.15.5:53516) 2024-09-12T03:04:28.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.839+0000 7f51cdaeb640 1 -- 172.21.15.5:0/119121249 learned_addr learned my addr 172.21.15.5:0/119121249 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:04:28.845 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.839+0000 7f51cd2ea640 1 --2- 172.21.15.5:0/119121249 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555db39a3b40 0x555db39a9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:28.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.839+0000 7f51cdaeb640 1 -- 172.21.15.5:0/119121249 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555db39ad370 msgr2=0x555db39aa5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:28.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.839+0000 7f51cdaeb640 1 --2- 172.21.15.5:0/119121249 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555db39ad370 0x555db39aa5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:28.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.839+0000 7f51cdaeb640 1 -- 172.21.15.5:0/119121249 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555db39a3b40 msgr2=0x555db39a9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:28.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.839+0000 7f51cdaeb640 1 --2- 172.21.15.5:0/119121249 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555db39a3b40 0x555db39a9ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:28.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.839+0000 7f51cdaeb640 1 -- 172.21.15.5:0/119121249 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555db36793c0 con 0x555db39a25d0 2024-09-12T03:04:28.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.839+0000 7f51cd2ea640 1 --2- 172.21.15.5:0/119121249 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555db39a3b40 0x555db39a9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:04:28.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.840+0000 7f51cdaeb640 1 --2- 172.21.15.5:0/119121249 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555db39a25d0 0x555db39acd40 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f51c000ab20 tx=0x7f51c009bca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ffadc6cfd50d725 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:28.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.840+0000 7f51ad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T03:04:28.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.840+0000 7f51ad7fa640 1 -- 172.21.15.5:0/119121249 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51c009b910 con 0x555db39a25d0 2024-09-12T03:04:28.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.840+0000 7f51ad7fa640 1 -- 172.21.15.5:0/119121249 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f51c009c230 con 0x555db39a25d0 2024-09-12T03:04:28.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.840+0000 7f51d2128740 1 -- 172.21.15.5:0/119121249 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51bc0047a0 con 0x555db39a25d0 2024-09-12T03:04:28.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.840+0000 7f51ad7fa640 1 -- 172.21.15.5:0/119121249 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f51c009b730 con 0x555db39a25d0 2024-09-12T03:04:28.846 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.840+0000 7f51d2128740 10 client.16842 fetch_fsmap learned FSMap version 33 2024-09-12T03:04:28.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.840+0000 7f51d2128740 1 -- 172.21.15.5:0/119121249 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x555db39bbb70 con 0x555db39a25d0 2024-09-12T03:04:28.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.840+0000 7f51ad7fa640 1 -- 172.21.15.5:0/119121249 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51bc0047a0 con 0x555db39a25d0 2024-09-12T03:04:28.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.840+0000 7f51ad7fa640 1 -- 172.21.15.5:0/119121249 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 33) ==== 8374+0+0 (secure 0 0 0) 0x7f51c00b1070 con 0x555db39a25d0 2024-09-12T03:04:28.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.840+0000 7f51d2128740 10 client.16842 fetch_fsmap finished waiting for FSMap version 33 2024-09-12T03:04:28.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.840+0000 7f51d2128740 10 client.16842 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:04:28.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.840+0000 7f51d2128740 20 client.16842 populate_metadata read hostname 'smithi005' 2024-09-12T03:04:28.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.840+0000 7f51d2128740 1 --2- 172.21.15.5:0/119121249 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x555db39bcf80 0x555db39bf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:28.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.840+0000 7f51d2128740 4 client.16842 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-12T03:04:28.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.840+0000 7f51d2128740 1 -- 172.21.15.5:0/119121249 --> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555db3915860 con 0x555db39bcf80 2024-09-12T03:04:28.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.841+0000 7f51cd2ea640 1 --2- 172.21.15.5:0/119121249 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x555db39bcf80 0x555db39bf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:28.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.841+0000 7f51cd2ea640 1 --2- 172.21.15.5:0/119121249 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x555db39bcf80 0x555db39bf3f0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa2b0883d5cfd4e7 server_cookie=a4f7fc2d95bce40f in_seq=0 out_seq=0 2024-09-12T03:04:28.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.841+0000 7f51ad7fa640 10 client.16842 ms_handle_connect on v2:172.21.15.148:6838/2317600000 2024-09-12T03:04:28.847 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.842+0000 7f51ad7fa640 1 -- 172.21.15.5:0/119121249 <== mds.0 v2:172.21.15.148:6838/2317600000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555db3915860 con 0x555db39bcf80 2024-09-12T03:04:28.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.842+0000 7f51ad7fa640 10 client.16842 handle_command_reply: tid=0 2024-09-12T03:04:28.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.842+0000 7f51d2128740 1 client.16842 shutdown 2024-09-12T03:04:28.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.842+0000 7f51d2128740 1 -- 172.21.15.5:0/119121249 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555db39a25d0 msgr2=0x555db39acd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:28.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.842+0000 7f51d2128740 1 --2- 172.21.15.5:0/119121249 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555db39a25d0 0x555db39acd40 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f51c000ab20 tx=0x7f51c009bca0 comp rx=0 tx=0).stop 2024-09-12T03:04:28.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.842+0000 7f51d2128740 1 -- 172.21.15.5:0/119121249 shutdown_connections 2024-09-12T03:04:28.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.842+0000 7f51d2128740 1 --2- 172.21.15.5:0/119121249 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555db39ad370 0x555db39aa5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:28.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.842+0000 7f51d2128740 1 --2- 172.21.15.5:0/119121249 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555db39a3b40 0x555db39a9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:28.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.842+0000 7f51d2128740 1 --2- 172.21.15.5:0/119121249 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x555db39bcf80 0x555db39bf3f0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:28.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.842+0000 7f51d2128740 1 --2- 172.21.15.5:0/119121249 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555db39a25d0 0x555db39acd40 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:28.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.842+0000 7f51d2128740 1 -- 172.21.15.5:0/119121249 >> 172.21.15.5:0/119121249 conn(0x555db396b310 msgr2=0x555db39a4960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:28.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.842+0000 7f51ad7fa640 0 client.16842 ms_handle_reset on v2:172.21.15.148:6838/2317600000 2024-09-12T03:04:28.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.842+0000 7f51d2128740 1 -- 172.21.15.5:0/119121249 shutdown_connections 2024-09-12T03:04:28.848 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.842+0000 7f51d2128740 1 -- 172.21.15.5:0/119121249 wait complete. 2024-09-12T03:04:28.849 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.842+0000 7f51d2128740 20 client.16842 trim_cache size 0 max 16384 2024-09-12T03:04:28.857 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.855+0000 7f51d2128740 1 Processor -- start 2024-09-12T03:04:28.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.856+0000 7f51d2128740 1 -- start start 2024-09-12T03:04:28.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.856+0000 7f51d2128740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555db39a25d0 0x555db39a5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:28.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.856+0000 7f51d2128740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555db39a3b40 0x555db39a5660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:28.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.856+0000 7f51d2128740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555db39ad370 0x555db39c4c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:28.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.856+0000 7f51d2128740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f51c81af550 con 0x555db39ad370 2024-09-12T03:04:28.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.856+0000 7f51d2128740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f51c81b1120 con 0x555db39a25d0 2024-09-12T03:04:28.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.856+0000 7f51d2128740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f51c81afc50 con 0x555db39a3b40 2024-09-12T03:04:28.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.856+0000 7f51ce2ec640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555db39ad370 0x555db39c4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:28.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.856+0000 7f51ce2ec640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555db39ad370 0x555db39c4c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53524/0 (socket says 172.21.15.5:53524) 2024-09-12T03:04:28.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.856+0000 7f51ce2ec640 1 -- 172.21.15.5:0/423672437 learned_addr learned my addr 172.21.15.5:0/423672437 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:04:28.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.856+0000 7f51cd2ea640 1 --2- 172.21.15.5:0/423672437 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555db39a3b40 0x555db39a5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:28.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.856+0000 7f51cdaeb640 1 --2- 172.21.15.5:0/423672437 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555db39a25d0 0x555db39a5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:28.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.856+0000 7f51ce2ec640 1 -- 172.21.15.5:0/423672437 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555db39a3b40 msgr2=0x555db39a5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:28.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.856+0000 7f51ce2ec640 1 --2- 172.21.15.5:0/423672437 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555db39a3b40 0x555db39a5660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:28.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.856+0000 7f51ce2ec640 1 -- 172.21.15.5:0/423672437 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555db39a25d0 msgr2=0x555db39a5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:28.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.856+0000 7f51ce2ec640 1 --2- 172.21.15.5:0/423672437 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555db39a25d0 0x555db39a5120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:28.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.856+0000 7f51ce2ec640 1 -- 172.21.15.5:0/423672437 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555db39aa950 con 0x555db39ad370 2024-09-12T03:04:28.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.856+0000 7f51ce2ec640 1 --2- 172.21.15.5:0/423672437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555db39ad370 0x555db39c4c90 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f51b80052d0 tx=0x7f51b8009660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99534203ecdb89f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:28.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.857+0000 7f51af7fe640 1 -- 172.21.15.5:0/423672437 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51b804f070 con 0x555db39ad370 2024-09-12T03:04:28.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.857+0000 7f51af7fe640 1 -- 172.21.15.5:0/423672437 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51b80035f0 con 0x555db39ad370 2024-09-12T03:04:28.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.857+0000 7f51af7fe640 1 -- 172.21.15.5:0/423672437 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51b8035070 con 0x555db39ad370 2024-09-12T03:04:28.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.857+0000 7f51d2128740 1 -- 172.21.15.5:0/423672437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555db39ad370 msgr2=0x555db39c4c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:28.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.857+0000 7f51d2128740 1 --2- 172.21.15.5:0/423672437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555db39ad370 0x555db39c4c90 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f51b80052d0 tx=0x7f51b8009660 comp rx=0 tx=0).stop 2024-09-12T03:04:28.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.857+0000 7f51d2128740 1 -- 172.21.15.5:0/423672437 shutdown_connections 2024-09-12T03:04:28.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.857+0000 7f51d2128740 1 --2- 172.21.15.5:0/423672437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555db39ad370 0x555db39c4c90 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:28.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.857+0000 7f51d2128740 1 --2- 172.21.15.5:0/423672437 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555db39a3b40 0x555db39a5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:28.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.857+0000 7f51d2128740 1 --2- 172.21.15.5:0/423672437 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555db39a25d0 0x555db39a5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:28.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.857+0000 7f51d2128740 1 -- 172.21.15.5:0/423672437 >> 172.21.15.5:0/423672437 conn(0x555db396b310 msgr2=0x555db399dd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:28.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.857+0000 7f51d2128740 1 -- 172.21.15.5:0/423672437 shutdown_connections 2024-09-12T03:04:28.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.857+0000 7f51d2128740 1 -- 172.21.15.5:0/423672437 wait complete. 2024-09-12T03:04:28.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.858+0000 7f51d2128740 1 -- start start 2024-09-12T03:04:28.860 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.858+0000 7f51d2128740 1 Processor -- start 2024-09-12T03:04:28.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.858+0000 7f51d2128740 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555db39a25d0 0x555db39b54b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:28.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.858+0000 7f51d2128740 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555db39a3b40 0x555db39b59f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:28.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.858+0000 7f51d2128740 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555db39ad370 0x555db39b2480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:28.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.858+0000 7f51d2128740 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f51c81b0a50 con 0x555db39a3b40 2024-09-12T03:04:28.863 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.858+0000 7f51d2128740 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f51c81b0350 con 0x555db39a25d0 2024-09-12T03:04:28.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.858+0000 7f51d2128740 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f51c81ad530 con 0x555db39ad370 2024-09-12T03:04:28.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.858+0000 7f51d2128740 10 client.0 _ll_register_callbacks cb 0x7f51c00185b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-12T03:04:28.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.858+0000 7f51cd2ea640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555db39a3b40 0x555db39b59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:28.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.858+0000 7f51cd2ea640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555db39a3b40 0x555db39b59f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53526/0 (socket says 172.21.15.5:53526) 2024-09-12T03:04:28.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.858+0000 7f51cd2ea640 1 -- 172.21.15.5:0/1262164284 learned_addr learned my addr 172.21.15.5:0/1262164284 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:04:28.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.858+0000 7f51ce2ec640 1 --2- 172.21.15.5:0/1262164284 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555db39ad370 0x555db39b2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:28.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.858+0000 7f51cdaeb640 1 --2- 172.21.15.5:0/1262164284 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555db39a25d0 0x555db39b54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:28.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.859+0000 7f51cd2ea640 1 -- 172.21.15.5:0/1262164284 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555db39ad370 msgr2=0x555db39b2480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:28.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.859+0000 7f51cd2ea640 1 --2- 172.21.15.5:0/1262164284 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555db39ad370 0x555db39b2480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:28.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.859+0000 7f51cd2ea640 1 -- 172.21.15.5:0/1262164284 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555db39a25d0 msgr2=0x555db39b54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-12T03:04:28.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.859+0000 7f51cd2ea640 1 --2- 172.21.15.5:0/1262164284 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555db39a25d0 0x555db39b54b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:28.864 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.859+0000 7f51cd2ea640 1 -- 172.21.15.5:0/1262164284 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555db39b29c0 con 0x555db39a3b40 2024-09-12T03:04:28.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.859+0000 7f51cd2ea640 1 --2- 172.21.15.5:0/1262164284 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555db39a3b40 0x555db39b59f0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f51bc00cb60 tx=0x7f51bc00c350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7d35c29cb44ebe4 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:28.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.859+0000 7f51ad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2024-09-12T03:04:28.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.859+0000 7f51ad7fa640 1 -- 172.21.15.5:0/1262164284 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51bc014070 con 0x555db39a3b40 2024-09-12T03:04:28.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.859+0000 7f51ad7fa640 1 -- 172.21.15.5:0/1262164284 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f51bc005240 con 0x555db39a3b40 2024-09-12T03:04:28.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.859+0000 7f51d2128740 1 -- 172.21.15.5:0/1262164284 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51b80035f0 con 0x555db39a3b40 2024-09-12T03:04:28.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.859+0000 7f51ad7fa640 1 -- 172.21.15.5:0/1262164284 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f51bc005440 con 0x555db39a3b40 2024-09-12T03:04:28.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.859+0000 7f51d2128740 10 client.16848 fetch_fsmap learned FSMap version 33 2024-09-12T03:04:28.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.859+0000 7f51d2128740 1 -- 172.21.15.5:0/1262164284 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x555db3a282d0 con 0x555db39a3b40 2024-09-12T03:04:28.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.860+0000 7f51ad7fa640 1 -- 172.21.15.5:0/1262164284 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51b80035f0 con 0x555db39a3b40 2024-09-12T03:04:28.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.860+0000 7f51ad7fa640 1 -- 172.21.15.5:0/1262164284 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 33) ==== 8374+0+0 (secure 0 0 0) 0x7f51bc004980 con 0x555db39a3b40 2024-09-12T03:04:28.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.860+0000 7f51d2128740 10 client.16848 fetch_fsmap finished waiting for FSMap version 33 2024-09-12T03:04:28.865 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.860+0000 7f51d2128740 10 client.16848 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-09-12T03:04:28.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.860+0000 7f51d2128740 20 client.16848 populate_metadata read hostname 'smithi005' 2024-09-12T03:04:28.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.860+0000 7f51d2128740 1 --2- 172.21.15.5:0/1262164284 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x555db3a28820 0x555db3a2ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:28.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.860+0000 7f51d2128740 4 client.16848 mds_command: new command op to 24502 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-12T03:04:28.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.860+0000 7f51d2128740 1 -- 172.21.15.5:0/1262164284 --> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555db390fd70 con 0x555db3a28820 2024-09-12T03:04:28.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.860+0000 7f51cdaeb640 1 --2- 172.21.15.5:0/1262164284 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x555db3a28820 0x555db3a2ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:28.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.861+0000 7f51cdaeb640 1 --2- 172.21.15.5:0/1262164284 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x555db3a28820 0x555db3a2ac40 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3bcfe5652c7c007e server_cookie=e92c50dd14370a6 in_seq=0 out_seq=0 2024-09-12T03:04:28.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.861+0000 7f51ad7fa640 10 client.16848 ms_handle_connect on v2:172.21.15.148:6838/2317600000 2024-09-12T03:04:28.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.861+0000 7f51ad7fa640 1 -- 172.21.15.5:0/1262164284 <== mds.0 v2:172.21.15.148:6838/2317600000 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555db390fd70 con 0x555db3a28820 2024-09-12T03:04:28.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.861+0000 7f51ad7fa640 10 client.16848 handle_command_reply: tid=0 2024-09-12T03:04:28.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.861+0000 7f51d2128740 1 client.16848 shutdown 2024-09-12T03:04:28.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.861+0000 7f51d2128740 1 -- 172.21.15.5:0/1262164284 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555db39a3b40 msgr2=0x555db39b59f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:28.866 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.861+0000 7f51d2128740 1 --2- 172.21.15.5:0/1262164284 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555db39a3b40 0x555db39b59f0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f51bc00cb60 tx=0x7f51bc00c350 comp rx=0 tx=0).stop 2024-09-12T03:04:28.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.861+0000 7f51d2128740 1 -- 172.21.15.5:0/1262164284 shutdown_connections 2024-09-12T03:04:28.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.861+0000 7f51d2128740 1 --2- 172.21.15.5:0/1262164284 >> [v2:172.21.15.148:6838/2317600000,v1:172.21.15.148:6839/2317600000] conn(0x555db3a28820 0x555db3a2ac40 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:28.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.861+0000 7f51d2128740 1 --2- 172.21.15.5:0/1262164284 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555db39a3b40 0x555db39b59f0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:28.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.861+0000 7f51d2128740 1 --2- 172.21.15.5:0/1262164284 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x555db39ad370 0x555db39b2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:28.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.861+0000 7f51ad7fa640 0 client.16848 ms_handle_reset on v2:172.21.15.148:6838/2317600000 2024-09-12T03:04:28.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.861+0000 7f51d2128740 1 --2- 172.21.15.5:0/1262164284 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x555db39a25d0 0x555db39b54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:28.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.861+0000 7f51d2128740 1 -- 172.21.15.5:0/1262164284 >> 172.21.15.5:0/1262164284 conn(0x555db396b310 msgr2=0x555db39ab850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:28.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.862+0000 7f51d2128740 1 -- 172.21.15.5:0/1262164284 shutdown_connections 2024-09-12T03:04:28.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.862+0000 7f51d2128740 1 -- 172.21.15.5:0/1262164284 wait complete. 2024-09-12T03:04:28.867 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.862+0000 7f51d2128740 20 client.16848 trim_cache size 0 max 16384 2024-09-12T03:04:28.868 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2024-09-12T03:04:28.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.862+0000 7f51cfd4f640 1 -- 172.21.15.5:0/2128317287 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f51b00a2c00 msgr2=0x7f51b00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:28.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.862+0000 7f51cfd4f640 1 --2- 172.21.15.5:0/2128317287 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f51b00a2c00 0x7f51b00a50c0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f51b8009230 tx=0x7f51b802a480 comp rx=0 tx=0).stop 2024-09-12T03:04:28.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.862+0000 7f51cfd4f640 1 -- 172.21.15.5:0/2128317287 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51c8100e20 msgr2=0x7f51c819a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:28.868 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.862+0000 7f51cfd4f640 1 --2- 172.21.15.5:0/2128317287 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51c8100e20 0x7f51c819a520 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f51c000cce0 tx=0x7f51c000a620 comp rx=0 tx=0).stop 2024-09-12T03:04:28.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.862+0000 7f51cfd4f640 1 -- 172.21.15.5:0/2128317287 shutdown_connections 2024-09-12T03:04:28.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.862+0000 7f51cfd4f640 1 --2- 172.21.15.5:0/2128317287 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f51c8102ef0 0x7f51c819edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:28.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.862+0000 7f51cfd4f640 1 --2- 172.21.15.5:0/2128317287 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f51c8102570 0x7f51c819aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:28.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.862+0000 7f51cfd4f640 1 --2- 172.21.15.5:0/2128317287 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f51b00a2c00 0x7f51b00a50c0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:28.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.862+0000 7f51cfd4f640 1 --2- 172.21.15.5:0/2128317287 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51c8100e20 0x7f51c819a520 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:28.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.862+0000 7f51cfd4f640 1 -- 172.21.15.5:0/2128317287 >> 172.21.15.5:0/2128317287 conn(0x7f51c80fc7d0 msgr2=0x7f51c806a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:28.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.862+0000 7f51cfd4f640 1 -- 172.21.15.5:0/2128317287 shutdown_connections 2024-09-12T03:04:28.869 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:28.862+0000 7f51cfd4f640 1 -- 172.21.15.5:0/2128317287 wait complete. 2024-09-12T03:04:29.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:04:28 smithi005 ceph-mon[31197]: pgmap v522: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-12T03:04:29.165 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:04:28 smithi148 ceph-mon[35090]: pgmap v522: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-12T03:04:29.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:04:28 smithi146 ceph-mon[35011]: pgmap v522: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-09-12T03:04:29.539 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa/tasks/fwd_scrub.py", line 49, in do_scrub self.sleep_unless_stopped(self.sleep_between_iterations) File "/home/teuthworker/src/git.ceph.com_ceph_086c737fa75c22d521a3932c59b31c5e1d1c9ef4/qa/tasks/thrasher.py", line 47, in sleep_unless_stopped raise self.Stopped() tasks.thrasher.ThrasherGreenlet.Stopped 2024-09-12T03:04:29.540 INFO:tasks.fwd_scrub:done joining 2024-09-12T03:04:29.541 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-09-12T03:04:29.552 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-12T03:04:29.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:29.667+0000 7f2344ca6640 1 -- 172.21.15.5:0/3385910863 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f23401077f0 msgr2=0x7f2340109c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:29.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:29.667+0000 7f2344ca6640 1 --2- 172.21.15.5:0/3385910863 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f23401077f0 0x7f2340109c60 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f2338009900 tx=0x7f233802a410 comp rx=0 tx=0).stop 2024-09-12T03:04:29.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:29.667+0000 7f2344ca6640 1 -- 172.21.15.5:0/3385910863 shutdown_connections 2024-09-12T03:04:29.675 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:29.667+0000 7f2344ca6640 1 --2- 172.21.15.5:0/3385910863 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f23401077f0 0x7f2340109c60 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:29.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:29.667+0000 7f2344ca6640 1 --2- 172.21.15.5:0/3385910863 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2340104ec0 0x7f23401072b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:29.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:29.667+0000 7f2344ca6640 1 --2- 172.21.15.5:0/3385910863 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f2340102590 0x7f2340104980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:29.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:29.667+0000 7f2344ca6640 1 -- 172.21.15.5:0/3385910863 >> 172.21.15.5:0/3385910863 conn(0x7f23400fc170 msgr2=0x7f23400fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:29.676 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:29.667+0000 7f2344ca6640 1 -- 172.21.15.5:0/3385910863 shutdown_connections 2024-09-12T03:04:29.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:29.667+0000 7f2344ca6640 1 -- 172.21.15.5:0/3385910863 wait complete. 2024-09-12T03:04:29.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:29.668+0000 7f2344ca6640 1 Processor -- start 2024-09-12T03:04:29.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:29.668+0000 7f2344ca6640 1 -- start start 2024-09-12T03:04:29.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:29.668+0000 7f2344ca6640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2340102590 0x7f2340197e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:29.677 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:29.668+0000 7f2344ca6640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2340104ec0 0x7f23401983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:29.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:29.668+0000 7f2344ca6640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f23401077f0 0x7f234019f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:29.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:29.668+0000 7f2344ca6640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f234010d7d0 con 0x7f2340102590 2024-09-12T03:04:29.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:29.668+0000 7f2344ca6640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f234010d650 con 0x7f23401077f0 2024-09-12T03:04:29.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:29.668+0000 7f2344ca6640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f234010d950 con 0x7f2340104ec0 2024-09-12T03:04:29.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:29.668+0000 7f233e59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2340102590 0x7f2340197e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:29.678 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:29.668+0000 7f233e59c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2340102590 0x7f2340197e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53554/0 (socket says 172.21.15.5:53554) 2024-09-12T03:04:29.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:29.668+0000 7f233e59c640 1 -- 172.21.15.5:0/106445109 learned_addr learned my addr 172.21.15.5:0/106445109 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:04:29.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:29.668+0000 7f233dd9b640 1 --2- 172.21.15.5:0/106445109 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2340104ec0 0x7f23401983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:29.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:29.669+0000 7f233ed9d640 1 --2- 172.21.15.5:0/106445109 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f23401077f0 0x7f234019f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:29.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:29.669+0000 7f233e59c640 1 -- 172.21.15.5:0/106445109 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2340104ec0 msgr2=0x7f23401983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:29.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:29.669+0000 7f233e59c640 1 --2- 172.21.15.5:0/106445109 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2340104ec0 0x7f23401983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:29.679 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:29.669+0000 7f233e59c640 1 -- 172.21.15.5:0/106445109 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f23401077f0 msgr2=0x7f234019f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:29.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:29.669+0000 7f233e59c640 1 --2- 172.21.15.5:0/106445109 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f23401077f0 0x7f234019f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:29.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:29.669+0000 7f233e59c640 1 -- 172.21.15.5:0/106445109 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f234019fab0 con 0x7f2340102590 2024-09-12T03:04:29.680 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:29.669+0000 7f233dd9b640 1 --2- 172.21.15.5:0/106445109 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2340104ec0 0x7f23401983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:04:29.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:29.669+0000 7f233ed9d640 1 --2- 172.21.15.5:0/106445109 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f23401077f0 0x7f234019f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-12T03:04:29.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:29.669+0000 7f233e59c640 1 --2- 172.21.15.5:0/106445109 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2340102590 0x7f2340197e90 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f233000eab0 tx=0x7f233000ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:29.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:29.669+0000 7f232f7fe640 1 -- 172.21.15.5:0/106445109 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f233000ce40 con 0x7f2340102590 2024-09-12T03:04:29.681 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:29.669+0000 7f232f7fe640 1 -- 172.21.15.5:0/106445109 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2330011950 con 0x7f2340102590 2024-09-12T03:04:29.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:29.669+0000 7f2344ca6640 1 -- 172.21.15.5:0/106445109 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f234019fda0 con 0x7f2340102590 2024-09-12T03:04:29.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:29.669+0000 7f232f7fe640 1 -- 172.21.15.5:0/106445109 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2330010ae0 con 0x7f2340102590 2024-09-12T03:04:29.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:29.669+0000 7f2344ca6640 1 -- 172.21.15.5:0/106445109 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23401a0210 con 0x7f2340102590 2024-09-12T03:04:29.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:29.670+0000 7f2344ca6640 1 -- 172.21.15.5:0/106445109 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23400fdc60 con 0x7f2340102590 2024-09-12T03:04:29.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:29.671+0000 7f232f7fe640 1 -- 172.21.15.5:0/106445109 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f2330011b00 con 0x7f2340102590 2024-09-12T03:04:29.682 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:29.672+0000 7f232f7fe640 1 --2- 172.21.15.5:0/106445109 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f23280a2b90 0x7f23280a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:29.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:29.672+0000 7f233dd9b640 1 --2- 172.21.15.5:0/106445109 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f23280a2b90 0x7f23280a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:29.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:29.672+0000 7f232f7fe640 1 -- 172.21.15.5:0/106445109 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f23300cf870 con 0x7f2340102590 2024-09-12T03:04:29.683 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:29.673+0000 7f233dd9b640 1 --2- 172.21.15.5:0/106445109 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f23280a2b90 0x7f23280a5050 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f23401993a0 tx=0x7f2334005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:29.684 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:29.676+0000 7f232f7fe640 1 -- 172.21.15.5:0/106445109 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2330016030 con 0x7f2340102590 2024-09-12T03:04:29.855 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:29.854+0000 7f2344ca6640 1 -- 172.21.15.5:0/106445109 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2340062ef0 con 0x7f2340102590 2024-09-12T03:04:29.856 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:29.856+0000 7f232f7fe640 1 -- 172.21.15.5:0/106445109 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+22656 (secure 0 0 0) 0x7f233009c780 con 0x7f2340102590 2024-09-12T03:04:29.856 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-12T03:04:29.856 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":83,"fsid":"618305f8-70b1-11ef-bcea-c7b262605968","created":"2024-09-12T02:48:42.352146+0000","modified":"2024-09-12T02:56:31.026079+0000","last_up_change":"2024-09-12T02:55:27.817161+0000","last_in_change":"2024-09-12T02:55:03.745281+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T02:51:47.970639+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T02:56:12.834271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T02:56:14.951437+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17da560a-6aa2-4f2c-a95f-67481dce4d3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":4248507573},{"type":"v1","addr":"172.21.15.5:6803","nonce":4248507573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":4248507573},{"type":"v1","addr":"172.21.15.5:6805","nonce":4248507573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":4248507573},{"type":"v1","addr":"172.21.15.5:6809","nonce":4248507573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":4248507573},{"type":"v1","addr":"172.21.15.5:6807","nonce":4248507573}]},"public_addr":"172.21.15.5:6803/4248507573","cluster_addr":"172.21.15.5:6805/4248507573","heartbeat_back_addr":"172.21.15.5:6809/4248507573","heartbeat_front_addr":"172.21.15.5:6807/4248507573","state":["exists","up"]},{"osd":1,"uuid":"a3237006-99af-4fc1-860c-d4917032647f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":1105990287},{"type":"v1","addr":"172.21.15.146:6801","nonce":1105990287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":1105990287},{"type":"v1","addr":"172.21.15.146:6803","nonce":1105990287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":1105990287},{"type":"v1","addr":"172.21.15.146:6807","nonce":1105990287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":1105990287},{"type":"v1","addr":"172.21.15.146:6805","nonce":1105990287}]},"public_addr":"172.21.15.146:6801/1105990287","cluster_addr":"172.21.15.146:6803/1105990287","heartbeat_back_addr":"172.21.15.146:6807/1105990287","heartbeat_front_addr":"172.21.15.146:6805/1105990287","state":["exists","up"]},{"osd":2,"uuid":"ea3ab84a-7739-4975-8244-31ddd5aad80d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3378174313},{"type":"v1","addr":"172.21.15.148:6801","nonce":3378174313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3378174313},{"type":"v1","addr":"172.21.15.148:6803","nonce":3378174313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3378174313},{"type":"v1","addr":"172.21.15.148:6807","nonce":3378174313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3378174313},{"type":"v1","addr":"172.21.15.148:6805","nonce":3378174313}]},"public_addr":"172.21.15.148:6801/3378174313","cluster_addr":"172.21.15.148:6803/3378174313","heartbeat_back_addr":"172.21.15.148:6807/3378174313","heartbeat_front_addr":"172.21.15.148:6805/3378174313","state":["exists","up"]},{"osd":3,"uuid":"4396a9ee-f3f3-410b-9fb1-124f1e33a1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":11741938},{"type":"v1","addr":"172.21.15.5:6811","nonce":11741938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":11741938},{"type":"v1","addr":"172.21.15.5:6813","nonce":11741938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":11741938},{"type":"v1","addr":"172.21.15.5:6817","nonce":11741938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":11741938},{"type":"v1","addr":"172.21.15.5:6815","nonce":11741938}]},"public_addr":"172.21.15.5:6811/11741938","cluster_addr":"172.21.15.5:6813/11741938","heartbeat_back_addr":"172.21.15.5:6817/11741938","heartbeat_front_addr":"172.21.15.5:6815/11741938","state":["exists","up"]},{"osd":4,"uuid":"989d76e7-c742-49d0-994a-c58f2771b1d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":3274923851},{"type":"v1","addr":"172.21.15.146:6809","nonce":3274923851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":3274923851},{"type":"v1","addr":"172.21.15.146:6811","nonce":3274923851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3274923851},{"type":"v1","addr":"172.21.15.146:6815","nonce":3274923851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":3274923851},{"type":"v1","addr":"172.21.15.146:6813","nonce":3274923851}]},"public_addr":"172.21.15.146:6809/3274923851","cluster_addr":"172.21.15.146:6811/3274923851","heartbeat_back_addr":"172.21.15.146:6815/3274923851","heartbeat_front_addr":"172.21.15.146:6813/3274923851","state":["exists","up"]},{"osd":5,"uuid":"64ada7d4-70a0-4b83-9af8-032be544608d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":2771670476},{"type":"v1","addr":"172.21.15.148:6809","nonce":2771670476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":2771670476},{"type":"v1","addr":"172.21.15.148:6811","nonce":2771670476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":2771670476},{"type":"v1","addr":"172.21.15.148:6815","nonce":2771670476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":2771670476},{"type":"v1","addr":"172.21.15.148:6813","nonce":2771670476}]},"public_addr":"172.21.15.148:6809/2771670476","cluster_addr":"172.21.15.148:6811/2771670476","heartbeat_back_addr":"172.21.15.148:6815/2771670476","heartbeat_front_addr":"172.21.15.148:6813/2771670476","state":["exists","up"]},{"osd":6,"uuid":"a70a88fd-2e4b-480c-bb98-62636cf70cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3176073169},{"type":"v1","addr":"172.21.15.5:6819","nonce":3176073169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":3176073169},{"type":"v1","addr":"172.21.15.5:6821","nonce":3176073169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":3176073169},{"type":"v1","addr":"172.21.15.5:6825","nonce":3176073169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3176073169},{"type":"v1","addr":"172.21.15.5:6823","nonce":3176073169}]},"public_addr":"172.21.15.5:6819/3176073169","cluster_addr":"172.21.15.5:6821/3176073169","heartbeat_back_addr":"172.21.15.5:6825/3176073169","heartbeat_front_addr":"172.21.15.5:6823/3176073169","state":["exists","up"]},{"osd":7,"uuid":"49451e8d-c741-4ad5-b8f5-89bad8d4d129","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2209508838},{"type":"v1","addr":"172.21.15.146:6817","nonce":2209508838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2209508838},{"type":"v1","addr":"172.21.15.146:6819","nonce":2209508838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2209508838},{"type":"v1","addr":"172.21.15.146:6823","nonce":2209508838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2209508838},{"type":"v1","addr":"172.21.15.146:6821","nonce":2209508838}]},"public_addr":"172.21.15.146:6817/2209508838","cluster_addr":"172.21.15.146:6819/2209508838","heartbeat_back_addr":"172.21.15.146:6823/2209508838","heartbeat_front_addr":"172.21.15.146:6821/2209508838","state":["exists","up"]},{"osd":8,"uuid":"57bfc121-a75e-4a7f-b1dc-64074b178b03","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":651045756},{"type":"v1","addr":"172.21.15.148:6817","nonce":651045756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":651045756},{"type":"v1","addr":"172.21.15.148:6819","nonce":651045756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":651045756},{"type":"v1","addr":"172.21.15.148:6823","nonce":651045756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":651045756},{"type":"v1","addr":"172.21.15.148:6821","nonce":651045756}]},"public_addr":"172.21.15.148:6817/651045756","cluster_addr":"172.21.15.148:6819/651045756","heartbeat_back_addr":"172.21.15.148:6823/651045756","heartbeat_front_addr":"172.21.15.148:6821/651045756","state":["exists","up"]},{"osd":9,"uuid":"d9726168-7494-4bf8-b433-350e85bfd461","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1961857360},{"type":"v1","addr":"172.21.15.5:6827","nonce":1961857360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1961857360},{"type":"v1","addr":"172.21.15.5:6829","nonce":1961857360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":1961857360},{"type":"v1","addr":"172.21.15.5:6833","nonce":1961857360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1961857360},{"type":"v1","addr":"172.21.15.5:6831","nonce":1961857360}]},"public_addr":"172.21.15.5:6827/1961857360","cluster_addr":"172.21.15.5:6829/1961857360","heartbeat_back_addr":"172.21.15.5:6833/1961857360","heartbeat_front_addr":"172.21.15.5:6831/1961857360","state":["exists","up"]},{"osd":10,"uuid":"519ec60f-3606-48ac-9043-6f59a798766d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":2041733103},{"type":"v1","addr":"172.21.15.146:6825","nonce":2041733103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":2041733103},{"type":"v1","addr":"172.21.15.146:6827","nonce":2041733103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":2041733103},{"type":"v1","addr":"172.21.15.146:6831","nonce":2041733103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":2041733103},{"type":"v1","addr":"172.21.15.146:6829","nonce":2041733103}]},"public_addr":"172.21.15.146:6825/2041733103","cluster_addr":"172.21.15.146:6827/2041733103","heartbeat_back_addr":"172.21.15.146:6831/2041733103","heartbeat_front_addr":"172.21.15.146:6829/2041733103","state":["exists","up"]},{"osd":11,"uuid":"0f92d0cb-ba54-4bc3-a6d0-3c54d910556a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":839290773},{"type":"v1","addr":"172.21.15.148:6825","nonce":839290773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":839290773},{"type":"v1","addr":"172.21.15.148:6827","nonce":839290773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":839290773},{"type":"v1","addr":"172.21.15.148:6831","nonce":839290773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":839290773},{"type":"v1","addr":"172.21.15.148:6829","nonce":839290773}]},"public_addr":"172.21.15.148:6825/839290773","cluster_addr":"172.21.15.148:6827/839290773","heartbeat_back_addr":"172.21.15.148:6831/839290773","heartbeat_front_addr":"172.21.15.148:6829/839290773","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:50:54.101755+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:51:19.037385+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:51:44.142837+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:52:09.150887+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:52:33.705079+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:52:58.845233+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:53:23.068339+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:53:47.520398+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:54:11.965370+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:54:36.776075+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:55:01.292236+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:55:25.346589+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:6800/3125821798":"2024-09-13T02:49:29.462216+0000","172.21.15.5:0/3479911451":"2024-09-13T02:49:29.462216+0000","172.21.15.5:0/1547753207":"2024-09-13T02:49:03.300872+0000","172.21.15.5:0/1905762627":"2024-09-13T02:49:03.300872+0000","172.21.15.5:0/4142340280":"2024-09-13T02:49:29.462216+0000","172.21.15.5:6801/3125821798":"2024-09-13T02:49:29.462216+0000","172.21.15.5:6801/2595781680":"2024-09-13T02:49:03.300872+0000","172.21.15.5:0/2273691311":"2024-09-13T02:49:29.462216+0000","172.21.15.5:6800/2595781680":"2024-09-13T02:49:03.300872+0000","172.21.15.5:0/1196201217":"2024-09-13T02:49:03.300872+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:04:29.858 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:29.858+0000 7f2344ca6640 1 -- 172.21.15.5:0/106445109 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f23280a2b90 msgr2=0x7f23280a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:29.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:29.858+0000 7f2344ca6640 1 --2- 172.21.15.5:0/106445109 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f23280a2b90 0x7f23280a5050 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f23401993a0 tx=0x7f2334005f90 comp rx=0 tx=0).stop 2024-09-12T03:04:29.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:29.858+0000 7f2344ca6640 1 -- 172.21.15.5:0/106445109 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2340102590 msgr2=0x7f2340197e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:29.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:29.858+0000 7f2344ca6640 1 --2- 172.21.15.5:0/106445109 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2340102590 0x7f2340197e90 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f233000eab0 tx=0x7f233000ef70 comp rx=0 tx=0).stop 2024-09-12T03:04:29.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:29.858+0000 7f2344ca6640 1 -- 172.21.15.5:0/106445109 shutdown_connections 2024-09-12T03:04:29.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:29.858+0000 7f2344ca6640 1 --2- 172.21.15.5:0/106445109 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f23401077f0 0x7f234019f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:29.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:29.858+0000 7f2344ca6640 1 --2- 172.21.15.5:0/106445109 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f2340104ec0 0x7f23401983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:29.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:29.858+0000 7f2344ca6640 1 --2- 172.21.15.5:0/106445109 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f23280a2b90 0x7f23280a5050 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:29.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:29.858+0000 7f2344ca6640 1 --2- 172.21.15.5:0/106445109 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2340102590 0x7f2340197e90 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:29.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:29.858+0000 7f2344ca6640 1 -- 172.21.15.5:0/106445109 >> 172.21.15.5:0/106445109 conn(0x7f23400fc170 msgr2=0x7f2340102f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:29.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:29.858+0000 7f2344ca6640 1 -- 172.21.15.5:0/106445109 shutdown_connections 2024-09-12T03:04:29.859 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:29.858+0000 7f2344ca6640 1 -- 172.21.15.5:0/106445109 wait complete. 2024-09-12T03:04:29.873 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-12T02:51:47.970639+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-12T02:56:12.834271+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-12T02:56:14.951437+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-12T03:04:29.873 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-12T03:04:29.982 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:29.980+0000 7f7a7aa27640 1 -- 172.21.15.5:0/3102198121 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a74104f80 msgr2=0x7f7a74107370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:29.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:29.980+0000 7f7a7aa27640 1 --2- 172.21.15.5:0/3102198121 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a74104f80 0x7f7a74107370 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f7a68009900 tx=0x7f7a6802a410 comp rx=0 tx=0).stop 2024-09-12T03:04:29.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:29.980+0000 7f7a7aa27640 1 -- 172.21.15.5:0/3102198121 shutdown_connections 2024-09-12T03:04:29.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:29.980+0000 7f7a7aa27640 1 --2- 172.21.15.5:0/3102198121 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a74104f80 0x7f7a74107370 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:29.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:29.980+0000 7f7a7aa27640 1 --2- 172.21.15.5:0/3102198121 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7a741078b0 0x7f7a74109ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:29.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:29.980+0000 7f7a7aa27640 1 --2- 172.21.15.5:0/3102198121 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7a74102650 0x7f7a74104a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:29.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:29.980+0000 7f7a7aa27640 1 -- 172.21.15.5:0/3102198121 >> 172.21.15.5:0/3102198121 conn(0x7f7a740fc190 msgr2=0x7f7a740fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:29.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:29.980+0000 7f7a7aa27640 1 -- 172.21.15.5:0/3102198121 shutdown_connections 2024-09-12T03:04:29.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:29.980+0000 7f7a7aa27640 1 -- 172.21.15.5:0/3102198121 wait complete. 2024-09-12T03:04:29.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:29.981+0000 7f7a7aa27640 1 Processor -- start 2024-09-12T03:04:29.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:29.981+0000 7f7a7aa27640 1 -- start start 2024-09-12T03:04:29.983 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:29.981+0000 7f7a7aa27640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7a74102650 0x7f7a74197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:29.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:29.981+0000 7f7a7aa27640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a74104f80 0x7f7a74198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:29.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:29.981+0000 7f7a7aa27640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7a741078b0 0x7f7a7419f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:29.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:29.981+0000 7f7a7aa27640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f7a7410d770 con 0x7f7a74104f80 2024-09-12T03:04:29.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:29.981+0000 7f7a7aa27640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f7a7410d5f0 con 0x7f7a74102650 2024-09-12T03:04:29.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:29.981+0000 7f7a7aa27640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f7a7410d8f0 con 0x7f7a741078b0 2024-09-12T03:04:29.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:29.981+0000 7f7a737fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a74104f80 0x7f7a74198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:29.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:29.981+0000 7f7a737fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a74104f80 0x7f7a74198370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53574/0 (socket says 172.21.15.5:53574) 2024-09-12T03:04:29.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:29.981+0000 7f7a737fe640 1 -- 172.21.15.5:0/3144379292 learned_addr learned my addr 172.21.15.5:0/3144379292 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:04:29.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:29.981+0000 7f7a73fff640 1 --2- 172.21.15.5:0/3144379292 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7a74102650 0x7f7a74197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:29.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:29.981+0000 7f7a78fc4640 1 --2- 172.21.15.5:0/3144379292 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7a741078b0 0x7f7a7419f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:29.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:29.982+0000 7f7a737fe640 1 -- 172.21.15.5:0/3144379292 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7a741078b0 msgr2=0x7f7a7419f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:29.984 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:29.982+0000 7f7a737fe640 1 --2- 172.21.15.5:0/3144379292 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7a741078b0 0x7f7a7419f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:29.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:29.982+0000 7f7a737fe640 1 -- 172.21.15.5:0/3144379292 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7a74102650 msgr2=0x7f7a74197e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:29.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:29.982+0000 7f7a737fe640 1 --2- 172.21.15.5:0/3144379292 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7a74102650 0x7f7a74197e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:29.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:29.982+0000 7f7a737fe640 1 -- 172.21.15.5:0/3144379292 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a7419fa50 con 0x7f7a74104f80 2024-09-12T03:04:29.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:29.982+0000 7f7a737fe640 1 --2- 172.21.15.5:0/3144379292 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a74104f80 0x7f7a74198370 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f7a68031610 tx=0x7f7a680043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:29.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:29.982+0000 7f7a717fa640 1 -- 172.21.15.5:0/3144379292 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a6803f030 con 0x7f7a74104f80 2024-09-12T03:04:29.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:29.982+0000 7f7a7aa27640 1 -- 172.21.15.5:0/3144379292 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a7419fce0 con 0x7f7a74104f80 2024-09-12T03:04:29.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:29.982+0000 7f7a717fa640 1 -- 172.21.15.5:0/3144379292 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a68003ce0 con 0x7f7a74104f80 2024-09-12T03:04:29.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:29.982+0000 7f7a717fa640 1 -- 172.21.15.5:0/3144379292 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a68035070 con 0x7f7a74104f80 2024-09-12T03:04:29.985 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:29.982+0000 7f7a7aa27640 1 -- 172.21.15.5:0/3144379292 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a741a01d0 con 0x7f7a74104f80 2024-09-12T03:04:29.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:29.983+0000 7f7a7aa27640 1 -- 172.21.15.5:0/3144379292 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a740fdc80 con 0x7f7a74104f80 2024-09-12T03:04:29.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:29.984+0000 7f7a717fa640 1 -- 172.21.15.5:0/3144379292 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f7a68042030 con 0x7f7a74104f80 2024-09-12T03:04:29.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:29.985+0000 7f7a717fa640 1 --2- 172.21.15.5:0/3144379292 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f7a5c0a2b90 0x7f7a5c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:29.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:29.985+0000 7f7a717fa640 1 -- 172.21.15.5:0/3144379292 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f7a680ef870 con 0x7f7a74104f80 2024-09-12T03:04:29.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:29.985+0000 7f7a73fff640 1 --2- 172.21.15.5:0/3144379292 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f7a5c0a2b90 0x7f7a5c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:29.986 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:29.986+0000 7f7a73fff640 1 --2- 172.21.15.5:0/3144379292 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f7a5c0a2b90 0x7f7a5c0a5050 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f7a6400bbe0 tx=0x7f7a6400b390 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:29.988 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:29.988+0000 7f7a717fa640 1 -- 172.21.15.5:0/3144379292 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7a680bc780 con 0x7f7a74104f80 2024-09-12T03:04:30.165 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:04:29 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/106445109' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:04:30.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.166+0000 7f7a7aa27640 1 -- 172.21.15.5:0/3144379292 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7a7402d970 con 0x7f7a74104f80 2024-09-12T03:04:30.166 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.166+0000 7f7a717fa640 1 -- 172.21.15.5:0/3144379292 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) ==== 93+0+10 (secure 0 0 0) 0x7f7a680bc080 con 0x7f7a74104f80 2024-09-12T03:04:30.166 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2024-09-12T03:04:30.168 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.168+0000 7f7a7aa27640 1 -- 172.21.15.5:0/3144379292 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f7a5c0a2b90 msgr2=0x7f7a5c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:30.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.168+0000 7f7a7aa27640 1 --2- 172.21.15.5:0/3144379292 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f7a5c0a2b90 0x7f7a5c0a5050 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f7a6400bbe0 tx=0x7f7a6400b390 comp rx=0 tx=0).stop 2024-09-12T03:04:30.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.168+0000 7f7a7aa27640 1 -- 172.21.15.5:0/3144379292 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a74104f80 msgr2=0x7f7a74198370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:30.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.168+0000 7f7a7aa27640 1 --2- 172.21.15.5:0/3144379292 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a74104f80 0x7f7a74198370 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f7a68031610 tx=0x7f7a680043b0 comp rx=0 tx=0).stop 2024-09-12T03:04:30.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.168+0000 7f7a7aa27640 1 -- 172.21.15.5:0/3144379292 shutdown_connections 2024-09-12T03:04:30.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.168+0000 7f7a7aa27640 1 --2- 172.21.15.5:0/3144379292 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f7a5c0a2b90 0x7f7a5c0a5050 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:30.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.168+0000 7f7a7aa27640 1 --2- 172.21.15.5:0/3144379292 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7a74104f80 0x7f7a74198370 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:30.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.168+0000 7f7a7aa27640 1 --2- 172.21.15.5:0/3144379292 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f7a741078b0 0x7f7a7419f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:30.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.168+0000 7f7a7aa27640 1 --2- 172.21.15.5:0/3144379292 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f7a74102650 0x7f7a74197e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:30.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.168+0000 7f7a7aa27640 1 -- 172.21.15.5:0/3144379292 >> 172.21.15.5:0/3144379292 conn(0x7f7a740fc190 msgr2=0x7f7a74102fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:30.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.168+0000 7f7a7aa27640 1 -- 172.21.15.5:0/3144379292 shutdown_connections 2024-09-12T03:04:30.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.168+0000 7f7a7aa27640 1 -- 172.21.15.5:0/3144379292 wait complete. 2024-09-12T03:04:30.178 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:04:29 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/106445109' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:04:30.179 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-12T03:04:30.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:04:29 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/106445109' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:04:30.287 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.285+0000 7fc6bbbc4640 1 -- 172.21.15.5:0/964155189 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc6b4101c20 msgr2=0x7fc6b41020a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:30.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.285+0000 7fc6bbbc4640 1 --2- 172.21.15.5:0/964155189 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc6b4101c20 0x7fc6b41020a0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fc6a8009900 tx=0x7fc6a802a410 comp rx=0 tx=0).stop 2024-09-12T03:04:30.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.285+0000 7fc6bbbc4640 1 -- 172.21.15.5:0/964155189 shutdown_connections 2024-09-12T03:04:30.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.285+0000 7fc6bbbc4640 1 --2- 172.21.15.5:0/964155189 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc6b4101c20 0x7fc6b41020a0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:30.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.285+0000 7fc6bbbc4640 1 --2- 172.21.15.5:0/964155189 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc6b41025e0 0x7fc6b4108e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:30.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.285+0000 7fc6bbbc4640 1 --2- 172.21.15.5:0/964155189 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc6b4100a30 0x7fc6b4100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:30.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.285+0000 7fc6bbbc4640 1 -- 172.21.15.5:0/964155189 >> 172.21.15.5:0/964155189 conn(0x7fc6b40fc1b0 msgr2=0x7fc6b40fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:30.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.286+0000 7fc6bbbc4640 1 -- 172.21.15.5:0/964155189 shutdown_connections 2024-09-12T03:04:30.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.286+0000 7fc6bbbc4640 1 -- 172.21.15.5:0/964155189 wait complete. 2024-09-12T03:04:30.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.286+0000 7fc6bbbc4640 1 Processor -- start 2024-09-12T03:04:30.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.286+0000 7fc6bbbc4640 1 -- start start 2024-09-12T03:04:30.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.286+0000 7fc6bbbc4640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc6b4100a30 0x7fc6b41049c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:30.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.286+0000 7fc6bbbc4640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc6b4101c20 0x7fc6b4103000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:30.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.286+0000 7fc6bbbc4640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc6b41025e0 0x7fc6b4103540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:30.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.286+0000 7fc6bbbc4640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fc6b410c700 con 0x7fc6b41025e0 2024-09-12T03:04:30.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.286+0000 7fc6bbbc4640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fc6b410c580 con 0x7fc6b4101c20 2024-09-12T03:04:30.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.286+0000 7fc6bbbc4640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fc6b410c880 con 0x7fc6b4100a30 2024-09-12T03:04:30.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.286+0000 7fc6ba161640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc6b41025e0 0x7fc6b4103540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:30.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.286+0000 7fc6ba161640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc6b41025e0 0x7fc6b4103540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53594/0 (socket says 172.21.15.5:53594) 2024-09-12T03:04:30.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.286+0000 7fc6ba161640 1 -- 172.21.15.5:0/3595542477 learned_addr learned my addr 172.21.15.5:0/3595542477 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:04:30.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.287+0000 7fc6b915f640 1 --2- 172.21.15.5:0/3595542477 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc6b4101c20 0x7fc6b4103000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:30.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.287+0000 7fc6b9960640 1 --2- 172.21.15.5:0/3595542477 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc6b4100a30 0x7fc6b41049c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:30.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.287+0000 7fc6ba161640 1 -- 172.21.15.5:0/3595542477 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc6b4100a30 msgr2=0x7fc6b41049c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:30.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.287+0000 7fc6ba161640 1 --2- 172.21.15.5:0/3595542477 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc6b4100a30 0x7fc6b41049c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:30.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.287+0000 7fc6ba161640 1 -- 172.21.15.5:0/3595542477 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc6b4101c20 msgr2=0x7fc6b4103000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:30.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.287+0000 7fc6ba161640 1 --2- 172.21.15.5:0/3595542477 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc6b4101c20 0x7fc6b4103000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:30.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.287+0000 7fc6ba161640 1 -- 172.21.15.5:0/3595542477 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6b4103dd0 con 0x7fc6b41025e0 2024-09-12T03:04:30.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.287+0000 7fc6b915f640 1 --2- 172.21.15.5:0/3595542477 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc6b4101c20 0x7fc6b4103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:04:30.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.287+0000 7fc6ba161640 1 --2- 172.21.15.5:0/3595542477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc6b41025e0 0x7fc6b4103540 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fc6ac00eb10 tx=0x7fc6ac00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:30.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.287+0000 7fc6a6ffd640 1 -- 172.21.15.5:0/3595542477 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6ac004b20 con 0x7fc6b41025e0 2024-09-12T03:04:30.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.287+0000 7fc6a6ffd640 1 -- 172.21.15.5:0/3595542477 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc6ac004cc0 con 0x7fc6b41025e0 2024-09-12T03:04:30.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.287+0000 7fc6bbbc4640 1 -- 172.21.15.5:0/3595542477 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6b41a2060 con 0x7fc6b41025e0 2024-09-12T03:04:30.290 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.287+0000 7fc6a6ffd640 1 -- 172.21.15.5:0/3595542477 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6ac010760 con 0x7fc6b41025e0 2024-09-12T03:04:30.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.287+0000 7fc6bbbc4640 1 -- 172.21.15.5:0/3595542477 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6b41a2530 con 0x7fc6b41025e0 2024-09-12T03:04:30.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.288+0000 7fc6bbbc4640 1 -- 172.21.15.5:0/3595542477 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6b4100e30 con 0x7fc6b41025e0 2024-09-12T03:04:30.291 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.289+0000 7fc6a6ffd640 1 -- 172.21.15.5:0/3595542477 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fc6ac005020 con 0x7fc6b41025e0 2024-09-12T03:04:30.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.290+0000 7fc6a6ffd640 1 --2- 172.21.15.5:0/3595542477 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fc6a00a2b90 0x7fc6a00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:30.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.290+0000 7fc6b9960640 1 --2- 172.21.15.5:0/3595542477 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fc6a00a2b90 0x7fc6a00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:30.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.290+0000 7fc6a6ffd640 1 -- 172.21.15.5:0/3595542477 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fc6ac0cf7d0 con 0x7fc6b41025e0 2024-09-12T03:04:30.292 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.290+0000 7fc6b9960640 1 --2- 172.21.15.5:0/3595542477 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fc6a00a2b90 0x7fc6a00a5050 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fc6b4101a80 tx=0x7fc6b0005f90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:30.294 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.294+0000 7fc6a6ffd640 1 -- 172.21.15.5:0/3595542477 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc6ac09c6e0 con 0x7fc6b41025e0 2024-09-12T03:04:30.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.468+0000 7fc6bbbc4640 1 -- 172.21.15.5:0/3595542477 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fc6b41a2a10 con 0x7fc6b41025e0 2024-09-12T03:04:30.469 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.469+0000 7fc6a6ffd640 1 -- 172.21.15.5:0/3595542477 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v83) ==== 104+0+11 (secure 0 0 0) 0x7fc6ac09bfe0 con 0x7fc6b41025e0 2024-09-12T03:04:30.469 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2024-09-12T03:04:30.471 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.471+0000 7fc6bbbc4640 1 -- 172.21.15.5:0/3595542477 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fc6a00a2b90 msgr2=0x7fc6a00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:30.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.471+0000 7fc6bbbc4640 1 --2- 172.21.15.5:0/3595542477 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fc6a00a2b90 0x7fc6a00a5050 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fc6b4101a80 tx=0x7fc6b0005f90 comp rx=0 tx=0).stop 2024-09-12T03:04:30.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.471+0000 7fc6bbbc4640 1 -- 172.21.15.5:0/3595542477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc6b41025e0 msgr2=0x7fc6b4103540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:30.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.471+0000 7fc6bbbc4640 1 --2- 172.21.15.5:0/3595542477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc6b41025e0 0x7fc6b4103540 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fc6ac00eb10 tx=0x7fc6ac00c2d0 comp rx=0 tx=0).stop 2024-09-12T03:04:30.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.471+0000 7fc6bbbc4640 1 -- 172.21.15.5:0/3595542477 shutdown_connections 2024-09-12T03:04:30.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.471+0000 7fc6bbbc4640 1 --2- 172.21.15.5:0/3595542477 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fc6a00a2b90 0x7fc6a00a5050 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:30.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.471+0000 7fc6bbbc4640 1 --2- 172.21.15.5:0/3595542477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc6b41025e0 0x7fc6b4103540 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:30.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.471+0000 7fc6bbbc4640 1 --2- 172.21.15.5:0/3595542477 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fc6b4101c20 0x7fc6b4103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:30.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.471+0000 7fc6bbbc4640 1 --2- 172.21.15.5:0/3595542477 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fc6b4100a30 0x7fc6b41049c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:30.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.471+0000 7fc6bbbc4640 1 -- 172.21.15.5:0/3595542477 >> 172.21.15.5:0/3595542477 conn(0x7fc6b40fc1b0 msgr2=0x7fc6b406a410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:30.472 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.471+0000 7fc6bbbc4640 1 -- 172.21.15.5:0/3595542477 shutdown_connections 2024-09-12T03:04:30.473 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.471+0000 7fc6bbbc4640 1 -- 172.21.15.5:0/3595542477 wait complete. 2024-09-12T03:04:30.482 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-12T03:04:30.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.588+0000 7fad1b28b640 1 -- 172.21.15.5:0/3403758939 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad141013a0 msgr2=0x7fad14101830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:30.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.588+0000 7fad1b28b640 1 --2- 172.21.15.5:0/3403758939 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad141013a0 0x7fad14101830 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fad0c009ca0 tx=0x7fad0c02a8c0 comp rx=0 tx=0).stop 2024-09-12T03:04:30.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.588+0000 7fad1b28b640 1 -- 172.21.15.5:0/3403758939 shutdown_connections 2024-09-12T03:04:30.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.588+0000 7fad1b28b640 1 --2- 172.21.15.5:0/3403758939 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad141013a0 0x7fad14101830 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:30.591 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.588+0000 7fad1b28b640 1 --2- 172.21.15.5:0/3403758939 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fad14069f60 0x7fad141006f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:30.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.588+0000 7fad1b28b640 1 --2- 172.21.15.5:0/3403758939 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad14069ad0 0x7fad141001b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:30.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.588+0000 7fad1b28b640 1 -- 172.21.15.5:0/3403758939 >> 172.21.15.5:0/3403758939 conn(0x7fad140fb950 msgr2=0x7fad140fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:30.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.589+0000 7fad1b28b640 1 -- 172.21.15.5:0/3403758939 shutdown_connections 2024-09-12T03:04:30.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.589+0000 7fad1b28b640 1 -- 172.21.15.5:0/3403758939 wait complete. 2024-09-12T03:04:30.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.589+0000 7fad1b28b640 1 Processor -- start 2024-09-12T03:04:30.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.589+0000 7fad1b28b640 1 -- start start 2024-09-12T03:04:30.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.589+0000 7fad1b28b640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fad14069ad0 0x7fad14197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:30.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.589+0000 7fad1b28b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad14069f60 0x7fad14198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:30.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.589+0000 7fad1b28b640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad141013a0 0x7fad1419f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:30.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.589+0000 7fad1b28b640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fad1410c6e0 con 0x7fad14069f60 2024-09-12T03:04:30.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.589+0000 7fad1b28b640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fad1410c560 con 0x7fad14069ad0 2024-09-12T03:04:30.592 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.589+0000 7fad1b28b640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fad1410c860 con 0x7fad141013a0 2024-09-12T03:04:30.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.589+0000 7fad18826640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad14069f60 0x7fad14198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:30.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.589+0000 7fad18826640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad14069f60 0x7fad14198450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53602/0 (socket says 172.21.15.5:53602) 2024-09-12T03:04:30.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.589+0000 7fad18826640 1 -- 172.21.15.5:0/3195632178 learned_addr learned my addr 172.21.15.5:0/3195632178 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:04:30.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.590+0000 7fad19828640 1 --2- 172.21.15.5:0/3195632178 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad141013a0 0x7fad1419f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:30.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.590+0000 7fad19027640 1 --2- 172.21.15.5:0/3195632178 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fad14069ad0 0x7fad14197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:30.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.590+0000 7fad18826640 1 -- 172.21.15.5:0/3195632178 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad141013a0 msgr2=0x7fad1419f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:30.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.590+0000 7fad18826640 1 --2- 172.21.15.5:0/3195632178 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad141013a0 0x7fad1419f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:30.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.590+0000 7fad18826640 1 -- 172.21.15.5:0/3195632178 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fad14069ad0 msgr2=0x7fad14197f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:30.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.590+0000 7fad18826640 1 --2- 172.21.15.5:0/3195632178 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fad14069ad0 0x7fad14197f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:30.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.590+0000 7fad18826640 1 -- 172.21.15.5:0/3195632178 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad1419fb30 con 0x7fad14069f60 2024-09-12T03:04:30.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.590+0000 7fad19828640 1 --2- 172.21.15.5:0/3195632178 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad141013a0 0x7fad1419f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:04:30.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.590+0000 7fad19027640 1 --2- 172.21.15.5:0/3195632178 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fad14069ad0 0x7fad14197f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:04:30.593 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.590+0000 7fad18826640 1 --2- 172.21.15.5:0/3195632178 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad14069f60 0x7fad14198450 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fad0400c970 tx=0x7fad0400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:30.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.590+0000 7fad0a7fc640 1 -- 172.21.15.5:0/3195632178 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad04007bb0 con 0x7fad14069f60 2024-09-12T03:04:30.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.590+0000 7fad0a7fc640 1 -- 172.21.15.5:0/3195632178 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad04007d50 con 0x7fad14069f60 2024-09-12T03:04:30.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.590+0000 7fad1b28b640 1 -- 172.21.15.5:0/3195632178 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad1419fe20 con 0x7fad14069f60 2024-09-12T03:04:30.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.590+0000 7fad0a7fc640 1 -- 172.21.15.5:0/3195632178 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad0400f450 con 0x7fad14069f60 2024-09-12T03:04:30.594 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.590+0000 7fad1b28b640 1 -- 172.21.15.5:0/3195632178 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad141a0330 con 0x7fad14069f60 2024-09-12T03:04:30.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.591+0000 7fad1b28b640 1 -- 172.21.15.5:0/3195632178 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad14109000 con 0x7fad14069f60 2024-09-12T03:04:30.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.592+0000 7fad0a7fc640 1 -- 172.21.15.5:0/3195632178 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fad040046c0 con 0x7fad14069f60 2024-09-12T03:04:30.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.593+0000 7fad0a7fc640 1 --2- 172.21.15.5:0/3195632178 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fad000a2c10 0x7fad000a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:30.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.593+0000 7fad19027640 1 --2- 172.21.15.5:0/3195632178 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fad000a2c10 0x7fad000a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:30.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.593+0000 7fad0a7fc640 1 -- 172.21.15.5:0/3195632178 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fad040cf0d0 con 0x7fad14069f60 2024-09-12T03:04:30.595 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.594+0000 7fad19027640 1 --2- 172.21.15.5:0/3195632178 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fad000a2c10 0x7fad000a50d0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fad100058c0 tx=0x7fad10005ec0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:30.597 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.596+0000 7fad0a7fc640 1 -- 172.21.15.5:0/3195632178 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fad04014030 con 0x7fad14069f60 2024-09-12T03:04:30.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.771+0000 7fad1b28b640 1 -- 172.21.15.5:0/3195632178 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fad1402da10 con 0x7fad14069f60 2024-09-12T03:04:30.771 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.771+0000 7fad0a7fc640 1 -- 172.21.15.5:0/3195632178 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v83) ==== 100+0+11 (secure 0 0 0) 0x7fad0409c060 con 0x7fad14069f60 2024-09-12T03:04:30.771 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2024-09-12T03:04:30.773 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.773+0000 7fad1b28b640 1 -- 172.21.15.5:0/3195632178 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fad000a2c10 msgr2=0x7fad000a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:30.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.773+0000 7fad1b28b640 1 --2- 172.21.15.5:0/3195632178 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fad000a2c10 0x7fad000a50d0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fad100058c0 tx=0x7fad10005ec0 comp rx=0 tx=0).stop 2024-09-12T03:04:30.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.773+0000 7fad1b28b640 1 -- 172.21.15.5:0/3195632178 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad14069f60 msgr2=0x7fad14198450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:30.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.773+0000 7fad1b28b640 1 --2- 172.21.15.5:0/3195632178 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad14069f60 0x7fad14198450 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fad0400c970 tx=0x7fad0400ce30 comp rx=0 tx=0).stop 2024-09-12T03:04:30.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.773+0000 7fad1b28b640 1 -- 172.21.15.5:0/3195632178 shutdown_connections 2024-09-12T03:04:30.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.773+0000 7fad1b28b640 1 --2- 172.21.15.5:0/3195632178 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fad000a2c10 0x7fad000a50d0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:30.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.773+0000 7fad1b28b640 1 --2- 172.21.15.5:0/3195632178 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad14069f60 0x7fad14198450 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:30.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.773+0000 7fad1b28b640 1 --2- 172.21.15.5:0/3195632178 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fad141013a0 0x7fad1419f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:30.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.773+0000 7fad1b28b640 1 --2- 172.21.15.5:0/3195632178 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fad14069ad0 0x7fad14197f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:30.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.773+0000 7fad1b28b640 1 -- 172.21.15.5:0/3195632178 >> 172.21.15.5:0/3195632178 conn(0x7fad140fb950 msgr2=0x7fad14102c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:30.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.773+0000 7fad1b28b640 1 -- 172.21.15.5:0/3195632178 shutdown_connections 2024-09-12T03:04:30.774 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.773+0000 7fad1b28b640 1 -- 172.21.15.5:0/3195632178 wait complete. 2024-09-12T03:04:30.784 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-09-12T03:04:30.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.888+0000 7f8b34e27640 1 -- 172.21.15.5:0/2614850805 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b30100540 msgr2=0x7f8b3010a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:30.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.888+0000 7f8b34e27640 1 --2- 172.21.15.5:0/2614850805 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b30100540 0x7f8b3010a0f0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f8b28009cd0 tx=0x7f8b2802a8e0 comp rx=0 tx=0).stop 2024-09-12T03:04:30.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.888+0000 7f8b34e27640 1 -- 172.21.15.5:0/2614850805 shutdown_connections 2024-09-12T03:04:30.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.888+0000 7f8b34e27640 1 --2- 172.21.15.5:0/2614850805 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b30100540 0x7f8b3010a0f0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:30.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.888+0000 7f8b34e27640 1 --2- 172.21.15.5:0/2614850805 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8b300ffb80 0x7f8b30100000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:30.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.888+0000 7f8b34e27640 1 --2- 172.21.15.5:0/2614850805 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b30103110 0x7f8b30103510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:30.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.888+0000 7f8b34e27640 1 -- 172.21.15.5:0/2614850805 >> 172.21.15.5:0/2614850805 conn(0x7f8b300fb950 msgr2=0x7f8b300fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:30.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.888+0000 7f8b34e27640 1 -- 172.21.15.5:0/2614850805 shutdown_connections 2024-09-12T03:04:30.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.888+0000 7f8b34e27640 1 -- 172.21.15.5:0/2614850805 wait complete. 2024-09-12T03:04:30.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.888+0000 7f8b34e27640 1 Processor -- start 2024-09-12T03:04:30.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.888+0000 7f8b34e27640 1 -- start start 2024-09-12T03:04:30.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.889+0000 7f8b34e27640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b300ffb80 0x7f8b30197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:30.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.889+0000 7f8b34e27640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b30100540 0x7f8b301983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:30.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.889+0000 7f8b34e27640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8b30103110 0x7f8b3019f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:30.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.889+0000 7f8b34e27640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f8b3010d8a0 con 0x7f8b30100540 2024-09-12T03:04:30.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.889+0000 7f8b34e27640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f8b3010d720 con 0x7f8b30103110 2024-09-12T03:04:30.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.889+0000 7f8b34e27640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f8b3010da20 con 0x7f8b300ffb80 2024-09-12T03:04:30.891 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.889+0000 7f8b27fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b30100540 0x7f8b301983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:30.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.889+0000 7f8b27fff640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b30100540 0x7f8b301983e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53614/0 (socket says 172.21.15.5:53614) 2024-09-12T03:04:30.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.889+0000 7f8b27fff640 1 -- 172.21.15.5:0/3586055389 learned_addr learned my addr 172.21.15.5:0/3586055389 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:04:30.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.889+0000 7f8b2ed9d640 1 --2- 172.21.15.5:0/3586055389 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8b30103110 0x7f8b3019f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:30.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.889+0000 7f8b2e59c640 1 --2- 172.21.15.5:0/3586055389 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b300ffb80 0x7f8b30197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:30.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.889+0000 7f8b27fff640 1 -- 172.21.15.5:0/3586055389 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b300ffb80 msgr2=0x7f8b30197ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:30.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.889+0000 7f8b27fff640 1 --2- 172.21.15.5:0/3586055389 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b300ffb80 0x7f8b30197ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:30.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.889+0000 7f8b27fff640 1 -- 172.21.15.5:0/3586055389 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8b30103110 msgr2=0x7f8b3019f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:30.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.889+0000 7f8b27fff640 1 --2- 172.21.15.5:0/3586055389 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8b30103110 0x7f8b3019f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:30.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.889+0000 7f8b27fff640 1 -- 172.21.15.5:0/3586055389 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b3019fac0 con 0x7f8b30100540 2024-09-12T03:04:30.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.889+0000 7f8b27fff640 1 --2- 172.21.15.5:0/3586055389 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b30100540 0x7f8b301983e0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f8b1800b4a0 tx=0x7f8b1800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:30.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.889+0000 7f8b277fe640 1 -- 172.21.15.5:0/3586055389 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b18014070 con 0x7f8b30100540 2024-09-12T03:04:30.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.890+0000 7f8b34e27640 1 -- 172.21.15.5:0/3586055389 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b3019fdb0 con 0x7f8b30100540 2024-09-12T03:04:30.892 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.890+0000 7f8b277fe640 1 -- 172.21.15.5:0/3586055389 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b18010af0 con 0x7f8b30100540 2024-09-12T03:04:30.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.890+0000 7f8b277fe640 1 -- 172.21.15.5:0/3586055389 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b18017d10 con 0x7f8b30100540 2024-09-12T03:04:30.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.890+0000 7f8b34e27640 1 -- 172.21.15.5:0/3586055389 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b301a0300 con 0x7f8b30100540 2024-09-12T03:04:30.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.890+0000 7f8b34e27640 1 -- 172.21.15.5:0/3586055389 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b30108fd0 con 0x7f8b30100540 2024-09-12T03:04:30.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.892+0000 7f8b277fe640 1 -- 172.21.15.5:0/3586055389 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f8b18017070 con 0x7f8b30100540 2024-09-12T03:04:30.893 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.892+0000 7f8b277fe640 1 --2- 172.21.15.5:0/3586055389 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f8b1c0a2b90 0x7f8b1c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:30.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.893+0000 7f8b277fe640 1 -- 172.21.15.5:0/3586055389 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f8b180ce6f0 con 0x7f8b30100540 2024-09-12T03:04:30.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.893+0000 7f8b2e59c640 1 --2- 172.21.15.5:0/3586055389 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f8b1c0a2b90 0x7f8b1c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:30.894 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.893+0000 7f8b2e59c640 1 --2- 172.21.15.5:0/3586055389 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f8b1c0a2b90 0x7f8b1c0a5050 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f8b200058c0 tx=0x7f8b2000a2d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:30.895 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:30.895+0000 7f8b277fe640 1 -- 172.21.15.5:0/3586055389 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8b1809afb0 con 0x7f8b30100540 2024-09-12T03:04:31.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:04:30 smithi005 ceph-mon[31197]: pgmap v523: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 170 B/s wr, 11 op/s 2024-09-12T03:04:31.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:04:30 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/3144379292' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-12T03:04:31.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:04:30 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/3595542477' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-12T03:04:31.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:04:30 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/3195632178' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-12T03:04:31.120 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:31.120+0000 7f8b34e27640 1 -- 172.21.15.5:0/3586055389 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f8b3002d9c0 con 0x7f8b30100540 2024-09-12T03:04:31.128 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:31.127+0000 7f8b277fe640 1 -- 172.21.15.5:0/3586055389 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v20) ==== 81+0+740362 (secure 0 0 0) 0x7f8b1809f2b0 con 0x7f8b30100540 2024-09-12T03:04:31.145 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-12T03:04:31.145 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-12T03:04:31.146 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 20, 2024-09-12T03:04:31.146 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.146 INFO:teuthology.orchestra.run.smithi005.stdout: "active_gid": 14148, 2024-09-12T03:04:31.146 INFO:teuthology.orchestra.run.smithi005.stdout: "active_name": "x", 2024-09-12T03:04:31.146 INFO:teuthology.orchestra.run.smithi005.stdout: "active_addrs": { 2024-09-12T03:04:31.147 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-12T03:04:31.147 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-12T03:04:31.147 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-12T03:04:31.147 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6800", 2024-09-12T03:04:31.147 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 3780631642 2024-09-12T03:04:31.147 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.148 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-12T03:04:31.148 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2024-09-12T03:04:31.148 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6801", 2024-09-12T03:04:31.148 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 3780631642 2024-09-12T03:04:31.148 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:31.149 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-12T03:04:31.149 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.149 INFO:teuthology.orchestra.run.smithi005.stdout: "active_addr": "172.21.15.5:6801/3780631642", 2024-09-12T03:04:31.149 INFO:teuthology.orchestra.run.smithi005.stdout: "active_change": "2024-09-12T02:49:29.462328+0000", 2024-09-12T03:04:31.149 INFO:teuthology.orchestra.run.smithi005.stdout: "active_mgr_features": 4540701547738038271, 2024-09-12T03:04:31.149 INFO:teuthology.orchestra.run.smithi005.stdout: "available": true, 2024-09-12T03:04:31.149 INFO:teuthology.orchestra.run.smithi005.stdout: "standbys": [ 2024-09-12T03:04:31.150 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-12T03:04:31.150 INFO:teuthology.orchestra.run.smithi005.stdout: "gid": 24115, 2024-09-12T03:04:31.150 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "z", 2024-09-12T03:04:31.150 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr_features": 4540701547738038271, 2024-09-12T03:04:31.150 INFO:teuthology.orchestra.run.smithi005.stdout: "available_modules": [ 2024-09-12T03:04:31.150 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-12T03:04:31.151 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "alerts", 2024-09-12T03:04:31.151 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-12T03:04:31.151 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-12T03:04:31.151 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-12T03:04:31.151 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-09-12T03:04:31.151 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-09-12T03:04:31.151 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-12T03:04:31.152 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.152 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.152 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-09-12T03:04:31.152 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.152 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.152 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.153 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How frequently to reexamine health status", 2024-09-12T03:04:31.156 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.156 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.156 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.156 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.157 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-12T03:04:31.157 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-12T03:04:31.157 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.157 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.157 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.157 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.158 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.158 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.158 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.158 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:31.158 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:31.159 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:31.159 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:31.159 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:31.159 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:31.159 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.159 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.160 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.160 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.160 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.161 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.161 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-12T03:04:31.161 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-12T03:04:31.161 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.161 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.162 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.162 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.162 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.162 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.162 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.162 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.163 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.163 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.163 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.163 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.163 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-12T03:04:31.163 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-12T03:04:31.164 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.164 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.164 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.164 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-12T03:04:31.164 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.164 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.164 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.165 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:31.165 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:31.165 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:31.165 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:31.165 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:31.165 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:31.165 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.166 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.166 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.166 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.166 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.166 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.166 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-12T03:04:31.167 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-12T03:04:31.167 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.167 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.167 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.167 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.167 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.167 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.168 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.168 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.168 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.168 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.168 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.168 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.168 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_destination": { 2024-09-12T03:04:31.169 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_destination", 2024-09-12T03:04:31.169 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.169 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.169 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.169 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.169 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.169 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.170 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.170 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Email address to send alerts to", 2024-09-12T03:04:31.170 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.170 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.170 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.170 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.170 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_from_name": { 2024-09-12T03:04:31.170 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_from_name", 2024-09-12T03:04:31.171 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.171 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.171 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.171 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "Ceph", 2024-09-12T03:04:31.172 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.172 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.172 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.172 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Email From: name", 2024-09-12T03:04:31.172 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.172 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.172 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.173 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.173 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_host": { 2024-09-12T03:04:31.173 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_host", 2024-09-12T03:04:31.173 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.173 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.173 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.174 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.174 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.174 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.174 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.174 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SMTP server", 2024-09-12T03:04:31.174 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.174 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.175 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.175 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.175 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_password": { 2024-09-12T03:04:31.175 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_password", 2024-09-12T03:04:31.175 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.175 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.175 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.176 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.176 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.176 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.176 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.176 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Password to authenticate with", 2024-09-12T03:04:31.177 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.177 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.177 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.177 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.177 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_port": { 2024-09-12T03:04:31.178 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_port", 2024-09-12T03:04:31.178 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-12T03:04:31.178 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.178 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.178 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "465", 2024-09-12T03:04:31.179 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.179 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.179 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.179 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SMTP port", 2024-09-12T03:04:31.179 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.179 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.180 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.180 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.180 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_sender": { 2024-09-12T03:04:31.180 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_sender", 2024-09-12T03:04:31.180 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.181 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.181 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.181 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.181 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.181 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.182 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.182 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SMTP envelope sender", 2024-09-12T03:04:31.182 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.182 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.182 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.183 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.183 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_ssl": { 2024-09-12T03:04:31.183 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_ssl", 2024-09-12T03:04:31.183 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.183 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.184 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.184 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-12T03:04:31.184 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.184 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.184 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.184 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-12T03:04:31.185 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.185 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.185 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.185 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.185 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_user": { 2024-09-12T03:04:31.185 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_user", 2024-09-12T03:04:31.186 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.186 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.186 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.186 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.186 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.187 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.187 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.187 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "User to authenticate as", 2024-09-12T03:04:31.187 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.187 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.188 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.188 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:31.188 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:31.188 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.188 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-12T03:04:31.188 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "balancer", 2024-09-12T03:04:31.189 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-12T03:04:31.189 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-12T03:04:31.189 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-12T03:04:31.189 INFO:teuthology.orchestra.run.smithi005.stdout: "active": { 2024-09-12T03:04:31.189 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "active", 2024-09-12T03:04:31.189 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.190 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.190 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.190 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-12T03:04:31.190 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.191 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.191 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.191 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "automatically balance PGs across cluster", 2024-09-12T03:04:31.191 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.192 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.192 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.192 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.192 INFO:teuthology.orchestra.run.smithi005.stdout: "begin_time": { 2024-09-12T03:04:31.192 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "begin_time", 2024-09-12T03:04:31.193 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.193 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.193 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.193 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0000", 2024-09-12T03:04:31.193 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.194 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.194 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.194 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "beginning time of day to automatically balance", 2024-09-12T03:04:31.194 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-12T03:04:31.194 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.194 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.195 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.195 INFO:teuthology.orchestra.run.smithi005.stdout: "begin_weekday": { 2024-09-12T03:04:31.195 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "begin_weekday", 2024-09-12T03:04:31.195 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-09-12T03:04:31.195 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.196 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.196 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-09-12T03:04:31.196 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "0", 2024-09-12T03:04:31.196 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "6", 2024-09-12T03:04:31.196 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.197 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-12T03:04:31.197 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-12T03:04:31.197 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.197 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.197 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.197 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_compat_max_iterations": { 2024-09-12T03:04:31.198 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crush_compat_max_iterations", 2024-09-12T03:04:31.198 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-09-12T03:04:31.198 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.198 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.198 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "25", 2024-09-12T03:04:31.198 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2024-09-12T03:04:31.199 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "250", 2024-09-12T03:04:31.199 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.199 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-12T03:04:31.199 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.199 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.200 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.200 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.200 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_compat_metrics": { 2024-09-12T03:04:31.200 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crush_compat_metrics", 2024-09-12T03:04:31.200 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.201 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.201 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.201 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "pgs,objects,bytes", 2024-09-12T03:04:31.201 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.201 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.201 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.202 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-12T03:04:31.202 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-12T03:04:31.202 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.202 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.202 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.202 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_compat_step": { 2024-09-12T03:04:31.202 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crush_compat_step", 2024-09-12T03:04:31.202 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-09-12T03:04:31.203 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.203 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.203 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0.5", 2024-09-12T03:04:31.203 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "0.001", 2024-09-12T03:04:31.203 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "0.999", 2024-09-12T03:04:31.203 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.203 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "aggressiveness of optimization", 2024-09-12T03:04:31.203 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-12T03:04:31.203 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.203 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.203 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.203 INFO:teuthology.orchestra.run.smithi005.stdout: "end_time": { 2024-09-12T03:04:31.203 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "end_time", 2024-09-12T03:04:31.203 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.204 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.204 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.204 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "2359", 2024-09-12T03:04:31.204 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.204 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.204 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.204 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "ending time of day to automatically balance", 2024-09-12T03:04:31.204 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-12T03:04:31.204 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.204 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.204 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.204 INFO:teuthology.orchestra.run.smithi005.stdout: "end_weekday": { 2024-09-12T03:04:31.204 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "end_weekday", 2024-09-12T03:04:31.205 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-09-12T03:04:31.205 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.205 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.205 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-09-12T03:04:31.205 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "0", 2024-09-12T03:04:31.205 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "6", 2024-09-12T03:04:31.205 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.205 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-12T03:04:31.205 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-12T03:04:31.205 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.205 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.205 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.205 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-12T03:04:31.206 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-12T03:04:31.206 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.206 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.206 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.206 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.206 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.206 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.206 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.206 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:31.206 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:31.206 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:31.206 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:31.206 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:31.207 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:31.207 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.207 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.207 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.207 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.207 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.207 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.207 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-12T03:04:31.207 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-12T03:04:31.207 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.207 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.208 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.208 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.208 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.208 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.208 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.208 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.208 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.208 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.208 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.208 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.209 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-12T03:04:31.209 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-12T03:04:31.209 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.209 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.209 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.209 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-12T03:04:31.209 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.209 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.209 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.209 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:31.209 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:31.209 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:31.209 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:31.210 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:31.210 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:31.210 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.210 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.210 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.210 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.210 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.210 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.210 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-12T03:04:31.210 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-12T03:04:31.210 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.210 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.210 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.210 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.211 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.211 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.211 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.211 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.211 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.211 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.211 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.211 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.211 INFO:teuthology.orchestra.run.smithi005.stdout: "min_score": { 2024-09-12T03:04:31.211 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "min_score", 2024-09-12T03:04:31.211 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-09-12T03:04:31.211 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.211 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.212 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-09-12T03:04:31.212 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.212 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.212 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.212 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-12T03:04:31.212 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.212 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.212 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.212 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.212 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": { 2024-09-12T03:04:31.212 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mode", 2024-09-12T03:04:31.212 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.212 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.213 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.213 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "upmap", 2024-09-12T03:04:31.213 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.213 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.213 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.213 INFO:teuthology.orchestra.run.smithi005.stdout: "crush-compat", 2024-09-12T03:04:31.213 INFO:teuthology.orchestra.run.smithi005.stdout: "none", 2024-09-12T03:04:31.213 INFO:teuthology.orchestra.run.smithi005.stdout: "read", 2024-09-12T03:04:31.213 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap", 2024-09-12T03:04:31.213 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap-read" 2024-09-12T03:04:31.213 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.213 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Balancer mode", 2024-09-12T03:04:31.214 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.214 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.214 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.214 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.214 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_ids": { 2024-09-12T03:04:31.214 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pool_ids", 2024-09-12T03:04:31.214 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.214 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.214 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.214 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.214 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.214 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.214 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.215 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-12T03:04:31.215 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.215 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.215 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.215 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.215 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-09-12T03:04:31.215 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-09-12T03:04:31.215 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-12T03:04:31.215 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.215 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.215 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-09-12T03:04:31.215 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.215 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.215 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.216 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-12T03:04:31.216 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.216 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.216 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.216 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.216 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap_max_deviation": { 2024-09-12T03:04:31.216 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "upmap_max_deviation", 2024-09-12T03:04:31.216 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-12T03:04:31.216 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.216 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.216 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2024-09-12T03:04:31.216 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2024-09-12T03:04:31.216 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.217 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.217 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-12T03:04:31.217 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-12T03:04:31.217 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.217 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.217 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.217 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap_max_optimizations": { 2024-09-12T03:04:31.217 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "upmap_max_optimizations", 2024-09-12T03:04:31.217 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-09-12T03:04:31.217 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.217 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.217 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-09-12T03:04:31.217 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.218 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.218 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.218 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-12T03:04:31.218 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.218 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.218 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.218 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:31.218 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:31.218 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.218 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-12T03:04:31.218 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cephadm", 2024-09-12T03:04:31.218 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-12T03:04:31.218 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-12T03:04:31.218 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-12T03:04:31.219 INFO:teuthology.orchestra.run.smithi005.stdout: "agent_down_multiplier": { 2024-09-12T03:04:31.219 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "agent_down_multiplier", 2024-09-12T03:04:31.219 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-09-12T03:04:31.219 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.219 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.219 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "3.0", 2024-09-12T03:04:31.219 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.219 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.219 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.219 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-12T03:04:31.219 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.219 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.219 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.220 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.220 INFO:teuthology.orchestra.run.smithi005.stdout: "agent_refresh_rate": { 2024-09-12T03:04:31.220 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "agent_refresh_rate", 2024-09-12T03:04:31.220 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-12T03:04:31.220 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.220 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.220 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "20", 2024-09-12T03:04:31.220 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.220 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.220 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.220 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-12T03:04:31.220 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.220 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.221 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.221 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.221 INFO:teuthology.orchestra.run.smithi005.stdout: "agent_starting_port": { 2024-09-12T03:04:31.221 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "agent_starting_port", 2024-09-12T03:04:31.221 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-12T03:04:31.221 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.221 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.221 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "4721", 2024-09-12T03:04:31.221 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.221 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.221 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.221 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-12T03:04:31.222 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.222 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.222 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.222 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.222 INFO:teuthology.orchestra.run.smithi005.stdout: "allow_ptrace": { 2024-09-12T03:04:31.222 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "allow_ptrace", 2024-09-12T03:04:31.222 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.222 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.222 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.222 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.222 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.222 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.222 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.223 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-12T03:04:31.223 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-12T03:04:31.223 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.223 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.223 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.223 INFO:teuthology.orchestra.run.smithi005.stdout: "autotune_interval": { 2024-09-12T03:04:31.223 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "autotune_interval", 2024-09-12T03:04:31.223 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-12T03:04:31.223 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.223 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.223 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-09-12T03:04:31.223 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.223 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.224 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.224 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-12T03:04:31.224 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.224 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.224 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.224 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.224 INFO:teuthology.orchestra.run.smithi005.stdout: "autotune_memory_target_ratio": { 2024-09-12T03:04:31.224 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "autotune_memory_target_ratio", 2024-09-12T03:04:31.224 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-09-12T03:04:31.224 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.224 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.224 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0.7", 2024-09-12T03:04:31.224 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.225 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.225 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.225 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-12T03:04:31.225 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.225 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.225 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.225 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.225 INFO:teuthology.orchestra.run.smithi005.stdout: "cephadm_log_destination": { 2024-09-12T03:04:31.225 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cephadm_log_destination", 2024-09-12T03:04:31.225 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.225 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.225 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.225 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.226 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.226 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.226 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.226 INFO:teuthology.orchestra.run.smithi005.stdout: "file", 2024-09-12T03:04:31.226 INFO:teuthology.orchestra.run.smithi005.stdout: "file,syslog", 2024-09-12T03:04:31.226 INFO:teuthology.orchestra.run.smithi005.stdout: "syslog" 2024-09-12T03:04:31.226 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.226 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-12T03:04:31.226 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.226 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.226 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.226 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.226 INFO:teuthology.orchestra.run.smithi005.stdout: "cgroups_split": { 2024-09-12T03:04:31.226 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cgroups_split", 2024-09-12T03:04:31.227 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.227 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.227 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.227 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-12T03:04:31.227 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.227 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.227 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.227 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-12T03:04:31.227 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.227 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.227 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.227 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.228 INFO:teuthology.orchestra.run.smithi005.stdout: "config_checks_enabled": { 2024-09-12T03:04:31.228 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "config_checks_enabled", 2024-09-12T03:04:31.228 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.228 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.228 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.228 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.228 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.228 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.228 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.228 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-12T03:04:31.228 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.228 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.228 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.229 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.229 INFO:teuthology.orchestra.run.smithi005.stdout: "config_dashboard": { 2024-09-12T03:04:31.229 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "config_dashboard", 2024-09-12T03:04:31.229 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.229 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.229 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.229 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-12T03:04:31.229 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.229 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.229 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.229 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-12T03:04:31.229 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.229 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.230 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.230 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.230 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_alertmanager": { 2024-09-12T03:04:31.230 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_alertmanager", 2024-09-12T03:04:31.230 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.230 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.230 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.230 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-12T03:04:31.230 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.230 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.230 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.230 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2024-09-12T03:04:31.230 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.231 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.231 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.231 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.231 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_base": { 2024-09-12T03:04:31.231 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_base", 2024-09-12T03:04:31.231 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.231 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.231 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.231 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-12T03:04:31.231 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.231 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.231 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.231 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Container image name, without the tag", 2024-09-12T03:04:31.231 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.232 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.232 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.232 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.232 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_elasticsearch": { 2024-09-12T03:04:31.232 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_elasticsearch", 2024-09-12T03:04:31.232 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.232 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.232 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.232 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-12T03:04:31.232 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.232 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.232 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.232 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "elasticsearch container image", 2024-09-12T03:04:31.233 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.233 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.233 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.233 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.233 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_grafana": { 2024-09-12T03:04:31.233 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_grafana", 2024-09-12T03:04:31.233 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.233 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.233 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.233 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-12T03:04:31.233 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.233 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.233 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.234 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2024-09-12T03:04:31.234 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.234 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.234 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.234 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.234 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_haproxy": { 2024-09-12T03:04:31.234 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_haproxy", 2024-09-12T03:04:31.234 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.234 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.234 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.234 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-12T03:04:31.235 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.235 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.235 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.235 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "HAproxy container image", 2024-09-12T03:04:31.235 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.235 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.235 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.235 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.235 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_jaeger_agent": { 2024-09-12T03:04:31.235 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_jaeger_agent", 2024-09-12T03:04:31.235 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.235 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.235 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.236 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-12T03:04:31.236 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.236 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.236 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.236 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Jaeger agent container image", 2024-09-12T03:04:31.236 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.236 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.236 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.236 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.236 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_jaeger_collector": { 2024-09-12T03:04:31.236 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_jaeger_collector", 2024-09-12T03:04:31.236 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.236 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.237 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.237 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-12T03:04:31.237 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.237 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.237 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.237 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Jaeger collector container image", 2024-09-12T03:04:31.237 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.237 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.237 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.237 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.237 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_jaeger_query": { 2024-09-12T03:04:31.237 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_jaeger_query", 2024-09-12T03:04:31.237 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.238 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.238 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.238 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-12T03:04:31.238 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.238 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.238 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.238 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Jaeger query container image", 2024-09-12T03:04:31.238 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.238 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.238 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.238 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.238 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_keepalived": { 2024-09-12T03:04:31.238 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_keepalived", 2024-09-12T03:04:31.238 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.239 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.239 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.239 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-12T03:04:31.239 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.239 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.239 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.239 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Keepalived container image", 2024-09-12T03:04:31.239 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.239 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.239 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.239 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.239 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_loki": { 2024-09-12T03:04:31.239 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_loki", 2024-09-12T03:04:31.240 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.240 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.240 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.240 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-12T03:04:31.240 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.240 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.240 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.240 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Loki container image", 2024-09-12T03:04:31.240 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.240 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.240 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.240 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.240 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_node_exporter": { 2024-09-12T03:04:31.241 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_node_exporter", 2024-09-12T03:04:31.241 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.241 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.241 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.241 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-12T03:04:31.241 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.241 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.241 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.241 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2024-09-12T03:04:31.241 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.241 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.241 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.241 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.241 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_nvmeof": { 2024-09-12T03:04:31.242 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_nvmeof", 2024-09-12T03:04:31.242 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.242 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.242 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.242 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-12T03:04:31.242 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.242 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.242 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.242 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Nvme-of container image", 2024-09-12T03:04:31.242 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.242 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.242 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.242 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.242 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_prometheus": { 2024-09-12T03:04:31.243 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_prometheus", 2024-09-12T03:04:31.243 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.243 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.243 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.243 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-12T03:04:31.243 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.243 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.243 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.243 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2024-09-12T03:04:31.243 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.243 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.243 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.243 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.244 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_promtail": { 2024-09-12T03:04:31.244 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_promtail", 2024-09-12T03:04:31.244 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.244 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.244 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.244 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-12T03:04:31.244 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.244 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.244 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.244 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Promtail container image", 2024-09-12T03:04:31.244 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.244 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.244 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.245 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.245 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_samba": { 2024-09-12T03:04:31.245 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_samba", 2024-09-12T03:04:31.245 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.245 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.245 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.245 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-12T03:04:31.245 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.245 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.245 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.245 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Samba/SMB container image", 2024-09-12T03:04:31.245 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.245 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.246 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.246 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.246 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_snmp_gateway": { 2024-09-12T03:04:31.246 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_snmp_gateway", 2024-09-12T03:04:31.246 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.246 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.246 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.246 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-12T03:04:31.246 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.246 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.246 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.246 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SNMP Gateway container image", 2024-09-12T03:04:31.246 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.246 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.247 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.247 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.247 INFO:teuthology.orchestra.run.smithi005.stdout: "container_init": { 2024-09-12T03:04:31.247 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_init", 2024-09-12T03:04:31.247 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.247 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.247 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.247 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-12T03:04:31.247 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.247 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.247 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.248 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Run podman/docker with `--init`", 2024-09-12T03:04:31.248 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.248 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.248 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.248 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.248 INFO:teuthology.orchestra.run.smithi005.stdout: "daemon_cache_timeout": { 2024-09-12T03:04:31.248 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "daemon_cache_timeout", 2024-09-12T03:04:31.248 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-12T03:04:31.248 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.248 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.248 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-09-12T03:04:31.248 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.248 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.249 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.249 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-12T03:04:31.249 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.249 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.249 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.249 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.249 INFO:teuthology.orchestra.run.smithi005.stdout: "default_cephadm_command_timeout": { 2024-09-12T03:04:31.249 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "default_cephadm_command_timeout", 2024-09-12T03:04:31.249 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-12T03:04:31.249 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.249 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.249 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "900", 2024-09-12T03:04:31.249 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.250 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.250 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.250 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-12T03:04:31.250 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.250 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.250 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.250 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.250 INFO:teuthology.orchestra.run.smithi005.stdout: "default_registry": { 2024-09-12T03:04:31.250 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "default_registry", 2024-09-12T03:04:31.250 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.250 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.250 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.251 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io", 2024-09-12T03:04:31.251 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.251 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.251 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.251 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-12T03:04:31.251 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.251 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.251 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.251 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.251 INFO:teuthology.orchestra.run.smithi005.stdout: "device_cache_timeout": { 2024-09-12T03:04:31.251 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "device_cache_timeout", 2024-09-12T03:04:31.251 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-12T03:04:31.251 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.252 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.252 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "1800", 2024-09-12T03:04:31.252 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.252 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.252 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.252 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "seconds to cache device inventory", 2024-09-12T03:04:31.252 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.252 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.252 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.252 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.252 INFO:teuthology.orchestra.run.smithi005.stdout: "device_enhanced_scan": { 2024-09-12T03:04:31.252 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "device_enhanced_scan", 2024-09-12T03:04:31.252 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.253 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.253 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.253 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.253 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.253 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.253 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.253 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-12T03:04:31.253 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.253 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.253 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.253 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.253 INFO:teuthology.orchestra.run.smithi005.stdout: "facts_cache_timeout": { 2024-09-12T03:04:31.253 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "facts_cache_timeout", 2024-09-12T03:04:31.254 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-12T03:04:31.254 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.254 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.254 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-09-12T03:04:31.254 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.254 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.254 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.254 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "seconds to cache host facts data", 2024-09-12T03:04:31.254 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.254 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.254 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.254 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.254 INFO:teuthology.orchestra.run.smithi005.stdout: "grafana_dashboards_path": { 2024-09-12T03:04:31.255 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "grafana_dashboards_path", 2024-09-12T03:04:31.255 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.255 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.255 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.255 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-12T03:04:31.255 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.255 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.255 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.255 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-12T03:04:31.255 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.255 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.255 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.255 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.256 INFO:teuthology.orchestra.run.smithi005.stdout: "host_check_interval": { 2024-09-12T03:04:31.256 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "host_check_interval", 2024-09-12T03:04:31.256 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-12T03:04:31.256 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.256 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.256 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-09-12T03:04:31.256 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.256 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.256 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.256 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to perform a host check", 2024-09-12T03:04:31.256 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.256 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.256 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.257 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.257 INFO:teuthology.orchestra.run.smithi005.stdout: "hw_monitoring": { 2024-09-12T03:04:31.257 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "hw_monitoring", 2024-09-12T03:04:31.257 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.257 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.257 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.257 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.257 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.257 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.257 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.257 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-12T03:04:31.257 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.257 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.258 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.258 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.258 INFO:teuthology.orchestra.run.smithi005.stdout: "inventory_list_all": { 2024-09-12T03:04:31.258 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "inventory_list_all", 2024-09-12T03:04:31.258 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.258 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.258 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.258 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.258 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.258 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.258 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.258 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-12T03:04:31.258 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.258 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.259 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.259 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.259 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-12T03:04:31.259 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-12T03:04:31.259 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.259 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.259 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.259 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.259 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.259 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.259 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.259 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:31.259 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:31.260 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:31.260 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:31.260 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:31.260 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:31.260 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.260 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.260 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.260 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.260 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.260 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.260 INFO:teuthology.orchestra.run.smithi005.stdout: "log_refresh_metadata": { 2024-09-12T03:04:31.260 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_refresh_metadata", 2024-09-12T03:04:31.260 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.260 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.261 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.261 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.261 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.261 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.261 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.261 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-12T03:04:31.261 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.261 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.261 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.261 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.262 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-12T03:04:31.262 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-12T03:04:31.262 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.262 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.262 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.262 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-12T03:04:31.262 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.262 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.262 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.262 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-12T03:04:31.262 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.262 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.262 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.262 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.263 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-12T03:04:31.263 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-12T03:04:31.263 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.263 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.263 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.263 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-12T03:04:31.263 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.263 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.263 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.263 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:31.263 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:31.263 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:31.263 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:31.264 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:31.264 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:31.264 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.264 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.264 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.264 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.264 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.264 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.264 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-12T03:04:31.264 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-12T03:04:31.264 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.264 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.264 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.265 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.265 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.265 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.265 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.265 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.265 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.265 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.265 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.265 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.265 INFO:teuthology.orchestra.run.smithi005.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-12T03:04:31.265 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-12T03:04:31.265 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.265 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.266 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.266 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.266 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.266 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.266 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.266 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-12T03:04:31.266 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.266 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.266 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.266 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.266 INFO:teuthology.orchestra.run.smithi005.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-12T03:04:31.266 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-12T03:04:31.266 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.266 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.267 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.267 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "*", 2024-09-12T03:04:31.267 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.267 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.267 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.267 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-12T03:04:31.267 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.267 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.267 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.267 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.267 INFO:teuthology.orchestra.run.smithi005.stdout: "max_count_per_host": { 2024-09-12T03:04:31.267 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_count_per_host", 2024-09-12T03:04:31.267 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-12T03:04:31.268 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.268 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.268 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-09-12T03:04:31.268 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.268 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.268 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.268 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "max number of daemons per service per host", 2024-09-12T03:04:31.268 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.268 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.268 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.268 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.268 INFO:teuthology.orchestra.run.smithi005.stdout: "max_osd_draining_count": { 2024-09-12T03:04:31.268 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_osd_draining_count", 2024-09-12T03:04:31.269 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-12T03:04:31.269 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.269 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.269 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-09-12T03:04:31.269 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.269 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.269 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.269 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-12T03:04:31.269 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.269 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.269 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.269 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.269 INFO:teuthology.orchestra.run.smithi005.stdout: "migration_current": { 2024-09-12T03:04:31.269 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "migration_current", 2024-09-12T03:04:31.270 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-12T03:04:31.270 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.270 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.270 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.270 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.270 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.270 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.270 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "internal - do not modify", 2024-09-12T03:04:31.270 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.270 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.270 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.270 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.270 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": { 2024-09-12T03:04:31.270 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mode", 2024-09-12T03:04:31.271 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.271 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.271 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.271 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "root", 2024-09-12T03:04:31.271 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.271 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.271 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.271 INFO:teuthology.orchestra.run.smithi005.stdout: "cephadm-package", 2024-09-12T03:04:31.271 INFO:teuthology.orchestra.run.smithi005.stdout: "root" 2024-09-12T03:04:31.271 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.271 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "mode for remote execution of cephadm", 2024-09-12T03:04:31.271 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.271 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.272 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.272 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.272 INFO:teuthology.orchestra.run.smithi005.stdout: "oob_default_addr": { 2024-09-12T03:04:31.272 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "oob_default_addr", 2024-09-12T03:04:31.272 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.272 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.272 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.272 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "169.254.1.1", 2024-09-12T03:04:31.272 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.272 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.272 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.272 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-12T03:04:31.272 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.273 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.273 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.273 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.273 INFO:teuthology.orchestra.run.smithi005.stdout: "prometheus_alerts_path": { 2024-09-12T03:04:31.273 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "prometheus_alerts_path", 2024-09-12T03:04:31.273 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.273 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.273 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.273 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-12T03:04:31.273 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.273 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.273 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.273 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-12T03:04:31.273 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.274 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.274 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.274 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.274 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_insecure": { 2024-09-12T03:04:31.274 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_insecure", 2024-09-12T03:04:31.274 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.274 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.274 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.274 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.274 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.274 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.274 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.275 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-12T03:04:31.275 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.275 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.275 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.275 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.275 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_password": { 2024-09-12T03:04:31.275 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_password", 2024-09-12T03:04:31.275 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.275 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.275 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.275 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.275 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.275 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.276 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.276 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-12T03:04:31.276 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.276 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.276 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.276 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.276 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_url": { 2024-09-12T03:04:31.276 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_url", 2024-09-12T03:04:31.276 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.276 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.276 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.276 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.276 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.277 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.277 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.277 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-12T03:04:31.277 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.277 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.277 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.277 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.277 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_username": { 2024-09-12T03:04:31.277 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_username", 2024-09-12T03:04:31.277 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.277 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.277 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.277 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.278 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.278 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.278 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.278 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-12T03:04:31.278 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.278 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.278 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.278 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.278 INFO:teuthology.orchestra.run.smithi005.stdout: "secure_monitoring_stack": { 2024-09-12T03:04:31.278 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "secure_monitoring_stack", 2024-09-12T03:04:31.278 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.278 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.278 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.279 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.279 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.279 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.279 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.279 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-12T03:04:31.279 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.279 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.279 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.279 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.279 INFO:teuthology.orchestra.run.smithi005.stdout: "service_discovery_port": { 2024-09-12T03:04:31.279 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "service_discovery_port", 2024-09-12T03:04:31.279 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-12T03:04:31.279 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.280 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.280 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8765", 2024-09-12T03:04:31.280 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.280 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.280 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.280 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "cephadm service discovery port", 2024-09-12T03:04:31.280 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.280 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.280 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.280 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.280 INFO:teuthology.orchestra.run.smithi005.stdout: "ssh_config_file": { 2024-09-12T03:04:31.280 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssh_config_file", 2024-09-12T03:04:31.280 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.281 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.281 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.281 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.281 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.281 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.281 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.281 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-12T03:04:31.281 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.281 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.281 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.281 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.281 INFO:teuthology.orchestra.run.smithi005.stdout: "use_agent": { 2024-09-12T03:04:31.281 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "use_agent", 2024-09-12T03:04:31.281 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.282 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.282 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.282 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.282 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.282 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.282 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.282 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-12T03:04:31.282 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.282 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.282 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.282 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.282 INFO:teuthology.orchestra.run.smithi005.stdout: "use_repo_digest": { 2024-09-12T03:04:31.282 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "use_repo_digest", 2024-09-12T03:04:31.283 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.283 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.283 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.283 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-12T03:04:31.283 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.283 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.283 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.283 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-12T03:04:31.283 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.283 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.283 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.283 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.283 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_on_failed_host_check": { 2024-09-12T03:04:31.284 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_on_failed_host_check", 2024-09-12T03:04:31.284 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.284 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.284 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.284 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-12T03:04:31.284 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.284 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.284 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.284 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise a health warning if the host check fails", 2024-09-12T03:04:31.284 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.284 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.284 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.284 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.285 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_on_stray_daemons": { 2024-09-12T03:04:31.285 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_on_stray_daemons", 2024-09-12T03:04:31.285 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.285 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.285 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.285 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-12T03:04:31.285 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.285 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.285 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.285 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-12T03:04:31.285 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.285 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.285 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.286 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.286 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_on_stray_hosts": { 2024-09-12T03:04:31.286 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_on_stray_hosts", 2024-09-12T03:04:31.286 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.286 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.286 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.286 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-12T03:04:31.286 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.286 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.286 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.286 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-12T03:04:31.286 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.286 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.286 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.287 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:31.287 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:31.287 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.287 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-12T03:04:31.287 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crash", 2024-09-12T03:04:31.287 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-12T03:04:31.287 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-12T03:04:31.287 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-12T03:04:31.287 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-12T03:04:31.287 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-12T03:04:31.287 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.288 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.288 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.288 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.288 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.288 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.288 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.288 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:31.288 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:31.288 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:31.288 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:31.288 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:31.288 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:31.288 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.289 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.289 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.289 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.289 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.289 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.289 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-12T03:04:31.289 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-12T03:04:31.289 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.289 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.289 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.289 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.289 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.289 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.290 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.290 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.290 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.290 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.290 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.290 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.290 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-12T03:04:31.290 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-12T03:04:31.290 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.290 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.290 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.291 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-12T03:04:31.291 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.291 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.291 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.291 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:31.291 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:31.291 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:31.291 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:31.291 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:31.291 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:31.291 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.291 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.292 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.292 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.292 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.292 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.292 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-12T03:04:31.292 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-12T03:04:31.292 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.292 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.292 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.292 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.292 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.292 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.292 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.293 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.293 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.293 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.293 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.293 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.293 INFO:teuthology.orchestra.run.smithi005.stdout: "retain_interval": { 2024-09-12T03:04:31.293 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "retain_interval", 2024-09-12T03:04:31.293 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-12T03:04:31.293 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.293 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.293 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "31536000", 2024-09-12T03:04:31.293 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.294 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.294 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.294 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-12T03:04:31.294 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.294 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.294 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.294 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.294 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_recent_interval": { 2024-09-12T03:04:31.294 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_recent_interval", 2024-09-12T03:04:31.294 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-12T03:04:31.295 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.295 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.295 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "1209600", 2024-09-12T03:04:31.295 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.295 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.295 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.295 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-12T03:04:31.295 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.295 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.295 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.295 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:31.295 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:31.295 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.296 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-12T03:04:31.296 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "dashboard", 2024-09-12T03:04:31.296 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-12T03:04:31.296 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-12T03:04:31.296 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-12T03:04:31.296 INFO:teuthology.orchestra.run.smithi005.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-12T03:04:31.296 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-12T03:04:31.296 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-12T03:04:31.296 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.296 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.296 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-09-12T03:04:31.296 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.296 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.297 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.297 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.297 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.297 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.297 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.297 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.297 INFO:teuthology.orchestra.run.smithi005.stdout: "ALERTMANAGER_API_HOST": { 2024-09-12T03:04:31.297 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-12T03:04:31.297 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.297 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.297 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.297 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.297 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.297 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.298 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.298 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.298 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.298 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.298 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.298 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.298 INFO:teuthology.orchestra.run.smithi005.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-12T03:04:31.298 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-12T03:04:31.298 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.298 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.298 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.298 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-12T03:04:31.299 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.299 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.299 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.299 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.299 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.299 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.299 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.299 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.299 INFO:teuthology.orchestra.run.smithi005.stdout: "AUDIT_API_ENABLED": { 2024-09-12T03:04:31.299 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "AUDIT_API_ENABLED", 2024-09-12T03:04:31.299 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.299 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.299 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.300 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.300 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.300 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.300 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.300 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.300 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.300 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.300 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.300 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.300 INFO:teuthology.orchestra.run.smithi005.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-12T03:04:31.300 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-12T03:04:31.300 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.300 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.301 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.301 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-12T03:04:31.301 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.301 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.301 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.301 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.301 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.301 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.301 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.301 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.302 INFO:teuthology.orchestra.run.smithi005.stdout: "ENABLE_BROWSABLE_API": { 2024-09-12T03:04:31.302 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-12T03:04:31.302 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.302 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.302 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.302 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-12T03:04:31.302 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.302 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.302 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.302 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.302 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.303 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.303 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.303 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.303 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-12T03:04:31.303 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-12T03:04:31.303 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.303 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.303 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.303 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-12T03:04:31.303 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.303 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.303 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.303 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.304 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.304 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.304 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.304 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.304 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-12T03:04:31.304 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-12T03:04:31.304 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.304 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.304 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.304 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-12T03:04:31.304 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.304 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.305 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.305 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.305 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.305 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.305 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.305 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.305 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-12T03:04:31.305 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-12T03:04:31.305 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.305 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.305 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.305 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-12T03:04:31.305 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.306 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.306 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.306 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.306 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.306 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.306 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.306 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.306 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-12T03:04:31.306 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-12T03:04:31.306 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.306 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.306 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.306 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-12T03:04:31.307 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.307 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.307 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.307 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.307 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.307 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.307 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.307 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.307 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-12T03:04:31.307 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-12T03:04:31.307 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.307 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.307 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.308 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-12T03:04:31.308 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.308 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.308 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.308 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.308 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.308 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.308 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.308 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.308 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-12T03:04:31.308 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-12T03:04:31.308 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.308 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.309 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.309 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-12T03:04:31.309 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.309 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.309 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.309 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.309 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.309 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.309 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.309 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.309 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-12T03:04:31.309 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-12T03:04:31.309 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.310 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.310 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.310 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-12T03:04:31.310 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.310 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.310 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.310 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.310 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.310 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.310 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.310 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.310 INFO:teuthology.orchestra.run.smithi005.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-12T03:04:31.310 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-12T03:04:31.311 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.311 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.311 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.311 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.311 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.311 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.311 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.311 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.311 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.311 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.311 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.311 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.311 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_PASSWORD": { 2024-09-12T03:04:31.312 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-12T03:04:31.312 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.312 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.312 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.312 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "admin", 2024-09-12T03:04:31.312 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.312 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.312 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.312 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.312 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.312 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.312 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.312 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.313 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-12T03:04:31.313 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-12T03:04:31.313 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.313 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.313 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.313 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-12T03:04:31.313 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.313 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.313 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.313 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.313 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.313 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.313 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.313 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.314 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_URL": { 2024-09-12T03:04:31.314 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_URL", 2024-09-12T03:04:31.314 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.314 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.314 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.314 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.314 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.314 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.314 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.314 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.314 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.314 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.314 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.314 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.315 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_USERNAME": { 2024-09-12T03:04:31.315 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-12T03:04:31.315 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.315 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.315 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.315 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "admin", 2024-09-12T03:04:31.315 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.315 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.315 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.315 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.315 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.315 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.315 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.315 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.316 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-12T03:04:31.316 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-12T03:04:31.316 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.316 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.316 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.316 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.316 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.316 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.316 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.316 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.316 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.316 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.316 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.317 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.317 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-12T03:04:31.317 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-12T03:04:31.317 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.317 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.317 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.317 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.317 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.317 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.317 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.317 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.317 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.318 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.318 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.318 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.318 INFO:teuthology.orchestra.run.smithi005.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-12T03:04:31.318 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-12T03:04:31.318 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.318 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.318 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.318 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-12T03:04:31.318 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.318 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.318 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.318 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.318 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.319 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.319 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.319 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.319 INFO:teuthology.orchestra.run.smithi005.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-12T03:04:31.319 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-12T03:04:31.319 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.319 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.319 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.319 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.319 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.319 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.319 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.319 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.320 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.320 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.320 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.320 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.320 INFO:teuthology.orchestra.run.smithi005.stdout: "PROMETHEUS_API_HOST": { 2024-09-12T03:04:31.320 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-12T03:04:31.320 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.320 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.320 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.320 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.320 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.320 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.320 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.321 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.321 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.321 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.321 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.321 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.321 INFO:teuthology.orchestra.run.smithi005.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-12T03:04:31.321 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-12T03:04:31.321 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.321 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.321 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.321 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-12T03:04:31.321 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.321 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.322 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.322 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.322 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.322 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.322 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.322 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.322 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-12T03:04:31.322 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-12T03:04:31.322 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.322 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.322 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.322 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.322 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.322 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.323 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.323 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.323 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.323 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.323 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.323 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.323 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-12T03:04:31.323 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-12T03:04:31.323 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.323 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.323 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.323 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.323 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.324 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.324 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.324 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.324 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.324 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.324 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.324 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.324 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-12T03:04:31.324 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-12T03:04:31.324 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.324 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.324 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.324 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-12T03:04:31.325 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.325 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.325 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.325 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.325 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.325 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.325 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.325 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.325 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-12T03:04:31.325 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-12T03:04:31.325 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.325 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.325 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.325 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-12T03:04:31.326 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.326 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.326 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.326 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.326 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.326 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.326 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.326 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.326 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-12T03:04:31.326 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-12T03:04:31.326 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.326 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.326 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.326 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.327 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.327 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.327 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.327 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.327 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.327 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.327 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.327 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.327 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-12T03:04:31.327 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-12T03:04:31.327 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.327 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.327 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.328 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.328 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.328 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.328 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.328 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.328 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.328 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.328 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.328 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.328 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-12T03:04:31.328 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-12T03:04:31.328 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.328 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.329 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.329 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.329 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.329 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.329 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.329 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.329 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.329 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.329 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.329 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.329 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_ENABLED": { 2024-09-12T03:04:31.329 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-12T03:04:31.329 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.329 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.330 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.330 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-12T03:04:31.330 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.330 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.330 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.330 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.330 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.330 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.330 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.330 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.330 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-12T03:04:31.330 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-12T03:04:31.331 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.331 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.331 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.331 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-12T03:04:31.331 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.331 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.331 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.331 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.331 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.331 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.331 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.331 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.331 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-12T03:04:31.332 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-12T03:04:31.332 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-12T03:04:31.332 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.332 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.332 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-09-12T03:04:31.332 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.332 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.332 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.332 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.332 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.332 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.332 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.332 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.333 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-12T03:04:31.333 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-12T03:04:31.333 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-12T03:04:31.333 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.333 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.333 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8", 2024-09-12T03:04:31.333 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.333 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.333 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.333 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.333 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.333 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.333 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.334 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.334 INFO:teuthology.orchestra.run.smithi005.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-12T03:04:31.334 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-12T03:04:31.334 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-12T03:04:31.334 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.334 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.334 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "45", 2024-09-12T03:04:31.334 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.334 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.334 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.334 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.334 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.334 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.335 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.335 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.335 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_ACCESS_KEY": { 2024-09-12T03:04:31.335 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-12T03:04:31.335 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.335 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.335 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.335 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.335 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.335 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.335 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.335 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.335 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.336 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.336 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.336 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.336 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-12T03:04:31.336 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-12T03:04:31.336 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.336 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.336 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.336 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "admin", 2024-09-12T03:04:31.336 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.336 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.336 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.336 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.337 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.337 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.337 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.337 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.337 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_SECRET_KEY": { 2024-09-12T03:04:31.337 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-12T03:04:31.337 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.337 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.337 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.337 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.337 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.337 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.337 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.338 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.338 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.338 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.338 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.338 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.338 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_SSL_VERIFY": { 2024-09-12T03:04:31.338 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-12T03:04:31.338 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.338 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.338 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.338 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-12T03:04:31.338 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.338 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.338 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.339 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.339 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.339 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.339 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.339 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.339 INFO:teuthology.orchestra.run.smithi005.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-12T03:04:31.339 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-12T03:04:31.339 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-12T03:04:31.339 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.339 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.339 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-09-12T03:04:31.339 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.339 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.339 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.340 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.340 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.340 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.340 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.340 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.340 INFO:teuthology.orchestra.run.smithi005.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-12T03:04:31.340 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-12T03:04:31.340 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-12T03:04:31.340 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.340 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.340 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-09-12T03:04:31.340 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.340 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.341 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.341 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.341 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.341 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.341 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.341 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.341 INFO:teuthology.orchestra.run.smithi005.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-12T03:04:31.341 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-12T03:04:31.341 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-12T03:04:31.341 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.341 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.341 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2024-09-12T03:04:31.341 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.341 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.342 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.342 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.342 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.342 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.342 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.342 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.342 INFO:teuthology.orchestra.run.smithi005.stdout: "cross_origin_url": { 2024-09-12T03:04:31.342 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cross_origin_url", 2024-09-12T03:04:31.342 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.342 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.342 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.342 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.342 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.343 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.343 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.343 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.343 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.343 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.343 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.343 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.343 INFO:teuthology.orchestra.run.smithi005.stdout: "crt_file": { 2024-09-12T03:04:31.343 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crt_file", 2024-09-12T03:04:31.343 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.343 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.343 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.343 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.343 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.344 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.344 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.344 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.344 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.344 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.344 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.344 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.344 INFO:teuthology.orchestra.run.smithi005.stdout: "debug": { 2024-09-12T03:04:31.344 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "debug", 2024-09-12T03:04:31.344 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.344 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.344 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.345 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.345 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.345 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.345 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.345 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Enable/disable debug options", 2024-09-12T03:04:31.345 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.345 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.345 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.345 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.345 INFO:teuthology.orchestra.run.smithi005.stdout: "jwt_token_ttl": { 2024-09-12T03:04:31.345 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "jwt_token_ttl", 2024-09-12T03:04:31.345 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-12T03:04:31.345 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.346 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.346 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "28800", 2024-09-12T03:04:31.346 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.346 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.346 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.346 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.346 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.346 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.346 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.346 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.346 INFO:teuthology.orchestra.run.smithi005.stdout: "key_file": { 2024-09-12T03:04:31.346 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "key_file", 2024-09-12T03:04:31.346 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.347 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.347 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.347 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.347 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.347 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.347 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.347 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.347 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.347 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.347 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.347 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.347 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-12T03:04:31.347 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-12T03:04:31.348 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.348 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.348 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.348 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.348 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.348 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.348 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.348 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:31.348 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:31.348 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:31.348 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:31.348 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:31.348 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:31.349 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.349 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.349 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.349 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.349 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.349 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.349 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-12T03:04:31.349 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-12T03:04:31.349 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.349 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.349 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.349 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.349 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.349 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.350 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.350 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.350 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.350 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.350 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.350 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.350 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-12T03:04:31.350 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-12T03:04:31.350 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.350 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.350 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.350 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-12T03:04:31.350 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.351 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.351 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.351 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:31.351 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:31.351 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:31.351 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:31.351 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:31.351 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:31.351 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.351 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.351 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.351 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.351 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.352 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.352 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-12T03:04:31.352 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-12T03:04:31.352 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.352 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.352 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.352 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.352 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.352 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.352 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.352 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.352 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.352 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.353 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.353 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.353 INFO:teuthology.orchestra.run.smithi005.stdout: "motd": { 2024-09-12T03:04:31.353 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "motd", 2024-09-12T03:04:31.353 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.353 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.353 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.353 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.353 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.353 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.353 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.353 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "The message of the day", 2024-09-12T03:04:31.353 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.353 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.354 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.354 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.354 INFO:teuthology.orchestra.run.smithi005.stdout: "redirect_resolve_ip_addr": { 2024-09-12T03:04:31.354 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "redirect_resolve_ip_addr", 2024-09-12T03:04:31.354 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.354 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.354 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.354 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.354 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.354 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.354 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.354 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.354 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.355 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.355 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.355 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.355 INFO:teuthology.orchestra.run.smithi005.stdout: "server_addr": { 2024-09-12T03:04:31.355 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_addr", 2024-09-12T03:04:31.355 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.355 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.355 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.355 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "::", 2024-09-12T03:04:31.355 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.355 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.355 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.355 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.355 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.356 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.356 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.356 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.356 INFO:teuthology.orchestra.run.smithi005.stdout: "server_port": { 2024-09-12T03:04:31.356 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_port", 2024-09-12T03:04:31.356 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-12T03:04:31.356 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.356 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.356 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8080", 2024-09-12T03:04:31.356 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.356 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.356 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.356 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.357 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.357 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.357 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.357 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.357 INFO:teuthology.orchestra.run.smithi005.stdout: "ssl": { 2024-09-12T03:04:31.357 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssl", 2024-09-12T03:04:31.357 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.357 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.357 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.357 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-12T03:04:31.357 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.357 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.357 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.357 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.358 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.358 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.358 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.358 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.358 INFO:teuthology.orchestra.run.smithi005.stdout: "ssl_server_port": { 2024-09-12T03:04:31.358 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssl_server_port", 2024-09-12T03:04:31.358 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-12T03:04:31.358 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.358 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.358 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8443", 2024-09-12T03:04:31.358 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.358 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.358 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.359 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.359 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.359 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.359 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.359 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.359 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_behaviour": { 2024-09-12T03:04:31.359 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_behaviour", 2024-09-12T03:04:31.359 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.359 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.359 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.359 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "redirect", 2024-09-12T03:04:31.359 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.360 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.360 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.360 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:31.360 INFO:teuthology.orchestra.run.smithi005.stdout: "redirect" 2024-09-12T03:04:31.360 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.360 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.360 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.360 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.360 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.360 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.360 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_error_status_code": { 2024-09-12T03:04:31.360 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_error_status_code", 2024-09-12T03:04:31.360 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-12T03:04:31.361 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.361 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.361 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "500", 2024-09-12T03:04:31.361 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "400", 2024-09-12T03:04:31.361 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "599", 2024-09-12T03:04:31.361 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.361 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.361 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.361 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.361 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.361 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.361 INFO:teuthology.orchestra.run.smithi005.stdout: "url_prefix": { 2024-09-12T03:04:31.361 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "url_prefix", 2024-09-12T03:04:31.362 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.362 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.362 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.362 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.362 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.362 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.362 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.362 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.362 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.362 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.362 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.362 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:31.362 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:31.363 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.364 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-12T03:04:31.364 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "devicehealth", 2024-09-12T03:04:31.364 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-12T03:04:31.364 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-12T03:04:31.364 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-12T03:04:31.364 INFO:teuthology.orchestra.run.smithi005.stdout: "enable_monitoring": { 2024-09-12T03:04:31.364 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enable_monitoring", 2024-09-12T03:04:31.364 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.364 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.364 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.364 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-12T03:04:31.364 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.365 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.365 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.365 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "monitor device health metrics", 2024-09-12T03:04:31.365 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.365 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.365 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.365 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.365 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-12T03:04:31.365 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-12T03:04:31.365 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.365 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.365 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.365 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.366 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.366 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.366 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.366 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:31.366 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:31.366 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:31.366 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:31.366 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:31.366 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:31.366 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.366 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.366 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.366 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.367 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.367 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.367 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-12T03:04:31.367 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-12T03:04:31.367 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.367 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.367 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.367 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.367 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.367 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.367 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.367 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.367 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.367 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.368 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.368 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.368 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-12T03:04:31.368 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-12T03:04:31.368 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.368 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.368 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.368 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-12T03:04:31.368 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.368 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.368 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.368 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:31.368 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:31.369 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:31.369 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:31.369 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:31.369 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:31.369 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.369 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.369 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.369 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.369 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.369 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.369 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-12T03:04:31.369 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-12T03:04:31.369 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.369 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.370 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.370 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.370 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.370 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.370 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.370 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.370 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.370 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.370 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.370 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.370 INFO:teuthology.orchestra.run.smithi005.stdout: "mark_out_threshold": { 2024-09-12T03:04:31.370 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mark_out_threshold", 2024-09-12T03:04:31.370 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-12T03:04:31.371 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.371 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.371 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "2419200", 2024-09-12T03:04:31.371 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.371 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.371 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.371 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-12T03:04:31.371 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.371 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.371 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.371 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.371 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_name": { 2024-09-12T03:04:31.371 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pool_name", 2024-09-12T03:04:31.372 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.372 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.372 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.372 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "device_health_metrics", 2024-09-12T03:04:31.372 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.372 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.372 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.372 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-12T03:04:31.372 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.372 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.372 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.372 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.372 INFO:teuthology.orchestra.run.smithi005.stdout: "retention_period": { 2024-09-12T03:04:31.372 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "retention_period", 2024-09-12T03:04:31.373 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-12T03:04:31.373 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.373 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.373 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "15552000", 2024-09-12T03:04:31.373 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.373 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.373 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.373 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how long to retain device health metrics", 2024-09-12T03:04:31.373 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.373 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.373 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.373 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.373 INFO:teuthology.orchestra.run.smithi005.stdout: "scrape_frequency": { 2024-09-12T03:04:31.373 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "scrape_frequency", 2024-09-12T03:04:31.374 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-12T03:04:31.374 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.374 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.374 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "86400", 2024-09-12T03:04:31.374 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.374 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.374 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.374 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-12T03:04:31.374 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.374 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.374 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.375 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.375 INFO:teuthology.orchestra.run.smithi005.stdout: "self_heal": { 2024-09-12T03:04:31.375 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "self_heal", 2024-09-12T03:04:31.375 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.375 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.375 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.375 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-12T03:04:31.375 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.375 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.375 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.375 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-12T03:04:31.375 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.375 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.375 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.376 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.376 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-09-12T03:04:31.376 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-09-12T03:04:31.376 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-12T03:04:31.376 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.376 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.376 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-09-12T03:04:31.376 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.376 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.376 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.376 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to wake up and check device health", 2024-09-12T03:04:31.376 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.376 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.377 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.377 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.377 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_threshold": { 2024-09-12T03:04:31.377 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_threshold", 2024-09-12T03:04:31.377 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-12T03:04:31.377 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.377 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.377 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "7257600", 2024-09-12T03:04:31.377 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.377 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.377 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.377 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-12T03:04:31.377 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.378 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.378 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.378 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:31.378 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:31.378 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.378 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-12T03:04:31.378 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "diskprediction_local", 2024-09-12T03:04:31.378 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-12T03:04:31.378 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-12T03:04:31.378 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-12T03:04:31.378 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-12T03:04:31.378 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-12T03:04:31.378 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.379 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.379 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.379 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.379 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.379 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.379 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.379 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:31.379 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:31.379 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:31.379 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:31.379 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:31.379 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:31.379 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.380 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.380 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.380 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.380 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.380 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.380 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-12T03:04:31.380 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-12T03:04:31.380 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.380 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.380 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.380 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.380 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.380 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.381 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.381 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.381 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.381 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.381 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.381 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.381 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-12T03:04:31.381 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-12T03:04:31.381 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.381 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.381 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.381 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-12T03:04:31.381 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.381 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.382 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.382 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:31.382 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:31.382 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:31.382 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:31.382 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:31.382 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:31.382 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.382 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.382 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.382 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.382 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.382 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.383 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-12T03:04:31.383 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-12T03:04:31.383 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.383 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.383 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.383 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.383 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.383 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.383 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.383 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.383 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.383 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.383 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.383 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.384 INFO:teuthology.orchestra.run.smithi005.stdout: "predict_interval": { 2024-09-12T03:04:31.384 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "predict_interval", 2024-09-12T03:04:31.384 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.384 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.384 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.384 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "86400", 2024-09-12T03:04:31.384 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.384 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.384 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.384 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.384 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.384 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.384 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.385 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.385 INFO:teuthology.orchestra.run.smithi005.stdout: "predictor_model": { 2024-09-12T03:04:31.385 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "predictor_model", 2024-09-12T03:04:31.385 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.385 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.385 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.385 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "prophetstor", 2024-09-12T03:04:31.385 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.385 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.385 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.385 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.385 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.385 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.386 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.386 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.386 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-09-12T03:04:31.386 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-09-12T03:04:31.386 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.386 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.386 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.386 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-09-12T03:04:31.386 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.386 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.386 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.386 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.386 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.386 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.387 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.387 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:31.387 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:31.387 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.387 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-12T03:04:31.387 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "influx", 2024-09-12T03:04:31.387 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": false, 2024-09-12T03:04:31.387 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "influxdb python module not found", 2024-09-12T03:04:31.387 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-12T03:04:31.387 INFO:teuthology.orchestra.run.smithi005.stdout: "batch_size": { 2024-09-12T03:04:31.387 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "batch_size", 2024-09-12T03:04:31.387 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-12T03:04:31.387 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.388 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.388 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5000", 2024-09-12T03:04:31.388 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.388 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.388 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.388 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-12T03:04:31.388 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.388 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.388 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.388 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.388 INFO:teuthology.orchestra.run.smithi005.stdout: "database": { 2024-09-12T03:04:31.388 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "database", 2024-09-12T03:04:31.389 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.389 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.389 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.389 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "ceph", 2024-09-12T03:04:31.389 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.389 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.389 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.389 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-12T03:04:31.389 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.389 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.389 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.389 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.389 INFO:teuthology.orchestra.run.smithi005.stdout: "hostname": { 2024-09-12T03:04:31.390 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "hostname", 2024-09-12T03:04:31.390 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.390 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.390 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.390 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.390 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.390 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.390 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.390 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "InfluxDB server hostname", 2024-09-12T03:04:31.390 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.391 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.391 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.391 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.391 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-09-12T03:04:31.391 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-09-12T03:04:31.391 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-12T03:04:31.391 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.391 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.391 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "30", 2024-09-12T03:04:31.391 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "5", 2024-09-12T03:04:31.391 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.391 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.392 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-12T03:04:31.392 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.392 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.392 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.392 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.392 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-12T03:04:31.392 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-12T03:04:31.392 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.392 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.392 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.392 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.392 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.393 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.393 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.393 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:31.393 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:31.393 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:31.393 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:31.393 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:31.393 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:31.393 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.393 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.393 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.393 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.393 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.394 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.394 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-12T03:04:31.394 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-12T03:04:31.394 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.394 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.394 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.394 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.394 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.394 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.394 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.394 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.394 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.394 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.395 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.395 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.395 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-12T03:04:31.395 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-12T03:04:31.395 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.395 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.395 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.395 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-12T03:04:31.395 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.395 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.395 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.395 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:31.395 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:31.396 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:31.396 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:31.396 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:31.396 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:31.396 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.396 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.396 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.396 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.396 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.396 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.396 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-12T03:04:31.396 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-12T03:04:31.396 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.397 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.397 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.397 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.397 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.397 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.397 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.397 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.397 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.397 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.397 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.397 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.397 INFO:teuthology.orchestra.run.smithi005.stdout: "password": { 2024-09-12T03:04:31.397 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "password", 2024-09-12T03:04:31.397 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.398 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.398 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.398 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.398 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.398 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.398 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.398 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "password of InfluxDB server user", 2024-09-12T03:04:31.398 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.398 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.398 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.398 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.398 INFO:teuthology.orchestra.run.smithi005.stdout: "port": { 2024-09-12T03:04:31.398 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "port", 2024-09-12T03:04:31.399 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-12T03:04:31.399 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.399 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.399 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8086", 2024-09-12T03:04:31.399 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.399 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.399 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.399 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "InfluxDB server port", 2024-09-12T03:04:31.399 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.399 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.399 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.399 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.399 INFO:teuthology.orchestra.run.smithi005.stdout: "ssl": { 2024-09-12T03:04:31.399 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssl", 2024-09-12T03:04:31.400 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.400 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.400 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.400 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "false", 2024-09-12T03:04:31.400 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.400 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.400 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.400 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-12T03:04:31.400 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.400 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.400 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.400 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.400 INFO:teuthology.orchestra.run.smithi005.stdout: "threads": { 2024-09-12T03:04:31.401 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "threads", 2024-09-12T03:04:31.401 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-12T03:04:31.401 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.401 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.401 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2024-09-12T03:04:31.401 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2024-09-12T03:04:31.401 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "32", 2024-09-12T03:04:31.401 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.401 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-12T03:04:31.401 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.402 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.402 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.402 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.402 INFO:teuthology.orchestra.run.smithi005.stdout: "username": { 2024-09-12T03:04:31.402 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "username", 2024-09-12T03:04:31.402 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.402 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.402 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.402 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.402 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.402 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.402 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.402 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "username of InfluxDB server user", 2024-09-12T03:04:31.403 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.403 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.403 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.403 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.403 INFO:teuthology.orchestra.run.smithi005.stdout: "verify_ssl": { 2024-09-12T03:04:31.403 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "verify_ssl", 2024-09-12T03:04:31.403 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.403 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.403 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.403 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "true", 2024-09-12T03:04:31.403 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.403 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.404 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.404 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-12T03:04:31.404 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.404 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.404 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.404 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:31.404 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:31.404 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.404 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-12T03:04:31.404 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "insights", 2024-09-12T03:04:31.404 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-12T03:04:31.404 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-12T03:04:31.404 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-12T03:04:31.405 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-12T03:04:31.405 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-12T03:04:31.405 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.405 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.405 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.405 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.405 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.405 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.405 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.405 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:31.405 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:31.405 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:31.405 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:31.406 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:31.406 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:31.406 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.406 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.406 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.406 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.406 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.406 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.406 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-12T03:04:31.406 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-12T03:04:31.406 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.406 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.406 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.406 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.407 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.407 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.407 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.407 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.407 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.407 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.407 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.407 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.407 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-12T03:04:31.407 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-12T03:04:31.407 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.407 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.407 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.408 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-12T03:04:31.408 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.408 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.408 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.408 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:31.408 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:31.408 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:31.408 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:31.408 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:31.408 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:31.408 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.408 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.408 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.409 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.409 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.409 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.409 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-12T03:04:31.409 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-12T03:04:31.409 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.409 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.409 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.409 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.409 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.409 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.409 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.409 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.409 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.410 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.410 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.410 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:31.410 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:31.410 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.410 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-12T03:04:31.410 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "iostat", 2024-09-12T03:04:31.410 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-12T03:04:31.410 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-12T03:04:31.410 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-12T03:04:31.410 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-12T03:04:31.410 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-12T03:04:31.410 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.411 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.411 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.411 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.411 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.411 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.411 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.411 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:31.411 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:31.411 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:31.411 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:31.411 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:31.411 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:31.411 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.411 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.412 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.412 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.412 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.412 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.412 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-12T03:04:31.412 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-12T03:04:31.412 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.412 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.412 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.412 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.412 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.412 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.412 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.413 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.413 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.413 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.413 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.413 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.413 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-12T03:04:31.413 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-12T03:04:31.413 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.413 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.413 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.413 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-12T03:04:31.413 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.413 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.413 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.414 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:31.414 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:31.414 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:31.414 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:31.414 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:31.414 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:31.414 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.414 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.414 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.414 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.414 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.414 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.414 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-12T03:04:31.415 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-12T03:04:31.415 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.415 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.415 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.415 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.415 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.415 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.415 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.415 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.415 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.415 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.415 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.415 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:31.415 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:31.416 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.416 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-12T03:04:31.416 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "k8sevents", 2024-09-12T03:04:31.416 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-12T03:04:31.416 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-12T03:04:31.416 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-12T03:04:31.416 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_event_retention_days": { 2024-09-12T03:04:31.416 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ceph_event_retention_days", 2024-09-12T03:04:31.416 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-12T03:04:31.416 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.416 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.416 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "7", 2024-09-12T03:04:31.416 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.417 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.417 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.417 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-12T03:04:31.417 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.417 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.417 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.417 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.417 INFO:teuthology.orchestra.run.smithi005.stdout: "config_check_secs": { 2024-09-12T03:04:31.417 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "config_check_secs", 2024-09-12T03:04:31.417 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-12T03:04:31.417 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.417 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.417 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-09-12T03:04:31.417 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "10", 2024-09-12T03:04:31.418 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.418 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.418 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-12T03:04:31.418 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.418 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.418 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.418 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.418 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-12T03:04:31.418 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-12T03:04:31.418 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.418 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.419 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.419 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.419 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.419 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.419 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.419 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:31.419 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:31.419 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:31.419 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:31.419 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:31.419 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:31.419 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.419 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.419 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.420 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.420 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.420 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.420 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-12T03:04:31.420 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-12T03:04:31.420 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.420 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.420 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.420 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.420 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.420 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.420 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.421 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.421 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.421 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.421 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.421 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.421 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-12T03:04:31.421 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-12T03:04:31.421 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.421 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.421 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.421 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-12T03:04:31.421 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.421 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.422 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.422 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:31.422 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:31.422 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:31.422 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:31.422 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:31.422 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:31.422 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.422 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.422 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.422 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.422 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.422 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.423 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-12T03:04:31.423 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-12T03:04:31.423 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.423 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.423 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.423 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.423 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.423 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.423 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.423 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.423 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.423 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.423 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.424 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:31.424 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:31.424 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.424 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-12T03:04:31.424 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "localpool", 2024-09-12T03:04:31.424 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-12T03:04:31.424 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-12T03:04:31.424 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-12T03:04:31.424 INFO:teuthology.orchestra.run.smithi005.stdout: "failure_domain": { 2024-09-12T03:04:31.424 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "failure_domain", 2024-09-12T03:04:31.424 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.424 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.424 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.425 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "host", 2024-09-12T03:04:31.425 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.425 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.425 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.425 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "failure domain for any created local pool", 2024-09-12T03:04:31.425 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-12T03:04:31.425 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.425 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.425 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.425 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-12T03:04:31.425 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-12T03:04:31.425 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.425 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.425 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.426 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.426 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.426 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.426 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.426 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:31.426 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:31.426 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:31.426 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:31.426 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:31.426 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:31.426 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.426 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.426 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.427 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.427 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.427 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.427 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-12T03:04:31.427 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-12T03:04:31.427 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.427 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.427 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.427 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.427 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.427 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.427 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.427 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.427 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.428 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.428 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.428 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.428 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-12T03:04:31.428 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-12T03:04:31.428 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.428 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.428 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.428 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-12T03:04:31.428 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.428 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.428 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.428 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:31.428 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:31.429 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:31.429 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:31.429 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:31.429 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:31.429 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.429 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.429 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.429 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.429 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.429 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.429 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-12T03:04:31.429 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-12T03:04:31.429 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.430 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.430 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.430 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.430 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.430 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.430 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.430 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.430 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.430 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.430 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.430 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.430 INFO:teuthology.orchestra.run.smithi005.stdout: "min_size": { 2024-09-12T03:04:31.430 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "min_size", 2024-09-12T03:04:31.430 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-12T03:04:31.431 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.431 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.431 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.431 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.431 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.431 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.431 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "default min_size for any created local pool", 2024-09-12T03:04:31.431 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-12T03:04:31.431 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.431 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.431 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.431 INFO:teuthology.orchestra.run.smithi005.stdout: "num_rep": { 2024-09-12T03:04:31.431 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "num_rep", 2024-09-12T03:04:31.431 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-12T03:04:31.432 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.432 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.432 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "3", 2024-09-12T03:04:31.432 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.432 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.432 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.432 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "default replica count for any created local pool", 2024-09-12T03:04:31.432 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.432 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.432 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.432 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.433 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num": { 2024-09-12T03:04:31.433 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pg_num", 2024-09-12T03:04:31.433 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-12T03:04:31.433 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.433 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.433 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "128", 2024-09-12T03:04:31.433 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.433 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.433 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.433 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "default pg_num for any created local pool", 2024-09-12T03:04:31.433 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.434 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.434 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.434 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.434 INFO:teuthology.orchestra.run.smithi005.stdout: "prefix": { 2024-09-12T03:04:31.434 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "prefix", 2024-09-12T03:04:31.434 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.434 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.434 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.434 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.434 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.434 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.434 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.435 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "name prefix for any created local pool", 2024-09-12T03:04:31.435 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.435 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.435 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.435 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.435 INFO:teuthology.orchestra.run.smithi005.stdout: "subtree": { 2024-09-12T03:04:31.435 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "subtree", 2024-09-12T03:04:31.435 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.435 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.435 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.435 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "rack", 2024-09-12T03:04:31.435 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.435 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.435 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.436 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-12T03:04:31.436 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-12T03:04:31.436 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.436 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.436 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:31.436 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:31.436 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.436 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-12T03:04:31.436 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mds_autoscaler", 2024-09-12T03:04:31.436 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-12T03:04:31.436 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-12T03:04:31.436 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-12T03:04:31.436 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-12T03:04:31.437 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-12T03:04:31.437 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.437 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.437 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.437 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.437 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.437 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.437 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.437 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:31.437 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:31.437 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:31.437 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:31.437 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:31.438 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:31.438 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.438 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.438 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.438 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.438 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.438 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.438 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-12T03:04:31.438 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-12T03:04:31.438 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.438 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.438 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.438 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.439 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.439 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.439 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.439 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.439 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.439 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.439 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.439 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.439 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-12T03:04:31.439 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-12T03:04:31.439 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.439 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.439 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.440 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-12T03:04:31.440 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.440 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.440 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.440 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:31.440 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:31.440 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:31.440 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:31.440 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:31.440 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:31.440 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.440 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.440 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.441 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.441 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.441 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.441 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-12T03:04:31.441 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-12T03:04:31.441 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.441 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.441 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.441 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.441 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.441 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.441 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.441 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.441 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.442 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.442 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.442 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:31.442 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:31.442 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.442 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-12T03:04:31.442 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mirroring", 2024-09-12T03:04:31.442 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-12T03:04:31.442 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-12T03:04:31.442 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-12T03:04:31.442 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-12T03:04:31.442 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-12T03:04:31.442 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.442 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.443 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.443 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.443 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.443 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.443 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.443 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:31.443 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:31.443 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:31.443 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:31.443 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:31.443 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:31.443 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.443 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.444 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.444 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.444 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.444 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.444 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-12T03:04:31.444 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-12T03:04:31.444 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.444 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.444 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.444 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.444 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.444 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.444 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.444 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.445 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.445 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.445 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.445 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.445 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-12T03:04:31.445 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-12T03:04:31.445 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.445 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.445 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.445 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-12T03:04:31.445 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.445 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.445 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.446 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:31.446 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:31.446 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:31.446 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:31.446 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:31.446 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:31.446 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.446 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.446 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.446 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.446 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.446 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.446 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-12T03:04:31.447 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-12T03:04:31.447 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.447 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.447 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.447 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.447 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.447 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.447 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.447 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.447 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.447 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.448 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.448 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:31.448 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:31.448 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.448 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-12T03:04:31.448 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "nfs", 2024-09-12T03:04:31.448 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-12T03:04:31.448 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-12T03:04:31.448 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-12T03:04:31.448 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-12T03:04:31.448 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-12T03:04:31.448 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.448 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.449 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.449 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.449 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.449 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.449 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.449 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:31.449 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:31.449 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:31.449 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:31.449 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:31.449 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:31.449 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.450 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.450 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.450 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.450 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.450 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.450 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-12T03:04:31.450 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-12T03:04:31.450 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.450 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.450 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.450 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.450 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.450 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.451 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.451 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.451 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.451 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.451 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.451 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.451 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-12T03:04:31.451 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-12T03:04:31.451 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.451 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.451 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.451 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-12T03:04:31.451 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.452 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.452 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.452 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:31.452 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:31.452 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:31.452 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:31.452 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:31.452 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:31.452 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.452 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.452 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.452 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.452 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.453 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.453 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-12T03:04:31.453 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-12T03:04:31.453 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.453 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.453 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.453 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.453 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.453 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.453 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.453 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.453 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.453 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.454 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.454 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:31.454 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:31.454 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.454 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-12T03:04:31.454 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "orchestrator", 2024-09-12T03:04:31.454 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-12T03:04:31.454 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-12T03:04:31.454 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-12T03:04:31.454 INFO:teuthology.orchestra.run.smithi005.stdout: "fail_fs": { 2024-09-12T03:04:31.454 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "fail_fs", 2024-09-12T03:04:31.454 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.455 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.455 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.455 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.455 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.455 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.455 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.455 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-12T03:04:31.455 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.455 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.455 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.455 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.455 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-12T03:04:31.455 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-12T03:04:31.455 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.456 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.456 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.456 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.456 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.456 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.456 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.456 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:31.456 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:31.456 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:31.456 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:31.456 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:31.456 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:31.456 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.457 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.457 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.457 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.457 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.457 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.457 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-12T03:04:31.457 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-12T03:04:31.457 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.457 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.457 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.457 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.457 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.457 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.457 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.458 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.458 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.458 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.458 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.458 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.458 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-12T03:04:31.458 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-12T03:04:31.458 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.458 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.458 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.458 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-12T03:04:31.458 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.458 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.459 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.459 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:31.459 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:31.459 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:31.459 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:31.459 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:31.459 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:31.459 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.459 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.459 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.459 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.459 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.459 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.459 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-12T03:04:31.460 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-12T03:04:31.460 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.460 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.460 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.460 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.460 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.460 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.460 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.460 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.460 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.460 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.460 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.460 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.461 INFO:teuthology.orchestra.run.smithi005.stdout: "orchestrator": { 2024-09-12T03:04:31.461 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "orchestrator", 2024-09-12T03:04:31.461 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.461 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.461 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.461 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.461 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.461 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.461 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.461 INFO:teuthology.orchestra.run.smithi005.stdout: "cephadm", 2024-09-12T03:04:31.461 INFO:teuthology.orchestra.run.smithi005.stdout: "rook", 2024-09-12T03:04:31.461 INFO:teuthology.orchestra.run.smithi005.stdout: "test_orchestrator" 2024-09-12T03:04:31.461 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.461 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Orchestrator backend", 2024-09-12T03:04:31.462 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.462 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.462 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.462 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:31.462 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:31.462 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.462 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-12T03:04:31.462 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "osd_perf_query", 2024-09-12T03:04:31.462 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-12T03:04:31.462 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-12T03:04:31.462 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-12T03:04:31.463 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-12T03:04:31.463 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-12T03:04:31.463 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.463 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.463 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.463 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.463 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.463 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.463 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.463 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:31.463 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:31.463 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:31.463 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:31.464 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:31.464 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:31.464 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.464 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.464 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.464 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.464 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.464 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.464 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-12T03:04:31.464 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-12T03:04:31.464 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.464 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.464 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.465 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.465 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.465 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.465 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.465 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.465 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.465 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.465 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.465 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.465 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-12T03:04:31.465 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-12T03:04:31.465 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.465 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.466 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.466 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-12T03:04:31.466 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.466 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.466 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.466 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:31.466 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:31.466 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:31.466 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:31.466 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:31.466 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:31.466 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.466 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.466 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.467 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.467 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.467 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.467 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-12T03:04:31.467 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-12T03:04:31.467 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.467 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.467 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.467 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.467 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.467 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.467 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.467 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.468 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.468 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.468 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.468 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:31.468 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:31.468 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.468 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-12T03:04:31.468 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "osd_support", 2024-09-12T03:04:31.468 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-12T03:04:31.468 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-12T03:04:31.468 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-12T03:04:31.468 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-12T03:04:31.468 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-12T03:04:31.469 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.469 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.469 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.469 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.469 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.469 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.469 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.469 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:31.469 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:31.469 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:31.469 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:31.469 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:31.469 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:31.470 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.470 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.470 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.470 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.470 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.470 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.470 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-12T03:04:31.470 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-12T03:04:31.470 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.470 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.470 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.470 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.470 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.470 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.471 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.471 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.471 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.471 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.471 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.471 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.471 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-12T03:04:31.471 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-12T03:04:31.471 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.471 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.471 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.471 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-12T03:04:31.471 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.472 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.472 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.472 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:31.472 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:31.472 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:31.472 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:31.472 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:31.472 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:31.472 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.472 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.472 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.472 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.472 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.472 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.473 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-12T03:04:31.473 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-12T03:04:31.473 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.473 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.473 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.473 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.473 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.473 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.473 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.473 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.473 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.473 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.473 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.474 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:31.474 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:31.474 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.474 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-12T03:04:31.474 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pg_autoscaler", 2024-09-12T03:04:31.474 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-12T03:04:31.474 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-12T03:04:31.474 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-12T03:04:31.474 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-12T03:04:31.474 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-12T03:04:31.474 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.474 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.474 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.474 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.475 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.475 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.475 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.475 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:31.475 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:31.475 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:31.475 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:31.475 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:31.475 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:31.475 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.475 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.475 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.475 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.475 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.476 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.476 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-12T03:04:31.476 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-12T03:04:31.476 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.476 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.476 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.476 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.476 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.476 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.476 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.476 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.476 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.476 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.477 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.477 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.477 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-12T03:04:31.477 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-12T03:04:31.477 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.477 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.477 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.477 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-12T03:04:31.477 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.477 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.477 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.478 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:31.478 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:31.478 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:31.478 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:31.478 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:31.478 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:31.478 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.478 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.478 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.478 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.478 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.478 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.478 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-12T03:04:31.478 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-12T03:04:31.479 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.479 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.479 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.479 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.479 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.479 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.479 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.479 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.479 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.479 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.479 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.479 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.479 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-09-12T03:04:31.480 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-09-12T03:04:31.480 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-12T03:04:31.480 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.480 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.480 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-09-12T03:04:31.480 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.480 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.480 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.480 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.480 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.480 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.480 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.480 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.481 INFO:teuthology.orchestra.run.smithi005.stdout: "threshold": { 2024-09-12T03:04:31.481 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "threshold", 2024-09-12T03:04:31.481 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-09-12T03:04:31.481 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.481 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.481 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "3.0", 2024-09-12T03:04:31.481 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1.0", 2024-09-12T03:04:31.481 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.481 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.481 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "scaling threshold", 2024-09-12T03:04:31.481 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-12T03:04:31.481 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.482 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.482 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:31.482 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:31.482 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.482 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-12T03:04:31.482 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "progress", 2024-09-12T03:04:31.482 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-12T03:04:31.482 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-12T03:04:31.482 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-12T03:04:31.482 INFO:teuthology.orchestra.run.smithi005.stdout: "allow_pg_recovery_event": { 2024-09-12T03:04:31.482 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "allow_pg_recovery_event", 2024-09-12T03:04:31.482 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.483 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.483 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.483 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.483 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.483 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.483 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.483 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-12T03:04:31.483 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.483 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.483 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.483 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.483 INFO:teuthology.orchestra.run.smithi005.stdout: "enabled": { 2024-09-12T03:04:31.484 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enabled", 2024-09-12T03:04:31.484 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.484 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.484 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.484 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-12T03:04:31.484 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.484 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.484 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.484 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.484 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.484 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.484 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.485 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.485 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-12T03:04:31.485 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-12T03:04:31.485 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.485 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.485 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.485 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.485 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.485 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.485 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.485 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:31.485 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:31.485 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:31.485 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:31.486 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:31.486 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:31.486 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.486 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.486 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.486 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.486 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.486 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.486 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-12T03:04:31.486 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-12T03:04:31.486 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.486 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.486 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.487 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.487 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.487 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.487 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.487 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.487 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.487 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.487 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.487 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.487 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-12T03:04:31.487 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-12T03:04:31.487 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.487 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.488 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.488 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-12T03:04:31.488 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.488 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.488 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.488 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:31.488 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:31.488 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:31.488 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:31.488 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:31.488 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:31.488 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.489 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.489 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.489 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.489 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.489 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.489 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-12T03:04:31.489 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-12T03:04:31.489 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.489 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.489 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.489 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.489 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.489 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.490 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.490 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.490 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.490 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.490 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.490 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.490 INFO:teuthology.orchestra.run.smithi005.stdout: "max_completed_events": { 2024-09-12T03:04:31.490 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_completed_events", 2024-09-12T03:04:31.490 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-12T03:04:31.490 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.490 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.490 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "50", 2024-09-12T03:04:31.490 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.491 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.491 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.491 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "number of past completed events to remember", 2024-09-12T03:04:31.491 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.491 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.491 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.491 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.491 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-09-12T03:04:31.491 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-09-12T03:04:31.491 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-12T03:04:31.491 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.492 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.492 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2024-09-12T03:04:31.492 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.492 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.492 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.492 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how long the module is going to sleep", 2024-09-12T03:04:31.492 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.492 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.492 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.492 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:31.492 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:31.493 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.493 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-12T03:04:31.493 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "prometheus", 2024-09-12T03:04:31.493 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-12T03:04:31.493 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-12T03:04:31.493 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-12T03:04:31.493 INFO:teuthology.orchestra.run.smithi005.stdout: "cache": { 2024-09-12T03:04:31.493 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cache", 2024-09-12T03:04:31.493 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.493 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.493 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.493 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-12T03:04:31.493 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.494 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.494 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.494 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.494 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.494 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.494 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.494 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.494 INFO:teuthology.orchestra.run.smithi005.stdout: "exclude_perf_counters": { 2024-09-12T03:04:31.494 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "exclude_perf_counters", 2024-09-12T03:04:31.494 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.494 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.494 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.494 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-12T03:04:31.495 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.495 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.495 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.495 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-12T03:04:31.495 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-12T03:04:31.495 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.495 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.495 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.495 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-12T03:04:31.495 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-12T03:04:31.495 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.495 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.495 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.496 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.496 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.496 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.496 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.496 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:31.496 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:31.496 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:31.496 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:31.496 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:31.496 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:31.496 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.496 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.496 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.497 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.497 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.497 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.497 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-12T03:04:31.497 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-12T03:04:31.497 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.497 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.497 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.497 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.497 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.497 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.497 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.497 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.498 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.498 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.498 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.498 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.498 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-12T03:04:31.498 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-12T03:04:31.498 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.498 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.498 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.498 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-12T03:04:31.498 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.498 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.498 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.499 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:31.499 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:31.499 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:31.499 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:31.499 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:31.499 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:31.499 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.499 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.499 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.499 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.499 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.499 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.499 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-12T03:04:31.500 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-12T03:04:31.500 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.500 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.500 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.500 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.500 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.500 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.500 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.500 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.500 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.500 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.500 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.500 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.500 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_stats_pools": { 2024-09-12T03:04:31.501 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rbd_stats_pools", 2024-09-12T03:04:31.501 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.501 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.501 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.501 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.501 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.501 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.501 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.501 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.501 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.501 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.501 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.501 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.502 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-12T03:04:31.502 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-12T03:04:31.502 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-12T03:04:31.502 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.502 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.502 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "300", 2024-09-12T03:04:31.502 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.502 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.502 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.502 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.502 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.502 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.503 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.503 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.503 INFO:teuthology.orchestra.run.smithi005.stdout: "scrape_interval": { 2024-09-12T03:04:31.503 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "scrape_interval", 2024-09-12T03:04:31.503 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-09-12T03:04:31.503 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.503 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.503 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "15.0", 2024-09-12T03:04:31.503 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.503 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.503 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.503 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.503 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.503 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.504 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.504 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.504 INFO:teuthology.orchestra.run.smithi005.stdout: "server_addr": { 2024-09-12T03:04:31.504 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_addr", 2024-09-12T03:04:31.504 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.504 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.504 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.504 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "::", 2024-09-12T03:04:31.504 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.504 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.504 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.504 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-12T03:04:31.504 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.505 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.505 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.505 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.505 INFO:teuthology.orchestra.run.smithi005.stdout: "server_port": { 2024-09-12T03:04:31.505 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_port", 2024-09-12T03:04:31.505 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-12T03:04:31.505 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.505 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.505 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "9283", 2024-09-12T03:04:31.505 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.505 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.505 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.505 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-12T03:04:31.505 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.506 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.506 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.506 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.506 INFO:teuthology.orchestra.run.smithi005.stdout: "stale_cache_strategy": { 2024-09-12T03:04:31.506 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "stale_cache_strategy", 2024-09-12T03:04:31.506 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.506 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.506 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.506 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "log", 2024-09-12T03:04:31.506 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.506 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.506 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.507 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.507 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.507 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.507 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.507 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.507 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_behaviour": { 2024-09-12T03:04:31.507 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_behaviour", 2024-09-12T03:04:31.507 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.507 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.507 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.507 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "default", 2024-09-12T03:04:31.507 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.507 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.508 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.508 INFO:teuthology.orchestra.run.smithi005.stdout: "default", 2024-09-12T03:04:31.508 INFO:teuthology.orchestra.run.smithi005.stdout: "error" 2024-09-12T03:04:31.508 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.508 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.508 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.508 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.508 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.508 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.508 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_error_status_code": { 2024-09-12T03:04:31.508 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_error_status_code", 2024-09-12T03:04:31.508 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-12T03:04:31.508 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.509 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.509 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "500", 2024-09-12T03:04:31.509 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "400", 2024-09-12T03:04:31.509 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "599", 2024-09-12T03:04:31.509 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.509 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.509 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.509 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.509 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.509 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:31.509 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:31.509 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.509 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-12T03:04:31.510 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rbd_support", 2024-09-12T03:04:31.510 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-12T03:04:31.510 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-12T03:04:31.510 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-12T03:04:31.510 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-12T03:04:31.510 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-12T03:04:31.510 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.510 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.510 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.510 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.510 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.510 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.510 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.511 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:31.511 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:31.511 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:31.511 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:31.511 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:31.511 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:31.511 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.511 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.511 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.511 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.511 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.511 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.511 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-12T03:04:31.511 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-12T03:04:31.512 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.512 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.512 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.512 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.512 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.512 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.512 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.512 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.512 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.512 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.512 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.512 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.513 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-12T03:04:31.513 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-12T03:04:31.513 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.513 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.513 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.513 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-12T03:04:31.513 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.513 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.513 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.513 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:31.513 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:31.513 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:31.513 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:31.514 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:31.514 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:31.514 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.514 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.514 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.514 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.514 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.514 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.514 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-12T03:04:31.514 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-12T03:04:31.514 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.514 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.514 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.515 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.515 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.515 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.515 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.515 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.515 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.515 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.515 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.515 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.515 INFO:teuthology.orchestra.run.smithi005.stdout: "max_concurrent_snap_create": { 2024-09-12T03:04:31.515 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_concurrent_snap_create", 2024-09-12T03:04:31.515 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-12T03:04:31.515 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.515 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.516 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-09-12T03:04:31.516 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.516 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.516 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.516 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.516 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.516 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.516 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.516 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.516 INFO:teuthology.orchestra.run.smithi005.stdout: "mirror_snapshot_schedule": { 2024-09-12T03:04:31.516 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mirror_snapshot_schedule", 2024-09-12T03:04:31.516 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.516 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.517 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.517 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.517 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.517 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.517 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.517 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.517 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.517 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.517 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.517 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.517 INFO:teuthology.orchestra.run.smithi005.stdout: "trash_purge_schedule": { 2024-09-12T03:04:31.517 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "trash_purge_schedule", 2024-09-12T03:04:31.517 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.518 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.518 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.518 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.518 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.518 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.518 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.518 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.518 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.518 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.518 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.518 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:31.518 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:31.518 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.519 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-12T03:04:31.519 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "restful", 2024-09-12T03:04:31.519 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-12T03:04:31.519 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-12T03:04:31.519 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-12T03:04:31.519 INFO:teuthology.orchestra.run.smithi005.stdout: "enable_auth": { 2024-09-12T03:04:31.519 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enable_auth", 2024-09-12T03:04:31.519 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.519 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.519 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.519 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-12T03:04:31.519 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.519 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.520 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.520 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.520 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.520 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.520 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.520 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.520 INFO:teuthology.orchestra.run.smithi005.stdout: "key_file": { 2024-09-12T03:04:31.520 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "key_file", 2024-09-12T03:04:31.520 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.520 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.520 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.520 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.520 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.521 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.521 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.521 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.521 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.521 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.521 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.521 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.521 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-12T03:04:31.521 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-12T03:04:31.522 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.522 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.522 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.522 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.522 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.522 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.522 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.522 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:31.522 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:31.522 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:31.522 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:31.522 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:31.522 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:31.523 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.523 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.523 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.523 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.523 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.523 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.523 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-12T03:04:31.523 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-12T03:04:31.523 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.523 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.523 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.523 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.523 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.524 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.524 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.524 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.524 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.524 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.524 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.524 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.524 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-12T03:04:31.524 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-12T03:04:31.524 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.524 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.524 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.524 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-12T03:04:31.525 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.525 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.525 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.525 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:31.525 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:31.525 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:31.525 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:31.525 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:31.525 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:31.525 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.525 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.525 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.525 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.526 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.526 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.526 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-12T03:04:31.526 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-12T03:04:31.526 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.526 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.526 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.526 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.526 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.526 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.526 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.526 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.526 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.527 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.527 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.527 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.527 INFO:teuthology.orchestra.run.smithi005.stdout: "server_addr": { 2024-09-12T03:04:31.527 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_addr", 2024-09-12T03:04:31.527 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.527 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.527 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.527 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.527 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.527 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.527 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.527 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.528 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.528 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.528 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.528 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.528 INFO:teuthology.orchestra.run.smithi005.stdout: "server_port": { 2024-09-12T03:04:31.528 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_port", 2024-09-12T03:04:31.528 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.528 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.528 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.528 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.528 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.528 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.528 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.529 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.529 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.529 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.529 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.529 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:31.529 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:31.529 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.529 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-12T03:04:31.529 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rgw", 2024-09-12T03:04:31.529 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-12T03:04:31.529 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-12T03:04:31.529 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-12T03:04:31.529 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-12T03:04:31.529 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-12T03:04:31.530 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.530 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.530 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.530 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.530 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.530 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.530 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.530 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:31.530 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:31.530 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:31.530 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:31.530 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:31.530 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:31.531 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.531 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.531 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.531 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.531 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.531 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.531 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-12T03:04:31.531 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-12T03:04:31.531 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.531 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.531 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.531 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.531 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.531 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.532 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.532 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.532 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.532 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.532 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.532 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.532 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-12T03:04:31.532 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-12T03:04:31.532 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.532 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.532 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.532 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-12T03:04:31.532 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.533 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.533 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.533 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:31.533 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:31.533 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:31.533 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:31.533 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:31.533 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:31.533 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.533 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.533 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.533 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.533 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.534 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.534 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-12T03:04:31.534 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-12T03:04:31.534 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.534 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.534 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.534 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.534 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.534 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.534 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.534 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.534 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.534 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.535 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.535 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:31.535 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:31.535 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.535 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-12T03:04:31.535 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rook", 2024-09-12T03:04:31.535 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-12T03:04:31.535 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-12T03:04:31.535 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-12T03:04:31.535 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-12T03:04:31.535 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-12T03:04:31.535 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.535 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.536 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.536 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.536 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.536 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.536 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.536 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:31.536 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:31.536 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:31.536 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:31.536 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:31.536 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:31.536 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.537 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.537 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.537 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.537 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.537 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.537 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-12T03:04:31.537 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-12T03:04:31.537 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.537 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.537 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.537 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.537 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.537 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.538 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.538 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.538 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.538 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.538 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.538 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.538 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-12T03:04:31.538 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-12T03:04:31.538 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.538 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.538 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.538 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-12T03:04:31.538 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.539 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.539 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.539 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:31.539 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:31.539 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:31.539 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:31.539 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:31.539 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:31.539 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.539 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.539 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.539 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.539 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.540 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.540 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-12T03:04:31.540 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-12T03:04:31.540 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.540 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.540 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.540 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.540 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.540 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.540 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.540 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.540 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.540 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.541 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.541 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.541 INFO:teuthology.orchestra.run.smithi005.stdout: "storage_class": { 2024-09-12T03:04:31.541 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "storage_class", 2024-09-12T03:04:31.541 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.541 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.541 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.541 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "local", 2024-09-12T03:04:31.541 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.541 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.541 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.541 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-12T03:04:31.541 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.542 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.542 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.542 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:31.542 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:31.542 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.542 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-12T03:04:31.542 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "selftest", 2024-09-12T03:04:31.542 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-12T03:04:31.542 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-12T03:04:31.542 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-12T03:04:31.542 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-12T03:04:31.542 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-12T03:04:31.542 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.543 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.543 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.543 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.543 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.543 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.543 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.543 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:31.543 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:31.543 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:31.543 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:31.543 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:31.543 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:31.543 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.544 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.544 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.544 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.544 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.544 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.544 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-12T03:04:31.544 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-12T03:04:31.544 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.544 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.544 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.544 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.544 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.544 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.545 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.545 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.545 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.545 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.545 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.545 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.545 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-12T03:04:31.545 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-12T03:04:31.545 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.545 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.545 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.545 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-12T03:04:31.545 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.545 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.546 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.546 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:31.546 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:31.546 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:31.546 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:31.546 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:31.546 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:31.546 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.546 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.546 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.546 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.546 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.546 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.547 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-12T03:04:31.547 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-12T03:04:31.547 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.547 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.547 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.547 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.547 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.547 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.547 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.547 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.547 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.547 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.547 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.548 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.548 INFO:teuthology.orchestra.run.smithi005.stdout: "roption1": { 2024-09-12T03:04:31.548 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "roption1", 2024-09-12T03:04:31.548 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.548 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.548 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.548 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.548 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.548 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.548 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.548 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.548 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.548 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.548 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.549 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.549 INFO:teuthology.orchestra.run.smithi005.stdout: "roption2": { 2024-09-12T03:04:31.549 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "roption2", 2024-09-12T03:04:31.549 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.549 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.549 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.549 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "xyz", 2024-09-12T03:04:31.549 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.549 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.549 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.549 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.549 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.549 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.550 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.550 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.550 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption1": { 2024-09-12T03:04:31.550 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption1", 2024-09-12T03:04:31.550 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.550 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.550 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.550 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.550 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.550 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.550 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.550 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.550 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.550 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.551 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.551 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.551 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption2": { 2024-09-12T03:04:31.551 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption2", 2024-09-12T03:04:31.551 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-12T03:04:31.551 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.551 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.551 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.551 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.551 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.551 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.551 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.552 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.552 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.552 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.552 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.552 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption3": { 2024-09-12T03:04:31.552 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption3", 2024-09-12T03:04:31.552 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-09-12T03:04:31.552 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.552 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.552 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.552 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.552 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.552 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.553 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.553 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.553 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.553 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.553 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.553 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption4": { 2024-09-12T03:04:31.553 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption4", 2024-09-12T03:04:31.553 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.553 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.553 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.553 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.553 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.553 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.554 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.554 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.554 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.554 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.554 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.554 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.554 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption5": { 2024-09-12T03:04:31.554 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption5", 2024-09-12T03:04:31.554 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.554 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.554 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.554 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.554 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.554 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.555 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.555 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.555 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.555 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.555 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.555 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.555 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption6": { 2024-09-12T03:04:31.555 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption6", 2024-09-12T03:04:31.555 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.555 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.555 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.555 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-12T03:04:31.555 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.556 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.556 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.556 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.556 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.556 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.556 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.556 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.556 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption7": { 2024-09-12T03:04:31.556 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption7", 2024-09-12T03:04:31.556 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-12T03:04:31.556 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.556 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.556 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.556 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2024-09-12T03:04:31.557 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "42", 2024-09-12T03:04:31.557 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.557 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.557 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.557 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.557 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.557 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.557 INFO:teuthology.orchestra.run.smithi005.stdout: "testkey": { 2024-09-12T03:04:31.557 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "testkey", 2024-09-12T03:04:31.557 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.557 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.557 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.557 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.558 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.558 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.558 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.558 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.558 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.558 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.558 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.558 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.558 INFO:teuthology.orchestra.run.smithi005.stdout: "testlkey": { 2024-09-12T03:04:31.558 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "testlkey", 2024-09-12T03:04:31.558 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.558 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.558 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.558 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.559 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.559 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.559 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.559 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.559 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.559 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.559 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.559 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.559 INFO:teuthology.orchestra.run.smithi005.stdout: "testnewline": { 2024-09-12T03:04:31.559 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "testnewline", 2024-09-12T03:04:31.559 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.559 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.559 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.560 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.560 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.560 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.560 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.560 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.560 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.560 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.560 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.560 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:31.560 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:31.560 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.560 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-12T03:04:31.560 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "snap_schedule", 2024-09-12T03:04:31.561 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-12T03:04:31.561 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-12T03:04:31.561 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-12T03:04:31.561 INFO:teuthology.orchestra.run.smithi005.stdout: "allow_m_granularity": { 2024-09-12T03:04:31.561 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "allow_m_granularity", 2024-09-12T03:04:31.561 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.561 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.561 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.561 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.561 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.561 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.561 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.561 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "allow minute scheduled snapshots", 2024-09-12T03:04:31.561 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.562 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.562 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.562 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.562 INFO:teuthology.orchestra.run.smithi005.stdout: "dump_on_update": { 2024-09-12T03:04:31.562 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "dump_on_update", 2024-09-12T03:04:31.562 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.562 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.562 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.562 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.562 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.562 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.562 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.562 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "dump database to debug log on update", 2024-09-12T03:04:31.563 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.563 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.563 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.563 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.563 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-12T03:04:31.563 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-12T03:04:31.563 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.563 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.563 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.563 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.563 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.563 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.563 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.564 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:31.564 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:31.564 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:31.564 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:31.564 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:31.564 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:31.564 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.564 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.564 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.564 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.564 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.564 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.564 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-12T03:04:31.564 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-12T03:04:31.565 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.565 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.565 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.565 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.565 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.565 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.565 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.565 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.565 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.565 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.565 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.565 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.565 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-12T03:04:31.566 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-12T03:04:31.566 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.566 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.566 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.566 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-12T03:04:31.566 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.566 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.566 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.566 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:31.566 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:31.566 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:31.566 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:31.567 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:31.567 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:31.567 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.567 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.567 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.567 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.567 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.567 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.567 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-12T03:04:31.567 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-12T03:04:31.567 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.567 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.568 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.568 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.568 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.568 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.568 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.568 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.568 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.568 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.568 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.568 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:31.568 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:31.568 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.568 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-12T03:04:31.569 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "stats", 2024-09-12T03:04:31.569 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-12T03:04:31.569 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-12T03:04:31.569 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-12T03:04:31.569 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-12T03:04:31.569 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-12T03:04:31.569 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.569 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.569 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.569 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.569 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.569 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.569 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.569 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:31.570 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:31.570 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:31.570 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:31.570 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:31.570 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:31.570 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.570 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.570 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.570 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.570 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.570 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.570 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-12T03:04:31.570 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-12T03:04:31.571 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.571 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.571 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.571 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.571 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.571 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.571 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.571 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.571 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.571 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.571 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.571 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.571 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-12T03:04:31.572 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-12T03:04:31.572 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.572 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.572 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.572 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-12T03:04:31.572 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.572 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.572 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.572 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:31.572 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:31.572 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:31.572 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:31.572 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:31.573 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:31.573 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.573 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.573 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.573 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.573 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.573 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.573 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-12T03:04:31.573 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-12T03:04:31.573 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.573 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.573 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.573 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.573 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.574 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.574 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.574 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.574 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.574 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.574 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.574 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:31.574 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:31.574 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.574 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-12T03:04:31.574 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "status", 2024-09-12T03:04:31.574 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-12T03:04:31.574 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-12T03:04:31.575 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-12T03:04:31.575 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-12T03:04:31.575 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-12T03:04:31.575 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.575 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.575 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.575 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.575 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.575 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.575 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.575 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:31.575 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:31.575 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:31.575 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:31.576 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:31.576 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:31.576 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.576 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.576 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.576 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.576 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.576 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.576 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-12T03:04:31.576 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-12T03:04:31.576 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.576 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.576 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.577 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.577 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.577 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.577 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.577 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.577 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.577 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.577 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.577 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.577 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-12T03:04:31.577 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-12T03:04:31.577 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.577 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.577 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.578 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-12T03:04:31.578 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.578 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.578 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.578 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:31.578 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:31.578 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:31.578 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:31.578 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:31.578 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:31.578 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.578 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.578 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.579 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.579 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.579 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.579 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-12T03:04:31.579 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-12T03:04:31.579 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.579 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.579 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.579 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.579 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.579 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.579 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.579 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.580 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.580 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.580 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.580 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:31.580 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:31.580 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.580 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-12T03:04:31.580 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "telegraf", 2024-09-12T03:04:31.580 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-12T03:04:31.580 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-12T03:04:31.580 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-12T03:04:31.580 INFO:teuthology.orchestra.run.smithi005.stdout: "address": { 2024-09-12T03:04:31.580 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "address", 2024-09-12T03:04:31.581 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.581 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.581 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.581 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-12T03:04:31.581 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.581 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.581 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.581 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.581 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.581 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.581 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.582 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.582 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-09-12T03:04:31.582 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-09-12T03:04:31.582 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-12T03:04:31.582 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.582 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.582 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "15", 2024-09-12T03:04:31.582 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.582 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.582 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.582 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.582 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.582 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.583 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.583 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.583 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-12T03:04:31.583 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-12T03:04:31.583 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.583 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.583 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.583 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.583 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.583 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.583 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.583 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:31.583 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:31.584 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:31.584 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:31.584 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:31.584 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:31.584 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.584 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.584 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.584 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.584 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.584 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.584 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-12T03:04:31.584 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-12T03:04:31.584 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.585 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.585 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.585 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.585 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.585 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.585 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.585 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.585 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.585 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.585 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.585 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.585 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-12T03:04:31.585 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-12T03:04:31.586 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.586 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.586 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.586 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-12T03:04:31.586 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.586 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.586 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.586 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:31.586 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:31.586 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:31.586 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:31.586 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:31.586 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:31.587 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.587 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.587 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.587 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.587 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.587 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.587 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-12T03:04:31.587 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-12T03:04:31.587 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.587 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.587 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.587 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.587 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.588 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.588 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.588 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.588 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.588 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.588 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.588 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:31.588 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:31.588 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.588 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-12T03:04:31.588 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "telemetry", 2024-09-12T03:04:31.588 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-12T03:04:31.588 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-12T03:04:31.589 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-12T03:04:31.589 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_basic": { 2024-09-12T03:04:31.589 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_basic", 2024-09-12T03:04:31.589 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.589 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.589 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.589 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-12T03:04:31.589 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.589 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.589 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.589 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-12T03:04:31.589 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.589 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.590 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.590 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.590 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_crash": { 2024-09-12T03:04:31.590 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_crash", 2024-09-12T03:04:31.590 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.590 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.590 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.590 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-12T03:04:31.590 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.590 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.590 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.590 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-12T03:04:31.590 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.590 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.591 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.591 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.591 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_device": { 2024-09-12T03:04:31.591 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_device", 2024-09-12T03:04:31.591 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.591 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.591 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.591 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-12T03:04:31.591 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.591 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.591 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.591 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-12T03:04:31.591 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.592 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.592 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.592 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.592 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_ident": { 2024-09-12T03:04:31.592 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_ident", 2024-09-12T03:04:31.592 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.592 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.592 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.592 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.592 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.592 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.592 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.592 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-12T03:04:31.593 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.593 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.593 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.593 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.593 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_perf": { 2024-09-12T03:04:31.593 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_perf", 2024-09-12T03:04:31.593 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.593 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.593 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.593 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.593 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.593 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.593 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.593 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-12T03:04:31.594 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.594 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.594 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.594 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.594 INFO:teuthology.orchestra.run.smithi005.stdout: "contact": { 2024-09-12T03:04:31.594 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "contact", 2024-09-12T03:04:31.594 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.594 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.594 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.594 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.594 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.594 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.594 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.594 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.595 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.595 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.595 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.595 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.595 INFO:teuthology.orchestra.run.smithi005.stdout: "description": { 2024-09-12T03:04:31.595 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "description", 2024-09-12T03:04:31.595 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.595 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.595 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.595 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.595 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.595 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.596 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.596 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.596 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.596 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.596 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.596 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.596 INFO:teuthology.orchestra.run.smithi005.stdout: "device_url": { 2024-09-12T03:04:31.596 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "device_url", 2024-09-12T03:04:31.596 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.596 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.596 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.596 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-12T03:04:31.597 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.597 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.597 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.597 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.597 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.597 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.597 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.597 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.597 INFO:teuthology.orchestra.run.smithi005.stdout: "enabled": { 2024-09-12T03:04:31.597 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enabled", 2024-09-12T03:04:31.597 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.597 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.597 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.598 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.598 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.598 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.598 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.598 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.598 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.598 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.598 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.598 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.598 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-09-12T03:04:31.598 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-09-12T03:04:31.598 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-12T03:04:31.598 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.599 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.599 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "24", 2024-09-12T03:04:31.599 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "8", 2024-09-12T03:04:31.599 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.599 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.599 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.599 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.599 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.599 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.599 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.599 INFO:teuthology.orchestra.run.smithi005.stdout: "last_opt_revision": { 2024-09-12T03:04:31.599 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "last_opt_revision", 2024-09-12T03:04:31.599 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-12T03:04:31.599 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.600 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.600 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "1", 2024-09-12T03:04:31.600 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.600 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.600 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.600 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.600 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.600 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.600 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.600 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.600 INFO:teuthology.orchestra.run.smithi005.stdout: "leaderboard": { 2024-09-12T03:04:31.600 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "leaderboard", 2024-09-12T03:04:31.600 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.601 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.601 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.601 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.601 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.601 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.601 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.601 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.601 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.601 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.601 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.601 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.601 INFO:teuthology.orchestra.run.smithi005.stdout: "leaderboard_description": { 2024-09-12T03:04:31.602 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "leaderboard_description", 2024-09-12T03:04:31.602 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.602 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.602 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.602 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.602 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.602 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.602 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.602 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.602 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.602 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.602 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.602 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.602 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-12T03:04:31.603 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-12T03:04:31.603 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.603 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.603 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.603 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.603 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.603 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.603 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.603 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:31.603 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:31.603 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:31.603 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:31.603 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:31.604 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:31.604 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.604 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.604 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.604 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.604 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.604 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.604 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-12T03:04:31.604 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-12T03:04:31.604 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.604 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.604 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.604 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.605 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.605 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.605 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.605 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.605 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.605 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.605 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.605 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.605 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-12T03:04:31.605 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-12T03:04:31.605 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.605 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.605 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.606 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-12T03:04:31.606 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.606 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.606 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.606 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:31.606 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:31.606 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:31.606 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:31.606 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:31.606 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:31.606 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.606 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.606 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.607 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.607 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.607 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.607 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-12T03:04:31.607 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-12T03:04:31.607 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.607 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.607 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.607 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.607 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.607 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.607 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.607 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.608 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.608 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.608 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.608 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.608 INFO:teuthology.orchestra.run.smithi005.stdout: "organization": { 2024-09-12T03:04:31.608 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "organization", 2024-09-12T03:04:31.608 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.608 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.608 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.608 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.608 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.608 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.609 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.609 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.609 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.609 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.609 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.609 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.609 INFO:teuthology.orchestra.run.smithi005.stdout: "proxy": { 2024-09-12T03:04:31.609 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "proxy", 2024-09-12T03:04:31.609 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.609 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.609 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.609 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.609 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.609 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.610 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.610 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.610 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.610 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.610 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.610 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.610 INFO:teuthology.orchestra.run.smithi005.stdout: "url": { 2024-09-12T03:04:31.610 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "url", 2024-09-12T03:04:31.610 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.610 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.610 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.610 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-12T03:04:31.611 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.611 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.611 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.611 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.611 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.611 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.611 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.611 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:31.611 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:31.611 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.611 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-12T03:04:31.611 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "test_orchestrator", 2024-09-12T03:04:31.611 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-12T03:04:31.612 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-12T03:04:31.612 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-12T03:04:31.612 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-12T03:04:31.612 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-12T03:04:31.612 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.612 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.612 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.612 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.612 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.612 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.612 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.612 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:31.612 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:31.613 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:31.613 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:31.613 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:31.613 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:31.613 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.613 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.613 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.613 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.613 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.613 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.613 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-12T03:04:31.613 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-12T03:04:31.613 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.614 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.614 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.614 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.614 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.614 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.614 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.614 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.614 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.614 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.614 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.614 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.614 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-12T03:04:31.614 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-12T03:04:31.615 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.615 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.615 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.615 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-12T03:04:31.615 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.615 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.615 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.615 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:31.615 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:31.615 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:31.615 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:31.615 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:31.615 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:31.616 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.616 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.616 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.616 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.616 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.616 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.616 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-12T03:04:31.616 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-12T03:04:31.616 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.616 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.616 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.616 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.616 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.616 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.617 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.617 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.617 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.617 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.617 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.617 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:31.617 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:31.617 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.617 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-12T03:04:31.617 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "volumes", 2024-09-12T03:04:31.617 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-12T03:04:31.617 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-12T03:04:31.617 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-12T03:04:31.618 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-12T03:04:31.618 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-12T03:04:31.618 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.618 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.618 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.618 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.618 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.618 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.618 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.618 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:31.618 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:31.618 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:31.618 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:31.618 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:31.619 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:31.619 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.619 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.619 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.619 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.619 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.619 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.619 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-12T03:04:31.619 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-12T03:04:31.619 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.619 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.619 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.619 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.620 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.620 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.620 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.620 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.620 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.620 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.620 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.620 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.620 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-12T03:04:31.620 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-12T03:04:31.620 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.620 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.620 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.621 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-12T03:04:31.621 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.621 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.621 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.621 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:31.621 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:31.621 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:31.621 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:31.621 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:31.621 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:31.621 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.621 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.621 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.621 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.622 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.622 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.622 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-12T03:04:31.622 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-12T03:04:31.622 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.622 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.622 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.622 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.622 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.622 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.622 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.622 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.622 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.623 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.623 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.623 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.623 INFO:teuthology.orchestra.run.smithi005.stdout: "max_concurrent_clones": { 2024-09-12T03:04:31.623 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_concurrent_clones", 2024-09-12T03:04:31.623 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-12T03:04:31.623 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.623 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.623 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "4", 2024-09-12T03:04:31.623 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.623 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.623 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.623 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-12T03:04:31.623 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.624 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.624 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.624 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.624 INFO:teuthology.orchestra.run.smithi005.stdout: "periodic_async_work": { 2024-09-12T03:04:31.624 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "periodic_async_work", 2024-09-12T03:04:31.624 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.624 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.624 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.624 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.624 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.624 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.624 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.624 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Periodically check for async work", 2024-09-12T03:04:31.625 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.625 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.625 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.625 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.625 INFO:teuthology.orchestra.run.smithi005.stdout: "snapshot_clone_delay": { 2024-09-12T03:04:31.625 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "snapshot_clone_delay", 2024-09-12T03:04:31.625 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-12T03:04:31.625 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.625 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.625 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-09-12T03:04:31.625 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.625 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.626 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.626 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-12T03:04:31.626 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.626 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.626 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.626 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.626 INFO:teuthology.orchestra.run.smithi005.stdout: "snapshot_clone_no_wait": { 2024-09-12T03:04:31.626 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "snapshot_clone_no_wait", 2024-09-12T03:04:31.626 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.626 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.626 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.626 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-12T03:04:31.626 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.627 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.627 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.627 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-12T03:04:31.627 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.627 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.627 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.627 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:31.627 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:31.627 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.627 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-12T03:04:31.627 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix", 2024-09-12T03:04:31.627 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-12T03:04:31.627 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-12T03:04:31.628 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-12T03:04:31.628 INFO:teuthology.orchestra.run.smithi005.stdout: "discovery_interval": { 2024-09-12T03:04:31.628 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "discovery_interval", 2024-09-12T03:04:31.628 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-09-12T03:04:31.628 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.628 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.628 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "100", 2024-09-12T03:04:31.628 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.628 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.628 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.628 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.628 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.628 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.629 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.629 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.629 INFO:teuthology.orchestra.run.smithi005.stdout: "identifier": { 2024-09-12T03:04:31.629 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "identifier", 2024-09-12T03:04:31.629 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.629 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.629 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.629 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.629 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.629 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.629 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.629 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.629 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.630 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.630 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.630 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.630 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-09-12T03:04:31.630 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-09-12T03:04:31.630 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-12T03:04:31.630 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.630 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.630 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-09-12T03:04:31.630 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.630 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.630 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.630 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.631 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.631 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.631 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.631 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.631 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-12T03:04:31.631 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-12T03:04:31.631 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.631 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.631 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.631 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.631 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.631 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.631 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.631 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:31.632 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:31.632 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:31.632 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:31.632 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:31.632 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:31.632 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.632 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.632 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.632 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.632 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.632 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.632 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-12T03:04:31.633 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-12T03:04:31.633 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.633 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.633 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.633 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.633 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.633 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.633 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.633 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.633 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.633 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.633 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.633 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.633 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-12T03:04:31.634 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-12T03:04:31.634 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.634 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.634 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.634 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-12T03:04:31.634 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.634 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.634 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.634 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:31.634 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:31.634 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:31.634 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:31.635 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:31.635 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:31.635 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.635 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.635 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.635 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.635 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.635 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.635 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-12T03:04:31.635 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-12T03:04:31.635 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.635 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.635 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.635 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.636 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.636 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.636 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.636 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.636 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.636 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.636 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.636 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.636 INFO:teuthology.orchestra.run.smithi005.stdout: "zabbix_host": { 2024-09-12T03:04:31.636 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix_host", 2024-09-12T03:04:31.636 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.636 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.636 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.637 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.637 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.637 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.637 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.637 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.637 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.637 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.637 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.637 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.637 INFO:teuthology.orchestra.run.smithi005.stdout: "zabbix_port": { 2024-09-12T03:04:31.637 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix_port", 2024-09-12T03:04:31.637 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-12T03:04:31.637 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.638 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.638 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10051", 2024-09-12T03:04:31.638 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.638 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.638 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.638 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.638 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.638 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.638 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.638 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.638 INFO:teuthology.orchestra.run.smithi005.stdout: "zabbix_sender": { 2024-09-12T03:04:31.638 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix_sender", 2024-09-12T03:04:31.638 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.638 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.639 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.639 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-12T03:04:31.639 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.639 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.639 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.639 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.639 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.639 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.639 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.639 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:31.639 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:31.639 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:31.640 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-12T03:04:31.640 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.640 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-12T03:04:31.640 INFO:teuthology.orchestra.run.smithi005.stdout: "gid": 24119, 2024-09-12T03:04:31.640 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "y", 2024-09-12T03:04:31.640 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr_features": 4540701547738038271, 2024-09-12T03:04:31.640 INFO:teuthology.orchestra.run.smithi005.stdout: "available_modules": [ 2024-09-12T03:04:31.640 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-12T03:04:31.640 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "alerts", 2024-09-12T03:04:31.640 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-12T03:04:31.640 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-12T03:04:31.640 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-12T03:04:31.640 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-09-12T03:04:31.641 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-09-12T03:04:31.641 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-12T03:04:31.641 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.641 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.641 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-09-12T03:04:31.641 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.641 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.641 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.641 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How frequently to reexamine health status", 2024-09-12T03:04:31.641 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.641 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.641 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.641 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.642 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-12T03:04:31.642 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-12T03:04:31.642 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.642 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.642 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.642 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.642 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.642 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.642 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.642 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:31.642 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:31.642 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:31.642 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:31.643 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:31.643 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:31.643 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.643 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.643 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.643 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.643 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.643 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.643 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-12T03:04:31.643 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-12T03:04:31.643 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.643 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.643 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.644 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.644 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.644 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.644 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.644 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.644 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.644 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.644 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.644 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.644 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-12T03:04:31.644 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-12T03:04:31.644 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.644 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.645 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.645 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-12T03:04:31.645 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.645 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.645 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.645 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:31.645 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:31.645 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:31.645 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:31.645 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:31.645 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:31.645 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.645 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.646 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.646 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.646 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.646 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.646 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-12T03:04:31.646 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-12T03:04:31.646 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.646 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.646 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.646 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.646 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.646 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.646 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.647 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.647 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.647 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.647 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.647 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.647 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_destination": { 2024-09-12T03:04:31.647 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_destination", 2024-09-12T03:04:31.647 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.647 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.647 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.647 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.647 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.647 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.648 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.648 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Email address to send alerts to", 2024-09-12T03:04:31.648 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.648 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.648 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.648 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.648 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_from_name": { 2024-09-12T03:04:31.648 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_from_name", 2024-09-12T03:04:31.648 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.648 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.648 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.648 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "Ceph", 2024-09-12T03:04:31.648 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.648 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.649 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.649 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Email From: name", 2024-09-12T03:04:31.649 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.649 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.649 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.649 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.649 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_host": { 2024-09-12T03:04:31.649 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_host", 2024-09-12T03:04:31.649 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.649 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.649 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.649 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.649 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.650 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.650 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.650 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SMTP server", 2024-09-12T03:04:31.650 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.650 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.650 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.650 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.650 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_password": { 2024-09-12T03:04:31.650 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_password", 2024-09-12T03:04:31.650 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.650 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.650 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.650 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.650 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.651 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.651 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.651 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Password to authenticate with", 2024-09-12T03:04:31.651 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.651 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.651 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.651 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.651 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_port": { 2024-09-12T03:04:31.651 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_port", 2024-09-12T03:04:31.651 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-12T03:04:31.651 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.651 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.651 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "465", 2024-09-12T03:04:31.652 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.652 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.652 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.652 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SMTP port", 2024-09-12T03:04:31.652 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.652 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.652 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.652 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.652 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_sender": { 2024-09-12T03:04:31.652 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_sender", 2024-09-12T03:04:31.652 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.652 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.652 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.652 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.653 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.653 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.653 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.653 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SMTP envelope sender", 2024-09-12T03:04:31.653 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.653 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.653 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.653 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.653 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_ssl": { 2024-09-12T03:04:31.653 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_ssl", 2024-09-12T03:04:31.653 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.653 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.653 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.654 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-12T03:04:31.654 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.654 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.654 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.654 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-12T03:04:31.654 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.654 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.654 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.654 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.654 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_user": { 2024-09-12T03:04:31.654 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_user", 2024-09-12T03:04:31.654 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.654 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.655 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.655 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.655 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.655 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.655 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.655 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "User to authenticate as", 2024-09-12T03:04:31.655 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.655 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.655 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.655 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:31.655 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:31.655 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.655 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-12T03:04:31.656 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "balancer", 2024-09-12T03:04:31.656 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-12T03:04:31.656 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-12T03:04:31.656 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-12T03:04:31.656 INFO:teuthology.orchestra.run.smithi005.stdout: "active": { 2024-09-12T03:04:31.656 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "active", 2024-09-12T03:04:31.656 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.656 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.656 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.656 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-12T03:04:31.656 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.656 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.656 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.657 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "automatically balance PGs across cluster", 2024-09-12T03:04:31.657 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.657 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.657 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.657 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.657 INFO:teuthology.orchestra.run.smithi005.stdout: "begin_time": { 2024-09-12T03:04:31.657 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "begin_time", 2024-09-12T03:04:31.657 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.657 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.657 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.657 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0000", 2024-09-12T03:04:31.657 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.657 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.657 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.658 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "beginning time of day to automatically balance", 2024-09-12T03:04:31.658 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-12T03:04:31.658 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.658 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.658 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.658 INFO:teuthology.orchestra.run.smithi005.stdout: "begin_weekday": { 2024-09-12T03:04:31.658 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "begin_weekday", 2024-09-12T03:04:31.658 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-09-12T03:04:31.658 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.658 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.658 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-09-12T03:04:31.658 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "0", 2024-09-12T03:04:31.658 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "6", 2024-09-12T03:04:31.659 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.659 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-12T03:04:31.659 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-12T03:04:31.659 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.659 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.659 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.659 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_compat_max_iterations": { 2024-09-12T03:04:31.659 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crush_compat_max_iterations", 2024-09-12T03:04:31.659 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-09-12T03:04:31.659 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.659 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.659 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "25", 2024-09-12T03:04:31.659 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2024-09-12T03:04:31.659 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "250", 2024-09-12T03:04:31.660 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.660 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-12T03:04:31.660 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.660 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.660 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.660 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.660 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_compat_metrics": { 2024-09-12T03:04:31.660 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crush_compat_metrics", 2024-09-12T03:04:31.660 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.660 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.660 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.660 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "pgs,objects,bytes", 2024-09-12T03:04:31.660 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.661 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.661 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.661 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-12T03:04:31.661 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-12T03:04:31.661 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.661 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.661 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.661 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_compat_step": { 2024-09-12T03:04:31.661 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crush_compat_step", 2024-09-12T03:04:31.661 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-09-12T03:04:31.661 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.661 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.661 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0.5", 2024-09-12T03:04:31.662 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "0.001", 2024-09-12T03:04:31.662 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "0.999", 2024-09-12T03:04:31.662 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.662 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "aggressiveness of optimization", 2024-09-12T03:04:31.662 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-12T03:04:31.662 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.662 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.662 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.662 INFO:teuthology.orchestra.run.smithi005.stdout: "end_time": { 2024-09-12T03:04:31.662 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "end_time", 2024-09-12T03:04:31.662 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.662 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.662 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.662 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "2359", 2024-09-12T03:04:31.663 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.663 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.663 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.663 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "ending time of day to automatically balance", 2024-09-12T03:04:31.663 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-12T03:04:31.663 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.663 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.663 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.663 INFO:teuthology.orchestra.run.smithi005.stdout: "end_weekday": { 2024-09-12T03:04:31.663 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "end_weekday", 2024-09-12T03:04:31.663 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-09-12T03:04:31.663 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.663 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.664 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-09-12T03:04:31.664 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "0", 2024-09-12T03:04:31.664 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "6", 2024-09-12T03:04:31.664 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.664 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-12T03:04:31.664 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-12T03:04:31.664 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.664 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.664 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.664 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-12T03:04:31.664 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-12T03:04:31.664 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.664 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.665 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.665 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.665 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.665 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.665 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.665 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:31.665 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:31.665 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:31.665 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:31.665 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:31.665 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:31.665 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.665 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.666 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.666 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.666 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.666 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.666 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-12T03:04:31.666 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-12T03:04:31.666 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.666 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.666 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.666 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.666 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.666 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.666 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.667 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.667 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.667 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.667 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.667 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.667 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-12T03:04:31.667 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-12T03:04:31.667 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.667 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.667 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.667 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-12T03:04:31.667 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.667 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.668 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.668 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:31.668 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:31.668 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:31.668 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:31.668 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:31.668 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:31.668 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.668 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.668 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.668 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.668 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.669 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.669 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-12T03:04:31.669 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-12T03:04:31.669 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.669 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.669 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.669 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.669 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.669 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.669 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.669 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.669 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.669 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.669 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.670 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.670 INFO:teuthology.orchestra.run.smithi005.stdout: "min_score": { 2024-09-12T03:04:31.670 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "min_score", 2024-09-12T03:04:31.670 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-09-12T03:04:31.670 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.670 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.670 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-09-12T03:04:31.670 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.670 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.670 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.670 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-12T03:04:31.670 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.670 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.671 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.671 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.671 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": { 2024-09-12T03:04:31.671 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mode", 2024-09-12T03:04:31.671 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.671 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.671 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.671 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "upmap", 2024-09-12T03:04:31.671 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.671 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.671 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.671 INFO:teuthology.orchestra.run.smithi005.stdout: "crush-compat", 2024-09-12T03:04:31.671 INFO:teuthology.orchestra.run.smithi005.stdout: "none", 2024-09-12T03:04:31.671 INFO:teuthology.orchestra.run.smithi005.stdout: "read", 2024-09-12T03:04:31.672 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap", 2024-09-12T03:04:31.672 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap-read" 2024-09-12T03:04:31.672 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.672 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Balancer mode", 2024-09-12T03:04:31.672 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.672 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.672 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.672 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.672 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_ids": { 2024-09-12T03:04:31.672 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pool_ids", 2024-09-12T03:04:31.672 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.672 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.673 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.673 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.673 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.673 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.673 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.673 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-12T03:04:31.673 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.673 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.673 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.673 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.673 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-09-12T03:04:31.673 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-09-12T03:04:31.673 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-12T03:04:31.674 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.674 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.674 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-09-12T03:04:31.674 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.674 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.674 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.674 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-12T03:04:31.674 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.674 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.674 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.674 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.674 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap_max_deviation": { 2024-09-12T03:04:31.674 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "upmap_max_deviation", 2024-09-12T03:04:31.675 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-12T03:04:31.675 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.675 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.675 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2024-09-12T03:04:31.675 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2024-09-12T03:04:31.675 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.675 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.675 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-12T03:04:31.675 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-12T03:04:31.675 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.675 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.675 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.675 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap_max_optimizations": { 2024-09-12T03:04:31.676 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "upmap_max_optimizations", 2024-09-12T03:04:31.676 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-09-12T03:04:31.676 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.676 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.676 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-09-12T03:04:31.676 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.676 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.676 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.676 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-12T03:04:31.676 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.676 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.676 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.676 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:31.677 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:31.677 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.677 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-12T03:04:31.677 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cephadm", 2024-09-12T03:04:31.677 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-12T03:04:31.677 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-12T03:04:31.677 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-12T03:04:31.677 INFO:teuthology.orchestra.run.smithi005.stdout: "agent_down_multiplier": { 2024-09-12T03:04:31.677 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "agent_down_multiplier", 2024-09-12T03:04:31.677 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-09-12T03:04:31.677 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.677 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.677 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "3.0", 2024-09-12T03:04:31.678 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.678 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.678 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.678 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-12T03:04:31.678 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.678 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.678 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.678 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.678 INFO:teuthology.orchestra.run.smithi005.stdout: "agent_refresh_rate": { 2024-09-12T03:04:31.678 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "agent_refresh_rate", 2024-09-12T03:04:31.678 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-12T03:04:31.678 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.678 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.678 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "20", 2024-09-12T03:04:31.679 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.679 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.679 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.679 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-12T03:04:31.679 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.679 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.679 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.679 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.679 INFO:teuthology.orchestra.run.smithi005.stdout: "agent_starting_port": { 2024-09-12T03:04:31.679 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "agent_starting_port", 2024-09-12T03:04:31.679 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-12T03:04:31.679 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.680 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.680 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "4721", 2024-09-12T03:04:31.680 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.680 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.680 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.680 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-12T03:04:31.680 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.680 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.680 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.680 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.680 INFO:teuthology.orchestra.run.smithi005.stdout: "allow_ptrace": { 2024-09-12T03:04:31.680 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "allow_ptrace", 2024-09-12T03:04:31.680 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.681 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.681 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.681 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.681 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.681 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.681 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.681 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-12T03:04:31.681 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-12T03:04:31.681 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.681 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.681 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.682 INFO:teuthology.orchestra.run.smithi005.stdout: "autotune_interval": { 2024-09-12T03:04:31.682 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "autotune_interval", 2024-09-12T03:04:31.682 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-12T03:04:31.682 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.682 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.682 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-09-12T03:04:31.682 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.682 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.682 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.682 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-12T03:04:31.682 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.682 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.683 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.683 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.683 INFO:teuthology.orchestra.run.smithi005.stdout: "autotune_memory_target_ratio": { 2024-09-12T03:04:31.683 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "autotune_memory_target_ratio", 2024-09-12T03:04:31.683 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-09-12T03:04:31.683 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.683 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.683 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0.7", 2024-09-12T03:04:31.683 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.683 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.683 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.683 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-12T03:04:31.683 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.684 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.684 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.684 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.684 INFO:teuthology.orchestra.run.smithi005.stdout: "cephadm_log_destination": { 2024-09-12T03:04:31.684 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cephadm_log_destination", 2024-09-12T03:04:31.684 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.684 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.684 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.684 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.684 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.684 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.684 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.684 INFO:teuthology.orchestra.run.smithi005.stdout: "file", 2024-09-12T03:04:31.685 INFO:teuthology.orchestra.run.smithi005.stdout: "file,syslog", 2024-09-12T03:04:31.685 INFO:teuthology.orchestra.run.smithi005.stdout: "syslog" 2024-09-12T03:04:31.685 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.685 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-12T03:04:31.685 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.685 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.685 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.685 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.685 INFO:teuthology.orchestra.run.smithi005.stdout: "cgroups_split": { 2024-09-12T03:04:31.685 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cgroups_split", 2024-09-12T03:04:31.685 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.685 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.685 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.686 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-12T03:04:31.686 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.686 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.686 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.686 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-12T03:04:31.686 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.686 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.686 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.686 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.686 INFO:teuthology.orchestra.run.smithi005.stdout: "config_checks_enabled": { 2024-09-12T03:04:31.686 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "config_checks_enabled", 2024-09-12T03:04:31.686 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.687 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.687 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.687 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.687 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.687 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.687 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.687 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-12T03:04:31.687 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.687 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.687 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.687 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.687 INFO:teuthology.orchestra.run.smithi005.stdout: "config_dashboard": { 2024-09-12T03:04:31.687 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "config_dashboard", 2024-09-12T03:04:31.688 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.688 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.688 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.688 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-12T03:04:31.688 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.688 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.688 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.688 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-12T03:04:31.688 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.688 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.688 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.688 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.688 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_alertmanager": { 2024-09-12T03:04:31.689 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_alertmanager", 2024-09-12T03:04:31.689 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.689 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.689 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.689 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-12T03:04:31.689 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.689 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.689 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.689 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2024-09-12T03:04:31.689 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.689 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.689 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.689 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.689 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_base": { 2024-09-12T03:04:31.690 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_base", 2024-09-12T03:04:31.690 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.690 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.690 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.690 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-12T03:04:31.690 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.690 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.690 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.690 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Container image name, without the tag", 2024-09-12T03:04:31.690 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.690 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.690 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.690 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.691 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_elasticsearch": { 2024-09-12T03:04:31.691 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_elasticsearch", 2024-09-12T03:04:31.691 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.691 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.691 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.691 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-12T03:04:31.691 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.691 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.691 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.691 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "elasticsearch container image", 2024-09-12T03:04:31.691 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.691 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.691 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.692 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.692 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_grafana": { 2024-09-12T03:04:31.692 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_grafana", 2024-09-12T03:04:31.692 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.692 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.692 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.692 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-12T03:04:31.692 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.692 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.692 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.692 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2024-09-12T03:04:31.692 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.692 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.693 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.693 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.693 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_haproxy": { 2024-09-12T03:04:31.693 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_haproxy", 2024-09-12T03:04:31.693 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.693 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.693 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.693 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-12T03:04:31.693 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.693 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.693 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.693 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "HAproxy container image", 2024-09-12T03:04:31.693 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.694 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.694 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.694 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.694 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_jaeger_agent": { 2024-09-12T03:04:31.694 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_jaeger_agent", 2024-09-12T03:04:31.694 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.694 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.694 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.694 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-12T03:04:31.694 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.694 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.694 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.695 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Jaeger agent container image", 2024-09-12T03:04:31.695 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.695 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.695 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.695 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.695 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_jaeger_collector": { 2024-09-12T03:04:31.695 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_jaeger_collector", 2024-09-12T03:04:31.695 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.695 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.695 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.695 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-12T03:04:31.695 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.695 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.696 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.696 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Jaeger collector container image", 2024-09-12T03:04:31.696 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.696 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.696 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.696 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.696 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_jaeger_query": { 2024-09-12T03:04:31.696 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_jaeger_query", 2024-09-12T03:04:31.696 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.696 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.696 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.696 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-12T03:04:31.696 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.697 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.697 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.697 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Jaeger query container image", 2024-09-12T03:04:31.697 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.697 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.697 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.697 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.697 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_keepalived": { 2024-09-12T03:04:31.697 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_keepalived", 2024-09-12T03:04:31.697 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.697 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.697 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.697 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-12T03:04:31.698 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.698 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.698 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.698 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Keepalived container image", 2024-09-12T03:04:31.698 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.698 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.698 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.698 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.698 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_loki": { 2024-09-12T03:04:31.698 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_loki", 2024-09-12T03:04:31.698 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.698 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.698 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.699 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-12T03:04:31.699 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.699 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.699 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.699 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Loki container image", 2024-09-12T03:04:31.699 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.699 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.699 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.699 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.699 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_node_exporter": { 2024-09-12T03:04:31.699 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_node_exporter", 2024-09-12T03:04:31.699 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.699 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.700 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.700 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-12T03:04:31.700 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.700 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.700 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.700 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2024-09-12T03:04:31.700 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.700 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.700 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.700 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.700 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_nvmeof": { 2024-09-12T03:04:31.700 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_nvmeof", 2024-09-12T03:04:31.701 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.701 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.701 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.701 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-12T03:04:31.701 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.701 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.701 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.701 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Nvme-of container image", 2024-09-12T03:04:31.701 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.701 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.701 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.701 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.701 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_prometheus": { 2024-09-12T03:04:31.701 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_prometheus", 2024-09-12T03:04:31.702 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.702 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.702 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.702 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-12T03:04:31.702 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.702 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.702 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.702 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2024-09-12T03:04:31.702 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.702 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.702 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.702 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.702 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_promtail": { 2024-09-12T03:04:31.703 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_promtail", 2024-09-12T03:04:31.703 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.703 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.703 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.703 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-12T03:04:31.703 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.703 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.703 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.703 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Promtail container image", 2024-09-12T03:04:31.703 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.703 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.703 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.703 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.704 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_samba": { 2024-09-12T03:04:31.704 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_samba", 2024-09-12T03:04:31.704 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.704 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.704 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.704 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-12T03:04:31.704 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.704 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.704 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.704 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Samba/SMB container image", 2024-09-12T03:04:31.704 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.704 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.705 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.705 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.705 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_snmp_gateway": { 2024-09-12T03:04:31.705 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_snmp_gateway", 2024-09-12T03:04:31.705 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.705 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.705 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.705 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-12T03:04:31.705 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.705 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.705 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.705 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SNMP Gateway container image", 2024-09-12T03:04:31.705 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.706 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.706 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.706 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.706 INFO:teuthology.orchestra.run.smithi005.stdout: "container_init": { 2024-09-12T03:04:31.706 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_init", 2024-09-12T03:04:31.706 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.706 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.706 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.706 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-12T03:04:31.706 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.706 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.706 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.706 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Run podman/docker with `--init`", 2024-09-12T03:04:31.706 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.706 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.707 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.707 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.707 INFO:teuthology.orchestra.run.smithi005.stdout: "daemon_cache_timeout": { 2024-09-12T03:04:31.707 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "daemon_cache_timeout", 2024-09-12T03:04:31.707 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-12T03:04:31.707 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.707 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.707 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-09-12T03:04:31.707 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.707 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.707 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.707 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-12T03:04:31.708 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.708 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.708 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.708 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.708 INFO:teuthology.orchestra.run.smithi005.stdout: "default_cephadm_command_timeout": { 2024-09-12T03:04:31.708 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "default_cephadm_command_timeout", 2024-09-12T03:04:31.708 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-12T03:04:31.708 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.708 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.708 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "900", 2024-09-12T03:04:31.708 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.708 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.708 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.709 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-12T03:04:31.709 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.709 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.709 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.709 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.709 INFO:teuthology.orchestra.run.smithi005.stdout: "default_registry": { 2024-09-12T03:04:31.709 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "default_registry", 2024-09-12T03:04:31.709 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.709 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.709 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.709 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io", 2024-09-12T03:04:31.709 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.710 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.710 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.710 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-12T03:04:31.710 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.710 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.710 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.710 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.710 INFO:teuthology.orchestra.run.smithi005.stdout: "device_cache_timeout": { 2024-09-12T03:04:31.710 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "device_cache_timeout", 2024-09-12T03:04:31.710 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-12T03:04:31.710 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.710 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.710 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "1800", 2024-09-12T03:04:31.711 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.711 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.711 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.711 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "seconds to cache device inventory", 2024-09-12T03:04:31.711 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.711 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.711 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.711 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.711 INFO:teuthology.orchestra.run.smithi005.stdout: "device_enhanced_scan": { 2024-09-12T03:04:31.711 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "device_enhanced_scan", 2024-09-12T03:04:31.711 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.711 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.711 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.712 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.712 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.712 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.712 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.712 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-12T03:04:31.712 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.712 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.712 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.712 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.712 INFO:teuthology.orchestra.run.smithi005.stdout: "facts_cache_timeout": { 2024-09-12T03:04:31.712 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "facts_cache_timeout", 2024-09-12T03:04:31.712 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-12T03:04:31.712 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.713 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.713 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-09-12T03:04:31.713 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.713 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.713 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.713 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "seconds to cache host facts data", 2024-09-12T03:04:31.713 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.713 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.713 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.713 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.713 INFO:teuthology.orchestra.run.smithi005.stdout: "grafana_dashboards_path": { 2024-09-12T03:04:31.713 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "grafana_dashboards_path", 2024-09-12T03:04:31.714 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.714 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.714 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.714 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-12T03:04:31.714 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.714 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.714 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.714 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-12T03:04:31.714 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.714 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.714 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.714 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.714 INFO:teuthology.orchestra.run.smithi005.stdout: "host_check_interval": { 2024-09-12T03:04:31.715 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "host_check_interval", 2024-09-12T03:04:31.715 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-12T03:04:31.715 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.715 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.715 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-09-12T03:04:31.715 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.715 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.715 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.715 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to perform a host check", 2024-09-12T03:04:31.715 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.715 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.716 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.716 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.716 INFO:teuthology.orchestra.run.smithi005.stdout: "hw_monitoring": { 2024-09-12T03:04:31.716 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "hw_monitoring", 2024-09-12T03:04:31.716 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.716 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.716 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.716 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.716 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.716 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.716 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.716 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-12T03:04:31.716 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.717 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.717 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.717 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.717 INFO:teuthology.orchestra.run.smithi005.stdout: "inventory_list_all": { 2024-09-12T03:04:31.717 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "inventory_list_all", 2024-09-12T03:04:31.717 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.717 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.717 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.717 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.717 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.717 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.717 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.717 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-12T03:04:31.718 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.718 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.718 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.718 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.718 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-12T03:04:31.718 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-12T03:04:31.718 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.718 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.718 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.718 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.718 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.718 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.718 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.718 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:31.719 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:31.719 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:31.719 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:31.719 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:31.719 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:31.719 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.719 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.719 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.719 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.719 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.719 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.719 INFO:teuthology.orchestra.run.smithi005.stdout: "log_refresh_metadata": { 2024-09-12T03:04:31.719 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_refresh_metadata", 2024-09-12T03:04:31.720 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.720 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.720 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.720 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.720 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.720 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.720 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.720 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-12T03:04:31.720 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.720 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.720 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.720 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.721 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-12T03:04:31.721 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-12T03:04:31.721 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.721 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.721 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.721 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-12T03:04:31.721 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.721 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.721 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.721 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-12T03:04:31.721 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.721 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.721 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.722 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.722 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-12T03:04:31.722 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-12T03:04:31.722 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.722 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.722 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.722 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-12T03:04:31.722 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.722 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.722 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.722 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:31.722 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:31.722 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:31.723 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:31.723 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:31.723 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:31.723 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.723 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.723 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.723 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.723 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.723 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.723 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-12T03:04:31.723 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-12T03:04:31.723 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.723 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.724 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.724 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.724 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.724 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.724 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.724 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.724 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.724 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.724 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.724 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.724 INFO:teuthology.orchestra.run.smithi005.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-12T03:04:31.724 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-12T03:04:31.724 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.725 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.725 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.725 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.725 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.725 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.725 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.725 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-12T03:04:31.725 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.725 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.725 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.725 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.725 INFO:teuthology.orchestra.run.smithi005.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-12T03:04:31.725 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-12T03:04:31.726 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.726 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.726 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.726 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "*", 2024-09-12T03:04:31.726 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.726 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.726 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.726 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-12T03:04:31.726 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.726 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.726 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.726 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.726 INFO:teuthology.orchestra.run.smithi005.stdout: "max_count_per_host": { 2024-09-12T03:04:31.726 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_count_per_host", 2024-09-12T03:04:31.727 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-12T03:04:31.727 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.727 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.727 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-09-12T03:04:31.727 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.727 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.727 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.727 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "max number of daemons per service per host", 2024-09-12T03:04:31.727 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.727 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.727 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.727 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.727 INFO:teuthology.orchestra.run.smithi005.stdout: "max_osd_draining_count": { 2024-09-12T03:04:31.728 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_osd_draining_count", 2024-09-12T03:04:31.728 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-12T03:04:31.728 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.728 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.728 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-09-12T03:04:31.728 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.728 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.728 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.728 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-12T03:04:31.728 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.728 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.728 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.728 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.729 INFO:teuthology.orchestra.run.smithi005.stdout: "migration_current": { 2024-09-12T03:04:31.729 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "migration_current", 2024-09-12T03:04:31.729 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-12T03:04:31.729 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.729 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.729 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.729 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.729 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.729 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.729 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "internal - do not modify", 2024-09-12T03:04:31.729 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.729 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.729 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.730 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.730 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": { 2024-09-12T03:04:31.730 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mode", 2024-09-12T03:04:31.730 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.730 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.730 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.730 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "root", 2024-09-12T03:04:31.730 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.730 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.730 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.730 INFO:teuthology.orchestra.run.smithi005.stdout: "cephadm-package", 2024-09-12T03:04:31.730 INFO:teuthology.orchestra.run.smithi005.stdout: "root" 2024-09-12T03:04:31.730 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.730 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "mode for remote execution of cephadm", 2024-09-12T03:04:31.731 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.731 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.731 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.731 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.731 INFO:teuthology.orchestra.run.smithi005.stdout: "oob_default_addr": { 2024-09-12T03:04:31.731 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "oob_default_addr", 2024-09-12T03:04:31.731 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.731 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.731 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.731 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "169.254.1.1", 2024-09-12T03:04:31.731 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.731 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.731 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.731 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-12T03:04:31.732 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.732 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.732 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.732 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.732 INFO:teuthology.orchestra.run.smithi005.stdout: "prometheus_alerts_path": { 2024-09-12T03:04:31.732 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "prometheus_alerts_path", 2024-09-12T03:04:31.732 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.732 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.732 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.732 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-12T03:04:31.732 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.732 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.733 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.733 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-12T03:04:31.733 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.733 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.733 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.733 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.733 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_insecure": { 2024-09-12T03:04:31.733 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_insecure", 2024-09-12T03:04:31.733 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.733 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.733 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.733 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.733 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.733 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.734 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.734 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-12T03:04:31.734 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.734 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.734 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.734 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.734 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_password": { 2024-09-12T03:04:31.734 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_password", 2024-09-12T03:04:31.734 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.734 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.734 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.734 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.735 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.735 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.735 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.735 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-12T03:04:31.735 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.735 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.735 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.735 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.735 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_url": { 2024-09-12T03:04:31.735 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_url", 2024-09-12T03:04:31.735 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.735 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.735 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.736 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.736 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.736 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.736 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.736 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-12T03:04:31.736 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.736 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.736 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.736 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.736 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_username": { 2024-09-12T03:04:31.736 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_username", 2024-09-12T03:04:31.736 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.737 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.737 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.737 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.737 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.737 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.737 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.737 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-12T03:04:31.737 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.737 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.737 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.737 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.737 INFO:teuthology.orchestra.run.smithi005.stdout: "secure_monitoring_stack": { 2024-09-12T03:04:31.737 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "secure_monitoring_stack", 2024-09-12T03:04:31.738 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.738 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.738 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.738 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.738 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.738 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.738 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.738 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-12T03:04:31.738 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.738 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.738 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.738 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.738 INFO:teuthology.orchestra.run.smithi005.stdout: "service_discovery_port": { 2024-09-12T03:04:31.738 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "service_discovery_port", 2024-09-12T03:04:31.739 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-12T03:04:31.739 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.739 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.739 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8765", 2024-09-12T03:04:31.739 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.739 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.739 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.739 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "cephadm service discovery port", 2024-09-12T03:04:31.739 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.739 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.739 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.739 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.739 INFO:teuthology.orchestra.run.smithi005.stdout: "ssh_config_file": { 2024-09-12T03:04:31.740 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssh_config_file", 2024-09-12T03:04:31.740 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.740 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.740 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.740 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.740 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.740 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.740 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.740 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-12T03:04:31.740 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.740 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.740 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.740 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.741 INFO:teuthology.orchestra.run.smithi005.stdout: "use_agent": { 2024-09-12T03:04:31.741 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "use_agent", 2024-09-12T03:04:31.741 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.741 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.741 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.741 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.741 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.741 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.741 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.741 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-12T03:04:31.741 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.741 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.741 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.742 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.742 INFO:teuthology.orchestra.run.smithi005.stdout: "use_repo_digest": { 2024-09-12T03:04:31.742 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "use_repo_digest", 2024-09-12T03:04:31.742 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.742 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.742 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.742 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-12T03:04:31.742 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.742 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.742 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.742 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-12T03:04:31.742 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.742 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.742 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.743 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.743 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_on_failed_host_check": { 2024-09-12T03:04:31.743 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_on_failed_host_check", 2024-09-12T03:04:31.743 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.743 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.743 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.743 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-12T03:04:31.743 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.743 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.743 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.743 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise a health warning if the host check fails", 2024-09-12T03:04:31.743 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.743 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.744 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.744 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.744 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_on_stray_daemons": { 2024-09-12T03:04:31.744 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_on_stray_daemons", 2024-09-12T03:04:31.744 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.744 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.744 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.744 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-12T03:04:31.744 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.744 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.744 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.744 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-12T03:04:31.744 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.744 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.745 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.745 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.745 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_on_stray_hosts": { 2024-09-12T03:04:31.745 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_on_stray_hosts", 2024-09-12T03:04:31.745 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.745 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.745 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.745 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-12T03:04:31.745 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.745 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.745 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.745 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-12T03:04:31.745 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.746 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.746 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.746 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:31.746 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:31.746 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.746 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-12T03:04:31.746 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crash", 2024-09-12T03:04:31.746 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-12T03:04:31.746 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-12T03:04:31.746 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-12T03:04:31.746 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-12T03:04:31.746 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-12T03:04:31.746 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.746 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.747 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.747 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.747 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.747 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.747 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.747 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:31.747 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:31.747 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:31.747 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:31.747 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:31.747 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:31.747 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.748 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.748 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.748 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.748 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.748 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.748 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-12T03:04:31.748 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-12T03:04:31.748 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.748 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.748 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.748 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.748 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.748 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.749 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.749 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.749 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.749 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.749 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.749 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.749 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-12T03:04:31.749 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-12T03:04:31.749 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.749 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.749 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.749 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-12T03:04:31.750 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.750 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.750 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.750 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:31.750 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:31.750 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:31.750 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:31.750 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:31.750 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:31.750 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.750 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.750 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.750 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.751 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.751 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.751 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-12T03:04:31.751 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-12T03:04:31.751 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.751 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.751 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.751 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.751 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.751 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.751 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.751 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.751 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.752 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.752 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.752 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.752 INFO:teuthology.orchestra.run.smithi005.stdout: "retain_interval": { 2024-09-12T03:04:31.752 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "retain_interval", 2024-09-12T03:04:31.752 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-12T03:04:31.752 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.752 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.752 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "31536000", 2024-09-12T03:04:31.752 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.752 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.752 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.752 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-12T03:04:31.753 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.753 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.753 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.753 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.753 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_recent_interval": { 2024-09-12T03:04:31.753 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_recent_interval", 2024-09-12T03:04:31.753 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-12T03:04:31.753 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.753 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.753 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "1209600", 2024-09-12T03:04:31.753 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.753 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.753 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.754 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-12T03:04:31.754 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.754 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.754 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.754 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:31.754 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:31.754 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.754 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-12T03:04:31.754 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "dashboard", 2024-09-12T03:04:31.754 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-12T03:04:31.754 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-12T03:04:31.754 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-12T03:04:31.755 INFO:teuthology.orchestra.run.smithi005.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-12T03:04:31.755 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-12T03:04:31.755 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-12T03:04:31.755 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.755 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.755 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-09-12T03:04:31.755 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.755 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.755 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.755 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.755 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.755 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.755 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.756 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.756 INFO:teuthology.orchestra.run.smithi005.stdout: "ALERTMANAGER_API_HOST": { 2024-09-12T03:04:31.756 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-12T03:04:31.756 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.756 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.756 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.756 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.756 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.756 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.756 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.756 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.756 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.756 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.757 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.757 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.757 INFO:teuthology.orchestra.run.smithi005.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-12T03:04:31.757 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-12T03:04:31.757 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.757 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.757 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.757 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-12T03:04:31.757 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.757 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.757 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.757 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.757 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.758 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.758 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.758 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.758 INFO:teuthology.orchestra.run.smithi005.stdout: "AUDIT_API_ENABLED": { 2024-09-12T03:04:31.758 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "AUDIT_API_ENABLED", 2024-09-12T03:04:31.758 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.758 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.758 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.758 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.758 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.758 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.758 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.758 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.759 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.759 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.759 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.759 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.759 INFO:teuthology.orchestra.run.smithi005.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-12T03:04:31.759 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-12T03:04:31.759 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.759 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.759 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.759 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-12T03:04:31.759 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.759 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.759 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.759 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.760 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.760 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.760 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.760 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.760 INFO:teuthology.orchestra.run.smithi005.stdout: "ENABLE_BROWSABLE_API": { 2024-09-12T03:04:31.760 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-12T03:04:31.760 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.760 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.760 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.760 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-12T03:04:31.760 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.760 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.760 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.761 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.761 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.761 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.761 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.761 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.761 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-12T03:04:31.761 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-12T03:04:31.761 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.761 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.761 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.761 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-12T03:04:31.761 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.761 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.762 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.762 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.762 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.762 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.762 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.762 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.762 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-12T03:04:31.762 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-12T03:04:31.762 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.762 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.762 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.762 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-12T03:04:31.763 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.763 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.763 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.763 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.763 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.763 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.763 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.763 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.763 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-12T03:04:31.763 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-12T03:04:31.763 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.763 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.763 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.764 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-12T03:04:31.764 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.764 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.764 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.764 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.764 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.764 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.764 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.764 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.764 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-12T03:04:31.764 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-12T03:04:31.764 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.765 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.765 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.765 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-12T03:04:31.765 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.765 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.765 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.765 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.765 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.765 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.765 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.765 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.765 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-12T03:04:31.765 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-12T03:04:31.766 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.766 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.766 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.766 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-12T03:04:31.766 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.766 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.766 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.766 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.766 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.766 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.766 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.766 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.766 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-12T03:04:31.766 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-12T03:04:31.767 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.767 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.767 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.767 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-12T03:04:31.767 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.767 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.767 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.767 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.767 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.767 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.767 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.767 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.767 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-12T03:04:31.768 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-12T03:04:31.768 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.768 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.768 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.768 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-12T03:04:31.768 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.768 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.768 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.768 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.768 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.768 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.768 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.768 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.769 INFO:teuthology.orchestra.run.smithi005.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-12T03:04:31.769 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-12T03:04:31.769 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.769 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.769 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.769 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.769 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.769 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.769 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.769 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.769 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.769 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.769 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.770 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.770 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_PASSWORD": { 2024-09-12T03:04:31.770 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-12T03:04:31.770 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.770 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.770 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.770 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "admin", 2024-09-12T03:04:31.770 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.770 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.770 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.770 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.770 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.770 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.770 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.771 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.771 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-12T03:04:31.771 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-12T03:04:31.771 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.771 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.771 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.771 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-12T03:04:31.771 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.771 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.771 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.771 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.771 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.771 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.772 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.772 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.772 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_URL": { 2024-09-12T03:04:31.772 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_URL", 2024-09-12T03:04:31.772 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.772 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.772 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.772 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.772 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.772 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.772 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.772 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.772 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.773 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.773 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.773 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.773 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_USERNAME": { 2024-09-12T03:04:31.773 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-12T03:04:31.773 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.773 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.773 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.773 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "admin", 2024-09-12T03:04:31.773 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.773 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.773 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.773 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.774 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.774 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.774 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.775 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.776 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-12T03:04:31.776 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-12T03:04:31.776 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.776 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.776 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.776 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.776 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.776 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.776 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.776 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.776 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.776 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.777 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.777 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.777 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-12T03:04:31.777 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-12T03:04:31.777 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.777 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.777 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.777 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.777 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.778 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.778 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.778 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.778 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.778 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.778 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.778 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.778 INFO:teuthology.orchestra.run.smithi005.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-12T03:04:31.778 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-12T03:04:31.778 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.779 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.779 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.779 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-12T03:04:31.779 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.779 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.779 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.779 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.779 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.779 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.779 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.779 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.779 INFO:teuthology.orchestra.run.smithi005.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-12T03:04:31.779 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-12T03:04:31.780 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.780 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.780 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.780 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.780 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.780 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.780 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.780 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.780 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.780 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.780 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.780 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.780 INFO:teuthology.orchestra.run.smithi005.stdout: "PROMETHEUS_API_HOST": { 2024-09-12T03:04:31.780 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-12T03:04:31.781 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.781 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.781 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.781 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.781 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.781 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.781 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.781 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.781 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.781 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.781 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.781 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.782 INFO:teuthology.orchestra.run.smithi005.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-12T03:04:31.782 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-12T03:04:31.782 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.782 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.782 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.782 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-12T03:04:31.782 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.782 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.782 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.782 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.782 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.782 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.782 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.783 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.783 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-12T03:04:31.783 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-12T03:04:31.783 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.783 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.783 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.783 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.783 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.783 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.783 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.783 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.783 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.783 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.784 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.784 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.784 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-12T03:04:31.784 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-12T03:04:31.784 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.784 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.784 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.784 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.784 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.784 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.784 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.784 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.785 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.785 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.785 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.785 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.785 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-12T03:04:31.785 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-12T03:04:31.785 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.785 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.785 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.785 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-12T03:04:31.785 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.785 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.785 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.786 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.786 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.786 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.786 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.786 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.786 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-12T03:04:31.786 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-12T03:04:31.786 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.786 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.786 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.786 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-12T03:04:31.786 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.787 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.787 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.787 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.787 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.787 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.787 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.787 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.787 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-12T03:04:31.787 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-12T03:04:31.787 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.787 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.787 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.787 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.788 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.788 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.788 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.788 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.788 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.788 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.788 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.788 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.788 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-12T03:04:31.788 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-12T03:04:31.788 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.788 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.788 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.789 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.789 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.789 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.789 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.789 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.789 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.789 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.789 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.789 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.789 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-12T03:04:31.789 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-12T03:04:31.789 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.789 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.790 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.790 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.790 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.790 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.790 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.790 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.790 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.790 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.790 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.790 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.790 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_ENABLED": { 2024-09-12T03:04:31.790 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-12T03:04:31.790 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.790 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.791 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.791 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-12T03:04:31.791 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.791 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.791 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.791 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.791 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.791 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.791 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.791 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.791 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-12T03:04:31.791 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-12T03:04:31.791 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.791 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.791 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.792 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-12T03:04:31.792 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.792 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.792 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.792 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.792 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.792 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.792 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.792 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.792 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-12T03:04:31.792 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-12T03:04:31.792 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-12T03:04:31.792 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.792 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.793 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-09-12T03:04:31.793 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.793 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.793 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.793 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.793 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.793 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.793 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.793 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.793 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-12T03:04:31.793 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-12T03:04:31.793 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-12T03:04:31.793 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.793 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.794 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8", 2024-09-12T03:04:31.794 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.794 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.794 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.794 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.794 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.794 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.794 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.794 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.794 INFO:teuthology.orchestra.run.smithi005.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-12T03:04:31.794 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-12T03:04:31.794 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-12T03:04:31.794 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.794 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.794 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "45", 2024-09-12T03:04:31.795 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.795 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.795 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.795 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.795 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.795 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.795 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.795 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.795 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_ACCESS_KEY": { 2024-09-12T03:04:31.795 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-12T03:04:31.795 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.795 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.795 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.795 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.796 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.796 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.796 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.796 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.796 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.796 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.796 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.796 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.796 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-12T03:04:31.796 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-12T03:04:31.796 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.796 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.796 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.797 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "admin", 2024-09-12T03:04:31.797 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.797 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.797 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.797 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.797 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.797 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.797 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.797 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.797 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_SECRET_KEY": { 2024-09-12T03:04:31.797 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-12T03:04:31.797 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.797 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.798 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.798 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.798 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.798 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.798 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.798 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.798 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.798 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.798 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.798 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.798 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_SSL_VERIFY": { 2024-09-12T03:04:31.798 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-12T03:04:31.799 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.799 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.799 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.799 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-12T03:04:31.799 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.799 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.799 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.799 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.799 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.799 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.799 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.799 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.799 INFO:teuthology.orchestra.run.smithi005.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-12T03:04:31.799 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-12T03:04:31.800 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-12T03:04:31.800 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.800 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.800 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-09-12T03:04:31.800 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.800 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.800 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.800 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.800 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.800 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.800 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.800 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.800 INFO:teuthology.orchestra.run.smithi005.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-12T03:04:31.801 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-12T03:04:31.801 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-12T03:04:31.801 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.801 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.801 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-09-12T03:04:31.801 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.801 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.801 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.801 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.801 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.801 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.801 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.801 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.802 INFO:teuthology.orchestra.run.smithi005.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-12T03:04:31.802 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-12T03:04:31.802 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-12T03:04:31.802 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.802 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.802 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2024-09-12T03:04:31.802 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.802 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.802 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.802 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.802 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.802 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.802 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.803 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.803 INFO:teuthology.orchestra.run.smithi005.stdout: "cross_origin_url": { 2024-09-12T03:04:31.803 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cross_origin_url", 2024-09-12T03:04:31.803 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.803 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.803 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.803 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.803 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.803 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.803 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.803 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.803 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.803 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.804 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.804 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.804 INFO:teuthology.orchestra.run.smithi005.stdout: "crt_file": { 2024-09-12T03:04:31.804 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crt_file", 2024-09-12T03:04:31.804 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.804 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.804 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.804 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.804 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.804 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.804 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.804 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.804 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.805 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.805 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.805 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.805 INFO:teuthology.orchestra.run.smithi005.stdout: "debug": { 2024-09-12T03:04:31.805 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "debug", 2024-09-12T03:04:31.805 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.805 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.805 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.805 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.805 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.805 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.805 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.806 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Enable/disable debug options", 2024-09-12T03:04:31.806 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.806 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.806 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.806 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.806 INFO:teuthology.orchestra.run.smithi005.stdout: "jwt_token_ttl": { 2024-09-12T03:04:31.806 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "jwt_token_ttl", 2024-09-12T03:04:31.806 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-12T03:04:31.806 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.806 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.806 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "28800", 2024-09-12T03:04:31.806 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.807 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.807 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.807 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.807 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.807 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.807 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.807 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.807 INFO:teuthology.orchestra.run.smithi005.stdout: "key_file": { 2024-09-12T03:04:31.807 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "key_file", 2024-09-12T03:04:31.807 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.807 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.807 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.807 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.807 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.808 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.808 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.808 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.808 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.808 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.808 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.808 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.808 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-12T03:04:31.808 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-12T03:04:31.808 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.808 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.808 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.808 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.809 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.809 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.809 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.809 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:31.809 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:31.809 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:31.809 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:31.809 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:31.809 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:31.809 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.809 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.809 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.809 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.809 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.810 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.810 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-12T03:04:31.810 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-12T03:04:31.810 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.810 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.810 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.810 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.810 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.810 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.810 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.810 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.810 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.810 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.810 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.811 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.811 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-12T03:04:31.811 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-12T03:04:31.811 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.811 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.811 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.811 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-12T03:04:31.811 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.811 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.811 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.811 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:31.811 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:31.811 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:31.811 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:31.812 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:31.812 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:31.812 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.812 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.812 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.812 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.812 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.812 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.812 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-12T03:04:31.812 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-12T03:04:31.812 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.812 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.812 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.813 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.813 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.813 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.813 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.813 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.813 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.813 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.813 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.813 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.813 INFO:teuthology.orchestra.run.smithi005.stdout: "motd": { 2024-09-12T03:04:31.813 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "motd", 2024-09-12T03:04:31.813 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.813 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.814 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.814 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.814 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.814 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.814 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.814 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "The message of the day", 2024-09-12T03:04:31.814 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.814 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.814 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.814 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.814 INFO:teuthology.orchestra.run.smithi005.stdout: "redirect_resolve_ip_addr": { 2024-09-12T03:04:31.814 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "redirect_resolve_ip_addr", 2024-09-12T03:04:31.814 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.815 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.815 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.815 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.815 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.815 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.815 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.815 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.815 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.815 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.815 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.815 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.815 INFO:teuthology.orchestra.run.smithi005.stdout: "server_addr": { 2024-09-12T03:04:31.815 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_addr", 2024-09-12T03:04:31.816 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.816 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.816 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.816 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "::", 2024-09-12T03:04:31.816 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.816 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.816 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.816 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.816 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.816 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.816 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.816 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.816 INFO:teuthology.orchestra.run.smithi005.stdout: "server_port": { 2024-09-12T03:04:31.817 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_port", 2024-09-12T03:04:31.817 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-12T03:04:31.817 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.817 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.817 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8080", 2024-09-12T03:04:31.817 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.817 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.817 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.817 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.817 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.817 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.817 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.817 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.817 INFO:teuthology.orchestra.run.smithi005.stdout: "ssl": { 2024-09-12T03:04:31.818 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssl", 2024-09-12T03:04:31.818 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.818 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.818 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.818 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-12T03:04:31.818 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.818 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.818 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.818 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.818 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.818 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.818 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.818 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.819 INFO:teuthology.orchestra.run.smithi005.stdout: "ssl_server_port": { 2024-09-12T03:04:31.819 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssl_server_port", 2024-09-12T03:04:31.819 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-12T03:04:31.819 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.819 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.819 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8443", 2024-09-12T03:04:31.819 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.819 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.819 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.819 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.819 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.819 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.819 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.820 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.820 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_behaviour": { 2024-09-12T03:04:31.820 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_behaviour", 2024-09-12T03:04:31.820 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.820 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.820 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.820 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "redirect", 2024-09-12T03:04:31.820 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.820 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.820 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.820 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:31.820 INFO:teuthology.orchestra.run.smithi005.stdout: "redirect" 2024-09-12T03:04:31.820 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.821 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.821 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.821 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.821 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.821 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.821 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_error_status_code": { 2024-09-12T03:04:31.821 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_error_status_code", 2024-09-12T03:04:31.821 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-12T03:04:31.821 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.821 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.821 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "500", 2024-09-12T03:04:31.821 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "400", 2024-09-12T03:04:31.821 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "599", 2024-09-12T03:04:31.822 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.822 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.822 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.822 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.822 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.822 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.822 INFO:teuthology.orchestra.run.smithi005.stdout: "url_prefix": { 2024-09-12T03:04:31.822 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "url_prefix", 2024-09-12T03:04:31.822 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.822 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.822 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.822 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.822 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.822 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.823 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.823 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.823 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.823 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.823 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.823 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:31.823 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:31.823 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.823 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-12T03:04:31.823 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "devicehealth", 2024-09-12T03:04:31.823 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-12T03:04:31.823 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-12T03:04:31.824 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-12T03:04:31.824 INFO:teuthology.orchestra.run.smithi005.stdout: "enable_monitoring": { 2024-09-12T03:04:31.824 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enable_monitoring", 2024-09-12T03:04:31.824 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.824 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.824 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.824 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-12T03:04:31.824 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.824 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.824 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.824 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "monitor device health metrics", 2024-09-12T03:04:31.824 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.824 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.825 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.825 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.825 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-12T03:04:31.825 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-12T03:04:31.825 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.825 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.825 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.825 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.825 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.825 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.825 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.825 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:31.825 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:31.826 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:31.826 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:31.826 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:31.826 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:31.826 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.826 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.826 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.826 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.826 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.826 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.826 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-12T03:04:31.826 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-12T03:04:31.826 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.827 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.827 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.827 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.827 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.827 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.827 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.827 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.827 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.827 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.827 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.827 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.827 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-12T03:04:31.827 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-12T03:04:31.828 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.828 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.828 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.828 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-12T03:04:31.828 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.828 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.828 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.828 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:31.828 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:31.828 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:31.828 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:31.828 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:31.828 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:31.828 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.829 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.829 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.829 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.829 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.829 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.829 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-12T03:04:31.829 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-12T03:04:31.829 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.829 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.829 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.829 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.829 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.829 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.829 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.830 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.830 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.830 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.830 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.830 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.830 INFO:teuthology.orchestra.run.smithi005.stdout: "mark_out_threshold": { 2024-09-12T03:04:31.830 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mark_out_threshold", 2024-09-12T03:04:31.830 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-12T03:04:31.830 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.830 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.830 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "2419200", 2024-09-12T03:04:31.830 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.830 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.830 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.831 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-12T03:04:31.831 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.831 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.831 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.831 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.831 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_name": { 2024-09-12T03:04:31.831 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pool_name", 2024-09-12T03:04:31.831 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.831 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.831 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.831 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "device_health_metrics", 2024-09-12T03:04:31.831 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.831 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.831 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.832 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-12T03:04:31.832 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.832 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.832 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.832 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.832 INFO:teuthology.orchestra.run.smithi005.stdout: "retention_period": { 2024-09-12T03:04:31.832 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "retention_period", 2024-09-12T03:04:31.832 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-12T03:04:31.832 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.832 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.832 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "15552000", 2024-09-12T03:04:31.832 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.833 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.833 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.833 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how long to retain device health metrics", 2024-09-12T03:04:31.833 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.833 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.833 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.833 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.833 INFO:teuthology.orchestra.run.smithi005.stdout: "scrape_frequency": { 2024-09-12T03:04:31.833 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "scrape_frequency", 2024-09-12T03:04:31.833 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-12T03:04:31.833 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.833 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.833 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "86400", 2024-09-12T03:04:31.833 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.834 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.834 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.834 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-12T03:04:31.834 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.834 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.834 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.834 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.834 INFO:teuthology.orchestra.run.smithi005.stdout: "self_heal": { 2024-09-12T03:04:31.834 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "self_heal", 2024-09-12T03:04:31.834 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.834 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.834 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.835 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-12T03:04:31.835 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.835 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.835 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.835 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-12T03:04:31.835 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.835 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.835 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.835 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.835 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-09-12T03:04:31.835 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-09-12T03:04:31.835 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-12T03:04:31.836 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.836 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.836 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-09-12T03:04:31.836 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.836 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.836 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.836 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to wake up and check device health", 2024-09-12T03:04:31.836 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.836 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.836 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.836 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.836 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_threshold": { 2024-09-12T03:04:31.836 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_threshold", 2024-09-12T03:04:31.837 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-12T03:04:31.837 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.837 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.837 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "7257600", 2024-09-12T03:04:31.837 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.837 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.837 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.837 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-12T03:04:31.837 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.837 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.837 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.837 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:31.837 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:31.837 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.838 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-12T03:04:31.838 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "diskprediction_local", 2024-09-12T03:04:31.838 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-12T03:04:31.838 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-12T03:04:31.838 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-12T03:04:31.838 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-12T03:04:31.838 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-12T03:04:31.838 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.838 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.838 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.838 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.838 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.838 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.839 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.839 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:31.839 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:31.839 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:31.839 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:31.839 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:31.839 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:31.839 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.839 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.839 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.839 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.839 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.839 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.839 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-12T03:04:31.840 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-12T03:04:31.840 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.840 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.840 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.840 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.840 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.840 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.840 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.840 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.840 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.840 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.840 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.840 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.841 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-12T03:04:31.841 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-12T03:04:31.841 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.841 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.841 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.841 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-12T03:04:31.841 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.841 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.841 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.841 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:31.841 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:31.841 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:31.841 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:31.841 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:31.842 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:31.842 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.842 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.842 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.842 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.842 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.842 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.842 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-12T03:04:31.842 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-12T03:04:31.842 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.842 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.842 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.842 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.842 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.843 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.843 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.843 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.843 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.843 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.843 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.843 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.843 INFO:teuthology.orchestra.run.smithi005.stdout: "predict_interval": { 2024-09-12T03:04:31.843 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "predict_interval", 2024-09-12T03:04:31.843 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.843 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.843 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.843 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "86400", 2024-09-12T03:04:31.843 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.843 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.844 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.844 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.844 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.844 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.844 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.844 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.844 INFO:teuthology.orchestra.run.smithi005.stdout: "predictor_model": { 2024-09-12T03:04:31.844 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "predictor_model", 2024-09-12T03:04:31.844 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.844 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.844 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.844 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "prophetstor", 2024-09-12T03:04:31.844 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.845 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.845 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.845 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.845 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.845 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.845 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.845 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.845 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-09-12T03:04:31.845 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-09-12T03:04:31.845 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.845 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.845 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.845 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-09-12T03:04:31.846 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.846 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.846 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.846 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.846 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.846 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.846 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.846 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:31.846 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:31.846 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.846 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-12T03:04:31.846 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "influx", 2024-09-12T03:04:31.846 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": false, 2024-09-12T03:04:31.846 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "influxdb python module not found", 2024-09-12T03:04:31.847 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-12T03:04:31.847 INFO:teuthology.orchestra.run.smithi005.stdout: "batch_size": { 2024-09-12T03:04:31.847 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "batch_size", 2024-09-12T03:04:31.847 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-12T03:04:31.847 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.847 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.847 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5000", 2024-09-12T03:04:31.847 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.847 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.847 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.847 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-12T03:04:31.847 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.847 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.847 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.848 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.848 INFO:teuthology.orchestra.run.smithi005.stdout: "database": { 2024-09-12T03:04:31.848 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "database", 2024-09-12T03:04:31.848 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.848 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.848 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.848 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "ceph", 2024-09-12T03:04:31.848 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.848 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.848 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.848 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-12T03:04:31.848 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.849 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.849 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.849 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.849 INFO:teuthology.orchestra.run.smithi005.stdout: "hostname": { 2024-09-12T03:04:31.849 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "hostname", 2024-09-12T03:04:31.849 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.849 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.849 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.849 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.849 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.849 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.849 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.849 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "InfluxDB server hostname", 2024-09-12T03:04:31.849 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.850 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.850 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.850 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.850 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-09-12T03:04:31.850 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-09-12T03:04:31.850 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-12T03:04:31.850 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.850 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.850 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "30", 2024-09-12T03:04:31.850 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "5", 2024-09-12T03:04:31.850 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.850 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.850 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-12T03:04:31.851 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.851 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.851 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.851 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.851 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-12T03:04:31.851 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-12T03:04:31.851 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.851 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.851 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.851 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.851 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.851 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.851 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.851 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:31.851 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:31.852 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:31.852 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:31.852 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:31.852 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:31.852 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.852 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.852 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.852 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.852 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.852 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.852 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-12T03:04:31.852 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-12T03:04:31.852 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.853 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.853 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.853 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.853 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.853 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.853 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.853 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.853 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.853 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.853 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.853 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.853 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-12T03:04:31.853 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-12T03:04:31.853 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.854 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.854 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.854 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-12T03:04:31.854 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.854 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.854 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.854 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:31.854 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:31.854 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:31.854 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:31.854 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:31.854 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:31.854 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.855 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.855 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.855 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.855 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.855 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.855 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-12T03:04:31.855 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-12T03:04:31.855 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.855 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.855 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.855 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.855 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.856 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.856 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.856 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.856 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.856 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.856 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.856 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.856 INFO:teuthology.orchestra.run.smithi005.stdout: "password": { 2024-09-12T03:04:31.856 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "password", 2024-09-12T03:04:31.856 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.856 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.856 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.856 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.857 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.857 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.857 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.857 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "password of InfluxDB server user", 2024-09-12T03:04:31.857 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.857 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.857 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.857 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.857 INFO:teuthology.orchestra.run.smithi005.stdout: "port": { 2024-09-12T03:04:31.857 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "port", 2024-09-12T03:04:31.857 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-12T03:04:31.857 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.857 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.858 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8086", 2024-09-12T03:04:31.858 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.858 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.858 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.858 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "InfluxDB server port", 2024-09-12T03:04:31.858 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.858 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.858 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.858 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.858 INFO:teuthology.orchestra.run.smithi005.stdout: "ssl": { 2024-09-12T03:04:31.858 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssl", 2024-09-12T03:04:31.858 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.858 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.858 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.859 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "false", 2024-09-12T03:04:31.859 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.859 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.859 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.859 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-12T03:04:31.859 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.859 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.859 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.859 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.859 INFO:teuthology.orchestra.run.smithi005.stdout: "threads": { 2024-09-12T03:04:31.859 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "threads", 2024-09-12T03:04:31.859 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-12T03:04:31.859 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.859 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.860 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2024-09-12T03:04:31.860 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2024-09-12T03:04:31.860 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "32", 2024-09-12T03:04:31.860 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.860 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-12T03:04:31.860 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.860 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.860 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.860 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.860 INFO:teuthology.orchestra.run.smithi005.stdout: "username": { 2024-09-12T03:04:31.860 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "username", 2024-09-12T03:04:31.860 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.860 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.861 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.861 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.861 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.861 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.861 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.861 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "username of InfluxDB server user", 2024-09-12T03:04:31.861 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.861 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.861 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.861 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.861 INFO:teuthology.orchestra.run.smithi005.stdout: "verify_ssl": { 2024-09-12T03:04:31.861 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "verify_ssl", 2024-09-12T03:04:31.861 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.861 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.862 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.862 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "true", 2024-09-12T03:04:31.862 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.862 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.862 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.862 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-12T03:04:31.862 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.862 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.862 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.862 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:31.862 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:31.862 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.862 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-12T03:04:31.863 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "insights", 2024-09-12T03:04:31.863 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-12T03:04:31.863 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-12T03:04:31.863 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-12T03:04:31.863 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-12T03:04:31.863 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-12T03:04:31.863 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.863 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.863 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.863 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.863 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.863 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.863 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.864 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:31.864 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:31.864 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:31.864 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:31.864 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:31.864 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:31.864 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.864 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.864 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.864 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.864 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.864 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.864 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-12T03:04:31.865 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-12T03:04:31.865 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.865 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.865 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.865 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.865 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.865 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.865 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.865 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.865 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.865 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.865 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.865 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.866 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-12T03:04:31.866 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-12T03:04:31.866 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.866 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.866 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.866 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-12T03:04:31.866 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.866 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.866 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.866 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:31.866 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:31.866 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:31.866 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:31.867 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:31.867 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:31.867 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.867 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.867 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.867 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.867 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.867 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.867 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-12T03:04:31.867 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-12T03:04:31.867 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.867 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.867 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.867 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.868 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.868 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.868 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.868 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.868 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.868 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.868 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.868 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:31.868 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:31.868 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.868 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-12T03:04:31.868 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "iostat", 2024-09-12T03:04:31.868 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-12T03:04:31.869 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-12T03:04:31.869 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-12T03:04:31.869 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-12T03:04:31.869 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-12T03:04:31.869 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.869 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.869 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.869 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.869 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.869 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.869 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.869 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:31.869 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:31.870 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:31.870 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:31.870 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:31.870 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:31.870 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.870 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.870 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.870 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.870 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.870 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.870 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-12T03:04:31.870 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-12T03:04:31.870 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.871 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.871 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.871 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.871 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.871 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.871 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.871 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.871 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.871 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.871 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.871 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.871 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-12T03:04:31.871 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-12T03:04:31.871 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.872 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.872 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.872 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-12T03:04:31.872 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.872 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.872 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.872 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:31.872 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:31.872 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:31.872 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:31.872 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:31.872 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:31.873 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.873 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.873 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.873 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.873 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.873 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.873 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-12T03:04:31.873 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-12T03:04:31.873 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.873 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.873 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.873 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.873 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.873 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.874 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.874 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.874 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.874 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.874 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.874 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:31.874 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:31.874 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.874 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-12T03:04:31.874 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "k8sevents", 2024-09-12T03:04:31.874 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-12T03:04:31.874 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-12T03:04:31.874 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-12T03:04:31.874 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_event_retention_days": { 2024-09-12T03:04:31.875 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ceph_event_retention_days", 2024-09-12T03:04:31.875 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-12T03:04:31.875 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.875 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.875 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "7", 2024-09-12T03:04:31.875 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.875 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.875 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.875 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-12T03:04:31.875 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.875 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.875 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.875 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.876 INFO:teuthology.orchestra.run.smithi005.stdout: "config_check_secs": { 2024-09-12T03:04:31.876 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "config_check_secs", 2024-09-12T03:04:31.876 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-12T03:04:31.876 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.876 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.876 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-09-12T03:04:31.876 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "10", 2024-09-12T03:04:31.876 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.876 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.876 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-12T03:04:31.876 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.876 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.876 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.877 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.877 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-12T03:04:31.877 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-12T03:04:31.877 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.877 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.877 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.877 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.877 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.877 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.877 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.877 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:31.877 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:31.877 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:31.878 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:31.878 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:31.878 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:31.878 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.878 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.878 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.878 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.878 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.878 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.878 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-12T03:04:31.878 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-12T03:04:31.878 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.878 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.879 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.879 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.879 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.879 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.879 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.879 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.879 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.879 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.879 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.879 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.879 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-12T03:04:31.879 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-12T03:04:31.879 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.879 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.880 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.880 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-12T03:04:31.880 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.880 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.880 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.880 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:31.880 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:31.880 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:31.880 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:31.880 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:31.880 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:31.880 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.881 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.881 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.881 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.881 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.881 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.881 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-12T03:04:31.881 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-12T03:04:31.881 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.881 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.881 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.881 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.881 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.881 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.881 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.882 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.882 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.882 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.882 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.882 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:31.882 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:31.882 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.882 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-12T03:04:31.882 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "localpool", 2024-09-12T03:04:31.882 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-12T03:04:31.882 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-12T03:04:31.882 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-12T03:04:31.882 INFO:teuthology.orchestra.run.smithi005.stdout: "failure_domain": { 2024-09-12T03:04:31.883 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "failure_domain", 2024-09-12T03:04:31.883 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.883 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.883 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.883 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "host", 2024-09-12T03:04:31.883 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.883 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.883 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.883 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "failure domain for any created local pool", 2024-09-12T03:04:31.883 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-12T03:04:31.883 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.883 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.883 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.884 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-12T03:04:31.884 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-12T03:04:31.884 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.884 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.884 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.884 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.884 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.884 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.884 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.884 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:31.884 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:31.884 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:31.884 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:31.885 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:31.885 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:31.885 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.885 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.885 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.885 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.885 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.885 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.885 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-12T03:04:31.885 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-12T03:04:31.885 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.885 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.885 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.885 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.886 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.886 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.886 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.886 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.886 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.886 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.886 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.886 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.886 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-12T03:04:31.886 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-12T03:04:31.886 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.886 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.886 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.887 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-12T03:04:31.887 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.887 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.887 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.887 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:31.887 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:31.887 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:31.887 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:31.887 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:31.887 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:31.887 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.887 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.887 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.887 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.888 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.888 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.888 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-12T03:04:31.888 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-12T03:04:31.888 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.888 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.888 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.888 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.888 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.888 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.888 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.888 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.888 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.889 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.889 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.889 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.889 INFO:teuthology.orchestra.run.smithi005.stdout: "min_size": { 2024-09-12T03:04:31.889 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "min_size", 2024-09-12T03:04:31.889 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-12T03:04:31.889 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.889 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.889 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.889 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.889 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.889 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.889 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "default min_size for any created local pool", 2024-09-12T03:04:31.890 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-12T03:04:31.890 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.890 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.890 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.890 INFO:teuthology.orchestra.run.smithi005.stdout: "num_rep": { 2024-09-12T03:04:31.890 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "num_rep", 2024-09-12T03:04:31.890 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-12T03:04:31.890 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.890 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.890 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "3", 2024-09-12T03:04:31.890 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.890 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.890 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.891 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "default replica count for any created local pool", 2024-09-12T03:04:31.891 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.891 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.891 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.891 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.891 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num": { 2024-09-12T03:04:31.891 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pg_num", 2024-09-12T03:04:31.891 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-12T03:04:31.891 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.891 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.891 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "128", 2024-09-12T03:04:31.892 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.892 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.892 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.892 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "default pg_num for any created local pool", 2024-09-12T03:04:31.892 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.892 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.892 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.892 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.892 INFO:teuthology.orchestra.run.smithi005.stdout: "prefix": { 2024-09-12T03:04:31.892 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "prefix", 2024-09-12T03:04:31.892 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.892 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.893 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.893 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.893 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.893 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.893 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.893 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "name prefix for any created local pool", 2024-09-12T03:04:31.893 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.893 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.893 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.893 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.893 INFO:teuthology.orchestra.run.smithi005.stdout: "subtree": { 2024-09-12T03:04:31.893 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "subtree", 2024-09-12T03:04:31.893 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.894 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.894 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.894 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "rack", 2024-09-12T03:04:31.894 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.894 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.894 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.894 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-12T03:04:31.894 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-12T03:04:31.894 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.894 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.894 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:31.894 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:31.894 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.895 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-12T03:04:31.895 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mds_autoscaler", 2024-09-12T03:04:31.895 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-12T03:04:31.895 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-12T03:04:31.895 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-12T03:04:31.895 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-12T03:04:31.895 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-12T03:04:31.895 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.895 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.895 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.895 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.895 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.895 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.895 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.896 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:31.896 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:31.896 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:31.896 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:31.896 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:31.896 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:31.896 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.896 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.896 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.896 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.896 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.896 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.896 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-12T03:04:31.897 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-12T03:04:31.897 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.897 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.897 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.897 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.897 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.897 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.897 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.897 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.897 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.897 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.897 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.897 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.898 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-12T03:04:31.898 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-12T03:04:31.898 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.898 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.898 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.898 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-12T03:04:31.898 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.898 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.898 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.898 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:31.898 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:31.898 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:31.898 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:31.899 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:31.899 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:31.899 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.899 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.899 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.899 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.899 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.899 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.899 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-12T03:04:31.899 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-12T03:04:31.899 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.899 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.899 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.900 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.900 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.900 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.900 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.900 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.900 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.900 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.900 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.900 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:31.900 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:31.900 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.900 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-12T03:04:31.900 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mirroring", 2024-09-12T03:04:31.901 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-12T03:04:31.901 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-12T03:04:31.901 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-12T03:04:31.901 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-12T03:04:31.901 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-12T03:04:31.901 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.901 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.901 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.901 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.901 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.901 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.901 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.901 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:31.902 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:31.902 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:31.902 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:31.902 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:31.902 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:31.902 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.902 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.902 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.902 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.902 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.902 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.902 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-12T03:04:31.902 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-12T03:04:31.902 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.902 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.903 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.903 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.903 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.903 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.903 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.903 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.903 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.903 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.903 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.903 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.903 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-12T03:04:31.903 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-12T03:04:31.903 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.903 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.904 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.904 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-12T03:04:31.904 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.904 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.904 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.904 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:31.904 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:31.904 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:31.904 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:31.904 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:31.904 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:31.904 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.904 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.904 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.904 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.905 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.905 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.905 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-12T03:04:31.905 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-12T03:04:31.905 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.905 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.905 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.905 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.905 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.905 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.905 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.905 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.906 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.906 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.906 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.906 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:31.906 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:31.906 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.906 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-12T03:04:31.906 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "nfs", 2024-09-12T03:04:31.906 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-12T03:04:31.906 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-12T03:04:31.906 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-12T03:04:31.906 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-12T03:04:31.906 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-12T03:04:31.907 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.907 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.907 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.907 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.907 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.907 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.907 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.907 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:31.907 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:31.907 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:31.907 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:31.907 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:31.908 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:31.908 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.908 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.908 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.908 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.908 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.908 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.908 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-12T03:04:31.908 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-12T03:04:31.908 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.908 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.909 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.909 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.909 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.909 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.909 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.909 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.909 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.909 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.909 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.909 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.909 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-12T03:04:31.909 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-12T03:04:31.909 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.910 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.910 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.910 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-12T03:04:31.910 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.910 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.910 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.910 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:31.910 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:31.910 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:31.910 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:31.910 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:31.910 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:31.910 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.911 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.911 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.911 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.911 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.911 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.911 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-12T03:04:31.911 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-12T03:04:31.911 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.911 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.911 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.911 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.911 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.911 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.912 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.912 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.912 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.912 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.912 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.912 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:31.912 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:31.912 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.912 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-12T03:04:31.912 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "orchestrator", 2024-09-12T03:04:31.912 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-12T03:04:31.912 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-12T03:04:31.912 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-12T03:04:31.913 INFO:teuthology.orchestra.run.smithi005.stdout: "fail_fs": { 2024-09-12T03:04:31.913 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "fail_fs", 2024-09-12T03:04:31.913 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.913 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.913 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.913 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.913 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.913 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.913 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.913 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-12T03:04:31.913 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.913 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.913 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.914 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.914 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-12T03:04:31.914 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-12T03:04:31.914 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.914 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.914 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.914 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.914 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.914 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.914 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.914 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:31.914 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:31.914 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:31.915 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:31.915 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:31.915 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:31.915 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.915 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.915 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.915 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.915 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.915 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.915 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-12T03:04:31.915 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-12T03:04:31.915 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.915 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.916 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.916 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.916 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.916 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.916 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.916 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.916 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.916 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.916 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.916 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.916 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-12T03:04:31.916 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-12T03:04:31.917 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.917 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.917 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.917 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-12T03:04:31.917 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.917 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.917 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.917 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:31.917 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:31.917 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:31.917 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:31.917 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:31.917 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:31.917 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.918 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.918 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.918 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.918 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.918 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.918 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-12T03:04:31.918 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-12T03:04:31.918 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.918 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.918 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.918 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.918 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.918 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.919 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.919 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.919 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.919 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.919 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.919 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.919 INFO:teuthology.orchestra.run.smithi005.stdout: "orchestrator": { 2024-09-12T03:04:31.919 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "orchestrator", 2024-09-12T03:04:31.919 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.919 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.919 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.919 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.919 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.920 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.920 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.920 INFO:teuthology.orchestra.run.smithi005.stdout: "cephadm", 2024-09-12T03:04:31.920 INFO:teuthology.orchestra.run.smithi005.stdout: "rook", 2024-09-12T03:04:31.920 INFO:teuthology.orchestra.run.smithi005.stdout: "test_orchestrator" 2024-09-12T03:04:31.920 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.920 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Orchestrator backend", 2024-09-12T03:04:31.920 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.920 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.920 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.920 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:31.920 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:31.920 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.920 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-12T03:04:31.921 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "osd_perf_query", 2024-09-12T03:04:31.921 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-12T03:04:31.921 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-12T03:04:31.921 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-12T03:04:31.921 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-12T03:04:31.921 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-12T03:04:31.921 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.921 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.921 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.921 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.921 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.921 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.922 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.922 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:31.922 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:31.922 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:31.922 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:31.922 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:31.922 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:31.922 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.922 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.922 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.922 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.922 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.922 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.923 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-12T03:04:31.923 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-12T03:04:31.923 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.923 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.923 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.923 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.923 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.923 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.923 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.923 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.923 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.923 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.923 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.923 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.924 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-12T03:04:31.924 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-12T03:04:31.924 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.924 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.924 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.924 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-12T03:04:31.924 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.924 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.924 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.924 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:31.924 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:31.924 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:31.925 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:31.925 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:31.925 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:31.925 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.925 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.925 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.925 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.925 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.925 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.925 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-12T03:04:31.925 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-12T03:04:31.925 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.925 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.926 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.926 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.926 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.926 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.926 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.926 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.926 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.926 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.926 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.926 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:31.926 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:31.926 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.926 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-12T03:04:31.927 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "osd_support", 2024-09-12T03:04:31.927 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-12T03:04:31.927 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-12T03:04:31.927 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-12T03:04:31.927 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-12T03:04:31.927 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-12T03:04:31.927 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.927 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.927 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.927 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.927 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.927 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.928 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.928 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:31.928 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:31.928 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:31.928 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:31.928 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:31.928 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:31.928 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.928 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.928 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.928 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.928 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.929 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.929 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-12T03:04:31.929 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-12T03:04:31.929 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.929 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.929 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.929 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.929 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.929 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.929 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.929 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.929 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.929 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.930 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.930 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.930 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-12T03:04:31.930 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-12T03:04:31.930 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.930 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.930 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.930 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-12T03:04:31.930 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.930 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.930 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.930 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:31.931 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:31.931 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:31.931 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:31.931 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:31.931 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:31.931 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.931 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.931 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.931 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.931 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.931 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.931 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-12T03:04:31.931 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-12T03:04:31.932 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.932 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.932 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.932 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.932 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.932 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.932 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.932 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.932 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.932 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.932 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.932 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:31.932 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:31.933 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.933 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-12T03:04:31.933 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pg_autoscaler", 2024-09-12T03:04:31.933 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-12T03:04:31.933 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-12T03:04:31.933 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-12T03:04:31.933 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-12T03:04:31.933 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-12T03:04:31.933 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.933 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.933 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.933 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.933 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.934 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.934 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.934 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:31.934 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:31.934 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:31.934 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:31.934 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:31.934 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:31.934 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.934 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.934 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.934 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.934 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.934 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.935 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-12T03:04:31.935 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-12T03:04:31.935 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.935 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.935 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.935 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.935 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.935 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.935 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.935 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.935 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.935 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.935 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.936 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.936 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-12T03:04:31.936 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-12T03:04:31.936 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.936 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.936 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.936 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-12T03:04:31.936 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.936 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.936 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.936 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:31.936 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:31.937 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:31.937 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:31.937 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:31.937 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:31.937 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.937 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.937 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.937 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.937 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.937 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.937 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-12T03:04:31.937 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-12T03:04:31.938 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.938 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.938 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.938 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.938 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.938 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.938 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.938 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.938 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.938 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.938 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.938 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.938 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-09-12T03:04:31.939 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-09-12T03:04:31.939 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-12T03:04:31.939 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.939 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.939 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-09-12T03:04:31.939 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.939 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.939 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.939 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.939 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.939 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.939 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.939 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.940 INFO:teuthology.orchestra.run.smithi005.stdout: "threshold": { 2024-09-12T03:04:31.940 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "threshold", 2024-09-12T03:04:31.940 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-09-12T03:04:31.940 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.940 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.940 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "3.0", 2024-09-12T03:04:31.940 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1.0", 2024-09-12T03:04:31.940 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.940 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.940 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "scaling threshold", 2024-09-12T03:04:31.940 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-12T03:04:31.940 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.940 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.941 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:31.941 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:31.941 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.941 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-12T03:04:31.941 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "progress", 2024-09-12T03:04:31.941 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-12T03:04:31.941 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-12T03:04:31.941 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-12T03:04:31.941 INFO:teuthology.orchestra.run.smithi005.stdout: "allow_pg_recovery_event": { 2024-09-12T03:04:31.941 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "allow_pg_recovery_event", 2024-09-12T03:04:31.941 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.941 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.941 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.942 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.942 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.942 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.942 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.942 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-12T03:04:31.942 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.942 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.942 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.942 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.942 INFO:teuthology.orchestra.run.smithi005.stdout: "enabled": { 2024-09-12T03:04:31.942 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enabled", 2024-09-12T03:04:31.942 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.942 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.942 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.943 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-12T03:04:31.943 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.943 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.943 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.943 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.943 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.943 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.943 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.943 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.943 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-12T03:04:31.943 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-12T03:04:31.943 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.943 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.944 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.944 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.944 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.944 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.944 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.944 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:31.944 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:31.944 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:31.944 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:31.944 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:31.944 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:31.944 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.944 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.945 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.945 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.945 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.945 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.945 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-12T03:04:31.945 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-12T03:04:31.945 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.945 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.945 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.945 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.945 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.945 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.945 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.945 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.946 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.946 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.946 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.946 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.946 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-12T03:04:31.946 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-12T03:04:31.946 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.946 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.946 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.946 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-12T03:04:31.946 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.946 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.946 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.947 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:31.947 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:31.947 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:31.947 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:31.947 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:31.947 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:31.947 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.947 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.947 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.947 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.947 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.947 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.947 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-12T03:04:31.947 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-12T03:04:31.947 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.948 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.948 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.948 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.948 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.948 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.948 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.948 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.948 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.948 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.948 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.948 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.948 INFO:teuthology.orchestra.run.smithi005.stdout: "max_completed_events": { 2024-09-12T03:04:31.948 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_completed_events", 2024-09-12T03:04:31.948 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-12T03:04:31.949 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.949 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.949 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "50", 2024-09-12T03:04:31.949 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.949 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.949 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.949 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "number of past completed events to remember", 2024-09-12T03:04:31.949 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.949 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.949 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.949 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.949 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-09-12T03:04:31.949 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-09-12T03:04:31.949 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-12T03:04:31.950 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.950 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.950 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2024-09-12T03:04:31.950 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.950 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.950 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.950 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how long the module is going to sleep", 2024-09-12T03:04:31.950 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.950 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.950 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.950 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:31.950 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:31.950 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.950 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-12T03:04:31.951 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "prometheus", 2024-09-12T03:04:31.951 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-12T03:04:31.951 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-12T03:04:31.951 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-12T03:04:31.951 INFO:teuthology.orchestra.run.smithi005.stdout: "cache": { 2024-09-12T03:04:31.951 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cache", 2024-09-12T03:04:31.951 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.951 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.951 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.951 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-12T03:04:31.951 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.951 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.952 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.952 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.952 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.952 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.952 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.952 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.952 INFO:teuthology.orchestra.run.smithi005.stdout: "exclude_perf_counters": { 2024-09-12T03:04:31.952 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "exclude_perf_counters", 2024-09-12T03:04:31.952 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.952 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.952 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.952 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-12T03:04:31.952 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.952 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.953 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.953 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-12T03:04:31.953 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-12T03:04:31.953 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.953 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.953 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.953 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-12T03:04:31.953 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-12T03:04:31.953 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.953 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.953 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.953 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.953 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.953 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.954 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.954 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:31.954 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:31.954 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:31.954 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:31.954 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:31.954 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:31.954 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.954 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.954 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.954 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.954 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.954 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.955 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-12T03:04:31.955 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-12T03:04:31.955 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.955 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.955 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.955 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.955 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.955 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.955 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.955 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.955 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.955 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.955 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.955 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.955 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-12T03:04:31.956 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-12T03:04:31.956 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.956 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.956 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.956 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-12T03:04:31.956 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.956 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.956 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.956 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:31.956 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:31.956 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:31.956 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:31.957 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:31.957 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:31.957 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.957 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.957 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.957 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.957 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.957 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.957 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-12T03:04:31.957 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-12T03:04:31.957 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.957 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.957 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.958 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.958 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.958 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.958 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.958 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.958 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.958 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.958 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.958 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.958 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_stats_pools": { 2024-09-12T03:04:31.958 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rbd_stats_pools", 2024-09-12T03:04:31.958 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.958 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.958 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.959 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.959 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.959 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.959 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.959 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.959 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.959 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.959 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.959 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.959 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-12T03:04:31.959 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-12T03:04:31.959 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-12T03:04:31.959 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.959 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.960 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "300", 2024-09-12T03:04:31.960 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.960 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.960 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.960 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.960 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.960 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.960 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.960 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.960 INFO:teuthology.orchestra.run.smithi005.stdout: "scrape_interval": { 2024-09-12T03:04:31.960 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "scrape_interval", 2024-09-12T03:04:31.960 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-09-12T03:04:31.961 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.961 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.961 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "15.0", 2024-09-12T03:04:31.961 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.961 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.961 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.961 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.961 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.961 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.961 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.961 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.961 INFO:teuthology.orchestra.run.smithi005.stdout: "server_addr": { 2024-09-12T03:04:31.962 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_addr", 2024-09-12T03:04:31.962 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.962 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.962 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.962 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "::", 2024-09-12T03:04:31.962 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.962 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.962 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.962 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-12T03:04:31.962 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.962 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.962 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.962 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.963 INFO:teuthology.orchestra.run.smithi005.stdout: "server_port": { 2024-09-12T03:04:31.963 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_port", 2024-09-12T03:04:31.963 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-12T03:04:31.963 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.963 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.963 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "9283", 2024-09-12T03:04:31.963 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.963 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.963 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.963 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-12T03:04:31.963 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.963 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.963 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.963 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.964 INFO:teuthology.orchestra.run.smithi005.stdout: "stale_cache_strategy": { 2024-09-12T03:04:31.964 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "stale_cache_strategy", 2024-09-12T03:04:31.964 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.964 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.964 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.964 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "log", 2024-09-12T03:04:31.964 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.964 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.964 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.964 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.964 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.964 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.964 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.965 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.965 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_behaviour": { 2024-09-12T03:04:31.965 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_behaviour", 2024-09-12T03:04:31.965 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.965 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.965 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.965 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "default", 2024-09-12T03:04:31.965 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.965 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.965 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.965 INFO:teuthology.orchestra.run.smithi005.stdout: "default", 2024-09-12T03:04:31.966 INFO:teuthology.orchestra.run.smithi005.stdout: "error" 2024-09-12T03:04:31.966 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.966 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.966 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.966 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.966 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.966 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.966 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_error_status_code": { 2024-09-12T03:04:31.966 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_error_status_code", 2024-09-12T03:04:31.966 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-12T03:04:31.966 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.966 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.966 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "500", 2024-09-12T03:04:31.967 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "400", 2024-09-12T03:04:31.967 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "599", 2024-09-12T03:04:31.967 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.967 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.967 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.967 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.967 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.967 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:31.967 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:31.967 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.967 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-12T03:04:31.967 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rbd_support", 2024-09-12T03:04:31.967 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-12T03:04:31.968 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-12T03:04:31.968 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-12T03:04:31.968 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-12T03:04:31.968 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-12T03:04:31.968 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.968 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.968 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.968 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.968 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.968 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.968 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.968 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:31.968 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:31.969 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:31.969 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:31.969 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:31.969 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:31.969 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.969 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.969 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.969 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.969 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.969 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.969 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-12T03:04:31.969 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-12T03:04:31.969 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.969 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.970 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.970 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.970 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.970 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.970 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.970 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.970 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.970 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.970 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.970 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.970 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-12T03:04:31.970 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-12T03:04:31.971 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.971 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.971 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.971 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-12T03:04:31.971 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.971 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.971 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.971 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:31.971 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:31.971 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:31.971 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:31.971 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:31.971 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:31.971 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.972 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.972 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.972 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.972 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.972 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.972 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-12T03:04:31.972 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-12T03:04:31.972 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.972 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.972 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.972 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.972 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.972 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.973 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.973 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.973 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.973 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.973 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.973 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.973 INFO:teuthology.orchestra.run.smithi005.stdout: "max_concurrent_snap_create": { 2024-09-12T03:04:31.973 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_concurrent_snap_create", 2024-09-12T03:04:31.973 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-12T03:04:31.973 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.973 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.973 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-09-12T03:04:31.973 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.973 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.974 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.974 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.975 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.975 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.976 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.976 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.976 INFO:teuthology.orchestra.run.smithi005.stdout: "mirror_snapshot_schedule": { 2024-09-12T03:04:31.976 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mirror_snapshot_schedule", 2024-09-12T03:04:31.976 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.976 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.976 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.976 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.976 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.976 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.976 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.976 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.976 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.977 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.977 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.977 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.977 INFO:teuthology.orchestra.run.smithi005.stdout: "trash_purge_schedule": { 2024-09-12T03:04:31.977 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "trash_purge_schedule", 2024-09-12T03:04:31.977 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.977 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.977 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.977 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.977 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.977 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.977 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.977 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.977 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.978 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.978 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.978 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:31.978 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:31.978 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.978 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-12T03:04:31.978 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "restful", 2024-09-12T03:04:31.978 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-12T03:04:31.978 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-12T03:04:31.978 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-12T03:04:31.978 INFO:teuthology.orchestra.run.smithi005.stdout: "enable_auth": { 2024-09-12T03:04:31.978 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enable_auth", 2024-09-12T03:04:31.978 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.978 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.979 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.979 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-12T03:04:31.979 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.979 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.979 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.979 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.979 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.979 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.979 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.979 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.979 INFO:teuthology.orchestra.run.smithi005.stdout: "key_file": { 2024-09-12T03:04:31.979 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "key_file", 2024-09-12T03:04:31.979 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.980 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.980 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.980 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.980 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.980 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.980 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.980 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.980 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.980 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.980 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.980 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.980 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-12T03:04:31.980 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-12T03:04:31.981 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.981 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.981 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.981 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.981 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.981 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.981 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.981 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:31.981 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:31.981 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:31.981 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:31.981 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:31.981 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:31.982 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.982 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.982 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.982 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.982 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.982 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.982 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-12T03:04:31.982 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-12T03:04:31.982 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.982 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.982 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.982 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.982 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.983 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.983 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.983 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.983 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.983 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.983 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.983 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.983 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-12T03:04:31.983 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-12T03:04:31.983 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.983 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.983 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.983 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-12T03:04:31.984 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.984 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.984 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.984 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:31.984 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:31.984 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:31.984 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:31.984 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:31.984 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:31.984 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.984 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.984 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.984 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.985 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.985 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.985 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-12T03:04:31.985 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-12T03:04:31.985 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.985 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.985 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.985 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.985 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.985 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.985 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.985 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.985 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.986 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.986 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.986 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.986 INFO:teuthology.orchestra.run.smithi005.stdout: "server_addr": { 2024-09-12T03:04:31.986 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_addr", 2024-09-12T03:04:31.986 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.986 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.986 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.986 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.986 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.986 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.986 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.986 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.987 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.987 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.987 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.987 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.987 INFO:teuthology.orchestra.run.smithi005.stdout: "server_port": { 2024-09-12T03:04:31.987 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_port", 2024-09-12T03:04:31.987 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.987 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.987 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:31.987 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.987 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.987 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.987 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.987 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.988 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.988 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.988 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.988 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:31.988 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:31.988 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.988 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-12T03:04:31.988 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rgw", 2024-09-12T03:04:31.988 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-12T03:04:31.988 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-12T03:04:31.988 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-12T03:04:31.988 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-12T03:04:31.988 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-12T03:04:31.989 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.989 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.989 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.989 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.989 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.989 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.989 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.989 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:31.989 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:31.989 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:31.989 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:31.989 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:31.989 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:31.990 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.990 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.990 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.990 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.990 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.990 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.990 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-12T03:04:31.990 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-12T03:04:31.990 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.990 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.990 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.990 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.990 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.990 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.991 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.991 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.991 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.991 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.991 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.991 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.991 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-12T03:04:31.991 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-12T03:04:31.991 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.991 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.991 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.991 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-12T03:04:31.991 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.991 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.991 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.992 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:31.992 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:31.992 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:31.992 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:31.992 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:31.992 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:31.992 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.992 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.992 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.992 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.992 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.992 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.992 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-12T03:04:31.993 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-12T03:04:31.993 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.993 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.993 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.993 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.993 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.993 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.993 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.993 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.993 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.993 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.993 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.994 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:31.994 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:31.994 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.994 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-12T03:04:31.994 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rook", 2024-09-12T03:04:31.994 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-12T03:04:31.994 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-12T03:04:31.994 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-12T03:04:31.994 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-12T03:04:31.994 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-12T03:04:31.994 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.994 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.994 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.994 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:31.995 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.995 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.995 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.995 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:31.995 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:31.995 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:31.995 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:31.995 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:31.995 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:31.995 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.995 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.995 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.995 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.996 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.996 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.996 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-12T03:04:31.996 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-12T03:04:31.996 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.996 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.996 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.996 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.996 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.996 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.996 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.996 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.997 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.997 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.997 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.997 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.997 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-12T03:04:31.997 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-12T03:04:31.997 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:31.997 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.997 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.997 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-12T03:04:31.997 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.997 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.997 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:31.998 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:31.998 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:31.998 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:31.998 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:31.998 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:31.998 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:31.998 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:31.998 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.998 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.998 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.998 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.998 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:31.998 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-12T03:04:31.999 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-12T03:04:31.999 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:31.999 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:31.999 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:31.999 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:31.999 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:31.999 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:31.999 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:31.999 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:31.999 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:31.999 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:31.999 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:31.999 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.000 INFO:teuthology.orchestra.run.smithi005.stdout: "storage_class": { 2024-09-12T03:04:32.000 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "storage_class", 2024-09-12T03:04:32.000 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.000 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.000 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.000 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "local", 2024-09-12T03:04:32.000 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.000 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.000 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.000 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-12T03:04:32.000 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.000 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.000 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.001 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:32.001 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:32.001 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.001 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-12T03:04:32.001 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "selftest", 2024-09-12T03:04:32.001 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-12T03:04:32.001 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-12T03:04:32.001 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-12T03:04:32.001 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-12T03:04:32.001 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-12T03:04:32.001 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.001 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.001 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.002 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:32.002 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.002 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.002 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:32.002 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:32.002 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:32.002 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:32.002 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:32.002 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:32.002 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:32.002 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:32.002 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.002 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.002 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.003 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.003 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.003 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-12T03:04:32.003 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-12T03:04:32.003 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.003 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.003 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.003 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:32.003 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.003 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.003 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.003 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.003 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.004 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.004 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.004 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.004 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-12T03:04:32.004 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-12T03:04:32.004 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.004 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.004 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.004 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-12T03:04:32.004 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.004 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.004 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:32.004 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:32.005 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:32.005 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:32.005 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:32.005 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:32.005 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:32.005 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:32.005 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.005 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.005 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.005 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.005 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.005 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-12T03:04:32.005 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-12T03:04:32.005 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.006 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.006 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.006 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:32.006 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.006 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.006 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.006 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.006 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.006 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.006 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.006 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.006 INFO:teuthology.orchestra.run.smithi005.stdout: "roption1": { 2024-09-12T03:04:32.006 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "roption1", 2024-09-12T03:04:32.007 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.007 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.007 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.007 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:32.007 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.007 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.007 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.007 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.007 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.007 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.007 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.007 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.007 INFO:teuthology.orchestra.run.smithi005.stdout: "roption2": { 2024-09-12T03:04:32.007 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "roption2", 2024-09-12T03:04:32.008 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.008 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.008 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.008 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "xyz", 2024-09-12T03:04:32.008 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.008 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.008 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.008 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.008 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.008 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.008 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.008 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.008 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption1": { 2024-09-12T03:04:32.009 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption1", 2024-09-12T03:04:32.009 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.009 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.009 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.009 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:32.009 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.009 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.009 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.009 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.009 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.009 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.009 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.009 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.009 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption2": { 2024-09-12T03:04:32.010 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption2", 2024-09-12T03:04:32.010 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-12T03:04:32.010 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.010 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.010 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:32.010 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.010 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.010 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.010 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.010 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.010 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.010 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.011 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.011 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption3": { 2024-09-12T03:04:32.011 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption3", 2024-09-12T03:04:32.011 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-09-12T03:04:32.011 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.011 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.011 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:32.011 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.011 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.011 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.011 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.011 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.011 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.012 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.012 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.012 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption4": { 2024-09-12T03:04:32.012 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption4", 2024-09-12T03:04:32.012 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.012 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.012 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.012 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:32.012 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.012 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.012 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.012 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.012 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.013 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.013 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.013 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.013 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption5": { 2024-09-12T03:04:32.013 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption5", 2024-09-12T03:04:32.013 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.013 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.013 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.013 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:32.013 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.013 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.013 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.013 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.014 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.014 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.014 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.014 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.014 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption6": { 2024-09-12T03:04:32.014 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption6", 2024-09-12T03:04:32.014 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.014 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.014 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.014 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-12T03:04:32.014 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.014 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.014 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.015 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.015 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.015 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.015 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.015 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.015 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption7": { 2024-09-12T03:04:32.015 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption7", 2024-09-12T03:04:32.015 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-12T03:04:32.015 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.015 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.015 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:32.015 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2024-09-12T03:04:32.015 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "42", 2024-09-12T03:04:32.016 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.016 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.016 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.016 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.016 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.016 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.016 INFO:teuthology.orchestra.run.smithi005.stdout: "testkey": { 2024-09-12T03:04:32.016 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "testkey", 2024-09-12T03:04:32.016 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.016 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.016 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.016 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:32.016 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.016 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.017 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.017 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.017 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.017 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.017 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.017 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.017 INFO:teuthology.orchestra.run.smithi005.stdout: "testlkey": { 2024-09-12T03:04:32.017 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "testlkey", 2024-09-12T03:04:32.017 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.017 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.017 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.017 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:32.017 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.017 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.018 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.018 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.018 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.018 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.018 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.018 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.018 INFO:teuthology.orchestra.run.smithi005.stdout: "testnewline": { 2024-09-12T03:04:32.018 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "testnewline", 2024-09-12T03:04:32.018 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.018 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.018 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.018 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:32.018 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.018 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.019 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.019 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.019 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.019 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.019 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.019 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:32.019 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:32.019 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.019 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-12T03:04:32.019 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "snap_schedule", 2024-09-12T03:04:32.019 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-12T03:04:32.019 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-12T03:04:32.019 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-12T03:04:32.019 INFO:teuthology.orchestra.run.smithi005.stdout: "allow_m_granularity": { 2024-09-12T03:04:32.020 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "allow_m_granularity", 2024-09-12T03:04:32.020 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.020 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.020 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.020 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:32.020 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.020 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.020 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.020 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "allow minute scheduled snapshots", 2024-09-12T03:04:32.020 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.020 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.020 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.020 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.021 INFO:teuthology.orchestra.run.smithi005.stdout: "dump_on_update": { 2024-09-12T03:04:32.021 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "dump_on_update", 2024-09-12T03:04:32.021 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.021 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.021 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.021 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:32.021 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.021 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.021 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.021 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "dump database to debug log on update", 2024-09-12T03:04:32.021 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.021 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.021 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.022 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.022 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-12T03:04:32.022 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-12T03:04:32.022 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.022 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.022 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.022 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:32.022 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.022 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.022 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:32.022 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:32.022 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:32.022 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:32.023 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:32.023 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:32.023 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:32.023 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:32.023 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.023 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.023 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.023 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.023 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.023 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-12T03:04:32.023 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-12T03:04:32.023 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.023 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.023 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.024 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:32.024 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.024 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.024 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.024 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.024 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.024 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.024 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.024 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.024 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-12T03:04:32.024 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-12T03:04:32.024 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.024 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.025 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.025 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-12T03:04:32.025 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.025 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.025 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:32.025 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:32.025 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:32.025 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:32.025 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:32.025 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:32.025 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:32.025 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:32.025 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.026 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.026 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.026 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.026 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.026 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-12T03:04:32.026 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-12T03:04:32.026 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.026 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.026 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.026 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:32.026 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.026 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.026 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.027 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.027 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.027 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.027 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.027 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:32.027 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:32.027 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.027 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-12T03:04:32.027 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "stats", 2024-09-12T03:04:32.027 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-12T03:04:32.027 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-12T03:04:32.027 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-12T03:04:32.027 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-12T03:04:32.027 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-12T03:04:32.028 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.028 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.028 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.028 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:32.028 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.028 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.028 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:32.028 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:32.028 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:32.028 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:32.028 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:32.028 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:32.028 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:32.029 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:32.029 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.029 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.029 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.029 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.029 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.029 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-12T03:04:32.029 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-12T03:04:32.029 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.029 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.029 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.029 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:32.029 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.030 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.030 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.030 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.030 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.030 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.030 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.030 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.030 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-12T03:04:32.030 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-12T03:04:32.030 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.030 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.030 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.030 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-12T03:04:32.031 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.031 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.031 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:32.031 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:32.031 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:32.031 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:32.031 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:32.031 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:32.031 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:32.031 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:32.031 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.031 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.031 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.031 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.032 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.032 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-12T03:04:32.032 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-12T03:04:32.032 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.032 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.032 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.032 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:32.032 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.032 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.032 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.032 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.032 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.032 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.033 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.033 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:32.033 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:32.033 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.033 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-12T03:04:32.033 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "status", 2024-09-12T03:04:32.033 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-12T03:04:32.033 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-12T03:04:32.033 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-12T03:04:32.033 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-12T03:04:32.033 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-12T03:04:32.033 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.033 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.034 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.034 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:32.034 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.034 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.034 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:32.034 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:32.034 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:32.034 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:32.034 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:32.034 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:32.034 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:32.034 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:32.034 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.035 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.035 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.035 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.035 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.035 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-12T03:04:32.035 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-12T03:04:32.035 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.035 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.035 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.035 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:32.035 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.035 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.035 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.036 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.036 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.036 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.036 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.036 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.036 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-12T03:04:32.036 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-12T03:04:32.036 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.036 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.036 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.036 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-12T03:04:32.036 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.036 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.037 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:32.037 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:32.037 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:32.037 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:32.037 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:32.037 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:32.037 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:32.037 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:32.037 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.037 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.037 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.037 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.038 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.038 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-12T03:04:32.038 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-12T03:04:32.038 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.038 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.038 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.038 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:32.038 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.038 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.038 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.038 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.038 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.038 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.039 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.039 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:32.039 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:32.039 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.039 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-12T03:04:32.039 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "telegraf", 2024-09-12T03:04:32.039 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-12T03:04:32.039 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-12T03:04:32.039 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-12T03:04:32.039 INFO:teuthology.orchestra.run.smithi005.stdout: "address": { 2024-09-12T03:04:32.039 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "address", 2024-09-12T03:04:32.039 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.040 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.040 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.040 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-12T03:04:32.040 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.040 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.040 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.040 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.040 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.040 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.040 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.040 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.040 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-09-12T03:04:32.040 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-09-12T03:04:32.041 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-12T03:04:32.041 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.041 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.041 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "15", 2024-09-12T03:04:32.041 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.041 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.041 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.041 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.041 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.041 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.041 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.041 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.041 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-12T03:04:32.042 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-12T03:04:32.042 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.042 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.042 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.042 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:32.042 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.042 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.042 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:32.042 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:32.042 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:32.042 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:32.042 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:32.043 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:32.043 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:32.043 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:32.043 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.043 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.043 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.043 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.043 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.043 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-12T03:04:32.043 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-12T03:04:32.043 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.043 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.043 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.044 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:32.044 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.044 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.044 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.044 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.044 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.044 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.044 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.044 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.044 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-12T03:04:32.044 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-12T03:04:32.044 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.044 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.045 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.045 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-12T03:04:32.045 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.045 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.045 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:32.045 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:32.045 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:32.045 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:32.045 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:32.045 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:32.045 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:32.045 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:32.045 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.046 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.046 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.046 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.046 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.046 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-12T03:04:32.046 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-12T03:04:32.046 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.046 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.046 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.046 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:32.046 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.046 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.046 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.046 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.047 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.047 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.047 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.047 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:32.047 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:32.047 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.047 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-12T03:04:32.047 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "telemetry", 2024-09-12T03:04:32.047 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-12T03:04:32.047 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-12T03:04:32.047 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-12T03:04:32.047 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_basic": { 2024-09-12T03:04:32.047 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_basic", 2024-09-12T03:04:32.048 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.048 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.048 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.048 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-12T03:04:32.048 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.048 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.048 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.048 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-12T03:04:32.048 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.048 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.048 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.048 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.048 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_crash": { 2024-09-12T03:04:32.049 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_crash", 2024-09-12T03:04:32.049 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.049 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.049 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.049 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-12T03:04:32.049 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.049 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.049 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.049 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-12T03:04:32.049 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.049 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.049 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.049 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.049 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_device": { 2024-09-12T03:04:32.050 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_device", 2024-09-12T03:04:32.050 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.050 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.050 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.050 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-12T03:04:32.050 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.050 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.050 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.050 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-12T03:04:32.050 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.050 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.050 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.050 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.050 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_ident": { 2024-09-12T03:04:32.051 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_ident", 2024-09-12T03:04:32.051 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.051 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.051 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.051 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:32.051 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.051 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.051 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.051 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-12T03:04:32.051 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.051 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.051 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.051 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.051 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_perf": { 2024-09-12T03:04:32.052 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_perf", 2024-09-12T03:04:32.052 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.052 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.052 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.052 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:32.052 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.052 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.052 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.052 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-12T03:04:32.052 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.052 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.052 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.052 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.053 INFO:teuthology.orchestra.run.smithi005.stdout: "contact": { 2024-09-12T03:04:32.053 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "contact", 2024-09-12T03:04:32.053 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.053 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.053 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.053 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:32.053 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.053 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.053 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.053 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.053 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.053 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.053 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.053 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.054 INFO:teuthology.orchestra.run.smithi005.stdout: "description": { 2024-09-12T03:04:32.054 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "description", 2024-09-12T03:04:32.054 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.054 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.054 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.054 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:32.054 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.054 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.054 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.054 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.054 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.054 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.054 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.055 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.055 INFO:teuthology.orchestra.run.smithi005.stdout: "device_url": { 2024-09-12T03:04:32.055 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "device_url", 2024-09-12T03:04:32.055 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.055 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.055 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.055 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-12T03:04:32.055 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.055 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.055 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.055 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.055 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.055 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.056 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.056 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.056 INFO:teuthology.orchestra.run.smithi005.stdout: "enabled": { 2024-09-12T03:04:32.056 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enabled", 2024-09-12T03:04:32.056 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.056 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.056 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.056 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:32.056 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.056 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.056 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.056 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.056 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.057 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.057 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.057 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.057 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-09-12T03:04:32.057 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-09-12T03:04:32.057 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-12T03:04:32.057 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.057 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.057 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "24", 2024-09-12T03:04:32.057 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "8", 2024-09-12T03:04:32.057 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.057 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.057 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.058 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.058 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.058 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.058 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.058 INFO:teuthology.orchestra.run.smithi005.stdout: "last_opt_revision": { 2024-09-12T03:04:32.058 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "last_opt_revision", 2024-09-12T03:04:32.058 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-12T03:04:32.058 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.058 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.058 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "1", 2024-09-12T03:04:32.058 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.058 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.058 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.058 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.059 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.059 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.059 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.059 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.059 INFO:teuthology.orchestra.run.smithi005.stdout: "leaderboard": { 2024-09-12T03:04:32.059 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "leaderboard", 2024-09-12T03:04:32.059 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.059 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.059 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.059 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:32.059 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.059 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.059 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.059 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.059 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.060 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.060 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.060 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.060 INFO:teuthology.orchestra.run.smithi005.stdout: "leaderboard_description": { 2024-09-12T03:04:32.060 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "leaderboard_description", 2024-09-12T03:04:32.060 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.060 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.060 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.060 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:32.060 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.060 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.060 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.060 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.060 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.061 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.061 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.061 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.061 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-12T03:04:32.061 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-12T03:04:32.061 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.061 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.061 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.061 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:32.061 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.061 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.061 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:32.061 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:32.062 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:32.062 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:32.062 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:32.062 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:32.062 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:32.062 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:32.062 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.062 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.062 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.062 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.062 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.062 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-12T03:04:32.062 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-12T03:04:32.062 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.063 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.063 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.063 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:32.063 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.063 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.063 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.063 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.063 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.063 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.063 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.063 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.063 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-12T03:04:32.063 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-12T03:04:32.063 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.063 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.064 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.064 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-12T03:04:32.064 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.064 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.064 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:32.064 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:32.064 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:32.064 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:32.064 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:32.064 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:32.064 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:32.064 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:32.064 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.065 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.065 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.065 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.065 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.065 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-12T03:04:32.065 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-12T03:04:32.065 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.065 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.065 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.065 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:32.065 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.065 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.065 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.065 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.066 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.066 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.066 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.066 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.066 INFO:teuthology.orchestra.run.smithi005.stdout: "organization": { 2024-09-12T03:04:32.066 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "organization", 2024-09-12T03:04:32.066 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.066 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.066 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.066 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:32.066 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.066 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.066 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.067 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.067 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.067 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.067 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.067 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.067 INFO:teuthology.orchestra.run.smithi005.stdout: "proxy": { 2024-09-12T03:04:32.067 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "proxy", 2024-09-12T03:04:32.067 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.067 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.067 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.067 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:32.067 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.067 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.067 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.067 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.068 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.068 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.068 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.068 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.068 INFO:teuthology.orchestra.run.smithi005.stdout: "url": { 2024-09-12T03:04:32.068 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "url", 2024-09-12T03:04:32.068 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.068 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.068 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.068 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-12T03:04:32.068 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.068 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.068 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.069 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.069 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.069 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.069 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.069 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:32.069 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:32.069 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.069 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-12T03:04:32.069 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "test_orchestrator", 2024-09-12T03:04:32.069 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-12T03:04:32.069 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-12T03:04:32.069 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-12T03:04:32.069 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-12T03:04:32.070 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-12T03:04:32.070 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.070 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.070 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.070 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:32.070 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.070 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.070 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:32.070 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:32.070 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:32.070 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:32.070 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:32.070 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:32.070 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:32.071 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:32.071 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.071 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.071 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.071 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.071 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.071 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-12T03:04:32.071 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-12T03:04:32.071 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.071 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.071 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.071 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:32.071 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.071 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.072 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.072 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.072 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.072 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.072 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.072 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.072 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-12T03:04:32.072 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-12T03:04:32.072 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.072 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.072 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.072 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-12T03:04:32.072 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.073 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.073 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:32.073 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:32.073 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:32.073 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:32.073 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:32.073 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:32.073 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:32.073 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:32.073 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.073 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.073 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.073 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.074 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.074 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-12T03:04:32.074 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-12T03:04:32.074 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.074 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.074 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.074 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:32.074 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.074 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.074 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.074 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.074 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.074 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.074 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.075 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:32.075 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:32.075 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.075 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-12T03:04:32.075 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "volumes", 2024-09-12T03:04:32.075 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-12T03:04:32.075 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-12T03:04:32.075 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-12T03:04:32.075 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-12T03:04:32.075 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-12T03:04:32.075 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.075 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.075 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.076 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:32.076 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.076 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.076 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:32.076 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:32.076 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:32.076 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:32.076 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:32.076 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:32.076 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:32.076 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:32.076 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.076 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.077 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.077 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.077 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.077 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-12T03:04:32.077 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-12T03:04:32.077 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.077 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.077 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.077 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:32.077 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.077 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.077 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.077 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.078 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.078 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.078 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.078 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.078 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-12T03:04:32.078 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-12T03:04:32.078 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.078 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.078 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.078 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-12T03:04:32.078 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.078 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.078 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:32.078 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:32.078 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:32.079 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:32.079 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:32.079 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:32.079 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:32.079 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:32.079 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.079 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.079 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.079 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.079 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.079 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-12T03:04:32.079 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-12T03:04:32.079 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.080 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.080 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.080 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:32.080 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.080 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.080 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.080 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.080 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.080 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.080 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.080 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.080 INFO:teuthology.orchestra.run.smithi005.stdout: "max_concurrent_clones": { 2024-09-12T03:04:32.080 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_concurrent_clones", 2024-09-12T03:04:32.080 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-12T03:04:32.081 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.081 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.081 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "4", 2024-09-12T03:04:32.081 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.081 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.081 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.081 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-12T03:04:32.081 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.081 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.081 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.081 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.081 INFO:teuthology.orchestra.run.smithi005.stdout: "periodic_async_work": { 2024-09-12T03:04:32.081 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "periodic_async_work", 2024-09-12T03:04:32.081 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.082 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.082 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.082 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:32.082 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.082 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.082 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.082 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Periodically check for async work", 2024-09-12T03:04:32.082 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.082 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.082 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.082 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.082 INFO:teuthology.orchestra.run.smithi005.stdout: "snapshot_clone_delay": { 2024-09-12T03:04:32.082 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "snapshot_clone_delay", 2024-09-12T03:04:32.082 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-12T03:04:32.083 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.083 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.083 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-09-12T03:04:32.083 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.083 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.083 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.083 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-12T03:04:32.083 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.083 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.083 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.083 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.083 INFO:teuthology.orchestra.run.smithi005.stdout: "snapshot_clone_no_wait": { 2024-09-12T03:04:32.083 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "snapshot_clone_no_wait", 2024-09-12T03:04:32.084 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.084 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.084 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.084 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-12T03:04:32.084 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.084 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.084 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.084 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-12T03:04:32.084 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.084 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.084 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.084 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:32.084 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:32.085 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.085 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-12T03:04:32.085 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix", 2024-09-12T03:04:32.085 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-12T03:04:32.085 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-12T03:04:32.085 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-12T03:04:32.085 INFO:teuthology.orchestra.run.smithi005.stdout: "discovery_interval": { 2024-09-12T03:04:32.085 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "discovery_interval", 2024-09-12T03:04:32.085 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-09-12T03:04:32.085 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.085 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.085 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "100", 2024-09-12T03:04:32.085 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.086 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.086 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.086 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.086 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.086 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.086 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.086 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.086 INFO:teuthology.orchestra.run.smithi005.stdout: "identifier": { 2024-09-12T03:04:32.086 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "identifier", 2024-09-12T03:04:32.086 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.086 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.086 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.086 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:32.086 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.087 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.087 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.087 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.087 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.087 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.087 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.087 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.087 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-09-12T03:04:32.087 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-09-12T03:04:32.087 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-12T03:04:32.087 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.087 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.087 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-09-12T03:04:32.087 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.088 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.088 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.088 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.088 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.088 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.088 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.088 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.088 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-12T03:04:32.088 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-12T03:04:32.088 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.088 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.088 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.088 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:32.089 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.089 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.089 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:32.089 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:32.089 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:32.089 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:32.089 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:32.089 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:32.089 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:32.089 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:32.089 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.089 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.089 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.090 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.090 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.090 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-12T03:04:32.090 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-12T03:04:32.090 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.090 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.090 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.090 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:32.090 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.090 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.090 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.090 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.090 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.090 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.091 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.091 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.091 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-12T03:04:32.091 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-12T03:04:32.091 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.091 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.091 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.091 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-12T03:04:32.091 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.091 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.091 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:32.091 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:32.091 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:32.091 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:32.092 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:32.092 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:32.092 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:32.092 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:32.092 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.092 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.092 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.092 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.092 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.092 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-12T03:04:32.092 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-12T03:04:32.092 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.092 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.093 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.093 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:32.093 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.093 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.093 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.093 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.093 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.093 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.093 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.093 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.093 INFO:teuthology.orchestra.run.smithi005.stdout: "zabbix_host": { 2024-09-12T03:04:32.093 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix_host", 2024-09-12T03:04:32.093 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.093 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.094 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.094 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:32.094 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.094 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.094 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.094 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.094 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.094 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.094 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.094 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.094 INFO:teuthology.orchestra.run.smithi005.stdout: "zabbix_port": { 2024-09-12T03:04:32.094 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix_port", 2024-09-12T03:04:32.094 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-12T03:04:32.094 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.095 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.095 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10051", 2024-09-12T03:04:32.095 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.095 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.095 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.095 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.095 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.095 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.095 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.095 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.095 INFO:teuthology.orchestra.run.smithi005.stdout: "zabbix_sender": { 2024-09-12T03:04:32.095 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix_sender", 2024-09-12T03:04:32.095 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.096 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.096 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.096 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-12T03:04:32.096 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.096 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.096 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.096 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.096 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.096 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.096 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.096 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:32.096 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:32.096 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:32.096 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-12T03:04:32.097 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:32.097 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:32.097 INFO:teuthology.orchestra.run.smithi005.stdout: "modules": [ 2024-09-12T03:04:32.097 INFO:teuthology.orchestra.run.smithi005.stdout: "cephadm", 2024-09-12T03:04:32.097 INFO:teuthology.orchestra.run.smithi005.stdout: "dashboard", 2024-09-12T03:04:32.097 INFO:teuthology.orchestra.run.smithi005.stdout: "iostat", 2024-09-12T03:04:32.097 INFO:teuthology.orchestra.run.smithi005.stdout: "nfs", 2024-09-12T03:04:32.097 INFO:teuthology.orchestra.run.smithi005.stdout: "restful" 2024-09-12T03:04:32.097 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:32.097 INFO:teuthology.orchestra.run.smithi005.stdout: "available_modules": [ 2024-09-12T03:04:32.097 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-12T03:04:32.097 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "alerts", 2024-09-12T03:04:32.097 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-12T03:04:32.097 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-12T03:04:32.098 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-12T03:04:32.098 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-09-12T03:04:32.098 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-09-12T03:04:32.098 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-12T03:04:32.098 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.098 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.098 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-09-12T03:04:32.098 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.098 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.098 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.098 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How frequently to reexamine health status", 2024-09-12T03:04:32.099 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.099 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.099 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.099 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.099 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-12T03:04:32.099 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-12T03:04:32.099 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.099 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.099 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.099 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:32.099 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.099 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.099 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:32.100 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:32.100 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:32.100 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:32.100 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:32.100 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:32.100 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:32.100 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:32.100 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.100 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.100 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.100 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.100 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.100 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-12T03:04:32.101 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-12T03:04:32.101 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.101 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.101 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.101 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:32.101 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.101 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.101 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.101 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.101 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.101 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.101 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.101 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.102 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-12T03:04:32.102 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-12T03:04:32.102 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.102 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.102 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.102 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-12T03:04:32.102 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.102 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.102 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:32.102 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:32.102 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:32.102 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:32.102 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:32.103 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:32.103 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:32.103 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:32.103 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.103 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.103 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.103 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.103 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.103 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-12T03:04:32.103 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-12T03:04:32.103 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.103 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.103 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.104 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:32.104 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.104 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.104 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.104 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.104 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.104 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.104 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.104 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.104 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_destination": { 2024-09-12T03:04:32.104 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_destination", 2024-09-12T03:04:32.104 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.104 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.105 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.105 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:32.105 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.105 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.105 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.105 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Email address to send alerts to", 2024-09-12T03:04:32.105 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.105 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.105 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.105 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.105 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_from_name": { 2024-09-12T03:04:32.105 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_from_name", 2024-09-12T03:04:32.106 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.106 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.106 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.106 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "Ceph", 2024-09-12T03:04:32.106 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.106 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.106 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.106 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Email From: name", 2024-09-12T03:04:32.106 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.106 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.106 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.106 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.106 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_host": { 2024-09-12T03:04:32.106 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_host", 2024-09-12T03:04:32.107 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.107 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.107 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.107 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:32.107 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.107 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.107 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.107 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SMTP server", 2024-09-12T03:04:32.107 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.107 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.107 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.107 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.107 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_password": { 2024-09-12T03:04:32.108 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_password", 2024-09-12T03:04:32.108 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.108 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.108 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.108 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:32.108 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.108 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.108 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.108 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Password to authenticate with", 2024-09-12T03:04:32.108 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.108 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.108 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.108 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.109 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_port": { 2024-09-12T03:04:32.109 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_port", 2024-09-12T03:04:32.109 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-12T03:04:32.109 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.109 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.109 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "465", 2024-09-12T03:04:32.109 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.109 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.109 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.109 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SMTP port", 2024-09-12T03:04:32.109 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.109 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.109 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.109 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.110 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_sender": { 2024-09-12T03:04:32.110 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_sender", 2024-09-12T03:04:32.110 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.110 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.110 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.110 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:32.110 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.110 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.110 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.110 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SMTP envelope sender", 2024-09-12T03:04:32.110 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.110 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.110 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.111 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.111 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_ssl": { 2024-09-12T03:04:32.111 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_ssl", 2024-09-12T03:04:32.111 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.111 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.111 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.111 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-12T03:04:32.111 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.111 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.111 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.111 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-12T03:04:32.111 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.111 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.111 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.112 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.112 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_user": { 2024-09-12T03:04:32.112 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_user", 2024-09-12T03:04:32.112 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.112 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.112 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.112 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:32.112 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.112 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.112 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.112 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "User to authenticate as", 2024-09-12T03:04:32.112 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.112 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.113 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.113 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:32.113 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:32.113 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.113 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-12T03:04:32.113 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "balancer", 2024-09-12T03:04:32.113 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-12T03:04:32.113 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-12T03:04:32.113 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-12T03:04:32.113 INFO:teuthology.orchestra.run.smithi005.stdout: "active": { 2024-09-12T03:04:32.113 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "active", 2024-09-12T03:04:32.113 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.113 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.113 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.114 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-12T03:04:32.114 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.114 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.114 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.114 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "automatically balance PGs across cluster", 2024-09-12T03:04:32.114 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.114 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.114 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.114 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.114 INFO:teuthology.orchestra.run.smithi005.stdout: "begin_time": { 2024-09-12T03:04:32.114 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "begin_time", 2024-09-12T03:04:32.114 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.115 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.115 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.115 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0000", 2024-09-12T03:04:32.115 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.115 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.115 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.115 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "beginning time of day to automatically balance", 2024-09-12T03:04:32.115 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-12T03:04:32.115 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.115 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.115 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.115 INFO:teuthology.orchestra.run.smithi005.stdout: "begin_weekday": { 2024-09-12T03:04:32.115 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "begin_weekday", 2024-09-12T03:04:32.115 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-09-12T03:04:32.116 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.116 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.116 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-09-12T03:04:32.116 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "0", 2024-09-12T03:04:32.116 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "6", 2024-09-12T03:04:32.116 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.116 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-12T03:04:32.116 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-12T03:04:32.116 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.116 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.116 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.116 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_compat_max_iterations": { 2024-09-12T03:04:32.117 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crush_compat_max_iterations", 2024-09-12T03:04:32.117 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-09-12T03:04:32.117 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.117 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.117 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "25", 2024-09-12T03:04:32.117 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2024-09-12T03:04:32.117 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "250", 2024-09-12T03:04:32.117 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.117 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-12T03:04:32.117 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.117 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.117 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.117 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.117 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_compat_metrics": { 2024-09-12T03:04:32.118 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crush_compat_metrics", 2024-09-12T03:04:32.118 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.118 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.118 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.118 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "pgs,objects,bytes", 2024-09-12T03:04:32.118 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.118 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.118 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.118 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-12T03:04:32.118 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-12T03:04:32.118 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.118 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.118 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.118 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_compat_step": { 2024-09-12T03:04:32.119 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crush_compat_step", 2024-09-12T03:04:32.119 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-09-12T03:04:32.119 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.119 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.119 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0.5", 2024-09-12T03:04:32.119 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "0.001", 2024-09-12T03:04:32.119 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "0.999", 2024-09-12T03:04:32.119 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.119 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "aggressiveness of optimization", 2024-09-12T03:04:32.119 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-12T03:04:32.119 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.119 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.119 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.119 INFO:teuthology.orchestra.run.smithi005.stdout: "end_time": { 2024-09-12T03:04:32.120 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "end_time", 2024-09-12T03:04:32.120 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.120 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.120 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.120 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "2359", 2024-09-12T03:04:32.120 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.120 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.120 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.120 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "ending time of day to automatically balance", 2024-09-12T03:04:32.120 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-12T03:04:32.120 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.120 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.120 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.121 INFO:teuthology.orchestra.run.smithi005.stdout: "end_weekday": { 2024-09-12T03:04:32.121 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "end_weekday", 2024-09-12T03:04:32.121 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-09-12T03:04:32.121 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.121 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.121 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-09-12T03:04:32.121 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "0", 2024-09-12T03:04:32.121 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "6", 2024-09-12T03:04:32.121 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.121 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-12T03:04:32.121 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-12T03:04:32.121 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.121 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.121 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.122 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-12T03:04:32.122 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-12T03:04:32.122 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.122 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.122 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.122 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:32.122 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.122 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.122 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:32.122 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:32.122 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:32.122 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:32.122 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:32.123 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:32.123 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:32.123 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:32.123 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.123 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.123 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.123 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.123 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.123 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-12T03:04:32.123 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-12T03:04:32.123 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.123 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.123 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.123 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:32.124 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.124 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.124 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.124 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.124 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.124 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.124 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.124 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.124 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-12T03:04:32.124 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-12T03:04:32.124 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.124 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.124 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.124 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-12T03:04:32.125 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.125 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.125 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:32.125 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:32.125 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:32.125 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:32.125 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:32.125 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:32.125 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:32.125 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:32.125 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.125 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.125 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.125 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.126 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.126 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-12T03:04:32.126 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-12T03:04:32.126 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.126 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.126 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.126 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:32.126 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.126 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.126 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.126 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.126 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.126 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.126 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.127 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.127 INFO:teuthology.orchestra.run.smithi005.stdout: "min_score": { 2024-09-12T03:04:32.127 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "min_score", 2024-09-12T03:04:32.127 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-09-12T03:04:32.127 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.127 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.127 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-09-12T03:04:32.127 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.127 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.127 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.127 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-12T03:04:32.127 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.127 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.127 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.128 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.128 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": { 2024-09-12T03:04:32.128 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mode", 2024-09-12T03:04:32.128 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.128 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.128 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.128 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "upmap", 2024-09-12T03:04:32.128 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.128 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.128 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:32.128 INFO:teuthology.orchestra.run.smithi005.stdout: "crush-compat", 2024-09-12T03:04:32.128 INFO:teuthology.orchestra.run.smithi005.stdout: "none", 2024-09-12T03:04:32.128 INFO:teuthology.orchestra.run.smithi005.stdout: "read", 2024-09-12T03:04:32.129 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap", 2024-09-12T03:04:32.129 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap-read" 2024-09-12T03:04:32.129 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:32.129 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Balancer mode", 2024-09-12T03:04:32.129 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.129 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.129 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.129 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.129 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_ids": { 2024-09-12T03:04:32.129 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pool_ids", 2024-09-12T03:04:32.129 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.129 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.129 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.129 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:32.130 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.130 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.130 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.130 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-12T03:04:32.130 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.130 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.130 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.130 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.130 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-09-12T03:04:32.130 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-09-12T03:04:32.130 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-12T03:04:32.130 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.130 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.130 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-09-12T03:04:32.131 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.131 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.131 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.131 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-12T03:04:32.131 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.131 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.131 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.131 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.131 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap_max_deviation": { 2024-09-12T03:04:32.131 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "upmap_max_deviation", 2024-09-12T03:04:32.131 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-12T03:04:32.131 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.131 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.132 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2024-09-12T03:04:32.132 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2024-09-12T03:04:32.132 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.132 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.132 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-12T03:04:32.132 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-12T03:04:32.132 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.132 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.132 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.132 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap_max_optimizations": { 2024-09-12T03:04:32.132 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "upmap_max_optimizations", 2024-09-12T03:04:32.132 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-09-12T03:04:32.132 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.133 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.133 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-09-12T03:04:32.133 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.133 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.133 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.133 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-12T03:04:32.133 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.133 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.133 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.133 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:32.133 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:32.133 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.133 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-12T03:04:32.133 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cephadm", 2024-09-12T03:04:32.134 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-12T03:04:32.134 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-12T03:04:32.134 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-12T03:04:32.134 INFO:teuthology.orchestra.run.smithi005.stdout: "agent_down_multiplier": { 2024-09-12T03:04:32.134 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "agent_down_multiplier", 2024-09-12T03:04:32.134 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-09-12T03:04:32.134 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.134 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.134 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "3.0", 2024-09-12T03:04:32.134 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.134 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.134 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.135 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-12T03:04:32.135 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.135 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.135 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.135 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.135 INFO:teuthology.orchestra.run.smithi005.stdout: "agent_refresh_rate": { 2024-09-12T03:04:32.135 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "agent_refresh_rate", 2024-09-12T03:04:32.135 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-12T03:04:32.135 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.135 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.135 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "20", 2024-09-12T03:04:32.135 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.135 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.136 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.136 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-12T03:04:32.136 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.136 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.136 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.136 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.136 INFO:teuthology.orchestra.run.smithi005.stdout: "agent_starting_port": { 2024-09-12T03:04:32.136 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "agent_starting_port", 2024-09-12T03:04:32.136 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-12T03:04:32.136 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.136 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.136 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "4721", 2024-09-12T03:04:32.136 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.137 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.137 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.137 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-12T03:04:32.137 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.137 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.137 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.137 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.137 INFO:teuthology.orchestra.run.smithi005.stdout: "allow_ptrace": { 2024-09-12T03:04:32.137 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "allow_ptrace", 2024-09-12T03:04:32.137 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.137 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.137 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.137 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:32.138 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.138 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.138 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.138 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-12T03:04:32.138 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-12T03:04:32.138 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.138 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.138 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.138 INFO:teuthology.orchestra.run.smithi005.stdout: "autotune_interval": { 2024-09-12T03:04:32.138 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "autotune_interval", 2024-09-12T03:04:32.138 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-12T03:04:32.138 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.138 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.139 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-09-12T03:04:32.139 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.139 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.139 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.139 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-12T03:04:32.139 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.139 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.139 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.139 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.139 INFO:teuthology.orchestra.run.smithi005.stdout: "autotune_memory_target_ratio": { 2024-09-12T03:04:32.139 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "autotune_memory_target_ratio", 2024-09-12T03:04:32.139 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-09-12T03:04:32.139 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.140 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.140 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0.7", 2024-09-12T03:04:32.140 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.140 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.140 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.140 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-12T03:04:32.140 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.140 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.140 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.140 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.140 INFO:teuthology.orchestra.run.smithi005.stdout: "cephadm_log_destination": { 2024-09-12T03:04:32.140 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cephadm_log_destination", 2024-09-12T03:04:32.140 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.141 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.141 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.141 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:32.141 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.141 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.141 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:32.141 INFO:teuthology.orchestra.run.smithi005.stdout: "file", 2024-09-12T03:04:32.141 INFO:teuthology.orchestra.run.smithi005.stdout: "file,syslog", 2024-09-12T03:04:32.141 INFO:teuthology.orchestra.run.smithi005.stdout: "syslog" 2024-09-12T03:04:32.141 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:32.141 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-12T03:04:32.141 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.141 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.142 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.142 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.142 INFO:teuthology.orchestra.run.smithi005.stdout: "cgroups_split": { 2024-09-12T03:04:32.142 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cgroups_split", 2024-09-12T03:04:32.142 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.142 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.142 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.142 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-12T03:04:32.142 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.142 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.142 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.142 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-12T03:04:32.142 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.142 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.143 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.143 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.143 INFO:teuthology.orchestra.run.smithi005.stdout: "config_checks_enabled": { 2024-09-12T03:04:32.143 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "config_checks_enabled", 2024-09-12T03:04:32.143 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.143 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.143 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.143 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:32.143 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.143 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.143 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.143 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-12T03:04:32.143 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.143 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.144 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.144 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.144 INFO:teuthology.orchestra.run.smithi005.stdout: "config_dashboard": { 2024-09-12T03:04:32.144 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "config_dashboard", 2024-09-12T03:04:32.144 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.144 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.144 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.144 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-12T03:04:32.144 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.144 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.144 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.144 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-12T03:04:32.144 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.145 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.145 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.145 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.145 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_alertmanager": { 2024-09-12T03:04:32.145 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_alertmanager", 2024-09-12T03:04:32.145 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.145 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.145 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.145 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-12T03:04:32.145 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.145 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.145 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.145 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2024-09-12T03:04:32.146 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.146 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.146 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.146 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.146 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_base": { 2024-09-12T03:04:32.146 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_base", 2024-09-12T03:04:32.146 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.146 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.146 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.146 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-12T03:04:32.146 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.146 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.146 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.147 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Container image name, without the tag", 2024-09-12T03:04:32.147 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.147 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.147 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.147 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.147 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_elasticsearch": { 2024-09-12T03:04:32.147 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_elasticsearch", 2024-09-12T03:04:32.147 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.147 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.147 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.147 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-12T03:04:32.147 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.147 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.148 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.148 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "elasticsearch container image", 2024-09-12T03:04:32.148 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.148 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.148 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.148 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.148 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_grafana": { 2024-09-12T03:04:32.148 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_grafana", 2024-09-12T03:04:32.148 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.148 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.148 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.148 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-12T03:04:32.148 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.149 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.149 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.149 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2024-09-12T03:04:32.149 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.149 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.149 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.149 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.149 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_haproxy": { 2024-09-12T03:04:32.149 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_haproxy", 2024-09-12T03:04:32.149 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.149 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.149 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.149 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-12T03:04:32.150 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.150 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.150 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.150 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "HAproxy container image", 2024-09-12T03:04:32.150 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.150 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.150 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.150 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.150 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_jaeger_agent": { 2024-09-12T03:04:32.150 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_jaeger_agent", 2024-09-12T03:04:32.150 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.150 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.150 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.150 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-12T03:04:32.151 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.151 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.151 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.151 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Jaeger agent container image", 2024-09-12T03:04:32.151 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.151 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.151 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.151 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.151 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_jaeger_collector": { 2024-09-12T03:04:32.151 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_jaeger_collector", 2024-09-12T03:04:32.151 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.151 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.151 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.151 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-12T03:04:32.151 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.152 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.152 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.152 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Jaeger collector container image", 2024-09-12T03:04:32.152 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.152 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.152 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.152 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.152 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_jaeger_query": { 2024-09-12T03:04:32.152 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_jaeger_query", 2024-09-12T03:04:32.152 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.152 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.152 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.152 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-12T03:04:32.153 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.153 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.153 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.153 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Jaeger query container image", 2024-09-12T03:04:32.153 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.153 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.153 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.153 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.153 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_keepalived": { 2024-09-12T03:04:32.153 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_keepalived", 2024-09-12T03:04:32.153 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.153 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.153 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.154 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-12T03:04:32.154 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.154 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.154 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.154 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Keepalived container image", 2024-09-12T03:04:32.154 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.154 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.154 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.154 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.154 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_loki": { 2024-09-12T03:04:32.154 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_loki", 2024-09-12T03:04:32.154 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.154 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.154 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.155 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-12T03:04:32.155 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.155 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.155 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.155 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Loki container image", 2024-09-12T03:04:32.155 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.155 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.155 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.155 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.155 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_node_exporter": { 2024-09-12T03:04:32.155 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_node_exporter", 2024-09-12T03:04:32.155 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.155 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.156 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.156 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-12T03:04:32.156 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.156 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.156 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.156 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2024-09-12T03:04:32.156 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.156 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.156 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.156 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.156 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_nvmeof": { 2024-09-12T03:04:32.156 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_nvmeof", 2024-09-12T03:04:32.156 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.157 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.157 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.157 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-12T03:04:32.157 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.157 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.157 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.157 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Nvme-of container image", 2024-09-12T03:04:32.157 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.157 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.157 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.157 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.157 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_prometheus": { 2024-09-12T03:04:32.157 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_prometheus", 2024-09-12T03:04:32.158 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.158 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.158 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.158 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-12T03:04:32.158 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.158 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.158 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.158 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2024-09-12T03:04:32.158 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.158 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.158 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.158 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.158 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_promtail": { 2024-09-12T03:04:32.159 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_promtail", 2024-09-12T03:04:32.159 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.159 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.159 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.159 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-12T03:04:32.159 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.159 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.159 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.159 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Promtail container image", 2024-09-12T03:04:32.159 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.159 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.159 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.159 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.160 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_samba": { 2024-09-12T03:04:32.160 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_samba", 2024-09-12T03:04:32.160 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.160 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.160 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.160 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-12T03:04:32.160 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.160 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.160 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.160 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Samba/SMB container image", 2024-09-12T03:04:32.160 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.160 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.160 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.160 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.161 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_snmp_gateway": { 2024-09-12T03:04:32.161 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_snmp_gateway", 2024-09-12T03:04:32.161 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.161 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.161 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.161 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-12T03:04:32.161 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.161 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.161 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.161 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SNMP Gateway container image", 2024-09-12T03:04:32.161 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.161 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.161 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.161 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.162 INFO:teuthology.orchestra.run.smithi005.stdout: "container_init": { 2024-09-12T03:04:32.162 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_init", 2024-09-12T03:04:32.162 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.162 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.162 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.162 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-12T03:04:32.162 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.162 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.162 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.162 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Run podman/docker with `--init`", 2024-09-12T03:04:32.162 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.162 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.162 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.163 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.163 INFO:teuthology.orchestra.run.smithi005.stdout: "daemon_cache_timeout": { 2024-09-12T03:04:32.163 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "daemon_cache_timeout", 2024-09-12T03:04:32.163 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-12T03:04:32.163 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.163 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.163 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-09-12T03:04:32.163 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.163 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.163 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.163 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-12T03:04:32.163 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.163 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.163 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.164 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.164 INFO:teuthology.orchestra.run.smithi005.stdout: "default_cephadm_command_timeout": { 2024-09-12T03:04:32.164 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "default_cephadm_command_timeout", 2024-09-12T03:04:32.164 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-12T03:04:32.164 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.164 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.164 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "900", 2024-09-12T03:04:32.164 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.164 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.164 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.164 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-12T03:04:32.164 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.164 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.164 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.165 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.165 INFO:teuthology.orchestra.run.smithi005.stdout: "default_registry": { 2024-09-12T03:04:32.165 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "default_registry", 2024-09-12T03:04:32.165 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.165 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.165 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.165 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io", 2024-09-12T03:04:32.165 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.165 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.165 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.165 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-12T03:04:32.165 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.165 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.166 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.166 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.166 INFO:teuthology.orchestra.run.smithi005.stdout: "device_cache_timeout": { 2024-09-12T03:04:32.166 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "device_cache_timeout", 2024-09-12T03:04:32.166 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-12T03:04:32.166 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.166 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.166 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "1800", 2024-09-12T03:04:32.166 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.166 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.166 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.166 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "seconds to cache device inventory", 2024-09-12T03:04:32.166 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.166 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.167 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.167 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.167 INFO:teuthology.orchestra.run.smithi005.stdout: "device_enhanced_scan": { 2024-09-12T03:04:32.167 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "device_enhanced_scan", 2024-09-12T03:04:32.167 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.167 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.167 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.167 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:32.167 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.167 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.167 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.167 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-12T03:04:32.167 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.167 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.168 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.168 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.168 INFO:teuthology.orchestra.run.smithi005.stdout: "facts_cache_timeout": { 2024-09-12T03:04:32.168 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "facts_cache_timeout", 2024-09-12T03:04:32.168 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-12T03:04:32.168 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.168 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.168 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-09-12T03:04:32.168 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.168 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.168 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.168 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "seconds to cache host facts data", 2024-09-12T03:04:32.168 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.169 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.169 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.169 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.169 INFO:teuthology.orchestra.run.smithi005.stdout: "grafana_dashboards_path": { 2024-09-12T03:04:32.169 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "grafana_dashboards_path", 2024-09-12T03:04:32.169 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.169 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.169 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.169 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-12T03:04:32.169 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.169 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.169 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.169 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-12T03:04:32.170 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.170 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.170 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.170 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.170 INFO:teuthology.orchestra.run.smithi005.stdout: "host_check_interval": { 2024-09-12T03:04:32.170 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "host_check_interval", 2024-09-12T03:04:32.170 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-12T03:04:32.170 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.170 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.170 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-09-12T03:04:32.170 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.170 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.170 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.171 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to perform a host check", 2024-09-12T03:04:32.171 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.171 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.171 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.171 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.171 INFO:teuthology.orchestra.run.smithi005.stdout: "hw_monitoring": { 2024-09-12T03:04:32.171 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "hw_monitoring", 2024-09-12T03:04:32.171 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.171 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.171 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.171 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:32.171 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.171 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.171 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.172 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-12T03:04:32.172 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.172 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.172 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.172 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.172 INFO:teuthology.orchestra.run.smithi005.stdout: "inventory_list_all": { 2024-09-12T03:04:32.172 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "inventory_list_all", 2024-09-12T03:04:32.172 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.172 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.172 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.172 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:32.172 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.172 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.173 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.173 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-12T03:04:32.173 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.173 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.173 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.173 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.173 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-12T03:04:32.173 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-12T03:04:32.173 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.173 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.173 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.173 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:32.173 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.173 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.174 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:32.174 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:32.174 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:32.174 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:32.174 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:32.174 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:32.174 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:32.174 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:32.174 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.174 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.174 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.174 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.174 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.174 INFO:teuthology.orchestra.run.smithi005.stdout: "log_refresh_metadata": { 2024-09-12T03:04:32.175 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_refresh_metadata", 2024-09-12T03:04:32.175 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.175 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.175 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.175 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:32.175 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.175 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.175 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.175 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-12T03:04:32.175 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.175 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.175 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.175 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.175 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-12T03:04:32.176 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-12T03:04:32.176 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.176 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.176 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.176 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-12T03:04:32.176 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.176 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.176 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.176 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-12T03:04:32.176 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.176 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.176 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.176 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.177 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-12T03:04:32.177 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-12T03:04:32.177 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.177 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.177 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.177 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-12T03:04:32.177 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.177 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.177 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:32.177 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:32.177 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:32.177 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:32.177 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:32.177 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:32.178 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:32.178 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:32.178 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.178 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.178 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.178 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.178 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.179 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-12T03:04:32.180 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-12T03:04:32.180 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.180 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.180 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.180 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:32.180 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.180 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.180 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.180 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.180 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.180 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.180 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.181 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.181 INFO:teuthology.orchestra.run.smithi005.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-12T03:04:32.181 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-12T03:04:32.181 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.181 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.181 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.181 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:32.181 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.181 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.181 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.181 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-12T03:04:32.181 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.181 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.182 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.182 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.182 INFO:teuthology.orchestra.run.smithi005.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-12T03:04:32.182 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-12T03:04:32.182 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.182 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.182 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.182 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "*", 2024-09-12T03:04:32.182 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.182 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.182 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.183 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-12T03:04:32.183 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.183 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.183 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.183 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.183 INFO:teuthology.orchestra.run.smithi005.stdout: "max_count_per_host": { 2024-09-12T03:04:32.183 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_count_per_host", 2024-09-12T03:04:32.183 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-12T03:04:32.183 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.183 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.183 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-09-12T03:04:32.183 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.183 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.183 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.183 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "max number of daemons per service per host", 2024-09-12T03:04:32.184 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.184 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.184 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.184 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.184 INFO:teuthology.orchestra.run.smithi005.stdout: "max_osd_draining_count": { 2024-09-12T03:04:32.184 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_osd_draining_count", 2024-09-12T03:04:32.184 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-12T03:04:32.184 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.184 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.184 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-09-12T03:04:32.184 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.184 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.185 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.185 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-12T03:04:32.185 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.185 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.185 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.185 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.185 INFO:teuthology.orchestra.run.smithi005.stdout: "migration_current": { 2024-09-12T03:04:32.185 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "migration_current", 2024-09-12T03:04:32.185 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-12T03:04:32.185 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.185 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.185 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:32.185 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.186 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.186 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.186 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "internal - do not modify", 2024-09-12T03:04:32.186 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.186 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.186 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.186 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.186 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": { 2024-09-12T03:04:32.186 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mode", 2024-09-12T03:04:32.186 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.186 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.186 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.186 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "root", 2024-09-12T03:04:32.187 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.187 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.187 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:32.187 INFO:teuthology.orchestra.run.smithi005.stdout: "cephadm-package", 2024-09-12T03:04:32.187 INFO:teuthology.orchestra.run.smithi005.stdout: "root" 2024-09-12T03:04:32.187 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:32.187 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "mode for remote execution of cephadm", 2024-09-12T03:04:32.187 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.187 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.187 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.187 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.187 INFO:teuthology.orchestra.run.smithi005.stdout: "oob_default_addr": { 2024-09-12T03:04:32.187 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "oob_default_addr", 2024-09-12T03:04:32.187 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.188 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.188 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.188 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "169.254.1.1", 2024-09-12T03:04:32.188 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.188 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.188 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.188 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-12T03:04:32.188 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.188 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.188 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.188 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.188 INFO:teuthology.orchestra.run.smithi005.stdout: "prometheus_alerts_path": { 2024-09-12T03:04:32.188 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "prometheus_alerts_path", 2024-09-12T03:04:32.189 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.189 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.189 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.189 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-12T03:04:32.189 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.189 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.189 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.189 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-12T03:04:32.189 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.189 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.189 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.189 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.189 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_insecure": { 2024-09-12T03:04:32.190 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_insecure", 2024-09-12T03:04:32.190 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.190 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.190 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.190 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:32.190 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.190 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.190 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.190 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-12T03:04:32.190 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.190 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.190 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.190 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.191 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_password": { 2024-09-12T03:04:32.191 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_password", 2024-09-12T03:04:32.191 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.191 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.191 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.191 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:32.191 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.191 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.191 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.191 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-12T03:04:32.191 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.191 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.191 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.192 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.192 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_url": { 2024-09-12T03:04:32.192 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_url", 2024-09-12T03:04:32.192 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.192 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.192 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.192 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:32.192 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.192 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.192 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.192 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-12T03:04:32.192 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.193 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.193 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.193 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.193 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_username": { 2024-09-12T03:04:32.193 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_username", 2024-09-12T03:04:32.193 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.193 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.193 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.193 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:32.193 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.193 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.193 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.194 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-12T03:04:32.194 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.194 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.194 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.194 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.194 INFO:teuthology.orchestra.run.smithi005.stdout: "secure_monitoring_stack": { 2024-09-12T03:04:32.194 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "secure_monitoring_stack", 2024-09-12T03:04:32.194 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.194 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.194 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.194 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:32.194 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.194 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.195 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.195 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-12T03:04:32.195 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.195 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.195 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.195 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.195 INFO:teuthology.orchestra.run.smithi005.stdout: "service_discovery_port": { 2024-09-12T03:04:32.195 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "service_discovery_port", 2024-09-12T03:04:32.195 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-12T03:04:32.195 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.195 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.195 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8765", 2024-09-12T03:04:32.196 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.196 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.196 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.196 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "cephadm service discovery port", 2024-09-12T03:04:32.196 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.196 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.196 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.196 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.196 INFO:teuthology.orchestra.run.smithi005.stdout: "ssh_config_file": { 2024-09-12T03:04:32.196 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssh_config_file", 2024-09-12T03:04:32.196 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.196 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.196 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.197 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:32.197 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.197 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.197 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.197 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-12T03:04:32.197 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.197 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.197 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.197 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.197 INFO:teuthology.orchestra.run.smithi005.stdout: "use_agent": { 2024-09-12T03:04:32.197 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "use_agent", 2024-09-12T03:04:32.197 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.197 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.198 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.198 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:32.198 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.198 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.198 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.198 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-12T03:04:32.198 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.198 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.198 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.198 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.198 INFO:teuthology.orchestra.run.smithi005.stdout: "use_repo_digest": { 2024-09-12T03:04:32.198 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "use_repo_digest", 2024-09-12T03:04:32.198 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.199 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.199 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.199 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-12T03:04:32.199 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.199 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.199 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.199 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-12T03:04:32.199 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.199 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.199 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.199 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.199 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_on_failed_host_check": { 2024-09-12T03:04:32.199 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_on_failed_host_check", 2024-09-12T03:04:32.200 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.200 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.200 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.200 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-12T03:04:32.200 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.200 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.200 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.200 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise a health warning if the host check fails", 2024-09-12T03:04:32.200 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.200 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.200 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.200 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.200 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_on_stray_daemons": { 2024-09-12T03:04:32.201 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_on_stray_daemons", 2024-09-12T03:04:32.201 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.201 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.201 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.201 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-12T03:04:32.201 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.201 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.201 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.201 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-12T03:04:32.201 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.201 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.201 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.201 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.202 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_on_stray_hosts": { 2024-09-12T03:04:32.202 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_on_stray_hosts", 2024-09-12T03:04:32.202 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.202 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.202 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.202 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-12T03:04:32.202 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.202 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.202 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.202 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-12T03:04:32.202 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.202 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.202 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.203 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:32.203 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:32.203 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.203 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-12T03:04:32.203 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crash", 2024-09-12T03:04:32.203 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-12T03:04:32.203 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-12T03:04:32.203 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-12T03:04:32.203 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-12T03:04:32.203 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-12T03:04:32.203 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.203 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.203 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.203 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:32.204 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.204 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.204 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:32.204 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:32.204 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:32.204 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:32.204 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:32.204 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:32.204 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:32.204 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:32.204 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.204 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.204 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.204 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.205 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.205 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-12T03:04:32.205 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-12T03:04:32.205 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.205 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.205 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.205 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:32.205 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.205 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.205 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.205 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.205 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.205 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.206 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.206 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.206 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-12T03:04:32.206 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-12T03:04:32.206 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.206 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.206 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.206 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-12T03:04:32.206 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.206 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.206 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:32.206 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:32.206 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:32.207 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:32.207 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:32.207 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:32.207 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:32.207 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:32.207 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.207 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.207 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.207 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.207 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.207 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-12T03:04:32.207 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-12T03:04:32.207 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.208 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.208 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.208 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:32.208 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.208 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.208 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.208 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.208 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.208 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.208 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.208 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.208 INFO:teuthology.orchestra.run.smithi005.stdout: "retain_interval": { 2024-09-12T03:04:32.208 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "retain_interval", 2024-09-12T03:04:32.208 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-12T03:04:32.209 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.209 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.209 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "31536000", 2024-09-12T03:04:32.209 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.209 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.209 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.209 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-12T03:04:32.209 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.209 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.209 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.209 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.209 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_recent_interval": { 2024-09-12T03:04:32.210 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_recent_interval", 2024-09-12T03:04:32.210 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-12T03:04:32.210 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.210 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.210 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "1209600", 2024-09-12T03:04:32.210 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.210 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.210 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.210 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-12T03:04:32.210 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.210 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.210 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.211 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:32.211 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:32.211 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.211 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-12T03:04:32.211 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "dashboard", 2024-09-12T03:04:32.211 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-12T03:04:32.211 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-12T03:04:32.211 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-12T03:04:32.211 INFO:teuthology.orchestra.run.smithi005.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-12T03:04:32.211 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-12T03:04:32.211 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-12T03:04:32.211 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.212 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.212 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-09-12T03:04:32.212 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.212 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.212 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.212 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.212 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.212 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.212 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.212 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.212 INFO:teuthology.orchestra.run.smithi005.stdout: "ALERTMANAGER_API_HOST": { 2024-09-12T03:04:32.213 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-12T03:04:32.213 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.213 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.213 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.213 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:32.213 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.213 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.213 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.213 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.213 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.213 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.213 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.213 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.214 INFO:teuthology.orchestra.run.smithi005.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-12T03:04:32.214 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-12T03:04:32.214 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.214 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.214 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.214 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-12T03:04:32.214 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.214 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.214 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.214 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.214 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.214 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.214 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.214 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.215 INFO:teuthology.orchestra.run.smithi005.stdout: "AUDIT_API_ENABLED": { 2024-09-12T03:04:32.215 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "AUDIT_API_ENABLED", 2024-09-12T03:04:32.215 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.215 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.215 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.215 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:32.215 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.215 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.215 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.215 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.215 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.215 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.215 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.216 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.216 INFO:teuthology.orchestra.run.smithi005.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-12T03:04:32.216 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-12T03:04:32.216 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.216 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.216 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.216 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-12T03:04:32.216 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.216 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.216 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.216 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.216 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.216 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.217 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.217 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.217 INFO:teuthology.orchestra.run.smithi005.stdout: "ENABLE_BROWSABLE_API": { 2024-09-12T03:04:32.217 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-12T03:04:32.217 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.217 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.217 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.217 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-12T03:04:32.217 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.217 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.217 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.217 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.218 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.218 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.218 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.218 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.218 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-12T03:04:32.218 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-12T03:04:32.218 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.218 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.218 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.218 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-12T03:04:32.218 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.218 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.218 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.219 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.219 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.219 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.219 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.219 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.219 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-12T03:04:32.219 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-12T03:04:32.219 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.219 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.219 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.219 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-12T03:04:32.219 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.219 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.220 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.220 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.220 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.220 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.220 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.220 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.220 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-12T03:04:32.220 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-12T03:04:32.220 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.220 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.220 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.220 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-12T03:04:32.220 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.221 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.221 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.221 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.221 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.221 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.221 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.221 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.221 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-12T03:04:32.221 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-12T03:04:32.221 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.221 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.221 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.221 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-12T03:04:32.221 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.222 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.222 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.222 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.222 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.222 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.222 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.222 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.222 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-12T03:04:32.222 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-12T03:04:32.222 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.222 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.222 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.223 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-12T03:04:32.223 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.223 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.223 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.223 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.223 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.223 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.223 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.223 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.223 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-12T03:04:32.223 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-12T03:04:32.223 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.223 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.223 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.224 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-12T03:04:32.224 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.224 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.224 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.224 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.224 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.224 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.224 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.224 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.224 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-12T03:04:32.224 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-12T03:04:32.224 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.224 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.225 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.225 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-12T03:04:32.225 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.225 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.225 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.225 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.225 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.225 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.225 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.225 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.225 INFO:teuthology.orchestra.run.smithi005.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-12T03:04:32.225 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-12T03:04:32.226 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.226 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.226 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.226 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:32.226 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.226 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.226 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.226 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.226 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.226 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.226 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.226 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.227 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_PASSWORD": { 2024-09-12T03:04:32.227 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-12T03:04:32.227 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.227 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.227 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.227 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "admin", 2024-09-12T03:04:32.227 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.227 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.227 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.227 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.227 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.227 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.227 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.227 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.228 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-12T03:04:32.228 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-12T03:04:32.228 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.228 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.228 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.228 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-12T03:04:32.228 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.228 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.228 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.228 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.228 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.228 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.229 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.229 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.229 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_URL": { 2024-09-12T03:04:32.229 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_URL", 2024-09-12T03:04:32.229 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.229 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.229 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.229 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:32.229 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.229 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.229 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.229 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.230 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.230 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.230 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.230 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.230 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_USERNAME": { 2024-09-12T03:04:32.230 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-12T03:04:32.230 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.230 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.230 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.230 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "admin", 2024-09-12T03:04:32.230 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.231 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.231 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.231 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.231 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.231 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.231 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.231 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.231 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-12T03:04:32.231 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-12T03:04:32.231 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.231 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.231 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.232 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:32.232 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.232 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.232 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.232 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.232 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.232 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.232 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.232 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.232 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-12T03:04:32.232 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-12T03:04:32.233 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.233 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.233 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.233 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:32.233 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.233 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.233 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.233 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.233 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.233 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.233 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.234 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.234 INFO:teuthology.orchestra.run.smithi005.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-12T03:04:32.234 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-12T03:04:32.234 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.234 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.234 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.234 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-12T03:04:32.234 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.234 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.234 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.234 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.235 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.235 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.235 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.235 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.235 INFO:teuthology.orchestra.run.smithi005.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-12T03:04:32.235 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-12T03:04:32.235 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.235 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.235 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.235 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:32.235 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.235 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.235 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.236 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.236 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.236 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.236 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.236 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.236 INFO:teuthology.orchestra.run.smithi005.stdout: "PROMETHEUS_API_HOST": { 2024-09-12T03:04:32.236 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-12T03:04:32.236 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.236 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.236 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.236 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:32.236 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.237 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.237 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.237 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.237 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.237 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.237 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.237 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.237 INFO:teuthology.orchestra.run.smithi005.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-12T03:04:32.237 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-12T03:04:32.237 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.237 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.237 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.238 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-12T03:04:32.238 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.238 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.238 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.238 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.238 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.238 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.238 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.238 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.238 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-12T03:04:32.238 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-12T03:04:32.238 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.238 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.238 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.239 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:32.239 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.239 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.239 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.239 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.239 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.239 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.239 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.239 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.239 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-12T03:04:32.239 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-12T03:04:32.239 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.239 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.240 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.240 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:32.240 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.240 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.240 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.240 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.240 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.240 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.240 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.240 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.240 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-12T03:04:32.240 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-12T03:04:32.240 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.241 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.241 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.241 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-12T03:04:32.241 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.241 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.241 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.241 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.241 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.241 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.241 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.241 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.241 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-12T03:04:32.241 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-12T03:04:32.242 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.242 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.242 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.242 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-12T03:04:32.242 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.242 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.242 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.242 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.242 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.242 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.242 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.242 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.242 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-12T03:04:32.243 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-12T03:04:32.243 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.243 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.243 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.243 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:32.243 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.243 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.243 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.243 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.243 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.243 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.243 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.243 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.244 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-12T03:04:32.244 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-12T03:04:32.244 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.244 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.244 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.244 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:32.244 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.244 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.244 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.244 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.244 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.244 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.245 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.245 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.245 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-12T03:04:32.245 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-12T03:04:32.245 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.245 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.245 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.245 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:32.245 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.245 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.245 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.245 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.245 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.246 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.246 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.246 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.246 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_ENABLED": { 2024-09-12T03:04:32.246 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-12T03:04:32.246 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.246 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.246 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.246 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-12T03:04:32.246 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.246 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.246 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.246 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.246 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.247 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.247 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.247 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.247 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-12T03:04:32.247 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-12T03:04:32.247 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.247 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.247 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.247 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-12T03:04:32.247 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.247 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.247 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.248 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.248 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.248 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.248 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.248 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.248 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-12T03:04:32.248 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-12T03:04:32.248 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-12T03:04:32.248 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.248 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.248 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-09-12T03:04:32.248 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.248 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.249 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.249 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.249 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.249 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.249 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.249 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.249 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-12T03:04:32.249 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-12T03:04:32.249 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-12T03:04:32.249 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.249 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.249 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8", 2024-09-12T03:04:32.249 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.250 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.250 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.250 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.250 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.250 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.250 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.250 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.250 INFO:teuthology.orchestra.run.smithi005.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-12T03:04:32.250 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-12T03:04:32.250 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-12T03:04:32.250 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.250 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.250 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "45", 2024-09-12T03:04:32.251 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.251 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.251 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.251 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.251 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.251 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.251 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.251 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.251 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_ACCESS_KEY": { 2024-09-12T03:04:32.251 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-12T03:04:32.251 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.251 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.251 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.252 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:32.252 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.252 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.252 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.252 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.252 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.252 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.252 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.252 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.252 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-12T03:04:32.252 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-12T03:04:32.252 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.252 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.253 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.253 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "admin", 2024-09-12T03:04:32.253 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.253 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.253 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.253 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.253 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.253 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.253 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.253 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.253 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_SECRET_KEY": { 2024-09-12T03:04:32.253 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-12T03:04:32.254 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.254 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.254 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.254 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:32.254 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.254 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.254 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.254 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.254 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.254 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.254 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.254 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.254 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_SSL_VERIFY": { 2024-09-12T03:04:32.254 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-12T03:04:32.255 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.255 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.255 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.255 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-12T03:04:32.255 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.255 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.255 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.255 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.255 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.255 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.255 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.255 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.255 INFO:teuthology.orchestra.run.smithi005.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-12T03:04:32.256 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-12T03:04:32.256 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-12T03:04:32.256 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.256 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.256 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-09-12T03:04:32.256 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.256 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.256 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.256 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.256 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.256 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.256 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.256 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.257 INFO:teuthology.orchestra.run.smithi005.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-12T03:04:32.257 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-12T03:04:32.257 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-12T03:04:32.257 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.257 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.257 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-09-12T03:04:32.257 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.257 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.257 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.257 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.257 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.257 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.257 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.257 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.258 INFO:teuthology.orchestra.run.smithi005.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-12T03:04:32.258 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-12T03:04:32.258 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-12T03:04:32.258 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.258 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.258 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2024-09-12T03:04:32.258 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.258 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.258 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.258 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.258 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.258 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.258 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.259 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.259 INFO:teuthology.orchestra.run.smithi005.stdout: "cross_origin_url": { 2024-09-12T03:04:32.259 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cross_origin_url", 2024-09-12T03:04:32.259 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.259 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.259 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.259 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:32.259 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.259 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.259 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.259 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.259 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.259 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.259 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.260 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.260 INFO:teuthology.orchestra.run.smithi005.stdout: "crt_file": { 2024-09-12T03:04:32.260 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crt_file", 2024-09-12T03:04:32.260 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.260 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.260 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.260 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:32.260 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.260 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.260 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.260 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.260 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.260 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.261 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.261 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.261 INFO:teuthology.orchestra.run.smithi005.stdout: "debug": { 2024-09-12T03:04:32.261 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "debug", 2024-09-12T03:04:32.261 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.261 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.261 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.261 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:32.261 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.261 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.261 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.261 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Enable/disable debug options", 2024-09-12T03:04:32.261 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.262 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.262 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.262 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.262 INFO:teuthology.orchestra.run.smithi005.stdout: "jwt_token_ttl": { 2024-09-12T03:04:32.262 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "jwt_token_ttl", 2024-09-12T03:04:32.262 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-12T03:04:32.262 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.262 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.262 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "28800", 2024-09-12T03:04:32.262 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.262 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.262 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.262 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.262 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.263 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.263 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.263 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.263 INFO:teuthology.orchestra.run.smithi005.stdout: "key_file": { 2024-09-12T03:04:32.263 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "key_file", 2024-09-12T03:04:32.263 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.263 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.263 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.263 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:32.263 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.263 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.263 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.263 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.264 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.264 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.264 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.264 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.264 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-12T03:04:32.264 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-12T03:04:32.264 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.264 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.264 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.264 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:32.264 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.264 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.265 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:32.265 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:32.265 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:32.265 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:32.265 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:32.265 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:32.265 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:32.265 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:32.265 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.265 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.265 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.265 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.265 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.266 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-12T03:04:32.266 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-12T03:04:32.266 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.266 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.266 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.266 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:32.266 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.266 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.266 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.266 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.266 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.266 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.266 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.267 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.267 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-12T03:04:32.267 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-12T03:04:32.267 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.267 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.267 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.267 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-12T03:04:32.267 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.267 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.267 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:32.267 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:32.267 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:32.267 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:32.267 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:32.268 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:32.268 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:32.268 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:32.268 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.268 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.268 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.268 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.268 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.268 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-12T03:04:32.268 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-12T03:04:32.268 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.268 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.268 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.269 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:32.269 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.269 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.269 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.269 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.269 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.269 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.269 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.269 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.269 INFO:teuthology.orchestra.run.smithi005.stdout: "motd": { 2024-09-12T03:04:32.269 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "motd", 2024-09-12T03:04:32.269 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.269 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.270 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.270 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:32.270 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.270 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.270 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.270 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "The message of the day", 2024-09-12T03:04:32.270 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.270 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.270 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.270 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.270 INFO:teuthology.orchestra.run.smithi005.stdout: "redirect_resolve_ip_addr": { 2024-09-12T03:04:32.270 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "redirect_resolve_ip_addr", 2024-09-12T03:04:32.270 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.270 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.270 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.271 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:32.271 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.271 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.271 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.271 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.271 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.271 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.271 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.271 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.271 INFO:teuthology.orchestra.run.smithi005.stdout: "server_addr": { 2024-09-12T03:04:32.271 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_addr", 2024-09-12T03:04:32.271 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.271 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.271 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.272 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "::", 2024-09-12T03:04:32.272 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.272 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.272 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.272 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.272 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.272 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.272 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.272 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.272 INFO:teuthology.orchestra.run.smithi005.stdout: "server_port": { 2024-09-12T03:04:32.272 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_port", 2024-09-12T03:04:32.272 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-12T03:04:32.273 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.273 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.273 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8080", 2024-09-12T03:04:32.273 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.273 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.273 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.273 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.273 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.273 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.273 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.273 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.273 INFO:teuthology.orchestra.run.smithi005.stdout: "ssl": { 2024-09-12T03:04:32.274 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssl", 2024-09-12T03:04:32.274 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.274 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.274 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.274 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-12T03:04:32.274 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.274 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.274 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.274 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.274 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.274 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.274 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.275 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.275 INFO:teuthology.orchestra.run.smithi005.stdout: "ssl_server_port": { 2024-09-12T03:04:32.275 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssl_server_port", 2024-09-12T03:04:32.275 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-12T03:04:32.275 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.275 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.275 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8443", 2024-09-12T03:04:32.275 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.275 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.275 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.275 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.275 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.276 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.276 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.276 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.276 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_behaviour": { 2024-09-12T03:04:32.276 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_behaviour", 2024-09-12T03:04:32.276 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.276 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.276 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.276 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "redirect", 2024-09-12T03:04:32.276 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.276 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.276 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:32.276 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:32.277 INFO:teuthology.orchestra.run.smithi005.stdout: "redirect" 2024-09-12T03:04:32.277 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:32.277 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.277 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.277 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.277 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.277 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.277 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_error_status_code": { 2024-09-12T03:04:32.277 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_error_status_code", 2024-09-12T03:04:32.277 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-12T03:04:32.277 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.277 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.277 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "500", 2024-09-12T03:04:32.278 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "400", 2024-09-12T03:04:32.278 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "599", 2024-09-12T03:04:32.278 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.278 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.278 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.278 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.278 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.278 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.278 INFO:teuthology.orchestra.run.smithi005.stdout: "url_prefix": { 2024-09-12T03:04:32.278 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "url_prefix", 2024-09-12T03:04:32.278 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.278 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.278 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.279 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:32.279 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.279 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.279 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.279 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.279 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.279 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.279 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.279 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:32.279 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:32.279 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.279 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-12T03:04:32.279 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "devicehealth", 2024-09-12T03:04:32.280 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-12T03:04:32.280 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-12T03:04:32.280 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-12T03:04:32.280 INFO:teuthology.orchestra.run.smithi005.stdout: "enable_monitoring": { 2024-09-12T03:04:32.280 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enable_monitoring", 2024-09-12T03:04:32.280 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.280 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.280 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.280 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-12T03:04:32.280 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.280 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.280 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.280 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "monitor device health metrics", 2024-09-12T03:04:32.280 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.281 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.281 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.281 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.281 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-12T03:04:32.281 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-12T03:04:32.281 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.281 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.281 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.281 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:32.281 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.281 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.281 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:32.281 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:32.281 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:32.282 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:32.282 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:32.282 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:32.282 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:32.282 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:32.282 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.282 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.282 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.282 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.282 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.282 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-12T03:04:32.282 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-12T03:04:32.282 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.283 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.283 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.283 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:32.283 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.283 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.283 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.283 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.283 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.283 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.283 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.283 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.283 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-12T03:04:32.283 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-12T03:04:32.284 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.284 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.284 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.284 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-12T03:04:32.284 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.284 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.284 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:32.284 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:32.284 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:32.284 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:32.284 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:32.284 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:32.284 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:32.285 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:32.285 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.285 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.285 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.285 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.285 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.285 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-12T03:04:32.285 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-12T03:04:32.285 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.285 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.285 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.285 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:32.285 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.286 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.286 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.286 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.286 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.286 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.286 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.286 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.286 INFO:teuthology.orchestra.run.smithi005.stdout: "mark_out_threshold": { 2024-09-12T03:04:32.286 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mark_out_threshold", 2024-09-12T03:04:32.286 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-12T03:04:32.286 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.286 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.286 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "2419200", 2024-09-12T03:04:32.286 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.287 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.287 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.287 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-12T03:04:32.287 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.287 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.287 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.287 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.287 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_name": { 2024-09-12T03:04:32.287 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pool_name", 2024-09-12T03:04:32.287 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.287 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.288 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.288 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "device_health_metrics", 2024-09-12T03:04:32.288 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.288 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.288 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.288 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-12T03:04:32.288 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.288 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.288 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.288 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.288 INFO:teuthology.orchestra.run.smithi005.stdout: "retention_period": { 2024-09-12T03:04:32.288 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "retention_period", 2024-09-12T03:04:32.289 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-12T03:04:32.289 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.289 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.289 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "15552000", 2024-09-12T03:04:32.289 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.289 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.289 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.289 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how long to retain device health metrics", 2024-09-12T03:04:32.289 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.289 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.289 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.289 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.290 INFO:teuthology.orchestra.run.smithi005.stdout: "scrape_frequency": { 2024-09-12T03:04:32.290 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "scrape_frequency", 2024-09-12T03:04:32.290 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-12T03:04:32.290 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.290 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.290 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "86400", 2024-09-12T03:04:32.290 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.290 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.290 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.290 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-12T03:04:32.290 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.290 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.290 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.291 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.291 INFO:teuthology.orchestra.run.smithi005.stdout: "self_heal": { 2024-09-12T03:04:32.291 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "self_heal", 2024-09-12T03:04:32.291 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.291 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.291 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.291 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-12T03:04:32.291 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.291 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.291 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.291 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-12T03:04:32.291 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.291 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.292 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.292 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.292 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-09-12T03:04:32.292 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-09-12T03:04:32.292 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-12T03:04:32.292 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.292 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.292 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-09-12T03:04:32.292 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.292 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.292 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.292 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to wake up and check device health", 2024-09-12T03:04:32.292 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.293 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.293 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.293 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.293 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_threshold": { 2024-09-12T03:04:32.293 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_threshold", 2024-09-12T03:04:32.293 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-12T03:04:32.293 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.293 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.293 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "7257600", 2024-09-12T03:04:32.293 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.293 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.293 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.293 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-12T03:04:32.294 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.294 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.294 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.294 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:32.294 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:32.294 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.294 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-12T03:04:32.294 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "diskprediction_local", 2024-09-12T03:04:32.294 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-12T03:04:32.294 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-12T03:04:32.294 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-12T03:04:32.294 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-12T03:04:32.295 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-12T03:04:32.295 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.295 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.295 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.295 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:32.295 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.295 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.295 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:32.295 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:32.295 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:32.295 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:32.295 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:32.295 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:32.295 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:32.296 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:32.296 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.296 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.296 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.296 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.296 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.296 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-12T03:04:32.296 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-12T03:04:32.296 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.296 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.296 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.296 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:32.296 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.297 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.297 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.297 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.297 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.297 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.297 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.297 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.297 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-12T03:04:32.297 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-12T03:04:32.297 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.297 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.297 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.297 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-12T03:04:32.298 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.298 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.298 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:32.298 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:32.298 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:32.298 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:32.298 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:32.298 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:32.298 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:32.298 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:32.298 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.298 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.298 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.298 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.299 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.299 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-12T03:04:32.299 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-12T03:04:32.299 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.299 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.299 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.299 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:32.299 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.299 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.299 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.299 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.299 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.299 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.299 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.300 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.300 INFO:teuthology.orchestra.run.smithi005.stdout: "predict_interval": { 2024-09-12T03:04:32.300 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "predict_interval", 2024-09-12T03:04:32.300 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.300 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.300 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.300 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "86400", 2024-09-12T03:04:32.300 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.300 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.300 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.301 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.301 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.301 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.301 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.301 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.301 INFO:teuthology.orchestra.run.smithi005.stdout: "predictor_model": { 2024-09-12T03:04:32.301 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "predictor_model", 2024-09-12T03:04:32.301 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.301 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.301 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.301 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "prophetstor", 2024-09-12T03:04:32.301 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.301 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.302 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.302 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.302 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.302 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.302 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.302 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.302 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-09-12T03:04:32.302 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-09-12T03:04:32.302 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.302 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.302 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.302 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2024-09-12T03:04:32.302 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.302 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.303 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.303 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.303 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.303 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.303 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.303 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:32.303 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:32.303 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.303 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-12T03:04:32.303 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "influx", 2024-09-12T03:04:32.303 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": false, 2024-09-12T03:04:32.303 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "influxdb python module not found", 2024-09-12T03:04:32.303 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-12T03:04:32.303 INFO:teuthology.orchestra.run.smithi005.stdout: "batch_size": { 2024-09-12T03:04:32.304 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "batch_size", 2024-09-12T03:04:32.304 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-12T03:04:32.304 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.304 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.304 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5000", 2024-09-12T03:04:32.304 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.304 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.304 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.304 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-12T03:04:32.305 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.305 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.305 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.305 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.305 INFO:teuthology.orchestra.run.smithi005.stdout: "database": { 2024-09-12T03:04:32.305 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "database", 2024-09-12T03:04:32.305 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.305 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.305 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.305 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "ceph", 2024-09-12T03:04:32.305 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.305 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.305 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.306 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-12T03:04:32.306 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.306 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.306 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.306 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.306 INFO:teuthology.orchestra.run.smithi005.stdout: "hostname": { 2024-09-12T03:04:32.306 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "hostname", 2024-09-12T03:04:32.306 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.306 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.306 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.306 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:32.306 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.307 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.307 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.307 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "InfluxDB server hostname", 2024-09-12T03:04:32.307 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.307 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.307 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.307 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.307 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-09-12T03:04:32.307 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-09-12T03:04:32.307 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-12T03:04:32.307 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.307 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.307 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "30", 2024-09-12T03:04:32.308 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "5", 2024-09-12T03:04:32.308 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.308 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.308 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-12T03:04:32.308 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.308 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.308 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.308 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.308 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-12T03:04:32.308 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-12T03:04:32.308 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.308 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.308 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.309 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:32.309 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.309 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.309 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:32.309 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:32.309 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:32.309 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:32.309 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:32.309 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:32.309 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:32.309 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:32.309 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.309 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.310 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.310 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.310 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.310 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-12T03:04:32.310 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-12T03:04:32.310 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.310 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.310 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.310 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:32.310 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.310 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.310 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.310 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.311 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.311 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.311 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.311 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.311 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-12T03:04:32.311 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-12T03:04:32.311 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.311 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.311 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.311 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-12T03:04:32.311 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.311 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.311 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:32.311 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:32.312 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:32.312 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:32.312 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:32.312 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:32.312 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:32.312 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:32.312 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.312 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.312 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.312 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.312 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.312 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-12T03:04:32.312 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-12T03:04:32.313 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.313 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.313 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.313 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:32.313 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.313 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.313 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.313 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.313 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.313 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.313 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.313 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.313 INFO:teuthology.orchestra.run.smithi005.stdout: "password": { 2024-09-12T03:04:32.313 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "password", 2024-09-12T03:04:32.314 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.314 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.314 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.314 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:32.314 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.314 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.314 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.314 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "password of InfluxDB server user", 2024-09-12T03:04:32.314 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.314 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.314 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.314 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.314 INFO:teuthology.orchestra.run.smithi005.stdout: "port": { 2024-09-12T03:04:32.314 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "port", 2024-09-12T03:04:32.315 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-12T03:04:32.315 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.315 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.315 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8086", 2024-09-12T03:04:32.315 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.315 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.315 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.315 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "InfluxDB server port", 2024-09-12T03:04:32.315 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.315 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.315 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.315 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.315 INFO:teuthology.orchestra.run.smithi005.stdout: "ssl": { 2024-09-12T03:04:32.316 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssl", 2024-09-12T03:04:32.316 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.316 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.316 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.316 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "false", 2024-09-12T03:04:32.316 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.316 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.316 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.316 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-12T03:04:32.316 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.316 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.316 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.316 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.316 INFO:teuthology.orchestra.run.smithi005.stdout: "threads": { 2024-09-12T03:04:32.317 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "threads", 2024-09-12T03:04:32.317 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-12T03:04:32.317 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.317 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.317 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2024-09-12T03:04:32.317 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2024-09-12T03:04:32.317 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "32", 2024-09-12T03:04:32.317 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.317 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-12T03:04:32.317 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.317 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.317 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.317 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.318 INFO:teuthology.orchestra.run.smithi005.stdout: "username": { 2024-09-12T03:04:32.318 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "username", 2024-09-12T03:04:32.318 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.318 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.318 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.318 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:32.318 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.318 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.318 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.318 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "username of InfluxDB server user", 2024-09-12T03:04:32.318 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.318 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.319 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.319 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.319 INFO:teuthology.orchestra.run.smithi005.stdout: "verify_ssl": { 2024-09-12T03:04:32.319 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "verify_ssl", 2024-09-12T03:04:32.319 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.319 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.319 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.319 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "true", 2024-09-12T03:04:32.319 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.319 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.319 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.319 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-12T03:04:32.319 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.320 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.320 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.320 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:32.320 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:32.320 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.320 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-12T03:04:32.320 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "insights", 2024-09-12T03:04:32.320 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-12T03:04:32.320 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-12T03:04:32.320 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-12T03:04:32.320 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-12T03:04:32.320 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-12T03:04:32.320 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.321 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.321 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.321 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:32.321 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.321 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.321 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:32.321 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:32.321 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:32.321 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:32.321 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:32.321 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:32.321 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:32.321 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:32.322 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.322 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.322 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.322 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.322 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.322 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-12T03:04:32.322 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-12T03:04:32.322 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.322 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.322 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.322 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:32.322 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.322 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.322 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.323 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.323 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.323 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.323 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.323 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.323 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-12T03:04:32.323 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-12T03:04:32.323 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.323 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.323 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.323 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-12T03:04:32.323 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.324 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.324 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:32.324 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:32.324 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:32.324 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:32.324 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:32.324 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:32.324 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:32.324 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:32.324 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.324 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.324 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.324 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.325 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.325 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-12T03:04:32.325 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-12T03:04:32.325 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.325 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.325 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.325 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:32.325 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.325 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.325 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.325 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.325 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.326 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.326 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.326 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:32.326 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:32.326 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.326 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-12T03:04:32.326 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "iostat", 2024-09-12T03:04:32.326 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-12T03:04:32.326 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-12T03:04:32.326 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-12T03:04:32.326 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-12T03:04:32.326 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-12T03:04:32.326 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.326 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.327 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.327 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:32.327 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.327 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.327 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:32.327 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:32.327 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:32.327 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:32.327 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:32.327 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:32.327 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:32.327 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:32.327 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.327 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.328 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.328 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.328 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.328 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-12T03:04:32.328 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-12T03:04:32.328 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.328 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.328 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.328 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:32.328 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.328 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.328 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.328 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.329 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.329 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.329 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.329 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.329 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-12T03:04:32.329 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-12T03:04:32.329 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.329 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.329 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.329 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-12T03:04:32.329 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.329 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.329 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:32.329 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:32.330 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:32.330 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:32.330 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:32.330 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:32.330 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:32.330 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:32.330 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.330 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.330 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.330 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.330 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.330 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-12T03:04:32.330 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-12T03:04:32.331 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.331 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.331 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.331 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:32.331 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.331 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.331 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.331 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.331 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.331 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.331 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.331 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:32.331 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:32.332 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.332 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-12T03:04:32.332 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "k8sevents", 2024-09-12T03:04:32.332 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-12T03:04:32.332 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-12T03:04:32.332 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-12T03:04:32.332 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_event_retention_days": { 2024-09-12T03:04:32.332 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ceph_event_retention_days", 2024-09-12T03:04:32.332 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-12T03:04:32.332 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.332 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.332 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "7", 2024-09-12T03:04:32.332 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.333 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.333 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.333 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-12T03:04:32.333 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.333 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.333 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.333 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.333 INFO:teuthology.orchestra.run.smithi005.stdout: "config_check_secs": { 2024-09-12T03:04:32.333 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "config_check_secs", 2024-09-12T03:04:32.333 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-12T03:04:32.333 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.333 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.333 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-09-12T03:04:32.334 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "10", 2024-09-12T03:04:32.334 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.334 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.334 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-12T03:04:32.334 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.334 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.334 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.334 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.334 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-12T03:04:32.334 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-12T03:04:32.334 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.334 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.334 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.334 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:32.334 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.335 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.335 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:32.335 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:32.335 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:32.335 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:32.335 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:32.335 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:32.335 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:32.335 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:32.335 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.335 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.335 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.335 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.335 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.336 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-12T03:04:32.336 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-12T03:04:32.336 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.336 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.336 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.336 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:32.336 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.336 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.336 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.336 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.336 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.336 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.337 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.337 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.337 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-12T03:04:32.337 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-12T03:04:32.337 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.337 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.337 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.337 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-12T03:04:32.337 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.337 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.337 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:32.337 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:32.338 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:32.338 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:32.338 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:32.338 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:32.338 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:32.338 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:32.338 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.338 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.338 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.338 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.338 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.338 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-12T03:04:32.338 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-12T03:04:32.339 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.339 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.339 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.339 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:32.339 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.339 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.339 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.339 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.339 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.339 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.339 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.339 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:32.339 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:32.340 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.340 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-12T03:04:32.340 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "localpool", 2024-09-12T03:04:32.340 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-12T03:04:32.340 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-12T03:04:32.340 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-12T03:04:32.340 INFO:teuthology.orchestra.run.smithi005.stdout: "failure_domain": { 2024-09-12T03:04:32.340 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "failure_domain", 2024-09-12T03:04:32.340 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.340 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.340 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.340 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "host", 2024-09-12T03:04:32.340 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.341 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.341 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.341 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "failure domain for any created local pool", 2024-09-12T03:04:32.341 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-12T03:04:32.341 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.341 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.341 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.341 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-12T03:04:32.341 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-12T03:04:32.341 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.341 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.341 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.341 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:32.342 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.342 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.342 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:32.342 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:32.342 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:32.342 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:32.342 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:32.342 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:32.342 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:32.342 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:32.342 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.342 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.342 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.343 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.343 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.343 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-12T03:04:32.343 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-12T03:04:32.343 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.343 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.343 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.343 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:32.343 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.343 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.343 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.343 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.343 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.344 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.344 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.344 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.344 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-12T03:04:32.344 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-12T03:04:32.344 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.344 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.344 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.344 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-12T03:04:32.344 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.344 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.344 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:32.344 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:32.345 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:32.345 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:32.345 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:32.345 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:32.345 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:32.345 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:32.345 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.345 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.345 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.345 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.345 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.345 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-12T03:04:32.345 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-12T03:04:32.346 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.346 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.346 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.346 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:32.346 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.346 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.346 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.346 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.346 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.346 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.346 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.346 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.346 INFO:teuthology.orchestra.run.smithi005.stdout: "min_size": { 2024-09-12T03:04:32.347 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "min_size", 2024-09-12T03:04:32.347 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-12T03:04:32.347 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.347 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.347 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:32.347 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.347 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.347 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.347 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "default min_size for any created local pool", 2024-09-12T03:04:32.347 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-12T03:04:32.347 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.347 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.347 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.347 INFO:teuthology.orchestra.run.smithi005.stdout: "num_rep": { 2024-09-12T03:04:32.348 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "num_rep", 2024-09-12T03:04:32.348 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-12T03:04:32.348 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.348 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.348 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "3", 2024-09-12T03:04:32.348 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.348 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.348 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.348 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "default replica count for any created local pool", 2024-09-12T03:04:32.348 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.348 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.348 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.348 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.349 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num": { 2024-09-12T03:04:32.349 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pg_num", 2024-09-12T03:04:32.349 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-12T03:04:32.349 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.349 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.349 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "128", 2024-09-12T03:04:32.349 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.349 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.349 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.349 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "default pg_num for any created local pool", 2024-09-12T03:04:32.349 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.349 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.349 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.350 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.350 INFO:teuthology.orchestra.run.smithi005.stdout: "prefix": { 2024-09-12T03:04:32.350 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "prefix", 2024-09-12T03:04:32.350 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.350 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.350 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.350 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:32.350 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.350 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.350 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.350 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "name prefix for any created local pool", 2024-09-12T03:04:32.350 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.350 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.350 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.351 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.351 INFO:teuthology.orchestra.run.smithi005.stdout: "subtree": { 2024-09-12T03:04:32.351 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "subtree", 2024-09-12T03:04:32.351 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.351 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.351 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.351 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "rack", 2024-09-12T03:04:32.351 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.351 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.351 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.351 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-12T03:04:32.351 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-12T03:04:32.351 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.351 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.352 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:32.352 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:32.352 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.352 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-12T03:04:32.352 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mds_autoscaler", 2024-09-12T03:04:32.352 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-12T03:04:32.352 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-12T03:04:32.352 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-12T03:04:32.352 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-12T03:04:32.352 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-12T03:04:32.352 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.352 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.352 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.353 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:32.353 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.353 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.353 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:32.353 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:32.353 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:32.353 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:32.353 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:32.353 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:32.353 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:32.353 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:32.353 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.353 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.353 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.354 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.354 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.354 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-12T03:04:32.354 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-12T03:04:32.354 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.354 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.354 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.354 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:32.354 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.354 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.354 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.354 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.354 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.355 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.355 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.355 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.355 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-12T03:04:32.355 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-12T03:04:32.355 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.355 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.355 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.355 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-12T03:04:32.355 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.355 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.355 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:32.355 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:32.355 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:32.356 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:32.356 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:32.356 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:32.356 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:32.356 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:32.356 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.356 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.356 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.356 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.356 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.356 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-12T03:04:32.356 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-12T03:04:32.356 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.357 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.357 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.357 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:32.357 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.357 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.357 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.357 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.357 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.357 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.357 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.357 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:32.357 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:32.357 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.357 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-12T03:04:32.358 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mirroring", 2024-09-12T03:04:32.358 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-12T03:04:32.358 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-12T03:04:32.358 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-12T03:04:32.358 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-12T03:04:32.358 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-12T03:04:32.358 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.358 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.358 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.358 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:32.358 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.358 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.358 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:32.359 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:32.359 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:32.359 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:32.359 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:32.359 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:32.359 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:32.359 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:32.359 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.359 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.359 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.359 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.359 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.359 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-12T03:04:32.359 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-12T03:04:32.360 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.360 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.360 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.360 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:32.360 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.360 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.360 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.360 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.360 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.360 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.360 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.360 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.360 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-12T03:04:32.361 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-12T03:04:32.361 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.361 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.361 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.361 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-12T03:04:32.361 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.361 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.361 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:32.361 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:32.361 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:32.361 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:32.361 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:32.361 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:32.362 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:32.362 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:32.362 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.362 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.362 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.362 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.362 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.362 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-12T03:04:32.362 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-12T03:04:32.362 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.362 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.362 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.362 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:32.363 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.363 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.363 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.363 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.363 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.363 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.363 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.363 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:32.363 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:32.363 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.363 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-12T03:04:32.363 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "nfs", 2024-09-12T03:04:32.364 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-12T03:04:32.364 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-12T03:04:32.364 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-12T03:04:32.364 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-12T03:04:32.364 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-12T03:04:32.364 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.364 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.364 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.364 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:32.364 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.364 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.364 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:32.365 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:32.365 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:32.365 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:32.365 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:32.365 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:32.365 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:32.365 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:32.365 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.365 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.365 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.366 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.366 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.366 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-12T03:04:32.366 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-12T03:04:32.366 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.366 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.366 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.366 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:32.366 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.366 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.366 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.367 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.367 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.367 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.367 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.367 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.367 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-12T03:04:32.367 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-12T03:04:32.367 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.367 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.367 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.368 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-12T03:04:32.368 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.368 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.368 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:32.368 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:32.368 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:32.368 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:32.368 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:32.368 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:32.368 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:32.368 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:32.368 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.369 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.369 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.369 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.369 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.369 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-12T03:04:32.369 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-12T03:04:32.369 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.369 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.369 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.369 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:32.369 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.369 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.369 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.369 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.370 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.370 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.370 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.370 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:32.370 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:32.370 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.370 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-12T03:04:32.370 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "orchestrator", 2024-09-12T03:04:32.370 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-12T03:04:32.370 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-12T03:04:32.370 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-12T03:04:32.370 INFO:teuthology.orchestra.run.smithi005.stdout: "fail_fs": { 2024-09-12T03:04:32.370 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "fail_fs", 2024-09-12T03:04:32.371 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.371 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.371 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.371 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:32.371 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.371 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.371 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.371 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-12T03:04:32.371 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.371 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.371 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.371 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.371 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-12T03:04:32.371 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-12T03:04:32.372 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.372 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.372 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.372 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:32.372 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.372 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.372 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:32.372 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:32.372 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:32.372 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:32.372 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:32.372 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:32.373 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:32.373 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:32.373 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.373 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.373 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.373 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.373 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.373 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-12T03:04:32.373 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-12T03:04:32.373 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.373 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.373 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.373 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:32.374 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.374 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.374 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.374 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.374 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.374 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.374 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.374 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.374 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-12T03:04:32.374 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-12T03:04:32.374 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.374 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.374 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.374 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-12T03:04:32.375 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.375 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.375 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:32.375 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:32.375 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:32.375 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:32.375 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:32.375 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:32.375 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:32.375 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:32.375 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.375 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.375 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.376 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.376 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.376 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-12T03:04:32.376 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-12T03:04:32.376 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.376 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.376 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.376 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:32.376 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.376 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.376 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.376 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.377 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.377 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.377 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.377 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.377 INFO:teuthology.orchestra.run.smithi005.stdout: "orchestrator": { 2024-09-12T03:04:32.377 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "orchestrator", 2024-09-12T03:04:32.377 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.377 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.377 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.377 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:32.377 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.377 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.378 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:32.378 INFO:teuthology.orchestra.run.smithi005.stdout: "cephadm", 2024-09-12T03:04:32.378 INFO:teuthology.orchestra.run.smithi005.stdout: "rook", 2024-09-12T03:04:32.378 INFO:teuthology.orchestra.run.smithi005.stdout: "test_orchestrator" 2024-09-12T03:04:32.378 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:32.378 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Orchestrator backend", 2024-09-12T03:04:32.378 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.378 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.378 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.378 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:32.378 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:32.378 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.378 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-12T03:04:32.379 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "osd_perf_query", 2024-09-12T03:04:32.379 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-12T03:04:32.379 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-12T03:04:32.379 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-12T03:04:32.379 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-12T03:04:32.379 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-12T03:04:32.379 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.379 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.379 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.379 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:32.379 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.379 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.380 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:32.380 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:32.380 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:32.380 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:32.380 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:32.380 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:32.380 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:32.380 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:32.380 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.380 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.380 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.380 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.380 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.381 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-12T03:04:32.381 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-12T03:04:32.381 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.381 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.381 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.381 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:32.381 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.381 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.381 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.381 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.381 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.381 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.381 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.382 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.382 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-12T03:04:32.383 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-12T03:04:32.383 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.383 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.384 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.384 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-12T03:04:32.384 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.384 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.384 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:32.384 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:32.384 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:32.384 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:32.384 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:32.384 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:32.384 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:32.384 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:32.384 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.385 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.385 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.385 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.385 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.385 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-12T03:04:32.385 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-12T03:04:32.385 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.385 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.385 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.385 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:32.385 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.385 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.385 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.386 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.386 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.386 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.386 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.386 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:32.386 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:32.386 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.386 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-12T03:04:32.386 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "osd_support", 2024-09-12T03:04:32.386 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-12T03:04:32.386 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-12T03:04:32.386 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-12T03:04:32.387 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-12T03:04:32.387 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-12T03:04:32.387 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.387 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.387 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.387 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:32.387 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.387 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.387 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:32.387 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:32.387 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:32.387 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:32.387 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:32.387 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:32.388 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:32.388 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:32.388 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.388 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.388 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.388 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.388 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.388 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-12T03:04:32.388 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-12T03:04:32.388 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.389 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.389 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.389 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:32.389 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.389 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.389 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.389 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.389 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.389 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.389 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.389 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.389 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-12T03:04:32.389 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-12T03:04:32.390 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.390 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.390 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.390 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-12T03:04:32.390 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.390 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.390 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:32.390 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:32.390 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:32.390 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:32.390 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:32.390 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:32.390 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:32.391 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:32.391 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.391 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.391 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.391 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.391 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.391 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-12T03:04:32.391 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-12T03:04:32.391 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.391 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.391 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.391 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:32.392 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.392 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.392 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.392 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.392 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.392 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.392 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.392 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:32.392 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:32.392 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.392 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-12T03:04:32.392 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pg_autoscaler", 2024-09-12T03:04:32.393 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-12T03:04:32.393 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-12T03:04:32.393 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-12T03:04:32.393 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-12T03:04:32.393 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-12T03:04:32.393 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.393 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.393 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.393 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:32.393 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.393 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.393 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:32.393 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:32.394 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:32.394 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:32.394 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:32.394 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:32.394 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:32.394 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:32.394 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.394 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.394 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.394 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.394 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.395 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-12T03:04:32.395 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-12T03:04:32.395 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.395 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.395 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.395 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:32.395 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.395 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.395 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.395 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.395 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.395 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.396 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.396 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.396 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-12T03:04:32.396 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-12T03:04:32.396 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.396 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.396 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.396 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-12T03:04:32.396 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.396 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.396 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:32.396 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:32.396 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:32.397 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:32.397 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:32.397 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:32.397 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:32.397 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:32.397 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.397 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.397 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.397 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.397 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.397 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-12T03:04:32.397 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-12T03:04:32.398 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.398 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.398 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.398 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:32.398 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.398 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.398 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.398 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.398 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.398 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.398 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.398 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.398 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-09-12T03:04:32.399 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-09-12T03:04:32.399 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-12T03:04:32.399 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.399 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.399 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-09-12T03:04:32.399 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.399 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.399 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.399 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.399 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.399 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.399 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.399 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.400 INFO:teuthology.orchestra.run.smithi005.stdout: "threshold": { 2024-09-12T03:04:32.400 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "threshold", 2024-09-12T03:04:32.400 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-09-12T03:04:32.400 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.400 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.400 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "3.0", 2024-09-12T03:04:32.400 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1.0", 2024-09-12T03:04:32.400 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.400 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.400 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "scaling threshold", 2024-09-12T03:04:32.400 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-12T03:04:32.400 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.401 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.401 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:32.401 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:32.401 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.401 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-12T03:04:32.401 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "progress", 2024-09-12T03:04:32.401 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-12T03:04:32.401 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-12T03:04:32.401 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-12T03:04:32.401 INFO:teuthology.orchestra.run.smithi005.stdout: "allow_pg_recovery_event": { 2024-09-12T03:04:32.401 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "allow_pg_recovery_event", 2024-09-12T03:04:32.401 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.401 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.402 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.402 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:32.402 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.402 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.402 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.402 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-12T03:04:32.402 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.402 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.402 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.402 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.402 INFO:teuthology.orchestra.run.smithi005.stdout: "enabled": { 2024-09-12T03:04:32.402 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enabled", 2024-09-12T03:04:32.402 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.402 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.403 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.403 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-12T03:04:32.403 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.403 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.403 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.403 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.403 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.403 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.403 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.403 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.403 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-12T03:04:32.403 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-12T03:04:32.403 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.404 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.404 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.404 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:32.404 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.404 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.404 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:32.404 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:32.404 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:32.404 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:32.404 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:32.404 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:32.404 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:32.404 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:32.405 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.405 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.405 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.405 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.405 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.405 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-12T03:04:32.405 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-12T03:04:32.405 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.405 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.405 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.405 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:32.405 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.405 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.405 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.406 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.406 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.406 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.406 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.406 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.406 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-12T03:04:32.406 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-12T03:04:32.406 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.406 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.406 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.406 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-12T03:04:32.406 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.406 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.407 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:32.407 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:32.407 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:32.407 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:32.407 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:32.407 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:32.407 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:32.407 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:32.407 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.407 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.407 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.407 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.407 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.407 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-12T03:04:32.408 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-12T03:04:32.408 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.408 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.408 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.408 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:32.408 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.408 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.408 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.408 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.408 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.408 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.408 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.408 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.409 INFO:teuthology.orchestra.run.smithi005.stdout: "max_completed_events": { 2024-09-12T03:04:32.409 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_completed_events", 2024-09-12T03:04:32.409 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-12T03:04:32.409 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.409 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.409 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "50", 2024-09-12T03:04:32.409 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.409 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.409 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.409 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "number of past completed events to remember", 2024-09-12T03:04:32.409 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.409 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.409 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.410 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.410 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2024-09-12T03:04:32.410 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2024-09-12T03:04:32.410 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-12T03:04:32.410 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.410 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.410 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2024-09-12T03:04:32.410 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.410 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.410 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.410 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how long the module is going to sleep", 2024-09-12T03:04:32.410 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.410 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.410 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.411 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:32.411 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:32.411 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.411 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-12T03:04:32.411 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "prometheus", 2024-09-12T03:04:32.411 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-12T03:04:32.411 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-12T03:04:32.411 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-12T03:04:32.411 INFO:teuthology.orchestra.run.smithi005.stdout: "cache": { 2024-09-12T03:04:32.411 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cache", 2024-09-12T03:04:32.411 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.411 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.411 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.412 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-12T03:04:32.412 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.412 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.412 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.412 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.412 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.412 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.412 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.412 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.412 INFO:teuthology.orchestra.run.smithi005.stdout: "exclude_perf_counters": { 2024-09-12T03:04:32.412 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "exclude_perf_counters", 2024-09-12T03:04:32.412 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.413 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.413 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.413 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-12T03:04:32.413 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.413 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.413 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.413 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-12T03:04:32.413 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-12T03:04:32.413 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.413 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.413 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.413 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-12T03:04:32.413 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-12T03:04:32.414 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.414 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.414 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.414 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:32.414 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.414 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.414 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:32.414 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:32.414 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:32.414 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:32.414 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:32.414 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:32.414 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:32.415 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:32.415 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.415 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.415 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.415 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.415 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.415 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-12T03:04:32.415 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-12T03:04:32.415 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.415 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.415 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.415 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:32.416 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.416 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.416 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.416 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.416 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.416 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.416 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.416 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.416 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-12T03:04:32.416 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-12T03:04:32.416 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.416 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.416 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.417 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-12T03:04:32.417 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.417 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.417 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:32.417 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:32.417 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:32.417 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:32.417 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:32.417 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:32.417 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:32.417 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:32.417 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.417 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.418 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.418 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.418 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.418 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-12T03:04:32.418 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-12T03:04:32.418 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.418 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.418 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.418 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:32.418 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.418 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.418 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.418 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.419 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.419 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.419 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.419 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.419 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_stats_pools": { 2024-09-12T03:04:32.419 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rbd_stats_pools", 2024-09-12T03:04:32.419 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.419 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.419 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.419 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:32.419 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.419 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.419 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.419 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.420 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.420 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.420 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.420 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.420 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-12T03:04:32.420 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-12T03:04:32.420 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-12T03:04:32.420 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.420 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.420 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "300", 2024-09-12T03:04:32.420 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.420 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.420 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.421 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.421 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.421 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.421 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.421 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.421 INFO:teuthology.orchestra.run.smithi005.stdout: "scrape_interval": { 2024-09-12T03:04:32.421 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "scrape_interval", 2024-09-12T03:04:32.421 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-09-12T03:04:32.421 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.421 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.421 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "15.0", 2024-09-12T03:04:32.421 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.421 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.422 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.422 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.422 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.422 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.422 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.422 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.422 INFO:teuthology.orchestra.run.smithi005.stdout: "server_addr": { 2024-09-12T03:04:32.422 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_addr", 2024-09-12T03:04:32.422 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.422 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.422 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.422 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "::", 2024-09-12T03:04:32.422 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.422 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.423 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.423 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-12T03:04:32.423 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.423 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.423 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.423 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.423 INFO:teuthology.orchestra.run.smithi005.stdout: "server_port": { 2024-09-12T03:04:32.423 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_port", 2024-09-12T03:04:32.423 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-12T03:04:32.423 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.423 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.423 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "9283", 2024-09-12T03:04:32.423 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.424 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.424 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.424 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-12T03:04:32.424 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.424 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.424 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.424 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.424 INFO:teuthology.orchestra.run.smithi005.stdout: "stale_cache_strategy": { 2024-09-12T03:04:32.424 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "stale_cache_strategy", 2024-09-12T03:04:32.424 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.424 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.424 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.424 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "log", 2024-09-12T03:04:32.424 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.425 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.425 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.425 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.425 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.425 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.425 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.425 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.425 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_behaviour": { 2024-09-12T03:04:32.425 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_behaviour", 2024-09-12T03:04:32.425 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.425 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.425 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.425 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "default", 2024-09-12T03:04:32.426 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.426 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.426 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:32.426 INFO:teuthology.orchestra.run.smithi005.stdout: "default", 2024-09-12T03:04:32.426 INFO:teuthology.orchestra.run.smithi005.stdout: "error" 2024-09-12T03:04:32.426 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:32.426 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.426 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.426 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.426 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.426 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.426 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_error_status_code": { 2024-09-12T03:04:32.427 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_error_status_code", 2024-09-12T03:04:32.427 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-12T03:04:32.427 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.427 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.427 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "500", 2024-09-12T03:04:32.427 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "400", 2024-09-12T03:04:32.427 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "599", 2024-09-12T03:04:32.427 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.427 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.427 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.427 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.427 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.427 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:32.427 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:32.428 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.428 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-12T03:04:32.428 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rbd_support", 2024-09-12T03:04:32.428 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-12T03:04:32.428 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-12T03:04:32.428 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-12T03:04:32.428 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-12T03:04:32.428 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-12T03:04:32.428 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.428 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.428 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.428 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:32.429 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.429 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.429 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:32.429 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:32.429 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:32.429 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:32.429 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:32.429 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:32.429 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:32.429 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:32.429 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.429 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.430 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.430 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.430 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.430 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-12T03:04:32.430 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-12T03:04:32.430 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.430 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.430 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.430 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:32.430 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.430 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.430 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.431 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.431 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.431 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.431 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.431 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.431 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-12T03:04:32.431 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-12T03:04:32.431 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.431 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.431 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.431 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-12T03:04:32.431 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.431 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.432 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:32.432 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:32.432 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:32.432 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:32.432 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:32.432 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:32.432 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:32.432 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:32.432 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.432 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.432 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.432 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.432 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.433 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-12T03:04:32.433 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-12T03:04:32.433 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.433 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.433 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.433 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:32.433 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.433 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.433 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.433 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.433 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.433 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.433 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.434 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.434 INFO:teuthology.orchestra.run.smithi005.stdout: "max_concurrent_snap_create": { 2024-09-12T03:04:32.434 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_concurrent_snap_create", 2024-09-12T03:04:32.434 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-12T03:04:32.434 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.434 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.434 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2024-09-12T03:04:32.434 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.434 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.434 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.434 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.434 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.434 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.435 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.435 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.435 INFO:teuthology.orchestra.run.smithi005.stdout: "mirror_snapshot_schedule": { 2024-09-12T03:04:32.435 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mirror_snapshot_schedule", 2024-09-12T03:04:32.435 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.435 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.435 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.435 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:32.435 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.435 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.435 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.435 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.435 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.435 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.436 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.436 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.436 INFO:teuthology.orchestra.run.smithi005.stdout: "trash_purge_schedule": { 2024-09-12T03:04:32.436 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "trash_purge_schedule", 2024-09-12T03:04:32.436 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.436 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.436 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.436 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:32.436 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.436 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.436 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.436 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.436 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.437 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.437 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.437 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:32.437 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:32.437 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.437 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-12T03:04:32.437 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "restful", 2024-09-12T03:04:32.437 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-12T03:04:32.437 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-12T03:04:32.437 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-12T03:04:32.437 INFO:teuthology.orchestra.run.smithi005.stdout: "enable_auth": { 2024-09-12T03:04:32.437 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enable_auth", 2024-09-12T03:04:32.437 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.438 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.438 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.438 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-12T03:04:32.438 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.438 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.438 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.438 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.438 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.438 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.438 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.438 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.438 INFO:teuthology.orchestra.run.smithi005.stdout: "key_file": { 2024-09-12T03:04:32.438 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "key_file", 2024-09-12T03:04:32.439 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.439 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.439 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.439 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:32.439 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.439 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.439 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.439 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.439 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.439 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.439 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.439 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.439 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-12T03:04:32.439 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-12T03:04:32.440 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.440 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.440 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.440 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:32.440 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.440 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.440 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:32.440 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:32.440 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:32.440 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:32.440 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:32.440 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:32.440 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:32.440 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:32.441 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.441 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.441 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.441 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.441 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.441 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-12T03:04:32.441 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-12T03:04:32.441 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.441 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.441 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.441 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:32.441 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.441 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.441 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.442 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.442 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.442 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.442 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.442 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.442 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-12T03:04:32.442 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-12T03:04:32.442 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.442 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.442 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.442 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-12T03:04:32.442 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.442 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.442 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:32.443 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:32.443 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:32.443 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:32.443 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:32.443 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:32.443 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:32.443 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:32.443 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.443 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.443 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.443 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.443 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.443 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-12T03:04:32.443 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-12T03:04:32.444 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.444 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.444 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.444 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:32.444 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.444 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.444 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.444 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.444 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.444 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.444 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.444 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.444 INFO:teuthology.orchestra.run.smithi005.stdout: "server_addr": { 2024-09-12T03:04:32.445 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_addr", 2024-09-12T03:04:32.445 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.445 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.445 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.445 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:32.445 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.445 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.445 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.445 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.445 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.445 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.445 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.446 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.446 INFO:teuthology.orchestra.run.smithi005.stdout: "server_port": { 2024-09-12T03:04:32.446 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_port", 2024-09-12T03:04:32.446 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.446 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.446 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.446 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:32.446 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.446 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.446 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.446 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.446 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.446 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.446 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.447 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:32.447 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:32.447 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.447 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-12T03:04:32.447 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rgw", 2024-09-12T03:04:32.447 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-12T03:04:32.447 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-12T03:04:32.447 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-12T03:04:32.447 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-12T03:04:32.447 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-12T03:04:32.447 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.447 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.447 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.448 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:32.448 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.448 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.448 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:32.448 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:32.448 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:32.448 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:32.448 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:32.448 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:32.448 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:32.448 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:32.448 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.448 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.449 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.449 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.449 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.449 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-12T03:04:32.449 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-12T03:04:32.449 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.449 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.449 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.449 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:32.449 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.449 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.449 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.449 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.450 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.450 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.450 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.450 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.450 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-12T03:04:32.450 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-12T03:04:32.450 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.450 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.450 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.450 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-12T03:04:32.450 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.450 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.450 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:32.451 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:32.451 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:32.451 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:32.451 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:32.451 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:32.451 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:32.451 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:32.451 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.451 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.451 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.451 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.451 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.451 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-12T03:04:32.452 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-12T03:04:32.452 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.452 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.452 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.452 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:32.452 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.452 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.452 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.452 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.452 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.452 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.452 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.452 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:32.453 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:32.453 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.453 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-12T03:04:32.453 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rook", 2024-09-12T03:04:32.453 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-12T03:04:32.453 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-12T03:04:32.453 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-12T03:04:32.453 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-12T03:04:32.453 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-12T03:04:32.453 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.453 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.453 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.453 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:32.454 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.454 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.454 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:32.454 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:32.454 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:32.454 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:32.454 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:32.454 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:32.454 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:32.454 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:32.454 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.454 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.454 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.455 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.455 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.455 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-12T03:04:32.455 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-12T03:04:32.455 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.455 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.455 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.455 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:32.455 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.455 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.455 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.455 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.455 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.455 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.456 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.456 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.456 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-12T03:04:32.456 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-12T03:04:32.456 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.456 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.456 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.456 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-12T03:04:32.456 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.456 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.456 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:32.456 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:32.457 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:32.457 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:32.457 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:32.457 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:32.457 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:32.457 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:32.457 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.457 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.457 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.457 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.457 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.457 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-12T03:04:32.457 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-12T03:04:32.458 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.458 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.458 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.458 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:32.458 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.458 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.458 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.458 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.458 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.458 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.458 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.458 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.458 INFO:teuthology.orchestra.run.smithi005.stdout: "storage_class": { 2024-09-12T03:04:32.458 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "storage_class", 2024-09-12T03:04:32.459 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.459 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.459 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.459 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "local", 2024-09-12T03:04:32.459 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.459 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.459 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.459 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-12T03:04:32.459 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.459 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.459 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.459 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:32.459 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:32.460 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.460 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-12T03:04:32.460 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "selftest", 2024-09-12T03:04:32.460 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-12T03:04:32.460 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-12T03:04:32.460 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-12T03:04:32.460 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-12T03:04:32.460 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-12T03:04:32.460 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.460 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.460 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.460 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:32.460 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.460 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.461 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:32.461 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:32.461 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:32.461 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:32.461 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:32.461 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:32.461 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:32.461 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:32.461 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.461 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.461 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.461 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.461 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.462 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-12T03:04:32.462 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-12T03:04:32.462 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.462 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.462 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.462 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:32.462 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.462 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.462 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.462 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.462 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.462 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.462 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.463 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.463 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-12T03:04:32.463 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-12T03:04:32.463 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.463 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.463 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.463 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-12T03:04:32.463 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.463 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.463 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:32.463 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:32.463 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:32.463 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:32.463 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:32.464 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:32.464 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:32.464 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:32.464 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.464 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.464 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.464 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.464 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.464 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-12T03:04:32.464 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-12T03:04:32.464 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.464 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.464 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.464 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:32.465 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.465 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.465 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.465 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.465 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.465 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.465 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.465 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.465 INFO:teuthology.orchestra.run.smithi005.stdout: "roption1": { 2024-09-12T03:04:32.465 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "roption1", 2024-09-12T03:04:32.465 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.465 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.465 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.466 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:32.466 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.466 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.466 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.466 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.466 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.466 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.466 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.466 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.466 INFO:teuthology.orchestra.run.smithi005.stdout: "roption2": { 2024-09-12T03:04:32.466 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "roption2", 2024-09-12T03:04:32.466 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.466 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.467 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.467 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "xyz", 2024-09-12T03:04:32.467 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.467 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.467 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.467 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.467 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.467 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.467 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.467 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.467 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption1": { 2024-09-12T03:04:32.467 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption1", 2024-09-12T03:04:32.467 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.468 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.468 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.468 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:32.468 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.468 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.468 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.468 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.468 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.468 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.468 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.468 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.468 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption2": { 2024-09-12T03:04:32.468 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption2", 2024-09-12T03:04:32.469 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-12T03:04:32.469 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.469 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.469 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:32.469 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.469 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.469 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.469 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.469 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.469 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.469 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.469 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.469 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption3": { 2024-09-12T03:04:32.470 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption3", 2024-09-12T03:04:32.470 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2024-09-12T03:04:32.470 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.470 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.470 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:32.470 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.470 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.470 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.470 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.470 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.470 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.470 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.470 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.471 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption4": { 2024-09-12T03:04:32.471 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption4", 2024-09-12T03:04:32.471 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.471 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.471 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.471 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:32.471 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.471 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.471 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.471 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.471 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.471 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.471 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.472 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.472 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption5": { 2024-09-12T03:04:32.472 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption5", 2024-09-12T03:04:32.472 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.472 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.472 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.472 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:32.472 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.472 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.472 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.472 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.472 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.472 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.473 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.473 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.473 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption6": { 2024-09-12T03:04:32.473 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption6", 2024-09-12T03:04:32.473 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.473 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.473 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.473 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-12T03:04:32.473 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.473 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.473 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.473 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.473 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.474 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.474 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.474 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.474 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption7": { 2024-09-12T03:04:32.474 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption7", 2024-09-12T03:04:32.474 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-12T03:04:32.474 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.474 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.474 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:32.474 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2024-09-12T03:04:32.474 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "42", 2024-09-12T03:04:32.474 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.475 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.475 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.475 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.475 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.475 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.475 INFO:teuthology.orchestra.run.smithi005.stdout: "testkey": { 2024-09-12T03:04:32.475 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "testkey", 2024-09-12T03:04:32.475 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.475 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.475 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.475 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:32.475 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.475 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.475 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.476 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.476 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.476 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.476 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.476 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.476 INFO:teuthology.orchestra.run.smithi005.stdout: "testlkey": { 2024-09-12T03:04:32.476 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "testlkey", 2024-09-12T03:04:32.476 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.476 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.476 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.476 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:32.476 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.476 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.477 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.477 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.477 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.477 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.477 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.477 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.477 INFO:teuthology.orchestra.run.smithi005.stdout: "testnewline": { 2024-09-12T03:04:32.477 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "testnewline", 2024-09-12T03:04:32.477 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.477 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.477 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.477 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:32.477 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.477 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.478 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.478 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.478 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.478 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.478 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.478 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:32.478 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:32.478 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.478 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-12T03:04:32.478 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "snap_schedule", 2024-09-12T03:04:32.478 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-12T03:04:32.478 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-12T03:04:32.478 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-12T03:04:32.478 INFO:teuthology.orchestra.run.smithi005.stdout: "allow_m_granularity": { 2024-09-12T03:04:32.479 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "allow_m_granularity", 2024-09-12T03:04:32.479 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.479 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.479 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.479 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:32.479 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.479 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.479 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.479 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "allow minute scheduled snapshots", 2024-09-12T03:04:32.479 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.479 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.479 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.479 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.479 INFO:teuthology.orchestra.run.smithi005.stdout: "dump_on_update": { 2024-09-12T03:04:32.480 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "dump_on_update", 2024-09-12T03:04:32.480 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.480 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.480 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.480 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:32.480 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.480 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.480 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.480 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "dump database to debug log on update", 2024-09-12T03:04:32.480 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.480 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.480 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.480 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.481 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-12T03:04:32.481 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-12T03:04:32.481 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.481 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.481 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.481 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:32.481 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.481 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.481 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:32.481 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:32.481 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:32.481 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:32.481 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:32.481 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:32.482 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:32.482 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:32.482 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.482 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.482 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.482 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.482 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.482 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-12T03:04:32.482 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-12T03:04:32.482 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.482 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.482 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.482 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:32.482 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.483 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.483 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.483 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.483 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.483 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.483 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.483 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.483 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-12T03:04:32.483 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-12T03:04:32.483 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.483 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.483 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.483 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-12T03:04:32.484 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.484 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.484 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:32.484 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:32.484 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:32.484 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:32.484 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:32.484 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:32.484 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:32.484 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:32.484 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.484 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.484 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.485 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.485 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.485 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-12T03:04:32.485 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-12T03:04:32.485 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.485 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.485 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.485 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:32.485 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.485 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.485 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.485 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.485 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.486 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.486 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.486 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:32.486 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:32.486 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.486 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-12T03:04:32.486 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "stats", 2024-09-12T03:04:32.486 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-12T03:04:32.486 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-12T03:04:32.486 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-12T03:04:32.486 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-12T03:04:32.486 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-12T03:04:32.486 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.486 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.487 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.487 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:32.487 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.487 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.487 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:32.487 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:32.487 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:32.487 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:32.487 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:32.487 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:32.487 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:32.487 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:32.487 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.487 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.488 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.488 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.488 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.488 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-12T03:04:32.488 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-12T03:04:32.488 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.488 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.488 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.488 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:32.488 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.488 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.488 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.488 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.489 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.489 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.489 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.489 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.489 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-12T03:04:32.489 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-12T03:04:32.489 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.489 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.489 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.489 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-12T03:04:32.489 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.489 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.489 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:32.490 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:32.490 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:32.490 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:32.490 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:32.490 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:32.490 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:32.490 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:32.490 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.490 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.490 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.490 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.490 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.490 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-12T03:04:32.491 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-12T03:04:32.491 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.491 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.491 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.491 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:32.491 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.491 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.491 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.491 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.491 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.491 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.491 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.491 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:32.492 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:32.492 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.492 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-12T03:04:32.492 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "status", 2024-09-12T03:04:32.492 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-12T03:04:32.492 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-12T03:04:32.492 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-12T03:04:32.492 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-12T03:04:32.492 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-12T03:04:32.492 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.492 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.492 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.492 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:32.493 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.493 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.493 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:32.493 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:32.493 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:32.493 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:32.493 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:32.493 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:32.493 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:32.493 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:32.493 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.493 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.493 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.494 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.494 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.494 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-12T03:04:32.494 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-12T03:04:32.494 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.494 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.494 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.494 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:32.494 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.494 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.494 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.494 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.494 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.495 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.495 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.495 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.495 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-12T03:04:32.495 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-12T03:04:32.495 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.495 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.495 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.495 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-12T03:04:32.495 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.495 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.495 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:32.495 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:32.496 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:32.496 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:32.496 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:32.496 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:32.496 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:32.496 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:32.496 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.496 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.496 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.496 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.496 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.496 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-12T03:04:32.496 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-12T03:04:32.497 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.497 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.497 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.497 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:32.497 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.497 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.497 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.497 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.497 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.497 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.497 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.497 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:32.497 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:32.498 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.498 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-12T03:04:32.498 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "telegraf", 2024-09-12T03:04:32.498 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-12T03:04:32.498 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-12T03:04:32.498 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-12T03:04:32.498 INFO:teuthology.orchestra.run.smithi005.stdout: "address": { 2024-09-12T03:04:32.498 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "address", 2024-09-12T03:04:32.498 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.498 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.498 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.498 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-12T03:04:32.498 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.499 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.499 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.499 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.499 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.499 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.499 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.499 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.499 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-09-12T03:04:32.499 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-09-12T03:04:32.499 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-12T03:04:32.499 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.499 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.499 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "15", 2024-09-12T03:04:32.499 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.499 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.500 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.500 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.500 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.500 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.500 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.500 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.500 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-12T03:04:32.500 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-12T03:04:32.500 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.500 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.500 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.500 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:32.500 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.501 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.501 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:32.501 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:32.501 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:32.501 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:32.501 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:32.501 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:32.501 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:32.501 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:32.501 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.501 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.501 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.501 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.501 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.502 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-12T03:04:32.502 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-12T03:04:32.502 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.502 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.502 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.502 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:32.502 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.502 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.502 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.502 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.502 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.502 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.502 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.503 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.503 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-12T03:04:32.503 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-12T03:04:32.503 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.503 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.503 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.503 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-12T03:04:32.503 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.503 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.503 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:32.503 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:32.503 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:32.504 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:32.504 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:32.504 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:32.504 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:32.504 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:32.504 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.504 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.504 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.504 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.504 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.504 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-12T03:04:32.504 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-12T03:04:32.504 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.505 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.505 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.505 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:32.505 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.505 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.505 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.505 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.505 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.505 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.505 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.505 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:32.505 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:32.506 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.506 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-12T03:04:32.506 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "telemetry", 2024-09-12T03:04:32.506 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-12T03:04:32.506 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-12T03:04:32.506 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-12T03:04:32.506 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_basic": { 2024-09-12T03:04:32.506 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_basic", 2024-09-12T03:04:32.506 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.506 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.506 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.506 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-12T03:04:32.506 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.507 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.507 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.507 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-12T03:04:32.507 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.507 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.507 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.507 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.507 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_crash": { 2024-09-12T03:04:32.507 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_crash", 2024-09-12T03:04:32.507 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.507 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.507 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.507 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-12T03:04:32.508 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.508 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.508 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.508 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-12T03:04:32.508 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.508 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.508 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.508 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.508 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_device": { 2024-09-12T03:04:32.508 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_device", 2024-09-12T03:04:32.508 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.508 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.508 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.509 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-12T03:04:32.509 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.509 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.509 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.509 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-12T03:04:32.509 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.509 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.509 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.509 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.509 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_ident": { 2024-09-12T03:04:32.509 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_ident", 2024-09-12T03:04:32.509 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.509 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.510 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.510 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:32.510 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.510 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.510 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.510 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-12T03:04:32.510 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.510 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.510 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.510 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.510 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_perf": { 2024-09-12T03:04:32.510 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_perf", 2024-09-12T03:04:32.510 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.510 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.511 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.511 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:32.511 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.511 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.511 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.511 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-12T03:04:32.511 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.511 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.511 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.511 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.511 INFO:teuthology.orchestra.run.smithi005.stdout: "contact": { 2024-09-12T03:04:32.511 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "contact", 2024-09-12T03:04:32.511 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.512 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.512 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.512 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:32.512 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.512 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.512 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.512 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.512 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.512 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.512 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.512 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.512 INFO:teuthology.orchestra.run.smithi005.stdout: "description": { 2024-09-12T03:04:32.513 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "description", 2024-09-12T03:04:32.513 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.513 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.513 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.513 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:32.513 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.513 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.513 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.513 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.513 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.513 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.513 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.513 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.513 INFO:teuthology.orchestra.run.smithi005.stdout: "device_url": { 2024-09-12T03:04:32.514 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "device_url", 2024-09-12T03:04:32.514 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.514 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.514 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.514 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-12T03:04:32.514 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.514 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.514 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.514 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.514 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.514 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.514 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.514 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.515 INFO:teuthology.orchestra.run.smithi005.stdout: "enabled": { 2024-09-12T03:04:32.515 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enabled", 2024-09-12T03:04:32.515 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.515 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.515 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.515 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:32.515 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.515 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.515 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.515 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.515 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.515 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.515 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.516 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.516 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-09-12T03:04:32.516 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-09-12T03:04:32.516 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-12T03:04:32.516 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.516 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.516 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "24", 2024-09-12T03:04:32.516 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "8", 2024-09-12T03:04:32.516 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.516 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.516 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.516 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.516 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.517 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.517 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.517 INFO:teuthology.orchestra.run.smithi005.stdout: "last_opt_revision": { 2024-09-12T03:04:32.517 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "last_opt_revision", 2024-09-12T03:04:32.517 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-12T03:04:32.517 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.517 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.517 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "1", 2024-09-12T03:04:32.517 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.517 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.517 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.517 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.517 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.518 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.518 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.518 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.518 INFO:teuthology.orchestra.run.smithi005.stdout: "leaderboard": { 2024-09-12T03:04:32.518 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "leaderboard", 2024-09-12T03:04:32.518 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.518 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.518 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.518 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:32.518 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.518 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.518 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.518 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.519 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.519 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.519 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.519 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.519 INFO:teuthology.orchestra.run.smithi005.stdout: "leaderboard_description": { 2024-09-12T03:04:32.519 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "leaderboard_description", 2024-09-12T03:04:32.519 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.519 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.519 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.519 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:32.519 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.519 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.519 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.520 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.520 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.520 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.520 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.520 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.520 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-12T03:04:32.520 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-12T03:04:32.520 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.520 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.520 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.520 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:32.520 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.520 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.521 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:32.521 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:32.521 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:32.521 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:32.521 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:32.521 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:32.521 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:32.521 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:32.521 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.521 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.521 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.521 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.521 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.522 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-12T03:04:32.522 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-12T03:04:32.522 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.522 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.522 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.522 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:32.522 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.522 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.522 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.522 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.522 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.522 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.522 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.522 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.523 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-12T03:04:32.523 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-12T03:04:32.523 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.523 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.523 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.523 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-12T03:04:32.523 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.523 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.523 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:32.523 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:32.523 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:32.523 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:32.523 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:32.524 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:32.524 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:32.524 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:32.524 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.524 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.524 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.524 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.524 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.524 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-12T03:04:32.524 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-12T03:04:32.524 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.524 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.524 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.525 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:32.525 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.525 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.525 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.525 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.525 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.525 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.525 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.525 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.525 INFO:teuthology.orchestra.run.smithi005.stdout: "organization": { 2024-09-12T03:04:32.525 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "organization", 2024-09-12T03:04:32.525 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.525 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.526 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.526 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:32.526 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.526 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.526 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.526 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.526 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.526 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.526 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.526 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.526 INFO:teuthology.orchestra.run.smithi005.stdout: "proxy": { 2024-09-12T03:04:32.526 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "proxy", 2024-09-12T03:04:32.526 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.526 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.527 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.527 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:32.527 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.527 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.527 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.527 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.527 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.527 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.527 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.527 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.527 INFO:teuthology.orchestra.run.smithi005.stdout: "url": { 2024-09-12T03:04:32.527 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "url", 2024-09-12T03:04:32.527 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.527 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.528 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.528 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-12T03:04:32.528 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.528 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.528 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.528 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.528 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.528 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.528 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.528 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:32.528 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:32.528 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.528 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-12T03:04:32.529 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "test_orchestrator", 2024-09-12T03:04:32.529 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-12T03:04:32.529 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-12T03:04:32.529 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-12T03:04:32.529 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-12T03:04:32.529 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-12T03:04:32.529 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.529 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.529 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.529 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:32.529 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.529 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.529 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:32.529 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:32.530 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:32.530 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:32.530 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:32.530 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:32.530 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:32.530 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:32.530 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.530 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.530 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.530 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.530 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.530 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-12T03:04:32.530 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-12T03:04:32.531 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.531 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.531 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.531 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:32.531 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.531 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.531 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.531 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.531 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.531 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.531 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.531 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.531 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-12T03:04:32.531 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-12T03:04:32.532 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.532 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.532 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.532 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-12T03:04:32.532 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.532 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.532 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:32.532 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:32.532 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:32.532 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:32.532 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:32.532 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:32.532 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:32.532 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:32.533 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.533 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.533 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.533 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.533 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.533 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-12T03:04:32.533 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-12T03:04:32.533 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.533 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.533 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.533 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:32.533 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.533 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.534 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.534 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.534 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.534 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.534 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.534 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:32.534 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:32.534 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.534 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-12T03:04:32.534 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "volumes", 2024-09-12T03:04:32.534 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-12T03:04:32.534 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-12T03:04:32.534 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-12T03:04:32.534 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-12T03:04:32.535 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-12T03:04:32.535 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.535 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.535 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.535 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:32.535 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.535 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.535 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:32.535 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:32.535 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:32.535 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:32.535 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:32.535 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:32.536 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:32.536 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:32.536 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.536 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.536 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.536 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.536 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.536 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-12T03:04:32.536 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-12T03:04:32.536 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.536 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.536 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.536 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:32.536 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.537 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.537 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.537 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.537 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.537 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.537 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.537 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.537 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-12T03:04:32.537 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-12T03:04:32.537 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.537 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.537 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.537 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-12T03:04:32.538 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.538 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.538 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:32.538 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:32.538 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:32.538 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:32.538 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:32.538 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:32.538 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:32.538 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:32.538 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.538 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.538 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.538 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.539 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.539 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-12T03:04:32.539 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-12T03:04:32.539 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.539 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.539 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.539 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:32.539 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.539 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.539 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.539 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.539 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.539 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.540 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.540 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.540 INFO:teuthology.orchestra.run.smithi005.stdout: "max_concurrent_clones": { 2024-09-12T03:04:32.540 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_concurrent_clones", 2024-09-12T03:04:32.540 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-12T03:04:32.540 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.540 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.540 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "4", 2024-09-12T03:04:32.540 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.540 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.540 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.540 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-12T03:04:32.540 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.541 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.541 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.541 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.541 INFO:teuthology.orchestra.run.smithi005.stdout: "periodic_async_work": { 2024-09-12T03:04:32.541 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "periodic_async_work", 2024-09-12T03:04:32.541 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.541 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.541 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.541 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:32.541 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.541 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.541 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.541 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Periodically check for async work", 2024-09-12T03:04:32.541 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.542 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.542 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.542 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.542 INFO:teuthology.orchestra.run.smithi005.stdout: "snapshot_clone_delay": { 2024-09-12T03:04:32.542 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "snapshot_clone_delay", 2024-09-12T03:04:32.542 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-12T03:04:32.542 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.542 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.542 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2024-09-12T03:04:32.542 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.542 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.542 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.542 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-12T03:04:32.543 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.543 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.543 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.543 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.543 INFO:teuthology.orchestra.run.smithi005.stdout: "snapshot_clone_no_wait": { 2024-09-12T03:04:32.543 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "snapshot_clone_no_wait", 2024-09-12T03:04:32.543 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.543 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.543 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.543 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2024-09-12T03:04:32.543 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.543 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.543 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.544 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-12T03:04:32.544 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.544 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.544 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.544 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:32.544 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:32.544 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.544 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-12T03:04:32.544 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix", 2024-09-12T03:04:32.544 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2024-09-12T03:04:32.544 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2024-09-12T03:04:32.544 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2024-09-12T03:04:32.544 INFO:teuthology.orchestra.run.smithi005.stdout: "discovery_interval": { 2024-09-12T03:04:32.545 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "discovery_interval", 2024-09-12T03:04:32.545 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2024-09-12T03:04:32.545 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.545 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.545 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "100", 2024-09-12T03:04:32.545 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.545 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.545 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.545 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.545 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.545 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.545 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.545 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.545 INFO:teuthology.orchestra.run.smithi005.stdout: "identifier": { 2024-09-12T03:04:32.546 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "identifier", 2024-09-12T03:04:32.546 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.546 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.546 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.546 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:32.546 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.546 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.546 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.546 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.546 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.546 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.546 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.546 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.547 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2024-09-12T03:04:32.547 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2024-09-12T03:04:32.547 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2024-09-12T03:04:32.547 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.547 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.547 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2024-09-12T03:04:32.547 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.547 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.547 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.547 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.547 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.547 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.547 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.547 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.548 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2024-09-12T03:04:32.548 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2024-09-12T03:04:32.548 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.548 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.548 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.548 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:32.548 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.548 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.548 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:32.548 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:32.548 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:32.548 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:32.548 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:32.549 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:32.549 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:32.549 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:32.549 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.549 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.549 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.549 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.549 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.549 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2024-09-12T03:04:32.549 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2024-09-12T03:04:32.549 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.549 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.549 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.549 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:32.550 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.550 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.550 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.550 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.550 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.550 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.550 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.550 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.550 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2024-09-12T03:04:32.550 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2024-09-12T03:04:32.550 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.550 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.550 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.551 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2024-09-12T03:04:32.551 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.551 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.551 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2024-09-12T03:04:32.551 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2024-09-12T03:04:32.551 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2024-09-12T03:04:32.551 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2024-09-12T03:04:32.551 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2024-09-12T03:04:32.551 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2024-09-12T03:04:32.551 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2024-09-12T03:04:32.551 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:32.551 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.551 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.551 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.552 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.552 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.552 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2024-09-12T03:04:32.552 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2024-09-12T03:04:32.552 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2024-09-12T03:04:32.552 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.552 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2024-09-12T03:04:32.552 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2024-09-12T03:04:32.552 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.552 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.552 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.552 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.552 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.553 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.553 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.553 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.553 INFO:teuthology.orchestra.run.smithi005.stdout: "zabbix_host": { 2024-09-12T03:04:32.553 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix_host", 2024-09-12T03:04:32.553 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.553 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.553 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.553 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2024-09-12T03:04:32.553 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.553 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.553 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.553 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.553 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.554 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.554 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.554 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.554 INFO:teuthology.orchestra.run.smithi005.stdout: "zabbix_port": { 2024-09-12T03:04:32.554 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix_port", 2024-09-12T03:04:32.554 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2024-09-12T03:04:32.554 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.554 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.554 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10051", 2024-09-12T03:04:32.554 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.554 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.554 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.554 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.554 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.555 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.555 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.555 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.555 INFO:teuthology.orchestra.run.smithi005.stdout: "zabbix_sender": { 2024-09-12T03:04:32.555 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix_sender", 2024-09-12T03:04:32.555 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2024-09-12T03:04:32.555 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2024-09-12T03:04:32.555 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2024-09-12T03:04:32.555 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-12T03:04:32.555 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2024-09-12T03:04:32.555 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2024-09-12T03:04:32.555 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2024-09-12T03:04:32.555 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2024-09-12T03:04:32.556 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2024-09-12T03:04:32.556 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2024-09-12T03:04:32.556 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2024-09-12T03:04:32.556 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:32.556 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:32.556 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:32.556 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:32.556 INFO:teuthology.orchestra.run.smithi005.stdout: "services": { 2024-09-12T03:04:32.556 INFO:teuthology.orchestra.run.smithi005.stdout: "dashboard": "https://172.21.15.5:8443/" 2024-09-12T03:04:32.556 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.556 INFO:teuthology.orchestra.run.smithi005.stdout: "always_on_modules": { 2024-09-12T03:04:32.556 INFO:teuthology.orchestra.run.smithi005.stdout: "octopus": [ 2024-09-12T03:04:32.556 INFO:teuthology.orchestra.run.smithi005.stdout: "balancer", 2024-09-12T03:04:32.556 INFO:teuthology.orchestra.run.smithi005.stdout: "crash", 2024-09-12T03:04:32.557 INFO:teuthology.orchestra.run.smithi005.stdout: "devicehealth", 2024-09-12T03:04:32.557 INFO:teuthology.orchestra.run.smithi005.stdout: "orchestrator", 2024-09-12T03:04:32.557 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_autoscaler", 2024-09-12T03:04:32.557 INFO:teuthology.orchestra.run.smithi005.stdout: "progress", 2024-09-12T03:04:32.557 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_support", 2024-09-12T03:04:32.557 INFO:teuthology.orchestra.run.smithi005.stdout: "status", 2024-09-12T03:04:32.557 INFO:teuthology.orchestra.run.smithi005.stdout: "telemetry", 2024-09-12T03:04:32.557 INFO:teuthology.orchestra.run.smithi005.stdout: "volumes" 2024-09-12T03:04:32.557 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:32.557 INFO:teuthology.orchestra.run.smithi005.stdout: "pacific": [ 2024-09-12T03:04:32.557 INFO:teuthology.orchestra.run.smithi005.stdout: "balancer", 2024-09-12T03:04:32.557 INFO:teuthology.orchestra.run.smithi005.stdout: "crash", 2024-09-12T03:04:32.557 INFO:teuthology.orchestra.run.smithi005.stdout: "devicehealth", 2024-09-12T03:04:32.557 INFO:teuthology.orchestra.run.smithi005.stdout: "orchestrator", 2024-09-12T03:04:32.558 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_autoscaler", 2024-09-12T03:04:32.558 INFO:teuthology.orchestra.run.smithi005.stdout: "progress", 2024-09-12T03:04:32.558 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_support", 2024-09-12T03:04:32.558 INFO:teuthology.orchestra.run.smithi005.stdout: "status", 2024-09-12T03:04:32.558 INFO:teuthology.orchestra.run.smithi005.stdout: "telemetry", 2024-09-12T03:04:32.558 INFO:teuthology.orchestra.run.smithi005.stdout: "volumes" 2024-09-12T03:04:32.558 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:32.558 INFO:teuthology.orchestra.run.smithi005.stdout: "quincy": [ 2024-09-12T03:04:32.558 INFO:teuthology.orchestra.run.smithi005.stdout: "balancer", 2024-09-12T03:04:32.558 INFO:teuthology.orchestra.run.smithi005.stdout: "crash", 2024-09-12T03:04:32.558 INFO:teuthology.orchestra.run.smithi005.stdout: "devicehealth", 2024-09-12T03:04:32.558 INFO:teuthology.orchestra.run.smithi005.stdout: "orchestrator", 2024-09-12T03:04:32.558 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_autoscaler", 2024-09-12T03:04:32.559 INFO:teuthology.orchestra.run.smithi005.stdout: "progress", 2024-09-12T03:04:32.559 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_support", 2024-09-12T03:04:32.559 INFO:teuthology.orchestra.run.smithi005.stdout: "status", 2024-09-12T03:04:32.559 INFO:teuthology.orchestra.run.smithi005.stdout: "telemetry", 2024-09-12T03:04:32.559 INFO:teuthology.orchestra.run.smithi005.stdout: "volumes" 2024-09-12T03:04:32.559 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:32.559 INFO:teuthology.orchestra.run.smithi005.stdout: "reef": [ 2024-09-12T03:04:32.559 INFO:teuthology.orchestra.run.smithi005.stdout: "balancer", 2024-09-12T03:04:32.559 INFO:teuthology.orchestra.run.smithi005.stdout: "crash", 2024-09-12T03:04:32.559 INFO:teuthology.orchestra.run.smithi005.stdout: "devicehealth", 2024-09-12T03:04:32.559 INFO:teuthology.orchestra.run.smithi005.stdout: "orchestrator", 2024-09-12T03:04:32.559 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_autoscaler", 2024-09-12T03:04:32.559 INFO:teuthology.orchestra.run.smithi005.stdout: "progress", 2024-09-12T03:04:32.559 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_support", 2024-09-12T03:04:32.560 INFO:teuthology.orchestra.run.smithi005.stdout: "status", 2024-09-12T03:04:32.560 INFO:teuthology.orchestra.run.smithi005.stdout: "telemetry", 2024-09-12T03:04:32.560 INFO:teuthology.orchestra.run.smithi005.stdout: "volumes" 2024-09-12T03:04:32.560 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2024-09-12T03:04:32.560 INFO:teuthology.orchestra.run.smithi005.stdout: "squid": [ 2024-09-12T03:04:32.560 INFO:teuthology.orchestra.run.smithi005.stdout: "balancer", 2024-09-12T03:04:32.560 INFO:teuthology.orchestra.run.smithi005.stdout: "crash", 2024-09-12T03:04:32.560 INFO:teuthology.orchestra.run.smithi005.stdout: "devicehealth", 2024-09-12T03:04:32.560 INFO:teuthology.orchestra.run.smithi005.stdout: "orchestrator", 2024-09-12T03:04:32.560 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_autoscaler", 2024-09-12T03:04:32.560 INFO:teuthology.orchestra.run.smithi005.stdout: "progress", 2024-09-12T03:04:32.560 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_support", 2024-09-12T03:04:32.560 INFO:teuthology.orchestra.run.smithi005.stdout: "status", 2024-09-12T03:04:32.561 INFO:teuthology.orchestra.run.smithi005.stdout: "telemetry", 2024-09-12T03:04:32.561 INFO:teuthology.orchestra.run.smithi005.stdout: "volumes" 2024-09-12T03:04:32.561 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-12T03:04:32.561 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.561 INFO:teuthology.orchestra.run.smithi005.stdout: "last_failure_osd_epoch": 3, 2024-09-12T03:04:32.561 INFO:teuthology.orchestra.run.smithi005.stdout: "active_clients": [ 2024-09-12T03:04:32.561 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-12T03:04:32.561 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "devicehealth", 2024-09-12T03:04:32.561 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-12T03:04:32.561 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-12T03:04:32.561 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-12T03:04:32.561 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:0", 2024-09-12T03:04:32.561 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2840186524 2024-09-12T03:04:32.561 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:32.562 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-12T03:04:32.562 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.562 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-12T03:04:32.562 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "libcephsqlite", 2024-09-12T03:04:32.562 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-12T03:04:32.562 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-12T03:04:32.562 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-12T03:04:32.562 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:0", 2024-09-12T03:04:32.562 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2937547841 2024-09-12T03:04:32.562 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:32.562 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-12T03:04:32.562 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.562 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-12T03:04:32.563 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rbd_support", 2024-09-12T03:04:32.563 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-12T03:04:32.563 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-12T03:04:32.563 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-12T03:04:32.563 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:0", 2024-09-12T03:04:32.563 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 3087412097 2024-09-12T03:04:32.563 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:32.563 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-12T03:04:32.563 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:32.563 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-12T03:04:32.563 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "volumes", 2024-09-12T03:04:32.563 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2024-09-12T03:04:32.563 INFO:teuthology.orchestra.run.smithi005.stdout: { 2024-09-12T03:04:32.564 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2024-09-12T03:04:32.564 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:0", 2024-09-12T03:04:32.564 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 411874225 2024-09-12T03:04:32.564 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:32.564 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-12T03:04:32.564 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:32.564 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2024-09-12T03:04:32.564 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-12T03:04:32.566 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:04:30 smithi146 ceph-mon[35011]: pgmap v523: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 170 B/s wr, 11 op/s 2024-09-12T03:04:32.566 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:04:30 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/3144379292' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-12T03:04:32.566 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:04:30 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/3595542477' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-12T03:04:32.566 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:04:30 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/3195632178' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-12T03:04:32.566 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:04:31 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/3586055389' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-12T03:04:32.566 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:04:30 smithi148 ceph-mon[35090]: pgmap v523: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 170 B/s wr, 11 op/s 2024-09-12T03:04:32.567 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:04:30 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/3144379292' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-12T03:04:32.567 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:04:30 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/3595542477' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-12T03:04:32.567 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:04:30 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/3195632178' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-12T03:04:32.567 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:04:31 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/3586055389' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-12T03:04:32.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:31.133+0000 7f8b34e27640 1 -- 172.21.15.5:0/3586055389 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f8b1c0a2b90 msgr2=0x7f8b1c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:32.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:31.133+0000 7f8b34e27640 1 --2- 172.21.15.5:0/3586055389 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f8b1c0a2b90 0x7f8b1c0a5050 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f8b200058c0 tx=0x7f8b2000a2d0 comp rx=0 tx=0).stop 2024-09-12T03:04:32.567 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:31.133+0000 7f8b34e27640 1 -- 172.21.15.5:0/3586055389 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b30100540 msgr2=0x7f8b301983e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:32.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:31.133+0000 7f8b34e27640 1 --2- 172.21.15.5:0/3586055389 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b30100540 0x7f8b301983e0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f8b1800b4a0 tx=0x7f8b1800b960 comp rx=0 tx=0).stop 2024-09-12T03:04:32.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:31.133+0000 7f8b34e27640 1 -- 172.21.15.5:0/3586055389 shutdown_connections 2024-09-12T03:04:32.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:31.133+0000 7f8b34e27640 1 --2- 172.21.15.5:0/3586055389 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f8b1c0a2b90 0x7f8b1c0a5050 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:32.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:31.133+0000 7f8b34e27640 1 --2- 172.21.15.5:0/3586055389 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b30100540 0x7f8b301983e0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:32.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:31.133+0000 7f8b34e27640 1 --2- 172.21.15.5:0/3586055389 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f8b30103110 0x7f8b3019f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:32.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:31.133+0000 7f8b34e27640 1 --2- 172.21.15.5:0/3586055389 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f8b300ffb80 0x7f8b30197ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:32.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:31.133+0000 7f8b34e27640 1 -- 172.21.15.5:0/3586055389 >> 172.21.15.5:0/3586055389 conn(0x7f8b300fb950 msgr2=0x7f8b301061a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:32.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:31.134+0000 7f8b34e27640 1 -- 172.21.15.5:0/3586055389 shutdown_connections 2024-09-12T03:04:32.568 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:31.134+0000 7f8b34e27640 1 -- 172.21.15.5:0/3586055389 wait complete. 2024-09-12T03:04:32.638 DEBUG:tasks.check_counter:Getting stats from a 2024-09-12T03:04:32.638 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph daemon mds.a perf dump 2024-09-12T03:04:32.639 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:04:31 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/3586055389' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-12T03:04:32.910 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:04:33.032 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:04:32 smithi005 ceph-mon[31197]: pgmap v524: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-12T03:04:33.165 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:04:32 smithi148 ceph-mon[35090]: pgmap v524: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-12T03:04:33.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:04:32 smithi146 ceph-mon[35011]: pgmap v524: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-09-12T03:04:33.404 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-12T03:04:33.404 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T03:04:33.405 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 979, 2024-09-12T03:04:33.405 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 979, 2024-09-12T03:04:33.405 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 295749, 2024-09-12T03:04:33.405 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 310499, 2024-09-12T03:04:33.405 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 6, 2024-09-12T03:04:33.405 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 2, 2024-09-12T03:04:33.405 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.079957651, 2024-09-12T03:04:33.405 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.024973997, 2024-09-12T03:04:33.405 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.023085588, 2024-09-12T03:04:33.405 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.024288451, 2024-09-12T03:04:33.405 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:04:33.405 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 978, 2024-09-12T03:04:33.405 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.031953286, 2024-09-12T03:04:33.406 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000032672 2024-09-12T03:04:33.406 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:33.406 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:04:33.406 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-12T03:04:33.406 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-12T03:04:33.406 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-12T03:04:33.407 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:33.407 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-09-12T03:04:33.408 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 448 2024-09-12T03:04:33.408 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:33.408 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T03:04:33.408 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 162, 2024-09-12T03:04:33.408 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 149, 2024-09-12T03:04:33.408 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 803694, 2024-09-12T03:04:33.408 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 87038, 2024-09-12T03:04:33.408 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 4, 2024-09-12T03:04:33.408 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 3, 2024-09-12T03:04:33.408 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.024028450, 2024-09-12T03:04:33.409 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.009297365, 2024-09-12T03:04:33.409 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.021031569, 2024-09-12T03:04:33.409 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.000131144, 2024-09-12T03:04:33.409 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:04:33.409 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 148, 2024-09-12T03:04:33.409 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.009061329, 2024-09-12T03:04:33.409 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000061225 2024-09-12T03:04:33.409 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:33.409 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:04:33.409 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-12T03:04:33.409 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-12T03:04:33.409 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-12T03:04:33.410 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:33.410 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 802304, 2024-09-12T03:04:33.410 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 86016 2024-09-12T03:04:33.410 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:33.410 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T03:04:33.410 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 395, 2024-09-12T03:04:33.410 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 499, 2024-09-12T03:04:33.410 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 97106, 2024-09-12T03:04:33.410 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 388332, 2024-09-12T03:04:33.410 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 5, 2024-09-12T03:04:33.410 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 4, 2024-09-12T03:04:33.410 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.040618041, 2024-09-12T03:04:33.410 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.014427932, 2024-09-12T03:04:33.410 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.009518878, 2024-09-12T03:04:33.411 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.012608564, 2024-09-12T03:04:33.411 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:04:33.411 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 497, 2024-09-12T03:04:33.411 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.016122571, 2024-09-12T03:04:33.411 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000032439 2024-09-12T03:04:33.411 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:33.411 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:04:33.411 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-12T03:04:33.411 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-12T03:04:33.411 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-12T03:04:33.411 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:33.411 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-12T03:04:33.411 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 263040 2024-09-12T03:04:33.412 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:33.412 INFO:teuthology.orchestra.run.smithi005.stdout: "cct": { 2024-09-12T03:04:33.412 INFO:teuthology.orchestra.run.smithi005.stdout: "total_workers": 1, 2024-09-12T03:04:33.412 INFO:teuthology.orchestra.run.smithi005.stdout: "unhealthy_workers": 0 2024-09-12T03:04:33.412 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:33.412 INFO:teuthology.orchestra.run.smithi005.stdout: "finisher-MDSRank": { 2024-09-12T03:04:33.412 INFO:teuthology.orchestra.run.smithi005.stdout: "queue_len": 0, 2024-09-12T03:04:33.412 INFO:teuthology.orchestra.run.smithi005.stdout: "complete_latency": { 2024-09-12T03:04:33.412 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 2270, 2024-09-12T03:04:33.412 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.055352453, 2024-09-12T03:04:33.412 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000024384 2024-09-12T03:04:33.412 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:33.412 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:33.413 INFO:teuthology.orchestra.run.smithi005.stdout: "finisher-PurgeQueue": { 2024-09-12T03:04:33.413 INFO:teuthology.orchestra.run.smithi005.stdout: "queue_len": 0, 2024-09-12T03:04:33.413 INFO:teuthology.orchestra.run.smithi005.stdout: "complete_latency": { 2024-09-12T03:04:33.413 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-12T03:04:33.413 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-12T03:04:33.413 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-12T03:04:33.413 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:33.413 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:33.413 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": { 2024-09-12T03:04:33.413 INFO:teuthology.orchestra.run.smithi005.stdout: "request": 0, 2024-09-12T03:04:33.413 INFO:teuthology.orchestra.run.smithi005.stdout: "reply": 0, 2024-09-12T03:04:33.413 INFO:teuthology.orchestra.run.smithi005.stdout: "reply_latency": { 2024-09-12T03:04:33.413 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-12T03:04:33.413 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-12T03:04:33.414 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-12T03:04:33.414 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:33.414 INFO:teuthology.orchestra.run.smithi005.stdout: "slow_reply": 0, 2024-09-12T03:04:33.414 INFO:teuthology.orchestra.run.smithi005.stdout: "forward": 0, 2024-09-12T03:04:33.414 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_fetch_complete": 0, 2024-09-12T03:04:33.414 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_fetch_keys": 0, 2024-09-12T03:04:33.414 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_commit": 0, 2024-09-12T03:04:33.414 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_split": 0, 2024-09-12T03:04:33.414 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_merge": 0, 2024-09-12T03:04:33.414 INFO:teuthology.orchestra.run.smithi005.stdout: "inodes": 0, 2024-09-12T03:04:33.414 INFO:teuthology.orchestra.run.smithi005.stdout: "inodes_top": 0, 2024-09-12T03:04:33.414 INFO:teuthology.orchestra.run.smithi005.stdout: "inodes_bottom": 0, 2024-09-12T03:04:33.414 INFO:teuthology.orchestra.run.smithi005.stdout: "inodes_pin_tail": 0, 2024-09-12T03:04:33.415 INFO:teuthology.orchestra.run.smithi005.stdout: "inodes_pinned": 0, 2024-09-12T03:04:33.415 INFO:teuthology.orchestra.run.smithi005.stdout: "inodes_expired": 0, 2024-09-12T03:04:33.415 INFO:teuthology.orchestra.run.smithi005.stdout: "inodes_with_caps": 0, 2024-09-12T03:04:33.415 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": 0, 2024-09-12T03:04:33.415 INFO:teuthology.orchestra.run.smithi005.stdout: "subtrees": 1, 2024-09-12T03:04:33.415 INFO:teuthology.orchestra.run.smithi005.stdout: "traverse": 0, 2024-09-12T03:04:33.415 INFO:teuthology.orchestra.run.smithi005.stdout: "traverse_hit": 0, 2024-09-12T03:04:33.415 INFO:teuthology.orchestra.run.smithi005.stdout: "traverse_forward": 0, 2024-09-12T03:04:33.415 INFO:teuthology.orchestra.run.smithi005.stdout: "traverse_discover": 0, 2024-09-12T03:04:33.415 INFO:teuthology.orchestra.run.smithi005.stdout: "traverse_dir_fetch": 0, 2024-09-12T03:04:33.415 INFO:teuthology.orchestra.run.smithi005.stdout: "traverse_remote_ino": 0, 2024-09-12T03:04:33.415 INFO:teuthology.orchestra.run.smithi005.stdout: "traverse_lock": 0, 2024-09-12T03:04:33.415 INFO:teuthology.orchestra.run.smithi005.stdout: "load_cent": 0, 2024-09-12T03:04:33.416 INFO:teuthology.orchestra.run.smithi005.stdout: "q": 0, 2024-09-12T03:04:33.416 INFO:teuthology.orchestra.run.smithi005.stdout: "exported": 0, 2024-09-12T03:04:33.416 INFO:teuthology.orchestra.run.smithi005.stdout: "exported_inodes": 0, 2024-09-12T03:04:33.416 INFO:teuthology.orchestra.run.smithi005.stdout: "imported": 0, 2024-09-12T03:04:33.416 INFO:teuthology.orchestra.run.smithi005.stdout: "imported_inodes": 0, 2024-09-12T03:04:33.416 INFO:teuthology.orchestra.run.smithi005.stdout: "openino_dir_fetch": 0, 2024-09-12T03:04:33.416 INFO:teuthology.orchestra.run.smithi005.stdout: "openino_backtrace_fetch": 0, 2024-09-12T03:04:33.416 INFO:teuthology.orchestra.run.smithi005.stdout: "openino_peer_discover": 0, 2024-09-12T03:04:33.416 INFO:teuthology.orchestra.run.smithi005.stdout: "root_rfiles": 0, 2024-09-12T03:04:33.416 INFO:teuthology.orchestra.run.smithi005.stdout: "root_rbytes": 0, 2024-09-12T03:04:33.416 INFO:teuthology.orchestra.run.smithi005.stdout: "root_rsnaps": 0, 2024-09-12T03:04:33.416 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_backtrace_fetch": 0, 2024-09-12T03:04:33.416 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_set_tag": 0, 2024-09-12T03:04:33.416 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_backtrace_repaired": 0, 2024-09-12T03:04:33.417 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_inotable_repaired": 0, 2024-09-12T03:04:33.417 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_dir_inodes": 0, 2024-09-12T03:04:33.417 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_dir_base_inodes": 0, 2024-09-12T03:04:33.417 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_dirfrag_rstats": 0, 2024-09-12T03:04:33.417 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_file_inodes": 0, 2024-09-12T03:04:33.417 INFO:teuthology.orchestra.run.smithi005.stdout: "handle_inode_file_caps": 0, 2024-09-12T03:04:33.417 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_cap_op_revoke": 0, 2024-09-12T03:04:33.417 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_cap_op_grant": 0, 2024-09-12T03:04:33.417 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_cap_op_trunc": 0, 2024-09-12T03:04:33.417 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-12T03:04:33.417 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-12T03:04:33.417 INFO:teuthology.orchestra.run.smithi005.stdout: "handle_client_caps": 0, 2024-09-12T03:04:33.417 INFO:teuthology.orchestra.run.smithi005.stdout: "handle_client_caps_dirty": 0, 2024-09-12T03:04:33.418 INFO:teuthology.orchestra.run.smithi005.stdout: "handle_client_cap_release": 0, 2024-09-12T03:04:33.418 INFO:teuthology.orchestra.run.smithi005.stdout: "process_request_cap_release": 0 2024-09-12T03:04:33.418 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:33.418 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_cache": { 2024-09-12T03:04:33.418 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_update": 0, 2024-09-12T03:04:33.418 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_update_receipt": 0, 2024-09-12T03:04:33.418 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_try_discover": 0, 2024-09-12T03:04:33.418 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_send_discover": 0, 2024-09-12T03:04:33.418 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_handle_discover": 0, 2024-09-12T03:04:33.418 INFO:teuthology.orchestra.run.smithi005.stdout: "num_strays": 0, 2024-09-12T03:04:33.418 INFO:teuthology.orchestra.run.smithi005.stdout: "num_strays_delayed": 0, 2024-09-12T03:04:33.418 INFO:teuthology.orchestra.run.smithi005.stdout: "num_strays_enqueuing": 0, 2024-09-12T03:04:33.418 INFO:teuthology.orchestra.run.smithi005.stdout: "strays_created": 0, 2024-09-12T03:04:33.418 INFO:teuthology.orchestra.run.smithi005.stdout: "strays_enqueued": 0, 2024-09-12T03:04:33.419 INFO:teuthology.orchestra.run.smithi005.stdout: "strays_reintegrated": 0, 2024-09-12T03:04:33.419 INFO:teuthology.orchestra.run.smithi005.stdout: "strays_migrated": 0, 2024-09-12T03:04:33.419 INFO:teuthology.orchestra.run.smithi005.stdout: "num_recovering_processing": 0, 2024-09-12T03:04:33.419 INFO:teuthology.orchestra.run.smithi005.stdout: "num_recovering_enqueued": 0, 2024-09-12T03:04:33.419 INFO:teuthology.orchestra.run.smithi005.stdout: "num_recovering_prioritized": 0, 2024-09-12T03:04:33.419 INFO:teuthology.orchestra.run.smithi005.stdout: "recovery_started": 0, 2024-09-12T03:04:33.419 INFO:teuthology.orchestra.run.smithi005.stdout: "recovery_completed": 0, 2024-09-12T03:04:33.419 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_quiesce_path": 0, 2024-09-12T03:04:33.419 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_quiesce_inode": 0, 2024-09-12T03:04:33.419 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_enqueue_scrub": 0, 2024-09-12T03:04:33.419 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_exportdir": 0, 2024-09-12T03:04:33.419 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_flush": 0, 2024-09-12T03:04:33.419 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_fragmentdir": 0, 2024-09-12T03:04:33.419 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_fragstats": 0, 2024-09-12T03:04:33.420 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_inodestats": 0 2024-09-12T03:04:33.420 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:33.420 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_log": { 2024-09-12T03:04:33.420 INFO:teuthology.orchestra.run.smithi005.stdout: "evlrg": 0, 2024-09-12T03:04:33.420 INFO:teuthology.orchestra.run.smithi005.stdout: "evadd": 0, 2024-09-12T03:04:33.420 INFO:teuthology.orchestra.run.smithi005.stdout: "evex": 0, 2024-09-12T03:04:33.420 INFO:teuthology.orchestra.run.smithi005.stdout: "evtrm": 0, 2024-09-12T03:04:33.420 INFO:teuthology.orchestra.run.smithi005.stdout: "ev": 5, 2024-09-12T03:04:33.420 INFO:teuthology.orchestra.run.smithi005.stdout: "evexg": 0, 2024-09-12T03:04:33.420 INFO:teuthology.orchestra.run.smithi005.stdout: "evexd": 0, 2024-09-12T03:04:33.420 INFO:teuthology.orchestra.run.smithi005.stdout: "segadd": 0, 2024-09-12T03:04:33.420 INFO:teuthology.orchestra.run.smithi005.stdout: "segex": 0, 2024-09-12T03:04:33.420 INFO:teuthology.orchestra.run.smithi005.stdout: "segtrm": 0, 2024-09-12T03:04:33.421 INFO:teuthology.orchestra.run.smithi005.stdout: "seg": 2, 2024-09-12T03:04:33.421 INFO:teuthology.orchestra.run.smithi005.stdout: "segmjr": 2, 2024-09-12T03:04:33.421 INFO:teuthology.orchestra.run.smithi005.stdout: "segexg": 0, 2024-09-12T03:04:33.421 INFO:teuthology.orchestra.run.smithi005.stdout: "segexd": 0, 2024-09-12T03:04:33.421 INFO:teuthology.orchestra.run.smithi005.stdout: "expos": 4194304, 2024-09-12T03:04:33.421 INFO:teuthology.orchestra.run.smithi005.stdout: "wrpos": 4195734, 2024-09-12T03:04:33.421 INFO:teuthology.orchestra.run.smithi005.stdout: "rdpos": 4195426, 2024-09-12T03:04:33.421 INFO:teuthology.orchestra.run.smithi005.stdout: "jlat": { 2024-09-12T03:04:33.421 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-12T03:04:33.421 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-12T03:04:33.421 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-12T03:04:33.421 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:33.421 INFO:teuthology.orchestra.run.smithi005.stdout: "replayed": 5 2024-09-12T03:04:33.421 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:33.422 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_mem": { 2024-09-12T03:04:33.422 INFO:teuthology.orchestra.run.smithi005.stdout: "ino": 3, 2024-09-12T03:04:33.422 INFO:teuthology.orchestra.run.smithi005.stdout: "ino+": 3, 2024-09-12T03:04:33.422 INFO:teuthology.orchestra.run.smithi005.stdout: "ino-": 0, 2024-09-12T03:04:33.422 INFO:teuthology.orchestra.run.smithi005.stdout: "dir": 1, 2024-09-12T03:04:33.422 INFO:teuthology.orchestra.run.smithi005.stdout: "dir+": 1, 2024-09-12T03:04:33.422 INFO:teuthology.orchestra.run.smithi005.stdout: "dir-": 0, 2024-09-12T03:04:33.422 INFO:teuthology.orchestra.run.smithi005.stdout: "dn": 0, 2024-09-12T03:04:33.422 INFO:teuthology.orchestra.run.smithi005.stdout: "dn+": 0, 2024-09-12T03:04:33.422 INFO:teuthology.orchestra.run.smithi005.stdout: "dn-": 0, 2024-09-12T03:04:33.422 INFO:teuthology.orchestra.run.smithi005.stdout: "cap": 0, 2024-09-12T03:04:33.422 INFO:teuthology.orchestra.run.smithi005.stdout: "cap+": 0, 2024-09-12T03:04:33.422 INFO:teuthology.orchestra.run.smithi005.stdout: "cap-": 0, 2024-09-12T03:04:33.422 INFO:teuthology.orchestra.run.smithi005.stdout: "rss": 50472, 2024-09-12T03:04:33.423 INFO:teuthology.orchestra.run.smithi005.stdout: "heap": 215328 2024-09-12T03:04:33.423 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:33.423 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_server": { 2024-09-12T03:04:33.423 INFO:teuthology.orchestra.run.smithi005.stdout: "dispatch_client_request": 0, 2024-09-12T03:04:33.423 INFO:teuthology.orchestra.run.smithi005.stdout: "dispatch_server_request": 0, 2024-09-12T03:04:33.423 INFO:teuthology.orchestra.run.smithi005.stdout: "handle_client_request": 0, 2024-09-12T03:04:33.423 INFO:teuthology.orchestra.run.smithi005.stdout: "handle_client_session": 0, 2024-09-12T03:04:33.423 INFO:teuthology.orchestra.run.smithi005.stdout: "handle_peer_request": 0, 2024-09-12T03:04:33.423 INFO:teuthology.orchestra.run.smithi005.stdout: "req_create_latency": { 2024-09-12T03:04:33.423 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-12T03:04:33.423 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-12T03:04:33.423 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-12T03:04:33.423 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:33.423 INFO:teuthology.orchestra.run.smithi005.stdout: "req_getattr_latency": { 2024-09-12T03:04:33.424 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-12T03:04:33.424 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-12T03:04:33.424 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-12T03:04:33.424 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:33.424 INFO:teuthology.orchestra.run.smithi005.stdout: "req_getfilelock_latency": { 2024-09-12T03:04:33.424 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-12T03:04:33.424 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-12T03:04:33.424 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-12T03:04:33.424 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:33.425 INFO:teuthology.orchestra.run.smithi005.stdout: "req_link_latency": { 2024-09-12T03:04:33.425 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-12T03:04:33.425 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-12T03:04:33.425 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-12T03:04:33.425 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:33.425 INFO:teuthology.orchestra.run.smithi005.stdout: "req_lookup_latency": { 2024-09-12T03:04:33.425 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-12T03:04:33.425 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-12T03:04:33.425 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-12T03:04:33.425 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:33.425 INFO:teuthology.orchestra.run.smithi005.stdout: "req_lookuphash_latency": { 2024-09-12T03:04:33.425 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-12T03:04:33.425 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-12T03:04:33.426 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-12T03:04:33.426 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:33.426 INFO:teuthology.orchestra.run.smithi005.stdout: "req_lookupino_latency": { 2024-09-12T03:04:33.426 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-12T03:04:33.426 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-12T03:04:33.426 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-12T03:04:33.426 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:33.426 INFO:teuthology.orchestra.run.smithi005.stdout: "req_lookupname_latency": { 2024-09-12T03:04:33.426 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-12T03:04:33.426 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-12T03:04:33.426 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-12T03:04:33.426 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:33.426 INFO:teuthology.orchestra.run.smithi005.stdout: "req_lookupparent_latency": { 2024-09-12T03:04:33.427 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-12T03:04:33.427 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-12T03:04:33.427 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-12T03:04:33.427 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:33.427 INFO:teuthology.orchestra.run.smithi005.stdout: "req_lookupsnap_latency": { 2024-09-12T03:04:33.427 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-12T03:04:33.427 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-12T03:04:33.427 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-12T03:04:33.427 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:33.427 INFO:teuthology.orchestra.run.smithi005.stdout: "req_lssnap_latency": { 2024-09-12T03:04:33.427 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-12T03:04:33.427 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-12T03:04:33.428 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-12T03:04:33.428 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:33.428 INFO:teuthology.orchestra.run.smithi005.stdout: "req_mkdir_latency": { 2024-09-12T03:04:33.428 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-12T03:04:33.429 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-12T03:04:33.429 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-12T03:04:33.429 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:33.429 INFO:teuthology.orchestra.run.smithi005.stdout: "req_mknod_latency": { 2024-09-12T03:04:33.429 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-12T03:04:33.429 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-12T03:04:33.429 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-12T03:04:33.429 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:33.429 INFO:teuthology.orchestra.run.smithi005.stdout: "req_mksnap_latency": { 2024-09-12T03:04:33.429 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-12T03:04:33.429 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-12T03:04:33.429 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-12T03:04:33.430 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:33.430 INFO:teuthology.orchestra.run.smithi005.stdout: "req_open_latency": { 2024-09-12T03:04:33.430 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-12T03:04:33.430 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-12T03:04:33.430 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-12T03:04:33.430 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:33.430 INFO:teuthology.orchestra.run.smithi005.stdout: "req_readdir_latency": { 2024-09-12T03:04:33.430 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-12T03:04:33.430 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-12T03:04:33.430 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-12T03:04:33.430 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:33.430 INFO:teuthology.orchestra.run.smithi005.stdout: "req_rename_latency": { 2024-09-12T03:04:33.430 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-12T03:04:33.430 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-12T03:04:33.431 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-12T03:04:33.431 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:33.431 INFO:teuthology.orchestra.run.smithi005.stdout: "req_renamesnap_latency": { 2024-09-12T03:04:33.431 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-12T03:04:33.431 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-12T03:04:33.431 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-12T03:04:33.431 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:33.431 INFO:teuthology.orchestra.run.smithi005.stdout: "req_snapdiff_latency": { 2024-09-12T03:04:33.431 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-12T03:04:33.431 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-12T03:04:33.431 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-12T03:04:33.431 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:33.431 INFO:teuthology.orchestra.run.smithi005.stdout: "req_rmdir_latency": { 2024-09-12T03:04:33.431 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-12T03:04:33.432 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-12T03:04:33.432 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-12T03:04:33.432 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:33.432 INFO:teuthology.orchestra.run.smithi005.stdout: "req_rmsnap_latency": { 2024-09-12T03:04:33.432 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-12T03:04:33.432 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-12T03:04:33.432 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-12T03:04:33.432 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:33.432 INFO:teuthology.orchestra.run.smithi005.stdout: "req_rmxattr_latency": { 2024-09-12T03:04:33.432 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-12T03:04:33.432 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-12T03:04:33.432 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-12T03:04:33.432 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:33.432 INFO:teuthology.orchestra.run.smithi005.stdout: "req_setattr_latency": { 2024-09-12T03:04:33.433 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-12T03:04:33.433 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-12T03:04:33.433 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-12T03:04:33.433 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:33.433 INFO:teuthology.orchestra.run.smithi005.stdout: "req_setdirlayout_latency": { 2024-09-12T03:04:33.433 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-12T03:04:33.433 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-12T03:04:33.433 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-12T03:04:33.433 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:33.433 INFO:teuthology.orchestra.run.smithi005.stdout: "req_setfilelock_latency": { 2024-09-12T03:04:33.433 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-12T03:04:33.433 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-12T03:04:33.433 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-12T03:04:33.434 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:33.434 INFO:teuthology.orchestra.run.smithi005.stdout: "req_setlayout_latency": { 2024-09-12T03:04:33.434 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-12T03:04:33.434 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-12T03:04:33.434 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-12T03:04:33.434 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:33.434 INFO:teuthology.orchestra.run.smithi005.stdout: "req_setxattr_latency": { 2024-09-12T03:04:33.434 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-12T03:04:33.434 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-12T03:04:33.434 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-12T03:04:33.434 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:33.434 INFO:teuthology.orchestra.run.smithi005.stdout: "req_symlink_latency": { 2024-09-12T03:04:33.434 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-12T03:04:33.434 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-12T03:04:33.435 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-12T03:04:33.435 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:33.435 INFO:teuthology.orchestra.run.smithi005.stdout: "req_unlink_latency": { 2024-09-12T03:04:33.435 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-12T03:04:33.435 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-12T03:04:33.435 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-12T03:04:33.435 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:33.435 INFO:teuthology.orchestra.run.smithi005.stdout: "cap_revoke_eviction": 0, 2024-09-12T03:04:33.435 INFO:teuthology.orchestra.run.smithi005.stdout: "cap_acquisition_throttle": 0, 2024-09-12T03:04:33.435 INFO:teuthology.orchestra.run.smithi005.stdout: "req_getvxattr_latency": { 2024-09-12T03:04:33.435 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-12T03:04:33.435 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-12T03:04:33.435 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-12T03:04:33.436 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:33.436 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:33.436 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_sessions": { 2024-09-12T03:04:33.436 INFO:teuthology.orchestra.run.smithi005.stdout: "session_count": 1, 2024-09-12T03:04:33.436 INFO:teuthology.orchestra.run.smithi005.stdout: "session_add": 2, 2024-09-12T03:04:33.436 INFO:teuthology.orchestra.run.smithi005.stdout: "session_remove": 1, 2024-09-12T03:04:33.436 INFO:teuthology.orchestra.run.smithi005.stdout: "sessions_open": 1, 2024-09-12T03:04:33.436 INFO:teuthology.orchestra.run.smithi005.stdout: "sessions_stale": 0, 2024-09-12T03:04:33.436 INFO:teuthology.orchestra.run.smithi005.stdout: "total_load": 0, 2024-09-12T03:04:33.436 INFO:teuthology.orchestra.run.smithi005.stdout: "average_load": 0, 2024-09-12T03:04:33.436 INFO:teuthology.orchestra.run.smithi005.stdout: "avg_session_uptime": 1387, 2024-09-12T03:04:33.436 INFO:teuthology.orchestra.run.smithi005.stdout: "mdthresh_evicted": 0 2024-09-12T03:04:33.436 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:33.437 INFO:teuthology.orchestra.run.smithi005.stdout: "mempool": { 2024-09-12T03:04:33.437 INFO:teuthology.orchestra.run.smithi005.stdout: "bloom_filter_bytes": 0, 2024-09-12T03:04:33.437 INFO:teuthology.orchestra.run.smithi005.stdout: "bloom_filter_items": 0, 2024-09-12T03:04:33.437 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T03:04:33.437 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_alloc_items": 0, 2024-09-12T03:04:33.437 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T03:04:33.437 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_data_items": 0, 2024-09-12T03:04:33.437 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T03:04:33.437 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T03:04:33.437 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T03:04:33.437 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T03:04:33.437 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T03:04:33.438 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_other_items": 0, 2024-09-12T03:04:33.438 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T03:04:33.438 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T03:04:33.438 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_extent_bytes": 0, 2024-09-12T03:04:33.438 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_extent_items": 0, 2024-09-12T03:04:33.438 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_blob_bytes": 0, 2024-09-12T03:04:33.438 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_blob_items": 0, 2024-09-12T03:04:33.438 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T03:04:33.438 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T03:04:33.438 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T03:04:33.438 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T03:04:33.438 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T03:04:33.438 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_fsck_items": 0, 2024-09-12T03:04:33.438 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_txc_bytes": 0, 2024-09-12T03:04:33.439 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_txc_items": 0, 2024-09-12T03:04:33.439 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T03:04:33.439 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T03:04:33.439 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_bytes": 0, 2024-09-12T03:04:33.439 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_items": 0, 2024-09-12T03:04:33.439 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_bytes": 0, 2024-09-12T03:04:33.439 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_items": 0, 2024-09-12T03:04:33.439 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T03:04:33.439 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_reader_items": 0, 2024-09-12T03:04:33.439 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T03:04:33.439 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_writer_items": 0, 2024-09-12T03:04:33.439 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_anon_bytes": 197480, 2024-09-12T03:04:33.439 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_anon_items": 55, 2024-09-12T03:04:33.439 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_meta_bytes": 0, 2024-09-12T03:04:33.440 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_meta_items": 0, 2024-09-12T03:04:33.440 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_bytes": 0, 2024-09-12T03:04:33.440 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_items": 0, 2024-09-12T03:04:33.440 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mapbl_bytes": 0, 2024-09-12T03:04:33.440 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mapbl_items": 0, 2024-09-12T03:04:33.440 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pglog_bytes": 0, 2024-09-12T03:04:33.440 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pglog_items": 0, 2024-09-12T03:04:33.440 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_bytes": 7496, 2024-09-12T03:04:33.440 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_items": 154, 2024-09-12T03:04:33.440 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T03:04:33.440 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_mapping_items": 0, 2024-09-12T03:04:33.440 INFO:teuthology.orchestra.run.smithi005.stdout: "pgmap_bytes": 0, 2024-09-12T03:04:33.440 INFO:teuthology.orchestra.run.smithi005.stdout: "pgmap_items": 0, 2024-09-12T03:04:33.441 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_co_bytes": 8512, 2024-09-12T03:04:33.441 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_co_items": 13, 2024-09-12T03:04:33.441 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_1_bytes": 0, 2024-09-12T03:04:33.441 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_1_items": 0, 2024-09-12T03:04:33.441 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_2_bytes": 0, 2024-09-12T03:04:33.441 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_2_items": 0 2024-09-12T03:04:33.441 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:33.441 INFO:teuthology.orchestra.run.smithi005.stdout: "objecter": { 2024-09-12T03:04:33.441 INFO:teuthology.orchestra.run.smithi005.stdout: "op_active": 0, 2024-09-12T03:04:33.441 INFO:teuthology.orchestra.run.smithi005.stdout: "op_laggy": 0, 2024-09-12T03:04:33.441 INFO:teuthology.orchestra.run.smithi005.stdout: "op_send": 1372, 2024-09-12T03:04:33.441 INFO:teuthology.orchestra.run.smithi005.stdout: "op_send_bytes": 21, 2024-09-12T03:04:33.442 INFO:teuthology.orchestra.run.smithi005.stdout: "op_resend": 3, 2024-09-12T03:04:33.442 INFO:teuthology.orchestra.run.smithi005.stdout: "op_reply": 1372, 2024-09-12T03:04:33.442 INFO:teuthology.orchestra.run.smithi005.stdout: "op_latency": { 2024-09-12T03:04:33.442 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 1372, 2024-09-12T03:04:33.442 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.713984811, 2024-09-12T03:04:33.442 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000520397 2024-09-12T03:04:33.442 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:33.443 INFO:teuthology.orchestra.run.smithi005.stdout: "op_inflight": 0, 2024-09-12T03:04:33.443 INFO:teuthology.orchestra.run.smithi005.stdout: "oplen_avg": { 2024-09-12T03:04:33.443 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 1372, 2024-09-12T03:04:33.443 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 1373 2024-09-12T03:04:33.443 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:33.443 INFO:teuthology.orchestra.run.smithi005.stdout: "op": 1372, 2024-09-12T03:04:33.443 INFO:teuthology.orchestra.run.smithi005.stdout: "op_r": 1372, 2024-09-12T03:04:33.443 INFO:teuthology.orchestra.run.smithi005.stdout: "op_w": 0, 2024-09-12T03:04:33.443 INFO:teuthology.orchestra.run.smithi005.stdout: "op_rmw": 0, 2024-09-12T03:04:33.443 INFO:teuthology.orchestra.run.smithi005.stdout: "op_pg": 0, 2024-09-12T03:04:33.443 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_stat": 878, 2024-09-12T03:04:33.444 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_create": 0, 2024-09-12T03:04:33.444 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_read": 492, 2024-09-12T03:04:33.444 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_write": 0, 2024-09-12T03:04:33.444 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_writefull": 0, 2024-09-12T03:04:33.444 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_writesame": 0, 2024-09-12T03:04:33.444 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_append": 0, 2024-09-12T03:04:33.444 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_zero": 0, 2024-09-12T03:04:33.444 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_truncate": 0, 2024-09-12T03:04:33.444 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_delete": 0, 2024-09-12T03:04:33.444 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_mapext": 0, 2024-09-12T03:04:33.444 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_sparse_read": 0, 2024-09-12T03:04:33.444 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_clonerange": 0, 2024-09-12T03:04:33.444 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_getxattr": 1, 2024-09-12T03:04:33.444 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_setxattr": 0, 2024-09-12T03:04:33.445 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_cmpxattr": 0, 2024-09-12T03:04:33.445 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_rmxattr": 0, 2024-09-12T03:04:33.445 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_resetxattrs": 0, 2024-09-12T03:04:33.445 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_call": 0, 2024-09-12T03:04:33.445 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_watch": 0, 2024-09-12T03:04:33.445 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_notify": 0, 2024-09-12T03:04:33.445 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_src_cmpxattr": 0, 2024-09-12T03:04:33.445 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_pgls": 0, 2024-09-12T03:04:33.446 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_pgls_filter": 0, 2024-09-12T03:04:33.446 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_other": 0, 2024-09-12T03:04:33.446 INFO:teuthology.orchestra.run.smithi005.stdout: "linger_active": 0, 2024-09-12T03:04:33.446 INFO:teuthology.orchestra.run.smithi005.stdout: "linger_send": 0, 2024-09-12T03:04:33.446 INFO:teuthology.orchestra.run.smithi005.stdout: "linger_resend": 0, 2024-09-12T03:04:33.446 INFO:teuthology.orchestra.run.smithi005.stdout: "linger_ping": 0, 2024-09-12T03:04:33.446 INFO:teuthology.orchestra.run.smithi005.stdout: "poolop_active": 0, 2024-09-12T03:04:33.446 INFO:teuthology.orchestra.run.smithi005.stdout: "poolop_send": 0, 2024-09-12T03:04:33.446 INFO:teuthology.orchestra.run.smithi005.stdout: "poolop_resend": 0, 2024-09-12T03:04:33.446 INFO:teuthology.orchestra.run.smithi005.stdout: "poolstat_active": 0, 2024-09-12T03:04:33.447 INFO:teuthology.orchestra.run.smithi005.stdout: "poolstat_send": 0, 2024-09-12T03:04:33.447 INFO:teuthology.orchestra.run.smithi005.stdout: "poolstat_resend": 0, 2024-09-12T03:04:33.447 INFO:teuthology.orchestra.run.smithi005.stdout: "statfs_active": 0, 2024-09-12T03:04:33.447 INFO:teuthology.orchestra.run.smithi005.stdout: "statfs_send": 0, 2024-09-12T03:04:33.447 INFO:teuthology.orchestra.run.smithi005.stdout: "statfs_resend": 0, 2024-09-12T03:04:33.447 INFO:teuthology.orchestra.run.smithi005.stdout: "command_active": 0, 2024-09-12T03:04:33.447 INFO:teuthology.orchestra.run.smithi005.stdout: "command_send": 0, 2024-09-12T03:04:33.447 INFO:teuthology.orchestra.run.smithi005.stdout: "command_resend": 0, 2024-09-12T03:04:33.447 INFO:teuthology.orchestra.run.smithi005.stdout: "map_epoch": 83, 2024-09-12T03:04:33.447 INFO:teuthology.orchestra.run.smithi005.stdout: "map_full": 0, 2024-09-12T03:04:33.447 INFO:teuthology.orchestra.run.smithi005.stdout: "map_inc": 2, 2024-09-12T03:04:33.447 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_sessions": 6, 2024-09-12T03:04:33.447 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_session_open": 6, 2024-09-12T03:04:33.447 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_session_close": 0, 2024-09-12T03:04:33.448 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_laggy": 0, 2024-09-12T03:04:33.448 INFO:teuthology.orchestra.run.smithi005.stdout: "omap_wr": 0, 2024-09-12T03:04:33.448 INFO:teuthology.orchestra.run.smithi005.stdout: "omap_rd": 2, 2024-09-12T03:04:33.448 INFO:teuthology.orchestra.run.smithi005.stdout: "omap_del": 0 2024-09-12T03:04:33.448 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:33.448 INFO:teuthology.orchestra.run.smithi005.stdout: "oft": { 2024-09-12T03:04:33.448 INFO:teuthology.orchestra.run.smithi005.stdout: "omap_total_objs": 0, 2024-09-12T03:04:33.448 INFO:teuthology.orchestra.run.smithi005.stdout: "omap_total_kv_pairs": 0, 2024-09-12T03:04:33.448 INFO:teuthology.orchestra.run.smithi005.stdout: "omap_total_updates": 0, 2024-09-12T03:04:33.448 INFO:teuthology.orchestra.run.smithi005.stdout: "omap_total_removes": 0 2024-09-12T03:04:33.448 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:33.448 INFO:teuthology.orchestra.run.smithi005.stdout: "osd-slow-ops": { 2024-09-12T03:04:33.448 INFO:teuthology.orchestra.run.smithi005.stdout: "slow_ops_count": 0 2024-09-12T03:04:33.448 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:33.449 INFO:teuthology.orchestra.run.smithi005.stdout: "purge_queue": { 2024-09-12T03:04:33.449 INFO:teuthology.orchestra.run.smithi005.stdout: "pq_executing_ops": 0, 2024-09-12T03:04:33.449 INFO:teuthology.orchestra.run.smithi005.stdout: "pq_executing_ops_high_water": 0, 2024-09-12T03:04:33.449 INFO:teuthology.orchestra.run.smithi005.stdout: "pq_executing": 0, 2024-09-12T03:04:33.449 INFO:teuthology.orchestra.run.smithi005.stdout: "pq_executing_high_water": 0, 2024-09-12T03:04:33.449 INFO:teuthology.orchestra.run.smithi005.stdout: "pq_executed_ops": 0, 2024-09-12T03:04:33.449 INFO:teuthology.orchestra.run.smithi005.stdout: "pq_executed": 0, 2024-09-12T03:04:33.449 INFO:teuthology.orchestra.run.smithi005.stdout: "pq_item_in_journal": 0 2024-09-12T03:04:33.449 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:33.449 INFO:teuthology.orchestra.run.smithi005.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T03:04:33.449 INFO:teuthology.orchestra.run.smithi005.stdout: "val": 0, 2024-09-12T03:04:33.449 INFO:teuthology.orchestra.run.smithi005.stdout: "max": 104857600, 2024-09-12T03:04:33.449 INFO:teuthology.orchestra.run.smithi005.stdout: "get_started": 0, 2024-09-12T03:04:33.449 INFO:teuthology.orchestra.run.smithi005.stdout: "get": 1532, 2024-09-12T03:04:33.450 INFO:teuthology.orchestra.run.smithi005.stdout: "get_sum": 1047548, 2024-09-12T03:04:33.450 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_fail": 0, 2024-09-12T03:04:33.450 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_success": 1532, 2024-09-12T03:04:33.450 INFO:teuthology.orchestra.run.smithi005.stdout: "take": 0, 2024-09-12T03:04:33.450 INFO:teuthology.orchestra.run.smithi005.stdout: "take_sum": 0, 2024-09-12T03:04:33.450 INFO:teuthology.orchestra.run.smithi005.stdout: "put": 1532, 2024-09-12T03:04:33.450 INFO:teuthology.orchestra.run.smithi005.stdout: "put_sum": 1047548, 2024-09-12T03:04:33.450 INFO:teuthology.orchestra.run.smithi005.stdout: "wait": { 2024-09-12T03:04:33.450 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-12T03:04:33.450 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-12T03:04:33.450 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-12T03:04:33.450 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:33.450 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:33.450 INFO:teuthology.orchestra.run.smithi005.stdout: "throttle-objecter_bytes": { 2024-09-12T03:04:33.451 INFO:teuthology.orchestra.run.smithi005.stdout: "val": 0, 2024-09-12T03:04:33.451 INFO:teuthology.orchestra.run.smithi005.stdout: "max": 104857600, 2024-09-12T03:04:33.451 INFO:teuthology.orchestra.run.smithi005.stdout: "get_started": 0, 2024-09-12T03:04:33.451 INFO:teuthology.orchestra.run.smithi005.stdout: "get": 0, 2024-09-12T03:04:33.451 INFO:teuthology.orchestra.run.smithi005.stdout: "get_sum": 0, 2024-09-12T03:04:33.451 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_fail": 0, 2024-09-12T03:04:33.451 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_success": 0, 2024-09-12T03:04:33.451 INFO:teuthology.orchestra.run.smithi005.stdout: "take": 1372, 2024-09-12T03:04:33.451 INFO:teuthology.orchestra.run.smithi005.stdout: "take_sum": 1435, 2024-09-12T03:04:33.451 INFO:teuthology.orchestra.run.smithi005.stdout: "put": 1372, 2024-09-12T03:04:33.451 INFO:teuthology.orchestra.run.smithi005.stdout: "put_sum": 1435, 2024-09-12T03:04:33.451 INFO:teuthology.orchestra.run.smithi005.stdout: "wait": { 2024-09-12T03:04:33.451 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-12T03:04:33.452 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-12T03:04:33.452 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-12T03:04:33.452 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:33.452 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:33.452 INFO:teuthology.orchestra.run.smithi005.stdout: "throttle-objecter_ops": { 2024-09-12T03:04:33.452 INFO:teuthology.orchestra.run.smithi005.stdout: "val": 0, 2024-09-12T03:04:33.452 INFO:teuthology.orchestra.run.smithi005.stdout: "max": 1024, 2024-09-12T03:04:33.452 INFO:teuthology.orchestra.run.smithi005.stdout: "get_started": 0, 2024-09-12T03:04:33.452 INFO:teuthology.orchestra.run.smithi005.stdout: "get": 0, 2024-09-12T03:04:33.452 INFO:teuthology.orchestra.run.smithi005.stdout: "get_sum": 0, 2024-09-12T03:04:33.452 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_fail": 0, 2024-09-12T03:04:33.453 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_success": 0, 2024-09-12T03:04:33.453 INFO:teuthology.orchestra.run.smithi005.stdout: "take": 1372, 2024-09-12T03:04:33.453 INFO:teuthology.orchestra.run.smithi005.stdout: "take_sum": 1372, 2024-09-12T03:04:33.453 INFO:teuthology.orchestra.run.smithi005.stdout: "put": 1372, 2024-09-12T03:04:33.453 INFO:teuthology.orchestra.run.smithi005.stdout: "put_sum": 1372, 2024-09-12T03:04:33.453 INFO:teuthology.orchestra.run.smithi005.stdout: "wait": { 2024-09-12T03:04:33.453 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-12T03:04:33.453 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-12T03:04:33.453 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-12T03:04:33.453 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:33.453 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:33.453 INFO:teuthology.orchestra.run.smithi005.stdout: "throttle-write_buf_throttle": { 2024-09-12T03:04:33.453 INFO:teuthology.orchestra.run.smithi005.stdout: "val": 0, 2024-09-12T03:04:33.453 INFO:teuthology.orchestra.run.smithi005.stdout: "max": 3758096384, 2024-09-12T03:04:33.454 INFO:teuthology.orchestra.run.smithi005.stdout: "get_started": 0, 2024-09-12T03:04:33.454 INFO:teuthology.orchestra.run.smithi005.stdout: "get": 0, 2024-09-12T03:04:33.454 INFO:teuthology.orchestra.run.smithi005.stdout: "get_sum": 0, 2024-09-12T03:04:33.454 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_fail": 0, 2024-09-12T03:04:33.454 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_success": 0, 2024-09-12T03:04:33.454 INFO:teuthology.orchestra.run.smithi005.stdout: "take": 0, 2024-09-12T03:04:33.454 INFO:teuthology.orchestra.run.smithi005.stdout: "take_sum": 0, 2024-09-12T03:04:33.454 INFO:teuthology.orchestra.run.smithi005.stdout: "put": 0, 2024-09-12T03:04:33.454 INFO:teuthology.orchestra.run.smithi005.stdout: "put_sum": 0, 2024-09-12T03:04:33.454 INFO:teuthology.orchestra.run.smithi005.stdout: "wait": { 2024-09-12T03:04:33.454 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-12T03:04:33.454 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-12T03:04:33.454 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-12T03:04:33.454 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:33.455 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:33.455 INFO:teuthology.orchestra.run.smithi005.stdout: "throttle-write_buf_throttle-0x556c19984b40": { 2024-09-12T03:04:33.455 INFO:teuthology.orchestra.run.smithi005.stdout: "val": 0, 2024-09-12T03:04:33.455 INFO:teuthology.orchestra.run.smithi005.stdout: "max": 3758096384, 2024-09-12T03:04:33.455 INFO:teuthology.orchestra.run.smithi005.stdout: "get_started": 0, 2024-09-12T03:04:33.455 INFO:teuthology.orchestra.run.smithi005.stdout: "get": 0, 2024-09-12T03:04:33.455 INFO:teuthology.orchestra.run.smithi005.stdout: "get_sum": 0, 2024-09-12T03:04:33.455 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_fail": 0, 2024-09-12T03:04:33.455 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_success": 0, 2024-09-12T03:04:33.455 INFO:teuthology.orchestra.run.smithi005.stdout: "take": 0, 2024-09-12T03:04:33.455 INFO:teuthology.orchestra.run.smithi005.stdout: "take_sum": 0, 2024-09-12T03:04:33.455 INFO:teuthology.orchestra.run.smithi005.stdout: "put": 0, 2024-09-12T03:04:33.455 INFO:teuthology.orchestra.run.smithi005.stdout: "put_sum": 0, 2024-09-12T03:04:33.455 INFO:teuthology.orchestra.run.smithi005.stdout: "wait": { 2024-09-12T03:04:33.455 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-12T03:04:33.456 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-12T03:04:33.456 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-12T03:04:33.456 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:33.456 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:33.456 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-12T03:04:34.164 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-09-12T03:04:34.164 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-09-12T03:04:34.164 INFO:tasks.check_counter:Daemon mds.a mds.dir_split=0 2024-09-12T03:04:34.164 DEBUG:tasks.check_counter:Getting stats from d 2024-09-12T03:04:34.164 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph daemon mds.d perf dump 2024-09-12T03:04:34.420 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:04:34.843 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-12T03:04:34.844 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T03:04:34.844 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 153, 2024-09-12T03:04:34.844 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 146, 2024-09-12T03:04:34.844 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 736320, 2024-09-12T03:04:34.844 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 88480, 2024-09-12T03:04:34.844 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 4, 2024-09-12T03:04:34.844 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 0, 2024-09-12T03:04:34.844 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.026402943, 2024-09-12T03:04:34.844 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.009776122, 2024-09-12T03:04:34.844 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.021665027, 2024-09-12T03:04:34.844 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:04:34.845 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:04:34.845 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 146, 2024-09-12T03:04:34.845 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.011218349, 2024-09-12T03:04:34.845 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000076838 2024-09-12T03:04:34.845 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:34.845 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:04:34.845 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-12T03:04:34.845 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-12T03:04:34.845 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-12T03:04:34.845 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:34.845 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 736320, 2024-09-12T03:04:34.845 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 88480 2024-09-12T03:04:34.845 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:34.846 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T03:04:34.846 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 1, 2024-09-12T03:04:34.846 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 109, 2024-09-12T03:04:34.846 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 160, 2024-09-12T03:04:34.846 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 97712, 2024-09-12T03:04:34.846 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 3, 2024-09-12T03:04:34.846 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 2, 2024-09-12T03:04:34.846 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.008160662, 2024-09-12T03:04:34.846 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.006007613, 2024-09-12T03:04:34.846 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.001092741, 2024-09-12T03:04:34.846 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:04:34.846 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:04:34.846 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 107, 2024-09-12T03:04:34.846 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.004791929, 2024-09-12T03:04:34.847 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000044784 2024-09-12T03:04:34.847 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:34.847 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:04:34.847 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-12T03:04:34.847 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-12T03:04:34.847 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-12T03:04:34.847 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:34.847 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-12T03:04:34.847 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 97712 2024-09-12T03:04:34.847 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:34.847 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T03:04:34.847 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 0, 2024-09-12T03:04:34.847 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 2, 2024-09-12T03:04:34.847 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 0, 2024-09-12T03:04:34.848 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 0, 2024-09-12T03:04:34.848 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 2, 2024-09-12T03:04:34.848 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 1, 2024-09-12T03:04:34.848 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.000365707, 2024-09-12T03:04:34.848 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.000000000, 2024-09-12T03:04:34.848 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.000284680, 2024-09-12T03:04:34.848 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:04:34.848 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:04:34.848 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-12T03:04:34.848 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-12T03:04:34.848 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-12T03:04:34.848 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:34.848 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:04:34.849 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-12T03:04:34.849 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-12T03:04:34.849 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-12T03:04:34.849 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:34.849 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-12T03:04:34.849 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 0 2024-09-12T03:04:34.849 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:34.849 INFO:teuthology.orchestra.run.smithi005.stdout: "cct": { 2024-09-12T03:04:34.849 INFO:teuthology.orchestra.run.smithi005.stdout: "total_workers": 0, 2024-09-12T03:04:34.849 INFO:teuthology.orchestra.run.smithi005.stdout: "unhealthy_workers": 0 2024-09-12T03:04:34.849 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:34.849 INFO:teuthology.orchestra.run.smithi005.stdout: "mempool": { 2024-09-12T03:04:34.849 INFO:teuthology.orchestra.run.smithi005.stdout: "bloom_filter_bytes": 0, 2024-09-12T03:04:34.850 INFO:teuthology.orchestra.run.smithi005.stdout: "bloom_filter_items": 0, 2024-09-12T03:04:34.850 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T03:04:34.850 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_alloc_items": 0, 2024-09-12T03:04:34.850 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T03:04:34.850 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_data_items": 0, 2024-09-12T03:04:34.850 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T03:04:34.850 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T03:04:34.850 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T03:04:34.850 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T03:04:34.850 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T03:04:34.850 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_other_items": 0, 2024-09-12T03:04:34.850 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T03:04:34.850 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T03:04:34.850 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_extent_bytes": 0, 2024-09-12T03:04:34.850 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_extent_items": 0, 2024-09-12T03:04:34.851 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_blob_bytes": 0, 2024-09-12T03:04:34.851 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_blob_items": 0, 2024-09-12T03:04:34.851 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T03:04:34.851 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T03:04:34.851 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T03:04:34.851 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T03:04:34.851 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T03:04:34.851 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_fsck_items": 0, 2024-09-12T03:04:34.851 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_txc_bytes": 0, 2024-09-12T03:04:34.851 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_txc_items": 0, 2024-09-12T03:04:34.851 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T03:04:34.851 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T03:04:34.851 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_bytes": 0, 2024-09-12T03:04:34.852 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_items": 0, 2024-09-12T03:04:34.852 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_bytes": 0, 2024-09-12T03:04:34.852 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_items": 0, 2024-09-12T03:04:34.852 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T03:04:34.852 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_reader_items": 0, 2024-09-12T03:04:34.852 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T03:04:34.852 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_writer_items": 0, 2024-09-12T03:04:34.852 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_anon_bytes": 149001, 2024-09-12T03:04:34.852 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_anon_items": 31, 2024-09-12T03:04:34.852 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_meta_bytes": 0, 2024-09-12T03:04:34.852 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_meta_items": 0, 2024-09-12T03:04:34.852 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_bytes": 0, 2024-09-12T03:04:34.852 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_items": 0, 2024-09-12T03:04:34.852 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mapbl_bytes": 0, 2024-09-12T03:04:34.853 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mapbl_items": 0, 2024-09-12T03:04:34.853 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pglog_bytes": 0, 2024-09-12T03:04:34.853 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pglog_items": 0, 2024-09-12T03:04:34.853 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_bytes": 0, 2024-09-12T03:04:34.853 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_items": 0, 2024-09-12T03:04:34.853 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T03:04:34.853 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_mapping_items": 0, 2024-09-12T03:04:34.853 INFO:teuthology.orchestra.run.smithi005.stdout: "pgmap_bytes": 0, 2024-09-12T03:04:34.853 INFO:teuthology.orchestra.run.smithi005.stdout: "pgmap_items": 0, 2024-09-12T03:04:34.853 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_co_bytes": 928, 2024-09-12T03:04:34.853 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_co_items": 2, 2024-09-12T03:04:34.853 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_1_bytes": 0, 2024-09-12T03:04:34.853 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_1_items": 0, 2024-09-12T03:04:34.854 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_2_bytes": 0, 2024-09-12T03:04:34.854 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_2_items": 0 2024-09-12T03:04:34.854 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:34.854 INFO:teuthology.orchestra.run.smithi005.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T03:04:34.854 INFO:teuthology.orchestra.run.smithi005.stdout: "val": 0, 2024-09-12T03:04:34.854 INFO:teuthology.orchestra.run.smithi005.stdout: "max": 104857600, 2024-09-12T03:04:34.854 INFO:teuthology.orchestra.run.smithi005.stdout: "get_started": 0, 2024-09-12T03:04:34.854 INFO:teuthology.orchestra.run.smithi005.stdout: "get": 150, 2024-09-12T03:04:34.855 INFO:teuthology.orchestra.run.smithi005.stdout: "get_sum": 712285, 2024-09-12T03:04:34.855 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_fail": 0, 2024-09-12T03:04:34.855 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_success": 150, 2024-09-12T03:04:34.855 INFO:teuthology.orchestra.run.smithi005.stdout: "take": 0, 2024-09-12T03:04:34.855 INFO:teuthology.orchestra.run.smithi005.stdout: "take_sum": 0, 2024-09-12T03:04:34.855 INFO:teuthology.orchestra.run.smithi005.stdout: "put": 150, 2024-09-12T03:04:34.855 INFO:teuthology.orchestra.run.smithi005.stdout: "put_sum": 712285, 2024-09-12T03:04:34.855 INFO:teuthology.orchestra.run.smithi005.stdout: "wait": { 2024-09-12T03:04:34.855 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-12T03:04:34.855 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-12T03:04:34.855 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-12T03:04:34.855 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:34.856 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:34.856 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-12T03:04:35.165 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:04:34 smithi148 ceph-mon[35090]: pgmap v525: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 85 B/s wr, 10 op/s 2024-09-12T03:04:35.183 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:04:34 smithi005 ceph-mon[31197]: pgmap v525: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 85 B/s wr, 10 op/s 2024-09-12T03:04:35.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:04:34 smithi146 ceph-mon[35011]: pgmap v525: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 85 B/s wr, 10 op/s 2024-09-12T03:04:35.550 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-09-12T03:04:35.550 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-09-12T03:04:35.550 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.d 2024-09-12T03:04:35.550 DEBUG:tasks.check_counter:Getting stats from g 2024-09-12T03:04:35.550 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph daemon mds.g perf dump 2024-09-12T03:04:35.809 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:04:36.296 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-12T03:04:36.296 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T03:04:36.296 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 489, 2024-09-12T03:04:36.297 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 491, 2024-09-12T03:04:36.297 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 164249, 2024-09-12T03:04:36.297 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 155503, 2024-09-12T03:04:36.297 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 6, 2024-09-12T03:04:36.297 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 2, 2024-09-12T03:04:36.297 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.050212159, 2024-09-12T03:04:36.297 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.020183945, 2024-09-12T03:04:36.297 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.011274386, 2024-09-12T03:04:36.297 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.013295936, 2024-09-12T03:04:36.297 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:04:36.297 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 489, 2024-09-12T03:04:36.297 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.021097160, 2024-09-12T03:04:36.298 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000043143 2024-09-12T03:04:36.298 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:36.298 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:04:36.298 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-12T03:04:36.298 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-12T03:04:36.298 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-12T03:04:36.298 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:36.298 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-12T03:04:36.298 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 0 2024-09-12T03:04:36.298 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:36.298 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T03:04:36.298 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 909, 2024-09-12T03:04:36.298 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 1013, 2024-09-12T03:04:36.298 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 305505, 2024-09-12T03:04:36.299 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 552985, 2024-09-12T03:04:36.299 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 5, 2024-09-12T03:04:36.299 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 4, 2024-09-12T03:04:36.299 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.063014028, 2024-09-12T03:04:36.299 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.020807724, 2024-09-12T03:04:36.299 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.017121258, 2024-09-12T03:04:36.299 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.020289904, 2024-09-12T03:04:36.299 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:04:36.299 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 1012, 2024-09-12T03:04:36.299 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.034345941, 2024-09-12T03:04:36.299 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000033938 2024-09-12T03:04:36.299 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:36.299 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:04:36.300 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-12T03:04:36.300 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-12T03:04:36.300 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-12T03:04:36.300 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:36.300 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 3184, 2024-09-12T03:04:36.300 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 265456 2024-09-12T03:04:36.300 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:36.300 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T03:04:36.300 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 163, 2024-09-12T03:04:36.300 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 150, 2024-09-12T03:04:36.300 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 803924, 2024-09-12T03:04:36.300 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 87605, 2024-09-12T03:04:36.301 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 4, 2024-09-12T03:04:36.301 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 3, 2024-09-12T03:04:36.301 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.024511549, 2024-09-12T03:04:36.301 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.009434578, 2024-09-12T03:04:36.301 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.021925386, 2024-09-12T03:04:36.301 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.000066869, 2024-09-12T03:04:36.301 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:04:36.301 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 149, 2024-09-12T03:04:36.301 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.008429351, 2024-09-12T03:04:36.301 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000056572 2024-09-12T03:04:36.301 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:36.301 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:04:36.301 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-12T03:04:36.302 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-12T03:04:36.302 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-12T03:04:36.302 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:36.302 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 802576, 2024-09-12T03:04:36.302 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 86640 2024-09-12T03:04:36.302 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:36.302 INFO:teuthology.orchestra.run.smithi005.stdout: "cct": { 2024-09-12T03:04:36.302 INFO:teuthology.orchestra.run.smithi005.stdout: "total_workers": 1, 2024-09-12T03:04:36.302 INFO:teuthology.orchestra.run.smithi005.stdout: "unhealthy_workers": 0 2024-09-12T03:04:36.302 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:36.302 INFO:teuthology.orchestra.run.smithi005.stdout: "finisher-MDSRank": { 2024-09-12T03:04:36.302 INFO:teuthology.orchestra.run.smithi005.stdout: "queue_len": 0, 2024-09-12T03:04:36.302 INFO:teuthology.orchestra.run.smithi005.stdout: "complete_latency": { 2024-09-12T03:04:36.303 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 2381, 2024-09-12T03:04:36.303 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.053857994, 2024-09-12T03:04:36.303 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000022619 2024-09-12T03:04:36.303 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:36.303 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:36.303 INFO:teuthology.orchestra.run.smithi005.stdout: "finisher-PurgeQueue": { 2024-09-12T03:04:36.303 INFO:teuthology.orchestra.run.smithi005.stdout: "queue_len": 0, 2024-09-12T03:04:36.303 INFO:teuthology.orchestra.run.smithi005.stdout: "complete_latency": { 2024-09-12T03:04:36.303 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-12T03:04:36.303 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-12T03:04:36.303 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-12T03:04:36.303 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:36.303 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:36.304 INFO:teuthology.orchestra.run.smithi005.stdout: "mds": { 2024-09-12T03:04:36.304 INFO:teuthology.orchestra.run.smithi005.stdout: "request": 0, 2024-09-12T03:04:36.304 INFO:teuthology.orchestra.run.smithi005.stdout: "reply": 0, 2024-09-12T03:04:36.304 INFO:teuthology.orchestra.run.smithi005.stdout: "reply_latency": { 2024-09-12T03:04:36.304 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-12T03:04:36.304 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-12T03:04:36.304 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-12T03:04:36.304 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:36.304 INFO:teuthology.orchestra.run.smithi005.stdout: "slow_reply": 0, 2024-09-12T03:04:36.304 INFO:teuthology.orchestra.run.smithi005.stdout: "forward": 0, 2024-09-12T03:04:36.304 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_fetch_complete": 0, 2024-09-12T03:04:36.304 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_fetch_keys": 0, 2024-09-12T03:04:36.304 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_commit": 0, 2024-09-12T03:04:36.305 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_split": 0, 2024-09-12T03:04:36.305 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_merge": 0, 2024-09-12T03:04:36.305 INFO:teuthology.orchestra.run.smithi005.stdout: "inodes": 7, 2024-09-12T03:04:36.305 INFO:teuthology.orchestra.run.smithi005.stdout: "inodes_top": 0, 2024-09-12T03:04:36.305 INFO:teuthology.orchestra.run.smithi005.stdout: "inodes_bottom": 7, 2024-09-12T03:04:36.305 INFO:teuthology.orchestra.run.smithi005.stdout: "inodes_pin_tail": 0, 2024-09-12T03:04:36.305 INFO:teuthology.orchestra.run.smithi005.stdout: "inodes_pinned": 7, 2024-09-12T03:04:36.305 INFO:teuthology.orchestra.run.smithi005.stdout: "inodes_expired": 0, 2024-09-12T03:04:36.305 INFO:teuthology.orchestra.run.smithi005.stdout: "inodes_with_caps": 0, 2024-09-12T03:04:36.305 INFO:teuthology.orchestra.run.smithi005.stdout: "caps": 0, 2024-09-12T03:04:36.305 INFO:teuthology.orchestra.run.smithi005.stdout: "subtrees": 1, 2024-09-12T03:04:36.305 INFO:teuthology.orchestra.run.smithi005.stdout: "traverse": 0, 2024-09-12T03:04:36.305 INFO:teuthology.orchestra.run.smithi005.stdout: "traverse_hit": 0, 2024-09-12T03:04:36.306 INFO:teuthology.orchestra.run.smithi005.stdout: "traverse_forward": 0, 2024-09-12T03:04:36.306 INFO:teuthology.orchestra.run.smithi005.stdout: "traverse_discover": 0, 2024-09-12T03:04:36.306 INFO:teuthology.orchestra.run.smithi005.stdout: "traverse_dir_fetch": 0, 2024-09-12T03:04:36.306 INFO:teuthology.orchestra.run.smithi005.stdout: "traverse_remote_ino": 0, 2024-09-12T03:04:36.306 INFO:teuthology.orchestra.run.smithi005.stdout: "traverse_lock": 0, 2024-09-12T03:04:36.306 INFO:teuthology.orchestra.run.smithi005.stdout: "load_cent": 0, 2024-09-12T03:04:36.306 INFO:teuthology.orchestra.run.smithi005.stdout: "q": 0, 2024-09-12T03:04:36.306 INFO:teuthology.orchestra.run.smithi005.stdout: "exported": 0, 2024-09-12T03:04:36.306 INFO:teuthology.orchestra.run.smithi005.stdout: "exported_inodes": 0, 2024-09-12T03:04:36.306 INFO:teuthology.orchestra.run.smithi005.stdout: "imported": 0, 2024-09-12T03:04:36.306 INFO:teuthology.orchestra.run.smithi005.stdout: "imported_inodes": 0, 2024-09-12T03:04:36.306 INFO:teuthology.orchestra.run.smithi005.stdout: "openino_dir_fetch": 0, 2024-09-12T03:04:36.307 INFO:teuthology.orchestra.run.smithi005.stdout: "openino_backtrace_fetch": 0, 2024-09-12T03:04:36.307 INFO:teuthology.orchestra.run.smithi005.stdout: "openino_peer_discover": 0, 2024-09-12T03:04:36.307 INFO:teuthology.orchestra.run.smithi005.stdout: "root_rfiles": 0, 2024-09-12T03:04:36.307 INFO:teuthology.orchestra.run.smithi005.stdout: "root_rbytes": 0, 2024-09-12T03:04:36.307 INFO:teuthology.orchestra.run.smithi005.stdout: "root_rsnaps": 0, 2024-09-12T03:04:36.307 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_backtrace_fetch": 0, 2024-09-12T03:04:36.307 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_set_tag": 0, 2024-09-12T03:04:36.307 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_backtrace_repaired": 0, 2024-09-12T03:04:36.307 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_inotable_repaired": 0, 2024-09-12T03:04:36.307 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_dir_inodes": 0, 2024-09-12T03:04:36.307 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_dir_base_inodes": 0, 2024-09-12T03:04:36.307 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_dirfrag_rstats": 0, 2024-09-12T03:04:36.307 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_file_inodes": 0, 2024-09-12T03:04:36.308 INFO:teuthology.orchestra.run.smithi005.stdout: "handle_inode_file_caps": 0, 2024-09-12T03:04:36.308 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_cap_op_revoke": 0, 2024-09-12T03:04:36.308 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_cap_op_grant": 0, 2024-09-12T03:04:36.308 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_cap_op_trunc": 0, 2024-09-12T03:04:36.308 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-12T03:04:36.308 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-12T03:04:36.308 INFO:teuthology.orchestra.run.smithi005.stdout: "handle_client_caps": 0, 2024-09-12T03:04:36.308 INFO:teuthology.orchestra.run.smithi005.stdout: "handle_client_caps_dirty": 0, 2024-09-12T03:04:36.308 INFO:teuthology.orchestra.run.smithi005.stdout: "handle_client_cap_release": 0, 2024-09-12T03:04:36.308 INFO:teuthology.orchestra.run.smithi005.stdout: "process_request_cap_release": 0 2024-09-12T03:04:36.308 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:36.308 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_cache": { 2024-09-12T03:04:36.308 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_update": 0, 2024-09-12T03:04:36.309 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_update_receipt": 0, 2024-09-12T03:04:36.309 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_try_discover": 0, 2024-09-12T03:04:36.309 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_send_discover": 0, 2024-09-12T03:04:36.309 INFO:teuthology.orchestra.run.smithi005.stdout: "dir_handle_discover": 0, 2024-09-12T03:04:36.309 INFO:teuthology.orchestra.run.smithi005.stdout: "num_strays": 0, 2024-09-12T03:04:36.309 INFO:teuthology.orchestra.run.smithi005.stdout: "num_strays_delayed": 0, 2024-09-12T03:04:36.309 INFO:teuthology.orchestra.run.smithi005.stdout: "num_strays_enqueuing": 0, 2024-09-12T03:04:36.309 INFO:teuthology.orchestra.run.smithi005.stdout: "strays_created": 0, 2024-09-12T03:04:36.309 INFO:teuthology.orchestra.run.smithi005.stdout: "strays_enqueued": 0, 2024-09-12T03:04:36.309 INFO:teuthology.orchestra.run.smithi005.stdout: "strays_reintegrated": 0, 2024-09-12T03:04:36.309 INFO:teuthology.orchestra.run.smithi005.stdout: "strays_migrated": 0, 2024-09-12T03:04:36.309 INFO:teuthology.orchestra.run.smithi005.stdout: "num_recovering_processing": 0, 2024-09-12T03:04:36.309 INFO:teuthology.orchestra.run.smithi005.stdout: "num_recovering_enqueued": 0, 2024-09-12T03:04:36.310 INFO:teuthology.orchestra.run.smithi005.stdout: "num_recovering_prioritized": 0, 2024-09-12T03:04:36.310 INFO:teuthology.orchestra.run.smithi005.stdout: "recovery_started": 0, 2024-09-12T03:04:36.310 INFO:teuthology.orchestra.run.smithi005.stdout: "recovery_completed": 0, 2024-09-12T03:04:36.310 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_quiesce_path": 0, 2024-09-12T03:04:36.310 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_quiesce_inode": 0, 2024-09-12T03:04:36.310 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_enqueue_scrub": 0, 2024-09-12T03:04:36.310 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_exportdir": 0, 2024-09-12T03:04:36.310 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_flush": 0, 2024-09-12T03:04:36.310 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_fragmentdir": 0, 2024-09-12T03:04:36.310 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_fragstats": 0, 2024-09-12T03:04:36.310 INFO:teuthology.orchestra.run.smithi005.stdout: "ireq_inodestats": 0 2024-09-12T03:04:36.310 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:36.311 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_log": { 2024-09-12T03:04:36.311 INFO:teuthology.orchestra.run.smithi005.stdout: "evlrg": 0, 2024-09-12T03:04:36.311 INFO:teuthology.orchestra.run.smithi005.stdout: "evadd": 0, 2024-09-12T03:04:36.311 INFO:teuthology.orchestra.run.smithi005.stdout: "evex": 0, 2024-09-12T03:04:36.311 INFO:teuthology.orchestra.run.smithi005.stdout: "evtrm": 0, 2024-09-12T03:04:36.311 INFO:teuthology.orchestra.run.smithi005.stdout: "ev": 47, 2024-09-12T03:04:36.311 INFO:teuthology.orchestra.run.smithi005.stdout: "evexg": 0, 2024-09-12T03:04:36.311 INFO:teuthology.orchestra.run.smithi005.stdout: "evexd": 0, 2024-09-12T03:04:36.311 INFO:teuthology.orchestra.run.smithi005.stdout: "segadd": 0, 2024-09-12T03:04:36.311 INFO:teuthology.orchestra.run.smithi005.stdout: "segex": 0, 2024-09-12T03:04:36.311 INFO:teuthology.orchestra.run.smithi005.stdout: "segtrm": 0, 2024-09-12T03:04:36.311 INFO:teuthology.orchestra.run.smithi005.stdout: "seg": 2, 2024-09-12T03:04:36.311 INFO:teuthology.orchestra.run.smithi005.stdout: "segmjr": 2, 2024-09-12T03:04:36.311 INFO:teuthology.orchestra.run.smithi005.stdout: "segexg": 0, 2024-09-12T03:04:36.312 INFO:teuthology.orchestra.run.smithi005.stdout: "segexd": 0, 2024-09-12T03:04:36.312 INFO:teuthology.orchestra.run.smithi005.stdout: "expos": 4194304, 2024-09-12T03:04:36.312 INFO:teuthology.orchestra.run.smithi005.stdout: "wrpos": 4266369, 2024-09-12T03:04:36.312 INFO:teuthology.orchestra.run.smithi005.stdout: "rdpos": 4265231, 2024-09-12T03:04:36.312 INFO:teuthology.orchestra.run.smithi005.stdout: "jlat": { 2024-09-12T03:04:36.312 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-12T03:04:36.312 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-12T03:04:36.312 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-12T03:04:36.312 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:36.312 INFO:teuthology.orchestra.run.smithi005.stdout: "replayed": 47 2024-09-12T03:04:36.312 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:36.312 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_mem": { 2024-09-12T03:04:36.312 INFO:teuthology.orchestra.run.smithi005.stdout: "ino": 4, 2024-09-12T03:04:36.313 INFO:teuthology.orchestra.run.smithi005.stdout: "ino+": 26, 2024-09-12T03:04:36.313 INFO:teuthology.orchestra.run.smithi005.stdout: "ino-": 22, 2024-09-12T03:04:36.313 INFO:teuthology.orchestra.run.smithi005.stdout: "dir": 2, 2024-09-12T03:04:36.313 INFO:teuthology.orchestra.run.smithi005.stdout: "dir+": 19, 2024-09-12T03:04:36.313 INFO:teuthology.orchestra.run.smithi005.stdout: "dir-": 17, 2024-09-12T03:04:36.313 INFO:teuthology.orchestra.run.smithi005.stdout: "dn": 7, 2024-09-12T03:04:36.313 INFO:teuthology.orchestra.run.smithi005.stdout: "dn+": 30, 2024-09-12T03:04:36.313 INFO:teuthology.orchestra.run.smithi005.stdout: "dn-": 23, 2024-09-12T03:04:36.313 INFO:teuthology.orchestra.run.smithi005.stdout: "cap": 0, 2024-09-12T03:04:36.313 INFO:teuthology.orchestra.run.smithi005.stdout: "cap+": 0, 2024-09-12T03:04:36.313 INFO:teuthology.orchestra.run.smithi005.stdout: "cap-": 0, 2024-09-12T03:04:36.313 INFO:teuthology.orchestra.run.smithi005.stdout: "rss": 50764, 2024-09-12T03:04:36.313 INFO:teuthology.orchestra.run.smithi005.stdout: "heap": 215328 2024-09-12T03:04:36.313 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:36.314 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_server": { 2024-09-12T03:04:36.314 INFO:teuthology.orchestra.run.smithi005.stdout: "dispatch_client_request": 0, 2024-09-12T03:04:36.314 INFO:teuthology.orchestra.run.smithi005.stdout: "dispatch_server_request": 0, 2024-09-12T03:04:36.314 INFO:teuthology.orchestra.run.smithi005.stdout: "handle_client_request": 0, 2024-09-12T03:04:36.314 INFO:teuthology.orchestra.run.smithi005.stdout: "handle_client_session": 0, 2024-09-12T03:04:36.314 INFO:teuthology.orchestra.run.smithi005.stdout: "handle_peer_request": 0, 2024-09-12T03:04:36.314 INFO:teuthology.orchestra.run.smithi005.stdout: "req_create_latency": { 2024-09-12T03:04:36.314 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-12T03:04:36.314 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-12T03:04:36.314 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-12T03:04:36.314 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:36.314 INFO:teuthology.orchestra.run.smithi005.stdout: "req_getattr_latency": { 2024-09-12T03:04:36.314 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-12T03:04:36.315 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-12T03:04:36.315 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-12T03:04:36.315 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:36.315 INFO:teuthology.orchestra.run.smithi005.stdout: "req_getfilelock_latency": { 2024-09-12T03:04:36.315 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-12T03:04:36.315 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-12T03:04:36.315 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-12T03:04:36.315 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:36.315 INFO:teuthology.orchestra.run.smithi005.stdout: "req_link_latency": { 2024-09-12T03:04:36.315 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-12T03:04:36.315 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-12T03:04:36.315 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-12T03:04:36.315 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:36.315 INFO:teuthology.orchestra.run.smithi005.stdout: "req_lookup_latency": { 2024-09-12T03:04:36.316 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-12T03:04:36.316 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-12T03:04:36.316 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-12T03:04:36.316 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:36.316 INFO:teuthology.orchestra.run.smithi005.stdout: "req_lookuphash_latency": { 2024-09-12T03:04:36.316 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-12T03:04:36.316 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-12T03:04:36.316 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-12T03:04:36.316 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:36.317 INFO:teuthology.orchestra.run.smithi005.stdout: "req_lookupino_latency": { 2024-09-12T03:04:36.317 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-12T03:04:36.317 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-12T03:04:36.317 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-12T03:04:36.317 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:36.317 INFO:teuthology.orchestra.run.smithi005.stdout: "req_lookupname_latency": { 2024-09-12T03:04:36.317 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-12T03:04:36.317 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-12T03:04:36.317 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-12T03:04:36.317 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:36.318 INFO:teuthology.orchestra.run.smithi005.stdout: "req_lookupparent_latency": { 2024-09-12T03:04:36.318 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-12T03:04:36.318 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-12T03:04:36.318 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-12T03:04:36.318 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:36.318 INFO:teuthology.orchestra.run.smithi005.stdout: "req_lookupsnap_latency": { 2024-09-12T03:04:36.318 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-12T03:04:36.318 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-12T03:04:36.318 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-12T03:04:36.319 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:36.319 INFO:teuthology.orchestra.run.smithi005.stdout: "req_lssnap_latency": { 2024-09-12T03:04:36.319 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-12T03:04:36.319 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-12T03:04:36.319 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-12T03:04:36.319 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:36.319 INFO:teuthology.orchestra.run.smithi005.stdout: "req_mkdir_latency": { 2024-09-12T03:04:36.319 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-12T03:04:36.320 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-12T03:04:36.320 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-12T03:04:36.320 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:36.320 INFO:teuthology.orchestra.run.smithi005.stdout: "req_mknod_latency": { 2024-09-12T03:04:36.320 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-12T03:04:36.321 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-12T03:04:36.321 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-12T03:04:36.321 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:36.321 INFO:teuthology.orchestra.run.smithi005.stdout: "req_mksnap_latency": { 2024-09-12T03:04:36.321 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-12T03:04:36.321 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-12T03:04:36.321 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-12T03:04:36.321 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:36.321 INFO:teuthology.orchestra.run.smithi005.stdout: "req_open_latency": { 2024-09-12T03:04:36.321 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-12T03:04:36.321 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-12T03:04:36.321 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-12T03:04:36.321 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:36.322 INFO:teuthology.orchestra.run.smithi005.stdout: "req_readdir_latency": { 2024-09-12T03:04:36.322 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-12T03:04:36.322 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-12T03:04:36.322 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-12T03:04:36.322 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:36.322 INFO:teuthology.orchestra.run.smithi005.stdout: "req_rename_latency": { 2024-09-12T03:04:36.322 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-12T03:04:36.322 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-12T03:04:36.322 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-12T03:04:36.322 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:36.322 INFO:teuthology.orchestra.run.smithi005.stdout: "req_renamesnap_latency": { 2024-09-12T03:04:36.322 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-12T03:04:36.322 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-12T03:04:36.322 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-12T03:04:36.323 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:36.323 INFO:teuthology.orchestra.run.smithi005.stdout: "req_snapdiff_latency": { 2024-09-12T03:04:36.323 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-12T03:04:36.323 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-12T03:04:36.323 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-12T03:04:36.323 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:36.323 INFO:teuthology.orchestra.run.smithi005.stdout: "req_rmdir_latency": { 2024-09-12T03:04:36.323 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-12T03:04:36.323 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-12T03:04:36.323 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-12T03:04:36.323 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:36.323 INFO:teuthology.orchestra.run.smithi005.stdout: "req_rmsnap_latency": { 2024-09-12T03:04:36.323 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-12T03:04:36.324 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-12T03:04:36.324 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-12T03:04:36.324 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:36.324 INFO:teuthology.orchestra.run.smithi005.stdout: "req_rmxattr_latency": { 2024-09-12T03:04:36.324 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-12T03:04:36.324 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-12T03:04:36.324 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-12T03:04:36.324 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:36.324 INFO:teuthology.orchestra.run.smithi005.stdout: "req_setattr_latency": { 2024-09-12T03:04:36.324 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-12T03:04:36.324 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-12T03:04:36.324 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-12T03:04:36.324 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:36.325 INFO:teuthology.orchestra.run.smithi005.stdout: "req_setdirlayout_latency": { 2024-09-12T03:04:36.325 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-12T03:04:36.325 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-12T03:04:36.325 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-12T03:04:36.325 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:36.325 INFO:teuthology.orchestra.run.smithi005.stdout: "req_setfilelock_latency": { 2024-09-12T03:04:36.325 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-12T03:04:36.325 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-12T03:04:36.325 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-12T03:04:36.325 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:36.325 INFO:teuthology.orchestra.run.smithi005.stdout: "req_setlayout_latency": { 2024-09-12T03:04:36.325 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-12T03:04:36.325 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-12T03:04:36.326 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-12T03:04:36.326 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:36.326 INFO:teuthology.orchestra.run.smithi005.stdout: "req_setxattr_latency": { 2024-09-12T03:04:36.326 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-12T03:04:36.326 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-12T03:04:36.326 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-12T03:04:36.326 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:36.326 INFO:teuthology.orchestra.run.smithi005.stdout: "req_symlink_latency": { 2024-09-12T03:04:36.326 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-12T03:04:36.326 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-12T03:04:36.326 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-12T03:04:36.326 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:36.327 INFO:teuthology.orchestra.run.smithi005.stdout: "req_unlink_latency": { 2024-09-12T03:04:36.327 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-12T03:04:36.327 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-12T03:04:36.327 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-12T03:04:36.327 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:36.327 INFO:teuthology.orchestra.run.smithi005.stdout: "cap_revoke_eviction": 0, 2024-09-12T03:04:36.327 INFO:teuthology.orchestra.run.smithi005.stdout: "cap_acquisition_throttle": 0, 2024-09-12T03:04:36.327 INFO:teuthology.orchestra.run.smithi005.stdout: "req_getvxattr_latency": { 2024-09-12T03:04:36.327 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-12T03:04:36.327 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-12T03:04:36.327 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-12T03:04:36.327 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:36.327 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:36.328 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_sessions": { 2024-09-12T03:04:36.328 INFO:teuthology.orchestra.run.smithi005.stdout: "session_count": 1, 2024-09-12T03:04:36.328 INFO:teuthology.orchestra.run.smithi005.stdout: "session_add": 2, 2024-09-12T03:04:36.328 INFO:teuthology.orchestra.run.smithi005.stdout: "session_remove": 1, 2024-09-12T03:04:36.328 INFO:teuthology.orchestra.run.smithi005.stdout: "sessions_open": 2, 2024-09-12T03:04:36.328 INFO:teuthology.orchestra.run.smithi005.stdout: "sessions_stale": 0, 2024-09-12T03:04:36.328 INFO:teuthology.orchestra.run.smithi005.stdout: "total_load": 0, 2024-09-12T03:04:36.328 INFO:teuthology.orchestra.run.smithi005.stdout: "average_load": 0, 2024-09-12T03:04:36.328 INFO:teuthology.orchestra.run.smithi005.stdout: "avg_session_uptime": 2397, 2024-09-12T03:04:36.328 INFO:teuthology.orchestra.run.smithi005.stdout: "mdthresh_evicted": 0 2024-09-12T03:04:36.328 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:36.328 INFO:teuthology.orchestra.run.smithi005.stdout: "mempool": { 2024-09-12T03:04:36.328 INFO:teuthology.orchestra.run.smithi005.stdout: "bloom_filter_bytes": 0, 2024-09-12T03:04:36.328 INFO:teuthology.orchestra.run.smithi005.stdout: "bloom_filter_items": 0, 2024-09-12T03:04:36.329 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T03:04:36.329 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_alloc_items": 0, 2024-09-12T03:04:36.329 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T03:04:36.329 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_data_items": 0, 2024-09-12T03:04:36.329 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T03:04:36.329 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T03:04:36.329 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T03:04:36.329 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T03:04:36.329 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T03:04:36.329 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_other_items": 0, 2024-09-12T03:04:36.329 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T03:04:36.329 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T03:04:36.330 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_extent_bytes": 0, 2024-09-12T03:04:36.330 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_extent_items": 0, 2024-09-12T03:04:36.330 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_blob_bytes": 0, 2024-09-12T03:04:36.330 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_blob_items": 0, 2024-09-12T03:04:36.330 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T03:04:36.330 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T03:04:36.330 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T03:04:36.330 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T03:04:36.330 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T03:04:36.330 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_fsck_items": 0, 2024-09-12T03:04:36.330 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_txc_bytes": 0, 2024-09-12T03:04:36.330 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_txc_items": 0, 2024-09-12T03:04:36.330 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T03:04:36.330 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T03:04:36.331 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_bytes": 0, 2024-09-12T03:04:36.331 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_items": 0, 2024-09-12T03:04:36.331 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_bytes": 0, 2024-09-12T03:04:36.331 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_items": 0, 2024-09-12T03:04:36.331 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T03:04:36.331 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_reader_items": 0, 2024-09-12T03:04:36.331 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T03:04:36.331 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_writer_items": 0, 2024-09-12T03:04:36.331 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_anon_bytes": 197463, 2024-09-12T03:04:36.331 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_anon_items": 54, 2024-09-12T03:04:36.331 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_meta_bytes": 0, 2024-09-12T03:04:36.331 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_meta_items": 0, 2024-09-12T03:04:36.331 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_bytes": 0, 2024-09-12T03:04:36.332 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_items": 0, 2024-09-12T03:04:36.332 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mapbl_bytes": 0, 2024-09-12T03:04:36.332 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mapbl_items": 0, 2024-09-12T03:04:36.332 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pglog_bytes": 0, 2024-09-12T03:04:36.332 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pglog_items": 0, 2024-09-12T03:04:36.332 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_bytes": 7496, 2024-09-12T03:04:36.332 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_items": 154, 2024-09-12T03:04:36.332 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T03:04:36.332 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_mapping_items": 0, 2024-09-12T03:04:36.332 INFO:teuthology.orchestra.run.smithi005.stdout: "pgmap_bytes": 0, 2024-09-12T03:04:36.332 INFO:teuthology.orchestra.run.smithi005.stdout: "pgmap_items": 0, 2024-09-12T03:04:36.332 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_co_bytes": 16984, 2024-09-12T03:04:36.332 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_co_items": 57, 2024-09-12T03:04:36.333 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_1_bytes": 0, 2024-09-12T03:04:36.333 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_1_items": 0, 2024-09-12T03:04:36.333 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_2_bytes": 0, 2024-09-12T03:04:36.333 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_2_items": 0 2024-09-12T03:04:36.333 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:36.333 INFO:teuthology.orchestra.run.smithi005.stdout: "objecter": { 2024-09-12T03:04:36.333 INFO:teuthology.orchestra.run.smithi005.stdout: "op_active": 0, 2024-09-12T03:04:36.333 INFO:teuthology.orchestra.run.smithi005.stdout: "op_laggy": 0, 2024-09-12T03:04:36.333 INFO:teuthology.orchestra.run.smithi005.stdout: "op_send": 1396, 2024-09-12T03:04:36.333 INFO:teuthology.orchestra.run.smithi005.stdout: "op_send_bytes": 21, 2024-09-12T03:04:36.333 INFO:teuthology.orchestra.run.smithi005.stdout: "op_resend": 3, 2024-09-12T03:04:36.333 INFO:teuthology.orchestra.run.smithi005.stdout: "op_reply": 1396, 2024-09-12T03:04:36.333 INFO:teuthology.orchestra.run.smithi005.stdout: "op_latency": { 2024-09-12T03:04:36.333 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 1396, 2024-09-12T03:04:36.334 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.740984303, 2024-09-12T03:04:36.334 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000530791 2024-09-12T03:04:36.334 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:36.334 INFO:teuthology.orchestra.run.smithi005.stdout: "op_inflight": 0, 2024-09-12T03:04:36.334 INFO:teuthology.orchestra.run.smithi005.stdout: "oplen_avg": { 2024-09-12T03:04:36.334 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 1396, 2024-09-12T03:04:36.334 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 1397 2024-09-12T03:04:36.334 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:36.334 INFO:teuthology.orchestra.run.smithi005.stdout: "op": 1396, 2024-09-12T03:04:36.334 INFO:teuthology.orchestra.run.smithi005.stdout: "op_r": 1396, 2024-09-12T03:04:36.334 INFO:teuthology.orchestra.run.smithi005.stdout: "op_w": 0, 2024-09-12T03:04:36.334 INFO:teuthology.orchestra.run.smithi005.stdout: "op_rmw": 0, 2024-09-12T03:04:36.334 INFO:teuthology.orchestra.run.smithi005.stdout: "op_pg": 0, 2024-09-12T03:04:36.334 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_stat": 882, 2024-09-12T03:04:36.335 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_create": 0, 2024-09-12T03:04:36.335 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_read": 512, 2024-09-12T03:04:36.335 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_write": 0, 2024-09-12T03:04:36.335 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_writefull": 0, 2024-09-12T03:04:36.335 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_writesame": 0, 2024-09-12T03:04:36.335 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_append": 0, 2024-09-12T03:04:36.335 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_zero": 0, 2024-09-12T03:04:36.335 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_truncate": 0, 2024-09-12T03:04:36.335 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_delete": 0, 2024-09-12T03:04:36.335 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_mapext": 0, 2024-09-12T03:04:36.335 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_sparse_read": 0, 2024-09-12T03:04:36.335 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_clonerange": 0, 2024-09-12T03:04:36.335 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_getxattr": 1, 2024-09-12T03:04:36.336 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_setxattr": 0, 2024-09-12T03:04:36.336 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_cmpxattr": 0, 2024-09-12T03:04:36.336 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_rmxattr": 0, 2024-09-12T03:04:36.336 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_resetxattrs": 0, 2024-09-12T03:04:36.336 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_call": 0, 2024-09-12T03:04:36.336 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_watch": 0, 2024-09-12T03:04:36.336 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_notify": 0, 2024-09-12T03:04:36.336 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_src_cmpxattr": 0, 2024-09-12T03:04:36.336 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_pgls": 0, 2024-09-12T03:04:36.336 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_pgls_filter": 0, 2024-09-12T03:04:36.336 INFO:teuthology.orchestra.run.smithi005.stdout: "osdop_other": 0, 2024-09-12T03:04:36.336 INFO:teuthology.orchestra.run.smithi005.stdout: "linger_active": 0, 2024-09-12T03:04:36.336 INFO:teuthology.orchestra.run.smithi005.stdout: "linger_send": 0, 2024-09-12T03:04:36.337 INFO:teuthology.orchestra.run.smithi005.stdout: "linger_resend": 0, 2024-09-12T03:04:36.337 INFO:teuthology.orchestra.run.smithi005.stdout: "linger_ping": 0, 2024-09-12T03:04:36.337 INFO:teuthology.orchestra.run.smithi005.stdout: "poolop_active": 0, 2024-09-12T03:04:36.337 INFO:teuthology.orchestra.run.smithi005.stdout: "poolop_send": 0, 2024-09-12T03:04:36.337 INFO:teuthology.orchestra.run.smithi005.stdout: "poolop_resend": 0, 2024-09-12T03:04:36.337 INFO:teuthology.orchestra.run.smithi005.stdout: "poolstat_active": 0, 2024-09-12T03:04:36.337 INFO:teuthology.orchestra.run.smithi005.stdout: "poolstat_send": 0, 2024-09-12T03:04:36.337 INFO:teuthology.orchestra.run.smithi005.stdout: "poolstat_resend": 0, 2024-09-12T03:04:36.337 INFO:teuthology.orchestra.run.smithi005.stdout: "statfs_active": 0, 2024-09-12T03:04:36.337 INFO:teuthology.orchestra.run.smithi005.stdout: "statfs_send": 0, 2024-09-12T03:04:36.337 INFO:teuthology.orchestra.run.smithi005.stdout: "statfs_resend": 0, 2024-09-12T03:04:36.337 INFO:teuthology.orchestra.run.smithi005.stdout: "command_active": 0, 2024-09-12T03:04:36.337 INFO:teuthology.orchestra.run.smithi005.stdout: "command_send": 0, 2024-09-12T03:04:36.338 INFO:teuthology.orchestra.run.smithi005.stdout: "command_resend": 0, 2024-09-12T03:04:36.338 INFO:teuthology.orchestra.run.smithi005.stdout: "map_epoch": 83, 2024-09-12T03:04:36.338 INFO:teuthology.orchestra.run.smithi005.stdout: "map_full": 0, 2024-09-12T03:04:36.338 INFO:teuthology.orchestra.run.smithi005.stdout: "map_inc": 2, 2024-09-12T03:04:36.338 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_sessions": 6, 2024-09-12T03:04:36.338 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_session_open": 6, 2024-09-12T03:04:36.338 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_session_close": 0, 2024-09-12T03:04:36.338 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_laggy": 0, 2024-09-12T03:04:36.338 INFO:teuthology.orchestra.run.smithi005.stdout: "omap_wr": 0, 2024-09-12T03:04:36.338 INFO:teuthology.orchestra.run.smithi005.stdout: "omap_rd": 2, 2024-09-12T03:04:36.338 INFO:teuthology.orchestra.run.smithi005.stdout: "omap_del": 0 2024-09-12T03:04:36.338 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:36.338 INFO:teuthology.orchestra.run.smithi005.stdout: "oft": { 2024-09-12T03:04:36.339 INFO:teuthology.orchestra.run.smithi005.stdout: "omap_total_objs": 0, 2024-09-12T03:04:36.339 INFO:teuthology.orchestra.run.smithi005.stdout: "omap_total_kv_pairs": 0, 2024-09-12T03:04:36.339 INFO:teuthology.orchestra.run.smithi005.stdout: "omap_total_updates": 0, 2024-09-12T03:04:36.339 INFO:teuthology.orchestra.run.smithi005.stdout: "omap_total_removes": 0 2024-09-12T03:04:36.339 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:36.339 INFO:teuthology.orchestra.run.smithi005.stdout: "osd-slow-ops": { 2024-09-12T03:04:36.339 INFO:teuthology.orchestra.run.smithi005.stdout: "slow_ops_count": 0 2024-09-12T03:04:36.339 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:36.339 INFO:teuthology.orchestra.run.smithi005.stdout: "purge_queue": { 2024-09-12T03:04:36.339 INFO:teuthology.orchestra.run.smithi005.stdout: "pq_executing_ops": 0, 2024-09-12T03:04:36.339 INFO:teuthology.orchestra.run.smithi005.stdout: "pq_executing_ops_high_water": 0, 2024-09-12T03:04:36.339 INFO:teuthology.orchestra.run.smithi005.stdout: "pq_executing": 0, 2024-09-12T03:04:36.339 INFO:teuthology.orchestra.run.smithi005.stdout: "pq_executing_high_water": 0, 2024-09-12T03:04:36.340 INFO:teuthology.orchestra.run.smithi005.stdout: "pq_executed_ops": 0, 2024-09-12T03:04:36.340 INFO:teuthology.orchestra.run.smithi005.stdout: "pq_executed": 0, 2024-09-12T03:04:36.340 INFO:teuthology.orchestra.run.smithi005.stdout: "pq_item_in_journal": 0 2024-09-12T03:04:36.340 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:36.340 INFO:teuthology.orchestra.run.smithi005.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T03:04:36.340 INFO:teuthology.orchestra.run.smithi005.stdout: "val": 0, 2024-09-12T03:04:36.340 INFO:teuthology.orchestra.run.smithi005.stdout: "max": 104857600, 2024-09-12T03:04:36.340 INFO:teuthology.orchestra.run.smithi005.stdout: "get_started": 0, 2024-09-12T03:04:36.340 INFO:teuthology.orchestra.run.smithi005.stdout: "get": 1557, 2024-09-12T03:04:36.340 INFO:teuthology.orchestra.run.smithi005.stdout: "get_sum": 1122375, 2024-09-12T03:04:36.340 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_fail": 0, 2024-09-12T03:04:36.340 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_success": 1557, 2024-09-12T03:04:36.340 INFO:teuthology.orchestra.run.smithi005.stdout: "take": 0, 2024-09-12T03:04:36.340 INFO:teuthology.orchestra.run.smithi005.stdout: "take_sum": 0, 2024-09-12T03:04:36.341 INFO:teuthology.orchestra.run.smithi005.stdout: "put": 1557, 2024-09-12T03:04:36.341 INFO:teuthology.orchestra.run.smithi005.stdout: "put_sum": 1122375, 2024-09-12T03:04:36.341 INFO:teuthology.orchestra.run.smithi005.stdout: "wait": { 2024-09-12T03:04:36.341 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-12T03:04:36.341 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-12T03:04:36.341 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-12T03:04:36.341 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:36.341 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:36.341 INFO:teuthology.orchestra.run.smithi005.stdout: "throttle-objecter_bytes": { 2024-09-12T03:04:36.341 INFO:teuthology.orchestra.run.smithi005.stdout: "val": 0, 2024-09-12T03:04:36.341 INFO:teuthology.orchestra.run.smithi005.stdout: "max": 104857600, 2024-09-12T03:04:36.341 INFO:teuthology.orchestra.run.smithi005.stdout: "get_started": 0, 2024-09-12T03:04:36.341 INFO:teuthology.orchestra.run.smithi005.stdout: "get": 0, 2024-09-12T03:04:36.342 INFO:teuthology.orchestra.run.smithi005.stdout: "get_sum": 0, 2024-09-12T03:04:36.342 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_fail": 0, 2024-09-12T03:04:36.342 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_success": 0, 2024-09-12T03:04:36.342 INFO:teuthology.orchestra.run.smithi005.stdout: "take": 1396, 2024-09-12T03:04:36.342 INFO:teuthology.orchestra.run.smithi005.stdout: "take_sum": 72070, 2024-09-12T03:04:36.342 INFO:teuthology.orchestra.run.smithi005.stdout: "put": 1396, 2024-09-12T03:04:36.342 INFO:teuthology.orchestra.run.smithi005.stdout: "put_sum": 72070, 2024-09-12T03:04:36.342 INFO:teuthology.orchestra.run.smithi005.stdout: "wait": { 2024-09-12T03:04:36.342 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-12T03:04:36.342 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-12T03:04:36.342 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-12T03:04:36.342 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:36.342 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:36.342 INFO:teuthology.orchestra.run.smithi005.stdout: "throttle-objecter_ops": { 2024-09-12T03:04:36.343 INFO:teuthology.orchestra.run.smithi005.stdout: "val": 0, 2024-09-12T03:04:36.343 INFO:teuthology.orchestra.run.smithi005.stdout: "max": 1024, 2024-09-12T03:04:36.343 INFO:teuthology.orchestra.run.smithi005.stdout: "get_started": 0, 2024-09-12T03:04:36.343 INFO:teuthology.orchestra.run.smithi005.stdout: "get": 0, 2024-09-12T03:04:36.343 INFO:teuthology.orchestra.run.smithi005.stdout: "get_sum": 0, 2024-09-12T03:04:36.343 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_fail": 0, 2024-09-12T03:04:36.343 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_success": 0, 2024-09-12T03:04:36.343 INFO:teuthology.orchestra.run.smithi005.stdout: "take": 1396, 2024-09-12T03:04:36.343 INFO:teuthology.orchestra.run.smithi005.stdout: "take_sum": 1396, 2024-09-12T03:04:36.343 INFO:teuthology.orchestra.run.smithi005.stdout: "put": 1396, 2024-09-12T03:04:36.343 INFO:teuthology.orchestra.run.smithi005.stdout: "put_sum": 1396, 2024-09-12T03:04:36.343 INFO:teuthology.orchestra.run.smithi005.stdout: "wait": { 2024-09-12T03:04:36.343 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-12T03:04:36.344 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-12T03:04:36.344 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-12T03:04:36.344 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:36.344 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:36.344 INFO:teuthology.orchestra.run.smithi005.stdout: "throttle-write_buf_throttle": { 2024-09-12T03:04:36.344 INFO:teuthology.orchestra.run.smithi005.stdout: "val": 0, 2024-09-12T03:04:36.344 INFO:teuthology.orchestra.run.smithi005.stdout: "max": 3758096384, 2024-09-12T03:04:36.344 INFO:teuthology.orchestra.run.smithi005.stdout: "get_started": 0, 2024-09-12T03:04:36.344 INFO:teuthology.orchestra.run.smithi005.stdout: "get": 0, 2024-09-12T03:04:36.344 INFO:teuthology.orchestra.run.smithi005.stdout: "get_sum": 0, 2024-09-12T03:04:36.344 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_fail": 0, 2024-09-12T03:04:36.344 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_success": 0, 2024-09-12T03:04:36.344 INFO:teuthology.orchestra.run.smithi005.stdout: "take": 0, 2024-09-12T03:04:36.345 INFO:teuthology.orchestra.run.smithi005.stdout: "take_sum": 0, 2024-09-12T03:04:36.345 INFO:teuthology.orchestra.run.smithi005.stdout: "put": 0, 2024-09-12T03:04:36.345 INFO:teuthology.orchestra.run.smithi005.stdout: "put_sum": 0, 2024-09-12T03:04:36.345 INFO:teuthology.orchestra.run.smithi005.stdout: "wait": { 2024-09-12T03:04:36.345 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-12T03:04:36.345 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-12T03:04:36.345 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-12T03:04:36.345 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:36.345 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:36.345 INFO:teuthology.orchestra.run.smithi005.stdout: "throttle-write_buf_throttle-0x55f507168b40": { 2024-09-12T03:04:36.345 INFO:teuthology.orchestra.run.smithi005.stdout: "val": 0, 2024-09-12T03:04:36.345 INFO:teuthology.orchestra.run.smithi005.stdout: "max": 3758096384, 2024-09-12T03:04:36.345 INFO:teuthology.orchestra.run.smithi005.stdout: "get_started": 0, 2024-09-12T03:04:36.345 INFO:teuthology.orchestra.run.smithi005.stdout: "get": 0, 2024-09-12T03:04:36.346 INFO:teuthology.orchestra.run.smithi005.stdout: "get_sum": 0, 2024-09-12T03:04:36.346 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_fail": 0, 2024-09-12T03:04:36.346 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_success": 0, 2024-09-12T03:04:36.346 INFO:teuthology.orchestra.run.smithi005.stdout: "take": 0, 2024-09-12T03:04:36.346 INFO:teuthology.orchestra.run.smithi005.stdout: "take_sum": 0, 2024-09-12T03:04:36.346 INFO:teuthology.orchestra.run.smithi005.stdout: "put": 0, 2024-09-12T03:04:36.346 INFO:teuthology.orchestra.run.smithi005.stdout: "put_sum": 0, 2024-09-12T03:04:36.346 INFO:teuthology.orchestra.run.smithi005.stdout: "wait": { 2024-09-12T03:04:36.346 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-12T03:04:36.346 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-12T03:04:36.346 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-12T03:04:36.346 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:36.346 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:36.346 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-12T03:04:37.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:04:36 smithi005 ceph-mon[31197]: pgmap v526: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-12T03:04:37.096 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-09-12T03:04:37.096 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-09-12T03:04:37.096 INFO:tasks.check_counter:Daemon mds.g mds.dir_split=0 2024-09-12T03:04:37.097 DEBUG:tasks.check_counter:Getting stats from j 2024-09-12T03:04:37.097 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph daemon mds.j perf dump 2024-09-12T03:04:37.165 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:04:36 smithi148 ceph-mon[35090]: pgmap v526: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-12T03:04:37.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:04:36 smithi146 ceph-mon[35011]: pgmap v526: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-12T03:04:37.353 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:04:37.871 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2024-09-12T03:04:37.871 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T03:04:37.871 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 151, 2024-09-12T03:04:37.872 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 146, 2024-09-12T03:04:37.872 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 733840, 2024-09-12T03:04:37.872 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 89312, 2024-09-12T03:04:37.872 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 4, 2024-09-12T03:04:37.872 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 0, 2024-09-12T03:04:37.872 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.025280355, 2024-09-12T03:04:37.872 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.010339202, 2024-09-12T03:04:37.873 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.019688031, 2024-09-12T03:04:37.873 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:04:37.873 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:04:37.873 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 145, 2024-09-12T03:04:37.873 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.007487988, 2024-09-12T03:04:37.873 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000051641 2024-09-12T03:04:37.873 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:37.874 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:04:37.874 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-12T03:04:37.874 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-12T03:04:37.874 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-12T03:04:37.874 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:37.874 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 733840, 2024-09-12T03:04:37.874 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 89312 2024-09-12T03:04:37.875 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:37.875 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T03:04:37.875 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 5, 2024-09-12T03:04:37.875 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 113, 2024-09-12T03:04:37.875 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 3184, 2024-09-12T03:04:37.875 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 98512, 2024-09-12T03:04:37.875 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 3, 2024-09-12T03:04:37.876 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 2, 2024-09-12T03:04:37.876 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.010894865, 2024-09-12T03:04:37.876 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.006697151, 2024-09-12T03:04:37.876 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.003255385, 2024-09-12T03:04:37.876 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:04:37.876 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:04:37.876 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 112, 2024-09-12T03:04:37.877 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.009271938, 2024-09-12T03:04:37.877 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000082785 2024-09-12T03:04:37.877 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:37.877 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:04:37.877 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-12T03:04:37.877 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-12T03:04:37.877 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-12T03:04:37.878 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:37.878 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 3184, 2024-09-12T03:04:37.878 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 98512 2024-09-12T03:04:37.878 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:37.878 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T03:04:37.878 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 0, 2024-09-12T03:04:37.878 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 2, 2024-09-12T03:04:37.879 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 0, 2024-09-12T03:04:37.879 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 0, 2024-09-12T03:04:37.879 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 2, 2024-09-12T03:04:37.879 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 1, 2024-09-12T03:04:37.879 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.000382400, 2024-09-12T03:04:37.879 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.000000000, 2024-09-12T03:04:37.879 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.000307879, 2024-09-12T03:04:37.879 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-12T03:04:37.880 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:04:37.880 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-12T03:04:37.880 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-12T03:04:37.880 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-12T03:04:37.880 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:37.880 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:04:37.881 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-12T03:04:37.881 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-12T03:04:37.881 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-12T03:04:37.881 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:37.881 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-12T03:04:37.881 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 0 2024-09-12T03:04:37.881 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:37.881 INFO:teuthology.orchestra.run.smithi005.stdout: "cct": { 2024-09-12T03:04:37.881 INFO:teuthology.orchestra.run.smithi005.stdout: "total_workers": 0, 2024-09-12T03:04:37.881 INFO:teuthology.orchestra.run.smithi005.stdout: "unhealthy_workers": 0 2024-09-12T03:04:37.881 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:37.882 INFO:teuthology.orchestra.run.smithi005.stdout: "mempool": { 2024-09-12T03:04:37.882 INFO:teuthology.orchestra.run.smithi005.stdout: "bloom_filter_bytes": 0, 2024-09-12T03:04:37.882 INFO:teuthology.orchestra.run.smithi005.stdout: "bloom_filter_items": 0, 2024-09-12T03:04:37.882 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T03:04:37.882 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_alloc_items": 0, 2024-09-12T03:04:37.882 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T03:04:37.882 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_data_items": 0, 2024-09-12T03:04:37.883 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T03:04:37.883 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T03:04:37.883 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T03:04:37.883 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T03:04:37.883 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T03:04:37.883 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_other_items": 0, 2024-09-12T03:04:37.883 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T03:04:37.883 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T03:04:37.883 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_extent_bytes": 0, 2024-09-12T03:04:37.884 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_extent_items": 0, 2024-09-12T03:04:37.884 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_blob_bytes": 0, 2024-09-12T03:04:37.884 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_blob_items": 0, 2024-09-12T03:04:37.884 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T03:04:37.884 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T03:04:37.884 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T03:04:37.884 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T03:04:37.884 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T03:04:37.884 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_fsck_items": 0, 2024-09-12T03:04:37.885 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_txc_bytes": 0, 2024-09-12T03:04:37.885 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_txc_items": 0, 2024-09-12T03:04:37.885 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T03:04:37.885 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T03:04:37.885 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_bytes": 0, 2024-09-12T03:04:37.885 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_items": 0, 2024-09-12T03:04:37.885 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_bytes": 0, 2024-09-12T03:04:37.886 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_items": 0, 2024-09-12T03:04:37.886 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T03:04:37.886 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_reader_items": 0, 2024-09-12T03:04:37.886 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T03:04:37.886 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_writer_items": 0, 2024-09-12T03:04:37.886 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_anon_bytes": 148967, 2024-09-12T03:04:37.886 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_anon_items": 29, 2024-09-12T03:04:37.887 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_meta_bytes": 0, 2024-09-12T03:04:37.887 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_meta_items": 0, 2024-09-12T03:04:37.887 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_bytes": 0, 2024-09-12T03:04:37.887 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_items": 0, 2024-09-12T03:04:37.887 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mapbl_bytes": 0, 2024-09-12T03:04:37.887 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mapbl_items": 0, 2024-09-12T03:04:37.887 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pglog_bytes": 0, 2024-09-12T03:04:37.887 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pglog_items": 0, 2024-09-12T03:04:37.887 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_bytes": 0, 2024-09-12T03:04:37.887 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_items": 0, 2024-09-12T03:04:37.887 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T03:04:37.887 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_mapping_items": 0, 2024-09-12T03:04:37.887 INFO:teuthology.orchestra.run.smithi005.stdout: "pgmap_bytes": 0, 2024-09-12T03:04:37.888 INFO:teuthology.orchestra.run.smithi005.stdout: "pgmap_items": 0, 2024-09-12T03:04:37.888 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_co_bytes": 928, 2024-09-12T03:04:37.888 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_co_items": 2, 2024-09-12T03:04:37.888 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_1_bytes": 0, 2024-09-12T03:04:37.888 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_1_items": 0, 2024-09-12T03:04:37.888 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_2_bytes": 0, 2024-09-12T03:04:37.888 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_2_items": 0 2024-09-12T03:04:37.888 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2024-09-12T03:04:37.888 INFO:teuthology.orchestra.run.smithi005.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T03:04:37.888 INFO:teuthology.orchestra.run.smithi005.stdout: "val": 0, 2024-09-12T03:04:37.889 INFO:teuthology.orchestra.run.smithi005.stdout: "max": 104857600, 2024-09-12T03:04:37.889 INFO:teuthology.orchestra.run.smithi005.stdout: "get_started": 0, 2024-09-12T03:04:37.889 INFO:teuthology.orchestra.run.smithi005.stdout: "get": 152, 2024-09-12T03:04:37.889 INFO:teuthology.orchestra.run.smithi005.stdout: "get_sum": 712545, 2024-09-12T03:04:37.889 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_fail": 0, 2024-09-12T03:04:37.889 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_success": 152, 2024-09-12T03:04:37.889 INFO:teuthology.orchestra.run.smithi005.stdout: "take": 0, 2024-09-12T03:04:37.889 INFO:teuthology.orchestra.run.smithi005.stdout: "take_sum": 0, 2024-09-12T03:04:37.889 INFO:teuthology.orchestra.run.smithi005.stdout: "put": 152, 2024-09-12T03:04:37.889 INFO:teuthology.orchestra.run.smithi005.stdout: "put_sum": 712545, 2024-09-12T03:04:37.889 INFO:teuthology.orchestra.run.smithi005.stdout: "wait": { 2024-09-12T03:04:37.889 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2024-09-12T03:04:37.890 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2024-09-12T03:04:37.890 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2024-09-12T03:04:37.890 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:37.890 INFO:teuthology.orchestra.run.smithi005.stdout: } 2024-09-12T03:04:37.890 INFO:teuthology.orchestra.run.smithi005.stdout:} 2024-09-12T03:04:38.559 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-09-12T03:04:38.559 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-09-12T03:04:38.559 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.j 2024-09-12T03:04:38.560 DEBUG:tasks.check_counter:Getting stats from b 2024-09-12T03:04:38.560 DEBUG:teuthology.orchestra.run.smithi146:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph daemon mds.b perf dump 2024-09-12T03:04:38.820 INFO:teuthology.orchestra.run.smithi146.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T03:04:39.165 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:04:38 smithi148 ceph-mon[35090]: pgmap v527: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-12T03:04:39.190 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:04:38 smithi146 ceph-mon[35011]: pgmap v527: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-12T03:04:39.299 INFO:teuthology.orchestra.run.smithi146.stdout:{ 2024-09-12T03:04:39.299 INFO:teuthology.orchestra.run.smithi146.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T03:04:39.299 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_messages": 45, 2024-09-12T03:04:39.299 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_messages": 47, 2024-09-12T03:04:39.299 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_bytes": 11223, 2024-09-12T03:04:39.299 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_bytes": 87303, 2024-09-12T03:04:39.299 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_created_connections": 15, 2024-09-12T03:04:39.300 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_active_connections": 3, 2024-09-12T03:04:39.300 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_total_time": 0.011507727, 2024-09-12T03:04:39.300 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_send_time": 0.001517518, 2024-09-12T03:04:39.300 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_recv_time": 0.006896907, 2024-09-12T03:04:39.300 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_fast_dispatch_time": 0.000517945, 2024-09-12T03:04:39.300 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:04:39.300 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 45, 2024-09-12T03:04:39.300 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.017626889, 2024-09-12T03:04:39.300 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000391708 2024-09-12T03:04:39.300 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:39.300 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:04:39.300 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 39, 2024-09-12T03:04:39.301 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000005764, 2024-09-12T03:04:39.301 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000147 2024-09-12T03:04:39.301 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:39.301 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_encrypted_bytes": 4480, 2024-09-12T03:04:39.301 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_encrypted_bytes": 2784 2024-09-12T03:04:39.301 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:39.301 INFO:teuthology.orchestra.run.smithi146.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T03:04:39.301 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_messages": 220, 2024-09-12T03:04:39.301 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_messages": 202, 2024-09-12T03:04:39.301 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_bytes": 825437, 2024-09-12T03:04:39.301 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_bytes": 292387, 2024-09-12T03:04:39.301 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_created_connections": 21, 2024-09-12T03:04:39.301 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_active_connections": 4, 2024-09-12T03:04:39.301 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_total_time": 0.032667047, 2024-09-12T03:04:39.302 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_send_time": 0.010307818, 2024-09-12T03:04:39.302 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_recv_time": 0.027738719, 2024-09-12T03:04:39.302 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_fast_dispatch_time": 0.000237414, 2024-09-12T03:04:39.302 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:04:39.302 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 202, 2024-09-12T03:04:39.302 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.025392246, 2024-09-12T03:04:39.302 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000125704 2024-09-12T03:04:39.302 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:39.302 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:04:39.302 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 16, 2024-09-12T03:04:39.302 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000001139, 2024-09-12T03:04:39.302 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000071 2024-09-12T03:04:39.303 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:39.303 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_encrypted_bytes": 819840, 2024-09-12T03:04:39.303 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_encrypted_bytes": 91104 2024-09-12T03:04:39.303 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:39.303 INFO:teuthology.orchestra.run.smithi146.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T03:04:39.303 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_messages": 626, 2024-09-12T03:04:39.303 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_messages": 611, 2024-09-12T03:04:39.303 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_bytes": 211909, 2024-09-12T03:04:39.303 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_bytes": 466503, 2024-09-12T03:04:39.303 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_created_connections": 6, 2024-09-12T03:04:39.303 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_active_connections": 4, 2024-09-12T03:04:39.303 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_total_time": 0.053107715, 2024-09-12T03:04:39.303 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_send_time": 0.022508246, 2024-09-12T03:04:39.304 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_recv_time": 0.031193570, 2024-09-12T03:04:39.304 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_fast_dispatch_time": 0.002786764, 2024-09-12T03:04:39.304 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:04:39.304 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 609, 2024-09-12T03:04:39.304 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.027780577, 2024-09-12T03:04:39.304 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000045616 2024-09-12T03:04:39.304 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:39.304 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:04:39.304 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 1087, 2024-09-12T03:04:39.304 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000273344, 2024-09-12T03:04:39.304 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000251 2024-09-12T03:04:39.304 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:39.304 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-12T03:04:39.305 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_encrypted_bytes": 266448 2024-09-12T03:04:39.305 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:39.305 INFO:teuthology.orchestra.run.smithi146.stdout: "cct": { 2024-09-12T03:04:39.305 INFO:teuthology.orchestra.run.smithi146.stdout: "total_workers": 1, 2024-09-12T03:04:39.305 INFO:teuthology.orchestra.run.smithi146.stdout: "unhealthy_workers": 0 2024-09-12T03:04:39.305 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:39.305 INFO:teuthology.orchestra.run.smithi146.stdout: "finisher-MDSRank": { 2024-09-12T03:04:39.305 INFO:teuthology.orchestra.run.smithi146.stdout: "queue_len": 0, 2024-09-12T03:04:39.305 INFO:teuthology.orchestra.run.smithi146.stdout: "complete_latency": { 2024-09-12T03:04:39.305 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 40, 2024-09-12T03:04:39.305 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.001495366, 2024-09-12T03:04:39.305 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000037384 2024-09-12T03:04:39.305 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-12T03:04:39.306 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:39.306 INFO:teuthology.orchestra.run.smithi146.stdout: "finisher-PurgeQueue": { 2024-09-12T03:04:39.306 INFO:teuthology.orchestra.run.smithi146.stdout: "queue_len": 0, 2024-09-12T03:04:39.306 INFO:teuthology.orchestra.run.smithi146.stdout: "complete_latency": { 2024-09-12T03:04:39.306 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 2, 2024-09-12T03:04:39.306 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000015326, 2024-09-12T03:04:39.306 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000007663 2024-09-12T03:04:39.306 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-12T03:04:39.306 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:39.306 INFO:teuthology.orchestra.run.smithi146.stdout: "mds": { 2024-09-12T03:04:39.306 INFO:teuthology.orchestra.run.smithi146.stdout: "request": 8, 2024-09-12T03:04:39.306 INFO:teuthology.orchestra.run.smithi146.stdout: "reply": 0, 2024-09-12T03:04:39.306 INFO:teuthology.orchestra.run.smithi146.stdout: "reply_latency": { 2024-09-12T03:04:39.306 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:39.307 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:39.307 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:39.307 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:39.307 INFO:teuthology.orchestra.run.smithi146.stdout: "slow_reply": 0, 2024-09-12T03:04:39.307 INFO:teuthology.orchestra.run.smithi146.stdout: "forward": 8, 2024-09-12T03:04:39.307 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_fetch_complete": 0, 2024-09-12T03:04:39.307 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_fetch_keys": 0, 2024-09-12T03:04:39.307 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_commit": 11, 2024-09-12T03:04:39.307 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_split": 0, 2024-09-12T03:04:39.307 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_merge": 0, 2024-09-12T03:04:39.307 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes": 15, 2024-09-12T03:04:39.307 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes_top": 0, 2024-09-12T03:04:39.307 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes_bottom": 15, 2024-09-12T03:04:39.307 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes_pin_tail": 0, 2024-09-12T03:04:39.308 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes_pinned": 13, 2024-09-12T03:04:39.308 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes_expired": 6, 2024-09-12T03:04:39.308 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes_with_caps": 0, 2024-09-12T03:04:39.308 INFO:teuthology.orchestra.run.smithi146.stdout: "caps": 0, 2024-09-12T03:04:39.308 INFO:teuthology.orchestra.run.smithi146.stdout: "subtrees": 3, 2024-09-12T03:04:39.308 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse": 20, 2024-09-12T03:04:39.308 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse_hit": 11, 2024-09-12T03:04:39.308 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse_forward": 8, 2024-09-12T03:04:39.308 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse_discover": 1, 2024-09-12T03:04:39.308 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse_dir_fetch": 0, 2024-09-12T03:04:39.308 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse_remote_ino": 0, 2024-09-12T03:04:39.308 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse_lock": 0, 2024-09-12T03:04:39.308 INFO:teuthology.orchestra.run.smithi146.stdout: "load_cent": 0, 2024-09-12T03:04:39.309 INFO:teuthology.orchestra.run.smithi146.stdout: "q": 0, 2024-09-12T03:04:39.309 INFO:teuthology.orchestra.run.smithi146.stdout: "exported": 0, 2024-09-12T03:04:39.309 INFO:teuthology.orchestra.run.smithi146.stdout: "exported_inodes": 0, 2024-09-12T03:04:39.309 INFO:teuthology.orchestra.run.smithi146.stdout: "imported": 0, 2024-09-12T03:04:39.309 INFO:teuthology.orchestra.run.smithi146.stdout: "imported_inodes": 0, 2024-09-12T03:04:39.309 INFO:teuthology.orchestra.run.smithi146.stdout: "openino_dir_fetch": 0, 2024-09-12T03:04:39.309 INFO:teuthology.orchestra.run.smithi146.stdout: "openino_backtrace_fetch": 0, 2024-09-12T03:04:39.309 INFO:teuthology.orchestra.run.smithi146.stdout: "openino_peer_discover": 0, 2024-09-12T03:04:39.309 INFO:teuthology.orchestra.run.smithi146.stdout: "root_rfiles": 4570, 2024-09-12T03:04:39.309 INFO:teuthology.orchestra.run.smithi146.stdout: "root_rbytes": 147481294, 2024-09-12T03:04:39.309 INFO:teuthology.orchestra.run.smithi146.stdout: "root_rsnaps": 0, 2024-09-12T03:04:39.309 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_backtrace_fetch": 0, 2024-09-12T03:04:39.309 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_set_tag": 0, 2024-09-12T03:04:39.309 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_backtrace_repaired": 0, 2024-09-12T03:04:39.310 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_inotable_repaired": 0, 2024-09-12T03:04:39.310 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_dir_inodes": 0, 2024-09-12T03:04:39.310 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_dir_base_inodes": 0, 2024-09-12T03:04:39.310 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_dirfrag_rstats": 0, 2024-09-12T03:04:39.310 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_file_inodes": 0, 2024-09-12T03:04:39.310 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_inode_file_caps": 0, 2024-09-12T03:04:39.310 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_cap_op_revoke": 0, 2024-09-12T03:04:39.310 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_cap_op_grant": 0, 2024-09-12T03:04:39.310 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_cap_op_trunc": 0, 2024-09-12T03:04:39.310 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-12T03:04:39.310 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-12T03:04:39.310 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_client_caps": 0, 2024-09-12T03:04:39.310 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_client_caps_dirty": 0, 2024-09-12T03:04:39.310 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_client_cap_release": 0, 2024-09-12T03:04:39.311 INFO:teuthology.orchestra.run.smithi146.stdout: "process_request_cap_release": 0 2024-09-12T03:04:39.311 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:39.311 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_cache": { 2024-09-12T03:04:39.311 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_update": 0, 2024-09-12T03:04:39.311 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_update_receipt": 3, 2024-09-12T03:04:39.311 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_try_discover": 2, 2024-09-12T03:04:39.311 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_send_discover": 3, 2024-09-12T03:04:39.311 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_handle_discover": 0, 2024-09-12T03:04:39.311 INFO:teuthology.orchestra.run.smithi146.stdout: "num_strays": 0, 2024-09-12T03:04:39.311 INFO:teuthology.orchestra.run.smithi146.stdout: "num_strays_delayed": 0, 2024-09-12T03:04:39.311 INFO:teuthology.orchestra.run.smithi146.stdout: "num_strays_enqueuing": 0, 2024-09-12T03:04:39.311 INFO:teuthology.orchestra.run.smithi146.stdout: "strays_created": 0, 2024-09-12T03:04:39.311 INFO:teuthology.orchestra.run.smithi146.stdout: "strays_enqueued": 0, 2024-09-12T03:04:39.311 INFO:teuthology.orchestra.run.smithi146.stdout: "strays_reintegrated": 0, 2024-09-12T03:04:39.312 INFO:teuthology.orchestra.run.smithi146.stdout: "strays_migrated": 0, 2024-09-12T03:04:39.312 INFO:teuthology.orchestra.run.smithi146.stdout: "num_recovering_processing": 0, 2024-09-12T03:04:39.312 INFO:teuthology.orchestra.run.smithi146.stdout: "num_recovering_enqueued": 0, 2024-09-12T03:04:39.312 INFO:teuthology.orchestra.run.smithi146.stdout: "num_recovering_prioritized": 0, 2024-09-12T03:04:39.312 INFO:teuthology.orchestra.run.smithi146.stdout: "recovery_started": 0, 2024-09-12T03:04:39.312 INFO:teuthology.orchestra.run.smithi146.stdout: "recovery_completed": 0, 2024-09-12T03:04:39.312 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_quiesce_path": 5, 2024-09-12T03:04:39.312 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_quiesce_inode": 26, 2024-09-12T03:04:39.312 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_enqueue_scrub": 0, 2024-09-12T03:04:39.312 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_exportdir": 0, 2024-09-12T03:04:39.312 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_flush": 0, 2024-09-12T03:04:39.312 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_fragmentdir": 0, 2024-09-12T03:04:39.312 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_fragstats": 0, 2024-09-12T03:04:39.313 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_inodestats": 0 2024-09-12T03:04:39.313 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:39.313 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_log": { 2024-09-12T03:04:39.313 INFO:teuthology.orchestra.run.smithi146.stdout: "evlrg": 0, 2024-09-12T03:04:39.313 INFO:teuthology.orchestra.run.smithi146.stdout: "evadd": 5, 2024-09-12T03:04:39.313 INFO:teuthology.orchestra.run.smithi146.stdout: "evex": 0, 2024-09-12T03:04:39.313 INFO:teuthology.orchestra.run.smithi146.stdout: "evtrm": 0, 2024-09-12T03:04:39.313 INFO:teuthology.orchestra.run.smithi146.stdout: "ev": 5, 2024-09-12T03:04:39.313 INFO:teuthology.orchestra.run.smithi146.stdout: "evexg": 0, 2024-09-12T03:04:39.313 INFO:teuthology.orchestra.run.smithi146.stdout: "evexd": 0, 2024-09-12T03:04:39.313 INFO:teuthology.orchestra.run.smithi146.stdout: "segadd": 2, 2024-09-12T03:04:39.313 INFO:teuthology.orchestra.run.smithi146.stdout: "segex": 0, 2024-09-12T03:04:39.313 INFO:teuthology.orchestra.run.smithi146.stdout: "segtrm": 0, 2024-09-12T03:04:39.314 INFO:teuthology.orchestra.run.smithi146.stdout: "seg": 2, 2024-09-12T03:04:39.314 INFO:teuthology.orchestra.run.smithi146.stdout: "segmjr": 2, 2024-09-12T03:04:39.314 INFO:teuthology.orchestra.run.smithi146.stdout: "segexg": 0, 2024-09-12T03:04:39.314 INFO:teuthology.orchestra.run.smithi146.stdout: "segexd": 0, 2024-09-12T03:04:39.314 INFO:teuthology.orchestra.run.smithi146.stdout: "expos": 4194304, 2024-09-12T03:04:39.314 INFO:teuthology.orchestra.run.smithi146.stdout: "wrpos": 4195734, 2024-09-12T03:04:39.314 INFO:teuthology.orchestra.run.smithi146.stdout: "rdpos": 0, 2024-09-12T03:04:39.314 INFO:teuthology.orchestra.run.smithi146.stdout: "jlat": { 2024-09-12T03:04:39.314 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 4, 2024-09-12T03:04:39.314 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.006660125, 2024-09-12T03:04:39.314 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.001665031 2024-09-12T03:04:39.314 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:39.314 INFO:teuthology.orchestra.run.smithi146.stdout: "replayed": 0 2024-09-12T03:04:39.315 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:39.315 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_mem": { 2024-09-12T03:04:39.315 INFO:teuthology.orchestra.run.smithi146.stdout: "ino": 19, 2024-09-12T03:04:39.315 INFO:teuthology.orchestra.run.smithi146.stdout: "ino+": 22, 2024-09-12T03:04:39.315 INFO:teuthology.orchestra.run.smithi146.stdout: "ino-": 3, 2024-09-12T03:04:39.315 INFO:teuthology.orchestra.run.smithi146.stdout: "dir": 18, 2024-09-12T03:04:39.315 INFO:teuthology.orchestra.run.smithi146.stdout: "dir+": 22, 2024-09-12T03:04:39.315 INFO:teuthology.orchestra.run.smithi146.stdout: "dir-": 4, 2024-09-12T03:04:39.315 INFO:teuthology.orchestra.run.smithi146.stdout: "dn": 15, 2024-09-12T03:04:39.315 INFO:teuthology.orchestra.run.smithi146.stdout: "dn+": 21, 2024-09-12T03:04:39.315 INFO:teuthology.orchestra.run.smithi146.stdout: "dn-": 6, 2024-09-12T03:04:39.315 INFO:teuthology.orchestra.run.smithi146.stdout: "cap": 0, 2024-09-12T03:04:39.315 INFO:teuthology.orchestra.run.smithi146.stdout: "cap+": 0, 2024-09-12T03:04:39.315 INFO:teuthology.orchestra.run.smithi146.stdout: "cap-": 0, 2024-09-12T03:04:39.316 INFO:teuthology.orchestra.run.smithi146.stdout: "rss": 50844, 2024-09-12T03:04:39.316 INFO:teuthology.orchestra.run.smithi146.stdout: "heap": 223520 2024-09-12T03:04:39.316 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:39.316 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_server": { 2024-09-12T03:04:39.316 INFO:teuthology.orchestra.run.smithi146.stdout: "dispatch_client_request": 8, 2024-09-12T03:04:39.316 INFO:teuthology.orchestra.run.smithi146.stdout: "dispatch_server_request": 0, 2024-09-12T03:04:39.316 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_client_request": 8, 2024-09-12T03:04:39.316 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_client_session": 9, 2024-09-12T03:04:39.316 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_peer_request": 0, 2024-09-12T03:04:39.316 INFO:teuthology.orchestra.run.smithi146.stdout: "req_create_latency": { 2024-09-12T03:04:39.316 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:39.316 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:39.316 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:39.317 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:39.317 INFO:teuthology.orchestra.run.smithi146.stdout: "req_getattr_latency": { 2024-09-12T03:04:39.317 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:39.317 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:39.317 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:39.317 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:39.317 INFO:teuthology.orchestra.run.smithi146.stdout: "req_getfilelock_latency": { 2024-09-12T03:04:39.317 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:39.317 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:39.317 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:39.317 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:39.317 INFO:teuthology.orchestra.run.smithi146.stdout: "req_link_latency": { 2024-09-12T03:04:39.317 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:39.318 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:39.318 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:39.318 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:39.318 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lookup_latency": { 2024-09-12T03:04:39.318 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:39.318 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:39.318 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:39.318 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:39.318 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lookuphash_latency": { 2024-09-12T03:04:39.318 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:39.318 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:39.318 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:39.318 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:39.318 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lookupino_latency": { 2024-09-12T03:04:39.319 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:39.319 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:39.319 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:39.319 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:39.319 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lookupname_latency": { 2024-09-12T03:04:39.319 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:39.319 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:39.319 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:39.319 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:39.319 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lookupparent_latency": { 2024-09-12T03:04:39.319 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:39.319 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:39.319 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:39.320 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:39.320 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lookupsnap_latency": { 2024-09-12T03:04:39.320 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:39.320 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:39.320 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:39.320 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:39.320 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lssnap_latency": { 2024-09-12T03:04:39.320 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:39.320 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:39.320 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:39.320 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:39.320 INFO:teuthology.orchestra.run.smithi146.stdout: "req_mkdir_latency": { 2024-09-12T03:04:39.320 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:39.320 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:39.321 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:39.321 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:39.321 INFO:teuthology.orchestra.run.smithi146.stdout: "req_mknod_latency": { 2024-09-12T03:04:39.321 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:39.321 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:39.321 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:39.321 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:39.321 INFO:teuthology.orchestra.run.smithi146.stdout: "req_mksnap_latency": { 2024-09-12T03:04:39.321 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:39.321 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:39.321 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:39.321 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:39.322 INFO:teuthology.orchestra.run.smithi146.stdout: "req_open_latency": { 2024-09-12T03:04:39.322 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:39.322 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:39.322 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:39.322 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:39.322 INFO:teuthology.orchestra.run.smithi146.stdout: "req_readdir_latency": { 2024-09-12T03:04:39.322 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:39.322 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:39.322 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:39.322 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:39.322 INFO:teuthology.orchestra.run.smithi146.stdout: "req_rename_latency": { 2024-09-12T03:04:39.322 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:39.322 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:39.323 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:39.323 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:39.323 INFO:teuthology.orchestra.run.smithi146.stdout: "req_renamesnap_latency": { 2024-09-12T03:04:39.323 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:39.323 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:39.323 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:39.323 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:39.323 INFO:teuthology.orchestra.run.smithi146.stdout: "req_snapdiff_latency": { 2024-09-12T03:04:39.323 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:39.323 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:39.323 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:39.323 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:39.323 INFO:teuthology.orchestra.run.smithi146.stdout: "req_rmdir_latency": { 2024-09-12T03:04:39.324 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:39.324 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:39.324 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:39.324 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:39.324 INFO:teuthology.orchestra.run.smithi146.stdout: "req_rmsnap_latency": { 2024-09-12T03:04:39.324 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:39.324 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:39.324 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:39.324 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:39.324 INFO:teuthology.orchestra.run.smithi146.stdout: "req_rmxattr_latency": { 2024-09-12T03:04:39.324 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:39.324 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:39.324 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:39.325 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:39.325 INFO:teuthology.orchestra.run.smithi146.stdout: "req_setattr_latency": { 2024-09-12T03:04:39.325 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:39.325 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:39.325 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:39.325 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:39.325 INFO:teuthology.orchestra.run.smithi146.stdout: "req_setdirlayout_latency": { 2024-09-12T03:04:39.325 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:39.325 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:39.325 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:39.325 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:39.325 INFO:teuthology.orchestra.run.smithi146.stdout: "req_setfilelock_latency": { 2024-09-12T03:04:39.325 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:39.325 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:39.326 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:39.326 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:39.326 INFO:teuthology.orchestra.run.smithi146.stdout: "req_setlayout_latency": { 2024-09-12T03:04:39.326 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:39.326 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:39.326 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:39.326 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:39.326 INFO:teuthology.orchestra.run.smithi146.stdout: "req_setxattr_latency": { 2024-09-12T03:04:39.326 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:39.326 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:39.326 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:39.326 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:39.326 INFO:teuthology.orchestra.run.smithi146.stdout: "req_symlink_latency": { 2024-09-12T03:04:39.327 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:39.327 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:39.327 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:39.327 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:39.327 INFO:teuthology.orchestra.run.smithi146.stdout: "req_unlink_latency": { 2024-09-12T03:04:39.327 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:39.327 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:39.327 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:39.327 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:39.327 INFO:teuthology.orchestra.run.smithi146.stdout: "cap_revoke_eviction": 0, 2024-09-12T03:04:39.327 INFO:teuthology.orchestra.run.smithi146.stdout: "cap_acquisition_throttle": 0, 2024-09-12T03:04:39.327 INFO:teuthology.orchestra.run.smithi146.stdout: "req_getvxattr_latency": { 2024-09-12T03:04:39.327 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:39.328 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:39.328 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:39.328 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-12T03:04:39.328 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:39.328 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_sessions": { 2024-09-12T03:04:39.328 INFO:teuthology.orchestra.run.smithi146.stdout: "session_count": 1, 2024-09-12T03:04:39.328 INFO:teuthology.orchestra.run.smithi146.stdout: "session_add": 2, 2024-09-12T03:04:39.328 INFO:teuthology.orchestra.run.smithi146.stdout: "session_remove": 1, 2024-09-12T03:04:39.328 INFO:teuthology.orchestra.run.smithi146.stdout: "sessions_open": 1, 2024-09-12T03:04:39.328 INFO:teuthology.orchestra.run.smithi146.stdout: "sessions_stale": 0, 2024-09-12T03:04:39.328 INFO:teuthology.orchestra.run.smithi146.stdout: "total_load": 0, 2024-09-12T03:04:39.328 INFO:teuthology.orchestra.run.smithi146.stdout: "average_load": 0, 2024-09-12T03:04:39.328 INFO:teuthology.orchestra.run.smithi146.stdout: "avg_session_uptime": 252, 2024-09-12T03:04:39.328 INFO:teuthology.orchestra.run.smithi146.stdout: "mdthresh_evicted": 0 2024-09-12T03:04:39.329 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:39.329 INFO:teuthology.orchestra.run.smithi146.stdout: "mempool": { 2024-09-12T03:04:39.329 INFO:teuthology.orchestra.run.smithi146.stdout: "bloom_filter_bytes": 0, 2024-09-12T03:04:39.329 INFO:teuthology.orchestra.run.smithi146.stdout: "bloom_filter_items": 0, 2024-09-12T03:04:39.329 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T03:04:39.329 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_alloc_items": 0, 2024-09-12T03:04:39.329 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T03:04:39.329 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_data_items": 0, 2024-09-12T03:04:39.329 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T03:04:39.329 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T03:04:39.329 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T03:04:39.329 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T03:04:39.329 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T03:04:39.329 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_other_items": 0, 2024-09-12T03:04:39.330 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T03:04:39.330 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T03:04:39.330 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_extent_bytes": 0, 2024-09-12T03:04:39.330 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_extent_items": 0, 2024-09-12T03:04:39.330 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_blob_bytes": 0, 2024-09-12T03:04:39.330 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_blob_items": 0, 2024-09-12T03:04:39.330 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T03:04:39.330 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T03:04:39.330 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T03:04:39.330 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T03:04:39.330 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T03:04:39.330 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_fsck_items": 0, 2024-09-12T03:04:39.330 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_txc_bytes": 0, 2024-09-12T03:04:39.331 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_txc_items": 0, 2024-09-12T03:04:39.331 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T03:04:39.331 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T03:04:39.331 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_writing_bytes": 0, 2024-09-12T03:04:39.331 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_writing_items": 0, 2024-09-12T03:04:39.331 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_bytes": 0, 2024-09-12T03:04:39.331 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_items": 0, 2024-09-12T03:04:39.331 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T03:04:39.331 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_file_reader_items": 0, 2024-09-12T03:04:39.331 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T03:04:39.331 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_file_writer_items": 0, 2024-09-12T03:04:39.331 INFO:teuthology.orchestra.run.smithi146.stdout: "buffer_anon_bytes": 221959, 2024-09-12T03:04:39.331 INFO:teuthology.orchestra.run.smithi146.stdout: "buffer_anon_items": 68, 2024-09-12T03:04:39.331 INFO:teuthology.orchestra.run.smithi146.stdout: "buffer_meta_bytes": 0, 2024-09-12T03:04:39.332 INFO:teuthology.orchestra.run.smithi146.stdout: "buffer_meta_items": 0, 2024-09-12T03:04:39.332 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_bytes": 0, 2024-09-12T03:04:39.332 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_items": 0, 2024-09-12T03:04:39.332 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_mapbl_bytes": 0, 2024-09-12T03:04:39.332 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_mapbl_items": 0, 2024-09-12T03:04:39.332 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_pglog_bytes": 0, 2024-09-12T03:04:39.332 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_pglog_items": 0, 2024-09-12T03:04:39.332 INFO:teuthology.orchestra.run.smithi146.stdout: "osdmap_bytes": 7496, 2024-09-12T03:04:39.332 INFO:teuthology.orchestra.run.smithi146.stdout: "osdmap_items": 154, 2024-09-12T03:04:39.332 INFO:teuthology.orchestra.run.smithi146.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T03:04:39.332 INFO:teuthology.orchestra.run.smithi146.stdout: "osdmap_mapping_items": 0, 2024-09-12T03:04:39.332 INFO:teuthology.orchestra.run.smithi146.stdout: "pgmap_bytes": 0, 2024-09-12T03:04:39.332 INFO:teuthology.orchestra.run.smithi146.stdout: "pgmap_items": 0, 2024-09-12T03:04:39.332 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_co_bytes": 76701, 2024-09-12T03:04:39.333 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_co_items": 305, 2024-09-12T03:04:39.333 INFO:teuthology.orchestra.run.smithi146.stdout: "unittest_1_bytes": 0, 2024-09-12T03:04:39.333 INFO:teuthology.orchestra.run.smithi146.stdout: "unittest_1_items": 0, 2024-09-12T03:04:39.333 INFO:teuthology.orchestra.run.smithi146.stdout: "unittest_2_bytes": 0, 2024-09-12T03:04:39.333 INFO:teuthology.orchestra.run.smithi146.stdout: "unittest_2_items": 0 2024-09-12T03:04:39.333 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:39.333 INFO:teuthology.orchestra.run.smithi146.stdout: "objecter": { 2024-09-12T03:04:39.333 INFO:teuthology.orchestra.run.smithi146.stdout: "op_active": 0, 2024-09-12T03:04:39.333 INFO:teuthology.orchestra.run.smithi146.stdout: "op_laggy": 0, 2024-09-12T03:04:39.333 INFO:teuthology.orchestra.run.smithi146.stdout: "op_send": 40, 2024-09-12T03:04:39.333 INFO:teuthology.orchestra.run.smithi146.stdout: "op_send_bytes": 11732, 2024-09-12T03:04:39.333 INFO:teuthology.orchestra.run.smithi146.stdout: "op_resend": 28, 2024-09-12T03:04:39.333 INFO:teuthology.orchestra.run.smithi146.stdout: "op_reply": 40, 2024-09-12T03:04:39.333 INFO:teuthology.orchestra.run.smithi146.stdout: "op_latency": { 2024-09-12T03:04:39.334 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 40, 2024-09-12T03:04:39.334 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.200996841, 2024-09-12T03:04:39.334 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.005024921 2024-09-12T03:04:39.334 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:39.334 INFO:teuthology.orchestra.run.smithi146.stdout: "op_inflight": 0, 2024-09-12T03:04:39.334 INFO:teuthology.orchestra.run.smithi146.stdout: "oplen_avg": { 2024-09-12T03:04:39.334 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 40, 2024-09-12T03:04:39.334 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 62 2024-09-12T03:04:39.334 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:39.334 INFO:teuthology.orchestra.run.smithi146.stdout: "op": 40, 2024-09-12T03:04:39.334 INFO:teuthology.orchestra.run.smithi146.stdout: "op_r": 0, 2024-09-12T03:04:39.334 INFO:teuthology.orchestra.run.smithi146.stdout: "op_w": 40, 2024-09-12T03:04:39.334 INFO:teuthology.orchestra.run.smithi146.stdout: "op_rmw": 0, 2024-09-12T03:04:39.334 INFO:teuthology.orchestra.run.smithi146.stdout: "op_pg": 0, 2024-09-12T03:04:39.335 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_stat": 0, 2024-09-12T03:04:39.335 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_create": 10, 2024-09-12T03:04:39.335 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_read": 0, 2024-09-12T03:04:39.335 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_write": 4, 2024-09-12T03:04:39.335 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_writefull": 7, 2024-09-12T03:04:39.335 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_writesame": 0, 2024-09-12T03:04:39.335 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_append": 0, 2024-09-12T03:04:39.335 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_zero": 0, 2024-09-12T03:04:39.335 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_truncate": 0, 2024-09-12T03:04:39.335 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_delete": 6, 2024-09-12T03:04:39.335 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_mapext": 0, 2024-09-12T03:04:39.335 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_sparse_read": 0, 2024-09-12T03:04:39.335 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_clonerange": 0, 2024-09-12T03:04:39.335 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_getxattr": 0, 2024-09-12T03:04:39.336 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_setxattr": 20, 2024-09-12T03:04:39.336 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_cmpxattr": 0, 2024-09-12T03:04:39.336 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_rmxattr": 0, 2024-09-12T03:04:39.336 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_resetxattrs": 0, 2024-09-12T03:04:39.336 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_call": 0, 2024-09-12T03:04:39.336 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_watch": 0, 2024-09-12T03:04:39.336 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_notify": 0, 2024-09-12T03:04:39.336 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_src_cmpxattr": 0, 2024-09-12T03:04:39.336 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_pgls": 0, 2024-09-12T03:04:39.336 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_pgls_filter": 0, 2024-09-12T03:04:39.336 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_other": 0, 2024-09-12T03:04:39.336 INFO:teuthology.orchestra.run.smithi146.stdout: "linger_active": 0, 2024-09-12T03:04:39.336 INFO:teuthology.orchestra.run.smithi146.stdout: "linger_send": 0, 2024-09-12T03:04:39.336 INFO:teuthology.orchestra.run.smithi146.stdout: "linger_resend": 0, 2024-09-12T03:04:39.337 INFO:teuthology.orchestra.run.smithi146.stdout: "linger_ping": 0, 2024-09-12T03:04:39.337 INFO:teuthology.orchestra.run.smithi146.stdout: "poolop_active": 0, 2024-09-12T03:04:39.337 INFO:teuthology.orchestra.run.smithi146.stdout: "poolop_send": 0, 2024-09-12T03:04:39.337 INFO:teuthology.orchestra.run.smithi146.stdout: "poolop_resend": 0, 2024-09-12T03:04:39.337 INFO:teuthology.orchestra.run.smithi146.stdout: "poolstat_active": 0, 2024-09-12T03:04:39.337 INFO:teuthology.orchestra.run.smithi146.stdout: "poolstat_send": 0, 2024-09-12T03:04:39.337 INFO:teuthology.orchestra.run.smithi146.stdout: "poolstat_resend": 0, 2024-09-12T03:04:39.337 INFO:teuthology.orchestra.run.smithi146.stdout: "statfs_active": 0, 2024-09-12T03:04:39.337 INFO:teuthology.orchestra.run.smithi146.stdout: "statfs_send": 0, 2024-09-12T03:04:39.337 INFO:teuthology.orchestra.run.smithi146.stdout: "statfs_resend": 0, 2024-09-12T03:04:39.337 INFO:teuthology.orchestra.run.smithi146.stdout: "command_active": 0, 2024-09-12T03:04:39.337 INFO:teuthology.orchestra.run.smithi146.stdout: "command_send": 0, 2024-09-12T03:04:39.337 INFO:teuthology.orchestra.run.smithi146.stdout: "command_resend": 0, 2024-09-12T03:04:39.337 INFO:teuthology.orchestra.run.smithi146.stdout: "map_epoch": 83, 2024-09-12T03:04:39.338 INFO:teuthology.orchestra.run.smithi146.stdout: "map_full": 0, 2024-09-12T03:04:39.338 INFO:teuthology.orchestra.run.smithi146.stdout: "map_inc": 2, 2024-09-12T03:04:39.338 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_sessions": 8, 2024-09-12T03:04:39.338 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_session_open": 8, 2024-09-12T03:04:39.338 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_session_close": 0, 2024-09-12T03:04:39.338 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_laggy": 0, 2024-09-12T03:04:39.338 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_wr": 14, 2024-09-12T03:04:39.338 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_rd": 0, 2024-09-12T03:04:39.338 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_del": 1 2024-09-12T03:04:39.338 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:39.338 INFO:teuthology.orchestra.run.smithi146.stdout: "oft": { 2024-09-12T03:04:39.338 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_total_objs": 1, 2024-09-12T03:04:39.338 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_total_kv_pairs": 0, 2024-09-12T03:04:39.339 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_total_updates": 0, 2024-09-12T03:04:39.339 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_total_removes": 0 2024-09-12T03:04:39.339 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:39.339 INFO:teuthology.orchestra.run.smithi146.stdout: "osd-slow-ops": { 2024-09-12T03:04:39.339 INFO:teuthology.orchestra.run.smithi146.stdout: "slow_ops_count": 31 2024-09-12T03:04:39.339 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:39.339 INFO:teuthology.orchestra.run.smithi146.stdout: "purge_queue": { 2024-09-12T03:04:39.339 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_executing_ops": 0, 2024-09-12T03:04:39.339 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_executing_ops_high_water": 0, 2024-09-12T03:04:39.339 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_executing": 0, 2024-09-12T03:04:39.339 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_executing_high_water": 0, 2024-09-12T03:04:39.339 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_executed_ops": 0, 2024-09-12T03:04:39.339 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_executed": 0, 2024-09-12T03:04:39.339 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_item_in_journal": 0 2024-09-12T03:04:39.340 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:39.340 INFO:teuthology.orchestra.run.smithi146.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T03:04:39.340 INFO:teuthology.orchestra.run.smithi146.stdout: "val": 0, 2024-09-12T03:04:39.340 INFO:teuthology.orchestra.run.smithi146.stdout: "max": 104857600, 2024-09-12T03:04:39.340 INFO:teuthology.orchestra.run.smithi146.stdout: "get_started": 0, 2024-09-12T03:04:39.340 INFO:teuthology.orchestra.run.smithi146.stdout: "get": 887, 2024-09-12T03:04:39.340 INFO:teuthology.orchestra.run.smithi146.stdout: "get_sum": 955530, 2024-09-12T03:04:39.340 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_fail": 0, 2024-09-12T03:04:39.340 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_success": 887, 2024-09-12T03:04:39.340 INFO:teuthology.orchestra.run.smithi146.stdout: "take": 0, 2024-09-12T03:04:39.340 INFO:teuthology.orchestra.run.smithi146.stdout: "take_sum": 0, 2024-09-12T03:04:39.340 INFO:teuthology.orchestra.run.smithi146.stdout: "put": 887, 2024-09-12T03:04:39.340 INFO:teuthology.orchestra.run.smithi146.stdout: "put_sum": 955530, 2024-09-12T03:04:39.341 INFO:teuthology.orchestra.run.smithi146.stdout: "wait": { 2024-09-12T03:04:39.341 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:39.341 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:39.341 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:39.341 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-12T03:04:39.341 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:39.341 INFO:teuthology.orchestra.run.smithi146.stdout: "throttle-objecter_bytes": { 2024-09-12T03:04:39.341 INFO:teuthology.orchestra.run.smithi146.stdout: "val": 0, 2024-09-12T03:04:39.341 INFO:teuthology.orchestra.run.smithi146.stdout: "max": 104857600, 2024-09-12T03:04:39.341 INFO:teuthology.orchestra.run.smithi146.stdout: "get_started": 0, 2024-09-12T03:04:39.341 INFO:teuthology.orchestra.run.smithi146.stdout: "get": 0, 2024-09-12T03:04:39.341 INFO:teuthology.orchestra.run.smithi146.stdout: "get_sum": 0, 2024-09-12T03:04:39.341 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_fail": 0, 2024-09-12T03:04:39.341 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_success": 0, 2024-09-12T03:04:39.342 INFO:teuthology.orchestra.run.smithi146.stdout: "take": 40, 2024-09-12T03:04:39.342 INFO:teuthology.orchestra.run.smithi146.stdout: "take_sum": 11732, 2024-09-12T03:04:39.342 INFO:teuthology.orchestra.run.smithi146.stdout: "put": 40, 2024-09-12T03:04:39.342 INFO:teuthology.orchestra.run.smithi146.stdout: "put_sum": 11732, 2024-09-12T03:04:39.342 INFO:teuthology.orchestra.run.smithi146.stdout: "wait": { 2024-09-12T03:04:39.342 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:39.343 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:04:38 smithi005 ceph-mon[31197]: pgmap v527: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-12T03:04:39.343 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:39.343 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:39.343 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-12T03:04:39.343 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:39.343 INFO:teuthology.orchestra.run.smithi146.stdout: "throttle-objecter_ops": { 2024-09-12T03:04:39.343 INFO:teuthology.orchestra.run.smithi146.stdout: "val": 0, 2024-09-12T03:04:39.343 INFO:teuthology.orchestra.run.smithi146.stdout: "max": 1024, 2024-09-12T03:04:39.343 INFO:teuthology.orchestra.run.smithi146.stdout: "get_started": 0, 2024-09-12T03:04:39.343 INFO:teuthology.orchestra.run.smithi146.stdout: "get": 0, 2024-09-12T03:04:39.343 INFO:teuthology.orchestra.run.smithi146.stdout: "get_sum": 0, 2024-09-12T03:04:39.343 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_fail": 0, 2024-09-12T03:04:39.344 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_success": 0, 2024-09-12T03:04:39.344 INFO:teuthology.orchestra.run.smithi146.stdout: "take": 40, 2024-09-12T03:04:39.344 INFO:teuthology.orchestra.run.smithi146.stdout: "take_sum": 40, 2024-09-12T03:04:39.344 INFO:teuthology.orchestra.run.smithi146.stdout: "put": 40, 2024-09-12T03:04:39.344 INFO:teuthology.orchestra.run.smithi146.stdout: "put_sum": 40, 2024-09-12T03:04:39.344 INFO:teuthology.orchestra.run.smithi146.stdout: "wait": { 2024-09-12T03:04:39.344 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:39.344 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:39.344 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:39.344 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-12T03:04:39.344 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:39.344 INFO:teuthology.orchestra.run.smithi146.stdout: "throttle-write_buf_throttle": { 2024-09-12T03:04:39.344 INFO:teuthology.orchestra.run.smithi146.stdout: "val": 0, 2024-09-12T03:04:39.345 INFO:teuthology.orchestra.run.smithi146.stdout: "max": 3758096384, 2024-09-12T03:04:39.345 INFO:teuthology.orchestra.run.smithi146.stdout: "get_started": 0, 2024-09-12T03:04:39.345 INFO:teuthology.orchestra.run.smithi146.stdout: "get": 0, 2024-09-12T03:04:39.345 INFO:teuthology.orchestra.run.smithi146.stdout: "get_sum": 0, 2024-09-12T03:04:39.345 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_fail": 0, 2024-09-12T03:04:39.345 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_success": 0, 2024-09-12T03:04:39.345 INFO:teuthology.orchestra.run.smithi146.stdout: "take": 0, 2024-09-12T03:04:39.345 INFO:teuthology.orchestra.run.smithi146.stdout: "take_sum": 0, 2024-09-12T03:04:39.345 INFO:teuthology.orchestra.run.smithi146.stdout: "put": 0, 2024-09-12T03:04:39.345 INFO:teuthology.orchestra.run.smithi146.stdout: "put_sum": 0, 2024-09-12T03:04:39.345 INFO:teuthology.orchestra.run.smithi146.stdout: "wait": { 2024-09-12T03:04:39.345 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:39.345 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:39.346 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:39.346 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-12T03:04:39.346 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:39.346 INFO:teuthology.orchestra.run.smithi146.stdout: "throttle-write_buf_throttle-0x55e2440da960": { 2024-09-12T03:04:39.346 INFO:teuthology.orchestra.run.smithi146.stdout: "val": 0, 2024-09-12T03:04:39.346 INFO:teuthology.orchestra.run.smithi146.stdout: "max": 3758096384, 2024-09-12T03:04:39.346 INFO:teuthology.orchestra.run.smithi146.stdout: "get_started": 0, 2024-09-12T03:04:39.346 INFO:teuthology.orchestra.run.smithi146.stdout: "get": 5, 2024-09-12T03:04:39.346 INFO:teuthology.orchestra.run.smithi146.stdout: "get_sum": 1430, 2024-09-12T03:04:39.346 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_fail": 0, 2024-09-12T03:04:39.346 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_success": 5, 2024-09-12T03:04:39.348 INFO:teuthology.orchestra.run.smithi146.stdout: "take": 0, 2024-09-12T03:04:39.349 INFO:teuthology.orchestra.run.smithi146.stdout: "take_sum": 0, 2024-09-12T03:04:39.349 INFO:teuthology.orchestra.run.smithi146.stdout: "put": 4, 2024-09-12T03:04:39.349 INFO:teuthology.orchestra.run.smithi146.stdout: "put_sum": 1430, 2024-09-12T03:04:39.349 INFO:teuthology.orchestra.run.smithi146.stdout: "wait": { 2024-09-12T03:04:39.349 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:39.349 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:39.349 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:39.349 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-12T03:04:39.349 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-12T03:04:39.349 INFO:teuthology.orchestra.run.smithi146.stdout:} 2024-09-12T03:04:40.141 INFO:tasks.check_counter:Daemon mds.b mds.exported=0 2024-09-12T03:04:40.141 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2024-09-12T03:04:40.141 INFO:tasks.check_counter:Daemon mds.b mds.dir_split=0 2024-09-12T03:04:40.141 DEBUG:tasks.check_counter:Getting stats from e 2024-09-12T03:04:40.141 DEBUG:teuthology.orchestra.run.smithi146:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph daemon mds.e perf dump 2024-09-12T03:04:40.404 INFO:teuthology.orchestra.run.smithi146.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T03:04:40.896 INFO:teuthology.orchestra.run.smithi146.stdout:{ 2024-09-12T03:04:40.896 INFO:teuthology.orchestra.run.smithi146.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T03:04:40.897 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_messages": 1078, 2024-09-12T03:04:40.897 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_messages": 1080, 2024-09-12T03:04:40.897 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_bytes": 50247291, 2024-09-12T03:04:40.897 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_bytes": 342814, 2024-09-12T03:04:40.897 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_created_connections": 8, 2024-09-12T03:04:40.897 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_active_connections": 4, 2024-09-12T03:04:40.897 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_total_time": 0.111771349, 2024-09-12T03:04:40.897 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_send_time": 0.032558589, 2024-09-12T03:04:40.897 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_recv_time": 0.045747341, 2024-09-12T03:04:40.897 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_fast_dispatch_time": 0.025219450, 2024-09-12T03:04:40.898 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:04:40.898 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 1078, 2024-09-12T03:04:40.898 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.034877203, 2024-09-12T03:04:40.898 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000032353 2024-09-12T03:04:40.898 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:40.898 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:04:40.898 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:40.898 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:40.898 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:40.898 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:40.898 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-12T03:04:40.899 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_encrypted_bytes": 0 2024-09-12T03:04:40.899 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:40.899 INFO:teuthology.orchestra.run.smithi146.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T03:04:40.899 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_messages": 463, 2024-09-12T03:04:40.899 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_messages": 450, 2024-09-12T03:04:40.899 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_bytes": 38634750, 2024-09-12T03:04:40.899 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_bytes": 184528, 2024-09-12T03:04:40.899 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_created_connections": 5, 2024-09-12T03:04:40.899 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_active_connections": 4, 2024-09-12T03:04:40.899 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_total_time": 0.065975472, 2024-09-12T03:04:40.899 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_send_time": 0.015366382, 2024-09-12T03:04:40.899 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_recv_time": 0.049521930, 2024-09-12T03:04:40.900 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_fast_dispatch_time": 0.007277916, 2024-09-12T03:04:40.900 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:04:40.900 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 449, 2024-09-12T03:04:40.900 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.018101273, 2024-09-12T03:04:40.900 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000040314 2024-09-12T03:04:40.900 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:40.900 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:04:40.900 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:40.900 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:40.900 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:40.901 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:40.901 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_encrypted_bytes": 804096, 2024-09-12T03:04:40.901 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_encrypted_bytes": 90016 2024-09-12T03:04:40.901 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:40.901 INFO:teuthology.orchestra.run.smithi146.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T03:04:40.901 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_messages": 416, 2024-09-12T03:04:40.901 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_messages": 523, 2024-09-12T03:04:40.901 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_bytes": 29469607, 2024-09-12T03:04:40.901 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_bytes": 398299, 2024-09-12T03:04:40.901 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_created_connections": 6, 2024-09-12T03:04:40.901 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_active_connections": 5, 2024-09-12T03:04:40.902 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_total_time": 0.057078115, 2024-09-12T03:04:40.902 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_send_time": 0.013450306, 2024-09-12T03:04:40.902 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_recv_time": 0.028602010, 2024-09-12T03:04:40.902 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_fast_dispatch_time": 0.011110705, 2024-09-12T03:04:40.902 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:04:40.902 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 522, 2024-09-12T03:04:40.902 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.021953584, 2024-09-12T03:04:40.902 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000042056 2024-09-12T03:04:40.902 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:40.902 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:04:40.902 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:40.903 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:40.903 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:40.903 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:40.903 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_encrypted_bytes": 3184, 2024-09-12T03:04:40.903 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_encrypted_bytes": 267600 2024-09-12T03:04:40.903 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:40.903 INFO:teuthology.orchestra.run.smithi146.stdout: "cct": { 2024-09-12T03:04:40.903 INFO:teuthology.orchestra.run.smithi146.stdout: "total_workers": 1, 2024-09-12T03:04:40.903 INFO:teuthology.orchestra.run.smithi146.stdout: "unhealthy_workers": 0 2024-09-12T03:04:40.903 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:40.903 INFO:teuthology.orchestra.run.smithi146.stdout: "finisher-MDSRank": { 2024-09-12T03:04:40.904 INFO:teuthology.orchestra.run.smithi146.stdout: "queue_len": 0, 2024-09-12T03:04:40.904 INFO:teuthology.orchestra.run.smithi146.stdout: "complete_latency": { 2024-09-12T03:04:40.904 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 2949, 2024-09-12T03:04:40.904 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.063651936, 2024-09-12T03:04:40.904 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000021584 2024-09-12T03:04:40.904 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-12T03:04:40.904 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:40.904 INFO:teuthology.orchestra.run.smithi146.stdout: "finisher-PurgeQueue": { 2024-09-12T03:04:40.904 INFO:teuthology.orchestra.run.smithi146.stdout: "queue_len": 0, 2024-09-12T03:04:40.904 INFO:teuthology.orchestra.run.smithi146.stdout: "complete_latency": { 2024-09-12T03:04:40.904 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:40.905 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:40.905 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:40.905 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-12T03:04:40.905 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:40.905 INFO:teuthology.orchestra.run.smithi146.stdout: "mds": { 2024-09-12T03:04:40.905 INFO:teuthology.orchestra.run.smithi146.stdout: "request": 0, 2024-09-12T03:04:40.905 INFO:teuthology.orchestra.run.smithi146.stdout: "reply": 0, 2024-09-12T03:04:40.905 INFO:teuthology.orchestra.run.smithi146.stdout: "reply_latency": { 2024-09-12T03:04:40.905 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:40.905 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:40.905 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:40.906 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:40.906 INFO:teuthology.orchestra.run.smithi146.stdout: "slow_reply": 0, 2024-09-12T03:04:40.906 INFO:teuthology.orchestra.run.smithi146.stdout: "forward": 0, 2024-09-12T03:04:40.906 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_fetch_complete": 0, 2024-09-12T03:04:40.906 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_fetch_keys": 0, 2024-09-12T03:04:40.906 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_commit": 0, 2024-09-12T03:04:40.906 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_split": 0, 2024-09-12T03:04:40.906 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_merge": 0, 2024-09-12T03:04:40.906 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes": 7316, 2024-09-12T03:04:40.906 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes_top": 0, 2024-09-12T03:04:40.906 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes_bottom": 7316, 2024-09-12T03:04:40.907 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes_pin_tail": 0, 2024-09-12T03:04:40.907 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes_pinned": 7316, 2024-09-12T03:04:40.907 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes_expired": 0, 2024-09-12T03:04:40.907 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes_with_caps": 0, 2024-09-12T03:04:40.907 INFO:teuthology.orchestra.run.smithi146.stdout: "caps": 0, 2024-09-12T03:04:40.907 INFO:teuthology.orchestra.run.smithi146.stdout: "subtrees": 2, 2024-09-12T03:04:40.907 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse": 0, 2024-09-12T03:04:40.907 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse_hit": 0, 2024-09-12T03:04:40.907 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse_forward": 0, 2024-09-12T03:04:40.907 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse_discover": 0, 2024-09-12T03:04:40.907 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse_dir_fetch": 0, 2024-09-12T03:04:40.908 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse_remote_ino": 0, 2024-09-12T03:04:40.908 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse_lock": 0, 2024-09-12T03:04:40.908 INFO:teuthology.orchestra.run.smithi146.stdout: "load_cent": 0, 2024-09-12T03:04:40.908 INFO:teuthology.orchestra.run.smithi146.stdout: "q": 0, 2024-09-12T03:04:40.908 INFO:teuthology.orchestra.run.smithi146.stdout: "exported": 0, 2024-09-12T03:04:40.908 INFO:teuthology.orchestra.run.smithi146.stdout: "exported_inodes": 0, 2024-09-12T03:04:40.908 INFO:teuthology.orchestra.run.smithi146.stdout: "imported": 0, 2024-09-12T03:04:40.908 INFO:teuthology.orchestra.run.smithi146.stdout: "imported_inodes": 0, 2024-09-12T03:04:40.908 INFO:teuthology.orchestra.run.smithi146.stdout: "openino_dir_fetch": 0, 2024-09-12T03:04:40.908 INFO:teuthology.orchestra.run.smithi146.stdout: "openino_backtrace_fetch": 0, 2024-09-12T03:04:40.908 INFO:teuthology.orchestra.run.smithi146.stdout: "openino_peer_discover": 0, 2024-09-12T03:04:40.909 INFO:teuthology.orchestra.run.smithi146.stdout: "root_rfiles": 4, 2024-09-12T03:04:40.909 INFO:teuthology.orchestra.run.smithi146.stdout: "root_rbytes": 234, 2024-09-12T03:04:40.909 INFO:teuthology.orchestra.run.smithi146.stdout: "root_rsnaps": 0, 2024-09-12T03:04:40.909 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_backtrace_fetch": 0, 2024-09-12T03:04:40.909 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_set_tag": 0, 2024-09-12T03:04:40.909 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_backtrace_repaired": 0, 2024-09-12T03:04:40.909 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_inotable_repaired": 0, 2024-09-12T03:04:40.909 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_dir_inodes": 0, 2024-09-12T03:04:40.909 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_dir_base_inodes": 0, 2024-09-12T03:04:40.909 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_dirfrag_rstats": 0, 2024-09-12T03:04:40.909 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_file_inodes": 0, 2024-09-12T03:04:40.910 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_inode_file_caps": 0, 2024-09-12T03:04:40.910 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_cap_op_revoke": 0, 2024-09-12T03:04:40.910 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_cap_op_grant": 0, 2024-09-12T03:04:40.910 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_cap_op_trunc": 0, 2024-09-12T03:04:40.910 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-12T03:04:40.910 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-12T03:04:40.910 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_client_caps": 0, 2024-09-12T03:04:40.910 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_client_caps_dirty": 0, 2024-09-12T03:04:40.910 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_client_cap_release": 0, 2024-09-12T03:04:40.910 INFO:teuthology.orchestra.run.smithi146.stdout: "process_request_cap_release": 0 2024-09-12T03:04:40.910 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:40.910 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_cache": { 2024-09-12T03:04:40.910 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_update": 0, 2024-09-12T03:04:40.911 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_update_receipt": 0, 2024-09-12T03:04:40.911 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_try_discover": 0, 2024-09-12T03:04:40.911 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_send_discover": 0, 2024-09-12T03:04:40.911 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_handle_discover": 0, 2024-09-12T03:04:40.911 INFO:teuthology.orchestra.run.smithi146.stdout: "num_strays": 0, 2024-09-12T03:04:40.911 INFO:teuthology.orchestra.run.smithi146.stdout: "num_strays_delayed": 0, 2024-09-12T03:04:40.911 INFO:teuthology.orchestra.run.smithi146.stdout: "num_strays_enqueuing": 0, 2024-09-12T03:04:40.911 INFO:teuthology.orchestra.run.smithi146.stdout: "strays_created": 0, 2024-09-12T03:04:40.911 INFO:teuthology.orchestra.run.smithi146.stdout: "strays_enqueued": 0, 2024-09-12T03:04:40.911 INFO:teuthology.orchestra.run.smithi146.stdout: "strays_reintegrated": 0, 2024-09-12T03:04:40.911 INFO:teuthology.orchestra.run.smithi146.stdout: "strays_migrated": 0, 2024-09-12T03:04:40.912 INFO:teuthology.orchestra.run.smithi146.stdout: "num_recovering_processing": 0, 2024-09-12T03:04:40.912 INFO:teuthology.orchestra.run.smithi146.stdout: "num_recovering_enqueued": 0, 2024-09-12T03:04:40.912 INFO:teuthology.orchestra.run.smithi146.stdout: "num_recovering_prioritized": 0, 2024-09-12T03:04:40.912 INFO:teuthology.orchestra.run.smithi146.stdout: "recovery_started": 0, 2024-09-12T03:04:40.912 INFO:teuthology.orchestra.run.smithi146.stdout: "recovery_completed": 0, 2024-09-12T03:04:40.912 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_quiesce_path": 0, 2024-09-12T03:04:40.912 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_quiesce_inode": 0, 2024-09-12T03:04:40.912 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_enqueue_scrub": 0, 2024-09-12T03:04:40.912 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_exportdir": 0, 2024-09-12T03:04:40.912 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_flush": 0, 2024-09-12T03:04:40.912 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_fragmentdir": 0, 2024-09-12T03:04:40.913 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_fragstats": 0, 2024-09-12T03:04:40.913 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_inodestats": 0 2024-09-12T03:04:40.913 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:40.913 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_log": { 2024-09-12T03:04:40.913 INFO:teuthology.orchestra.run.smithi146.stdout: "evlrg": 0, 2024-09-12T03:04:40.913 INFO:teuthology.orchestra.run.smithi146.stdout: "evadd": 0, 2024-09-12T03:04:40.913 INFO:teuthology.orchestra.run.smithi146.stdout: "evex": 0, 2024-09-12T03:04:40.913 INFO:teuthology.orchestra.run.smithi146.stdout: "evtrm": 0, 2024-09-12T03:04:40.913 INFO:teuthology.orchestra.run.smithi146.stdout: "ev": 56323, 2024-09-12T03:04:40.913 INFO:teuthology.orchestra.run.smithi146.stdout: "evexg": 0, 2024-09-12T03:04:40.913 INFO:teuthology.orchestra.run.smithi146.stdout: "evexd": 0, 2024-09-12T03:04:40.913 INFO:teuthology.orchestra.run.smithi146.stdout: "segadd": 0, 2024-09-12T03:04:40.914 INFO:teuthology.orchestra.run.smithi146.stdout: "segex": 0, 2024-09-12T03:04:40.914 INFO:teuthology.orchestra.run.smithi146.stdout: "segtrm": 0, 2024-09-12T03:04:40.914 INFO:teuthology.orchestra.run.smithi146.stdout: "seg": 68, 2024-09-12T03:04:40.914 INFO:teuthology.orchestra.run.smithi146.stdout: "segmjr": 6, 2024-09-12T03:04:40.914 INFO:teuthology.orchestra.run.smithi146.stdout: "segexg": 0, 2024-09-12T03:04:40.914 INFO:teuthology.orchestra.run.smithi146.stdout: "segexd": 0, 2024-09-12T03:04:40.914 INFO:teuthology.orchestra.run.smithi146.stdout: "expos": 4194304, 2024-09-12T03:04:40.914 INFO:teuthology.orchestra.run.smithi146.stdout: "wrpos": 121226016, 2024-09-12T03:04:40.914 INFO:teuthology.orchestra.run.smithi146.stdout: "rdpos": 121225417, 2024-09-12T03:04:40.914 INFO:teuthology.orchestra.run.smithi146.stdout: "jlat": { 2024-09-12T03:04:40.914 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:40.914 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:40.914 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:40.915 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:40.915 INFO:teuthology.orchestra.run.smithi146.stdout: "replayed": 56323 2024-09-12T03:04:40.915 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:40.915 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_mem": { 2024-09-12T03:04:40.915 INFO:teuthology.orchestra.run.smithi146.stdout: "ino": 27, 2024-09-12T03:04:40.915 INFO:teuthology.orchestra.run.smithi146.stdout: "ino+": 7297, 2024-09-12T03:04:40.915 INFO:teuthology.orchestra.run.smithi146.stdout: "ino-": 7270, 2024-09-12T03:04:40.915 INFO:teuthology.orchestra.run.smithi146.stdout: "dir": 22, 2024-09-12T03:04:40.915 INFO:teuthology.orchestra.run.smithi146.stdout: "dir+": 288, 2024-09-12T03:04:40.915 INFO:teuthology.orchestra.run.smithi146.stdout: "dir-": 266, 2024-09-12T03:04:40.915 INFO:teuthology.orchestra.run.smithi146.stdout: "dn": 7316, 2024-09-12T03:04:40.916 INFO:teuthology.orchestra.run.smithi146.stdout: "dn+": 12079, 2024-09-12T03:04:40.916 INFO:teuthology.orchestra.run.smithi146.stdout: "dn-": 4763, 2024-09-12T03:04:40.916 INFO:teuthology.orchestra.run.smithi146.stdout: "cap": 0, 2024-09-12T03:04:40.916 INFO:teuthology.orchestra.run.smithi146.stdout: "cap+": 0, 2024-09-12T03:04:40.916 INFO:teuthology.orchestra.run.smithi146.stdout: "cap-": 0, 2024-09-12T03:04:40.916 INFO:teuthology.orchestra.run.smithi146.stdout: "rss": 70548, 2024-09-12T03:04:40.916 INFO:teuthology.orchestra.run.smithi146.stdout: "heap": 215328 2024-09-12T03:04:40.916 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:40.916 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_server": { 2024-09-12T03:04:40.916 INFO:teuthology.orchestra.run.smithi146.stdout: "dispatch_client_request": 0, 2024-09-12T03:04:40.916 INFO:teuthology.orchestra.run.smithi146.stdout: "dispatch_server_request": 0, 2024-09-12T03:04:40.916 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_client_request": 0, 2024-09-12T03:04:40.917 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_client_session": 0, 2024-09-12T03:04:40.917 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_peer_request": 0, 2024-09-12T03:04:40.917 INFO:teuthology.orchestra.run.smithi146.stdout: "req_create_latency": { 2024-09-12T03:04:40.917 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:40.917 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:40.917 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:40.917 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:40.917 INFO:teuthology.orchestra.run.smithi146.stdout: "req_getattr_latency": { 2024-09-12T03:04:40.917 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:40.917 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:40.917 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:40.918 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:40.918 INFO:teuthology.orchestra.run.smithi146.stdout: "req_getfilelock_latency": { 2024-09-12T03:04:40.918 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:40.918 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:40.918 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:40.918 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:40.918 INFO:teuthology.orchestra.run.smithi146.stdout: "req_link_latency": { 2024-09-12T03:04:40.918 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:40.918 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:40.918 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:40.918 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:40.918 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lookup_latency": { 2024-09-12T03:04:40.918 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:40.919 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:40.919 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:40.919 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:40.919 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lookuphash_latency": { 2024-09-12T03:04:40.919 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:40.919 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:40.919 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:40.919 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:40.919 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lookupino_latency": { 2024-09-12T03:04:40.919 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:40.920 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:40.920 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:40.920 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:40.920 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lookupname_latency": { 2024-09-12T03:04:40.920 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:40.920 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:40.920 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:40.920 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:40.920 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lookupparent_latency": { 2024-09-12T03:04:40.920 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:40.920 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:40.920 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:40.921 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:40.921 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lookupsnap_latency": { 2024-09-12T03:04:40.921 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:40.921 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:40.921 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:40.921 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:40.921 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lssnap_latency": { 2024-09-12T03:04:40.921 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:40.921 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:40.921 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:40.921 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:40.922 INFO:teuthology.orchestra.run.smithi146.stdout: "req_mkdir_latency": { 2024-09-12T03:04:40.922 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:40.922 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:40.922 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:40.922 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:40.922 INFO:teuthology.orchestra.run.smithi146.stdout: "req_mknod_latency": { 2024-09-12T03:04:40.922 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:40.923 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:40.923 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:40.923 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:40.923 INFO:teuthology.orchestra.run.smithi146.stdout: "req_mksnap_latency": { 2024-09-12T03:04:40.923 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:40.923 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:40.923 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:40.923 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:40.923 INFO:teuthology.orchestra.run.smithi146.stdout: "req_open_latency": { 2024-09-12T03:04:40.923 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:40.923 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:40.923 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:40.923 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:40.924 INFO:teuthology.orchestra.run.smithi146.stdout: "req_readdir_latency": { 2024-09-12T03:04:40.924 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:40.924 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:40.924 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:40.924 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:40.924 INFO:teuthology.orchestra.run.smithi146.stdout: "req_rename_latency": { 2024-09-12T03:04:40.924 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:40.924 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:40.924 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:40.924 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:40.924 INFO:teuthology.orchestra.run.smithi146.stdout: "req_renamesnap_latency": { 2024-09-12T03:04:40.924 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:40.925 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:40.925 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:40.925 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:40.925 INFO:teuthology.orchestra.run.smithi146.stdout: "req_snapdiff_latency": { 2024-09-12T03:04:40.925 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:40.925 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:40.925 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:40.925 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:40.925 INFO:teuthology.orchestra.run.smithi146.stdout: "req_rmdir_latency": { 2024-09-12T03:04:40.925 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:40.925 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:40.925 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:40.926 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:40.926 INFO:teuthology.orchestra.run.smithi146.stdout: "req_rmsnap_latency": { 2024-09-12T03:04:40.926 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:40.926 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:40.926 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:40.926 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:40.926 INFO:teuthology.orchestra.run.smithi146.stdout: "req_rmxattr_latency": { 2024-09-12T03:04:40.926 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:40.926 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:40.926 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:40.926 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:40.927 INFO:teuthology.orchestra.run.smithi146.stdout: "req_setattr_latency": { 2024-09-12T03:04:40.927 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:40.927 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:40.927 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:40.927 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:40.927 INFO:teuthology.orchestra.run.smithi146.stdout: "req_setdirlayout_latency": { 2024-09-12T03:04:40.927 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:40.927 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:40.927 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:40.927 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:40.927 INFO:teuthology.orchestra.run.smithi146.stdout: "req_setfilelock_latency": { 2024-09-12T03:04:40.927 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:40.927 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:40.928 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:40.928 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:40.928 INFO:teuthology.orchestra.run.smithi146.stdout: "req_setlayout_latency": { 2024-09-12T03:04:40.928 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:40.928 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:40.928 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:40.928 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:40.928 INFO:teuthology.orchestra.run.smithi146.stdout: "req_setxattr_latency": { 2024-09-12T03:04:40.928 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:40.928 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:40.928 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:40.928 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:40.929 INFO:teuthology.orchestra.run.smithi146.stdout: "req_symlink_latency": { 2024-09-12T03:04:40.929 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:40.929 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:40.929 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:40.929 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:40.929 INFO:teuthology.orchestra.run.smithi146.stdout: "req_unlink_latency": { 2024-09-12T03:04:40.929 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:40.929 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:40.929 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:40.929 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:40.930 INFO:teuthology.orchestra.run.smithi146.stdout: "cap_revoke_eviction": 0, 2024-09-12T03:04:40.930 INFO:teuthology.orchestra.run.smithi146.stdout: "cap_acquisition_throttle": 0, 2024-09-12T03:04:40.930 INFO:teuthology.orchestra.run.smithi146.stdout: "req_getvxattr_latency": { 2024-09-12T03:04:40.930 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:40.930 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:40.930 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:40.930 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-12T03:04:40.930 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:40.930 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_sessions": { 2024-09-12T03:04:40.930 INFO:teuthology.orchestra.run.smithi146.stdout: "session_count": 1, 2024-09-12T03:04:40.930 INFO:teuthology.orchestra.run.smithi146.stdout: "session_add": 3, 2024-09-12T03:04:40.930 INFO:teuthology.orchestra.run.smithi146.stdout: "session_remove": 2, 2024-09-12T03:04:40.931 INFO:teuthology.orchestra.run.smithi146.stdout: "sessions_open": 2, 2024-09-12T03:04:40.931 INFO:teuthology.orchestra.run.smithi146.stdout: "sessions_stale": 0, 2024-09-12T03:04:40.931 INFO:teuthology.orchestra.run.smithi146.stdout: "total_load": 0, 2024-09-12T03:04:40.931 INFO:teuthology.orchestra.run.smithi146.stdout: "average_load": 0, 2024-09-12T03:04:40.931 INFO:teuthology.orchestra.run.smithi146.stdout: "avg_session_uptime": 2895, 2024-09-12T03:04:40.931 INFO:teuthology.orchestra.run.smithi146.stdout: "mdthresh_evicted": 0 2024-09-12T03:04:40.931 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:40.931 INFO:teuthology.orchestra.run.smithi146.stdout: "mempool": { 2024-09-12T03:04:40.931 INFO:teuthology.orchestra.run.smithi146.stdout: "bloom_filter_bytes": 0, 2024-09-12T03:04:40.931 INFO:teuthology.orchestra.run.smithi146.stdout: "bloom_filter_items": 0, 2024-09-12T03:04:40.931 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T03:04:40.932 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_alloc_items": 0, 2024-09-12T03:04:40.932 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T03:04:40.932 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_data_items": 0, 2024-09-12T03:04:40.932 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T03:04:40.932 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T03:04:40.932 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T03:04:40.932 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T03:04:40.932 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T03:04:40.932 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_other_items": 0, 2024-09-12T03:04:40.932 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T03:04:40.932 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T03:04:40.932 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_extent_bytes": 0, 2024-09-12T03:04:40.933 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_extent_items": 0, 2024-09-12T03:04:40.933 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_blob_bytes": 0, 2024-09-12T03:04:40.933 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_blob_items": 0, 2024-09-12T03:04:40.933 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T03:04:40.933 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T03:04:40.933 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T03:04:40.933 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T03:04:40.933 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T03:04:40.933 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_fsck_items": 0, 2024-09-12T03:04:40.933 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_txc_bytes": 0, 2024-09-12T03:04:40.933 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_txc_items": 0, 2024-09-12T03:04:40.933 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T03:04:40.934 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T03:04:40.934 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_writing_bytes": 0, 2024-09-12T03:04:40.934 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_writing_items": 0, 2024-09-12T03:04:40.934 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_bytes": 0, 2024-09-12T03:04:40.934 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_items": 0, 2024-09-12T03:04:40.934 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T03:04:40.934 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_file_reader_items": 0, 2024-09-12T03:04:40.934 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T03:04:40.934 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_file_writer_items": 0, 2024-09-12T03:04:40.934 INFO:teuthology.orchestra.run.smithi146.stdout: "buffer_anon_bytes": 230715, 2024-09-12T03:04:40.934 INFO:teuthology.orchestra.run.smithi146.stdout: "buffer_anon_items": 76, 2024-09-12T03:04:40.935 INFO:teuthology.orchestra.run.smithi146.stdout: "buffer_meta_bytes": 0, 2024-09-12T03:04:40.935 INFO:teuthology.orchestra.run.smithi146.stdout: "buffer_meta_items": 0, 2024-09-12T03:04:40.935 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_bytes": 0, 2024-09-12T03:04:40.935 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_items": 0, 2024-09-12T03:04:40.935 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_mapbl_bytes": 0, 2024-09-12T03:04:40.935 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_mapbl_items": 0, 2024-09-12T03:04:40.935 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_pglog_bytes": 0, 2024-09-12T03:04:40.935 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_pglog_items": 0, 2024-09-12T03:04:40.935 INFO:teuthology.orchestra.run.smithi146.stdout: "osdmap_bytes": 7496, 2024-09-12T03:04:40.935 INFO:teuthology.orchestra.run.smithi146.stdout: "osdmap_items": 154, 2024-09-12T03:04:40.935 INFO:teuthology.orchestra.run.smithi146.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T03:04:40.935 INFO:teuthology.orchestra.run.smithi146.stdout: "osdmap_mapping_items": 0, 2024-09-12T03:04:40.936 INFO:teuthology.orchestra.run.smithi146.stdout: "pgmap_bytes": 0, 2024-09-12T03:04:40.936 INFO:teuthology.orchestra.run.smithi146.stdout: "pgmap_items": 0, 2024-09-12T03:04:40.936 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_co_bytes": 5240087, 2024-09-12T03:04:40.936 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_co_items": 30050, 2024-09-12T03:04:40.936 INFO:teuthology.orchestra.run.smithi146.stdout: "unittest_1_bytes": 0, 2024-09-12T03:04:40.936 INFO:teuthology.orchestra.run.smithi146.stdout: "unittest_1_items": 0, 2024-09-12T03:04:40.936 INFO:teuthology.orchestra.run.smithi146.stdout: "unittest_2_bytes": 0, 2024-09-12T03:04:40.936 INFO:teuthology.orchestra.run.smithi146.stdout: "unittest_2_items": 0 2024-09-12T03:04:40.936 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:40.936 INFO:teuthology.orchestra.run.smithi146.stdout: "objecter": { 2024-09-12T03:04:40.936 INFO:teuthology.orchestra.run.smithi146.stdout: "op_active": 0, 2024-09-12T03:04:40.937 INFO:teuthology.orchestra.run.smithi146.stdout: "op_laggy": 0, 2024-09-12T03:04:40.937 INFO:teuthology.orchestra.run.smithi146.stdout: "op_send": 1786, 2024-09-12T03:04:40.937 INFO:teuthology.orchestra.run.smithi146.stdout: "op_send_bytes": 26, 2024-09-12T03:04:40.937 INFO:teuthology.orchestra.run.smithi146.stdout: "op_resend": 4, 2024-09-12T03:04:40.937 INFO:teuthology.orchestra.run.smithi146.stdout: "op_reply": 1786, 2024-09-12T03:04:40.937 INFO:teuthology.orchestra.run.smithi146.stdout: "op_latency": { 2024-09-12T03:04:40.937 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 1786, 2024-09-12T03:04:40.937 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 1.095982702, 2024-09-12T03:04:40.937 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000613652 2024-09-12T03:04:40.937 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:40.937 INFO:teuthology.orchestra.run.smithi146.stdout: "op_inflight": 0, 2024-09-12T03:04:40.937 INFO:teuthology.orchestra.run.smithi146.stdout: "oplen_avg": { 2024-09-12T03:04:40.938 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 1786, 2024-09-12T03:04:40.938 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 1787 2024-09-12T03:04:40.938 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:40.938 INFO:teuthology.orchestra.run.smithi146.stdout: "op": 1786, 2024-09-12T03:04:40.938 INFO:teuthology.orchestra.run.smithi146.stdout: "op_r": 1786, 2024-09-12T03:04:40.938 INFO:teuthology.orchestra.run.smithi146.stdout: "op_w": 0, 2024-09-12T03:04:40.938 INFO:teuthology.orchestra.run.smithi146.stdout: "op_rmw": 0, 2024-09-12T03:04:40.938 INFO:teuthology.orchestra.run.smithi146.stdout: "op_pg": 0, 2024-09-12T03:04:40.938 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_stat": 1047, 2024-09-12T03:04:40.938 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_create": 0, 2024-09-12T03:04:40.938 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_read": 736, 2024-09-12T03:04:40.939 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_write": 0, 2024-09-12T03:04:40.939 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_writefull": 0, 2024-09-12T03:04:40.939 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_writesame": 0, 2024-09-12T03:04:40.939 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_append": 0, 2024-09-12T03:04:40.939 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_zero": 0, 2024-09-12T03:04:40.939 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_truncate": 0, 2024-09-12T03:04:40.939 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_delete": 0, 2024-09-12T03:04:40.939 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_mapext": 0, 2024-09-12T03:04:40.939 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_sparse_read": 0, 2024-09-12T03:04:40.939 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_clonerange": 0, 2024-09-12T03:04:40.939 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_getxattr": 2, 2024-09-12T03:04:40.940 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_setxattr": 0, 2024-09-12T03:04:40.940 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_cmpxattr": 0, 2024-09-12T03:04:40.940 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_rmxattr": 0, 2024-09-12T03:04:40.940 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_resetxattrs": 0, 2024-09-12T03:04:40.940 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_call": 0, 2024-09-12T03:04:40.940 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_watch": 0, 2024-09-12T03:04:40.940 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_notify": 0, 2024-09-12T03:04:40.940 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_src_cmpxattr": 0, 2024-09-12T03:04:40.940 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_pgls": 0, 2024-09-12T03:04:40.940 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_pgls_filter": 0, 2024-09-12T03:04:40.940 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_other": 0, 2024-09-12T03:04:40.941 INFO:teuthology.orchestra.run.smithi146.stdout: "linger_active": 0, 2024-09-12T03:04:40.941 INFO:teuthology.orchestra.run.smithi146.stdout: "linger_send": 0, 2024-09-12T03:04:40.941 INFO:teuthology.orchestra.run.smithi146.stdout: "linger_resend": 0, 2024-09-12T03:04:40.941 INFO:teuthology.orchestra.run.smithi146.stdout: "linger_ping": 0, 2024-09-12T03:04:40.941 INFO:teuthology.orchestra.run.smithi146.stdout: "poolop_active": 0, 2024-09-12T03:04:40.941 INFO:teuthology.orchestra.run.smithi146.stdout: "poolop_send": 0, 2024-09-12T03:04:40.941 INFO:teuthology.orchestra.run.smithi146.stdout: "poolop_resend": 0, 2024-09-12T03:04:40.941 INFO:teuthology.orchestra.run.smithi146.stdout: "poolstat_active": 0, 2024-09-12T03:04:40.941 INFO:teuthology.orchestra.run.smithi146.stdout: "poolstat_send": 0, 2024-09-12T03:04:40.941 INFO:teuthology.orchestra.run.smithi146.stdout: "poolstat_resend": 0, 2024-09-12T03:04:40.941 INFO:teuthology.orchestra.run.smithi146.stdout: "statfs_active": 0, 2024-09-12T03:04:40.942 INFO:teuthology.orchestra.run.smithi146.stdout: "statfs_send": 0, 2024-09-12T03:04:40.942 INFO:teuthology.orchestra.run.smithi146.stdout: "statfs_resend": 0, 2024-09-12T03:04:40.942 INFO:teuthology.orchestra.run.smithi146.stdout: "command_active": 0, 2024-09-12T03:04:40.942 INFO:teuthology.orchestra.run.smithi146.stdout: "command_send": 0, 2024-09-12T03:04:40.942 INFO:teuthology.orchestra.run.smithi146.stdout: "command_resend": 0, 2024-09-12T03:04:40.942 INFO:teuthology.orchestra.run.smithi146.stdout: "map_epoch": 83, 2024-09-12T03:04:40.942 INFO:teuthology.orchestra.run.smithi146.stdout: "map_full": 0, 2024-09-12T03:04:40.942 INFO:teuthology.orchestra.run.smithi146.stdout: "map_inc": 2, 2024-09-12T03:04:40.942 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_sessions": 10, 2024-09-12T03:04:40.942 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_session_open": 10, 2024-09-12T03:04:40.942 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_session_close": 0, 2024-09-12T03:04:40.942 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_laggy": 0, 2024-09-12T03:04:40.943 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_wr": 0, 2024-09-12T03:04:40.943 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_rd": 2, 2024-09-12T03:04:40.943 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_del": 0 2024-09-12T03:04:40.943 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:40.943 INFO:teuthology.orchestra.run.smithi146.stdout: "oft": { 2024-09-12T03:04:40.943 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_total_objs": 0, 2024-09-12T03:04:40.943 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_total_kv_pairs": 0, 2024-09-12T03:04:40.943 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_total_updates": 0, 2024-09-12T03:04:40.943 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_total_removes": 0 2024-09-12T03:04:40.943 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:40.943 INFO:teuthology.orchestra.run.smithi146.stdout: "osd-slow-ops": { 2024-09-12T03:04:40.943 INFO:teuthology.orchestra.run.smithi146.stdout: "slow_ops_count": 0 2024-09-12T03:04:40.944 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:40.944 INFO:teuthology.orchestra.run.smithi146.stdout: "purge_queue": { 2024-09-12T03:04:40.944 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_executing_ops": 0, 2024-09-12T03:04:40.944 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_executing_ops_high_water": 0, 2024-09-12T03:04:40.944 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_executing": 0, 2024-09-12T03:04:40.944 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_executing_high_water": 0, 2024-09-12T03:04:40.944 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_executed_ops": 0, 2024-09-12T03:04:40.944 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_executed": 0, 2024-09-12T03:04:40.944 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_item_in_journal": 0 2024-09-12T03:04:40.944 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:40.944 INFO:teuthology.orchestra.run.smithi146.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T03:04:40.945 INFO:teuthology.orchestra.run.smithi146.stdout: "val": 0, 2024-09-12T03:04:40.945 INFO:teuthology.orchestra.run.smithi146.stdout: "max": 104857600, 2024-09-12T03:04:40.945 INFO:teuthology.orchestra.run.smithi146.stdout: "get_started": 0, 2024-09-12T03:04:40.945 INFO:teuthology.orchestra.run.smithi146.stdout: "get": 1953, 2024-09-12T03:04:40.945 INFO:teuthology.orchestra.run.smithi146.stdout: "get_sum": 118164399, 2024-09-12T03:04:40.945 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_fail": 0, 2024-09-12T03:04:40.945 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_success": 1953, 2024-09-12T03:04:40.945 INFO:teuthology.orchestra.run.smithi146.stdout: "take": 0, 2024-09-12T03:04:40.945 INFO:teuthology.orchestra.run.smithi146.stdout: "take_sum": 0, 2024-09-12T03:04:40.945 INFO:teuthology.orchestra.run.smithi146.stdout: "put": 1953, 2024-09-12T03:04:40.945 INFO:teuthology.orchestra.run.smithi146.stdout: "put_sum": 118164399, 2024-09-12T03:04:40.946 INFO:teuthology.orchestra.run.smithi146.stdout: "wait": { 2024-09-12T03:04:40.946 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:40.946 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:40.946 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:40.946 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-12T03:04:40.946 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:40.946 INFO:teuthology.orchestra.run.smithi146.stdout: "throttle-objecter_bytes": { 2024-09-12T03:04:40.946 INFO:teuthology.orchestra.run.smithi146.stdout: "val": 0, 2024-09-12T03:04:40.946 INFO:teuthology.orchestra.run.smithi146.stdout: "max": 104857600, 2024-09-12T03:04:40.946 INFO:teuthology.orchestra.run.smithi146.stdout: "get_started": 0, 2024-09-12T03:04:40.947 INFO:teuthology.orchestra.run.smithi146.stdout: "get": 0, 2024-09-12T03:04:40.947 INFO:teuthology.orchestra.run.smithi146.stdout: "get_sum": 0, 2024-09-12T03:04:40.947 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_fail": 0, 2024-09-12T03:04:40.947 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_success": 0, 2024-09-12T03:04:40.947 INFO:teuthology.orchestra.run.smithi146.stdout: "take": 1786, 2024-09-12T03:04:40.947 INFO:teuthology.orchestra.run.smithi146.stdout: "take_sum": 117048810, 2024-09-12T03:04:40.947 INFO:teuthology.orchestra.run.smithi146.stdout: "put": 1786, 2024-09-12T03:04:40.947 INFO:teuthology.orchestra.run.smithi146.stdout: "put_sum": 117048810, 2024-09-12T03:04:40.947 INFO:teuthology.orchestra.run.smithi146.stdout: "wait": { 2024-09-12T03:04:40.947 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:40.947 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:40.948 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:40.948 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-12T03:04:40.948 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:40.948 INFO:teuthology.orchestra.run.smithi146.stdout: "throttle-objecter_ops": { 2024-09-12T03:04:40.948 INFO:teuthology.orchestra.run.smithi146.stdout: "val": 0, 2024-09-12T03:04:40.948 INFO:teuthology.orchestra.run.smithi146.stdout: "max": 1024, 2024-09-12T03:04:40.948 INFO:teuthology.orchestra.run.smithi146.stdout: "get_started": 0, 2024-09-12T03:04:40.948 INFO:teuthology.orchestra.run.smithi146.stdout: "get": 0, 2024-09-12T03:04:40.948 INFO:teuthology.orchestra.run.smithi146.stdout: "get_sum": 0, 2024-09-12T03:04:40.948 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_fail": 0, 2024-09-12T03:04:40.948 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_success": 0, 2024-09-12T03:04:40.949 INFO:teuthology.orchestra.run.smithi146.stdout: "take": 1786, 2024-09-12T03:04:40.949 INFO:teuthology.orchestra.run.smithi146.stdout: "take_sum": 1786, 2024-09-12T03:04:40.949 INFO:teuthology.orchestra.run.smithi146.stdout: "put": 1786, 2024-09-12T03:04:40.949 INFO:teuthology.orchestra.run.smithi146.stdout: "put_sum": 1786, 2024-09-12T03:04:40.949 INFO:teuthology.orchestra.run.smithi146.stdout: "wait": { 2024-09-12T03:04:40.949 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:40.949 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:40.949 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:40.949 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-12T03:04:40.949 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:40.949 INFO:teuthology.orchestra.run.smithi146.stdout: "throttle-write_buf_throttle": { 2024-09-12T03:04:40.950 INFO:teuthology.orchestra.run.smithi146.stdout: "val": 0, 2024-09-12T03:04:40.950 INFO:teuthology.orchestra.run.smithi146.stdout: "max": 3758096384, 2024-09-12T03:04:40.950 INFO:teuthology.orchestra.run.smithi146.stdout: "get_started": 0, 2024-09-12T03:04:40.950 INFO:teuthology.orchestra.run.smithi146.stdout: "get": 0, 2024-09-12T03:04:40.950 INFO:teuthology.orchestra.run.smithi146.stdout: "get_sum": 0, 2024-09-12T03:04:40.950 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_fail": 0, 2024-09-12T03:04:40.950 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_success": 0, 2024-09-12T03:04:40.950 INFO:teuthology.orchestra.run.smithi146.stdout: "take": 0, 2024-09-12T03:04:40.950 INFO:teuthology.orchestra.run.smithi146.stdout: "take_sum": 0, 2024-09-12T03:04:40.950 INFO:teuthology.orchestra.run.smithi146.stdout: "put": 0, 2024-09-12T03:04:40.950 INFO:teuthology.orchestra.run.smithi146.stdout: "put_sum": 0, 2024-09-12T03:04:40.950 INFO:teuthology.orchestra.run.smithi146.stdout: "wait": { 2024-09-12T03:04:40.950 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:40.951 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:40.951 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:40.951 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-12T03:04:40.951 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:40.951 INFO:teuthology.orchestra.run.smithi146.stdout: "throttle-write_buf_throttle-0x55e7b1e1eb40": { 2024-09-12T03:04:40.951 INFO:teuthology.orchestra.run.smithi146.stdout: "val": 0, 2024-09-12T03:04:40.951 INFO:teuthology.orchestra.run.smithi146.stdout: "max": 3758096384, 2024-09-12T03:04:40.951 INFO:teuthology.orchestra.run.smithi146.stdout: "get_started": 0, 2024-09-12T03:04:40.951 INFO:teuthology.orchestra.run.smithi146.stdout: "get": 0, 2024-09-12T03:04:40.951 INFO:teuthology.orchestra.run.smithi146.stdout: "get_sum": 0, 2024-09-12T03:04:40.951 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_fail": 0, 2024-09-12T03:04:40.951 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_success": 0, 2024-09-12T03:04:40.952 INFO:teuthology.orchestra.run.smithi146.stdout: "take": 0, 2024-09-12T03:04:40.952 INFO:teuthology.orchestra.run.smithi146.stdout: "take_sum": 0, 2024-09-12T03:04:40.952 INFO:teuthology.orchestra.run.smithi146.stdout: "put": 0, 2024-09-12T03:04:40.952 INFO:teuthology.orchestra.run.smithi146.stdout: "put_sum": 0, 2024-09-12T03:04:40.952 INFO:teuthology.orchestra.run.smithi146.stdout: "wait": { 2024-09-12T03:04:40.952 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:40.952 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:40.952 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:40.952 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-12T03:04:40.952 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-12T03:04:40.952 INFO:teuthology.orchestra.run.smithi146.stdout:} 2024-09-12T03:04:41.165 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:04:40 smithi148 ceph-mon[35090]: pgmap v528: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-12T03:04:41.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:04:40 smithi146 ceph-mon[35011]: pgmap v528: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-12T03:04:41.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:04:40 smithi005 ceph-mon[31197]: pgmap v528: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-12T03:04:41.764 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-09-12T03:04:41.764 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-09-12T03:04:41.764 INFO:tasks.check_counter:Daemon mds.e mds.dir_split=0 2024-09-12T03:04:41.764 DEBUG:tasks.check_counter:Getting stats from h 2024-09-12T03:04:41.765 DEBUG:teuthology.orchestra.run.smithi146:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph daemon mds.h perf dump 2024-09-12T03:04:42.014 INFO:teuthology.orchestra.run.smithi146.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T03:04:42.512 INFO:teuthology.orchestra.run.smithi146.stdout:{ 2024-09-12T03:04:42.512 INFO:teuthology.orchestra.run.smithi146.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T03:04:42.513 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_messages": 148, 2024-09-12T03:04:42.513 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_messages": 179, 2024-09-12T03:04:42.513 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_bytes": 43201, 2024-09-12T03:04:42.513 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_bytes": 241409, 2024-09-12T03:04:42.513 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_created_connections": 18, 2024-09-12T03:04:42.514 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_active_connections": 5, 2024-09-12T03:04:42.514 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_total_time": 0.022950587, 2024-09-12T03:04:42.514 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_send_time": 0.005825920, 2024-09-12T03:04:42.514 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_recv_time": 0.012669749, 2024-09-12T03:04:42.514 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_fast_dispatch_time": 0.001803254, 2024-09-12T03:04:42.514 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:04:42.515 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 175, 2024-09-12T03:04:42.515 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.017909871, 2024-09-12T03:04:42.515 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000102342 2024-09-12T03:04:42.515 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:42.515 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:04:42.515 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 133, 2024-09-12T03:04:42.516 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000020732, 2024-09-12T03:04:42.516 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000155 2024-09-12T03:04:42.516 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:42.516 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_encrypted_bytes": 22832, 2024-09-12T03:04:42.516 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_encrypted_bytes": 44400 2024-09-12T03:04:42.517 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:42.517 INFO:teuthology.orchestra.run.smithi146.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T03:04:42.517 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_messages": 1273, 2024-09-12T03:04:42.517 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_messages": 966, 2024-09-12T03:04:42.517 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_bytes": 395789, 2024-09-12T03:04:42.517 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_bytes": 586198, 2024-09-12T03:04:42.517 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_created_connections": 10, 2024-09-12T03:04:42.517 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_active_connections": 5, 2024-09-12T03:04:42.517 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_total_time": 0.087425845, 2024-09-12T03:04:42.517 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_send_time": 0.034235321, 2024-09-12T03:04:42.518 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_recv_time": 0.061337253, 2024-09-12T03:04:42.518 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_fast_dispatch_time": 0.003379327, 2024-09-12T03:04:42.518 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:04:42.518 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 967, 2024-09-12T03:04:42.518 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.043862668, 2024-09-12T03:04:42.518 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000045359 2024-09-12T03:04:42.518 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:42.518 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:04:42.518 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 2030, 2024-09-12T03:04:42.518 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000375867, 2024-09-12T03:04:42.518 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000185 2024-09-12T03:04:42.518 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:42.519 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_encrypted_bytes": 3184, 2024-09-12T03:04:42.519 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_encrypted_bytes": 267744 2024-09-12T03:04:42.519 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:42.519 INFO:teuthology.orchestra.run.smithi146.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T03:04:42.519 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_messages": 260, 2024-09-12T03:04:42.519 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_messages": 247, 2024-09-12T03:04:42.519 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_bytes": 827303, 2024-09-12T03:04:42.519 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_bytes": 266412, 2024-09-12T03:04:42.519 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_created_connections": 19, 2024-09-12T03:04:42.519 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_active_connections": 6, 2024-09-12T03:04:42.520 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_total_time": 0.034675428, 2024-09-12T03:04:42.520 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_send_time": 0.011591377, 2024-09-12T03:04:42.520 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_recv_time": 0.030172532, 2024-09-12T03:04:42.520 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_fast_dispatch_time": 0.000935188, 2024-09-12T03:04:42.520 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:04:42.520 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 246, 2024-09-12T03:04:42.520 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.025670240, 2024-09-12T03:04:42.520 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000104350 2024-09-12T03:04:42.520 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:42.520 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:04:42.520 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 19, 2024-09-12T03:04:42.520 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000002221, 2024-09-12T03:04:42.521 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000116 2024-09-12T03:04:42.521 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:42.521 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_encrypted_bytes": 814544, 2024-09-12T03:04:42.521 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_encrypted_bytes": 100976 2024-09-12T03:04:42.521 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:42.521 INFO:teuthology.orchestra.run.smithi146.stdout: "cct": { 2024-09-12T03:04:42.521 INFO:teuthology.orchestra.run.smithi146.stdout: "total_workers": 1, 2024-09-12T03:04:42.521 INFO:teuthology.orchestra.run.smithi146.stdout: "unhealthy_workers": 0 2024-09-12T03:04:42.521 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:42.521 INFO:teuthology.orchestra.run.smithi146.stdout: "finisher-MDSRank": { 2024-09-12T03:04:42.521 INFO:teuthology.orchestra.run.smithi146.stdout: "queue_len": 0, 2024-09-12T03:04:42.521 INFO:teuthology.orchestra.run.smithi146.stdout: "complete_latency": { 2024-09-12T03:04:42.522 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 135, 2024-09-12T03:04:42.522 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.013610663, 2024-09-12T03:04:42.522 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000100819 2024-09-12T03:04:42.522 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-12T03:04:42.522 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:42.522 INFO:teuthology.orchestra.run.smithi146.stdout: "finisher-PurgeQueue": { 2024-09-12T03:04:42.522 INFO:teuthology.orchestra.run.smithi146.stdout: "queue_len": 0, 2024-09-12T03:04:42.522 INFO:teuthology.orchestra.run.smithi146.stdout: "complete_latency": { 2024-09-12T03:04:42.522 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 33, 2024-09-12T03:04:42.522 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.001041785, 2024-09-12T03:04:42.522 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000031569 2024-09-12T03:04:42.522 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-12T03:04:42.523 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:42.523 INFO:teuthology.orchestra.run.smithi146.stdout: "mds": { 2024-09-12T03:04:42.523 INFO:teuthology.orchestra.run.smithi146.stdout: "request": 26, 2024-09-12T03:04:42.523 INFO:teuthology.orchestra.run.smithi146.stdout: "reply": 19, 2024-09-12T03:04:42.523 INFO:teuthology.orchestra.run.smithi146.stdout: "reply_latency": { 2024-09-12T03:04:42.523 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 19, 2024-09-12T03:04:42.523 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.020544154, 2024-09-12T03:04:42.523 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.001081271 2024-09-12T03:04:42.523 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:42.523 INFO:teuthology.orchestra.run.smithi146.stdout: "slow_reply": 0, 2024-09-12T03:04:42.523 INFO:teuthology.orchestra.run.smithi146.stdout: "forward": 7, 2024-09-12T03:04:42.523 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_fetch_complete": 0, 2024-09-12T03:04:42.524 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_fetch_keys": 0, 2024-09-12T03:04:42.524 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_commit": 11, 2024-09-12T03:04:42.524 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_split": 0, 2024-09-12T03:04:42.524 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_merge": 0, 2024-09-12T03:04:42.524 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes": 16, 2024-09-12T03:04:42.524 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes_top": 0, 2024-09-12T03:04:42.524 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes_bottom": 16, 2024-09-12T03:04:42.524 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes_pin_tail": 0, 2024-09-12T03:04:42.524 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes_pinned": 14, 2024-09-12T03:04:42.524 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes_expired": 18, 2024-09-12T03:04:42.524 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes_with_caps": 1, 2024-09-12T03:04:42.524 INFO:teuthology.orchestra.run.smithi146.stdout: "caps": 1, 2024-09-12T03:04:42.524 INFO:teuthology.orchestra.run.smithi146.stdout: "subtrees": 3, 2024-09-12T03:04:42.525 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse": 64, 2024-09-12T03:04:42.525 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse_hit": 36, 2024-09-12T03:04:42.525 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse_forward": 6, 2024-09-12T03:04:42.525 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse_discover": 3, 2024-09-12T03:04:42.525 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse_dir_fetch": 0, 2024-09-12T03:04:42.525 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse_remote_ino": 0, 2024-09-12T03:04:42.525 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse_lock": 0, 2024-09-12T03:04:42.525 INFO:teuthology.orchestra.run.smithi146.stdout: "load_cent": 0, 2024-09-12T03:04:42.525 INFO:teuthology.orchestra.run.smithi146.stdout: "q": 0, 2024-09-12T03:04:42.525 INFO:teuthology.orchestra.run.smithi146.stdout: "exported": 2, 2024-09-12T03:04:42.525 INFO:teuthology.orchestra.run.smithi146.stdout: "exported_inodes": 0, 2024-09-12T03:04:42.526 INFO:teuthology.orchestra.run.smithi146.stdout: "imported": 2, 2024-09-12T03:04:42.526 INFO:teuthology.orchestra.run.smithi146.stdout: "imported_inodes": 5, 2024-09-12T03:04:42.526 INFO:teuthology.orchestra.run.smithi146.stdout: "openino_dir_fetch": 0, 2024-09-12T03:04:42.526 INFO:teuthology.orchestra.run.smithi146.stdout: "openino_backtrace_fetch": 29, 2024-09-12T03:04:42.526 INFO:teuthology.orchestra.run.smithi146.stdout: "openino_peer_discover": 0, 2024-09-12T03:04:42.526 INFO:teuthology.orchestra.run.smithi146.stdout: "root_rfiles": 4570, 2024-09-12T03:04:42.526 INFO:teuthology.orchestra.run.smithi146.stdout: "root_rbytes": 147481294, 2024-09-12T03:04:42.526 INFO:teuthology.orchestra.run.smithi146.stdout: "root_rsnaps": 0, 2024-09-12T03:04:42.526 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_backtrace_fetch": 29, 2024-09-12T03:04:42.526 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_set_tag": 0, 2024-09-12T03:04:42.526 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_backtrace_repaired": 0, 2024-09-12T03:04:42.526 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_inotable_repaired": 0, 2024-09-12T03:04:42.526 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_dir_inodes": 0, 2024-09-12T03:04:42.527 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_dir_base_inodes": 0, 2024-09-12T03:04:42.527 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_dirfrag_rstats": 0, 2024-09-12T03:04:42.527 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_file_inodes": 29, 2024-09-12T03:04:42.527 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_inode_file_caps": 0, 2024-09-12T03:04:42.527 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_cap_op_revoke": 22, 2024-09-12T03:04:42.527 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_cap_op_grant": 22, 2024-09-12T03:04:42.527 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_cap_op_trunc": 0, 2024-09-12T03:04:42.527 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-12T03:04:42.527 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-12T03:04:42.527 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_client_caps": 29, 2024-09-12T03:04:42.528 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_client_caps_dirty": 6, 2024-09-12T03:04:42.528 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_client_cap_release": 5, 2024-09-12T03:04:42.528 INFO:teuthology.orchestra.run.smithi146.stdout: "process_request_cap_release": 16 2024-09-12T03:04:42.528 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:42.528 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_cache": { 2024-09-12T03:04:42.528 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_update": 0, 2024-09-12T03:04:42.528 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_update_receipt": 0, 2024-09-12T03:04:42.528 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_try_discover": 0, 2024-09-12T03:04:42.528 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_send_discover": 4, 2024-09-12T03:04:42.528 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_handle_discover": 0, 2024-09-12T03:04:42.528 INFO:teuthology.orchestra.run.smithi146.stdout: "num_strays": 0, 2024-09-12T03:04:42.528 INFO:teuthology.orchestra.run.smithi146.stdout: "num_strays_delayed": 0, 2024-09-12T03:04:42.529 INFO:teuthology.orchestra.run.smithi146.stdout: "num_strays_enqueuing": 0, 2024-09-12T03:04:42.529 INFO:teuthology.orchestra.run.smithi146.stdout: "strays_created": 6, 2024-09-12T03:04:42.529 INFO:teuthology.orchestra.run.smithi146.stdout: "strays_enqueued": 6, 2024-09-12T03:04:42.529 INFO:teuthology.orchestra.run.smithi146.stdout: "strays_reintegrated": 0, 2024-09-12T03:04:42.529 INFO:teuthology.orchestra.run.smithi146.stdout: "strays_migrated": 0, 2024-09-12T03:04:42.529 INFO:teuthology.orchestra.run.smithi146.stdout: "num_recovering_processing": 0, 2024-09-12T03:04:42.529 INFO:teuthology.orchestra.run.smithi146.stdout: "num_recovering_enqueued": 0, 2024-09-12T03:04:42.529 INFO:teuthology.orchestra.run.smithi146.stdout: "num_recovering_prioritized": 0, 2024-09-12T03:04:42.529 INFO:teuthology.orchestra.run.smithi146.stdout: "recovery_started": 0, 2024-09-12T03:04:42.529 INFO:teuthology.orchestra.run.smithi146.stdout: "recovery_completed": 0, 2024-09-12T03:04:42.529 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_quiesce_path": 5, 2024-09-12T03:04:42.529 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_quiesce_inode": 80, 2024-09-12T03:04:42.529 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_enqueue_scrub": 0, 2024-09-12T03:04:42.530 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_exportdir": 2, 2024-09-12T03:04:42.530 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_flush": 0, 2024-09-12T03:04:42.530 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_fragmentdir": 0, 2024-09-12T03:04:42.530 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_fragstats": 0, 2024-09-12T03:04:42.530 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_inodestats": 0 2024-09-12T03:04:42.530 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:42.530 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_log": { 2024-09-12T03:04:42.530 INFO:teuthology.orchestra.run.smithi146.stdout: "evlrg": 0, 2024-09-12T03:04:42.530 INFO:teuthology.orchestra.run.smithi146.stdout: "evadd": 47, 2024-09-12T03:04:42.530 INFO:teuthology.orchestra.run.smithi146.stdout: "evex": 0, 2024-09-12T03:04:42.530 INFO:teuthology.orchestra.run.smithi146.stdout: "evtrm": 0, 2024-09-12T03:04:42.530 INFO:teuthology.orchestra.run.smithi146.stdout: "ev": 47, 2024-09-12T03:04:42.530 INFO:teuthology.orchestra.run.smithi146.stdout: "evexg": 0, 2024-09-12T03:04:42.530 INFO:teuthology.orchestra.run.smithi146.stdout: "evexd": 0, 2024-09-12T03:04:42.531 INFO:teuthology.orchestra.run.smithi146.stdout: "segadd": 2, 2024-09-12T03:04:42.531 INFO:teuthology.orchestra.run.smithi146.stdout: "segex": 0, 2024-09-12T03:04:42.531 INFO:teuthology.orchestra.run.smithi146.stdout: "segtrm": 0, 2024-09-12T03:04:42.531 INFO:teuthology.orchestra.run.smithi146.stdout: "seg": 2, 2024-09-12T03:04:42.531 INFO:teuthology.orchestra.run.smithi146.stdout: "segmjr": 2, 2024-09-12T03:04:42.531 INFO:teuthology.orchestra.run.smithi146.stdout: "segexg": 0, 2024-09-12T03:04:42.531 INFO:teuthology.orchestra.run.smithi146.stdout: "segexd": 0, 2024-09-12T03:04:42.531 INFO:teuthology.orchestra.run.smithi146.stdout: "expos": 4194304, 2024-09-12T03:04:42.531 INFO:teuthology.orchestra.run.smithi146.stdout: "wrpos": 4266369, 2024-09-12T03:04:42.531 INFO:teuthology.orchestra.run.smithi146.stdout: "rdpos": 0, 2024-09-12T03:04:42.531 INFO:teuthology.orchestra.run.smithi146.stdout: "jlat": { 2024-09-12T03:04:42.531 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 28, 2024-09-12T03:04:42.531 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.058023163, 2024-09-12T03:04:42.532 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.002072255 2024-09-12T03:04:42.532 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:42.532 INFO:teuthology.orchestra.run.smithi146.stdout: "replayed": 0 2024-09-12T03:04:42.532 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:42.532 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_mem": { 2024-09-12T03:04:42.532 INFO:teuthology.orchestra.run.smithi146.stdout: "ino": 20, 2024-09-12T03:04:42.532 INFO:teuthology.orchestra.run.smithi146.stdout: "ino+": 35, 2024-09-12T03:04:42.532 INFO:teuthology.orchestra.run.smithi146.stdout: "ino-": 15, 2024-09-12T03:04:42.532 INFO:teuthology.orchestra.run.smithi146.stdout: "dir": 19, 2024-09-12T03:04:42.532 INFO:teuthology.orchestra.run.smithi146.stdout: "dir+": 29, 2024-09-12T03:04:42.532 INFO:teuthology.orchestra.run.smithi146.stdout: "dir-": 10, 2024-09-12T03:04:42.532 INFO:teuthology.orchestra.run.smithi146.stdout: "dn": 16, 2024-09-12T03:04:42.532 INFO:teuthology.orchestra.run.smithi146.stdout: "dn+": 46, 2024-09-12T03:04:42.533 INFO:teuthology.orchestra.run.smithi146.stdout: "dn-": 30, 2024-09-12T03:04:42.533 INFO:teuthology.orchestra.run.smithi146.stdout: "cap": 1, 2024-09-12T03:04:42.533 INFO:teuthology.orchestra.run.smithi146.stdout: "cap+": 16, 2024-09-12T03:04:42.533 INFO:teuthology.orchestra.run.smithi146.stdout: "cap-": 15, 2024-09-12T03:04:42.533 INFO:teuthology.orchestra.run.smithi146.stdout: "rss": 51584, 2024-09-12T03:04:42.533 INFO:teuthology.orchestra.run.smithi146.stdout: "heap": 223520 2024-09-12T03:04:42.533 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:42.533 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_server": { 2024-09-12T03:04:42.533 INFO:teuthology.orchestra.run.smithi146.stdout: "dispatch_client_request": 43, 2024-09-12T03:04:42.533 INFO:teuthology.orchestra.run.smithi146.stdout: "dispatch_server_request": 2, 2024-09-12T03:04:42.533 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_client_request": 26, 2024-09-12T03:04:42.533 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_client_session": 10, 2024-09-12T03:04:42.533 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_peer_request": 14, 2024-09-12T03:04:42.533 INFO:teuthology.orchestra.run.smithi146.stdout: "req_create_latency": { 2024-09-12T03:04:42.534 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 2, 2024-09-12T03:04:42.534 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.005435473, 2024-09-12T03:04:42.534 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.002717736 2024-09-12T03:04:42.534 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:42.534 INFO:teuthology.orchestra.run.smithi146.stdout: "req_getattr_latency": { 2024-09-12T03:04:42.534 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 1, 2024-09-12T03:04:42.534 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000910787, 2024-09-12T03:04:42.534 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000910787 2024-09-12T03:04:42.534 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:42.534 INFO:teuthology.orchestra.run.smithi146.stdout: "req_getfilelock_latency": { 2024-09-12T03:04:42.534 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:42.534 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:42.534 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:42.534 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:42.535 INFO:teuthology.orchestra.run.smithi146.stdout: "req_link_latency": { 2024-09-12T03:04:42.535 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:42.535 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:42.535 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:42.535 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:42.535 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lookup_latency": { 2024-09-12T03:04:42.535 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 7, 2024-09-12T03:04:42.535 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.001839659, 2024-09-12T03:04:42.535 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000262808 2024-09-12T03:04:42.535 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:42.535 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lookuphash_latency": { 2024-09-12T03:04:42.536 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:42.536 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:42.536 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:42.536 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:42.536 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lookupino_latency": { 2024-09-12T03:04:42.536 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:42.536 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:42.536 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:42.536 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:42.536 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lookupname_latency": { 2024-09-12T03:04:42.536 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:42.536 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:42.537 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:42.537 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:42.537 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lookupparent_latency": { 2024-09-12T03:04:42.537 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:42.537 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:42.537 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:42.537 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:42.537 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lookupsnap_latency": { 2024-09-12T03:04:42.537 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:42.537 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:42.537 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:42.537 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:42.537 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lssnap_latency": { 2024-09-12T03:04:42.537 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:42.538 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:42.538 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:42.538 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:42.539 INFO:teuthology.orchestra.run.smithi146.stdout: "req_mkdir_latency": { 2024-09-12T03:04:42.539 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:42.539 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:42.539 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:42.539 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:42.539 INFO:teuthology.orchestra.run.smithi146.stdout: "req_mknod_latency": { 2024-09-12T03:04:42.539 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:42.539 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:42.539 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:42.539 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:42.540 INFO:teuthology.orchestra.run.smithi146.stdout: "req_mksnap_latency": { 2024-09-12T03:04:42.540 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:42.540 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:42.540 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:42.540 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:42.540 INFO:teuthology.orchestra.run.smithi146.stdout: "req_open_latency": { 2024-09-12T03:04:42.540 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:42.540 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:42.541 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:42.541 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:42.541 INFO:teuthology.orchestra.run.smithi146.stdout: "req_readdir_latency": { 2024-09-12T03:04:42.541 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 3, 2024-09-12T03:04:42.541 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.001087500, 2024-09-12T03:04:42.541 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000362500 2024-09-12T03:04:42.542 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:42.542 INFO:teuthology.orchestra.run.smithi146.stdout: "req_rename_latency": { 2024-09-12T03:04:42.542 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:42.542 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:42.542 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:42.542 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:42.543 INFO:teuthology.orchestra.run.smithi146.stdout: "req_renamesnap_latency": { 2024-09-12T03:04:42.543 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:42.543 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:42.543 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:42.543 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:42.543 INFO:teuthology.orchestra.run.smithi146.stdout: "req_snapdiff_latency": { 2024-09-12T03:04:42.544 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:42.544 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:42.544 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:42.544 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:42.544 INFO:teuthology.orchestra.run.smithi146.stdout: "req_rmdir_latency": { 2024-09-12T03:04:42.544 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:42.544 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:42.545 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:42.545 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:42.545 INFO:teuthology.orchestra.run.smithi146.stdout: "req_rmsnap_latency": { 2024-09-12T03:04:42.545 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:42.545 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:42.545 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:42.546 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:42.546 INFO:teuthology.orchestra.run.smithi146.stdout: "req_rmxattr_latency": { 2024-09-12T03:04:42.546 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:42.546 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:42.546 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:42.546 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:42.547 INFO:teuthology.orchestra.run.smithi146.stdout: "req_setattr_latency": { 2024-09-12T03:04:42.547 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:42.547 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:42.547 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:42.547 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:42.547 INFO:teuthology.orchestra.run.smithi146.stdout: "req_setdirlayout_latency": { 2024-09-12T03:04:42.548 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:42.548 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:42.548 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:42.548 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:42.548 INFO:teuthology.orchestra.run.smithi146.stdout: "req_setfilelock_latency": { 2024-09-12T03:04:42.548 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:42.549 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:42.549 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:42.549 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:42.549 INFO:teuthology.orchestra.run.smithi146.stdout: "req_setlayout_latency": { 2024-09-12T03:04:42.549 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:42.550 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:42.550 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:42.550 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:42.550 INFO:teuthology.orchestra.run.smithi146.stdout: "req_setxattr_latency": { 2024-09-12T03:04:42.550 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:42.550 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:42.551 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:42.551 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:42.551 INFO:teuthology.orchestra.run.smithi146.stdout: "req_symlink_latency": { 2024-09-12T03:04:42.551 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:42.551 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:42.551 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:42.551 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:42.552 INFO:teuthology.orchestra.run.smithi146.stdout: "req_unlink_latency": { 2024-09-12T03:04:42.552 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 6, 2024-09-12T03:04:42.552 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.011270735, 2024-09-12T03:04:42.552 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.001878455 2024-09-12T03:04:42.552 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:42.552 INFO:teuthology.orchestra.run.smithi146.stdout: "cap_revoke_eviction": 0, 2024-09-12T03:04:42.553 INFO:teuthology.orchestra.run.smithi146.stdout: "cap_acquisition_throttle": 0, 2024-09-12T03:04:42.553 INFO:teuthology.orchestra.run.smithi146.stdout: "req_getvxattr_latency": { 2024-09-12T03:04:42.553 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:42.553 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:42.553 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:42.553 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-12T03:04:42.554 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:42.554 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_sessions": { 2024-09-12T03:04:42.554 INFO:teuthology.orchestra.run.smithi146.stdout: "session_count": 1, 2024-09-12T03:04:42.554 INFO:teuthology.orchestra.run.smithi146.stdout: "session_add": 2, 2024-09-12T03:04:42.554 INFO:teuthology.orchestra.run.smithi146.stdout: "session_remove": 1, 2024-09-12T03:04:42.554 INFO:teuthology.orchestra.run.smithi146.stdout: "sessions_open": 1, 2024-09-12T03:04:42.555 INFO:teuthology.orchestra.run.smithi146.stdout: "sessions_stale": 0, 2024-09-12T03:04:42.555 INFO:teuthology.orchestra.run.smithi146.stdout: "total_load": 10, 2024-09-12T03:04:42.555 INFO:teuthology.orchestra.run.smithi146.stdout: "average_load": 10, 2024-09-12T03:04:42.555 INFO:teuthology.orchestra.run.smithi146.stdout: "avg_session_uptime": 398, 2024-09-12T03:04:42.555 INFO:teuthology.orchestra.run.smithi146.stdout: "mdthresh_evicted": 0 2024-09-12T03:04:42.555 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:42.556 INFO:teuthology.orchestra.run.smithi146.stdout: "mempool": { 2024-09-12T03:04:42.556 INFO:teuthology.orchestra.run.smithi146.stdout: "bloom_filter_bytes": 0, 2024-09-12T03:04:42.556 INFO:teuthology.orchestra.run.smithi146.stdout: "bloom_filter_items": 0, 2024-09-12T03:04:42.556 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T03:04:42.556 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_alloc_items": 0, 2024-09-12T03:04:42.557 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T03:04:42.557 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_data_items": 0, 2024-09-12T03:04:42.557 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T03:04:42.557 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T03:04:42.557 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T03:04:42.557 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T03:04:42.558 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T03:04:42.558 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_other_items": 0, 2024-09-12T03:04:42.558 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T03:04:42.558 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T03:04:42.558 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_extent_bytes": 0, 2024-09-12T03:04:42.558 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_extent_items": 0, 2024-09-12T03:04:42.559 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_blob_bytes": 0, 2024-09-12T03:04:42.559 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_blob_items": 0, 2024-09-12T03:04:42.559 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T03:04:42.559 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T03:04:42.559 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T03:04:42.559 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T03:04:42.560 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T03:04:42.560 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_fsck_items": 0, 2024-09-12T03:04:42.560 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_txc_bytes": 0, 2024-09-12T03:04:42.560 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_txc_items": 0, 2024-09-12T03:04:42.560 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T03:04:42.561 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T03:04:42.561 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_writing_bytes": 0, 2024-09-12T03:04:42.561 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_writing_items": 0, 2024-09-12T03:04:42.561 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_bytes": 0, 2024-09-12T03:04:42.561 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_items": 0, 2024-09-12T03:04:42.561 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T03:04:42.562 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_file_reader_items": 0, 2024-09-12T03:04:42.562 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T03:04:42.562 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_file_writer_items": 0, 2024-09-12T03:04:42.562 INFO:teuthology.orchestra.run.smithi146.stdout: "buffer_anon_bytes": 279127, 2024-09-12T03:04:42.562 INFO:teuthology.orchestra.run.smithi146.stdout: "buffer_anon_items": 92, 2024-09-12T03:04:42.562 INFO:teuthology.orchestra.run.smithi146.stdout: "buffer_meta_bytes": 0, 2024-09-12T03:04:42.563 INFO:teuthology.orchestra.run.smithi146.stdout: "buffer_meta_items": 0, 2024-09-12T03:04:42.563 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_bytes": 0, 2024-09-12T03:04:42.563 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_items": 0, 2024-09-12T03:04:42.563 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_mapbl_bytes": 0, 2024-09-12T03:04:42.563 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_mapbl_items": 0, 2024-09-12T03:04:42.563 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_pglog_bytes": 0, 2024-09-12T03:04:42.564 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_pglog_items": 0, 2024-09-12T03:04:42.564 INFO:teuthology.orchestra.run.smithi146.stdout: "osdmap_bytes": 7496, 2024-09-12T03:04:42.564 INFO:teuthology.orchestra.run.smithi146.stdout: "osdmap_items": 154, 2024-09-12T03:04:42.564 INFO:teuthology.orchestra.run.smithi146.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T03:04:42.564 INFO:teuthology.orchestra.run.smithi146.stdout: "osdmap_mapping_items": 0, 2024-09-12T03:04:42.564 INFO:teuthology.orchestra.run.smithi146.stdout: "pgmap_bytes": 0, 2024-09-12T03:04:42.565 INFO:teuthology.orchestra.run.smithi146.stdout: "pgmap_items": 0, 2024-09-12T03:04:42.565 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_co_bytes": 81381, 2024-09-12T03:04:42.565 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_co_items": 335, 2024-09-12T03:04:42.565 INFO:teuthology.orchestra.run.smithi146.stdout: "unittest_1_bytes": 0, 2024-09-12T03:04:42.565 INFO:teuthology.orchestra.run.smithi146.stdout: "unittest_1_items": 0, 2024-09-12T03:04:42.565 INFO:teuthology.orchestra.run.smithi146.stdout: "unittest_2_bytes": 0, 2024-09-12T03:04:42.566 INFO:teuthology.orchestra.run.smithi146.stdout: "unittest_2_items": 0 2024-09-12T03:04:42.566 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:42.566 INFO:teuthology.orchestra.run.smithi146.stdout: "objecter": { 2024-09-12T03:04:42.566 INFO:teuthology.orchestra.run.smithi146.stdout: "op_active": 0, 2024-09-12T03:04:42.566 INFO:teuthology.orchestra.run.smithi146.stdout: "op_laggy": 0, 2024-09-12T03:04:42.566 INFO:teuthology.orchestra.run.smithi146.stdout: "op_send": 136, 2024-09-12T03:04:42.567 INFO:teuthology.orchestra.run.smithi146.stdout: "op_send_bytes": 86766, 2024-09-12T03:04:42.567 INFO:teuthology.orchestra.run.smithi146.stdout: "op_resend": 28, 2024-09-12T03:04:42.567 INFO:teuthology.orchestra.run.smithi146.stdout: "op_reply": 136, 2024-09-12T03:04:42.567 INFO:teuthology.orchestra.run.smithi146.stdout: "op_latency": { 2024-09-12T03:04:42.567 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 136, 2024-09-12T03:04:42.567 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.355994369, 2024-09-12T03:04:42.568 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.002617605 2024-09-12T03:04:42.568 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:42.568 INFO:teuthology.orchestra.run.smithi146.stdout: "op_inflight": 0, 2024-09-12T03:04:42.568 INFO:teuthology.orchestra.run.smithi146.stdout: "oplen_avg": { 2024-09-12T03:04:42.568 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 136, 2024-09-12T03:04:42.568 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 175 2024-09-12T03:04:42.569 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:42.569 INFO:teuthology.orchestra.run.smithi146.stdout: "op": 136, 2024-09-12T03:04:42.569 INFO:teuthology.orchestra.run.smithi146.stdout: "op_r": 32, 2024-09-12T03:04:42.569 INFO:teuthology.orchestra.run.smithi146.stdout: "op_w": 104, 2024-09-12T03:04:42.569 INFO:teuthology.orchestra.run.smithi146.stdout: "op_rmw": 0, 2024-09-12T03:04:42.570 INFO:teuthology.orchestra.run.smithi146.stdout: "op_pg": 0, 2024-09-12T03:04:42.570 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_stat": 0, 2024-09-12T03:04:42.570 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_create": 10, 2024-09-12T03:04:42.570 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_read": 3, 2024-09-12T03:04:42.570 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_write": 31, 2024-09-12T03:04:42.570 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_writefull": 16, 2024-09-12T03:04:42.571 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_writesame": 0, 2024-09-12T03:04:42.571 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_append": 0, 2024-09-12T03:04:42.571 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_zero": 0, 2024-09-12T03:04:42.571 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_truncate": 0, 2024-09-12T03:04:42.571 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_delete": 18, 2024-09-12T03:04:42.571 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_mapext": 0, 2024-09-12T03:04:42.572 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_sparse_read": 0, 2024-09-12T03:04:42.572 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_clonerange": 0, 2024-09-12T03:04:42.572 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_getxattr": 29, 2024-09-12T03:04:42.572 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_setxattr": 20, 2024-09-12T03:04:42.572 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_cmpxattr": 0, 2024-09-12T03:04:42.572 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_rmxattr": 0, 2024-09-12T03:04:42.573 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_resetxattrs": 0, 2024-09-12T03:04:42.573 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_call": 0, 2024-09-12T03:04:42.573 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_watch": 0, 2024-09-12T03:04:42.573 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_notify": 0, 2024-09-12T03:04:42.573 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_src_cmpxattr": 0, 2024-09-12T03:04:42.573 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_pgls": 0, 2024-09-12T03:04:42.573 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_pgls_filter": 0, 2024-09-12T03:04:42.574 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_other": 0, 2024-09-12T03:04:42.574 INFO:teuthology.orchestra.run.smithi146.stdout: "linger_active": 0, 2024-09-12T03:04:42.574 INFO:teuthology.orchestra.run.smithi146.stdout: "linger_send": 0, 2024-09-12T03:04:42.574 INFO:teuthology.orchestra.run.smithi146.stdout: "linger_resend": 0, 2024-09-12T03:04:42.574 INFO:teuthology.orchestra.run.smithi146.stdout: "linger_ping": 0, 2024-09-12T03:04:42.574 INFO:teuthology.orchestra.run.smithi146.stdout: "poolop_active": 0, 2024-09-12T03:04:42.575 INFO:teuthology.orchestra.run.smithi146.stdout: "poolop_send": 0, 2024-09-12T03:04:42.575 INFO:teuthology.orchestra.run.smithi146.stdout: "poolop_resend": 0, 2024-09-12T03:04:42.575 INFO:teuthology.orchestra.run.smithi146.stdout: "poolstat_active": 0, 2024-09-12T03:04:42.575 INFO:teuthology.orchestra.run.smithi146.stdout: "poolstat_send": 0, 2024-09-12T03:04:42.575 INFO:teuthology.orchestra.run.smithi146.stdout: "poolstat_resend": 0, 2024-09-12T03:04:42.575 INFO:teuthology.orchestra.run.smithi146.stdout: "statfs_active": 0, 2024-09-12T03:04:42.576 INFO:teuthology.orchestra.run.smithi146.stdout: "statfs_send": 0, 2024-09-12T03:04:42.576 INFO:teuthology.orchestra.run.smithi146.stdout: "statfs_resend": 0, 2024-09-12T03:04:42.576 INFO:teuthology.orchestra.run.smithi146.stdout: "command_active": 0, 2024-09-12T03:04:42.576 INFO:teuthology.orchestra.run.smithi146.stdout: "command_send": 0, 2024-09-12T03:04:42.576 INFO:teuthology.orchestra.run.smithi146.stdout: "command_resend": 0, 2024-09-12T03:04:42.577 INFO:teuthology.orchestra.run.smithi146.stdout: "map_epoch": 83, 2024-09-12T03:04:42.577 INFO:teuthology.orchestra.run.smithi146.stdout: "map_full": 0, 2024-09-12T03:04:42.577 INFO:teuthology.orchestra.run.smithi146.stdout: "map_inc": 2, 2024-09-12T03:04:42.577 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_sessions": 12, 2024-09-12T03:04:42.577 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_session_open": 12, 2024-09-12T03:04:42.577 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_session_close": 0, 2024-09-12T03:04:42.577 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_laggy": 0, 2024-09-12T03:04:42.578 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_wr": 39, 2024-09-12T03:04:42.578 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_rd": 0, 2024-09-12T03:04:42.578 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_del": 9 2024-09-12T03:04:42.578 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:42.578 INFO:teuthology.orchestra.run.smithi146.stdout: "oft": { 2024-09-12T03:04:42.578 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_total_objs": 1, 2024-09-12T03:04:42.578 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_total_kv_pairs": 0, 2024-09-12T03:04:42.579 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_total_updates": 0, 2024-09-12T03:04:42.579 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_total_removes": 0 2024-09-12T03:04:42.579 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:42.579 INFO:teuthology.orchestra.run.smithi146.stdout: "osd-slow-ops": { 2024-09-12T03:04:42.579 INFO:teuthology.orchestra.run.smithi146.stdout: "slow_ops_count": 85 2024-09-12T03:04:42.579 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:42.579 INFO:teuthology.orchestra.run.smithi146.stdout: "purge_queue": { 2024-09-12T03:04:42.579 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_executing_ops": 0, 2024-09-12T03:04:42.579 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_executing_ops_high_water": 5, 2024-09-12T03:04:42.579 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_executing": 0, 2024-09-12T03:04:42.580 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_executing_high_water": 5, 2024-09-12T03:04:42.580 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_executed_ops": 6, 2024-09-12T03:04:42.580 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_executed": 6, 2024-09-12T03:04:42.580 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_item_in_journal": 0 2024-09-12T03:04:42.580 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:42.580 INFO:teuthology.orchestra.run.smithi146.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T03:04:42.580 INFO:teuthology.orchestra.run.smithi146.stdout: "val": 0, 2024-09-12T03:04:42.580 INFO:teuthology.orchestra.run.smithi146.stdout: "max": 104857600, 2024-09-12T03:04:42.580 INFO:teuthology.orchestra.run.smithi146.stdout: "get_started": 0, 2024-09-12T03:04:42.580 INFO:teuthology.orchestra.run.smithi146.stdout: "get": 1677, 2024-09-12T03:04:42.581 INFO:teuthology.orchestra.run.smithi146.stdout: "get_sum": 1098943, 2024-09-12T03:04:42.581 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_fail": 0, 2024-09-12T03:04:42.581 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_success": 1677, 2024-09-12T03:04:42.581 INFO:teuthology.orchestra.run.smithi146.stdout: "take": 0, 2024-09-12T03:04:42.581 INFO:teuthology.orchestra.run.smithi146.stdout: "take_sum": 0, 2024-09-12T03:04:42.581 INFO:teuthology.orchestra.run.smithi146.stdout: "put": 1677, 2024-09-12T03:04:42.581 INFO:teuthology.orchestra.run.smithi146.stdout: "put_sum": 1098943, 2024-09-12T03:04:42.581 INFO:teuthology.orchestra.run.smithi146.stdout: "wait": { 2024-09-12T03:04:42.581 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:42.581 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:42.582 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:42.582 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-12T03:04:42.582 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:42.582 INFO:teuthology.orchestra.run.smithi146.stdout: "throttle-objecter_bytes": { 2024-09-12T03:04:42.582 INFO:teuthology.orchestra.run.smithi146.stdout: "val": 0, 2024-09-12T03:04:42.582 INFO:teuthology.orchestra.run.smithi146.stdout: "max": 104857600, 2024-09-12T03:04:42.582 INFO:teuthology.orchestra.run.smithi146.stdout: "get_started": 0, 2024-09-12T03:04:42.582 INFO:teuthology.orchestra.run.smithi146.stdout: "get": 0, 2024-09-12T03:04:42.582 INFO:teuthology.orchestra.run.smithi146.stdout: "get_sum": 0, 2024-09-12T03:04:42.583 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_fail": 0, 2024-09-12T03:04:42.583 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_success": 0, 2024-09-12T03:04:42.583 INFO:teuthology.orchestra.run.smithi146.stdout: "take": 136, 2024-09-12T03:04:42.583 INFO:teuthology.orchestra.run.smithi146.stdout: "take_sum": 87372, 2024-09-12T03:04:42.583 INFO:teuthology.orchestra.run.smithi146.stdout: "put": 136, 2024-09-12T03:04:42.583 INFO:teuthology.orchestra.run.smithi146.stdout: "put_sum": 87372, 2024-09-12T03:04:42.583 INFO:teuthology.orchestra.run.smithi146.stdout: "wait": { 2024-09-12T03:04:42.583 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:42.583 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:42.583 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:42.583 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-12T03:04:42.583 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:42.583 INFO:teuthology.orchestra.run.smithi146.stdout: "throttle-objecter_ops": { 2024-09-12T03:04:42.584 INFO:teuthology.orchestra.run.smithi146.stdout: "val": 0, 2024-09-12T03:04:42.584 INFO:teuthology.orchestra.run.smithi146.stdout: "max": 1024, 2024-09-12T03:04:42.584 INFO:teuthology.orchestra.run.smithi146.stdout: "get_started": 0, 2024-09-12T03:04:42.584 INFO:teuthology.orchestra.run.smithi146.stdout: "get": 0, 2024-09-12T03:04:42.584 INFO:teuthology.orchestra.run.smithi146.stdout: "get_sum": 0, 2024-09-12T03:04:42.584 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_fail": 0, 2024-09-12T03:04:42.584 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_success": 0, 2024-09-12T03:04:42.584 INFO:teuthology.orchestra.run.smithi146.stdout: "take": 136, 2024-09-12T03:04:42.584 INFO:teuthology.orchestra.run.smithi146.stdout: "take_sum": 136, 2024-09-12T03:04:42.584 INFO:teuthology.orchestra.run.smithi146.stdout: "put": 136, 2024-09-12T03:04:42.584 INFO:teuthology.orchestra.run.smithi146.stdout: "put_sum": 136, 2024-09-12T03:04:42.584 INFO:teuthology.orchestra.run.smithi146.stdout: "wait": { 2024-09-12T03:04:42.584 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:42.584 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:42.585 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:42.585 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-12T03:04:42.585 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:42.585 INFO:teuthology.orchestra.run.smithi146.stdout: "throttle-write_buf_throttle": { 2024-09-12T03:04:42.585 INFO:teuthology.orchestra.run.smithi146.stdout: "val": 0, 2024-09-12T03:04:42.585 INFO:teuthology.orchestra.run.smithi146.stdout: "max": 3758096384, 2024-09-12T03:04:42.585 INFO:teuthology.orchestra.run.smithi146.stdout: "get_started": 0, 2024-09-12T03:04:42.585 INFO:teuthology.orchestra.run.smithi146.stdout: "get": 6, 2024-09-12T03:04:42.585 INFO:teuthology.orchestra.run.smithi146.stdout: "get_sum": 606, 2024-09-12T03:04:42.585 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_fail": 0, 2024-09-12T03:04:42.585 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_success": 6, 2024-09-12T03:04:42.585 INFO:teuthology.orchestra.run.smithi146.stdout: "take": 0, 2024-09-12T03:04:42.585 INFO:teuthology.orchestra.run.smithi146.stdout: "take_sum": 0, 2024-09-12T03:04:42.585 INFO:teuthology.orchestra.run.smithi146.stdout: "put": 3, 2024-09-12T03:04:42.586 INFO:teuthology.orchestra.run.smithi146.stdout: "put_sum": 606, 2024-09-12T03:04:42.586 INFO:teuthology.orchestra.run.smithi146.stdout: "wait": { 2024-09-12T03:04:42.586 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:42.586 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:42.586 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:42.586 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-12T03:04:42.586 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:42.586 INFO:teuthology.orchestra.run.smithi146.stdout: "throttle-write_buf_throttle-0x56199d22a960": { 2024-09-12T03:04:42.586 INFO:teuthology.orchestra.run.smithi146.stdout: "val": 0, 2024-09-12T03:04:42.586 INFO:teuthology.orchestra.run.smithi146.stdout: "max": 3758096384, 2024-09-12T03:04:42.586 INFO:teuthology.orchestra.run.smithi146.stdout: "get_started": 0, 2024-09-12T03:04:42.586 INFO:teuthology.orchestra.run.smithi146.stdout: "get": 47, 2024-09-12T03:04:42.586 INFO:teuthology.orchestra.run.smithi146.stdout: "get_sum": 72065, 2024-09-12T03:04:42.586 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_fail": 0, 2024-09-12T03:04:42.587 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_success": 47, 2024-09-12T03:04:42.587 INFO:teuthology.orchestra.run.smithi146.stdout: "take": 0, 2024-09-12T03:04:42.587 INFO:teuthology.orchestra.run.smithi146.stdout: "take_sum": 0, 2024-09-12T03:04:42.587 INFO:teuthology.orchestra.run.smithi146.stdout: "put": 28, 2024-09-12T03:04:42.587 INFO:teuthology.orchestra.run.smithi146.stdout: "put_sum": 72065, 2024-09-12T03:04:42.587 INFO:teuthology.orchestra.run.smithi146.stdout: "wait": { 2024-09-12T03:04:42.587 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:42.587 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:42.587 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:42.587 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-12T03:04:42.587 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-12T03:04:42.587 INFO:teuthology.orchestra.run.smithi146.stdout:} 2024-09-12T03:04:43.138 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:04:42 smithi146 ceph-mon[35011]: pgmap v529: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T03:04:43.165 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:04:42 smithi148 ceph-mon[35090]: pgmap v529: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T03:04:43.167 INFO:tasks.check_counter:Daemon mds.h mds.exported=2 2024-09-12T03:04:43.167 INFO:tasks.check_counter:Daemon mds.h mds.imported=2 2024-09-12T03:04:43.167 INFO:tasks.check_counter:Daemon mds.h mds.dir_split=0 2024-09-12T03:04:43.167 DEBUG:tasks.check_counter:Getting stats from k 2024-09-12T03:04:43.168 DEBUG:teuthology.orchestra.run.smithi146:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph daemon mds.k perf dump 2024-09-12T03:04:43.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:04:42 smithi005 ceph-mon[31197]: pgmap v529: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T03:04:43.415 INFO:teuthology.orchestra.run.smithi146.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.b/config 2024-09-12T03:04:43.960 INFO:teuthology.orchestra.run.smithi146.stdout:{ 2024-09-12T03:04:43.960 INFO:teuthology.orchestra.run.smithi146.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T03:04:43.960 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_messages": 170, 2024-09-12T03:04:43.961 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_messages": 157, 2024-09-12T03:04:43.961 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_bytes": 806482, 2024-09-12T03:04:43.961 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_bytes": 92436, 2024-09-12T03:04:43.961 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_created_connections": 5, 2024-09-12T03:04:43.961 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_active_connections": 1, 2024-09-12T03:04:43.961 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_total_time": 0.026597505, 2024-09-12T03:04:43.961 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_send_time": 0.009768619, 2024-09-12T03:04:43.961 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_recv_time": 0.024362469, 2024-09-12T03:04:43.961 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_fast_dispatch_time": 0.000016551, 2024-09-12T03:04:43.961 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:04:43.961 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 156, 2024-09-12T03:04:43.961 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.009471584, 2024-09-12T03:04:43.961 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000060715 2024-09-12T03:04:43.962 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:43.962 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:04:43.962 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:43.962 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:43.962 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:43.962 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:43.962 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_encrypted_bytes": 805648, 2024-09-12T03:04:43.962 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_encrypted_bytes": 92112 2024-09-12T03:04:43.962 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:43.962 INFO:teuthology.orchestra.run.smithi146.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T03:04:43.962 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_messages": 411, 2024-09-12T03:04:43.962 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_messages": 520, 2024-09-12T03:04:43.962 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_bytes": 103082, 2024-09-12T03:04:43.963 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_bytes": 397065, 2024-09-12T03:04:43.963 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_created_connections": 5, 2024-09-12T03:04:43.963 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_active_connections": 4, 2024-09-12T03:04:43.963 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_total_time": 0.042030475, 2024-09-12T03:04:43.963 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_send_time": 0.014600236, 2024-09-12T03:04:43.963 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_recv_time": 0.010864927, 2024-09-12T03:04:43.963 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_fast_dispatch_time": 0.012396793, 2024-09-12T03:04:43.963 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:04:43.963 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 519, 2024-09-12T03:04:43.963 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.023995026, 2024-09-12T03:04:43.963 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000046233 2024-09-12T03:04:43.963 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:43.963 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:04:43.963 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:43.964 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:43.964 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:43.964 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:43.964 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_encrypted_bytes": 3184, 2024-09-12T03:04:43.964 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_encrypted_bytes": 267952 2024-09-12T03:04:43.964 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:43.964 INFO:teuthology.orchestra.run.smithi146.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T03:04:43.964 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_messages": 1004, 2024-09-12T03:04:43.964 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_messages": 1006, 2024-09-12T03:04:43.964 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_bytes": 334566, 2024-09-12T03:04:43.964 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_bytes": 319330, 2024-09-12T03:04:43.964 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_created_connections": 4, 2024-09-12T03:04:43.965 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_active_connections": 3, 2024-09-12T03:04:43.965 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_total_time": 0.092121107, 2024-09-12T03:04:43.965 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_send_time": 0.033160971, 2024-09-12T03:04:43.965 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_recv_time": 0.023543084, 2024-09-12T03:04:43.965 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_running_fast_dispatch_time": 0.027136957, 2024-09-12T03:04:43.965 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:04:43.965 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 1004, 2024-09-12T03:04:43.965 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.037034671, 2024-09-12T03:04:43.965 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000036887 2024-09-12T03:04:43.965 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:43.965 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:04:43.965 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:43.965 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:43.965 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:43.966 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:43.966 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-12T03:04:43.966 INFO:teuthology.orchestra.run.smithi146.stdout: "msgr_send_encrypted_bytes": 0 2024-09-12T03:04:43.966 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:43.966 INFO:teuthology.orchestra.run.smithi146.stdout: "cct": { 2024-09-12T03:04:43.966 INFO:teuthology.orchestra.run.smithi146.stdout: "total_workers": 1, 2024-09-12T03:04:43.966 INFO:teuthology.orchestra.run.smithi146.stdout: "unhealthy_workers": 0 2024-09-12T03:04:43.966 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:43.966 INFO:teuthology.orchestra.run.smithi146.stdout: "finisher-MDSRank": { 2024-09-12T03:04:43.966 INFO:teuthology.orchestra.run.smithi146.stdout: "queue_len": 0, 2024-09-12T03:04:43.966 INFO:teuthology.orchestra.run.smithi146.stdout: "complete_latency": { 2024-09-12T03:04:43.966 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 2379, 2024-09-12T03:04:43.966 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.061614654, 2024-09-12T03:04:43.967 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000025899 2024-09-12T03:04:43.967 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-12T03:04:43.967 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:43.967 INFO:teuthology.orchestra.run.smithi146.stdout: "finisher-PurgeQueue": { 2024-09-12T03:04:43.967 INFO:teuthology.orchestra.run.smithi146.stdout: "queue_len": 0, 2024-09-12T03:04:43.967 INFO:teuthology.orchestra.run.smithi146.stdout: "complete_latency": { 2024-09-12T03:04:43.967 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:43.967 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:43.967 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:43.967 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-12T03:04:43.968 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:43.968 INFO:teuthology.orchestra.run.smithi146.stdout: "mds": { 2024-09-12T03:04:43.968 INFO:teuthology.orchestra.run.smithi146.stdout: "request": 0, 2024-09-12T03:04:43.968 INFO:teuthology.orchestra.run.smithi146.stdout: "reply": 0, 2024-09-12T03:04:43.968 INFO:teuthology.orchestra.run.smithi146.stdout: "reply_latency": { 2024-09-12T03:04:43.968 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:43.968 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:43.968 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:43.968 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:43.968 INFO:teuthology.orchestra.run.smithi146.stdout: "slow_reply": 0, 2024-09-12T03:04:43.968 INFO:teuthology.orchestra.run.smithi146.stdout: "forward": 0, 2024-09-12T03:04:43.968 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_fetch_complete": 0, 2024-09-12T03:04:43.968 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_fetch_keys": 0, 2024-09-12T03:04:43.969 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_commit": 0, 2024-09-12T03:04:43.969 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_split": 0, 2024-09-12T03:04:43.969 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_merge": 0, 2024-09-12T03:04:43.969 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes": 6, 2024-09-12T03:04:43.969 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes_top": 0, 2024-09-12T03:04:43.969 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes_bottom": 6, 2024-09-12T03:04:43.969 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes_pin_tail": 0, 2024-09-12T03:04:43.969 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes_pinned": 6, 2024-09-12T03:04:43.969 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes_expired": 0, 2024-09-12T03:04:43.969 INFO:teuthology.orchestra.run.smithi146.stdout: "inodes_with_caps": 0, 2024-09-12T03:04:43.969 INFO:teuthology.orchestra.run.smithi146.stdout: "caps": 0, 2024-09-12T03:04:43.969 INFO:teuthology.orchestra.run.smithi146.stdout: "subtrees": 1, 2024-09-12T03:04:43.969 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse": 0, 2024-09-12T03:04:43.970 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse_hit": 0, 2024-09-12T03:04:43.970 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse_forward": 0, 2024-09-12T03:04:43.970 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse_discover": 0, 2024-09-12T03:04:43.970 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse_dir_fetch": 0, 2024-09-12T03:04:43.970 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse_remote_ino": 0, 2024-09-12T03:04:43.970 INFO:teuthology.orchestra.run.smithi146.stdout: "traverse_lock": 0, 2024-09-12T03:04:43.970 INFO:teuthology.orchestra.run.smithi146.stdout: "load_cent": 0, 2024-09-12T03:04:43.970 INFO:teuthology.orchestra.run.smithi146.stdout: "q": 0, 2024-09-12T03:04:43.970 INFO:teuthology.orchestra.run.smithi146.stdout: "exported": 0, 2024-09-12T03:04:43.970 INFO:teuthology.orchestra.run.smithi146.stdout: "exported_inodes": 0, 2024-09-12T03:04:43.970 INFO:teuthology.orchestra.run.smithi146.stdout: "imported": 0, 2024-09-12T03:04:43.970 INFO:teuthology.orchestra.run.smithi146.stdout: "imported_inodes": 0, 2024-09-12T03:04:43.970 INFO:teuthology.orchestra.run.smithi146.stdout: "openino_dir_fetch": 0, 2024-09-12T03:04:43.970 INFO:teuthology.orchestra.run.smithi146.stdout: "openino_backtrace_fetch": 0, 2024-09-12T03:04:43.971 INFO:teuthology.orchestra.run.smithi146.stdout: "openino_peer_discover": 0, 2024-09-12T03:04:43.971 INFO:teuthology.orchestra.run.smithi146.stdout: "root_rfiles": 0, 2024-09-12T03:04:43.971 INFO:teuthology.orchestra.run.smithi146.stdout: "root_rbytes": 0, 2024-09-12T03:04:43.971 INFO:teuthology.orchestra.run.smithi146.stdout: "root_rsnaps": 0, 2024-09-12T03:04:43.971 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_backtrace_fetch": 0, 2024-09-12T03:04:43.971 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_set_tag": 0, 2024-09-12T03:04:43.971 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_backtrace_repaired": 0, 2024-09-12T03:04:43.971 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_inotable_repaired": 0, 2024-09-12T03:04:43.971 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_dir_inodes": 0, 2024-09-12T03:04:43.971 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_dir_base_inodes": 0, 2024-09-12T03:04:43.971 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_dirfrag_rstats": 0, 2024-09-12T03:04:43.971 INFO:teuthology.orchestra.run.smithi146.stdout: "scrub_file_inodes": 0, 2024-09-12T03:04:43.971 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_inode_file_caps": 0, 2024-09-12T03:04:43.971 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_cap_op_revoke": 0, 2024-09-12T03:04:43.972 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_cap_op_grant": 0, 2024-09-12T03:04:43.972 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_cap_op_trunc": 0, 2024-09-12T03:04:43.972 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-12T03:04:43.972 INFO:teuthology.orchestra.run.smithi146.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-12T03:04:43.972 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_client_caps": 0, 2024-09-12T03:04:43.972 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_client_caps_dirty": 0, 2024-09-12T03:04:43.972 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_client_cap_release": 0, 2024-09-12T03:04:43.972 INFO:teuthology.orchestra.run.smithi146.stdout: "process_request_cap_release": 0 2024-09-12T03:04:43.972 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:43.972 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_cache": { 2024-09-12T03:04:43.972 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_update": 0, 2024-09-12T03:04:43.972 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_update_receipt": 0, 2024-09-12T03:04:43.972 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_try_discover": 0, 2024-09-12T03:04:43.973 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_send_discover": 0, 2024-09-12T03:04:43.973 INFO:teuthology.orchestra.run.smithi146.stdout: "dir_handle_discover": 0, 2024-09-12T03:04:43.973 INFO:teuthology.orchestra.run.smithi146.stdout: "num_strays": 0, 2024-09-12T03:04:43.973 INFO:teuthology.orchestra.run.smithi146.stdout: "num_strays_delayed": 0, 2024-09-12T03:04:43.973 INFO:teuthology.orchestra.run.smithi146.stdout: "num_strays_enqueuing": 0, 2024-09-12T03:04:43.973 INFO:teuthology.orchestra.run.smithi146.stdout: "strays_created": 0, 2024-09-12T03:04:43.973 INFO:teuthology.orchestra.run.smithi146.stdout: "strays_enqueued": 0, 2024-09-12T03:04:43.973 INFO:teuthology.orchestra.run.smithi146.stdout: "strays_reintegrated": 0, 2024-09-12T03:04:43.973 INFO:teuthology.orchestra.run.smithi146.stdout: "strays_migrated": 0, 2024-09-12T03:04:43.973 INFO:teuthology.orchestra.run.smithi146.stdout: "num_recovering_processing": 0, 2024-09-12T03:04:43.973 INFO:teuthology.orchestra.run.smithi146.stdout: "num_recovering_enqueued": 0, 2024-09-12T03:04:43.973 INFO:teuthology.orchestra.run.smithi146.stdout: "num_recovering_prioritized": 0, 2024-09-12T03:04:43.973 INFO:teuthology.orchestra.run.smithi146.stdout: "recovery_started": 0, 2024-09-12T03:04:43.974 INFO:teuthology.orchestra.run.smithi146.stdout: "recovery_completed": 0, 2024-09-12T03:04:43.974 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_quiesce_path": 0, 2024-09-12T03:04:43.974 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_quiesce_inode": 0, 2024-09-12T03:04:43.974 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_enqueue_scrub": 0, 2024-09-12T03:04:43.974 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_exportdir": 0, 2024-09-12T03:04:43.974 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_flush": 0, 2024-09-12T03:04:43.974 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_fragmentdir": 0, 2024-09-12T03:04:43.974 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_fragstats": 0, 2024-09-12T03:04:43.974 INFO:teuthology.orchestra.run.smithi146.stdout: "ireq_inodestats": 0 2024-09-12T03:04:43.974 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:43.974 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_log": { 2024-09-12T03:04:43.974 INFO:teuthology.orchestra.run.smithi146.stdout: "evlrg": 0, 2024-09-12T03:04:43.974 INFO:teuthology.orchestra.run.smithi146.stdout: "evadd": 0, 2024-09-12T03:04:43.974 INFO:teuthology.orchestra.run.smithi146.stdout: "evex": 0, 2024-09-12T03:04:43.975 INFO:teuthology.orchestra.run.smithi146.stdout: "evtrm": 0, 2024-09-12T03:04:43.975 INFO:teuthology.orchestra.run.smithi146.stdout: "ev": 24, 2024-09-12T03:04:43.975 INFO:teuthology.orchestra.run.smithi146.stdout: "evexg": 0, 2024-09-12T03:04:43.975 INFO:teuthology.orchestra.run.smithi146.stdout: "evexd": 0, 2024-09-12T03:04:43.975 INFO:teuthology.orchestra.run.smithi146.stdout: "segadd": 0, 2024-09-12T03:04:43.975 INFO:teuthology.orchestra.run.smithi146.stdout: "segex": 0, 2024-09-12T03:04:43.975 INFO:teuthology.orchestra.run.smithi146.stdout: "segtrm": 0, 2024-09-12T03:04:43.975 INFO:teuthology.orchestra.run.smithi146.stdout: "seg": 2, 2024-09-12T03:04:43.975 INFO:teuthology.orchestra.run.smithi146.stdout: "segmjr": 2, 2024-09-12T03:04:43.975 INFO:teuthology.orchestra.run.smithi146.stdout: "segexg": 0, 2024-09-12T03:04:43.975 INFO:teuthology.orchestra.run.smithi146.stdout: "segexd": 0, 2024-09-12T03:04:43.975 INFO:teuthology.orchestra.run.smithi146.stdout: "expos": 4194304, 2024-09-12T03:04:43.975 INFO:teuthology.orchestra.run.smithi146.stdout: "wrpos": 4229211, 2024-09-12T03:04:43.976 INFO:teuthology.orchestra.run.smithi146.stdout: "rdpos": 4227560, 2024-09-12T03:04:43.976 INFO:teuthology.orchestra.run.smithi146.stdout: "jlat": { 2024-09-12T03:04:43.976 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:43.976 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:43.976 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:43.976 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:43.976 INFO:teuthology.orchestra.run.smithi146.stdout: "replayed": 24 2024-09-12T03:04:43.976 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:43.976 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_mem": { 2024-09-12T03:04:43.976 INFO:teuthology.orchestra.run.smithi146.stdout: "ino": 4, 2024-09-12T03:04:43.976 INFO:teuthology.orchestra.run.smithi146.stdout: "ino+": 19, 2024-09-12T03:04:43.976 INFO:teuthology.orchestra.run.smithi146.stdout: "ino-": 15, 2024-09-12T03:04:43.976 INFO:teuthology.orchestra.run.smithi146.stdout: "dir": 2, 2024-09-12T03:04:43.976 INFO:teuthology.orchestra.run.smithi146.stdout: "dir+": 13, 2024-09-12T03:04:43.977 INFO:teuthology.orchestra.run.smithi146.stdout: "dir-": 11, 2024-09-12T03:04:43.977 INFO:teuthology.orchestra.run.smithi146.stdout: "dn": 6, 2024-09-12T03:04:43.977 INFO:teuthology.orchestra.run.smithi146.stdout: "dn+": 25, 2024-09-12T03:04:43.977 INFO:teuthology.orchestra.run.smithi146.stdout: "dn-": 19, 2024-09-12T03:04:43.977 INFO:teuthology.orchestra.run.smithi146.stdout: "cap": 0, 2024-09-12T03:04:43.977 INFO:teuthology.orchestra.run.smithi146.stdout: "cap+": 0, 2024-09-12T03:04:43.977 INFO:teuthology.orchestra.run.smithi146.stdout: "cap-": 0, 2024-09-12T03:04:43.977 INFO:teuthology.orchestra.run.smithi146.stdout: "rss": 48480, 2024-09-12T03:04:43.977 INFO:teuthology.orchestra.run.smithi146.stdout: "heap": 215328 2024-09-12T03:04:43.977 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:43.977 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_server": { 2024-09-12T03:04:43.977 INFO:teuthology.orchestra.run.smithi146.stdout: "dispatch_client_request": 0, 2024-09-12T03:04:43.977 INFO:teuthology.orchestra.run.smithi146.stdout: "dispatch_server_request": 0, 2024-09-12T03:04:43.978 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_client_request": 0, 2024-09-12T03:04:43.978 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_client_session": 0, 2024-09-12T03:04:43.978 INFO:teuthology.orchestra.run.smithi146.stdout: "handle_peer_request": 0, 2024-09-12T03:04:43.978 INFO:teuthology.orchestra.run.smithi146.stdout: "req_create_latency": { 2024-09-12T03:04:43.978 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:43.978 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:43.978 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:43.978 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:43.978 INFO:teuthology.orchestra.run.smithi146.stdout: "req_getattr_latency": { 2024-09-12T03:04:43.978 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:43.978 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:43.978 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:43.978 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:43.979 INFO:teuthology.orchestra.run.smithi146.stdout: "req_getfilelock_latency": { 2024-09-12T03:04:43.979 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:43.979 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:43.979 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:43.979 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:43.979 INFO:teuthology.orchestra.run.smithi146.stdout: "req_link_latency": { 2024-09-12T03:04:43.979 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:43.979 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:43.979 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:43.979 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:43.979 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lookup_latency": { 2024-09-12T03:04:43.979 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:43.979 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:43.979 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:43.980 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:43.980 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lookuphash_latency": { 2024-09-12T03:04:43.980 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:43.980 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:43.980 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:43.980 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:43.980 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lookupino_latency": { 2024-09-12T03:04:43.980 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:43.980 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:43.980 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:43.980 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:43.980 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lookupname_latency": { 2024-09-12T03:04:43.980 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:43.980 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:43.981 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:43.981 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:43.981 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lookupparent_latency": { 2024-09-12T03:04:43.981 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:43.981 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:43.981 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:43.981 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:43.981 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lookupsnap_latency": { 2024-09-12T03:04:43.981 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:43.981 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:43.981 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:43.981 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:43.981 INFO:teuthology.orchestra.run.smithi146.stdout: "req_lssnap_latency": { 2024-09-12T03:04:43.981 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:43.982 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:43.982 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:43.982 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:43.982 INFO:teuthology.orchestra.run.smithi146.stdout: "req_mkdir_latency": { 2024-09-12T03:04:43.982 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:43.983 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:43.983 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:43.983 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:43.983 INFO:teuthology.orchestra.run.smithi146.stdout: "req_mknod_latency": { 2024-09-12T03:04:43.983 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:43.983 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:43.983 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:43.983 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:43.983 INFO:teuthology.orchestra.run.smithi146.stdout: "req_mksnap_latency": { 2024-09-12T03:04:43.983 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:43.983 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:43.984 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:43.984 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:43.984 INFO:teuthology.orchestra.run.smithi146.stdout: "req_open_latency": { 2024-09-12T03:04:43.984 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:43.984 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:43.984 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:43.984 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:43.984 INFO:teuthology.orchestra.run.smithi146.stdout: "req_readdir_latency": { 2024-09-12T03:04:43.984 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:43.984 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:43.984 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:43.984 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:43.984 INFO:teuthology.orchestra.run.smithi146.stdout: "req_rename_latency": { 2024-09-12T03:04:43.985 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:43.985 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:43.985 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:43.985 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:43.985 INFO:teuthology.orchestra.run.smithi146.stdout: "req_renamesnap_latency": { 2024-09-12T03:04:43.985 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:43.985 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:43.985 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:43.985 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:43.985 INFO:teuthology.orchestra.run.smithi146.stdout: "req_snapdiff_latency": { 2024-09-12T03:04:43.985 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:43.985 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:43.985 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:43.986 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:43.986 INFO:teuthology.orchestra.run.smithi146.stdout: "req_rmdir_latency": { 2024-09-12T03:04:43.986 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:43.986 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:43.986 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:43.986 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:43.986 INFO:teuthology.orchestra.run.smithi146.stdout: "req_rmsnap_latency": { 2024-09-12T03:04:43.986 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:43.986 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:43.986 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:43.986 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:43.986 INFO:teuthology.orchestra.run.smithi146.stdout: "req_rmxattr_latency": { 2024-09-12T03:04:43.986 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:43.986 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:43.987 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:43.987 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:43.987 INFO:teuthology.orchestra.run.smithi146.stdout: "req_setattr_latency": { 2024-09-12T03:04:43.987 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:43.987 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:43.987 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:43.987 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:43.987 INFO:teuthology.orchestra.run.smithi146.stdout: "req_setdirlayout_latency": { 2024-09-12T03:04:43.987 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:43.987 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:43.987 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:43.987 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:43.987 INFO:teuthology.orchestra.run.smithi146.stdout: "req_setfilelock_latency": { 2024-09-12T03:04:43.987 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:43.988 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:43.988 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:43.988 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:43.988 INFO:teuthology.orchestra.run.smithi146.stdout: "req_setlayout_latency": { 2024-09-12T03:04:43.988 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:43.988 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:43.988 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:43.988 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:43.988 INFO:teuthology.orchestra.run.smithi146.stdout: "req_setxattr_latency": { 2024-09-12T03:04:43.988 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:43.988 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:43.988 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:43.988 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:43.989 INFO:teuthology.orchestra.run.smithi146.stdout: "req_symlink_latency": { 2024-09-12T03:04:43.989 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:43.989 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:43.989 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:43.989 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:43.989 INFO:teuthology.orchestra.run.smithi146.stdout: "req_unlink_latency": { 2024-09-12T03:04:43.989 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:43.989 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:43.989 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:43.989 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:43.989 INFO:teuthology.orchestra.run.smithi146.stdout: "cap_revoke_eviction": 0, 2024-09-12T03:04:43.989 INFO:teuthology.orchestra.run.smithi146.stdout: "cap_acquisition_throttle": 0, 2024-09-12T03:04:43.989 INFO:teuthology.orchestra.run.smithi146.stdout: "req_getvxattr_latency": { 2024-09-12T03:04:43.989 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:43.990 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:43.990 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:43.990 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-12T03:04:43.990 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:43.990 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_sessions": { 2024-09-12T03:04:43.990 INFO:teuthology.orchestra.run.smithi146.stdout: "session_count": 1, 2024-09-12T03:04:43.990 INFO:teuthology.orchestra.run.smithi146.stdout: "session_add": 3, 2024-09-12T03:04:43.990 INFO:teuthology.orchestra.run.smithi146.stdout: "session_remove": 2, 2024-09-12T03:04:43.990 INFO:teuthology.orchestra.run.smithi146.stdout: "sessions_open": 1, 2024-09-12T03:04:43.990 INFO:teuthology.orchestra.run.smithi146.stdout: "sessions_stale": 0, 2024-09-12T03:04:43.990 INFO:teuthology.orchestra.run.smithi146.stdout: "total_load": 0, 2024-09-12T03:04:43.990 INFO:teuthology.orchestra.run.smithi146.stdout: "average_load": 0, 2024-09-12T03:04:43.990 INFO:teuthology.orchestra.run.smithi146.stdout: "avg_session_uptime": 1650, 2024-09-12T03:04:43.990 INFO:teuthology.orchestra.run.smithi146.stdout: "mdthresh_evicted": 0 2024-09-12T03:04:43.991 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:43.991 INFO:teuthology.orchestra.run.smithi146.stdout: "mempool": { 2024-09-12T03:04:43.991 INFO:teuthology.orchestra.run.smithi146.stdout: "bloom_filter_bytes": 0, 2024-09-12T03:04:43.991 INFO:teuthology.orchestra.run.smithi146.stdout: "bloom_filter_items": 0, 2024-09-12T03:04:43.991 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T03:04:43.991 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_alloc_items": 0, 2024-09-12T03:04:43.991 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T03:04:43.991 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_data_items": 0, 2024-09-12T03:04:43.991 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T03:04:43.991 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T03:04:43.991 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T03:04:43.991 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T03:04:43.991 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T03:04:43.992 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_other_items": 0, 2024-09-12T03:04:43.992 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T03:04:43.992 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T03:04:43.992 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_extent_bytes": 0, 2024-09-12T03:04:43.992 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_extent_items": 0, 2024-09-12T03:04:43.992 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_blob_bytes": 0, 2024-09-12T03:04:43.992 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_blob_items": 0, 2024-09-12T03:04:43.992 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T03:04:43.992 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T03:04:43.992 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T03:04:43.992 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T03:04:43.992 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T03:04:43.992 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_fsck_items": 0, 2024-09-12T03:04:43.993 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_txc_bytes": 0, 2024-09-12T03:04:43.993 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_txc_items": 0, 2024-09-12T03:04:43.993 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T03:04:43.993 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T03:04:43.993 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_writing_bytes": 0, 2024-09-12T03:04:43.993 INFO:teuthology.orchestra.run.smithi146.stdout: "bluestore_writing_items": 0, 2024-09-12T03:04:43.993 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_bytes": 0, 2024-09-12T03:04:43.993 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_items": 0, 2024-09-12T03:04:43.993 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T03:04:43.993 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_file_reader_items": 0, 2024-09-12T03:04:43.993 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T03:04:43.993 INFO:teuthology.orchestra.run.smithi146.stdout: "bluefs_file_writer_items": 0, 2024-09-12T03:04:43.993 INFO:teuthology.orchestra.run.smithi146.stdout: "buffer_anon_bytes": 189383, 2024-09-12T03:04:43.993 INFO:teuthology.orchestra.run.smithi146.stdout: "buffer_anon_items": 50, 2024-09-12T03:04:43.994 INFO:teuthology.orchestra.run.smithi146.stdout: "buffer_meta_bytes": 0, 2024-09-12T03:04:43.994 INFO:teuthology.orchestra.run.smithi146.stdout: "buffer_meta_items": 0, 2024-09-12T03:04:43.994 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_bytes": 0, 2024-09-12T03:04:43.994 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_items": 0, 2024-09-12T03:04:43.994 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_mapbl_bytes": 0, 2024-09-12T03:04:43.994 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_mapbl_items": 0, 2024-09-12T03:04:43.994 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_pglog_bytes": 0, 2024-09-12T03:04:43.994 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_pglog_items": 0, 2024-09-12T03:04:43.994 INFO:teuthology.orchestra.run.smithi146.stdout: "osdmap_bytes": 7496, 2024-09-12T03:04:43.994 INFO:teuthology.orchestra.run.smithi146.stdout: "osdmap_items": 154, 2024-09-12T03:04:43.994 INFO:teuthology.orchestra.run.smithi146.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T03:04:43.994 INFO:teuthology.orchestra.run.smithi146.stdout: "osdmap_mapping_items": 0, 2024-09-12T03:04:43.994 INFO:teuthology.orchestra.run.smithi146.stdout: "pgmap_bytes": 0, 2024-09-12T03:04:43.994 INFO:teuthology.orchestra.run.smithi146.stdout: "pgmap_items": 0, 2024-09-12T03:04:43.995 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_co_bytes": 16272, 2024-09-12T03:04:43.995 INFO:teuthology.orchestra.run.smithi146.stdout: "mds_co_items": 52, 2024-09-12T03:04:43.995 INFO:teuthology.orchestra.run.smithi146.stdout: "unittest_1_bytes": 0, 2024-09-12T03:04:43.995 INFO:teuthology.orchestra.run.smithi146.stdout: "unittest_1_items": 0, 2024-09-12T03:04:43.995 INFO:teuthology.orchestra.run.smithi146.stdout: "unittest_2_bytes": 0, 2024-09-12T03:04:43.995 INFO:teuthology.orchestra.run.smithi146.stdout: "unittest_2_items": 0 2024-09-12T03:04:43.995 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:43.995 INFO:teuthology.orchestra.run.smithi146.stdout: "objecter": { 2024-09-12T03:04:43.995 INFO:teuthology.orchestra.run.smithi146.stdout: "op_active": 0, 2024-09-12T03:04:43.995 INFO:teuthology.orchestra.run.smithi146.stdout: "op_laggy": 0, 2024-09-12T03:04:43.995 INFO:teuthology.orchestra.run.smithi146.stdout: "op_send": 1411, 2024-09-12T03:04:43.995 INFO:teuthology.orchestra.run.smithi146.stdout: "op_send_bytes": 21, 2024-09-12T03:04:43.995 INFO:teuthology.orchestra.run.smithi146.stdout: "op_resend": 3, 2024-09-12T03:04:43.995 INFO:teuthology.orchestra.run.smithi146.stdout: "op_reply": 1411, 2024-09-12T03:04:43.996 INFO:teuthology.orchestra.run.smithi146.stdout: "op_latency": { 2024-09-12T03:04:43.996 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 1411, 2024-09-12T03:04:43.996 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.637989918, 2024-09-12T03:04:43.996 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000452154 2024-09-12T03:04:43.996 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:43.996 INFO:teuthology.orchestra.run.smithi146.stdout: "op_inflight": 0, 2024-09-12T03:04:43.996 INFO:teuthology.orchestra.run.smithi146.stdout: "oplen_avg": { 2024-09-12T03:04:43.996 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 1411, 2024-09-12T03:04:43.996 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 1412 2024-09-12T03:04:43.996 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:43.996 INFO:teuthology.orchestra.run.smithi146.stdout: "op": 1411, 2024-09-12T03:04:43.996 INFO:teuthology.orchestra.run.smithi146.stdout: "op_r": 1411, 2024-09-12T03:04:43.996 INFO:teuthology.orchestra.run.smithi146.stdout: "op_w": 0, 2024-09-12T03:04:43.996 INFO:teuthology.orchestra.run.smithi146.stdout: "op_rmw": 0, 2024-09-12T03:04:43.997 INFO:teuthology.orchestra.run.smithi146.stdout: "op_pg": 0, 2024-09-12T03:04:43.997 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_stat": 900, 2024-09-12T03:04:43.997 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_create": 0, 2024-09-12T03:04:43.997 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_read": 509, 2024-09-12T03:04:43.997 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_write": 0, 2024-09-12T03:04:43.997 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_writefull": 0, 2024-09-12T03:04:43.997 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_writesame": 0, 2024-09-12T03:04:43.997 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_append": 0, 2024-09-12T03:04:43.997 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_zero": 0, 2024-09-12T03:04:43.997 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_truncate": 0, 2024-09-12T03:04:43.997 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_delete": 0, 2024-09-12T03:04:43.997 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_mapext": 0, 2024-09-12T03:04:43.997 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_sparse_read": 0, 2024-09-12T03:04:43.997 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_clonerange": 0, 2024-09-12T03:04:43.998 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_getxattr": 1, 2024-09-12T03:04:43.998 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_setxattr": 0, 2024-09-12T03:04:43.998 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_cmpxattr": 0, 2024-09-12T03:04:43.998 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_rmxattr": 0, 2024-09-12T03:04:43.998 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_resetxattrs": 0, 2024-09-12T03:04:43.998 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_call": 0, 2024-09-12T03:04:43.998 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_watch": 0, 2024-09-12T03:04:43.998 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_notify": 0, 2024-09-12T03:04:43.998 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_src_cmpxattr": 0, 2024-09-12T03:04:43.998 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_pgls": 0, 2024-09-12T03:04:43.998 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_pgls_filter": 0, 2024-09-12T03:04:43.998 INFO:teuthology.orchestra.run.smithi146.stdout: "osdop_other": 0, 2024-09-12T03:04:43.998 INFO:teuthology.orchestra.run.smithi146.stdout: "linger_active": 0, 2024-09-12T03:04:43.998 INFO:teuthology.orchestra.run.smithi146.stdout: "linger_send": 0, 2024-09-12T03:04:43.999 INFO:teuthology.orchestra.run.smithi146.stdout: "linger_resend": 0, 2024-09-12T03:04:43.999 INFO:teuthology.orchestra.run.smithi146.stdout: "linger_ping": 0, 2024-09-12T03:04:43.999 INFO:teuthology.orchestra.run.smithi146.stdout: "poolop_active": 0, 2024-09-12T03:04:43.999 INFO:teuthology.orchestra.run.smithi146.stdout: "poolop_send": 0, 2024-09-12T03:04:43.999 INFO:teuthology.orchestra.run.smithi146.stdout: "poolop_resend": 0, 2024-09-12T03:04:43.999 INFO:teuthology.orchestra.run.smithi146.stdout: "poolstat_active": 0, 2024-09-12T03:04:43.999 INFO:teuthology.orchestra.run.smithi146.stdout: "poolstat_send": 0, 2024-09-12T03:04:43.999 INFO:teuthology.orchestra.run.smithi146.stdout: "poolstat_resend": 0, 2024-09-12T03:04:43.999 INFO:teuthology.orchestra.run.smithi146.stdout: "statfs_active": 0, 2024-09-12T03:04:43.999 INFO:teuthology.orchestra.run.smithi146.stdout: "statfs_send": 0, 2024-09-12T03:04:43.999 INFO:teuthology.orchestra.run.smithi146.stdout: "statfs_resend": 0, 2024-09-12T03:04:43.999 INFO:teuthology.orchestra.run.smithi146.stdout: "command_active": 0, 2024-09-12T03:04:43.999 INFO:teuthology.orchestra.run.smithi146.stdout: "command_send": 0, 2024-09-12T03:04:43.999 INFO:teuthology.orchestra.run.smithi146.stdout: "command_resend": 0, 2024-09-12T03:04:43.999 INFO:teuthology.orchestra.run.smithi146.stdout: "map_epoch": 83, 2024-09-12T03:04:44.000 INFO:teuthology.orchestra.run.smithi146.stdout: "map_full": 0, 2024-09-12T03:04:44.000 INFO:teuthology.orchestra.run.smithi146.stdout: "map_inc": 2, 2024-09-12T03:04:44.000 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_sessions": 5, 2024-09-12T03:04:44.000 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_session_open": 5, 2024-09-12T03:04:44.000 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_session_close": 0, 2024-09-12T03:04:44.000 INFO:teuthology.orchestra.run.smithi146.stdout: "osd_laggy": 0, 2024-09-12T03:04:44.000 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_wr": 0, 2024-09-12T03:04:44.000 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_rd": 2, 2024-09-12T03:04:44.000 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_del": 0 2024-09-12T03:04:44.000 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:44.000 INFO:teuthology.orchestra.run.smithi146.stdout: "oft": { 2024-09-12T03:04:44.000 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_total_objs": 0, 2024-09-12T03:04:44.000 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_total_kv_pairs": 0, 2024-09-12T03:04:44.001 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_total_updates": 0, 2024-09-12T03:04:44.001 INFO:teuthology.orchestra.run.smithi146.stdout: "omap_total_removes": 0 2024-09-12T03:04:44.001 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:44.001 INFO:teuthology.orchestra.run.smithi146.stdout: "osd-slow-ops": { 2024-09-12T03:04:44.001 INFO:teuthology.orchestra.run.smithi146.stdout: "slow_ops_count": 0 2024-09-12T03:04:44.001 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:44.001 INFO:teuthology.orchestra.run.smithi146.stdout: "purge_queue": { 2024-09-12T03:04:44.001 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_executing_ops": 0, 2024-09-12T03:04:44.001 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_executing_ops_high_water": 0, 2024-09-12T03:04:44.001 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_executing": 0, 2024-09-12T03:04:44.001 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_executing_high_water": 0, 2024-09-12T03:04:44.001 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_executed_ops": 0, 2024-09-12T03:04:44.001 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_executed": 0, 2024-09-12T03:04:44.001 INFO:teuthology.orchestra.run.smithi146.stdout: "pq_item_in_journal": 0 2024-09-12T03:04:44.002 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:44.002 INFO:teuthology.orchestra.run.smithi146.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T03:04:44.002 INFO:teuthology.orchestra.run.smithi146.stdout: "val": 0, 2024-09-12T03:04:44.002 INFO:teuthology.orchestra.run.smithi146.stdout: "max": 104857600, 2024-09-12T03:04:44.002 INFO:teuthology.orchestra.run.smithi146.stdout: "get_started": 0, 2024-09-12T03:04:44.002 INFO:teuthology.orchestra.run.smithi146.stdout: "get": 1581, 2024-09-12T03:04:44.002 INFO:teuthology.orchestra.run.smithi146.stdout: "get_sum": 1090212, 2024-09-12T03:04:44.002 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_fail": 0, 2024-09-12T03:04:44.002 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_success": 1581, 2024-09-12T03:04:44.002 INFO:teuthology.orchestra.run.smithi146.stdout: "take": 0, 2024-09-12T03:04:44.002 INFO:teuthology.orchestra.run.smithi146.stdout: "take_sum": 0, 2024-09-12T03:04:44.002 INFO:teuthology.orchestra.run.smithi146.stdout: "put": 1581, 2024-09-12T03:04:44.002 INFO:teuthology.orchestra.run.smithi146.stdout: "put_sum": 1090212, 2024-09-12T03:04:44.002 INFO:teuthology.orchestra.run.smithi146.stdout: "wait": { 2024-09-12T03:04:44.003 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:44.003 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:44.003 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:44.003 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-12T03:04:44.003 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:44.003 INFO:teuthology.orchestra.run.smithi146.stdout: "throttle-objecter_bytes": { 2024-09-12T03:04:44.003 INFO:teuthology.orchestra.run.smithi146.stdout: "val": 0, 2024-09-12T03:04:44.003 INFO:teuthology.orchestra.run.smithi146.stdout: "max": 104857600, 2024-09-12T03:04:44.003 INFO:teuthology.orchestra.run.smithi146.stdout: "get_started": 0, 2024-09-12T03:04:44.003 INFO:teuthology.orchestra.run.smithi146.stdout: "get": 0, 2024-09-12T03:04:44.003 INFO:teuthology.orchestra.run.smithi146.stdout: "get_sum": 0, 2024-09-12T03:04:44.003 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_fail": 0, 2024-09-12T03:04:44.003 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_success": 0, 2024-09-12T03:04:44.003 INFO:teuthology.orchestra.run.smithi146.stdout: "take": 1411, 2024-09-12T03:04:44.004 INFO:teuthology.orchestra.run.smithi146.stdout: "take_sum": 34912, 2024-09-12T03:04:44.004 INFO:teuthology.orchestra.run.smithi146.stdout: "put": 1411, 2024-09-12T03:04:44.004 INFO:teuthology.orchestra.run.smithi146.stdout: "put_sum": 34912, 2024-09-12T03:04:44.004 INFO:teuthology.orchestra.run.smithi146.stdout: "wait": { 2024-09-12T03:04:44.004 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:44.004 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:44.005 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:44.005 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-12T03:04:44.005 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:44.005 INFO:teuthology.orchestra.run.smithi146.stdout: "throttle-objecter_ops": { 2024-09-12T03:04:44.005 INFO:teuthology.orchestra.run.smithi146.stdout: "val": 0, 2024-09-12T03:04:44.005 INFO:teuthology.orchestra.run.smithi146.stdout: "max": 1024, 2024-09-12T03:04:44.005 INFO:teuthology.orchestra.run.smithi146.stdout: "get_started": 0, 2024-09-12T03:04:44.005 INFO:teuthology.orchestra.run.smithi146.stdout: "get": 0, 2024-09-12T03:04:44.005 INFO:teuthology.orchestra.run.smithi146.stdout: "get_sum": 0, 2024-09-12T03:04:44.005 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_fail": 0, 2024-09-12T03:04:44.005 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_success": 0, 2024-09-12T03:04:44.005 INFO:teuthology.orchestra.run.smithi146.stdout: "take": 1411, 2024-09-12T03:04:44.005 INFO:teuthology.orchestra.run.smithi146.stdout: "take_sum": 1411, 2024-09-12T03:04:44.005 INFO:teuthology.orchestra.run.smithi146.stdout: "put": 1411, 2024-09-12T03:04:44.005 INFO:teuthology.orchestra.run.smithi146.stdout: "put_sum": 1411, 2024-09-12T03:04:44.006 INFO:teuthology.orchestra.run.smithi146.stdout: "wait": { 2024-09-12T03:04:44.006 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:44.006 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:44.006 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:44.006 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-12T03:04:44.006 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:44.006 INFO:teuthology.orchestra.run.smithi146.stdout: "throttle-write_buf_throttle": { 2024-09-12T03:04:44.006 INFO:teuthology.orchestra.run.smithi146.stdout: "val": 0, 2024-09-12T03:04:44.006 INFO:teuthology.orchestra.run.smithi146.stdout: "max": 3758096384, 2024-09-12T03:04:44.006 INFO:teuthology.orchestra.run.smithi146.stdout: "get_started": 0, 2024-09-12T03:04:44.006 INFO:teuthology.orchestra.run.smithi146.stdout: "get": 0, 2024-09-12T03:04:44.006 INFO:teuthology.orchestra.run.smithi146.stdout: "get_sum": 0, 2024-09-12T03:04:44.007 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_fail": 0, 2024-09-12T03:04:44.007 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_success": 0, 2024-09-12T03:04:44.007 INFO:teuthology.orchestra.run.smithi146.stdout: "take": 0, 2024-09-12T03:04:44.007 INFO:teuthology.orchestra.run.smithi146.stdout: "take_sum": 0, 2024-09-12T03:04:44.007 INFO:teuthology.orchestra.run.smithi146.stdout: "put": 0, 2024-09-12T03:04:44.007 INFO:teuthology.orchestra.run.smithi146.stdout: "put_sum": 0, 2024-09-12T03:04:44.007 INFO:teuthology.orchestra.run.smithi146.stdout: "wait": { 2024-09-12T03:04:44.007 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:44.007 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:44.007 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:44.007 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-12T03:04:44.007 INFO:teuthology.orchestra.run.smithi146.stdout: }, 2024-09-12T03:04:44.008 INFO:teuthology.orchestra.run.smithi146.stdout: "throttle-write_buf_throttle-0x556d67c1cb40": { 2024-09-12T03:04:44.008 INFO:teuthology.orchestra.run.smithi146.stdout: "val": 0, 2024-09-12T03:04:44.008 INFO:teuthology.orchestra.run.smithi146.stdout: "max": 3758096384, 2024-09-12T03:04:44.008 INFO:teuthology.orchestra.run.smithi146.stdout: "get_started": 0, 2024-09-12T03:04:44.008 INFO:teuthology.orchestra.run.smithi146.stdout: "get": 0, 2024-09-12T03:04:44.008 INFO:teuthology.orchestra.run.smithi146.stdout: "get_sum": 0, 2024-09-12T03:04:44.008 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_fail": 0, 2024-09-12T03:04:44.008 INFO:teuthology.orchestra.run.smithi146.stdout: "get_or_fail_success": 0, 2024-09-12T03:04:44.008 INFO:teuthology.orchestra.run.smithi146.stdout: "take": 0, 2024-09-12T03:04:44.008 INFO:teuthology.orchestra.run.smithi146.stdout: "take_sum": 0, 2024-09-12T03:04:44.008 INFO:teuthology.orchestra.run.smithi146.stdout: "put": 0, 2024-09-12T03:04:44.008 INFO:teuthology.orchestra.run.smithi146.stdout: "put_sum": 0, 2024-09-12T03:04:44.008 INFO:teuthology.orchestra.run.smithi146.stdout: "wait": { 2024-09-12T03:04:44.008 INFO:teuthology.orchestra.run.smithi146.stdout: "avgcount": 0, 2024-09-12T03:04:44.009 INFO:teuthology.orchestra.run.smithi146.stdout: "sum": 0.000000000, 2024-09-12T03:04:44.009 INFO:teuthology.orchestra.run.smithi146.stdout: "avgtime": 0.000000000 2024-09-12T03:04:44.009 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-12T03:04:44.009 INFO:teuthology.orchestra.run.smithi146.stdout: } 2024-09-12T03:04:44.009 INFO:teuthology.orchestra.run.smithi146.stdout:} 2024-09-12T03:04:44.698 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-09-12T03:04:44.698 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-09-12T03:04:44.698 INFO:tasks.check_counter:Daemon mds.k mds.dir_split=0 2024-09-12T03:04:44.698 DEBUG:tasks.check_counter:Getting stats from c 2024-09-12T03:04:44.698 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph daemon mds.c perf dump 2024-09-12T03:04:44.959 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T03:04:45.081 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:04:44 smithi148 ceph-mon[35090]: pgmap v530: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-12T03:04:45.206 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:04:44 smithi146 ceph-mon[35011]: pgmap v530: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-12T03:04:45.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:04:44 smithi005 ceph-mon[31197]: pgmap v530: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-12T03:04:45.537 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-12T03:04:45.538 INFO:teuthology.orchestra.run.smithi148.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T03:04:45.538 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_messages": 101, 2024-09-12T03:04:45.538 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages": 115, 2024-09-12T03:04:45.538 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_bytes": 25439, 2024-09-12T03:04:45.538 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_bytes": 335400, 2024-09-12T03:04:45.538 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_created_connections": 32, 2024-09-12T03:04:45.538 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_active_connections": 5, 2024-09-12T03:04:45.538 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_total_time": 0.020058501, 2024-09-12T03:04:45.538 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_send_time": 0.003634276, 2024-09-12T03:04:45.538 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_recv_time": 0.012864559, 2024-09-12T03:04:45.538 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_fast_dispatch_time": 0.000950642, 2024-09-12T03:04:45.538 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:04:45.539 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 111, 2024-09-12T03:04:45.539 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.017470473, 2024-09-12T03:04:45.539 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000157391 2024-09-12T03:04:45.539 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:45.539 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:04:45.539 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 95, 2024-09-12T03:04:45.539 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000012659, 2024-09-12T03:04:45.539 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000133 2024-09-12T03:04:45.539 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:45.539 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_encrypted_bytes": 11840, 2024-09-12T03:04:45.539 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_encrypted_bytes": 17904 2024-09-12T03:04:45.539 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:45.539 INFO:teuthology.orchestra.run.smithi148.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T03:04:45.540 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_messages": 1045, 2024-09-12T03:04:45.540 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages": 836, 2024-09-12T03:04:45.540 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_bytes": 332807, 2024-09-12T03:04:45.540 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_bytes": 520856, 2024-09-12T03:04:45.540 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_created_connections": 7, 2024-09-12T03:04:45.540 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_active_connections": 4, 2024-09-12T03:04:45.540 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_total_time": 0.081302486, 2024-09-12T03:04:45.540 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_send_time": 0.037605828, 2024-09-12T03:04:45.540 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_recv_time": 0.047337515, 2024-09-12T03:04:45.540 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_fast_dispatch_time": 0.002867386, 2024-09-12T03:04:45.540 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:04:45.540 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 837, 2024-09-12T03:04:45.540 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.042961091, 2024-09-12T03:04:45.541 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000051327 2024-09-12T03:04:45.541 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:45.541 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:04:45.541 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 1674, 2024-09-12T03:04:45.541 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000308182, 2024-09-12T03:04:45.541 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000184 2024-09-12T03:04:45.541 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:45.541 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_encrypted_bytes": 3184, 2024-09-12T03:04:45.541 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_encrypted_bytes": 269536 2024-09-12T03:04:45.541 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:45.541 INFO:teuthology.orchestra.run.smithi148.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T03:04:45.541 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_messages": 242, 2024-09-12T03:04:45.541 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages": 223, 2024-09-12T03:04:45.542 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_bytes": 837837, 2024-09-12T03:04:45.542 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_bytes": 117478, 2024-09-12T03:04:45.542 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_created_connections": 6, 2024-09-12T03:04:45.542 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_active_connections": 5, 2024-09-12T03:04:45.542 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_total_time": 0.029384563, 2024-09-12T03:04:45.542 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_send_time": 0.010719364, 2024-09-12T03:04:45.542 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_recv_time": 0.025803100, 2024-09-12T03:04:45.542 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_fast_dispatch_time": 0.000869566, 2024-09-12T03:04:45.542 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:04:45.542 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 222, 2024-09-12T03:04:45.542 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.020229173, 2024-09-12T03:04:45.542 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000091122 2024-09-12T03:04:45.542 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:45.543 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:04:45.543 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:45.543 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:45.543 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:45.543 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:45.543 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_encrypted_bytes": 830064, 2024-09-12T03:04:45.543 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_encrypted_bytes": 99824 2024-09-12T03:04:45.543 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:45.543 INFO:teuthology.orchestra.run.smithi148.stdout: "cct": { 2024-09-12T03:04:45.543 INFO:teuthology.orchestra.run.smithi148.stdout: "total_workers": 1, 2024-09-12T03:04:45.543 INFO:teuthology.orchestra.run.smithi148.stdout: "unhealthy_workers": 0 2024-09-12T03:04:45.543 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:45.543 INFO:teuthology.orchestra.run.smithi148.stdout: "finisher-MDSRank": { 2024-09-12T03:04:45.544 INFO:teuthology.orchestra.run.smithi148.stdout: "queue_len": 0, 2024-09-12T03:04:45.544 INFO:teuthology.orchestra.run.smithi148.stdout: "complete_latency": { 2024-09-12T03:04:45.544 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 94, 2024-09-12T03:04:45.544 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.006375221, 2024-09-12T03:04:45.544 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000067821 2024-09-12T03:04:45.544 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-12T03:04:45.544 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:45.544 INFO:teuthology.orchestra.run.smithi148.stdout: "finisher-PurgeQueue": { 2024-09-12T03:04:45.544 INFO:teuthology.orchestra.run.smithi148.stdout: "queue_len": 0, 2024-09-12T03:04:45.544 INFO:teuthology.orchestra.run.smithi148.stdout: "complete_latency": { 2024-09-12T03:04:45.544 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 16, 2024-09-12T03:04:45.544 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000303632, 2024-09-12T03:04:45.545 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000018977 2024-09-12T03:04:45.545 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-12T03:04:45.545 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:45.545 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": { 2024-09-12T03:04:45.545 INFO:teuthology.orchestra.run.smithi148.stdout: "request": 17, 2024-09-12T03:04:45.545 INFO:teuthology.orchestra.run.smithi148.stdout: "reply": 8, 2024-09-12T03:04:45.545 INFO:teuthology.orchestra.run.smithi148.stdout: "reply_latency": { 2024-09-12T03:04:45.545 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 8, 2024-09-12T03:04:45.545 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.008843348, 2024-09-12T03:04:45.545 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.001105418 2024-09-12T03:04:45.545 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:45.545 INFO:teuthology.orchestra.run.smithi148.stdout: "slow_reply": 0, 2024-09-12T03:04:45.545 INFO:teuthology.orchestra.run.smithi148.stdout: "forward": 9, 2024-09-12T03:04:45.545 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_fetch_complete": 0, 2024-09-12T03:04:45.546 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_fetch_keys": 0, 2024-09-12T03:04:45.546 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_commit": 11, 2024-09-12T03:04:45.546 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_split": 0, 2024-09-12T03:04:45.546 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_merge": 0, 2024-09-12T03:04:45.546 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes": 15, 2024-09-12T03:04:45.546 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_top": 0, 2024-09-12T03:04:45.546 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_bottom": 15, 2024-09-12T03:04:45.546 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_pin_tail": 0, 2024-09-12T03:04:45.546 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_pinned": 13, 2024-09-12T03:04:45.546 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_expired": 12, 2024-09-12T03:04:45.546 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_with_caps": 0, 2024-09-12T03:04:45.546 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": 0, 2024-09-12T03:04:45.546 INFO:teuthology.orchestra.run.smithi148.stdout: "subtrees": 3, 2024-09-12T03:04:45.547 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse": 37, 2024-09-12T03:04:45.547 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_hit": 21, 2024-09-12T03:04:45.547 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_forward": 8, 2024-09-12T03:04:45.547 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_discover": 3, 2024-09-12T03:04:45.547 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_dir_fetch": 0, 2024-09-12T03:04:45.547 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_remote_ino": 0, 2024-09-12T03:04:45.547 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_lock": 0, 2024-09-12T03:04:45.547 INFO:teuthology.orchestra.run.smithi148.stdout: "load_cent": 0, 2024-09-12T03:04:45.547 INFO:teuthology.orchestra.run.smithi148.stdout: "q": 0, 2024-09-12T03:04:45.547 INFO:teuthology.orchestra.run.smithi148.stdout: "exported": 1, 2024-09-12T03:04:45.547 INFO:teuthology.orchestra.run.smithi148.stdout: "exported_inodes": 0, 2024-09-12T03:04:45.547 INFO:teuthology.orchestra.run.smithi148.stdout: "imported": 1, 2024-09-12T03:04:45.547 INFO:teuthology.orchestra.run.smithi148.stdout: "imported_inodes": 3, 2024-09-12T03:04:45.548 INFO:teuthology.orchestra.run.smithi148.stdout: "openino_dir_fetch": 0, 2024-09-12T03:04:45.548 INFO:teuthology.orchestra.run.smithi148.stdout: "openino_backtrace_fetch": 12, 2024-09-12T03:04:45.548 INFO:teuthology.orchestra.run.smithi148.stdout: "openino_peer_discover": 0, 2024-09-12T03:04:45.548 INFO:teuthology.orchestra.run.smithi148.stdout: "root_rfiles": 4570, 2024-09-12T03:04:45.548 INFO:teuthology.orchestra.run.smithi148.stdout: "root_rbytes": 147481294, 2024-09-12T03:04:45.548 INFO:teuthology.orchestra.run.smithi148.stdout: "root_rsnaps": 0, 2024-09-12T03:04:45.548 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_backtrace_fetch": 12, 2024-09-12T03:04:45.548 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_set_tag": 0, 2024-09-12T03:04:45.548 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_backtrace_repaired": 0, 2024-09-12T03:04:45.548 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_inotable_repaired": 0, 2024-09-12T03:04:45.548 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_dir_inodes": 6, 2024-09-12T03:04:45.548 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_dir_base_inodes": 0, 2024-09-12T03:04:45.548 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_dirfrag_rstats": 6, 2024-09-12T03:04:45.548 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_file_inodes": 6, 2024-09-12T03:04:45.549 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_inode_file_caps": 0, 2024-09-12T03:04:45.549 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_revoke": 9, 2024-09-12T03:04:45.549 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_grant": 7, 2024-09-12T03:04:45.549 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_trunc": 0, 2024-09-12T03:04:45.549 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-12T03:04:45.549 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-12T03:04:45.549 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_caps": 10, 2024-09-12T03:04:45.549 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_caps_dirty": 1, 2024-09-12T03:04:45.549 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_cap_release": 2, 2024-09-12T03:04:45.550 INFO:teuthology.orchestra.run.smithi148.stdout: "process_request_cap_release": 5 2024-09-12T03:04:45.550 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:45.550 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_cache": { 2024-09-12T03:04:45.550 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_update": 0, 2024-09-12T03:04:45.550 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_update_receipt": 0, 2024-09-12T03:04:45.550 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_try_discover": 0, 2024-09-12T03:04:45.550 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_send_discover": 4, 2024-09-12T03:04:45.550 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_handle_discover": 0, 2024-09-12T03:04:45.550 INFO:teuthology.orchestra.run.smithi148.stdout: "num_strays": 0, 2024-09-12T03:04:45.550 INFO:teuthology.orchestra.run.smithi148.stdout: "num_strays_delayed": 0, 2024-09-12T03:04:45.550 INFO:teuthology.orchestra.run.smithi148.stdout: "num_strays_enqueuing": 0, 2024-09-12T03:04:45.550 INFO:teuthology.orchestra.run.smithi148.stdout: "strays_created": 2, 2024-09-12T03:04:45.550 INFO:teuthology.orchestra.run.smithi148.stdout: "strays_enqueued": 2, 2024-09-12T03:04:45.550 INFO:teuthology.orchestra.run.smithi148.stdout: "strays_reintegrated": 0, 2024-09-12T03:04:45.551 INFO:teuthology.orchestra.run.smithi148.stdout: "strays_migrated": 0, 2024-09-12T03:04:45.551 INFO:teuthology.orchestra.run.smithi148.stdout: "num_recovering_processing": 0, 2024-09-12T03:04:45.551 INFO:teuthology.orchestra.run.smithi148.stdout: "num_recovering_enqueued": 0, 2024-09-12T03:04:45.551 INFO:teuthology.orchestra.run.smithi148.stdout: "num_recovering_prioritized": 0, 2024-09-12T03:04:45.551 INFO:teuthology.orchestra.run.smithi148.stdout: "recovery_started": 0, 2024-09-12T03:04:45.551 INFO:teuthology.orchestra.run.smithi148.stdout: "recovery_completed": 0, 2024-09-12T03:04:45.551 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_quiesce_path": 5, 2024-09-12T03:04:45.551 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_quiesce_inode": 55, 2024-09-12T03:04:45.551 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_enqueue_scrub": 0, 2024-09-12T03:04:45.551 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_exportdir": 1, 2024-09-12T03:04:45.551 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_flush": 0, 2024-09-12T03:04:45.552 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_fragmentdir": 0, 2024-09-12T03:04:45.552 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_fragstats": 0, 2024-09-12T03:04:45.552 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_inodestats": 0 2024-09-12T03:04:45.552 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:45.552 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_log": { 2024-09-12T03:04:45.552 INFO:teuthology.orchestra.run.smithi148.stdout: "evlrg": 0, 2024-09-12T03:04:45.552 INFO:teuthology.orchestra.run.smithi148.stdout: "evadd": 20, 2024-09-12T03:04:45.552 INFO:teuthology.orchestra.run.smithi148.stdout: "evex": 0, 2024-09-12T03:04:45.552 INFO:teuthology.orchestra.run.smithi148.stdout: "evtrm": 0, 2024-09-12T03:04:45.552 INFO:teuthology.orchestra.run.smithi148.stdout: "ev": 20, 2024-09-12T03:04:45.552 INFO:teuthology.orchestra.run.smithi148.stdout: "evexg": 0, 2024-09-12T03:04:45.552 INFO:teuthology.orchestra.run.smithi148.stdout: "evexd": 0, 2024-09-12T03:04:45.552 INFO:teuthology.orchestra.run.smithi148.stdout: "segadd": 2, 2024-09-12T03:04:45.553 INFO:teuthology.orchestra.run.smithi148.stdout: "segex": 0, 2024-09-12T03:04:45.553 INFO:teuthology.orchestra.run.smithi148.stdout: "segtrm": 0, 2024-09-12T03:04:45.553 INFO:teuthology.orchestra.run.smithi148.stdout: "seg": 2, 2024-09-12T03:04:45.553 INFO:teuthology.orchestra.run.smithi148.stdout: "segmjr": 2, 2024-09-12T03:04:45.553 INFO:teuthology.orchestra.run.smithi148.stdout: "segexg": 0, 2024-09-12T03:04:45.553 INFO:teuthology.orchestra.run.smithi148.stdout: "segexd": 0, 2024-09-12T03:04:45.553 INFO:teuthology.orchestra.run.smithi148.stdout: "expos": 4194304, 2024-09-12T03:04:45.553 INFO:teuthology.orchestra.run.smithi148.stdout: "wrpos": 4226471, 2024-09-12T03:04:45.553 INFO:teuthology.orchestra.run.smithi148.stdout: "rdpos": 0, 2024-09-12T03:04:45.553 INFO:teuthology.orchestra.run.smithi148.stdout: "jlat": { 2024-09-12T03:04:45.553 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 17, 2024-09-12T03:04:45.553 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.156626064, 2024-09-12T03:04:45.553 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.009213297 2024-09-12T03:04:45.553 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:45.554 INFO:teuthology.orchestra.run.smithi148.stdout: "replayed": 0 2024-09-12T03:04:45.554 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:45.554 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_mem": { 2024-09-12T03:04:45.554 INFO:teuthology.orchestra.run.smithi148.stdout: "ino": 19, 2024-09-12T03:04:45.554 INFO:teuthology.orchestra.run.smithi148.stdout: "ino+": 27, 2024-09-12T03:04:45.554 INFO:teuthology.orchestra.run.smithi148.stdout: "ino-": 8, 2024-09-12T03:04:45.554 INFO:teuthology.orchestra.run.smithi148.stdout: "dir": 18, 2024-09-12T03:04:45.554 INFO:teuthology.orchestra.run.smithi148.stdout: "dir+": 26, 2024-09-12T03:04:45.554 INFO:teuthology.orchestra.run.smithi148.stdout: "dir-": 8, 2024-09-12T03:04:45.554 INFO:teuthology.orchestra.run.smithi148.stdout: "dn": 15, 2024-09-12T03:04:45.554 INFO:teuthology.orchestra.run.smithi148.stdout: "dn+": 33, 2024-09-12T03:04:45.554 INFO:teuthology.orchestra.run.smithi148.stdout: "dn-": 18, 2024-09-12T03:04:45.554 INFO:teuthology.orchestra.run.smithi148.stdout: "cap": 0, 2024-09-12T03:04:45.554 INFO:teuthology.orchestra.run.smithi148.stdout: "cap+": 7, 2024-09-12T03:04:45.555 INFO:teuthology.orchestra.run.smithi148.stdout: "cap-": 7, 2024-09-12T03:04:45.555 INFO:teuthology.orchestra.run.smithi148.stdout: "rss": 51328, 2024-09-12T03:04:45.555 INFO:teuthology.orchestra.run.smithi148.stdout: "heap": 223520 2024-09-12T03:04:45.555 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:45.555 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_server": { 2024-09-12T03:04:45.555 INFO:teuthology.orchestra.run.smithi148.stdout: "dispatch_client_request": 28, 2024-09-12T03:04:45.555 INFO:teuthology.orchestra.run.smithi148.stdout: "dispatch_server_request": 1, 2024-09-12T03:04:45.555 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_request": 17, 2024-09-12T03:04:45.555 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_session": 9, 2024-09-12T03:04:45.555 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_peer_request": 5, 2024-09-12T03:04:45.555 INFO:teuthology.orchestra.run.smithi148.stdout: "req_create_latency": { 2024-09-12T03:04:45.555 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:45.555 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:45.556 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:45.556 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:45.556 INFO:teuthology.orchestra.run.smithi148.stdout: "req_getattr_latency": { 2024-09-12T03:04:45.556 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 1, 2024-09-12T03:04:45.556 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.001519513, 2024-09-12T03:04:45.556 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.001519513 2024-09-12T03:04:45.556 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:45.556 INFO:teuthology.orchestra.run.smithi148.stdout: "req_getfilelock_latency": { 2024-09-12T03:04:45.556 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:45.556 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:45.556 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:45.556 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:45.556 INFO:teuthology.orchestra.run.smithi148.stdout: "req_link_latency": { 2024-09-12T03:04:45.557 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:45.557 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:45.557 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:45.557 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:45.557 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookup_latency": { 2024-09-12T03:04:45.557 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 3, 2024-09-12T03:04:45.557 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.001134994, 2024-09-12T03:04:45.557 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000378331 2024-09-12T03:04:45.557 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:45.557 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookuphash_latency": { 2024-09-12T03:04:45.557 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:45.557 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:45.557 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:45.557 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:45.558 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookupino_latency": { 2024-09-12T03:04:45.558 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:45.558 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:45.558 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:45.558 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:45.558 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookupname_latency": { 2024-09-12T03:04:45.558 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:45.558 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:45.558 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:45.558 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:45.558 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookupparent_latency": { 2024-09-12T03:04:45.558 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:45.558 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:45.558 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:45.559 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:45.559 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookupsnap_latency": { 2024-09-12T03:04:45.559 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:45.559 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:45.559 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:45.559 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:45.559 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lssnap_latency": { 2024-09-12T03:04:45.559 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:45.559 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:45.559 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:45.559 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:45.559 INFO:teuthology.orchestra.run.smithi148.stdout: "req_mkdir_latency": { 2024-09-12T03:04:45.560 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:45.560 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:45.560 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:45.560 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:45.560 INFO:teuthology.orchestra.run.smithi148.stdout: "req_mknod_latency": { 2024-09-12T03:04:45.561 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:45.561 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:45.561 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:45.561 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:45.561 INFO:teuthology.orchestra.run.smithi148.stdout: "req_mksnap_latency": { 2024-09-12T03:04:45.561 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:45.561 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:45.561 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:45.561 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:45.561 INFO:teuthology.orchestra.run.smithi148.stdout: "req_open_latency": { 2024-09-12T03:04:45.561 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:45.561 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:45.561 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:45.562 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:45.562 INFO:teuthology.orchestra.run.smithi148.stdout: "req_readdir_latency": { 2024-09-12T03:04:45.562 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 2, 2024-09-12T03:04:45.562 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000555389, 2024-09-12T03:04:45.562 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000277694 2024-09-12T03:04:45.562 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:45.562 INFO:teuthology.orchestra.run.smithi148.stdout: "req_rename_latency": { 2024-09-12T03:04:45.562 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:45.562 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:45.562 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:45.562 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:45.563 INFO:teuthology.orchestra.run.smithi148.stdout: "req_renamesnap_latency": { 2024-09-12T03:04:45.563 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:45.563 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:45.563 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:45.563 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:45.563 INFO:teuthology.orchestra.run.smithi148.stdout: "req_snapdiff_latency": { 2024-09-12T03:04:45.563 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:45.563 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:45.563 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:45.563 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:45.563 INFO:teuthology.orchestra.run.smithi148.stdout: "req_rmdir_latency": { 2024-09-12T03:04:45.563 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 1, 2024-09-12T03:04:45.563 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.004791133, 2024-09-12T03:04:45.563 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.004791133 2024-09-12T03:04:45.564 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:45.564 INFO:teuthology.orchestra.run.smithi148.stdout: "req_rmsnap_latency": { 2024-09-12T03:04:45.564 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:45.564 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:45.564 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:45.564 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:45.564 INFO:teuthology.orchestra.run.smithi148.stdout: "req_rmxattr_latency": { 2024-09-12T03:04:45.564 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:45.564 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:45.564 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:45.564 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:45.564 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setattr_latency": { 2024-09-12T03:04:45.564 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:45.565 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:45.565 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:45.565 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:45.565 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setdirlayout_latency": { 2024-09-12T03:04:45.565 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:45.565 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:45.565 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:45.565 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:45.565 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setfilelock_latency": { 2024-09-12T03:04:45.565 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:45.565 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:45.565 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:45.565 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:45.565 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setlayout_latency": { 2024-09-12T03:04:45.566 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:45.566 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:45.566 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:45.566 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:45.566 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setxattr_latency": { 2024-09-12T03:04:45.566 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:45.566 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:45.566 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:45.566 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:45.566 INFO:teuthology.orchestra.run.smithi148.stdout: "req_symlink_latency": { 2024-09-12T03:04:45.566 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:45.566 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:45.566 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:45.566 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:45.567 INFO:teuthology.orchestra.run.smithi148.stdout: "req_unlink_latency": { 2024-09-12T03:04:45.567 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 1, 2024-09-12T03:04:45.567 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000842319, 2024-09-12T03:04:45.567 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000842319 2024-09-12T03:04:45.567 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:45.567 INFO:teuthology.orchestra.run.smithi148.stdout: "cap_revoke_eviction": 0, 2024-09-12T03:04:45.567 INFO:teuthology.orchestra.run.smithi148.stdout: "cap_acquisition_throttle": 0, 2024-09-12T03:04:45.567 INFO:teuthology.orchestra.run.smithi148.stdout: "req_getvxattr_latency": { 2024-09-12T03:04:45.567 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:45.567 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:45.567 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:45.567 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-12T03:04:45.567 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:45.567 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_sessions": { 2024-09-12T03:04:45.568 INFO:teuthology.orchestra.run.smithi148.stdout: "session_count": 1, 2024-09-12T03:04:45.568 INFO:teuthology.orchestra.run.smithi148.stdout: "session_add": 2, 2024-09-12T03:04:45.568 INFO:teuthology.orchestra.run.smithi148.stdout: "session_remove": 1, 2024-09-12T03:04:45.568 INFO:teuthology.orchestra.run.smithi148.stdout: "sessions_open": 1, 2024-09-12T03:04:45.568 INFO:teuthology.orchestra.run.smithi148.stdout: "sessions_stale": 0, 2024-09-12T03:04:45.568 INFO:teuthology.orchestra.run.smithi148.stdout: "total_load": 4, 2024-09-12T03:04:45.568 INFO:teuthology.orchestra.run.smithi148.stdout: "average_load": 4, 2024-09-12T03:04:45.568 INFO:teuthology.orchestra.run.smithi148.stdout: "avg_session_uptime": 1636, 2024-09-12T03:04:45.568 INFO:teuthology.orchestra.run.smithi148.stdout: "mdthresh_evicted": 0 2024-09-12T03:04:45.568 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:45.568 INFO:teuthology.orchestra.run.smithi148.stdout: "mempool": { 2024-09-12T03:04:45.568 INFO:teuthology.orchestra.run.smithi148.stdout: "bloom_filter_bytes": 0, 2024-09-12T03:04:45.568 INFO:teuthology.orchestra.run.smithi148.stdout: "bloom_filter_items": 0, 2024-09-12T03:04:45.569 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T03:04:45.569 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_alloc_items": 0, 2024-09-12T03:04:45.569 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T03:04:45.569 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_data_items": 0, 2024-09-12T03:04:45.569 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T03:04:45.569 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T03:04:45.569 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T03:04:45.569 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T03:04:45.569 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T03:04:45.569 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_other_items": 0, 2024-09-12T03:04:45.569 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T03:04:45.569 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T03:04:45.569 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_extent_bytes": 0, 2024-09-12T03:04:45.570 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_extent_items": 0, 2024-09-12T03:04:45.570 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_blob_bytes": 0, 2024-09-12T03:04:45.570 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_blob_items": 0, 2024-09-12T03:04:45.570 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T03:04:45.570 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T03:04:45.570 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T03:04:45.570 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T03:04:45.570 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T03:04:45.570 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_fsck_items": 0, 2024-09-12T03:04:45.570 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_txc_bytes": 0, 2024-09-12T03:04:45.570 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_txc_items": 0, 2024-09-12T03:04:45.570 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T03:04:45.570 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T03:04:45.571 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_bytes": 0, 2024-09-12T03:04:45.571 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_items": 0, 2024-09-12T03:04:45.571 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_bytes": 0, 2024-09-12T03:04:45.571 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_items": 0, 2024-09-12T03:04:45.571 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T03:04:45.571 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_reader_items": 0, 2024-09-12T03:04:45.571 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T03:04:45.571 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_writer_items": 0, 2024-09-12T03:04:45.571 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_anon_bytes": 262183, 2024-09-12T03:04:45.571 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_anon_items": 82, 2024-09-12T03:04:45.571 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_meta_bytes": 0, 2024-09-12T03:04:45.571 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_meta_items": 0, 2024-09-12T03:04:45.571 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_bytes": 0, 2024-09-12T03:04:45.571 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_items": 0, 2024-09-12T03:04:45.572 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_mapbl_bytes": 0, 2024-09-12T03:04:45.572 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_mapbl_items": 0, 2024-09-12T03:04:45.572 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_pglog_bytes": 0, 2024-09-12T03:04:45.572 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_pglog_items": 0, 2024-09-12T03:04:45.572 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_bytes": 7496, 2024-09-12T03:04:45.572 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_items": 154, 2024-09-12T03:04:45.572 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T03:04:45.572 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_mapping_items": 0, 2024-09-12T03:04:45.572 INFO:teuthology.orchestra.run.smithi148.stdout: "pgmap_bytes": 0, 2024-09-12T03:04:45.572 INFO:teuthology.orchestra.run.smithi148.stdout: "pgmap_items": 0, 2024-09-12T03:04:45.572 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_co_bytes": 76917, 2024-09-12T03:04:45.572 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_co_items": 332, 2024-09-12T03:04:45.572 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_1_bytes": 0, 2024-09-12T03:04:45.573 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_1_items": 0, 2024-09-12T03:04:45.573 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_2_bytes": 0, 2024-09-12T03:04:45.573 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_2_items": 0 2024-09-12T03:04:45.573 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:45.573 INFO:teuthology.orchestra.run.smithi148.stdout: "objecter": { 2024-09-12T03:04:45.573 INFO:teuthology.orchestra.run.smithi148.stdout: "op_active": 0, 2024-09-12T03:04:45.573 INFO:teuthology.orchestra.run.smithi148.stdout: "op_laggy": 0, 2024-09-12T03:04:45.573 INFO:teuthology.orchestra.run.smithi148.stdout: "op_send": 88, 2024-09-12T03:04:45.573 INFO:teuthology.orchestra.run.smithi148.stdout: "op_send_bytes": 44798, 2024-09-12T03:04:45.573 INFO:teuthology.orchestra.run.smithi148.stdout: "op_resend": 33, 2024-09-12T03:04:45.573 INFO:teuthology.orchestra.run.smithi148.stdout: "op_reply": 88, 2024-09-12T03:04:45.573 INFO:teuthology.orchestra.run.smithi148.stdout: "op_latency": { 2024-09-12T03:04:45.573 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 88, 2024-09-12T03:04:45.573 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.443988793, 2024-09-12T03:04:45.574 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.005045327 2024-09-12T03:04:45.574 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:45.574 INFO:teuthology.orchestra.run.smithi148.stdout: "op_inflight": 0, 2024-09-12T03:04:45.574 INFO:teuthology.orchestra.run.smithi148.stdout: "oplen_avg": { 2024-09-12T03:04:45.574 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 88, 2024-09-12T03:04:45.574 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 118 2024-09-12T03:04:45.574 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:45.574 INFO:teuthology.orchestra.run.smithi148.stdout: "op": 88, 2024-09-12T03:04:45.574 INFO:teuthology.orchestra.run.smithi148.stdout: "op_r": 13, 2024-09-12T03:04:45.574 INFO:teuthology.orchestra.run.smithi148.stdout: "op_w": 75, 2024-09-12T03:04:45.574 INFO:teuthology.orchestra.run.smithi148.stdout: "op_rmw": 0, 2024-09-12T03:04:45.574 INFO:teuthology.orchestra.run.smithi148.stdout: "op_pg": 0, 2024-09-12T03:04:45.574 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_stat": 0, 2024-09-12T03:04:45.574 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_create": 10, 2024-09-12T03:04:45.575 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_read": 1, 2024-09-12T03:04:45.575 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_write": 18, 2024-09-12T03:04:45.575 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_writefull": 12, 2024-09-12T03:04:45.575 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_writesame": 0, 2024-09-12T03:04:45.575 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_append": 0, 2024-09-12T03:04:45.575 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_zero": 0, 2024-09-12T03:04:45.575 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_truncate": 0, 2024-09-12T03:04:45.575 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_delete": 14, 2024-09-12T03:04:45.575 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_mapext": 0, 2024-09-12T03:04:45.575 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_sparse_read": 0, 2024-09-12T03:04:45.575 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_clonerange": 0, 2024-09-12T03:04:45.575 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_getxattr": 12, 2024-09-12T03:04:45.575 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_setxattr": 20, 2024-09-12T03:04:45.575 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_cmpxattr": 0, 2024-09-12T03:04:45.575 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_rmxattr": 0, 2024-09-12T03:04:45.576 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_resetxattrs": 0, 2024-09-12T03:04:45.576 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_call": 0, 2024-09-12T03:04:45.576 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_watch": 0, 2024-09-12T03:04:45.576 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_notify": 0, 2024-09-12T03:04:45.576 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_src_cmpxattr": 0, 2024-09-12T03:04:45.576 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_pgls": 0, 2024-09-12T03:04:45.576 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_pgls_filter": 0, 2024-09-12T03:04:45.576 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_other": 0, 2024-09-12T03:04:45.576 INFO:teuthology.orchestra.run.smithi148.stdout: "linger_active": 0, 2024-09-12T03:04:45.576 INFO:teuthology.orchestra.run.smithi148.stdout: "linger_send": 0, 2024-09-12T03:04:45.577 INFO:teuthology.orchestra.run.smithi148.stdout: "linger_resend": 0, 2024-09-12T03:04:45.577 INFO:teuthology.orchestra.run.smithi148.stdout: "linger_ping": 0, 2024-09-12T03:04:45.577 INFO:teuthology.orchestra.run.smithi148.stdout: "poolop_active": 0, 2024-09-12T03:04:45.577 INFO:teuthology.orchestra.run.smithi148.stdout: "poolop_send": 0, 2024-09-12T03:04:45.577 INFO:teuthology.orchestra.run.smithi148.stdout: "poolop_resend": 0, 2024-09-12T03:04:45.577 INFO:teuthology.orchestra.run.smithi148.stdout: "poolstat_active": 0, 2024-09-12T03:04:45.577 INFO:teuthology.orchestra.run.smithi148.stdout: "poolstat_send": 0, 2024-09-12T03:04:45.577 INFO:teuthology.orchestra.run.smithi148.stdout: "poolstat_resend": 0, 2024-09-12T03:04:45.577 INFO:teuthology.orchestra.run.smithi148.stdout: "statfs_active": 0, 2024-09-12T03:04:45.577 INFO:teuthology.orchestra.run.smithi148.stdout: "statfs_send": 0, 2024-09-12T03:04:45.577 INFO:teuthology.orchestra.run.smithi148.stdout: "statfs_resend": 0, 2024-09-12T03:04:45.577 INFO:teuthology.orchestra.run.smithi148.stdout: "command_active": 0, 2024-09-12T03:04:45.578 INFO:teuthology.orchestra.run.smithi148.stdout: "command_send": 0, 2024-09-12T03:04:45.578 INFO:teuthology.orchestra.run.smithi148.stdout: "command_resend": 0, 2024-09-12T03:04:45.578 INFO:teuthology.orchestra.run.smithi148.stdout: "map_epoch": 83, 2024-09-12T03:04:45.578 INFO:teuthology.orchestra.run.smithi148.stdout: "map_full": 0, 2024-09-12T03:04:45.578 INFO:teuthology.orchestra.run.smithi148.stdout: "map_inc": 2, 2024-09-12T03:04:45.578 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_sessions": 10, 2024-09-12T03:04:45.578 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_session_open": 10, 2024-09-12T03:04:45.578 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_session_close": 0, 2024-09-12T03:04:45.578 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_laggy": 0, 2024-09-12T03:04:45.578 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_wr": 26, 2024-09-12T03:04:45.578 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_rd": 0, 2024-09-12T03:04:45.578 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_del": 5 2024-09-12T03:04:45.578 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:45.578 INFO:teuthology.orchestra.run.smithi148.stdout: "oft": { 2024-09-12T03:04:45.579 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_total_objs": 1, 2024-09-12T03:04:45.579 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_total_kv_pairs": 0, 2024-09-12T03:04:45.579 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_total_updates": 0, 2024-09-12T03:04:45.579 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_total_removes": 0 2024-09-12T03:04:45.579 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:45.579 INFO:teuthology.orchestra.run.smithi148.stdout: "osd-slow-ops": { 2024-09-12T03:04:45.579 INFO:teuthology.orchestra.run.smithi148.stdout: "slow_ops_count": 60 2024-09-12T03:04:45.579 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:45.579 INFO:teuthology.orchestra.run.smithi148.stdout: "purge_queue": { 2024-09-12T03:04:45.579 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executing_ops": 0, 2024-09-12T03:04:45.579 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executing_ops_high_water": 2, 2024-09-12T03:04:45.579 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executing": 0, 2024-09-12T03:04:45.579 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executing_high_water": 2, 2024-09-12T03:04:45.579 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executed_ops": 2, 2024-09-12T03:04:45.580 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executed": 2, 2024-09-12T03:04:45.580 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_item_in_journal": 0 2024-09-12T03:04:45.580 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:45.580 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T03:04:45.580 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-09-12T03:04:45.580 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 104857600, 2024-09-12T03:04:45.580 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-09-12T03:04:45.580 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 1384, 2024-09-12T03:04:45.580 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 1056163, 2024-09-12T03:04:45.580 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-09-12T03:04:45.580 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 1384, 2024-09-12T03:04:45.580 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 0, 2024-09-12T03:04:45.580 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 0, 2024-09-12T03:04:45.580 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 1384, 2024-09-12T03:04:45.581 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 1056163, 2024-09-12T03:04:45.581 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-09-12T03:04:45.581 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:45.581 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:45.581 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:45.581 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-12T03:04:45.581 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:45.581 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-objecter_bytes": { 2024-09-12T03:04:45.581 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-09-12T03:04:45.581 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 104857600, 2024-09-12T03:04:45.581 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-09-12T03:04:45.581 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 0, 2024-09-12T03:04:45.581 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 0, 2024-09-12T03:04:45.582 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-09-12T03:04:45.582 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 0, 2024-09-12T03:04:45.582 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 88, 2024-09-12T03:04:45.582 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 45000, 2024-09-12T03:04:45.582 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 88, 2024-09-12T03:04:45.582 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 45000, 2024-09-12T03:04:45.582 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-09-12T03:04:45.582 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:45.582 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:45.582 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:45.582 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-12T03:04:45.582 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:45.582 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-objecter_ops": { 2024-09-12T03:04:45.582 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-09-12T03:04:45.583 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 1024, 2024-09-12T03:04:45.583 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-09-12T03:04:45.583 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 0, 2024-09-12T03:04:45.583 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 0, 2024-09-12T03:04:45.583 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-09-12T03:04:45.583 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 0, 2024-09-12T03:04:45.583 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 88, 2024-09-12T03:04:45.583 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 88, 2024-09-12T03:04:45.583 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 88, 2024-09-12T03:04:45.583 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 88, 2024-09-12T03:04:45.583 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-09-12T03:04:45.583 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:45.583 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:45.584 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:45.584 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-12T03:04:45.584 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:45.584 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-write_buf_throttle": { 2024-09-12T03:04:45.584 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-09-12T03:04:45.584 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 3758096384, 2024-09-12T03:04:45.584 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-09-12T03:04:45.584 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 2, 2024-09-12T03:04:45.584 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 202, 2024-09-12T03:04:45.584 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-09-12T03:04:45.584 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 2, 2024-09-12T03:04:45.584 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 0, 2024-09-12T03:04:45.584 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 0, 2024-09-12T03:04:45.584 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 1, 2024-09-12T03:04:45.585 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 202, 2024-09-12T03:04:45.585 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-09-12T03:04:45.585 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:45.585 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:45.585 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:45.585 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-12T03:04:45.585 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:45.585 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-write_buf_throttle-0x560645c50960": { 2024-09-12T03:04:45.585 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-09-12T03:04:45.585 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 3758096384, 2024-09-12T03:04:45.585 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-09-12T03:04:45.585 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 20, 2024-09-12T03:04:45.585 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 32167, 2024-09-12T03:04:45.586 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-09-12T03:04:45.586 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 20, 2024-09-12T03:04:45.586 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 0, 2024-09-12T03:04:45.586 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 0, 2024-09-12T03:04:45.586 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 17, 2024-09-12T03:04:45.586 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 32167, 2024-09-12T03:04:45.586 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-09-12T03:04:45.586 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:45.586 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:45.586 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:45.586 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-12T03:04:45.586 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-12T03:04:45.586 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-12T03:04:46.394 INFO:tasks.check_counter:Daemon mds.c mds.exported=1 2024-09-12T03:04:46.395 INFO:tasks.check_counter:Daemon mds.c mds.imported=1 2024-09-12T03:04:46.395 INFO:tasks.check_counter:Daemon mds.c mds.dir_split=0 2024-09-12T03:04:46.395 DEBUG:tasks.check_counter:Getting stats from f 2024-09-12T03:04:46.395 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph daemon mds.f perf dump 2024-09-12T03:04:46.652 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T03:04:47.094 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-12T03:04:47.094 INFO:teuthology.orchestra.run.smithi148.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T03:04:47.095 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_messages": 892, 2024-09-12T03:04:47.095 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages": 681, 2024-09-12T03:04:47.095 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_bytes": 287460, 2024-09-12T03:04:47.095 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_bytes": 385329, 2024-09-12T03:04:47.095 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_created_connections": 17, 2024-09-12T03:04:47.095 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_active_connections": 3, 2024-09-12T03:04:47.095 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_total_time": 0.074440479, 2024-09-12T03:04:47.095 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_send_time": 0.030739698, 2024-09-12T03:04:47.095 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_recv_time": 0.043163537, 2024-09-12T03:04:47.095 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_fast_dispatch_time": 0.003484485, 2024-09-12T03:04:47.095 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:04:47.095 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 681, 2024-09-12T03:04:47.096 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.036764923, 2024-09-12T03:04:47.096 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000053986 2024-09-12T03:04:47.096 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:47.096 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:04:47.096 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 1453, 2024-09-12T03:04:47.096 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000286463, 2024-09-12T03:04:47.096 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000197 2024-09-12T03:04:47.096 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:47.096 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-12T03:04:47.096 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_encrypted_bytes": 0 2024-09-12T03:04:47.096 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:47.096 INFO:teuthology.orchestra.run.smithi148.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T03:04:47.096 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_messages": 46, 2024-09-12T03:04:47.097 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages": 154, 2024-09-12T03:04:47.097 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_bytes": 12485, 2024-09-12T03:04:47.097 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_bytes": 435703, 2024-09-12T03:04:47.097 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_created_connections": 19, 2024-09-12T03:04:47.097 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_active_connections": 7, 2024-09-12T03:04:47.097 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_total_time": 0.015992582, 2024-09-12T03:04:47.097 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_send_time": 0.005025889, 2024-09-12T03:04:47.097 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_recv_time": 0.008717595, 2024-09-12T03:04:47.097 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_fast_dispatch_time": 0.000775918, 2024-09-12T03:04:47.097 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:04:47.097 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 153, 2024-09-12T03:04:47.097 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.021493922, 2024-09-12T03:04:47.097 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000140483 2024-09-12T03:04:47.097 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:47.098 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:04:47.098 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 12, 2024-09-12T03:04:47.098 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000915, 2024-09-12T03:04:47.098 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000076 2024-09-12T03:04:47.098 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:47.098 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_encrypted_bytes": 3184, 2024-09-12T03:04:47.098 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_encrypted_bytes": 271296 2024-09-12T03:04:47.098 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:47.099 INFO:teuthology.orchestra.run.smithi148.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T03:04:47.099 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_messages": 268, 2024-09-12T03:04:47.099 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages": 263, 2024-09-12T03:04:47.099 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_bytes": 841764, 2024-09-12T03:04:47.099 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_bytes": 150811, 2024-09-12T03:04:47.099 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_created_connections": 11, 2024-09-12T03:04:47.099 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_active_connections": 6, 2024-09-12T03:04:47.100 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_total_time": 0.035077973, 2024-09-12T03:04:47.100 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_send_time": 0.012008198, 2024-09-12T03:04:47.100 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_recv_time": 0.030901505, 2024-09-12T03:04:47.100 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_fast_dispatch_time": 0.000623854, 2024-09-12T03:04:47.100 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:04:47.100 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 260, 2024-09-12T03:04:47.100 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.015033108, 2024-09-12T03:04:47.100 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000057819 2024-09-12T03:04:47.100 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:47.100 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:04:47.100 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 62, 2024-09-12T03:04:47.100 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000010624, 2024-09-12T03:04:47.100 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000171 2024-09-12T03:04:47.101 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:47.101 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_encrypted_bytes": 834016, 2024-09-12T03:04:47.101 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_encrypted_bytes": 116208 2024-09-12T03:04:47.101 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:47.101 INFO:teuthology.orchestra.run.smithi148.stdout: "cct": { 2024-09-12T03:04:47.101 INFO:teuthology.orchestra.run.smithi148.stdout: "total_workers": 1, 2024-09-12T03:04:47.101 INFO:teuthology.orchestra.run.smithi148.stdout: "unhealthy_workers": 0 2024-09-12T03:04:47.101 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:47.102 INFO:teuthology.orchestra.run.smithi148.stdout: "finisher-MDSRank": { 2024-09-12T03:04:47.102 INFO:teuthology.orchestra.run.smithi148.stdout: "queue_len": 0, 2024-09-12T03:04:47.102 INFO:teuthology.orchestra.run.smithi148.stdout: "complete_latency": { 2024-09-12T03:04:47.102 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 92, 2024-09-12T03:04:47.102 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.006504638, 2024-09-12T03:04:47.102 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000070702 2024-09-12T03:04:47.102 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-12T03:04:47.102 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:47.102 INFO:teuthology.orchestra.run.smithi148.stdout: "finisher-PurgeQueue": { 2024-09-12T03:04:47.102 INFO:teuthology.orchestra.run.smithi148.stdout: "queue_len": 0, 2024-09-12T03:04:47.102 INFO:teuthology.orchestra.run.smithi148.stdout: "complete_latency": { 2024-09-12T03:04:47.102 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 20, 2024-09-12T03:04:47.102 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000499666, 2024-09-12T03:04:47.102 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000024983 2024-09-12T03:04:47.103 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-12T03:04:47.103 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:47.103 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": { 2024-09-12T03:04:47.103 INFO:teuthology.orchestra.run.smithi148.stdout: "request": 13, 2024-09-12T03:04:47.103 INFO:teuthology.orchestra.run.smithi148.stdout: "reply": 10, 2024-09-12T03:04:47.103 INFO:teuthology.orchestra.run.smithi148.stdout: "reply_latency": { 2024-09-12T03:04:47.103 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 10, 2024-09-12T03:04:47.103 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.006939213, 2024-09-12T03:04:47.103 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000693921 2024-09-12T03:04:47.103 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:47.103 INFO:teuthology.orchestra.run.smithi148.stdout: "slow_reply": 0, 2024-09-12T03:04:47.103 INFO:teuthology.orchestra.run.smithi148.stdout: "forward": 3, 2024-09-12T03:04:47.103 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_fetch_complete": 0, 2024-09-12T03:04:47.104 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_fetch_keys": 0, 2024-09-12T03:04:47.104 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_commit": 11, 2024-09-12T03:04:47.104 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_split": 0, 2024-09-12T03:04:47.104 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_merge": 0, 2024-09-12T03:04:47.104 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes": 15, 2024-09-12T03:04:47.104 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_top": 1, 2024-09-12T03:04:47.104 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_bottom": 14, 2024-09-12T03:04:47.104 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_pin_tail": 0, 2024-09-12T03:04:47.104 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_pinned": 13, 2024-09-12T03:04:47.104 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_expired": 8, 2024-09-12T03:04:47.104 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_with_caps": 0, 2024-09-12T03:04:47.104 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": 0, 2024-09-12T03:04:47.104 INFO:teuthology.orchestra.run.smithi148.stdout: "subtrees": 3, 2024-09-12T03:04:47.105 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse": 37, 2024-09-12T03:04:47.105 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_hit": 25, 2024-09-12T03:04:47.105 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_forward": 3, 2024-09-12T03:04:47.105 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_discover": 1, 2024-09-12T03:04:47.105 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_dir_fetch": 0, 2024-09-12T03:04:47.105 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_remote_ino": 0, 2024-09-12T03:04:47.105 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_lock": 0, 2024-09-12T03:04:47.105 INFO:teuthology.orchestra.run.smithi148.stdout: "load_cent": 0, 2024-09-12T03:04:47.105 INFO:teuthology.orchestra.run.smithi148.stdout: "q": 0, 2024-09-12T03:04:47.105 INFO:teuthology.orchestra.run.smithi148.stdout: "exported": 1, 2024-09-12T03:04:47.105 INFO:teuthology.orchestra.run.smithi148.stdout: "exported_inodes": 0, 2024-09-12T03:04:47.105 INFO:teuthology.orchestra.run.smithi148.stdout: "imported": 1, 2024-09-12T03:04:47.105 INFO:teuthology.orchestra.run.smithi148.stdout: "imported_inodes": 9, 2024-09-12T03:04:47.106 INFO:teuthology.orchestra.run.smithi148.stdout: "openino_dir_fetch": 0, 2024-09-12T03:04:47.106 INFO:teuthology.orchestra.run.smithi148.stdout: "openino_backtrace_fetch": 25, 2024-09-12T03:04:47.106 INFO:teuthology.orchestra.run.smithi148.stdout: "openino_peer_discover": 0, 2024-09-12T03:04:47.106 INFO:teuthology.orchestra.run.smithi148.stdout: "root_rfiles": 4570, 2024-09-12T03:04:47.106 INFO:teuthology.orchestra.run.smithi148.stdout: "root_rbytes": 147481294, 2024-09-12T03:04:47.106 INFO:teuthology.orchestra.run.smithi148.stdout: "root_rsnaps": 0, 2024-09-12T03:04:47.106 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_backtrace_fetch": 25, 2024-09-12T03:04:47.106 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_set_tag": 0, 2024-09-12T03:04:47.106 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_backtrace_repaired": 0, 2024-09-12T03:04:47.106 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_inotable_repaired": 0, 2024-09-12T03:04:47.106 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_dir_inodes": 0, 2024-09-12T03:04:47.106 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_dir_base_inodes": 0, 2024-09-12T03:04:47.106 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_dirfrag_rstats": 0, 2024-09-12T03:04:47.106 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_file_inodes": 25, 2024-09-12T03:04:47.107 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_inode_file_caps": 0, 2024-09-12T03:04:47.107 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_revoke": 1, 2024-09-12T03:04:47.107 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_grant": 6, 2024-09-12T03:04:47.107 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_trunc": 0, 2024-09-12T03:04:47.107 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-12T03:04:47.107 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-12T03:04:47.107 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_caps": 1, 2024-09-12T03:04:47.107 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_caps_dirty": 0, 2024-09-12T03:04:47.107 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_cap_release": 1, 2024-09-12T03:04:47.107 INFO:teuthology.orchestra.run.smithi148.stdout: "process_request_cap_release": 10 2024-09-12T03:04:47.107 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:47.107 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_cache": { 2024-09-12T03:04:47.107 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_update": 0, 2024-09-12T03:04:47.107 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_update_receipt": 0, 2024-09-12T03:04:47.108 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_try_discover": 0, 2024-09-12T03:04:47.108 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_send_discover": 2, 2024-09-12T03:04:47.108 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_handle_discover": 0, 2024-09-12T03:04:47.108 INFO:teuthology.orchestra.run.smithi148.stdout: "num_strays": 0, 2024-09-12T03:04:47.108 INFO:teuthology.orchestra.run.smithi148.stdout: "num_strays_delayed": 0, 2024-09-12T03:04:47.108 INFO:teuthology.orchestra.run.smithi148.stdout: "num_strays_enqueuing": 0, 2024-09-12T03:04:47.108 INFO:teuthology.orchestra.run.smithi148.stdout: "strays_created": 5, 2024-09-12T03:04:47.108 INFO:teuthology.orchestra.run.smithi148.stdout: "strays_enqueued": 5, 2024-09-12T03:04:47.108 INFO:teuthology.orchestra.run.smithi148.stdout: "strays_reintegrated": 0, 2024-09-12T03:04:47.108 INFO:teuthology.orchestra.run.smithi148.stdout: "strays_migrated": 0, 2024-09-12T03:04:47.108 INFO:teuthology.orchestra.run.smithi148.stdout: "num_recovering_processing": 0, 2024-09-12T03:04:47.108 INFO:teuthology.orchestra.run.smithi148.stdout: "num_recovering_enqueued": 0, 2024-09-12T03:04:47.108 INFO:teuthology.orchestra.run.smithi148.stdout: "num_recovering_prioritized": 0, 2024-09-12T03:04:47.108 INFO:teuthology.orchestra.run.smithi148.stdout: "recovery_started": 0, 2024-09-12T03:04:47.109 INFO:teuthology.orchestra.run.smithi148.stdout: "recovery_completed": 0, 2024-09-12T03:04:47.109 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_quiesce_path": 5, 2024-09-12T03:04:47.109 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_quiesce_inode": 44, 2024-09-12T03:04:47.109 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_enqueue_scrub": 0, 2024-09-12T03:04:47.109 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_exportdir": 1, 2024-09-12T03:04:47.109 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_flush": 0, 2024-09-12T03:04:47.109 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_fragmentdir": 0, 2024-09-12T03:04:47.109 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_fragstats": 0, 2024-09-12T03:04:47.109 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_inodestats": 0 2024-09-12T03:04:47.109 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:47.110 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_log": { 2024-09-12T03:04:47.110 INFO:teuthology.orchestra.run.smithi148.stdout: "evlrg": 0, 2024-09-12T03:04:47.110 INFO:teuthology.orchestra.run.smithi148.stdout: "evadd": 24, 2024-09-12T03:04:47.110 INFO:teuthology.orchestra.run.smithi148.stdout: "evex": 0, 2024-09-12T03:04:47.110 INFO:teuthology.orchestra.run.smithi148.stdout: "evtrm": 0, 2024-09-12T03:04:47.110 INFO:teuthology.orchestra.run.smithi148.stdout: "ev": 24, 2024-09-12T03:04:47.110 INFO:teuthology.orchestra.run.smithi148.stdout: "evexg": 0, 2024-09-12T03:04:47.110 INFO:teuthology.orchestra.run.smithi148.stdout: "evexd": 0, 2024-09-12T03:04:47.110 INFO:teuthology.orchestra.run.smithi148.stdout: "segadd": 2, 2024-09-12T03:04:47.110 INFO:teuthology.orchestra.run.smithi148.stdout: "segex": 0, 2024-09-12T03:04:47.110 INFO:teuthology.orchestra.run.smithi148.stdout: "segtrm": 0, 2024-09-12T03:04:47.110 INFO:teuthology.orchestra.run.smithi148.stdout: "seg": 2, 2024-09-12T03:04:47.110 INFO:teuthology.orchestra.run.smithi148.stdout: "segmjr": 2, 2024-09-12T03:04:47.110 INFO:teuthology.orchestra.run.smithi148.stdout: "segexg": 0, 2024-09-12T03:04:47.110 INFO:teuthology.orchestra.run.smithi148.stdout: "segexd": 0, 2024-09-12T03:04:47.111 INFO:teuthology.orchestra.run.smithi148.stdout: "expos": 4194304, 2024-09-12T03:04:47.111 INFO:teuthology.orchestra.run.smithi148.stdout: "wrpos": 4229211, 2024-09-12T03:04:47.111 INFO:teuthology.orchestra.run.smithi148.stdout: "rdpos": 0, 2024-09-12T03:04:47.111 INFO:teuthology.orchestra.run.smithi148.stdout: "jlat": { 2024-09-12T03:04:47.111 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 13, 2024-09-12T03:04:47.111 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.025046711, 2024-09-12T03:04:47.111 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.001926670 2024-09-12T03:04:47.111 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:47.111 INFO:teuthology.orchestra.run.smithi148.stdout: "replayed": 0 2024-09-12T03:04:47.111 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:47.111 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_mem": { 2024-09-12T03:04:47.112 INFO:teuthology.orchestra.run.smithi148.stdout: "ino": 19, 2024-09-12T03:04:47.112 INFO:teuthology.orchestra.run.smithi148.stdout: "ino+": 28, 2024-09-12T03:04:47.112 INFO:teuthology.orchestra.run.smithi148.stdout: "ino-": 9, 2024-09-12T03:04:47.112 INFO:teuthology.orchestra.run.smithi148.stdout: "dir": 18, 2024-09-12T03:04:47.112 INFO:teuthology.orchestra.run.smithi148.stdout: "dir+": 23, 2024-09-12T03:04:47.112 INFO:teuthology.orchestra.run.smithi148.stdout: "dir-": 5, 2024-09-12T03:04:47.112 INFO:teuthology.orchestra.run.smithi148.stdout: "dn": 15, 2024-09-12T03:04:47.112 INFO:teuthology.orchestra.run.smithi148.stdout: "dn+": 37, 2024-09-12T03:04:47.112 INFO:teuthology.orchestra.run.smithi148.stdout: "dn-": 22, 2024-09-12T03:04:47.112 INFO:teuthology.orchestra.run.smithi148.stdout: "cap": 0, 2024-09-12T03:04:47.112 INFO:teuthology.orchestra.run.smithi148.stdout: "cap+": 6, 2024-09-12T03:04:47.112 INFO:teuthology.orchestra.run.smithi148.stdout: "cap-": 6, 2024-09-12T03:04:47.112 INFO:teuthology.orchestra.run.smithi148.stdout: "rss": 51112, 2024-09-12T03:04:47.113 INFO:teuthology.orchestra.run.smithi148.stdout: "heap": 223520 2024-09-12T03:04:47.113 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:47.113 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_server": { 2024-09-12T03:04:47.113 INFO:teuthology.orchestra.run.smithi148.stdout: "dispatch_client_request": 20, 2024-09-12T03:04:47.113 INFO:teuthology.orchestra.run.smithi148.stdout: "dispatch_server_request": 1, 2024-09-12T03:04:47.113 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_request": 13, 2024-09-12T03:04:47.113 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_session": 13, 2024-09-12T03:04:47.113 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_peer_request": 8, 2024-09-12T03:04:47.113 INFO:teuthology.orchestra.run.smithi148.stdout: "req_create_latency": { 2024-09-12T03:04:47.113 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:47.113 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:47.113 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:47.113 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:47.113 INFO:teuthology.orchestra.run.smithi148.stdout: "req_getattr_latency": { 2024-09-12T03:04:47.114 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:47.114 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:47.114 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:47.114 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:47.114 INFO:teuthology.orchestra.run.smithi148.stdout: "req_getfilelock_latency": { 2024-09-12T03:04:47.114 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:47.114 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:47.114 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:47.114 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:47.114 INFO:teuthology.orchestra.run.smithi148.stdout: "req_link_latency": { 2024-09-12T03:04:47.114 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:47.114 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:47.114 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:47.114 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:47.114 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookup_latency": { 2024-09-12T03:04:47.115 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 4, 2024-09-12T03:04:47.115 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000869356, 2024-09-12T03:04:47.115 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000217339 2024-09-12T03:04:47.115 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:47.115 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookuphash_latency": { 2024-09-12T03:04:47.115 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:47.115 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:47.115 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:47.115 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:47.115 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookupino_latency": { 2024-09-12T03:04:47.115 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:47.115 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:47.115 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:47.116 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:47.116 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookupname_latency": { 2024-09-12T03:04:47.116 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:47.116 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:47.116 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:47.116 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:47.116 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookupparent_latency": { 2024-09-12T03:04:47.116 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:47.116 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:47.116 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:47.116 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:47.116 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookupsnap_latency": { 2024-09-12T03:04:47.116 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:47.116 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:47.117 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:47.117 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:47.117 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lssnap_latency": { 2024-09-12T03:04:47.117 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:47.117 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:47.117 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:47.117 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:47.117 INFO:teuthology.orchestra.run.smithi148.stdout: "req_mkdir_latency": { 2024-09-12T03:04:47.118 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:47.118 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:47.118 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:47.118 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:47.118 INFO:teuthology.orchestra.run.smithi148.stdout: "req_mknod_latency": { 2024-09-12T03:04:47.118 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:47.118 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:47.118 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:47.118 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:47.118 INFO:teuthology.orchestra.run.smithi148.stdout: "req_mksnap_latency": { 2024-09-12T03:04:47.118 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:47.119 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:47.119 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:47.119 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:47.119 INFO:teuthology.orchestra.run.smithi148.stdout: "req_open_latency": { 2024-09-12T03:04:47.119 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:47.119 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:47.119 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:47.119 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:47.119 INFO:teuthology.orchestra.run.smithi148.stdout: "req_readdir_latency": { 2024-09-12T03:04:47.119 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 1, 2024-09-12T03:04:47.119 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000299250, 2024-09-12T03:04:47.119 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000299250 2024-09-12T03:04:47.119 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:47.119 INFO:teuthology.orchestra.run.smithi148.stdout: "req_rename_latency": { 2024-09-12T03:04:47.120 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:47.120 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:47.120 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:47.120 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:47.120 INFO:teuthology.orchestra.run.smithi148.stdout: "req_renamesnap_latency": { 2024-09-12T03:04:47.120 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:47.120 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:47.120 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:47.120 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:47.120 INFO:teuthology.orchestra.run.smithi148.stdout: "req_snapdiff_latency": { 2024-09-12T03:04:47.120 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:47.120 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:47.120 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:47.121 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:47.121 INFO:teuthology.orchestra.run.smithi148.stdout: "req_rmdir_latency": { 2024-09-12T03:04:47.121 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:47.121 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:47.121 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:47.121 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:47.121 INFO:teuthology.orchestra.run.smithi148.stdout: "req_rmsnap_latency": { 2024-09-12T03:04:47.121 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:47.121 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:47.121 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:47.121 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:47.121 INFO:teuthology.orchestra.run.smithi148.stdout: "req_rmxattr_latency": { 2024-09-12T03:04:47.121 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:47.121 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:47.122 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:47.122 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:47.122 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setattr_latency": { 2024-09-12T03:04:47.122 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:47.122 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:47.122 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:47.122 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:47.122 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setdirlayout_latency": { 2024-09-12T03:04:47.122 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:47.122 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:47.122 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:47.122 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:47.122 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setfilelock_latency": { 2024-09-12T03:04:47.123 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:47.123 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:47.123 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:47.123 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:47.123 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setlayout_latency": { 2024-09-12T03:04:47.123 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:47.123 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:47.123 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:47.123 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:47.123 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setxattr_latency": { 2024-09-12T03:04:47.123 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:47.123 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:47.123 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:47.123 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:47.124 INFO:teuthology.orchestra.run.smithi148.stdout: "req_symlink_latency": { 2024-09-12T03:04:47.124 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:47.124 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:47.124 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:47.124 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:47.124 INFO:teuthology.orchestra.run.smithi148.stdout: "req_unlink_latency": { 2024-09-12T03:04:47.124 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 5, 2024-09-12T03:04:47.124 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.005770607, 2024-09-12T03:04:47.124 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.001154121 2024-09-12T03:04:47.124 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:47.124 INFO:teuthology.orchestra.run.smithi148.stdout: "cap_revoke_eviction": 0, 2024-09-12T03:04:47.124 INFO:teuthology.orchestra.run.smithi148.stdout: "cap_acquisition_throttle": 0, 2024-09-12T03:04:47.124 INFO:teuthology.orchestra.run.smithi148.stdout: "req_getvxattr_latency": { 2024-09-12T03:04:47.124 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:47.125 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:47.125 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:47.125 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-12T03:04:47.125 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:47.125 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_sessions": { 2024-09-12T03:04:47.125 INFO:teuthology.orchestra.run.smithi148.stdout: "session_count": 1, 2024-09-12T03:04:47.125 INFO:teuthology.orchestra.run.smithi148.stdout: "session_add": 3, 2024-09-12T03:04:47.125 INFO:teuthology.orchestra.run.smithi148.stdout: "session_remove": 2, 2024-09-12T03:04:47.125 INFO:teuthology.orchestra.run.smithi148.stdout: "sessions_open": 1, 2024-09-12T03:04:47.125 INFO:teuthology.orchestra.run.smithi148.stdout: "sessions_stale": 0, 2024-09-12T03:04:47.125 INFO:teuthology.orchestra.run.smithi148.stdout: "total_load": 9, 2024-09-12T03:04:47.125 INFO:teuthology.orchestra.run.smithi148.stdout: "average_load": 9, 2024-09-12T03:04:47.125 INFO:teuthology.orchestra.run.smithi148.stdout: "avg_session_uptime": 266, 2024-09-12T03:04:47.126 INFO:teuthology.orchestra.run.smithi148.stdout: "mdthresh_evicted": 0 2024-09-12T03:04:47.126 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:47.126 INFO:teuthology.orchestra.run.smithi148.stdout: "mempool": { 2024-09-12T03:04:47.126 INFO:teuthology.orchestra.run.smithi148.stdout: "bloom_filter_bytes": 0, 2024-09-12T03:04:47.126 INFO:teuthology.orchestra.run.smithi148.stdout: "bloom_filter_items": 0, 2024-09-12T03:04:47.126 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T03:04:47.126 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_alloc_items": 0, 2024-09-12T03:04:47.126 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T03:04:47.126 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_data_items": 0, 2024-09-12T03:04:47.126 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T03:04:47.126 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T03:04:47.126 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T03:04:47.126 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T03:04:47.127 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T03:04:47.127 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_other_items": 0, 2024-09-12T03:04:47.127 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T03:04:47.127 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T03:04:47.127 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_extent_bytes": 0, 2024-09-12T03:04:47.127 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_extent_items": 0, 2024-09-12T03:04:47.127 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_blob_bytes": 0, 2024-09-12T03:04:47.127 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_blob_items": 0, 2024-09-12T03:04:47.127 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T03:04:47.127 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T03:04:47.127 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T03:04:47.127 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T03:04:47.127 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T03:04:47.127 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_fsck_items": 0, 2024-09-12T03:04:47.127 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_txc_bytes": 0, 2024-09-12T03:04:47.128 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_txc_items": 0, 2024-09-12T03:04:47.128 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T03:04:47.128 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T03:04:47.128 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_bytes": 0, 2024-09-12T03:04:47.128 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_items": 0, 2024-09-12T03:04:47.128 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_bytes": 0, 2024-09-12T03:04:47.128 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_items": 0, 2024-09-12T03:04:47.128 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T03:04:47.128 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_reader_items": 0, 2024-09-12T03:04:47.128 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T03:04:47.128 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_writer_items": 0, 2024-09-12T03:04:47.128 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_anon_bytes": 271191, 2024-09-12T03:04:47.128 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_anon_items": 90, 2024-09-12T03:04:47.128 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_meta_bytes": 0, 2024-09-12T03:04:47.129 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_meta_items": 0, 2024-09-12T03:04:47.129 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_bytes": 0, 2024-09-12T03:04:47.129 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_items": 0, 2024-09-12T03:04:47.129 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_mapbl_bytes": 0, 2024-09-12T03:04:47.129 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_mapbl_items": 0, 2024-09-12T03:04:47.129 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_pglog_bytes": 0, 2024-09-12T03:04:47.129 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_pglog_items": 0, 2024-09-12T03:04:47.129 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_bytes": 7496, 2024-09-12T03:04:47.129 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_items": 154, 2024-09-12T03:04:47.129 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T03:04:47.129 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_mapping_items": 0, 2024-09-12T03:04:47.129 INFO:teuthology.orchestra.run.smithi148.stdout: "pgmap_bytes": 0, 2024-09-12T03:04:47.129 INFO:teuthology.orchestra.run.smithi148.stdout: "pgmap_items": 0, 2024-09-12T03:04:47.129 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_co_bytes": 76781, 2024-09-12T03:04:47.130 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_co_items": 315, 2024-09-12T03:04:47.130 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_1_bytes": 0, 2024-09-12T03:04:47.130 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_1_items": 0, 2024-09-12T03:04:47.130 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_2_bytes": 0, 2024-09-12T03:04:47.130 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_2_items": 0 2024-09-12T03:04:47.130 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:47.130 INFO:teuthology.orchestra.run.smithi148.stdout: "objecter": { 2024-09-12T03:04:47.130 INFO:teuthology.orchestra.run.smithi148.stdout: "op_active": 0, 2024-09-12T03:04:47.130 INFO:teuthology.orchestra.run.smithi148.stdout: "op_laggy": 0, 2024-09-12T03:04:47.130 INFO:teuthology.orchestra.run.smithi148.stdout: "op_send": 91, 2024-09-12T03:04:47.130 INFO:teuthology.orchestra.run.smithi148.stdout: "op_send_bytes": 46224, 2024-09-12T03:04:47.130 INFO:teuthology.orchestra.run.smithi148.stdout: "op_resend": 33, 2024-09-12T03:04:47.130 INFO:teuthology.orchestra.run.smithi148.stdout: "op_reply": 91, 2024-09-12T03:04:47.130 INFO:teuthology.orchestra.run.smithi148.stdout: "op_latency": { 2024-09-12T03:04:47.131 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 91, 2024-09-12T03:04:47.131 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.274993086, 2024-09-12T03:04:47.131 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.003021902 2024-09-12T03:04:47.131 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:47.131 INFO:teuthology.orchestra.run.smithi148.stdout: "op_inflight": 0, 2024-09-12T03:04:47.131 INFO:teuthology.orchestra.run.smithi148.stdout: "oplen_avg": { 2024-09-12T03:04:47.131 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 91, 2024-09-12T03:04:47.131 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 113 2024-09-12T03:04:47.131 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:47.131 INFO:teuthology.orchestra.run.smithi148.stdout: "op": 91, 2024-09-12T03:04:47.131 INFO:teuthology.orchestra.run.smithi148.stdout: "op_r": 26, 2024-09-12T03:04:47.131 INFO:teuthology.orchestra.run.smithi148.stdout: "op_w": 65, 2024-09-12T03:04:47.131 INFO:teuthology.orchestra.run.smithi148.stdout: "op_rmw": 0, 2024-09-12T03:04:47.131 INFO:teuthology.orchestra.run.smithi148.stdout: "op_pg": 0, 2024-09-12T03:04:47.132 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_stat": 0, 2024-09-12T03:04:47.132 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_create": 10, 2024-09-12T03:04:47.132 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_read": 1, 2024-09-12T03:04:47.132 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_write": 14, 2024-09-12T03:04:47.132 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_writefull": 11, 2024-09-12T03:04:47.132 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_writesame": 0, 2024-09-12T03:04:47.132 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_append": 0, 2024-09-12T03:04:47.132 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_zero": 0, 2024-09-12T03:04:47.132 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_truncate": 0, 2024-09-12T03:04:47.132 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_delete": 17, 2024-09-12T03:04:47.132 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_mapext": 0, 2024-09-12T03:04:47.132 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_sparse_read": 0, 2024-09-12T03:04:47.132 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_clonerange": 0, 2024-09-12T03:04:47.132 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_getxattr": 25, 2024-09-12T03:04:47.133 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_setxattr": 20, 2024-09-12T03:04:47.133 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_cmpxattr": 0, 2024-09-12T03:04:47.133 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_rmxattr": 0, 2024-09-12T03:04:47.133 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_resetxattrs": 0, 2024-09-12T03:04:47.133 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_call": 0, 2024-09-12T03:04:47.133 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_watch": 0, 2024-09-12T03:04:47.133 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_notify": 0, 2024-09-12T03:04:47.133 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_src_cmpxattr": 0, 2024-09-12T03:04:47.133 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_pgls": 0, 2024-09-12T03:04:47.133 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_pgls_filter": 0, 2024-09-12T03:04:47.133 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_other": 0, 2024-09-12T03:04:47.133 INFO:teuthology.orchestra.run.smithi148.stdout: "linger_active": 0, 2024-09-12T03:04:47.133 INFO:teuthology.orchestra.run.smithi148.stdout: "linger_send": 0, 2024-09-12T03:04:47.134 INFO:teuthology.orchestra.run.smithi148.stdout: "linger_resend": 0, 2024-09-12T03:04:47.134 INFO:teuthology.orchestra.run.smithi148.stdout: "linger_ping": 0, 2024-09-12T03:04:47.134 INFO:teuthology.orchestra.run.smithi148.stdout: "poolop_active": 0, 2024-09-12T03:04:47.136 INFO:teuthology.orchestra.run.smithi148.stdout: "poolop_send": 0, 2024-09-12T03:04:47.136 INFO:teuthology.orchestra.run.smithi148.stdout: "poolop_resend": 0, 2024-09-12T03:04:47.136 INFO:teuthology.orchestra.run.smithi148.stdout: "poolstat_active": 0, 2024-09-12T03:04:47.136 INFO:teuthology.orchestra.run.smithi148.stdout: "poolstat_send": 0, 2024-09-12T03:04:47.136 INFO:teuthology.orchestra.run.smithi148.stdout: "poolstat_resend": 0, 2024-09-12T03:04:47.136 INFO:teuthology.orchestra.run.smithi148.stdout: "statfs_active": 0, 2024-09-12T03:04:47.136 INFO:teuthology.orchestra.run.smithi148.stdout: "statfs_send": 0, 2024-09-12T03:04:47.136 INFO:teuthology.orchestra.run.smithi148.stdout: "statfs_resend": 0, 2024-09-12T03:04:47.136 INFO:teuthology.orchestra.run.smithi148.stdout: "command_active": 0, 2024-09-12T03:04:47.136 INFO:teuthology.orchestra.run.smithi148.stdout: "command_send": 0, 2024-09-12T03:04:47.136 INFO:teuthology.orchestra.run.smithi148.stdout: "command_resend": 0, 2024-09-12T03:04:47.137 INFO:teuthology.orchestra.run.smithi148.stdout: "map_epoch": 83, 2024-09-12T03:04:47.137 INFO:teuthology.orchestra.run.smithi148.stdout: "map_full": 0, 2024-09-12T03:04:47.137 INFO:teuthology.orchestra.run.smithi148.stdout: "map_inc": 2, 2024-09-12T03:04:47.137 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_sessions": 11, 2024-09-12T03:04:47.137 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_session_open": 11, 2024-09-12T03:04:47.137 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_session_close": 0, 2024-09-12T03:04:47.137 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_laggy": 0, 2024-09-12T03:04:47.137 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_wr": 14, 2024-09-12T03:04:47.137 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_rd": 0, 2024-09-12T03:04:47.137 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_del": 1 2024-09-12T03:04:47.137 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:47.137 INFO:teuthology.orchestra.run.smithi148.stdout: "oft": { 2024-09-12T03:04:47.137 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_total_objs": 1, 2024-09-12T03:04:47.137 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_total_kv_pairs": 0, 2024-09-12T03:04:47.137 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_total_updates": 0, 2024-09-12T03:04:47.138 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_total_removes": 0 2024-09-12T03:04:47.138 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:47.138 INFO:teuthology.orchestra.run.smithi148.stdout: "osd-slow-ops": { 2024-09-12T03:04:47.138 INFO:teuthology.orchestra.run.smithi148.stdout: "slow_ops_count": 49 2024-09-12T03:04:47.138 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:47.138 INFO:teuthology.orchestra.run.smithi148.stdout: "purge_queue": { 2024-09-12T03:04:47.138 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executing_ops": 0, 2024-09-12T03:04:47.138 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executing_ops_high_water": 5, 2024-09-12T03:04:47.138 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executing": 0, 2024-09-12T03:04:47.138 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executing_high_water": 5, 2024-09-12T03:04:47.138 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executed_ops": 5, 2024-09-12T03:04:47.138 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executed": 5, 2024-09-12T03:04:47.139 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_item_in_journal": 0 2024-09-12T03:04:47.139 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:47.139 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T03:04:47.139 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-09-12T03:04:47.139 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 104857600, 2024-09-12T03:04:47.139 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-09-12T03:04:47.139 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 1202, 2024-09-12T03:04:47.139 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 1018739, 2024-09-12T03:04:47.139 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-09-12T03:04:47.139 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 1202, 2024-09-12T03:04:47.139 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 0, 2024-09-12T03:04:47.139 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 0, 2024-09-12T03:04:47.139 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 1202, 2024-09-12T03:04:47.139 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 1018739, 2024-09-12T03:04:47.139 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-09-12T03:04:47.140 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:47.140 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:47.140 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:47.140 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-12T03:04:47.140 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:47.140 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-objecter_bytes": { 2024-09-12T03:04:47.140 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-09-12T03:04:47.140 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 104857600, 2024-09-12T03:04:47.140 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-09-12T03:04:47.140 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 0, 2024-09-12T03:04:47.140 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 0, 2024-09-12T03:04:47.140 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-09-12T03:04:47.140 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 0, 2024-09-12T03:04:47.141 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 91, 2024-09-12T03:04:47.141 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 46729, 2024-09-12T03:04:47.141 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 91, 2024-09-12T03:04:47.141 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 46729, 2024-09-12T03:04:47.141 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-09-12T03:04:47.141 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:47.141 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:47.141 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:47.141 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-12T03:04:47.141 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:47.141 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-objecter_ops": { 2024-09-12T03:04:47.141 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-09-12T03:04:47.141 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 1024, 2024-09-12T03:04:47.142 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-09-12T03:04:47.142 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 0, 2024-09-12T03:04:47.142 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 0, 2024-09-12T03:04:47.142 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-09-12T03:04:47.142 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 0, 2024-09-12T03:04:47.142 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 91, 2024-09-12T03:04:47.142 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 91, 2024-09-12T03:04:47.142 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 91, 2024-09-12T03:04:47.142 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 91, 2024-09-12T03:04:47.142 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-09-12T03:04:47.142 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:47.142 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:47.142 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:47.142 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-12T03:04:47.143 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:47.143 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-write_buf_throttle": { 2024-09-12T03:04:47.143 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-09-12T03:04:47.143 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 3758096384, 2024-09-12T03:04:47.143 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-09-12T03:04:47.143 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 5, 2024-09-12T03:04:47.143 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 505, 2024-09-12T03:04:47.143 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-09-12T03:04:47.143 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 5, 2024-09-12T03:04:47.143 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 0, 2024-09-12T03:04:47.143 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 0, 2024-09-12T03:04:47.143 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 1, 2024-09-12T03:04:47.143 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 505, 2024-09-12T03:04:47.143 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-09-12T03:04:47.144 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:47.144 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:47.144 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:47.144 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-12T03:04:47.144 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:47.144 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-write_buf_throttle-0x55587e04e960": { 2024-09-12T03:04:47.144 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-09-12T03:04:47.144 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 3758096384, 2024-09-12T03:04:47.144 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-09-12T03:04:47.144 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 24, 2024-09-12T03:04:47.144 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 34907, 2024-09-12T03:04:47.144 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-09-12T03:04:47.144 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 24, 2024-09-12T03:04:47.144 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 0, 2024-09-12T03:04:47.144 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 0, 2024-09-12T03:04:47.144 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 13, 2024-09-12T03:04:47.145 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 34907, 2024-09-12T03:04:47.145 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-09-12T03:04:47.145 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:47.145 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:47.145 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:47.145 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-12T03:04:47.145 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-12T03:04:47.145 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-12T03:04:47.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:04:46 smithi005 ceph-mon[31197]: pgmap v531: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-12T03:04:47.415 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:04:46 smithi148 ceph-mon[35090]: pgmap v531: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-12T03:04:47.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:04:46 smithi146 ceph-mon[35011]: pgmap v531: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-09-12T03:04:47.989 INFO:tasks.check_counter:Daemon mds.f mds.exported=1 2024-09-12T03:04:47.989 INFO:tasks.check_counter:Daemon mds.f mds.imported=1 2024-09-12T03:04:47.990 INFO:tasks.check_counter:Daemon mds.f mds.dir_split=0 2024-09-12T03:04:47.990 DEBUG:tasks.check_counter:Getting stats from i 2024-09-12T03:04:47.990 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph daemon mds.i perf dump 2024-09-12T03:04:48.239 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T03:04:48.734 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-12T03:04:48.735 INFO:teuthology.orchestra.run.smithi148.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T03:04:48.735 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_messages": 79302, 2024-09-12T03:04:48.735 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages": 138562, 2024-09-12T03:04:48.735 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_bytes": 28059441, 2024-09-12T03:04:48.735 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_bytes": 87409284, 2024-09-12T03:04:48.735 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_created_connections": 30, 2024-09-12T03:04:48.735 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_active_connections": 3, 2024-09-12T03:04:48.735 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_total_time": 5.449373245, 2024-09-12T03:04:48.735 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_send_time": 2.508050997, 2024-09-12T03:04:48.735 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_recv_time": 3.604949463, 2024-09-12T03:04:48.735 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_fast_dispatch_time": 0.517885155, 2024-09-12T03:04:48.735 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:04:48.736 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 138561, 2024-09-12T03:04:48.736 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 3.393624235, 2024-09-12T03:04:48.736 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000024491 2024-09-12T03:04:48.736 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:48.736 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:04:48.736 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 133516, 2024-09-12T03:04:48.736 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.025471739, 2024-09-12T03:04:48.736 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000190 2024-09-12T03:04:48.736 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:48.736 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_encrypted_bytes": 21394496, 2024-09-12T03:04:48.736 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_encrypted_bytes": 52849840 2024-09-12T03:04:48.736 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:48.736 INFO:teuthology.orchestra.run.smithi148.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T03:04:48.737 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_messages": 49371, 2024-09-12T03:04:48.737 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages": 49596, 2024-09-12T03:04:48.737 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_bytes": 13246981, 2024-09-12T03:04:48.737 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_bytes": 63941346, 2024-09-12T03:04:48.737 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_created_connections": 14, 2024-09-12T03:04:48.737 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_active_connections": 6, 2024-09-12T03:04:48.737 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_total_time": 2.700115078, 2024-09-12T03:04:48.737 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_send_time": 0.957360255, 2024-09-12T03:04:48.737 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_recv_time": 0.676461768, 2024-09-12T03:04:48.737 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_fast_dispatch_time": 0.890147458, 2024-09-12T03:04:48.737 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:04:48.737 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 49595, 2024-09-12T03:04:48.737 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 2.224624327, 2024-09-12T03:04:48.738 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000044855 2024-09-12T03:04:48.738 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:48.738 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:04:48.738 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 1080, 2024-09-12T03:04:48.738 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000254959, 2024-09-12T03:04:48.738 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000236 2024-09-12T03:04:48.738 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:48.738 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_encrypted_bytes": 3184, 2024-09-12T03:04:48.738 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_encrypted_bytes": 276448 2024-09-12T03:04:48.738 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:48.738 INFO:teuthology.orchestra.run.smithi148.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T03:04:48.738 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_messages": 24855, 2024-09-12T03:04:48.738 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages": 25497, 2024-09-12T03:04:48.738 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_bytes": 6430680, 2024-09-12T03:04:48.739 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_bytes": 38226529, 2024-09-12T03:04:48.739 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_created_connections": 79, 2024-09-12T03:04:48.739 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_active_connections": 10, 2024-09-12T03:04:48.739 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_total_time": 1.479082196, 2024-09-12T03:04:48.739 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_send_time": 0.492054503, 2024-09-12T03:04:48.739 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_recv_time": 0.422469550, 2024-09-12T03:04:48.739 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_fast_dispatch_time": 0.473116334, 2024-09-12T03:04:48.739 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:04:48.739 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 25495, 2024-09-12T03:04:48.739 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 1.227182276, 2024-09-12T03:04:48.739 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000048134 2024-09-12T03:04:48.739 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:48.739 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:04:48.740 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 3837, 2024-09-12T03:04:48.740 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000788833, 2024-09-12T03:04:48.740 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000205 2024-09-12T03:04:48.740 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:48.740 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_encrypted_bytes": 41104, 2024-09-12T03:04:48.740 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_encrypted_bytes": 1856 2024-09-12T03:04:48.740 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:48.740 INFO:teuthology.orchestra.run.smithi148.stdout: "cct": { 2024-09-12T03:04:48.740 INFO:teuthology.orchestra.run.smithi148.stdout: "total_workers": 1, 2024-09-12T03:04:48.740 INFO:teuthology.orchestra.run.smithi148.stdout: "unhealthy_workers": 0 2024-09-12T03:04:48.740 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:48.740 INFO:teuthology.orchestra.run.smithi148.stdout: "finisher-MDSRank": { 2024-09-12T03:04:48.740 INFO:teuthology.orchestra.run.smithi148.stdout: "queue_len": 0, 2024-09-12T03:04:48.741 INFO:teuthology.orchestra.run.smithi148.stdout: "complete_latency": { 2024-09-12T03:04:48.741 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 103775, 2024-09-12T03:04:48.741 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 38.983040077, 2024-09-12T03:04:48.741 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000375649 2024-09-12T03:04:48.741 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-12T03:04:48.741 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:48.741 INFO:teuthology.orchestra.run.smithi148.stdout: "finisher-PurgeQueue": { 2024-09-12T03:04:48.741 INFO:teuthology.orchestra.run.smithi148.stdout: "queue_len": 0, 2024-09-12T03:04:48.741 INFO:teuthology.orchestra.run.smithi148.stdout: "complete_latency": { 2024-09-12T03:04:48.741 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 13036, 2024-09-12T03:04:48.741 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 2.940463996, 2024-09-12T03:04:48.741 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000225564 2024-09-12T03:04:48.742 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-12T03:04:48.742 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:48.742 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": { 2024-09-12T03:04:48.742 INFO:teuthology.orchestra.run.smithi148.stdout: "request": 24744, 2024-09-12T03:04:48.742 INFO:teuthology.orchestra.run.smithi148.stdout: "reply": 24741, 2024-09-12T03:04:48.742 INFO:teuthology.orchestra.run.smithi148.stdout: "reply_latency": { 2024-09-12T03:04:48.742 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 24741, 2024-09-12T03:04:48.742 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 83.359101529, 2024-09-12T03:04:48.742 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.003369269 2024-09-12T03:04:48.742 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:48.742 INFO:teuthology.orchestra.run.smithi148.stdout: "slow_reply": 0, 2024-09-12T03:04:48.742 INFO:teuthology.orchestra.run.smithi148.stdout: "forward": 3, 2024-09-12T03:04:48.742 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_fetch_complete": 0, 2024-09-12T03:04:48.742 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_fetch_keys": 0, 2024-09-12T03:04:48.743 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_commit": 221, 2024-09-12T03:04:48.743 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_split": 25, 2024-09-12T03:04:48.743 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_merge": 3, 2024-09-12T03:04:48.743 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes": 712, 2024-09-12T03:04:48.743 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_top": 477, 2024-09-12T03:04:48.743 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_bottom": 213, 2024-09-12T03:04:48.743 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_pin_tail": 22, 2024-09-12T03:04:48.743 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_pinned": 30, 2024-09-12T03:04:48.743 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_expired": 23, 2024-09-12T03:04:48.743 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_with_caps": 3, 2024-09-12T03:04:48.743 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": 3, 2024-09-12T03:04:48.743 INFO:teuthology.orchestra.run.smithi148.stdout: "subtrees": 2, 2024-09-12T03:04:48.743 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse": 25853, 2024-09-12T03:04:48.743 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_hit": 25239, 2024-09-12T03:04:48.744 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_forward": 1, 2024-09-12T03:04:48.744 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_discover": 0, 2024-09-12T03:04:48.744 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_dir_fetch": 0, 2024-09-12T03:04:48.744 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_remote_ino": 0, 2024-09-12T03:04:48.744 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_lock": 1, 2024-09-12T03:04:48.744 INFO:teuthology.orchestra.run.smithi148.stdout: "load_cent": 0, 2024-09-12T03:04:48.744 INFO:teuthology.orchestra.run.smithi148.stdout: "q": 0, 2024-09-12T03:04:48.744 INFO:teuthology.orchestra.run.smithi148.stdout: "exported": 4, 2024-09-12T03:04:48.744 INFO:teuthology.orchestra.run.smithi148.stdout: "exported_inodes": 17, 2024-09-12T03:04:48.744 INFO:teuthology.orchestra.run.smithi148.stdout: "imported": 4, 2024-09-12T03:04:48.744 INFO:teuthology.orchestra.run.smithi148.stdout: "imported_inodes": 0, 2024-09-12T03:04:48.744 INFO:teuthology.orchestra.run.smithi148.stdout: "openino_dir_fetch": 0, 2024-09-12T03:04:48.744 INFO:teuthology.orchestra.run.smithi148.stdout: "openino_backtrace_fetch": 27277, 2024-09-12T03:04:48.745 INFO:teuthology.orchestra.run.smithi148.stdout: "openino_peer_discover": 0, 2024-09-12T03:04:48.745 INFO:teuthology.orchestra.run.smithi148.stdout: "root_rfiles": 4, 2024-09-12T03:04:48.745 INFO:teuthology.orchestra.run.smithi148.stdout: "root_rbytes": 234, 2024-09-12T03:04:48.745 INFO:teuthology.orchestra.run.smithi148.stdout: "root_rsnaps": 0, 2024-09-12T03:04:48.745 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_backtrace_fetch": 27277, 2024-09-12T03:04:48.745 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_set_tag": 0, 2024-09-12T03:04:48.745 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_backtrace_repaired": 0, 2024-09-12T03:04:48.745 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_inotable_repaired": 0, 2024-09-12T03:04:48.745 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_dir_inodes": 408, 2024-09-12T03:04:48.745 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_dir_base_inodes": 8, 2024-09-12T03:04:48.746 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_dirfrag_rstats": 400, 2024-09-12T03:04:48.746 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_file_inodes": 26869, 2024-09-12T03:04:48.746 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_inode_file_caps": 12, 2024-09-12T03:04:48.746 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_revoke": 17463, 2024-09-12T03:04:48.746 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_grant": 23840, 2024-09-12T03:04:48.746 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_trunc": 7863, 2024-09-12T03:04:48.746 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-12T03:04:48.746 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-12T03:04:48.746 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_caps": 27360, 2024-09-12T03:04:48.746 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_caps_dirty": 14457, 2024-09-12T03:04:48.746 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_cap_release": 581, 2024-09-12T03:04:48.746 INFO:teuthology.orchestra.run.smithi148.stdout: "process_request_cap_release": 16950 2024-09-12T03:04:48.746 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:48.746 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_cache": { 2024-09-12T03:04:48.747 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_update": 1, 2024-09-12T03:04:48.747 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_update_receipt": 0, 2024-09-12T03:04:48.747 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_try_discover": 0, 2024-09-12T03:04:48.747 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_send_discover": 0, 2024-09-12T03:04:48.747 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_handle_discover": 13, 2024-09-12T03:04:48.747 INFO:teuthology.orchestra.run.smithi148.stdout: "num_strays": 4, 2024-09-12T03:04:48.747 INFO:teuthology.orchestra.run.smithi148.stdout: "num_strays_delayed": 0, 2024-09-12T03:04:48.747 INFO:teuthology.orchestra.run.smithi148.stdout: "num_strays_enqueuing": 0, 2024-09-12T03:04:48.747 INFO:teuthology.orchestra.run.smithi148.stdout: "strays_created": 7266, 2024-09-12T03:04:48.747 INFO:teuthology.orchestra.run.smithi148.stdout: "strays_enqueued": 7259, 2024-09-12T03:04:48.747 INFO:teuthology.orchestra.run.smithi148.stdout: "strays_reintegrated": 3, 2024-09-12T03:04:48.747 INFO:teuthology.orchestra.run.smithi148.stdout: "strays_migrated": 0, 2024-09-12T03:04:48.747 INFO:teuthology.orchestra.run.smithi148.stdout: "num_recovering_processing": 0, 2024-09-12T03:04:48.747 INFO:teuthology.orchestra.run.smithi148.stdout: "num_recovering_enqueued": 0, 2024-09-12T03:04:48.748 INFO:teuthology.orchestra.run.smithi148.stdout: "num_recovering_prioritized": 0, 2024-09-12T03:04:48.748 INFO:teuthology.orchestra.run.smithi148.stdout: "recovery_started": 0, 2024-09-12T03:04:48.748 INFO:teuthology.orchestra.run.smithi148.stdout: "recovery_completed": 0, 2024-09-12T03:04:48.748 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_quiesce_path": 5, 2024-09-12T03:04:48.748 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_quiesce_inode": 18275, 2024-09-12T03:04:48.748 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_enqueue_scrub": 8, 2024-09-12T03:04:48.748 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_exportdir": 4, 2024-09-12T03:04:48.748 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_flush": 0, 2024-09-12T03:04:48.748 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_fragmentdir": 28, 2024-09-12T03:04:48.748 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_fragstats": 0, 2024-09-12T03:04:48.749 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_inodestats": 0 2024-09-12T03:04:48.749 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:48.749 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_log": { 2024-09-12T03:04:48.749 INFO:teuthology.orchestra.run.smithi148.stdout: "evlrg": 0, 2024-09-12T03:04:48.749 INFO:teuthology.orchestra.run.smithi148.stdout: "evadd": 56323, 2024-09-12T03:04:48.749 INFO:teuthology.orchestra.run.smithi148.stdout: "evex": 0, 2024-09-12T03:04:48.749 INFO:teuthology.orchestra.run.smithi148.stdout: "evtrm": 0, 2024-09-12T03:04:48.749 INFO:teuthology.orchestra.run.smithi148.stdout: "ev": 56323, 2024-09-12T03:04:48.749 INFO:teuthology.orchestra.run.smithi148.stdout: "evexg": 0, 2024-09-12T03:04:48.749 INFO:teuthology.orchestra.run.smithi148.stdout: "evexd": 0, 2024-09-12T03:04:48.749 INFO:teuthology.orchestra.run.smithi148.stdout: "segadd": 68, 2024-09-12T03:04:48.749 INFO:teuthology.orchestra.run.smithi148.stdout: "segex": 0, 2024-09-12T03:04:48.749 INFO:teuthology.orchestra.run.smithi148.stdout: "segtrm": 0, 2024-09-12T03:04:48.750 INFO:teuthology.orchestra.run.smithi148.stdout: "seg": 68, 2024-09-12T03:04:48.750 INFO:teuthology.orchestra.run.smithi148.stdout: "segmjr": 6, 2024-09-12T03:04:48.750 INFO:teuthology.orchestra.run.smithi148.stdout: "segexg": 0, 2024-09-12T03:04:48.750 INFO:teuthology.orchestra.run.smithi148.stdout: "segexd": 0, 2024-09-12T03:04:48.750 INFO:teuthology.orchestra.run.smithi148.stdout: "expos": 4194304, 2024-09-12T03:04:48.750 INFO:teuthology.orchestra.run.smithi148.stdout: "wrpos": 121226016, 2024-09-12T03:04:48.750 INFO:teuthology.orchestra.run.smithi148.stdout: "rdpos": 0, 2024-09-12T03:04:48.750 INFO:teuthology.orchestra.run.smithi148.stdout: "jlat": { 2024-09-12T03:04:48.750 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 24933, 2024-09-12T03:04:48.750 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 93.062343931, 2024-09-12T03:04:48.750 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.003732496 2024-09-12T03:04:48.750 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:48.750 INFO:teuthology.orchestra.run.smithi148.stdout: "replayed": 0 2024-09-12T03:04:48.750 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:48.751 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_mem": { 2024-09-12T03:04:48.751 INFO:teuthology.orchestra.run.smithi148.stdout: "ino": 27, 2024-09-12T03:04:48.751 INFO:teuthology.orchestra.run.smithi148.stdout: "ino+": 7305, 2024-09-12T03:04:48.751 INFO:teuthology.orchestra.run.smithi148.stdout: "ino-": 7278, 2024-09-12T03:04:48.751 INFO:teuthology.orchestra.run.smithi148.stdout: "dir": 22, 2024-09-12T03:04:48.751 INFO:teuthology.orchestra.run.smithi148.stdout: "dir+": 288, 2024-09-12T03:04:48.751 INFO:teuthology.orchestra.run.smithi148.stdout: "dir-": 266, 2024-09-12T03:04:48.751 INFO:teuthology.orchestra.run.smithi148.stdout: "dn": 712, 2024-09-12T03:04:48.751 INFO:teuthology.orchestra.run.smithi148.stdout: "dn+": 12511, 2024-09-12T03:04:48.751 INFO:teuthology.orchestra.run.smithi148.stdout: "dn-": 11799, 2024-09-12T03:04:48.751 INFO:teuthology.orchestra.run.smithi148.stdout: "cap": 3, 2024-09-12T03:04:48.751 INFO:teuthology.orchestra.run.smithi148.stdout: "cap+": 16122, 2024-09-12T03:04:48.751 INFO:teuthology.orchestra.run.smithi148.stdout: "cap-": 16119, 2024-09-12T03:04:48.751 INFO:teuthology.orchestra.run.smithi148.stdout: "rss": 113984, 2024-09-12T03:04:48.752 INFO:teuthology.orchestra.run.smithi148.stdout: "heap": 231712 2024-09-12T03:04:48.752 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:48.752 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_server": { 2024-09-12T03:04:48.752 INFO:teuthology.orchestra.run.smithi148.stdout: "dispatch_client_request": 42943, 2024-09-12T03:04:48.752 INFO:teuthology.orchestra.run.smithi148.stdout: "dispatch_server_request": 19, 2024-09-12T03:04:48.752 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_request": 24744, 2024-09-12T03:04:48.752 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_session": 21, 2024-09-12T03:04:48.752 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_peer_request": 46, 2024-09-12T03:04:48.752 INFO:teuthology.orchestra.run.smithi148.stdout: "req_create_latency": { 2024-09-12T03:04:48.752 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 7205, 2024-09-12T03:04:48.752 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 24.234862120, 2024-09-12T03:04:48.752 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.003363617 2024-09-12T03:04:48.752 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:48.752 INFO:teuthology.orchestra.run.smithi148.stdout: "req_getattr_latency": { 2024-09-12T03:04:48.753 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 22, 2024-09-12T03:04:48.753 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.903936431, 2024-09-12T03:04:48.753 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.041088019 2024-09-12T03:04:48.753 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:48.753 INFO:teuthology.orchestra.run.smithi148.stdout: "req_getfilelock_latency": { 2024-09-12T03:04:48.753 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:48.753 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:48.753 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:48.753 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:48.753 INFO:teuthology.orchestra.run.smithi148.stdout: "req_link_latency": { 2024-09-12T03:04:48.753 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 3, 2024-09-12T03:04:48.753 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.003707521, 2024-09-12T03:04:48.753 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.001235840 2024-09-12T03:04:48.754 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:48.754 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookup_latency": { 2024-09-12T03:04:48.754 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 209, 2024-09-12T03:04:48.754 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.752875848, 2024-09-12T03:04:48.754 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.003602276 2024-09-12T03:04:48.754 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:48.754 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookuphash_latency": { 2024-09-12T03:04:48.754 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:48.754 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:48.754 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:48.754 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:48.754 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookupino_latency": { 2024-09-12T03:04:48.754 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 2, 2024-09-12T03:04:48.754 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000319105, 2024-09-12T03:04:48.755 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000159552 2024-09-12T03:04:48.755 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:48.755 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookupname_latency": { 2024-09-12T03:04:48.755 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:48.755 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:48.755 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:48.755 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:48.755 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookupparent_latency": { 2024-09-12T03:04:48.755 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:48.755 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:48.755 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:48.756 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:48.756 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookupsnap_latency": { 2024-09-12T03:04:48.756 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:48.756 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:48.756 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:48.756 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:48.757 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lssnap_latency": { 2024-09-12T03:04:48.757 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:48.757 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:48.757 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:48.757 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:48.757 INFO:teuthology.orchestra.run.smithi148.stdout: "req_mkdir_latency": { 2024-09-12T03:04:48.757 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 73, 2024-09-12T03:04:48.757 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.062824906, 2024-09-12T03:04:48.757 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000860615 2024-09-12T03:04:48.757 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:48.758 INFO:teuthology.orchestra.run.smithi148.stdout: "req_mknod_latency": { 2024-09-12T03:04:48.758 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:48.758 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:48.758 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:48.758 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:48.758 INFO:teuthology.orchestra.run.smithi148.stdout: "req_mksnap_latency": { 2024-09-12T03:04:48.758 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:48.758 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:48.758 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:48.758 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:48.758 INFO:teuthology.orchestra.run.smithi148.stdout: "req_open_latency": { 2024-09-12T03:04:48.758 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 430, 2024-09-12T03:04:48.758 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.089221344, 2024-09-12T03:04:48.758 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000207491 2024-09-12T03:04:48.759 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:48.759 INFO:teuthology.orchestra.run.smithi148.stdout: "req_readdir_latency": { 2024-09-12T03:04:48.759 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 475, 2024-09-12T03:04:48.759 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 1.309124660, 2024-09-12T03:04:48.759 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.002756051 2024-09-12T03:04:48.759 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:48.759 INFO:teuthology.orchestra.run.smithi148.stdout: "req_rename_latency": { 2024-09-12T03:04:48.759 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 94, 2024-09-12T03:04:48.759 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.176205461, 2024-09-12T03:04:48.759 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.001874526 2024-09-12T03:04:48.759 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:48.759 INFO:teuthology.orchestra.run.smithi148.stdout: "req_renamesnap_latency": { 2024-09-12T03:04:48.759 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:48.759 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:48.760 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:48.760 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:48.760 INFO:teuthology.orchestra.run.smithi148.stdout: "req_snapdiff_latency": { 2024-09-12T03:04:48.760 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:48.760 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:48.760 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:48.760 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:48.760 INFO:teuthology.orchestra.run.smithi148.stdout: "req_rmdir_latency": { 2024-09-12T03:04:48.761 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 72, 2024-09-12T03:04:48.761 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 2.499463235, 2024-09-12T03:04:48.761 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.034714767 2024-09-12T03:04:48.761 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:48.761 INFO:teuthology.orchestra.run.smithi148.stdout: "req_rmsnap_latency": { 2024-09-12T03:04:48.762 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:48.762 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:48.762 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:48.762 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:48.762 INFO:teuthology.orchestra.run.smithi148.stdout: "req_rmxattr_latency": { 2024-09-12T03:04:48.762 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:48.762 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:48.762 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:48.762 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:48.763 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setattr_latency": { 2024-09-12T03:04:48.763 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 8995, 2024-09-12T03:04:48.763 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 28.335486836, 2024-09-12T03:04:48.763 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.003150137 2024-09-12T03:04:48.763 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:48.763 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setdirlayout_latency": { 2024-09-12T03:04:48.763 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:48.763 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:48.763 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:48.763 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:48.763 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setfilelock_latency": { 2024-09-12T03:04:48.763 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 8, 2024-09-12T03:04:48.763 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.002768934, 2024-09-12T03:04:48.764 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000346116 2024-09-12T03:04:48.764 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:48.764 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setlayout_latency": { 2024-09-12T03:04:48.764 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:48.764 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:48.764 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:48.764 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:48.764 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setxattr_latency": { 2024-09-12T03:04:48.764 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 8, 2024-09-12T03:04:48.764 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.020411245, 2024-09-12T03:04:48.764 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.002551405 2024-09-12T03:04:48.764 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:48.764 INFO:teuthology.orchestra.run.smithi148.stdout: "req_symlink_latency": { 2024-09-12T03:04:48.765 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 6, 2024-09-12T03:04:48.765 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.003495532, 2024-09-12T03:04:48.765 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000582588 2024-09-12T03:04:48.765 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:48.765 INFO:teuthology.orchestra.run.smithi148.stdout: "req_unlink_latency": { 2024-09-12T03:04:48.765 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 7139, 2024-09-12T03:04:48.765 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 24.964398351, 2024-09-12T03:04:48.765 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.003496904 2024-09-12T03:04:48.765 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:48.765 INFO:teuthology.orchestra.run.smithi148.stdout: "cap_revoke_eviction": 0, 2024-09-12T03:04:48.765 INFO:teuthology.orchestra.run.smithi148.stdout: "cap_acquisition_throttle": 0, 2024-09-12T03:04:48.765 INFO:teuthology.orchestra.run.smithi148.stdout: "req_getvxattr_latency": { 2024-09-12T03:04:48.765 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:48.766 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:48.766 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:48.766 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-12T03:04:48.766 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:48.766 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_sessions": { 2024-09-12T03:04:48.766 INFO:teuthology.orchestra.run.smithi148.stdout: "session_count": 1, 2024-09-12T03:04:48.766 INFO:teuthology.orchestra.run.smithi148.stdout: "session_add": 3, 2024-09-12T03:04:48.766 INFO:teuthology.orchestra.run.smithi148.stdout: "session_remove": 2, 2024-09-12T03:04:48.766 INFO:teuthology.orchestra.run.smithi148.stdout: "sessions_open": 1, 2024-09-12T03:04:48.766 INFO:teuthology.orchestra.run.smithi148.stdout: "sessions_stale": 0, 2024-09-12T03:04:48.766 INFO:teuthology.orchestra.run.smithi148.stdout: "total_load": 8427, 2024-09-12T03:04:48.766 INFO:teuthology.orchestra.run.smithi148.stdout: "average_load": 8427, 2024-09-12T03:04:48.766 INFO:teuthology.orchestra.run.smithi148.stdout: "avg_session_uptime": 405, 2024-09-12T03:04:48.766 INFO:teuthology.orchestra.run.smithi148.stdout: "mdthresh_evicted": 0 2024-09-12T03:04:48.767 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:48.767 INFO:teuthology.orchestra.run.smithi148.stdout: "mempool": { 2024-09-12T03:04:48.767 INFO:teuthology.orchestra.run.smithi148.stdout: "bloom_filter_bytes": 0, 2024-09-12T03:04:48.767 INFO:teuthology.orchestra.run.smithi148.stdout: "bloom_filter_items": 0, 2024-09-12T03:04:48.767 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T03:04:48.767 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_alloc_items": 0, 2024-09-12T03:04:48.767 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T03:04:48.767 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_data_items": 0, 2024-09-12T03:04:48.767 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T03:04:48.767 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T03:04:48.767 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T03:04:48.767 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T03:04:48.767 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T03:04:48.767 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_other_items": 0, 2024-09-12T03:04:48.768 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T03:04:48.768 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T03:04:48.768 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_extent_bytes": 0, 2024-09-12T03:04:48.768 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_extent_items": 0, 2024-09-12T03:04:48.768 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_blob_bytes": 0, 2024-09-12T03:04:48.768 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_blob_items": 0, 2024-09-12T03:04:48.768 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T03:04:48.768 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T03:04:48.768 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T03:04:48.768 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T03:04:48.768 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T03:04:48.768 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_fsck_items": 0, 2024-09-12T03:04:48.768 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_txc_bytes": 0, 2024-09-12T03:04:48.769 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_txc_items": 0, 2024-09-12T03:04:48.769 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T03:04:48.769 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T03:04:48.769 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_bytes": 0, 2024-09-12T03:04:48.769 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_items": 0, 2024-09-12T03:04:48.769 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_bytes": 0, 2024-09-12T03:04:48.769 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_items": 0, 2024-09-12T03:04:48.769 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T03:04:48.769 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_reader_items": 0, 2024-09-12T03:04:48.769 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T03:04:48.769 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_writer_items": 0, 2024-09-12T03:04:48.769 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_anon_bytes": 319243, 2024-09-12T03:04:48.769 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_anon_items": 118, 2024-09-12T03:04:48.769 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_meta_bytes": 0, 2024-09-12T03:04:48.770 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_meta_items": 0, 2024-09-12T03:04:48.770 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_bytes": 0, 2024-09-12T03:04:48.770 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_items": 0, 2024-09-12T03:04:48.770 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_mapbl_bytes": 0, 2024-09-12T03:04:48.770 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_mapbl_items": 0, 2024-09-12T03:04:48.770 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_pglog_bytes": 0, 2024-09-12T03:04:48.770 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_pglog_items": 0, 2024-09-12T03:04:48.770 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_bytes": 7496, 2024-09-12T03:04:48.770 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_items": 154, 2024-09-12T03:04:48.770 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T03:04:48.770 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_mapping_items": 0, 2024-09-12T03:04:48.770 INFO:teuthology.orchestra.run.smithi148.stdout: "pgmap_bytes": 0, 2024-09-12T03:04:48.770 INFO:teuthology.orchestra.run.smithi148.stdout: "pgmap_items": 0, 2024-09-12T03:04:48.770 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_co_bytes": 636246, 2024-09-12T03:04:48.771 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_co_items": 8831, 2024-09-12T03:04:48.771 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_1_bytes": 0, 2024-09-12T03:04:48.771 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_1_items": 0, 2024-09-12T03:04:48.771 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_2_bytes": 0, 2024-09-12T03:04:48.771 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_2_items": 0 2024-09-12T03:04:48.771 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:48.771 INFO:teuthology.orchestra.run.smithi148.stdout: "objecter": { 2024-09-12T03:04:48.771 INFO:teuthology.orchestra.run.smithi148.stdout: "op_active": 0, 2024-09-12T03:04:48.771 INFO:teuthology.orchestra.run.smithi148.stdout: "op_laggy": 0, 2024-09-12T03:04:48.771 INFO:teuthology.orchestra.run.smithi148.stdout: "op_send": 97541, 2024-09-12T03:04:48.771 INFO:teuthology.orchestra.run.smithi148.stdout: "op_send_bytes": 121540247, 2024-09-12T03:04:48.771 INFO:teuthology.orchestra.run.smithi148.stdout: "op_resend": 34, 2024-09-12T03:04:48.771 INFO:teuthology.orchestra.run.smithi148.stdout: "op_reply": 97541, 2024-09-12T03:04:48.771 INFO:teuthology.orchestra.run.smithi148.stdout: "op_latency": { 2024-09-12T03:04:48.772 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 97541, 2024-09-12T03:04:48.772 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 211.999693225, 2024-09-12T03:04:48.772 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.002173441 2024-09-12T03:04:48.772 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:48.772 INFO:teuthology.orchestra.run.smithi148.stdout: "op_inflight": 0, 2024-09-12T03:04:48.772 INFO:teuthology.orchestra.run.smithi148.stdout: "oplen_avg": { 2024-09-12T03:04:48.772 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 97541, 2024-09-12T03:04:48.772 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 97903 2024-09-12T03:04:48.772 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:48.772 INFO:teuthology.orchestra.run.smithi148.stdout: "op": 97541, 2024-09-12T03:04:48.772 INFO:teuthology.orchestra.run.smithi148.stdout: "op_r": 27361, 2024-09-12T03:04:48.772 INFO:teuthology.orchestra.run.smithi148.stdout: "op_w": 70180, 2024-09-12T03:04:48.772 INFO:teuthology.orchestra.run.smithi148.stdout: "op_rmw": 0, 2024-09-12T03:04:48.773 INFO:teuthology.orchestra.run.smithi148.stdout: "op_pg": 0, 2024-09-12T03:04:48.773 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_stat": 0, 2024-09-12T03:04:48.773 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_create": 11, 2024-09-12T03:04:48.773 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_read": 76, 2024-09-12T03:04:48.773 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_write": 25001, 2024-09-12T03:04:48.773 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_writefull": 65, 2024-09-12T03:04:48.773 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_writesame": 0, 2024-09-12T03:04:48.773 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_append": 0, 2024-09-12T03:04:48.773 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_zero": 0, 2024-09-12T03:04:48.773 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_truncate": 9, 2024-09-12T03:04:48.773 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_delete": 8028, 2024-09-12T03:04:48.773 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_mapext": 0, 2024-09-12T03:04:48.773 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_sparse_read": 0, 2024-09-12T03:04:48.773 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_clonerange": 0, 2024-09-12T03:04:48.774 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_getxattr": 27285, 2024-09-12T03:04:48.774 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_setxattr": 22, 2024-09-12T03:04:48.774 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_cmpxattr": 0, 2024-09-12T03:04:48.774 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_rmxattr": 0, 2024-09-12T03:04:48.774 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_resetxattrs": 0, 2024-09-12T03:04:48.774 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_call": 0, 2024-09-12T03:04:48.774 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_watch": 0, 2024-09-12T03:04:48.774 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_notify": 0, 2024-09-12T03:04:48.774 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_src_cmpxattr": 0, 2024-09-12T03:04:48.774 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_pgls": 0, 2024-09-12T03:04:48.774 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_pgls_filter": 0, 2024-09-12T03:04:48.774 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_other": 36717, 2024-09-12T03:04:48.774 INFO:teuthology.orchestra.run.smithi148.stdout: "linger_active": 0, 2024-09-12T03:04:48.774 INFO:teuthology.orchestra.run.smithi148.stdout: "linger_send": 0, 2024-09-12T03:04:48.775 INFO:teuthology.orchestra.run.smithi148.stdout: "linger_resend": 0, 2024-09-12T03:04:48.775 INFO:teuthology.orchestra.run.smithi148.stdout: "linger_ping": 0, 2024-09-12T03:04:48.775 INFO:teuthology.orchestra.run.smithi148.stdout: "poolop_active": 0, 2024-09-12T03:04:48.775 INFO:teuthology.orchestra.run.smithi148.stdout: "poolop_send": 0, 2024-09-12T03:04:48.775 INFO:teuthology.orchestra.run.smithi148.stdout: "poolop_resend": 0, 2024-09-12T03:04:48.775 INFO:teuthology.orchestra.run.smithi148.stdout: "poolstat_active": 0, 2024-09-12T03:04:48.775 INFO:teuthology.orchestra.run.smithi148.stdout: "poolstat_send": 0, 2024-09-12T03:04:48.775 INFO:teuthology.orchestra.run.smithi148.stdout: "poolstat_resend": 0, 2024-09-12T03:04:48.775 INFO:teuthology.orchestra.run.smithi148.stdout: "statfs_active": 0, 2024-09-12T03:04:48.775 INFO:teuthology.orchestra.run.smithi148.stdout: "statfs_send": 0, 2024-09-12T03:04:48.775 INFO:teuthology.orchestra.run.smithi148.stdout: "statfs_resend": 0, 2024-09-12T03:04:48.775 INFO:teuthology.orchestra.run.smithi148.stdout: "command_active": 0, 2024-09-12T03:04:48.775 INFO:teuthology.orchestra.run.smithi148.stdout: "command_send": 0, 2024-09-12T03:04:48.775 INFO:teuthology.orchestra.run.smithi148.stdout: "command_resend": 0, 2024-09-12T03:04:48.776 INFO:teuthology.orchestra.run.smithi148.stdout: "map_epoch": 83, 2024-09-12T03:04:48.776 INFO:teuthology.orchestra.run.smithi148.stdout: "map_full": 0, 2024-09-12T03:04:48.776 INFO:teuthology.orchestra.run.smithi148.stdout: "map_inc": 2, 2024-09-12T03:04:48.776 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_sessions": 12, 2024-09-12T03:04:48.776 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_session_open": 12, 2024-09-12T03:04:48.776 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_session_close": 0, 2024-09-12T03:04:48.776 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_laggy": 0, 2024-09-12T03:04:48.776 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_wr": 630, 2024-09-12T03:04:48.776 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_rd": 0, 2024-09-12T03:04:48.776 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_del": 59 2024-09-12T03:04:48.776 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:48.776 INFO:teuthology.orchestra.run.smithi148.stdout: "oft": { 2024-09-12T03:04:48.776 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_total_objs": 1, 2024-09-12T03:04:48.777 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_total_kv_pairs": 0, 2024-09-12T03:04:48.777 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_total_updates": 0, 2024-09-12T03:04:48.777 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_total_removes": 0 2024-09-12T03:04:48.777 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:48.777 INFO:teuthology.orchestra.run.smithi148.stdout: "osd-slow-ops": { 2024-09-12T03:04:48.777 INFO:teuthology.orchestra.run.smithi148.stdout: "slow_ops_count": 13898 2024-09-12T03:04:48.777 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:48.777 INFO:teuthology.orchestra.run.smithi148.stdout: "purge_queue": { 2024-09-12T03:04:48.777 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executing_ops": 0, 2024-09-12T03:04:48.777 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executing_ops_high_water": 70, 2024-09-12T03:04:48.777 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executing": 0, 2024-09-12T03:04:48.781 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executing_high_water": 6, 2024-09-12T03:04:48.781 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executed_ops": 7449, 2024-09-12T03:04:48.781 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executed": 7259, 2024-09-12T03:04:48.781 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_item_in_journal": 0 2024-09-12T03:04:48.781 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:48.781 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T03:04:48.782 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-09-12T03:04:48.782 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 104857600, 2024-09-12T03:04:48.782 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-09-12T03:04:48.782 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 153524, 2024-09-12T03:04:48.782 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 31346288, 2024-09-12T03:04:48.782 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-09-12T03:04:48.782 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 153524, 2024-09-12T03:04:48.782 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 0, 2024-09-12T03:04:48.782 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 0, 2024-09-12T03:04:48.782 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 153524, 2024-09-12T03:04:48.782 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 31346288, 2024-09-12T03:04:48.782 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-09-12T03:04:48.782 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:48.783 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:48.783 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:48.783 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-12T03:04:48.783 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:48.783 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-objecter_bytes": { 2024-09-12T03:04:48.783 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-09-12T03:04:48.783 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 104857600, 2024-09-12T03:04:48.783 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-09-12T03:04:48.783 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 0, 2024-09-12T03:04:48.783 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 0, 2024-09-12T03:04:48.783 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-09-12T03:04:48.784 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 0, 2024-09-12T03:04:48.784 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 97541, 2024-09-12T03:04:48.784 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 122273454, 2024-09-12T03:04:48.784 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 97541, 2024-09-12T03:04:48.784 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 122273454, 2024-09-12T03:04:48.784 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-09-12T03:04:48.784 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:48.784 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:48.784 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:48.784 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-12T03:04:48.784 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:48.784 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-objecter_ops": { 2024-09-12T03:04:48.784 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-09-12T03:04:48.785 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 1024, 2024-09-12T03:04:48.785 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-09-12T03:04:48.785 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 0, 2024-09-12T03:04:48.785 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 0, 2024-09-12T03:04:48.785 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-09-12T03:04:48.785 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 0, 2024-09-12T03:04:48.785 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 97541, 2024-09-12T03:04:48.785 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 97541, 2024-09-12T03:04:48.785 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 97541, 2024-09-12T03:04:48.785 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 97541, 2024-09-12T03:04:48.785 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-09-12T03:04:48.785 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:48.785 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:48.786 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:48.786 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-12T03:04:48.786 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:48.786 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-write_buf_throttle": { 2024-09-12T03:04:48.786 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-09-12T03:04:48.786 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 3758096384, 2024-09-12T03:04:48.786 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-09-12T03:04:48.786 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 7259, 2024-09-12T03:04:48.786 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 733207, 2024-09-12T03:04:48.786 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-09-12T03:04:48.786 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 7259, 2024-09-12T03:04:48.786 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 0, 2024-09-12T03:04:48.786 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 0, 2024-09-12T03:04:48.787 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 68, 2024-09-12T03:04:48.787 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 733207, 2024-09-12T03:04:48.787 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-09-12T03:04:48.787 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:48.787 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:48.787 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:48.787 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-12T03:04:48.787 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:48.787 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-write_buf_throttle-0x55c06f152960": { 2024-09-12T03:04:48.787 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-09-12T03:04:48.787 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 3758096384, 2024-09-12T03:04:48.787 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-09-12T03:04:48.787 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 56323, 2024-09-12T03:04:48.788 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 117031712, 2024-09-12T03:04:48.788 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-09-12T03:04:48.788 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 56323, 2024-09-12T03:04:48.788 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 0, 2024-09-12T03:04:48.788 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 0, 2024-09-12T03:04:48.788 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 24933, 2024-09-12T03:04:48.788 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 117031712, 2024-09-12T03:04:48.788 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-09-12T03:04:48.788 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:48.788 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:48.788 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:48.788 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-12T03:04:48.788 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-12T03:04:48.788 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-12T03:04:49.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:04:48 smithi005 ceph-mon[31197]: pgmap v532: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-12T03:04:49.373 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:04:48 smithi148 ceph-mon[35090]: pgmap v532: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-12T03:04:49.431 INFO:tasks.check_counter:Daemon mds.i mds.exported=4 2024-09-12T03:04:49.431 INFO:tasks.check_counter:Daemon mds.i mds.imported=4 2024-09-12T03:04:49.431 INFO:tasks.check_counter:Daemon mds.i mds.dir_split=25 2024-09-12T03:04:49.431 DEBUG:tasks.check_counter:Getting stats from l 2024-09-12T03:04:49.431 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph daemon mds.l perf dump 2024-09-12T03:04:49.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:04:48 smithi146 ceph-mon[35011]: pgmap v532: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-09-12T03:04:49.677 INFO:teuthology.orchestra.run.smithi148.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.c/config 2024-09-12T03:04:50.261 INFO:teuthology.orchestra.run.smithi148.stdout:{ 2024-09-12T03:04:50.261 INFO:teuthology.orchestra.run.smithi148.stdout: "AsyncMessenger::Worker-0": { 2024-09-12T03:04:50.261 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_messages": 673, 2024-09-12T03:04:50.261 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages": 660, 2024-09-12T03:04:50.262 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_bytes": 974864, 2024-09-12T03:04:50.262 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_bytes": 253028, 2024-09-12T03:04:50.262 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_created_connections": 5, 2024-09-12T03:04:50.262 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_active_connections": 1, 2024-09-12T03:04:50.262 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_total_time": 0.080129521, 2024-09-12T03:04:50.262 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_send_time": 0.026989834, 2024-09-12T03:04:50.262 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_recv_time": 0.036352404, 2024-09-12T03:04:50.262 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_fast_dispatch_time": 0.017528903, 2024-09-12T03:04:50.262 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:04:50.262 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 659, 2024-09-12T03:04:50.262 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.034143705, 2024-09-12T03:04:50.262 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000051811 2024-09-12T03:04:50.262 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:50.263 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:04:50.263 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:50.263 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:50.263 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:50.263 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:50.263 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_encrypted_bytes": 806192, 2024-09-12T03:04:50.263 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_encrypted_bytes": 93392 2024-09-12T03:04:50.263 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:50.263 INFO:teuthology.orchestra.run.smithi148.stdout: "AsyncMessenger::Worker-1": { 2024-09-12T03:04:50.263 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_messages": 7, 2024-09-12T03:04:50.263 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages": 118, 2024-09-12T03:04:50.263 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_bytes": 3740, 2024-09-12T03:04:50.263 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_bytes": 270618, 2024-09-12T03:04:50.264 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_created_connections": 4, 2024-09-12T03:04:50.264 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_active_connections": 3, 2024-09-12T03:04:50.264 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_total_time": 0.008620500, 2024-09-12T03:04:50.264 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_send_time": 0.004348757, 2024-09-12T03:04:50.264 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_recv_time": 0.003219703, 2024-09-12T03:04:50.264 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_fast_dispatch_time": 0.000068052, 2024-09-12T03:04:50.264 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:04:50.264 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 117, 2024-09-12T03:04:50.264 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.011030287, 2024-09-12T03:04:50.264 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000094275 2024-09-12T03:04:50.264 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:50.264 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:04:50.265 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:50.265 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:50.265 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:50.265 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:50.265 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_encrypted_bytes": 3184, 2024-09-12T03:04:50.265 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_encrypted_bytes": 269920 2024-09-12T03:04:50.265 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:50.265 INFO:teuthology.orchestra.run.smithi148.stdout: "AsyncMessenger::Worker-2": { 2024-09-12T03:04:50.265 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_messages": 925, 2024-09-12T03:04:50.265 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages": 926, 2024-09-12T03:04:50.265 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_bytes": 268574, 2024-09-12T03:04:50.265 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_bytes": 293839, 2024-09-12T03:04:50.265 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_created_connections": 4, 2024-09-12T03:04:50.266 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_active_connections": 3, 2024-09-12T03:04:50.266 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_total_time": 0.063823158, 2024-09-12T03:04:50.266 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_send_time": 0.019650273, 2024-09-12T03:04:50.266 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_recv_time": 0.018667669, 2024-09-12T03:04:50.266 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_running_fast_dispatch_time": 0.021149180, 2024-09-12T03:04:50.266 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_messages_queue_lat": { 2024-09-12T03:04:50.266 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 924, 2024-09-12T03:04:50.266 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.031641497, 2024-09-12T03:04:50.266 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000034244 2024-09-12T03:04:50.266 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:50.266 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_handle_ack_lat": { 2024-09-12T03:04:50.266 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:50.266 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:50.267 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:50.267 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:50.267 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-12T03:04:50.267 INFO:teuthology.orchestra.run.smithi148.stdout: "msgr_send_encrypted_bytes": 0 2024-09-12T03:04:50.267 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:50.267 INFO:teuthology.orchestra.run.smithi148.stdout: "cct": { 2024-09-12T03:04:50.267 INFO:teuthology.orchestra.run.smithi148.stdout: "total_workers": 1, 2024-09-12T03:04:50.267 INFO:teuthology.orchestra.run.smithi148.stdout: "unhealthy_workers": 0 2024-09-12T03:04:50.267 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:50.267 INFO:teuthology.orchestra.run.smithi148.stdout: "finisher-MDSRank": { 2024-09-12T03:04:50.267 INFO:teuthology.orchestra.run.smithi148.stdout: "queue_len": 0, 2024-09-12T03:04:50.267 INFO:teuthology.orchestra.run.smithi148.stdout: "complete_latency": { 2024-09-12T03:04:50.267 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 2441, 2024-09-12T03:04:50.268 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.062364839, 2024-09-12T03:04:50.268 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000025548 2024-09-12T03:04:50.268 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-12T03:04:50.268 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:50.268 INFO:teuthology.orchestra.run.smithi148.stdout: "finisher-PurgeQueue": { 2024-09-12T03:04:50.268 INFO:teuthology.orchestra.run.smithi148.stdout: "queue_len": 0, 2024-09-12T03:04:50.268 INFO:teuthology.orchestra.run.smithi148.stdout: "complete_latency": { 2024-09-12T03:04:50.268 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:50.268 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:50.268 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:50.268 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-12T03:04:50.268 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:50.269 INFO:teuthology.orchestra.run.smithi148.stdout: "mds": { 2024-09-12T03:04:50.269 INFO:teuthology.orchestra.run.smithi148.stdout: "request": 0, 2024-09-12T03:04:50.269 INFO:teuthology.orchestra.run.smithi148.stdout: "reply": 0, 2024-09-12T03:04:50.269 INFO:teuthology.orchestra.run.smithi148.stdout: "reply_latency": { 2024-09-12T03:04:50.269 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:50.269 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:50.269 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:50.269 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:50.269 INFO:teuthology.orchestra.run.smithi148.stdout: "slow_reply": 0, 2024-09-12T03:04:50.269 INFO:teuthology.orchestra.run.smithi148.stdout: "forward": 0, 2024-09-12T03:04:50.269 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_fetch_complete": 0, 2024-09-12T03:04:50.269 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_fetch_keys": 0, 2024-09-12T03:04:50.269 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_commit": 0, 2024-09-12T03:04:50.269 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_split": 0, 2024-09-12T03:04:50.270 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_merge": 0, 2024-09-12T03:04:50.270 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes": 3, 2024-09-12T03:04:50.270 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_top": 0, 2024-09-12T03:04:50.270 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_bottom": 3, 2024-09-12T03:04:50.270 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_pin_tail": 0, 2024-09-12T03:04:50.270 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_pinned": 3, 2024-09-12T03:04:50.270 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_expired": 0, 2024-09-12T03:04:50.270 INFO:teuthology.orchestra.run.smithi148.stdout: "inodes_with_caps": 0, 2024-09-12T03:04:50.270 INFO:teuthology.orchestra.run.smithi148.stdout: "caps": 0, 2024-09-12T03:04:50.270 INFO:teuthology.orchestra.run.smithi148.stdout: "subtrees": 1, 2024-09-12T03:04:50.270 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse": 0, 2024-09-12T03:04:50.270 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_hit": 0, 2024-09-12T03:04:50.270 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_forward": 0, 2024-09-12T03:04:50.271 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_discover": 0, 2024-09-12T03:04:50.271 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_dir_fetch": 0, 2024-09-12T03:04:50.271 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_remote_ino": 0, 2024-09-12T03:04:50.271 INFO:teuthology.orchestra.run.smithi148.stdout: "traverse_lock": 0, 2024-09-12T03:04:50.271 INFO:teuthology.orchestra.run.smithi148.stdout: "load_cent": 0, 2024-09-12T03:04:50.271 INFO:teuthology.orchestra.run.smithi148.stdout: "q": 0, 2024-09-12T03:04:50.271 INFO:teuthology.orchestra.run.smithi148.stdout: "exported": 0, 2024-09-12T03:04:50.271 INFO:teuthology.orchestra.run.smithi148.stdout: "exported_inodes": 0, 2024-09-12T03:04:50.271 INFO:teuthology.orchestra.run.smithi148.stdout: "imported": 0, 2024-09-12T03:04:50.271 INFO:teuthology.orchestra.run.smithi148.stdout: "imported_inodes": 0, 2024-09-12T03:04:50.271 INFO:teuthology.orchestra.run.smithi148.stdout: "openino_dir_fetch": 0, 2024-09-12T03:04:50.271 INFO:teuthology.orchestra.run.smithi148.stdout: "openino_backtrace_fetch": 0, 2024-09-12T03:04:50.271 INFO:teuthology.orchestra.run.smithi148.stdout: "openino_peer_discover": 0, 2024-09-12T03:04:50.272 INFO:teuthology.orchestra.run.smithi148.stdout: "root_rfiles": 0, 2024-09-12T03:04:50.272 INFO:teuthology.orchestra.run.smithi148.stdout: "root_rbytes": 0, 2024-09-12T03:04:50.272 INFO:teuthology.orchestra.run.smithi148.stdout: "root_rsnaps": 0, 2024-09-12T03:04:50.272 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_backtrace_fetch": 0, 2024-09-12T03:04:50.272 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_set_tag": 0, 2024-09-12T03:04:50.272 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_backtrace_repaired": 0, 2024-09-12T03:04:50.272 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_inotable_repaired": 0, 2024-09-12T03:04:50.272 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_dir_inodes": 0, 2024-09-12T03:04:50.272 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_dir_base_inodes": 0, 2024-09-12T03:04:50.272 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_dirfrag_rstats": 0, 2024-09-12T03:04:50.272 INFO:teuthology.orchestra.run.smithi148.stdout: "scrub_file_inodes": 0, 2024-09-12T03:04:50.272 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_inode_file_caps": 0, 2024-09-12T03:04:50.272 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_revoke": 0, 2024-09-12T03:04:50.273 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_grant": 0, 2024-09-12T03:04:50.273 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_trunc": 0, 2024-09-12T03:04:50.273 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-12T03:04:50.273 INFO:teuthology.orchestra.run.smithi148.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-12T03:04:50.273 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_caps": 0, 2024-09-12T03:04:50.273 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_caps_dirty": 0, 2024-09-12T03:04:50.273 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_cap_release": 0, 2024-09-12T03:04:50.273 INFO:teuthology.orchestra.run.smithi148.stdout: "process_request_cap_release": 0 2024-09-12T03:04:50.273 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:50.273 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_cache": { 2024-09-12T03:04:50.273 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_update": 0, 2024-09-12T03:04:50.273 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_update_receipt": 0, 2024-09-12T03:04:50.273 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_try_discover": 0, 2024-09-12T03:04:50.274 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_send_discover": 0, 2024-09-12T03:04:50.274 INFO:teuthology.orchestra.run.smithi148.stdout: "dir_handle_discover": 0, 2024-09-12T03:04:50.274 INFO:teuthology.orchestra.run.smithi148.stdout: "num_strays": 0, 2024-09-12T03:04:50.274 INFO:teuthology.orchestra.run.smithi148.stdout: "num_strays_delayed": 0, 2024-09-12T03:04:50.274 INFO:teuthology.orchestra.run.smithi148.stdout: "num_strays_enqueuing": 0, 2024-09-12T03:04:50.274 INFO:teuthology.orchestra.run.smithi148.stdout: "strays_created": 0, 2024-09-12T03:04:50.274 INFO:teuthology.orchestra.run.smithi148.stdout: "strays_enqueued": 0, 2024-09-12T03:04:50.274 INFO:teuthology.orchestra.run.smithi148.stdout: "strays_reintegrated": 0, 2024-09-12T03:04:50.274 INFO:teuthology.orchestra.run.smithi148.stdout: "strays_migrated": 0, 2024-09-12T03:04:50.274 INFO:teuthology.orchestra.run.smithi148.stdout: "num_recovering_processing": 0, 2024-09-12T03:04:50.274 INFO:teuthology.orchestra.run.smithi148.stdout: "num_recovering_enqueued": 0, 2024-09-12T03:04:50.274 INFO:teuthology.orchestra.run.smithi148.stdout: "num_recovering_prioritized": 0, 2024-09-12T03:04:50.274 INFO:teuthology.orchestra.run.smithi148.stdout: "recovery_started": 0, 2024-09-12T03:04:50.274 INFO:teuthology.orchestra.run.smithi148.stdout: "recovery_completed": 0, 2024-09-12T03:04:50.275 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_quiesce_path": 0, 2024-09-12T03:04:50.275 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_quiesce_inode": 0, 2024-09-12T03:04:50.275 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_enqueue_scrub": 0, 2024-09-12T03:04:50.275 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_exportdir": 0, 2024-09-12T03:04:50.275 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_flush": 0, 2024-09-12T03:04:50.275 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_fragmentdir": 0, 2024-09-12T03:04:50.275 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_fragstats": 0, 2024-09-12T03:04:50.275 INFO:teuthology.orchestra.run.smithi148.stdout: "ireq_inodestats": 0 2024-09-12T03:04:50.275 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:50.275 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_log": { 2024-09-12T03:04:50.275 INFO:teuthology.orchestra.run.smithi148.stdout: "evlrg": 0, 2024-09-12T03:04:50.275 INFO:teuthology.orchestra.run.smithi148.stdout: "evadd": 0, 2024-09-12T03:04:50.275 INFO:teuthology.orchestra.run.smithi148.stdout: "evex": 0, 2024-09-12T03:04:50.275 INFO:teuthology.orchestra.run.smithi148.stdout: "evtrm": 0, 2024-09-12T03:04:50.276 INFO:teuthology.orchestra.run.smithi148.stdout: "ev": 20, 2024-09-12T03:04:50.276 INFO:teuthology.orchestra.run.smithi148.stdout: "evexg": 0, 2024-09-12T03:04:50.276 INFO:teuthology.orchestra.run.smithi148.stdout: "evexd": 0, 2024-09-12T03:04:50.276 INFO:teuthology.orchestra.run.smithi148.stdout: "segadd": 0, 2024-09-12T03:04:50.276 INFO:teuthology.orchestra.run.smithi148.stdout: "segex": 0, 2024-09-12T03:04:50.276 INFO:teuthology.orchestra.run.smithi148.stdout: "segtrm": 0, 2024-09-12T03:04:50.276 INFO:teuthology.orchestra.run.smithi148.stdout: "seg": 2, 2024-09-12T03:04:50.276 INFO:teuthology.orchestra.run.smithi148.stdout: "segmjr": 2, 2024-09-12T03:04:50.276 INFO:teuthology.orchestra.run.smithi148.stdout: "segexg": 0, 2024-09-12T03:04:50.276 INFO:teuthology.orchestra.run.smithi148.stdout: "segexd": 0, 2024-09-12T03:04:50.276 INFO:teuthology.orchestra.run.smithi148.stdout: "expos": 4194304, 2024-09-12T03:04:50.276 INFO:teuthology.orchestra.run.smithi148.stdout: "wrpos": 4226471, 2024-09-12T03:04:50.276 INFO:teuthology.orchestra.run.smithi148.stdout: "rdpos": 4224820, 2024-09-12T03:04:50.277 INFO:teuthology.orchestra.run.smithi148.stdout: "jlat": { 2024-09-12T03:04:50.277 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:50.277 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:50.277 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:50.277 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:50.277 INFO:teuthology.orchestra.run.smithi148.stdout: "replayed": 20 2024-09-12T03:04:50.277 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:50.277 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_mem": { 2024-09-12T03:04:50.277 INFO:teuthology.orchestra.run.smithi148.stdout: "ino": 4, 2024-09-12T03:04:50.277 INFO:teuthology.orchestra.run.smithi148.stdout: "ino+": 18, 2024-09-12T03:04:50.277 INFO:teuthology.orchestra.run.smithi148.stdout: "ino-": 14, 2024-09-12T03:04:50.277 INFO:teuthology.orchestra.run.smithi148.stdout: "dir": 2, 2024-09-12T03:04:50.277 INFO:teuthology.orchestra.run.smithi148.stdout: "dir+": 16, 2024-09-12T03:04:50.278 INFO:teuthology.orchestra.run.smithi148.stdout: "dir-": 14, 2024-09-12T03:04:50.278 INFO:teuthology.orchestra.run.smithi148.stdout: "dn": 3, 2024-09-12T03:04:50.278 INFO:teuthology.orchestra.run.smithi148.stdout: "dn+": 18, 2024-09-12T03:04:50.278 INFO:teuthology.orchestra.run.smithi148.stdout: "dn-": 15, 2024-09-12T03:04:50.278 INFO:teuthology.orchestra.run.smithi148.stdout: "cap": 0, 2024-09-12T03:04:50.278 INFO:teuthology.orchestra.run.smithi148.stdout: "cap+": 0, 2024-09-12T03:04:50.278 INFO:teuthology.orchestra.run.smithi148.stdout: "cap-": 0, 2024-09-12T03:04:50.278 INFO:teuthology.orchestra.run.smithi148.stdout: "rss": 50868, 2024-09-12T03:04:50.278 INFO:teuthology.orchestra.run.smithi148.stdout: "heap": 215328 2024-09-12T03:04:50.278 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:50.278 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_server": { 2024-09-12T03:04:50.278 INFO:teuthology.orchestra.run.smithi148.stdout: "dispatch_client_request": 0, 2024-09-12T03:04:50.279 INFO:teuthology.orchestra.run.smithi148.stdout: "dispatch_server_request": 0, 2024-09-12T03:04:50.279 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_request": 0, 2024-09-12T03:04:50.279 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_client_session": 0, 2024-09-12T03:04:50.279 INFO:teuthology.orchestra.run.smithi148.stdout: "handle_peer_request": 0, 2024-09-12T03:04:50.279 INFO:teuthology.orchestra.run.smithi148.stdout: "req_create_latency": { 2024-09-12T03:04:50.279 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:50.279 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:50.279 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:50.279 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:50.279 INFO:teuthology.orchestra.run.smithi148.stdout: "req_getattr_latency": { 2024-09-12T03:04:50.279 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:50.279 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:50.279 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:50.279 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:50.279 INFO:teuthology.orchestra.run.smithi148.stdout: "req_getfilelock_latency": { 2024-09-12T03:04:50.280 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:50.280 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:50.280 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:50.280 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:50.280 INFO:teuthology.orchestra.run.smithi148.stdout: "req_link_latency": { 2024-09-12T03:04:50.280 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:50.280 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:50.280 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:50.280 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:50.280 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookup_latency": { 2024-09-12T03:04:50.280 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:50.280 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:50.280 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:50.280 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:50.281 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookuphash_latency": { 2024-09-12T03:04:50.281 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:50.281 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:50.281 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:50.281 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:50.281 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookupino_latency": { 2024-09-12T03:04:50.281 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:50.281 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:50.281 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:50.281 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:50.281 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookupname_latency": { 2024-09-12T03:04:50.281 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:50.281 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:50.282 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:50.282 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:50.282 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookupparent_latency": { 2024-09-12T03:04:50.282 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:50.282 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:50.282 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:50.282 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:50.282 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lookupsnap_latency": { 2024-09-12T03:04:50.282 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:50.282 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:50.282 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:50.282 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:50.283 INFO:teuthology.orchestra.run.smithi148.stdout: "req_lssnap_latency": { 2024-09-12T03:04:50.283 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:50.283 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:50.283 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:50.283 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:50.283 INFO:teuthology.orchestra.run.smithi148.stdout: "req_mkdir_latency": { 2024-09-12T03:04:50.283 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:50.284 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:50.284 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:50.284 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:50.284 INFO:teuthology.orchestra.run.smithi148.stdout: "req_mknod_latency": { 2024-09-12T03:04:50.284 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:50.284 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:50.284 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:50.284 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:50.284 INFO:teuthology.orchestra.run.smithi148.stdout: "req_mksnap_latency": { 2024-09-12T03:04:50.284 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:50.285 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:50.285 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:50.285 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:50.285 INFO:teuthology.orchestra.run.smithi148.stdout: "req_open_latency": { 2024-09-12T03:04:50.285 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:50.285 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:50.285 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:50.285 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:50.285 INFO:teuthology.orchestra.run.smithi148.stdout: "req_readdir_latency": { 2024-09-12T03:04:50.285 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:50.285 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:50.285 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:50.285 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:50.286 INFO:teuthology.orchestra.run.smithi148.stdout: "req_rename_latency": { 2024-09-12T03:04:50.286 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:50.286 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:50.286 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:50.286 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:50.286 INFO:teuthology.orchestra.run.smithi148.stdout: "req_renamesnap_latency": { 2024-09-12T03:04:50.286 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:50.286 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:50.286 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:50.286 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:50.286 INFO:teuthology.orchestra.run.smithi148.stdout: "req_snapdiff_latency": { 2024-09-12T03:04:50.286 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:50.286 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:50.287 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:50.287 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:50.287 INFO:teuthology.orchestra.run.smithi148.stdout: "req_rmdir_latency": { 2024-09-12T03:04:50.287 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:50.287 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:50.287 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:50.287 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:50.287 INFO:teuthology.orchestra.run.smithi148.stdout: "req_rmsnap_latency": { 2024-09-12T03:04:50.287 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:50.287 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:50.287 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:50.287 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:50.287 INFO:teuthology.orchestra.run.smithi148.stdout: "req_rmxattr_latency": { 2024-09-12T03:04:50.288 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:50.288 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:50.288 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:50.288 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:50.288 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setattr_latency": { 2024-09-12T03:04:50.288 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:50.288 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:50.288 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:50.288 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:50.288 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setdirlayout_latency": { 2024-09-12T03:04:50.288 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:50.288 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:50.289 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:50.289 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:50.289 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setfilelock_latency": { 2024-09-12T03:04:50.289 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:50.289 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:50.289 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:50.289 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:50.289 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setlayout_latency": { 2024-09-12T03:04:50.289 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:50.289 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:50.289 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:50.289 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:50.289 INFO:teuthology.orchestra.run.smithi148.stdout: "req_setxattr_latency": { 2024-09-12T03:04:50.290 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:50.290 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:50.290 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:50.290 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:50.290 INFO:teuthology.orchestra.run.smithi148.stdout: "req_symlink_latency": { 2024-09-12T03:04:50.290 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:50.290 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:50.290 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:50.290 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:50.290 INFO:teuthology.orchestra.run.smithi148.stdout: "req_unlink_latency": { 2024-09-12T03:04:50.290 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:50.290 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:50.290 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:50.291 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:50.291 INFO:teuthology.orchestra.run.smithi148.stdout: "cap_revoke_eviction": 0, 2024-09-12T03:04:50.291 INFO:teuthology.orchestra.run.smithi148.stdout: "cap_acquisition_throttle": 0, 2024-09-12T03:04:50.291 INFO:teuthology.orchestra.run.smithi148.stdout: "req_getvxattr_latency": { 2024-09-12T03:04:50.291 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:50.291 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:50.291 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:50.291 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-12T03:04:50.291 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:50.291 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_sessions": { 2024-09-12T03:04:50.291 INFO:teuthology.orchestra.run.smithi148.stdout: "session_count": 1, 2024-09-12T03:04:50.291 INFO:teuthology.orchestra.run.smithi148.stdout: "session_add": 2, 2024-09-12T03:04:50.291 INFO:teuthology.orchestra.run.smithi148.stdout: "session_remove": 1, 2024-09-12T03:04:50.292 INFO:teuthology.orchestra.run.smithi148.stdout: "sessions_open": 1, 2024-09-12T03:04:50.292 INFO:teuthology.orchestra.run.smithi148.stdout: "sessions_stale": 0, 2024-09-12T03:04:50.292 INFO:teuthology.orchestra.run.smithi148.stdout: "total_load": 0, 2024-09-12T03:04:50.292 INFO:teuthology.orchestra.run.smithi148.stdout: "average_load": 0, 2024-09-12T03:04:50.292 INFO:teuthology.orchestra.run.smithi148.stdout: "avg_session_uptime": 1639, 2024-09-12T03:04:50.292 INFO:teuthology.orchestra.run.smithi148.stdout: "mdthresh_evicted": 0 2024-09-12T03:04:50.292 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:50.292 INFO:teuthology.orchestra.run.smithi148.stdout: "mempool": { 2024-09-12T03:04:50.292 INFO:teuthology.orchestra.run.smithi148.stdout: "bloom_filter_bytes": 0, 2024-09-12T03:04:50.292 INFO:teuthology.orchestra.run.smithi148.stdout: "bloom_filter_items": 0, 2024-09-12T03:04:50.292 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_alloc_bytes": 0, 2024-09-12T03:04:50.292 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_alloc_items": 0, 2024-09-12T03:04:50.293 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_data_bytes": 0, 2024-09-12T03:04:50.293 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_data_items": 0, 2024-09-12T03:04:50.293 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-12T03:04:50.293 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_onode_items": 0, 2024-09-12T03:04:50.293 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-12T03:04:50.293 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_meta_items": 0, 2024-09-12T03:04:50.293 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_other_bytes": 0, 2024-09-12T03:04:50.293 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_other_items": 0, 2024-09-12T03:04:50.293 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-12T03:04:50.293 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_cache_buffer_items": 0, 2024-09-12T03:04:50.293 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_extent_bytes": 0, 2024-09-12T03:04:50.293 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_extent_items": 0, 2024-09-12T03:04:50.293 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_blob_bytes": 0, 2024-09-12T03:04:50.293 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_blob_items": 0, 2024-09-12T03:04:50.294 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-12T03:04:50.294 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_shared_blob_items": 0, 2024-09-12T03:04:50.294 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-12T03:04:50.294 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_inline_bl_items": 0, 2024-09-12T03:04:50.294 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_fsck_bytes": 0, 2024-09-12T03:04:50.294 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_fsck_items": 0, 2024-09-12T03:04:50.294 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_txc_bytes": 0, 2024-09-12T03:04:50.294 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_txc_items": 0, 2024-09-12T03:04:50.294 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-12T03:04:50.294 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_deferred_items": 0, 2024-09-12T03:04:50.294 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_bytes": 0, 2024-09-12T03:04:50.294 INFO:teuthology.orchestra.run.smithi148.stdout: "bluestore_writing_items": 0, 2024-09-12T03:04:50.294 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_bytes": 0, 2024-09-12T03:04:50.295 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_items": 0, 2024-09-12T03:04:50.295 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_reader_bytes": 0, 2024-09-12T03:04:50.295 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_reader_items": 0, 2024-09-12T03:04:50.295 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_writer_bytes": 0, 2024-09-12T03:04:50.295 INFO:teuthology.orchestra.run.smithi148.stdout: "bluefs_file_writer_items": 0, 2024-09-12T03:04:50.295 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_anon_bytes": 182351, 2024-09-12T03:04:50.295 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_anon_items": 48, 2024-09-12T03:04:50.295 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_meta_bytes": 0, 2024-09-12T03:04:50.295 INFO:teuthology.orchestra.run.smithi148.stdout: "buffer_meta_items": 0, 2024-09-12T03:04:50.295 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_bytes": 0, 2024-09-12T03:04:50.295 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_items": 0, 2024-09-12T03:04:50.295 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_mapbl_bytes": 0, 2024-09-12T03:04:50.295 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_mapbl_items": 0, 2024-09-12T03:04:50.296 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_pglog_bytes": 0, 2024-09-12T03:04:50.296 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_pglog_items": 0, 2024-09-12T03:04:50.296 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_bytes": 7496, 2024-09-12T03:04:50.296 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_items": 154, 2024-09-12T03:04:50.296 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_mapping_bytes": 0, 2024-09-12T03:04:50.296 INFO:teuthology.orchestra.run.smithi148.stdout: "osdmap_mapping_items": 0, 2024-09-12T03:04:50.296 INFO:teuthology.orchestra.run.smithi148.stdout: "pgmap_bytes": 0, 2024-09-12T03:04:50.296 INFO:teuthology.orchestra.run.smithi148.stdout: "pgmap_items": 0, 2024-09-12T03:04:50.296 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_co_bytes": 14160, 2024-09-12T03:04:50.296 INFO:teuthology.orchestra.run.smithi148.stdout: "mds_co_items": 40, 2024-09-12T03:04:50.296 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_1_bytes": 0, 2024-09-12T03:04:50.296 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_1_items": 0, 2024-09-12T03:04:50.296 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_2_bytes": 0, 2024-09-12T03:04:50.297 INFO:teuthology.orchestra.run.smithi148.stdout: "unittest_2_items": 0 2024-09-12T03:04:50.297 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:50.297 INFO:teuthology.orchestra.run.smithi148.stdout: "objecter": { 2024-09-12T03:04:50.297 INFO:teuthology.orchestra.run.smithi148.stdout: "op_active": 0, 2024-09-12T03:04:50.297 INFO:teuthology.orchestra.run.smithi148.stdout: "op_laggy": 0, 2024-09-12T03:04:50.297 INFO:teuthology.orchestra.run.smithi148.stdout: "op_send": 1428, 2024-09-12T03:04:50.297 INFO:teuthology.orchestra.run.smithi148.stdout: "op_send_bytes": 21, 2024-09-12T03:04:50.297 INFO:teuthology.orchestra.run.smithi148.stdout: "op_resend": 3, 2024-09-12T03:04:50.297 INFO:teuthology.orchestra.run.smithi148.stdout: "op_reply": 1428, 2024-09-12T03:04:50.297 INFO:teuthology.orchestra.run.smithi148.stdout: "op_latency": { 2024-09-12T03:04:50.297 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 1428, 2024-09-12T03:04:50.297 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.854978493, 2024-09-12T03:04:50.297 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000598724 2024-09-12T03:04:50.298 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:50.298 INFO:teuthology.orchestra.run.smithi148.stdout: "op_inflight": 0, 2024-09-12T03:04:50.298 INFO:teuthology.orchestra.run.smithi148.stdout: "oplen_avg": { 2024-09-12T03:04:50.298 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 1428, 2024-09-12T03:04:50.298 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 1429 2024-09-12T03:04:50.298 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:50.298 INFO:teuthology.orchestra.run.smithi148.stdout: "op": 1428, 2024-09-12T03:04:50.298 INFO:teuthology.orchestra.run.smithi148.stdout: "op_r": 1428, 2024-09-12T03:04:50.298 INFO:teuthology.orchestra.run.smithi148.stdout: "op_w": 0, 2024-09-12T03:04:50.298 INFO:teuthology.orchestra.run.smithi148.stdout: "op_rmw": 0, 2024-09-12T03:04:50.298 INFO:teuthology.orchestra.run.smithi148.stdout: "op_pg": 0, 2024-09-12T03:04:50.298 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_stat": 908, 2024-09-12T03:04:50.298 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_create": 0, 2024-09-12T03:04:50.298 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_read": 518, 2024-09-12T03:04:50.299 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_write": 0, 2024-09-12T03:04:50.299 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_writefull": 0, 2024-09-12T03:04:50.299 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_writesame": 0, 2024-09-12T03:04:50.299 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_append": 0, 2024-09-12T03:04:50.299 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_zero": 0, 2024-09-12T03:04:50.299 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_truncate": 0, 2024-09-12T03:04:50.299 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_delete": 0, 2024-09-12T03:04:50.299 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_mapext": 0, 2024-09-12T03:04:50.299 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_sparse_read": 0, 2024-09-12T03:04:50.299 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_clonerange": 0, 2024-09-12T03:04:50.299 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_getxattr": 1, 2024-09-12T03:04:50.299 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_setxattr": 0, 2024-09-12T03:04:50.299 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_cmpxattr": 0, 2024-09-12T03:04:50.299 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_rmxattr": 0, 2024-09-12T03:04:50.300 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_resetxattrs": 0, 2024-09-12T03:04:50.300 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_call": 0, 2024-09-12T03:04:50.300 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_watch": 0, 2024-09-12T03:04:50.300 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_notify": 0, 2024-09-12T03:04:50.300 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_src_cmpxattr": 0, 2024-09-12T03:04:50.300 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_pgls": 0, 2024-09-12T03:04:50.300 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_pgls_filter": 0, 2024-09-12T03:04:50.300 INFO:teuthology.orchestra.run.smithi148.stdout: "osdop_other": 0, 2024-09-12T03:04:50.300 INFO:teuthology.orchestra.run.smithi148.stdout: "linger_active": 0, 2024-09-12T03:04:50.300 INFO:teuthology.orchestra.run.smithi148.stdout: "linger_send": 0, 2024-09-12T03:04:50.300 INFO:teuthology.orchestra.run.smithi148.stdout: "linger_resend": 0, 2024-09-12T03:04:50.300 INFO:teuthology.orchestra.run.smithi148.stdout: "linger_ping": 0, 2024-09-12T03:04:50.300 INFO:teuthology.orchestra.run.smithi148.stdout: "poolop_active": 0, 2024-09-12T03:04:50.301 INFO:teuthology.orchestra.run.smithi148.stdout: "poolop_send": 0, 2024-09-12T03:04:50.301 INFO:teuthology.orchestra.run.smithi148.stdout: "poolop_resend": 0, 2024-09-12T03:04:50.301 INFO:teuthology.orchestra.run.smithi148.stdout: "poolstat_active": 0, 2024-09-12T03:04:50.301 INFO:teuthology.orchestra.run.smithi148.stdout: "poolstat_send": 0, 2024-09-12T03:04:50.301 INFO:teuthology.orchestra.run.smithi148.stdout: "poolstat_resend": 0, 2024-09-12T03:04:50.301 INFO:teuthology.orchestra.run.smithi148.stdout: "statfs_active": 0, 2024-09-12T03:04:50.301 INFO:teuthology.orchestra.run.smithi148.stdout: "statfs_send": 0, 2024-09-12T03:04:50.301 INFO:teuthology.orchestra.run.smithi148.stdout: "statfs_resend": 0, 2024-09-12T03:04:50.301 INFO:teuthology.orchestra.run.smithi148.stdout: "command_active": 0, 2024-09-12T03:04:50.301 INFO:teuthology.orchestra.run.smithi148.stdout: "command_send": 0, 2024-09-12T03:04:50.301 INFO:teuthology.orchestra.run.smithi148.stdout: "command_resend": 0, 2024-09-12T03:04:50.301 INFO:teuthology.orchestra.run.smithi148.stdout: "map_epoch": 83, 2024-09-12T03:04:50.301 INFO:teuthology.orchestra.run.smithi148.stdout: "map_full": 0, 2024-09-12T03:04:50.302 INFO:teuthology.orchestra.run.smithi148.stdout: "map_inc": 2, 2024-09-12T03:04:50.302 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_sessions": 4, 2024-09-12T03:04:50.302 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_session_open": 4, 2024-09-12T03:04:50.302 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_session_close": 0, 2024-09-12T03:04:50.302 INFO:teuthology.orchestra.run.smithi148.stdout: "osd_laggy": 0, 2024-09-12T03:04:50.302 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_wr": 0, 2024-09-12T03:04:50.302 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_rd": 2, 2024-09-12T03:04:50.302 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_del": 0 2024-09-12T03:04:50.302 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:50.302 INFO:teuthology.orchestra.run.smithi148.stdout: "oft": { 2024-09-12T03:04:50.302 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_total_objs": 0, 2024-09-12T03:04:50.302 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_total_kv_pairs": 0, 2024-09-12T03:04:50.302 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_total_updates": 0, 2024-09-12T03:04:50.302 INFO:teuthology.orchestra.run.smithi148.stdout: "omap_total_removes": 0 2024-09-12T03:04:50.303 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:50.303 INFO:teuthology.orchestra.run.smithi148.stdout: "osd-slow-ops": { 2024-09-12T03:04:50.303 INFO:teuthology.orchestra.run.smithi148.stdout: "slow_ops_count": 0 2024-09-12T03:04:50.303 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:50.303 INFO:teuthology.orchestra.run.smithi148.stdout: "purge_queue": { 2024-09-12T03:04:50.303 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executing_ops": 0, 2024-09-12T03:04:50.303 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executing_ops_high_water": 0, 2024-09-12T03:04:50.303 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executing": 0, 2024-09-12T03:04:50.303 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executing_high_water": 0, 2024-09-12T03:04:50.303 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executed_ops": 0, 2024-09-12T03:04:50.303 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_executed": 0, 2024-09-12T03:04:50.303 INFO:teuthology.orchestra.run.smithi148.stdout: "pq_item_in_journal": 0 2024-09-12T03:04:50.303 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:50.303 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-12T03:04:50.304 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-09-12T03:04:50.304 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 104857600, 2024-09-12T03:04:50.304 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-09-12T03:04:50.304 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 1601, 2024-09-12T03:04:50.304 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 1091356, 2024-09-12T03:04:50.304 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-09-12T03:04:50.304 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 1601, 2024-09-12T03:04:50.304 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 0, 2024-09-12T03:04:50.304 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 0, 2024-09-12T03:04:50.304 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 1601, 2024-09-12T03:04:50.304 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 1091356, 2024-09-12T03:04:50.304 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-09-12T03:04:50.305 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:50.305 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:50.305 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:50.305 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-12T03:04:50.305 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:50.305 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-objecter_bytes": { 2024-09-12T03:04:50.305 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-09-12T03:04:50.305 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 104857600, 2024-09-12T03:04:50.305 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-09-12T03:04:50.305 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 0, 2024-09-12T03:04:50.305 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 0, 2024-09-12T03:04:50.305 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-09-12T03:04:50.305 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 0, 2024-09-12T03:04:50.305 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 1428, 2024-09-12T03:04:50.306 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 32172, 2024-09-12T03:04:50.306 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 1428, 2024-09-12T03:04:50.306 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 32172, 2024-09-12T03:04:50.306 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-09-12T03:04:50.306 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:50.306 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:50.306 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:50.306 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-12T03:04:50.306 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:50.306 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-objecter_ops": { 2024-09-12T03:04:50.306 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-09-12T03:04:50.306 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 1024, 2024-09-12T03:04:50.306 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-09-12T03:04:50.307 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 0, 2024-09-12T03:04:50.307 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 0, 2024-09-12T03:04:50.307 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-09-12T03:04:50.307 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 0, 2024-09-12T03:04:50.307 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 1428, 2024-09-12T03:04:50.307 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 1428, 2024-09-12T03:04:50.307 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 1428, 2024-09-12T03:04:50.307 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 1428, 2024-09-12T03:04:50.307 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-09-12T03:04:50.307 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:50.307 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:50.307 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:50.307 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-12T03:04:50.307 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:50.308 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-write_buf_throttle": { 2024-09-12T03:04:50.308 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-09-12T03:04:50.308 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 3758096384, 2024-09-12T03:04:50.308 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-09-12T03:04:50.308 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 0, 2024-09-12T03:04:50.308 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 0, 2024-09-12T03:04:50.308 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-09-12T03:04:50.308 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 0, 2024-09-12T03:04:50.308 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 0, 2024-09-12T03:04:50.308 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 0, 2024-09-12T03:04:50.308 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 0, 2024-09-12T03:04:50.308 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 0, 2024-09-12T03:04:50.308 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-09-12T03:04:50.309 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:50.309 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:50.309 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:50.309 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-12T03:04:50.309 INFO:teuthology.orchestra.run.smithi148.stdout: }, 2024-09-12T03:04:50.309 INFO:teuthology.orchestra.run.smithi148.stdout: "throttle-write_buf_throttle-0x55e228bb0b40": { 2024-09-12T03:04:50.309 INFO:teuthology.orchestra.run.smithi148.stdout: "val": 0, 2024-09-12T03:04:50.309 INFO:teuthology.orchestra.run.smithi148.stdout: "max": 3758096384, 2024-09-12T03:04:50.309 INFO:teuthology.orchestra.run.smithi148.stdout: "get_started": 0, 2024-09-12T03:04:50.309 INFO:teuthology.orchestra.run.smithi148.stdout: "get": 0, 2024-09-12T03:04:50.309 INFO:teuthology.orchestra.run.smithi148.stdout: "get_sum": 0, 2024-09-12T03:04:50.309 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_fail": 0, 2024-09-12T03:04:50.309 INFO:teuthology.orchestra.run.smithi148.stdout: "get_or_fail_success": 0, 2024-09-12T03:04:50.309 INFO:teuthology.orchestra.run.smithi148.stdout: "take": 0, 2024-09-12T03:04:50.310 INFO:teuthology.orchestra.run.smithi148.stdout: "take_sum": 0, 2024-09-12T03:04:50.310 INFO:teuthology.orchestra.run.smithi148.stdout: "put": 0, 2024-09-12T03:04:50.310 INFO:teuthology.orchestra.run.smithi148.stdout: "put_sum": 0, 2024-09-12T03:04:50.310 INFO:teuthology.orchestra.run.smithi148.stdout: "wait": { 2024-09-12T03:04:50.310 INFO:teuthology.orchestra.run.smithi148.stdout: "avgcount": 0, 2024-09-12T03:04:50.310 INFO:teuthology.orchestra.run.smithi148.stdout: "sum": 0.000000000, 2024-09-12T03:04:50.310 INFO:teuthology.orchestra.run.smithi148.stdout: "avgtime": 0.000000000 2024-09-12T03:04:50.310 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-12T03:04:50.310 INFO:teuthology.orchestra.run.smithi148.stdout: } 2024-09-12T03:04:50.310 INFO:teuthology.orchestra.run.smithi148.stdout:} 2024-09-12T03:04:51.087 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:04:50 smithi148 ceph-mon[35090]: pgmap v533: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-12T03:04:51.114 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-09-12T03:04:51.114 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-09-12T03:04:51.115 INFO:tasks.check_counter:Daemon mds.l mds.dir_split=0 2024-09-12T03:04:51.115 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-09-12T03:04:51.126 INFO:tasks.kclient:Unmounting kernel clients... 2024-09-12T03:04:51.126 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-09-12T03:04:51.126 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:04:51.148 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2024-09-12T03:04:51.149 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-12T03:04:51.210 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph osd blocklist ls 2024-09-12T03:04:51.277 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:04:50 smithi005 ceph-mon[31197]: pgmap v533: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-12T03:04:51.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:04:50 smithi146 ceph-mon[35011]: pgmap v533: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-09-12T03:04:51.519 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:04:52.035 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:52.030+0000 7f28d5667640 1 -- 172.21.15.5:0/1837529962 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28d0103740 msgr2=0x7f28d0107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:52.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:52.030+0000 7f28d5667640 1 --2- 172.21.15.5:0/1837529962 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28d0103740 0x7f28d0107b80 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f28c4009930 tx=0x7f28c402a420 comp rx=0 tx=0).stop 2024-09-12T03:04:52.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:52.030+0000 7f28d5667640 1 -- 172.21.15.5:0/1837529962 shutdown_connections 2024-09-12T03:04:52.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:52.030+0000 7f28d5667640 1 --2- 172.21.15.5:0/1837529962 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28d0103740 0x7f28d0107b80 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:52.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:52.030+0000 7f28d5667640 1 --2- 172.21.15.5:0/1837529962 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28d0108290 0x7f28d010ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:52.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:52.030+0000 7f28d5667640 1 --2- 172.21.15.5:0/1837529962 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f28d0102d90 0x7f28d0103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:52.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:52.030+0000 7f28d5667640 1 -- 172.21.15.5:0/1837529962 >> 172.21.15.5:0/1837529962 conn(0x7f28d00fc7d0 msgr2=0x7f28d00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:52.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:52.030+0000 7f28d5667640 1 -- 172.21.15.5:0/1837529962 shutdown_connections 2024-09-12T03:04:52.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:52.030+0000 7f28d5667640 1 -- 172.21.15.5:0/1837529962 wait complete. 2024-09-12T03:04:52.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:52.030+0000 7f28d5667640 1 Processor -- start 2024-09-12T03:04:52.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:52.031+0000 7f28d5667640 1 -- start start 2024-09-12T03:04:52.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:52.031+0000 7f28d5667640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28d0102d90 0x7f28d01a0570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:52.036 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:52.031+0000 7f28d5667640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28d0103740 0x7f28d01a0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:52.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:52.031+0000 7f28d5667640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f28d0108290 0x7f28d019a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:52.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:52.031+0000 7f28d5667640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f28d0112090 con 0x7f28d0103740 2024-09-12T03:04:52.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:52.031+0000 7f28d5667640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f28d0111f10 con 0x7f28d0108290 2024-09-12T03:04:52.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:52.031+0000 7f28d5667640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f28d0112210 con 0x7f28d0102d90 2024-09-12T03:04:52.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:52.031+0000 7f28ceffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28d0102d90 0x7f28d01a0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:52.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:52.031+0000 7f28ce7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28d0103740 0x7f28d01a0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:52.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:52.031+0000 7f28ceffd640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28d0102d90 0x7f28d01a0570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.148:3300/0 says I am v2:172.21.15.5:35952/0 (socket says 172.21.15.5:35952) 2024-09-12T03:04:52.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:52.031+0000 7f28ce7fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28d0103740 0x7f28d01a0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46702/0 (socket says 172.21.15.5:46702) 2024-09-12T03:04:52.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:52.031+0000 7f28ceffd640 1 -- 172.21.15.5:0/1559838231 learned_addr learned my addr 172.21.15.5:0/1559838231 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:04:52.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:52.031+0000 7f28cf7fe640 1 --2- 172.21.15.5:0/1559838231 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f28d0108290 0x7f28d019a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:52.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:52.031+0000 7f28ceffd640 1 -- 172.21.15.5:0/1559838231 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f28d0108290 msgr2=0x7f28d019a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:52.037 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:52.031+0000 7f28ceffd640 1 --2- 172.21.15.5:0/1559838231 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f28d0108290 0x7f28d019a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:52.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:52.031+0000 7f28ceffd640 1 -- 172.21.15.5:0/1559838231 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28d0103740 msgr2=0x7f28d01a0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:52.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:52.031+0000 7f28ceffd640 1 --2- 172.21.15.5:0/1559838231 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28d0103740 0x7f28d01a0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:52.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:52.031+0000 7f28ceffd640 1 -- 172.21.15.5:0/1559838231 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28d019aeb0 con 0x7f28d0102d90 2024-09-12T03:04:52.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:52.031+0000 7f28ce7fc640 1 --2- 172.21.15.5:0/1559838231 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28d0103740 0x7f28d01a0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:04:52.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:52.032+0000 7f28ceffd640 1 --2- 172.21.15.5:0/1559838231 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28d0102d90 0x7f28d01a0570 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f28c000c890 tx=0x7f28c000cd50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:52.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:52.032+0000 7f28b7fff640 1 -- 172.21.15.5:0/1559838231 <== mon.1 v2:172.21.15.148:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c0004220 con 0x7f28d0102d90 2024-09-12T03:04:52.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:52.032+0000 7f28b7fff640 1 -- 172.21.15.5:0/1559838231 <== mon.1 v2:172.21.15.148:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28c00043c0 con 0x7f28d0102d90 2024-09-12T03:04:52.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:52.032+0000 7f28d5667640 1 -- 172.21.15.5:0/1559838231 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28d019b1a0 con 0x7f28d0102d90 2024-09-12T03:04:52.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:52.032+0000 7f28b7fff640 1 -- 172.21.15.5:0/1559838231 <== mon.1 v2:172.21.15.148:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c00049e0 con 0x7f28d0102d90 2024-09-12T03:04:52.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:52.032+0000 7f28d5667640 1 -- 172.21.15.5:0/1559838231 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28d01a74f0 con 0x7f28d0102d90 2024-09-12T03:04:52.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:52.033+0000 7f28d5667640 1 -- 172.21.15.5:0/1559838231 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28d006a7f0 con 0x7f28d0102d90 2024-09-12T03:04:52.038 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:52.034+0000 7f28b7fff640 1 -- 172.21.15.5:0/1559838231 <== mon.1 v2:172.21.15.148:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f28c0004c40 con 0x7f28d0102d90 2024-09-12T03:04:52.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:52.035+0000 7f28b7fff640 1 --2- 172.21.15.5:0/1559838231 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f28b80a2b80 0x7f28b80a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:52.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:52.035+0000 7f28ce7fc640 1 --2- 172.21.15.5:0/1559838231 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f28b80a2b80 0x7f28b80a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:52.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:52.035+0000 7f28b7fff640 1 -- 172.21.15.5:0/1559838231 <== mon.1 v2:172.21.15.148:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f28c00cf070 con 0x7f28d0102d90 2024-09-12T03:04:52.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:52.035+0000 7f28ce7fc640 1 --2- 172.21.15.5:0/1559838231 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f28b80a2b80 0x7f28b80a5040 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f28c4004770 tx=0x7f28c40058d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:52.039 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:52.039+0000 7f28b7fff640 1 -- 172.21.15.5:0/1559838231 <== mon.1 v2:172.21.15.148:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f28c0014030 con 0x7f28d0102d90 2024-09-12T03:04:52.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:52.229+0000 7f28d5667640 1 -- 172.21.15.5:0/1559838231 --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f28d019bb70 con 0x7f28d0102d90 2024-09-12T03:04:52.229 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:52.229+0000 7f28b7fff640 1 -- 172.21.15.5:0/1559838231 <== mon.1 v2:172.21.15.148:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v83) ==== 81+0+582 (secure 0 0 0) 0x7f28c009b930 con 0x7f28d0102d90 2024-09-12T03:04:52.230 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:6800/3125821798 2024-09-13T02:49:29.462216+0000 2024-09-12T03:04:52.230 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/3479911451 2024-09-13T02:49:29.462216+0000 2024-09-12T03:04:52.230 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/1547753207 2024-09-13T02:49:03.300872+0000 2024-09-12T03:04:52.230 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/1905762627 2024-09-13T02:49:03.300872+0000 2024-09-12T03:04:52.230 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/4142340280 2024-09-13T02:49:29.462216+0000 2024-09-12T03:04:52.230 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:6801/3125821798 2024-09-13T02:49:29.462216+0000 2024-09-12T03:04:52.230 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:6801/2595781680 2024-09-13T02:49:03.300872+0000 2024-09-12T03:04:52.230 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/2273691311 2024-09-13T02:49:29.462216+0000 2024-09-12T03:04:52.230 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:6800/2595781680 2024-09-13T02:49:03.300872+0000 2024-09-12T03:04:52.230 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/1196201217 2024-09-13T02:49:03.300872+0000 2024-09-12T03:04:52.231 INFO:teuthology.orchestra.run.smithi005.stderr:listed 10 entries 2024-09-12T03:04:52.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:52.232+0000 7f28d5667640 1 -- 172.21.15.5:0/1559838231 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f28b80a2b80 msgr2=0x7f28b80a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:52.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:52.232+0000 7f28d5667640 1 --2- 172.21.15.5:0/1559838231 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f28b80a2b80 0x7f28b80a5040 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f28c4004770 tx=0x7f28c40058d0 comp rx=0 tx=0).stop 2024-09-12T03:04:52.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:52.232+0000 7f28d5667640 1 -- 172.21.15.5:0/1559838231 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28d0102d90 msgr2=0x7f28d01a0570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:52.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:52.232+0000 7f28d5667640 1 --2- 172.21.15.5:0/1559838231 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28d0102d90 0x7f28d01a0570 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f28c000c890 tx=0x7f28c000cd50 comp rx=0 tx=0).stop 2024-09-12T03:04:52.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:52.232+0000 7f28d5667640 1 -- 172.21.15.5:0/1559838231 shutdown_connections 2024-09-12T03:04:52.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:52.232+0000 7f28d5667640 1 --2- 172.21.15.5:0/1559838231 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f28b80a2b80 0x7f28b80a5040 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:52.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:52.232+0000 7f28d5667640 1 --2- 172.21.15.5:0/1559838231 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28d0103740 0x7f28d01a0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:52.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:52.232+0000 7f28d5667640 1 --2- 172.21.15.5:0/1559838231 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f28d0108290 0x7f28d019a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:52.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:52.232+0000 7f28d5667640 1 --2- 172.21.15.5:0/1559838231 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f28d0102d90 0x7f28d01a0570 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:52.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:52.232+0000 7f28d5667640 1 -- 172.21.15.5:0/1559838231 >> 172.21.15.5:0/1559838231 conn(0x7f28d00fc7d0 msgr2=0x7f28d0100da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:52.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:52.232+0000 7f28d5667640 1 -- 172.21.15.5:0/1559838231 shutdown_connections 2024-09-12T03:04:52.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:52.232+0000 7f28d5667640 1 -- 172.21.15.5:0/1559838231 wait complete. 2024-09-12T03:04:53.001 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-12T03:04:53.001 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:04:53.001 DEBUG:teuthology.orchestra.run.smithi005:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-12T03:04:53.089 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi005.front.sepia.ceph.com 2024-09-12T03:04:53.089 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:04:53.089 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-12T03:04:53.133 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:04:53.133 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list 2024-09-12T03:04:53.169 INFO:teuthology.orchestra.run.smithi005.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-12T03:04:53.170 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:04:53.170 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-12T03:04:53.239 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:04:52 smithi005 ceph-mon[31197]: pgmap v534: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T03:04:53.239 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:04:52 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/1559838231' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-12T03:04:53.243 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-12T03:04:53.243 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link delete ceph-brx 2024-09-12T03:04:53.335 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-12T03:04:53.346 INFO:tasks.cephadm:Teardown begin 2024-09-12T03:04:53.347 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-12T03:04:53.347 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:04:53.415 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:04:52 smithi148 ceph-mon[35090]: pgmap v534: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T03:04:53.415 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:04:52 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/1559838231' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-12T03:04:53.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:04:52 smithi146 ceph-mon[35011]: pgmap v534: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-09-12T03:04:53.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:04:52 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/1559838231' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-12T03:04:53.605 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:04:54.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:54.062+0000 7f9875054640 1 -- 172.21.15.5:0/399233817 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9870102790 msgr2=0x7f98701067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:54.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:54.062+0000 7f9875054640 1 --2- 172.21.15.5:0/399233817 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9870102790 0x7f98701067e0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f9864009900 tx=0x7f986402a410 comp rx=0 tx=0).stop 2024-09-12T03:04:54.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:54.062+0000 7f9875054640 1 -- 172.21.15.5:0/399233817 shutdown_connections 2024-09-12T03:04:54.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:54.062+0000 7f9875054640 1 --2- 172.21.15.5:0/399233817 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9870102790 0x7f98701067e0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:54.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:54.062+0000 7f9875054640 1 --2- 172.21.15.5:0/399233817 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9870106f10 0x7f987010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:54.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:54.062+0000 7f9875054640 1 --2- 172.21.15.5:0/399233817 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9870101de0 0x7f98701021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:54.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:54.062+0000 7f9875054640 1 -- 172.21.15.5:0/399233817 >> 172.21.15.5:0/399233817 conn(0x7f98700fd8c0 msgr2=0x7f98700ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:54.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:54.062+0000 7f9875054640 1 -- 172.21.15.5:0/399233817 shutdown_connections 2024-09-12T03:04:54.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:54.062+0000 7f9875054640 1 -- 172.21.15.5:0/399233817 wait complete. 2024-09-12T03:04:54.064 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:54.063+0000 7f9875054640 1 Processor -- start 2024-09-12T03:04:54.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:54.063+0000 7f9875054640 1 -- start start 2024-09-12T03:04:54.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:54.063+0000 7f9875054640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9870101de0 0x7f9870198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:54.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:54.063+0000 7f9875054640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9870102790 0x7f98701989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:54.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:54.063+0000 7f9875054640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9870106f10 0x7f987019cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:54.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:54.063+0000 7f9875054640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f9870111f90 con 0x7f9870106f10 2024-09-12T03:04:54.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:54.063+0000 7f9875054640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f9870111e10 con 0x7f9870102790 2024-09-12T03:04:54.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:54.063+0000 7f9875054640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f9870112110 con 0x7f9870101de0 2024-09-12T03:04:54.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:54.063+0000 7f986f59e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9870106f10 0x7f987019cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:54.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:54.063+0000 7f986e59c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9870102790 0x7f98701989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:54.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:54.063+0000 7f986e59c640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9870102790 0x7f98701989d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.146:3300/0 says I am v2:172.21.15.5:39428/0 (socket says 172.21.15.5:39428) 2024-09-12T03:04:54.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:54.063+0000 7f986e59c640 1 -- 172.21.15.5:0/220264713 learned_addr learned my addr 172.21.15.5:0/220264713 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:04:54.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:54.063+0000 7f986ed9d640 1 --2- 172.21.15.5:0/220264713 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9870101de0 0x7f9870198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:54.065 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:54.064+0000 7f986e59c640 1 -- 172.21.15.5:0/220264713 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9870101de0 msgr2=0x7f9870198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:54.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:54.064+0000 7f986e59c640 1 --2- 172.21.15.5:0/220264713 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9870101de0 0x7f9870198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:54.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:54.064+0000 7f986e59c640 1 -- 172.21.15.5:0/220264713 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9870106f10 msgr2=0x7f987019cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:54.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:54.064+0000 7f986e59c640 1 --2- 172.21.15.5:0/220264713 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9870106f10 0x7f987019cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:54.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:54.064+0000 7f986e59c640 1 -- 172.21.15.5:0/220264713 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f987019d440 con 0x7f9870102790 2024-09-12T03:04:54.066 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:54.064+0000 7f986f59e640 1 --2- 172.21.15.5:0/220264713 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9870106f10 0x7f987019cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-12T03:04:54.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:54.064+0000 7f986e59c640 1 --2- 172.21.15.5:0/220264713 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9870102790 0x7f98701989d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f9864006010 tx=0x7f98640042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:54.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:54.064+0000 7f9857fff640 1 -- 172.21.15.5:0/220264713 <== mon.2 v2:172.21.15.146:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f986403f030 con 0x7f9870102790 2024-09-12T03:04:54.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:54.064+0000 7f9875054640 1 -- 172.21.15.5:0/220264713 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f987019d6d0 con 0x7f9870102790 2024-09-12T03:04:54.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:54.064+0000 7f9857fff640 1 -- 172.21.15.5:0/220264713 <== mon.2 v2:172.21.15.146:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9864003ce0 con 0x7f9870102790 2024-09-12T03:04:54.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:54.064+0000 7f9857fff640 1 -- 172.21.15.5:0/220264713 <== mon.2 v2:172.21.15.146:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9864035070 con 0x7f9870102790 2024-09-12T03:04:54.067 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:54.064+0000 7f9875054640 1 -- 172.21.15.5:0/220264713 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98701a5010 con 0x7f9870102790 2024-09-12T03:04:54.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:54.065+0000 7f9875054640 1 -- 172.21.15.5:0/220264713 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98701021c0 con 0x7f9870102790 2024-09-12T03:04:54.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:54.066+0000 7f9857fff640 1 -- 172.21.15.5:0/220264713 <== mon.2 v2:172.21.15.146:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f9864042030 con 0x7f9870102790 2024-09-12T03:04:54.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:54.067+0000 7f9857fff640 1 --2- 172.21.15.5:0/220264713 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f98580a2b80 0x7f98580a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:54.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:54.067+0000 7f986ed9d640 1 --2- 172.21.15.5:0/220264713 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f98580a2b80 0x7f98580a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:54.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:54.067+0000 7f9857fff640 1 -- 172.21.15.5:0/220264713 <== mon.2 v2:172.21.15.146:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f98640ef6a0 con 0x7f9870102790 2024-09-12T03:04:54.068 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:54.068+0000 7f986ed9d640 1 --2- 172.21.15.5:0/220264713 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f98580a2b80 0x7f98580a5040 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f9860005d00 tx=0x7f986000b410 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:54.072 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:54.071+0000 7f9857fff640 1 -- 172.21.15.5:0/220264713 <== mon.2 v2:172.21.15.146:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f98640bc020 con 0x7f9870102790 2024-09-12T03:04:54.285 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:54.285+0000 7f9875054640 1 -- 172.21.15.5:0/220264713 --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f98701997f0 con 0x7f9870102790 2024-09-12T03:04:54.286 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:54.286+0000 7f9857fff640 1 -- 172.21.15.5:0/220264713 <== mon.2 v2:172.21.15.146:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11969 (secure 0 0 0) 0x7f98701997f0 con 0x7f9870102790 2024-09-12T03:04:54.286 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-12T03:04:54.286 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":33,"btime":"2024-09-12T03:03:57:799392+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/4135232521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":4135232521},{"type":"v1","addr":"172.21.15.5:6835","nonce":4135232521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/4290221940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":4290221940},{"type":"v1","addr":"172.21.15.5:6837","nonce":4290221940}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T02:56:16.595562+0000","modified":"2024-09-12T03:03:57.585077+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24502,"mds_1":24487,"mds_2":24481,"mds_3":24479,"mds_4":24458},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/26447341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":26447341},{"type":"v1","addr":"172.21.15.5:6839","nonce":26447341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2393443295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2393443295},{"type":"v1","addr":"172.21.15.5:6841","nonce":2393443295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6833/3686970137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3686970137},{"type":"v1","addr":"172.21.15.146:6833","nonce":3686970137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6833/568512209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":568512209},{"type":"v1","addr":"172.21.15.148:6833","nonce":568512209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6835/2540740420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2540740420},{"type":"v1","addr":"172.21.15.146:6835","nonce":2540740420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.146:6837/1673370937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":1673370937},{"type":"v1","addr":"172.21.15.146:6837","nonce":1673370937}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":3,"incarnation":22,"state":"up:active","state_seq":6,"addr":"172.21.15.146:6839/3036767228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":3036767228},{"type":"v1","addr":"172.21.15.146:6839","nonce":3036767228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":2,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.148:6835/4195086427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":4195086427},{"type":"v1","addr":"172.21.15.148:6835","nonce":4195086427}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.148:6837/2606873922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2606873922},{"type":"v1","addr":"172.21.15.148:6837","nonce":2606873922}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6839/2317600000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":2317600000},{"type":"v1","addr":"172.21.15.148:6839","nonce":2317600000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24502,"qdb_cluster":[24458,24479,24481,24487,24502]},"id":1}]} 2024-09-12T03:04:54.287 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 33 2024-09-12T03:04:54.288 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:54.288+0000 7f9875054640 1 -- 172.21.15.5:0/220264713 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f98580a2b80 msgr2=0x7f98580a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:54.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:54.288+0000 7f9875054640 1 --2- 172.21.15.5:0/220264713 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f98580a2b80 0x7f98580a5040 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f9860005d00 tx=0x7f986000b410 comp rx=0 tx=0).stop 2024-09-12T03:04:54.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:54.288+0000 7f9875054640 1 -- 172.21.15.5:0/220264713 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9870102790 msgr2=0x7f98701989d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:54.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:54.288+0000 7f9875054640 1 --2- 172.21.15.5:0/220264713 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9870102790 0x7f98701989d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f9864006010 tx=0x7f98640042f0 comp rx=0 tx=0).stop 2024-09-12T03:04:54.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:54.288+0000 7f9875054640 1 -- 172.21.15.5:0/220264713 shutdown_connections 2024-09-12T03:04:54.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:54.288+0000 7f9875054640 1 --2- 172.21.15.5:0/220264713 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f98580a2b80 0x7f98580a5040 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:54.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:54.288+0000 7f9875054640 1 --2- 172.21.15.5:0/220264713 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9870106f10 0x7f987019cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:54.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:54.288+0000 7f9875054640 1 --2- 172.21.15.5:0/220264713 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f9870102790 0x7f98701989d0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:54.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:54.288+0000 7f9875054640 1 --2- 172.21.15.5:0/220264713 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f9870101de0 0x7f9870198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:54.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:54.288+0000 7f9875054640 1 -- 172.21.15.5:0/220264713 >> 172.21.15.5:0/220264713 conn(0x7f98700fd8c0 msgr2=0x7f98700fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:54.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:54.288+0000 7f9875054640 1 -- 172.21.15.5:0/220264713 shutdown_connections 2024-09-12T03:04:54.289 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:54.288+0000 7f9875054640 1 -- 172.21.15.5:0/220264713 wait complete. 2024-09-12T03:04:54.946 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-12T03:04:55.216 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:04:55.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:04:54 smithi005 ceph-mon[31197]: pgmap v535: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-12T03:04:55.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:04:54 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/220264713' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:04:55.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:04:54 smithi148 ceph-mon[35090]: pgmap v535: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-12T03:04:55.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:04:54 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/220264713' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:04:55.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:04:54 smithi146 ceph-mon[35011]: pgmap v535: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-09-12T03:04:55.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:04:54 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/220264713' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:04:55.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:55.694+0000 7fcf3792d640 1 -- 172.21.15.5:0/3078240988 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf30106f50 msgr2=0x7fcf3010eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:55.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:55.694+0000 7fcf3792d640 1 --2- 172.21.15.5:0/3078240988 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf30106f50 0x7fcf3010eaf0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fcf28009930 tx=0x7fcf2802a420 comp rx=0 tx=0).stop 2024-09-12T03:04:55.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:55.694+0000 7fcf3792d640 1 -- 172.21.15.5:0/3078240988 shutdown_connections 2024-09-12T03:04:55.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:55.694+0000 7fcf3792d640 1 --2- 172.21.15.5:0/3078240988 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf30106f50 0x7fcf3010eaf0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:55.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:55.694+0000 7fcf3792d640 1 --2- 172.21.15.5:0/3078240988 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcf30100ca0 0x7fcf30106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:55.695 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:55.694+0000 7fcf3792d640 1 --2- 172.21.15.5:0/3078240988 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcf301002f0 0x7fcf301006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:55.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:55.694+0000 7fcf3792d640 1 -- 172.21.15.5:0/3078240988 >> 172.21.15.5:0/3078240988 conn(0x7fcf30077210 msgr2=0x7fcf300fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:55.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:55.694+0000 7fcf3792d640 1 -- 172.21.15.5:0/3078240988 shutdown_connections 2024-09-12T03:04:55.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:55.694+0000 7fcf3792d640 1 -- 172.21.15.5:0/3078240988 wait complete. 2024-09-12T03:04:55.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:55.695+0000 7fcf3792d640 1 Processor -- start 2024-09-12T03:04:55.696 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:55.695+0000 7fcf3792d640 1 -- start start 2024-09-12T03:04:55.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:55.695+0000 7fcf3792d640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcf301002f0 0x7fcf30101270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:55.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:55.695+0000 7fcf3792d640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcf30100ca0 0x7fcf301017b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:55.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:55.695+0000 7fcf3792d640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf30106f50 0x7fcf30101cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:55.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:55.695+0000 7fcf3792d640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7fcf30111fa0 con 0x7fcf30106f50 2024-09-12T03:04:55.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:55.695+0000 7fcf3792d640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7fcf30111e20 con 0x7fcf30100ca0 2024-09-12T03:04:55.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:55.695+0000 7fcf3792d640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7fcf30112120 con 0x7fcf301002f0 2024-09-12T03:04:55.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:55.695+0000 7fcf35eca640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf30106f50 0x7fcf30101cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:55.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:55.695+0000 7fcf35eca640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf30106f50 0x7fcf30101cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59928/0 (socket says 172.21.15.5:59928) 2024-09-12T03:04:55.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:55.695+0000 7fcf35eca640 1 -- 172.21.15.5:0/949214090 learned_addr learned my addr 172.21.15.5:0/949214090 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:04:55.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:55.695+0000 7fcf356c9640 1 --2- 172.21.15.5:0/949214090 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcf301002f0 0x7fcf30101270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:55.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:55.696+0000 7fcf34ec8640 1 --2- 172.21.15.5:0/949214090 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcf30100ca0 0x7fcf301017b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:55.698 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:55.696+0000 7fcf35eca640 1 -- 172.21.15.5:0/949214090 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcf301002f0 msgr2=0x7fcf30101270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:55.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:55.696+0000 7fcf35eca640 1 --2- 172.21.15.5:0/949214090 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcf301002f0 0x7fcf30101270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:55.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:55.696+0000 7fcf35eca640 1 -- 172.21.15.5:0/949214090 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcf30100ca0 msgr2=0x7fcf301017b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:55.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:55.696+0000 7fcf35eca640 1 --2- 172.21.15.5:0/949214090 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcf30100ca0 0x7fcf301017b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:55.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:55.696+0000 7fcf35eca640 1 -- 172.21.15.5:0/949214090 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf301a6920 con 0x7fcf30106f50 2024-09-12T03:04:55.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:55.696+0000 7fcf356c9640 1 --2- 172.21.15.5:0/949214090 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcf301002f0 0x7fcf30101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:04:55.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:55.696+0000 7fcf35eca640 1 --2- 172.21.15.5:0/949214090 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf30106f50 0x7fcf30101cf0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fcf28005ab0 tx=0x7fcf280042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:55.699 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:55.696+0000 7fcf2e7fc640 1 -- 172.21.15.5:0/949214090 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf2803f030 con 0x7fcf30106f50 2024-09-12T03:04:55.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:55.696+0000 7fcf3792d640 1 -- 172.21.15.5:0/949214090 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf301a6bb0 con 0x7fcf30106f50 2024-09-12T03:04:55.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:55.696+0000 7fcf2e7fc640 1 -- 172.21.15.5:0/949214090 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf28003ce0 con 0x7fcf30106f50 2024-09-12T03:04:55.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:55.696+0000 7fcf2e7fc640 1 -- 172.21.15.5:0/949214090 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf28035070 con 0x7fcf30106f50 2024-09-12T03:04:55.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:55.696+0000 7fcf3792d640 1 -- 172.21.15.5:0/949214090 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf301a7090 con 0x7fcf30106f50 2024-09-12T03:04:55.700 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:55.697+0000 7fcf3792d640 1 -- 172.21.15.5:0/949214090 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf301006d0 con 0x7fcf30106f50 2024-09-12T03:04:55.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:55.698+0000 7fcf2e7fc640 1 -- 172.21.15.5:0/949214090 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7fcf28042030 con 0x7fcf30106f50 2024-09-12T03:04:55.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:55.699+0000 7fcf2e7fc640 1 --2- 172.21.15.5:0/949214090 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fcf180a2b80 0x7fcf180a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:55.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:55.699+0000 7fcf356c9640 1 --2- 172.21.15.5:0/949214090 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fcf180a2b80 0x7fcf180a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:55.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:55.699+0000 7fcf2e7fc640 1 -- 172.21.15.5:0/949214090 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7fcf280efa40 con 0x7fcf30106f50 2024-09-12T03:04:55.701 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:55.700+0000 7fcf356c9640 1 --2- 172.21.15.5:0/949214090 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fcf180a2b80 0x7fcf180a5040 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fcf20005fd0 tx=0x7fcf20009970 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:55.702 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:55.702+0000 7fcf2e7fc640 1 -- 172.21.15.5:0/949214090 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcf280bc950 con 0x7fcf30106f50 2024-09-12T03:04:55.914 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:55.914+0000 7fcf3792d640 1 -- 172.21.15.5:0/949214090 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcf3010a120 con 0x7fcf30106f50 2024-09-12T03:04:55.915 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:55.915+0000 7fcf2e7fc640 1 -- 172.21.15.5:0/949214090 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11969 (secure 0 0 0) 0x7fcf280bc250 con 0x7fcf30106f50 2024-09-12T03:04:55.915 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-12T03:04:55.915 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":33,"btime":"2024-09-12T03:03:57:799392+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/4135232521","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":4135232521},{"type":"v1","addr":"172.21.15.5:6835","nonce":4135232521}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/4290221940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":4290221940},{"type":"v1","addr":"172.21.15.5:6837","nonce":4290221940}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-12T02:56:16.595562+0000","modified":"2024-09-12T03:03:57.585077+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24502,"mds_1":24487,"mds_2":24481,"mds_3":24479,"mds_4":24458},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14601":{"gid":14601,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6839/26447341","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":26447341},{"type":"v1","addr":"172.21.15.5:6839","nonce":26447341}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14613":{"gid":14613,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.5:6841/2393443295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2393443295},{"type":"v1","addr":"172.21.15.5:6841","nonce":2393443295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6833/3686970137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6832","nonce":3686970137},{"type":"v1","addr":"172.21.15.146:6833","nonce":3686970137}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24439":{"gid":24439,"name":"l","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.148:6833/568512209","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6832","nonce":568512209},{"type":"v1","addr":"172.21.15.148:6833","nonce":568512209}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.146:6835/2540740420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6834","nonce":2540740420},{"type":"v1","addr":"172.21.15.146:6835","nonce":2540740420}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24458":{"gid":24458,"name":"h","rank":4,"incarnation":24,"state":"up:active","state_seq":9,"addr":"172.21.15.146:6837/1673370937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6836","nonce":1673370937},{"type":"v1","addr":"172.21.15.146:6837","nonce":1673370937}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24479":{"gid":24479,"name":"b","rank":3,"incarnation":22,"state":"up:active","state_seq":6,"addr":"172.21.15.146:6839/3036767228","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6838","nonce":3036767228},{"type":"v1","addr":"172.21.15.146:6839","nonce":3036767228}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24481":{"gid":24481,"name":"f","rank":2,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.148:6835/4195086427","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6834","nonce":4195086427},{"type":"v1","addr":"172.21.15.148:6835","nonce":4195086427}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24487":{"gid":24487,"name":"c","rank":1,"incarnation":18,"state":"up:active","state_seq":9,"addr":"172.21.15.148:6837/2606873922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6836","nonce":2606873922},{"type":"v1","addr":"172.21.15.148:6837","nonce":2606873922}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24502":{"gid":24502,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.148:6839/2317600000","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6838","nonce":2317600000},{"type":"v1","addr":"172.21.15.148:6839","nonce":2317600000}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24502,"qdb_cluster":[24458,24479,24481,24487,24502]},"id":1}]} 2024-09-12T03:04:55.916 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 33 2024-09-12T03:04:55.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:55.917+0000 7fcf3792d640 1 -- 172.21.15.5:0/949214090 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fcf180a2b80 msgr2=0x7fcf180a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:55.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:55.917+0000 7fcf3792d640 1 --2- 172.21.15.5:0/949214090 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fcf180a2b80 0x7fcf180a5040 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fcf20005fd0 tx=0x7fcf20009970 comp rx=0 tx=0).stop 2024-09-12T03:04:55.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:55.917+0000 7fcf3792d640 1 -- 172.21.15.5:0/949214090 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf30106f50 msgr2=0x7fcf30101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:55.917 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:55.917+0000 7fcf3792d640 1 --2- 172.21.15.5:0/949214090 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf30106f50 0x7fcf30101cf0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fcf28005ab0 tx=0x7fcf280042f0 comp rx=0 tx=0).stop 2024-09-12T03:04:55.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:55.917+0000 7fcf3792d640 1 -- 172.21.15.5:0/949214090 shutdown_connections 2024-09-12T03:04:55.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:55.917+0000 7fcf3792d640 1 --2- 172.21.15.5:0/949214090 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7fcf180a2b80 0x7fcf180a5040 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:55.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:55.917+0000 7fcf3792d640 1 --2- 172.21.15.5:0/949214090 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf30106f50 0x7fcf30101cf0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:55.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:55.917+0000 7fcf3792d640 1 --2- 172.21.15.5:0/949214090 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7fcf30100ca0 0x7fcf301017b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:55.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:55.917+0000 7fcf3792d640 1 --2- 172.21.15.5:0/949214090 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7fcf301002f0 0x7fcf30101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:55.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:55.917+0000 7fcf3792d640 1 -- 172.21.15.5:0/949214090 >> 172.21.15.5:0/949214090 conn(0x7fcf30077210 msgr2=0x7fcf30105810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:55.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:55.917+0000 7fcf3792d640 1 -- 172.21.15.5:0/949214090 shutdown_connections 2024-09-12T03:04:55.918 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:55.917+0000 7fcf3792d640 1 -- 172.21.15.5:0/949214090 wait complete. 2024-09-12T03:04:56.310 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:04:55 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/949214090' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:04:56.415 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:04:55 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/949214090' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:04:56.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:04:55 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/949214090' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-12T03:04:56.498 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-12T03:04:56.757 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:04:57.144 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:04:56 smithi005 ceph-mon[31197]: pgmap v536: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-12T03:04:57.168 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:04:56 smithi148 ceph-mon[35090]: pgmap v536: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-12T03:04:57.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:57.226+0000 7f5860061640 1 -- 172.21.15.5:0/1511890492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5858102790 msgr2=0x7f58581067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:57.227 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:57.226+0000 7f5860061640 1 --2- 172.21.15.5:0/1511890492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5858102790 0x7f58581067e0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f5848009900 tx=0x7f584802a410 comp rx=0 tx=0).stop 2024-09-12T03:04:57.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:57.226+0000 7f5860061640 1 -- 172.21.15.5:0/1511890492 shutdown_connections 2024-09-12T03:04:57.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:57.226+0000 7f5860061640 1 --2- 172.21.15.5:0/1511890492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5858102790 0x7f58581067e0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:57.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:57.226+0000 7f5860061640 1 --2- 172.21.15.5:0/1511890492 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5858106f10 0x7f585810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:57.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:57.226+0000 7f5860061640 1 --2- 172.21.15.5:0/1511890492 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5858101de0 0x7f58581021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:57.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:57.226+0000 7f5860061640 1 -- 172.21.15.5:0/1511890492 >> 172.21.15.5:0/1511890492 conn(0x7f58580fd8c0 msgr2=0x7f58580ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:57.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:57.227+0000 7f5860061640 1 -- 172.21.15.5:0/1511890492 shutdown_connections 2024-09-12T03:04:57.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:57.227+0000 7f5860061640 1 -- 172.21.15.5:0/1511890492 wait complete. 2024-09-12T03:04:57.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:57.227+0000 7f5860061640 1 Processor -- start 2024-09-12T03:04:57.228 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:57.227+0000 7f5860061640 1 -- start start 2024-09-12T03:04:57.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:57.228+0000 7f5860061640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5858101de0 0x7f58581983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:57.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:57.228+0000 7f5860061640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5858102790 0x7f5858198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:57.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:57.228+0000 7f5860061640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5858106f10 0x7f585819cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:57.231 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:57.228+0000 7f5860061640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5858111f70 con 0x7f5858102790 2024-09-12T03:04:57.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:57.228+0000 7f5860061640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f5858111df0 con 0x7f5858101de0 2024-09-12T03:04:57.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:57.228+0000 7f5860061640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f58581120f0 con 0x7f5858106f10 2024-09-12T03:04:57.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:57.228+0000 7f585d5fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5858102790 0x7f5858198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:57.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:57.228+0000 7f585d5fc640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5858102790 0x7f5858198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59942/0 (socket says 172.21.15.5:59942) 2024-09-12T03:04:57.232 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:57.228+0000 7f585d5fc640 1 -- 172.21.15.5:0/1139926005 learned_addr learned my addr 172.21.15.5:0/1139926005 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:04:57.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:57.228+0000 7f585ddfd640 1 --2- 172.21.15.5:0/1139926005 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5858101de0 0x7f58581983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:57.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:57.228+0000 7f585e5fe640 1 --2- 172.21.15.5:0/1139926005 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5858106f10 0x7f585819cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:57.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:57.228+0000 7f585d5fc640 1 -- 172.21.15.5:0/1139926005 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5858106f10 msgr2=0x7f585819cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:57.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:57.228+0000 7f585d5fc640 1 --2- 172.21.15.5:0/1139926005 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5858106f10 0x7f585819cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:57.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:57.228+0000 7f585d5fc640 1 -- 172.21.15.5:0/1139926005 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5858101de0 msgr2=0x7f58581983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:57.233 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:57.228+0000 7f585d5fc640 1 --2- 172.21.15.5:0/1139926005 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5858101de0 0x7f58581983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:57.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:57.228+0000 7f585d5fc640 1 -- 172.21.15.5:0/1139926005 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f585819d380 con 0x7f5858102790 2024-09-12T03:04:57.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:57.228+0000 7f585e5fe640 1 --2- 172.21.15.5:0/1139926005 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5858106f10 0x7f585819cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:04:57.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:57.228+0000 7f585d5fc640 1 --2- 172.21.15.5:0/1139926005 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5858102790 0x7f5858198910 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f5848005ab0 tx=0x7f58480042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:57.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:57.229+0000 7f584effd640 1 -- 172.21.15.5:0/1139926005 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f584803f030 con 0x7f5858102790 2024-09-12T03:04:57.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:57.229+0000 7f584effd640 1 -- 172.21.15.5:0/1139926005 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5848003ce0 con 0x7f5858102790 2024-09-12T03:04:57.234 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:57.229+0000 7f584effd640 1 -- 172.21.15.5:0/1139926005 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5848035070 con 0x7f5858102790 2024-09-12T03:04:57.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:57.229+0000 7f5860061640 1 -- 172.21.15.5:0/1139926005 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f585819d610 con 0x7f5858102790 2024-09-12T03:04:57.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:57.229+0000 7f5860061640 1 -- 172.21.15.5:0/1139926005 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58581a4f50 con 0x7f5858102790 2024-09-12T03:04:57.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:57.230+0000 7f5860061640 1 -- 172.21.15.5:0/1139926005 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58581032c0 con 0x7f5858102790 2024-09-12T03:04:57.235 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:57.231+0000 7f584effd640 1 -- 172.21.15.5:0/1139926005 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f5848042030 con 0x7f5858102790 2024-09-12T03:04:57.236 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:57.232+0000 7f584effd640 1 --2- 172.21.15.5:0/1139926005 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f58440a2b80 0x7f58440a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:57.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:57.232+0000 7f585ddfd640 1 --2- 172.21.15.5:0/1139926005 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f58440a2b80 0x7f58440a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:57.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:57.232+0000 7f584effd640 1 -- 172.21.15.5:0/1139926005 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f58480efba0 con 0x7f5858102790 2024-09-12T03:04:57.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:57.232+0000 7f585ddfd640 1 --2- 172.21.15.5:0/1139926005 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f58440a2b80 0x7f58440a5040 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f58540058c0 tx=0x7f585400a250 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:57.237 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:57.235+0000 7f584effd640 1 -- 172.21.15.5:0/1139926005 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f58480bcb10 con 0x7f5858102790 2024-09-12T03:04:57.414 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:57.413+0000 7f5860061640 1 -- 172.21.15.5:0/1139926005 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f585810a150 con 0x7f5858102790 2024-09-12T03:04:57.415 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:57.415+0000 7f584effd640 1 -- 172.21.15.5:0/1139926005 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+22656 (secure 0 0 0) 0x7f58480bc410 con 0x7f5858102790 2024-09-12T03:04:57.415 INFO:teuthology.orchestra.run.smithi005.stdout: 2024-09-12T03:04:57.416 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":83,"fsid":"618305f8-70b1-11ef-bcea-c7b262605968","created":"2024-09-12T02:48:42.352146+0000","modified":"2024-09-12T02:56:31.026079+0000","last_up_change":"2024-09-12T02:55:27.817161+0000","last_in_change":"2024-09-12T02:55:03.745281+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-12T02:51:47.970639+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-12T02:56:12.834271+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-12T02:56:14.951437+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"17da560a-6aa2-4f2c-a95f-67481dce4d3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":4248507573},{"type":"v1","addr":"172.21.15.5:6803","nonce":4248507573}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":4248507573},{"type":"v1","addr":"172.21.15.5:6805","nonce":4248507573}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":4248507573},{"type":"v1","addr":"172.21.15.5:6809","nonce":4248507573}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":4248507573},{"type":"v1","addr":"172.21.15.5:6807","nonce":4248507573}]},"public_addr":"172.21.15.5:6803/4248507573","cluster_addr":"172.21.15.5:6805/4248507573","heartbeat_back_addr":"172.21.15.5:6809/4248507573","heartbeat_front_addr":"172.21.15.5:6807/4248507573","state":["exists","up"]},{"osd":1,"uuid":"a3237006-99af-4fc1-860c-d4917032647f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6800","nonce":1105990287},{"type":"v1","addr":"172.21.15.146:6801","nonce":1105990287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6802","nonce":1105990287},{"type":"v1","addr":"172.21.15.146:6803","nonce":1105990287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6806","nonce":1105990287},{"type":"v1","addr":"172.21.15.146:6807","nonce":1105990287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6804","nonce":1105990287},{"type":"v1","addr":"172.21.15.146:6805","nonce":1105990287}]},"public_addr":"172.21.15.146:6801/1105990287","cluster_addr":"172.21.15.146:6803/1105990287","heartbeat_back_addr":"172.21.15.146:6807/1105990287","heartbeat_front_addr":"172.21.15.146:6805/1105990287","state":["exists","up"]},{"osd":2,"uuid":"ea3ab84a-7739-4975-8244-31ddd5aad80d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6800","nonce":3378174313},{"type":"v1","addr":"172.21.15.148:6801","nonce":3378174313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6802","nonce":3378174313},{"type":"v1","addr":"172.21.15.148:6803","nonce":3378174313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6806","nonce":3378174313},{"type":"v1","addr":"172.21.15.148:6807","nonce":3378174313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6804","nonce":3378174313},{"type":"v1","addr":"172.21.15.148:6805","nonce":3378174313}]},"public_addr":"172.21.15.148:6801/3378174313","cluster_addr":"172.21.15.148:6803/3378174313","heartbeat_back_addr":"172.21.15.148:6807/3378174313","heartbeat_front_addr":"172.21.15.148:6805/3378174313","state":["exists","up"]},{"osd":3,"uuid":"4396a9ee-f3f3-410b-9fb1-124f1e33a1f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":11741938},{"type":"v1","addr":"172.21.15.5:6811","nonce":11741938}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":11741938},{"type":"v1","addr":"172.21.15.5:6813","nonce":11741938}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":11741938},{"type":"v1","addr":"172.21.15.5:6817","nonce":11741938}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":11741938},{"type":"v1","addr":"172.21.15.5:6815","nonce":11741938}]},"public_addr":"172.21.15.5:6811/11741938","cluster_addr":"172.21.15.5:6813/11741938","heartbeat_back_addr":"172.21.15.5:6817/11741938","heartbeat_front_addr":"172.21.15.5:6815/11741938","state":["exists","up"]},{"osd":4,"uuid":"989d76e7-c742-49d0-994a-c58f2771b1d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6808","nonce":3274923851},{"type":"v1","addr":"172.21.15.146:6809","nonce":3274923851}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6810","nonce":3274923851},{"type":"v1","addr":"172.21.15.146:6811","nonce":3274923851}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6814","nonce":3274923851},{"type":"v1","addr":"172.21.15.146:6815","nonce":3274923851}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6812","nonce":3274923851},{"type":"v1","addr":"172.21.15.146:6813","nonce":3274923851}]},"public_addr":"172.21.15.146:6809/3274923851","cluster_addr":"172.21.15.146:6811/3274923851","heartbeat_back_addr":"172.21.15.146:6815/3274923851","heartbeat_front_addr":"172.21.15.146:6813/3274923851","state":["exists","up"]},{"osd":5,"uuid":"64ada7d4-70a0-4b83-9af8-032be544608d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6808","nonce":2771670476},{"type":"v1","addr":"172.21.15.148:6809","nonce":2771670476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6810","nonce":2771670476},{"type":"v1","addr":"172.21.15.148:6811","nonce":2771670476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6814","nonce":2771670476},{"type":"v1","addr":"172.21.15.148:6815","nonce":2771670476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6812","nonce":2771670476},{"type":"v1","addr":"172.21.15.148:6813","nonce":2771670476}]},"public_addr":"172.21.15.148:6809/2771670476","cluster_addr":"172.21.15.148:6811/2771670476","heartbeat_back_addr":"172.21.15.148:6815/2771670476","heartbeat_front_addr":"172.21.15.148:6813/2771670476","state":["exists","up"]},{"osd":6,"uuid":"a70a88fd-2e4b-480c-bb98-62636cf70cc6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3176073169},{"type":"v1","addr":"172.21.15.5:6819","nonce":3176073169}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":3176073169},{"type":"v1","addr":"172.21.15.5:6821","nonce":3176073169}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":3176073169},{"type":"v1","addr":"172.21.15.5:6825","nonce":3176073169}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3176073169},{"type":"v1","addr":"172.21.15.5:6823","nonce":3176073169}]},"public_addr":"172.21.15.5:6819/3176073169","cluster_addr":"172.21.15.5:6821/3176073169","heartbeat_back_addr":"172.21.15.5:6825/3176073169","heartbeat_front_addr":"172.21.15.5:6823/3176073169","state":["exists","up"]},{"osd":7,"uuid":"49451e8d-c741-4ad5-b8f5-89bad8d4d129","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6816","nonce":2209508838},{"type":"v1","addr":"172.21.15.146:6817","nonce":2209508838}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6818","nonce":2209508838},{"type":"v1","addr":"172.21.15.146:6819","nonce":2209508838}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6822","nonce":2209508838},{"type":"v1","addr":"172.21.15.146:6823","nonce":2209508838}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6820","nonce":2209508838},{"type":"v1","addr":"172.21.15.146:6821","nonce":2209508838}]},"public_addr":"172.21.15.146:6817/2209508838","cluster_addr":"172.21.15.146:6819/2209508838","heartbeat_back_addr":"172.21.15.146:6823/2209508838","heartbeat_front_addr":"172.21.15.146:6821/2209508838","state":["exists","up"]},{"osd":8,"uuid":"57bfc121-a75e-4a7f-b1dc-64074b178b03","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6816","nonce":651045756},{"type":"v1","addr":"172.21.15.148:6817","nonce":651045756}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6818","nonce":651045756},{"type":"v1","addr":"172.21.15.148:6819","nonce":651045756}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6822","nonce":651045756},{"type":"v1","addr":"172.21.15.148:6823","nonce":651045756}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6820","nonce":651045756},{"type":"v1","addr":"172.21.15.148:6821","nonce":651045756}]},"public_addr":"172.21.15.148:6817/651045756","cluster_addr":"172.21.15.148:6819/651045756","heartbeat_back_addr":"172.21.15.148:6823/651045756","heartbeat_front_addr":"172.21.15.148:6821/651045756","state":["exists","up"]},{"osd":9,"uuid":"d9726168-7494-4bf8-b433-350e85bfd461","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":1961857360},{"type":"v1","addr":"172.21.15.5:6827","nonce":1961857360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":1961857360},{"type":"v1","addr":"172.21.15.5:6829","nonce":1961857360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":1961857360},{"type":"v1","addr":"172.21.15.5:6833","nonce":1961857360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":1961857360},{"type":"v1","addr":"172.21.15.5:6831","nonce":1961857360}]},"public_addr":"172.21.15.5:6827/1961857360","cluster_addr":"172.21.15.5:6829/1961857360","heartbeat_back_addr":"172.21.15.5:6833/1961857360","heartbeat_front_addr":"172.21.15.5:6831/1961857360","state":["exists","up"]},{"osd":10,"uuid":"519ec60f-3606-48ac-9043-6f59a798766d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6824","nonce":2041733103},{"type":"v1","addr":"172.21.15.146:6825","nonce":2041733103}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6826","nonce":2041733103},{"type":"v1","addr":"172.21.15.146:6827","nonce":2041733103}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6830","nonce":2041733103},{"type":"v1","addr":"172.21.15.146:6831","nonce":2041733103}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.146:6828","nonce":2041733103},{"type":"v1","addr":"172.21.15.146:6829","nonce":2041733103}]},"public_addr":"172.21.15.146:6825/2041733103","cluster_addr":"172.21.15.146:6827/2041733103","heartbeat_back_addr":"172.21.15.146:6831/2041733103","heartbeat_front_addr":"172.21.15.146:6829/2041733103","state":["exists","up"]},{"osd":11,"uuid":"0f92d0cb-ba54-4bc3-a6d0-3c54d910556a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6824","nonce":839290773},{"type":"v1","addr":"172.21.15.148:6825","nonce":839290773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6826","nonce":839290773},{"type":"v1","addr":"172.21.15.148:6827","nonce":839290773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6830","nonce":839290773},{"type":"v1","addr":"172.21.15.148:6831","nonce":839290773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.148:6828","nonce":839290773},{"type":"v1","addr":"172.21.15.148:6829","nonce":839290773}]},"public_addr":"172.21.15.148:6825/839290773","cluster_addr":"172.21.15.148:6827/839290773","heartbeat_back_addr":"172.21.15.148:6831/839290773","heartbeat_front_addr":"172.21.15.148:6829/839290773","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:50:54.101755+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:51:19.037385+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:51:44.142837+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:52:09.150887+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:52:33.705079+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:52:58.845233+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:53:23.068339+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:53:47.520398+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:54:11.965370+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:54:36.776075+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:55:01.292236+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-12T02:55:25.346589+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:6800/3125821798":"2024-09-13T02:49:29.462216+0000","172.21.15.5:0/3479911451":"2024-09-13T02:49:29.462216+0000","172.21.15.5:0/1547753207":"2024-09-13T02:49:03.300872+0000","172.21.15.5:0/1905762627":"2024-09-13T02:49:03.300872+0000","172.21.15.5:0/4142340280":"2024-09-13T02:49:29.462216+0000","172.21.15.5:6801/3125821798":"2024-09-13T02:49:29.462216+0000","172.21.15.5:6801/2595781680":"2024-09-13T02:49:03.300872+0000","172.21.15.5:0/2273691311":"2024-09-13T02:49:29.462216+0000","172.21.15.5:6800/2595781680":"2024-09-13T02:49:03.300872+0000","172.21.15.5:0/1196201217":"2024-09-13T02:49:03.300872+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-12T03:04:57.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:57.417+0000 7f5860061640 1 -- 172.21.15.5:0/1139926005 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f58440a2b80 msgr2=0x7f58440a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:57.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:57.417+0000 7f5860061640 1 --2- 172.21.15.5:0/1139926005 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f58440a2b80 0x7f58440a5040 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f58540058c0 tx=0x7f585400a250 comp rx=0 tx=0).stop 2024-09-12T03:04:57.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:57.417+0000 7f5860061640 1 -- 172.21.15.5:0/1139926005 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5858102790 msgr2=0x7f5858198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:57.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:57.417+0000 7f5860061640 1 --2- 172.21.15.5:0/1139926005 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5858102790 0x7f5858198910 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f5848005ab0 tx=0x7f58480042f0 comp rx=0 tx=0).stop 2024-09-12T03:04:57.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:57.417+0000 7f5860061640 1 -- 172.21.15.5:0/1139926005 shutdown_connections 2024-09-12T03:04:57.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:57.417+0000 7f5860061640 1 --2- 172.21.15.5:0/1139926005 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f58440a2b80 0x7f58440a5040 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:57.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:57.417+0000 7f5860061640 1 --2- 172.21.15.5:0/1139926005 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5858102790 0x7f5858198910 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:57.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:57.417+0000 7f5860061640 1 --2- 172.21.15.5:0/1139926005 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5858106f10 0x7f585819cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:57.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:57.417+0000 7f5860061640 1 --2- 172.21.15.5:0/1139926005 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5858101de0 0x7f58581983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:57.418 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:57.417+0000 7f5860061640 1 -- 172.21.15.5:0/1139926005 >> 172.21.15.5:0/1139926005 conn(0x7f58580fd8c0 msgr2=0x7f58580fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:57.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:57.418+0000 7f5860061640 1 -- 172.21.15.5:0/1139926005 shutdown_connections 2024-09-12T03:04:57.419 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:57.418+0000 7f5860061640 1 -- 172.21.15.5:0/1139926005 wait complete. 2024-09-12T03:04:57.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:04:56 smithi146 ceph-mon[35011]: pgmap v536: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 597 B/s wr, 11 op/s 2024-09-12T03:04:58.174 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:04:57 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/1139926005' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:04:58.206 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph fs fail cephfs 2024-09-12T03:04:58.415 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:04:57 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/1139926005' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:04:58.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:04:57 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/1139926005' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-12T03:04:58.468 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:04:58.935 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:58.925+0000 7f55c26d3640 1 -- 172.21.15.5:0/3760644098 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55bc106a80 msgr2=0x7f55bc106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:58.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:58.925+0000 7f55c26d3640 1 --2- 172.21.15.5:0/3760644098 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55bc106a80 0x7f55bc106e60 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f55ac009930 tx=0x7f55ac02a420 comp rx=0 tx=0).stop 2024-09-12T03:04:58.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:58.925+0000 7f55c26d3640 1 -- 172.21.15.5:0/3760644098 shutdown_connections 2024-09-12T03:04:58.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:58.925+0000 7f55c26d3640 1 --2- 172.21.15.5:0/3760644098 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55bc101420 0x7f55bc10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:58.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:58.925+0000 7f55c26d3640 1 --2- 172.21.15.5:0/3760644098 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f55bc100a80 0x7f55bc100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:58.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:58.925+0000 7f55c26d3640 1 --2- 172.21.15.5:0/3760644098 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55bc106a80 0x7f55bc106e60 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:58.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:58.925+0000 7f55c26d3640 1 -- 172.21.15.5:0/3760644098 >> 172.21.15.5:0/3760644098 conn(0x7f55bc0fc7a0 msgr2=0x7f55bc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:58.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:58.926+0000 7f55c26d3640 1 -- 172.21.15.5:0/3760644098 shutdown_connections 2024-09-12T03:04:58.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:58.926+0000 7f55c26d3640 1 -- 172.21.15.5:0/3760644098 wait complete. 2024-09-12T03:04:58.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:58.926+0000 7f55c26d3640 1 Processor -- start 2024-09-12T03:04:58.936 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:58.926+0000 7f55c26d3640 1 -- start start 2024-09-12T03:04:58.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:58.927+0000 7f55c26d3640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f55bc100a80 0x7f55bc1984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:58.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:58.927+0000 7f55c26d3640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55bc101420 0x7f55bc1989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:58.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:58.927+0000 7f55c26d3640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55bc106a80 0x7f55bc19cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:58.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:58.927+0000 7f55c26d3640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f55bc111fa0 con 0x7f55bc101420 2024-09-12T03:04:58.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:58.927+0000 7f55c26d3640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f55bc111e20 con 0x7f55bc100a80 2024-09-12T03:04:58.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:58.927+0000 7f55c26d3640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f55bc112120 con 0x7f55bc106a80 2024-09-12T03:04:58.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:58.927+0000 7f55bb7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55bc101420 0x7f55bc1989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:58.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:58.927+0000 7f55bb7fe640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55bc101420 0x7f55bc1989e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59954/0 (socket says 172.21.15.5:59954) 2024-09-12T03:04:58.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:58.927+0000 7f55bb7fe640 1 -- 172.21.15.5:0/87838218 learned_addr learned my addr 172.21.15.5:0/87838218 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:04:58.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:58.927+0000 7f55bbfff640 1 --2- 172.21.15.5:0/87838218 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f55bc100a80 0x7f55bc1984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:58.937 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:58.927+0000 7f55c0c70640 1 --2- 172.21.15.5:0/87838218 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55bc106a80 0x7f55bc19cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:58.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:58.927+0000 7f55bb7fe640 1 -- 172.21.15.5:0/87838218 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55bc106a80 msgr2=0x7f55bc19cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:58.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:58.927+0000 7f55bb7fe640 1 --2- 172.21.15.5:0/87838218 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55bc106a80 0x7f55bc19cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:58.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:58.927+0000 7f55bb7fe640 1 -- 172.21.15.5:0/87838218 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f55bc100a80 msgr2=0x7f55bc1984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:58.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:58.927+0000 7f55bb7fe640 1 --2- 172.21.15.5:0/87838218 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f55bc100a80 0x7f55bc1984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:58.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:58.927+0000 7f55bb7fe640 1 -- 172.21.15.5:0/87838218 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55bc19d450 con 0x7f55bc101420 2024-09-12T03:04:58.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:58.927+0000 7f55c0c70640 1 --2- 172.21.15.5:0/87838218 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55bc106a80 0x7f55bc19cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:04:58.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:58.928+0000 7f55bb7fe640 1 --2- 172.21.15.5:0/87838218 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55bc101420 0x7f55bc1989e0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f55b000c890 tx=0x7f55b000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:58.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:58.928+0000 7f55b97fa640 1 -- 172.21.15.5:0/87838218 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b0004220 con 0x7f55bc101420 2024-09-12T03:04:58.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:58.928+0000 7f55c26d3640 1 -- 172.21.15.5:0/87838218 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55bc19d740 con 0x7f55bc101420 2024-09-12T03:04:58.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:58.928+0000 7f55b97fa640 1 -- 172.21.15.5:0/87838218 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55b00043c0 con 0x7f55bc101420 2024-09-12T03:04:58.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:58.928+0000 7f55b97fa640 1 -- 172.21.15.5:0/87838218 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b0004a00 con 0x7f55bc101420 2024-09-12T03:04:58.938 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:58.928+0000 7f55c26d3640 1 -- 172.21.15.5:0/87838218 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55bc1a5020 con 0x7f55bc101420 2024-09-12T03:04:58.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:58.929+0000 7f55c26d3640 1 -- 172.21.15.5:0/87838218 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55bc100ee0 con 0x7f55bc101420 2024-09-12T03:04:58.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:58.930+0000 7f55b97fa640 1 -- 172.21.15.5:0/87838218 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f55b00079b0 con 0x7f55bc101420 2024-09-12T03:04:58.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:58.931+0000 7f55b97fa640 1 --2- 172.21.15.5:0/87838218 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f55a40a2b80 0x7f55a40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:04:58.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:58.931+0000 7f55bbfff640 1 --2- 172.21.15.5:0/87838218 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f55a40a2b80 0x7f55a40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:04:58.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:58.931+0000 7f55b97fa640 1 -- 172.21.15.5:0/87838218 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8211+0+0 (secure 0 0 0) 0x7f55b00cf2b0 con 0x7f55bc101420 2024-09-12T03:04:58.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:58.931+0000 7f55bbfff640 1 --2- 172.21.15.5:0/87838218 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f55a40a2b80 0x7f55a40a5040 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f55ac009900 tx=0x7f55ac0058d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:04:58.939 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:58.934+0000 7f55b97fa640 1 -- 172.21.15.5:0/87838218 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f55b0014030 con 0x7f55bc101420 2024-09-12T03:04:59.147 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:59.146+0000 7f55c26d3640 1 -- 172.21.15.5:0/87838218 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f55bc199800 con 0x7f55bc101420 2024-09-12T03:04:59.147 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:04:58 smithi005 ceph-mon[31197]: pgmap v537: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 853 B/s wr, 11 op/s 2024-09-12T03:04:59.149 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:59.149+0000 7f55b97fa640 1 -- 172.21.15.5:0/87838218 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v34) ==== 161+0+0 (secure 0 0 0) 0x7f55bc199800 con 0x7f55bc101420 2024-09-12T03:04:59.149 INFO:teuthology.orchestra.run.smithi005.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-12T03:04:59.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:59.153+0000 7f55c26d3640 1 -- 172.21.15.5:0/87838218 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f55a40a2b80 msgr2=0x7f55a40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:59.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:59.153+0000 7f55c26d3640 1 --2- 172.21.15.5:0/87838218 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f55a40a2b80 0x7f55a40a5040 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f55ac009900 tx=0x7f55ac0058d0 comp rx=0 tx=0).stop 2024-09-12T03:04:59.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:59.153+0000 7f55c26d3640 1 -- 172.21.15.5:0/87838218 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55bc101420 msgr2=0x7f55bc1989e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:04:59.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:59.153+0000 7f55c26d3640 1 --2- 172.21.15.5:0/87838218 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55bc101420 0x7f55bc1989e0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f55b000c890 tx=0x7f55b000cd50 comp rx=0 tx=0).stop 2024-09-12T03:04:59.153 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:59.153+0000 7f55c26d3640 1 -- 172.21.15.5:0/87838218 shutdown_connections 2024-09-12T03:04:59.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:59.153+0000 7f55c26d3640 1 --2- 172.21.15.5:0/87838218 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f55a40a2b80 0x7f55a40a5040 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:59.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:59.153+0000 7f55c26d3640 1 --2- 172.21.15.5:0/87838218 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55bc101420 0x7f55bc1989e0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:59.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:59.153+0000 7f55c26d3640 1 --2- 172.21.15.5:0/87838218 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f55bc106a80 0x7f55bc19cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:59.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:59.153+0000 7f55c26d3640 1 --2- 172.21.15.5:0/87838218 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f55bc100a80 0x7f55bc1984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:04:59.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:59.153+0000 7f55c26d3640 1 -- 172.21.15.5:0/87838218 >> 172.21.15.5:0/87838218 conn(0x7f55bc0fc7a0 msgr2=0x7f55bc0fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:04:59.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:59.153+0000 7f55c26d3640 1 -- 172.21.15.5:0/87838218 shutdown_connections 2024-09-12T03:04:59.154 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:04:59.153+0000 7f55c26d3640 1 -- 172.21.15.5:0/87838218 wait complete. 2024-09-12T03:04:59.415 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:04:58 smithi148 ceph-mon[35090]: pgmap v537: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 853 B/s wr, 11 op/s 2024-09-12T03:04:59.416 INFO:journalctl@ceph.mds.l.smithi148.stdout:Sep 12 03:04:59 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-l[64649]: ignoring --setuser ceph since I am not root 2024-09-12T03:04:59.416 INFO:journalctl@ceph.mds.l.smithi148.stdout:Sep 12 03:04:59 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-l[64649]: ignoring --setgroup ceph since I am not root 2024-09-12T03:04:59.416 INFO:journalctl@ceph.mds.l.smithi148.stdout:Sep 12 03:04:59 smithi148 ceph-mds[64653]: -- 172.21.15.148:0/907094674 <== mon.1 v2:172.21.15.148:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c12164f860 con 0x55c12162c800 2024-09-12T03:04:59.416 INFO:journalctl@ceph.mds.l.smithi148.stdout:Sep 12 03:04:59 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-l[64649]: starting mds.l at 2024-09-12T03:04:59.416 INFO:journalctl@ceph.mds.c.smithi148.stdout:Sep 12 03:04:59 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-c[65251]: ignoring --setuser ceph since I am not root 2024-09-12T03:04:59.416 INFO:journalctl@ceph.mds.c.smithi148.stdout:Sep 12 03:04:59 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-c[65251]: ignoring --setgroup ceph since I am not root 2024-09-12T03:04:59.416 INFO:journalctl@ceph.mds.c.smithi148.stdout:Sep 12 03:04:59 smithi148 ceph-mds[65255]: -- 172.21.15.148:0/22833947 <== mon.1 v2:172.21.15.148:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558e5852bc20 con 0x558e58509400 2024-09-12T03:04:59.417 INFO:journalctl@ceph.mds.c.smithi148.stdout:Sep 12 03:04:59 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-c[65251]: starting mds.c at 2024-09-12T03:04:59.421 INFO:journalctl@ceph.mds.f.smithi148.stdout:Sep 12 03:04:59 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-f[64950]: ignoring --setuser ceph since I am not root 2024-09-12T03:04:59.421 INFO:journalctl@ceph.mds.f.smithi148.stdout:Sep 12 03:04:59 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-f[64950]: ignoring --setgroup ceph since I am not root 2024-09-12T03:04:59.421 INFO:journalctl@ceph.mds.f.smithi148.stdout:Sep 12 03:04:59 smithi148 ceph-mds[64954]: -- 172.21.15.148:0/83896109 <== mon.1 v2:172.21.15.148:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556072b99860 con 0x556072b77000 2024-09-12T03:04:59.422 INFO:journalctl@ceph.mds.f.smithi148.stdout:Sep 12 03:04:59 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-f[64950]: starting mds.f at 2024-09-12T03:04:59.422 INFO:journalctl@ceph.mds.i.smithi148.stdout:Sep 12 03:04:59 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-i[65552]: ignoring --setuser ceph since I am not root 2024-09-12T03:04:59.422 INFO:journalctl@ceph.mds.i.smithi148.stdout:Sep 12 03:04:59 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-i[65552]: ignoring --setgroup ceph since I am not root 2024-09-12T03:04:59.422 INFO:journalctl@ceph.mds.i.smithi148.stdout:Sep 12 03:04:59 smithi148 ceph-mds[65556]: -- 172.21.15.148:0/2064386421 <== mon.1 v2:172.21.15.148:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56383d669860 con 0x56383d647400 2024-09-12T03:04:59.422 INFO:journalctl@ceph.mds.i.smithi148.stdout:Sep 12 03:04:59 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-i[65552]: starting mds.i at 2024-09-12T03:04:59.448 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:04:59 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mon-a[31193]: 2024-09-12T03:04:59.147+0000 7f60dc5f1640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-12T03:04:59.448 INFO:journalctl@ceph.mds.g.smithi005.stdout:Sep 12 03:04:59 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-g[69561]: ignoring --setuser ceph since I am not root 2024-09-12T03:04:59.448 INFO:journalctl@ceph.mds.g.smithi005.stdout:Sep 12 03:04:59 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-g[69561]: ignoring --setgroup ceph since I am not root 2024-09-12T03:04:59.448 INFO:journalctl@ceph.mds.g.smithi005.stdout:Sep 12 03:04:59 smithi005 ceph-mds[69583]: -- 172.21.15.5:0/348320949 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55fdce8a8d00 con 0x55fdce87c800 2024-09-12T03:04:59.448 INFO:journalctl@ceph.mds.g.smithi005.stdout:Sep 12 03:04:59 smithi005 ceph-mds[69583]: -- 172.21.15.5:0/348320949 <== mon.0 v2:172.21.15.5:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55fdce89f860 con 0x55fdce87c800 2024-09-12T03:04:59.448 INFO:journalctl@ceph.mds.g.smithi005.stdout:Sep 12 03:04:59 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-g[69561]: starting mds.g at 2024-09-12T03:04:59.449 INFO:journalctl@ceph.mds.a.smithi005.stdout:Sep 12 03:04:59 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-a[69981]: ignoring --setuser ceph since I am not root 2024-09-12T03:04:59.449 INFO:journalctl@ceph.mds.a.smithi005.stdout:Sep 12 03:04:59 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-a[69981]: ignoring --setgroup ceph since I am not root 2024-09-12T03:04:59.449 INFO:journalctl@ceph.mds.a.smithi005.stdout:Sep 12 03:04:59 smithi005 ceph-mds[69985]: -- 172.21.15.5:0/53400793 <== mon.1 v2:172.21.15.148:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55eea33c1860 con 0x55eea339e800 2024-09-12T03:04:59.449 INFO:journalctl@ceph.mds.a.smithi005.stdout:Sep 12 03:04:59 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-a[69981]: starting mds.a at 2024-09-12T03:04:59.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:04:58 smithi146 ceph-mon[35011]: pgmap v537: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 853 B/s wr, 11 op/s 2024-09-12T03:04:59.456 INFO:journalctl@ceph.mds.b.smithi146.stdout:Sep 12 03:04:59 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-b[65422]: ignoring --setuser ceph since I am not root 2024-09-12T03:04:59.456 INFO:journalctl@ceph.mds.b.smithi146.stdout:Sep 12 03:04:59 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-b[65422]: ignoring --setgroup ceph since I am not root 2024-09-12T03:04:59.457 INFO:journalctl@ceph.mds.b.smithi146.stdout:Sep 12 03:04:59 smithi146 ceph-mds[65426]: -- 172.21.15.146:0/3018127757 <== mon.2 v2:172.21.15.146:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561107feb860 con 0x561107fc8800 2024-09-12T03:04:59.457 INFO:journalctl@ceph.mds.b.smithi146.stdout:Sep 12 03:04:59 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-b[65422]: starting mds.b at 2024-09-12T03:04:59.457 INFO:journalctl@ceph.mds.e.smithi146.stdout:Sep 12 03:04:59 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-e[64819]: ignoring --setuser ceph since I am not root 2024-09-12T03:04:59.457 INFO:journalctl@ceph.mds.e.smithi146.stdout:Sep 12 03:04:59 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-e[64819]: ignoring --setgroup ceph since I am not root 2024-09-12T03:04:59.457 INFO:journalctl@ceph.mds.e.smithi146.stdout:Sep 12 03:04:59 smithi146 ceph-mds[64823]: -- 172.21.15.146:0/1692571060 <== mon.1 v2:172.21.15.148:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5593b21e7860 con 0x5593b21c4800 2024-09-12T03:04:59.457 INFO:journalctl@ceph.mds.e.smithi146.stdout:Sep 12 03:04:59 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-e[64819]: starting mds.e at 2024-09-12T03:04:59.458 INFO:journalctl@ceph.mds.k.smithi146.stdout:Sep 12 03:04:59 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-k[64500]: ignoring --setuser ceph since I am not root 2024-09-12T03:04:59.458 INFO:journalctl@ceph.mds.k.smithi146.stdout:Sep 12 03:04:59 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-k[64500]: ignoring --setgroup ceph since I am not root 2024-09-12T03:04:59.458 INFO:journalctl@ceph.mds.k.smithi146.stdout:Sep 12 03:04:59 smithi146 ceph-mds[64504]: -- 172.21.15.146:0/1998606502 <== mon.0 v2:172.21.15.5:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5644d2ba9860 con 0x5644d2b86800 2024-09-12T03:04:59.458 INFO:journalctl@ceph.mds.k.smithi146.stdout:Sep 12 03:04:59 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-k[64500]: starting mds.k at 2024-09-12T03:04:59.458 INFO:journalctl@ceph.mds.k.smithi146.stdout:Sep 12 03:04:59 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-k[64500]: 2024-09-12T03:04:59.196+0000 7fdcda685640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6832/0: (98) Address already in use 2024-09-12T03:04:59.458 INFO:journalctl@ceph.mds.h.smithi146.stdout:Sep 12 03:04:59 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-h[65118]: ignoring --setuser ceph since I am not root 2024-09-12T03:04:59.458 INFO:journalctl@ceph.mds.h.smithi146.stdout:Sep 12 03:04:59 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-h[65118]: ignoring --setgroup ceph since I am not root 2024-09-12T03:04:59.458 INFO:journalctl@ceph.mds.h.smithi146.stdout:Sep 12 03:04:59 smithi146 ceph-mds[65122]: -- 172.21.15.146:0/2102600035 <== mon.0 v2:172.21.15.5:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55bac21d3860 con 0x55bac21b1000 2024-09-12T03:04:59.458 INFO:journalctl@ceph.mds.h.smithi146.stdout:Sep 12 03:04:59 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-h[65118]: starting mds.h at 2024-09-12T03:04:59.459 INFO:journalctl@ceph.mds.h.smithi146.stdout:Sep 12 03:04:59 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-h[65118]: 2024-09-12T03:04:59.196+0000 7f0b296ac640 -1 PosixStack listen unable to listen on v1:0.0.0.0:6834/0: (98) Address already in use 2024-09-12T03:04:59.894 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-09-12T03:05:00.164 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:05:00.274 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:04:59 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/87838218' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-12T03:05:00.274 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:04:59 smithi005 ceph-mon[31197]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-12T03:05:00.274 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:04:59 smithi005 ceph-mon[31197]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-12T03:05:00.274 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:04:59 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/87838218' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-12T03:05:00.274 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:04:59 smithi005 ceph-mon[31197]: osdmap e84: 12 total, 12 up, 12 in 2024-09-12T03:05:00.274 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:04:59 smithi005 ceph-mon[31197]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-12T03:05:00.275 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:04:59 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-12T03:05:00.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:04:59 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/87838218' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-12T03:05:00.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:04:59 smithi148 ceph-mon[35090]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-12T03:05:00.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:04:59 smithi148 ceph-mon[35090]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-12T03:05:00.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:04:59 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/87838218' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-12T03:05:00.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:04:59 smithi148 ceph-mon[35090]: osdmap e84: 12 total, 12 up, 12 in 2024-09-12T03:05:00.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:04:59 smithi148 ceph-mon[35090]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-12T03:05:00.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:04:59 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-12T03:05:00.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:04:59 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/87838218' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-12T03:05:00.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:04:59 smithi146 ceph-mon[35011]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-12T03:05:00.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:04:59 smithi146 ceph-mon[35011]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-12T03:05:00.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:04:59 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/87838218' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-12T03:05:00.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:04:59 smithi146 ceph-mon[35011]: osdmap e84: 12 total, 12 up, 12 in 2024-09-12T03:05:00.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:04:59 smithi146 ceph-mon[35011]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-09-12T03:05:00.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:04:59 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-12T03:05:00.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:00.578+0000 7f1ab8d65640 1 -- 172.21.15.5:0/446275894 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ab4106f10 msgr2=0x7f1ab410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:00.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:00.578+0000 7f1ab8d65640 1 --2- 172.21.15.5:0/446275894 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ab4106f10 0x7f1ab410ea90 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f1aac009ca0 tx=0x7f1aac02a8c0 comp rx=0 tx=0).stop 2024-09-12T03:05:00.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:00.578+0000 7f1ab8d65640 1 -- 172.21.15.5:0/446275894 shutdown_connections 2024-09-12T03:05:00.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:00.578+0000 7f1ab8d65640 1 --2- 172.21.15.5:0/446275894 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ab4106f10 0x7f1ab410ea90 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:00.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:00.578+0000 7f1ab8d65640 1 --2- 172.21.15.5:0/446275894 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ab4102790 0x7f1ab41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:00.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:00.578+0000 7f1ab8d65640 1 --2- 172.21.15.5:0/446275894 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1ab4101de0 0x7f1ab41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:00.602 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:00.578+0000 7f1ab8d65640 1 -- 172.21.15.5:0/446275894 >> 172.21.15.5:0/446275894 conn(0x7f1ab40fd8c0 msgr2=0x7f1ab40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:00.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:00.578+0000 7f1ab8d65640 1 -- 172.21.15.5:0/446275894 shutdown_connections 2024-09-12T03:05:00.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:00.578+0000 7f1ab8d65640 1 -- 172.21.15.5:0/446275894 wait complete. 2024-09-12T03:05:00.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:00.579+0000 7f1ab8d65640 1 Processor -- start 2024-09-12T03:05:00.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:00.579+0000 7f1ab8d65640 1 -- start start 2024-09-12T03:05:00.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:00.579+0000 7f1ab8d65640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1ab4101de0 0x7f1ab41983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:00.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:00.579+0000 7f1ab8d65640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ab4102790 0x7f1ab4198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:00.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:00.579+0000 7f1ab8d65640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ab4106f10 0x7f1ab419ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:00.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:00.579+0000 7f1ab8d65640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab4111f70 con 0x7f1ab4102790 2024-09-12T03:05:00.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:00.579+0000 7f1ab8d65640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab4111df0 con 0x7f1ab4101de0 2024-09-12T03:05:00.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:00.579+0000 7f1ab8d65640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab41120f0 con 0x7f1ab4106f10 2024-09-12T03:05:00.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:00.579+0000 7f1aa9d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ab4102790 0x7f1ab4198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:00.603 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:00.579+0000 7f1aa9d9b640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ab4102790 0x7f1ab4198930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60012/0 (socket says 172.21.15.5:60012) 2024-09-12T03:05:00.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:00.579+0000 7f1aa9d9b640 1 -- 172.21.15.5:0/4071027456 learned_addr learned my addr 172.21.15.5:0/4071027456 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:05:00.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:00.580+0000 7f1ab2d9d640 1 --2- 172.21.15.5:0/4071027456 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ab4106f10 0x7f1ab419ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:00.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:00.580+0000 7f1ab259c640 1 --2- 172.21.15.5:0/4071027456 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1ab4101de0 0x7f1ab41983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:00.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:00.580+0000 7f1aa9d9b640 1 -- 172.21.15.5:0/4071027456 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ab4106f10 msgr2=0x7f1ab419ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:00.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:00.580+0000 7f1aa9d9b640 1 --2- 172.21.15.5:0/4071027456 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ab4106f10 0x7f1ab419ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:00.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:00.580+0000 7f1aa9d9b640 1 -- 172.21.15.5:0/4071027456 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1ab4101de0 msgr2=0x7f1ab41983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:00.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:00.580+0000 7f1aa9d9b640 1 --2- 172.21.15.5:0/4071027456 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1ab4101de0 0x7f1ab41983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:00.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:00.580+0000 7f1aa9d9b640 1 -- 172.21.15.5:0/4071027456 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ab419d3a0 con 0x7f1ab4102790 2024-09-12T03:05:00.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:00.580+0000 7f1ab259c640 1 --2- 172.21.15.5:0/4071027456 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1ab4101de0 0x7f1ab41983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:05:00.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:00.580+0000 7f1aa9d9b640 1 --2- 172.21.15.5:0/4071027456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ab4102790 0x7f1ab4198930 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c00d770 tx=0x7f1a9c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:00.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:00.580+0000 7f1aabfff640 1 -- 172.21.15.5:0/4071027456 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a9c017070 con 0x7f1ab4102790 2024-09-12T03:05:00.604 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:00.580+0000 7f1ab8d65640 1 -- 172.21.15.5:0/4071027456 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ab419d690 con 0x7f1ab4102790 2024-09-12T03:05:00.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:00.580+0000 7f1aabfff640 1 -- 172.21.15.5:0/4071027456 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a9c005500 con 0x7f1ab4102790 2024-09-12T03:05:00.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:00.580+0000 7f1aabfff640 1 -- 172.21.15.5:0/4071027456 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a9c004e00 con 0x7f1ab4102790 2024-09-12T03:05:00.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:00.580+0000 7f1ab8d65640 1 -- 172.21.15.5:0/4071027456 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ab41a4f70 con 0x7f1ab4102790 2024-09-12T03:05:00.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:00.581+0000 7f1ab8d65640 1 -- 172.21.15.5:0/4071027456 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ab41021c0 con 0x7f1ab4102790 2024-09-12T03:05:00.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:00.582+0000 7f1aabfff640 1 -- 172.21.15.5:0/4071027456 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f1a9c004fa0 con 0x7f1ab4102790 2024-09-12T03:05:00.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:00.583+0000 7f1aabfff640 1 --2- 172.21.15.5:0/4071027456 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f1aa00a2b80 0x7f1aa00a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:00.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:00.584+0000 7f1ab259c640 1 --2- 172.21.15.5:0/4071027456 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f1aa00a2b80 0x7f1aa00a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:00.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:00.584+0000 7f1aabfff640 1 -- 172.21.15.5:0/4071027456 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8641+0+0 (secure 0 0 0) 0x7f1a9c0cec00 con 0x7f1ab4102790 2024-09-12T03:05:00.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:00.584+0000 7f1ab259c640 1 --2- 172.21.15.5:0/4071027456 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f1aa00a2b80 0x7f1aa00a5040 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f1aa40058c0 tx=0x7f1aa4005ec0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:00.605 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:00.588+0000 7f1aabfff640 1 -- 172.21.15.5:0/4071027456 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1a9c09b960 con 0x7f1ab4102790 2024-09-12T03:05:00.799 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:00.799+0000 7f1ab8d65640 1 -- 172.21.15.5:0/4071027456 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f1ab410a0f0 con 0x7f1ab4102790 2024-09-12T03:05:01.160 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:01.160+0000 7f1aabfff640 1 -- 172.21.15.5:0/4071027456 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v36) ==== 104+0+0 (secure 0 0 0) 0x7f1a9c09b260 con 0x7f1ab4102790 2024-09-12T03:05:01.162 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:01.162+0000 7f1ab8d65640 1 -- 172.21.15.5:0/4071027456 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f1aa00a2b80 msgr2=0x7f1aa00a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:01.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:01.162+0000 7f1ab8d65640 1 --2- 172.21.15.5:0/4071027456 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f1aa00a2b80 0x7f1aa00a5040 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f1aa40058c0 tx=0x7f1aa4005ec0 comp rx=0 tx=0).stop 2024-09-12T03:05:01.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:01.162+0000 7f1ab8d65640 1 -- 172.21.15.5:0/4071027456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ab4102790 msgr2=0x7f1ab4198930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:01.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:01.162+0000 7f1ab8d65640 1 --2- 172.21.15.5:0/4071027456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ab4102790 0x7f1ab4198930 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f1a9c00d770 tx=0x7f1a9c00dc30 comp rx=0 tx=0).stop 2024-09-12T03:05:01.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:01.163+0000 7f1ab8d65640 1 -- 172.21.15.5:0/4071027456 shutdown_connections 2024-09-12T03:05:01.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:01.163+0000 7f1ab8d65640 1 --2- 172.21.15.5:0/4071027456 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f1aa00a2b80 0x7f1aa00a5040 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:01.163 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:01.163+0000 7f1ab8d65640 1 --2- 172.21.15.5:0/4071027456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ab4102790 0x7f1ab4198930 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:01.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:01.163+0000 7f1ab8d65640 1 --2- 172.21.15.5:0/4071027456 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f1ab4106f10 0x7f1ab419ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:01.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:01.163+0000 7f1ab8d65640 1 --2- 172.21.15.5:0/4071027456 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f1ab4101de0 0x7f1ab41983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:01.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:01.163+0000 7f1ab8d65640 1 -- 172.21.15.5:0/4071027456 >> 172.21.15.5:0/4071027456 conn(0x7f1ab40fd8c0 msgr2=0x7f1ab40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:01.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:01.163+0000 7f1ab8d65640 1 -- 172.21.15.5:0/4071027456 shutdown_connections 2024-09-12T03:05:01.164 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:01.163+0000 7f1ab8d65640 1 -- 172.21.15.5:0/4071027456 wait complete. 2024-09-12T03:05:01.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:05:01 smithi148 ceph-mon[35090]: pgmap v539: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.3 KiB/s wr, 12 op/s 2024-09-12T03:05:01.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:05:01 smithi148 ceph-mon[35090]: mds.? [v2:172.21.15.5:6839/3877369789,v1:172.21.15.5:6841/3877369789] up:boot 2024-09-12T03:05:01.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:05:01 smithi148 ceph-mon[35090]: mds.? [v2:172.21.15.5:6838/3435517234,v1:172.21.15.5:6840/3435517234] up:boot 2024-09-12T03:05:01.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:05:01 smithi148 ceph-mon[35090]: mds.? [v2:172.21.15.148:6832/1403516521,v1:172.21.15.148:6834/1403516521] up:boot 2024-09-12T03:05:01.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:05:01 smithi148 ceph-mon[35090]: mds.? [v2:172.21.15.148:6833/170663569,v1:172.21.15.148:6835/170663569] up:boot 2024-09-12T03:05:01.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:05:01 smithi148 ceph-mon[35090]: mds.? [v2:172.21.15.146:6833/1327107008,v1:172.21.15.146:6834/1327107008] up:boot 2024-09-12T03:05:01.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:05:01 smithi148 ceph-mon[35090]: mds.? [v2:172.21.15.146:6832/1345714561,v1:172.21.15.146:6835/1345714561] up:boot 2024-09-12T03:05:01.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:05:01 smithi148 ceph-mon[35090]: mds.? [v2:172.21.15.146:6836/1931763137,v1:172.21.15.146:6837/1931763137] up:boot 2024-09-12T03:05:01.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:05:01 smithi148 ceph-mon[35090]: mds.? [v2:172.21.15.148:6836/4066002849,v1:172.21.15.148:6837/4066002849] up:boot 2024-09-12T03:05:01.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:05:01 smithi148 ceph-mon[35090]: mds.? [v2:172.21.15.146:6838/448642360,v1:172.21.15.146:6839/448642360] up:boot 2024-09-12T03:05:01.417 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:05:01 smithi148 ceph-mon[35090]: mds.? [v2:172.21.15.148:6838/2140461168,v1:172.21.15.148:6839/2140461168] up:boot 2024-09-12T03:05:01.417 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:05:01 smithi148 ceph-mon[35090]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-12T03:05:01.417 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:05:01 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-12T03:05:01.417 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:05:01 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-12T03:05:01.417 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:05:01 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T03:05:01.417 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:05:01 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-12T03:05:01.417 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:05:01 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-12T03:05:01.417 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:05:01 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-12T03:05:01.417 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:05:01 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-12T03:05:01.417 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:05:01 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-12T03:05:01.417 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:05:01 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-12T03:05:01.417 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:05:01 smithi148 ceph-mon[35090]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-12T03:05:01.417 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:05:01 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/4071027456' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-12T03:05:01.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:05:01 smithi146 ceph-mon[35011]: pgmap v539: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.3 KiB/s wr, 12 op/s 2024-09-12T03:05:01.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:05:01 smithi146 ceph-mon[35011]: mds.? [v2:172.21.15.5:6839/3877369789,v1:172.21.15.5:6841/3877369789] up:boot 2024-09-12T03:05:01.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:05:01 smithi146 ceph-mon[35011]: mds.? [v2:172.21.15.5:6838/3435517234,v1:172.21.15.5:6840/3435517234] up:boot 2024-09-12T03:05:01.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:05:01 smithi146 ceph-mon[35011]: mds.? [v2:172.21.15.148:6832/1403516521,v1:172.21.15.148:6834/1403516521] up:boot 2024-09-12T03:05:01.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:05:01 smithi146 ceph-mon[35011]: mds.? [v2:172.21.15.148:6833/170663569,v1:172.21.15.148:6835/170663569] up:boot 2024-09-12T03:05:01.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:05:01 smithi146 ceph-mon[35011]: mds.? [v2:172.21.15.146:6833/1327107008,v1:172.21.15.146:6834/1327107008] up:boot 2024-09-12T03:05:01.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:05:01 smithi146 ceph-mon[35011]: mds.? [v2:172.21.15.146:6832/1345714561,v1:172.21.15.146:6835/1345714561] up:boot 2024-09-12T03:05:01.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:05:01 smithi146 ceph-mon[35011]: mds.? [v2:172.21.15.146:6836/1931763137,v1:172.21.15.146:6837/1931763137] up:boot 2024-09-12T03:05:01.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:05:01 smithi146 ceph-mon[35011]: mds.? [v2:172.21.15.148:6836/4066002849,v1:172.21.15.148:6837/4066002849] up:boot 2024-09-12T03:05:01.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:05:01 smithi146 ceph-mon[35011]: mds.? [v2:172.21.15.146:6838/448642360,v1:172.21.15.146:6839/448642360] up:boot 2024-09-12T03:05:01.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:05:01 smithi146 ceph-mon[35011]: mds.? [v2:172.21.15.148:6838/2140461168,v1:172.21.15.148:6839/2140461168] up:boot 2024-09-12T03:05:01.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:05:01 smithi146 ceph-mon[35011]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-12T03:05:01.458 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:05:01 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-12T03:05:01.458 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:05:01 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-12T03:05:01.458 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:05:01 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T03:05:01.458 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:05:01 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-12T03:05:01.458 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:05:01 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-12T03:05:01.458 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:05:01 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-12T03:05:01.458 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:05:01 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-12T03:05:01.458 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:05:01 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-12T03:05:01.458 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:05:01 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-12T03:05:01.458 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:05:01 smithi146 ceph-mon[35011]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-12T03:05:01.458 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:05:01 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/4071027456' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-12T03:05:01.498 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:05:01 smithi005 ceph-mon[31197]: pgmap v539: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.3 KiB/s wr, 12 op/s 2024-09-12T03:05:01.498 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:05:01 smithi005 ceph-mon[31197]: mds.? [v2:172.21.15.5:6839/3877369789,v1:172.21.15.5:6841/3877369789] up:boot 2024-09-12T03:05:01.498 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:05:01 smithi005 ceph-mon[31197]: mds.? [v2:172.21.15.5:6838/3435517234,v1:172.21.15.5:6840/3435517234] up:boot 2024-09-12T03:05:01.498 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:05:01 smithi005 ceph-mon[31197]: mds.? [v2:172.21.15.148:6832/1403516521,v1:172.21.15.148:6834/1403516521] up:boot 2024-09-12T03:05:01.498 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:05:01 smithi005 ceph-mon[31197]: mds.? [v2:172.21.15.148:6833/170663569,v1:172.21.15.148:6835/170663569] up:boot 2024-09-12T03:05:01.498 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:05:01 smithi005 ceph-mon[31197]: mds.? [v2:172.21.15.146:6833/1327107008,v1:172.21.15.146:6834/1327107008] up:boot 2024-09-12T03:05:01.498 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:05:01 smithi005 ceph-mon[31197]: mds.? [v2:172.21.15.146:6832/1345714561,v1:172.21.15.146:6835/1345714561] up:boot 2024-09-12T03:05:01.499 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:05:01 smithi005 ceph-mon[31197]: mds.? [v2:172.21.15.146:6836/1931763137,v1:172.21.15.146:6837/1931763137] up:boot 2024-09-12T03:05:01.499 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:05:01 smithi005 ceph-mon[31197]: mds.? [v2:172.21.15.148:6836/4066002849,v1:172.21.15.148:6837/4066002849] up:boot 2024-09-12T03:05:01.499 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:05:01 smithi005 ceph-mon[31197]: mds.? [v2:172.21.15.146:6838/448642360,v1:172.21.15.146:6839/448642360] up:boot 2024-09-12T03:05:01.499 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:05:01 smithi005 ceph-mon[31197]: mds.? [v2:172.21.15.148:6838/2140461168,v1:172.21.15.148:6839/2140461168] up:boot 2024-09-12T03:05:01.499 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:05:01 smithi005 ceph-mon[31197]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-09-12T03:05:01.499 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:05:01 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-12T03:05:01.499 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:05:01 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-12T03:05:01.499 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:05:01 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-12T03:05:01.499 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:05:01 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-12T03:05:01.499 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:05:01 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-12T03:05:01.499 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:05:01 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-12T03:05:01.499 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:05:01 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-12T03:05:01.499 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:05:01 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-12T03:05:01.499 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:05:01 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-12T03:05:01.500 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:05:01 smithi005 ceph-mon[31197]: from='mgr.14148 172.21.15.5:0/3924670688' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-12T03:05:01.500 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:05:01 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/4071027456' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-12T03:05:01.841 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-12T03:05:02.107 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:05:02.415 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:05:02 smithi148 ceph-mon[35090]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-12T03:05:02.415 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:05:02 smithi148 ceph-mon[35090]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-12T03:05:02.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:05:02 smithi148 ceph-mon[35090]: Cluster is now healthy 2024-09-12T03:05:02.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:05:02 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/4071027456' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-12T03:05:02.416 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:05:02 smithi148 ceph-mon[35090]: fsmap 12 up:standby 2024-09-12T03:05:02.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:05:02 smithi146 ceph-mon[35011]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-12T03:05:02.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:05:02 smithi146 ceph-mon[35011]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-12T03:05:02.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:05:02 smithi146 ceph-mon[35011]: Cluster is now healthy 2024-09-12T03:05:02.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:05:02 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/4071027456' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-12T03:05:02.457 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:05:02 smithi146 ceph-mon[35011]: fsmap 12 up:standby 2024-09-12T03:05:02.484 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:05:02 smithi005 ceph-mon[31197]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-12T03:05:02.484 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:05:02 smithi005 ceph-mon[31197]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-12T03:05:02.484 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:05:02 smithi005 ceph-mon[31197]: Cluster is now healthy 2024-09-12T03:05:02.484 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:05:02 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/4071027456' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-12T03:05:02.484 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:05:02 smithi005 ceph-mon[31197]: fsmap 12 up:standby 2024-09-12T03:05:02.570 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:02.570+0000 7f5f607f0640 1 -- 172.21.15.5:0/3350954232 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f58102790 msgr2=0x7f5f581067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:02.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:02.570+0000 7f5f607f0640 1 --2- 172.21.15.5:0/3350954232 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f58102790 0x7f5f581067e0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f5f48009900 tx=0x7f5f4802a410 comp rx=0 tx=0).stop 2024-09-12T03:05:02.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:02.570+0000 7f5f607f0640 1 -- 172.21.15.5:0/3350954232 shutdown_connections 2024-09-12T03:05:02.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:02.570+0000 7f5f607f0640 1 --2- 172.21.15.5:0/3350954232 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f58102790 0x7f5f581067e0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:02.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:02.570+0000 7f5f607f0640 1 --2- 172.21.15.5:0/3350954232 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5f58106f10 0x7f5f5810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:02.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:02.570+0000 7f5f607f0640 1 --2- 172.21.15.5:0/3350954232 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f58101de0 0x7f5f581021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:02.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:02.570+0000 7f5f607f0640 1 -- 172.21.15.5:0/3350954232 >> 172.21.15.5:0/3350954232 conn(0x7f5f580fd8c0 msgr2=0x7f5f580ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:02.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:02.570+0000 7f5f607f0640 1 -- 172.21.15.5:0/3350954232 shutdown_connections 2024-09-12T03:05:02.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:02.570+0000 7f5f607f0640 1 -- 172.21.15.5:0/3350954232 wait complete. 2024-09-12T03:05:02.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:02.570+0000 7f5f607f0640 1 Processor -- start 2024-09-12T03:05:02.571 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:02.571+0000 7f5f607f0640 1 -- start start 2024-09-12T03:05:02.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:02.571+0000 7f5f607f0640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f58101de0 0x7f5f580744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:02.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:02.571+0000 7f5f607f0640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5f58102790 0x7f5f58074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:02.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:02.571+0000 7f5f607f0640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f58106f10 0x7f5f58074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:02.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:02.571+0000 7f5f607f0640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7f5f58111f70 con 0x7f5f58101de0 2024-09-12T03:05:02.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:02.571+0000 7f5f607f0640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7f5f58111df0 con 0x7f5f58102790 2024-09-12T03:05:02.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:02.571+0000 7f5f607f0640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7f5f581120f0 con 0x7f5f58106f10 2024-09-12T03:05:02.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:02.571+0000 7f5f5e58c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f58101de0 0x7f5f580744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:02.573 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:02.571+0000 7f5f5e58c640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f58101de0 0x7f5f580744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60032/0 (socket says 172.21.15.5:60032) 2024-09-12T03:05:02.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:02.571+0000 7f5f5e58c640 1 -- 172.21.15.5:0/2236571941 learned_addr learned my addr 172.21.15.5:0/2236571941 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:05:02.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:02.571+0000 7f5f5dd8b640 1 --2- 172.21.15.5:0/2236571941 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5f58102790 0x7f5f58074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:02.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:02.571+0000 7f5f5ed8d640 1 --2- 172.21.15.5:0/2236571941 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f58106f10 0x7f5f58074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:02.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:02.571+0000 7f5f5e58c640 1 -- 172.21.15.5:0/2236571941 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f58106f10 msgr2=0x7f5f58074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:02.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:02.571+0000 7f5f5e58c640 1 --2- 172.21.15.5:0/2236571941 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f58106f10 0x7f5f58074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:02.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:02.571+0000 7f5f5e58c640 1 -- 172.21.15.5:0/2236571941 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5f58102790 msgr2=0x7f5f58074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:02.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:02.571+0000 7f5f5e58c640 1 --2- 172.21.15.5:0/2236571941 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5f58102790 0x7f5f58074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:02.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:02.571+0000 7f5f5e58c640 1 -- 172.21.15.5:0/2236571941 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f581a6930 con 0x7f5f58101de0 2024-09-12T03:05:02.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:02.571+0000 7f5f5ed8d640 1 --2- 172.21.15.5:0/2236571941 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f58106f10 0x7f5f58074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:05:02.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:02.572+0000 7f5f5e58c640 1 --2- 172.21.15.5:0/2236571941 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f58101de0 0x7f5f580744f0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f5f5400d770 tx=0x7f5f5400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:02.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:02.572+0000 7f5f4f7fe640 1 -- 172.21.15.5:0/2236571941 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f54017070 con 0x7f5f58101de0 2024-09-12T03:05:02.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:02.572+0000 7f5f607f0640 1 -- 172.21.15.5:0/2236571941 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f581a6c20 con 0x7f5f58101de0 2024-09-12T03:05:02.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:02.572+0000 7f5f4f7fe640 1 -- 172.21.15.5:0/2236571941 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f54005500 con 0x7f5f58101de0 2024-09-12T03:05:02.574 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:02.572+0000 7f5f4f7fe640 1 -- 172.21.15.5:0/2236571941 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f54004e00 con 0x7f5f58101de0 2024-09-12T03:05:02.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:02.572+0000 7f5f607f0640 1 -- 172.21.15.5:0/2236571941 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f581a70e0 con 0x7f5f58101de0 2024-09-12T03:05:02.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:02.573+0000 7f5f607f0640 1 -- 172.21.15.5:0/2236571941 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f581032c0 con 0x7f5f58101de0 2024-09-12T03:05:02.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:02.574+0000 7f5f4f7fe640 1 -- 172.21.15.5:0/2236571941 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7f5f54004fa0 con 0x7f5f58101de0 2024-09-12T03:05:02.575 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:02.575+0000 7f5f4f7fe640 1 --2- 172.21.15.5:0/2236571941 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f5f440a2b80 0x7f5f440a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:02.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:02.575+0000 7f5f4f7fe640 1 -- 172.21.15.5:0/2236571941 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8641+0+0 (secure 0 0 0) 0x7f5f540cec80 con 0x7f5f58101de0 2024-09-12T03:05:02.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:02.575+0000 7f5f5dd8b640 1 --2- 172.21.15.5:0/2236571941 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f5f440a2b80 0x7f5f440a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:02.576 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:02.575+0000 7f5f5dd8b640 1 --2- 172.21.15.5:0/2236571941 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f5f440a2b80 0x7f5f440a5040 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f5f48009230 tx=0x7f5f480058d0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:02.579 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:02.579+0000 7f5f4f7fe640 1 -- 172.21.15.5:0/2236571941 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5f5409b9e0 con 0x7f5f58101de0 2024-09-12T03:05:02.757 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:02.757+0000 7f5f607f0640 1 -- 172.21.15.5:0/2236571941 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f5f5810a0f0 con 0x7f5f58101de0 2024-09-12T03:05:03.165 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:03.164+0000 7f5f4f7fe640 1 -- 172.21.15.5:0/2236571941 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v85) ==== 181+0+0 (secure 0 0 0) 0x7f5f5409b2e0 con 0x7f5f58101de0 2024-09-12T03:05:03.165 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_metadata' removed 2024-09-12T03:05:03.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:03.168+0000 7f5f607f0640 1 -- 172.21.15.5:0/2236571941 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f5f440a2b80 msgr2=0x7f5f440a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:03.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:03.168+0000 7f5f607f0640 1 --2- 172.21.15.5:0/2236571941 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f5f440a2b80 0x7f5f440a5040 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f5f48009230 tx=0x7f5f480058d0 comp rx=0 tx=0).stop 2024-09-12T03:05:03.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:03.168+0000 7f5f607f0640 1 -- 172.21.15.5:0/2236571941 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f58101de0 msgr2=0x7f5f580744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:03.169 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:03.168+0000 7f5f607f0640 1 --2- 172.21.15.5:0/2236571941 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f58101de0 0x7f5f580744f0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f5f5400d770 tx=0x7f5f5400dc30 comp rx=0 tx=0).stop 2024-09-12T03:05:03.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:03.168+0000 7f5f607f0640 1 -- 172.21.15.5:0/2236571941 shutdown_connections 2024-09-12T03:05:03.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:03.168+0000 7f5f607f0640 1 --2- 172.21.15.5:0/2236571941 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7f5f58106f10 0x7f5f58074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:03.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:03.168+0000 7f5f607f0640 1 --2- 172.21.15.5:0/2236571941 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7f5f58102790 0x7f5f58074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:03.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:03.168+0000 7f5f607f0640 1 --2- 172.21.15.5:0/2236571941 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7f5f440a2b80 0x7f5f440a5040 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:03.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:03.168+0000 7f5f607f0640 1 --2- 172.21.15.5:0/2236571941 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f58101de0 0x7f5f580744f0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:03.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:03.168+0000 7f5f607f0640 1 -- 172.21.15.5:0/2236571941 >> 172.21.15.5:0/2236571941 conn(0x7f5f580fd8c0 msgr2=0x7f5f580fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:03.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:03.169+0000 7f5f607f0640 1 -- 172.21.15.5:0/2236571941 shutdown_connections 2024-09-12T03:05:03.170 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:03.169+0000 7f5f607f0640 1 -- 172.21.15.5:0/2236571941 wait complete. 2024-09-12T03:05:03.454 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:05:03 smithi005 ceph-mon[31197]: pgmap v540: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.3 KiB/s wr, 11 op/s 2024-09-12T03:05:03.454 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:05:03 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/2236571941' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T03:05:03.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:05:03 smithi146 ceph-mon[35011]: pgmap v540: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.3 KiB/s wr, 11 op/s 2024-09-12T03:05:03.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:05:03 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/2236571941' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T03:05:03.665 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:05:03 smithi148 ceph-mon[35090]: pgmap v540: 97 pgs: 97 active+clean; 113 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.3 KiB/s wr, 11 op/s 2024-09-12T03:05:03.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:05:03 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/2236571941' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T03:05:03.849 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc shell --fsid 618305f8-70b1-11ef-bcea-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-12T03:05:04.122 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/mon.a/config 2024-09-12T03:05:04.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:05:04 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/2236571941' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-12T03:05:04.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:05:04 smithi146 ceph-mon[35011]: osdmap e85: 12 total, 12 up, 12 in 2024-09-12T03:05:04.487 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:05:04 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/2236571941' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-12T03:05:04.487 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:05:04 smithi005 ceph-mon[31197]: osdmap e85: 12 total, 12 up, 12 in 2024-09-12T03:05:04.665 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:05:04 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/2236571941' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-12T03:05:04.665 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:05:04 smithi148 ceph-mon[35090]: osdmap e85: 12 total, 12 up, 12 in 2024-09-12T03:05:04.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:04.703+0000 7efffd06e640 1 -- 172.21.15.5:0/625112290 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efff8101420 msgr2=0x7efff810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:04.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:04.703+0000 7efffd06e640 1 --2- 172.21.15.5:0/625112290 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efff8101420 0x7efff810d950 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7efff0009cd0 tx=0x7efff002a8e0 comp rx=0 tx=0).stop 2024-09-12T03:05:04.704 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:04.703+0000 7efffd06e640 1 -- 172.21.15.5:0/625112290 shutdown_connections 2024-09-12T03:05:04.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:04.703+0000 7efffd06e640 1 --2- 172.21.15.5:0/625112290 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efff8101420 0x7efff810d950 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:04.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:04.703+0000 7efffd06e640 1 --2- 172.21.15.5:0/625112290 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efff8100a80 0x7efff8100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:04.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:04.703+0000 7efffd06e640 1 --2- 172.21.15.5:0/625112290 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efff8106a80 0x7efff8106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:04.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:04.703+0000 7efffd06e640 1 -- 172.21.15.5:0/625112290 >> 172.21.15.5:0/625112290 conn(0x7efff80fc7a0 msgr2=0x7efff80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:04.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:04.703+0000 7efffd06e640 1 -- 172.21.15.5:0/625112290 shutdown_connections 2024-09-12T03:05:04.705 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:04.703+0000 7efffd06e640 1 -- 172.21.15.5:0/625112290 wait complete. 2024-09-12T03:05:04.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:04.704+0000 7efffd06e640 1 Processor -- start 2024-09-12T03:05:04.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:04.704+0000 7efffd06e640 1 -- start start 2024-09-12T03:05:04.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:04.704+0000 7efffd06e640 1 --2- >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efff8100a80 0x7efff8198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:04.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:04.704+0000 7efffd06e640 1 --2- >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efff8101420 0x7efff81989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:04.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:04.704+0000 7efffd06e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efff8106a80 0x7efff819cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:04.707 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:04.704+0000 7efffd06e640 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 -- 0x7efff8110ea0 con 0x7efff8106a80 2024-09-12T03:05:04.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:04.704+0000 7efffd06e640 1 -- --> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] -- mon_getmap magic: 0 -- 0x7efff8110d20 con 0x7efff8101420 2024-09-12T03:05:04.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:04.704+0000 7efffd06e640 1 -- --> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] -- mon_getmap magic: 0 -- 0x7efff8111020 con 0x7efff8100a80 2024-09-12T03:05:04.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:04.704+0000 7efff759e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efff8106a80 0x7efff819cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:04.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:04.704+0000 7efff759e640 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efff8106a80 0x7efff819cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54852/0 (socket says 172.21.15.5:54852) 2024-09-12T03:05:04.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:04.704+0000 7efff759e640 1 -- 172.21.15.5:0/3779341686 learned_addr learned my addr 172.21.15.5:0/3779341686 (peer_addr_for_me v2:172.21.15.5:0/0) 2024-09-12T03:05:04.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:04.704+0000 7effee59c640 1 --2- 172.21.15.5:0/3779341686 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efff8101420 0x7efff81989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:04.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:04.704+0000 7efff6d9d640 1 --2- 172.21.15.5:0/3779341686 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efff8100a80 0x7efff8198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:04.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:04.705+0000 7efff759e640 1 -- 172.21.15.5:0/3779341686 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efff8100a80 msgr2=0x7efff8198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:04.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:04.705+0000 7efff759e640 1 --2- 172.21.15.5:0/3779341686 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efff8100a80 0x7efff8198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:04.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:04.705+0000 7efff759e640 1 -- 172.21.15.5:0/3779341686 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efff8101420 msgr2=0x7efff81989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:04.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:04.705+0000 7efff759e640 1 --2- 172.21.15.5:0/3779341686 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efff8101420 0x7efff81989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:04.708 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:04.705+0000 7efff759e640 1 -- 172.21.15.5:0/3779341686 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efff819d430 con 0x7efff8106a80 2024-09-12T03:05:04.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:04.705+0000 7effee59c640 1 --2- 172.21.15.5:0/3779341686 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efff8101420 0x7efff81989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:05:04.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:04.705+0000 7efff6d9d640 1 --2- 172.21.15.5:0/3779341686 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efff8100a80 0x7efff8198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-12T03:05:04.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:04.705+0000 7efff759e640 1 --2- 172.21.15.5:0/3779341686 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efff8106a80 0x7efff819cd50 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7efff0005cc0 tx=0x7efff0003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:04.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:04.705+0000 7efff4d99640 1 -- 172.21.15.5:0/3779341686 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff0004040 con 0x7efff8106a80 2024-09-12T03:05:04.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:04.705+0000 7efff4d99640 1 -- 172.21.15.5:0/3779341686 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efff00046c0 con 0x7efff8106a80 2024-09-12T03:05:04.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:04.705+0000 7efffd06e640 1 -- 172.21.15.5:0/3779341686 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efff819d6c0 con 0x7efff8106a80 2024-09-12T03:05:04.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:04.705+0000 7efff4d99640 1 -- 172.21.15.5:0/3779341686 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff0046480 con 0x7efff8106a80 2024-09-12T03:05:04.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:04.705+0000 7efffd06e640 1 -- 172.21.15.5:0/3779341686 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efff81a5000 con 0x7efff8106a80 2024-09-12T03:05:04.709 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:04.706+0000 7efffd06e640 1 -- 172.21.15.5:0/3779341686 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efff8100ee0 con 0x7efff8106a80 2024-09-12T03:05:04.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:04.707+0000 7efff4d99640 1 -- 172.21.15.5:0/3779341686 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 20) ==== 137832+0+0 (secure 0 0 0) 0x7efff00305a0 con 0x7efff8106a80 2024-09-12T03:05:04.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:04.708+0000 7efff4d99640 1 --2- 172.21.15.5:0/3779341686 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7effe40a2b80 0x7effe40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-12T03:05:04.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:04.708+0000 7efff6d9d640 1 --2- 172.21.15.5:0/3779341686 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7effe40a2b80 0x7effe40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-12T03:05:04.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:04.708+0000 7efff4d99640 1 -- 172.21.15.5:0/3779341686 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8087+0+0 (secure 0 0 0) 0x7efff002e030 con 0x7efff8106a80 2024-09-12T03:05:04.710 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:04.709+0000 7efff6d9d640 1 --2- 172.21.15.5:0/3779341686 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7effe40a2b80 0x7effe40a5040 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7effe8009930 tx=0x7effe8008040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-12T03:05:04.712 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:04.711+0000 7efff4d99640 1 -- 172.21.15.5:0/3779341686 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efff004b060 con 0x7efff8106a80 2024-09-12T03:05:04.890 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:04.888+0000 7efffd06e640 1 -- 172.21.15.5:0/3779341686 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7efff81a5230 con 0x7efff8106a80 2024-09-12T03:05:05.182 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:05.180+0000 7efff4d99640 1 -- 172.21.15.5:0/3779341686 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v86) ==== 169+0+0 (secure 0 0 0) 0x7efff00bd9e0 con 0x7efff8106a80 2024-09-12T03:05:05.182 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_data' removed 2024-09-12T03:05:05.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:05.184+0000 7efffd06e640 1 -- 172.21.15.5:0/3779341686 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7effe40a2b80 msgr2=0x7effe40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:05.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:05.184+0000 7efffd06e640 1 --2- 172.21.15.5:0/3779341686 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7effe40a2b80 0x7effe40a5040 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7effe8009930 tx=0x7effe8008040 comp rx=0 tx=0).stop 2024-09-12T03:05:05.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:05.184+0000 7efffd06e640 1 -- 172.21.15.5:0/3779341686 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efff8106a80 msgr2=0x7efff819cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-12T03:05:05.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:05.184+0000 7efffd06e640 1 --2- 172.21.15.5:0/3779341686 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efff8106a80 0x7efff819cd50 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7efff0005cc0 tx=0x7efff0003970 comp rx=0 tx=0).stop 2024-09-12T03:05:05.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:05.185+0000 7efffd06e640 1 -- 172.21.15.5:0/3779341686 shutdown_connections 2024-09-12T03:05:05.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:05.185+0000 7efffd06e640 1 --2- 172.21.15.5:0/3779341686 >> [v2:172.21.15.5:6800/3780631642,v1:172.21.15.5:6801/3780631642] conn(0x7effe40a2b80 0x7effe40a5040 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:05.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:05.185+0000 7efffd06e640 1 --2- 172.21.15.5:0/3779341686 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efff8106a80 0x7efff819cd50 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:05.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:05.185+0000 7efffd06e640 1 --2- 172.21.15.5:0/3779341686 >> [v2:172.21.15.146:3300/0,v1:172.21.15.146:6789/0] conn(0x7efff8101420 0x7efff81989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:05.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:05.185+0000 7efffd06e640 1 --2- 172.21.15.5:0/3779341686 >> [v2:172.21.15.148:3300/0,v1:172.21.15.148:6789/0] conn(0x7efff8100a80 0x7efff8198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-12T03:05:05.186 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:05.185+0000 7efffd06e640 1 -- 172.21.15.5:0/3779341686 >> 172.21.15.5:0/3779341686 conn(0x7efff80fc7a0 msgr2=0x7efff806a830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-12T03:05:05.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:05.185+0000 7efffd06e640 1 -- 172.21.15.5:0/3779341686 shutdown_connections 2024-09-12T03:05:05.187 INFO:teuthology.orchestra.run.smithi005.stderr:2024-09-12T03:05:05.185+0000 7efffd06e640 1 -- 172.21.15.5:0/3779341686 wait complete. 2024-09-12T03:05:05.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:05:05 smithi146 ceph-mon[35011]: pgmap v542: 65 pgs: 65 active+clean; 577 KiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:05:05.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:05:05 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/3779341686' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T03:05:05.505 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:05:05 smithi005 ceph-mon[31197]: pgmap v542: 65 pgs: 65 active+clean; 577 KiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:05:05.505 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:05:05 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/3779341686' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T03:05:05.665 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:05:05 smithi148 ceph-mon[35090]: pgmap v542: 65 pgs: 65 active+clean; 577 KiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-12T03:05:05.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:05:05 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/3779341686' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-12T03:05:05.799 DEBUG:teuthology.orchestra.run.smithi005:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-12T03:05:05.830 DEBUG:teuthology.orchestra.run.smithi146:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-12T03:05:05.864 DEBUG:teuthology.orchestra.run.smithi148:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-12T03:05:05.902 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-12T03:05:05.902 DEBUG:teuthology.orchestra.run.smithi005:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-12T03:05:05.920 DEBUG:teuthology.orchestra.run.smithi146:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-12T03:05:05.938 DEBUG:teuthology.orchestra.run.smithi148:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-12T03:05:05.959 INFO:tasks.cephadm:Stopping all daemons... 2024-09-12T03:05:05.960 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-12T03:05:05.960 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ceph-618305f8-70b1-11ef-bcea-c7b262605968@mon.a 2024-09-12T03:05:06.060 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:05:06 smithi005 systemd[1]: Stopping Ceph mon.a for 618305f8-70b1-11ef-bcea-c7b262605968... 2024-09-12T03:05:06.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:05:06 smithi146 ceph-mon[35011]: from='client.? 172.21.15.5:0/3779341686' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-12T03:05:06.456 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:05:06 smithi146 ceph-mon[35011]: osdmap e86: 12 total, 12 up, 12 in 2024-09-12T03:05:06.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:05:06 smithi005 ceph-mon[31197]: from='client.? 172.21.15.5:0/3779341686' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-12T03:05:06.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:05:06 smithi005 ceph-mon[31197]: osdmap e86: 12 total, 12 up, 12 in 2024-09-12T03:05:06.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:05:06 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mon-a[31193]: 2024-09-12T03:05:06.317+0000 7f60e1dfc640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:05:06.560 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:05:06 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mon-a[31193]: 2024-09-12T03:05:06.317+0000 7f60e1dfc640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-12T03:05:06.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:05:06 smithi005 podman[117925]: 2024-09-12 03:05:06.390604898 +0000 UTC m=+0.225341780 container died 62861099a2acfdd3da3b829c4cbab84623c36b01d03f445532a93ebf5c6735dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-mon-a, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-12T03:05:06.665 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:05:06 smithi148 ceph-mon[35090]: from='client.? 172.21.15.5:0/3779341686' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-12T03:05:06.666 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:05:06 smithi148 ceph-mon[35090]: osdmap e86: 12 total, 12 up, 12 in 2024-09-12T03:05:07.026 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:05:06 smithi005 podman[117925]: 2024-09-12 03:05:06.733973904 +0000 UTC m=+0.568710790 container cleanup 62861099a2acfdd3da3b829c4cbab84623c36b01d03f445532a93ebf5c6735dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-mon-a, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T03:05:07.026 INFO:journalctl@ceph.mon.a.smithi005.stdout:Sep 12 03:05:06 smithi005 bash[117925]: ceph-618305f8-70b1-11ef-bcea-c7b262605968-mon-a 2024-09-12T03:05:07.167 DEBUG:teuthology.orchestra.run.smithi005:> sudo pkill -f 'journalctl -f -n 0 -u ceph-618305f8-70b1-11ef-bcea-c7b262605968@mon.a.service' 2024-09-12T03:05:07.220 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:05:07.220 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-12T03:05:07.220 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-09-12T03:05:07.220 DEBUG:teuthology.orchestra.run.smithi146:> sudo systemctl stop ceph-618305f8-70b1-11ef-bcea-c7b262605968@mon.b 2024-09-12T03:05:07.516 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:05:07 smithi146 systemd[1]: Stopping Ceph mon.b for 618305f8-70b1-11ef-bcea-c7b262605968... 2024-09-12T03:05:07.956 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:05:07 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mon-b[35007]: 2024-09-12T03:05:07.516+0000 7fb4997cc640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:05:07.956 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:05:07 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mon-b[35007]: 2024-09-12T03:05:07.516+0000 7fb4997cc640 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-09-12T03:05:07.956 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:05:07 smithi146 podman[68897]: 2024-09-12 03:05:07.71051622 +0000 UTC m=+0.288238352 container died 50219fda65c4452d947aba33e6b0e42a99d42fd23b4badf735fc1c2b5357c25d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-mon-b, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-09-12T03:05:08.266 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:05:08 smithi146 podman[68897]: 2024-09-12 03:05:08.040155164 +0000 UTC m=+0.617877297 container cleanup 50219fda65c4452d947aba33e6b0e42a99d42fd23b4badf735fc1c2b5357c25d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-mon-b, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-12T03:05:08.266 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:05:08 smithi146 bash[68897]: ceph-618305f8-70b1-11ef-bcea-c7b262605968-mon-b 2024-09-12T03:05:08.516 DEBUG:teuthology.orchestra.run.smithi146:> sudo pkill -f 'journalctl -f -n 0 -u ceph-618305f8-70b1-11ef-bcea-c7b262605968@mon.b.service' 2024-09-12T03:05:08.538 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:05:08 smithi146 podman[68910]: 2024-09-12 03:05:08.266356904 +0000 UTC m=+0.550527797 container remove 50219fda65c4452d947aba33e6b0e42a99d42fd23b4badf735fc1c2b5357c25d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-mon-b, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-12T03:05:08.538 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:05:08 smithi146 systemd[1]: ceph-618305f8-70b1-11ef-bcea-c7b262605968@mon.b.service: Deactivated successfully. 2024-09-12T03:05:08.538 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:05:08 smithi146 systemd[1]: Stopped Ceph mon.b for 618305f8-70b1-11ef-bcea-c7b262605968. 2024-09-12T03:05:08.538 INFO:journalctl@ceph.mon.b.smithi146.stdout:Sep 12 03:05:08 smithi146 systemd[1]: ceph-618305f8-70b1-11ef-bcea-c7b262605968@mon.b.service: Consumed 9.537s CPU time. 2024-09-12T03:05:08.561 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:05:08.561 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-09-12T03:05:08.562 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-09-12T03:05:08.562 DEBUG:teuthology.orchestra.run.smithi148:> sudo systemctl stop ceph-618305f8-70b1-11ef-bcea-c7b262605968@mon.c 2024-09-12T03:05:08.866 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:05:08 smithi148 systemd[1]: Stopping Ceph mon.c for 618305f8-70b1-11ef-bcea-c7b262605968... 2024-09-12T03:05:09.165 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:05:08 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mon-c[35086]: 2024-09-12T03:05:08.864+0000 7fe9480c6640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:05:09.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:05:08 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mon-c[35086]: 2024-09-12T03:05:08.864+0000 7fe9480c6640 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-09-12T03:05:09.166 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:05:09 smithi148 podman[68897]: 2024-09-12 03:05:09.101035874 +0000 UTC m=+0.338450436 container died faab55de658f2675968742874a556c0ab1a65b38e664778596137e7746fd9011 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-mon-c, io.buildah.version=1.37.2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-12T03:05:09.642 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:05:09 smithi148 podman[68897]: 2024-09-12 03:05:09.432829995 +0000 UTC m=+0.670244512 container cleanup faab55de658f2675968742874a556c0ab1a65b38e664778596137e7746fd9011 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-mon-c, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T03:05:09.642 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:05:09 smithi148 bash[68897]: ceph-618305f8-70b1-11ef-bcea-c7b262605968-mon-c 2024-09-12T03:05:09.855 DEBUG:teuthology.orchestra.run.smithi148:> sudo pkill -f 'journalctl -f -n 0 -u ceph-618305f8-70b1-11ef-bcea-c7b262605968@mon.c.service' 2024-09-12T03:05:09.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:05:09 smithi148 podman[68909]: 2024-09-12 03:05:09.642435263 +0000 UTC m=+0.535289229 container remove faab55de658f2675968742874a556c0ab1a65b38e664778596137e7746fd9011 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-mon-c, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-12T03:05:09.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:05:09 smithi148 systemd[1]: ceph-618305f8-70b1-11ef-bcea-c7b262605968@mon.c.service: Deactivated successfully. 2024-09-12T03:05:09.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:05:09 smithi148 systemd[1]: Stopped Ceph mon.c for 618305f8-70b1-11ef-bcea-c7b262605968. 2024-09-12T03:05:09.916 INFO:journalctl@ceph.mon.c.smithi148.stdout:Sep 12 03:05:09 smithi148 systemd[1]: ceph-618305f8-70b1-11ef-bcea-c7b262605968@mon.c.service: Consumed 9.636s CPU time. 2024-09-12T03:05:09.931 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:05:09.932 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-09-12T03:05:09.932 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-12T03:05:09.932 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ceph-618305f8-70b1-11ef-bcea-c7b262605968@mgr.x 2024-09-12T03:05:10.238 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 03:05:09 smithi005 systemd[1]: Stopping Ceph mgr.x for 618305f8-70b1-11ef-bcea-c7b262605968... 2024-09-12T03:05:10.497 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 03:05:10 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: 2024-09-12T03:05:10.238+0000 7ff9216a6640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:05:10.498 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 03:05:10 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x[31423]: 2024-09-12T03:05:10.238+0000 7ff9216a6640 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-09-12T03:05:10.498 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 03:05:10 smithi005 podman[118045]: 2024-09-12 03:05:10.269834135 +0000 UTC m=+0.134903688 container died 929a8c33032e68c39cb5f45436bea78f8b9569c82e7e400dbd5f966b6c9228ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=) 2024-09-12T03:05:10.810 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 03:05:10 smithi005 podman[118045]: 2024-09-12 03:05:10.697323401 +0000 UTC m=+0.562392953 container cleanup 929a8c33032e68c39cb5f45436bea78f8b9569c82e7e400dbd5f966b6c9228ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-09-12T03:05:10.810 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 03:05:10 smithi005 bash[118045]: ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x 2024-09-12T03:05:11.294 DEBUG:teuthology.orchestra.run.smithi005:> sudo pkill -f 'journalctl -f -n 0 -u ceph-618305f8-70b1-11ef-bcea-c7b262605968@mgr.x.service' 2024-09-12T03:05:11.310 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 03:05:11 smithi005 podman[118058]: 2024-09-12 03:05:11.049154034 +0000 UTC m=+0.776760862 container remove 929a8c33032e68c39cb5f45436bea78f8b9569c82e7e400dbd5f966b6c9228ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:b53b6d3e9f4baeb64d08dfb72b9cb1112ce886fc, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-x, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True) 2024-09-12T03:05:11.310 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 03:05:11 smithi005 systemd[1]: ceph-618305f8-70b1-11ef-bcea-c7b262605968@mgr.x.service: Deactivated successfully. 2024-09-12T03:05:11.310 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 03:05:11 smithi005 systemd[1]: Stopped Ceph mgr.x for 618305f8-70b1-11ef-bcea-c7b262605968. 2024-09-12T03:05:11.310 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Sep 12 03:05:11 smithi005 systemd[1]: ceph-618305f8-70b1-11ef-bcea-c7b262605968@mgr.x.service: Consumed 46.791s CPU time. 2024-09-12T03:05:11.331 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:05:11.331 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-12T03:05:11.331 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-12T03:05:11.332 DEBUG:teuthology.orchestra.run.smithi146:> sudo systemctl stop ceph-618305f8-70b1-11ef-bcea-c7b262605968@mgr.y 2024-09-12T03:05:11.629 INFO:journalctl@ceph.mgr.y.smithi146.stdout:Sep 12 03:05:11 smithi146 systemd[1]: Stopping Ceph mgr.y for 618305f8-70b1-11ef-bcea-c7b262605968... 2024-09-12T03:05:11.956 INFO:journalctl@ceph.mgr.y.smithi146.stdout:Sep 12 03:05:11 smithi146 podman[69020]: 2024-09-12 03:05:11.665069636 +0000 UTC m=+0.140300310 container died 44071781cc3203a9c7001dd091a183c375a76de4586c000a7f655e6c8beb0c7b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True) 2024-09-12T03:05:12.312 INFO:journalctl@ceph.mgr.y.smithi146.stdout:Sep 12 03:05:12 smithi146 podman[69020]: 2024-09-12 03:05:12.103201933 +0000 UTC m=+0.578432606 container cleanup 44071781cc3203a9c7001dd091a183c375a76de4586c000a7f655e6c8beb0c7b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-y, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS) 2024-09-12T03:05:12.312 INFO:journalctl@ceph.mgr.y.smithi146.stdout:Sep 12 03:05:12 smithi146 bash[69020]: ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-y 2024-09-12T03:05:12.530 DEBUG:teuthology.orchestra.run.smithi146:> sudo pkill -f 'journalctl -f -n 0 -u ceph-618305f8-70b1-11ef-bcea-c7b262605968@mgr.y.service' 2024-09-12T03:05:12.565 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:05:12.565 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-12T03:05:12.565 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-09-12T03:05:12.565 DEBUG:teuthology.orchestra.run.smithi148:> sudo systemctl stop ceph-618305f8-70b1-11ef-bcea-c7b262605968@mgr.z 2024-09-12T03:05:12.915 INFO:journalctl@ceph.mgr.z.smithi148.stdout:Sep 12 03:05:12 smithi148 systemd[1]: Stopping Ceph mgr.z for 618305f8-70b1-11ef-bcea-c7b262605968... 2024-09-12T03:05:13.185 INFO:journalctl@ceph.mgr.z.smithi148.stdout:Sep 12 03:05:12 smithi148 podman[69020]: 2024-09-12 03:05:12.950012325 +0000 UTC m=+0.186003033 container died c05a997c2d4fe74dc6427091f5e8e13875beedc85e2b12334da9c4a4a0fbf65c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-z, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-12T03:05:13.597 INFO:journalctl@ceph.mgr.z.smithi148.stdout:Sep 12 03:05:13 smithi148 podman[69020]: 2024-09-12 03:05:13.337343593 +0000 UTC m=+0.573334304 container cleanup c05a997c2d4fe74dc6427091f5e8e13875beedc85e2b12334da9c4a4a0fbf65c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-z, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-12T03:05:13.597 INFO:journalctl@ceph.mgr.z.smithi148.stdout:Sep 12 03:05:13 smithi148 bash[69020]: ceph-618305f8-70b1-11ef-bcea-c7b262605968-mgr-z 2024-09-12T03:05:13.728 DEBUG:teuthology.orchestra.run.smithi148:> sudo pkill -f 'journalctl -f -n 0 -u ceph-618305f8-70b1-11ef-bcea-c7b262605968@mgr.z.service' 2024-09-12T03:05:13.764 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:05:13.764 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-09-12T03:05:13.764 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-12T03:05:13.765 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ceph-618305f8-70b1-11ef-bcea-c7b262605968@osd.0 2024-09-12T03:05:14.060 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 12 03:05:13 smithi005 systemd[1]: Stopping Ceph osd.0 for 618305f8-70b1-11ef-bcea-c7b262605968... 2024-09-12T03:05:14.560 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 12 03:05:14 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-0[42657]: 2024-09-12T03:05:14.067+0000 7f63eae18640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:05:14.560 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 12 03:05:14 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-0[42657]: 2024-09-12T03:05:14.067+0000 7f63eae18640 -1 osd.0 86 *** Got signal Terminated *** 2024-09-12T03:05:14.560 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 12 03:05:14 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-0[42657]: 2024-09-12T03:05:14.067+0000 7f63eae18640 -1 osd.0 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T03:05:19.661 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 12 03:05:19 smithi005 podman[118167]: 2024-09-12 03:05:19.391992066 +0000 UTC m=+5.426314642 container died 67834bd03595a1cb3a0d429223ab2e7c065d8345017db5b6c591f356f4686664 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0) 2024-09-12T03:05:20.060 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 12 03:05:19 smithi005 podman[118167]: 2024-09-12 03:05:19.844132098 +0000 UTC m=+5.878454672 container cleanup 67834bd03595a1cb3a0d429223ab2e7c065d8345017db5b6c591f356f4686664 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-0, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731) 2024-09-12T03:05:20.060 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 12 03:05:19 smithi005 bash[118167]: ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-0 2024-09-12T03:05:20.471 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 12 03:05:20 smithi005 podman[118187]: 2024-09-12 03:05:20.137283346 +0000 UTC m=+0.740577695 container remove 67834bd03595a1cb3a0d429223ab2e7c065d8345017db5b6c591f356f4686664 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3) 2024-09-12T03:05:20.806 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 12 03:05:20 smithi005 podman[118276]: 2024-09-12 03:05:20.471440961 +0000 UTC m=+0.097323549 container create bc2995098504109b96079991068cf22790b02d2e671cf8dea6eb43f1a6559949 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-0-deactivate, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-12T03:05:20.807 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 12 03:05:20 smithi005 podman[118276]: 2024-09-12 03:05:20.390413125 +0000 UTC m=+0.016295683 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:05:20.807 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 12 03:05:20 smithi005 podman[118276]: 2024-09-12 03:05:20.707265466 +0000 UTC m=+0.333148013 container init bc2995098504109b96079991068cf22790b02d2e671cf8dea6eb43f1a6559949 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-0-deactivate, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-12T03:05:20.807 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 12 03:05:20 smithi005 podman[118276]: 2024-09-12 03:05:20.710677384 +0000 UTC m=+0.336559943 container start bc2995098504109b96079991068cf22790b02d2e671cf8dea6eb43f1a6559949 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.37.2) 2024-09-12T03:05:20.807 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 12 03:05:20 smithi005 podman[118276]: 2024-09-12 03:05:20.807110774 +0000 UTC m=+0.432993337 container attach bc2995098504109b96079991068cf22790b02d2e671cf8dea6eb43f1a6559949 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-0-deactivate, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T03:05:21.060 INFO:journalctl@ceph.osd.0.smithi005.stdout:Sep 12 03:05:20 smithi005 podman[118276]: 2024-09-12 03:05:20.91344668 +0000 UTC m=+0.539329234 container died bc2995098504109b96079991068cf22790b02d2e671cf8dea6eb43f1a6559949 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-0-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-12T03:05:21.568 DEBUG:teuthology.orchestra.run.smithi005:> sudo pkill -f 'journalctl -f -n 0 -u ceph-618305f8-70b1-11ef-bcea-c7b262605968@osd.0.service' 2024-09-12T03:05:21.609 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:05:21.609 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-12T03:05:21.609 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-12T03:05:21.610 DEBUG:teuthology.orchestra.run.smithi146:> sudo systemctl stop ceph-618305f8-70b1-11ef-bcea-c7b262605968@osd.1 2024-09-12T03:05:21.915 INFO:journalctl@ceph.osd.1.smithi146.stdout:Sep 12 03:05:21 smithi146 systemd[1]: Stopping Ceph osd.1 for 618305f8-70b1-11ef-bcea-c7b262605968... 2024-09-12T03:05:22.206 INFO:journalctl@ceph.osd.1.smithi146.stdout:Sep 12 03:05:21 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-1[39529]: 2024-09-12T03:05:21.914+0000 7f281b75e640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:05:22.206 INFO:journalctl@ceph.osd.1.smithi146.stdout:Sep 12 03:05:21 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-1[39529]: 2024-09-12T03:05:21.914+0000 7f281b75e640 -1 osd.1 86 *** Got signal Terminated *** 2024-09-12T03:05:22.206 INFO:journalctl@ceph.osd.1.smithi146.stdout:Sep 12 03:05:21 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-1[39529]: 2024-09-12T03:05:21.914+0000 7f281b75e640 -1 osd.1 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T03:05:27.667 INFO:journalctl@ceph.osd.1.smithi146.stdout:Sep 12 03:05:27 smithi146 podman[69147]: 2024-09-12 03:05:27.333554395 +0000 UTC m=+5.532532692 container died d193e2b4cafe3d61ac3efe5f8ead04ccae0e781ca65df9e218f692b29dbc75c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-1, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-12T03:05:27.956 INFO:journalctl@ceph.osd.1.smithi146.stdout:Sep 12 03:05:27 smithi146 podman[69147]: 2024-09-12 03:05:27.740034529 +0000 UTC m=+5.939012786 container cleanup d193e2b4cafe3d61ac3efe5f8ead04ccae0e781ca65df9e218f692b29dbc75c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-1, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-12T03:05:27.956 INFO:journalctl@ceph.osd.1.smithi146.stdout:Sep 12 03:05:27 smithi146 bash[69147]: ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-1 2024-09-12T03:05:28.305 INFO:journalctl@ceph.osd.1.smithi146.stdout:Sep 12 03:05:27 smithi146 podman[69162]: 2024-09-12 03:05:27.982848775 +0000 UTC m=+0.642059395 container remove d193e2b4cafe3d61ac3efe5f8ead04ccae0e781ca65df9e218f692b29dbc75c5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-1, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2) 2024-09-12T03:05:28.612 INFO:journalctl@ceph.osd.1.smithi146.stdout:Sep 12 03:05:28 smithi146 podman[69251]: 2024-09-12 03:05:28.208325968 +0000 UTC m=+0.015997430 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:05:28.612 INFO:journalctl@ceph.osd.1.smithi146.stdout:Sep 12 03:05:28 smithi146 podman[69251]: 2024-09-12 03:05:28.310738603 +0000 UTC m=+0.118410059 container create b4115939044c84c02d421842d25c8d726aa50938853673435842b33f1461186f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-12T03:05:28.613 INFO:journalctl@ceph.osd.1.smithi146.stdout:Sep 12 03:05:28 smithi146 podman[69251]: 2024-09-12 03:05:28.538464113 +0000 UTC m=+0.346135583 container init b4115939044c84c02d421842d25c8d726aa50938853673435842b33f1461186f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-1-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-12T03:05:28.613 INFO:journalctl@ceph.osd.1.smithi146.stdout:Sep 12 03:05:28 smithi146 podman[69251]: 2024-09-12 03:05:28.542198955 +0000 UTC m=+0.349870415 container start b4115939044c84c02d421842d25c8d726aa50938853673435842b33f1461186f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T03:05:28.613 INFO:journalctl@ceph.osd.1.smithi146.stdout:Sep 12 03:05:28 smithi146 podman[69251]: 2024-09-12 03:05:28.613037254 +0000 UTC m=+0.420708711 container attach b4115939044c84c02d421842d25c8d726aa50938853673435842b33f1461186f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.build-date=20240731, ceph=True, RELEASE=squid-b53b6d3, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-12T03:05:28.956 INFO:journalctl@ceph.osd.1.smithi146.stdout:Sep 12 03:05:28 smithi146 podman[69251]: 2024-09-12 03:05:28.727027816 +0000 UTC m=+0.534699275 container died b4115939044c84c02d421842d25c8d726aa50938853673435842b33f1461186f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T03:05:29.364 DEBUG:teuthology.orchestra.run.smithi146:> sudo pkill -f 'journalctl -f -n 0 -u ceph-618305f8-70b1-11ef-bcea-c7b262605968@osd.1.service' 2024-09-12T03:05:29.403 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:05:29.404 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-12T03:05:29.404 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-12T03:05:29.404 DEBUG:teuthology.orchestra.run.smithi148:> sudo systemctl stop ceph-618305f8-70b1-11ef-bcea-c7b262605968@osd.2 2024-09-12T03:05:29.708 INFO:journalctl@ceph.osd.2.smithi148.stdout:Sep 12 03:05:29 smithi148 systemd[1]: Stopping Ceph osd.2 for 618305f8-70b1-11ef-bcea-c7b262605968... 2024-09-12T03:05:30.165 INFO:journalctl@ceph.osd.2.smithi148.stdout:Sep 12 03:05:29 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-2[40096]: 2024-09-12T03:05:29.707+0000 7fb2a4140640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:05:30.166 INFO:journalctl@ceph.osd.2.smithi148.stdout:Sep 12 03:05:29 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-2[40096]: 2024-09-12T03:05:29.707+0000 7fb2a4140640 -1 osd.2 86 *** Got signal Terminated *** 2024-09-12T03:05:30.166 INFO:journalctl@ceph.osd.2.smithi148.stdout:Sep 12 03:05:29 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-2[40096]: 2024-09-12T03:05:29.707+0000 7fb2a4140640 -1 osd.2 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T03:05:35.400 INFO:journalctl@ceph.osd.2.smithi148.stdout:Sep 12 03:05:35 smithi148 podman[69154]: 2024-09-12 03:05:35.128782176 +0000 UTC m=+5.529301551 container died a58b36cf2e8cf3f866fe1a3aae88b6b90d52147e56c09cb88b81f02ed0a0f4eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-2, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-12T03:05:35.665 INFO:journalctl@ceph.osd.2.smithi148.stdout:Sep 12 03:05:35 smithi148 podman[69154]: 2024-09-12 03:05:35.490649942 +0000 UTC m=+5.891169315 container cleanup a58b36cf2e8cf3f866fe1a3aae88b6b90d52147e56c09cb88b81f02ed0a0f4eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-2, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-12T03:05:35.665 INFO:journalctl@ceph.osd.2.smithi148.stdout:Sep 12 03:05:35 smithi148 bash[69154]: ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-2 2024-09-12T03:05:36.105 INFO:journalctl@ceph.osd.2.smithi148.stdout:Sep 12 03:05:35 smithi148 podman[69171]: 2024-09-12 03:05:35.783835963 +0000 UTC m=+0.646996919 container remove a58b36cf2e8cf3f866fe1a3aae88b6b90d52147e56c09cb88b81f02ed0a0f4eb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-2, org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T03:05:36.369 INFO:journalctl@ceph.osd.2.smithi148.stdout:Sep 12 03:05:36 smithi148 podman[69256]: 2024-09-12 03:05:36.008350923 +0000 UTC m=+0.015943161 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:05:36.369 INFO:journalctl@ceph.osd.2.smithi148.stdout:Sep 12 03:05:36 smithi148 podman[69256]: 2024-09-12 03:05:36.117853076 +0000 UTC m=+0.125445349 container create 33ec2d9b97ebe7cfc2d52b47486c9c143e6d8f8de59d154fe50075fadbed195e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-2-deactivate, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-12T03:05:36.666 INFO:journalctl@ceph.osd.2.smithi148.stdout:Sep 12 03:05:36 smithi148 podman[69256]: 2024-09-12 03:05:36.370004105 +0000 UTC m=+0.377596340 container init 33ec2d9b97ebe7cfc2d52b47486c9c143e6d8f8de59d154fe50075fadbed195e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-12T03:05:36.666 INFO:journalctl@ceph.osd.2.smithi148.stdout:Sep 12 03:05:36 smithi148 podman[69256]: 2024-09-12 03:05:36.373409702 +0000 UTC m=+0.381001939 container start 33ec2d9b97ebe7cfc2d52b47486c9c143e6d8f8de59d154fe50075fadbed195e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS) 2024-09-12T03:05:36.666 INFO:journalctl@ceph.osd.2.smithi148.stdout:Sep 12 03:05:36 smithi148 podman[69256]: 2024-09-12 03:05:36.453264233 +0000 UTC m=+0.460856466 container attach 33ec2d9b97ebe7cfc2d52b47486c9c143e6d8f8de59d154fe50075fadbed195e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-09-12T03:05:36.666 INFO:journalctl@ceph.osd.2.smithi148.stdout:Sep 12 03:05:36 smithi148 podman[69256]: 2024-09-12 03:05:36.571075323 +0000 UTC m=+0.578667555 container died 33ec2d9b97ebe7cfc2d52b47486c9c143e6d8f8de59d154fe50075fadbed195e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS) 2024-09-12T03:05:37.209 DEBUG:teuthology.orchestra.run.smithi148:> sudo pkill -f 'journalctl -f -n 0 -u ceph-618305f8-70b1-11ef-bcea-c7b262605968@osd.2.service' 2024-09-12T03:05:37.250 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:05:37.251 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-12T03:05:37.251 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-12T03:05:37.251 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ceph-618305f8-70b1-11ef-bcea-c7b262605968@osd.3 2024-09-12T03:05:37.560 INFO:journalctl@ceph.osd.3.smithi005.stdout:Sep 12 03:05:37 smithi005 systemd[1]: Stopping Ceph osd.3 for 618305f8-70b1-11ef-bcea-c7b262605968... 2024-09-12T03:05:38.060 INFO:journalctl@ceph.osd.3.smithi005.stdout:Sep 12 03:05:37 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-3[49554]: 2024-09-12T03:05:37.571+0000 7f971bb37640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:05:38.060 INFO:journalctl@ceph.osd.3.smithi005.stdout:Sep 12 03:05:37 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-3[49554]: 2024-09-12T03:05:37.571+0000 7f971bb37640 -1 osd.3 86 *** Got signal Terminated *** 2024-09-12T03:05:38.060 INFO:journalctl@ceph.osd.3.smithi005.stdout:Sep 12 03:05:37 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-3[49554]: 2024-09-12T03:05:37.571+0000 7f971bb37640 -1 osd.3 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T03:05:40.206 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:05:39 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:05:39.942+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:18.916404+0000 front 2024-09-12T03:05:18.916319+0000 (oldest deadline 2024-09-12T03:05:39.416329+0000) 2024-09-12T03:05:40.560 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:05:40 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:05:40.243+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:14.571718+0000 front 2024-09-12T03:05:14.571768+0000 (oldest deadline 2024-09-12T03:05:39.772780+0000) 2024-09-12T03:05:41.456 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:05:40 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:05:40.956+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:18.916404+0000 front 2024-09-12T03:05:18.916319+0000 (oldest deadline 2024-09-12T03:05:39.416329+0000) 2024-09-12T03:05:41.560 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:05:41 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:05:41.247+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:14.571718+0000 front 2024-09-12T03:05:14.571768+0000 (oldest deadline 2024-09-12T03:05:39.772780+0000) 2024-09-12T03:05:42.060 INFO:journalctl@ceph.osd.3.smithi005.stdout:Sep 12 03:05:41 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-3[49554]: 2024-09-12T03:05:41.772+0000 7f971813c640 -1 osd.3 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:18.060412+0000 front 2024-09-12T03:05:18.060439+0000 (oldest deadline 2024-09-12T03:05:41.560548+0000) 2024-09-12T03:05:42.206 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:05:41 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:05:41.967+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:18.916404+0000 front 2024-09-12T03:05:18.916319+0000 (oldest deadline 2024-09-12T03:05:39.416329+0000) 2024-09-12T03:05:42.560 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:05:42 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:05:42.249+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:14.571718+0000 front 2024-09-12T03:05:14.571768+0000 (oldest deadline 2024-09-12T03:05:39.772780+0000) 2024-09-12T03:05:42.956 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:05:42 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:05:42.933+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:18.916404+0000 front 2024-09-12T03:05:18.916319+0000 (oldest deadline 2024-09-12T03:05:39.416329+0000) 2024-09-12T03:05:43.166 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 12 03:05:42 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-5[46768]: 2024-09-12T03:05:42.881+0000 7fcd0c8ad640 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:17.282199+0000 front 2024-09-12T03:05:17.282098+0000 (oldest deadline 2024-09-12T03:05:42.581953+0000) 2024-09-12T03:05:43.166 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:05:43 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:05:43.009+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:16.844053+0000 front 2024-09-12T03:05:16.843858+0000 (oldest deadline 2024-09-12T03:05:42.644259+0000) 2024-09-12T03:05:43.239 INFO:journalctl@ceph.osd.3.smithi005.stdout:Sep 12 03:05:42 smithi005 podman[118378]: 2024-09-12 03:05:42.975663931 +0000 UTC m=+5.524484810 container died f15ebd611d258cbc6946911db52ecf209f26aa8d2de8e69e676f946ef63a4156 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-3, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T03:05:43.239 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 12 03:05:42 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-6[56400]: 2024-09-12T03:05:42.908+0000 7f8e4a34d640 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:19.024576+0000 front 2024-09-12T03:05:19.024530+0000 (oldest deadline 2024-09-12T03:05:42.524750+0000) 2024-09-12T03:05:43.560 INFO:journalctl@ceph.osd.3.smithi005.stdout:Sep 12 03:05:43 smithi005 podman[118378]: 2024-09-12 03:05:43.413424657 +0000 UTC m=+5.962245533 container cleanup f15ebd611d258cbc6946911db52ecf209f26aa8d2de8e69e676f946ef63a4156 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-3, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-12T03:05:43.560 INFO:journalctl@ceph.osd.3.smithi005.stdout:Sep 12 03:05:43 smithi005 bash[118378]: ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-3 2024-09-12T03:05:43.560 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:05:43 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:05:43.251+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:14.571718+0000 front 2024-09-12T03:05:14.571768+0000 (oldest deadline 2024-09-12T03:05:39.772780+0000) 2024-09-12T03:05:43.896 INFO:journalctl@ceph.osd.3.smithi005.stdout:Sep 12 03:05:43 smithi005 podman[118406]: 2024-09-12 03:05:43.640277976 +0000 UTC m=+0.658785743 container remove f15ebd611d258cbc6946911db52ecf209f26aa8d2de8e69e676f946ef63a4156 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-12T03:05:44.198 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 12 03:05:43 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-6[56400]: 2024-09-12T03:05:43.896+0000 7f8e4a34d640 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:19.024576+0000 front 2024-09-12T03:05:19.024530+0000 (oldest deadline 2024-09-12T03:05:42.524750+0000) 2024-09-12T03:05:44.198 INFO:journalctl@ceph.osd.3.smithi005.stdout:Sep 12 03:05:44 smithi005 podman[118495]: 2024-09-12 03:05:44.087208406 +0000 UTC m=+0.108679214 container create 65b8b90e8fcb134d825fd7812bfca478c6828cf36b9f68d209bff63322911f08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731) 2024-09-12T03:05:44.199 INFO:journalctl@ceph.osd.3.smithi005.stdout:Sep 12 03:05:44 smithi005 podman[118495]: 2024-09-12 03:05:43.995023482 +0000 UTC m=+0.016494260 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:05:44.206 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:05:43 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:05:43.899+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:18.916404+0000 front 2024-09-12T03:05:18.916319+0000 (oldest deadline 2024-09-12T03:05:39.416329+0000) 2024-09-12T03:05:44.415 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 12 03:05:43 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-5[46768]: 2024-09-12T03:05:43.915+0000 7fcd0c8ad640 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:17.282199+0000 front 2024-09-12T03:05:17.282098+0000 (oldest deadline 2024-09-12T03:05:42.581953+0000) 2024-09-12T03:05:44.416 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:05:44 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:05:44.045+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:16.844053+0000 front 2024-09-12T03:05:16.843858+0000 (oldest deadline 2024-09-12T03:05:42.644259+0000) 2024-09-12T03:05:44.503 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:05:44 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:05:44.275+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:14.571718+0000 front 2024-09-12T03:05:14.571768+0000 (oldest deadline 2024-09-12T03:05:39.772780+0000) 2024-09-12T03:05:44.504 INFO:journalctl@ceph.osd.3.smithi005.stdout:Sep 12 03:05:44 smithi005 podman[118495]: 2024-09-12 03:05:44.31006031 +0000 UTC m=+0.331531077 container init 65b8b90e8fcb134d825fd7812bfca478c6828cf36b9f68d209bff63322911f08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-12T03:05:44.504 INFO:journalctl@ceph.osd.3.smithi005.stdout:Sep 12 03:05:44 smithi005 podman[118495]: 2024-09-12 03:05:44.313463432 +0000 UTC m=+0.334934216 container start 65b8b90e8fcb134d825fd7812bfca478c6828cf36b9f68d209bff63322911f08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-3-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-12T03:05:44.504 INFO:journalctl@ceph.osd.3.smithi005.stdout:Sep 12 03:05:44 smithi005 podman[118495]: 2024-09-12 03:05:44.393266865 +0000 UTC m=+0.414737632 container attach 65b8b90e8fcb134d825fd7812bfca478c6828cf36b9f68d209bff63322911f08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-3-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T03:05:44.779 INFO:journalctl@ceph.osd.3.smithi005.stdout:Sep 12 03:05:44 smithi005 podman[118495]: 2024-09-12 03:05:44.504472289 +0000 UTC m=+0.525943067 container died 65b8b90e8fcb134d825fd7812bfca478c6828cf36b9f68d209bff63322911f08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2) 2024-09-12T03:05:45.060 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 12 03:05:44 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-6[56400]: 2024-09-12T03:05:44.869+0000 7f8e4a34d640 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:19.024576+0000 front 2024-09-12T03:05:19.024530+0000 (oldest deadline 2024-09-12T03:05:42.524750+0000) 2024-09-12T03:05:45.171 DEBUG:teuthology.orchestra.run.smithi005:> sudo pkill -f 'journalctl -f -n 0 -u ceph-618305f8-70b1-11ef-bcea-c7b262605968@osd.3.service' 2024-09-12T03:05:45.206 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:05:44 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:05:44.948+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:18.916404+0000 front 2024-09-12T03:05:18.916319+0000 (oldest deadline 2024-09-12T03:05:39.416329+0000) 2024-09-12T03:05:45.216 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:05:45.217 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-12T03:05:45.217 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-12T03:05:45.217 DEBUG:teuthology.orchestra.run.smithi146:> sudo systemctl stop ceph-618305f8-70b1-11ef-bcea-c7b262605968@osd.4 2024-09-12T03:05:45.416 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 12 03:05:44 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-5[46768]: 2024-09-12T03:05:44.948+0000 7fcd0c8ad640 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:17.282199+0000 front 2024-09-12T03:05:17.282098+0000 (oldest deadline 2024-09-12T03:05:42.581953+0000) 2024-09-12T03:05:45.416 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:05:45 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:05:45.024+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:16.844053+0000 front 2024-09-12T03:05:16.843858+0000 (oldest deadline 2024-09-12T03:05:42.644259+0000) 2024-09-12T03:05:45.544 INFO:journalctl@ceph.osd.4.smithi146.stdout:Sep 12 03:05:45 smithi146 systemd[1]: Stopping Ceph osd.4 for 618305f8-70b1-11ef-bcea-c7b262605968... 2024-09-12T03:05:45.560 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:05:45 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:05:45.323+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:14.571718+0000 front 2024-09-12T03:05:14.571768+0000 (oldest deadline 2024-09-12T03:05:39.772780+0000) 2024-09-12T03:05:45.956 INFO:journalctl@ceph.osd.4.smithi146.stdout:Sep 12 03:05:45 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-4[46161]: 2024-09-12T03:05:45.543+0000 7fd5f5ab5640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:05:45.956 INFO:journalctl@ceph.osd.4.smithi146.stdout:Sep 12 03:05:45 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-4[46161]: 2024-09-12T03:05:45.543+0000 7fd5f5ab5640 -1 osd.4 86 *** Got signal Terminated *** 2024-09-12T03:05:45.956 INFO:journalctl@ceph.osd.4.smithi146.stdout:Sep 12 03:05:45 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-4[46161]: 2024-09-12T03:05:45.543+0000 7fd5f5ab5640 -1 osd.4 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T03:05:46.165 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 12 03:05:45 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-5[46768]: 2024-09-12T03:05:45.908+0000 7fcd0c8ad640 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:17.282199+0000 front 2024-09-12T03:05:17.282098+0000 (oldest deadline 2024-09-12T03:05:42.581953+0000) 2024-09-12T03:05:46.166 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:05:46 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:05:46.042+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:16.844053+0000 front 2024-09-12T03:05:16.843858+0000 (oldest deadline 2024-09-12T03:05:42.644259+0000) 2024-09-12T03:05:46.295 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 12 03:05:45 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-6[56400]: 2024-09-12T03:05:45.906+0000 7f8e4a34d640 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:19.024576+0000 front 2024-09-12T03:05:19.024530+0000 (oldest deadline 2024-09-12T03:05:42.524750+0000) 2024-09-12T03:05:46.456 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:05:45 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:05:45.978+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:18.916404+0000 front 2024-09-12T03:05:18.916319+0000 (oldest deadline 2024-09-12T03:05:39.416329+0000) 2024-09-12T03:05:46.560 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:05:46 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:05:46.296+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:14.571718+0000 front 2024-09-12T03:05:14.571768+0000 (oldest deadline 2024-09-12T03:05:39.772780+0000) 2024-09-12T03:05:47.310 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 12 03:05:46 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-6[56400]: 2024-09-12T03:05:46.944+0000 7f8e4a34d640 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:19.024576+0000 front 2024-09-12T03:05:19.024530+0000 (oldest deadline 2024-09-12T03:05:42.524750+0000) 2024-09-12T03:05:47.415 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:05:47 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:05:47.077+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:16.844053+0000 front 2024-09-12T03:05:16.843858+0000 (oldest deadline 2024-09-12T03:05:42.644259+0000) 2024-09-12T03:05:47.416 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 12 03:05:46 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-5[46768]: 2024-09-12T03:05:46.926+0000 7fcd0c8ad640 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:17.282199+0000 front 2024-09-12T03:05:17.282098+0000 (oldest deadline 2024-09-12T03:05:42.581953+0000) 2024-09-12T03:05:47.456 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:05:47 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:05:47.013+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:18.916404+0000 front 2024-09-12T03:05:18.916319+0000 (oldest deadline 2024-09-12T03:05:39.416329+0000) 2024-09-12T03:05:47.810 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:05:47 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:05:47.319+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:14.571718+0000 front 2024-09-12T03:05:14.571768+0000 (oldest deadline 2024-09-12T03:05:39.772780+0000) 2024-09-12T03:05:48.123 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 12 03:05:47 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-5[46768]: 2024-09-12T03:05:47.944+0000 7fcd0c8ad640 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:17.282199+0000 front 2024-09-12T03:05:17.282098+0000 (oldest deadline 2024-09-12T03:05:42.581953+0000) 2024-09-12T03:05:48.123 INFO:journalctl@ceph.osd.8.smithi148.stdout:Sep 12 03:05:47 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-8[53762]: 2024-09-12T03:05:47.798+0000 7f7e0d911640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:25.877300+0000 front 2024-09-12T03:05:25.877333+0000 (oldest deadline 2024-09-12T03:05:47.577274+0000) 2024-09-12T03:05:48.295 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 12 03:05:47 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-6[56400]: 2024-09-12T03:05:47.924+0000 7f8e4a34d640 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:19.024576+0000 front 2024-09-12T03:05:19.024530+0000 (oldest deadline 2024-09-12T03:05:42.524750+0000) 2024-09-12T03:05:48.415 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:05:48 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:05:48.122+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:16.844053+0000 front 2024-09-12T03:05:16.843858+0000 (oldest deadline 2024-09-12T03:05:42.644259+0000) 2024-09-12T03:05:48.415 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:05:48 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:05:48.122+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:22.644805+0000 front 2024-09-12T03:05:22.644911+0000 (oldest deadline 2024-09-12T03:05:47.344653+0000) 2024-09-12T03:05:48.456 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:05:48 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:05:48.025+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:18.916404+0000 front 2024-09-12T03:05:18.916319+0000 (oldest deadline 2024-09-12T03:05:39.416329+0000) 2024-09-12T03:05:48.560 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:05:48 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:05:48.295+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:14.571718+0000 front 2024-09-12T03:05:14.571768+0000 (oldest deadline 2024-09-12T03:05:39.772780+0000) 2024-09-12T03:05:49.099 INFO:journalctl@ceph.osd.8.smithi148.stdout:Sep 12 03:05:48 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-8[53762]: 2024-09-12T03:05:48.814+0000 7f7e0d911640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:25.877300+0000 front 2024-09-12T03:05:25.877333+0000 (oldest deadline 2024-09-12T03:05:47.577274+0000) 2024-09-12T03:05:49.100 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 12 03:05:48 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-5[46768]: 2024-09-12T03:05:48.923+0000 7fcd0c8ad640 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:17.282199+0000 front 2024-09-12T03:05:17.282098+0000 (oldest deadline 2024-09-12T03:05:42.581953+0000) 2024-09-12T03:05:49.100 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 12 03:05:48 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-5[46768]: 2024-09-12T03:05:48.923+0000 7fcd0c8ad640 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:24.282932+0000 front 2024-09-12T03:05:24.282983+0000 (oldest deadline 2024-09-12T03:05:48.382896+0000) 2024-09-12T03:05:49.307 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 12 03:05:48 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-6[56400]: 2024-09-12T03:05:48.934+0000 7f8e4a34d640 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:19.024576+0000 front 2024-09-12T03:05:19.024530+0000 (oldest deadline 2024-09-12T03:05:42.524750+0000) 2024-09-12T03:05:49.307 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 12 03:05:48 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-6[56400]: 2024-09-12T03:05:48.934+0000 7f8e4a34d640 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:23.025982+0000 front 2024-09-12T03:05:23.025795+0000 (oldest deadline 2024-09-12T03:05:48.925748+0000) 2024-09-12T03:05:49.415 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:05:49 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:05:49.099+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:16.844053+0000 front 2024-09-12T03:05:16.843858+0000 (oldest deadline 2024-09-12T03:05:42.644259+0000) 2024-09-12T03:05:49.416 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:05:49 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:05:49.099+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:22.644805+0000 front 2024-09-12T03:05:22.644911+0000 (oldest deadline 2024-09-12T03:05:47.344653+0000) 2024-09-12T03:05:49.456 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:05:49 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:05:49.032+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:18.916404+0000 front 2024-09-12T03:05:18.916319+0000 (oldest deadline 2024-09-12T03:05:39.416329+0000) 2024-09-12T03:05:49.560 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:05:49 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:05:49.307+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:14.571718+0000 front 2024-09-12T03:05:14.571768+0000 (oldest deadline 2024-09-12T03:05:39.772780+0000) 2024-09-12T03:05:49.560 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:05:49 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:05:49.307+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:24.374445+0000 front 2024-09-12T03:05:24.374549+0000 (oldest deadline 2024-09-12T03:05:48.374868+0000) 2024-09-12T03:05:50.145 INFO:journalctl@ceph.osd.8.smithi148.stdout:Sep 12 03:05:49 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-8[53762]: 2024-09-12T03:05:49.806+0000 7f7e0d911640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:25.877300+0000 front 2024-09-12T03:05:25.877333+0000 (oldest deadline 2024-09-12T03:05:47.577274+0000) 2024-09-12T03:05:50.146 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 12 03:05:49 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-5[46768]: 2024-09-12T03:05:49.924+0000 7fcd0c8ad640 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:17.282199+0000 front 2024-09-12T03:05:17.282098+0000 (oldest deadline 2024-09-12T03:05:42.581953+0000) 2024-09-12T03:05:50.146 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 12 03:05:49 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-5[46768]: 2024-09-12T03:05:49.924+0000 7fcd0c8ad640 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:24.282932+0000 front 2024-09-12T03:05:24.282983+0000 (oldest deadline 2024-09-12T03:05:48.382896+0000) 2024-09-12T03:05:50.310 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 12 03:05:49 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-6[56400]: 2024-09-12T03:05:49.928+0000 7f8e4a34d640 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:19.024576+0000 front 2024-09-12T03:05:19.024530+0000 (oldest deadline 2024-09-12T03:05:42.524750+0000) 2024-09-12T03:05:50.310 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 12 03:05:49 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-6[56400]: 2024-09-12T03:05:49.928+0000 7f8e4a34d640 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:23.025982+0000 front 2024-09-12T03:05:23.025795+0000 (oldest deadline 2024-09-12T03:05:48.925748+0000) 2024-09-12T03:05:50.324 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:05:50 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:05:50.050+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:18.916404+0000 front 2024-09-12T03:05:18.916319+0000 (oldest deadline 2024-09-12T03:05:39.416329+0000) 2024-09-12T03:05:50.415 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:05:50 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:05:50.145+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:16.844053+0000 front 2024-09-12T03:05:16.843858+0000 (oldest deadline 2024-09-12T03:05:42.644259+0000) 2024-09-12T03:05:50.416 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:05:50 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:05:50.145+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:22.644805+0000 front 2024-09-12T03:05:22.644911+0000 (oldest deadline 2024-09-12T03:05:47.344653+0000) 2024-09-12T03:05:50.706 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 12 03:05:50 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-7[52718]: 2024-09-12T03:05:50.324+0000 7f331b9c0640 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:24.091027+0000 front 2024-09-12T03:05:24.090973+0000 (oldest deadline 2024-09-12T03:05:49.991085+0000) 2024-09-12T03:05:50.810 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:05:50 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:05:50.313+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:14.571718+0000 front 2024-09-12T03:05:14.571768+0000 (oldest deadline 2024-09-12T03:05:39.772780+0000) 2024-09-12T03:05:50.810 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:05:50 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:05:50.313+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:24.374445+0000 front 2024-09-12T03:05:24.374549+0000 (oldest deadline 2024-09-12T03:05:48.374868+0000) 2024-09-12T03:05:51.133 INFO:journalctl@ceph.osd.8.smithi148.stdout:Sep 12 03:05:50 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-8[53762]: 2024-09-12T03:05:50.807+0000 7f7e0d911640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:25.877300+0000 front 2024-09-12T03:05:25.877333+0000 (oldest deadline 2024-09-12T03:05:47.577274+0000) 2024-09-12T03:05:51.133 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 12 03:05:50 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-5[46768]: 2024-09-12T03:05:50.924+0000 7fcd0c8ad640 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:17.282199+0000 front 2024-09-12T03:05:17.282098+0000 (oldest deadline 2024-09-12T03:05:42.581953+0000) 2024-09-12T03:05:51.133 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 12 03:05:50 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-5[46768]: 2024-09-12T03:05:50.924+0000 7fcd0c8ad640 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:24.282932+0000 front 2024-09-12T03:05:24.282983+0000 (oldest deadline 2024-09-12T03:05:48.382896+0000) 2024-09-12T03:05:51.310 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 12 03:05:50 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-6[56400]: 2024-09-12T03:05:50.968+0000 7f8e4a34d640 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:19.024576+0000 front 2024-09-12T03:05:19.024530+0000 (oldest deadline 2024-09-12T03:05:42.524750+0000) 2024-09-12T03:05:51.310 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 12 03:05:50 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-6[56400]: 2024-09-12T03:05:50.968+0000 7f8e4a34d640 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:23.025982+0000 front 2024-09-12T03:05:23.025795+0000 (oldest deadline 2024-09-12T03:05:48.925748+0000) 2024-09-12T03:05:51.311 INFO:journalctl@ceph.osd.4.smithi146.stdout:Sep 12 03:05:51 smithi146 podman[69379]: 2024-09-12 03:05:51.033970796 +0000 UTC m=+5.614262973 container died b7728f11508585ecfa15462eb9593a106ed52de4b03cdbe7db73ec6c43dc0853 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-4, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-12T03:05:51.311 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:05:51 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:05:51.019+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:18.916404+0000 front 2024-09-12T03:05:18.916319+0000 (oldest deadline 2024-09-12T03:05:39.416329+0000) 2024-09-12T03:05:51.311 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:05:51 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:05:51.019+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:25.817651+0000 front 2024-09-12T03:05:25.817711+0000 (oldest deadline 2024-09-12T03:05:50.517693+0000) 2024-09-12T03:05:51.415 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:05:51 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:05:51.132+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:16.844053+0000 front 2024-09-12T03:05:16.843858+0000 (oldest deadline 2024-09-12T03:05:42.644259+0000) 2024-09-12T03:05:51.416 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:05:51 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:05:51.132+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:22.644805+0000 front 2024-09-12T03:05:22.644911+0000 (oldest deadline 2024-09-12T03:05:47.344653+0000) 2024-09-12T03:05:51.669 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 12 03:05:51 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-7[52718]: 2024-09-12T03:05:51.355+0000 7f331b9c0640 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:24.091027+0000 front 2024-09-12T03:05:24.090973+0000 (oldest deadline 2024-09-12T03:05:49.991085+0000) 2024-09-12T03:05:51.669 INFO:journalctl@ceph.osd.4.smithi146.stdout:Sep 12 03:05:51 smithi146 podman[69379]: 2024-09-12 03:05:51.402010753 +0000 UTC m=+5.982302918 container cleanup b7728f11508585ecfa15462eb9593a106ed52de4b03cdbe7db73ec6c43dc0853 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-4, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-b53b6d3, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T03:05:51.669 INFO:journalctl@ceph.osd.4.smithi146.stdout:Sep 12 03:05:51 smithi146 bash[69379]: ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-4 2024-09-12T03:05:51.810 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:05:51 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:05:51.341+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:14.571718+0000 front 2024-09-12T03:05:14.571768+0000 (oldest deadline 2024-09-12T03:05:39.772780+0000) 2024-09-12T03:05:51.810 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:05:51 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:05:51.341+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:24.374445+0000 front 2024-09-12T03:05:24.374549+0000 (oldest deadline 2024-09-12T03:05:48.374868+0000) 2024-09-12T03:05:51.956 INFO:journalctl@ceph.osd.4.smithi146.stdout:Sep 12 03:05:51 smithi146 podman[69395]: 2024-09-12 03:05:51.669635428 +0000 UTC m=+0.634098624 container remove b7728f11508585ecfa15462eb9593a106ed52de4b03cdbe7db73ec6c43dc0853 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-4, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-12T03:05:52.136 INFO:journalctl@ceph.osd.8.smithi148.stdout:Sep 12 03:05:51 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-8[53762]: 2024-09-12T03:05:51.849+0000 7f7e0d911640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:25.877300+0000 front 2024-09-12T03:05:25.877333+0000 (oldest deadline 2024-09-12T03:05:47.577274+0000) 2024-09-12T03:05:52.136 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 12 03:05:51 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-5[46768]: 2024-09-12T03:05:51.967+0000 7fcd0c8ad640 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:17.282199+0000 front 2024-09-12T03:05:17.282098+0000 (oldest deadline 2024-09-12T03:05:42.581953+0000) 2024-09-12T03:05:52.136 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 12 03:05:51 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-5[46768]: 2024-09-12T03:05:51.967+0000 7fcd0c8ad640 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:24.282932+0000 front 2024-09-12T03:05:24.282983+0000 (oldest deadline 2024-09-12T03:05:48.382896+0000) 2024-09-12T03:05:52.310 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 12 03:05:51 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-6[56400]: 2024-09-12T03:05:51.984+0000 7f8e4a34d640 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:19.024576+0000 front 2024-09-12T03:05:19.024530+0000 (oldest deadline 2024-09-12T03:05:42.524750+0000) 2024-09-12T03:05:52.310 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 12 03:05:51 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-6[56400]: 2024-09-12T03:05:51.984+0000 7f8e4a34d640 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:23.025982+0000 front 2024-09-12T03:05:23.025795+0000 (oldest deadline 2024-09-12T03:05:48.925748+0000) 2024-09-12T03:05:52.330 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:05:52 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:05:52.024+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:18.916404+0000 front 2024-09-12T03:05:18.916319+0000 (oldest deadline 2024-09-12T03:05:39.416329+0000) 2024-09-12T03:05:52.330 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:05:52 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:05:52.024+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:25.817651+0000 front 2024-09-12T03:05:25.817711+0000 (oldest deadline 2024-09-12T03:05:50.517693+0000) 2024-09-12T03:05:52.330 INFO:journalctl@ceph.osd.4.smithi146.stdout:Sep 12 03:05:52 smithi146 podman[69483]: 2024-09-12 03:05:51.915584718 +0000 UTC m=+0.015850213 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:05:52.330 INFO:journalctl@ceph.osd.4.smithi146.stdout:Sep 12 03:05:52 smithi146 podman[69483]: 2024-09-12 03:05:52.020552373 +0000 UTC m=+0.120817864 container create de80c4df8d06302cb0d75f6e444bcf06830d27ca890e22a90d03441289b88ab9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-12T03:05:52.330 INFO:journalctl@ceph.osd.4.smithi146.stdout:Sep 12 03:05:52 smithi146 podman[69483]: 2024-09-12 03:05:52.255970794 +0000 UTC m=+0.356236284 container init de80c4df8d06302cb0d75f6e444bcf06830d27ca890e22a90d03441289b88ab9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-12T03:05:52.330 INFO:journalctl@ceph.osd.4.smithi146.stdout:Sep 12 03:05:52 smithi146 podman[69483]: 2024-09-12 03:05:52.259302095 +0000 UTC m=+0.359567593 container start de80c4df8d06302cb0d75f6e444bcf06830d27ca890e22a90d03441289b88ab9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T03:05:52.331 INFO:journalctl@ceph.osd.4.smithi146.stdout:Sep 12 03:05:52 smithi146 podman[69483]: 2024-09-12 03:05:52.330879918 +0000 UTC m=+0.431145415 container attach de80c4df8d06302cb0d75f6e444bcf06830d27ca890e22a90d03441289b88ab9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T03:05:52.415 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:05:52 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:05:52.136+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:16.844053+0000 front 2024-09-12T03:05:16.843858+0000 (oldest deadline 2024-09-12T03:05:42.644259+0000) 2024-09-12T03:05:52.415 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:05:52 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:05:52.136+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:22.644805+0000 front 2024-09-12T03:05:22.644911+0000 (oldest deadline 2024-09-12T03:05:47.344653+0000) 2024-09-12T03:05:52.706 INFO:journalctl@ceph.osd.4.smithi146.stdout:Sep 12 03:05:52 smithi146 podman[69483]: 2024-09-12 03:05:52.44846842 +0000 UTC m=+0.548733910 container died de80c4df8d06302cb0d75f6e444bcf06830d27ca890e22a90d03441289b88ab9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-4-deactivate, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-12T03:05:52.707 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 12 03:05:52 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-7[52718]: 2024-09-12T03:05:52.349+0000 7f331b9c0640 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:24.091027+0000 front 2024-09-12T03:05:24.090973+0000 (oldest deadline 2024-09-12T03:05:49.991085+0000) 2024-09-12T03:05:52.810 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:05:52 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:05:52.328+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:14.571718+0000 front 2024-09-12T03:05:14.571768+0000 (oldest deadline 2024-09-12T03:05:39.772780+0000) 2024-09-12T03:05:52.810 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:05:52 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:05:52.328+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:24.374445+0000 front 2024-09-12T03:05:24.374549+0000 (oldest deadline 2024-09-12T03:05:48.374868+0000) 2024-09-12T03:05:53.020 INFO:journalctl@ceph.osd.4.smithi146.stdout:Sep 12 03:05:53 smithi146 podman[69507]: 2024-09-12 03:05:53.000804043 +0000 UTC m=+0.546184972 container remove de80c4df8d06302cb0d75f6e444bcf06830d27ca890e22a90d03441289b88ab9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2) 2024-09-12T03:05:53.025 DEBUG:teuthology.orchestra.run.smithi146:> sudo pkill -f 'journalctl -f -n 0 -u ceph-618305f8-70b1-11ef-bcea-c7b262605968@osd.4.service' 2024-09-12T03:05:53.064 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:05:53.064 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-12T03:05:53.065 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-12T03:05:53.065 DEBUG:teuthology.orchestra.run.smithi148:> sudo systemctl stop ceph-618305f8-70b1-11ef-bcea-c7b262605968@osd.5 2024-09-12T03:05:53.093 INFO:journalctl@ceph.osd.8.smithi148.stdout:Sep 12 03:05:52 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-8[53762]: 2024-09-12T03:05:52.816+0000 7f7e0d911640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:25.877300+0000 front 2024-09-12T03:05:25.877333+0000 (oldest deadline 2024-09-12T03:05:47.577274+0000) 2024-09-12T03:05:53.093 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 12 03:05:52 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-5[46768]: 2024-09-12T03:05:52.972+0000 7fcd0c8ad640 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:17.282199+0000 front 2024-09-12T03:05:17.282098+0000 (oldest deadline 2024-09-12T03:05:42.581953+0000) 2024-09-12T03:05:53.093 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 12 03:05:52 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-5[46768]: 2024-09-12T03:05:52.972+0000 7fcd0c8ad640 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:24.282932+0000 front 2024-09-12T03:05:24.282983+0000 (oldest deadline 2024-09-12T03:05:48.382896+0000) 2024-09-12T03:05:53.310 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 12 03:05:53 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-6[56400]: 2024-09-12T03:05:53.024+0000 7f8e4a34d640 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:19.024576+0000 front 2024-09-12T03:05:19.024530+0000 (oldest deadline 2024-09-12T03:05:42.524750+0000) 2024-09-12T03:05:53.310 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 12 03:05:53 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-6[56400]: 2024-09-12T03:05:53.024+0000 7f8e4a34d640 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:23.025982+0000 front 2024-09-12T03:05:23.025795+0000 (oldest deadline 2024-09-12T03:05:48.925748+0000) 2024-09-12T03:05:53.350 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:05:53 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:05:53.035+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:18.916404+0000 front 2024-09-12T03:05:18.916319+0000 (oldest deadline 2024-09-12T03:05:39.416329+0000) 2024-09-12T03:05:53.350 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:05:53 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:05:53.035+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:25.817651+0000 front 2024-09-12T03:05:25.817711+0000 (oldest deadline 2024-09-12T03:05:50.517693+0000) 2024-09-12T03:05:53.367 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 12 03:05:53 smithi148 systemd[1]: Stopping Ceph osd.5 for 618305f8-70b1-11ef-bcea-c7b262605968... 2024-09-12T03:05:53.368 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:05:53 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:05:53.111+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:16.844053+0000 front 2024-09-12T03:05:16.843858+0000 (oldest deadline 2024-09-12T03:05:42.644259+0000) 2024-09-12T03:05:53.368 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:05:53 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:05:53.111+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:22.644805+0000 front 2024-09-12T03:05:22.644911+0000 (oldest deadline 2024-09-12T03:05:47.344653+0000) 2024-09-12T03:05:53.665 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 12 03:05:53 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-5[46768]: 2024-09-12T03:05:53.367+0000 7fcd102a8640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:05:53.666 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 12 03:05:53 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-5[46768]: 2024-09-12T03:05:53.367+0000 7fcd102a8640 -1 osd.5 86 *** Got signal Terminated *** 2024-09-12T03:05:53.666 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 12 03:05:53 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-5[46768]: 2024-09-12T03:05:53.367+0000 7fcd102a8640 -1 osd.5 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T03:05:53.706 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 12 03:05:53 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-7[52718]: 2024-09-12T03:05:53.350+0000 7f331b9c0640 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:24.091027+0000 front 2024-09-12T03:05:24.090973+0000 (oldest deadline 2024-09-12T03:05:49.991085+0000) 2024-09-12T03:05:53.810 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:05:53 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:05:53.320+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:14.571718+0000 front 2024-09-12T03:05:14.571768+0000 (oldest deadline 2024-09-12T03:05:39.772780+0000) 2024-09-12T03:05:53.810 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:05:53 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:05:53.320+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:24.374445+0000 front 2024-09-12T03:05:24.374549+0000 (oldest deadline 2024-09-12T03:05:48.374868+0000) 2024-09-12T03:05:54.165 INFO:journalctl@ceph.osd.8.smithi148.stdout:Sep 12 03:05:53 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-8[53762]: 2024-09-12T03:05:53.830+0000 7f7e0d911640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:25.877300+0000 front 2024-09-12T03:05:25.877333+0000 (oldest deadline 2024-09-12T03:05:47.577274+0000) 2024-09-12T03:05:54.166 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:05:54 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:05:54.071+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:16.844053+0000 front 2024-09-12T03:05:16.843858+0000 (oldest deadline 2024-09-12T03:05:42.644259+0000) 2024-09-12T03:05:54.166 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:05:54 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:05:54.071+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:22.644805+0000 front 2024-09-12T03:05:22.644911+0000 (oldest deadline 2024-09-12T03:05:47.344653+0000) 2024-09-12T03:05:54.166 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 12 03:05:53 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-5[46768]: 2024-09-12T03:05:53.967+0000 7fcd0c8ad640 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:17.282199+0000 front 2024-09-12T03:05:17.282098+0000 (oldest deadline 2024-09-12T03:05:42.581953+0000) 2024-09-12T03:05:54.166 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 12 03:05:53 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-5[46768]: 2024-09-12T03:05:53.967+0000 7fcd0c8ad640 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:24.282932+0000 front 2024-09-12T03:05:24.282983+0000 (oldest deadline 2024-09-12T03:05:48.382896+0000) 2024-09-12T03:05:54.310 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 12 03:05:54 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-6[56400]: 2024-09-12T03:05:54.041+0000 7f8e4a34d640 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:19.024576+0000 front 2024-09-12T03:05:19.024530+0000 (oldest deadline 2024-09-12T03:05:42.524750+0000) 2024-09-12T03:05:54.310 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 12 03:05:54 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-6[56400]: 2024-09-12T03:05:54.041+0000 7f8e4a34d640 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:23.025982+0000 front 2024-09-12T03:05:23.025795+0000 (oldest deadline 2024-09-12T03:05:48.925748+0000) 2024-09-12T03:05:54.376 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:05:53 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:05:53.987+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:18.916404+0000 front 2024-09-12T03:05:18.916319+0000 (oldest deadline 2024-09-12T03:05:39.416329+0000) 2024-09-12T03:05:54.376 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:05:53 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:05:53.987+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:25.817651+0000 front 2024-09-12T03:05:25.817711+0000 (oldest deadline 2024-09-12T03:05:50.517693+0000) 2024-09-12T03:05:54.706 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 12 03:05:54 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-7[52718]: 2024-09-12T03:05:54.377+0000 7f331b9c0640 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:24.091027+0000 front 2024-09-12T03:05:24.090973+0000 (oldest deadline 2024-09-12T03:05:49.991085+0000) 2024-09-12T03:05:54.810 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:05:54 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:05:54.351+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:14.571718+0000 front 2024-09-12T03:05:14.571768+0000 (oldest deadline 2024-09-12T03:05:39.772780+0000) 2024-09-12T03:05:54.810 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:05:54 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:05:54.351+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:24.374445+0000 front 2024-09-12T03:05:24.374549+0000 (oldest deadline 2024-09-12T03:05:48.374868+0000) 2024-09-12T03:05:55.083 INFO:journalctl@ceph.osd.8.smithi148.stdout:Sep 12 03:05:54 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-8[53762]: 2024-09-12T03:05:54.822+0000 7f7e0d911640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:25.877300+0000 front 2024-09-12T03:05:25.877333+0000 (oldest deadline 2024-09-12T03:05:47.577274+0000) 2024-09-12T03:05:55.083 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 12 03:05:54 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-5[46768]: 2024-09-12T03:05:54.982+0000 7fcd0c8ad640 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:17.282199+0000 front 2024-09-12T03:05:17.282098+0000 (oldest deadline 2024-09-12T03:05:42.581953+0000) 2024-09-12T03:05:55.083 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 12 03:05:54 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-5[46768]: 2024-09-12T03:05:54.982+0000 7fcd0c8ad640 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:24.282932+0000 front 2024-09-12T03:05:24.282983+0000 (oldest deadline 2024-09-12T03:05:48.382896+0000) 2024-09-12T03:05:55.206 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:05:54 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:05:54.947+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:18.916404+0000 front 2024-09-12T03:05:18.916319+0000 (oldest deadline 2024-09-12T03:05:39.416329+0000) 2024-09-12T03:05:55.206 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:05:54 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:05:54.947+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:25.817651+0000 front 2024-09-12T03:05:25.817711+0000 (oldest deadline 2024-09-12T03:05:50.517693+0000) 2024-09-12T03:05:55.413 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 12 03:05:55 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-6[56400]: 2024-09-12T03:05:55.086+0000 7f8e4a34d640 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:19.024576+0000 front 2024-09-12T03:05:19.024530+0000 (oldest deadline 2024-09-12T03:05:42.524750+0000) 2024-09-12T03:05:55.413 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 12 03:05:55 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-6[56400]: 2024-09-12T03:05:55.086+0000 7f8e4a34d640 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:23.025982+0000 front 2024-09-12T03:05:23.025795+0000 (oldest deadline 2024-09-12T03:05:48.925748+0000) 2024-09-12T03:05:55.415 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:05:55 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:05:55.083+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:16.844053+0000 front 2024-09-12T03:05:16.843858+0000 (oldest deadline 2024-09-12T03:05:42.644259+0000) 2024-09-12T03:05:55.415 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:05:55 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:05:55.083+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:22.644805+0000 front 2024-09-12T03:05:22.644911+0000 (oldest deadline 2024-09-12T03:05:47.344653+0000) 2024-09-12T03:05:55.706 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 12 03:05:55 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-7[52718]: 2024-09-12T03:05:55.369+0000 7f331b9c0640 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:24.091027+0000 front 2024-09-12T03:05:24.090973+0000 (oldest deadline 2024-09-12T03:05:49.991085+0000) 2024-09-12T03:05:55.810 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:05:55 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:05:55.324+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:14.571718+0000 front 2024-09-12T03:05:14.571768+0000 (oldest deadline 2024-09-12T03:05:39.772780+0000) 2024-09-12T03:05:55.810 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:05:55 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:05:55.324+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:24.374445+0000 front 2024-09-12T03:05:24.374549+0000 (oldest deadline 2024-09-12T03:05:48.374868+0000) 2024-09-12T03:05:56.165 INFO:journalctl@ceph.osd.8.smithi148.stdout:Sep 12 03:05:55 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-8[53762]: 2024-09-12T03:05:55.865+0000 7f7e0d911640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:25.877300+0000 front 2024-09-12T03:05:25.877333+0000 (oldest deadline 2024-09-12T03:05:47.577274+0000) 2024-09-12T03:05:56.166 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:05:56 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:05:56.056+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:16.844053+0000 front 2024-09-12T03:05:16.843858+0000 (oldest deadline 2024-09-12T03:05:42.644259+0000) 2024-09-12T03:05:56.166 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:05:56 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:05:56.056+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:22.644805+0000 front 2024-09-12T03:05:22.644911+0000 (oldest deadline 2024-09-12T03:05:47.344653+0000) 2024-09-12T03:05:56.166 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 12 03:05:56 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-5[46768]: 2024-09-12T03:05:56.004+0000 7fcd0c8ad640 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:17.282199+0000 front 2024-09-12T03:05:17.282098+0000 (oldest deadline 2024-09-12T03:05:42.581953+0000) 2024-09-12T03:05:56.166 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 12 03:05:56 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-5[46768]: 2024-09-12T03:05:56.004+0000 7fcd0c8ad640 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:24.282932+0000 front 2024-09-12T03:05:24.282983+0000 (oldest deadline 2024-09-12T03:05:48.382896+0000) 2024-09-12T03:05:56.166 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 12 03:05:56 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-5[46768]: 2024-09-12T03:05:56.004+0000 7fcd0c8ad640 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:34.684832+0000 front 2024-09-12T03:05:34.684812+0000 (oldest deadline 2024-09-12T03:05:55.784974+0000) 2024-09-12T03:05:56.335 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:05:55 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:05:55.981+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:18.916404+0000 front 2024-09-12T03:05:18.916319+0000 (oldest deadline 2024-09-12T03:05:39.416329+0000) 2024-09-12T03:05:56.335 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:05:55 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:05:55.981+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:25.817651+0000 front 2024-09-12T03:05:25.817711+0000 (oldest deadline 2024-09-12T03:05:50.517693+0000) 2024-09-12T03:05:56.335 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:05:55 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:05:55.981+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:30.518204+0000 front 2024-09-12T03:05:30.518383+0000 (oldest deadline 2024-09-12T03:05:55.218163+0000) 2024-09-12T03:05:56.560 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:05:56 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:05:56.313+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:14.571718+0000 front 2024-09-12T03:05:14.571768+0000 (oldest deadline 2024-09-12T03:05:39.772780+0000) 2024-09-12T03:05:56.560 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:05:56 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:05:56.313+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:24.374445+0000 front 2024-09-12T03:05:24.374549+0000 (oldest deadline 2024-09-12T03:05:48.374868+0000) 2024-09-12T03:05:56.560 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 12 03:05:56 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-6[56400]: 2024-09-12T03:05:56.123+0000 7f8e4a34d640 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:19.024576+0000 front 2024-09-12T03:05:19.024530+0000 (oldest deadline 2024-09-12T03:05:42.524750+0000) 2024-09-12T03:05:56.560 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 12 03:05:56 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-6[56400]: 2024-09-12T03:05:56.123+0000 7f8e4a34d640 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:23.025982+0000 front 2024-09-12T03:05:23.025795+0000 (oldest deadline 2024-09-12T03:05:48.925748+0000) 2024-09-12T03:05:56.561 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 12 03:05:56 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-6[56400]: 2024-09-12T03:05:56.123+0000 7f8e4a34d640 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:31.628129+0000 front 2024-09-12T03:05:31.628211+0000 (oldest deadline 2024-09-12T03:05:55.127992+0000) 2024-09-12T03:05:56.706 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 12 03:05:56 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-7[52718]: 2024-09-12T03:05:56.336+0000 7f331b9c0640 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:24.091027+0000 front 2024-09-12T03:05:24.090973+0000 (oldest deadline 2024-09-12T03:05:49.991085+0000) 2024-09-12T03:05:57.166 INFO:journalctl@ceph.osd.8.smithi148.stdout:Sep 12 03:05:56 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-8[53762]: 2024-09-12T03:05:56.907+0000 7f7e0d911640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:25.877300+0000 front 2024-09-12T03:05:25.877333+0000 (oldest deadline 2024-09-12T03:05:47.577274+0000) 2024-09-12T03:05:57.166 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:05:57 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:05:57.091+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:16.844053+0000 front 2024-09-12T03:05:16.843858+0000 (oldest deadline 2024-09-12T03:05:42.644259+0000) 2024-09-12T03:05:57.166 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:05:57 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:05:57.091+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:22.644805+0000 front 2024-09-12T03:05:22.644911+0000 (oldest deadline 2024-09-12T03:05:47.344653+0000) 2024-09-12T03:05:57.166 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 12 03:05:57 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-5[46768]: 2024-09-12T03:05:56.999+0000 7fcd0c8ad640 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:17.282199+0000 front 2024-09-12T03:05:17.282098+0000 (oldest deadline 2024-09-12T03:05:42.581953+0000) 2024-09-12T03:05:57.166 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 12 03:05:57 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-5[46768]: 2024-09-12T03:05:56.999+0000 7fcd0c8ad640 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:24.282932+0000 front 2024-09-12T03:05:24.282983+0000 (oldest deadline 2024-09-12T03:05:48.382896+0000) 2024-09-12T03:05:57.167 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 12 03:05:57 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-5[46768]: 2024-09-12T03:05:56.999+0000 7fcd0c8ad640 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:34.684832+0000 front 2024-09-12T03:05:34.684812+0000 (oldest deadline 2024-09-12T03:05:55.784974+0000) 2024-09-12T03:05:57.206 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:05:56 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:05:56.955+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:18.916404+0000 front 2024-09-12T03:05:18.916319+0000 (oldest deadline 2024-09-12T03:05:39.416329+0000) 2024-09-12T03:05:57.206 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:05:56 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:05:56.955+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:25.817651+0000 front 2024-09-12T03:05:25.817711+0000 (oldest deadline 2024-09-12T03:05:50.517693+0000) 2024-09-12T03:05:57.206 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:05:56 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:05:56.955+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:30.518204+0000 front 2024-09-12T03:05:30.518383+0000 (oldest deadline 2024-09-12T03:05:55.218163+0000) 2024-09-12T03:05:57.560 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:05:57 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:05:57.264+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:14.571718+0000 front 2024-09-12T03:05:14.571768+0000 (oldest deadline 2024-09-12T03:05:39.772780+0000) 2024-09-12T03:05:57.560 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:05:57 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:05:57.264+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:24.374445+0000 front 2024-09-12T03:05:24.374549+0000 (oldest deadline 2024-09-12T03:05:48.374868+0000) 2024-09-12T03:05:57.560 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 12 03:05:57 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-6[56400]: 2024-09-12T03:05:57.156+0000 7f8e4a34d640 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:19.024576+0000 front 2024-09-12T03:05:19.024530+0000 (oldest deadline 2024-09-12T03:05:42.524750+0000) 2024-09-12T03:05:57.561 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 12 03:05:57 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-6[56400]: 2024-09-12T03:05:57.156+0000 7f8e4a34d640 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:23.025982+0000 front 2024-09-12T03:05:23.025795+0000 (oldest deadline 2024-09-12T03:05:48.925748+0000) 2024-09-12T03:05:57.561 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 12 03:05:57 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-6[56400]: 2024-09-12T03:05:57.156+0000 7f8e4a34d640 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:31.628129+0000 front 2024-09-12T03:05:31.628211+0000 (oldest deadline 2024-09-12T03:05:55.127992+0000) 2024-09-12T03:05:57.706 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 12 03:05:57 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-7[52718]: 2024-09-12T03:05:57.306+0000 7f331b9c0640 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:24.091027+0000 front 2024-09-12T03:05:24.090973+0000 (oldest deadline 2024-09-12T03:05:49.991085+0000) 2024-09-12T03:05:57.706 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 12 03:05:57 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-7[52718]: 2024-09-12T03:05:57.306+0000 7f331b9c0640 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:34.592906+0000 front 2024-09-12T03:05:34.592762+0000 (oldest deadline 2024-09-12T03:05:56.892580+0000) 2024-09-12T03:05:58.165 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 12 03:05:57 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-5[46768]: 2024-09-12T03:05:57.986+0000 7fcd0c8ad640 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:17.282199+0000 front 2024-09-12T03:05:17.282098+0000 (oldest deadline 2024-09-12T03:05:42.581953+0000) 2024-09-12T03:05:58.166 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 12 03:05:57 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-5[46768]: 2024-09-12T03:05:57.986+0000 7fcd0c8ad640 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:24.282932+0000 front 2024-09-12T03:05:24.282983+0000 (oldest deadline 2024-09-12T03:05:48.382896+0000) 2024-09-12T03:05:58.166 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 12 03:05:57 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-5[46768]: 2024-09-12T03:05:57.986+0000 7fcd0c8ad640 -1 osd.5 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:34.684832+0000 front 2024-09-12T03:05:34.684812+0000 (oldest deadline 2024-09-12T03:05:55.784974+0000) 2024-09-12T03:05:58.166 INFO:journalctl@ceph.osd.8.smithi148.stdout:Sep 12 03:05:57 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-8[53762]: 2024-09-12T03:05:57.904+0000 7f7e0d911640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:25.877300+0000 front 2024-09-12T03:05:25.877333+0000 (oldest deadline 2024-09-12T03:05:47.577274+0000) 2024-09-12T03:05:58.166 INFO:journalctl@ceph.osd.8.smithi148.stdout:Sep 12 03:05:57 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-8[53762]: 2024-09-12T03:05:57.904+0000 7f7e0d911640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:31.678293+0000 front 2024-09-12T03:05:31.678404+0000 (oldest deadline 2024-09-12T03:05:56.978470+0000) 2024-09-12T03:05:58.166 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:05:58 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:05:58.047+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:16.844053+0000 front 2024-09-12T03:05:16.843858+0000 (oldest deadline 2024-09-12T03:05:42.644259+0000) 2024-09-12T03:05:58.166 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:05:58 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:05:58.047+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:22.644805+0000 front 2024-09-12T03:05:22.644911+0000 (oldest deadline 2024-09-12T03:05:47.344653+0000) 2024-09-12T03:05:58.206 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:05:57 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:05:57.938+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:18.916404+0000 front 2024-09-12T03:05:18.916319+0000 (oldest deadline 2024-09-12T03:05:39.416329+0000) 2024-09-12T03:05:58.206 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:05:57 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:05:57.938+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:25.817651+0000 front 2024-09-12T03:05:25.817711+0000 (oldest deadline 2024-09-12T03:05:50.517693+0000) 2024-09-12T03:05:58.206 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:05:57 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:05:57.938+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:30.518204+0000 front 2024-09-12T03:05:30.518383+0000 (oldest deadline 2024-09-12T03:05:55.218163+0000) 2024-09-12T03:05:58.560 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 12 03:05:58 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-6[56400]: 2024-09-12T03:05:58.114+0000 7f8e4a34d640 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:19.024576+0000 front 2024-09-12T03:05:19.024530+0000 (oldest deadline 2024-09-12T03:05:42.524750+0000) 2024-09-12T03:05:58.560 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 12 03:05:58 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-6[56400]: 2024-09-12T03:05:58.114+0000 7f8e4a34d640 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:23.025982+0000 front 2024-09-12T03:05:23.025795+0000 (oldest deadline 2024-09-12T03:05:48.925748+0000) 2024-09-12T03:05:58.560 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 12 03:05:58 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-6[56400]: 2024-09-12T03:05:58.114+0000 7f8e4a34d640 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:31.628129+0000 front 2024-09-12T03:05:31.628211+0000 (oldest deadline 2024-09-12T03:05:55.127992+0000) 2024-09-12T03:05:58.560 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:05:58 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:05:58.257+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:14.571718+0000 front 2024-09-12T03:05:14.571768+0000 (oldest deadline 2024-09-12T03:05:39.772780+0000) 2024-09-12T03:05:58.560 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:05:58 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:05:58.257+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:24.374445+0000 front 2024-09-12T03:05:24.374549+0000 (oldest deadline 2024-09-12T03:05:48.374868+0000) 2024-09-12T03:05:58.706 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 12 03:05:58 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-7[52718]: 2024-09-12T03:05:58.277+0000 7f331b9c0640 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:24.091027+0000 front 2024-09-12T03:05:24.090973+0000 (oldest deadline 2024-09-12T03:05:49.991085+0000) 2024-09-12T03:05:58.706 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 12 03:05:58 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-7[52718]: 2024-09-12T03:05:58.277+0000 7f331b9c0640 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:34.592906+0000 front 2024-09-12T03:05:34.592762+0000 (oldest deadline 2024-09-12T03:05:56.892580+0000) 2024-09-12T03:05:59.152 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:05:59 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:05:59.066+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:16.844053+0000 front 2024-09-12T03:05:16.843858+0000 (oldest deadline 2024-09-12T03:05:42.644259+0000) 2024-09-12T03:05:59.152 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:05:59 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:05:59.066+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:22.644805+0000 front 2024-09-12T03:05:22.644911+0000 (oldest deadline 2024-09-12T03:05:47.344653+0000) 2024-09-12T03:05:59.152 INFO:journalctl@ceph.osd.8.smithi148.stdout:Sep 12 03:05:58 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-8[53762]: 2024-09-12T03:05:58.883+0000 7f7e0d911640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:25.877300+0000 front 2024-09-12T03:05:25.877333+0000 (oldest deadline 2024-09-12T03:05:47.577274+0000) 2024-09-12T03:05:59.153 INFO:journalctl@ceph.osd.8.smithi148.stdout:Sep 12 03:05:58 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-8[53762]: 2024-09-12T03:05:58.883+0000 7f7e0d911640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:31.678293+0000 front 2024-09-12T03:05:31.678404+0000 (oldest deadline 2024-09-12T03:05:56.978470+0000) 2024-09-12T03:05:59.153 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 12 03:05:58 smithi148 podman[69374]: 2024-09-12 03:05:58.918031844 +0000 UTC m=+5.642672318 container died 3b01fb95287ae3b70f057d4850c6054a2b164051c78ca40f0edf888dbb055bda (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-5, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T03:05:59.206 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:05:58 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:05:58.940+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:18.916404+0000 front 2024-09-12T03:05:18.916319+0000 (oldest deadline 2024-09-12T03:05:39.416329+0000) 2024-09-12T03:05:59.206 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:05:58 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:05:58.940+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:25.817651+0000 front 2024-09-12T03:05:25.817711+0000 (oldest deadline 2024-09-12T03:05:50.517693+0000) 2024-09-12T03:05:59.206 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:05:58 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:05:58.940+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:30.518204+0000 front 2024-09-12T03:05:30.518383+0000 (oldest deadline 2024-09-12T03:05:55.218163+0000) 2024-09-12T03:05:59.415 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 12 03:05:59 smithi148 podman[69374]: 2024-09-12 03:05:59.217428172 +0000 UTC m=+5.942068647 container cleanup 3b01fb95287ae3b70f057d4850c6054a2b164051c78ca40f0edf888dbb055bda (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-5, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2) 2024-09-12T03:05:59.416 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 12 03:05:59 smithi148 bash[69374]: ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-5 2024-09-12T03:05:59.560 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 12 03:05:59 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-6[56400]: 2024-09-12T03:05:59.118+0000 7f8e4a34d640 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:19.024576+0000 front 2024-09-12T03:05:19.024530+0000 (oldest deadline 2024-09-12T03:05:42.524750+0000) 2024-09-12T03:05:59.560 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 12 03:05:59 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-6[56400]: 2024-09-12T03:05:59.118+0000 7f8e4a34d640 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:23.025982+0000 front 2024-09-12T03:05:23.025795+0000 (oldest deadline 2024-09-12T03:05:48.925748+0000) 2024-09-12T03:05:59.560 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 12 03:05:59 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-6[56400]: 2024-09-12T03:05:59.118+0000 7f8e4a34d640 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:31.628129+0000 front 2024-09-12T03:05:31.628211+0000 (oldest deadline 2024-09-12T03:05:55.127992+0000) 2024-09-12T03:05:59.561 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:05:59 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:05:59.246+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:14.571718+0000 front 2024-09-12T03:05:14.571768+0000 (oldest deadline 2024-09-12T03:05:39.772780+0000) 2024-09-12T03:05:59.561 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:05:59 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:05:59.246+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:24.374445+0000 front 2024-09-12T03:05:24.374549+0000 (oldest deadline 2024-09-12T03:05:48.374868+0000) 2024-09-12T03:05:59.561 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:05:59 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:05:59.246+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:34.276237+0000 front 2024-09-12T03:05:34.276182+0000 (oldest deadline 2024-09-12T03:05:58.976098+0000) 2024-09-12T03:05:59.706 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 12 03:05:59 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-7[52718]: 2024-09-12T03:05:59.319+0000 7f331b9c0640 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:24.091027+0000 front 2024-09-12T03:05:24.090973+0000 (oldest deadline 2024-09-12T03:05:49.991085+0000) 2024-09-12T03:05:59.706 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 12 03:05:59 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-7[52718]: 2024-09-12T03:05:59.319+0000 7f331b9c0640 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:34.592906+0000 front 2024-09-12T03:05:34.592762+0000 (oldest deadline 2024-09-12T03:05:56.892580+0000) 2024-09-12T03:05:59.780 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 12 03:05:59 smithi148 podman[69389]: 2024-09-12 03:05:59.468571007 +0000 UTC m=+0.549713247 container remove 3b01fb95287ae3b70f057d4850c6054a2b164051c78ca40f0edf888dbb055bda (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-5, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3) 2024-09-12T03:06:00.037 INFO:journalctl@ceph.osd.8.smithi148.stdout:Sep 12 03:05:59 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-8[53762]: 2024-09-12T03:05:59.877+0000 7f7e0d911640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:25.877300+0000 front 2024-09-12T03:05:25.877333+0000 (oldest deadline 2024-09-12T03:05:47.577274+0000) 2024-09-12T03:06:00.038 INFO:journalctl@ceph.osd.8.smithi148.stdout:Sep 12 03:05:59 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-8[53762]: 2024-09-12T03:05:59.877+0000 7f7e0d911640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:31.678293+0000 front 2024-09-12T03:05:31.678404+0000 (oldest deadline 2024-09-12T03:05:56.978470+0000) 2024-09-12T03:06:00.038 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 12 03:05:59 smithi148 podman[69480]: 2024-09-12 03:05:59.683851604 +0000 UTC m=+0.016109061 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:06:00.038 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 12 03:05:59 smithi148 podman[69480]: 2024-09-12 03:05:59.785895077 +0000 UTC m=+0.118152528 container create ce5d9ecf030234bbf74078a99a645a9906bd86432e291acb6eb547af95518c6a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T03:06:00.289 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:05:59 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:05:59.958+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:18.916404+0000 front 2024-09-12T03:05:18.916319+0000 (oldest deadline 2024-09-12T03:05:39.416329+0000) 2024-09-12T03:06:00.289 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:05:59 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:05:59.958+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:25.817651+0000 front 2024-09-12T03:05:25.817711+0000 (oldest deadline 2024-09-12T03:05:50.517693+0000) 2024-09-12T03:06:00.289 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:05:59 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:05:59.958+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:30.518204+0000 front 2024-09-12T03:05:30.518383+0000 (oldest deadline 2024-09-12T03:05:55.218163+0000) 2024-09-12T03:06:00.416 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:00 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:00.112+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:16.844053+0000 front 2024-09-12T03:05:16.843858+0000 (oldest deadline 2024-09-12T03:05:42.644259+0000) 2024-09-12T03:06:00.416 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:00 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:00.112+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:22.644805+0000 front 2024-09-12T03:05:22.644911+0000 (oldest deadline 2024-09-12T03:05:47.344653+0000) 2024-09-12T03:06:00.416 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:00 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:00.112+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:31.846594+0000 front 2024-09-12T03:05:31.846677+0000 (oldest deadline 2024-09-12T03:05:59.447156+0000) 2024-09-12T03:06:00.416 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 12 03:06:00 smithi148 podman[69480]: 2024-09-12 03:06:00.038394761 +0000 UTC m=+0.370652205 container init ce5d9ecf030234bbf74078a99a645a9906bd86432e291acb6eb547af95518c6a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-5-deactivate, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-12T03:06:00.416 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 12 03:06:00 smithi148 podman[69480]: 2024-09-12 03:06:00.04171276 +0000 UTC m=+0.373970217 container start ce5d9ecf030234bbf74078a99a645a9906bd86432e291acb6eb547af95518c6a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T03:06:00.416 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 12 03:06:00 smithi148 podman[69480]: 2024-09-12 03:06:00.121639482 +0000 UTC m=+0.453896938 container attach ce5d9ecf030234bbf74078a99a645a9906bd86432e291acb6eb547af95518c6a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-5-deactivate, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-12T03:06:00.416 INFO:journalctl@ceph.osd.5.smithi148.stdout:Sep 12 03:06:00 smithi148 podman[69480]: 2024-09-12 03:06:00.231377144 +0000 UTC m=+0.563634600 container died ce5d9ecf030234bbf74078a99a645a9906bd86432e291acb6eb547af95518c6a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731) 2024-09-12T03:06:00.560 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 12 03:06:00 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-6[56400]: 2024-09-12T03:06:00.141+0000 7f8e4a34d640 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:19.024576+0000 front 2024-09-12T03:05:19.024530+0000 (oldest deadline 2024-09-12T03:05:42.524750+0000) 2024-09-12T03:06:00.560 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 12 03:06:00 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-6[56400]: 2024-09-12T03:06:00.141+0000 7f8e4a34d640 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:23.025982+0000 front 2024-09-12T03:05:23.025795+0000 (oldest deadline 2024-09-12T03:05:48.925748+0000) 2024-09-12T03:06:00.560 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 12 03:06:00 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-6[56400]: 2024-09-12T03:06:00.141+0000 7f8e4a34d640 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:31.628129+0000 front 2024-09-12T03:05:31.628211+0000 (oldest deadline 2024-09-12T03:05:55.127992+0000) 2024-09-12T03:06:00.560 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:00 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:00.231+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:14.571718+0000 front 2024-09-12T03:05:14.571768+0000 (oldest deadline 2024-09-12T03:05:39.772780+0000) 2024-09-12T03:06:00.560 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:00 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:00.231+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:24.374445+0000 front 2024-09-12T03:05:24.374549+0000 (oldest deadline 2024-09-12T03:05:48.374868+0000) 2024-09-12T03:06:00.561 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:00 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:00.231+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:34.276237+0000 front 2024-09-12T03:05:34.276182+0000 (oldest deadline 2024-09-12T03:05:58.976098+0000) 2024-09-12T03:06:00.706 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 12 03:06:00 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-7[52718]: 2024-09-12T03:06:00.289+0000 7f331b9c0640 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:24.091027+0000 front 2024-09-12T03:05:24.090973+0000 (oldest deadline 2024-09-12T03:05:49.991085+0000) 2024-09-12T03:06:00.706 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 12 03:06:00 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-7[52718]: 2024-09-12T03:06:00.289+0000 7f331b9c0640 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:34.592906+0000 front 2024-09-12T03:05:34.592762+0000 (oldest deadline 2024-09-12T03:05:56.892580+0000) 2024-09-12T03:06:00.847 DEBUG:teuthology.orchestra.run.smithi148:> sudo pkill -f 'journalctl -f -n 0 -u ceph-618305f8-70b1-11ef-bcea-c7b262605968@osd.5.service' 2024-09-12T03:06:00.928 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:06:00.928 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-12T03:06:00.928 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-12T03:06:00.929 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ceph-618305f8-70b1-11ef-bcea-c7b262605968@osd.6 2024-09-12T03:06:01.158 INFO:journalctl@ceph.osd.8.smithi148.stdout:Sep 12 03:06:00 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-8[53762]: 2024-09-12T03:06:00.868+0000 7f7e0d911640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:25.877300+0000 front 2024-09-12T03:05:25.877333+0000 (oldest deadline 2024-09-12T03:05:47.577274+0000) 2024-09-12T03:06:01.159 INFO:journalctl@ceph.osd.8.smithi148.stdout:Sep 12 03:06:00 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-8[53762]: 2024-09-12T03:06:00.868+0000 7f7e0d911640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:31.678293+0000 front 2024-09-12T03:05:31.678404+0000 (oldest deadline 2024-09-12T03:05:56.978470+0000) 2024-09-12T03:06:01.241 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 12 03:06:00 smithi005 systemd[1]: Stopping Ceph osd.6 for 618305f8-70b1-11ef-bcea-c7b262605968... 2024-09-12T03:06:01.241 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 12 03:06:01 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-6[56400]: 2024-09-12T03:06:01.117+0000 7f8e4a34d640 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:19.024576+0000 front 2024-09-12T03:05:19.024530+0000 (oldest deadline 2024-09-12T03:05:42.524750+0000) 2024-09-12T03:06:01.241 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 12 03:06:01 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-6[56400]: 2024-09-12T03:06:01.117+0000 7f8e4a34d640 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:23.025982+0000 front 2024-09-12T03:05:23.025795+0000 (oldest deadline 2024-09-12T03:05:48.925748+0000) 2024-09-12T03:06:01.241 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 12 03:06:01 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-6[56400]: 2024-09-12T03:06:01.117+0000 7f8e4a34d640 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:31.628129+0000 front 2024-09-12T03:05:31.628211+0000 (oldest deadline 2024-09-12T03:05:55.127992+0000) 2024-09-12T03:06:01.243 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:00 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:00.975+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:18.916404+0000 front 2024-09-12T03:05:18.916319+0000 (oldest deadline 2024-09-12T03:05:39.416329+0000) 2024-09-12T03:06:01.243 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:00 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:00.975+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:25.817651+0000 front 2024-09-12T03:05:25.817711+0000 (oldest deadline 2024-09-12T03:05:50.517693+0000) 2024-09-12T03:06:01.243 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:00 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:00.975+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:30.518204+0000 front 2024-09-12T03:05:30.518383+0000 (oldest deadline 2024-09-12T03:05:55.218163+0000) 2024-09-12T03:06:01.415 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:01 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:01.158+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:16.844053+0000 front 2024-09-12T03:05:16.843858+0000 (oldest deadline 2024-09-12T03:05:42.644259+0000) 2024-09-12T03:06:01.415 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:01 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:01.158+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:22.644805+0000 front 2024-09-12T03:05:22.644911+0000 (oldest deadline 2024-09-12T03:05:47.344653+0000) 2024-09-12T03:06:01.416 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:01 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:01.158+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:31.846594+0000 front 2024-09-12T03:05:31.846677+0000 (oldest deadline 2024-09-12T03:05:59.447156+0000) 2024-09-12T03:06:01.560 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 12 03:06:01 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-6[56400]: 2024-09-12T03:06:01.241+0000 7f8e4dd48640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:06:01.560 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 12 03:06:01 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-6[56400]: 2024-09-12T03:06:01.241+0000 7f8e4dd48640 -1 osd.6 86 *** Got signal Terminated *** 2024-09-12T03:06:01.560 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 12 03:06:01 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-6[56400]: 2024-09-12T03:06:01.241+0000 7f8e4dd48640 -1 osd.6 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T03:06:01.561 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:01 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:01.263+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:14.571718+0000 front 2024-09-12T03:05:14.571768+0000 (oldest deadline 2024-09-12T03:05:39.772780+0000) 2024-09-12T03:06:01.561 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:01 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:01.263+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:24.374445+0000 front 2024-09-12T03:05:24.374549+0000 (oldest deadline 2024-09-12T03:05:48.374868+0000) 2024-09-12T03:06:01.561 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:01 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:01.263+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:34.276237+0000 front 2024-09-12T03:05:34.276182+0000 (oldest deadline 2024-09-12T03:05:58.976098+0000) 2024-09-12T03:06:01.706 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 12 03:06:01 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-7[52718]: 2024-09-12T03:06:01.242+0000 7f331b9c0640 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:24.091027+0000 front 2024-09-12T03:05:24.090973+0000 (oldest deadline 2024-09-12T03:05:49.991085+0000) 2024-09-12T03:06:01.706 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 12 03:06:01 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-7[52718]: 2024-09-12T03:06:01.242+0000 7f331b9c0640 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:34.592906+0000 front 2024-09-12T03:05:34.592762+0000 (oldest deadline 2024-09-12T03:05:56.892580+0000) 2024-09-12T03:06:02.165 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:02 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:02.117+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:16.844053+0000 front 2024-09-12T03:05:16.843858+0000 (oldest deadline 2024-09-12T03:05:42.644259+0000) 2024-09-12T03:06:02.166 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:02 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:02.117+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:22.644805+0000 front 2024-09-12T03:05:22.644911+0000 (oldest deadline 2024-09-12T03:05:47.344653+0000) 2024-09-12T03:06:02.166 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:02 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:02.117+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:31.846594+0000 front 2024-09-12T03:05:31.846677+0000 (oldest deadline 2024-09-12T03:05:59.447156+0000) 2024-09-12T03:06:02.166 INFO:journalctl@ceph.osd.8.smithi148.stdout:Sep 12 03:06:01 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-8[53762]: 2024-09-12T03:06:01.899+0000 7f7e0d911640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:25.877300+0000 front 2024-09-12T03:05:25.877333+0000 (oldest deadline 2024-09-12T03:05:47.577274+0000) 2024-09-12T03:06:02.166 INFO:journalctl@ceph.osd.8.smithi148.stdout:Sep 12 03:06:01 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-8[53762]: 2024-09-12T03:06:01.899+0000 7f7e0d911640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:31.678293+0000 front 2024-09-12T03:05:31.678404+0000 (oldest deadline 2024-09-12T03:05:56.978470+0000) 2024-09-12T03:06:02.219 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:01 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:01.962+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:18.916404+0000 front 2024-09-12T03:05:18.916319+0000 (oldest deadline 2024-09-12T03:05:39.416329+0000) 2024-09-12T03:06:02.220 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:01 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:01.962+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:25.817651+0000 front 2024-09-12T03:05:25.817711+0000 (oldest deadline 2024-09-12T03:05:50.517693+0000) 2024-09-12T03:06:02.220 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:01 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:01.962+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:30.518204+0000 front 2024-09-12T03:05:30.518383+0000 (oldest deadline 2024-09-12T03:05:55.218163+0000) 2024-09-12T03:06:02.560 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 12 03:06:02 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-6[56400]: 2024-09-12T03:06:02.103+0000 7f8e4a34d640 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:19.024576+0000 front 2024-09-12T03:05:19.024530+0000 (oldest deadline 2024-09-12T03:05:42.524750+0000) 2024-09-12T03:06:02.560 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 12 03:06:02 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-6[56400]: 2024-09-12T03:06:02.103+0000 7f8e4a34d640 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:23.025982+0000 front 2024-09-12T03:05:23.025795+0000 (oldest deadline 2024-09-12T03:05:48.925748+0000) 2024-09-12T03:06:02.560 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 12 03:06:02 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-6[56400]: 2024-09-12T03:06:02.103+0000 7f8e4a34d640 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:31.628129+0000 front 2024-09-12T03:05:31.628211+0000 (oldest deadline 2024-09-12T03:05:55.127992+0000) 2024-09-12T03:06:02.560 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:02 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:02.222+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:14.571718+0000 front 2024-09-12T03:05:14.571768+0000 (oldest deadline 2024-09-12T03:05:39.772780+0000) 2024-09-12T03:06:02.561 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:02 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:02.222+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:24.374445+0000 front 2024-09-12T03:05:24.374549+0000 (oldest deadline 2024-09-12T03:05:48.374868+0000) 2024-09-12T03:06:02.561 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:02 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:02.222+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:34.276237+0000 front 2024-09-12T03:05:34.276182+0000 (oldest deadline 2024-09-12T03:05:58.976098+0000) 2024-09-12T03:06:02.706 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 12 03:06:02 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-7[52718]: 2024-09-12T03:06:02.219+0000 7f331b9c0640 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:24.091027+0000 front 2024-09-12T03:05:24.090973+0000 (oldest deadline 2024-09-12T03:05:49.991085+0000) 2024-09-12T03:06:02.706 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 12 03:06:02 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-7[52718]: 2024-09-12T03:06:02.219+0000 7f331b9c0640 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:34.592906+0000 front 2024-09-12T03:05:34.592762+0000 (oldest deadline 2024-09-12T03:05:56.892580+0000) 2024-09-12T03:06:03.165 INFO:journalctl@ceph.osd.8.smithi148.stdout:Sep 12 03:06:02 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-8[53762]: 2024-09-12T03:06:02.915+0000 7f7e0d911640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:25.877300+0000 front 2024-09-12T03:05:25.877333+0000 (oldest deadline 2024-09-12T03:05:47.577274+0000) 2024-09-12T03:06:03.166 INFO:journalctl@ceph.osd.8.smithi148.stdout:Sep 12 03:06:02 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-8[53762]: 2024-09-12T03:06:02.915+0000 7f7e0d911640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:31.678293+0000 front 2024-09-12T03:05:31.678404+0000 (oldest deadline 2024-09-12T03:05:56.978470+0000) 2024-09-12T03:06:03.166 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:03 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:03.089+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:16.844053+0000 front 2024-09-12T03:05:16.843858+0000 (oldest deadline 2024-09-12T03:05:42.644259+0000) 2024-09-12T03:06:03.166 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:03 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:03.089+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:22.644805+0000 front 2024-09-12T03:05:22.644911+0000 (oldest deadline 2024-09-12T03:05:47.344653+0000) 2024-09-12T03:06:03.166 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:03 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:03.089+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:31.846594+0000 front 2024-09-12T03:05:31.846677+0000 (oldest deadline 2024-09-12T03:05:59.447156+0000) 2024-09-12T03:06:03.456 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 12 03:06:03 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-7[52718]: 2024-09-12T03:06:03.198+0000 7f331b9c0640 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:24.091027+0000 front 2024-09-12T03:05:24.090973+0000 (oldest deadline 2024-09-12T03:05:49.991085+0000) 2024-09-12T03:06:03.456 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 12 03:06:03 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-7[52718]: 2024-09-12T03:06:03.198+0000 7f331b9c0640 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:34.592906+0000 front 2024-09-12T03:05:34.592762+0000 (oldest deadline 2024-09-12T03:05:56.892580+0000) 2024-09-12T03:06:03.457 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:02 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:02.987+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:18.916404+0000 front 2024-09-12T03:05:18.916319+0000 (oldest deadline 2024-09-12T03:05:39.416329+0000) 2024-09-12T03:06:03.457 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:02 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:02.987+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:25.817651+0000 front 2024-09-12T03:05:25.817711+0000 (oldest deadline 2024-09-12T03:05:50.517693+0000) 2024-09-12T03:06:03.457 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:02 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:02.987+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:30.518204+0000 front 2024-09-12T03:05:30.518383+0000 (oldest deadline 2024-09-12T03:05:55.218163+0000) 2024-09-12T03:06:03.560 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 12 03:06:03 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-6[56400]: 2024-09-12T03:06:03.135+0000 7f8e4a34d640 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:19.024576+0000 front 2024-09-12T03:05:19.024530+0000 (oldest deadline 2024-09-12T03:05:42.524750+0000) 2024-09-12T03:06:03.560 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 12 03:06:03 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-6[56400]: 2024-09-12T03:06:03.135+0000 7f8e4a34d640 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:23.025982+0000 front 2024-09-12T03:05:23.025795+0000 (oldest deadline 2024-09-12T03:05:48.925748+0000) 2024-09-12T03:06:03.560 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 12 03:06:03 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-6[56400]: 2024-09-12T03:06:03.135+0000 7f8e4a34d640 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:31.628129+0000 front 2024-09-12T03:05:31.628211+0000 (oldest deadline 2024-09-12T03:05:55.127992+0000) 2024-09-12T03:06:03.560 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:03 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:03.198+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:14.571718+0000 front 2024-09-12T03:05:14.571768+0000 (oldest deadline 2024-09-12T03:05:39.772780+0000) 2024-09-12T03:06:03.561 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:03 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:03.198+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:24.374445+0000 front 2024-09-12T03:05:24.374549+0000 (oldest deadline 2024-09-12T03:05:48.374868+0000) 2024-09-12T03:06:03.561 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:03 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:03.198+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:34.276237+0000 front 2024-09-12T03:05:34.276182+0000 (oldest deadline 2024-09-12T03:05:58.976098+0000) 2024-09-12T03:06:04.415 INFO:journalctl@ceph.osd.8.smithi148.stdout:Sep 12 03:06:03 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-8[53762]: 2024-09-12T03:06:03.933+0000 7f7e0d911640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:25.877300+0000 front 2024-09-12T03:05:25.877333+0000 (oldest deadline 2024-09-12T03:05:47.577274+0000) 2024-09-12T03:06:04.415 INFO:journalctl@ceph.osd.8.smithi148.stdout:Sep 12 03:06:03 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-8[53762]: 2024-09-12T03:06:03.933+0000 7f7e0d911640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:31.678293+0000 front 2024-09-12T03:05:31.678404+0000 (oldest deadline 2024-09-12T03:05:56.978470+0000) 2024-09-12T03:06:04.416 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:04 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:04.066+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:16.844053+0000 front 2024-09-12T03:05:16.843858+0000 (oldest deadline 2024-09-12T03:05:42.644259+0000) 2024-09-12T03:06:04.416 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:04 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:04.066+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:22.644805+0000 front 2024-09-12T03:05:22.644911+0000 (oldest deadline 2024-09-12T03:05:47.344653+0000) 2024-09-12T03:06:04.416 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:04 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:04.066+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:31.846594+0000 front 2024-09-12T03:05:31.846677+0000 (oldest deadline 2024-09-12T03:05:59.447156+0000) 2024-09-12T03:06:04.416 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:04 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:04.066+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-12T03:05:39.447493+0000 front 2024-09-12T03:05:39.447609+0000 (oldest deadline 2024-09-12T03:06:03.547590+0000) 2024-09-12T03:06:04.456 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 12 03:06:04 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-7[52718]: 2024-09-12T03:06:04.238+0000 7f331b9c0640 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:24.091027+0000 front 2024-09-12T03:05:24.090973+0000 (oldest deadline 2024-09-12T03:05:49.991085+0000) 2024-09-12T03:06:04.456 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 12 03:06:04 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-7[52718]: 2024-09-12T03:06:04.238+0000 7f331b9c0640 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:34.592906+0000 front 2024-09-12T03:05:34.592762+0000 (oldest deadline 2024-09-12T03:05:56.892580+0000) 2024-09-12T03:06:04.457 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:04 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:04.028+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:18.916404+0000 front 2024-09-12T03:05:18.916319+0000 (oldest deadline 2024-09-12T03:05:39.416329+0000) 2024-09-12T03:06:04.457 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:04 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:04.028+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:25.817651+0000 front 2024-09-12T03:05:25.817711+0000 (oldest deadline 2024-09-12T03:05:50.517693+0000) 2024-09-12T03:06:04.457 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:04 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:04.028+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:30.518204+0000 front 2024-09-12T03:05:30.518383+0000 (oldest deadline 2024-09-12T03:05:55.218163+0000) 2024-09-12T03:06:04.560 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 12 03:06:04 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-6[56400]: 2024-09-12T03:06:04.174+0000 7f8e4a34d640 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:19.024576+0000 front 2024-09-12T03:05:19.024530+0000 (oldest deadline 2024-09-12T03:05:42.524750+0000) 2024-09-12T03:06:04.560 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 12 03:06:04 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-6[56400]: 2024-09-12T03:06:04.174+0000 7f8e4a34d640 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:23.025982+0000 front 2024-09-12T03:05:23.025795+0000 (oldest deadline 2024-09-12T03:05:48.925748+0000) 2024-09-12T03:06:04.560 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 12 03:06:04 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-6[56400]: 2024-09-12T03:06:04.174+0000 7f8e4a34d640 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:31.628129+0000 front 2024-09-12T03:05:31.628211+0000 (oldest deadline 2024-09-12T03:05:55.127992+0000) 2024-09-12T03:06:04.560 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:04 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:04.199+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:14.571718+0000 front 2024-09-12T03:05:14.571768+0000 (oldest deadline 2024-09-12T03:05:39.772780+0000) 2024-09-12T03:06:04.561 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:04 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:04.199+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:24.374445+0000 front 2024-09-12T03:05:24.374549+0000 (oldest deadline 2024-09-12T03:05:48.374868+0000) 2024-09-12T03:06:04.561 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:04 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:04.199+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:34.276237+0000 front 2024-09-12T03:05:34.276182+0000 (oldest deadline 2024-09-12T03:05:58.976098+0000) 2024-09-12T03:06:05.165 INFO:journalctl@ceph.osd.8.smithi148.stdout:Sep 12 03:06:04 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-8[53762]: 2024-09-12T03:06:04.906+0000 7f7e0d911640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:25.877300+0000 front 2024-09-12T03:05:25.877333+0000 (oldest deadline 2024-09-12T03:05:47.577274+0000) 2024-09-12T03:06:05.165 INFO:journalctl@ceph.osd.8.smithi148.stdout:Sep 12 03:06:04 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-8[53762]: 2024-09-12T03:06:04.906+0000 7f7e0d911640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:31.678293+0000 front 2024-09-12T03:05:31.678404+0000 (oldest deadline 2024-09-12T03:05:56.978470+0000) 2024-09-12T03:06:05.166 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:05 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:05.094+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:16.844053+0000 front 2024-09-12T03:05:16.843858+0000 (oldest deadline 2024-09-12T03:05:42.644259+0000) 2024-09-12T03:06:05.166 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:05 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:05.094+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:22.644805+0000 front 2024-09-12T03:05:22.644911+0000 (oldest deadline 2024-09-12T03:05:47.344653+0000) 2024-09-12T03:06:05.166 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:05 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:05.094+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:31.846594+0000 front 2024-09-12T03:05:31.846677+0000 (oldest deadline 2024-09-12T03:05:59.447156+0000) 2024-09-12T03:06:05.166 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:05 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:05.094+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-12T03:05:39.447493+0000 front 2024-09-12T03:05:39.447609+0000 (oldest deadline 2024-09-12T03:06:03.547590+0000) 2024-09-12T03:06:05.456 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 12 03:06:05 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-7[52718]: 2024-09-12T03:06:05.259+0000 7f331b9c0640 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:24.091027+0000 front 2024-09-12T03:05:24.090973+0000 (oldest deadline 2024-09-12T03:05:49.991085+0000) 2024-09-12T03:06:05.456 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 12 03:06:05 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-7[52718]: 2024-09-12T03:06:05.259+0000 7f331b9c0640 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:34.592906+0000 front 2024-09-12T03:05:34.592762+0000 (oldest deadline 2024-09-12T03:05:56.892580+0000) 2024-09-12T03:06:05.457 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:05 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:05.059+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:18.916404+0000 front 2024-09-12T03:05:18.916319+0000 (oldest deadline 2024-09-12T03:05:39.416329+0000) 2024-09-12T03:06:05.457 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:05 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:05.059+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:25.817651+0000 front 2024-09-12T03:05:25.817711+0000 (oldest deadline 2024-09-12T03:05:50.517693+0000) 2024-09-12T03:06:05.457 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:05 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:05.059+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:30.518204+0000 front 2024-09-12T03:05:30.518383+0000 (oldest deadline 2024-09-12T03:05:55.218163+0000) 2024-09-12T03:06:05.560 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:05 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:05.196+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:14.571718+0000 front 2024-09-12T03:05:14.571768+0000 (oldest deadline 2024-09-12T03:05:39.772780+0000) 2024-09-12T03:06:05.560 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:05 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:05.196+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:24.374445+0000 front 2024-09-12T03:05:24.374549+0000 (oldest deadline 2024-09-12T03:05:48.374868+0000) 2024-09-12T03:06:05.560 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:05 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:05.196+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:34.276237+0000 front 2024-09-12T03:05:34.276182+0000 (oldest deadline 2024-09-12T03:05:58.976098+0000) 2024-09-12T03:06:05.560 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 12 03:06:05 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-6[56400]: 2024-09-12T03:06:05.150+0000 7f8e4a34d640 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:19.024576+0000 front 2024-09-12T03:05:19.024530+0000 (oldest deadline 2024-09-12T03:05:42.524750+0000) 2024-09-12T03:06:05.561 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 12 03:06:05 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-6[56400]: 2024-09-12T03:06:05.150+0000 7f8e4a34d640 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:23.025982+0000 front 2024-09-12T03:05:23.025795+0000 (oldest deadline 2024-09-12T03:05:48.925748+0000) 2024-09-12T03:06:05.561 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 12 03:06:05 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-6[56400]: 2024-09-12T03:06:05.150+0000 7f8e4a34d640 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:31.628129+0000 front 2024-09-12T03:05:31.628211+0000 (oldest deadline 2024-09-12T03:05:55.127992+0000) 2024-09-12T03:06:06.142 INFO:journalctl@ceph.osd.8.smithi148.stdout:Sep 12 03:06:05 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-8[53762]: 2024-09-12T03:06:05.863+0000 7f7e0d911640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:25.877300+0000 front 2024-09-12T03:05:25.877333+0000 (oldest deadline 2024-09-12T03:05:47.577274+0000) 2024-09-12T03:06:06.143 INFO:journalctl@ceph.osd.8.smithi148.stdout:Sep 12 03:06:05 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-8[53762]: 2024-09-12T03:06:05.863+0000 7f7e0d911640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:31.678293+0000 front 2024-09-12T03:05:31.678404+0000 (oldest deadline 2024-09-12T03:05:56.978470+0000) 2024-09-12T03:06:06.300 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:06 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:06.047+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:18.916404+0000 front 2024-09-12T03:05:18.916319+0000 (oldest deadline 2024-09-12T03:05:39.416329+0000) 2024-09-12T03:06:06.301 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:06 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:06.047+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:25.817651+0000 front 2024-09-12T03:05:25.817711+0000 (oldest deadline 2024-09-12T03:05:50.517693+0000) 2024-09-12T03:06:06.301 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:06 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:06.047+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:30.518204+0000 front 2024-09-12T03:05:30.518383+0000 (oldest deadline 2024-09-12T03:05:55.218163+0000) 2024-09-12T03:06:06.415 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:06 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:06.142+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:16.844053+0000 front 2024-09-12T03:05:16.843858+0000 (oldest deadline 2024-09-12T03:05:42.644259+0000) 2024-09-12T03:06:06.415 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:06 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:06.142+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:22.644805+0000 front 2024-09-12T03:05:22.644911+0000 (oldest deadline 2024-09-12T03:05:47.344653+0000) 2024-09-12T03:06:06.415 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:06 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:06.142+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:31.846594+0000 front 2024-09-12T03:05:31.846677+0000 (oldest deadline 2024-09-12T03:05:59.447156+0000) 2024-09-12T03:06:06.416 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:06 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:06.142+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-12T03:05:39.447493+0000 front 2024-09-12T03:05:39.447609+0000 (oldest deadline 2024-09-12T03:06:03.547590+0000) 2024-09-12T03:06:06.561 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 12 03:06:06 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-6[56400]: 2024-09-12T03:06:06.199+0000 7f8e4a34d640 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:19.024576+0000 front 2024-09-12T03:05:19.024530+0000 (oldest deadline 2024-09-12T03:05:42.524750+0000) 2024-09-12T03:06:06.561 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 12 03:06:06 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-6[56400]: 2024-09-12T03:06:06.199+0000 7f8e4a34d640 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:23.025982+0000 front 2024-09-12T03:05:23.025795+0000 (oldest deadline 2024-09-12T03:05:48.925748+0000) 2024-09-12T03:06:06.561 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 12 03:06:06 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-6[56400]: 2024-09-12T03:06:06.199+0000 7f8e4a34d640 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:31.628129+0000 front 2024-09-12T03:05:31.628211+0000 (oldest deadline 2024-09-12T03:05:55.127992+0000) 2024-09-12T03:06:06.561 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 12 03:06:06 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-6[56400]: 2024-09-12T03:06:06.199+0000 7f8e4a34d640 -1 osd.6 86 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-12T03:05:39.729375+0000 front 2024-09-12T03:05:39.729393+0000 (oldest deadline 2024-09-12T03:06:05.629506+0000) 2024-09-12T03:06:06.561 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:06 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:06.191+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:14.571718+0000 front 2024-09-12T03:05:14.571768+0000 (oldest deadline 2024-09-12T03:05:39.772780+0000) 2024-09-12T03:06:06.561 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:06 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:06.191+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:24.374445+0000 front 2024-09-12T03:05:24.374549+0000 (oldest deadline 2024-09-12T03:05:48.374868+0000) 2024-09-12T03:06:06.561 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:06 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:06.191+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:34.276237+0000 front 2024-09-12T03:05:34.276182+0000 (oldest deadline 2024-09-12T03:05:58.976098+0000) 2024-09-12T03:06:06.706 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 12 03:06:06 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-7[52718]: 2024-09-12T03:06:06.300+0000 7f331b9c0640 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:24.091027+0000 front 2024-09-12T03:05:24.090973+0000 (oldest deadline 2024-09-12T03:05:49.991085+0000) 2024-09-12T03:06:06.706 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 12 03:06:06 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-7[52718]: 2024-09-12T03:06:06.300+0000 7f331b9c0640 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:34.592906+0000 front 2024-09-12T03:05:34.592762+0000 (oldest deadline 2024-09-12T03:05:56.892580+0000) 2024-09-12T03:06:06.706 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 12 03:06:06 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-7[52718]: 2024-09-12T03:06:06.300+0000 7f331b9c0640 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-12T03:05:40.894302+0000 front 2024-09-12T03:05:40.894338+0000 (oldest deadline 2024-09-12T03:06:05.594083+0000) 2024-09-12T03:06:06.949 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 12 03:06:06 smithi005 podman[118597]: 2024-09-12 03:06:06.656548167 +0000 UTC m=+5.523876751 container died f3c8dc1cccdc1e03ae315710e71ac97c39c12a6aa49187c00c9c92bf913ced48 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-6, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=squid-b53b6d3) 2024-09-12T03:06:07.165 INFO:journalctl@ceph.osd.8.smithi148.stdout:Sep 12 03:06:06 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-8[53762]: 2024-09-12T03:06:06.826+0000 7f7e0d911640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:25.877300+0000 front 2024-09-12T03:05:25.877333+0000 (oldest deadline 2024-09-12T03:05:47.577274+0000) 2024-09-12T03:06:07.166 INFO:journalctl@ceph.osd.8.smithi148.stdout:Sep 12 03:06:06 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-8[53762]: 2024-09-12T03:06:06.826+0000 7f7e0d911640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:31.678293+0000 front 2024-09-12T03:05:31.678404+0000 (oldest deadline 2024-09-12T03:05:56.978470+0000) 2024-09-12T03:06:07.310 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 12 03:06:07 smithi005 podman[118597]: 2024-09-12 03:06:07.157854971 +0000 UTC m=+6.025183597 container cleanup f3c8dc1cccdc1e03ae315710e71ac97c39c12a6aa49187c00c9c92bf913ced48 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-6, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-12T03:06:07.310 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 12 03:06:07 smithi005 bash[118597]: ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-6 2024-09-12T03:06:07.310 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:07 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:07.183+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:14.571718+0000 front 2024-09-12T03:05:14.571768+0000 (oldest deadline 2024-09-12T03:05:39.772780+0000) 2024-09-12T03:06:07.310 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:07 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:07.183+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:24.374445+0000 front 2024-09-12T03:05:24.374549+0000 (oldest deadline 2024-09-12T03:05:48.374868+0000) 2024-09-12T03:06:07.311 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:07 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:07.183+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:34.276237+0000 front 2024-09-12T03:05:34.276182+0000 (oldest deadline 2024-09-12T03:05:58.976098+0000) 2024-09-12T03:06:07.456 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 12 03:06:07 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-7[52718]: 2024-09-12T03:06:07.276+0000 7f331b9c0640 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:24.091027+0000 front 2024-09-12T03:05:24.090973+0000 (oldest deadline 2024-09-12T03:05:49.991085+0000) 2024-09-12T03:06:07.456 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 12 03:06:07 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-7[52718]: 2024-09-12T03:06:07.276+0000 7f331b9c0640 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:34.592906+0000 front 2024-09-12T03:05:34.592762+0000 (oldest deadline 2024-09-12T03:05:56.892580+0000) 2024-09-12T03:06:07.456 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 12 03:06:07 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-7[52718]: 2024-09-12T03:06:07.276+0000 7f331b9c0640 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-12T03:05:40.894302+0000 front 2024-09-12T03:05:40.894338+0000 (oldest deadline 2024-09-12T03:06:05.594083+0000) 2024-09-12T03:06:07.457 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:07 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:07.032+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:18.916404+0000 front 2024-09-12T03:05:18.916319+0000 (oldest deadline 2024-09-12T03:05:39.416329+0000) 2024-09-12T03:06:07.457 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:07 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:07.032+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:25.817651+0000 front 2024-09-12T03:05:25.817711+0000 (oldest deadline 2024-09-12T03:05:50.517693+0000) 2024-09-12T03:06:07.457 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:07 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:07.032+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:30.518204+0000 front 2024-09-12T03:05:30.518383+0000 (oldest deadline 2024-09-12T03:05:55.218163+0000) 2024-09-12T03:06:07.457 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:07 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:07.032+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-12T03:05:42.120310+0000 front 2024-09-12T03:05:42.120454+0000 (oldest deadline 2024-09-12T03:06:06.820355+0000) 2024-09-12T03:06:07.665 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:07 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:07.167+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:16.844053+0000 front 2024-09-12T03:05:16.843858+0000 (oldest deadline 2024-09-12T03:05:42.644259+0000) 2024-09-12T03:06:07.666 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:07 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:07.167+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:22.644805+0000 front 2024-09-12T03:05:22.644911+0000 (oldest deadline 2024-09-12T03:05:47.344653+0000) 2024-09-12T03:06:07.666 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:07 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:07.167+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:31.846594+0000 front 2024-09-12T03:05:31.846677+0000 (oldest deadline 2024-09-12T03:05:59.447156+0000) 2024-09-12T03:06:07.666 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:07 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:07.167+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-12T03:05:39.447493+0000 front 2024-09-12T03:05:39.447609+0000 (oldest deadline 2024-09-12T03:06:03.547590+0000) 2024-09-12T03:06:07.709 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 12 03:06:07 smithi005 podman[118612]: 2024-09-12 03:06:07.384215258 +0000 UTC m=+0.720886871 container remove f3c8dc1cccdc1e03ae315710e71ac97c39c12a6aa49187c00c9c92bf913ced48 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-6, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-12T03:06:07.970 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 12 03:06:07 smithi005 podman[118700]: 2024-09-12 03:06:07.709937513 +0000 UTC m=+0.112350368 container create a9518ba00ae1aabf44c3dbfdb2e4d482d25e249db4ec65a005b5ecb8e398f668 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-6-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T03:06:07.970 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 12 03:06:07 smithi005 podman[118700]: 2024-09-12 03:06:07.614339719 +0000 UTC m=+0.016752574 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:06:08.166 INFO:journalctl@ceph.osd.8.smithi148.stdout:Sep 12 03:06:07 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-8[53762]: 2024-09-12T03:06:07.845+0000 7f7e0d911640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:25.877300+0000 front 2024-09-12T03:05:25.877333+0000 (oldest deadline 2024-09-12T03:05:47.577274+0000) 2024-09-12T03:06:08.166 INFO:journalctl@ceph.osd.8.smithi148.stdout:Sep 12 03:06:07 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-8[53762]: 2024-09-12T03:06:07.845+0000 7f7e0d911640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:31.678293+0000 front 2024-09-12T03:05:31.678404+0000 (oldest deadline 2024-09-12T03:05:56.978470+0000) 2024-09-12T03:06:08.166 INFO:journalctl@ceph.osd.8.smithi148.stdout:Sep 12 03:06:07 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-8[53762]: 2024-09-12T03:06:07.845+0000 7f7e0d911640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-12T03:05:37.479439+0000 front 2024-09-12T03:05:37.479495+0000 (oldest deadline 2024-09-12T03:06:06.879955+0000) 2024-09-12T03:06:08.310 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:08 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:08.205+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:14.571718+0000 front 2024-09-12T03:05:14.571768+0000 (oldest deadline 2024-09-12T03:05:39.772780+0000) 2024-09-12T03:06:08.310 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:08 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:08.205+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:24.374445+0000 front 2024-09-12T03:05:24.374549+0000 (oldest deadline 2024-09-12T03:05:48.374868+0000) 2024-09-12T03:06:08.311 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:08 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:08.205+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:34.276237+0000 front 2024-09-12T03:05:34.276182+0000 (oldest deadline 2024-09-12T03:05:58.976098+0000) 2024-09-12T03:06:08.311 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:08 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:08.205+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-12T03:05:42.377230+0000 front 2024-09-12T03:05:42.377348+0000 (oldest deadline 2024-09-12T03:06:07.677394+0000) 2024-09-12T03:06:08.311 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 12 03:06:07 smithi005 podman[118700]: 2024-09-12 03:06:07.971197493 +0000 UTC m=+0.373610348 container init a9518ba00ae1aabf44c3dbfdb2e4d482d25e249db4ec65a005b5ecb8e398f668 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-6-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-12T03:06:08.311 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 12 03:06:07 smithi005 podman[118700]: 2024-09-12 03:06:07.974876778 +0000 UTC m=+0.377289637 container start a9518ba00ae1aabf44c3dbfdb2e4d482d25e249db4ec65a005b5ecb8e398f668 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-6-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=) 2024-09-12T03:06:08.312 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 12 03:06:08 smithi005 podman[118700]: 2024-09-12 03:06:08.054349316 +0000 UTC m=+0.456762181 container attach a9518ba00ae1aabf44c3dbfdb2e4d482d25e249db4ec65a005b5ecb8e398f668 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-6-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-12T03:06:08.312 INFO:journalctl@ceph.osd.6.smithi005.stdout:Sep 12 03:06:08 smithi005 podman[118700]: 2024-09-12 03:06:08.167512274 +0000 UTC m=+0.569925129 container died a9518ba00ae1aabf44c3dbfdb2e4d482d25e249db4ec65a005b5ecb8e398f668 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-6-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-12T03:06:08.456 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 12 03:06:08 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-7[52718]: 2024-09-12T03:06:08.268+0000 7f331b9c0640 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:24.091027+0000 front 2024-09-12T03:05:24.090973+0000 (oldest deadline 2024-09-12T03:05:49.991085+0000) 2024-09-12T03:06:08.457 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 12 03:06:08 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-7[52718]: 2024-09-12T03:06:08.268+0000 7f331b9c0640 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:34.592906+0000 front 2024-09-12T03:05:34.592762+0000 (oldest deadline 2024-09-12T03:05:56.892580+0000) 2024-09-12T03:06:08.457 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 12 03:06:08 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-7[52718]: 2024-09-12T03:06:08.268+0000 7f331b9c0640 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-12T03:05:40.894302+0000 front 2024-09-12T03:05:40.894338+0000 (oldest deadline 2024-09-12T03:06:05.594083+0000) 2024-09-12T03:06:08.457 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:08 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:08.062+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:18.916404+0000 front 2024-09-12T03:05:18.916319+0000 (oldest deadline 2024-09-12T03:05:39.416329+0000) 2024-09-12T03:06:08.457 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:08 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:08.062+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:25.817651+0000 front 2024-09-12T03:05:25.817711+0000 (oldest deadline 2024-09-12T03:05:50.517693+0000) 2024-09-12T03:06:08.458 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:08 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:08.062+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:30.518204+0000 front 2024-09-12T03:05:30.518383+0000 (oldest deadline 2024-09-12T03:05:55.218163+0000) 2024-09-12T03:06:08.458 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:08 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:08.062+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-12T03:05:42.120310+0000 front 2024-09-12T03:05:42.120454+0000 (oldest deadline 2024-09-12T03:06:06.820355+0000) 2024-09-12T03:06:08.665 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:08 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:08.174+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:16.844053+0000 front 2024-09-12T03:05:16.843858+0000 (oldest deadline 2024-09-12T03:05:42.644259+0000) 2024-09-12T03:06:08.665 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:08 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:08.174+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:22.644805+0000 front 2024-09-12T03:05:22.644911+0000 (oldest deadline 2024-09-12T03:05:47.344653+0000) 2024-09-12T03:06:08.665 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:08 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:08.174+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:31.846594+0000 front 2024-09-12T03:05:31.846677+0000 (oldest deadline 2024-09-12T03:05:59.447156+0000) 2024-09-12T03:06:08.666 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:08 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:08.174+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-12T03:05:39.447493+0000 front 2024-09-12T03:05:39.447609+0000 (oldest deadline 2024-09-12T03:06:03.547590+0000) 2024-09-12T03:06:08.821 DEBUG:teuthology.orchestra.run.smithi005:> sudo pkill -f 'journalctl -f -n 0 -u ceph-618305f8-70b1-11ef-bcea-c7b262605968@osd.6.service' 2024-09-12T03:06:08.862 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:06:08.862 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-12T03:06:08.862 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-12T03:06:08.863 DEBUG:teuthology.orchestra.run.smithi146:> sudo systemctl stop ceph-618305f8-70b1-11ef-bcea-c7b262605968@osd.7 2024-09-12T03:06:09.132 INFO:journalctl@ceph.osd.8.smithi148.stdout:Sep 12 03:06:08 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-8[53762]: 2024-09-12T03:06:08.851+0000 7f7e0d911640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:25.877300+0000 front 2024-09-12T03:05:25.877333+0000 (oldest deadline 2024-09-12T03:05:47.577274+0000) 2024-09-12T03:06:09.133 INFO:journalctl@ceph.osd.8.smithi148.stdout:Sep 12 03:06:08 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-8[53762]: 2024-09-12T03:06:08.851+0000 7f7e0d911640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:31.678293+0000 front 2024-09-12T03:05:31.678404+0000 (oldest deadline 2024-09-12T03:05:56.978470+0000) 2024-09-12T03:06:09.133 INFO:journalctl@ceph.osd.8.smithi148.stdout:Sep 12 03:06:08 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-8[53762]: 2024-09-12T03:06:08.851+0000 7f7e0d911640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-12T03:05:37.479439+0000 front 2024-09-12T03:05:37.479495+0000 (oldest deadline 2024-09-12T03:06:06.879955+0000) 2024-09-12T03:06:09.133 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:09 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:09.132+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:16.844053+0000 front 2024-09-12T03:05:16.843858+0000 (oldest deadline 2024-09-12T03:05:42.644259+0000) 2024-09-12T03:06:09.133 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:09 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:09.132+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:22.644805+0000 front 2024-09-12T03:05:22.644911+0000 (oldest deadline 2024-09-12T03:05:47.344653+0000) 2024-09-12T03:06:09.133 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:09 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:09.132+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:31.846594+0000 front 2024-09-12T03:05:31.846677+0000 (oldest deadline 2024-09-12T03:05:59.447156+0000) 2024-09-12T03:06:09.133 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:09 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:09.132+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-12T03:05:39.447493+0000 front 2024-09-12T03:05:39.447609+0000 (oldest deadline 2024-09-12T03:06:03.547590+0000) 2024-09-12T03:06:09.177 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 12 03:06:08 smithi146 systemd[1]: Stopping Ceph osd.7 for 618305f8-70b1-11ef-bcea-c7b262605968... 2024-09-12T03:06:09.178 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:09 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:09.045+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:18.916404+0000 front 2024-09-12T03:05:18.916319+0000 (oldest deadline 2024-09-12T03:05:39.416329+0000) 2024-09-12T03:06:09.178 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:09 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:09.045+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:25.817651+0000 front 2024-09-12T03:05:25.817711+0000 (oldest deadline 2024-09-12T03:05:50.517693+0000) 2024-09-12T03:06:09.178 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:09 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:09.045+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:30.518204+0000 front 2024-09-12T03:05:30.518383+0000 (oldest deadline 2024-09-12T03:05:55.218163+0000) 2024-09-12T03:06:09.178 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:09 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:09.045+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-12T03:05:42.120310+0000 front 2024-09-12T03:05:42.120454+0000 (oldest deadline 2024-09-12T03:06:06.820355+0000) 2024-09-12T03:06:09.456 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 12 03:06:09 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-7[52718]: 2024-09-12T03:06:09.177+0000 7f331f3bb640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:06:09.456 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 12 03:06:09 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-7[52718]: 2024-09-12T03:06:09.177+0000 7f331f3bb640 -1 osd.7 86 *** Got signal Terminated *** 2024-09-12T03:06:09.457 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 12 03:06:09 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-7[52718]: 2024-09-12T03:06:09.177+0000 7f331f3bb640 -1 osd.7 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T03:06:09.457 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 12 03:06:09 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-7[52718]: 2024-09-12T03:06:09.262+0000 7f331b9c0640 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:24.091027+0000 front 2024-09-12T03:05:24.090973+0000 (oldest deadline 2024-09-12T03:05:49.991085+0000) 2024-09-12T03:06:09.457 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 12 03:06:09 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-7[52718]: 2024-09-12T03:06:09.262+0000 7f331b9c0640 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:34.592906+0000 front 2024-09-12T03:05:34.592762+0000 (oldest deadline 2024-09-12T03:05:56.892580+0000) 2024-09-12T03:06:09.457 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 12 03:06:09 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-7[52718]: 2024-09-12T03:06:09.262+0000 7f331b9c0640 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-12T03:05:40.894302+0000 front 2024-09-12T03:05:40.894338+0000 (oldest deadline 2024-09-12T03:06:05.594083+0000) 2024-09-12T03:06:09.473 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:09 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:09.220+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:14.571718+0000 front 2024-09-12T03:05:14.571768+0000 (oldest deadline 2024-09-12T03:05:39.772780+0000) 2024-09-12T03:06:09.473 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:09 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:09.220+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:24.374445+0000 front 2024-09-12T03:05:24.374549+0000 (oldest deadline 2024-09-12T03:05:48.374868+0000) 2024-09-12T03:06:09.473 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:09 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:09.220+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:34.276237+0000 front 2024-09-12T03:05:34.276182+0000 (oldest deadline 2024-09-12T03:05:58.976098+0000) 2024-09-12T03:06:09.474 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:09 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:09.220+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-12T03:05:42.377230+0000 front 2024-09-12T03:05:42.377348+0000 (oldest deadline 2024-09-12T03:06:07.677394+0000) 2024-09-12T03:06:10.134 INFO:journalctl@ceph.osd.8.smithi148.stdout:Sep 12 03:06:09 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-8[53762]: 2024-09-12T03:06:09.827+0000 7f7e0d911640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:25.877300+0000 front 2024-09-12T03:05:25.877333+0000 (oldest deadline 2024-09-12T03:05:47.577274+0000) 2024-09-12T03:06:10.134 INFO:journalctl@ceph.osd.8.smithi148.stdout:Sep 12 03:06:09 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-8[53762]: 2024-09-12T03:06:09.827+0000 7f7e0d911640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:31.678293+0000 front 2024-09-12T03:05:31.678404+0000 (oldest deadline 2024-09-12T03:05:56.978470+0000) 2024-09-12T03:06:10.134 INFO:journalctl@ceph.osd.8.smithi148.stdout:Sep 12 03:06:09 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-8[53762]: 2024-09-12T03:06:09.827+0000 7f7e0d911640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-12T03:05:37.479439+0000 front 2024-09-12T03:05:37.479495+0000 (oldest deadline 2024-09-12T03:06:06.879955+0000) 2024-09-12T03:06:10.415 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:10 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:10.134+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:16.844053+0000 front 2024-09-12T03:05:16.843858+0000 (oldest deadline 2024-09-12T03:05:42.644259+0000) 2024-09-12T03:06:10.416 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:10 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:10.134+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:22.644805+0000 front 2024-09-12T03:05:22.644911+0000 (oldest deadline 2024-09-12T03:05:47.344653+0000) 2024-09-12T03:06:10.416 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:10 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:10.134+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:31.846594+0000 front 2024-09-12T03:05:31.846677+0000 (oldest deadline 2024-09-12T03:05:59.447156+0000) 2024-09-12T03:06:10.416 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:10 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:10.134+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-12T03:05:39.447493+0000 front 2024-09-12T03:05:39.447609+0000 (oldest deadline 2024-09-12T03:06:03.547590+0000) 2024-09-12T03:06:10.456 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 12 03:06:10 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-7[52718]: 2024-09-12T03:06:10.217+0000 7f331b9c0640 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:24.091027+0000 front 2024-09-12T03:05:24.090973+0000 (oldest deadline 2024-09-12T03:05:49.991085+0000) 2024-09-12T03:06:10.456 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 12 03:06:10 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-7[52718]: 2024-09-12T03:06:10.217+0000 7f331b9c0640 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:34.592906+0000 front 2024-09-12T03:05:34.592762+0000 (oldest deadline 2024-09-12T03:05:56.892580+0000) 2024-09-12T03:06:10.457 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 12 03:06:10 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-7[52718]: 2024-09-12T03:06:10.217+0000 7f331b9c0640 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-12T03:05:40.894302+0000 front 2024-09-12T03:05:40.894338+0000 (oldest deadline 2024-09-12T03:06:05.594083+0000) 2024-09-12T03:06:10.457 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:10 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:10.055+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:18.916404+0000 front 2024-09-12T03:05:18.916319+0000 (oldest deadline 2024-09-12T03:05:39.416329+0000) 2024-09-12T03:06:10.457 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:10 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:10.055+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:25.817651+0000 front 2024-09-12T03:05:25.817711+0000 (oldest deadline 2024-09-12T03:05:50.517693+0000) 2024-09-12T03:06:10.457 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:10 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:10.055+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:30.518204+0000 front 2024-09-12T03:05:30.518383+0000 (oldest deadline 2024-09-12T03:05:55.218163+0000) 2024-09-12T03:06:10.457 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:10 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:10.055+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-12T03:05:42.120310+0000 front 2024-09-12T03:05:42.120454+0000 (oldest deadline 2024-09-12T03:06:06.820355+0000) 2024-09-12T03:06:10.560 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:10 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:10.189+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:14.571718+0000 front 2024-09-12T03:05:14.571768+0000 (oldest deadline 2024-09-12T03:05:39.772780+0000) 2024-09-12T03:06:10.560 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:10 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:10.189+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:24.374445+0000 front 2024-09-12T03:05:24.374549+0000 (oldest deadline 2024-09-12T03:05:48.374868+0000) 2024-09-12T03:06:10.560 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:10 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:10.189+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:34.276237+0000 front 2024-09-12T03:05:34.276182+0000 (oldest deadline 2024-09-12T03:05:58.976098+0000) 2024-09-12T03:06:10.560 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:10 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:10.189+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-12T03:05:42.377230+0000 front 2024-09-12T03:05:42.377348+0000 (oldest deadline 2024-09-12T03:06:07.677394+0000) 2024-09-12T03:06:11.161 INFO:journalctl@ceph.osd.8.smithi148.stdout:Sep 12 03:06:10 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-8[53762]: 2024-09-12T03:06:10.826+0000 7f7e0d911640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:25.877300+0000 front 2024-09-12T03:05:25.877333+0000 (oldest deadline 2024-09-12T03:05:47.577274+0000) 2024-09-12T03:06:11.161 INFO:journalctl@ceph.osd.8.smithi148.stdout:Sep 12 03:06:10 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-8[53762]: 2024-09-12T03:06:10.826+0000 7f7e0d911640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:31.678293+0000 front 2024-09-12T03:05:31.678404+0000 (oldest deadline 2024-09-12T03:05:56.978470+0000) 2024-09-12T03:06:11.161 INFO:journalctl@ceph.osd.8.smithi148.stdout:Sep 12 03:06:10 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-8[53762]: 2024-09-12T03:06:10.826+0000 7f7e0d911640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-12T03:05:37.479439+0000 front 2024-09-12T03:05:37.479495+0000 (oldest deadline 2024-09-12T03:06:06.879955+0000) 2024-09-12T03:06:11.358 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 12 03:06:11 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-7[52718]: 2024-09-12T03:06:11.233+0000 7f331b9c0640 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:24.091027+0000 front 2024-09-12T03:05:24.090973+0000 (oldest deadline 2024-09-12T03:05:49.991085+0000) 2024-09-12T03:06:11.358 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 12 03:06:11 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-7[52718]: 2024-09-12T03:06:11.233+0000 7f331b9c0640 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:34.592906+0000 front 2024-09-12T03:05:34.592762+0000 (oldest deadline 2024-09-12T03:05:56.892580+0000) 2024-09-12T03:06:11.358 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 12 03:06:11 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-7[52718]: 2024-09-12T03:06:11.233+0000 7f331b9c0640 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-12T03:05:40.894302+0000 front 2024-09-12T03:05:40.894338+0000 (oldest deadline 2024-09-12T03:06:05.594083+0000) 2024-09-12T03:06:11.358 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:11 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:11.022+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:18.916404+0000 front 2024-09-12T03:05:18.916319+0000 (oldest deadline 2024-09-12T03:05:39.416329+0000) 2024-09-12T03:06:11.358 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:11 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:11.022+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:25.817651+0000 front 2024-09-12T03:05:25.817711+0000 (oldest deadline 2024-09-12T03:05:50.517693+0000) 2024-09-12T03:06:11.358 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:11 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:11.022+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:30.518204+0000 front 2024-09-12T03:05:30.518383+0000 (oldest deadline 2024-09-12T03:05:55.218163+0000) 2024-09-12T03:06:11.358 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:11 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:11.022+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-12T03:05:42.120310+0000 front 2024-09-12T03:05:42.120454+0000 (oldest deadline 2024-09-12T03:06:06.820355+0000) 2024-09-12T03:06:11.415 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:11 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:11.161+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:16.844053+0000 front 2024-09-12T03:05:16.843858+0000 (oldest deadline 2024-09-12T03:05:42.644259+0000) 2024-09-12T03:06:11.416 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:11 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:11.161+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:22.644805+0000 front 2024-09-12T03:05:22.644911+0000 (oldest deadline 2024-09-12T03:05:47.344653+0000) 2024-09-12T03:06:11.416 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:11 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:11.161+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:31.846594+0000 front 2024-09-12T03:05:31.846677+0000 (oldest deadline 2024-09-12T03:05:59.447156+0000) 2024-09-12T03:06:11.416 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:11 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:11.161+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-12T03:05:39.447493+0000 front 2024-09-12T03:05:39.447609+0000 (oldest deadline 2024-09-12T03:06:03.547590+0000) 2024-09-12T03:06:11.560 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:11 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:11.176+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:14.571718+0000 front 2024-09-12T03:05:14.571768+0000 (oldest deadline 2024-09-12T03:05:39.772780+0000) 2024-09-12T03:06:11.560 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:11 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:11.176+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:24.374445+0000 front 2024-09-12T03:05:24.374549+0000 (oldest deadline 2024-09-12T03:05:48.374868+0000) 2024-09-12T03:06:11.560 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:11 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:11.176+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:34.276237+0000 front 2024-09-12T03:05:34.276182+0000 (oldest deadline 2024-09-12T03:05:58.976098+0000) 2024-09-12T03:06:11.560 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:11 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:11.176+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-12T03:05:42.377230+0000 front 2024-09-12T03:05:42.377348+0000 (oldest deadline 2024-09-12T03:06:07.677394+0000) 2024-09-12T03:06:12.165 INFO:journalctl@ceph.osd.8.smithi148.stdout:Sep 12 03:06:11 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-8[53762]: 2024-09-12T03:06:11.820+0000 7f7e0d911640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:25.877300+0000 front 2024-09-12T03:05:25.877333+0000 (oldest deadline 2024-09-12T03:05:47.577274+0000) 2024-09-12T03:06:12.166 INFO:journalctl@ceph.osd.8.smithi148.stdout:Sep 12 03:06:11 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-8[53762]: 2024-09-12T03:06:11.820+0000 7f7e0d911640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:31.678293+0000 front 2024-09-12T03:05:31.678404+0000 (oldest deadline 2024-09-12T03:05:56.978470+0000) 2024-09-12T03:06:12.166 INFO:journalctl@ceph.osd.8.smithi148.stdout:Sep 12 03:06:11 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-8[53762]: 2024-09-12T03:06:11.820+0000 7f7e0d911640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-12T03:05:37.479439+0000 front 2024-09-12T03:05:37.479495+0000 (oldest deadline 2024-09-12T03:06:06.879955+0000) 2024-09-12T03:06:12.166 INFO:journalctl@ceph.osd.8.smithi148.stdout:Sep 12 03:06:11 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-8[53762]: 2024-09-12T03:06:11.820+0000 7f7e0d911640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-09-12T03:05:46.880420+0000 front 2024-09-12T03:05:46.880527+0000 (oldest deadline 2024-09-12T03:06:11.580369+0000) 2024-09-12T03:06:12.373 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 12 03:06:12 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-7[52718]: 2024-09-12T03:06:12.195+0000 7f331b9c0640 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:24.091027+0000 front 2024-09-12T03:05:24.090973+0000 (oldest deadline 2024-09-12T03:05:49.991085+0000) 2024-09-12T03:06:12.373 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 12 03:06:12 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-7[52718]: 2024-09-12T03:06:12.195+0000 7f331b9c0640 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:34.592906+0000 front 2024-09-12T03:05:34.592762+0000 (oldest deadline 2024-09-12T03:05:56.892580+0000) 2024-09-12T03:06:12.373 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 12 03:06:12 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-7[52718]: 2024-09-12T03:06:12.195+0000 7f331b9c0640 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-12T03:05:40.894302+0000 front 2024-09-12T03:05:40.894338+0000 (oldest deadline 2024-09-12T03:06:05.594083+0000) 2024-09-12T03:06:12.373 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:12 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:12.050+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:18.916404+0000 front 2024-09-12T03:05:18.916319+0000 (oldest deadline 2024-09-12T03:05:39.416329+0000) 2024-09-12T03:06:12.374 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:12 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:12.050+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:25.817651+0000 front 2024-09-12T03:05:25.817711+0000 (oldest deadline 2024-09-12T03:05:50.517693+0000) 2024-09-12T03:06:12.374 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:12 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:12.050+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:30.518204+0000 front 2024-09-12T03:05:30.518383+0000 (oldest deadline 2024-09-12T03:05:55.218163+0000) 2024-09-12T03:06:12.374 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:12 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:12.050+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-12T03:05:42.120310+0000 front 2024-09-12T03:05:42.120454+0000 (oldest deadline 2024-09-12T03:06:06.820355+0000) 2024-09-12T03:06:12.560 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:12 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:12.131+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:14.571718+0000 front 2024-09-12T03:05:14.571768+0000 (oldest deadline 2024-09-12T03:05:39.772780+0000) 2024-09-12T03:06:12.560 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:12 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:12.131+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:24.374445+0000 front 2024-09-12T03:05:24.374549+0000 (oldest deadline 2024-09-12T03:05:48.374868+0000) 2024-09-12T03:06:12.560 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:12 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:12.131+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:34.276237+0000 front 2024-09-12T03:05:34.276182+0000 (oldest deadline 2024-09-12T03:05:58.976098+0000) 2024-09-12T03:06:12.560 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:12 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:12.131+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-12T03:05:42.377230+0000 front 2024-09-12T03:05:42.377348+0000 (oldest deadline 2024-09-12T03:06:07.677394+0000) 2024-09-12T03:06:12.665 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:12 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:12.202+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:16.844053+0000 front 2024-09-12T03:05:16.843858+0000 (oldest deadline 2024-09-12T03:05:42.644259+0000) 2024-09-12T03:06:12.666 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:12 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:12.202+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:22.644805+0000 front 2024-09-12T03:05:22.644911+0000 (oldest deadline 2024-09-12T03:05:47.344653+0000) 2024-09-12T03:06:12.666 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:12 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:12.202+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:31.846594+0000 front 2024-09-12T03:05:31.846677+0000 (oldest deadline 2024-09-12T03:05:59.447156+0000) 2024-09-12T03:06:12.666 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:12 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:12.202+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-12T03:05:39.447493+0000 front 2024-09-12T03:05:39.447609+0000 (oldest deadline 2024-09-12T03:06:03.547590+0000) 2024-09-12T03:06:13.165 INFO:journalctl@ceph.osd.8.smithi148.stdout:Sep 12 03:06:12 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-8[53762]: 2024-09-12T03:06:12.793+0000 7f7e0d911640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:25.877300+0000 front 2024-09-12T03:05:25.877333+0000 (oldest deadline 2024-09-12T03:05:47.577274+0000) 2024-09-12T03:06:13.165 INFO:journalctl@ceph.osd.8.smithi148.stdout:Sep 12 03:06:12 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-8[53762]: 2024-09-12T03:06:12.793+0000 7f7e0d911640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:31.678293+0000 front 2024-09-12T03:05:31.678404+0000 (oldest deadline 2024-09-12T03:05:56.978470+0000) 2024-09-12T03:06:13.166 INFO:journalctl@ceph.osd.8.smithi148.stdout:Sep 12 03:06:12 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-8[53762]: 2024-09-12T03:06:12.793+0000 7f7e0d911640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-12T03:05:37.479439+0000 front 2024-09-12T03:05:37.479495+0000 (oldest deadline 2024-09-12T03:06:06.879955+0000) 2024-09-12T03:06:13.166 INFO:journalctl@ceph.osd.8.smithi148.stdout:Sep 12 03:06:12 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-8[53762]: 2024-09-12T03:06:12.793+0000 7f7e0d911640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-09-12T03:05:46.880420+0000 front 2024-09-12T03:05:46.880527+0000 (oldest deadline 2024-09-12T03:06:11.580369+0000) 2024-09-12T03:06:13.433 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:13 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:13.236+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:16.844053+0000 front 2024-09-12T03:05:16.843858+0000 (oldest deadline 2024-09-12T03:05:42.644259+0000) 2024-09-12T03:06:13.434 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:13 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:13.236+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:22.644805+0000 front 2024-09-12T03:05:22.644911+0000 (oldest deadline 2024-09-12T03:05:47.344653+0000) 2024-09-12T03:06:13.434 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:13 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:13.236+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:31.846594+0000 front 2024-09-12T03:05:31.846677+0000 (oldest deadline 2024-09-12T03:05:59.447156+0000) 2024-09-12T03:06:13.434 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:13 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:13.236+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-12T03:05:39.447493+0000 front 2024-09-12T03:05:39.447609+0000 (oldest deadline 2024-09-12T03:06:03.547590+0000) 2024-09-12T03:06:13.434 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:13 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:13.236+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-09-12T03:05:48.248794+0000 front 2024-09-12T03:05:48.248779+0000 (oldest deadline 2024-09-12T03:06:12.348647+0000) 2024-09-12T03:06:13.457 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 12 03:06:13 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-7[52718]: 2024-09-12T03:06:13.233+0000 7f331b9c0640 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:24.091027+0000 front 2024-09-12T03:05:24.090973+0000 (oldest deadline 2024-09-12T03:05:49.991085+0000) 2024-09-12T03:06:13.457 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 12 03:06:13 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-7[52718]: 2024-09-12T03:06:13.233+0000 7f331b9c0640 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:34.592906+0000 front 2024-09-12T03:05:34.592762+0000 (oldest deadline 2024-09-12T03:05:56.892580+0000) 2024-09-12T03:06:13.457 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 12 03:06:13 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-7[52718]: 2024-09-12T03:06:13.233+0000 7f331b9c0640 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-12T03:05:40.894302+0000 front 2024-09-12T03:05:40.894338+0000 (oldest deadline 2024-09-12T03:06:05.594083+0000) 2024-09-12T03:06:13.457 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 12 03:06:13 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-7[52718]: 2024-09-12T03:06:13.233+0000 7f331b9c0640 -1 osd.7 86 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-09-12T03:05:48.395645+0000 front 2024-09-12T03:05:48.395734+0000 (oldest deadline 2024-09-12T03:06:12.996205+0000) 2024-09-12T03:06:13.457 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:13 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:13.070+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:18.916404+0000 front 2024-09-12T03:05:18.916319+0000 (oldest deadline 2024-09-12T03:05:39.416329+0000) 2024-09-12T03:06:13.458 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:13 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:13.070+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:25.817651+0000 front 2024-09-12T03:05:25.817711+0000 (oldest deadline 2024-09-12T03:05:50.517693+0000) 2024-09-12T03:06:13.458 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:13 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:13.070+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:30.518204+0000 front 2024-09-12T03:05:30.518383+0000 (oldest deadline 2024-09-12T03:05:55.218163+0000) 2024-09-12T03:06:13.458 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:13 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:13.070+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-12T03:05:42.120310+0000 front 2024-09-12T03:05:42.120454+0000 (oldest deadline 2024-09-12T03:06:06.820355+0000) 2024-09-12T03:06:13.560 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:13 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:13.091+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:14.571718+0000 front 2024-09-12T03:05:14.571768+0000 (oldest deadline 2024-09-12T03:05:39.772780+0000) 2024-09-12T03:06:13.560 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:13 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:13.091+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:24.374445+0000 front 2024-09-12T03:05:24.374549+0000 (oldest deadline 2024-09-12T03:05:48.374868+0000) 2024-09-12T03:06:13.560 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:13 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:13.091+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:34.276237+0000 front 2024-09-12T03:05:34.276182+0000 (oldest deadline 2024-09-12T03:05:58.976098+0000) 2024-09-12T03:06:13.560 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:13 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:13.091+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-12T03:05:42.377230+0000 front 2024-09-12T03:05:42.377348+0000 (oldest deadline 2024-09-12T03:06:07.677394+0000) 2024-09-12T03:06:13.560 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:13 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:13.091+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-09-12T03:05:47.678051+0000 front 2024-09-12T03:05:47.677984+0000 (oldest deadline 2024-09-12T03:06:12.377935+0000) 2024-09-12T03:06:14.165 INFO:journalctl@ceph.osd.8.smithi148.stdout:Sep 12 03:06:13 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-8[53762]: 2024-09-12T03:06:13.761+0000 7f7e0d911640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:25.877300+0000 front 2024-09-12T03:05:25.877333+0000 (oldest deadline 2024-09-12T03:05:47.577274+0000) 2024-09-12T03:06:14.166 INFO:journalctl@ceph.osd.8.smithi148.stdout:Sep 12 03:06:13 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-8[53762]: 2024-09-12T03:06:13.761+0000 7f7e0d911640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:31.678293+0000 front 2024-09-12T03:05:31.678404+0000 (oldest deadline 2024-09-12T03:05:56.978470+0000) 2024-09-12T03:06:14.166 INFO:journalctl@ceph.osd.8.smithi148.stdout:Sep 12 03:06:13 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-8[53762]: 2024-09-12T03:06:13.761+0000 7f7e0d911640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-12T03:05:37.479439+0000 front 2024-09-12T03:05:37.479495+0000 (oldest deadline 2024-09-12T03:06:06.879955+0000) 2024-09-12T03:06:14.166 INFO:journalctl@ceph.osd.8.smithi148.stdout:Sep 12 03:06:13 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-8[53762]: 2024-09-12T03:06:13.761+0000 7f7e0d911640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-09-12T03:05:46.880420+0000 front 2024-09-12T03:05:46.880527+0000 (oldest deadline 2024-09-12T03:06:11.580369+0000) 2024-09-12T03:06:14.310 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:14 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:14.050+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:14.571718+0000 front 2024-09-12T03:05:14.571768+0000 (oldest deadline 2024-09-12T03:05:39.772780+0000) 2024-09-12T03:06:14.310 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:14 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:14.050+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:24.374445+0000 front 2024-09-12T03:05:24.374549+0000 (oldest deadline 2024-09-12T03:05:48.374868+0000) 2024-09-12T03:06:14.310 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:14 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:14.050+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:34.276237+0000 front 2024-09-12T03:05:34.276182+0000 (oldest deadline 2024-09-12T03:05:58.976098+0000) 2024-09-12T03:06:14.310 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:14 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:14.050+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-12T03:05:42.377230+0000 front 2024-09-12T03:05:42.377348+0000 (oldest deadline 2024-09-12T03:06:07.677394+0000) 2024-09-12T03:06:14.310 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:14 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:14.050+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-09-12T03:05:47.678051+0000 front 2024-09-12T03:05:47.677984+0000 (oldest deadline 2024-09-12T03:06:12.377935+0000) 2024-09-12T03:06:14.456 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:14 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:14.116+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:18.916404+0000 front 2024-09-12T03:05:18.916319+0000 (oldest deadline 2024-09-12T03:05:39.416329+0000) 2024-09-12T03:06:14.456 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:14 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:14.116+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:25.817651+0000 front 2024-09-12T03:05:25.817711+0000 (oldest deadline 2024-09-12T03:05:50.517693+0000) 2024-09-12T03:06:14.456 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:14 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:14.116+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:30.518204+0000 front 2024-09-12T03:05:30.518383+0000 (oldest deadline 2024-09-12T03:05:55.218163+0000) 2024-09-12T03:06:14.456 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:14 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:14.116+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-12T03:05:42.120310+0000 front 2024-09-12T03:05:42.120454+0000 (oldest deadline 2024-09-12T03:06:06.820355+0000) 2024-09-12T03:06:14.456 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:14 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:14.116+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-09-12T03:05:47.921106+0000 front 2024-09-12T03:05:47.921133+0000 (oldest deadline 2024-09-12T03:06:13.821241+0000) 2024-09-12T03:06:14.665 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:14 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:14.269+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:16.844053+0000 front 2024-09-12T03:05:16.843858+0000 (oldest deadline 2024-09-12T03:05:42.644259+0000) 2024-09-12T03:06:14.665 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:14 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:14.269+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:22.644805+0000 front 2024-09-12T03:05:22.644911+0000 (oldest deadline 2024-09-12T03:05:47.344653+0000) 2024-09-12T03:06:14.666 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:14 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:14.269+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:31.846594+0000 front 2024-09-12T03:05:31.846677+0000 (oldest deadline 2024-09-12T03:05:59.447156+0000) 2024-09-12T03:06:14.666 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:14 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:14.269+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-12T03:05:39.447493+0000 front 2024-09-12T03:05:39.447609+0000 (oldest deadline 2024-09-12T03:06:03.547590+0000) 2024-09-12T03:06:14.666 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:14 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:14.269+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-09-12T03:05:48.248794+0000 front 2024-09-12T03:05:48.248779+0000 (oldest deadline 2024-09-12T03:06:12.348647+0000) 2024-09-12T03:06:14.894 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 12 03:06:14 smithi146 podman[69585]: 2024-09-12 03:06:14.635069573 +0000 UTC m=+5.577928543 container died f735ae846010eefc6b6c98101bc9268552eda6c67b5d5bed6502a0e91c3343c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-7, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-12T03:06:15.165 INFO:journalctl@ceph.osd.8.smithi148.stdout:Sep 12 03:06:14 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-8[53762]: 2024-09-12T03:06:14.763+0000 7f7e0d911640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:25.877300+0000 front 2024-09-12T03:05:25.877333+0000 (oldest deadline 2024-09-12T03:05:47.577274+0000) 2024-09-12T03:06:15.166 INFO:journalctl@ceph.osd.8.smithi148.stdout:Sep 12 03:06:14 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-8[53762]: 2024-09-12T03:06:14.763+0000 7f7e0d911640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:31.678293+0000 front 2024-09-12T03:05:31.678404+0000 (oldest deadline 2024-09-12T03:05:56.978470+0000) 2024-09-12T03:06:15.166 INFO:journalctl@ceph.osd.8.smithi148.stdout:Sep 12 03:06:14 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-8[53762]: 2024-09-12T03:06:14.763+0000 7f7e0d911640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-12T03:05:37.479439+0000 front 2024-09-12T03:05:37.479495+0000 (oldest deadline 2024-09-12T03:06:06.879955+0000) 2024-09-12T03:06:15.166 INFO:journalctl@ceph.osd.8.smithi148.stdout:Sep 12 03:06:14 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-8[53762]: 2024-09-12T03:06:14.763+0000 7f7e0d911640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-09-12T03:05:46.880420+0000 front 2024-09-12T03:05:46.880527+0000 (oldest deadline 2024-09-12T03:06:11.580369+0000) 2024-09-12T03:06:15.206 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 12 03:06:14 smithi146 podman[69585]: 2024-09-12 03:06:14.985535328 +0000 UTC m=+5.928394296 container cleanup f735ae846010eefc6b6c98101bc9268552eda6c67b5d5bed6502a0e91c3343c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-7, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-12T03:06:15.206 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 12 03:06:14 smithi146 bash[69585]: ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-7 2024-09-12T03:06:15.207 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:15 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:15.071+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:18.916404+0000 front 2024-09-12T03:05:18.916319+0000 (oldest deadline 2024-09-12T03:05:39.416329+0000) 2024-09-12T03:06:15.207 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:15 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:15.071+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:25.817651+0000 front 2024-09-12T03:05:25.817711+0000 (oldest deadline 2024-09-12T03:05:50.517693+0000) 2024-09-12T03:06:15.207 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:15 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:15.071+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:30.518204+0000 front 2024-09-12T03:05:30.518383+0000 (oldest deadline 2024-09-12T03:05:55.218163+0000) 2024-09-12T03:06:15.207 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:15 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:15.071+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-12T03:05:42.120310+0000 front 2024-09-12T03:05:42.120454+0000 (oldest deadline 2024-09-12T03:06:06.820355+0000) 2024-09-12T03:06:15.207 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:15 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:15.071+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-09-12T03:05:47.921106+0000 front 2024-09-12T03:05:47.921133+0000 (oldest deadline 2024-09-12T03:06:13.821241+0000) 2024-09-12T03:06:15.310 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:15 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:15.005+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:14.571718+0000 front 2024-09-12T03:05:14.571768+0000 (oldest deadline 2024-09-12T03:05:39.772780+0000) 2024-09-12T03:06:15.310 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:15 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:15.005+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:24.374445+0000 front 2024-09-12T03:05:24.374549+0000 (oldest deadline 2024-09-12T03:05:48.374868+0000) 2024-09-12T03:06:15.310 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:15 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:15.005+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:34.276237+0000 front 2024-09-12T03:05:34.276182+0000 (oldest deadline 2024-09-12T03:05:58.976098+0000) 2024-09-12T03:06:15.310 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:15 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:15.005+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-12T03:05:42.377230+0000 front 2024-09-12T03:05:42.377348+0000 (oldest deadline 2024-09-12T03:06:07.677394+0000) 2024-09-12T03:06:15.310 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:15 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:15.005+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-09-12T03:05:47.678051+0000 front 2024-09-12T03:05:47.677984+0000 (oldest deadline 2024-09-12T03:06:12.377935+0000) 2024-09-12T03:06:15.594 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 12 03:06:15 smithi146 podman[69614]: 2024-09-12 03:06:15.253367284 +0000 UTC m=+0.618763272 container remove f735ae846010eefc6b6c98101bc9268552eda6c67b5d5bed6502a0e91c3343c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-7, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD) 2024-09-12T03:06:15.605 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 12 03:06:15 smithi146 podman[69699]: 2024-09-12 03:06:15.49752682 +0000 UTC m=+0.015949997 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:06:15.666 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:15 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:15.289+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:16.844053+0000 front 2024-09-12T03:05:16.843858+0000 (oldest deadline 2024-09-12T03:05:42.644259+0000) 2024-09-12T03:06:15.666 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:15 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:15.289+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:22.644805+0000 front 2024-09-12T03:05:22.644911+0000 (oldest deadline 2024-09-12T03:05:47.344653+0000) 2024-09-12T03:06:15.666 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:15 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:15.289+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:31.846594+0000 front 2024-09-12T03:05:31.846677+0000 (oldest deadline 2024-09-12T03:05:59.447156+0000) 2024-09-12T03:06:15.666 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:15 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:15.289+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-12T03:05:39.447493+0000 front 2024-09-12T03:05:39.447609+0000 (oldest deadline 2024-09-12T03:06:03.547590+0000) 2024-09-12T03:06:15.666 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:15 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:15.289+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-09-12T03:05:48.248794+0000 front 2024-09-12T03:05:48.248779+0000 (oldest deadline 2024-09-12T03:06:12.348647+0000) 2024-09-12T03:06:15.898 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 12 03:06:15 smithi146 podman[69699]: 2024-09-12 03:06:15.604496822 +0000 UTC m=+0.122919991 container create 90d95306793de47e8957fa909a6409cdd7565cd017e5748f1978210bc8b5f570 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-7-deactivate, RELEASE=squid-b53b6d3, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T03:06:15.898 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 12 03:06:15 smithi146 podman[69699]: 2024-09-12 03:06:15.823333005 +0000 UTC m=+0.341756194 container init 90d95306793de47e8957fa909a6409cdd7565cd017e5748f1978210bc8b5f570 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-7-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T03:06:15.898 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 12 03:06:15 smithi146 podman[69699]: 2024-09-12 03:06:15.82660745 +0000 UTC m=+0.345030624 container start 90d95306793de47e8957fa909a6409cdd7565cd017e5748f1978210bc8b5f570 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-7-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-12T03:06:15.898 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 12 03:06:15 smithi146 podman[69699]: 2024-09-12 03:06:15.898175919 +0000 UTC m=+0.416599103 container attach 90d95306793de47e8957fa909a6409cdd7565cd017e5748f1978210bc8b5f570 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-7-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T03:06:16.165 INFO:journalctl@ceph.osd.8.smithi148.stdout:Sep 12 03:06:15 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-8[53762]: 2024-09-12T03:06:15.785+0000 7f7e0d911640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:25.877300+0000 front 2024-09-12T03:05:25.877333+0000 (oldest deadline 2024-09-12T03:05:47.577274+0000) 2024-09-12T03:06:16.165 INFO:journalctl@ceph.osd.8.smithi148.stdout:Sep 12 03:06:15 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-8[53762]: 2024-09-12T03:06:15.785+0000 7f7e0d911640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:31.678293+0000 front 2024-09-12T03:05:31.678404+0000 (oldest deadline 2024-09-12T03:05:56.978470+0000) 2024-09-12T03:06:16.166 INFO:journalctl@ceph.osd.8.smithi148.stdout:Sep 12 03:06:15 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-8[53762]: 2024-09-12T03:06:15.785+0000 7f7e0d911640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-12T03:05:37.479439+0000 front 2024-09-12T03:05:37.479495+0000 (oldest deadline 2024-09-12T03:06:06.879955+0000) 2024-09-12T03:06:16.166 INFO:journalctl@ceph.osd.8.smithi148.stdout:Sep 12 03:06:15 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-8[53762]: 2024-09-12T03:06:15.785+0000 7f7e0d911640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-09-12T03:05:46.880420+0000 front 2024-09-12T03:05:46.880527+0000 (oldest deadline 2024-09-12T03:06:11.580369+0000) 2024-09-12T03:06:16.206 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 12 03:06:16 smithi146 podman[69699]: 2024-09-12 03:06:16.018176519 +0000 UTC m=+0.536599698 container died 90d95306793de47e8957fa909a6409cdd7565cd017e5748f1978210bc8b5f570 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-7-deactivate, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux ) 2024-09-12T03:06:16.206 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:16 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:16.036+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:18.916404+0000 front 2024-09-12T03:05:18.916319+0000 (oldest deadline 2024-09-12T03:05:39.416329+0000) 2024-09-12T03:06:16.207 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:16 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:16.036+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:25.817651+0000 front 2024-09-12T03:05:25.817711+0000 (oldest deadline 2024-09-12T03:05:50.517693+0000) 2024-09-12T03:06:16.207 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:16 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:16.036+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:30.518204+0000 front 2024-09-12T03:05:30.518383+0000 (oldest deadline 2024-09-12T03:05:55.218163+0000) 2024-09-12T03:06:16.207 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:16 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:16.036+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-12T03:05:42.120310+0000 front 2024-09-12T03:05:42.120454+0000 (oldest deadline 2024-09-12T03:06:06.820355+0000) 2024-09-12T03:06:16.207 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:16 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:16.036+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-09-12T03:05:47.921106+0000 front 2024-09-12T03:05:47.921133+0000 (oldest deadline 2024-09-12T03:06:13.821241+0000) 2024-09-12T03:06:16.310 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:16 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:16.014+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:14.571718+0000 front 2024-09-12T03:05:14.571768+0000 (oldest deadline 2024-09-12T03:05:39.772780+0000) 2024-09-12T03:06:16.310 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:16 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:16.014+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:24.374445+0000 front 2024-09-12T03:05:24.374549+0000 (oldest deadline 2024-09-12T03:05:48.374868+0000) 2024-09-12T03:06:16.310 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:16 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:16.014+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:34.276237+0000 front 2024-09-12T03:05:34.276182+0000 (oldest deadline 2024-09-12T03:05:58.976098+0000) 2024-09-12T03:06:16.310 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:16 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:16.014+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-12T03:05:42.377230+0000 front 2024-09-12T03:05:42.377348+0000 (oldest deadline 2024-09-12T03:06:07.677394+0000) 2024-09-12T03:06:16.310 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:16 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:16.014+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-09-12T03:05:47.678051+0000 front 2024-09-12T03:05:47.677984+0000 (oldest deadline 2024-09-12T03:06:12.377935+0000) 2024-09-12T03:06:16.537 INFO:journalctl@ceph.osd.7.smithi146.stdout:Sep 12 03:06:16 smithi146 podman[69722]: 2024-09-12 03:06:16.526208653 +0000 UTC m=+0.501897354 container remove 90d95306793de47e8957fa909a6409cdd7565cd017e5748f1978210bc8b5f570 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-7-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-12T03:06:16.543 DEBUG:teuthology.orchestra.run.smithi146:> sudo pkill -f 'journalctl -f -n 0 -u ceph-618305f8-70b1-11ef-bcea-c7b262605968@osd.7.service' 2024-09-12T03:06:16.582 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:06:16.582 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-12T03:06:16.582 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-09-12T03:06:16.583 DEBUG:teuthology.orchestra.run.smithi148:> sudo systemctl stop ceph-618305f8-70b1-11ef-bcea-c7b262605968@osd.8 2024-09-12T03:06:16.612 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:16 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:16.272+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:16.844053+0000 front 2024-09-12T03:05:16.843858+0000 (oldest deadline 2024-09-12T03:05:42.644259+0000) 2024-09-12T03:06:16.613 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:16 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:16.272+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:22.644805+0000 front 2024-09-12T03:05:22.644911+0000 (oldest deadline 2024-09-12T03:05:47.344653+0000) 2024-09-12T03:06:16.613 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:16 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:16.272+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:31.846594+0000 front 2024-09-12T03:05:31.846677+0000 (oldest deadline 2024-09-12T03:05:59.447156+0000) 2024-09-12T03:06:16.613 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:16 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:16.272+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-12T03:05:39.447493+0000 front 2024-09-12T03:05:39.447609+0000 (oldest deadline 2024-09-12T03:06:03.547590+0000) 2024-09-12T03:06:16.613 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:16 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:16.272+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-09-12T03:05:48.248794+0000 front 2024-09-12T03:05:48.248779+0000 (oldest deadline 2024-09-12T03:06:12.348647+0000) 2024-09-12T03:06:16.877 INFO:journalctl@ceph.osd.8.smithi148.stdout:Sep 12 03:06:16 smithi148 systemd[1]: Stopping Ceph osd.8 for 618305f8-70b1-11ef-bcea-c7b262605968... 2024-09-12T03:06:16.878 INFO:journalctl@ceph.osd.8.smithi148.stdout:Sep 12 03:06:16 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-8[53762]: 2024-09-12T03:06:16.793+0000 7f7e0d911640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:25.877300+0000 front 2024-09-12T03:05:25.877333+0000 (oldest deadline 2024-09-12T03:05:47.577274+0000) 2024-09-12T03:06:16.878 INFO:journalctl@ceph.osd.8.smithi148.stdout:Sep 12 03:06:16 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-8[53762]: 2024-09-12T03:06:16.793+0000 7f7e0d911640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:31.678293+0000 front 2024-09-12T03:05:31.678404+0000 (oldest deadline 2024-09-12T03:05:56.978470+0000) 2024-09-12T03:06:16.878 INFO:journalctl@ceph.osd.8.smithi148.stdout:Sep 12 03:06:16 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-8[53762]: 2024-09-12T03:06:16.793+0000 7f7e0d911640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-12T03:05:37.479439+0000 front 2024-09-12T03:05:37.479495+0000 (oldest deadline 2024-09-12T03:06:06.879955+0000) 2024-09-12T03:06:16.878 INFO:journalctl@ceph.osd.8.smithi148.stdout:Sep 12 03:06:16 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-8[53762]: 2024-09-12T03:06:16.793+0000 7f7e0d911640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-09-12T03:05:46.880420+0000 front 2024-09-12T03:05:46.880527+0000 (oldest deadline 2024-09-12T03:06:11.580369+0000) 2024-09-12T03:06:16.878 INFO:journalctl@ceph.osd.8.smithi148.stdout:Sep 12 03:06:16 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-8[53762]: 2024-09-12T03:06:16.876+0000 7f7e15314640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:06:16.878 INFO:journalctl@ceph.osd.8.smithi148.stdout:Sep 12 03:06:16 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-8[53762]: 2024-09-12T03:06:16.876+0000 7f7e15314640 -1 osd.8 86 *** Got signal Terminated *** 2024-09-12T03:06:16.878 INFO:journalctl@ceph.osd.8.smithi148.stdout:Sep 12 03:06:16 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-8[53762]: 2024-09-12T03:06:16.876+0000 7f7e15314640 -1 osd.8 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T03:06:17.310 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:17 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:17.018+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:14.571718+0000 front 2024-09-12T03:05:14.571768+0000 (oldest deadline 2024-09-12T03:05:39.772780+0000) 2024-09-12T03:06:17.310 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:17 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:17.018+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:24.374445+0000 front 2024-09-12T03:05:24.374549+0000 (oldest deadline 2024-09-12T03:05:48.374868+0000) 2024-09-12T03:06:17.310 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:17 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:17.018+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:34.276237+0000 front 2024-09-12T03:05:34.276182+0000 (oldest deadline 2024-09-12T03:05:58.976098+0000) 2024-09-12T03:06:17.310 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:17 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:17.018+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-12T03:05:42.377230+0000 front 2024-09-12T03:05:42.377348+0000 (oldest deadline 2024-09-12T03:06:07.677394+0000) 2024-09-12T03:06:17.310 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:17 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:17.018+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-09-12T03:05:47.678051+0000 front 2024-09-12T03:05:47.677984+0000 (oldest deadline 2024-09-12T03:06:12.377935+0000) 2024-09-12T03:06:17.457 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:17 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:17.076+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:18.916404+0000 front 2024-09-12T03:05:18.916319+0000 (oldest deadline 2024-09-12T03:05:39.416329+0000) 2024-09-12T03:06:17.457 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:17 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:17.076+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:25.817651+0000 front 2024-09-12T03:05:25.817711+0000 (oldest deadline 2024-09-12T03:05:50.517693+0000) 2024-09-12T03:06:17.457 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:17 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:17.076+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:30.518204+0000 front 2024-09-12T03:05:30.518383+0000 (oldest deadline 2024-09-12T03:05:55.218163+0000) 2024-09-12T03:06:17.457 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:17 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:17.076+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-12T03:05:42.120310+0000 front 2024-09-12T03:05:42.120454+0000 (oldest deadline 2024-09-12T03:06:06.820355+0000) 2024-09-12T03:06:17.457 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:17 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:17.076+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-09-12T03:05:47.921106+0000 front 2024-09-12T03:05:47.921133+0000 (oldest deadline 2024-09-12T03:06:13.821241+0000) 2024-09-12T03:06:17.665 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:17 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:17.293+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:16.844053+0000 front 2024-09-12T03:05:16.843858+0000 (oldest deadline 2024-09-12T03:05:42.644259+0000) 2024-09-12T03:06:17.665 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:17 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:17.293+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:22.644805+0000 front 2024-09-12T03:05:22.644911+0000 (oldest deadline 2024-09-12T03:05:47.344653+0000) 2024-09-12T03:06:17.666 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:17 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:17.293+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:31.846594+0000 front 2024-09-12T03:05:31.846677+0000 (oldest deadline 2024-09-12T03:05:59.447156+0000) 2024-09-12T03:06:17.666 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:17 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:17.293+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-12T03:05:39.447493+0000 front 2024-09-12T03:05:39.447609+0000 (oldest deadline 2024-09-12T03:06:03.547590+0000) 2024-09-12T03:06:17.666 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:17 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:17.293+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-09-12T03:05:48.248794+0000 front 2024-09-12T03:05:48.248779+0000 (oldest deadline 2024-09-12T03:06:12.348647+0000) 2024-09-12T03:06:18.165 INFO:journalctl@ceph.osd.8.smithi148.stdout:Sep 12 03:06:17 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-8[53762]: 2024-09-12T03:06:17.821+0000 7f7e0d911640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:25.877300+0000 front 2024-09-12T03:05:25.877333+0000 (oldest deadline 2024-09-12T03:05:47.577274+0000) 2024-09-12T03:06:18.165 INFO:journalctl@ceph.osd.8.smithi148.stdout:Sep 12 03:06:17 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-8[53762]: 2024-09-12T03:06:17.821+0000 7f7e0d911640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:31.678293+0000 front 2024-09-12T03:05:31.678404+0000 (oldest deadline 2024-09-12T03:05:56.978470+0000) 2024-09-12T03:06:18.166 INFO:journalctl@ceph.osd.8.smithi148.stdout:Sep 12 03:06:17 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-8[53762]: 2024-09-12T03:06:17.821+0000 7f7e0d911640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-12T03:05:37.479439+0000 front 2024-09-12T03:05:37.479495+0000 (oldest deadline 2024-09-12T03:06:06.879955+0000) 2024-09-12T03:06:18.166 INFO:journalctl@ceph.osd.8.smithi148.stdout:Sep 12 03:06:17 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-8[53762]: 2024-09-12T03:06:17.821+0000 7f7e0d911640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-09-12T03:05:46.880420+0000 front 2024-09-12T03:05:46.880527+0000 (oldest deadline 2024-09-12T03:06:11.580369+0000) 2024-09-12T03:06:18.456 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:18 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:18.035+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:18.916404+0000 front 2024-09-12T03:05:18.916319+0000 (oldest deadline 2024-09-12T03:05:39.416329+0000) 2024-09-12T03:06:18.456 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:18 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:18.035+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:25.817651+0000 front 2024-09-12T03:05:25.817711+0000 (oldest deadline 2024-09-12T03:05:50.517693+0000) 2024-09-12T03:06:18.456 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:18 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:18.035+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:30.518204+0000 front 2024-09-12T03:05:30.518383+0000 (oldest deadline 2024-09-12T03:05:55.218163+0000) 2024-09-12T03:06:18.456 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:18 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:18.035+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-12T03:05:42.120310+0000 front 2024-09-12T03:05:42.120454+0000 (oldest deadline 2024-09-12T03:06:06.820355+0000) 2024-09-12T03:06:18.457 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:18 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:18.035+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-09-12T03:05:47.921106+0000 front 2024-09-12T03:05:47.921133+0000 (oldest deadline 2024-09-12T03:06:13.821241+0000) 2024-09-12T03:06:18.560 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:18 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:18.068+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:14.571718+0000 front 2024-09-12T03:05:14.571768+0000 (oldest deadline 2024-09-12T03:05:39.772780+0000) 2024-09-12T03:06:18.560 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:18 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:18.068+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:24.374445+0000 front 2024-09-12T03:05:24.374549+0000 (oldest deadline 2024-09-12T03:05:48.374868+0000) 2024-09-12T03:06:18.560 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:18 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:18.068+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:34.276237+0000 front 2024-09-12T03:05:34.276182+0000 (oldest deadline 2024-09-12T03:05:58.976098+0000) 2024-09-12T03:06:18.560 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:18 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:18.068+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-12T03:05:42.377230+0000 front 2024-09-12T03:05:42.377348+0000 (oldest deadline 2024-09-12T03:06:07.677394+0000) 2024-09-12T03:06:18.560 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:18 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:18.068+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-09-12T03:05:47.678051+0000 front 2024-09-12T03:05:47.677984+0000 (oldest deadline 2024-09-12T03:06:12.377935+0000) 2024-09-12T03:06:18.665 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:18 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:18.245+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:16.844053+0000 front 2024-09-12T03:05:16.843858+0000 (oldest deadline 2024-09-12T03:05:42.644259+0000) 2024-09-12T03:06:18.666 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:18 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:18.245+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:22.644805+0000 front 2024-09-12T03:05:22.644911+0000 (oldest deadline 2024-09-12T03:05:47.344653+0000) 2024-09-12T03:06:18.666 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:18 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:18.245+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:31.846594+0000 front 2024-09-12T03:05:31.846677+0000 (oldest deadline 2024-09-12T03:05:59.447156+0000) 2024-09-12T03:06:18.666 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:18 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:18.245+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-12T03:05:39.447493+0000 front 2024-09-12T03:05:39.447609+0000 (oldest deadline 2024-09-12T03:06:03.547590+0000) 2024-09-12T03:06:18.666 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:18 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:18.245+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-09-12T03:05:48.248794+0000 front 2024-09-12T03:05:48.248779+0000 (oldest deadline 2024-09-12T03:06:12.348647+0000) 2024-09-12T03:06:19.165 INFO:journalctl@ceph.osd.8.smithi148.stdout:Sep 12 03:06:18 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-8[53762]: 2024-09-12T03:06:18.819+0000 7f7e0d911640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:25.877300+0000 front 2024-09-12T03:05:25.877333+0000 (oldest deadline 2024-09-12T03:05:47.577274+0000) 2024-09-12T03:06:19.165 INFO:journalctl@ceph.osd.8.smithi148.stdout:Sep 12 03:06:18 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-8[53762]: 2024-09-12T03:06:18.819+0000 7f7e0d911640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:31.678293+0000 front 2024-09-12T03:05:31.678404+0000 (oldest deadline 2024-09-12T03:05:56.978470+0000) 2024-09-12T03:06:19.166 INFO:journalctl@ceph.osd.8.smithi148.stdout:Sep 12 03:06:18 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-8[53762]: 2024-09-12T03:06:18.819+0000 7f7e0d911640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-12T03:05:37.479439+0000 front 2024-09-12T03:05:37.479495+0000 (oldest deadline 2024-09-12T03:06:06.879955+0000) 2024-09-12T03:06:19.166 INFO:journalctl@ceph.osd.8.smithi148.stdout:Sep 12 03:06:18 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-8[53762]: 2024-09-12T03:06:18.819+0000 7f7e0d911640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-09-12T03:05:46.880420+0000 front 2024-09-12T03:05:46.880527+0000 (oldest deadline 2024-09-12T03:06:11.580369+0000) 2024-09-12T03:06:19.310 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:19 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:19.024+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:14.571718+0000 front 2024-09-12T03:05:14.571768+0000 (oldest deadline 2024-09-12T03:05:39.772780+0000) 2024-09-12T03:06:19.310 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:19 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:19.024+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:24.374445+0000 front 2024-09-12T03:05:24.374549+0000 (oldest deadline 2024-09-12T03:05:48.374868+0000) 2024-09-12T03:06:19.311 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:19 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:19.024+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:34.276237+0000 front 2024-09-12T03:05:34.276182+0000 (oldest deadline 2024-09-12T03:05:58.976098+0000) 2024-09-12T03:06:19.311 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:19 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:19.024+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-12T03:05:42.377230+0000 front 2024-09-12T03:05:42.377348+0000 (oldest deadline 2024-09-12T03:06:07.677394+0000) 2024-09-12T03:06:19.311 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:19 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:19.024+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-09-12T03:05:47.678051+0000 front 2024-09-12T03:05:47.677984+0000 (oldest deadline 2024-09-12T03:06:12.377935+0000) 2024-09-12T03:06:19.456 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:19 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:19.023+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:18.916404+0000 front 2024-09-12T03:05:18.916319+0000 (oldest deadline 2024-09-12T03:05:39.416329+0000) 2024-09-12T03:06:19.457 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:19 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:19.023+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:25.817651+0000 front 2024-09-12T03:05:25.817711+0000 (oldest deadline 2024-09-12T03:05:50.517693+0000) 2024-09-12T03:06:19.457 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:19 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:19.023+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:30.518204+0000 front 2024-09-12T03:05:30.518383+0000 (oldest deadline 2024-09-12T03:05:55.218163+0000) 2024-09-12T03:06:19.457 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:19 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:19.023+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-12T03:05:42.120310+0000 front 2024-09-12T03:05:42.120454+0000 (oldest deadline 2024-09-12T03:06:06.820355+0000) 2024-09-12T03:06:19.457 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:19 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:19.023+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-09-12T03:05:47.921106+0000 front 2024-09-12T03:05:47.921133+0000 (oldest deadline 2024-09-12T03:06:13.821241+0000) 2024-09-12T03:06:19.665 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:19 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:19.229+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:16.844053+0000 front 2024-09-12T03:05:16.843858+0000 (oldest deadline 2024-09-12T03:05:42.644259+0000) 2024-09-12T03:06:19.666 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:19 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:19.229+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:22.644805+0000 front 2024-09-12T03:05:22.644911+0000 (oldest deadline 2024-09-12T03:05:47.344653+0000) 2024-09-12T03:06:19.666 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:19 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:19.229+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:31.846594+0000 front 2024-09-12T03:05:31.846677+0000 (oldest deadline 2024-09-12T03:05:59.447156+0000) 2024-09-12T03:06:19.666 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:19 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:19.229+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-12T03:05:39.447493+0000 front 2024-09-12T03:05:39.447609+0000 (oldest deadline 2024-09-12T03:06:03.547590+0000) 2024-09-12T03:06:19.666 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:19 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:19.229+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-09-12T03:05:48.248794+0000 front 2024-09-12T03:05:48.248779+0000 (oldest deadline 2024-09-12T03:06:12.348647+0000) 2024-09-12T03:06:20.166 INFO:journalctl@ceph.osd.8.smithi148.stdout:Sep 12 03:06:19 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-8[53762]: 2024-09-12T03:06:19.797+0000 7f7e0d911640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:25.877300+0000 front 2024-09-12T03:05:25.877333+0000 (oldest deadline 2024-09-12T03:05:47.577274+0000) 2024-09-12T03:06:20.166 INFO:journalctl@ceph.osd.8.smithi148.stdout:Sep 12 03:06:19 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-8[53762]: 2024-09-12T03:06:19.797+0000 7f7e0d911640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:31.678293+0000 front 2024-09-12T03:05:31.678404+0000 (oldest deadline 2024-09-12T03:05:56.978470+0000) 2024-09-12T03:06:20.166 INFO:journalctl@ceph.osd.8.smithi148.stdout:Sep 12 03:06:19 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-8[53762]: 2024-09-12T03:06:19.797+0000 7f7e0d911640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-12T03:05:37.479439+0000 front 2024-09-12T03:05:37.479495+0000 (oldest deadline 2024-09-12T03:06:06.879955+0000) 2024-09-12T03:06:20.166 INFO:journalctl@ceph.osd.8.smithi148.stdout:Sep 12 03:06:19 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-8[53762]: 2024-09-12T03:06:19.797+0000 7f7e0d911640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-09-12T03:05:46.880420+0000 front 2024-09-12T03:05:46.880527+0000 (oldest deadline 2024-09-12T03:06:11.580369+0000) 2024-09-12T03:06:20.166 INFO:journalctl@ceph.osd.8.smithi148.stdout:Sep 12 03:06:19 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-8[53762]: 2024-09-12T03:06:19.797+0000 7f7e0d911640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.148:6812 osd.5 since back 2024-09-12T03:05:54.481240+0000 front 2024-09-12T03:05:54.481310+0000 (oldest deadline 2024-09-12T03:06:19.781465+0000) 2024-09-12T03:06:20.310 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:20 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:20.024+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:14.571718+0000 front 2024-09-12T03:05:14.571768+0000 (oldest deadline 2024-09-12T03:05:39.772780+0000) 2024-09-12T03:06:20.310 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:20 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:20.024+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:24.374445+0000 front 2024-09-12T03:05:24.374549+0000 (oldest deadline 2024-09-12T03:05:48.374868+0000) 2024-09-12T03:06:20.310 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:20 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:20.024+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:34.276237+0000 front 2024-09-12T03:05:34.276182+0000 (oldest deadline 2024-09-12T03:05:58.976098+0000) 2024-09-12T03:06:20.310 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:20 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:20.024+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-12T03:05:42.377230+0000 front 2024-09-12T03:05:42.377348+0000 (oldest deadline 2024-09-12T03:06:07.677394+0000) 2024-09-12T03:06:20.310 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:20 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:20.024+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-09-12T03:05:47.678051+0000 front 2024-09-12T03:05:47.677984+0000 (oldest deadline 2024-09-12T03:06:12.377935+0000) 2024-09-12T03:06:20.456 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:20 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:20.022+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:18.916404+0000 front 2024-09-12T03:05:18.916319+0000 (oldest deadline 2024-09-12T03:05:39.416329+0000) 2024-09-12T03:06:20.456 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:20 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:20.022+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:25.817651+0000 front 2024-09-12T03:05:25.817711+0000 (oldest deadline 2024-09-12T03:05:50.517693+0000) 2024-09-12T03:06:20.457 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:20 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:20.022+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:30.518204+0000 front 2024-09-12T03:05:30.518383+0000 (oldest deadline 2024-09-12T03:05:55.218163+0000) 2024-09-12T03:06:20.457 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:20 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:20.022+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-12T03:05:42.120310+0000 front 2024-09-12T03:05:42.120454+0000 (oldest deadline 2024-09-12T03:06:06.820355+0000) 2024-09-12T03:06:20.457 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:20 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:20.022+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-09-12T03:05:47.921106+0000 front 2024-09-12T03:05:47.921133+0000 (oldest deadline 2024-09-12T03:06:13.821241+0000) 2024-09-12T03:06:20.665 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:20 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:20.269+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:16.844053+0000 front 2024-09-12T03:05:16.843858+0000 (oldest deadline 2024-09-12T03:05:42.644259+0000) 2024-09-12T03:06:20.666 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:20 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:20.269+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:22.644805+0000 front 2024-09-12T03:05:22.644911+0000 (oldest deadline 2024-09-12T03:05:47.344653+0000) 2024-09-12T03:06:20.666 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:20 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:20.269+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:31.846594+0000 front 2024-09-12T03:05:31.846677+0000 (oldest deadline 2024-09-12T03:05:59.447156+0000) 2024-09-12T03:06:20.666 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:20 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:20.269+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-12T03:05:39.447493+0000 front 2024-09-12T03:05:39.447609+0000 (oldest deadline 2024-09-12T03:06:03.547590+0000) 2024-09-12T03:06:20.666 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:20 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:20.269+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-09-12T03:05:48.248794+0000 front 2024-09-12T03:05:48.248779+0000 (oldest deadline 2024-09-12T03:06:12.348647+0000) 2024-09-12T03:06:21.165 INFO:journalctl@ceph.osd.8.smithi148.stdout:Sep 12 03:06:20 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-8[53762]: 2024-09-12T03:06:20.789+0000 7f7e0d911640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:25.877300+0000 front 2024-09-12T03:05:25.877333+0000 (oldest deadline 2024-09-12T03:05:47.577274+0000) 2024-09-12T03:06:21.166 INFO:journalctl@ceph.osd.8.smithi148.stdout:Sep 12 03:06:20 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-8[53762]: 2024-09-12T03:06:20.789+0000 7f7e0d911640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:31.678293+0000 front 2024-09-12T03:05:31.678404+0000 (oldest deadline 2024-09-12T03:05:56.978470+0000) 2024-09-12T03:06:21.166 INFO:journalctl@ceph.osd.8.smithi148.stdout:Sep 12 03:06:20 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-8[53762]: 2024-09-12T03:06:20.789+0000 7f7e0d911640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-12T03:05:37.479439+0000 front 2024-09-12T03:05:37.479495+0000 (oldest deadline 2024-09-12T03:06:06.879955+0000) 2024-09-12T03:06:21.166 INFO:journalctl@ceph.osd.8.smithi148.stdout:Sep 12 03:06:20 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-8[53762]: 2024-09-12T03:06:20.789+0000 7f7e0d911640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-09-12T03:05:46.880420+0000 front 2024-09-12T03:05:46.880527+0000 (oldest deadline 2024-09-12T03:06:11.580369+0000) 2024-09-12T03:06:21.166 INFO:journalctl@ceph.osd.8.smithi148.stdout:Sep 12 03:06:20 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-8[53762]: 2024-09-12T03:06:20.789+0000 7f7e0d911640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.148:6812 osd.5 since back 2024-09-12T03:05:54.481240+0000 front 2024-09-12T03:05:54.481310+0000 (oldest deadline 2024-09-12T03:06:19.781465+0000) 2024-09-12T03:06:21.310 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:21 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:21.019+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:14.571718+0000 front 2024-09-12T03:05:14.571768+0000 (oldest deadline 2024-09-12T03:05:39.772780+0000) 2024-09-12T03:06:21.310 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:21 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:21.019+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:24.374445+0000 front 2024-09-12T03:05:24.374549+0000 (oldest deadline 2024-09-12T03:05:48.374868+0000) 2024-09-12T03:06:21.310 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:21 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:21.019+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:34.276237+0000 front 2024-09-12T03:05:34.276182+0000 (oldest deadline 2024-09-12T03:05:58.976098+0000) 2024-09-12T03:06:21.310 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:21 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:21.019+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-12T03:05:42.377230+0000 front 2024-09-12T03:05:42.377348+0000 (oldest deadline 2024-09-12T03:06:07.677394+0000) 2024-09-12T03:06:21.310 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:21 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:21.019+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-09-12T03:05:47.678051+0000 front 2024-09-12T03:05:47.677984+0000 (oldest deadline 2024-09-12T03:06:12.377935+0000) 2024-09-12T03:06:21.456 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:20 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:20.976+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:18.916404+0000 front 2024-09-12T03:05:18.916319+0000 (oldest deadline 2024-09-12T03:05:39.416329+0000) 2024-09-12T03:06:21.456 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:20 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:20.976+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:25.817651+0000 front 2024-09-12T03:05:25.817711+0000 (oldest deadline 2024-09-12T03:05:50.517693+0000) 2024-09-12T03:06:21.456 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:20 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:20.976+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:30.518204+0000 front 2024-09-12T03:05:30.518383+0000 (oldest deadline 2024-09-12T03:05:55.218163+0000) 2024-09-12T03:06:21.456 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:20 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:20.976+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-12T03:05:42.120310+0000 front 2024-09-12T03:05:42.120454+0000 (oldest deadline 2024-09-12T03:06:06.820355+0000) 2024-09-12T03:06:21.457 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:20 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:20.976+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-09-12T03:05:47.921106+0000 front 2024-09-12T03:05:47.921133+0000 (oldest deadline 2024-09-12T03:06:13.821241+0000) 2024-09-12T03:06:21.665 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:21 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:21.280+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:16.844053+0000 front 2024-09-12T03:05:16.843858+0000 (oldest deadline 2024-09-12T03:05:42.644259+0000) 2024-09-12T03:06:21.666 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:21 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:21.280+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:22.644805+0000 front 2024-09-12T03:05:22.644911+0000 (oldest deadline 2024-09-12T03:05:47.344653+0000) 2024-09-12T03:06:21.666 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:21 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:21.280+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:31.846594+0000 front 2024-09-12T03:05:31.846677+0000 (oldest deadline 2024-09-12T03:05:59.447156+0000) 2024-09-12T03:06:21.666 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:21 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:21.280+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-12T03:05:39.447493+0000 front 2024-09-12T03:05:39.447609+0000 (oldest deadline 2024-09-12T03:06:03.547590+0000) 2024-09-12T03:06:21.666 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:21 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:21.280+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-09-12T03:05:48.248794+0000 front 2024-09-12T03:05:48.248779+0000 (oldest deadline 2024-09-12T03:06:12.348647+0000) 2024-09-12T03:06:21.666 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:21 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:21.280+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.148:6812 osd.5 since back 2024-09-12T03:05:55.849475+0000 front 2024-09-12T03:05:55.849548+0000 (oldest deadline 2024-09-12T03:06:21.149634+0000) 2024-09-12T03:06:22.165 INFO:journalctl@ceph.osd.8.smithi148.stdout:Sep 12 03:06:21 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-8[53762]: 2024-09-12T03:06:21.836+0000 7f7e0d911640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:25.877300+0000 front 2024-09-12T03:05:25.877333+0000 (oldest deadline 2024-09-12T03:05:47.577274+0000) 2024-09-12T03:06:22.165 INFO:journalctl@ceph.osd.8.smithi148.stdout:Sep 12 03:06:21 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-8[53762]: 2024-09-12T03:06:21.836+0000 7f7e0d911640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:31.678293+0000 front 2024-09-12T03:05:31.678404+0000 (oldest deadline 2024-09-12T03:05:56.978470+0000) 2024-09-12T03:06:22.166 INFO:journalctl@ceph.osd.8.smithi148.stdout:Sep 12 03:06:21 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-8[53762]: 2024-09-12T03:06:21.836+0000 7f7e0d911640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-12T03:05:37.479439+0000 front 2024-09-12T03:05:37.479495+0000 (oldest deadline 2024-09-12T03:06:06.879955+0000) 2024-09-12T03:06:22.166 INFO:journalctl@ceph.osd.8.smithi148.stdout:Sep 12 03:06:21 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-8[53762]: 2024-09-12T03:06:21.836+0000 7f7e0d911640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-09-12T03:05:46.880420+0000 front 2024-09-12T03:05:46.880527+0000 (oldest deadline 2024-09-12T03:06:11.580369+0000) 2024-09-12T03:06:22.166 INFO:journalctl@ceph.osd.8.smithi148.stdout:Sep 12 03:06:21 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-8[53762]: 2024-09-12T03:06:21.836+0000 7f7e0d911640 -1 osd.8 86 heartbeat_check: no reply from 172.21.15.148:6812 osd.5 since back 2024-09-12T03:05:54.481240+0000 front 2024-09-12T03:05:54.481310+0000 (oldest deadline 2024-09-12T03:06:19.781465+0000) 2024-09-12T03:06:22.310 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:22 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:22.046+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:14.571718+0000 front 2024-09-12T03:05:14.571768+0000 (oldest deadline 2024-09-12T03:05:39.772780+0000) 2024-09-12T03:06:22.310 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:22 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:22.046+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:24.374445+0000 front 2024-09-12T03:05:24.374549+0000 (oldest deadline 2024-09-12T03:05:48.374868+0000) 2024-09-12T03:06:22.310 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:22 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:22.046+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:34.276237+0000 front 2024-09-12T03:05:34.276182+0000 (oldest deadline 2024-09-12T03:05:58.976098+0000) 2024-09-12T03:06:22.310 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:22 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:22.046+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-12T03:05:42.377230+0000 front 2024-09-12T03:05:42.377348+0000 (oldest deadline 2024-09-12T03:06:07.677394+0000) 2024-09-12T03:06:22.310 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:22 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:22.046+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-09-12T03:05:47.678051+0000 front 2024-09-12T03:05:47.677984+0000 (oldest deadline 2024-09-12T03:06:12.377935+0000) 2024-09-12T03:06:22.456 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:21 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:21.997+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:18.916404+0000 front 2024-09-12T03:05:18.916319+0000 (oldest deadline 2024-09-12T03:05:39.416329+0000) 2024-09-12T03:06:22.456 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:21 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:21.997+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:25.817651+0000 front 2024-09-12T03:05:25.817711+0000 (oldest deadline 2024-09-12T03:05:50.517693+0000) 2024-09-12T03:06:22.456 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:21 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:21.997+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:30.518204+0000 front 2024-09-12T03:05:30.518383+0000 (oldest deadline 2024-09-12T03:05:55.218163+0000) 2024-09-12T03:06:22.456 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:21 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:21.997+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-12T03:05:42.120310+0000 front 2024-09-12T03:05:42.120454+0000 (oldest deadline 2024-09-12T03:06:06.820355+0000) 2024-09-12T03:06:22.456 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:21 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:21.997+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-09-12T03:05:47.921106+0000 front 2024-09-12T03:05:47.921133+0000 (oldest deadline 2024-09-12T03:06:13.821241+0000) 2024-09-12T03:06:22.628 INFO:journalctl@ceph.osd.8.smithi148.stdout:Sep 12 03:06:22 smithi148 podman[69594]: 2024-09-12 03:06:22.360348378 +0000 UTC m=+5.576534600 container died f89cd9d61bd216076a9dfbc7202777ea2561e0f42be11085d05dfd405175684c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-8, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, RELEASE=squid-b53b6d3, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T03:06:22.628 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:22 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:22.310+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:16.844053+0000 front 2024-09-12T03:05:16.843858+0000 (oldest deadline 2024-09-12T03:05:42.644259+0000) 2024-09-12T03:06:22.628 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:22 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:22.310+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:22.644805+0000 front 2024-09-12T03:05:22.644911+0000 (oldest deadline 2024-09-12T03:05:47.344653+0000) 2024-09-12T03:06:22.628 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:22 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:22.310+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:31.846594+0000 front 2024-09-12T03:05:31.846677+0000 (oldest deadline 2024-09-12T03:05:59.447156+0000) 2024-09-12T03:06:22.629 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:22 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:22.310+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-12T03:05:39.447493+0000 front 2024-09-12T03:05:39.447609+0000 (oldest deadline 2024-09-12T03:06:03.547590+0000) 2024-09-12T03:06:22.629 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:22 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:22.310+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-09-12T03:05:48.248794+0000 front 2024-09-12T03:05:48.248779+0000 (oldest deadline 2024-09-12T03:06:12.348647+0000) 2024-09-12T03:06:22.629 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:22 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:22.310+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.148:6812 osd.5 since back 2024-09-12T03:05:55.849475+0000 front 2024-09-12T03:05:55.849548+0000 (oldest deadline 2024-09-12T03:06:21.149634+0000) 2024-09-12T03:06:22.916 INFO:journalctl@ceph.osd.8.smithi148.stdout:Sep 12 03:06:22 smithi148 podman[69594]: 2024-09-12 03:06:22.71879062 +0000 UTC m=+5.934976849 container cleanup f89cd9d61bd216076a9dfbc7202777ea2561e0f42be11085d05dfd405175684c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-8, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-12T03:06:22.916 INFO:journalctl@ceph.osd.8.smithi148.stdout:Sep 12 03:06:22 smithi148 bash[69594]: ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-8 2024-09-12T03:06:23.310 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:23 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:23.001+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:14.571718+0000 front 2024-09-12T03:05:14.571768+0000 (oldest deadline 2024-09-12T03:05:39.772780+0000) 2024-09-12T03:06:23.310 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:23 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:23.001+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:24.374445+0000 front 2024-09-12T03:05:24.374549+0000 (oldest deadline 2024-09-12T03:05:48.374868+0000) 2024-09-12T03:06:23.310 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:23 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:23.001+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:34.276237+0000 front 2024-09-12T03:05:34.276182+0000 (oldest deadline 2024-09-12T03:05:58.976098+0000) 2024-09-12T03:06:23.310 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:23 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:23.001+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-12T03:05:42.377230+0000 front 2024-09-12T03:05:42.377348+0000 (oldest deadline 2024-09-12T03:06:07.677394+0000) 2024-09-12T03:06:23.310 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:23 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:23.001+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-09-12T03:05:47.678051+0000 front 2024-09-12T03:05:47.677984+0000 (oldest deadline 2024-09-12T03:06:12.377935+0000) 2024-09-12T03:06:23.310 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:23 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:23.001+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.148:6812 osd.5 since back 2024-09-12T03:05:58.179806+0000 front 2024-09-12T03:05:58.179817+0000 (oldest deadline 2024-09-12T03:06:22.279613+0000) 2024-09-12T03:06:23.331 INFO:journalctl@ceph.osd.8.smithi148.stdout:Sep 12 03:06:23 smithi148 podman[69609]: 2024-09-12 03:06:23.00321132 +0000 UTC m=+0.642424881 container remove f89cd9d61bd216076a9dfbc7202777ea2561e0f42be11085d05dfd405175684c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, GIT_CLEAN=True, ceph=True) 2024-09-12T03:06:23.456 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:22 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:22.984+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:18.916404+0000 front 2024-09-12T03:05:18.916319+0000 (oldest deadline 2024-09-12T03:05:39.416329+0000) 2024-09-12T03:06:23.456 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:22 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:22.984+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:25.817651+0000 front 2024-09-12T03:05:25.817711+0000 (oldest deadline 2024-09-12T03:05:50.517693+0000) 2024-09-12T03:06:23.456 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:22 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:22.984+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:30.518204+0000 front 2024-09-12T03:05:30.518383+0000 (oldest deadline 2024-09-12T03:05:55.218163+0000) 2024-09-12T03:06:23.457 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:22 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:22.984+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-12T03:05:42.120310+0000 front 2024-09-12T03:05:42.120454+0000 (oldest deadline 2024-09-12T03:06:06.820355+0000) 2024-09-12T03:06:23.457 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:22 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:22.984+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-09-12T03:05:47.921106+0000 front 2024-09-12T03:05:47.921133+0000 (oldest deadline 2024-09-12T03:06:13.821241+0000) 2024-09-12T03:06:23.665 INFO:journalctl@ceph.osd.8.smithi148.stdout:Sep 12 03:06:23 smithi148 podman[69696]: 2024-09-12 03:06:23.234994439 +0000 UTC m=+0.015974009 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:06:23.666 INFO:journalctl@ceph.osd.8.smithi148.stdout:Sep 12 03:06:23 smithi148 podman[69696]: 2024-09-12 03:06:23.353896093 +0000 UTC m=+0.134875705 container create d2576378512d7b77361f4287abca03e3e550a6b99dbd4c017ca45b70c4341fd7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-8-deactivate, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-12T03:06:23.666 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:23 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:23.342+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:16.844053+0000 front 2024-09-12T03:05:16.843858+0000 (oldest deadline 2024-09-12T03:05:42.644259+0000) 2024-09-12T03:06:23.666 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:23 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:23.342+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:22.644805+0000 front 2024-09-12T03:05:22.644911+0000 (oldest deadline 2024-09-12T03:05:47.344653+0000) 2024-09-12T03:06:23.666 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:23 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:23.342+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:31.846594+0000 front 2024-09-12T03:05:31.846677+0000 (oldest deadline 2024-09-12T03:05:59.447156+0000) 2024-09-12T03:06:23.666 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:23 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:23.342+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-12T03:05:39.447493+0000 front 2024-09-12T03:05:39.447609+0000 (oldest deadline 2024-09-12T03:06:03.547590+0000) 2024-09-12T03:06:23.666 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:23 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:23.342+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-09-12T03:05:48.248794+0000 front 2024-09-12T03:05:48.248779+0000 (oldest deadline 2024-09-12T03:06:12.348647+0000) 2024-09-12T03:06:23.666 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:23 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:23.342+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.148:6812 osd.5 since back 2024-09-12T03:05:55.849475+0000 front 2024-09-12T03:05:55.849548+0000 (oldest deadline 2024-09-12T03:06:21.149634+0000) 2024-09-12T03:06:24.165 INFO:journalctl@ceph.osd.8.smithi148.stdout:Sep 12 03:06:23 smithi148 podman[69696]: 2024-09-12 03:06:23.683934186 +0000 UTC m=+0.464913756 container init d2576378512d7b77361f4287abca03e3e550a6b99dbd4c017ca45b70c4341fd7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-8-deactivate, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-12T03:06:24.166 INFO:journalctl@ceph.osd.8.smithi148.stdout:Sep 12 03:06:23 smithi148 podman[69696]: 2024-09-12 03:06:23.689470544 +0000 UTC m=+0.470450122 container start d2576378512d7b77361f4287abca03e3e550a6b99dbd4c017ca45b70c4341fd7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-8-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3) 2024-09-12T03:06:24.166 INFO:journalctl@ceph.osd.8.smithi148.stdout:Sep 12 03:06:23 smithi148 podman[69696]: 2024-09-12 03:06:23.91533176 +0000 UTC m=+0.696311343 container attach d2576378512d7b77361f4287abca03e3e550a6b99dbd4c017ca45b70c4341fd7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-8-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, ceph=True) 2024-09-12T03:06:24.166 INFO:journalctl@ceph.osd.8.smithi148.stdout:Sep 12 03:06:23 smithi148 podman[69696]: 2024-09-12 03:06:23.915548893 +0000 UTC m=+0.696528464 container died d2576378512d7b77361f4287abca03e3e550a6b99dbd4c017ca45b70c4341fd7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-8-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True) 2024-09-12T03:06:24.206 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:23 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:23.941+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:18.916404+0000 front 2024-09-12T03:05:18.916319+0000 (oldest deadline 2024-09-12T03:05:39.416329+0000) 2024-09-12T03:06:24.206 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:23 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:23.941+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:25.817651+0000 front 2024-09-12T03:05:25.817711+0000 (oldest deadline 2024-09-12T03:05:50.517693+0000) 2024-09-12T03:06:24.206 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:23 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:23.941+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:30.518204+0000 front 2024-09-12T03:05:30.518383+0000 (oldest deadline 2024-09-12T03:05:55.218163+0000) 2024-09-12T03:06:24.207 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:23 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:23.941+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-12T03:05:42.120310+0000 front 2024-09-12T03:05:42.120454+0000 (oldest deadline 2024-09-12T03:06:06.820355+0000) 2024-09-12T03:06:24.207 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:23 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:23.941+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-09-12T03:05:47.921106+0000 front 2024-09-12T03:05:47.921133+0000 (oldest deadline 2024-09-12T03:06:13.821241+0000) 2024-09-12T03:06:24.207 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:23 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:23.941+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.148:6812 osd.5 since back 2024-09-12T03:05:54.322143+0000 front 2024-09-12T03:05:54.322202+0000 (oldest deadline 2024-09-12T03:06:23.122455+0000) 2024-09-12T03:06:24.310 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:24 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:24.009+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:14.571718+0000 front 2024-09-12T03:05:14.571768+0000 (oldest deadline 2024-09-12T03:05:39.772780+0000) 2024-09-12T03:06:24.310 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:24 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:24.009+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:24.374445+0000 front 2024-09-12T03:05:24.374549+0000 (oldest deadline 2024-09-12T03:05:48.374868+0000) 2024-09-12T03:06:24.310 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:24 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:24.009+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:34.276237+0000 front 2024-09-12T03:05:34.276182+0000 (oldest deadline 2024-09-12T03:05:58.976098+0000) 2024-09-12T03:06:24.310 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:24 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:24.009+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-12T03:05:42.377230+0000 front 2024-09-12T03:05:42.377348+0000 (oldest deadline 2024-09-12T03:06:07.677394+0000) 2024-09-12T03:06:24.310 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:24 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:24.009+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-09-12T03:05:47.678051+0000 front 2024-09-12T03:05:47.677984+0000 (oldest deadline 2024-09-12T03:06:12.377935+0000) 2024-09-12T03:06:24.310 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:24 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:24.009+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.148:6812 osd.5 since back 2024-09-12T03:05:58.179806+0000 front 2024-09-12T03:05:58.179817+0000 (oldest deadline 2024-09-12T03:06:22.279613+0000) 2024-09-12T03:06:24.446 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:24 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:24.359+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:16.844053+0000 front 2024-09-12T03:05:16.843858+0000 (oldest deadline 2024-09-12T03:05:42.644259+0000) 2024-09-12T03:06:24.446 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:24 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:24.359+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:22.644805+0000 front 2024-09-12T03:05:22.644911+0000 (oldest deadline 2024-09-12T03:05:47.344653+0000) 2024-09-12T03:06:24.446 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:24 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:24.359+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:31.846594+0000 front 2024-09-12T03:05:31.846677+0000 (oldest deadline 2024-09-12T03:05:59.447156+0000) 2024-09-12T03:06:24.446 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:24 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:24.359+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-12T03:05:39.447493+0000 front 2024-09-12T03:05:39.447609+0000 (oldest deadline 2024-09-12T03:06:03.547590+0000) 2024-09-12T03:06:24.446 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:24 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:24.359+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-09-12T03:05:48.248794+0000 front 2024-09-12T03:05:48.248779+0000 (oldest deadline 2024-09-12T03:06:12.348647+0000) 2024-09-12T03:06:24.446 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:24 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:24.359+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.148:6812 osd.5 since back 2024-09-12T03:05:55.849475+0000 front 2024-09-12T03:05:55.849548+0000 (oldest deadline 2024-09-12T03:06:21.149634+0000) 2024-09-12T03:06:24.462 DEBUG:teuthology.orchestra.run.smithi148:> sudo pkill -f 'journalctl -f -n 0 -u ceph-618305f8-70b1-11ef-bcea-c7b262605968@osd.8.service' 2024-09-12T03:06:24.541 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:06:24.541 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-09-12T03:06:24.541 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-09-12T03:06:24.541 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ceph-618305f8-70b1-11ef-bcea-c7b262605968@osd.9 2024-09-12T03:06:24.835 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:24 smithi005 systemd[1]: Stopping Ceph osd.9 for 618305f8-70b1-11ef-bcea-c7b262605968... 2024-09-12T03:06:25.206 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:24 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:24.895+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:18.916404+0000 front 2024-09-12T03:05:18.916319+0000 (oldest deadline 2024-09-12T03:05:39.416329+0000) 2024-09-12T03:06:25.206 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:24 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:24.895+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:25.817651+0000 front 2024-09-12T03:05:25.817711+0000 (oldest deadline 2024-09-12T03:05:50.517693+0000) 2024-09-12T03:06:25.207 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:24 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:24.895+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:30.518204+0000 front 2024-09-12T03:05:30.518383+0000 (oldest deadline 2024-09-12T03:05:55.218163+0000) 2024-09-12T03:06:25.207 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:24 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:24.895+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-12T03:05:42.120310+0000 front 2024-09-12T03:05:42.120454+0000 (oldest deadline 2024-09-12T03:06:06.820355+0000) 2024-09-12T03:06:25.207 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:24 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:24.895+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-09-12T03:05:47.921106+0000 front 2024-09-12T03:05:47.921133+0000 (oldest deadline 2024-09-12T03:06:13.821241+0000) 2024-09-12T03:06:25.207 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:24 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:24.895+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.148:6812 osd.5 since back 2024-09-12T03:05:54.322143+0000 front 2024-09-12T03:05:54.322202+0000 (oldest deadline 2024-09-12T03:06:23.122455+0000) 2024-09-12T03:06:25.310 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:24 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:24.834+0000 7fe44e8c3640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:06:25.310 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:24 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:24.834+0000 7fe44e8c3640 -1 osd.9 86 *** Got signal Terminated *** 2024-09-12T03:06:25.310 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:24 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:24.834+0000 7fe44e8c3640 -1 osd.9 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T03:06:25.310 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:25 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:25.045+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:14.571718+0000 front 2024-09-12T03:05:14.571768+0000 (oldest deadline 2024-09-12T03:05:39.772780+0000) 2024-09-12T03:06:25.310 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:25 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:25.045+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:24.374445+0000 front 2024-09-12T03:05:24.374549+0000 (oldest deadline 2024-09-12T03:05:48.374868+0000) 2024-09-12T03:06:25.311 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:25 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:25.045+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:34.276237+0000 front 2024-09-12T03:05:34.276182+0000 (oldest deadline 2024-09-12T03:05:58.976098+0000) 2024-09-12T03:06:25.311 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:25 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:25.045+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-12T03:05:42.377230+0000 front 2024-09-12T03:05:42.377348+0000 (oldest deadline 2024-09-12T03:06:07.677394+0000) 2024-09-12T03:06:25.311 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:25 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:25.045+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-09-12T03:05:47.678051+0000 front 2024-09-12T03:05:47.677984+0000 (oldest deadline 2024-09-12T03:06:12.377935+0000) 2024-09-12T03:06:25.311 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:25 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:25.045+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.148:6812 osd.5 since back 2024-09-12T03:05:58.179806+0000 front 2024-09-12T03:05:58.179817+0000 (oldest deadline 2024-09-12T03:06:22.279613+0000) 2024-09-12T03:06:25.648 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:25 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:25.335+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:16.844053+0000 front 2024-09-12T03:05:16.843858+0000 (oldest deadline 2024-09-12T03:05:42.644259+0000) 2024-09-12T03:06:25.648 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:25 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:25.335+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:22.644805+0000 front 2024-09-12T03:05:22.644911+0000 (oldest deadline 2024-09-12T03:05:47.344653+0000) 2024-09-12T03:06:25.649 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:25 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:25.335+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:31.846594+0000 front 2024-09-12T03:05:31.846677+0000 (oldest deadline 2024-09-12T03:05:59.447156+0000) 2024-09-12T03:06:25.649 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:25 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:25.335+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-12T03:05:39.447493+0000 front 2024-09-12T03:05:39.447609+0000 (oldest deadline 2024-09-12T03:06:03.547590+0000) 2024-09-12T03:06:25.649 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:25 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:25.335+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-09-12T03:05:48.248794+0000 front 2024-09-12T03:05:48.248779+0000 (oldest deadline 2024-09-12T03:06:12.348647+0000) 2024-09-12T03:06:25.649 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:25 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:25.335+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.148:6812 osd.5 since back 2024-09-12T03:05:55.849475+0000 front 2024-09-12T03:05:55.849548+0000 (oldest deadline 2024-09-12T03:06:21.149634+0000) 2024-09-12T03:06:26.206 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:25 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:25.945+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:18.916404+0000 front 2024-09-12T03:05:18.916319+0000 (oldest deadline 2024-09-12T03:05:39.416329+0000) 2024-09-12T03:06:26.206 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:25 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:25.945+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:25.817651+0000 front 2024-09-12T03:05:25.817711+0000 (oldest deadline 2024-09-12T03:05:50.517693+0000) 2024-09-12T03:06:26.206 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:25 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:25.945+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:30.518204+0000 front 2024-09-12T03:05:30.518383+0000 (oldest deadline 2024-09-12T03:05:55.218163+0000) 2024-09-12T03:06:26.207 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:25 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:25.945+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-12T03:05:42.120310+0000 front 2024-09-12T03:05:42.120454+0000 (oldest deadline 2024-09-12T03:06:06.820355+0000) 2024-09-12T03:06:26.207 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:25 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:25.945+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-09-12T03:05:47.921106+0000 front 2024-09-12T03:05:47.921133+0000 (oldest deadline 2024-09-12T03:06:13.821241+0000) 2024-09-12T03:06:26.207 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:25 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:25.945+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.148:6812 osd.5 since back 2024-09-12T03:05:54.322143+0000 front 2024-09-12T03:05:54.322202+0000 (oldest deadline 2024-09-12T03:06:23.122455+0000) 2024-09-12T03:06:26.310 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:26 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:26.004+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:14.571718+0000 front 2024-09-12T03:05:14.571768+0000 (oldest deadline 2024-09-12T03:05:39.772780+0000) 2024-09-12T03:06:26.310 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:26 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:26.004+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:24.374445+0000 front 2024-09-12T03:05:24.374549+0000 (oldest deadline 2024-09-12T03:05:48.374868+0000) 2024-09-12T03:06:26.310 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:26 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:26.004+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:34.276237+0000 front 2024-09-12T03:05:34.276182+0000 (oldest deadline 2024-09-12T03:05:58.976098+0000) 2024-09-12T03:06:26.310 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:26 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:26.004+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-12T03:05:42.377230+0000 front 2024-09-12T03:05:42.377348+0000 (oldest deadline 2024-09-12T03:06:07.677394+0000) 2024-09-12T03:06:26.310 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:26 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:26.004+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-09-12T03:05:47.678051+0000 front 2024-09-12T03:05:47.677984+0000 (oldest deadline 2024-09-12T03:06:12.377935+0000) 2024-09-12T03:06:26.311 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:26 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:26.004+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.148:6812 osd.5 since back 2024-09-12T03:05:58.179806+0000 front 2024-09-12T03:05:58.179817+0000 (oldest deadline 2024-09-12T03:06:22.279613+0000) 2024-09-12T03:06:26.665 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:26 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:26.374+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:16.844053+0000 front 2024-09-12T03:05:16.843858+0000 (oldest deadline 2024-09-12T03:05:42.644259+0000) 2024-09-12T03:06:26.665 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:26 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:26.374+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:22.644805+0000 front 2024-09-12T03:05:22.644911+0000 (oldest deadline 2024-09-12T03:05:47.344653+0000) 2024-09-12T03:06:26.666 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:26 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:26.374+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:31.846594+0000 front 2024-09-12T03:05:31.846677+0000 (oldest deadline 2024-09-12T03:05:59.447156+0000) 2024-09-12T03:06:26.666 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:26 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:26.374+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-12T03:05:39.447493+0000 front 2024-09-12T03:05:39.447609+0000 (oldest deadline 2024-09-12T03:06:03.547590+0000) 2024-09-12T03:06:26.666 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:26 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:26.374+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-09-12T03:05:48.248794+0000 front 2024-09-12T03:05:48.248779+0000 (oldest deadline 2024-09-12T03:06:12.348647+0000) 2024-09-12T03:06:26.666 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:26 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:26.374+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.148:6812 osd.5 since back 2024-09-12T03:05:55.849475+0000 front 2024-09-12T03:05:55.849548+0000 (oldest deadline 2024-09-12T03:06:21.149634+0000) 2024-09-12T03:06:27.310 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:26 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:26.999+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:14.571718+0000 front 2024-09-12T03:05:14.571768+0000 (oldest deadline 2024-09-12T03:05:39.772780+0000) 2024-09-12T03:06:27.310 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:26 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:26.999+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:24.374445+0000 front 2024-09-12T03:05:24.374549+0000 (oldest deadline 2024-09-12T03:05:48.374868+0000) 2024-09-12T03:06:27.310 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:26 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:26.999+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:34.276237+0000 front 2024-09-12T03:05:34.276182+0000 (oldest deadline 2024-09-12T03:05:58.976098+0000) 2024-09-12T03:06:27.310 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:26 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:26.999+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-12T03:05:42.377230+0000 front 2024-09-12T03:05:42.377348+0000 (oldest deadline 2024-09-12T03:06:07.677394+0000) 2024-09-12T03:06:27.310 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:26 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:26.999+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-09-12T03:05:47.678051+0000 front 2024-09-12T03:05:47.677984+0000 (oldest deadline 2024-09-12T03:06:12.377935+0000) 2024-09-12T03:06:27.310 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:26 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:26.999+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.148:6812 osd.5 since back 2024-09-12T03:05:58.179806+0000 front 2024-09-12T03:05:58.179817+0000 (oldest deadline 2024-09-12T03:06:22.279613+0000) 2024-09-12T03:06:27.456 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:26 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:26.975+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:18.916404+0000 front 2024-09-12T03:05:18.916319+0000 (oldest deadline 2024-09-12T03:05:39.416329+0000) 2024-09-12T03:06:27.456 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:26 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:26.975+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:25.817651+0000 front 2024-09-12T03:05:25.817711+0000 (oldest deadline 2024-09-12T03:05:50.517693+0000) 2024-09-12T03:06:27.456 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:26 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:26.975+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:30.518204+0000 front 2024-09-12T03:05:30.518383+0000 (oldest deadline 2024-09-12T03:05:55.218163+0000) 2024-09-12T03:06:27.456 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:26 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:26.975+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-12T03:05:42.120310+0000 front 2024-09-12T03:05:42.120454+0000 (oldest deadline 2024-09-12T03:06:06.820355+0000) 2024-09-12T03:06:27.457 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:26 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:26.975+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-09-12T03:05:47.921106+0000 front 2024-09-12T03:05:47.921133+0000 (oldest deadline 2024-09-12T03:06:13.821241+0000) 2024-09-12T03:06:27.457 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:26 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:26.975+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.148:6812 osd.5 since back 2024-09-12T03:05:54.322143+0000 front 2024-09-12T03:05:54.322202+0000 (oldest deadline 2024-09-12T03:06:23.122455+0000) 2024-09-12T03:06:27.666 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:27 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:27.369+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:16.844053+0000 front 2024-09-12T03:05:16.843858+0000 (oldest deadline 2024-09-12T03:05:42.644259+0000) 2024-09-12T03:06:27.666 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:27 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:27.369+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:22.644805+0000 front 2024-09-12T03:05:22.644911+0000 (oldest deadline 2024-09-12T03:05:47.344653+0000) 2024-09-12T03:06:27.666 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:27 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:27.369+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:31.846594+0000 front 2024-09-12T03:05:31.846677+0000 (oldest deadline 2024-09-12T03:05:59.447156+0000) 2024-09-12T03:06:27.666 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:27 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:27.369+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-12T03:05:39.447493+0000 front 2024-09-12T03:05:39.447609+0000 (oldest deadline 2024-09-12T03:06:03.547590+0000) 2024-09-12T03:06:27.666 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:27 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:27.369+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-09-12T03:05:48.248794+0000 front 2024-09-12T03:05:48.248779+0000 (oldest deadline 2024-09-12T03:06:12.348647+0000) 2024-09-12T03:06:27.666 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:27 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:27.369+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.148:6812 osd.5 since back 2024-09-12T03:05:55.849475+0000 front 2024-09-12T03:05:55.849548+0000 (oldest deadline 2024-09-12T03:06:21.149634+0000) 2024-09-12T03:06:28.206 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:27 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:27.944+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:18.916404+0000 front 2024-09-12T03:05:18.916319+0000 (oldest deadline 2024-09-12T03:05:39.416329+0000) 2024-09-12T03:06:28.206 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:27 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:27.944+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:25.817651+0000 front 2024-09-12T03:05:25.817711+0000 (oldest deadline 2024-09-12T03:05:50.517693+0000) 2024-09-12T03:06:28.207 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:27 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:27.944+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:30.518204+0000 front 2024-09-12T03:05:30.518383+0000 (oldest deadline 2024-09-12T03:05:55.218163+0000) 2024-09-12T03:06:28.207 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:27 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:27.944+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-12T03:05:42.120310+0000 front 2024-09-12T03:05:42.120454+0000 (oldest deadline 2024-09-12T03:06:06.820355+0000) 2024-09-12T03:06:28.207 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:27 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:27.944+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-09-12T03:05:47.921106+0000 front 2024-09-12T03:05:47.921133+0000 (oldest deadline 2024-09-12T03:06:13.821241+0000) 2024-09-12T03:06:28.207 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:27 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:27.944+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.148:6812 osd.5 since back 2024-09-12T03:05:54.322143+0000 front 2024-09-12T03:05:54.322202+0000 (oldest deadline 2024-09-12T03:06:23.122455+0000) 2024-09-12T03:06:28.207 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:27 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:27.944+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-12T03:06:03.123072+0000 front 2024-09-12T03:06:03.123046+0000 (oldest deadline 2024-09-12T03:06:27.822843+0000) 2024-09-12T03:06:28.310 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:28 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:28.009+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:14.571718+0000 front 2024-09-12T03:05:14.571768+0000 (oldest deadline 2024-09-12T03:05:39.772780+0000) 2024-09-12T03:06:28.310 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:28 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:28.009+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:24.374445+0000 front 2024-09-12T03:05:24.374549+0000 (oldest deadline 2024-09-12T03:05:48.374868+0000) 2024-09-12T03:06:28.310 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:28 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:28.009+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:34.276237+0000 front 2024-09-12T03:05:34.276182+0000 (oldest deadline 2024-09-12T03:05:58.976098+0000) 2024-09-12T03:06:28.310 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:28 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:28.009+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-12T03:05:42.377230+0000 front 2024-09-12T03:05:42.377348+0000 (oldest deadline 2024-09-12T03:06:07.677394+0000) 2024-09-12T03:06:28.310 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:28 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:28.009+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-09-12T03:05:47.678051+0000 front 2024-09-12T03:05:47.677984+0000 (oldest deadline 2024-09-12T03:06:12.377935+0000) 2024-09-12T03:06:28.310 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:28 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:28.009+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.148:6812 osd.5 since back 2024-09-12T03:05:58.179806+0000 front 2024-09-12T03:05:58.179817+0000 (oldest deadline 2024-09-12T03:06:22.279613+0000) 2024-09-12T03:06:28.666 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:28 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:28.332+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:16.844053+0000 front 2024-09-12T03:05:16.843858+0000 (oldest deadline 2024-09-12T03:05:42.644259+0000) 2024-09-12T03:06:28.667 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:28 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:28.332+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:22.644805+0000 front 2024-09-12T03:05:22.644911+0000 (oldest deadline 2024-09-12T03:05:47.344653+0000) 2024-09-12T03:06:28.667 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:28 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:28.332+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:31.846594+0000 front 2024-09-12T03:05:31.846677+0000 (oldest deadline 2024-09-12T03:05:59.447156+0000) 2024-09-12T03:06:28.667 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:28 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:28.332+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-12T03:05:39.447493+0000 front 2024-09-12T03:05:39.447609+0000 (oldest deadline 2024-09-12T03:06:03.547590+0000) 2024-09-12T03:06:28.667 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:28 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:28.332+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-09-12T03:05:48.248794+0000 front 2024-09-12T03:05:48.248779+0000 (oldest deadline 2024-09-12T03:06:12.348647+0000) 2024-09-12T03:06:28.667 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:28 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:28.332+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.148:6812 osd.5 since back 2024-09-12T03:05:55.849475+0000 front 2024-09-12T03:05:55.849548+0000 (oldest deadline 2024-09-12T03:06:21.149634+0000) 2024-09-12T03:06:29.206 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:28 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:28.952+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:18.916404+0000 front 2024-09-12T03:05:18.916319+0000 (oldest deadline 2024-09-12T03:05:39.416329+0000) 2024-09-12T03:06:29.206 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:28 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:28.952+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:25.817651+0000 front 2024-09-12T03:05:25.817711+0000 (oldest deadline 2024-09-12T03:05:50.517693+0000) 2024-09-12T03:06:29.206 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:28 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:28.952+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:30.518204+0000 front 2024-09-12T03:05:30.518383+0000 (oldest deadline 2024-09-12T03:05:55.218163+0000) 2024-09-12T03:06:29.207 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:28 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:28.952+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-12T03:05:42.120310+0000 front 2024-09-12T03:05:42.120454+0000 (oldest deadline 2024-09-12T03:06:06.820355+0000) 2024-09-12T03:06:29.207 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:28 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:28.952+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-09-12T03:05:47.921106+0000 front 2024-09-12T03:05:47.921133+0000 (oldest deadline 2024-09-12T03:06:13.821241+0000) 2024-09-12T03:06:29.207 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:28 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:28.952+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.148:6812 osd.5 since back 2024-09-12T03:05:54.322143+0000 front 2024-09-12T03:05:54.322202+0000 (oldest deadline 2024-09-12T03:06:23.122455+0000) 2024-09-12T03:06:29.207 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:28 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:28.952+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-12T03:06:03.123072+0000 front 2024-09-12T03:06:03.123046+0000 (oldest deadline 2024-09-12T03:06:27.822843+0000) 2024-09-12T03:06:29.310 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:28 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:28.967+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:14.571718+0000 front 2024-09-12T03:05:14.571768+0000 (oldest deadline 2024-09-12T03:05:39.772780+0000) 2024-09-12T03:06:29.310 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:28 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:28.967+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:24.374445+0000 front 2024-09-12T03:05:24.374549+0000 (oldest deadline 2024-09-12T03:05:48.374868+0000) 2024-09-12T03:06:29.310 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:28 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:28.967+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:34.276237+0000 front 2024-09-12T03:05:34.276182+0000 (oldest deadline 2024-09-12T03:05:58.976098+0000) 2024-09-12T03:06:29.310 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:28 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:28.967+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-12T03:05:42.377230+0000 front 2024-09-12T03:05:42.377348+0000 (oldest deadline 2024-09-12T03:06:07.677394+0000) 2024-09-12T03:06:29.310 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:28 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:28.967+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-09-12T03:05:47.678051+0000 front 2024-09-12T03:05:47.677984+0000 (oldest deadline 2024-09-12T03:06:12.377935+0000) 2024-09-12T03:06:29.310 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:28 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:28.967+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.148:6812 osd.5 since back 2024-09-12T03:05:58.179806+0000 front 2024-09-12T03:05:58.179817+0000 (oldest deadline 2024-09-12T03:06:22.279613+0000) 2024-09-12T03:06:29.310 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:28 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9[63125]: 2024-09-12T03:06:28.967+0000 7fe446ec0640 -1 osd.9 86 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-12T03:06:05.681080+0000 front 2024-09-12T03:06:05.680971+0000 (oldest deadline 2024-09-12T03:06:28.580998+0000) 2024-09-12T03:06:29.665 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:29 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:29.367+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:16.844053+0000 front 2024-09-12T03:05:16.843858+0000 (oldest deadline 2024-09-12T03:05:42.644259+0000) 2024-09-12T03:06:29.666 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:29 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:29.367+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:22.644805+0000 front 2024-09-12T03:05:22.644911+0000 (oldest deadline 2024-09-12T03:05:47.344653+0000) 2024-09-12T03:06:29.666 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:29 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:29.367+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:31.846594+0000 front 2024-09-12T03:05:31.846677+0000 (oldest deadline 2024-09-12T03:05:59.447156+0000) 2024-09-12T03:06:29.666 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:29 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:29.367+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-12T03:05:39.447493+0000 front 2024-09-12T03:05:39.447609+0000 (oldest deadline 2024-09-12T03:06:03.547590+0000) 2024-09-12T03:06:29.666 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:29 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:29.367+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-09-12T03:05:48.248794+0000 front 2024-09-12T03:05:48.248779+0000 (oldest deadline 2024-09-12T03:06:12.348647+0000) 2024-09-12T03:06:29.666 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:29 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:29.367+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.148:6812 osd.5 since back 2024-09-12T03:05:55.849475+0000 front 2024-09-12T03:05:55.849548+0000 (oldest deadline 2024-09-12T03:06:21.149634+0000) 2024-09-12T03:06:30.206 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:29 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:29.918+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:18.916404+0000 front 2024-09-12T03:05:18.916319+0000 (oldest deadline 2024-09-12T03:05:39.416329+0000) 2024-09-12T03:06:30.207 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:29 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:29.918+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:25.817651+0000 front 2024-09-12T03:05:25.817711+0000 (oldest deadline 2024-09-12T03:05:50.517693+0000) 2024-09-12T03:06:30.207 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:29 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:29.918+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:30.518204+0000 front 2024-09-12T03:05:30.518383+0000 (oldest deadline 2024-09-12T03:05:55.218163+0000) 2024-09-12T03:06:30.207 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:29 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:29.918+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-12T03:05:42.120310+0000 front 2024-09-12T03:05:42.120454+0000 (oldest deadline 2024-09-12T03:06:06.820355+0000) 2024-09-12T03:06:30.207 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:29 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:29.918+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-09-12T03:05:47.921106+0000 front 2024-09-12T03:05:47.921133+0000 (oldest deadline 2024-09-12T03:06:13.821241+0000) 2024-09-12T03:06:30.207 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:29 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:29.918+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.148:6812 osd.5 since back 2024-09-12T03:05:54.322143+0000 front 2024-09-12T03:05:54.322202+0000 (oldest deadline 2024-09-12T03:06:23.122455+0000) 2024-09-12T03:06:30.207 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:29 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:29.918+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-12T03:06:03.123072+0000 front 2024-09-12T03:06:03.123046+0000 (oldest deadline 2024-09-12T03:06:27.822843+0000) 2024-09-12T03:06:30.560 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:30 smithi005 podman[118814]: 2024-09-12 03:06:30.248018142 +0000 UTC m=+5.499922947 container died 140fd33a3c1bb5ab78ba35369b25c94f02c54351968c8c8ca74ac733db19704d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9, GIT_CLEAN=True, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux ) 2024-09-12T03:06:30.665 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:30 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:30.377+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:16.844053+0000 front 2024-09-12T03:05:16.843858+0000 (oldest deadline 2024-09-12T03:05:42.644259+0000) 2024-09-12T03:06:30.665 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:30 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:30.377+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:22.644805+0000 front 2024-09-12T03:05:22.644911+0000 (oldest deadline 2024-09-12T03:05:47.344653+0000) 2024-09-12T03:06:30.666 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:30 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:30.377+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:31.846594+0000 front 2024-09-12T03:05:31.846677+0000 (oldest deadline 2024-09-12T03:05:59.447156+0000) 2024-09-12T03:06:30.666 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:30 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:30.377+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-12T03:05:39.447493+0000 front 2024-09-12T03:05:39.447609+0000 (oldest deadline 2024-09-12T03:06:03.547590+0000) 2024-09-12T03:06:30.666 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:30 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:30.378+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-09-12T03:05:48.248794+0000 front 2024-09-12T03:05:48.248779+0000 (oldest deadline 2024-09-12T03:06:12.348647+0000) 2024-09-12T03:06:30.666 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:30 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:30.378+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.148:6812 osd.5 since back 2024-09-12T03:05:55.849475+0000 front 2024-09-12T03:05:55.849548+0000 (oldest deadline 2024-09-12T03:06:21.149634+0000) 2024-09-12T03:06:30.877 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:30 smithi005 podman[118814]: 2024-09-12 03:06:30.610001275 +0000 UTC m=+5.861906116 container cleanup 140fd33a3c1bb5ab78ba35369b25c94f02c54351968c8c8ca74ac733db19704d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T03:06:30.877 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:30 smithi005 bash[118814]: ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9 2024-09-12T03:06:31.228 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:30 smithi005 podman[118831]: 2024-09-12 03:06:30.877983576 +0000 UTC m=+0.625796000 container remove 140fd33a3c1bb5ab78ba35369b25c94f02c54351968c8c8ca74ac733db19704d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3) 2024-09-12T03:06:31.456 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:30 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:30.962+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:18.916404+0000 front 2024-09-12T03:05:18.916319+0000 (oldest deadline 2024-09-12T03:05:39.416329+0000) 2024-09-12T03:06:31.456 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:30 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:30.962+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:25.817651+0000 front 2024-09-12T03:05:25.817711+0000 (oldest deadline 2024-09-12T03:05:50.517693+0000) 2024-09-12T03:06:31.456 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:30 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:30.962+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:30.518204+0000 front 2024-09-12T03:05:30.518383+0000 (oldest deadline 2024-09-12T03:05:55.218163+0000) 2024-09-12T03:06:31.456 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:30 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:30.962+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-12T03:05:42.120310+0000 front 2024-09-12T03:05:42.120454+0000 (oldest deadline 2024-09-12T03:06:06.820355+0000) 2024-09-12T03:06:31.457 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:30 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:30.962+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-09-12T03:05:47.921106+0000 front 2024-09-12T03:05:47.921133+0000 (oldest deadline 2024-09-12T03:06:13.821241+0000) 2024-09-12T03:06:31.457 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:30 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:30.962+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.148:6812 osd.5 since back 2024-09-12T03:05:54.322143+0000 front 2024-09-12T03:05:54.322202+0000 (oldest deadline 2024-09-12T03:06:23.122455+0000) 2024-09-12T03:06:31.457 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:30 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:30.962+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-12T03:06:03.123072+0000 front 2024-09-12T03:06:03.123046+0000 (oldest deadline 2024-09-12T03:06:27.822843+0000) 2024-09-12T03:06:31.539 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:31 smithi005 podman[118922]: 2024-09-12 03:06:31.228696383 +0000 UTC m=+0.111857687 container create 8a572ea0f3d66f48d47c22fe74601b384ee15c3c26034866128381bace27b991 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2) 2024-09-12T03:06:31.539 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:31 smithi005 podman[118922]: 2024-09-12 03:06:31.133293558 +0000 UTC m=+0.016454858 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:06:31.665 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:31 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:31.369+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:16.844053+0000 front 2024-09-12T03:05:16.843858+0000 (oldest deadline 2024-09-12T03:05:42.644259+0000) 2024-09-12T03:06:31.666 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:31 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:31.369+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:22.644805+0000 front 2024-09-12T03:05:22.644911+0000 (oldest deadline 2024-09-12T03:05:47.344653+0000) 2024-09-12T03:06:31.666 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:31 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:31.369+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:31.846594+0000 front 2024-09-12T03:05:31.846677+0000 (oldest deadline 2024-09-12T03:05:59.447156+0000) 2024-09-12T03:06:31.666 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:31 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:31.369+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-12T03:05:39.447493+0000 front 2024-09-12T03:05:39.447609+0000 (oldest deadline 2024-09-12T03:06:03.547590+0000) 2024-09-12T03:06:31.666 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:31 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:31.369+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-09-12T03:05:48.248794+0000 front 2024-09-12T03:05:48.248779+0000 (oldest deadline 2024-09-12T03:06:12.348647+0000) 2024-09-12T03:06:31.666 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:31 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:31.369+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.148:6812 osd.5 since back 2024-09-12T03:05:55.849475+0000 front 2024-09-12T03:05:55.849548+0000 (oldest deadline 2024-09-12T03:06:21.149634+0000) 2024-09-12T03:06:31.810 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:31 smithi005 podman[118922]: 2024-09-12 03:06:31.539701504 +0000 UTC m=+0.422862809 container init 8a572ea0f3d66f48d47c22fe74601b384ee15c3c26034866128381bace27b991 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9-deactivate, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, RELEASE=squid-b53b6d3, GIT_CLEAN=True) 2024-09-12T03:06:31.811 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:31 smithi005 podman[118922]: 2024-09-12 03:06:31.543652907 +0000 UTC m=+0.426814207 container start 8a572ea0f3d66f48d47c22fe74601b384ee15c3c26034866128381bace27b991 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-12T03:06:31.811 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:31 smithi005 podman[118922]: 2024-09-12 03:06:31.614265562 +0000 UTC m=+0.497426872 container attach 8a572ea0f3d66f48d47c22fe74601b384ee15c3c26034866128381bace27b991 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9-deactivate, RELEASE=squid-b53b6d3, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-09-12T03:06:31.811 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:31 smithi005 podman[118922]: 2024-09-12 03:06:31.736086957 +0000 UTC m=+0.619248262 container died 8a572ea0f3d66f48d47c22fe74601b384ee15c3c26034866128381bace27b991 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9-deactivate, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux ) 2024-09-12T03:06:32.267 INFO:journalctl@ceph.osd.9.smithi005.stdout:Sep 12 03:06:32 smithi005 podman[118945]: 2024-09-12 03:06:32.250861632 +0000 UTC m=+0.508256542 container remove 8a572ea0f3d66f48d47c22fe74601b384ee15c3c26034866128381bace27b991 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-9-deactivate, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS) 2024-09-12T03:06:32.273 DEBUG:teuthology.orchestra.run.smithi005:> sudo pkill -f 'journalctl -f -n 0 -u ceph-618305f8-70b1-11ef-bcea-c7b262605968@osd.9.service' 2024-09-12T03:06:32.314 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:06:32.314 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-09-12T03:06:32.315 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-09-12T03:06:32.315 DEBUG:teuthology.orchestra.run.smithi146:> sudo systemctl stop ceph-618305f8-70b1-11ef-bcea-c7b262605968@osd.10 2024-09-12T03:06:32.342 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:31 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:31.981+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:18.916404+0000 front 2024-09-12T03:05:18.916319+0000 (oldest deadline 2024-09-12T03:05:39.416329+0000) 2024-09-12T03:06:32.342 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:31 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:31.981+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:25.817651+0000 front 2024-09-12T03:05:25.817711+0000 (oldest deadline 2024-09-12T03:05:50.517693+0000) 2024-09-12T03:06:32.342 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:31 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:31.981+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:30.518204+0000 front 2024-09-12T03:05:30.518383+0000 (oldest deadline 2024-09-12T03:05:55.218163+0000) 2024-09-12T03:06:32.342 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:31 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:31.981+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-12T03:05:42.120310+0000 front 2024-09-12T03:05:42.120454+0000 (oldest deadline 2024-09-12T03:06:06.820355+0000) 2024-09-12T03:06:32.342 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:31 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:31.981+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-09-12T03:05:47.921106+0000 front 2024-09-12T03:05:47.921133+0000 (oldest deadline 2024-09-12T03:06:13.821241+0000) 2024-09-12T03:06:32.342 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:31 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:31.981+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.148:6812 osd.5 since back 2024-09-12T03:05:54.322143+0000 front 2024-09-12T03:05:54.322202+0000 (oldest deadline 2024-09-12T03:06:23.122455+0000) 2024-09-12T03:06:32.342 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:31 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:31.981+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-12T03:06:03.123072+0000 front 2024-09-12T03:06:03.123046+0000 (oldest deadline 2024-09-12T03:06:27.822843+0000) 2024-09-12T03:06:32.602 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:32 smithi146 systemd[1]: Stopping Ceph osd.10 for 618305f8-70b1-11ef-bcea-c7b262605968... 2024-09-12T03:06:32.665 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:32 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:32.323+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:16.844053+0000 front 2024-09-12T03:05:16.843858+0000 (oldest deadline 2024-09-12T03:05:42.644259+0000) 2024-09-12T03:06:32.666 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:32 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:32.323+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:22.644805+0000 front 2024-09-12T03:05:22.644911+0000 (oldest deadline 2024-09-12T03:05:47.344653+0000) 2024-09-12T03:06:32.666 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:32 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:32.323+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:31.846594+0000 front 2024-09-12T03:05:31.846677+0000 (oldest deadline 2024-09-12T03:05:59.447156+0000) 2024-09-12T03:06:32.666 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:32 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:32.323+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-12T03:05:39.447493+0000 front 2024-09-12T03:05:39.447609+0000 (oldest deadline 2024-09-12T03:06:03.547590+0000) 2024-09-12T03:06:32.666 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:32 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:32.323+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-09-12T03:05:48.248794+0000 front 2024-09-12T03:05:48.248779+0000 (oldest deadline 2024-09-12T03:06:12.348647+0000) 2024-09-12T03:06:32.666 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:32 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:32.323+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.148:6812 osd.5 since back 2024-09-12T03:05:55.849475+0000 front 2024-09-12T03:05:55.849548+0000 (oldest deadline 2024-09-12T03:06:21.149634+0000) 2024-09-12T03:06:32.956 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:32 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:32.602+0000 7f8b61e6d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:06:32.956 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:32 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:32.602+0000 7f8b61e6d640 -1 osd.10 86 *** Got signal Terminated *** 2024-09-12T03:06:32.956 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:32 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:32.602+0000 7f8b61e6d640 -1 osd.10 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T03:06:33.456 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:33 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:33.019+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:18.916404+0000 front 2024-09-12T03:05:18.916319+0000 (oldest deadline 2024-09-12T03:05:39.416329+0000) 2024-09-12T03:06:33.457 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:33 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:33.019+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:25.817651+0000 front 2024-09-12T03:05:25.817711+0000 (oldest deadline 2024-09-12T03:05:50.517693+0000) 2024-09-12T03:06:33.457 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:33 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:33.019+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:30.518204+0000 front 2024-09-12T03:05:30.518383+0000 (oldest deadline 2024-09-12T03:05:55.218163+0000) 2024-09-12T03:06:33.457 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:33 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:33.019+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-12T03:05:42.120310+0000 front 2024-09-12T03:05:42.120454+0000 (oldest deadline 2024-09-12T03:06:06.820355+0000) 2024-09-12T03:06:33.457 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:33 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:33.019+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-09-12T03:05:47.921106+0000 front 2024-09-12T03:05:47.921133+0000 (oldest deadline 2024-09-12T03:06:13.821241+0000) 2024-09-12T03:06:33.458 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:33 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:33.019+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.148:6812 osd.5 since back 2024-09-12T03:05:54.322143+0000 front 2024-09-12T03:05:54.322202+0000 (oldest deadline 2024-09-12T03:06:23.122455+0000) 2024-09-12T03:06:33.458 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:33 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:33.019+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-12T03:06:03.123072+0000 front 2024-09-12T03:06:03.123046+0000 (oldest deadline 2024-09-12T03:06:27.822843+0000) 2024-09-12T03:06:33.666 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:33 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:33.356+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:16.844053+0000 front 2024-09-12T03:05:16.843858+0000 (oldest deadline 2024-09-12T03:05:42.644259+0000) 2024-09-12T03:06:33.666 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:33 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:33.356+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:22.644805+0000 front 2024-09-12T03:05:22.644911+0000 (oldest deadline 2024-09-12T03:05:47.344653+0000) 2024-09-12T03:06:33.666 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:33 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:33.356+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:31.846594+0000 front 2024-09-12T03:05:31.846677+0000 (oldest deadline 2024-09-12T03:05:59.447156+0000) 2024-09-12T03:06:33.666 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:33 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:33.356+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-12T03:05:39.447493+0000 front 2024-09-12T03:05:39.447609+0000 (oldest deadline 2024-09-12T03:06:03.547590+0000) 2024-09-12T03:06:33.666 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:33 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:33.356+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-09-12T03:05:48.248794+0000 front 2024-09-12T03:05:48.248779+0000 (oldest deadline 2024-09-12T03:06:12.348647+0000) 2024-09-12T03:06:33.666 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:33 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:33.356+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.148:6812 osd.5 since back 2024-09-12T03:05:55.849475+0000 front 2024-09-12T03:05:55.849548+0000 (oldest deadline 2024-09-12T03:06:21.149634+0000) 2024-09-12T03:06:34.456 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:34 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:34.040+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:18.916404+0000 front 2024-09-12T03:05:18.916319+0000 (oldest deadline 2024-09-12T03:05:39.416329+0000) 2024-09-12T03:06:34.456 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:34 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:34.040+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:25.817651+0000 front 2024-09-12T03:05:25.817711+0000 (oldest deadline 2024-09-12T03:05:50.517693+0000) 2024-09-12T03:06:34.457 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:34 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:34.040+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:30.518204+0000 front 2024-09-12T03:05:30.518383+0000 (oldest deadline 2024-09-12T03:05:55.218163+0000) 2024-09-12T03:06:34.457 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:34 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:34.040+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-12T03:05:42.120310+0000 front 2024-09-12T03:05:42.120454+0000 (oldest deadline 2024-09-12T03:06:06.820355+0000) 2024-09-12T03:06:34.457 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:34 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:34.040+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-09-12T03:05:47.921106+0000 front 2024-09-12T03:05:47.921133+0000 (oldest deadline 2024-09-12T03:06:13.821241+0000) 2024-09-12T03:06:34.457 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:34 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:34.040+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.148:6812 osd.5 since back 2024-09-12T03:05:54.322143+0000 front 2024-09-12T03:05:54.322202+0000 (oldest deadline 2024-09-12T03:06:23.122455+0000) 2024-09-12T03:06:34.457 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:34 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:34.040+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-12T03:06:03.123072+0000 front 2024-09-12T03:06:03.123046+0000 (oldest deadline 2024-09-12T03:06:27.822843+0000) 2024-09-12T03:06:34.665 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:34 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:34.399+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:16.844053+0000 front 2024-09-12T03:05:16.843858+0000 (oldest deadline 2024-09-12T03:05:42.644259+0000) 2024-09-12T03:06:34.666 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:34 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:34.399+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:22.644805+0000 front 2024-09-12T03:05:22.644911+0000 (oldest deadline 2024-09-12T03:05:47.344653+0000) 2024-09-12T03:06:34.666 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:34 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:34.399+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:31.846594+0000 front 2024-09-12T03:05:31.846677+0000 (oldest deadline 2024-09-12T03:05:59.447156+0000) 2024-09-12T03:06:34.666 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:34 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:34.399+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-12T03:05:39.447493+0000 front 2024-09-12T03:05:39.447609+0000 (oldest deadline 2024-09-12T03:06:03.547590+0000) 2024-09-12T03:06:34.666 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:34 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:34.399+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-09-12T03:05:48.248794+0000 front 2024-09-12T03:05:48.248779+0000 (oldest deadline 2024-09-12T03:06:12.348647+0000) 2024-09-12T03:06:34.666 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:34 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:34.399+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.148:6812 osd.5 since back 2024-09-12T03:05:55.849475+0000 front 2024-09-12T03:05:55.849548+0000 (oldest deadline 2024-09-12T03:06:21.149634+0000) 2024-09-12T03:06:35.456 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:35 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:35.018+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:18.916404+0000 front 2024-09-12T03:05:18.916319+0000 (oldest deadline 2024-09-12T03:05:39.416329+0000) 2024-09-12T03:06:35.456 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:35 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:35.018+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:25.817651+0000 front 2024-09-12T03:05:25.817711+0000 (oldest deadline 2024-09-12T03:05:50.517693+0000) 2024-09-12T03:06:35.457 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:35 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:35.018+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:30.518204+0000 front 2024-09-12T03:05:30.518383+0000 (oldest deadline 2024-09-12T03:05:55.218163+0000) 2024-09-12T03:06:35.457 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:35 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:35.018+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-12T03:05:42.120310+0000 front 2024-09-12T03:05:42.120454+0000 (oldest deadline 2024-09-12T03:06:06.820355+0000) 2024-09-12T03:06:35.457 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:35 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:35.018+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-09-12T03:05:47.921106+0000 front 2024-09-12T03:05:47.921133+0000 (oldest deadline 2024-09-12T03:06:13.821241+0000) 2024-09-12T03:06:35.457 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:35 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:35.018+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.148:6812 osd.5 since back 2024-09-12T03:05:54.322143+0000 front 2024-09-12T03:05:54.322202+0000 (oldest deadline 2024-09-12T03:06:23.122455+0000) 2024-09-12T03:06:35.457 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:35 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:35.018+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-12T03:06:03.123072+0000 front 2024-09-12T03:06:03.123046+0000 (oldest deadline 2024-09-12T03:06:27.822843+0000) 2024-09-12T03:06:35.722 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:35 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:35.446+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:16.844053+0000 front 2024-09-12T03:05:16.843858+0000 (oldest deadline 2024-09-12T03:05:42.644259+0000) 2024-09-12T03:06:35.722 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:35 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:35.446+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:22.644805+0000 front 2024-09-12T03:05:22.644911+0000 (oldest deadline 2024-09-12T03:05:47.344653+0000) 2024-09-12T03:06:35.722 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:35 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:35.446+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:31.846594+0000 front 2024-09-12T03:05:31.846677+0000 (oldest deadline 2024-09-12T03:05:59.447156+0000) 2024-09-12T03:06:35.722 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:35 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:35.446+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-12T03:05:39.447493+0000 front 2024-09-12T03:05:39.447609+0000 (oldest deadline 2024-09-12T03:06:03.547590+0000) 2024-09-12T03:06:35.722 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:35 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:35.446+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-09-12T03:05:48.248794+0000 front 2024-09-12T03:05:48.248779+0000 (oldest deadline 2024-09-12T03:06:12.348647+0000) 2024-09-12T03:06:35.722 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:35 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:35.446+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.148:6812 osd.5 since back 2024-09-12T03:05:55.849475+0000 front 2024-09-12T03:05:55.849548+0000 (oldest deadline 2024-09-12T03:06:21.149634+0000) 2024-09-12T03:06:36.456 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:36 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:36.008+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:18.916404+0000 front 2024-09-12T03:05:18.916319+0000 (oldest deadline 2024-09-12T03:05:39.416329+0000) 2024-09-12T03:06:36.456 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:36 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:36.008+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:25.817651+0000 front 2024-09-12T03:05:25.817711+0000 (oldest deadline 2024-09-12T03:05:50.517693+0000) 2024-09-12T03:06:36.457 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:36 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:36.008+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:30.518204+0000 front 2024-09-12T03:05:30.518383+0000 (oldest deadline 2024-09-12T03:05:55.218163+0000) 2024-09-12T03:06:36.457 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:36 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:36.008+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-12T03:05:42.120310+0000 front 2024-09-12T03:05:42.120454+0000 (oldest deadline 2024-09-12T03:06:06.820355+0000) 2024-09-12T03:06:36.457 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:36 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:36.008+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-09-12T03:05:47.921106+0000 front 2024-09-12T03:05:47.921133+0000 (oldest deadline 2024-09-12T03:06:13.821241+0000) 2024-09-12T03:06:36.457 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:36 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:36.008+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.148:6812 osd.5 since back 2024-09-12T03:05:54.322143+0000 front 2024-09-12T03:05:54.322202+0000 (oldest deadline 2024-09-12T03:06:23.122455+0000) 2024-09-12T03:06:36.457 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:36 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:36.008+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-12T03:06:03.123072+0000 front 2024-09-12T03:06:03.123046+0000 (oldest deadline 2024-09-12T03:06:27.822843+0000) 2024-09-12T03:06:36.916 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:36 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:36.470+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:16.844053+0000 front 2024-09-12T03:05:16.843858+0000 (oldest deadline 2024-09-12T03:05:42.644259+0000) 2024-09-12T03:06:36.916 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:36 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:36.470+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:22.644805+0000 front 2024-09-12T03:05:22.644911+0000 (oldest deadline 2024-09-12T03:05:47.344653+0000) 2024-09-12T03:06:36.916 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:36 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:36.470+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:31.846594+0000 front 2024-09-12T03:05:31.846677+0000 (oldest deadline 2024-09-12T03:05:59.447156+0000) 2024-09-12T03:06:36.916 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:36 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:36.470+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-12T03:05:39.447493+0000 front 2024-09-12T03:05:39.447609+0000 (oldest deadline 2024-09-12T03:06:03.547590+0000) 2024-09-12T03:06:36.916 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:36 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:36.470+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-09-12T03:05:48.248794+0000 front 2024-09-12T03:05:48.248779+0000 (oldest deadline 2024-09-12T03:06:12.348647+0000) 2024-09-12T03:06:36.917 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:36 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:36.470+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.148:6812 osd.5 since back 2024-09-12T03:05:55.849475+0000 front 2024-09-12T03:05:55.849548+0000 (oldest deadline 2024-09-12T03:06:21.149634+0000) 2024-09-12T03:06:37.456 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:37 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:37.029+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:18.916404+0000 front 2024-09-12T03:05:18.916319+0000 (oldest deadline 2024-09-12T03:05:39.416329+0000) 2024-09-12T03:06:37.456 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:37 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:37.029+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:25.817651+0000 front 2024-09-12T03:05:25.817711+0000 (oldest deadline 2024-09-12T03:05:50.517693+0000) 2024-09-12T03:06:37.456 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:37 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:37.029+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:30.518204+0000 front 2024-09-12T03:05:30.518383+0000 (oldest deadline 2024-09-12T03:05:55.218163+0000) 2024-09-12T03:06:37.457 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:37 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:37.029+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-12T03:05:42.120310+0000 front 2024-09-12T03:05:42.120454+0000 (oldest deadline 2024-09-12T03:06:06.820355+0000) 2024-09-12T03:06:37.457 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:37 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:37.029+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-09-12T03:05:47.921106+0000 front 2024-09-12T03:05:47.921133+0000 (oldest deadline 2024-09-12T03:06:13.821241+0000) 2024-09-12T03:06:37.457 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:37 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:37.029+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.148:6812 osd.5 since back 2024-09-12T03:05:54.322143+0000 front 2024-09-12T03:05:54.322202+0000 (oldest deadline 2024-09-12T03:06:23.122455+0000) 2024-09-12T03:06:37.457 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:37 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10[59743]: 2024-09-12T03:06:37.029+0000 7f8b5a46a640 -1 osd.10 86 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2024-09-12T03:06:03.123072+0000 front 2024-09-12T03:06:03.123046+0000 (oldest deadline 2024-09-12T03:06:27.822843+0000) 2024-09-12T03:06:37.916 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:37 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:37.504+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:16.844053+0000 front 2024-09-12T03:05:16.843858+0000 (oldest deadline 2024-09-12T03:05:42.644259+0000) 2024-09-12T03:06:37.916 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:37 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:37.504+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:22.644805+0000 front 2024-09-12T03:05:22.644911+0000 (oldest deadline 2024-09-12T03:05:47.344653+0000) 2024-09-12T03:06:37.916 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:37 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:37.504+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:31.846594+0000 front 2024-09-12T03:05:31.846677+0000 (oldest deadline 2024-09-12T03:05:59.447156+0000) 2024-09-12T03:06:37.916 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:37 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:37.504+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-12T03:05:39.447493+0000 front 2024-09-12T03:05:39.447609+0000 (oldest deadline 2024-09-12T03:06:03.547590+0000) 2024-09-12T03:06:37.916 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:37 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:37.504+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-09-12T03:05:48.248794+0000 front 2024-09-12T03:05:48.248779+0000 (oldest deadline 2024-09-12T03:06:12.348647+0000) 2024-09-12T03:06:37.916 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:37 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:37.504+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.148:6812 osd.5 since back 2024-09-12T03:05:55.849475+0000 front 2024-09-12T03:05:55.849548+0000 (oldest deadline 2024-09-12T03:06:21.149634+0000) 2024-09-12T03:06:38.343 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:38 smithi146 podman[69803]: 2024-09-12 03:06:38.039806945 +0000 UTC m=+5.527569399 container died d1f11e3bd1e24a7f400a7a49ce6ebff64966aac87dbda87ee93d0ef19a1a5794 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T03:06:38.706 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:38 smithi146 podman[69803]: 2024-09-12 03:06:38.344068896 +0000 UTC m=+5.831831347 container cleanup d1f11e3bd1e24a7f400a7a49ce6ebff64966aac87dbda87ee93d0ef19a1a5794 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3) 2024-09-12T03:06:38.707 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:38 smithi146 bash[69803]: ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10 2024-09-12T03:06:38.707 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:38 smithi146 podman[69818]: 2024-09-12 03:06:38.553300918 +0000 UTC m=+0.509130632 container remove d1f11e3bd1e24a7f400a7a49ce6ebff64966aac87dbda87ee93d0ef19a1a5794 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2) 2024-09-12T03:06:38.915 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:38 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:38.460+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:16.844053+0000 front 2024-09-12T03:05:16.843858+0000 (oldest deadline 2024-09-12T03:05:42.644259+0000) 2024-09-12T03:06:38.916 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:38 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:38.460+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:22.644805+0000 front 2024-09-12T03:05:22.644911+0000 (oldest deadline 2024-09-12T03:05:47.344653+0000) 2024-09-12T03:06:38.916 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:38 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:38.460+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:31.846594+0000 front 2024-09-12T03:05:31.846677+0000 (oldest deadline 2024-09-12T03:05:59.447156+0000) 2024-09-12T03:06:38.916 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:38 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:38.460+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-12T03:05:39.447493+0000 front 2024-09-12T03:05:39.447609+0000 (oldest deadline 2024-09-12T03:06:03.547590+0000) 2024-09-12T03:06:38.916 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:38 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:38.460+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-09-12T03:05:48.248794+0000 front 2024-09-12T03:05:48.248779+0000 (oldest deadline 2024-09-12T03:06:12.348647+0000) 2024-09-12T03:06:38.916 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:38 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:38.460+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.148:6812 osd.5 since back 2024-09-12T03:05:55.849475+0000 front 2024-09-12T03:05:55.849548+0000 (oldest deadline 2024-09-12T03:06:21.149634+0000) 2024-09-12T03:06:38.916 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:38 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:38.460+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.146:6820 osd.7 since back 2024-09-12T03:06:12.351187+0000 front 2024-09-12T03:06:12.351311+0000 (oldest deadline 2024-09-12T03:06:38.251021+0000) 2024-09-12T03:06:39.139 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:38 smithi146 podman[69903]: 2024-09-12 03:06:38.854035699 +0000 UTC m=+0.104649417 container create 02ad42558b6515d5fc37efa318cf27ee84bc24e5e7b0e89b9ae8544bf0e9251e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T03:06:39.139 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:38 smithi146 podman[69903]: 2024-09-12 03:06:38.765665777 +0000 UTC m=+0.016279509 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:06:39.139 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:39 smithi146 podman[69903]: 2024-09-12 03:06:39.064664111 +0000 UTC m=+0.315277853 container init 02ad42558b6515d5fc37efa318cf27ee84bc24e5e7b0e89b9ae8544bf0e9251e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731) 2024-09-12T03:06:39.139 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:39 smithi146 podman[69903]: 2024-09-12 03:06:39.068429716 +0000 UTC m=+0.319043434 container start 02ad42558b6515d5fc37efa318cf27ee84bc24e5e7b0e89b9ae8544bf0e9251e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10-deactivate, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-12T03:06:39.139 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:39 smithi146 podman[69903]: 2024-09-12 03:06:39.13945163 +0000 UTC m=+0.390065356 container attach 02ad42558b6515d5fc37efa318cf27ee84bc24e5e7b0e89b9ae8544bf0e9251e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2) 2024-09-12T03:06:39.456 INFO:journalctl@ceph.osd.10.smithi146.stdout:Sep 12 03:06:39 smithi146 podman[69903]: 2024-09-12 03:06:39.262939113 +0000 UTC m=+0.513552836 container died 02ad42558b6515d5fc37efa318cf27ee84bc24e5e7b0e89b9ae8544bf0e9251e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-10-deactivate, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731) 2024-09-12T03:06:39.849 DEBUG:teuthology.orchestra.run.smithi146:> sudo pkill -f 'journalctl -f -n 0 -u ceph-618305f8-70b1-11ef-bcea-c7b262605968@osd.10.service' 2024-09-12T03:06:39.915 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:39 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:39.475+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:16.844053+0000 front 2024-09-12T03:05:16.843858+0000 (oldest deadline 2024-09-12T03:05:42.644259+0000) 2024-09-12T03:06:39.916 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:39 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:39.475+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:22.644805+0000 front 2024-09-12T03:05:22.644911+0000 (oldest deadline 2024-09-12T03:05:47.344653+0000) 2024-09-12T03:06:39.916 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:39 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:39.475+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:31.846594+0000 front 2024-09-12T03:05:31.846677+0000 (oldest deadline 2024-09-12T03:05:59.447156+0000) 2024-09-12T03:06:39.916 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:39 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:39.475+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-12T03:05:39.447493+0000 front 2024-09-12T03:05:39.447609+0000 (oldest deadline 2024-09-12T03:06:03.547590+0000) 2024-09-12T03:06:39.916 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:39 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:39.475+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-09-12T03:05:48.248794+0000 front 2024-09-12T03:05:48.248779+0000 (oldest deadline 2024-09-12T03:06:12.348647+0000) 2024-09-12T03:06:39.916 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:39 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:39.475+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.148:6812 osd.5 since back 2024-09-12T03:05:55.849475+0000 front 2024-09-12T03:05:55.849548+0000 (oldest deadline 2024-09-12T03:06:21.149634+0000) 2024-09-12T03:06:39.916 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:39 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:39.475+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.146:6820 osd.7 since back 2024-09-12T03:06:12.351187+0000 front 2024-09-12T03:06:12.351311+0000 (oldest deadline 2024-09-12T03:06:38.251021+0000) 2024-09-12T03:06:39.929 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:06:39.929 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-09-12T03:06:39.929 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-09-12T03:06:39.929 DEBUG:teuthology.orchestra.run.smithi148:> sudo systemctl stop ceph-618305f8-70b1-11ef-bcea-c7b262605968@osd.11 2024-09-12T03:06:40.229 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:39 smithi148 systemd[1]: Stopping Ceph osd.11 for 618305f8-70b1-11ef-bcea-c7b262605968... 2024-09-12T03:06:40.666 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:40 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:40.228+0000 7fb89b4f1640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:06:40.666 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:40 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:40.228+0000 7fb89b4f1640 -1 osd.11 86 *** Got signal Terminated *** 2024-09-12T03:06:40.666 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:40 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:40.228+0000 7fb89b4f1640 -1 osd.11 86 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-12T03:06:40.666 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:40 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:40.439+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:16.844053+0000 front 2024-09-12T03:05:16.843858+0000 (oldest deadline 2024-09-12T03:05:42.644259+0000) 2024-09-12T03:06:40.666 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:40 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:40.439+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:22.644805+0000 front 2024-09-12T03:05:22.644911+0000 (oldest deadline 2024-09-12T03:05:47.344653+0000) 2024-09-12T03:06:40.666 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:40 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:40.439+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:31.846594+0000 front 2024-09-12T03:05:31.846677+0000 (oldest deadline 2024-09-12T03:05:59.447156+0000) 2024-09-12T03:06:40.666 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:40 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:40.439+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-12T03:05:39.447493+0000 front 2024-09-12T03:05:39.447609+0000 (oldest deadline 2024-09-12T03:06:03.547590+0000) 2024-09-12T03:06:40.666 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:40 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:40.439+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-09-12T03:05:48.248794+0000 front 2024-09-12T03:05:48.248779+0000 (oldest deadline 2024-09-12T03:06:12.348647+0000) 2024-09-12T03:06:40.666 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:40 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:40.439+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.148:6812 osd.5 since back 2024-09-12T03:05:55.849475+0000 front 2024-09-12T03:05:55.849548+0000 (oldest deadline 2024-09-12T03:06:21.149634+0000) 2024-09-12T03:06:40.666 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:40 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:40.439+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.146:6820 osd.7 since back 2024-09-12T03:06:12.351187+0000 front 2024-09-12T03:06:12.351311+0000 (oldest deadline 2024-09-12T03:06:38.251021+0000) 2024-09-12T03:06:41.666 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:41 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:41.404+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:16.844053+0000 front 2024-09-12T03:05:16.843858+0000 (oldest deadline 2024-09-12T03:05:42.644259+0000) 2024-09-12T03:06:41.666 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:41 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:41.404+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:22.644805+0000 front 2024-09-12T03:05:22.644911+0000 (oldest deadline 2024-09-12T03:05:47.344653+0000) 2024-09-12T03:06:41.666 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:41 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:41.404+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:31.846594+0000 front 2024-09-12T03:05:31.846677+0000 (oldest deadline 2024-09-12T03:05:59.447156+0000) 2024-09-12T03:06:41.666 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:41 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:41.404+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-12T03:05:39.447493+0000 front 2024-09-12T03:05:39.447609+0000 (oldest deadline 2024-09-12T03:06:03.547590+0000) 2024-09-12T03:06:41.666 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:41 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:41.404+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-09-12T03:05:48.248794+0000 front 2024-09-12T03:05:48.248779+0000 (oldest deadline 2024-09-12T03:06:12.348647+0000) 2024-09-12T03:06:41.666 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:41 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:41.404+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.148:6812 osd.5 since back 2024-09-12T03:05:55.849475+0000 front 2024-09-12T03:05:55.849548+0000 (oldest deadline 2024-09-12T03:06:21.149634+0000) 2024-09-12T03:06:41.666 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:41 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:41.404+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.146:6820 osd.7 since back 2024-09-12T03:06:12.351187+0000 front 2024-09-12T03:06:12.351311+0000 (oldest deadline 2024-09-12T03:06:38.251021+0000) 2024-09-12T03:06:42.665 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:42 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:42.371+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:16.844053+0000 front 2024-09-12T03:05:16.843858+0000 (oldest deadline 2024-09-12T03:05:42.644259+0000) 2024-09-12T03:06:42.666 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:42 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:42.371+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:22.644805+0000 front 2024-09-12T03:05:22.644911+0000 (oldest deadline 2024-09-12T03:05:47.344653+0000) 2024-09-12T03:06:42.666 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:42 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:42.371+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:31.846594+0000 front 2024-09-12T03:05:31.846677+0000 (oldest deadline 2024-09-12T03:05:59.447156+0000) 2024-09-12T03:06:42.666 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:42 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:42.371+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-12T03:05:39.447493+0000 front 2024-09-12T03:05:39.447609+0000 (oldest deadline 2024-09-12T03:06:03.547590+0000) 2024-09-12T03:06:42.666 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:42 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:42.371+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-09-12T03:05:48.248794+0000 front 2024-09-12T03:05:48.248779+0000 (oldest deadline 2024-09-12T03:06:12.348647+0000) 2024-09-12T03:06:42.666 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:42 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:42.371+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.148:6812 osd.5 since back 2024-09-12T03:05:55.849475+0000 front 2024-09-12T03:05:55.849548+0000 (oldest deadline 2024-09-12T03:06:21.149634+0000) 2024-09-12T03:06:42.666 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:42 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:42.371+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.146:6820 osd.7 since back 2024-09-12T03:06:12.351187+0000 front 2024-09-12T03:06:12.351311+0000 (oldest deadline 2024-09-12T03:06:38.251021+0000) 2024-09-12T03:06:43.666 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:43 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:43.375+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:16.844053+0000 front 2024-09-12T03:05:16.843858+0000 (oldest deadline 2024-09-12T03:05:42.644259+0000) 2024-09-12T03:06:43.666 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:43 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:43.375+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:22.644805+0000 front 2024-09-12T03:05:22.644911+0000 (oldest deadline 2024-09-12T03:05:47.344653+0000) 2024-09-12T03:06:43.666 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:43 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:43.375+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:31.846594+0000 front 2024-09-12T03:05:31.846677+0000 (oldest deadline 2024-09-12T03:05:59.447156+0000) 2024-09-12T03:06:43.666 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:43 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:43.375+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-12T03:05:39.447493+0000 front 2024-09-12T03:05:39.447609+0000 (oldest deadline 2024-09-12T03:06:03.547590+0000) 2024-09-12T03:06:43.666 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:43 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:43.375+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-09-12T03:05:48.248794+0000 front 2024-09-12T03:05:48.248779+0000 (oldest deadline 2024-09-12T03:06:12.348647+0000) 2024-09-12T03:06:43.666 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:43 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:43.375+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.148:6812 osd.5 since back 2024-09-12T03:05:55.849475+0000 front 2024-09-12T03:05:55.849548+0000 (oldest deadline 2024-09-12T03:06:21.149634+0000) 2024-09-12T03:06:43.666 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:43 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:43.375+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.146:6820 osd.7 since back 2024-09-12T03:06:12.351187+0000 front 2024-09-12T03:06:12.351311+0000 (oldest deadline 2024-09-12T03:06:38.251021+0000) 2024-09-12T03:06:44.665 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:44 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:44.339+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2024-09-12T03:05:16.844053+0000 front 2024-09-12T03:05:16.843858+0000 (oldest deadline 2024-09-12T03:05:42.644259+0000) 2024-09-12T03:06:44.666 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:44 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:44.339+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.146:6804 osd.1 since back 2024-09-12T03:05:22.644805+0000 front 2024-09-12T03:05:22.644911+0000 (oldest deadline 2024-09-12T03:05:47.344653+0000) 2024-09-12T03:06:44.666 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:44 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:44.339+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.148:6804 osd.2 since back 2024-09-12T03:05:31.846594+0000 front 2024-09-12T03:05:31.846677+0000 (oldest deadline 2024-09-12T03:05:59.447156+0000) 2024-09-12T03:06:44.666 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:44 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:44.339+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2024-09-12T03:05:39.447493+0000 front 2024-09-12T03:05:39.447609+0000 (oldest deadline 2024-09-12T03:06:03.547590+0000) 2024-09-12T03:06:44.666 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:44 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:44.339+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.146:6812 osd.4 since back 2024-09-12T03:05:48.248794+0000 front 2024-09-12T03:05:48.248779+0000 (oldest deadline 2024-09-12T03:06:12.348647+0000) 2024-09-12T03:06:44.666 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:44 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:44.339+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.148:6812 osd.5 since back 2024-09-12T03:05:55.849475+0000 front 2024-09-12T03:05:55.849548+0000 (oldest deadline 2024-09-12T03:06:21.149634+0000) 2024-09-12T03:06:44.666 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:44 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:44.339+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.146:6820 osd.7 since back 2024-09-12T03:06:12.351187+0000 front 2024-09-12T03:06:12.351311+0000 (oldest deadline 2024-09-12T03:06:38.251021+0000) 2024-09-12T03:06:44.666 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:44 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11[60361]: 2024-09-12T03:06:44.339+0000 7fb893aee640 -1 osd.11 86 heartbeat_check: no reply from 172.21.15.148:6820 osd.8 since back 2024-09-12T03:06:21.151753+0000 front 2024-09-12T03:06:21.151777+0000 (oldest deadline 2024-09-12T03:06:43.451752+0000) 2024-09-12T03:06:45.915 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:45 smithi148 podman[69809]: 2024-09-12 03:06:45.660083991 +0000 UTC m=+5.539778565 container died 26d68867ac1769b7efd0f947cdabaada7cb840fef15a7ecd24fbde745a055394 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3) 2024-09-12T03:06:46.229 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:46 smithi148 podman[69809]: 2024-09-12 03:06:46.011855956 +0000 UTC m=+5.891550521 container cleanup 26d68867ac1769b7efd0f947cdabaada7cb840fef15a7ecd24fbde745a055394 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-12T03:06:46.229 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:46 smithi148 bash[69809]: ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11 2024-09-12T03:06:46.663 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:46 smithi148 podman[69840]: 2024-09-12 03:06:46.229745313 +0000 UTC m=+0.559144573 container remove 26d68867ac1769b7efd0f947cdabaada7cb840fef15a7ecd24fbde745a055394 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-12T03:06:46.915 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:46 smithi148 podman[69929]: 2024-09-12 03:06:46.663738482 +0000 UTC m=+0.108040999 container create 2a5c0709552170d68ced90b8aba2f6fa2d45b3ee65782f78c060a78dea122af6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11-deactivate, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2) 2024-09-12T03:06:46.915 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:46 smithi148 podman[69929]: 2024-09-12 03:06:46.571793391 +0000 UTC m=+0.016095921 image pull 54467abc55f864c4f73dfaf9c6ac3bb3e87b773123d5badaf790bee57321d594 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66 2024-09-12T03:06:46.915 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:46 smithi148 podman[69929]: 2024-09-12 03:06:46.916069981 +0000 UTC m=+0.360372499 container init 2a5c0709552170d68ced90b8aba2f6fa2d45b3ee65782f78c060a78dea122af6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-12T03:06:47.166 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:46 smithi148 podman[69929]: 2024-09-12 03:06:46.919323012 +0000 UTC m=+0.363625539 container start 2a5c0709552170d68ced90b8aba2f6fa2d45b3ee65782f78c060a78dea122af6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11-deactivate, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3) 2024-09-12T03:06:47.166 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:46 smithi148 podman[69929]: 2024-09-12 03:06:46.990962555 +0000 UTC m=+0.435265085 container attach 2a5c0709552170d68ced90b8aba2f6fa2d45b3ee65782f78c060a78dea122af6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11-deactivate, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T03:06:47.166 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:47 smithi148 podman[69929]: 2024-09-12 03:06:47.108965772 +0000 UTC m=+0.553268304 container died 2a5c0709552170d68ced90b8aba2f6fa2d45b3ee65782f78c060a78dea122af6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11-deactivate, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T03:06:47.632 DEBUG:teuthology.orchestra.run.smithi148:> sudo pkill -f 'journalctl -f -n 0 -u ceph-618305f8-70b1-11ef-bcea-c7b262605968@osd.11.service' 2024-09-12T03:06:47.655 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:47 smithi148 podman[69952]: 2024-09-12 03:06:47.602551258 +0000 UTC m=+0.487586385 container remove 2a5c0709552170d68ced90b8aba2f6fa2d45b3ee65782f78c060a78dea122af6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-osd-11-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, ceph=True, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-09-12T03:06:47.655 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:47 smithi148 systemd[1]: ceph-618305f8-70b1-11ef-bcea-c7b262605968@osd.11.service: Deactivated successfully. 2024-09-12T03:06:47.655 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:47 smithi148 systemd[1]: Stopped Ceph osd.11 for 618305f8-70b1-11ef-bcea-c7b262605968. 2024-09-12T03:06:47.656 INFO:journalctl@ceph.osd.11.smithi148.stdout:Sep 12 03:06:47 smithi148 systemd[1]: ceph-618305f8-70b1-11ef-bcea-c7b262605968@osd.11.service: Consumed 27.080s CPU time. 2024-09-12T03:06:47.670 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:06:47.670 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-09-12T03:06:47.670 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-09-12T03:06:47.671 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ceph-618305f8-70b1-11ef-bcea-c7b262605968@mds.a 2024-09-12T03:06:48.003 INFO:journalctl@ceph.mds.a.smithi005.stdout:Sep 12 03:06:47 smithi005 systemd[1]: Stopping Ceph mds.a for 618305f8-70b1-11ef-bcea-c7b262605968... 2024-09-12T03:06:48.310 INFO:journalctl@ceph.mds.a.smithi005.stdout:Sep 12 03:06:48 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-a[69981]: 2024-09-12T03:06:48.003+0000 7f782de23640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:06:48.310 INFO:journalctl@ceph.mds.a.smithi005.stdout:Sep 12 03:06:48 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-a[69981]: 2024-09-12T03:06:48.003+0000 7f782de23640 -1 mds.a *** got signal Terminated *** 2024-09-12T03:06:51.486 INFO:journalctl@ceph.mds.a.smithi005.stdout:Sep 12 03:06:51 smithi005 podman[119041]: 2024-09-12 03:06:51.216482108 +0000 UTC m=+3.328312241 container died be62dcbdf775072357f2fc3ef613231786d44dff6d2f9686dcd9296e7f68670d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-a, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T03:06:51.810 INFO:journalctl@ceph.mds.a.smithi005.stdout:Sep 12 03:06:51 smithi005 podman[119041]: 2024-09-12 03:06:51.694179096 +0000 UTC m=+3.806009232 container cleanup be62dcbdf775072357f2fc3ef613231786d44dff6d2f9686dcd9296e7f68670d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-a, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=squid-b53b6d3, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T03:06:51.810 INFO:journalctl@ceph.mds.a.smithi005.stdout:Sep 12 03:06:51 smithi005 bash[119041]: ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-a 2024-09-12T03:06:52.076 DEBUG:teuthology.orchestra.run.smithi005:> sudo pkill -f 'journalctl -f -n 0 -u ceph-618305f8-70b1-11ef-bcea-c7b262605968@mds.a.service' 2024-09-12T03:06:52.112 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:06:52.112 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-09-12T03:06:52.112 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-09-12T03:06:52.112 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ceph-618305f8-70b1-11ef-bcea-c7b262605968@mds.d 2024-09-12T03:06:52.560 INFO:journalctl@ceph.mds.d.smithi005.stdout:Sep 12 03:06:52 smithi005 systemd[1]: Stopping Ceph mds.d for 618305f8-70b1-11ef-bcea-c7b262605968... 2024-09-12T03:06:52.560 INFO:journalctl@ceph.mds.d.smithi005.stdout:Sep 12 03:06:52 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-d[68785]: 2024-09-12T03:06:52.422+0000 7fb30e3ba640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:06:52.560 INFO:journalctl@ceph.mds.d.smithi005.stdout:Sep 12 03:06:52 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-d[68785]: 2024-09-12T03:06:52.422+0000 7fb30e3ba640 -1 mds.d *** got signal Terminated *** 2024-09-12T03:06:53.739 INFO:journalctl@ceph.mds.d.smithi005.stdout:Sep 12 03:06:53 smithi005 podman[119164]: 2024-09-12 03:06:53.416415908 +0000 UTC m=+1.079339058 container died eaf2f38083f94da41b0fc877099d5d01d3241eca608a29e3ea6fe088949a8004 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-d, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2) 2024-09-12T03:06:53.739 INFO:journalctl@ceph.mds.d.smithi005.stdout:Sep 12 03:06:53 smithi005 podman[119164]: 2024-09-12 03:06:53.739718336 +0000 UTC m=+1.402641486 container cleanup eaf2f38083f94da41b0fc877099d5d01d3241eca608a29e3ea6fe088949a8004 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-d, RELEASE=squid-b53b6d3, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T03:06:54.060 INFO:journalctl@ceph.mds.d.smithi005.stdout:Sep 12 03:06:53 smithi005 bash[119164]: ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-d 2024-09-12T03:06:54.060 INFO:journalctl@ceph.mds.d.smithi005.stdout:Sep 12 03:06:53 smithi005 podman[119176]: 2024-09-12 03:06:53.890828076 +0000 UTC m=+0.467165859 container remove eaf2f38083f94da41b0fc877099d5d01d3241eca608a29e3ea6fe088949a8004 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-d, CEPH_POINT_RELEASE=, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-12T03:06:54.123 DEBUG:teuthology.orchestra.run.smithi005:> sudo pkill -f 'journalctl -f -n 0 -u ceph-618305f8-70b1-11ef-bcea-c7b262605968@mds.d.service' 2024-09-12T03:06:54.160 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:06:54.160 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-09-12T03:06:54.160 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-09-12T03:06:54.160 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ceph-618305f8-70b1-11ef-bcea-c7b262605968@mds.g 2024-09-12T03:06:54.485 INFO:journalctl@ceph.mds.g.smithi005.stdout:Sep 12 03:06:54 smithi005 systemd[1]: Stopping Ceph mds.g for 618305f8-70b1-11ef-bcea-c7b262605968... 2024-09-12T03:06:54.810 INFO:journalctl@ceph.mds.g.smithi005.stdout:Sep 12 03:06:54 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-g[69561]: 2024-09-12T03:06:54.485+0000 7f09a33f3640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:06:54.810 INFO:journalctl@ceph.mds.g.smithi005.stdout:Sep 12 03:06:54 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-g[69561]: 2024-09-12T03:06:54.485+0000 7f09a33f3640 -1 mds.g *** got signal Terminated *** 2024-09-12T03:06:58.768 INFO:journalctl@ceph.mds.g.smithi005.stdout:Sep 12 03:06:58 smithi005 podman[119286]: 2024-09-12 03:06:58.497699536 +0000 UTC m=+4.114982376 container died f37068bb74e3ae7bfbbd6738d4d33707d2ad5f23fa2da53d9af3a8ebd4f5d212 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-g, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-12T03:06:59.060 INFO:journalctl@ceph.mds.g.smithi005.stdout:Sep 12 03:06:58 smithi005 podman[119286]: 2024-09-12 03:06:58.968288941 +0000 UTC m=+4.585571814 container cleanup f37068bb74e3ae7bfbbd6738d4d33707d2ad5f23fa2da53d9af3a8ebd4f5d212 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-g, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-b53b6d3) 2024-09-12T03:06:59.060 INFO:journalctl@ceph.mds.g.smithi005.stdout:Sep 12 03:06:58 smithi005 bash[119286]: ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-g 2024-09-12T03:06:59.457 DEBUG:teuthology.orchestra.run.smithi005:> sudo pkill -f 'journalctl -f -n 0 -u ceph-618305f8-70b1-11ef-bcea-c7b262605968@mds.g.service' 2024-09-12T03:06:59.490 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:06:59.491 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-09-12T03:06:59.491 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-09-12T03:06:59.491 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ceph-618305f8-70b1-11ef-bcea-c7b262605968@mds.j 2024-09-12T03:06:59.810 INFO:journalctl@ceph.mds.j.smithi005.stdout:Sep 12 03:06:59 smithi005 systemd[1]: Stopping Ceph mds.j for 618305f8-70b1-11ef-bcea-c7b262605968... 2024-09-12T03:06:59.810 INFO:journalctl@ceph.mds.j.smithi005.stdout:Sep 12 03:06:59 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-j[68372]: 2024-09-12T03:06:59.797+0000 7f86eda55640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:06:59.810 INFO:journalctl@ceph.mds.j.smithi005.stdout:Sep 12 03:06:59 smithi005 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-j[68372]: 2024-09-12T03:06:59.797+0000 7f86eda55640 -1 mds.j *** got signal Terminated *** 2024-09-12T03:07:02.147 INFO:journalctl@ceph.mds.j.smithi005.stdout:Sep 12 03:07:01 smithi005 podman[119406]: 2024-09-12 03:07:01.881406972 +0000 UTC m=+2.166908285 container died 6307c2f6d51ab2a63267ac61623dbc66702827b4b88ea1a1882930c4d0fda744 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-j, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-12T03:07:02.560 INFO:journalctl@ceph.mds.j.smithi005.stdout:Sep 12 03:07:02 smithi005 podman[119406]: 2024-09-12 03:07:02.331041751 +0000 UTC m=+2.616543068 container cleanup 6307c2f6d51ab2a63267ac61623dbc66702827b4b88ea1a1882930c4d0fda744 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-j, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-12T03:07:02.560 INFO:journalctl@ceph.mds.j.smithi005.stdout:Sep 12 03:07:02 smithi005 bash[119406]: ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-j 2024-09-12T03:07:02.765 DEBUG:teuthology.orchestra.run.smithi005:> sudo pkill -f 'journalctl -f -n 0 -u ceph-618305f8-70b1-11ef-bcea-c7b262605968@mds.j.service' 2024-09-12T03:07:02.799 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:07:02.799 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-09-12T03:07:02.799 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-09-12T03:07:02.799 DEBUG:teuthology.orchestra.run.smithi146:> sudo systemctl stop ceph-618305f8-70b1-11ef-bcea-c7b262605968@mds.b 2024-09-12T03:07:03.193 INFO:journalctl@ceph.mds.b.smithi146.stdout:Sep 12 03:07:02 smithi146 systemd[1]: Stopping Ceph mds.b for 618305f8-70b1-11ef-bcea-c7b262605968... 2024-09-12T03:07:03.456 INFO:journalctl@ceph.mds.b.smithi146.stdout:Sep 12 03:07:03 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-b[65422]: 2024-09-12T03:07:03.192+0000 7f63746b0640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:07:03.456 INFO:journalctl@ceph.mds.b.smithi146.stdout:Sep 12 03:07:03 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-b[65422]: 2024-09-12T03:07:03.192+0000 7f63746b0640 -1 mds.b *** got signal Terminated *** 2024-09-12T03:07:07.456 INFO:journalctl@ceph.mds.b.smithi146.stdout:Sep 12 03:07:07 smithi146 podman[70005]: 2024-09-12 03:07:07.199900602 +0000 UTC m=+4.203050611 container died 4dd5146b28909eb32e1a24aa1f94938941a23fdf93fede8fbc2f721f733284d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-b, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-12T03:07:07.834 INFO:journalctl@ceph.mds.b.smithi146.stdout:Sep 12 03:07:07 smithi146 podman[70005]: 2024-09-12 03:07:07.642223755 +0000 UTC m=+4.645373764 container cleanup 4dd5146b28909eb32e1a24aa1f94938941a23fdf93fede8fbc2f721f733284d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-b, RELEASE=squid-b53b6d3, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD) 2024-09-12T03:07:07.835 INFO:journalctl@ceph.mds.b.smithi146.stdout:Sep 12 03:07:07 smithi146 bash[70005]: ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-b 2024-09-12T03:07:08.067 DEBUG:teuthology.orchestra.run.smithi146:> sudo pkill -f 'journalctl -f -n 0 -u ceph-618305f8-70b1-11ef-bcea-c7b262605968@mds.b.service' 2024-09-12T03:07:08.089 INFO:journalctl@ceph.mds.b.smithi146.stdout:Sep 12 03:07:07 smithi146 podman[70029]: 2024-09-12 03:07:07.834770189 +0000 UTC m=+0.628369589 container remove 4dd5146b28909eb32e1a24aa1f94938941a23fdf93fede8fbc2f721f733284d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-b, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-12T03:07:08.089 INFO:journalctl@ceph.mds.b.smithi146.stdout:Sep 12 03:07:08 smithi146 systemd[1]: ceph-618305f8-70b1-11ef-bcea-c7b262605968@mds.b.service: Deactivated successfully. 2024-09-12T03:07:08.089 INFO:journalctl@ceph.mds.b.smithi146.stdout:Sep 12 03:07:08 smithi146 systemd[1]: Stopped Ceph mds.b for 618305f8-70b1-11ef-bcea-c7b262605968. 2024-09-12T03:07:08.089 INFO:journalctl@ceph.mds.b.smithi146.stdout:Sep 12 03:07:08 smithi146 systemd[1]: ceph-618305f8-70b1-11ef-bcea-c7b262605968@mds.b.service: Consumed 1.597s CPU time. 2024-09-12T03:07:08.103 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:07:08.103 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-09-12T03:07:08.103 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-09-12T03:07:08.103 DEBUG:teuthology.orchestra.run.smithi146:> sudo systemctl stop ceph-618305f8-70b1-11ef-bcea-c7b262605968@mds.e 2024-09-12T03:07:08.429 INFO:journalctl@ceph.mds.e.smithi146.stdout:Sep 12 03:07:08 smithi146 systemd[1]: Stopping Ceph mds.e for 618305f8-70b1-11ef-bcea-c7b262605968... 2024-09-12T03:07:08.706 INFO:journalctl@ceph.mds.e.smithi146.stdout:Sep 12 03:07:08 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-e[64819]: 2024-09-12T03:07:08.427+0000 7f1a58b16640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:07:08.706 INFO:journalctl@ceph.mds.e.smithi146.stdout:Sep 12 03:07:08 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-e[64819]: 2024-09-12T03:07:08.427+0000 7f1a58b16640 -1 mds.e *** got signal Terminated *** 2024-09-12T03:07:11.478 INFO:journalctl@ceph.mds.e.smithi146.stdout:Sep 12 03:07:11 smithi146 podman[70138]: 2024-09-12 03:07:11.223846339 +0000 UTC m=+2.899465507 container died c6bdd085a6c147772cfabf791383e536d0b2b29a41da50c7b7199d39284bffe1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-e, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2) 2024-09-12T03:07:11.920 INFO:journalctl@ceph.mds.e.smithi146.stdout:Sep 12 03:07:11 smithi146 podman[70138]: 2024-09-12 03:07:11.553556113 +0000 UTC m=+3.229175270 container cleanup c6bdd085a6c147772cfabf791383e536d0b2b29a41da50c7b7199d39284bffe1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-e, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-12T03:07:11.920 INFO:journalctl@ceph.mds.e.smithi146.stdout:Sep 12 03:07:11 smithi146 bash[70138]: ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-e 2024-09-12T03:07:11.920 INFO:journalctl@ceph.mds.e.smithi146.stdout:Sep 12 03:07:11 smithi146 podman[70150]: 2024-09-12 03:07:11.721315857 +0000 UTC m=+0.492480734 container remove c6bdd085a6c147772cfabf791383e536d0b2b29a41da50c7b7199d39284bffe1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-e, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-12T03:07:11.943 DEBUG:teuthology.orchestra.run.smithi146:> sudo pkill -f 'journalctl -f -n 0 -u ceph-618305f8-70b1-11ef-bcea-c7b262605968@mds.e.service' 2024-09-12T03:07:11.977 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:07:11.977 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-09-12T03:07:11.977 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-09-12T03:07:11.977 DEBUG:teuthology.orchestra.run.smithi146:> sudo systemctl stop ceph-618305f8-70b1-11ef-bcea-c7b262605968@mds.h 2024-09-12T03:07:12.206 INFO:journalctl@ceph.mds.h.smithi146.stdout:Sep 12 03:07:12 smithi146 systemd[1]: Stopping Ceph mds.h for 618305f8-70b1-11ef-bcea-c7b262605968... 2024-09-12T03:07:12.706 INFO:journalctl@ceph.mds.h.smithi146.stdout:Sep 12 03:07:12 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-h[65118]: 2024-09-12T03:07:12.272+0000 7f0b276a8640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:07:12.706 INFO:journalctl@ceph.mds.h.smithi146.stdout:Sep 12 03:07:12 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-h[65118]: 2024-09-12T03:07:12.272+0000 7f0b276a8640 -1 mds.h *** got signal Terminated *** 2024-09-12T03:07:15.490 INFO:journalctl@ceph.mds.h.smithi146.stdout:Sep 12 03:07:15 smithi146 podman[70260]: 2024-09-12 03:07:15.21352162 +0000 UTC m=+3.019312720 container died f42ddd7f5459b111551df6b5ece3a3733b65634a19260ea36e67af718728a8c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-h, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-b53b6d3, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T03:07:15.849 INFO:journalctl@ceph.mds.h.smithi146.stdout:Sep 12 03:07:15 smithi146 podman[70260]: 2024-09-12 03:07:15.582133669 +0000 UTC m=+3.387924765 container cleanup f42ddd7f5459b111551df6b5ece3a3733b65634a19260ea36e67af718728a8c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-h, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.37.2, ceph=True, RELEASE=squid-b53b6d3, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T03:07:15.849 INFO:journalctl@ceph.mds.h.smithi146.stdout:Sep 12 03:07:15 smithi146 bash[70260]: ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-h 2024-09-12T03:07:16.000 DEBUG:teuthology.orchestra.run.smithi146:> sudo pkill -f 'journalctl -f -n 0 -u ceph-618305f8-70b1-11ef-bcea-c7b262605968@mds.h.service' 2024-09-12T03:07:16.034 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:07:16.034 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-09-12T03:07:16.035 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-09-12T03:07:16.035 DEBUG:teuthology.orchestra.run.smithi146:> sudo systemctl stop ceph-618305f8-70b1-11ef-bcea-c7b262605968@mds.k 2024-09-12T03:07:16.456 INFO:journalctl@ceph.mds.k.smithi146.stdout:Sep 12 03:07:16 smithi146 systemd[1]: Stopping Ceph mds.k for 618305f8-70b1-11ef-bcea-c7b262605968... 2024-09-12T03:07:16.456 INFO:journalctl@ceph.mds.k.smithi146.stdout:Sep 12 03:07:16 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-k[64500]: 2024-09-12T03:07:16.351+0000 7fdcd8681640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:07:16.456 INFO:journalctl@ceph.mds.k.smithi146.stdout:Sep 12 03:07:16 smithi146 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-k[64500]: 2024-09-12T03:07:16.351+0000 7fdcd8681640 -1 mds.k *** got signal Terminated *** 2024-09-12T03:07:19.476 INFO:journalctl@ceph.mds.k.smithi146.stdout:Sep 12 03:07:19 smithi146 podman[70381]: 2024-09-12 03:07:19.212834915 +0000 UTC m=+2.953071919 container died 889f13ae82d140d2d4ffadb8fe1751d38a7eed5a018267a9133d2877e6df5242 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-k, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-12T03:07:19.852 INFO:journalctl@ceph.mds.k.smithi146.stdout:Sep 12 03:07:19 smithi146 podman[70381]: 2024-09-12 03:07:19.576532045 +0000 UTC m=+3.316769048 container cleanup 889f13ae82d140d2d4ffadb8fe1751d38a7eed5a018267a9133d2877e6df5242 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-k, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2) 2024-09-12T03:07:19.852 INFO:journalctl@ceph.mds.k.smithi146.stdout:Sep 12 03:07:19 smithi146 bash[70381]: ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-k 2024-09-12T03:07:20.001 DEBUG:teuthology.orchestra.run.smithi146:> sudo pkill -f 'journalctl -f -n 0 -u ceph-618305f8-70b1-11ef-bcea-c7b262605968@mds.k.service' 2024-09-12T03:07:20.033 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:07:20.034 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-09-12T03:07:20.034 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-09-12T03:07:20.034 DEBUG:teuthology.orchestra.run.smithi148:> sudo systemctl stop ceph-618305f8-70b1-11ef-bcea-c7b262605968@mds.c 2024-09-12T03:07:20.393 INFO:journalctl@ceph.mds.c.smithi148.stdout:Sep 12 03:07:20 smithi148 systemd[1]: Stopping Ceph mds.c for 618305f8-70b1-11ef-bcea-c7b262605968... 2024-09-12T03:07:20.665 INFO:journalctl@ceph.mds.c.smithi148.stdout:Sep 12 03:07:20 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-c[65251]: 2024-09-12T03:07:20.410+0000 7f77990bc640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:07:20.666 INFO:journalctl@ceph.mds.c.smithi148.stdout:Sep 12 03:07:20 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-c[65251]: 2024-09-12T03:07:20.410+0000 7f77990bc640 -1 mds.c *** got signal Terminated *** 2024-09-12T03:07:23.485 INFO:journalctl@ceph.mds.c.smithi148.stdout:Sep 12 03:07:23 smithi148 podman[70034]: 2024-09-12 03:07:23.215783884 +0000 UTC m=+2.982252468 container died 3d621945b9f5be9ecb22d3441728918a2fa1feda51217d12a96cd2b4a18c9c77 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-c, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-09-12T03:07:23.853 INFO:journalctl@ceph.mds.c.smithi148.stdout:Sep 12 03:07:23 smithi148 podman[70034]: 2024-09-12 03:07:23.57717758 +0000 UTC m=+3.343646165 container cleanup 3d621945b9f5be9ecb22d3441728918a2fa1feda51217d12a96cd2b4a18c9c77 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-c, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, RELEASE=squid-b53b6d3, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-12T03:07:23.853 INFO:journalctl@ceph.mds.c.smithi148.stdout:Sep 12 03:07:23 smithi148 bash[70034]: ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-c 2024-09-12T03:07:24.005 DEBUG:teuthology.orchestra.run.smithi148:> sudo pkill -f 'journalctl -f -n 0 -u ceph-618305f8-70b1-11ef-bcea-c7b262605968@mds.c.service' 2024-09-12T03:07:24.040 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:07:24.040 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-09-12T03:07:24.040 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-09-12T03:07:24.041 DEBUG:teuthology.orchestra.run.smithi148:> sudo systemctl stop ceph-618305f8-70b1-11ef-bcea-c7b262605968@mds.f 2024-09-12T03:07:24.416 INFO:journalctl@ceph.mds.f.smithi148.stdout:Sep 12 03:07:24 smithi148 systemd[1]: Stopping Ceph mds.f for 618305f8-70b1-11ef-bcea-c7b262605968... 2024-09-12T03:07:24.416 INFO:journalctl@ceph.mds.f.smithi148.stdout:Sep 12 03:07:24 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-f[64950]: 2024-09-12T03:07:24.364+0000 7fc9ad49b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:07:24.416 INFO:journalctl@ceph.mds.f.smithi148.stdout:Sep 12 03:07:24 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-f[64950]: 2024-09-12T03:07:24.364+0000 7fc9ad49b640 -1 mds.f *** got signal Terminated *** 2024-09-12T03:07:27.514 INFO:journalctl@ceph.mds.f.smithi148.stdout:Sep 12 03:07:27 smithi148 podman[70168]: 2024-09-12 03:07:27.225154939 +0000 UTC m=+2.962708734 container died 3cb01b1b23a3bdaf7aee42a146972760fb89f5d0c4c1a0035de2bf7d949dc385 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-f, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b53b6d3, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-12T03:07:27.839 INFO:journalctl@ceph.mds.f.smithi148.stdout:Sep 12 03:07:27 smithi148 podman[70168]: 2024-09-12 03:07:27.614027688 +0000 UTC m=+3.351581471 container cleanup 3cb01b1b23a3bdaf7aee42a146972760fb89f5d0c4c1a0035de2bf7d949dc385 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-f, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, RELEASE=squid-b53b6d3, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-12T03:07:27.839 INFO:journalctl@ceph.mds.f.smithi148.stdout:Sep 12 03:07:27 smithi148 bash[70168]: ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-f 2024-09-12T03:07:27.980 DEBUG:teuthology.orchestra.run.smithi148:> sudo pkill -f 'journalctl -f -n 0 -u ceph-618305f8-70b1-11ef-bcea-c7b262605968@mds.f.service' 2024-09-12T03:07:28.013 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:07:28.013 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-09-12T03:07:28.014 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-09-12T03:07:28.014 DEBUG:teuthology.orchestra.run.smithi148:> sudo systemctl stop ceph-618305f8-70b1-11ef-bcea-c7b262605968@mds.i 2024-09-12T03:07:28.415 INFO:journalctl@ceph.mds.i.smithi148.stdout:Sep 12 03:07:28 smithi148 systemd[1]: Stopping Ceph mds.i for 618305f8-70b1-11ef-bcea-c7b262605968... 2024-09-12T03:07:28.416 INFO:journalctl@ceph.mds.i.smithi148.stdout:Sep 12 03:07:28 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-i[65552]: 2024-09-12T03:07:28.334+0000 7fa2b1809640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:07:28.416 INFO:journalctl@ceph.mds.i.smithi148.stdout:Sep 12 03:07:28 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-i[65552]: 2024-09-12T03:07:28.334+0000 7fa2b1809640 -1 mds.i *** got signal Terminated *** 2024-09-12T03:07:31.484 INFO:journalctl@ceph.mds.i.smithi148.stdout:Sep 12 03:07:31 smithi148 podman[70288]: 2024-09-12 03:07:31.231875792 +0000 UTC m=+2.993316850 container died 6c536fc6aff28c9b9923d921b3efad4e43b4a170a735b3b1221c884fcb39c0db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-i, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-12T03:07:31.759 INFO:journalctl@ceph.mds.i.smithi148.stdout:Sep 12 03:07:31 smithi148 podman[70288]: 2024-09-12 03:07:31.575825617 +0000 UTC m=+3.337266671 container cleanup 6c536fc6aff28c9b9923d921b3efad4e43b4a170a735b3b1221c884fcb39c0db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-i, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-b53b6d3, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-09-12T03:07:31.759 INFO:journalctl@ceph.mds.i.smithi148.stdout:Sep 12 03:07:31 smithi148 bash[70288]: ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-i 2024-09-12T03:07:31.999 DEBUG:teuthology.orchestra.run.smithi148:> sudo pkill -f 'journalctl -f -n 0 -u ceph-618305f8-70b1-11ef-bcea-c7b262605968@mds.i.service' 2024-09-12T03:07:32.020 INFO:journalctl@ceph.mds.i.smithi148.stdout:Sep 12 03:07:31 smithi148 podman[70301]: 2024-09-12 03:07:31.760045316 +0000 UTC m=+0.526488280 container remove 6c536fc6aff28c9b9923d921b3efad4e43b4a170a735b3b1221c884fcb39c0db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-i, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-12T03:07:32.020 INFO:journalctl@ceph.mds.i.smithi148.stdout:Sep 12 03:07:31 smithi148 systemd[1]: ceph-618305f8-70b1-11ef-bcea-c7b262605968@mds.i.service: Deactivated successfully. 2024-09-12T03:07:32.020 INFO:journalctl@ceph.mds.i.smithi148.stdout:Sep 12 03:07:31 smithi148 systemd[1]: Stopped Ceph mds.i for 618305f8-70b1-11ef-bcea-c7b262605968. 2024-09-12T03:07:32.020 INFO:journalctl@ceph.mds.i.smithi148.stdout:Sep 12 03:07:31 smithi148 systemd[1]: ceph-618305f8-70b1-11ef-bcea-c7b262605968@mds.i.service: Consumed 1min 57.626s CPU time. 2024-09-12T03:07:32.032 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:07:32.032 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-09-12T03:07:32.033 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-09-12T03:07:32.033 DEBUG:teuthology.orchestra.run.smithi148:> sudo systemctl stop ceph-618305f8-70b1-11ef-bcea-c7b262605968@mds.l 2024-09-12T03:07:32.378 INFO:journalctl@ceph.mds.l.smithi148.stdout:Sep 12 03:07:32 smithi148 systemd[1]: Stopping Ceph mds.l for 618305f8-70b1-11ef-bcea-c7b262605968... 2024-09-12T03:07:32.665 INFO:journalctl@ceph.mds.l.smithi148.stdout:Sep 12 03:07:32 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-l[64649]: 2024-09-12T03:07:32.378+0000 7f2a06105640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-12T03:07:32.665 INFO:journalctl@ceph.mds.l.smithi148.stdout:Sep 12 03:07:32 smithi148 ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-l[64649]: 2024-09-12T03:07:32.378+0000 7f2a06105640 -1 mds.l *** got signal Terminated *** 2024-09-12T03:07:35.487 INFO:journalctl@ceph.mds.l.smithi148.stdout:Sep 12 03:07:35 smithi148 podman[70411]: 2024-09-12 03:07:35.215627473 +0000 UTC m=+2.957298698 container died d3975d960a316984d08fff93587579a89f37aa124d7cc4c1116f469efcd4654c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-l, RELEASE=squid-b53b6d3, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0) 2024-09-12T03:07:35.846 INFO:journalctl@ceph.mds.l.smithi148.stdout:Sep 12 03:07:35 smithi148 podman[70411]: 2024-09-12 03:07:35.587693205 +0000 UTC m=+3.329364493 container cleanup d3975d960a316984d08fff93587579a89f37aa124d7cc4c1116f469efcd4654c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3be30d720e37ec4df7f667e60f22b4d221c4ce7b408e68b90f9ac10220535e66, name=ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-l, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, RELEASE=squid-b53b6d3) 2024-09-12T03:07:35.846 INFO:journalctl@ceph.mds.l.smithi148.stdout:Sep 12 03:07:35 smithi148 bash[70411]: ceph-618305f8-70b1-11ef-bcea-c7b262605968-mds-l 2024-09-12T03:07:36.008 DEBUG:teuthology.orchestra.run.smithi148:> sudo pkill -f 'journalctl -f -n 0 -u ceph-618305f8-70b1-11ef-bcea-c7b262605968@mds.l.service' 2024-09-12T03:07:36.040 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-12T03:07:36.040 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-09-12T03:07:36.040 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 618305f8-70b1-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-12T03:07:36.253 INFO:teuthology.orchestra.run.smithi005.stdout:Deleting cluster with fsid: 618305f8-70b1-11ef-bcea-c7b262605968 2024-09-12T03:07:38.862 DEBUG:teuthology.orchestra.run.smithi146:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 618305f8-70b1-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-12T03:07:39.068 INFO:teuthology.orchestra.run.smithi146.stdout:Deleting cluster with fsid: 618305f8-70b1-11ef-bcea-c7b262605968 2024-09-12T03:07:41.481 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 618305f8-70b1-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-12T03:07:41.684 INFO:teuthology.orchestra.run.smithi148.stdout:Deleting cluster with fsid: 618305f8-70b1-11ef-bcea-c7b262605968 2024-09-12T03:07:44.093 DEBUG:teuthology.orchestra.run.smithi005:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-12T03:07:44.128 DEBUG:teuthology.orchestra.run.smithi146:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-12T03:07:44.161 DEBUG:teuthology.orchestra.run.smithi148:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-12T03:07:44.189 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-12T03:07:44.191 DEBUG:teuthology.misc:Transferring archived files from smithi005:/var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901638/remote/smithi005/crash 2024-09-12T03:07:44.192 DEBUG:teuthology.orchestra.run.smithi005:> sudo tar c -f - -C /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/crash -- . 2024-09-12T03:07:44.233 INFO:teuthology.orchestra.run.smithi005.stderr:tar: /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-12T03:07:44.233 INFO:teuthology.orchestra.run.smithi005.stderr:tar: Error is not recoverable: exiting now 2024-09-12T03:07:44.235 DEBUG:teuthology.misc:Transferring archived files from smithi146:/var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901638/remote/smithi146/crash 2024-09-12T03:07:44.236 DEBUG:teuthology.orchestra.run.smithi146:> sudo tar c -f - -C /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/crash -- . 2024-09-12T03:07:44.330 INFO:teuthology.orchestra.run.smithi146.stderr:tar: /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-12T03:07:44.330 INFO:teuthology.orchestra.run.smithi146.stderr:tar: Error is not recoverable: exiting now 2024-09-12T03:07:44.332 DEBUG:teuthology.misc:Transferring archived files from smithi148:/var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901638/remote/smithi148/crash 2024-09-12T03:07:44.333 DEBUG:teuthology.orchestra.run.smithi148:> sudo tar c -f - -C /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/crash -- . 2024-09-12T03:07:44.373 INFO:teuthology.orchestra.run.smithi148.stderr:tar: /var/lib/ceph/618305f8-70b1-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-12T03:07:44.373 INFO:teuthology.orchestra.run.smithi148.stderr:tar: Error is not recoverable: exiting now 2024-09-12T03:07:44.374 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-12T03:07:44.375 DEBUG:teuthology.orchestra.run.smithi005:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-09-12T03:07:44.429 INFO:tasks.cephadm:Compressing logs... 2024-09-12T03:07:44.429 DEBUG:teuthology.orchestra.run.smithi005:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-12T03:07:44.432 DEBUG:teuthology.orchestra.run.smithi146:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-12T03:07:44.435 DEBUG:teuthology.orchestra.run.smithi148:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-12T03:07:44.457 INFO:teuthology.orchestra.run.smithi005.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph-osd.0.log 2024-09-12T03:07:44.457 INFO:teuthology.orchestra.run.smithi005.stderr:: No such file or directory 2024-09-12T03:07:44.457 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph-osd.9.log 2024-09-12T03:07:44.457 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-12T03:07:44.459 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-12T03:07:44.459 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph-osd.9.log: /var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph-mds.a.log 2024-09-12T03:07:44.459 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-12T03:07:44.459 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph.audit.log: /var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph-mon.a.log 2024-09-12T03:07:44.459 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph-mds.g.log 2024-09-12T03:07:44.460 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph-mon.a.log: 82.7% -- replaced with /var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-12T03:07:44.460 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph-osd.6.log 2024-09-12T03:07:44.466 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph-mds.g.log: /var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph-osd.6.log: 90.2% -- replaced with /var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-12T03:07:44.466 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph.log 2024-09-12T03:07:44.466 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph-mds.d.log 2024-09-12T03:07:44.468 INFO:teuthology.orchestra.run.smithi148.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-12T03:07:44.468 INFO:teuthology.orchestra.run.smithi148.stderr:: No such file or directory 2024-09-12T03:07:44.468 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-12T03:07:44.468 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph-mds.i.log 2024-09-12T03:07:44.468 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph-mon.c.log 2024-09-12T03:07:44.469 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph.audit.log: /var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-12T03:07:44.469 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph-osd.11.log 2024-09-12T03:07:44.469 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph-mon.c.log: /var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph-osd.2.log 2024-09-12T03:07:44.469 INFO:teuthology.orchestra.run.smithi148.stderr: 82.6% -- replaced with /var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-12T03:07:44.469 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph-osd.5.log 2024-09-12T03:07:44.469 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph.log 2024-09-12T03:07:44.469 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph-mds.l.log 2024-09-12T03:07:44.469 INFO:teuthology.orchestra.run.smithi148.stderr: 90.5% -- replaced with /var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-12T03:07:44.469 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph-osd.8.log 2024-09-12T03:07:44.469 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph-mds.l.log: 86.3% -- replaced with /var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-12T03:07:44.469 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph-mgr.z.log 2024-09-12T03:07:44.472 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph.log: 86.3% -- replaced with /var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-12T03:07:44.473 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph-mgr.x.log 2024-09-12T03:07:44.473 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph-mds.d.log: 94.7% -- replaced with /var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph-mds.d.log.gz 2024-09-12T03:07:44.473 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph-mds.j.log 2024-09-12T03:07:44.473 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph-mgr.x.log: 90.9% -- replaced with /var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-12T03:07:44.474 INFO:teuthology.orchestra.run.smithi146.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph-osd.10.log 2024-09-12T03:07:44.474 INFO:teuthology.orchestra.run.smithi146.stderr:: No such file or directory 2024-09-12T03:07:44.474 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph-osd.1.log 2024-09-12T03:07:44.474 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph-osd.7.log 2024-09-12T03:07:44.474 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-12T03:07:44.474 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph-mds.b.log 2024-09-12T03:07:44.474 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph-mds.k.log 2024-09-12T03:07:44.474 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph-volume.log: /var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph-osd.4.log 2024-09-12T03:07:44.475 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-12T03:07:44.475 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-12T03:07:44.475 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph.audit.log: 90.4% -- replaced with /var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-12T03:07:44.475 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph-mgr.y.log 2024-09-12T03:07:44.475 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph.cephadm.log: 82.6% -- replaced with /var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-12T03:07:44.475 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph.log 2024-09-12T03:07:44.476 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph-osd.8.log: 91.0% -- replaced with /var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-12T03:07:44.478 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph-mgr.y.log: 92.5% -- replaced with /var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph-mgr.y.log.gz 2024-09-12T03:07:44.478 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph-mds.h.log 2024-09-12T03:07:44.478 INFO:teuthology.orchestra.run.smithi146.stderr: 91.0% -- replaced with /var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-12T03:07:44.478 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph-mon.b.log 2024-09-12T03:07:44.479 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph-mds.h.log: 86.3% -- replaced with /var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-12T03:07:44.484 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph-osd.3.log 2024-09-12T03:07:44.486 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph-mds.c.log 2024-09-12T03:07:44.487 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph-mds.j.log: 94.6% -- replaced with /var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph-mds.j.log.gz 2024-09-12T03:07:44.489 INFO:teuthology.orchestra.run.smithi146.stderr:gzip -5 --verbose -- /var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph-mds.e.log 2024-09-12T03:07:44.490 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-12T03:07:44.492 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph-mgr.z.log: 92.6% -- replaced with /var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph-mgr.z.log.gz 2024-09-12T03:07:44.499 INFO:teuthology.orchestra.run.smithi148.stderr:gzip -5 --verbose -- /var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph-mds.f.log 2024-09-12T03:07:44.502 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-12T03:07:44.511 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph-osd.3.log: /var/log/ceph/cephadm.log: 94.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-12T03:07:44.511 INFO:teuthology.orchestra.run.smithi005.stderr: 94.1% -- replaced with /var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph-mds.a.log.gz 2024-09-12T03:07:44.512 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-12T03:07:44.514 INFO:teuthology.orchestra.run.smithi005.stderr: 93.6% -- replaced with /var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph-mds.g.log.gz 2024-09-12T03:07:44.516 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph-mds.f.log: 94.0% -- replaced with /var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph-mds.l.log.gz 2024-09-12T03:07:44.518 INFO:teuthology.orchestra.run.smithi148.stderr:/var/log/ceph/cephadm.log: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-12T03:07:44.519 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph-mds.e.log: 93.8% -- replaced with /var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph-mds.k.log.gz 2024-09-12T03:07:44.521 INFO:teuthology.orchestra.run.smithi146.stderr:/var/log/ceph/cephadm.log: 91.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-12T03:07:44.529 INFO:teuthology.orchestra.run.smithi146.stderr: 92.5% -- replaced with /var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph-mds.b.log.gz 2024-09-12T03:07:44.608 INFO:teuthology.orchestra.run.smithi148.stderr: 92.7% -- replaced with /var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph-mds.c.log.gz 2024-09-12T03:07:44.617 INFO:teuthology.orchestra.run.smithi148.stderr: 93.0% -- replaced with /var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph-mds.f.log.gz 2024-09-12T03:07:44.648 INFO:teuthology.orchestra.run.smithi146.stderr: 93.1% -- replaced with /var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph-mds.h.log.gz 2024-09-12T03:07:44.722 INFO:teuthology.orchestra.run.smithi005.stderr: 90.2% -- replaced with /var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph-mgr.x.log.gz 2024-09-12T03:07:44.750 INFO:teuthology.orchestra.run.smithi148.stderr: 91.7% -- replaced with /var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph-mon.c.log.gz 2024-09-12T03:07:44.810 INFO:teuthology.orchestra.run.smithi146.stderr: 91.7% -- replaced with /var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph-mon.b.log.gz 2024-09-12T03:07:45.463 INFO:teuthology.orchestra.run.smithi005.stderr: 90.4% -- replaced with /var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph-mon.a.log.gz 2024-09-12T03:07:46.720 INFO:teuthology.orchestra.run.smithi005.stderr: 94.4% -- replaced with /var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph-osd.9.log.gz 2024-09-12T03:07:46.867 INFO:teuthology.orchestra.run.smithi146.stderr: 96.0% -- replaced with /var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph-mds.e.log.gz 2024-09-12T03:07:47.102 INFO:teuthology.orchestra.run.smithi146.stderr: 94.1% -- replaced with /var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph-osd.4.log.gz 2024-09-12T03:07:47.387 INFO:teuthology.orchestra.run.smithi005.stderr: 93.9% -- replaced with /var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph-osd.0.log.gz 2024-09-12T03:07:47.390 INFO:teuthology.orchestra.run.smithi146.stderr: 93.8% -- replaced with /var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph-osd.10.log.gz 2024-09-12T03:07:48.148 INFO:teuthology.orchestra.run.smithi005.stderr: 94.0% -- replaced with /var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph-osd.3.log.gz 2024-09-12T03:07:48.224 INFO:teuthology.orchestra.run.smithi146.stderr: 94.5% -- replaced with /var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph-osd.1.log.gz 2024-09-12T03:07:48.353 INFO:teuthology.orchestra.run.smithi148.stderr: 93.9% -- replaced with /var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph-osd.11.log.gz 2024-09-12T03:07:49.277 INFO:teuthology.orchestra.run.smithi148.stderr: 94.2% -- replaced with /var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph-osd.2.log.gz 2024-09-12T03:07:49.401 INFO:teuthology.orchestra.run.smithi148.stderr: 94.1% -- replaced with /var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph-osd.8.log.gz 2024-09-12T03:07:49.813 INFO:teuthology.orchestra.run.smithi005.stderr: 94.0% -- replaced with /var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph-osd.6.log.gz 2024-09-12T03:07:49.815 INFO:teuthology.orchestra.run.smithi005.stderr: 2024-09-12T03:07:49.815 INFO:teuthology.orchestra.run.smithi005.stderr:real 0m5.369s 2024-09-12T03:07:49.815 INFO:teuthology.orchestra.run.smithi005.stderr:user 0m15.041s 2024-09-12T03:07:49.815 INFO:teuthology.orchestra.run.smithi005.stderr:sys 0m0.558s 2024-09-12T03:07:50.119 INFO:teuthology.orchestra.run.smithi148.stderr: 94.3% -- replaced with /var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph-osd.5.log.gz 2024-09-12T03:07:50.318 INFO:teuthology.orchestra.run.smithi146.stderr: 94.4% -- replaced with /var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph-osd.7.log.gz 2024-09-12T03:07:50.320 INFO:teuthology.orchestra.run.smithi146.stderr: 2024-09-12T03:07:50.320 INFO:teuthology.orchestra.run.smithi146.stderr:real 0m5.871s 2024-09-12T03:07:50.320 INFO:teuthology.orchestra.run.smithi146.stderr:user 0m17.489s 2024-09-12T03:07:50.320 INFO:teuthology.orchestra.run.smithi146.stderr:sys 0m0.672s 2024-09-12T03:08:23.963 INFO:teuthology.orchestra.run.smithi148.stderr: 93.8% -- replaced with /var/log/ceph/618305f8-70b1-11ef-bcea-c7b262605968/ceph-mds.i.log.gz 2024-09-12T03:08:23.964 INFO:teuthology.orchestra.run.smithi148.stderr: 2024-09-12T03:08:23.964 INFO:teuthology.orchestra.run.smithi148.stderr:real 0m39.515s 2024-09-12T03:08:23.965 INFO:teuthology.orchestra.run.smithi148.stderr:user 0m57.366s 2024-09-12T03:08:23.965 INFO:teuthology.orchestra.run.smithi148.stderr:sys 0m1.928s 2024-09-12T03:08:23.965 INFO:tasks.cephadm:Archiving logs... 2024-09-12T03:08:23.965 DEBUG:teuthology.misc:Transferring archived files from smithi005:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901638/remote/smithi005/log 2024-09-12T03:08:23.966 DEBUG:teuthology.orchestra.run.smithi005:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-12T03:08:25.973 DEBUG:teuthology.misc:Transferring archived files from smithi146:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901638/remote/smithi146/log 2024-09-12T03:08:25.974 DEBUG:teuthology.orchestra.run.smithi146:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-12T03:08:28.284 DEBUG:teuthology.misc:Transferring archived files from smithi148:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901638/remote/smithi148/log 2024-09-12T03:08:28.285 DEBUG:teuthology.orchestra.run.smithi148:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-12T03:08:36.352 INFO:tasks.cephadm:Removing cluster... 2024-09-12T03:08:36.352 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 618305f8-70b1-11ef-bcea-c7b262605968 --force 2024-09-12T03:08:36.566 INFO:teuthology.orchestra.run.smithi005.stdout:Deleting cluster with fsid: 618305f8-70b1-11ef-bcea-c7b262605968 2024-09-12T03:08:36.847 DEBUG:teuthology.orchestra.run.smithi146:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 618305f8-70b1-11ef-bcea-c7b262605968 --force 2024-09-12T03:08:37.052 INFO:teuthology.orchestra.run.smithi146.stdout:Deleting cluster with fsid: 618305f8-70b1-11ef-bcea-c7b262605968 2024-09-12T03:08:37.338 DEBUG:teuthology.orchestra.run.smithi148:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 618305f8-70b1-11ef-bcea-c7b262605968 --force 2024-09-12T03:08:37.541 INFO:teuthology.orchestra.run.smithi148.stdout:Deleting cluster with fsid: 618305f8-70b1-11ef-bcea-c7b262605968 2024-09-12T03:08:37.868 INFO:tasks.cephadm:Removing cephadm ... 2024-09-12T03:08:37.868 DEBUG:teuthology.orchestra.run.smithi005:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-12T03:08:37.888 DEBUG:teuthology.orchestra.run.smithi146:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-12T03:08:37.904 DEBUG:teuthology.orchestra.run.smithi148:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-12T03:08:37.920 INFO:tasks.cephadm:Teardown complete 2024-09-12T03:08:37.920 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-12T03:08:37.934 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-12T03:08:37.935 DEBUG:teuthology.orchestra.run.smithi005:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-12T03:08:37.938 DEBUG:teuthology.orchestra.run.smithi146:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-12T03:08:37.947 DEBUG:teuthology.orchestra.run.smithi148:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-12T03:08:38.057 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-12T03:08:38.069 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-12T03:08:38.069 DEBUG:teuthology.orchestra.run.smithi005:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-12T03:08:38.072 DEBUG:teuthology.orchestra.run.smithi146:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-12T03:08:38.075 DEBUG:teuthology.orchestra.run.smithi148:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-12T03:08:38.087 INFO:teuthology.orchestra.run.smithi146.stderr:bash: line 1: ntpq: command not found 2024-09-12T03:08:38.087 INFO:teuthology.orchestra.run.smithi005.stderr:bash: line 1: ntpq: command not found 2024-09-12T03:08:38.089 INFO:teuthology.orchestra.run.smithi148.stderr:bash: line 1: ntpq: command not found 2024-09-12T03:08:38.091 INFO:teuthology.orchestra.run.smithi146.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-12T03:08:38.091 INFO:teuthology.orchestra.run.smithi146.stdout:=============================================================================== 2024-09-12T03:08:38.091 INFO:teuthology.orchestra.run.smithi146.stdout:^* hv01.front.sepia.ceph.com 3 6 377 63 -3110us[-3109us] +/- 46ms 2024-09-12T03:08:38.091 INFO:teuthology.orchestra.run.smithi146.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 60 +1804us[+1804us] +/- 61ms 2024-09-12T03:08:38.091 INFO:teuthology.orchestra.run.smithi146.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 62 +2951us[+2951us] +/- 75ms 2024-09-12T03:08:38.091 INFO:teuthology.orchestra.run.smithi146.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T03:08:38.092 INFO:teuthology.orchestra.run.smithi005.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-12T03:08:38.093 INFO:teuthology.orchestra.run.smithi005.stdout:=============================================================================== 2024-09-12T03:08:38.093 INFO:teuthology.orchestra.run.smithi005.stdout:^* hv01.front.sepia.ceph.com 3 6 377 60 -3065us[-3076us] +/- 46ms 2024-09-12T03:08:38.093 INFO:teuthology.orchestra.run.smithi005.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 59 +1858us[+1858us] +/- 61ms 2024-09-12T03:08:38.093 INFO:teuthology.orchestra.run.smithi005.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 60 +3005us[+3005us] +/- 75ms 2024-09-12T03:08:38.093 INFO:teuthology.orchestra.run.smithi005.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T03:08:38.094 INFO:teuthology.orchestra.run.smithi148.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-12T03:08:38.095 INFO:teuthology.orchestra.run.smithi148.stdout:=============================================================================== 2024-09-12T03:08:38.095 INFO:teuthology.orchestra.run.smithi148.stdout:^* hv01.front.sepia.ceph.com 3 6 377 60 -3096us[-3095us] +/- 46ms 2024-09-12T03:08:38.095 INFO:teuthology.orchestra.run.smithi148.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 60 +1831us[+1832us] +/- 61ms 2024-09-12T03:08:38.095 INFO:teuthology.orchestra.run.smithi148.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 58 +2977us[+2977us] +/- 75ms 2024-09-12T03:08:38.095 INFO:teuthology.orchestra.run.smithi148.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-12T03:08:38.095 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-12T03:08:38.144 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-12T03:08:38.145 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-12T03:08:38.202 DEBUG:teuthology.orchestra.run.smithi005:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-12T03:08:38.291 DEBUG:teuthology.orchestra.run.smithi146:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-12T03:08:38.358 DEBUG:teuthology.orchestra.run.smithi148:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-12T03:08:38.425 DEBUG:teuthology.orchestra.run.smithi005:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-12T03:08:38.457 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T03:08:38.457 DEBUG:teuthology.orchestra.run.smithi146:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-12T03:08:38.486 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T03:08:38.486 DEBUG:teuthology.orchestra.run.smithi148:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-12T03:08:38.514 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T03:08:38.514 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-12T03:08:38.527 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-12T03:08:38.667 INFO:teuthology.task.internal:Duration was 1610.203929 seconds 2024-09-12T03:08:38.667 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-12T03:08:38.731 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-12T03:08:38.732 DEBUG:teuthology.orchestra.run.smithi005:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-12T03:08:38.735 DEBUG:teuthology.orchestra.run.smithi146:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-12T03:08:38.737 DEBUG:teuthology.orchestra.run.smithi148:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-12T03:08:38.774 INFO:teuthology.orchestra.run.smithi146.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-12T03:08:38.776 INFO:teuthology.orchestra.run.smithi148.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-12T03:08:38.778 INFO:teuthology.orchestra.run.smithi005.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-12T03:08:39.245 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-12T03:08:39.245 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi005.front.sepia.ceph.com 2024-09-12T03:08:39.245 DEBUG:teuthology.orchestra.run.smithi005:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-12T03:08:39.270 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi146.front.sepia.ceph.com 2024-09-12T03:08:39.270 DEBUG:teuthology.orchestra.run.smithi146:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-12T03:08:39.308 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi148.front.sepia.ceph.com 2024-09-12T03:08:39.308 DEBUG:teuthology.orchestra.run.smithi148:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-12T03:08:39.333 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-12T03:08:39.334 DEBUG:teuthology.orchestra.run.smithi005:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-12T03:08:39.337 DEBUG:teuthology.orchestra.run.smithi146:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-12T03:08:39.352 DEBUG:teuthology.orchestra.run.smithi148:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-12T03:08:39.401 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-12T03:08:39.402 DEBUG:teuthology.orchestra.run.smithi005:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-12T03:08:39.407 DEBUG:teuthology.orchestra.run.smithi146:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-12T03:08:39.417 DEBUG:teuthology.orchestra.run.smithi148:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-12T03:08:40.429 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-12T03:08:40.441 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-12T03:08:40.442 DEBUG:teuthology.orchestra.run.smithi005:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-12T03:08:40.468 DEBUG:teuthology.orchestra.run.smithi146:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-12T03:08:40.495 DEBUG:teuthology.orchestra.run.smithi148:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-12T03:08:40.522 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-12T03:08:40.535 DEBUG:teuthology.orchestra.run.smithi005:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-12T03:08:40.537 DEBUG:teuthology.orchestra.run.smithi146:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-12T03:08:40.540 DEBUG:teuthology.orchestra.run.smithi148:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-12T03:08:40.559 INFO:teuthology.orchestra.run.smithi005.stdout:kernel.core_pattern = core 2024-09-12T03:08:40.563 INFO:teuthology.orchestra.run.smithi146.stdout:kernel.core_pattern = core 2024-09-12T03:08:40.587 INFO:teuthology.orchestra.run.smithi148.stdout:kernel.core_pattern = core 2024-09-12T03:08:40.622 DEBUG:teuthology.orchestra.run.smithi005:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-12T03:08:40.637 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T03:08:40.638 DEBUG:teuthology.orchestra.run.smithi146:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-12T03:08:40.662 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T03:08:40.662 DEBUG:teuthology.orchestra.run.smithi148:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-12T03:08:40.677 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-12T03:08:40.678 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-12T03:08:40.689 INFO:teuthology.task.internal:Transferring archived files... 2024-09-12T03:08:40.690 DEBUG:teuthology.misc:Transferring archived files from smithi005:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901638/remote/smithi005 2024-09-12T03:08:40.690 DEBUG:teuthology.orchestra.run.smithi005:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-12T03:08:40.748 DEBUG:teuthology.misc:Transferring archived files from smithi146:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901638/remote/smithi146 2024-09-12T03:08:40.748 DEBUG:teuthology.orchestra.run.smithi146:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-12T03:08:40.785 DEBUG:teuthology.misc:Transferring archived files from smithi148:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-11_21:24:17-fs-squid-distro-default-smithi/7901638/remote/smithi148 2024-09-12T03:08:40.785 DEBUG:teuthology.orchestra.run.smithi148:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-12T03:08:40.821 INFO:teuthology.task.internal:Removing archive directory... 2024-09-12T03:08:40.821 DEBUG:teuthology.orchestra.run.smithi005:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-12T03:08:40.823 DEBUG:teuthology.orchestra.run.smithi146:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-12T03:08:40.826 DEBUG:teuthology.orchestra.run.smithi148:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-12T03:08:40.871 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-12T03:08:40.884 INFO:teuthology.task.internal:Not uploading archives. 2024-09-12T03:08:40.885 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-12T03:08:40.977 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-12T03:08:40.977 DEBUG:teuthology.orchestra.run.smithi005:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-12T03:08:40.980 DEBUG:teuthology.orchestra.run.smithi146:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-12T03:08:40.982 DEBUG:teuthology.orchestra.run.smithi148:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-12T03:08:40.993 INFO:teuthology.orchestra.run.smithi005.stdout: 265637 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 12 03:08 /home/ubuntu/cephtest 2024-09-12T03:08:40.995 INFO:teuthology.orchestra.run.smithi146.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 12 03:08 /home/ubuntu/cephtest 2024-09-12T03:08:40.997 INFO:teuthology.orchestra.run.smithi148.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 12 03:08 /home/ubuntu/cephtest 2024-09-12T03:08:40.998 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-12T03:08:41.057 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-12T03:08:41.267 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-quiesce/with-quiesce 6-workunit/suites/fsx}} duration: 1610.2039289474487 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-09-12T03:08:41.268 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-12T03:08:41.336 INFO:teuthology.run:pass